Trees & Forests
How a Tree Decides
8 min+70 XPStep 1/11
Twenty questions, played by a machine
A decision tree is a flowchart the data builds itself. Each node asks one yes/no question — balance < $2,000? — and routes the record left or right, until it lands in a leaf that votes on the answer.
To grow the tree, the algorithm tries every feature at every possible cut point and keeps the split that produces the purest children — the nodes whose labels end up most one-sided. A split that puts 95% of fraud on one side beats one that merely shuffles the mix.
insight
**Entropy** is the scoreboard for purity: a 50/50 node has maximum entropy, a 100% node has zero. Each split is chosen for the biggest entropy drop — called *information gain*.