Skip to content

Trees & Forests

How a Tree Decides

8 min+70 XPStep 1/11

Twenty questions, played by a machine

A decision tree is a flowchart the data builds itself. Each node asks one yes/no question — balance < $2,000? — and routes the record left or right, until it lands in a leaf that votes on the answer.

To grow the tree, the algorithm tries every feature at every possible cut point and keeps the split that produces the purest children — the nodes whose labels end up most one-sided. A split that puts 95% of fraud on one side beats one that merely shuffles the mix.

insight

**Entropy** is the scoreboard for purity: a 50/50 node has maximum entropy, a 100% node has zero. Each split is chosen for the biggest entropy drop — called *information gain*.