Skip to content

Trees & Forests

Gradient Boosting: Serial Self-Correction

9 min+80 XPStep 1/11

The opposite philosophy

Bagging grows independent trees in parallel and votes. Boosting grows trees in sequence — each one trained to fix what the ensemble currently gets wrong.

The ensemble predicts, the errors are measured, and the next small tree is fit to those residuals — literally answering 'where are we still wrong?' Add it to the ensemble, re-measure, repeat for hundreds of rounds.

analogy

A forest is a crowd voting in parallel. Boosting is a relay of editors, each correcting the errors the previous draft still contains.