Trees & Forests
Gradient Boosting: Serial Self-Correction
9 min+80 XPStep 1/11
The opposite philosophy
Bagging grows independent trees in parallel and votes. Boosting grows trees in sequence — each one trained to fix what the ensemble currently gets wrong.
The ensemble predicts, the errors are measured, and the next small tree is fit to those residuals — literally answering 'where are we still wrong?' Add it to the ensemble, re-measure, repeat for hundreds of rounds.
analogy
A forest is a crowd voting in parallel. Boosting is a relay of editors, each correcting the errors the previous draft still contains.