The Feedback Problem
Exploration: Bandits, Not Just Exploitation
11 min+110 XPStep 1/10
Your training data was collected by the enemy — your last model
Every logged interaction records what the previous system chose to show. Items never displayed have no data, so the model can't learn it would have loved them — the training set is rigged in favor of whatever already got exposure.
Pure exploitation is a lock-in spiral: show the safe hits, get clicks on the safe hits, learn the safe hits, show the safe hits again. The bandit framing breaks it: each item is a slot-machine arm, showing it is a pull, and the problem is balancing exploit (best known arm) against explore (learn what the other arms pay).
analogy
A recommender with zero exploration is polling people who already voted for the incumbent. The landslide you measure is the loop you built.