Evaluation Pitfalls

Aggregate accuracy hides almost everything you care about. A '96% accurate' classifier can be coin-flipping on the rare class that actually matters. Detection has its own metrics — IoU measures how much a predicted box overlaps the true one, and mAP averages precision across classes and IoU thresholds. An mAP at IoU 0.5 is lenient; at IoU 0.75, strict — papers report both for a reason.

One more trap: many vision benchmarks were scraped from the same web as the training sets. When test images leak into training data, you're measuring memory, not skill — contamination inflates numbers quietly.