Fairness in Clinical Models

The canonical case (Obermeyer et al., Science 2019): a widely-used care-management algorithm used healthcare cost as a proxy for health need. Looked neutral — but at equal sickness levels, Black patients had historically generated less cost (access barriers, fewer referrals), so equal-risk scores mapped to sicker Black patients needing the same threshold.

Fixing the label — predicting need (active chronic conditions) instead of cost — repaired most of the disparity. The lesson that generalizes: audit the label first. It's the most concentrated place where structural inequity becomes model output.