ML Forecasting & Evaluation
Forecast Metrics That Don't Lie
Every metric has a personality
MAE (mean absolute error) averages |actual − forecast|: interpretable, in the same units as your data, robust to occasional disasters. Optimize MAE and you're chasing the median outcome.
RMSE (root mean squared error) squares first: a 10-unit miss costs 100, while ten 1-unit misses cost 10. It punishes the occasional catastrophe — and optimizes toward the mean, which skewed error distributions make a genuinely different target.
Always check both: RMSE ≥ MAE by construction, and a wide gap between them means your errors are mostly small with a few monsters mixed in.
insight
MAE vs RMSE is a policy decision, not a math one. Do occasional big misses hurt disproportionately — a critical part stocked out — or is error just error? Pick the metric that prices mistakes the way your business does.