Regression Suites & CI Evals

The moment evals run only 'when someone remembers', they stop catching anything — the failure mode is always a change, and it always lands between manual runs. CI evals run the suite on every prompt edit, every model version bump, every retrieval change: the gate that says 'this change didn't break what we defined as working'.

The gate design matters: a threshold ('no metric may drop >2 points', 'no safety criterion may fail') turns evals from information into enforcement. Without gates, eval results are a report nobody acts on; with them, regressions block merges the way failing tests do.