Skip to content

Spec'ing the Probabilistic

Shipping AI Features Safely

7 min+70 XPStep 1/11

Ship behind flags and gates

The AI shipping checklist differs from classic features: a feature flag (instant kill without deploy), an eval gate (the suite must pass the bar before the flag opens), staged rollout — internal dogfood → alpha cohort → expansion — with acceptance metrics watched per stage, and a rollback story that includes the model version. 'Revert' means reverting prompt + model + retrieval config, not just code.

If you can't answer 'which model version served that output' from your logs, you're not ready to ship.