AWS ML Specialty
MLOps Domain
8 min+80 XPStep 1/12
Four ways to serve a model
- Real-time endpoint — persistent, millisecond latency, request/response (payloads up to ~6 MB)
- Serverless inference — spiky or idle traffic where cold starts are acceptable
- Batch transform — no endpoint at all; score whole datasets on a schedule
- Asynchronous inference — large payloads (up to ~1 GB) or long processing; queued, near-real-time
Plus multi-model endpoints: thousands of similar models (one per customer or store) share a single endpoint — SageMaker loads each on demand, slashing cost.
insight
Keyword map: 'cost + many similar models' → multi-model endpoint; 'unpredictable traffic' → serverless; 'score GBs nightly' → batch transform.