Cost & Latency

API pricing is per million tokens, split input/output — outputs usually run 3–5× the input price. Rough landscape: flagship models ~$3–15 per 1M input, $15–75 per 1M output; small models ~$0.10–0.50.

Example: a 2,000-token input + 500-token output request at $5/$15 per 1M costs ~$0.0175. Trivial — until a million requests a month turns it into a $17,500 line item, and a chatty system prompt or bloated retrieval silently doubles it.