Decoding Strategies

At every step, an LLM produces a probability distribution over all ~100k tokens in its vocabulary — not a single word. Something has to pick one. That picking rule is the decoding strategy, and it changes everything about the output.

Greedy decoding always takes the argmax — the single most likely token. Safe and deterministic, but boring: left alone it drifts into 'the best way is the best way is the best way…' loops.