The Eval Toolbox: Code, Model, Human

Code assertions — deterministic checks: is the output valid JSON, does it contain the required field, does the regex match. Free to run, perfectly reliable — and blind to anything that can't be asserted in code.

Model-graded — an LLM judges the output against a rubric: scalable to thousands of cases, catches open-ended quality — and inherits every bias and blind spot of the judge model.

Human eval — people rate the output: the gold standard for 'is this actually good' — at gold-standard prices, with gold-standard latency and inter-rater disagreement.

None is sufficient alone. The craft is allocating them.