Skip to content

LLM-as-Judge

Rubrics That Grade Like You

10 min+100 XPStep 1/10

A rubric is a spec, written for a judge

'Is the response good?' is not a rubric — it's a vibe dressed as a question. A rubric decomposes 'good' into atomic criteria, each independently checkable: 'cites a source for every factual claim', 'stays under 200 words', 'doesn't promise features that don't exist'.

Atomicity matters because judges grade criteria, not holism — a multi-part criterion ('accurate and concise and well-formatted') produces unexplainable grades where you can't tell which part failed. One criterion per question; the score is the pattern across them.

insight

A criterion you can't state as a checkable question isn't a criterion — it's a wish. 'Is it good?' fails; 'does every claim have a citation?' grades.