Machine Learning Foundations · entry 06/10 · 3 min read

Models grading models

There is no answer key for 'summarize this contract', so the field grades with models — cheap, fast, and wrong in specific, correctable ways.

Why it exists

Classification has labels; generation does not. Once you need to score thousands of open-ended outputs per release, human grading stops being affordable and a model becomes the grader — LLM-as-judge, reading a response against a rubric, or comparing two responses and picking the better. It works well enough to be the backbone of most eval pipelines in production, which is exactly why its failure modes are worth knowing precisely.

The biases are known, so control for them

Judges prefer longer answers regardless of quality. They prefer a position — first for some models, last for others — and the effect is large enough to flip verdicts on its own, so any pairwise comparison must be run in both orders, with disagreement scored as a tie rather than averaged away; averaging launders instability into a confident-looking midpoint. They may favor their own outputs, though that one is contested: control for answer quality and much of it disappears. They are lenient with fluent, confident prose, which is precisely the failure mode generation produces. And they reward surface conformity to the rubric over substance. None of that makes the method useless; all of it means an uncontrolled judge measures style.

Making a judge you can trust

Prefer comparison to scoring: "is A better than B" is a question models answer far more consistently than "rate this 1–10," where the scale drifts between runs — at the cost of the position bias above and of any absolute level, which is why most pipelines keep a rubric score for release gating and comparisons for ranking. Write the rubric as concrete, checkable criteria rather than adjectives — "cites a source for every numeric claim" beats "is accurate." Make the judge work through those criteria before it commits to a verdict: a score emitted first gets one token's worth of thought, and the explanation after it is by construction a rationalization. Then do the step that makes it science: calibrate the judge against human labels on a sample, measure agreement, and re-check it whenever you change judge models. A judge nobody validated is a random number generator with good manners.

Failure mode

Grading a model with itself, then optimizing against the score. Two compounding errors: self-preference may inflate the number, and optimizing against any fixed judge detaches it from what you wanted — your system learns the judge's blind spots, and the metric climbs while quality does not. Use a different model as judge than the one under test, and hold the line every eval needs: a set nobody optimizes against, read by a person, often enough to notice drift.