LLM-as-a-judge

Also known as: LLM judge

On this page 7

LLM-as-a-judge is an evaluation method in which a large language model grades the outputs of another AI system, or of itself, according to a written rubric, a reference answer or a pairwise preference question. In model evaluation, an LLM judge fills the gap between cheap automatic metrics such as BLEU or ROUGE, which miss meaning, and human evaluation, which is accurate but slow and expensive.

The approach became standard in artificial intelligence once instruction-following models proved able to assess open-ended qualities such as helpfulness, factual consistency, tone and reasoning quality. It now underpins many leaderboards, regression test suites and reward signals, yet its scores are only as trustworthy as the judge model, the prompt and the validation against humans.

How an LLM judge produces a score

An LLM judge receives a structured evaluation prompt containing the task, the candidate response, the scoring criteria and, optionally, a reference answer. It returns a score, a label or a preference, often preceded by a short written rationale that improves consistency when the model reasons before deciding.

Results are then parsed and aggregated over a dataset into metrics such as mean rubric score, pass rate or win rate against a baseline model. Because the judge is itself a stochastic model, practitioners typically set temperature to zero or average several samples to reduce variance.

The table below lists the main judging formats and what each measures.

FormatJudge inputOutput
Single-answer gradingOne response and a rubricScore on a fixed scale, such as 1 to 5
Pairwise comparisonTwo responses to the same promptPreferred response or tie
Reference-based gradingResponse plus a gold answerCorrect, partially correct or incorrect
Binary checklistResponse and yes or no criteriaPass or fail per criterion
Common LLM-as-a-judge formats

Where LLM judges fit among evaluation methods

LLM-as-a-judge sits between lexical metrics and expert human review in cost, speed and semantic sensitivity. Choosing among them depends on whether the task has a single correct answer and how much disagreement between careful humans is expected.

MethodStrengthMain weakness
Exact matchObjective and reproducibleUseless for free-form text
ROUGE and BLEUCheap n-gram overlapIgnores paraphrase and correctness
BERTScoreCaptures semantic similarityStill needs a reference text
LLM-as-a-judgeScales open-ended assessmentBiases and limited reliability
Human evaluationClosest to real user judgmentSlow, costly, variable agreement
LLM judges compared with other evaluation methods

Tasks with verifiable answers, such as arithmetic in GSM8K or code checked by unit tests in HumanEval, should be scored by deterministic checks. An LLM judge adds value mainly where quality is subjective or multidimensional, for example summaries, dialogue or explanations.

Known biases of LLM judges

Research and field experience have documented systematic biases that distort judge scores in predictable directions. Recognizing them is the first step toward a defensible evaluation pipeline.

BiasWhat happensTypical mitigation
Position biasFavors the first or second answer shownSwap order and average both runs
Verbosity biasPrefers longer responsesLength-controlled scoring or explicit rubric
Self-preference biasRates its own model family higherUse a judge from another model family
Authority biasTrusts confident or cited claimsProvide reference facts for checking
Format biasRewards headings and listsNormalize formatting before judging
Documented judge biases and mitigations

Judges also struggle with tasks harder than their own capability. A judge that cannot solve a difficult math or coding problem frequently fails to notice subtle errors in a candidate solution, so reference answers or tool-based verification become necessary for advanced reasoning benchmarks.

Designing a reliable judging setup

A dependable LLM-as-a-judge pipeline treats the judge as a measurement instrument that must be calibrated. The most important design choices concern the rubric, the judge model and the validation procedure.

  • Write specific, observable criteria instead of vague words such as good or helpful.
  • Prefer binary or low-resolution scales, which judges apply more consistently than 10-point scales.
  • Ask for a brief rationale before the verdict so the score follows from explicit reasoning.
  • Validate against a human-labeled subset, typically a few hundred items, and report agreement.
  • Freeze the judge version and prompt so that scores remain comparable over time.

Agreement with humans is usually reported with Cohen’s kappa, Spearman correlation or simple percent agreement. A judge whose agreement with experts approaches inter-annotator agreement between those experts is generally considered fit for purpose for that task, but not automatically for others.

ChoiceRecommended settingEffect
Judge temperatureZero or near zeroReduces score variance
Scoring scaleBinary or 3 to 5 pointsImproves consistency
Comparison orderBoth orders evaluatedCancels position bias
Judge familyDifferent from candidateLimits self-preference
Configuration choices and their effects

Risks of relying on automated judgment

When an LLM judge becomes an optimization target, for instance as a reward model in reinforcement learning, candidate models can learn to exploit judge weaknesses rather than improve. This is a form of Goodhart’s law: verbose, confident or flattering outputs may raise scores without raising real quality.

Other risks include cost at scale, since judging thousands of long outputs with a frontier model is not cheap, and non-reproducibility when a hosted judge model is updated silently. For high-stakes decisions such as safety sign-off or medical content, LLM judges should complement, not replace, expert review.

A practical safeguard is to monitor judge drift by rescoring a small, fixed calibration set whenever the judge model or prompt changes. If agreement with the stored human labels drops noticeably, earlier and later scores should not be compared until the judge has been recalibrated and revalidated.

A plain-language summary

LLM-as-a-judge means using one AI to grade another AI. Instead of asking people to read thousands of answers, a capable language model reads each answer, compares it with clear instructions and gives it a score.

It works a bit like a teaching assistant marking essays with an answer key. It is fast and usually sensible, but it has habits, such as liking longer essays. That is why the grades should be checked against real human opinions before anyone trusts them.

An example from practice

A software company runs a customer support assistant and changes its system prompt every week. Before each release, the team must confirm that answers remain accurate and polite across a fixed set of 1,200 real customer questions, which would take human reviewers several days.

The team builds a pairwise LLM judge from a different model family, asks it to compare the new and old answers in both orders, and uses a four-item rubric covering correctness, completeness, tone and policy compliance. Two support leads first label 300 pairs, and the judge agrees with them on about 85 percent of cases, close to their agreement with each other. The release check now finishes in under an hour, and only the disputed or low-scoring answers go to human review, which cuts review effort sharply while catching regressions before customers see them.