LLM as a Judge: When It Works and When It Lies

LLM as a judge means using one language model to score or compare another model's output against a rubric. It scales review cheaply, and the best-known study of the method found that strong judges agreed with human preferences over 80% of the time, about as often as humans agree with each other. It is only as trustworthy as the rubric and the calibration behind it.

That last sentence is the whole post. A judge model is a measuring instrument. An instrument nobody calibrated is a guess with a number attached, and the number makes the guess look more reliable than it is.

We use judge models in production, and we have also watched them be confidently wrong.

What is LLM as a judge?

Most teams evaluating AI output end up with three kinds of checks, and a judge model sits in the middle.

• Code-based checks. Exact match, string match, a schema validator, a test that runs. Fast, cheap and rigid. They cannot tell you whether a summary is faithful.
• Human review. Accurate and flexible, and slow and expensive. Nobody reads ten thousand outputs a night.
• A judge model. Fast, flexible and able to handle judgment calls. Also able to be wrong in ways that look right.

Anthropic's eval guide lays out the same three options and puts the trade-off plainly: LLM-based grading is "fast and flexible, scalable and suitable for complex judgment," with the instruction to "test to ensure reliability first then scale" (Claude docs, "Define success criteria and build evaluations", read October 7, 2026).

The paper that named the method, Zheng et al., "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena," was first posted in June 2023 and last revised in December 2023 (arXiv 2306.05685). It found that strong LLM judges like GPT-4 could match both controlled and crowdsourced human preferences "achieving over 80% agreement, the same level of agreement between humans."

Treat that as a 2023 result on chat-style answers, not a guarantee for your task. It tells you the method can work. It does not tell you your judge works.

When does a judge model actually work?

In our experience, a judge earns its keep under three conditions. All three show up in the vendor guidance too.

The question is narrow. "Does this reply contain patient health information, yes or no" is a question a judge can answer consistently. "Rate this reply from 1 to 10" is not. The Claude docs recommend making grading "empirical or specific," for example having the model output only "correct" or "incorrect," or a fixed 1 to 5 scale, because "purely qualitative evaluations are hard to assess quickly and at scale."

There is a written rubric with examples. A judge without a rubric fills the gap with its own taste. Write down what a pass looks like and what a fail looks like, and include one of each. The docs' own example rubric is blunt on purpose: if the answer does not mention a specific company name in the first sentence, it is graded incorrect. Blunt rubrics are a feature.

The judge is a different model from the one being graded. The Claude docs include this line in their sample grading code: "Generally best practice to use a different model to evaluate than the model used to generate the evaluated output." That matters because of the biases below.

If you cannot meet those three, you probably do not have an eval yet. You have a vibe check with an API bill.

Why do LLM judges get it wrong?

The same Zheng et al. paper that reported the 80% agreement also named the failure modes: "position, verbosity, and self-enhancement biases, as well as limited reasoning ability."

Position bias. When asked to compare two answers, a judge can favor whichever one appears first, or second, regardless of quality. Swap the order and the verdict flips. If your pairwise eval never swaps order, you do not know whether you measured quality or seating.

Verbosity bias. Judges tend to reward the longer answer. A padded response with three extra paragraphs can beat a tight, correct one. This is how an agent "improves" on your eval while getting worse for your users.

Self-enhancement bias. A judge can prefer output written by itself or by its own model family. Grade a model with itself and you may be measuring how much it likes its own style.

Limited reasoning. A judge can be confidently wrong on math or logic it could not have solved itself. If the judge would fail the task, it cannot reliably grade the task.

Here is what we do about each:

• Swap and require agreement. For pairwise comparisons, run the judge twice with the order reversed. Count a win only when both runs agree. Ties are information.
• Control for length. Put a length expectation in the rubric, or compare answers of similar length. If a longer answer wins, check whether it won on substance.
• Ask for the reason before the verdict. The Claude docs recommend a grader with thinking on, "so that it reasons before it produces an evaluation score."
• Keep the deterministic check. If code can verify it, let code verify it. Does the JSON parse, does the test pass, does the cited URL resolve. A judge is for what code cannot see, never a replacement for what it can.

How do you calibrate a judge before trusting it?

Calibration is the step most teams skip, and it is the one that decides whether your eval numbers mean anything.

Hand-label a small ground-truth set first. Before the judge grades anything at scale, have a person label a set of real outputs against the same rubric. Keep it small and real, drawn from the inputs your system actually sees.

Measure agreement. Run the judge on that set and compare. Where it agrees with your labels, you have some evidence. Where it disagrees, read every case. You will usually find one of three things: the rubric is ambiguous, the judge has a blind spot, or your own label was wrong.

Re-run it whenever something changes. A new judge model, a new rubric, a new prompt template or a new kind of input each resets your confidence. Calibration is not a one-time setup step. It is part of the release checklist.

Keep the disagreements. The cases where the judge and the human split are the best future eval cases you will ever get.

This is the same loop we describe in eval-driven agent development: the eval set grows from real failures, and the judge is one gate in the harness, not the whole test suite. It is also part of our agent orchestration methodology, where we decide early which checks are code, which are a model and which need a person.

What does a judge look like in a shipped product?

In Smile PreVue, the dental platform we build, a second AI model checks every smile simulation after it is generated. It compares the preview against the patient's own photo and against the treatment that was requested, and the result is shown to the clinic's team only.

A few design choices there map directly to the rules above:

• The rubric is narrow and named. The grader looks at specific things, like whether the patient's identity and framing were preserved, whether the lips were left alone, whether only the teeth changed, and whether the requested treatment and shade came through. It is told not to describe the patient.
• The verdict is a short ordinal label. The clinic sees Excellent, Good or Review, not a long free-text opinion.
• A failing grade names the problem. When a preview is flagged for review, the specific issue is shown and the dentist can regenerate with that fix written into the instructions, or regenerate as is. Earlier attempts are archived, not deleted.
• The judge does not slow the product. Grading runs in the background after the preview is already on screen.

The choice we would point other teams at is where the score goes. It stays with the clinical team. It never appears on what the patient receives, and it is not sold as an accuracy number. A judge score is a quality signal for the people operating the system. It is not a promise to the person on the other end, and it is not a clinical prediction about how the final dental work will look. Turning an internal quality signal into a public claim is how teams end up defending a number they never validated.

FAQ

Is LLM as a judge reliable? For narrow, rubric-based questions, once it has been checked against human labels, it can be. For open-ended "is this good" calls on its own, we would not rely on it.

Should the judge be a bigger model? Different matters more than bigger. Avoid grading a model with itself or its own family, and make sure the judge could solve the task it is grading.

Does a judge model replace human review? It replaces most of the volume. It does not replace the human-labeled set you calibrate against, or the spot checks on the cases where it disagrees with that set.

What does LLM-as-a-judge evaluation cost? Every graded output is another model call, and reasoning before the verdict adds tokens. Budget it like any other production token spend, and use code checks wherever they can do the job.

Where to start

If you have an agent in production and no idea whether last week's prompt change made it better or worse, the fix is not a fancier judge. It is a small labeled set, a narrow rubric and a judge you have checked against both. Before reaching for one at all, it is worth asking when a simple LLM call beats an agent, because a simpler system is easier to evaluate.

If you want help building evals into a system you are shipping, that is the kind of custom software for AI workflows we do. If you would rather do it yourself, you can learn to build evals with Claude one on one. Either way, tell us what you are building.