---
title: "Discussion on \"LLM-as-a-Judge: Using One Model to Grade Another (and Where It Fools You)\" | Hashnode"
description: "Discussion on \"LLM-as-a-Judge: Using One Model to Grade Another (and Where It Fools You)\". LLM-as-a-Judge"
url: "https://hashnode.com/posts/llm-as-a-judge-using-one-model-to-grade-another-and-where-it-fools-you/6aeafeb3f128828a14a16cc1"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [LLM-as-a-Judge: Using One Model to Grade Another (and Where It Fools You)](https://nagspidey.hashnode.dev/llm-as-a-judge-using-one-model-to-grade-another-and-where-it-fools-you?utm_source=hashnode&utm_medium=feed)

LLM-as-a-Judge

Nov 4 · 8 min read · nagspidey.hashnode.dev  
Read the article

-   [#llm](https://hashnode.com/tag/llm)
-   [#ai-evaluation](https://hashnode.com/tag/ai-evaluation)
-   [#machine-learning](https://hashnode.com/tag/machine-learning)
-   [#artificial-intelligence](https://hashnode.com/tag/artificial-intelligence)
-   [#software-engineering](https://hashnode.com/tag/software-engineering)

## Written by

[Nagappan S](https://hashnode.com/@imkarthik03)

@imkarthik03 · Chennai, India

  
[View profile](https://hashnode.com/@imkarthik03)

Building scalable products at the intersection of Full Stack Engineering & AI

I build scalable web apps, experiment with AI, and turn complex engineering problems into working products.

## Responses 2

Markdown supported · ⌘↵ to post

[Salman Parvez | ML Systems](https://hashnode.com/@ml-systems)

[Nov 4](https://hashnode.com/posts/llm-as-a-judge-using-one-model-to-grade-another-and-where-it-fools-you/6aeafeb3f128828a14a16cc1/comment/6aeb3aa66a76c5c99da82800)

Verbosity bias is the one I'd watch. A longer estimate isn't a better estimate, and a judge that rewards length will push everything toward padding. Checking the judge against a few human-graded answers is the step we won't skip at ML Systems.

[Nagappan S](https://hashnode.com/@imkarthik03)

[3d ago](https://hashnode.com/posts/llm-as-a-judge-using-one-model-to-grade-another-and-where-it-fools-you/6aeafeb3f128828a14a16cc1/comment/6aeb3aa66a76c5c99da82800#reply-6af1a5e524e8782be41d2a61)

Agreed on verbosity bias. One cheap extra check on your human-graded set: correlate the judge's scores with answer length. If that correlation is stronger than the one in the human scores, the bias is showing. The post suggests aiming for roughly 0.85 correlation with human raters and re-checking regularly.