---
title: "Discussion on \"Why Agent Evaluation Is Harder Than Model Evaluation\" | Hashnode"
description: "Discussion on \"Why Agent Evaluation Is Harder Than Model Evaluation\". The moment a system does multi-step work, model benchmarks stop predicting anything. What agent evaluation actually has to measure instead."
url: "https://hashnode.com/posts/why-agent-evaluation-is-harder-than-model-evaluation/6a6d5203846eddf6cc00a19b"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Why Agent Evaluation Is Harder Than Model Evaluation](https://pragmatic-engineer.hashnode.dev/why-agent-evaluation-is-harder-than-model-evaluation?utm_source=hashnode&utm_medium=feed)

The moment a system does multi-step work, model benchmarks stop predicting anything. What agent evaluation actually has to measure instead.

Aug 1 · 8 min read · pragmatic-engineer.hashnode.dev  
Updated Oct 29  
Read the article

-   [#ai](https://hashnode.com/tag/ai)
-   [#agentic-ai](https://hashnode.com/tag/agentic-ai)
-   [#llm](https://hashnode.com/tag/llm)
-   [#challenge](https://hashnode.com/tag/challenge)

## Written by

[Debashish Ghosal](https://hashnode.com/@deghosal)

@deghosal · Seattle, USA

  
[View profile](https://hashnode.com/@deghosal)

AI-Native Engineering Leader · 25 years in SDLC · Building production-grade agent systems with LangGraph, CrewAI, and circuit breakers · ex-Microsoft · WBD

Engineering leader with 25 years building CI/CD platforms, developer experience tooling, and cloud infrastructure at Warner Bros. Discovery, Microsoft, and Nordstrom. Currently building production-grade AI agent systems — circuit breakers, model routing, cost boundaries, and reliability patterns for LLM agent loops. Author of ai-loopguard, an open-source circuit breaker for agent loops.

## Responses 1

Markdown supported · ⌘↵ to post

[Kartik N V J K](https://hashnode.com/@kartik-nvj)

[Sep 18](https://hashnode.com/posts/why-agent-evaluation-is-harder-than-model-evaluation/6a6d5203846eddf6cc00a19b/comment/6aada5e6216271e6fa7fc235)

The integration realism point is what separates useful evals from academic exercises. An agent that passes every unit test in a sandbox can still fail the moment it touches a real API with rate limits and stale data. The hardest part is not writing the eval, it is building a test environment that is realistic enough to matter. What is your approach to mocking external dependencies?
