---
title: "Discussion on \"Evals — Why a Bad Eval Is Worse Than No Eval\" | Hashnode"
description: "Discussion on \"Evals — Why a Bad Eval Is Worse Than No Eval\". The first time I ran evals on my LLM playground, the agent scored lower than basic RAG — 7.5 vs 8.9. The agent is more capable: it has live tool data on top of retrieval, it can answer questions basic"
url: "https://hashnode.com/posts/evals-why-a-bad-eval-is-worse-than-no-eval/6a2c7a4051327aaa82812867"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Evals — Why a Bad Eval Is Worse Than No Eval](https://blog.davidhahn.co/evals-why-a-bad-eval-is-worse-than-no-eval?utm_source=hashnode&utm_medium=feed)

The first time I ran evals on my LLM playground, the agent scored lower than basic RAG — 7.5 vs 8.9. The agent is more capable: it has live tool data on top of retrieval, it can answer questions basic

Jun 12 · 6 min read · blog.davidhahn.co  
Read the article

## Written by

[David Hahn](https://hashnode.com/@davidhahn)

@davidhahn · Chicago, USA

  
[View profile](https://hashnode.com/@davidhahn)

10+ years in software engineering, with deep expertise in frontend. Now going deep on LLMs — streaming, RAG, tool use, and everything in between.

Senior engineer with deep roots in React, TypeScript, and production-scale UI — including 4 years at Apple. Now focused on applied AI engineering: working hands-on with LLM APIs, RAG pipelines, agents, and the full stack of tooling that modern AI products are built on. Writing here to document the build, share what's non-obvious, and connect with teams working on hard AI problems.

## Responses 1

Markdown supported · ⌘↵ to post

[Ken](https://hashnode.com/@kenerator)

[Jun 16](https://hashnode.com/posts/evals-why-a-bad-eval-is-worse-than-no-eval/6a2c7a4051327aaa82812867/comment/6a3194b0f5fb975051c6dd9d)

Good distinction between system failure and eval failure. The next thing I’d want is to make the eval itself a versioned artifact.

Each run should carry the fixture version, case type, scorer version, scoring dimensions, judge configuration if an LLM judge is used, and a small human-audited slice. Otherwise the score can look precise while the thing being measured keeps changing.

That also makes bad evals easier to debug. You can ask whether the system regressed, the case set is unrepresentative, the scorer is applying the wrong criterion, or the production workload drifted away from the fixture set.
