---
title: "#llm-evaluation articles | Hashnode"
description: "Explore 44 articles tagged #llm-evaluation on Hashnode. Tutorials, guides, and how-tos from the community."
url: "https://hashnode.com/tag/llm-evaluation"
---

Tag

# #llm-evaluation

44 posts · 0 followers

[AgentOps Lab](https://hashnode.com/@moneyprinterops) · [2d ago](https://hashnode.com/posts/eval-case-brief-before-you-score-the-agent-run/6af67af723037bb26d25a175)

### [Write an Eval Case Brief Before You Score the Agent Run](https://moneyprinterops.hashnode.dev/eval-case-brief-before-you-score-the-agent-run?utm_source=hashnode&utm_medium=feed)

Don't dump a failure into a blank chat and ask for a score. Write an eval case brief first: one behavior, sanitized inputs, and pass/fail signals.

7 min read · [moneyprinterops.hashnode.dev](https://moneyprinterops.hashnode.dev/eval-case-brief-before-you-score-the-agent-run?utm_source=hashnode&utm_medium=feed)

[Ignacio Di Rocco](https://hashnode.com/@idirocco) · [4d ago](https://hashnode.com/posts/resume-coach-pipeline-from-20-to-90-schema-validation-and-still-produced-questionable-resumes/6af3c7109eaeb3693740f74b)

### [Resume Coach pipeline, from 20% to 90% schema validation (and still produced questionable resumes)](https://idirocco.hashnode.dev/resume-coach-pipeline-from-20-to-90-schema-validation-and-still-produced-questionable-resumes?utm_source=hashnode&utm_medium=feed)

Generating Valid Synthetic Data Doesn't Mean Generating Good Data

5 min read · [idirocco.hashnode.dev](https://idirocco.hashnode.dev/resume-coach-pipeline-from-20-to-90-schema-validation-and-still-produced-questionable-resumes?utm_source=hashnode&utm_medium=feed)

[Ignacio Di Rocco](https://hashnode.com/@idirocco) · [Nov 3](https://hashnode.com/posts/why-generic-synthetic-examples-fail-in-category-specific-scenarios/6aea14df5ea170c1ad922669)

### [Why Generic Synthetic Examples Fail in Category-Specific Scenarios](https://idirocco.hashnode.dev/why-generic-synthetic-examples-fail-in-category-specific-scenarios?utm_source=hashnode&utm_medium=feed)

A targeted generator-prompt change, what 93.6% actually means, and lessons from AI Waiter

4 min read · [idirocco.hashnode.dev](https://idirocco.hashnode.dev/why-generic-synthetic-examples-fail-in-category-specific-scenarios?utm_source=hashnode&utm_medium=feed)

[Ignacio Di Rocco](https://hashnode.com/@idirocco) · [Oct 19](https://hashnode.com/posts/a-passing-eval-doesn-t-mean-your-ai-feature-is-reliable/6ad65d7184992924134251fd)

### [A Passing Eval Doesn't Mean Your AI Feature Is Reliable](https://idirocco.hashnode.dev/a-passing-eval-doesn-t-mean-your-ai-feature-is-reliable?utm_source=hashnode&utm_medium=feed)

How I think about evaluation sets, failure analysis, and the gap between a good score and a dependable AI product.

8 min read · [idirocco.hashnode.dev](https://idirocco.hashnode.dev/a-passing-eval-doesn-t-mean-your-ai-feature-is-reliable?utm_source=hashnode&utm_medium=feed)

[Ignacio Di Rocco](https://hashnode.com/@idirocco) · [Oct 15](https://hashnode.com/posts/my-llm-judge-went-from-70-to-96-agreement-with-humans-here-s-what-i-learned/6ad0d9d3385c982bf108df04)

### [My LLM Judge Went From 70% to 96% Agreement With Humans. Here’s What I Learned.](https://idirocco.hashnode.dev/my-llm-judge-went-from-70-to-96-agreement-with-humans-here-s-what-i-learned?utm_source=hashnode&utm_medium=feed)

What three iterations of an LLM-as-a-judge rubric taught me about evaluation design, regression testing, and synthetic data quality.

14 min read · [idirocco.hashnode.dev](https://idirocco.hashnode.dev/my-llm-judge-went-from-70-to-96-agreement-with-humans-here-s-what-i-learned?utm_source=hashnode&utm_medium=feed)

[Rahul Sai Indeevar V](https://hashnode.com/@rahul-ai) · [Oct 14](https://hashnode.com/posts/lll-evaluation/6acf6dd3d37319a358082b5b)

### [LLL Evaluation](https://rahul-ai.hashnode.dev/lll-evaluation?utm_source=hashnode&utm_medium=feed)

Starting with the basic question, "What is Evaluation ?"Evaluation refers to the measurement of different properties in system's outputs. What sort of properties are we interested in measuring? AI as

19 min read · [rahul-ai.hashnode.dev](https://rahul-ai.hashnode.dev/lll-evaluation?utm_source=hashnode&utm_medium=feed)

[Kartik N V J K](https://hashnode.com/@kartik-nvj) · [Oct 8](https://hashnode.com/posts/i-captured-a-hundred-agent-failures-a-week-and-fixed-three-the-bottleneck-was-curation/6ac7e31b10b9753715266452)

### [I captured a hundred agent failures a week and fixed three. The bottleneck was curation.](https://kartiknvjk.hashnode.dev/i-captured-a-hundred-agent-failures-a-week-and-fixed-three-the-bottleneck-was-curation?utm_source=hashnode&utm_medium=feed)

I had full traces and an eval set that was mostly the same bug on repeat. Here is the curation discipline that fixed it.

5 min read · [kartiknvjk.hashnode.dev](https://kartiknvjk.hashnode.dev/i-captured-a-hundred-agent-failures-a-week-and-fixed-three-the-bottleneck-was-curation?utm_source=hashnode&utm_medium=feed)

[Muhammad Azlaan Zubair](https://hashnode.com/@mdazlaanzubair) · [Sep 7](https://hashnode.com/posts/gpt-6-astra-alignment-is-not-authorization/6a9eea6e2e55493143d32421)

### [GPT-6 Astra Is More Aligned. I Still Wouldn’t Trust It With More Autonomy.](https://letinix.hashnode.dev/gpt-6-astra-alignment-is-not-authorization?utm_source=hashnode&utm_medium=feed)

Astra appears better at respecting boundaries. That is not the same as giving the model authority over those boundaries.

10 min read · [letinix.hashnode.dev](https://letinix.hashnode.dev/gpt-6-astra-alignment-is-not-authorization?utm_source=hashnode&utm_medium=feed)

[Muhammad Azlaan Zubair](https://hashnode.com/@mdazlaanzubair) · [Sep 5](https://hashnode.com/posts/the-model-is-not-the-architecture/6a9c24a1f32d22f8292de7ce)

### [The Model Is Not the Architecture](https://letinix.hashnode.dev/the-model-is-not-the-architecture?utm_source=hashnode&utm_medium=feed)

DeepMind's specialized systems show where reliable AI capability actually comes from

6 min read · [letinix.hashnode.dev](https://letinix.hashnode.dev/the-model-is-not-the-architecture?utm_source=hashnode&utm_medium=feed)

[Adam Jaber](https://hashnode.com/@adamjaber) · [Sep 3](https://hashnode.com/posts/why-does-ai-forget-context-window-explained/6a98d471e6b8e6a495f61c8e)

### [Why AI Forgets What You Said — and Why a Bigger Context Window Won't Fix It](https://simplyexplained.hashnode.dev/why-does-ai-forget-context-window-explained?utm_source=hashnode&utm_medium=feed)

AI has no memory — it re-reads your whole chat each turn, and even that has a poorly-attended dead zone in the middle. Here's why a bigger context window won't save you, and the skill that will.

8 min read · [simplyexplained.hashnode.dev](https://simplyexplained.hashnode.dev/why-does-ai-forget-context-window-explained?utm_source=hashnode&utm_medium=feed)

[3](https://hashnode.com/posts/why-does-ai-forget-context-window-explained/6a98d471e6b8e6a495f61c8e)
