---
title: "Discussion on \"Benchmarking an Agent Without Lying to Yourself\" | Hashnode"
description: "Discussion on \"Benchmarking an Agent Without Lying to Yourself\". Most agent benchmarks are not wrong on purpose. They are contaminated: the task set leaked into training, the fixtures moved between runs, the."
url: "https://hashnode.com/posts/benchmarking-an-agent-without-lying-to-yourself/6ad102340c908f302bae95eb"
---

[Feed](https://hashnode.com/feed) /

Discussion

# [Benchmarking an Agent Without Lying to Yourself](https://harrisonsec.hashnode.dev/benchmarking-an-agent-without-lying-to-yourself?utm_source=hashnode&utm_medium=feed)

Most agent benchmarks are not wrong on purpose. They are contaminated: the task set leaked into training, the fixtures moved between runs, the.

Oct 15 · 7 min read · harrisonsec.hashnode.dev  
Read the article

-   [#ai](https://hashnode.com/tag/ai)
-   [#machine-learning](https://hashnode.com/tag/machine-learning)
-   [#testing](https://hashnode.com/tag/testing)

## Written by

[Harrison Guo](https://hashnode.com/@harrisonguo)

@harrisonguo · WhiteRock, CA

  
[View profile](https://hashnode.com/@harrisonguo)

Security engineer and distributed systems practitioner. Writing about systems internals, performance, security research, and AI applications. Building harrisonsec.com.

## Responses

Markdown supported · ⌘↵ to post

No responses yet.