---
title: "Comment by Kartik N V J K on \"Don't trust the agent's PASS: a locked-test bench you can build in an hour\" | Hashnode"
description: "Locking the test file with a SHA-256 and disqualifying any run that touches the hash is the right instinct, because the moment the agent can edit the scorer, PA"
url: "https://hashnode.com/posts/dont-trust-agent-pass-locked-test-bench/6ac84ef3a3016099157b4006/comment/6ac93c9e14c76ac32523c602"
---

[Discussion](https://hashnode.com/posts/dont-trust-agent-pass-locked-test-bench/6ac84ef3a3016099157b4006)

Comment [Don't trust the agent's PASS: a locked-test bench you can build in an hour](https://hashnode.com/posts/dont-trust-agent-pass-locked-test-bench/6ac84ef3a3016099157b4006)

[Kartik N V J K](https://hashnode.com/@kartik-nvj)

[Oct 9](https://hashnode.com/posts/dont-trust-agent-pass-locked-test-bench/6ac84ef3a3016099157b4006/comment/6ac93c9e14c76ac32523c602)

Locking the test file with a SHA-256 and disqualifying any run that touches the hash is the right instinct, because the moment the agent can edit the scorer, PASS means nothing. The n=3 honesty is good to see too, most people would have shipped a ranking. I ran into the other half of this, a suite that stayed green while the agent still broke in production, and wrote up the CI/CD I moved to: [kartiknvjk.hashnode.dev/my-agent-passed-every-che…](https://kartiknvjk.hashnode.dev/my-agent-passed-every-check-and-still-broke-production-in-an-hour-here-s-the-ci-cd-i-run-now) . Are you adding held-out hidden tests on top of the locked file?
