---
title: "Comment by Ahmet Özel on \"Don't trust the agent's PASS: a locked-test bench you can build in an hour\" | Hashnode"
description: "Locking the test file protects the assertions, but the independent rerun should also prove that all six intended tests actually executed against the edited inve"
url: "https://hashnode.com/posts/dont-trust-agent-pass-locked-test-bench/6ac84ef3a3016099157b4006/comment/6ac888743ccafbedef013db4"
---

[Discussion](https://hashnode.com/posts/dont-trust-agent-pass-locked-test-bench/6ac84ef3a3016099157b4006)

Comment [Don't trust the agent's PASS: a locked-test bench you can build in an hour](https://hashnode.com/posts/dont-trust-agent-pass-locked-test-bench/6ac84ef3a3016099157b4006)

[Ahmet Özel](https://hashnode.com/@ahmetozel)

[Oct 9](https://hashnode.com/posts/dont-trust-agent-pass-locked-test-bench/6ac84ef3a3016099157b4006/comment/6ac888743ccafbedef013db4)

Locking the test file protects the assertions, but the independent rerun should also prove that all six intended tests actually executed against the edited inventory module. A green process exit can still come from empty discovery or importing an installed copy instead of the fresh workspace.

I would record discovered test IDs, executed count and the resolved module path alongside the test hash. Make a zero-test run fail the bench, and retain the six per-test outcomes rather than only Yes in the pilot table. Those checks preserve your small reproducible setup while making its PASS evidence about the intended implementation rather than just the test runner's exit status.

[Sai Ram Kotha](https://hashnode.com/@sairamkotha)

[Oct 9](https://hashnode.com/posts/dont-trust-agent-pass-locked-test-bench/6ac84ef3a3016099157b4006/comment/6ac888743ccafbedef013db4#reply-6ac8f942ef0c02600fec0797)

Good catch, and I'll say plainly that the pilot doesn't prove this. A zero-test run or a stale installed copy would also exit green. I'll record discovered IDs, executed count and the resolved module path next to the hash, fail on zero tests, and keep all six per-test results in the table. Thank you, this makes the evidence about the edited code rather than the runner.

[Ahmet Özel](https://hashnode.com/@ahmetozel)

[Oct 11](https://hashnode.com/posts/dont-trust-agent-pass-locked-test-bench/6ac84ef3a3016099157b4006/comment/6ac888743ccafbedef013db4#reply-6acb581728d3a78c4dbd562e)

Recording all six outcomes will make the revised table much easier to audit. I would count skipped and expected-failure cases separately from executed passes, so collecting six tests cannot accidentally be reported as six checks of the implementation.

One cheap positive control is to run the bench against the original known-broken module and require the intended failures before accepting any repaired run. That verifies the locked assertions can still detect the six bugs with the current runner and import configuration.
