---
title: "Sai Ram Kotha (@sairamkotha) | Hashnode"
description: "AI, Mobile. Read the latest articles by Sai Ram Kotha on Hashnode."
url: "https://hashnode.com/@sairamkotha"
---

@sairamkotha

# Sai Ram Kotha

@sairamkotha · Bengaluru, India · Joined October 2026

AI, Mobile

## Comments

[Good catch, and I'll say plainly that the pilot doesn't prove this. A zero-test run or a stale installed copy would also exit green. I'll record discovered IDs, executed count and the resolved module path next to the hash, fail on zero tests, and keep all six per-test results in the table. Thank you, this makes the evidence about the edited code rather than the runner.](https://hashnode.com/posts/dont-trust-agent-pass-locked-test-bench/6ac84ef3a3016099157b4006/comment/6ac888743ccafbedef013db4#reply-6ac8f942ef0c02600fec0797)

Reply · Article · Oct 9 · [Don't trust the agent's PASS: a locked-test bench you can build in an hour](https://hashnode.com/posts/dont-trust-agent-pass-locked-test-bench/6ac84ef3a3016099157b4006)

[This is the missing half. The pilot only showed the bench is reproducible, not that it discriminates. A held-out file with different inputs, locked by its own hash, would turn "passes" into "generalizes", and the visible-vs-hidden gap is a better cross-model number than either score. I'll add it as the next version of the bench and say so in the post. Which inputs would you pick first for the inventory module?](https://hashnode.com/posts/dont-trust-agent-pass-locked-test-bench/6ac84ef3a3016099157b4006/comment/6ac8cd93cdb37db53d67795c#reply-6ac8f926579ef132773cd0af)

Reply · Article · Oct 9 · [Don't trust the agent's PASS: a locked-test bench you can build in an hour](https://hashnode.com/posts/dont-trust-agent-pass-locked-test-bench/6ac84ef3a3016099157b4006)
