Locking the test file with a SHA-256 and disqualifying any run that touches the hash is the right instinct, because the moment the agent can edit the scorer, PASS means nothing. The n=3 honesty is good to see too, most people would have shipped a ranking. I ran into the other half of this, a suite that stayed green while the agent still broke in production, and wrote up the CI/CD I moved to: kartiknvjk.hashnode.dev/my-agent-passed-every-che… . Are you adding held-out hidden tests on top of the locked file?