Home
Feed

Extend

Hashnode Pro
Hashnode for Agents

Resources

The foreword
Hashnode gql skill

Buildful - AI coding agent that delivers frontier-level results at a fraction of the cost Bug0 - The AI-native e2e QA regression testing Hackathons Changelog Brand @hashnode on X Hashnode on LinkedIn Support - hello+support@hashnode.com Code of Conduct Terms Privacy Sitemap

Search Hashnode

Search posts, tags, users, and pages

Discussion on "The agent said "all tests pass." The test folder was 70% smaller." | Hashnode

Feed/

Discussion

Abdul Wasay

Full-stack engineer. I study how AI coding agents fake "done" and build tools that catch it.

Sep 24

The agent said "all tests pass." The test folder was 70% smaller.

A coding agent was asked to port a library from TypeScript to Go. It came back with a short report: all tests pass It was true, in a way. The tests that were left did pass. But in the maintainer's w

wasaybuilds.hashnode.dev5 min read

Responses2

Markdown supported · ⌘↵ to post

B·

Brian · AI News

I like AI

Sep 24

Trust checklists fail when they stay abstract. I score an AI tool on one real repo task with a known answer before I let it near production code.

Yeah, that's a solid way to do it. Funny thing though, the agent in this story basically passed its known answers too. It just memorised them, and it all fell apart when he added one new test case. So now I tweak the task a bit after it passes, just to see if it actually holds up.

Abdul Wasay

Full-stack engineer. I study how AI coding agents fake "done" and build tools that catch it.

Sep 24