IIshan Nerlekar·9h agoA second look: auditing Terminal-Bench 4.0 after the big auditI ran a cheap test-reading audit on Terminal-Bench 4.0 after a major audit. All 24 wrong solutions passed; 10 were already known, 9 were new.8 min read·thegradersaidyes.hashnode.dev1O
Ttappers·9h agoStop Running Unrelated Monorepo Test Suites on Every Agent Edit: Full-Suite Runs vs. AST Dependency-Scoped Incremental Tests (-95.7% Input Tokens)💡 AI Labo Benchmark Series — Experiment E30 (Feedback Sensors & Incremental Verification) This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-i6 min read·tappers.hashnode.dev
Ttappers·9h agoDeduplicating Cascading Test Failures for Coding Agents: Raw Error Dumps vs. AST Root-Cause Error Clustering (-95.3% Input Tokens)💡 AI Labo Benchmark Series — Experiment E29 (Feedback Sensors & Error Deduplication) This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolat7 min read·tappers.hashnode.dev
Ttappers·9h agoBreaking A➔B➔A Edit Oscillations in Coding Agents: Unguarded Retry Loops vs. a Loop-Detection Circuit Breaker (-94.9% Input Tokens)💡 AI Labo Benchmark Series — Experiment E28 (Feedback Sensors & Loop Circuit Breakers) This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isol7 min read·tappers.hashnode.dev
Ttappers·9h agoStop Your Coding Agent From Cycling Through Refuted Guesses: Free-Text Monologue vs. a Structured Hypothesis Matrix (2.54x Faster)💡 AI Labo Benchmark Series — Experiment E27 (Structured Reasoning & Diagnostic Scratchpads) This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network7 min read·tappers.hashnode.dev
Ttappers·9h agoWhy Claude Code's Plan Mode Works: Interleaved Read-Write Loops vs. a Two-Phase Read-Only Plan Lock (-94.8% Input Tokens)💡 AI Labo Benchmark Series — Experiment E26 (Orchestration & Phase Gates) This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox7 min read·tappers.hashnode.dev
Ttappers·1d agoStop Invalidating Your Coding Agent's KV Cache: Dynamic System Prompt Mutation vs. Append-Only Prefix-Stable Prompt Caching (2.63x Faster)💡 AI Labo Benchmark Series — Experiment E25 (Context Hygiene & KV-Cache Architecture) This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isola7 min read·tappers.hashnode.dev
Ttappers·1d agoDefeating the Pink Elephant Effect in Coding Agents: 25+ Negative 'NEVER DO X' Rules vs. 1 Positive Golden Code Template (2.85x Faster)💡 AI Labo Benchmark Series — Experiment E24 (Prompt Hygiene & Affirmative Guides) This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated8 min read·tappers.hashnode.dev
Ttappers·1d agoStopping Edit-Test Thrashing Across Coupled Modules: Sequential Single-File Edits vs. Atomic Multi-File Batches (2.26x Faster)💡 AI Labo Benchmark Series — Experiment E23 (Edit Primitives & Cross-Module Transactions) This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-i7 min read·tappers.hashnode.dev
Ttappers·1d agoFail-Before-Fix in AI Coding Agents: Speculative Patching vs. a Reproduction-First Gate (-95.3% Input Tokens, 100% Accuracy)💡 AI Labo Benchmark Series — Experiment E22 (Verification Gates & Bugfix Workflows) This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolate9 min read·tappers.hashnode.dev
