The pattern across all three rounds is the real finding here, every "obviously correct" hypothesis (split the big file, audit the cache key) was wrong in the same specific way: intuition pointed at the artifact's size, when the actual cost was in how many times an expensive operation got repeated across it. O(declarations × graph) collapsing to O(graph) via a shared visited set, and definitional unfolding cost hiding behind line count in Round 3, are structurally the same mistake wearing different clothes.
The cache bug is the one I'd flag as most instructive for anyone debugging CI generally: "read the timestamps before you audit the keys" is such a cheap check that gets skipped because key mismatches and eviction are the more sophisticated-sounding suspects. A composite action's internal step ordering being orthogonal to your own workflow's step ordering is exactly the kind of assumption that's invisible until you actually look at two adjacent log lines.
The PRD Loop section is the part I keep coming back to, though. "Make stopping honestly cheaper than pushing through dishonestly" reframes the whole agent-reliability problem correctly , it's not about trusting the agent to be honest, it's about removing every cheap path to a false "done" so honesty becomes the only remaining option. The miss report is the proof that worked: an agent writing "targets missed, this PR does not claim completion" and continuing on its own is only remarkable because the corner-cutting equilibrium was structurally blocked, not because the agent chose virtue unprompted.
Question on the completion audit: when it re-extracts conditions from the PRD independently and checks them against CI logs, does it ever have false negatives, flagging something as unmet when it actually was satisfied, just recorded in a place the auditor didn't check? Curious how much the auditor's own reliability had to be earned through the same kind of iteration as the main loop.
Great question — yes, false negatives happen, and the design accepts them deliberately. The auditor's verdicts distinguish unverified (evidence not found where it looked) from unmet, so the case you describe surfaces as the auditor's limitation and costs one round trip to resolve: point at the evidence, re-check. That asymmetry is intentional — a false "not done" costs a round trip; a false "done" corrupts the gate itself.
The direction we actually had to iterate on was the opposite: the auditor trusting records produced by the loop it audits (ledgers, checklists). The fix was an evidence hierarchy — "satisfied" may only be grounded in primary evidence (reading the actual code/statements, running the checks), and one mismatch between a record and reality disqualifies that whole document as evidence.
Your instinct is right: the auditor went through the same iteration as the loop — several discipline waves, plus an adversarial review that found fail-closed gaps in the auditor itself. The current rules are here if you want the details:
The "unverified vs. unmet" distinction is the key move, separating "the auditor couldn't confirm this" from "this genuinely failed" turns a false-negative into a cheap retry instead of a wrongful block. That asymmetry (round-trip cost vs. gate corruption) is the right thing to optimize for.
The evidence hierarchy detail, one mismatch disqualifying an entire document rather than just that claim, is a strict rule, but it makes sense given what it's guarding against. A partially-trustworthy record is arguably worse than an untrusted one, since it invites exactly the kind of selective trust that let the corner-cutting equilibrium happen in the first place.
Appreciate you linking the actual SKILL file, will take a look.
The pattern across all three rounds is the real finding here, every "obviously correct" hypothesis (split the big file, audit the cache key) was wrong in the same specific way: intuition pointed at the artifact's size, when the actual cost was in how many times an expensive operation got repeated across it. O(declarations × graph) collapsing to O(graph) via a shared visited set, and definitional unfolding cost hiding behind line count in Round 3, are structurally the same mistake wearing different clothes.
The cache bug is the one I'd flag as most instructive for anyone debugging CI generally: "read the timestamps before you audit the keys" is such a cheap check that gets skipped because key mismatches and eviction are the more sophisticated-sounding suspects. A composite action's internal step ordering being orthogonal to your own workflow's step ordering is exactly the kind of assumption that's invisible until you actually look at two adjacent log lines.
The PRD Loop section is the part I keep coming back to, though. "Make stopping honestly cheaper than pushing through dishonestly" reframes the whole agent-reliability problem correctly , it's not about trusting the agent to be honest, it's about removing every cheap path to a false "done" so honesty becomes the only remaining option. The miss report is the proof that worked: an agent writing "targets missed, this PR does not claim completion" and continuing on its own is only remarkable because the corner-cutting equilibrium was structurally blocked, not because the agent chose virtue unprompted.
Question on the completion audit: when it re-extracts conditions from the PRD independently and checks them against CI logs, does it ever have false negatives, flagging something as unmet when it actually was satisfied, just recorded in a place the auditor didn't check? Curious how much the auditor's own reliability had to be earned through the same kind of iteration as the main loop.