Discussion on "The Tests Passed. The Architecture Didn't."

Vlad Zoff · 2026-09-23T06:16:31.861Z

What happens when AI coding agents can implement a feature correctly but still change the architecture around it?

Guard's baseline accept --reason is close to what we landed on for a file our agent keeps growing: going over its size budget only passes if the same commit raises the budget with a written reason. A budget well above the actual size fails too, a blunter take on your "gradually tighten things".

Yeah, that's pretty much the kind of behavior I want from a baseline. Existing debt is one thing, but a change should have to explain why it's adding more. Having the exception and the reason in the same change also makes the later review much easier.

Agreed on the later review. For us the reason gets written while the author still remembers why; by the time it reaches review, does it usually settle things or start a discussion?

Observability without traces is guesswork. I log tool args latency and which policy gate fired so a bad run is a five minute debug not a day.

Yeah, logging which policy gate fired is really useful. Otherwise you just know that the run failed and end up digging through the whole thing to figure out why. Tool args and latency are useful for the same reason.

This tracks with something I've run into building agent tooling for ops workflows: tests catch behavior regressions but never structural ones, and once you're gluing multiple agent-driven changes together the drift compounds quietly. The baseline idea, flag new violations rather than the whole existing debt, is the right call, trying to get a real project to zero first is a nonstarter.

One thing I didn't see addressed: what happens when an agent's change is a legitimate, deliberate refactor that's supposed to move the policy itself, not just violate it. Say a requirement genuinely calls for introducing a new adapter layer that didn't exist before, the agent's diff will look exactly like a violation to Guard (a new import crossing an unapproved boundary) even though it's the correct outcome. Does Guard have any notion of "this change is proposing a policy update" versus "this change breaks an existing rule", or does that always fall back to a human editing the policy file after the fact? That boundary between fact-checking and rule-authoring seems like the part that'll matter most once agents start doing bigger structural work, not just feature slices within an existing shape.

That’s a good distinction, and I think this is actually one of the harder parts of the problem.

In the current version of Guard, a deliberate architectural change is not treated as implicitly changing policy. If an existing rule is violated, Guard will report it even when the change may be intentional. Policy changes are a separate, explicit step.

There is a propose flow that can generate evidence-based rule and contract proposals from the project state, and those proposals still have to be explicitly approved before becoming active policy. So the model is roughly:

change → detect against current policy → decide whether this is a violation or an intentional architecture change → update/approve policy → verify again.

I think keeping that boundary explicit is important. I don't want the agent that made the refactor to be able to silently rewrite the rules it is being checked against.

The part I'm interested in exploring is making that workflow more agent-friendly: letting the agent say “this requirement intentionally changes this boundary,” then having Guard produce a bounded policy proposal for human approval rather than treating it as either a hard failure or an unrestricted policy edit.

That feels like a useful middle ground between fact-checking and rule-authoring.

That propose flow sounds like the right shape for it. The part I'd watch once this runs inside a fully agent-driven pipeline is who's actually on the other end of the approval step. If the same agent that authored the refactor is also the one framing the evidence-based proposal, approval risks becoming a second pass by the same actor rather than a real check, especially under pressure to unblock a run. Is human approval on the propose flow enforced structurally right now, a separate account or a review step the agent literally can't skip, or is it currently just a convention that happens to be a person today?

That's a fair concern. Right now Guard doesn't have a separate identity layer that proves "a human approved this". The proposal and approval are separate steps, and a proposal doesn't become active policy by itself, but the host still controls who or what is allowed to perform that approval. So in a fully autonomous setup, you're right that giving the same agent permission to approve its own proposal would defeat the point. I'd rather make that boundary explicit than pretend the tool can solve it by itself.

The Tests Passed. The Architecture Didn't.

Written by

Responses9

Search Hashnode

The Tests Passed. The Architecture Didn't.

Written by

Responses9