Discussion on "Why My Dev Agent Only Knows One Product"

Burak Karamık · 2026-08-11T13:57:55.057Z

General-purpose coding agents demo well and fail quietly on a real backlog. So I built a narrow one: one product, one pipeline, no claim without proof. Here is what it does, how, and why. What this s

Your breakdown of the authentication piece is exactly right. The multi-service local environment setup, logging in as a user, recording the session, keeping credentials safe across runs, that's the kind of friction that sinks general-purpose agents but never shows up in demos. You solved it by narrowing scope to one pipeline. The other direction is giving the agent its own identity layer, so it carries credentials and session state as part of its runtime rather than wiring them in fresh each time. A programmable wallet and credential vault let the agent authenticate to any service without secrets in prompts or files. That kind of portable identity feels like the right direction for making agents that can truly own a ticket end to end.

Four words carrying the whole architecture: "Documentation persuades; enforcement decides." The scripts-do-the-paperwork split is the piece I want to ask about, specifically failure attribution. You list blaming your code for platform-layer behavior among the expensive early failures, and step 5 now says the agent checks the platform side before concluding the cause is ours. What does that check consist of mechanically? Layer-blame is the one judgment in your pipeline that seems hardest to script, since it requires knowing where your code's responsibility ends, and a wrong platform-blame produces the evidenced-but-misdirected ticket, which is your confident-wrongness failure wearing better clothes. Also want to name the design decision most pipelines miss: can't-reproduce as a first-class terminal state. An agent permitted to stop without a fix, on the record, is the difference between a backlog tool and a plausible-fix generator.

It's measurement, not judgment. In my case platform is queryable, so the agent asks it the same question the app asks, then asks again with one thing changed — same record, different user, or the same data without the filter the screen applies. If it's already wrong before our code touches it, the answer is obvious. That's what stops a misdirected ticket, not the diagnosis itself so it is basically re-try loop. Unfortunetely it isn't bulletproof. I hit a case where the platform accepted a filter and quietly ignored it, so both queries matched and "not our code" looked proven. So a probe has to show it could actually see what it claims to check. And agreed on can't-reproduce — it's usually a data problem, which is why the agent has to try arranging data before it's allowed to say it.

Narrowing to one product and one pipeline is the move I wish more people made, a general agent has too large a surface to ever really trust. "No claim without proof" is also what makes it evaluable: a narrow agent has a small enough action space that you can actually score whether each answer was grounded. Did the narrow scope cut your failure rate mostly by removing wrong tool calls, or by making retrieval land on the right context?

Mostly the tool surface, not the reasoning. All those tools are what make a general agent good at general work; for me they were clutter, and every extra path was another way to do the same thing slightly wrong.

Least subtle example: two browser providers installed, the wrong one quietly answering and recording nothing — runs looked fine and produced no evidence. No prompt catches that; a narrower tool surface does.

Retrieval barely applies ( for now at least ), since I don't really much do it: a short always-loaded rulebook plus a pass over what earlier runs learned. And no clean before/after number — it's observational and objective :) . What survived narrowing was environmental, like verifying against a build without the fix. That took enforcement and evidence checks, not fewer tools.

Search Hashnode

Why My Dev Agent Only Knows One Product

Responses(5)