Kartik N V J K

The stale-doc grounding failure is the one that quietly wrecks these cost models, the workflow looks cheap until it's confidently reporting off outdated sources. I liked that you put a real break-even number on it rather than hand-waving the savings. How often did you refresh the grounding docs to keep the output trustworthy?

Surfacing agent state is the half everyone underbuilds, users lose trust the moment the agent goes quiet and they can't tell if it's working or stuck. Even a rough live trace of what the agent is doing changes how much people trust the output. Does AG-UI handle streaming intermediate steps, or just final state?

Breaking the harness into four concrete jobs is the clarity I wish more write-ups had. I built one by hand recently and the loop itself was tiny, but the bugs all lived in the boring glue between those jobs. I wrote up those five bugs here: https://kartiknvjk.hashnode.dev/i-built-an-agent-harness-by-hand-the-loop-was-tiny-the-five-bugs-were-not

Excessive Agency climbing to number three matches what I'm seeing, the risk moved from what the model says to what it's allowed to do. Blast radius is the right lens, a wrong answer is cheap but a wrong action with tool access is not. How are you scoping agent permissions to keep that radius small?

An offline chunk auditor is something I wish I'd had sooner, most RAG debugging happens after retrieval already failed. The six structural checks are a good start, mid-sentence splits alone caused half my early recall problems. Are you planning any semantic checks on top of the structural ones, or keeping it deterministic on purpose?

Sharp point: more detectors without release discipline just adds noise you learn to ignore. Tying eval gates to the release itself did more for my reliability than any new metric. Are you gating deploys on the eval suite, or running it after the fact?

Starting deterministic is advice I wish I had taken sooner: a rules-based router is far easier to debug than a model deciding every hop. I only reach for an LLM router when the branching genuinely cannot be enumerated. Where did you draw the line between deterministic routing and letting the model decide?

Filling the window with the right tokens beats filling it with more tokens, every time I have measured it. Past a point, extra context actively hurt because the model latched onto the wrong passage. Are you ranking by relevance before packing the window, or trimming after retrieval?

Framing eval as a security boundary is exactly right: a weak eval is an open door for a tool-calling agent. I treat the eval suite like an allowlist, testing what the agent must refuse as rigorously as what it should do. I went deeper on vetting that boundary for MCP servers here: https://kartiknvjk.hashnode.dev/how-i-evaluate-mcp-servers-for-security-before-i-trust-them — how are you testing the refuse-path without a real exploit to replay?

The cost section resonates: retrieval quality and cost pull in opposite directions once you scale the index. I found reranking let me pull fewer candidates up front without losing recall, which dropped both latency and spend. Are you caching embeddings at the query level, or only on the document side?