KNKartik N V J Kinkartiknvjk.hashnode.dev路4d ago 路 5 min readMy agent passed every check and still broke production in an hour. Here's the CI/CD I run now. A while back my team merged what looked like a harmless change: a small refactor to our agent's tool-routing prompt. It passed lint. It passed unit tests. It passed our eval gate. Within the hour, too11R
KNKartik N V J Kinkartiknvjk.hashnode.dev路Jul 28 路 7 min readHow I generate LLM test cases that actually catch bugs I got tired of hand-writing test cases for our agent, so I did the obvious 2026 thing: I pointed an LLM at our docs and asked it to generate them. In an afternoon I had a few thousand. I felt incredib00
KNKartik N V J Kinkartiknvjk.hashnode.dev路Jul 23 路 7 min readEvaluating an intent classifier: what I check beyond accuracyI built the intent router that sits at the front of our support agent. It reads what a user typed and decides which pipeline handles it. I tested it on a nice balanced set, got 92 percent accuracy, an00
KNKartik N V J Kinkartiknvjk.hashnode.dev路Jul 21 路 6 min readEvaluating LangGraph agents: the four things I score per step I had a LangGraph agent running in production and a regression set of fifty queries it passed comfortably. Task completion sat at 0.93. I spot-checked the answers and they looked right. I genuinely th00
KNKartik N V J Kinkartiknvjk.hashnode.dev路Jul 14 路 7 min readI picked a coding agent off a leaderboard. It flopped on our codebase.Last year my team had to pick a coding agent, and I volunteered to run the evaluation. I felt good about it. I pulled up the public benchmark scores, lined up the contenders, took the one at the top, 00