KNKartik N V J Kinkartiknvjk.hashnode.dev路Sep 17 路 4 min readMy CI evals were green. A regression still paged me at 3 AM.The eval that finally caught a bad regression for me was not the one in CI. CI had run twelve hours earlier, against a frozen test set, and passed. The regression was sitting in live production traffi00
KNKartik N V J Kinkartiknvjk.hashnode.dev路Sep 2 路 8 min readLlamaIndex makes RAG easy to build and hard to debug. Here is how I evaluate it.I built a RAG app on LlamaIndex in about four lines. Wire an index to a query engine, point it at my documents, ask a question, get an answer. The first hundred queries were great. I was impressed wit11K
KNKartik N V J Kinkartiknvjk.hashnode.dev路Aug 26 路 6 min readAWS Bedrock's built-in eval graded my agent green. It only looks at a quarter of it.I built an agent on AWS Bedrock the low-code way. Pick a foundation model, wire an action group to a Lambda, attach a Knowledge Base, add a Guardrail. Then I ran Bedrock's built-in Model Evaluation jo00
KNKartik N V J Kinkartiknvjk.hashnode.dev路Aug 26 路 5 min readI uninstalled three AI coding CLIs in a week. The model was never the problem.I tried a bunch of terminal coding agents last month. Some I reopened every morning. Some I uninstalled by Friday. What surprised me is that the ones I dropped were not worse at writing code. The mode00
KNKartik N V J Kinkartiknvjk.hashnode.dev路Aug 21 路 8 min readI Built an Agent Harness by Hand. The Loop Was Tiny. The Five Bugs Were Not.I built an agent harness by hand before I reached for a framework, because I wanted to see the thing a framework hides. The loop turned out to be smaller than every tutorial makes it look, small enoug00