Senior Implementation Engineer at Atlan
I'm an engineer who builds systems and writes about what actually happens when I do. I'm currently working through the craft of AI agent engineering from the ground up - documenting the experiments, the failures, and the moments that change my mental model. No polished takes. Just the build log.
Markdown supported · ⌘↵ to post
Kartik N V J K
The finding that adding memory scored equal or worse is the useful kind of negative result, because it exposes that the cases were passing on general agent quality rather than recall. Locking a held-out split so prompt iteration cannot leak into final assessment is the discipline most eval harnesses drop first. Did judge variance stay small enough across the five providers that the deltas were trustworthy, or did you need multiple judge runs to separate signal from noise?