Thanks, Kartik. Yeah, I completely agree with this. Once the harness starts handling things like retries, context trimming, and step limits, those can definitely become failure points themselves.
I think the tricky part is figuring out what actually caused a long run to fail, whether it was the model, a tool call, or something the harness did along the way. Logging the model outputs, tool calls, retries, context changes, and step-by-step results would probably make that much easier to trace.
I’m still exploring this part of harnesses, but I think measuring and observing the harness itself is a really interesting next layer. Thanks for bringing this up.
Kartik N V J K
AI Developer | Making AI reliable, trustworthy & accessible to everyone | Active community contributor
Good demystification, the harness really is the part that decides whether an agent survives past a handful of steps. What I would add is that the harness itself needs measuring: once you have retries, context trimming, and step limits in there, each is a knob that can quietly wreck a long run. When a run falls apart at step 80, how do you localize whether it was the model or the harness that failed?