---
title: "Comment by Kartik N V J K on \"How to Set Up Evals for a Complex Multi-Agent System\" | Hashnode"
description: "Handoffs are where I lose the most time, especially the 'lost uncertainty' case you name: a specialist returns a hedged, evidence-light recommendation and the c"
url: "https://hashnode.com/posts/how-to-set-up-evals-for-a-complex-multi-agent-system/6ac46769848e3306fd9c5c03/comment/6ac6966a95ff0ef484394708"
---

[Discussion](https://hashnode.com/posts/how-to-set-up-evals-for-a-complex-multi-agent-system/6ac46769848e3306fd9c5c03)

Comment [How to Set Up Evals for a Complex Multi-Agent System](https://hashnode.com/posts/how-to-set-up-evals-for-a-complex-multi-agent-system/6ac46769848e3306fd9c5c03)

[Kartik N V J K](https://hashnode.com/@kartik-nvj)

[Oct 7](https://hashnode.com/posts/how-to-set-up-evals-for-a-complex-multi-agent-system/6ac46769848e3306fd9c5c03/comment/6ac6966a95ff0ef484394708)

Handoffs are where I lose the most time, especially the 'lost uncertainty' case you name: a specialist returns a hedged, evidence-light recommendation and the coordinator reads it as settled fact. Freezing the specialist reports so rival coordinators get identical input is the step I would have skipped. When you grade a handoff, do you assert on the preserved caveats directly, or only on whether the final plan stayed within authority?

[Gary Stafford](https://hashnode.com/@garystafford)

[Nov 10](https://hashnode.com/posts/how-to-set-up-evals-for-a-complex-multi-agent-system/6ac46769848e3306fd9c5c03/comment/6ac6966a95ff0ef484394708#reply-6af368f36f6fdf3b91b9a6b1)

Thanks, Kartik. We checked both. In the Mars replay, each Director configuration had access to the same recorded specialist reports. The deterministic checks verified that the Director proposed an allowed set of actions and submitted it for approval. The semantic rubric separately checked whether the Director distinguished projections from observations and disclosed material shortfalls.

For a dedicated handoff test, I’d make the caveat explicit in the fixture. Say a specialist reports normal air-processing throughput but says cabin air quality is unmeasured. I’d check that the Director carries that uncertainty into the plan or resolves it with new evidence. A plan can stay within its authority and still overstate what the team knows.
