---
title: "Comment by AI Developer on \"Building Trusted AI Agents with Statistical Rigor\" | Hashnode"
description: "Good framing of agent evals as a statistics problem. One suggestion on the latency section: agent latency and token cost are usually heavily right-skewed, so me"
url: "https://hashnode.com/posts/building-trusted-ai-agents-with-statistical-rigor/6aeb9cc0d1747a206c668fcf/comment/6aec0fa54392c688c0306bd7"
---

[Discussion](https://hashnode.com/posts/building-trusted-ai-agents-with-statistical-rigor/6aeb9cc0d1747a206c668fcf)

Comment [Building Trusted AI Agents with Statistical Rigor](https://hashnode.com/posts/building-trusted-ai-agents-with-statistical-rigor/6aeb9cc0d1747a206c668fcf)

[AI Developer](https://hashnode.com/@lycore)

[Nov 5](https://hashnode.com/posts/building-trusted-ai-agents-with-statistical-rigor/6aeb9cc0d1747a206c668fcf/comment/6aec0fa54392c688c0306bd7)

Good framing of agent evals as a statistics problem. One suggestion on the latency section: agent latency and token cost are usually heavily right-skewed, so mean and Z-scores can mislead because a few slow runs inflate the standard deviation and hide themselves. Percentiles (p50/p95/p99) or a robust measure like median absolute deviation tend to be safer for flagging outliers. For quality or pass-rate metrics with small sample sizes, a bootstrap or Wilson confidence interval helps avoid reading noise as regression in a CI gate. Since the same prompt can score differently across runs, how would you set the gate threshold: a fixed pass-rate floor, or a comparison against the previous version's distribution?