Mmarcuscheninvoicelatency.hashnode.dev·3d ago · 9 min readA month of failed calls, and my eval had the same name for all of themI spent a Monday morning sorting a spreadsheet that could not be sorted. Four hundred and eleven calls from the previous month had come back below our threshold. I wanted them grouped, because I had o00
Mmarcuscheninvoicelatency.hashnode.dev·4d ago · 6 min readThe voice A/B test that picked the worse agent, and won by 4 pointsWe ran a clean A/B test between two versions of a phone agent. Variant B won by 4 points on our success metric. We shipped B. Two weeks later the escalation rate to human agents had gone up, and the "00
Mmarcuscheninvoicelatency.hashnode.dev·Jul 26 · 5 min readOur voice agent scored 91 percent. The callers still hung up angry.Two weeks after we launched the support line, the dashboard was the color you want. Task-success rate: 91 percent. The agent booked the appointment, reset the password, quoted the balance. Green acros00
Mmarcuscheninvoicelatency.hashnode.dev·Jul 24 · 7 min readThe 1.8 seconds after "wait": the week our voice agent refused to stop talkingThe recording that finally made me understand the problem was eleven seconds long. A woman calls in to move a dentist appointment. She says "yeah so I need to push my Thursday." The agent starts readi00
Mmarcuscheninvoicelatency.hashnode.dev·Jul 19 · 16 min readA caller told our voice agent to ignore its instructions, and it did. The guardrail that fixed it had a 20 millisecond budget.Real time safety on a phone call is a latency problem before it is a safety problem, and most guardrail writeups forget that. Here is the incident, the tools I weighed, and what I shipped. TL;DR. A ca00
