Software geek and founder of Privatrak
Software geek and founder of Privatrak: no-code, cookieless product analytics. Writing about what I learn building it.
Markdown supported · ⌘↵ to post
Your runs table already measures something most agent setups can't: how often people take the agent's work back. The share of runs undone within a couple of weeks is close to what DORA calls change fail rate, the ratio of deployments that need immediate intervention, usually a rollback or a hotfix. Split by client and by tool, it would show whether one assistant gets funnels wrong more often than rules, which tells you where the server's instructions need work.
I'd count the quiet fixes too. A setup someone repaired by hand never shows up as an undo, but your conflict check already spots it: a human edit to a field a run set. Tracking both keeps a low undo rate from hiding a lot of manual rework.
Yeah, we've been thinking about that too, change fail rate is a good way to put it. Counting the quiet fixes is a good catch, the undo rate alone would look better than reality.
Kartik N V J K
Making the preview the real undo, rolled back, is the line I'm taking with me. A separate preview path would drift from the real one sooner or later. My worry is from the agent side: if someone undoes a run mid-session, the agent still believes its funnel exists. When its next write gets refused for joining an undone run, does the error say why, so it doesn't just open a fresh run and put everything back?