Evelyn Wyatt
Really interesting take on the idea of putting a lightweight decision layer in front of an AI agent. I especially liked the principle of “Jev chooses, code decides”—that separation makes a lot of sense when you’re dealing with actions where the cost of a wrong decision can be high.
The finance example was also a great way to make the architecture concrete. Having the model narrow down intent while leaving limits, balances, permissions, idempotency, and final confirmation to deterministic code feels like a much more realistic approach than asking an LLM to handle everything.
The cost discussion is interesting too. Saving money on routing only really matters if the extra dependency and latency don’t introduce new problems, so I appreciated that the post acknowledges that trade-off rather than presenting the decision layer as a free optimization. Great practical read!
Eval harnesses beat vibes. I keep a tiny golden set of tasks and refuse model upgrades until that set stays green. Speed without a harness is just confident failure.
This is one of the more concrete decision-layer writeups I've seen -- the fixed Choice/Score/Noul shapes are the right idea for cutting hallucination risk on format. One thing I'd want nailed down before trusting it for a money-movement gate though: format safety isn't calibration safety. A Score or Noul call can return a confident, well-formed 0.95 that's still wrong, and the piece only treats a timeout as a failure worth an explicit fallback. I'd want a low-confidence or out-of-distribution case handled the same way -- explicit "not checked," not silently trusted just because it parsed cleanly. Narrow classifiers gating write actions tend to fail quietly like that: they don't stop returning a shape, they just drift as the input distribution shifts, and nothing distinguishes "confident and right" from "confident and wrong" unless someone's actively watching calibration after launch, not just during the initial shadow-mode switch-on.
The decision-layer architecture also raises an interesting observability problem. If Jev sits between the agent and the tools, its value shouldn't be measured only by how much LLM cost it removes. You'd also want to know how often its decisions actually changed the downstream routing, how often the fallback path was triggered, and whether those decisions reduced unnecessary tool calls without increasing recovery work. That makes the decision layer measurable as an operational component rather than just a cheaper classifier. Over time, those signals could also reveal where the typed decision boundary is too narrow or where the agent is still sending cases that don't belong there.