Discussion on "How I Route Every Claude Code Prompt to the Right Model"

Satya Prakash · 2026-10-05T08:36:32.470Z

A small Python hook, three subagents and a CLAUDE.md file. Full code below, plus the parts that still don't work well. TL;DR I used to run one model for everything. That meant I either overpaid for si

Biasing to the stronger model on close scores is the right asymmetry, since over-spending on Opus just costs money while under-routing a hard debug costs you a wrong answer you trust. The part I would instrument is exactly the misroutes you list at the end: keyword matching drifts, and without logging routing decisions against outcomes you cannot tell if the router is actually saving anything. Are you tracking how often the deep tier gets a prompt Haiku would have nailed?

Agreed on the asymmetry. That's why close calls go up a tier: overspending costs money, but a confidently wrong debug costs much more.

To answer your question: no, not yet. Right now I only know which tier a prompt was sent to, not whether a cheaper model would have handled it. So I can't yet say how much the router saves.

My plan is to log every routing decision (prompt hash, scores, tier, escalated or not) to a JSONL file. Then I'll take a sample of prompts that went to Opus, re-run them on Haiku and Sonnet, and compare the answers. That shows how often Opus was overkill, and turns "it probably saves money" into an actual number.

Routing by prompt score is smart, but the costly failures are misroutes in one direction: a hard refactor sent to the cheap tier comes back confidently shallow and costs more in rework than the big model would have. Do you log the cases where you escalated manually after a cheap answer, and feed those back into the scoring rules?

iin1005h23

Yes, that's the failure that matters most. A shallow answer you trust costs more in rework than Opus would have.

Today there are two guards. Claude can move a task up a tier, and Haiku and Sonnet are told to stop and hand off if the work turns out harder than it looked. But I don't log manual escalations yet, so nothing feeds back into the scoring.

That's the next step. Every escalation will be logged with the original prompt and both tiers: automatic hand-offs, Claude moving a task up, and me re-running a task on a stronger model. Each one becomes a labelled example ("this prompt needed deep"). They go into a regression test set, and I'll tune the rules against that set rather than changing weights on gut feel.

Search Hashnode

How I Route Every Claude Code Prompt to the Right Model

Responses(4)