Mmajia1faninmajiafan.hashnode.dev·1h ago · 4 min readYou can't fix what you can't see: observability for LLM gatewaysYou can't fix what you can't see: observability for LLM gateways Routing and circuit breakers keep your LLM traffic alive when a provider degrades. But none of it matters if you can't tell what's actu00
Mmajia1faninmajiafan.hashnode.dev·Aug 10 · 4 min readA practical circuit-breaker for LLM API calls in productionWhen a model provider degrades, naive retries turn a small outage into a bill explosion. Here's the circuit-breaker pattern we use to contain it. If you've ever watched a single provider's 429s cascad00
