Model Routing: Sending Easy Requests to Small Models and Hard Ones to Big ModelsYou've probably noticed that some AI features feel instant and cheap to run, while others feel slow and expensive — even inside the same product. A one-line autocomplete suggestion and a "write me a f11d ago · 8 min readnagspidey.hashnode.dev
Guardrails: Validating What Goes Into (and Comes Out of) an LLMYou ship a support chatbot on top of an LLM. Week one, someone pastes a prompt injection hidden inside a "customer complaint" and gets it to reveal your system prompt. Week two, a different user asks 5d ago · 8 min readnagspidey.hashnode.dev
Rate Limits, Retries and Fallbacks: Keeping Your AI Feature Up When the Provider Isn'tIt's 2 AM and your phone buzzes. Your AI feature — the one that's been quietly working for three weeks — is throwing errors for every single user. You pull up the logs expecting a bug in your code. ThOct 21 · 10 min readnagspidey.hashnode.dev
Observability for AI Apps: What to Log When Your Product Is a Black BoxIt's 2am and a support ticket comes in: "the AI gave a completely wrong answer and then it took forever." You open your logs to figure out what happened. You see a POST /chat request. You see a 200 OKOct 19 · 8 min readnagspidey.hashnode.dev
Structured Output: Making an LLM Return JSON You Can Actually TrustYou ask the model to "return JSON" and it mostly does. Mostly. Then one day it wraps the JSON in a markdown code fence, or adds "Sure, here's the data you asked for:" before the {, or drops a trailingOct 14 · 8 min readnagspidey.hashnode.dev
