Jjasmineparkinjas-blogs.hashnode.dev·1d ago · 5 min readCost per action: the number your LLM spend dashboard cannot produceTL;DR. Cost dashboards for LLM systems report spend per model, per provider, per thousand calls. Finance asks what the support copilot costs per resolved ticket, and the dashboard cannot answer becaus00
Jjasmineparkinjas-blogs.hashnode.dev·4d ago · 6 min readOne tenant, one polling loop, and $9,100 we didn't budget for The page that mattered never fired. Finance sent it instead, on a Thursday, in Slack: "Projected LLM spend this month is 41% over. Is that expected?" It was not expected. It took me four hours to find00
Jjasmineparkinjas-blogs.hashnode.dev·Jul 17 · 18 min readOur LLM service had no backpressure. The provider got 7x slower and our p99 got 25x worse.TL;DR. Our summarization endpoint holds a p99 under 3 seconds. On 21/05 our provider degraded: median call time went from 620ms to about 4.3 seconds, roughly 7x. Our p99 went to 35 seconds, roughly 2500
Jjasmineparkinjas-blogs.hashnode.dev·Jul 14 · 7 min readSelf-hosted LLM observability: six stacks, weighed by what they cost you to run TL;DR: If your LLM traffic sends personal data, or your obs invoice scales with token volume, self-hosting the trace pipeline is worth costing out. Storage is the cheap part (roughly 60 GB and about 100
Jjasmineparkinjas-blogs.hashnode.dev·Jul 14 · 13 min readA $3,900 overnight bill from our LLM eval suite: the incident, and the spend guard I shipped after TL;DR. Our LLM-judge eval suite had no cost ceiling. It ran the full judge over 1,200 cases on every CI trigger. On 08/07 a dependency bot opened 41 pull requests between 01:00 and 04:00, and our merg00
