Designing a per-user AI cost meter: idempotency, cache pricing and aliasesAI apps have a margin problem that normal SaaS metrics hide: price per user is flat, cost per user is variable. I built FiscaLLM (MIT) to join the two. Most of the work wasn't the dashboard. It was ma6h ago · 3 min readtheaicommit.hashnode.dev
Why Speculative Decoding Slows Hybrid Mamba Models at Large Batch SizesEngineer's view. This is a database that rewrote the whole table file on every transaction so it could roll back. Real databases append each change to a log, checkpoint now and then, and roll back by 6h ago · 9 min readtheaicommit.hashnode.dev
EngAI: AI News for Software Engineers, One Swipe at a TimeAI ships faster than anyone can read it. A new model on Monday, an SDK change on Tuesday, a deprecation notice on Wednesday that breaks your build in March. Most of us follow it through newsletters an1d ago · 3 min readtheaicommit.hashnode.dev
How Sorting Your Prompts Speeds Up a Mixture-of-Experts Batch JobEngineer's view. This is a batch job that touches records in arrival order. Each record needs a different index page, so the cache keeps reloading pages. Sort by key and each page loads once per batch1d ago · 9 min readtheaicommit.hashnode.dev
How to Make Your Agent Check the Records It Changed Before It Says DoneEngineer's view. This is an endpoint that returns 200 after writing the wrong value. The handler logged success, and only a look at the row shows the bug. An agent's "done" is that 200: it comes from 12KA3d ago · 9 min readtheaicommit.hashnode.dev
