May 2 · 10 min read · The key-value cache is the most expensive part of running a large language model — and until now, nobody had solved it without sacrificing accuracy. At ICLR 2026, Google Research published TurboQuant: a two-stage compression algorithm that reduces KV...
Join discussionApr 14 · 6 min read · Google's TurboQuant: What It Is and Why It Actually Matters The numbers are absurd. For one user running a single Llama-3.1-8B model at 128,000 tokens of context, the KV cache alone chews up 16 gigabytes of VRAM. On a GPU that might have 24GB total. ...
Join discussion
Apr 7 · 8 min read · Google published the TurboQuant paper on March 25. It's April 7. There are already five independent implementations, a llama.cpp fork running 104B parameter models on a MacBook, and an active vLLM integration effort. Google hasn't released a single l...
Join discussion
