SSathvic Kollu·Jun 17How much VRAM do you actually need to run Llama 3 or Gemma locally?The weights fit. The KV cache is what OOMs you. Here's the math for Llama 3 and Gemma 2, plus a calculator.6 min read·sathvickollu.hashnode.dev1