Search posts, tags, users, and pages
The weights fit. The KV cache is what OOMs you. Here's the math for Llama 3 and Gemma 2, plus a calculator.
Sathvic Kollu
@sathvickollu · Bengaluru, India
Markdown supported · ⌘↵ to post
No responses yet.