Epoch 1: Try It, Time It, Then Buy It

TL;DR vLLM vs llama.cpp: on a 20–24 GB card, vLLM fit 1.83 users at 32K context. llama.cpp fit ~13, because it needs 4× less KV-cache memory for Gemma 4. On AMD, four concurrent requests is the swee