Pratyush Ranjan

KV Cache Optimisation in AI Inference

When we chat with an AI tool like Claude or ChatGPT and await for a response, large mathematical operations happen behind the scenes. Most of these tools that we use today have models that are built o