Essential Math & Concepts for LLM InferenceBack of the envelope calculations to estimate model's GPU memory requirements & insights into HW/SW optimizations12 min readΒ·venkat.euMay 31, 2024
The power of Mechanical Sympathy in Software EngineeringExploring Locality of Reference, LMAX Disruptor & Flash Attention22 min readΒ·venkat.euApr 18, 2024
CPU & GPU - The BasicsA digestible high-level overview of what happens in The Die16 min readΒ·venkat.euApr 8, 2024
OS Error: Too many open files. Understanding file and socket descriptors.Debugging resource leakage and optimizing server configuration7 min readΒ·venkat.euMar 26, 2024