GAGraham Andanjeinfata-fanaka.hashnode.dev·Aug 6 · 9 min readCache EvictionThe key-value cache is an inference optimization technique that eliminates redundant recomputation of past token representations during autoregressive generation. The KV cache memory footprint scales 00
GAGraham Andanjeinfata-fanaka.hashnode.dev·Jul 31 · 5 min readModel QuantizationQuantization is a model compression technique that reduces numerical precision of weights and activations from floating-point to lower-bit representations, decreasing model size and computational cost00
GAGraham Andanjeinfata-fanaka.hashnode.dev·Jul 30 · 3 min readNeural Architecture SearchAutomates the process of determining optimal model configurations by systematically exploring large spaces of possible architecture to identify those that best balance accuracy, computational cost, me00
GAGraham Andanjeinfata-fanaka.hashnode.dev·Jul 28 · 5 min readStructured approximationsApproximation-based compression techniques restructure model representation to reduce complexity while maintaining expressive power, complementing the pruning and distillation methods discussed earlie00
GAGraham Andanjeinfata-fanaka.hashnode.dev·Jul 27 · 3 min readKnowledge distillationKnowledge distillation involves using a large teacher model to train a smaller student model. The student model not only learns from the correct labels but also from the teacher's output distribution.00
