Speculative DecodingSpeculative Decoding is an inference optimization technique that pairs a large target model with a lightweight draft mechanism that quickly proposes several next tokens. The target model verifies thosSep 9 · 5 min readfata-fanaka.hashnode.dev
Cache EvictionThe key-value cache is an inference optimization technique that eliminates redundant recomputation of past token representations during autoregressive generation. The KV cache memory footprint scales Aug 6 · 9 min readfata-fanaka.hashnode.dev
Model QuantizationQuantization is a model compression technique that reduces numerical precision of weights and activations from floating-point to lower-bit representations, decreasing model size and computational costJul 31 · 5 min readfata-fanaka.hashnode.dev
Neural Architecture SearchAutomates the process of determining optimal model configurations by systematically exploring large spaces of possible architecture to identify those that best balance accuracy, computational cost, meJul 30 · 3 min readfata-fanaka.hashnode.dev
Structured approximationsApproximation-based compression techniques restructure model representation to reduce complexity while maintaining expressive power, complementing the pruning and distillation methods discussed earlieJul 28 · 5 min readfata-fanaka.hashnode.dev
