GAGraham Andanjeinfata-fanaka.hashnode.dev·1d ago · 5 min readModel QuantizationQuantization is a model compression technique that reduces numerical precision of weights and activations from floating-point to lower-bit representations, decreasing model size and computational cost00
GAGraham Andanjeinfata-fanaka.hashnode.dev·2d ago · 3 min readNeural Architecture SearchAutomates the process of determining optimal model configurations by systematically exploring large spaces of possible architecture to identify those that best balance accuracy, computational cost, me00
GAGraham Andanjeinfata-fanaka.hashnode.dev·4d ago · 5 min readStructured approximationsApproximation-based compression techniques restructure model representation to reduce complexity while maintaining expressive power, complementing the pruning and distillation methods discussed earlie00
GAGraham Andanjeinfata-fanaka.hashnode.dev·4d ago · 3 min readKnowledge distillationKnowledge distillation involves using a large teacher model to train a smaller student model. The student model not only learns from the correct labels but also from the teacher's output distribution.00
GAGraham Andanjeinfata-fanaka.hashnode.dev·Jul 25 · 3 min readModel PruningStructured model optimization Structured model optimization works in two key ways: Eliminating parameter redundancy. Structuring computations for efficient hardware execution through techniques like00
