Knowledge distillation involves using a large teacher model to train a smaller student model. The student model not only learns from the correct labels but also from the teacher's output distribution.
Markdown supported · ⌘↵ to post
No responses yet.