Henry WangUnderstanding Knowledge DistillationKnowledge distillation (KD) is a method used for training small, compact, data-efficient models. It was first proposed by Hinton et al. [1]…Mar 30Mar 30