Sitemap
Nerd For Tech

NFT is an Educational Media House. Our mission is to bring the invaluable knowledge and experiences of experts from all over the world to the novice. To know more about us, visit https://www.nerdfortech.org/.

Follow publication

Review — Model Distillation: Distilling the Knowledge in a Neural Network (Image Classification)

6 min readFeb 28, 2021

--

Higher Temperature for Distillation
@ Medium)

Outline

1. Higher Temperature for Model Distillation

1.1. Higher Temperature for Soft Targets

1.2. The Calculation of Gradients

2. Experimental Results

2.1. MNIST

2.2. Speech Recognition

Frame classification accuracy and Word Error Rate (WER)

2.3. JFT

Classification accuracy (top 1) on the JFT development set

2.4. Soft Targets as Regularizers

Frame classification accuracy and Word Error Rate (WER)

--

--

Nerd For Tech
Nerd For Tech

Published in Nerd For Tech

NFT is an Educational Media House. Our mission is to bring the invaluable knowledge and experiences of experts from all over the world to the novice. To know more about us, visit https://www.nerdfortech.org/.

Sik-Ho Tsang
Sik-Ho Tsang

Written by Sik-Ho Tsang

PhD, Researcher. I share what I learn. :) Linktree: https://linktr.ee/shtsang for Twitter, LinkedIn, etc.

No responses yet