ResNet (2015) | one minute summary
ResNets remain extremely relevant after 5.5 years
Published in
2 min readMay 15, 2021
He et al. introduced ResNet in their groundbreaking paper in late 2015, “Deep Residual Learning for Image Recognition”, and since then pretty much every new image classification model compares their results with ResNet or its variants.
Prerequisite knowledge: Convolutional neural networks, the vanishing gradient problem
- Why? Theoretically, making convolutional neural networks deeper is beneficial for image classification tasks, but plain networks that are too deep have a training accuracy degradation problem caused by vanishing gradients.
- What? This paper introduces a deep residual learning framework using unhindered skip/shortcut connections to avoid the vanishing gradient problem.
- How? In each residual learning building block, the input is split into two paths (one path that goes through all learning layers and another that is able to skip some of the layers) which are element-wise summed at the output. A series of residual learning blocks effectively simplifies the network to reduce the effect of vanishing gradients (as there are fewer layers to propagate through) and also constrains the skipped layers to learn a desired feature space (F(x)) based on the existing mapping (H(x)).
Related articles:
