Review: SPPNet —1st Runner Up (Object Detection), 2nd Runner Up (Image Classification) in ILSVRC 2014
In this story, SPPNet is reviewed. SPPNet has introduced a new technique in CNN called Spatial Pyramid Pooling (SPP) at the transition of convolutional layer and fully connected layer. This is a work from Microsoft.
In ILSVRC 2014, SPPNet has got 1st Runner Up in Object Detection, 2nd Runner Up in Image Classification, and 5th Place in Localization Task. And it got 2 papers in 2014 ECCV [1] and 2015 TPAMI [2] with over 1000 and 600 citations respectively. Thus, SPPNet is one of the worth reading deep learning papers. (Sik-Ho Tsang @ Medium)
Dataset
Classification: Over 15 millions labeled high-resolution images with around 22,000 categories. ILSVRC uses a subset of ImageNet of around 1000 images in each of 1000 categories. In all, there are roughly 1.3M/50k/100k images are used for the training/validation/testing sets
Detection: 200 categories. 450k/20k/40k images are used for the training/validation/testing sets
What are covered
- Spatial Pyramid Pooling (SPP)
- Multi-Size Training
- Full Image Representation
- Multi-View Testing
- Comparison with State-of-the-art Approaches (Classification)
- SPPNet in Object Detection
- Comparison with State-of-the-art Approaches (Detection)
1. Spatial Pyramid Pooling (SPP)
Conventionally, at the transition of conv layer and FC layer, there is one single pooling layer or even no pooling layer. In SPPNet, it suggests to have multiple pooling layers with different scales.
In the above figure, 3-level SPP is used. Suppose the conv5 layer has 256 feature maps. Then at SPP layer,
- first, each feature map is pooled to become one value (grey), thus 256-d vector is formed.
- Then, each feature map is pooled to have 4 values (green), and form a 4×256-d vector.
- Similarly, each feature map is pooled to have 16 values (blue), and form a 16×256-d vector.
- The above 3 vectors are concatenated to form a 1-d vector.
- Finally, this 1-d vector is going into FC layers as usual.
With SPP, we don’t need to crop the image to fixed size, like AlexNet, before going into CNN. Any image sizes can be inputted.
2. Multi-Size Training
With SPP, variable sizes are accepted as input, different sizes should be inputted into network to increase the robustness of the network during training.
However, for the effectiveness of the training process, only 224×224 and 180×180 images are used as input. Two networks, 180-network and 240-network are trained with shared parameters.
Authors replicated ZFNet [3], AlexNet [4] and Overfeat [5] with modifications as below (The number after it is the conv layer number):
4-level SPPNet is used here with the pyramid {6×6, 3×3, 2×2, 1×1}.
As shown above, with SPP only, the error rates have been improved for all models. With Multi-Size Training, the error rates improve further. (10-view means the 10-crop testing from [four corners + 1 center] and the corresponding horizontal flips)
3. Full Image Representation
As full image can also input into CNN with the use of SPP, authors compare full image input with only using 1 center crop input:
Top-1 error rates are all improved with full image as input.
4. Multi-View Testing
With full image support by using SPP, multi-view testing can be facilitated easily.
- Authors resize the image to 6 scales: {224, 256, 300, 360, 448, 560}
- For each scale, 18 views are generated: {1 center, 4 corners, 4 on the middle of each side} and the corresponding flips.
Thus, there are in total 96 views. - And 2 full-image views plus corresponding flips are also included.
SPPNet using OverFeat-7 obtains 9.14/9.08% Top-5 Error Rate on validation/test set which is the only one under 10% in the table.
5. Comparison with State-of-the-art Approaches (Classification)
11 models of SPPNet are used in testing. The outputs are averaged to get a more accurate prediction. This is a boosting or ensemble technique used in many CNN models such as LeNet, AlexNet, ZFNet.
8.06% error rate is obtained. Unfortunately, VGGNet and GoogLeNet have better performance with the use of deep models. Finally, SPPNet can only got 2nd runner-up in the classification task.
6. SPPNet in Object Detection
- Selective Search [6] is used to generate about 2k region proposals (bounding boxes) just like in R-CNN [7].
- The input image goes through SPPNet using ZFNet by ONLY one time.
- At the last conv layer, feature maps bounded by each region proposal is going into SPP layer then FC layer as shown below:
Compared with R-CNN, SPPNet processes the image at conv layers for only one time while R-CNN processes the image at conv layers for 2k times since there are 2k region proposal. The image below illustrates the idea:
After the FC layer for each bounding box, SVM and bounding box regressor are also needed, which is not an end-to-end learning architecture.
7. Comparison with State-of-the-art Approaches (Detection)
7.1 VOC 2007
In VOC 2007 as shown above, compared with R-CNN, SPPNet with 5 scales obtained higher mAP of 59.2%.
7.2 ILSVRC 2014
In ILSVRC 2014, SPPNet obtains 35.1% mAP and got 1st runner-up in object detection task.
Actually, Microsoft has contributed many state-of-the-art deep learning approaches in ILSVRC afterwards such as PReLUNet and ResNet. I will review them later on!
Of course, other networks would also be reviewed, please stay tuned!!!!
References
- [2014 ECCV] [SPPNet]
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition - [2015 TPAMI] [SPPNet]
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition - [2014 ECCV] [ZFNet]
Visualizing and Understanding Convolutional Networks - [2012 NIPS] [AlexNet]
ImageNet Classification with Deep Convolutional Neural Networks - [2014 ICLR] [OverFeat]
OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks - [2013 IJCV] [Selective Search]
Selective Search for Object Recognition - [2014 CVPR] [R-CNN]
Rich feature hierarchies for accurate object detection and semantic segmentation
My Reviews
- Review of ZFNet — Winner of ILSVRC 2013 (Image Classification)
- Review: AlexNet, CaffeNet — Winner of ILSVRC 2012 (Image Classification)
- Review: OverFeat — Winner of ILSVRC 2013 Localization Task (Object Detection)
- Review: R-CNN (Object Detection)
Join Coinmonks Telegram Channel and Youtube Channel get daily Crypto News
Also, Read
- Copy Trading | Crypto Tax Software
- Grid Trading | Crypto Hardware Wallet
- Crypto Telegram Signals | Crypto Trading Bot
- Best Crypto Exchange | Best Crypto Exchange in India
- Best Crypto APIs for Developers
- Best Crypto Lending Platform
- Free Crypto Signals | Crypto Trading Bots
- An ultimate guide to Leveraged Token
