The Advantages of DenseNet
The world of convolutional neural network architectures is quickly becoming more clustered and crowded. Most students focused on utilizing either the VGG or ResNet networks and rarely explore the other architectures. Often students believe that going above 50 layers is both unnecessary and computationally expensive. In this short article, I attempt to show the merits of abandoning the VGG or ResNet architecture and exploring the Densely Connected Convolutional Networks (DenseNet) architecture.
The DenseNet architecture was proposed in the seminal paper, “Densely Connected Convolutional Networks”[1]. This architecture resulted from the desire to improve higher layer architectures that were being developed. Specifically, improving the problem that many of the layers in high-layer networks were in a sense redundant.
The DenseNet architecture attempts to solve this problem by densely connecting all the layers. For non-experts, this means that each layer receives inputs from all the preceding layers and passes its own information to all subsequent layers, which means that the final output layer has direct information from every single layer including the very first layer. This right here is supposed to improve the problem of redundant layers.
From personal experience in my research, DenseNet architecture has several significant advantages over other architectures. First, the authors cite that their architecture beats the results of the other competing architectures in ImageNet. I can confirm this, my research in Near-Identical Images showed that the DenseNet architecture does provide the best representation of images. Secondly, the authors argue that their improved parameter efficiency makes the network easier to train. When compared to other network architectures of similar size this is true. I would advocate that the training time is in fact very competitive with the training time of some the lower-layer networks. The improved performance is certainly worth the few extra minutes of training.
I encourage the further exploration and usage of DenseNet. I will soon be providing a basic starting code of the Pre-trained network on GitHub.
References
[1] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” Proc. — 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, vol. 2017-Janua, pp. 2261–2269, 2017.
