Key Deep Learning Architectures: AlexNet

AlexNet [2012, paper by Krizhevsky et al.]

Main ideas

Why it is important

Brief description

ReLU nonlinearity

The benefits of ReLU (excerpt from the paper)

Local response normalization

Local response normalization formula from the paper
An example of local response normalization made in Excel by me.

Overlapping pooling

Overlapping pooling of the kind used by AlexNet. Source.

Data augmentation

Test time data augmentation

Dropout

Architecture

AlexNet architecture from paper. Color labeling is mine.

Additional readings

Return to list of architectures

Thanks for reading! If you enjoyed it, hit that clap button below and subscribe to updates on my website! It would mean a lot to me and encourage me to write more stories like this.