Deep Learning Best Practices (1) — Weight Initialization

Neerja Doshi
Mar 26, 2018 · 7 min read

Basics, weight initialization pitfalls & best practices

https://pixabay.com/photo-1600668/

Motivation

As a beginner at deep learning, one of the things I realized is that there isn’t much online documentation that covers all the deep learning tricks in one place. There are lots of small best practices, ranging from simple tricks like initializing weights, regularization…