Yuanrui DonginAI³ | Theory, Practice, BusinessResBlock — A trick to impove the modelIn deep learning, it’s common that the deeper network has stronger ability, and performance is better. However, the deeper network also…Sep 24, 2019Sep 24, 2019
Yuanrui DonginAI³ | Theory, Practice, BusinessUnderstanding DropoutWhen training deep neural networks, we always encounter two major shortcomings:Sep 20, 2019Sep 20, 2019
Yuanrui DonginAI³ | Theory, Practice, BusinessAWD-LSTMAWD-LSTM (ASGD Weight-Dropped LSTM) is one of the most popular language models. It has been used in many top papers, and its performance…Sep 9, 20191Sep 9, 20191
Yuanrui DonginAI³ | Theory, Practice, BusinessHyper Parameter—MomentumWhen we use the SGD (stochastic mini-batch gradient descent, commonly known as SGD in deep learning) to train parameters, sometimes it…Sep 9, 2019Sep 9, 2019
Yuanrui DonginAI³ | Theory, Practice, BusinessUsing softmax carefullyIn deep learning, softmax is a very common and important function, especially in multi-classification image recognition. Originally, it’s…Sep 2, 2019Sep 2, 2019
Yuanrui DonginAI³ | Theory, Practice, BusinessWhy Initializing a Neural Network is Important!Generally, neural network models rely on stochastic gradient descent for model training and parameter updating. The final performance of…Aug 20, 2019Aug 20, 2019
Yuanrui DonginAI³ | Theory, Practice, BusinessBroadcasting ApplicationFor the fastai Part 2 courses, it focous on how to rewrite the pytorch library. Therefroe, We know how the library works, and we can apply…Aug 19, 2019Aug 19, 2019