Yuanrui DonginAI³ | Theory, Practice, BusinessResBlock — A trick to impove the modelIn deep learning, it’s common that the deeper network has stronger ability, and performance is better. However, the deeper network also…3 min read·Sep 24, 2019----
Yuanrui DonginAI³ | Theory, Practice, BusinessUnderstanding DropoutWhen training deep neural networks, we always encounter two major shortcomings:3 min read·Sep 20, 2019----
Yuanrui DonginAI³ | Theory, Practice, BusinessAWD-LSTMAWD-LSTM (ASGD Weight-Dropped LSTM) is one of the most popular language models. It has been used in many top papers, and its performance…3 min read·Sep 9, 2019--1--1
Yuanrui DonginAI³ | Theory, Practice, BusinessHyper Parameter—MomentumWhen we use the SGD (stochastic mini-batch gradient descent, commonly known as SGD in deep learning) to train parameters, sometimes it…4 min read·Sep 9, 2019----
Yuanrui DonginAI³ | Theory, Practice, BusinessUsing softmax carefullyIn deep learning, softmax is a very common and important function, especially in multi-classification image recognition. Originally, it’s…3 min read·Sep 2, 2019----
Yuanrui DonginAI³ | Theory, Practice, BusinessWhy Initializing a Neural Network is Important!Generally, neural network models rely on stochastic gradient descent for model training and parameter updating. The final performance of…3 min read·Aug 20, 2019----
Yuanrui DonginAI³ | Theory, Practice, BusinessBroadcasting ApplicationFor the fastai Part 2 courses, it focous on how to rewrite the pytorch library. Therefroe, We know how the library works, and we can apply…3 min read·Aug 19, 2019----