Analytics Vidhya
Published in

Analytics Vidhya

Deep Learning Specialization Course

Course 2: Improving Deep Neural Networks: Hyperparameters tuning, Regularization and Optimization (Week 3 notes)

HyperParameter Tuning

Training the neural networks involve setting several hyperparameters. In this week we will learn to find a good setting for them.

  1. Tuning Process

Batch Normalization

One of the important ideas in deep learning is an algorithm called batch normalization that helps to train an algorithm faster.

  1. Normalizing the input features to mean zero and variance one, increases the speed similarly batch norm does that to hidden layers.
  2. Consider the scenario of covariate shift, where distribution changes with another dataset and the algorithm fail to generalize on that dataset. Similarly, in the neural network, if any we consider any hidden layer, the input values keep on changing resulting in covariate shift. Batch norm reduces the amount that the distribution of these hidden unit values shifts around.

Multiclass Classification

Generalization of logistic regression is known as softmax regression which can let us make predictions to identify more than two classes. In the below image, the outer layer will have four output units where we will try to identify the probability of each of the four classes.



Analytics Vidhya is a community of Analytics and Data Science professionals. We are building the next-gen data science ecosystem

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Madhuri Jain

Empowering ambitious mothers who are in their 20s and 30s to break through their limiting beliefs and reach their full potential.