DL : Metric Evaluations for Building a Model

Part 3 of Deep Learning Specialization

1. Single-Metric Evaluation

Formulate all of your metrics into single metric

2. Optimizing & Satisfying Metrics

Choose only 1 main-focus metric to maximize/minimize as optimizing metric

And the other metrics in acceptable range as satisfying metrics.

3. Train-Validate(Dev)-Test

Same data distribution (same data source)

splitting pattern when having small dataset (< 10k — 100k)

splitting pattern when having huge dataset (> 100k)

4. Bayes Error - Human Error

5. Bias — Variance

6. Train-TrainDev or Dev-Test

Different set of data distribution

7. DataDismatch — Overfitting To Dev

8. Improve Model Performance

8.1 Error analysis

Find cause of error by jumping into your data itself.

8.2 Orthogonalization

Improve model performance using clear, direct-to-the-point hyperparameters.

Bias (Human error — Train error)

• Bigger model
• Learning rate & Optimizer

Variance (Train error — Dev / TrainDev error)

• Bigger validate data
• Regularization

Related Articles

• Neural Network — Part 1.1
• Neural Network — Part 1.2
• Neural Network — Part 2
• Neural Network — Part 3
• CNN — Part 4.1
• CNN — Part 4.2
• RNN — Part 5.1
• RNN — Part 5.2

Reference

Deep Learning Specialization: Structuring Machine Learning Projects (Coursera)(Youtube)