A Summary from: How To Improve Deep Learning Performance

A summary from reading the post by Jason Brownlee post on machinelearningmastery.com. Here’s the original link:

The gains often get smaller the further down the list. For example, a new framing of your problem or more data is often going to give you more payoff than tuning the parameters of your best performing algorithm. Not always, but in general.

Improve Performance With Data

  1. Get More Data.
  2. Invent More Data.
  3. Rescale Your Data.
  4. Transform Your Data.
  5. Feature Selection.

Improve Performance With Algorithms.

  1. Spot-Check Algorithms.
  2. Steal From Literature.
  3. Resampling Methods.

Improve Performance With Algorithm Tuning.

  1. Diagnostics.
  2. Weight Initialization.
  3. Learning Rate.
  4. Activation Functions.
  5. Network Topology.
  6. Batches and Epochs.
  7. Regularization.
  8. Optimization and Loss.
  9. Early Stopping.

Improve Performance With Ensembles.

In fact, you can often get good performance from combining the predictions from multiple “good enough” models rather than from multiple highly tuned (and fragile) models.

  1. Combine Models.
  2. Combine Views.
  3. Stacking.