EXP — Hyperparameter Tuning

So I found myself in what I imagine to be a fairly common scenario, I have a single machine with multiple cores or GPU units, and I want to learn something about a machine learning model. I either want to find the parameters to obtain the best possible model for a particular dataset, or explore how a specific parameter affects the model performance.

You now face the harsh reality of exploratory analysis and hyperparameter optimization. It is not enough to play around with some parameters and use intuition to pick the best learning rates, dropout probabilities, number of neurons, and…