Harnessing Human Insight: The Power of Human-in-the-Loop Optimization with Optuna (PART-2)

Preferential Optimisation using Optuna Dashboard

Hyperparameter optimization holds the key to enhancing machine learning models, but navigating its complexities can be daunting. However, with the emergence of Human-in-the-Loop (HITL) optimization, the process becomes more intuitive and collaborative. In the previous part of this blog, we introduced HITL optimization using Optuna’s Objective Form Widget, bridging human expertise with algorithmic precision. You can refer to Part 1 for a detailed understanding.

Harnessing Human Insight: The Power of Human-in-the-Loop Optimization with Optuna(PART-1)

Just getting started with Optuna for hyperparameter tuning? In our introductory exploration, we uncovered a straightforward approach to hyperparameter tuning using optuna. This initial dive into hyperparameter optimization serves as a foundation for our deeper exploration. For detailed steps on getting started with Optuna, refer to the below article.

Optuna: Turbocharging Models with Hyper-parameter Magic

In this blog, we’re diving into another potent tool in Optuna’s optimisation toolkit: Preferential Optimization. We won’t just talk about what it is; we’ll roll up our sleeves and get hands-on with Python to really grasp how it works. So, buckle up as we explore Preferential Optimization in simple terms and then put it into action with some code!

Just like the previous part of this blog, we will also leverage Optuna and Optuna Dashboard to implement preferential optimisation in Generative Adversarial Networks (GANs) using the MNIST dataset as our canvas. Feel free to navigate to PART-1 of this blog to get a basic understanding of GANs.

Preferential Optimisation in Optuna

Preferential Optimization

HITL (Human-in-the-Loop) Optimization, represents a paradigm shift in the way we approach optimization tasks. It leverages the intuition and expertise of human operators, seamlessly integrating their insights into the optimization process. This symbiotic relationship between human expertise and automated algorithms holds immense potential for enhancing the efficiency and efficacy of optimization tasks across diverse domains.

Preferential optimization is a method for optimizing hyperparameters, focusing of human preferences, by determining which trial is superior when comparing a few. It differs from human-in-the-loop optimization utilising objective form widgets (Explored in PART-1), which relies on absolute evaluations, as it significantly reduces fluctuations in evaluators’ criteria, thus ensuring more consistent results.

Preferential Optimisation

In HITL optimization using Optuna Dashboard, there are primarily three components:

• A script that samples the hyperparameters from optuna and generates output for evaluation (Generated images in this case)
• Database and File Storage to store the experiment artefacts data (data, files, etc.).
• Optuna Dashboard for displaying the outputs stored in the storage and making evaluations.

Implementing preferential optimisation in Python

As a prerequisite for any optimisation in optuna, let’s start by defining an objective function.

Note: The objective function in preferential optimisation may or may not return any metric because it gives user control in determining which trial is superior when comparing a few unlike human-in-the-loop optimization utilising objective form widgets which allows user for evaluation to find the optimal hyperparameters based on a metric.

Here, we have defined an objective function almost similar to the one defined in the previous blog.

1. In the above objective function, a grid of new images is generated after each trial, These images are stored in the artefact store using upload_artifact function.
2. Then, the artefact ID that is generated is stored to the user attribute key, which is specified via register_preference_feedback_component.

Now, that the model training function is prepared, we’re poised to optimize the model through hyperparameter tuning by creating a study.

Optuna study

The above function creates an optuna study called “Preferential Optimisation”.

1. The n_generate parameter is the number of trials that should await human evaluation before the next trial is generated.
2. load_if_true allows a Study to exist and be reused and the experiment to be resumed if it has already been created.
3. Finally. Within the while loop, new trials are generated if the condition should_generate() returns True

Last but not least, we start the optimisation by defining the ‘main’ function.

• #1 — The FileSystemArtifactStore is created, which is one of the Artifact Store options used in the Optuna. Artefact Store is used to store artefacts (data, files, etc.) generated during Optuna trials(included in the train_GANs function). Also, the tmp folder is created to store the grid of newly generated images.
• #2 — start_optimisation() the function is called to run the trials for optimisation process.

Once you’ve executed the entire script, open a new terminal window and navigate to the root folder where the script is located. Then, run the Optuna dashboard locally by executing the following command:

optuna-dashboard sqlite:///pref_opt.db --artifact-dir ./artifact --port 5004

Now that you’ve executed the shell command, Get set to witness the preferential optimization process unfold on the Optuna dashboard.

Preferential Optimisation using Optuna

The above video showcases the Optuna dashboard in action as it navigates through the process of preferential optimization. It demonstrates the seamless integration of human feedback into the optimization loop and delivers a comprehensive visualisation of the results. However, there are few important points to notice from this demonstration, shedding light on the effectiveness and usability of Optuna in optimization scenarios.

• Optuna dashboard empowers the user to choose the best trial by knocking out the worst trials one after the other.
• The maximum number of concurrent trials is 6, defined by n_generate. A new trial is created when one of the existing trials concludes.
• If the number of trials awaiting human evaluation is less than n_generate , then study.should_generate() becomes true which starts the next trial.

This brings us to the end of the blog. Thank you for reading this blog post. I hope you found it informative and valuable for your optimization journey. If you have any questions or comments, please feel free to leave them below.

If you feel any need to visit the code or explore how each and every part of optimisation in optuna is implemented in Python, feel free to click the GitHub repo link.

Until then, happy optimizing!

References

• Optuna Documentation — https://optuna-dashboard.readthedocs.io/en/latest/tutorials/preferential-optimization.html
• Optuna Dashboard Documentation — https://optuna-dashboard.readthedocs.io/en/latest/