AutoSampler: Full Support for Multi-Objective & Constrained Optimization

Introduction

AutoSampler is an intelligent sampler that automatically selects an optimization algorithm implemented in Optuna for a given problem, enabling efficient optimization. As the most popular package on OptunaHub, it receives over 30,000 weekly downloads. The initial version of this feature, as stated in a blog post published shortly after its release, implemented special automatic selection rules for single-objective optimization. However, for multi-objective optimization and constrained optimization, it defaulted to Optuna’s standard sampler, with full support for these problem settings designated as future work.

With the development of Optuna v4.6 (to be released next month), we have enhanced AutoSampler to fully support multi-objective and constrained optimization — a long-requested feature. This blog post will introduce these new enhancements, explain how to use them, and present benchmark results demonstrating their performance. (Note: The new AutoSampler is available now.)

Enhanced Support for Multi-Objective & Constrained Optimization

The new AutoSampler builds upon its original design philosophy by automatically selecting the most suitable sampler for a given optimization problem. It analyzes factors such as the evaluation budget, search space, constraints, and the number of objectives to choose a sampler that empirically delivers results equal to or better than the default.

With this update, the selection rules now fully support multi-objective and constrained optimization. The logic automatically switches between several powerful samplers (see Figure 1):

• TPESampler: Best for complex search spaces involving categorical variables and conditional branching.
• GPSampler: Ideal for continuous and integer search spaces, now with full multi-objective and constraint support.
• NSGAIISampler & NSGAIIISampler: Scalable choices for problems with a high number of evaluations (NSGA-II) or many objectives (NSGA-III).

This significant enhancement was made possible by the full implementation of constraint and multi-objective support for GPSampler in Optuna v4.2-v4.5.

Figure 1: AutoSampler finally fully supports constrained optimization and multi-objective optimization.

How to Use AutoSampler

Getting started with the latest AutoSampler is straightforward. Here’s how you can set it up and integrate it into your Optuna study.

First, make sure you have the necessary packages installed. You’ll need optuna version 4.5 or higher. You can install optuna, optunahub, and the AutoSampler’s dependencies with the following commands:

pip install -U optuna optunahub
pip install -r https://hub.optuna.org/samplers/auto_sampler/requirements.txt

Once the packages are installed, you simply load the "samplers/auto_sampler” package using the optunahub.load_module function, instantiate the AutoSampler included in the package, and use it just like any other Optuna sampler. Here is a runnable example for a multi-objective optimization problem:

import optuna
import optunahub


def objective(trial: optuna.Trial) -> tuple[float]:
    x = trial.suggest_float("x", 0, 5)
    y = trial.suggest_float("y", 0, 3)
    return  4 * x**2 + 4 * y**2, (x - 5)**2 + (y - 5)**2

study = optuna.create_study(
    directions=["minimize", "minimize"],
    sampler=optunahub.load_module("samplers/auto_sampler").AutoSampler()
)
study.optimize(objective, n_trials=300)

print(study.best_trials)

If you’ve used AutoSampler before, your locally cached version might be outdated. To ensure you have the latest version, run load_module once with the force_reload=True flag.

python
>>> import optunahub; optunahub.load_module("samplers/auto_sampler", force_reload=True)

Performance Evaluation

We evaluated the latest AutoSampler against Optuna’s default sampler in both multi-objective and constrained optimization scenarios. The results highlight the benefits of its intelligent, adaptive approach.

Multi-Objective Optimization

For our multi-objective benchmark, we used the widely recognized WFG1 problem. As shown in Figure 2, the AutoSampler demonstrates superior search performance. Unlike the default sampler, which always uses the NSGA-II algorithm, AutoSampler dynamically switches between four powerful methods based on the problem’s characteristics (e.g., number of evaluations, objectives, and search space): GPSampler, TPESampler, NSGAIISampler, and NSGAIIISampler. This automatic selection allows it to find better solutions by adapting its strategy to the task at hand.

Figure 2: Experimental results for the WFG1 benchmark

Constrained Optimization

We also compared performance on constrained problems using the 5-dimensional Rotated Rastrigin function with three constraints in the bbob-constrained benchmark set. As seen in Figure 3, the AutoSampler again outperforms the default.

Its advantage comes from new selection rules that incorporate the powerful, constraint-aware GPSampler (introduced in Optuna v4.2). This enables it to navigate constrained search spaces much more effectively. Furthermore, for problems that are both constrained and multi-objective, it leverages the even newer GPSampler enhancements from Optuna v4.5.

Figure 3: Benchmark results for the Rotated Rastrigin function

Wrapping Up

In this update, we’ve unveiled a significantly enhanced AutoSampler with full support for multi-objective and constrained optimization. By integrating AutoSampler into your workflow, you can achieve performance that is empirically equal to or better than the default sampler, all without the complex task of manually selecting the right algorithm. This powerful convenience allows you to focus on your problem while Optuna handles the optimization strategy.

This release fulfills a key milestone on our Optuna v5 roadmap: “Feature upgrades for AutoSampler to enable multi-objective optimization and constrained optimization with automatic algorithm selection.” We are continuing to develop new features and enhance existing ones as we move closer to the official v5 release. We encourage you to take advantage of these new capabilities and stay tuned for more updates to come!