Have you ever found a bug or wanted to contribute to an open source project, and then opened a project page on GitHub without knowing where to start? Should I first try building the project? Must I run the test suite? Where are the contribution guidelines and do I have to go through them all?
Each project has its own culture or set of rules, there is no single best practice for contributing. For Optuna, we feel that streamlining the contribution experience is critical to the growth of the project. This blog aims to give a hint of how to get involved with Optuna, and in particular, how you could go about sending pull requests. Once you’ve read this article, we hope you’ll also feel confident in reporting issues or reaching out in general.
TL;DR
We are proud of this project and have been trying to make this project great since day one. We believe you will love it, however, we know there’s room for improvement.
These are sentences from the contribution guidelines. In other words, Optuna wants your pull request.
What’s Optuna?
Optuna is a hyperparameter optimization framework in Python. A previous blog explains more about the framework itself. Its source code is pure Python and supports Python 3.5 and up to and including 3.8.
This will soon change as 3.5 reaches its end-of-life, and we are expecting code refactoring following this event. For those following this post, contributions are welcome.
Sending a Pull Request to Optuna
We love to see the project grow. A pull request is a convenient way to accomplish this. Here we summarize some tips for the benevolent reader. If you go through the contribution guidelines, how to create a pull request for Optuna should become clear. However, experience tells us that these guidelines are not the perfect solution. Some of this complements the existing contribution guidelines.
Figuring Out What to Do
If you don’t have any particular change in mind, look for issues labeled contribution-welcome. These issues are waiting to be picked. If the descriptions aren’t clear, don’t hesitate to ask for details or where to start looking in the source code. For simpler tasks, look at the good first issue tickets. Again, don’t be shy to ask. These might involve local changes or changes to the documentation. We are happy to individually follow up from here if you would be interested in continuing to contribute.
If you have found an issue or have some change in mind, you might know what to do. It could be a good idea to discuss implementation details on issues first, but this is not a necessity if you plan to create a pull request.
Building Optuna and Making Changes
Once you know what to do, create a fork of the project, clone it, and build it as follows from the project root directory.
pip install -e .Changes are immediately reflected in your installed optuna module when installing in editable mode (-e). Since the project is written entirely in Python, no recompilation is needed. Recommended Python versions for development are 3.6 or 3.7, unless you are constrained to one or writing version specific changes. A limited set of features such as third party library integration modules for libraries that don’t support Python 3.5 might not work in that version. 3.6 and 3.7 are the most widely supported versions in Optuna.
Coding styles such as naming conventions, type hints, and formatters are covered in the coding guidelines. These are highly recommended to review, since they’re often not the most essential parts of your changes but can drag out the review process. One tip is to install a black plugin for your editor or IDE, if available. Optuna is formatted with black.
Running the Test Suite
Tests are located under tests/ with subdirectories resembling the corresponding submodules in optuna. For instance, tests/samplers_tests/test_grid.py includes the unit tests for optuna.samplers.GridSampler, a sampler class that can be used to do grid search. After making changes, try to at least run tests that seem relevant. If they are not there, you probably want to add them. Ideally, one should run the entire test suite provided by a project to make sure that changes don’t break existing code. It’s however not a requirement for Optuna. This is because the test suite is quite large, mainly due to the optional dependencies that Optuna comes with for various integration modules for PyTorch, TensorFlow, Keras, LightGBM, etc. We might later find out through CI that tests fail, but it is okay to work on those as they appear (see “Sending a Pull Request”). All tests eventually have to pass before changes are merged.
Unless you are working with the optional dependencies, having pytest installed is usually enough to run the tests.
pip install pytestpytest tests/samplers_tests/test_grid.py
If you can and want to run the entire test suite, you could run pip install -e “.[testing]” to install all optional dependencies, or make use of a container with the CircleCI CLI that takes care of all of those. This takes some time. There are also several official Docker images that you can use to start a container with everything already set up if you are comfortable with that.
Writing Documentation
Without saying, good documentation is crucial. If you can accompany code changes with relevant changes to the documentation, please do. Changes solely to documentation are highly appreciated as well. Optuna uses Sphinx. Again, it’s documented in the contribution guidelines but taking a look at existing documentation might also help.
Sending a Pull Request
You are more than welcome to send a pull request whenever you feel comfortable. You could wait until all changes look solid and tests pass, or could send a draft pull request when starting working on one. We are glad to see any kind. Some general tips when doing this include the following.
- Motivate changes in the pull request description
- Be descriptive. Although code should be self-documented, a good summary usually speeds up the review process
- Address known errors in your pull request without waiting for a review. CI results including failures can be checked from the pull request page. Common failures are complaints from the style checkers which are usually straightforward to fix
When you create a pull request, you might notice that CI jobs on CircleCI and GitHub Actions are started automatically. These jobs will run the test suite, check the coding style and the type hints, making sure that changes look solid. If you want to know exactly what is executed, take a look at our configuration files for CircleCI or GitHub Actions. CircleCI provides a CLI that allows you to verify changes locally and you can for instance run circleci build --job checks to check the coding style if you don’t want to push every change.
Reviews might be quick, or span several days or weeks. We try to go through them all making sure no ticket becomes stale. Some discussions might still take time to come to conclusions due to their nature of controversiality or complexity in implementation. During review, we try to strike a balance between speed and code quality. It’s a controversial topic and the operation is not perfected by any means. Let us know during the review or reach out if you think that this process can be improved overall.
Conclusion
Optuna, similar to many other open source projects, has its own set of contribution guidelines. The aim is not to raise the bar to take part, but to improve the experience in doing so. Given that, pull requests are always welcome, and if you feel uncertain in how to approach this project, follow a few simple steps and you should feel confident in doing so. We really look forward to seeing you.
If you would like to follow our latest news, don’t forget to check the @OptunaAutoML Twitter account. We also have Gitter chat rooms for general discussions as an alternative to GitHub.
