Dependency injection made easy in Python
Discover Opyoid, our new dependency injection library made in Python, featuring typings, modules, auto bindings and more !
Why do we need dependency injection ?
Let’s use an example project to illustrate this article. Imagine that you are writing code for a chatbot application. You want to reuse some classes between your bots so you don’t have to redo all your work every time. You start by writing the main classes:
The Chat class is fairly simple to understand. It needs a source that provides the user messages, then gives it to the answer generator to get the corresponding answer, and the output writer is responsible to handle the answer. You can optionally add a logger that will write the conversation to a file.
The UserMessageSource and OutputMessageWriter are abstract classes that can be implemented to customize the behavior of the chatbot.
A realistic scenario could be to create two different implementations. The CLI version reads the input from the console and writes the answers to it, this is useful for debugging on a local environment.
This simple version can read input from the CLI and print the answers, as well as log the conversation in a text file.
A production-ready version could be connected to a Message Queuing solution such as RabbitMQ to map a stream of user inputs into the corresponding bot answers. Since the implementation details are not really relevant to this article, here is a simple fake version:
In this version, we need a MqConfig instance that will be shared between the MqUserMessageSource and the MqOutputWriter. The ConversationLogger is not used, it could be added as a independent MQ consumer.
This code works, but has a few pain points:
- Creating a new configuration: implies instantiating a new Chat and all its dependencies. If this doesn’t seem like much work to you, remember it’s just an example, you probably wouldn’t want to do it with over 100 classes of dependencies.
- Adding/removing/swapping a feature: changing a group of logically linked dependencies can be tedious (you have to remove 2 classes and add 3 more to have a MQ-connected chatbot).
- Adding a dependency to a class breaks all configurations: if the conversation logger requires a log format, you would have to check all configurations using one and edit them manually.
- It’s a lot of boilerplate code, error-prone and without great value.
One solution to mitigate this is to use dependency injection to automatically link each class to its dependencies.
How does it work ?
Here are a few examples of how you can use opyoid to simplify your application.
As you can see, you create bindings to configure which classes should be instantiated, and opyoid does the rest.
- Here Chat requires an instance of UserMessageSource, and CliUserMessageSource is bound to this type so it gets instantiated when needed. Same thing for the OutputWriter, that is bound to the CLI version. When you want to bind a class to itself, such as Chat or AnswerGenerator, you don’t have to declare it twice.
- The ConversationLogger is bound to an instance of itself, it will be used directly when needed.
- All the bindings are given to an Injector, that you can then use to instantiate new objects.
- Notice that the only requirement for binding a class is that its constructor uses type hints, in our example we didn’t change the classes in any way.
- Also the order of the bindings does not matter, you can bind a class before or after its dependencies, so you don’t have to think about it.
What does the MQ version look like now ?
Notice how we used an InstanceBinding for the MqConfig, this is useful when you need to inject configuration dataclasses that contain primitive types such as strings, ints or booleans. This MqConfig has been automatically linked to all MQ connectors, so adding a new one would not require additional configuration except the class itself. You can also see that removing the ConversationLogger binding is not a problem since it has a default value in the Chat constructor, only the parameters without a default are required.
Further improvements
What if my code has no type hints ? Does that mean that I have to rewrite all my code so that I can use this ? And what if I depend on an external library, that I don’t have any control over ?
Be reassured, you can write your own provider to wrap those classes so they can be injected too.
Here the ChatProvider will be used to create every Chat instance needed, even if its constructor had no type hints. Notice the ProviderBinding in the Injector initialization.
Writing your own provider can also be used to add some logic on how some classes are instantiated.
Does that mean I have to create a single file with all my bindings ? I still have to duplicate a lot of bindings between my configurations.
That’s why you can use modules to group your bindings, so that you can reuse them across your configurations and just pick the ones you need.
Modules are usually used to group all classes related to a feature. By adding or removing the module, you can easily enable or disable this feature.
You can declare as many bindings as you want in a module, and install other modules as needed, we used it here to install the ChatModule into the CliChatModule.
Note that you can still add bindings on top of the modules in your injector.
Can you do better ? You talked about reducing boilerplate code, yet I still have to write all those bindings ?
Say no more:
When using the auto_bindings option, classes are automatically bound to themselves, so you only need to write the other bindings. You can still use modules and bindings of course, these auto bindings are only created as a last resort option. In the example we removed the ChatModule since it contained only ClassBindings.
Comparing it to the code we had in the beginning, we now have :
- automatic linking between classes and their dependencies, much less breaking changes when requirements change, automatic detection of missing dependencies
- no duplication between configurations
- boilerplate code reduced to the bare minimum
Why not reuse an existing library ?
We found that the Python ecosystem was lacking solid options for that. The best library we tested was pinject, but we wanted to take advantage of the typings to inject our classes. Another alternative is python-dependency-injector, but it still requires a lot of code to setup. Plus, it’s always more fun to do things on your own 😉.
So we decided to write a new dependency injection Python library, opyoid, to be able to easily resolve dependencies between classes and simplify the setup in large applications.
Our main goals when writing our library were:
- Automatically provide dependencies to each class
- Use typings to resolve them, as they are becoming the norm in Python
- Be able to inject third-party classes
- No mandatory decorators on all classes
We also added some more advanced features that we needed:
- A scope for each binding to be able to share some instances throughout the code while creating multiple instances of some other classes
- Providers to customize how classes are created or delay instantiation
- Annotations to be able to have multiple bindings for the same type
- And more…
Final words
And there you go, now you finally have a clean and readable code split into simple classes, without the hassle of writing boilerplate code. You can easily bind new classes where needed for each feature, create modules to group them together and easily activate them. You can check the project on Github for more examples and documentation.
Using dependency injection and especially with opyoid helped us tremendously to create modular and reusable applications, and developing on our codebase has never been so pleasurable.
Thanks for reading, you can leave a comment if you want to give some feedback on the article or the library 😊.
