Writing more modular code with dependency injection

Fall into the pit of success with the architect’s favorite pattern

Software engineer injecting a dependency — courtesy of GPT-4 ❤

You won’t get far into any book on software engineering without the author extolling the virtues of code modularity. One of the most powerful tools to force you to write more modular code is the use of dependency injection.

Example: Writing a Router

Let’s say that you were tasked with writing a simple router. This router should have a fetch() method, which grabs some data from a specific HTTP endpoint, and a send() method, which sends data to a different HTTP endpoint.

If we were to implement it in python, it might look something like this, with source being the endpoint you are getting data from, and dest being the endpoint you are sending that data to.

import json
import requests

class Router:
    def __init__(self, source: str, dest: str) -> None:
        self.source = source
        self.dest = dest

    def fetch(self) -> json:
        response = requests.get(self.source)
        return json.loads(response.text)

    def send(self, data: json) -> None:
        requests.post(self.dest, data=data)

Now, your boss comes back with another request from the customer. “Oh! Also, when we send out data, we might want to fetch it over a raw ZMQ socket in addition to HTTP”.

One obvious way to do this might be to implement separate fetch_http and fetch_zmq methods in the class, and then set a flag at construction time like so:

class Router:
    def __init__(self, source: str, dest: str, mode: str) -> None:
        self.source = source
        self.dest = dest
        self.mode = mode

    def fetch(self) -> json:
        if self.mode == "http":
            return self.fetch_http(self.source)
        elif self.mode == "zmq":
            return self.fetch_zmq(self.source)
    
    def fetch_http(self):
        # ...

    def fetch_http(self):
        # ...

This approach is fine, but it has a major drawback: to add functionality, we are adding complexity to the Router class. If we wanted to add additional mechanisms we might get data over, we would need additional if statements, and more flags. We’re bloating the class that is closest to the user, and adding complexity where it’s most visible and annoying.

Dependency Injection to the rescue

Alternatively, we could recognize that all we are trying to do is change the implementation of our “fetch” functionality, and so we could use dependency injection to split out this “fetch” functionality into its own component, which gets passed in to the Router class at runtime:

import abc
class Fetcher(abc.ABC):
    def __init__(self, endpoint: str) -> None:
        self.endpoint = endpoint

    @abc.abstractmethod
    def fetch(self) -> json:
        pass

Now, the implementation of our Router class becomes much simpler — almost trivial:

class Router:
    def __init__(self, fetcher: Fetcher, dest: str) -> None:
        self.fetcher = fetcher
        self.dest = dest

    def fetch(self) -> json:
        return self.fetcher.fetch()

And all the complexity has been moved to the specific implementations of our Fetcher class, with separate implementations for HTTP and ZMQ protocols:

class HTTPFetcher(Fetcher):
    def __init__(self, endpoint: str) -> None:
        super().__init__(endpoint)
    
    def fetch(self) -> json:
        # HTTP-specific implementation

class ZMQFetcher(Fetcher):
    def __init__(self, endpoint: str) -> None:
        super().__init__(endpoint)
    
    def fetch(self) -> json:
        # ZMQ-specific implementation

Now, when a user creates a Router object using HTTP, they would do something like this:

fetcher = HTTPFetcher("http://www.myapi.com/")
router = Router(fetcher=fetcher, dest="http://otherapi.com")

The customer is back again, with more requirements

Oh! I forgot to mention — we also want our destination to support all the same protocols as our sources.

No problem, we can simply do the exact same thing with the send() method that we did with the fetch method — inject it at runtime.

import abc
class Sender(abc.ABC):
    def __init__(self, endpoint: str) -> None:
        self.endpoint = endpoint

    @abc.abstractmethod
    def send(self, data: json) -> json:
        pass

And we now modify our Router class to use this interface:

class Router:
    def __init__(self, fetcher: Fetcher, sender: Sender) -> None:
        self.fetcher = fetcher
        self.sender = sender

    def fetch(self) -> json:
        return self.fetcher.fetch()
    
    def send(self) -> None:
        self.sender.send()

And provide some concrete implementations of our Sender classes:

class HTTPSender(Sender):
    def __init__(self, endpoint: str) -> None:
        super().__init__(endpoint)
    
    def send(self, data: json) -> None:
        # HTTP-specific implementation

class ZMQFetcher(Fetcher):
    def __init__(self, endpoint: str) -> None:
        super().__init__(endpoint)
    
    def fetch(self, data: json) -> None:
        # ZMQ-specific implementation

A fully modular Router

Now, we can do something really cool. We can mix and match our send and fetch functionalities to make completely different types of Router objects:

router1 = Router(sender=HTTPSender(...), fetcher=ZMQFetcher(...))
router2 = Router(sender=ZMQSender(...), fetcher=ZMQFetcher(...))
# and so on...

Dependency injection is such a powerful pattern because it forces us into a pit of success, forcing us to make our code modular. It’s almost impossible to use dependency injection without making your code more modular than it was before.

The pit of success — trust me, you want to be here.

Separation of concerns

In the example above, we got another thing for free: separation of concerns. The Fetcher classes were concerned only with fetching data, the Sender classes with sending data, and the router was merely a layer that composed and used those pieces.

The Router objects didn’t need to know anything about the actual protocols that would be used to access or send the data. This is the power of dependency injection.

Clear Interfaces

Using dependency injection also forces us to specify what the interfaces between our modules should be. In the example above, we define a fetch() interface and a send() interface. The router object has access to both of these, and uses them to perform its job.

Explicitly specifying interfaces for how our code should be allowed to talk to itself makes it much easier to extend in the future, and makes clear what each “module” of our software does, because each “module” now has a clearly-defined interface.

Closing Thoughts

Dependency injection, when done right, may be the single most powerful tool you have at your disposal to write more modular, extensible, and clean code. Few patterns are as far-reaching, useful, and powerful.

This is part of my series on dependency injection. For the previous story in the series, see Writing future-proof code with dependency injection.