SoftwareArch: Using Interfaces and Dependency Injection to Enable Testing and Extension

The use of interfaces is an easy to understand technique which enables the creation of both testable and extensible code. I have consistently found it to be the most powerful architectural design tool available.

The goal of this article is to cover what interfaces are, how they are used and how they enable extension and testability. Finally it should illustrate how interfaces can help to better manage software delivery and scheduling as well!

Interfaces

An Interface specifies a contract. Depending on language or framework they can be explicitly enforced or implicitly enforced. Go provides an example of explicit interface enforcement. Attempting to use something as an interface without that thing fully complying with the interface will result in a compile time error. Executing the example above results in the following error:

prog.go:22:85: cannot use BadPricer literal (type BadPricer) as type StockPricer in argument to isPricerHigherThan100:
	BadPricer does not implement StockPricer (missing CurrentPrice method)
Program exited.

Interfaces are a tool which decouple a caller, and a callee, through the use of a contract.

Let’s try and make this more concrete with example of an automatic stock trader. The stock trader will be invoked with a set buy price and a ticker symbol. It will then speak to an exchange in order to retrieve the current price for the symbol. Then if the ticker is under the buy price it will make a purchase.

A simple architectural sketch of the program may look similar to the diagram above. The above example shows that get current price has a direct dependency on HTTP in order to speak to the stock service. The Action state also has a direct dependency on HTTP. This requires that both states need to directly understand how to use HTTP to retrieve stock data and/or perform a trade.

The implementation may look like:

func analyze(ticker string, maxTradePrice float64) (bool, err) {
  resp, err := http.Get(
      "http://stock-service.com/currentprice/" + ticker
  )
  if err != nil {
  	// handle error
  }
  defer resp.Body.Close()
  body, err := ioutil.ReadAll(resp.Body)
  // ...
  currentPrice := parsePriceFromBody(body)
  var hasTraded bool
  var err error
  if currentPrice <= maximumTradePrice {
    err = doTrade(ticker, currentPrice)
    if err == nil {
      hasTraded = true
    }
  }
  return hasTraded, err
}

Here the caller (analyze) has a direct hard dependency on HTTP. It needs to know how to form HTTP requests. It needs to understand how to parse responses. It needs to understand how to deal with retries, timeouts, authentication, etc. It has a tight coupling to http. Any time we want to invoke analyze we need to invoke the http library.

How can an interface help here? We can use the contract an interface provides to specify a behavior and not a concrete implementation. More information about go-specific usage of interfaces can be found here.

type StockExchange interface {
  CurrentPrice(ticker string) float64
}

Above defines the concept of a StockExchange. It defines that the StockExchange supports a single function call CurrentPrice . I have found these 3 lines to be the most powerful architectural technique available. They allow us greater control of our application dependencies. They enable testing. They enable extension.

Dependency Injection

To fully realize the value of interfaces we’ll need to use a technique called Dependency Injection.

Dependency Injection specifies that the caller provides something that a callee needs. This usually takes the form of a caller configuring an object and then passing that object to the callee. The callee is then abstracted from the configuration and implementation. There is some level of indirection here. Consider an HTTP Rest service request. In order to implement a client we’ll need to use an HTTP library that understands how to handle forming, sending and receiving HTTP requests.

If we were to put the HTTP request behind an interface the caller could be decoupled and remain unaware that an HTTP request is actually occurring.

The caller would only invoke a generic function call. It could be a local call, a remote call, and HTTP call, an RPC call, etc. The caller is none the wiser, and usually doesn’t care as long as it gets the results it’s expecting. Below shows how dependency injection might look in our analyze method.

func analyze(se StockExchange, ticker string, maxTradePrice float64) (bool, error) {
  currentPrice := se.CurrentPrice(ticker)
  var hasTraded bool
  var err error
  if currentPrice <= maximumTradePrice {
    err = doTrade(ticker, currentPrice)
    if err == nil {
      hasTraded = true
    }
  }
  return hasTraded, err
}

I still can’t stop being astonished at what happens here. We’ve completed inverted our dependency tree and taken better control of our program. Not only that, but the implementation visually looks cleaner and is easier to understand. We can see clearly that the analysis needs to fetch a current price, check to see if that price qualifies and then makes a trade. Most importantly this enables decoupling of a caller and a callee. Since the caller and the implementation are decoupled by the interface, it enables extension of the interface with multiple different implementations. Interfaces enable multiple different concrete implementations and doesn’t require the callee code to change!

The get current price state of the program is only dependent on an StockExchange Interface. It knows nothing about communicate with the actual stock service, how prices are stored or how requests are made. It is blissfully unaware. This relationship goes both ways. The HTTPStockExchange Implementation also knows nothing about analysis. It doesn’t know about the context of where it is executed or when because it’s not called directly.

Since portions of the program (those that depend on interfaces) don’t need to change as concrete implementations are changed/added/removed, it future proofs the design. Suppose we discover that the StockService is very often unavailable.

How is the example above different from a function call? A function call would also clean up the implementation. The difference is that the function call would still need to invoke HTTP. analyze would just be delegating to a function that calls http instead of calling http directly. The power here comes from “injecting” ie the caller providing the interface to the callee. This is what creates the dependency inversion where the get prices is only dependent on an interface and not on an implementation.

Multiple implementations out of the box

As it stands we have an analyze function, and a StockExchange interface but we can’t really do anything useful. We have only declared our program. It’s not currently invokable because we don’t yet have any concrete implementations that fulfill our interface.

The diagram below just focuses on the get current price state and its dependency on the StockExchange interface. Below shows how two completely different implementations exist side by side and get current price is none the wiser. Additionally neither implementation is connected to the other, both only depend on the StockExchange interface.

Production

The original HTTP implementation already exists in our original implementation of analyze we just need to extract it an encapsulate it behind a concrete implementation of the interface.

type HTTPStockExchange struct {}
func (se HTTPStockExchange) CurrentPrice(ticker string) float64 {
  resp, err := http.Get(
      "http://stock-service.com/currentprice/" + ticker
  )
  if err != nil {
  	// handle error
  }
  defer resp.Body.Close()
  body, err := ioutil.ReadAll(resp.Body)
  // ...
  return parsePriceFromBody(body)
}

What was previously coupled to the analyze function now stands alone and fulfills the StockExchange interface, meaning we can pass it to analyze . Remember from the charts above, analyze no longer has a direct dependency on HTTP. Using the interface makes it so that analyze has no idea what’s happening behind the scenes. It only knows that its guaranteed to get passed an object that it can call CurrentPrice on.

We also get the general benefits of encapsulation here. Before when the http requests were coupled to analyze the only way to exercise our communication with the exchange over http was indirectly through the analyze method. While we could have encapsulated those calls behind a function and exercised the function independently, interfaces force us to decouple the caller and the callee. We can now test the actual HTTPStockExchange individually independent of its caller. This has massive resounding effects on the scope of our tests and how we understand and respond to test failures.

Testing

With our current code we have an HTTPStockService structure which allows us to individually verify that it can communicate with, and parse responses from, the Stock Service. But how are we going to test that analyze can do the correct thing with the response from the StockExchange interface, in a reliable repeatable way.

currentPrice := se.CurrentPrice(ticker)
 if currentPrice <= maxTradePrice {
    err := doTrade(ticker, currentPrice)
  }

We COULD use the HTTP implementation, but this comes with many downsides. Doing network calls in unit tests can be slow, especially to external services. Latencies and network flakiness could make tests unreliable. Additionally, if we would like a test that asserts we can make a trade and tests that assert we can filter out when we SHOULDN’T make a trade, finding production data that reliably fulfills those two conditions can be hard. We could choose a maxTradePrice that artificially simulates each condition, ie maxTradePrice := -100 will not trade and a maxTradePrice := 10000000 should reasonably result in a trade.

But what happens if we have a quota with the Stock Service? or what happens if we have to pay for access? Do we really , or should we have to, pay or use our quota for unit tests? Ideally we’ll be running tests extremely frequently, so they should be fast, cheap and reliable. I hope that the above paragraph explains why using an actual HTTP version doesn’t fulfill our testing goals!

There’s a better way and its enabled through the use of interfaces!

Having an interface allows us to carefully craft a StockExchange implementation which will allow us to exercise analyze quickly, safely and reliably.

type StubExchange struct {
   Price float64
}
func (se StubExchange) CurrentPrice(ticker string) float64 {
   return se.Price
}
func TestAnalyze_MakeTrade(t *testing.T) {
  se := StubExchange{Price: 10}
  maxTradePrice := 11
  traded, err := analyze(se, "TSLA", maxTradePrice)
  if err != nil {
     t.Errorf("expected err == nil received: %s", err)
  }
  if !traded {
    t.Error("expected traded == true")
  } 
}
func TestAnalyze_DontTrade(t *testing.T) {
  se := StubExchange{Price: 10}
  maxTradePrice := 9
  traded, err := analyze(se, "TSLA", maxTradePrice)
  // assertions
}

The above uses a stub exchange to trigger the desired branch in analyze . Each test then makes assertions to verify that analyze does the correct thing. While this is a test program, in my experiences components/architecture that leverage interfaces similar to this way end up having tests like this in actual code!!! Because of interfaces we’re able to use an in memory controllable StockExchange, which allows for reliable, easily configured, easy to understand, repeatable and lightning fast tests!!!

Decoupling — Caller Configuration

Now that we’ve covered how to use interfaces to decouple the caller and the callee and how to have multiple concrete implementations, we’re still missing a critical component. How do we configure and provide a specific implementation at a specific time? We can invoke the analyze function directly but what about the production configuration?

This is where Dependency Injection comes in.

func main() {
   var ticker = flag.String("ticker", "", "stock ticker symbol to trade for")
   var maxTradePrice = flag.Float64("maxtradeprice", "", "max price to pay for a share of the ticker symbol."
   se := HTTPStockExchange{}
  analyze(se, *ticker, *maxTradePrice)
}

Similar to our test, the specific concrete implementation of the StockExchange that analyze will be used is configured outside of analyze by its caller. It is then passed in (injected) into analyze. This allows it so that analyze knows NOTHING about how HTTPStockExchange is configured. Perhaps we want to expose the http domain that we will use for the exchange as a command line flag, analyze will not have to change. Or if we need to provide some sort of auth or token to access HTTPStockExchange which is pulled from the environment? Once again analyze will not have to change. Configuration takes place at the layer outside of analyze, completely insulating analyze from the responsibility of having to configure its dependencies. This enables a strict separation of concerns.

Deferring decisions

As if the above examples aren’t enough, there are many more benefits of Interfaces and DI. Interfaces enable deferring decisions about concrete implementations until later. While interfaces require us to make a decision about the behavior to support, the enable the deferment of decisions about specific implementations. Suppose we knew that we wanted to make automatic trades, but we still weren’t sure which Stock Provider to use? We see a similar class of decisions all the time in regards to data storage. Should our program use mysql, postgres, redis, filesystem, cassandra? These are ultimately implementation details, and interfaces empower us to push making these decisions off until later. They allow us to move forward with the business logic of our programs, but wait to make these technology specific decisions later!!

While this by itself leaves many possibilities open for us, it also does something magical for scheduling our project. Imagine if we move the other dependency to an exchange interface as well.

We’ve rearrange our architecture as a DAG so that as soon as we agree upon our stock exchange interface we’re able to move forward with the pipeline CONCURRENTLY with the HTTPStockExchange. We’ve created a situation where adding another person to the project will help the project move faster. Massaging our architecture like this allows us to better predict where, when, and how long we’ll be able to utilize more people to help us deliver faster. Additionally, because of the decoupled nature of interfaces, implementation interfaces is usually easy work to get bootstrapped on. We can develop, test, and verify an HTTPStockExchange completely independently of our program!!!!

Analyzing the architectural dependencies, and scheduling according to those dependencies, can massively accelerate projects. Using this exact technique, I’ve been able to deliver multi month projects, in a fraction of the time estimated for them. I hope to be able to write more about this specific technique in the future!

Future Proofing

By now it should start being clearer how interfaces and dependency injection can help future proof our designs. Perhaps we change our stock service provider, or we move to streaming and saving quotes in real time, or any number of different possibilities. analyze as is will support any implementation that is able to be coalesced into the StockExchange interface.

se.CurrentPrice(ticker)

This insulates us from changing in a large number of cases. Not all cases but many of the predictable cases we may encounter. In addition to protecting us from having to change analyze code, and reverify its core functionality, it makes it easy to provide new implementations, or switch providers along these lines. It also allows us to seamlessly extended, or update, concrete implementations we already have without having to change or reverify analyze !!!

I hope the examples above illustrate how decoupling through the use of interfaces completely inverts dependencies and isolates the caller from the callee. Since it is decoupled it doesn’t have a dependency on a specific implementation, instead it has a strict dependency on a specific behavior. That behavior can be fulfilled by many different implementations. This is the crux of the design and is also referred to as duck typing.

The concept of interfaces, and depending on behavior instead of implementation, is a concept so powerful go decided to adopt it interfaces as a language primitive, in a pretty radical way. In go, structures don’t have to explicitly declare compliance with an interface, that compliance will be evaluated and checked by go during compile time. This creates an additional level of decoupling because as a structure is declared it doesn’t need to know where or when or in what context it will be used.

I hope the example above shows how interfaces and DI are a powerful technique to use even from the very beginning of a project! Almost all projects I’ve worked on DO have multiple implementations required from the beginning: a production implementation and a test.

If you’ve enjoyed this at all, very shortly I’ll be publishing an article that illustrates designing, building and testing a full application using these techniques.

Thank you for reading!