INDIRECTION

A key design principle in a software engineer’s arsenal

• “All problems in computer science can be solved by another level of indirection.” [David Wheeler]
• “Most performance problems in computer science can be solved by removing a layer of indirection” [unknown]

Indirection is one of the most useful (and simple) techniques in a software engineer’s arsenal. It is a powerful technique for decoupling software units by introducing a layer between two existing units/layers, enabling the introduction of new behaviour, without directly impairing the existing units. See the figure below.

Indirection removes direct coupling between units and promotes:

• Extensibility. Facilitates functional extension that belongs neither in the client or target. Logging, metrics, transaction management, and remote method invocations are examples of cross domain responsibilities, seen as “plumbing” logic that should not pollute (or be tightly-coupled to) business logic. Aspect oriented programming is a popular solution for this problem.
• Modifying a target interface to meet client interface needs. This is useful when the target’s interface cannot change, possibly because others are already tightly coupled to it, or the target is not under your control (Control).
• Encapsulate technology specifics. Ensures client and target communicate embedding a technological coupling in them. This promotes the use of different technologies without affecting either party, and promotes Evolvability.
• Encapsulation of code or design complexity. You don’t always want clients/consumers to know of complex interactions of design choices. By ensuring clients remain decoupled from the how (or even the order that) something is done, we facilitate change.

INDIRECTION IN PRACTICE

Indirection is a fancy name for “wrapping”. Almost anything can be wrapped; for instance:

- SOA’s idea of reusing legacy systems by abstracting them away used Indirection. Ok, we probably now see it as an anti-pattern, but the sentiment was good, and it wasn’t the fault of Indirection.

- A Load Balancer acts as a layer of Indirection between consumer and web server and promotes (horizontal) Scalability and Resilience.

- The Java Virtual Machine (JVM) is a (well considered) platform agnostic “wrapper” above Operating System (OS) libraries, that encapsulates from us the complexities of which OS the software application is executing on.

- Queues are a form of Indirection (they are a Holding Pattern). They create a bulwark/bulkhead between two systems, which is particularly useful if those systems work at different speeds.

- Gateways and Edge Services use Indirection between (external) consumers and a (internal) consumable resource, such as an API. They are often used to inject “plumbing” logic like authorization or metrics into a solution.

- Enterprise Service Buses (ESBs) introduced a layer of indirection between consumer and target(s), and are often used to construct workflows and transactional logic into a request lifecycle.

- The Hub-and-Spoke Integration Architecture, where the Hub is the indirectional technique.

Indirection can, under the right circumstances improve Reuse (high cohesion), Flexibility, Maintainability, Extensibility, and Evolvability.

Indirection can be categorised into the following main groups (discussed next):

• Behavioural Extension.
• Interface Modification.
• Technology Encapsulation.
• Complexity Encapsulation.

BEHAVIOURAL EXTENSION

Indirection is commonly used to add behaviours not necessarily belonging to the client or target. These behaviours are typically added without direct awareness of either party. This solution promotes Reuse and Flexibility by reducing coupling (e.g. Assumptions); making each unit available in more contexts. See the figure below.

The Proxy and Decorator design patterns are canonical examples of the approach. Proxy uses indirection to add logic to determine whether to call a target. Whilst Decorator attaches behaviour(s) to the target behaviour, increasing its level of functionality without polluting it.

INTERFACE MODIFICATION

Indirection is also useful for changing a target interface. This is often called “wrapping”. See below.

Wrapping enables the wrapped target to be utilised even when its interface does not match one expected of the client.

The Adapter and Mediator design patterns use Indirection to modify how an interface looks. Clients utilise a target service (by applying Adapter or Mediator), even if the target does not match what is expected. Whilst Adapter modifies the target interface to match one expected by the client, Mediator modifies both client and target interfaces, enabling a two-way, “mediated” communication between them (Hub-and-Spoke is a form of Mediator).

TECHNOLOGY ENCAPSULATION

Indirection is also useful for technology encapsulation. See below.

Flexibility is a key architectural concern for many enterprises (it can promote Sellability, for instance), and technology independence often drives it. Indirection promotes loose coupling; enabling technologies to be more easily interchanged. The end result? Better Evolvability, an extended product lifetime, and thus increased ROI.

The Business Delegate and Service Adapter design patterns are examples of Technology Indirection, encapsulating EJB and Web Service technologies respectively. And whilst I’m not a huge advocate of this approach, I’ve also seen solutions that wrap legacy SOAP APIs with a REST equivalent to satisfy external consumers, which is a form of Technology Indirection (I’ll let you figure out why wrapping SOAP with REST, devoid of any deep thinking, might not be a good practice).

Technology Indirection also promotes target reuse by reducing assumptions in the target; see below.

The target is currently only available to clients through Technology A. Yet the target remains technologically agnostic, enabling us to wrap it with other technologies, without pollution. See below.

COMPLEXITY ENCAPSULATION

Some implementations may be so complex that understanding is almost impossible (e.g. it’s too complex to be read in isolation). Indirection can be useful here — by moving the complexity into another layer, we can reduce code complexity and promote maintainability. See below.

There’s a secondary reason that you might consider this solution. It can also be used to hide embarrassing design decisions that would hurt your Reputation if they were exposed to external consumers.

The Facade and Session/Service Facade design patterns are archetypal solutions that hide the complexities of subsystem communication through a layer of indirection. They also promote Reuse, enabling different clients to utilise the same Facade functionality without polluting them by embedding the same complex logic in multiple places. I’ve also seen a Gateway/Edge solution used to abstract the complexities in different security models away from consumer and underlying internal system, enabling Optionality in security model, and protecting the Evolvability of the internal system.

VIRTUAL MACHINES AND INDIRECTION

Virtual Machines (e.g. the Java Virtual Machine) operate at a high-level of Indirection. The layer sits between the client (the executing application) and the target platform (the Operating System the application runs on); see below.

Note that the client never directly interacts with the OS; it uses the standardised set of (JVM’s) libraries that are uniform across all operating systems. It hides OS complexities; the developer need not have a deep understanding of each and every OS library. This is key to Java’s platform independence (and its slogan “write once, run anywhere”); developers develop a single application that operates consistently across all platforms. The virtual machine is actually an enormous Adapter, adapting Java APIs to meet different OS libraries.

INDIRECTION AND PERFORMANCE

Introducing an indirectional layer between two software units will likely cause some performance degradation. In the Object-Oriented world, an indirectional layer causes (a) additional object creation and (b) extra processing to delegate to that object. At a higher level, it may involve some form of middleware (e.g. Queues) or Gateway/Edge Service to mediate between components. You must also consider the need to scale these indirectional layers to meet desired SLAs.

Is it important? That depends upon the context. For example, adding further layers to a workflow management system may not cause performance concerns; yet it might do in a 3D games engine, which is heavily reliant on high performance.

SUMMARY

Indirection is a key design principle which is heavily used in design patterns. It introduces a layer between client and target to handle a specific responsibility that would be inappropriate in the client or target. Indirection ensures the client and target remain decoupled, thereby promoting Flexibility, Reuse, Extensibility, Evolvability, and (to some extent) Maintainability.

Common Indirectional uses include:

• Adding new behaviours.
• Modifying interfaces.
• Technology encapsulation.
• Complexity encapsulation.

Several Design Patterns are indirectional by nature, including Proxy, Decorator, Adapter, Facade, and Mediator. It’s also heavily used in areas such as Edge Services, Load Balancing, PAAS.

Whilst Indirection is incredibly powerful, it comes at a price. As we add layers, we find performance degrades. Whether that is acceptable is context-specific.

FURTHER CONSIDERATIONS

• Assumptions
• Queues & Streams
• Gateways/Edge Services
• Hub-and-Spoke
• Control