The Gap Between Architecture and Development (Part 1)

Translating a programming language into code is trivial thanks to compilers and interpreters. However, translating architectural design into programming languages is something that we, as computer scientists, have not really solved yet.

In this post, I will try to convince you that there are many abstraction gaps that are filled by software engineers and architects with their contributions to the industry every day. However, while some gaps have been addressed appropriately, others are wide chasms that we ought to fill for the betterment of computer science as a discipline. In Part 2, I will present my humble contribution to fill what I consider to be the most important of these remaining abstraction gaps with my most recent project: Highwheel-Modules.

When you are developing software you are involved in a creative process not dissimilar from the act of translating between different languages. At the end of the day, every programming endeavor consists of taking concepts you have in your mind or that you read from a napkin written by your customer and elevated to the title of “Requirement Analysis Document”, and translating them into a form that can be understood and executed by a computer. This is not necessarily an easy feat. Machines, while predictable, work on a fundamentally different level of abstraction than we do and modelling our ideas in a way that is understandable by them takes patience, practice and gallons of coffee/tea.

This abstraction gap is the object of study of the well established field of programming languages. If you just look at the Wikipedia page for the list of programming languages you will find more than 600 of them, most invented and put in use in the last 50 years which is arguably a blip in the history of humankind.

While programming languages are a tremendously effective tool to translate certain abstractions into an executable format, they are not suitable for expressing concepts at higher level of abstraction. When we are designing a software system we are generally thinking about “things” that solve a portion of the problem and we connect these “things” together to solve the entirety of the issue.

The structure and the connections between these “things” is one of the primary concerns of software architecture and communicating these ideas and designs naturally translates into boxes and arrows diagrams like this:

Architecture sketch from https://c4model.com

There are many examples that illustrate how to communicate these designs effectively. Simon Brown is one of the main proponents of the C4 model for software architecture which consists roughly of representing a software system in 4 different levels of detail:

• Context Diagrams: represent the big picture, showing what are the users and external entities that interact with the software system
• Container Diagrams: represent the separate deployable elements of your system (containers) that execute code or store data.
• Component Diagrams: represent the building blocks of each container (components) and the relationships between one another.
• Core Diagrams: represent how the components are represented in code.

Separation of concerns

What I consider to be the main goal of the software architect is to identify the organization (the structure) of the parts of the system that minimizes the cost of change. The main instrument to achieve this is separation of concerns. Separation of concerns is an old concept in computer science that was probably first explicitly described by the legendary E.W. Dijkstra in his publication “On the role of scientific thought”.

Let me try to explain to you, what to my taste is characteristic for all intelligent thinking. It is, that one is willing to study in depth an aspect of one’s subject matter in isolation for the sake of its own consistency, all the time knowing that one is occupying oneself only with one of the aspects. […] nothing is gained - on the contrary! - by tackling these various aspects simultaneously. It is what I sometimes have called “the separation of concerns”, which, even if not perfectly possible, is yet the only available technique for effective ordering of one’s thoughts, that I know of. This is what I mean by “focussing one’s attention upon some aspect”: it does not mean ignoring the other aspects, it is just doing justice to the fact that from this aspect’s point of view, the other is irrelevant. It is being one- and multiple-track minded simultaneously.

Within a software system, separation of concerns translates into structuring the system components so that one component needs to change if and only if:

1. Its function changes.
2. The interface to other components required to accomplish its function changes.

These two reason for change are visually communicated by enclosing software components in “boxes” and by creating “arrows” between them.

Separation of concerns is arguably the driving force behind development best practices like SOLID principles by Robert Martin and Design Patterns by the GOF.

While it is possible to pontificate over engineering best practices for years and spend countless hours designing systems on whiteboards that are impeccable under a structural perspective, these ideals still have to be translated into code that needs to be converted into machine executable code. We currently have few reliable ways to do this.

Certain programming languages like JVM compiled languages as Java, Scala or Kotlin, make it easy to reason about your code in terms of components thanks to packages. Packages allow you to keep close classes and interfaces that are part of the same cohesive logical entity and should be used to separate the concerns of modules of the application. However, with the notable exception of simple publishing rules introduces with Java 9 and project Jigsaw, there are no tools available to the software architect to describe what constitutes a software module and what is this module allowed to depend upon.

This lack of tooling is what I consider to be the single most relevant impediment that prevents software architecture being represented, enforced and verified at the programming language level and, by analogy, the single most meaningful gap between architecture and development. Delaying a solution will just make the gap more significant and create more and more ivory tower architects in the industry.