The Software Design Principle that we all agree on — Part I

Software design principles and frameworks are guidelines of good software development. Each may have strengths and weaknesses and can be combined or adapted to address a particular aspect of the software requirement. In practice, design discussions can often be a source of immense frustration for those involved. The process of navigating through a myriad of opinions, preferences, and perspectives can feel like an endless maze, with each participant passionately advocating their own vision. As the conversation unfolds, it becomes increasingly challenging to reach a consensus. In such situations, teams are often under pressure to make decisions quickly. However, the overwhelming number of options and opinions can lead to “analysis paralysis,” where the team becomes stuck in a state of overthinking and indecision. This further complicates the process, leaving everyone feeling drained and disheartened.

We will delve into the rich history of design principles in search of a unifying design principle. By examining the evolution of these design practices and understanding their underlying foundations, I hope to identify a common thread that can bridge the gap between diverse perspectives and facilitate more harmonious and productive software design discussions. This exploration aims to foster a greater sense of collaboration and unity among team members, ultimately enhancing the creative process and leading to more effective and innovative solutions.

Software Architects debates.

Definitions

Software design is the process of planning a software system to meet user needs. It involves making decisions about architecture, components, and interfaces. Challenges include balancing competing factors like functionality, usability, security, cost, and more. Effective software design must anticipate future needs and changes to ensure long-term success.

Software design principles provide a framework for creating effective and successful software systems. They help to ensure that the designed software is functional and flexible for changes and technologies while fulfilling user needs. Adhering to design principles will result in an easy system to understand, modify, and maintain while reducing risks. Design principles also help improve the overall quality of software systems written, making it more secure, robust, resilient, and scalable.

Conceptual Integrity

In 1975, Fred Brooks in his book, “The Mythical Man Month” defined Conceptual Integrity, as coherence and consistency of the overall design and structure of a software system. Brooks emphasizes that achieving conceptual integrity is crucial for successful software development.

According to Brooks, Conceptual Integrity arises from unifying vision or conceptual framework that governs the system’s design. It is the result of a well-defined and well-executed architectural plan that guides the development process. The Conceptual Integrity ensures that the system components adhere to a common set of principles and conventions.

Examples for Conceptual Integrity

Unix file notation’s Conceptual Integrity is highlighted by its design, which emphasizes simplicity and consistency, with a unified approach to handling various data types, such as directories, files, and sockets. By treating everything as a file, Unix provides a consistent interface for interacting with different resources, allowing programmers to use the same set of commands and system calls.

Java’s Conceptual Integrity is largely due to its object-oriented programming (OOP) approach, where everything is treated as an object. This principle, inspired by Smalltalk, promotes consistency and simplicity in the language. By perceiving everything as an object, Java unifies the way developers interact with data and resources, making the language easier to understand and extend.

Einstein’s “Theory of Everything” can be considered an example of Conceptual Integrity in that it aims to provide a coherent and consistent framework to describe all fundamental forces of nature. By unifying these forces into a single theory, it would simplify our understanding of the universe and make it easier to comprehend the underlying principles governing the behavior of matter and energy. Stay tuned, we will come back to this later.

Brooks argues that Conceptual Integrity is essential for several reasons:

1. Ease of understanding: When a system exhibits Conceptual Integrity, its structure and behavior are coherent and predictable, making it easier for developers to understand, maintain, and modify the system. This reduces the complexity and cognitive load associated with working on the software.
2. Consistency: Conceptual Integrity helps ensure consistency throughout the system. Design decisions and implementation choices align with the underlying conceptual framework, resulting in a cohesive and unified system. Consistency enhances the user experience, promotes code reuse, and simplifies troubleshooting and debugging.
3. Communication and collaboration: Conceptual Integrity serves as a common language and reference point for the development team. It enables effective communication and collaboration by providing a shared understanding of the system’s goals, constraints, and design principles. This shared vision facilitates coordination among team members and fosters a cohesive development process.
4. Evolution and scalability: A system with Conceptual Integrity is more adaptable and scalable over time. As new features or requirements arise, the underlying architectural framework can accommodate changes with minimal disruption. Conceptual integrity ensures that modifications fit seamlessly into the existing structure, reducing the risk of introducing unforeseen side effects or breaking existing functionality.

Brooks later on claimed in his book, that Conceptual Integrity is “the most important consideration in system design”. He also emphasized in his updated version, published 20 years after the original, “I am more convinced than ever that Conceptual Integrity is central to product quality.”

Conceptual Integrity and the design team

According to Brooks, Conceptual Integrity states that the design must proceed from one mind, or from a small number of agreeing resonant minds. This means that one should include many minds (or even the entire team) in the analysis and design process. This is an important detail that should not be discounted by those who wish to do away with the architect’s role. Team input in analysis and design is essential for setting up a team, ensuring the soundness and quality of the analysis, and refactoring the design into something more polished. In fact, the earlier the architect or design team includes the entire team (or domain-team leads for exceptionally large teams), the higher quality the design will be.

The design team must be open-minded to constructive criticism; the architect(s) must be egoless.

Conceptual Integrity — The challenges

Leaky Abstraction:
When a software system becomes large and complex, maintaining Conceptual Integrity can be increasingly challenging. Joel Spolsky coined the term “leaky abstractions” to describe one of these challenges. An abstraction is a simplified representation of a system that explains it in an intuitive way. However, they are not perfect, and there is often some degree of leakage between the abstraction and the underlying system.

Spolsky’s article cites many examples of leaky abstractions that create problems for software development. For example, The SQL language omits the procedural steps for querying a database, allowing one to merely define what one wants. But certain SQL queries are thousands of times slower than other logically equivalent queries. A famous example of this is where some SQL servers are dramatically faster if you specify “WHERE a=b AND b=c AND a=c” than if you only specify “WHERE a=b AND b=c”. While both of the SQL expressions are logically the same, adding “a=c” at the end hints the query plan analyzer how to process the rows. The problem is that SQL coder is not supposed to care about the procedure, only the specification. This kind of caveats may cause horrible performance decrease. The SQL developer will have to understand what’s under the hood in order to figure out what was wrong.

Scaling up:
Another challenge is that maintaining Conceptual Integrity requires a shared understanding of the design among all members of the development team. This can be difficult to achieve in large software projects, where there may be many developers working on different parts of the system. Without a shared understanding of the design, it can be difficult to maintain consistency and ensure that the system is coherent.

Antipattern and patterns:
Design frameworks offer patterns and antipattern. CQRS is considered a pattern and “Golden hammer” is a well-known antipattern. These two patterns are an example of how a consistent method may sometime be inappropriate for all problems.

CQRS is a pattern that separates the write model from the read model. The idea is that by separating these responsibilities, you can create simpler and more scalable architecture. However, this separation can also lead to inconsistencies in the design. For example, if the write and read models use different data models or have different business logic, it can be difficult to maintain Conceptual Integrity and ensure that the system is consistent and coherent.

Similarly, the “Golden hammer” problem is a phenomenon where developers choose to use technology simply because it is popular, without considering whether it is the best fit for their particular use case. This can lead to inconsistency in the design, as the chosen technology may not be well-suited to the specific requirements of the system.

Is Conceptual Integrity a “Silver Bullet”?

Let’s get back to Albert Einstein’s “Unified Field Theory,” which attempts to conceptually integrate all the fundamental forces of physics between elementary particles into a single theoretical framework. Well, Einstein spent the latter part of his life searching for such a unified field theory but was unsuccessful. Theoretical physicists have not yet formulated a widely accepted, consistent theory that combines into a single theory, and the problem remain an outstanding field in physics today.

In his seminal paper, “No Silver Bullet”, Fred Brooks argues that there is no single solution or technology that can significantly reduce the complexity of software development. This idea contradicts his emphasis on Conceptual Integrity. The contradiction arises because “No Silver Bullet” acknowledges the inherent complexity of software development, suggesting that even with a strong, consistent vision, challenges persist. On the other hand, Conceptual Integrity implies that a consistent design can alleviate many difficulties in software development. While both ideas hold merit, they present contrasting perspectives on the role of design and the nature of complexity in software engineering.

Daniel Jackson in his essay “What’s Wrong with Git? A Conceptual Design Analysis”, explored Brooks Conceptual Integrity elements. He said that in fact, the elements are orthogonality, propriety and generality (I will go into more details in chapter II) . He argued that consistency is not a part of Conceptual Integrity however, it is essential when exposing an interface to the user. Examples include: (1) graphic user interface, (2) command line interface (3) API. All the external examples must be consistent but shouldn’t indicate the internal consistency of a system. To illustrate this point, consider that the human body is externally symmetrical, but internally, organs are arranged asymmetrically.

What now?

It seems that Conceptual Integrity is a key element in system design. We know that design teams need to decide fast, so in most cases there are many compromises leaking into the initial concept until one cannot notice the design intent. De-facto we understand that maintaining consistency is an irrational challenge that might take a Software design team on an endless pursuit, and unlike theoretical physics, they don’t have the benefit of making it a lifetime project. In fact, we are witnessing that Conceptual Integrity Consistency is a controversial concept among many Software design practitioner and theorist, and I can hardly argue that Conceptual Integrity Consistency is something everyone can agree on.

In the next part, I will continue our journey on design principles. It will explore the elements of conceptual integrity and will tie them to more recent design principles. It may possibly bring us closer to a principle that all can agree on.

References:

• Frederick P. Brooks, Jr. : The Design of Design Essays from a Computer Scientist
• Frederick P. Brooks, Jr. : The Mythical Man-Month, Essays on Software Engineering
• Frederick P. Brooks, Jr. : “No Silver Bullet” https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.117.315
• Joel Spolsky : “The leaky abstraction” https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-abstractions/
• Daniel Jackson — “what wrong with GIT?” a conceptual design https://dspace.mit.edu/handle/1721.1/105233
• C2 for Me : http://marc.tries.fed.wiki/view/c2-for-me/ward.eu.wiki.org/c2-for-me
• Martin Fowler.com, CQRS: https://martinfowler.com/bliki/CQRS.html
• AntiPatterns : refactoring software, architectures, and projects in crisis, Page 111 — “GOLDEN HAMMER”