Science and Engineering in Software Architecture

We are passing through tough times, “The moment everything changed” with COVID-19, quarantine, people trying adapt their day to day, companies not prepared to work remotely… On top of that, one thing that I have seen is a lack of science in different aspects. Lots of assumptions, lack of data and a complete chaos.

In engineering, one thing that we don’t want is chaos — WHAT?! But what about chaos mindset, chaos engineering? — don’t get me wrong, as engineers , we accept chaos and live with it, but build stable and resilient systems on top of it.

According to wikipedia, “Scientists study things by looking at them very carefully, by measuring them, and by doing experiments and tests. Scientists try to explain why things act the way they do, and predict what will happen” and also “Science uses mathematics and logic, which are sometimes called ‘formal sciences’.”, “ Science produces accurate facts, scientific laws and theories”. And as you well know, against facts there are no arguments.

Science is about exploring, experimentation, discovery and, of course, data. Please, data. Let’s base our propositions, our argues and discussions on concrete things.

“We don’t need experiment, we just need to built it. I know how and what to do”. It is not about that. It is about understanding the rules of the game, how to play, strategies and play it.

Architecture needs science. Before you think about components or technology, you need to think about your code, your algorithms. If your code is poor, there is no architecture that will scale or save it from failing in production, so things like data structures, time and space complexity are essential. Like design patterns, that you probably know many, its important to know different data structures (dictionary, dynamic array, BST, heap, etc.) and algorithms (sortings, Dijkstra, breadth-first search, a-star, etc.), understand the use cases and the concepts behind this things. If you have a good code, your architecture can leverage exponentially the potential of your software because you have built a strong foundation. Also, your code should walk side by side with your architecture decisions, so remember to always ask questions like: Will my code work running in parallel? Is there no shared processing that could fail using this strategy? Is my code isolated? Is my code immutable? What is the impact of this algorithm to the architecture?

So the algorithms, code complexity and data structures are important things to be part of a scientific method. You need to keep in mind since the conception of an idea or a new service design, given that it will make direct impact on business results. Also, science should be considered in all parts of the development lifecycle, not only on the foundation part but in the whole delivery (leads, front end engineer, back end engineers, etc.) and discovery (POs, BAs, UXs, etc.).

Ivory Tower

Don’t be an Ivory Tower Architect. Don’t be a PPT architect or the guy who thinks their job is to attend meetings. If there is nothing to discuss, He will invent a reason to set up a meeting.

I’m putting here some points that may help you not being that guy:

• Write code. Don’t be the guy who probably hasn’t written a line of code in 5 years. You need to eat code as breakfast;
• Discuss tough problems and solutions with others;
• Be available to help. You are not there just to write boxes on a presentation, you need to be there to unblock the road during implementation and help others in design/architecture matters.
• Have ownership;
• KISS (Keep It Simple, Stupid!);
• Less ego, more humility;
• Trust the development team. I see lots of architecture teams, that think that developers can’t think so they need to make all the hard work, write lots of requirements and libs so the developers can’t fail. By doing that, basically you are putting blinder on developers preventing them seeing to the rear and to the side, preventing developer’s evolution and the architectural thinking, making the architecture team a bottleneck and building an architecture completely coupled. If we are building a huge system, we need to think about scaling the development team and for that is essencial to give developers responsibility. So let they mess things up, failure is the best teacher. Teach them, and build a high bar team.

Ivory Tower vision

Scientific method

I’ve commented about scientific method before but you may be asking yourself how to apply it to the software engineering world. Actually this method doesn’t just apply to engineering, but also to the business discovery or ux discovery parts. The scientific method is a systematic way of learning, answering questions and solving problems. The key difference between the scientific method and other ways of acquiring knowledge are forming a hypothesis and then experimenting to validate a proposition.

Engineering needs different options. As architects, people expect us to have an ace up our sleeves, that will save the day with a magic solution. But theses “aces” don’t appear from day to night. Therefore, we need to research and experiment things to have options. But only having available options don’t say much, so we also need to compare that options (it’s always about tradeoffs), know their risks and constraints, and also their results.

I’m putting here five steps that guides me in my day-to-day dealing with real engineering problems:

• Problem / Concept: Start with a scenario and a problem that you want to solve. You need to ask questions about it to understand what really is the issue and why you want to fix it.
• Research: Research, think about the knowledge you need to investigate this problem. Google it, read articles, checkout new technologies, similar codes, ask for help.
• Hypothesis: This is a possible solution (or a group of solutions) based on the problem and your researches stating what is the expected outcome and the tradeoffs of this proposal.
• Experiment: Test and validate the hypothesis, write down a scenario, code and validate the outputs.
• Conclusion: Analyse the data from the experiments runned and compare them. With data the things seems more clear and obvious, as you obtained experience. If it’s not clear at all, now you have a more concrete and well defined scenario and you can involve others. It’s important to document this conclusions, so you, your team and the company can continuously learn in scale.

You may have noticed this is an algorithm, and it’s not a coincidance that its pretty similar to many other software methods, because all of them inherited from scientific method.
Do benchmarks and POCs. Benchmarks can be tested repeatedly and quickly without requiring manual work. They will help you to get rid quickly of assumptions and poor techniques, and direct your attention to successful techniques. POCs can help you validate an architecture solution on tough scenarios during experimentation, validate non functional requirements within a possible solution, mitigate “fogs” in your team and also give your team orientation and guidelines (you can check out this post https://medium.com/@gusdlim/poc-as-an-architecture-output-cf0ef4a42e10).

It’s easy to say things and judge people from the ivory tower, so get down from it and do more. Allow yourself to experiment, learn, evolve and bring more science to your team’s day to day.