Software Engineering Whirlpool
Software Evolution
The traditional analogy of considering software development as the construction of an architectural building is absolutely wrong. A better analogy is to regard software construction as the evolution of a biological being.
Hence what is constant is change and evolution. This can be described by an analogy to a whirlpool where the movement dynamics are driven by many factors and shape the software and the data in continuous manner.
This scheme can be used to keep in mind, all the important things that must be considered while designing good, and evolving, software solutions.
Foundations
Any biological software entity depends on good genetic foundations.
A software project must be based on excellent infrastructure which is provided by the team, the company and even the country or region. This good infrastructure must be a multilevel requirement involving performance, reliability and security. Regrettable is to forget that what support everything is company employees. Additionally, if they do good work they are usually invisible. They are the true pillars of the entire structure.
Above this level resides the Development Infrastructure, which includes dev tools, Slack-like and Git-like, and the set of processes that may conform a DevSecOp continual-operational environment.
Training is sometimes forgotten as well. At least training space should be allocated. The software evolution requires a continuous evolution in terms of technical skills as well as in career-development for team members.
Taming the world
The world is complex and fuzzy. The best we can do is to create a model of their inner workings. The uncertainty is pervasive and will always be there. Additionally, besides having a stochastic component, it is always changing. Embracing change is not only a necessity, is the way every software project should be tackled.
The first net to throw to the world is the Business Domain. It’s a fallacy to consider that business specialists are not needed. Someone familiar with the business jargon, someone who understand how to calm the evolving beasts of the business should be there to be the first filter to tame the lion.
An elephant is what we need to perceive. However, a bunch of blindfolded Business Analysts would have to understand the elephant only from the perceptual information they can get without seeing. Everyone would end up with their own representation.
The requirements have FURPS+ components. And yes, they are always evolving, changing, morphing. Transforming into something new, something different. And what usually drives this evolution are business ROIs. Behind every requirement there should be some connection to the business, some drive to reduce costs or increase revenues. Hence, diving deeper is what might be expected from everyone gathering requirements. And they should be sieved through a CBAM analysis: “what is the cost of this new gene expression?”. Nature performs this filtering by natural selection.
Consistency, Availability, Partioning, Usability, Supportability, Security: a striking balance, very hard to maintain and to evaluate. The balance itself can shift sometimes, depending on the business context and the corporate own evolution.
Designing a new being
Nature thrives for simplicity by building complexity. Things should be kept simple, but not simpler. Anything that can be used to foresee, as much as possible, the outcome of any decision, is absolutely welcomed: mocks, stubs, tools, simulators, emulators. Nature, on the other hand, can waste. And it does. Countless of species have come and go, many even without any trace behind. But a software project is resource-limited so it must fit a deadline, a budget.
Frameworks, components and patterns are here to help to constrain the required resources. These are the pillars of the tangible developed software. Do not reinvent the wheel (Nature did that many many times with evolutionary convergence, and so did humanity inventing writing five times).
Company’s Design Principles should be documented, as widespread knowledge. And they should be clear. They feed on the knowledge of tools, and patterns, and overall, previous experiences that are known to the business and to the “Architect”.
Evolving Features
The digital world claims for quantified requirements. Everything that must be built should be kept in some numeric form. Qualitative demands are subjective. These changing requirements feed a Product Backlog which is a lists of what needs to be done.
On the other side, the problem context inspires a Model Solution, which is balanced by the forces of the meta-characteristics. The Model is also a source of requirements that go back into the Product Backlog. Simplicity is the mantra.
Some things will be beyond your team knowledge. Or even beyond humanity knowledge. For those challenges, prototyping and iterating.
The Model is refined all the time, according to the feasibility of the Universe, and according to the fact that everything is changing (including our own understanding of the Universe). Separation of concerns, Encapsulation and defining Boundaries and Interfaces keep a healthy and reliable Model.
Construction
The business guides the priorities of the Product Backlog. This is fuel to the construction engine of the Development Cycle.
The time required to obtain a user story from the Product Backlog and devise how to implement the specified feature is the time task. Using the specified Development Environment, the compile cycle, which constructs a build, providing continuous availability and continuos integration. Keeping small batches, maintaining looser coupling between different components (so change is not widespread) and better coherence within components.
The result of this process is a deployed build, into Deployment, out into the wild.
Cleaning up waste
A system should never be deployed in a Big Bang, it should be increasingly deployed, step by step, in a controlled roll-out. Antipatterns should be avoided. A pattern for everything is a sign of warning. Ass-u-me, we assume something and we all loose. Git organize how everything is done, but commit-and-run is risky, always. The software rot. That is the time to evaluate a new version from scratch. Metaphors are good to explain something in a meeting, not as a basis to design something (oxymoron in this blog).
Testing
Quality Assurance, automated testing and regression tests are the scaffolding that allows the software to evolve in the real world. Without them, software would evolve into a useless state. The deployment of new versions arise tech risk, that must be controlled in a Risk Tracking and Management.
The bugs are notified, detected, analyzed and overall registered. A bug which is registered and understood is a substantial difference between something which is not even known. Bugs end up being back again into new elements of the Product Backlog.
The fixed-to-be bugs and all that is created from the creation process is part of the Technical Debt that is accumulated as long as the entity is alive, breathing and working, and evolving. It is like the clinical history of a patient that in some way is only accumulated.
The soul of the system
The soul of the system is the Architecture Solution which surrounds the most important asset which is the Data diamond. The Architecture Solution is the living organism that is always evolving, feeding from waves of change that come from everywhere.
Performance and Security Testing (pentests) are always performed that check the state of the living organism. It is never too early to improve performance. It is never too late to improve security. This is done through continual measurement and profiling of the entire solution. Measuring is objetive and quantitative. Tuning is fed back into the system, to adjust parameters and generate a new inflow of information that traverse the entire whirlpool. How the system performs, how it operates, into what it evolves, is indeed a new basic template model that can be harnessed, and from where it can be learned how to do things in the future.
References
- Mens, Software Evolution, Springer 2007
- Philippe Kruchten. 2003. The Rational Unified Process: An Introduction (3rd. ed.). Addison-Wesley Longman Publishing Co., Inc., USA.