Extreme Programming (Part Three): Incremental, Insured, Collective Engineering — Symbiont.io
By: Anna Savarin
Welcome back to our three-part series on XP, or eXtreme Programming, from the Mortgages team at Symbiont. Check out the earlier posts in the series, where my colleagues Alex Bennett and Jeffrey Lin shared the business owner and product manager perspectives. As an engineer on Symbiont’s Mortgages team, I will share my point of view on how our team has adopted XP practices and their impact on our work.
Getting Started: Benefits of Incremental System Design
Our client is asking our team to build powerful and complex software. Great! Where do we begin? Between the blinking cursor on our proverbial blank page and a fully functional product with thousands of users, hundreds of screens, and massive throughput of data, lies a long road. We are talking about years of work as well as teams of subject matter experts (SMEs), designers, and product, software, and infrastructure engineers.
Like any daunting task, the answer lies in breaking it down into smaller pieces. One way to tackle these pieces could be to build them one by one, to completion. However, those of us who write software know that life is not so simple: these pieces will interact with one another and reveal information about each other and the system as a whole. You cannot get away with building one and then calling it done — you will certainly learn something down the line that will cause you to revisit. So if you cannot predict how the pieces will interact, or how they will shape the overall design, it can be more effective to get to know your future system incrementally. In practice, this means building a small enough portion of each piece at a time to put a skeleton in place, something that partially works end-to-end. You can then continue to refine and adjust as you come back to your pieces and add more “meat” to them.
Often, even in agile environments, development teams are tempted to build components and modules in a depth-first sort of way. They overengineer the foundational layers, only to discover later that the target they have been optimizing for has diverged from how the business requirements and the code have evolved. We have all seen those giant architecture design documents and kick-off meetings followed by lengthy implementations. In the XP approach, your design will come together incrementally, as a continuous course correction informed by hindsight observation rather than a prediction. Your task as an engineer is to ensure that your system fits your requirements on exactly that day. The next day, when you inevitably learn something new, you will refactor your code to enhance your skeleton, and to make sure that your system has evolved to meet the new requirements.
Such an incremental approach has enormous advantages to both the engineering team and the client. When the time comes for you and your team to tease out architectural patterns, you do so based on facts of what is already written. Unlike heated and unproductive debates rooted in forward-looking intuition, this method facilitates convergence based on actual, existing code. It also offers early visibility into the system for engineers and for users. Engineers can de-risk parts of the system they do not understand by building a small enough portion to know which questions to ask and to answer.
From the client’s perspective, they have an improved experience as well. The incremental approach delivers value to the customer in the shape of small features that are enhanced incrementally. This establishes a continuous value stream consisting of these small pieces of functionality. Clients have the chance to step in and revise their requirements as development continues. This type of flexibility is key, as clients are also getting to know the future system, from the user perspective. This process gives them room to address the edge cases that inevitably come up. Engineers can take wrong turns that are low-cost. So can clients.
Test-Driven Development and the Safety Net of Automated Tests
With this description of the incremental approach, I hope to convey a sense of movement. The code is volatile; it bends and twists constantly. Refactoring is a continuous task, aimed not only at improving code maintainability, but also at formalizing the emerging patterns we observe, as well as fitting in new requirements that arise.
You might be wondering, how can we keep changing our code so much and be confident that our system still works? We have the comprehensive suite of our unit, integration, and end-to-end automated tests as insurance and crash pad. It is the result of the XP — and our — TDD (test-driven development) approach to writing software.
TDD is valuable not only for the resulting safety, but also for the mechanics of the development process itself. For an engineer, it can be easy to become overwhelmed when building or enhancing a feature: your mind can instantaneously sprawl out to dozens of use cases and edge cases, leaving you stuck and paralyzed. Writing a failing test somehow breaks this pattern, and focuses your attention on passing the one case. Perhaps this is because it puts you in the shoes of the consumer of your code — as a result of which you also design better interfaces. Writing that first failing test is much easier than that first line of code.
It is also easier to continue writing failing tests incrementally in order to keep working up to your requirements. You go from failing to passing — red to green — and, ding! You get that jolt of satisfaction, and you reach for more. Next test, and next test, and you are in that coveted developer flow, where you “wake up” hours later, with working code and an insurance policy for your future self.
The level of confidence you have when you go in for a refactor while insured by a test suite makes all the difference. Writing code becomes a source of pleasure, not anxiety. You are free to experiment, turn the whole thing upside down, make changes as deep as you like, and you can rest easy that your tests will validate that your code still works.
XP consists of many other practices and, frankly, I can rave about it for hours. In fact, there was too much to fit in one blog! In a follow-up piece, I’ll dive deeper into the collaborative aspect of XP and our unique approach to working together at Symbiont.
For all of its payoffs, XP is also hard. It sets a high bar for excellence of results, but it requires rigor and commitment from everyone involved. What I appreciate most about XP is that it embodies a kind of reverence and care for the product that you build. My sincere personal hope is that XP gains wider adoption in software development and everyone builds better products that users can trust as a result.
Originally published at https://symbiont.io on February 14, 2020.
