A year of Extreme Programming
About a year ago we decided to experiment with the otherwise left behind practices of XP. We had the CICD pipeline in place but our change lead time was in hours. We practiced unit testing religiously but the deployments sometimes had defects and were reverted. We had daily scrums but somehow it felt like the team was working in silos due to too many work in progress items. We had sonar and other code standard measuring tools in place but nitpicking in PRs didn't seem effective enough unless the person reviewing the code was given the context. So we decided to experiment with pair/mob programming, TDD and refactoring. I recommend that you read a bit about these practices if you are not familiar with them already, I don't want to bore you with the definitions :).
Starting the Journey
During the first quarter of 2020 we spent most of the time getting familiar with the practices. It was not difficult but it seemed different. While practicing TDD, writing tests before production code was new. It helped us think about the business use cases and acceptance criteria before we could think about the strategy to implement that feature. Covering each unit of our code in tests helped us to refactor more as we were confident enough that changes will not break anything or introduce new bugs.
Navigating the other person what to write when you know exactly how to proceed forward really tests your patience. The way navigation works differs from person to person. For some you can just say “filter this list of items to get the green ones” and they would know to convert the list to stream, apply the filter on color and then collect the stream back to a list (at least in Java). For others, you may have to navigate line by line “on line 70 type this.items.stream.filter …”. But we realized that these kind of situations used to come more often when the pair was unclear about the strategy to take, or they jumped directly to implementation without understanding the problem first.
There were times when the solution seemed simple at first, but once we dived into the implementation we realized it cannot be implemented with the initial strategy which was decided during the beginning. At that point we would take a break from implementation and switch back to notepad until we were clear on how to proceed forward.
Sometimes we would reach out directly to the product manager, designers or stakeholders to understand more about their requirements before we proceeded forward. These discussions during the mob sessions helped us to answer one simple question “What problem are we trying to solve?”
When we onboarded new people
Shortly after starting our journey with XP, we onboarded a couple of awesome people in the team. Instead of handing them confluence documents to read we decided to onboard them in a lean way.
After formal introductions with the team on their very first day, they started participating in the mob sessions. The new engineers were encouraged more to take the driver role. The navigator (a more experienced engineer) would guide them about the feature in hand and work collaboratively towards a solution. They would explain to them the value that new feature is going to produce when it gets merged into production. This way we were able to give them not only technical knowledge but also product background. More importantly, they were able to deliver value and create impact from their very first day.
When a new person joins a team they are reluctant to provide their input during code reviews, architecture discussions and product level decisions.
A beginner may think that they are not familiar enough with the system, or they are not experienced enough or maybe they are not sure how the team would react if they challenge directly. Some of these things may be true in one case or the other. Pair/Mob sessions helped us handle these situations. Firstly, when you discuss and debate with a person about how the implementation should be done they are more likely to provide their best input when nothing is implemented yet, as opposed to providing their input when everything is implemented and it comes as a pull request. Secondly, mobbing sessions are a way for the whole team to get together and solve one problem while discussing ideas. This created an environment for everyone to contribute, share their thoughts and ask questions. So the nature of XP promotes input from different people who are part of the session.
There are some cons to pair programming as well which were brought up in our retros. The most important one of them was that constant guidance from experienced members stopped new members from exploring different things themselves as they would get everything served on a silver platter without much work, they felt being micromanaged.
How was remote pairing
As the offices went to hibernation during the second quarter, we switched to remote working. By this time we had gotten our hands dirty with XP practices. Pairing remotely is a bit different. We faced a few challenges in the beginning.
We were not sure when and how should the driver and navigator switch roles. This was trivial while working in the office. We just had to switch seats after a particular time (mostly 15 minutes) even if the code was in RED state. While working remotely we had to ensure that we kept the code in GREEN state more often so that we could commit to the trunk without affecting the CI. And so while getting the code to GREEN we would often miss the switching deadlines.
Another challenge in the beginning was about collaborating and working together towards a common solution. XP practices encourage collaboration in the team to create a more synergistic outcome that cannot be created individually. If two people are working on the same problem with different ideas in mind they should be able to come together to create a third alternative which is much better than the initial individual ideas. And as Kim Scott puts it: “To come together for a better solution you need to care about the people you work with”. So we started having remote hangout sessions where everything else would be discussed apart from work.
Getting to know your peers is really important to create an atmosphere of feedback and learning. If you know your pair well you can bounce and discuss ideas with them in a candid manner. Else the session would be more like junior writing the code and the senior dictating them. This is not good. This will ultimately reduce feedback and innovation.
As you may have heard or experienced yourself that remote work brings silos with it.
Remote work reduces interaction and communication with your colleagues. It may seem fine during the early days. In fact it may even seem like a great setup for many of us. But as days pass you start to realize that silos and lack of communication hurts your productivity. You only meet the team during scrums, planning or hangout sessions. Rest of the time is spent working independently. We were lucky enough to not have faced these issues.
Following XP remotely allowed us to work collaboratively. We spent most of our day in zoom virtual rooms, solving problems together. We shuffled pairs often so that every one got a chance to work with each other.
Reducing change lead time
We were able to produce a significant dent on our change lead time. As at least two people were already working together on a feature so there was almost no need of creating pull requests for review. During the session the navigator ensured all code practices were being followed while guiding the driver. This reduced the defect rate in code as well.
Another thing that helped us to reduce CLT was to limit the WIP (Work In Progress) items. Too many wip items are a silent killer for a teams productivity. We realized that switching between tasks too much caused us to lose focus and so ultimately increased delivery time. So we re-evaluated our definition of Done and decided to mark tickets as done only when they had been released to our customers. If at some point we realized that we were not being able to release a feature to production due to some xyz reasons, we would work together as a team to resolve bottlenecks in mob sessions. And yes, that would mean not picking up any new thing unless the issue at hand is resolved. If the feature was stuck due to dependency on some other team then in that case it seemed meaningful to pick something else.
We defined CLT as the time between commit to production and the customer being able to use that feature.
On knowledge transfer
Pair programming helped us with getting the knowledge across the team instead of depending on one person. When a member of our team went on leaves we would always have another person who had the context of things which we wanted to work on. This speeded up the delivery. It also helped us in refactoring code more often as when we revisited our old code we were able to point out things which needed improvement before incorporating the new feature.
At Careem we have 100s of microservices. When people leave they usually transfer knowledge and ownership to the wider team about the things they worked on. Delivering that knowledge altogether in 60 mins knowledge transfer sessions is sort of just over populating too much information in too less time. It doesn’t serve the purpose, it relieves them of the responsibility. Following XP practices allowed us to transfer knowledge in real time rather than cramming it all up in one session.
Learnings along the way…
Some of the things written above might seem too good to be true. There are a few things which a team must consider to truly reap the benefits of XP.
- Create a culture of learning, feedback and reiterating. The way we were able to produce these results was by constantly reiterating and improving on things which we lacked upon. We would sometimes do a retrospective soon after an XP session to get an impromptu feedback. And these feedbacks from everyone in the team helped us to improve our next sessions.
- Measuring kpis to gauge performance of an activity helps in understanding if we are really going in the right direction or not. In our case, we started measuring CLT, incidents, reverts and software defects. This helped us in identifying if XP delivers what it promises.
- Create a culture of working together as a team. Synergy says 1+1=3. You cannot get that 3 unless you are able to produce another 1 which comes from working collaboratively and creating something which cannot be done individually. This involves the stages of listening patiently to ideas, clarifying them, debating on them, deciding on which one to pick, persuading that every one should be on the same page, executing on the decided idea and then finally sharing the learnings after execution.
We started this as an experiment for one sprint. Why don’t you try this in your team as well?
Acknowledgement
I would like to acknowledge and thank Lamak Qaizar for introducing the team to these practices. Without his constant guidance and efforts it is safe to say we would not have reached where we are today.
