This is how we roll
The developer was working on his task, focused and determined. Numerous commits were made and tests were written. Yet he did not see the bug crawling behind him. It remained in the darkness, waiting to break the app in the most violent way. When came the time to pass acceptance testing, the bug remained hidden, waiting for an opportunistic strike. As soon as the feature hit production, it was already too late. The amount of 500 errors exploded and forced the team to an embarrassing rollback.
This kind of scenario was occurring from time to time in our small dev team. At the time, we were only two developers and one product owner (who did some dev too). We are now four developers, one product owner and one product manager.
Over the course of a year, we came to an organization that dramatically reduced the number of bugs we shipped to production. It is based on two components:
- a process followed by the team to ship a feature
- a set of best practices that enhance code quality
In this post, I will present our development process and how we defined it. It will lead me to talk about the tools we use and why. Throughout the post, I will also list the practices enhancing our code quality, alongside some tools that help us in the process.
Like millions of development teams throughout the world, we use agile development methods. That means our team has very short development cycles, called iterations, over two-week periods.
Jeremy, our product owner, manages a backlog of tasks needed to improve the product. It is balanced with high priority tasks, such as bug fixes or features required by our current and potential clients. When those tasks are identified, he lists and prioritizes them with Julien, our product manager. Eventually, he selects a bundle of tasks he foresees for the next iteration. Jeremy manages each iteration with Target Process. It’s more powerful than Trello, and we can use it for free.
Once tasks are listed and prioritized, we have to estimate the time they require before they can hit production. All developers meet with Jeremy and review each planned task.
At this point, we regularly ask Jeremy for more information and discuss basic implementation. How would I get started? How would I organize my work? We simplify the workload estimates using t-shirt sizes:
- XS < 1d workload for a single developer,
- S for 1d,
- M for 2–4d,
- etc
The final description and estimation for each task are updated in Target Process for further reference.
This estimation meeting is essential for the team to elaborate on new features together, from architecture to workload and avoid pitfalls.
Even though we use Target Process as a virtual Kanban board, we also use a physical one. We realized it was more pleasant and easier to just look up and see what everybody was up to at a given time. It also helps focusing on what is important and removes the clutter created by backlog or future ideas.
Moreover, it is the central meeting point for our daily stand-up meeting. It is crucial to make it a ritual. It is done at the beginning of our workday at around 8:40am. Everyone talks about what he accomplished the previous day and what he plans to do today. This meeting is important to keep everyone on the same page in regard to the product development or business. It is also the right moment for each developer to share any concern he might have and ask for help (even though he is encouraged to ask for help any time during the day). It’s the best way to avoid the tunnelling effect a developer can experience when working on a long task alone.
Once the tasks for the current iteration are positioned on our board and ordered by priority, it is time for work! Each developer takes the first task in the list, and when done with it, takes the next task on top. In fact, as a developer, you will never know what you will end up coding. We believe this has two positive effects:
First, it forces developers to get their hands dirty with all aspects of the product, regardless of the technology. We use Ruby On Rails, Android, iOS and Windows Phone. The benefits? Increased code ownership and skills.
Second, it helps with keeping the same bearing during an iteration. We said what we do and we do what we said; in order.
When a new feature is a bit tricky, we like to peer program. One might think that it “uses” twice as much developers for one feature but the benefits outweigh the cons. First, you have two brains instead of one, it leads to a sort of instant code review. Questions are raised more often than when alone, and having to explain your train of thought exposes more easily the imperfections of your future implementation. Second, it is a good way to bring somebody up to speed on a new technology or on our code base. It is a very important part of our onboarding process. Last but not the least, it increases code ownership. Every member of the team should be able to work on any part of the project. We have nothing against specialization though. But we want to avoid the case when the developer is on vacations and nobody knows the code to fix; add to the mix pressure from angry clients and you have a very bad day for everybody!
Another way to avoid this caveat is to do code reviews. Another developer (not the one who developed the feature) looks into your code and searches for bugs, more elegant solutions, better variable and function naming, etc.
No line of code can be shipped to production unless it has been reviewed by a team member. Here is the process: when he is done developing the task and all tests are in the green, the developer creates “a diff” on Differential. It is a tool part of Facebook’s Phabricator, a collection of open-sourced tools that help teams manage their projects. It comprises a wiki, a bug tracker, a code review tool, a Kanban board, …
A diff is just an aggregated view of the commits for a given feature.
All team members are notified via Slack that a new diff is awaiting review. When one of them is available, he logs into the team’s Phabricator and reviews the code. We decided to do code reviews asynchronously, because it is not always easy to synchronize with another team member at a given time. But we do synchronous code reviews whenever possible.
Everybody is listed as a reviewer, so anyone can review and comment on a diff: we like to have as much input as possible from all team members.
At this stage, it is of paramount importance to apply “egoless programming”. The reviewee must keep in mind that the reviewer comments on the code and not on the developer. The reviewer must focus on the code and not the developer. Everyone must bear in mind that overall code cleanness is the goal.
Once the diff is accepted, the developer is notified via Slack and rebases his branch on master and pushes his code. He moves the ticket from “Code review” to “Done / Ready for demo”.
The product owner then makes sure the developed feature is in line with the user story he detailed and the customer’s expectations. If that is the case, he places the ticket in the “To be released” column and the feature will be shipped to production as soon as possible.
We used to ship only once or twice a sprint (two-week periods), so our deployments to production comprised many features, bug fixes and changes. It was quite difficult to pin down a bug when one occurred.
We now ship as often as we can, which is part of the Kanban methodology. Deployments are small, so when something goes wrong, it is very easy to find the guilty lines.
During a sprint’s course, things happen, bad or good. So at the end of a sprint, the whole team gathers to speak about the previous two weeks. It is a privileged moment when we celebrate our successes, bring new ideas to the table, or offer suggestions to improve both individually and as a group.
The key is to remain constructive at all time and keep in mind the things being said are not meant to harm. We use the “Keep Drop Start” approach. For that, we divide a board into 4 parts:
- A smiley face: the area where we celebrate, talk about the good things that we must keep doing
- A sad face: the things that annoy us or went wrong that we must drop.
- A light bulb: new ideas we would like to share to the others.
- A question mark: things that are unclear in our mind, that puzzle us.
We write what we think on post-its and pin them in the corresponding area on the board and elaborates more on the subjects. When everybody is done, we discuss each point and plan future actions. The best way to ensure things will be done is to have someone become owner of the task. The opening of this very blog is the result of a “light bulb idea” and I took the lead to write its first entry. And even if I am the one writing it, I ask help from every member of the team and from our friends at elCurator who are in the same office. Being the lead does not mean you are doing it on your own.
That’s it, you now know everything about the development process at Appaloosa! Of course, things that worked for 6 people will certainly not work for 12. As the team continues to grow and goes through sprint after sprint, our process keeps evolving.
If you’d like to discuss more on the subject, or if you think you’ll love working with us, contact us at feedback@appaloosa-store.com.
i++
