Growing While Remaining Productive
By Rich Luick and Gennadiy Shafranovich
We gave this talk at Droidcon 2018.
The Android team at Peloton has grown a lot recently. Our team has grown to 5 times the number of engineers it was 18 months ago, and what we have learned during this rapid growth is that teams need to work differently at different sizes. This is evident in both the ideal workflow and communication patterns of teams.
Small Teams
Smaller teams are often concerned with the rapid delivery of new features. They are trying to get new things shipped and are often only focused on one or two features at a time. Communication is easy and direct within the team as well as with other stakeholders such as product or QA.
Larger Teams
Larger teams may find themselves broken into sub-teams and working in parallel on a diverse set of features, often on different timelines. Communication is no longer direct across the greater team and yet is even more important since sub-teams can develop their own patterns that could end up clashing.
So, how do you grow while remaining productive?
First, we have to define what we mean by a team being “productive.” For us, this includes engineers being efficient when developing, having features be of a high enough quality to be releasable, and delivering said features to our end users. All of this needs to be done while not overloading any personnel or tooling involved in the process. Being able to make technical and process decisions as a group is part of being able to stay productive as a team.
Make a space where open discussions are held.
At the core of the process for making decisions together is a space where members of a team can have honest and open discussions. It needs to be an inclusive venue where any member of the team can contribute as little or as much as they are comfortable with. Individual seniority doesn’t matter in this venue, except for when it is used to backup a particular line of reasoning with examples and experience. No question or issue is too small to bring up either, including team processes..
At Peloton, this is a weekly one hour meeting where all Android team members are invited. A document is created after each of these for the next week’s agenda, and anyone can contribute to it throughout the week. The first thing we do in this gathering is review the previous week’s action items in order to keep everyone accountable for tasks that they signed up for. The rest of the meeting is spent in discussing issues on the agenda and making decisions as a group.
Your decisions must be actionable.
The decisions you arrive at as a team need to apply to all the members of your team. If individuals or groups are excluded, you risk creating tiers within your group. Some decisions can be made in the room with just a short discussion while others will require investigative work or a proof-of-concept project to back up the decision. Assigning this work to volunteers in the group as action items helps move the work forward. Remember, you will review these action items in following meetings and receive progress reports from the assigned volunteers. Once they are done and present their work, you can make a decision to proceed or not as a group.
Communicate your decisions…
The decisions you make won’t matter much if people are not aware of them. What convinces the rest of your group that the right thing is being done is knowing the “what” and the “why” of each decision. Documenting these improves communication and has the extra benefit of making onboarding of new team members much easier.
…and enforce them
The trust behind this process is that the decisions you make are enforced by the entire group. It shows your team that if they follow the process and bring up an issue or suggestion, it will be discussed and any outcome will be communicated and enforced.
Where possible, you should use tooling to do this. It is easier with things like code style since a simple lint rule can cause pull requests to be flagged. Enforcement becomes a bit harder with things like patterns (e.g. MVVM) or usages of specific libraries.
For decisions that are more difficult to automatically enforce, use code reviews. Your entire team is aware of the rules and should comment on code when these are violated. This is also a great way for new members of the team to learn the “right way” to do things. They are simply pointed to the right pieces of documentation as they go through their code review comments.
When things change, iterate.
While trust behind the process is enforcement, the real power of it is in iteration. The reality is that your environment will change over time. This can include the makeup of your team, your requirements, the ecosystem you are developing in, or all of the above.
Use your discussion venue to revisit previous decisions and check if they are still applicable to your current situation. Questioning your past decisions can also lead you to find better ways to solve the problem. A good sign that it is time to revisit a decision is when the original reasoning behind it no longer applies. Basically, if the answer to the question of “why did we do this” is something that may no longer apply, it is time to revisit that decision.
Iteration also allows you to make a good decision, and not wait to make a perfect one. Any move in your team that improves your current situation is a positive move. It also allows you to question the process itself. If the way you setup your discussion venue is no longer working out, discuss it, and see if the group can come up with a better solution.
Example: The `googleJavaFormat` plugin
Just like most growing teams, our Android team at Peloton had issues deciding exactly what code style we should follow. As most readers will know, this means religious arguments on the number of spaces to indent a line, where do curly braces go, and most of all what should be your the maximum line length. Just like most teams we used our discussion forum to work through these one by one.
However, during one of our weekly team meetings a team member suggested we use a highly opinionated formatter instead of trying to decide and enforce every formatting rule. We investigated what this would look like in practice and realized that while no one will be 100% happy with the outcome, the formatter completely removed the ambiguity. There is one way to format your code and that’s it. This completely removed the cognitive overhead that was connected to Java code styling and formatting.
Example: Github CODEOWNERS
At Peloton our Android team ran into the problem that all growing teams face. One engineer made a minor change and a fellow team member reviewed the change and approved it. This is not an issue of course, this is the regular course of work, but for the reviewer, the change did not include enough context to flag issues that may arise. The story then follows a very predictable course, ending with a bug in production. All of a sudden, a tiny change in the code base causes unexpected behavior somewhere else. The underlying cause has to do with the structure of the code base and the component changed. No component should be able to cause this big of an unexpected change.
The two engineers involved brought this up in one of our weekly team gatherings and the group proceeded to discuss long term solutions. Something that came out of the conversation was the usual “how can we prevent this for now?” question. Someone on the team had experience with GitHub’s CODEOWNERS feature and we decided to immediately add code owner definitions to sensitive parts of the code base. This means that team members with context on specific areas get added to pull requests and get a chance to glance at them to ensure strange side effects don’t happen. We also instituted a rule that if the author and the reviewer do not have full context on a component, and the code owners set up isn’t suggesting a reviewer, it is up to the author to find who to add in order to get a thorough review.
Considerations
We’ve had plenty of success using this process, but as we all know nothing is perfect. There will always be some things that require thought and consideration with any process, including this one.
Continued Growth
We’ve successfully scaled this process from 5 to roughly 20 engineers and found that it worked well within that range. There is the feeling that as we keep growing, it might be difficult to get everything done in one meeting. The plan here is to start to split up to more concentrated town squares and go from there.
Minority Opinions
It is nearly impossible to get a group of engineers to all agree on something. Even if we have a majority opinion we may end up with dissenters who are passionate about their point of view. One of the most important parts of this process is that everyone needs to be heard. All ideas need to be considered and we need to be ready to explain why something is not right for our team. Be sure to document the “why” here as this alleviates the stress of explaining these decisions to new team members.
Action Items With No Progress
Not having forward progress on action items can lead to losing trust in the process. Team members need to be held accountable for tasks assigned to them to ensure we maintain momentum. Remember to review these items at the start of each town hall and ask for assignees to provide an explanation for why certain tasks have not been completed.
Time
The town halls and all the work that comes out of them take time. It is imperative that the rest of the organization buys into the process and understands that time spent now is better than having to spend even more time later on. Every action item we complete should help diminish how long things take down the road.
But it’s worth it!
This process has allowed us to adapt to a growing team, with changing requirements, in an evolving ecosystem while delivering consistent, high quality work, with a low defect rate. Everyone has a voice and engineers are empowered to push for positive change. Finally, we have a clear process to improve and learn.
