Improving a distributed team efficiency

The James team have been running distributed, cross time zone Sprints for two months so far. We have two people in Hanoï, three in Lyon and one in Toulouse.

The James project tends to be complex and the team often have to deal, during these Sprints, with large code base changes. As any team in Linagora, we rely a lot on code reviews, that are done before merging changes, during the pull request. These pull request hence tends to be big, and often hard to review. We might need five review pass, or even more, to get these changes integrated to the code base. Without time zone issues, you can hope doing these passes in one or two days with a reactive team. However, with six hours of jet lag between France and Vietnam, the time for integrating pull requests increases to one or two weeks.

Thus, the James team started to adapt. I would like to present some of the changes we experiment here.

Effective pull requests

In this context, handling pull request the right way is important. Some rules that we follow are :

— Your PR should be good. Always re-read it as if you were a reviewer before opening it.
— Split in several reviews whenever you can. Smaller changesets are way easier to merge.
— Discuss design issues on the chat, before the review. This way the risks of conflicts during the review are reduced.
— Reviews matters more than code. Our job is not just writing code, but getting it integrated. We try to do as much reviews as we can during common hours between France and Vietnam. We also need to review all the Sprint open pull request before taking a new ticket. And if we have to handle some comment on a review, we have to delay “in progress” tickets.
— Make review comments as explicit as possible. If they raise some incomprehension, it might lead to at least one more review pass.

Recently, we found an innovating way of emphasizing our care about reviews. It is to be able to add work time on reviews as part of the Sprint time reporting. This way we encourage developers to do reviews, and consider it as their normal and expected job.

One last thing that we do not do yet, but that I would like the team to do… We should always leave a trace of a review we are doing, even when we don’t know what to think about the review (no comment, no approval, no change requested and no questions). Even if it just means leaving a comment telling that you reviewed the PR, and found nothing to say. I believe it will encourage all team members dedicating more time for the reviews.

Cherry-pick programming

Even like this, the delay of PRs is still high. Agile programming, in such situation, would solve this by pair programming. However, pair programming is not possible across time zones.

We then developed “cherry-pick programming”. The idea is, when the people in the other time zone is off, and you see some nice changes to add to the pull request, just do it on one of your branch, and propose it for cherry-pick in the review. Your co-worker can then directly comment your changeset, and you can correct your proposals without time zone issues. When the pull request owner wakes up and arrive at work, he already have a code snipset he can integrate to the pull request. This way we might avoid several “review rounds”.

In the “pros” of this technique :

— We get several team members involved on the same issue
— Agreement is easier to reach
— It is easy to follow, as one people, the pull request owner, put all the pieces together

In the “cons” :

— You need to wait the other location for changes you proposed being integrated :
— For “emergency merges” you still have some delays
— The James CI runs on open pull requests. If you do not open a pull request for testing, your code snipsets will not be tested by the CI until they are integrated to the original pull request.
— Educational purposes. It is not as good as pair programming. We tend to remember better things we solved ourselves, and not do the mistake again. Here, we shortcut the trial and error loop.

Switch of ownership

Another classical things we do is “switches of tasks/PR owner”.

The idea is someone, in a location, starts the job. He open a pull request PR-1.

Then France wakes up. A french developer might take back the task, work on it, and open a pull request PR-2.

Then Vietnam is back on line. A Vietnamese developer will continue the task, work on it and open a pull request PR-3… Until a pull request reaches agreement.

The switch can be discussed between two developers, or might be an initiative.

Clearly :

— This is a bit more messy. Keeping track of ownership requires a lot of communication. It is hard to keep track of the pull request history, and you can get comments across several pull requests.
— The switch of ownership can leave the impression “someone stole your toy”.
— And can prevent you from doing some tasks that might have made you progress

However :

— There is no delay. Ci runs on the pull request, and it might be merged even without agreement of several locations. This is a good way of finishing distributed Sprints.

Conclusion

In a distributed team, be active. Always try to think of what is the best way to get the reviews going faster. And time zone issues, that might be considered as a weakness, can be turned into a strength (in some way..). And even across continents, software development is a social activity that requires exchanges across team members. I hope that this presentation of the way the James team try to optimize these exchanges might give you ideas, in case you run in the same kind of issues than us.