In-Depth: The Costs And Benefits Of Pair Programming
Four important insights from scientific studies of pair programming versus solo programming
Pair programming is a common practice in Agile teams. Although it has been around for a long time (Coplien, 2015), it was popularized by Kent Beck as part of Extreme Programming (XP) (Beck & Andres, 2004). But the practice remains controversial. The notion of letting two developers pair on one task seems wasteful compared to solo programming where each developer work on their own task. However, proponents of pair programming argue that it increases quality, speed, and learning, and thus overall productivity.
We have always been a fan of pair programming. We’ve learned tons by working together on code with other developers and have been able to teach others through it. There is something very satisfying about passing the keyboard back and forth and sharing the mind space for a problem with someone else. So while our belief is certainly that pair programming is a critical practice in Agile software development, we also want to challenge our beliefs with empirical evidence.
So in this post, we explore the insights from over 25 academic studies that we read while preparing for this post. We summarize them in four core insights and also offer five practical recommendations. The insights may support your beliefs or challenge them.
This post is part of our “in-depth” series. Each post discusses scientific research that is relevant to our work with Scrum and Agile teams. We hope to contribute to more evidence-based conversations in our community and a stronger reliance on robust research over personal opinions. As you may imagine, it takes a lot of time to compile such evidence-based posts. If you think our content is valuable, and you think we should write more of it, you can support us on Patreon.
What is pair programming?
In its formal form, pair programming refers to the practice where two developers work together at a single workstation on a single problem (Beck, 2004). In this pair, one developer acts as the driver and writes the code while the other is the navigator and reviews each line of code. These pairs should be dynamic in that partners switch with other pairs frequently to spread learning across the team.
Beck (1999) originally reasoned that pair programming has three benefits over solo programming: 1) the quality of the code is higher because two minds are reviewing it simultaneously, 2) developers learn together and from each other, and 3) the overall duration to complete tasks are shorter because developers can work more effectively. Of course, these benefits weigh against the (superficial) cost of having two developers spend hours on a task that would otherwise be done by one developer.
Beck & Andres (2000) argue that pair programming has so many benefits that it should be practiced for all code that is written. Other authors take a more pragmatic approach and recommend it for critical tasks in particular. Many teams also practice ad-hoc forms of pair programming where the distinction between driver and navigator isn’t strictly upheld. There are also larger versions of pair programming, called mob programming or ensemble programming, where one driver has more navigators (usually the whole team).
A note on costs versus benefits
The main controversy of pair programming is whether the benefits outweigh the costs. Is it worth it, in the long run, to have two developers spend time on a task that would otherwise be done by a single developer? We’ve personally dealt with quite a few developers who felt that pair programming was wasteful. Similarly, management is not always on board because — in their minds — it translates to “twice the time for half the work”.
We’ve noticed that such skepticism is often brushed away by proponents of pair programming. We certainly have many times. While proponents often acknowledge that pair programming may result in more person-hours spent on a task, the increased quality and shared learning are completely worth it in their minds. Any skepticism towards pair programming is then framed as attempts to “squeeze as much work out of developers as possible”. But this is a simplistic, and probably unfair, characterization. It’s easy to step over such financial concerns if you’re not the one paying the salaries of developers.
We’ve had the pleasure to work with many company owners who were (retired) developers themselves. While they absolutely recognized the benefits of pair programming, they still found it hard to justify the business case sometimes. After all, the immediate costs of pair programming (more person-hours) are much more visible and obvious than the (expected) future benefits of increased quality, fewer bugs, and shared learning. This makes pair programming an investment strategy. And as all entrepreneurs know, sometimes you can make the case for an investment, and sometimes you can’t.
“It’s easy to step over such financial concerns if you’re not the one paying the salaries of developers.”
Our hope is that this post contributes to the debates around this controversy. We hope it will give you better ways to talk about the costs and benefits of pair programming in your organization. Scientific research also has things to say about what can make it more effective.
Insight #1: Pair programming takes slightly more time than solo programming
The first question is how pair programming compares to solo programming in terms of speed. Are two developers working as a pair faster on the same task as an individual developer? This is part of Beck’s (1999) original argument for pair programming, although it is not strongly held in more recent publications.
Many scientific studies take an experimental approach to this question. They bring developers into a controlled laboratory setting and ask them to solve a coding problem, either alone or in pairs. The researchers then measure the time it takes to solve the task or, in some cases, until a certain quality standard is met. In many cases, these studies are performed with students (computer sciences), while at other times professional developers are hired or invited to participate. Usually, these studies measure how much time it takes two developers to solve a coding problem compared to a single developer. In other cases, the solution needs to meet a certain quality level before being accepted.
The best way to take in the results of many scientific studies together is through a meta-analysis. Such analyses use advanced statistical techniques to aggregate the findings from many individual studies and draw broad conclusions. Several meta-analyses have been performed for pair programming. By far the most commonly cited study was performed by Hannay et. al. (2009). They aggregated the results of 18 empirical studies. They concluded that across these studies, pair programming took slightly more time than solo programming, and required moderately more effort (person-hours). However, a more recent meta-analysis by Salge & Berente (2016) of 15 empirical studies found no significant difference in the duration. This is consistent with a well-known study by Williams et. al. (2000) that showed a 15% increase in duration compared to solo programming. But the authors also found that the results improved as pairs worked together more, and could potentially become twice as fast as solo developers. However, this study was not peer-reviewed and used a small sample of students. We should note that this study used a small sample of students, and its conclusions are far less robust and generalizable than the aforementioned meta-analyses.
The majority of academic studies suggest that pair programmers require a bit more time to complete the same task as a solo developer. However, a consistent pattern across these studies is that the experience of developers and the complexity of the problem matter to the outcomes. Junior developers seem to become slightly faster when pairing (Hannay et. al. 2009). Pairing also decreases the duration compared to solo programming for complex problems (ibid).
“The majority of academic studies suggest that pair programmers require a bit more time to complete the same task as a solo developer. However, a consistent pattern across these studies is that the experience of developers and the complexity of the problem matter to the outcomes.”
So for speed alone, the overall picture that emerges from this is that developers who work in pairs are a little slower than developers who work alone on a task. So the evidence does not support the idea that pair programming immediately improves speed, and doesn’t make for a strong business case on its own.
One limitation of these studies is that they only consider the duration of coding, but not the cycle time of the task across all stages of development. Perhaps the duration of solo programming is shorter, but with the added code review that needs to happen afterward and the bugs that have to be fixed, it might still take longer than pair programming. Unfortunately, I have found no evidence to either support or reject this assertion.
Insight #2: Pair programming results in higher-quality code than solo programming, but not always
But what about quality? This is probably the most important benefit in the minds of the proponents, as well as Beck (1999) and other authors on pair programming (Coplien, 2015, Kim et. al. 2021, Williams et. al. 2003). The assumption here is that when two developers pair on a task, there is more encouragement to maintain quality standards related to clean code, testing, and documentation. If developers work alone, they may be more inclined to loosen their standards, particularly when pressed for time. Pair programming also acts as a real-time, ongoing code review, where both developers review and improve code as it is written.
Most scientific investigations of pair programming measure quality by testing the solution against existing test cases. Some studies measure quality through code metrics (such as SQALE). Again, the most consistent conclusions come from meta-analyses of many academic studies. Hannay et. al. (2009) aggregated 18 empirical studies and found a slight increase in quality when comparing pair programming with solo programming. Another meta-analysis of 15 empirical studies by Salge & Berente (2016) also found a moderate positive effect. To illustrate the difference, Williams et. al. (2000) found that solo developers passed between 73–78% of test cases with their solution, whereas paired developers passed 86–94%.
As with speed, there is clear evidence that the experience of developers and the complexity of the challenge moderate these effects. The jump in quality is most pronounced when junior developers pair up (Hannay et. al., 2009), most spectacularly by 149% for complex challenges. Intermediate developers also benefit from pair programming, but only on complex tasks (92% up). Interestingly, there was no clear benefit for senior developers. The quality of their code was mostly the same, paired or solo.
“Hannay et. al. (2009) aggregated 18 empirical studies and found a slight increase in quality when comparing pair programming with solo programming.”
So the overall picture that emerges for quality is that pair programming improves quality, particularly for junior and intermediate developers, and particularly for complex problems. Dybå et. al. (2014) summarize this as: “Two heads are better than one for achieving correctness on highly complex programming tasks“.
Insight #3: Pair programming results in more learning than solo programming
A third often-cited benefit of pair programming is that it allows developers to share learnings by working together. This matches our own experience; we’ve often learned most about how to write better code by working with another developer with another perspective. Obviously, there are many ways to measure learning. Most studies use a short survey to ask developers how much they learned from working on the code in pairs or solo.
Only the meta-analysis of Salge & Berente (2016) covers learning as an outcome of pairing. Based on 15 empirical studies, they conclude that there is indeed a moderate positive effect of pair programming on learning as compared to solo programming. However, it is unclear how the expertise and complexity of the problem play into this. Several studies point out that learning appears to be much higher when developers of different levels of experience are paired, but not too different.
“Based on 15 empirical studies, they conclude that there is indeed a moderate positive effect of pair programming on learning as compared to solo programming.”
So in terms of learning, it is safe to conclude that pair programming creates more learning opportunities for developers than solo programming.
Insight #4: Other benefits of pair programming
Up to this point, we covered speed/duration, quality, and learning. But what about other benefits of pair programming? Williams et. al. (2000) found that developers who paired up reported much higher satisfaction and enjoyment than developers who worked solo. 90% of the participants in their experiment preferred pairing over soloing afterward. A longitudinal study by Vanhanen, Lassenius & Mäntylä (2007) found that pair programming improved team spirit and work satisfaction. Interestingly, this study also supported the common observation that developers are often initially skeptical of pair programming, but become advocates of it after experiencing it.
Revisiting the business case for pair programming
The aforementioned results show that pair programming is beneficial for quality and learning, especially for junior and intermediate developers, and particularly for complex tasks. However, pair programming tends to be slower than solo programming, although this difference may diminish as developers get used to pairing (Williams et. al., 2000).
So what does this mean for the business case of pair programming? We think it’s reasonable to conclude that pair programming is more expensive than solo programming if we only consider person-hours. Simply put, if one developer costs two person-hours for the completion of a task, paired developers cost a bit more than four hours for that same task. Results may vary case by case, but this is the broad pattern that is supported by the evidence. However, the increased costs are offset by higher quality code, more learning, and potentially increased speed in the future as developers become more familiar with pairing.
“We think it’s reasonable to conclude that pair programming is more expensive than solo programming if we only consider person-hours.”
Unfortunately, both sides of the cost-benefit analysis of pair programming vary in how easily they are quantifiable in terms of money. The immediate costs in person-hours are clear, but the reduction in costs due to fewer bugs, more maintainable code, higher job satisfaction, and faster learning curves for developers is much harder to quantify right now.
However, many academic studies have shown that low-quality code increases costs in the (near) future as bugs occur more easily and are harder to resolve (Khomh et. al. 2012, Li & Shatnawi, 2007, Politowski, 2020). Furthermore, pair programming contributes to a shared understanding of quality and shared learning, which makes Scrum teams more effective at satisfying stakeholder needs (Verwijs & Russo, 2022).
Practical Recommendations
So what does all this mean in practice? Below, we outline recommendations that make the most sense in light of the evidence.
Recommendation #1: Focus on improved quality and learning
If you need to convince others of the benefits of pair programming, you do well to focus on increased quality and learning. The evidence clearly supports this, whereas any anticipated time-saving benefits are less clear and may or may not materialize in the future. However, you shouldn’t sidestep the increased costs of pair programming. Whether you like it or not, how much work costs is an important consideration in any decision that (commercial) businesses make, in particular for people in management positions or teams that have financial control. You would feel the same if you were paying the salaries and the bills.
A better narrative is to frame pair programming as an investment in quality and learning. This is particularly useful in a tight labor market where good developers are hard to come by. Pairing junior and intermediate developers with more experienced developers is an evidence-based way to increase learning and increase the skills of the existing workforce.
“A better narrative is to frame pair programming as an investment in quality and learning. This is particularly useful in a tight labor market where good developers are hard to come by.”
Recommendation #2: Encourage pairing on complex tasks
The evidence suggests that pairing is most beneficial for complex tasks, although the most junior developers also benefit from pairing on simpler tasks. For more experienced developers, pairing on simple tasks may even lead to worse outcomes than solo programming (Hannay et. al., 2009).
Thus, the evidence does not support the belief that pairing should be used to write all code, as recommended initially (Beck, 1999). Instead, as a tool, pair programming appears to be best suited for complex problems and to help developers who lack the experience to write good code on their own. Of course, the judgment of what is complex progresses along with experience. So this is ideally a judgment that teams should make together as they coordinate their work. The Daily Scrum is an ideal opportunity to make such judgments and issue pairing invitations.
Recommendation #3: Focus on the prevention of code smells
One area we did not cover in this post was how to make the most out of pair programming. One evidence-based recommendation is to focus pairing sessions on the prevention of code smells, particularly those that result in long and messy classes (e.g. “Blob Class” and “Spaghetti Code”). As we’ve written in another in-depth article, scientific evidence supports the belief that such code smells increase bugs, require more time to make changes, and make it harder to understand code.
Recommendation #4: Avoid big experience gaps
Most academic studies suggest that the pairing of developers with different levels of experience is more beneficial, as long as the gap is not too large. However, junior developers who are paired may also be more effective together than alone for tasks that are not too complex. Otherwise, there is the risk of the “blind leading the blind”. However, pairing a developer with very low experience with a developer with tons of experience also doesn’t seem to be very effective (Hannay et. al., 2009, Bowman et. al. 2019). So if you have the option, form pairs of mixed experience that are not too far apart.
Recommendation #5: Make work agreements
Good communication is clearly essential to pair programming (Zarb & Hughes, 2015). As with all the work that happens in teams, it is useful to make critical work agreements to make such interactions enjoyable and effective. For pair programming specifically, the patterns and guidelines by Zarb & Hughes (ibid) that we shared in another article are useful. Alternatively, you can make your own. You can find many tips in this in-depth article on the science of work agreements. Examples of such work agreements could be:
- “Tasks that are most suited to pair programming in our team are …“
- “We can invite others to pair with us by …“
- “We limit our pair programming sessions to … minutes”
Closing Words
Some people swear by pair programming, others hate it. The practice has been around since the early 1990s and has enjoyed a lot of attention from academics. Still, it is controversial for its trade-off between the costs of having two developers work on a task on the one hand, and quality and learning on the other.
To date, scientific evidence overwhelmingly shows that pair programming takes slightly more time than solo programming, against almost twice the effort in person-hours. However, this investment in time is offset by a clear increase in quality, learning, and enjoyment, which in turn speeds up development also. We also discussed how pairing seems most useful for complex tasks, and when developers lack experience.
We bookended this post with five practical recommendations to improve pair programming for your team.
Our hope is that this post gives you a better, evidence-based, starting point to discuss the costs and benefits of pair programming in your team and organization. Perhaps some of the results challenge your own beliefs and experiences, which may lead to some reflection. In any case, happy pair programming and solo programming!
This post took over 35 hours to research and write. If you think our content is valuable, and you think we should write more of it, you can support us on Patreon. Find more evidence-based posts here.
We thank all the authors of the referenced papers and studies for their work.
References
Beck, K. (1999). Embracing change with extreme programming. Computer, 32(10), 70–77.
Beck, K., & Andres, C. (2004). Extreme programming explained: Embrace change. 2-nd edition.
Bowman, N. A., Jarratt, L., Culver, K. C., & Segre, A. M. (2019, July). How prior programming experience affects students’ pair programming experiences and outcomes. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education (pp. 170–175).
Coplien, J. (1995). A development process generative pattern language. Pattern Languages of Program Design.
Dybå, T., Arisholm, E., Sjoberg, D. I., Hannay, J. E., & Shull, F. (2007). Are two heads better than one? On the effectiveness of pair programming. IEEE software, 24(6), 12–15.
Hannay, J. E., Dybå, T., Arisholm, E., & Sjøberg, D. I. (2009). The effectiveness of pair programming: A meta-analysis. Information and software technology, 51(7), 1110–1122.
Kim, G., Humble, J., Debois, P., Willis, J., & Forsgren, N. (2021). The DevOps handbook: How to create world-class agility, reliability, & security in technology organizations. IT Revolution.
Khomh, F., Di Penta, M., Guéhéneuc, Y. G., & Antoniol, G. (2012). An exploratory study of the impact of antipatterns on class change-and fault-proneness. Empirical Software Engineering, 17(3), 243–275.
Li, W., & Shatnawi, R. (2007). An empirical study of the bad smells and class error probability in the post-release object-oriented system evolution. Journal of systems and software, 80(7), 1120–1128.
Politowski, C., Khomh, F., Romano, S., Scanniello, G., Petrillo, F., Guéhéneuc, Y. G., & Maiga, A. (2020). A large-scale empirical study of the impact of spaghetti code and blob anti-patterns on program comprehension. Information and Software Technology, 122, 106278.
Salge, C. A. D. L., & Berente, N. (2016). Pair programming vs. solo programming: What do we know after 15 years of research?. In 2016 49th Hawaii International Conference on System Sciences (HICSS) (pp. 5398–5406). IEEE.
Williams, L., Kessler, R. R., Cunningham, W., & Jeffries, R. (2000). Strengthening the case for pair programming. IEEE software, 17(4), 19–25.
Vanhanen, J., Lassenius, C., & Mantyla, M. V. (2007, August). Issues and tactics when adopting pair programming: A longitudinal case study. In International Conference on Software Engineering Advances (ICSEA 2007) (pp. 70–70). IEEE.
Verwijs, C., & Russo, D. (2021). A theory of scrum team effectiveness. arXiv preprint arXiv:2105.12439.
Williams, L., & Kessler, R. R. (2003). Pair programming illuminated. Addison-Wesley Professional.
Zarb, M., & Hughes, J. (2015). Breaking the communication barrier: guidelines to aid communication within pair programming. Computer science education, 25(2), 120–151.
