What makes a great software engineer?

A few years ago my Ph.D. student Paul Li was looking for a dissertation topic. At the time, he was fascinated by the decisions that software engineers have to make and the data use to make them. At the same time, I was starting to manage a team of six engineers myself at AnswerDash and thinking about the broader issues of developer productivity and software team management. As it turns out, few researchers had holistically investigated what skills a software engineer needs, beyond vaguely talking about “good communication skills” and the ability to write “good code quickly”. Paul decided that this would make a great dissertation topic, and I agreed, and he set out to investigate what makes a great software engineer?

Paul defended his dissertation last week and just put the finishing touches on his dissertation document, describing three major studies of software engineering expertise, including an interview study of 59 senior engineers and architects at Microsoft, a survey of almost 2,000 Microsoft engineers, and another interview study of 46 non-engineers on their views of software engineers and their required skills. What did he discover?

While it’s hard to summarize the incredible nuance in his 250 pages of discovery, several key findings jumped out to me:

• Across all three studies, it was clear that both software engineers and non-software engineers viewed the ability to quickly and correctly write code that met requirements was viewed as a baseline requirement. It wasn’t enough to qualify someone as a “great” engineer, but rather a necessary but insufficient condition for working as an engineer.
• Engineers need to be pliable in their beliefs and their skills. In their view, the software industry moves fast enough that they quickly have to discard old knowledge and replace it with new knowledge, constantly learning not only about new languages, libraries, and architectures, but also new business ideas, market priorities and other workflows. An engineer that struggled to learn, learn quickly, and learn enthusiastically (rather than reluctantly) could not be a great engineer.
• Great engineers can reason about multiple levels of abstraction seamlessly and lucidly, linking the lowest levels of implementation to the highest levels of product impact in a marketplace. This ability was considered rare but extremely valuable, because it supported the central decision-making tasks of engineers, which required this multi-level abstract reasoning. Engineers who got lost in the details and couldn’t connect them to the larger system of concerns in the business were not considered great.
• Because software engineering was an inherently collaborative endeavor, engineers viewed the constructive facilitation of the work of others as critical. This was not only in a technical sense (providing information and updates to other engineers who depended on code an engineer maintained), but in a social sense (creating a psychologically safe environment in which ideas could be shared, helping others to learn productivity, etc.).
• Non-software engineers described many of the same desirable attributes for software engineers as software engineers did, but they also raised the issue of respect. They observed a clear ranking of different types of expertise in terms of value to the organization, with engineers of all kinds viewed as essential to shipping a product, but other types of expertise as peripheral. This revealed a bias toward functionality as value, overlooking other types of UX and product marketing value.
• Across the diverse organizations of Microsoft, there was broad agreement on the above, suggesting that either these are fundamental skills to software engineering, or fundamental skills to the culture of Microsoft.

Importantly, many of the attributes of great engineers that Paul discovered are not attributes that we know how to train for, measure, or relate to actual software engineering outcomes. Much of his work raises questions about how to use these discoveries in education, hiring, promotion, and research.

Of course, because the work was focused on Microsoft, there are a number of questions about it’s generalizability. It’s possible that these notions of engineering greatness are particular to Microsoft and might be different in other companies, particularly in non-software companies that have in-house software teams. It’s also possible that there are cultural differences in settings with different norms and values on interpersonal relationships. We need more research on other types of software companies to know just how fundamental these attributes are to software engineering expertise.

Paul’s work has already had significant impact, reaching tens of thousands of engineers through ACM Learning Webinar and impacting the design of the Academy for Software Engineering curriculum in New York. Microsoft has also been eager to apply some of his findings to hiring and recruiting.

Do these findings seem consistent for software companies you work in or are familiar with? If not, what’s different? We’re eager to hear your reactions!

Originally published at blogs.uw.edu on June 21, 2016.