GitHub Transformers: Tools & Workflows

GitHub began as a tool for hobbyists and evolved with organizations and on-premise enterprise products into a tool for professionals.

We’re going to share 9 key insights from GitHub’s annual Tools & Workflows survey, welcoming debate about “learn git v. less git,” text editor and IDE popularity, and shining a light onto some blind spots.

One finding, in particular, “Professionals solve their problems with support from human peers” surfaces a critical discovery about human behavior: the distinguishing factor in solving challenges getting started with Git and GitHub is that professionals have greater access to people who can help them solve their problems.

tl:dr:

1. People’s use of GitHub changes over time. Respondents were largely drawn to GitHub for open source, but their primary use of the tools transitioned for work.
2. Large majorities use Sublime, Vim, and Atom. Alignment with editors serves user needs.
3. Git is the greatest challenge. Learning Git was cited as the users’ greatest challenge getting started with GitHub, regardless of their previous experience with VCS.
4. Professional users are more likely than hobbyists to rely on low-tech to solve problems. Whiteboards and paper and human contact play an important role in software development.

What we care about and why

We’re able to answer many descriptive questions about GitHub use (e.g. whether people use the command line or the client to make a commit) by analyzing events from our user database. Our database, however, provides no insight into what our users do and the tools they use outside of GitHub, so to learn about these things, we have to ask and listen.

The descriptive statistics presented here summarize raw data and depict what is happening among a subset of our users, giving us insight into phenomena we care about. It’s important to know the number of people using particular tools, so that we can make informed decisions about how our product should work with others. Organizations like Stack Overflow have produced reports packed with data that illustrate trends, but our population is distinct from theirs.

Understanding tool choice, in addition to GitHub use, is helping our teams to better understand and serve our users’ needs. We expanded our inquiry beyond tool brand and type to get a better understanding of processes, behaviors, and human motivations driving tool choice and experience:

1. Who are these people by segment? (professional v. hobbyist users)
2. How do segments of people on GitHub choose their tools?
3. Where do segments of people on GitHub draw information upon which to inform choice?
4. Why do certain segments of people stick with or what motivates them to change tools?
5. What are people currently looking for (emerging trends by segment) from the Ecosystem?

A note on survey recruitment, the survey was delivered through:

• an in-app popover (logged-in/logged out)
• email to 5k inactives (1% response rate)
• email to 5k new account creators (1% response rate)
• email to 90k new account creators with a subset of questions (~1% response rate, analyzed in a separate report)

The following plot shows the distribution of account ages among our survey respondents (in blue), compared to the distribution of a random draw of 10,000 active users from the db. New account creators are clearly severely underrepresented in this data set, while long-term tenured users (3 years +) are overrepresented.

Note: This density plot shows the estimated probability distribution of account ages by groups. Similar to a smoothed histogram, the points on the curves higher on the Y axis indicate values that occur more frequently.

To account for disparity and bias in the respondents, we subset the survey questions and sent an email with a link to the shorter survey to 90,000 new account creators organized by segments of 30-, 60-, and 90-days of tenure (read more about the NAC effort here). We learned that reaching new account creators from within the application is challenging (they spend little to no time within the app), so we have a blind spot in this data.

Professional v. Hobbyist Users

For this report, we segmented respondents according to their self-reported primary use of GitHub into two main groups/audiences, asking:

Q. What do you primarily use GitHub for today?

1. Professionals – People who primarily use GitHub for their job (36%).
2. Hobbyists – People who primarily use GitHub for personal projects, a class, or exploring open source projects (64%).

To illustrate the ways in which these populations are similar and distinct, we analyze and visualize responses according to these personas throughout this post.

1. People’s use of GitHub transforms over time

We asked respondents:

Q. What originally brought you to GitHub?

as well as:

Q. What do you primarily use GitHub for today?

The following plot depicts the differences in what brought people to GitHub originally (3 years ago) vs. what keeps them on GitHub today. Exploring OSS is a large plurality of the original draw, but it falls behind personal projects and work today.

Insights:

• 47% of respondents cited a different primary use for GitHub today than when they originally joined.
• Most of these users came originally to explore open source. 46% of respondents originally signed up for GitHub in order to explore open source projects, but only 26% cite that as their primary use today.
  • The biggest movement has been towards professional use: 12% of respondents originally joined for a job, but 35% are primarily professional users today.

A note on “Transformers”

We looked more closely at the population who signed up as hobbyists (for OSS or personal projects or a class) and have transformed into professional (e.g. primarily for a job) users to see what we could learn about their behaviors and trajectory:

• 73% of current professionals started out as hobbyists.
• Transformers make up 26% of all respondents. They have older accounts: 3.8 years, on average, compared to 2.8 years among non-transformers. Even excluding new account creators, the gap in account age between these groups is about 1 year.
• Transformers are slightly older in human years on average than non-transformers (31 years old vs. 29 years old), and respondents from the United States are more likely to make the jump from hobbyist to professional than international respondents (32% vs. 22%).

Open text quotes:

Release management with SVN SUCKS. I needed a better way.

Wanted to Open Source a project at the time, this was what I stumbled upon.

Developer colleagues from work, who were using it for work. I didn’t have use for it back then as I wasn’t working a tech job (so: “my job” wouldn’t be accurate because it didn’t require me to know about GitHub).

2. Large majorities use VCS and Text Editors.

To better understand people and their tools, we asked:

Q. Which of the following are absolutely necessary to your current software development process?

Note: We use Cleveland dot plots to show point estimates from multiple response questions. Respondents could select more than one response, so the estimates for each group may sum to more than 1.

The Y axis is always ordered based on the frequency of responses among professionals.

Insights:

• VCS is an essential part of the development toolkit for 84% of the entire sample, including 90% of professionals. This is partly a function of our sample being selected from VCS (e.g. GitHub) users.
• Text editors are a close second, with 84% of respondents noting they are essential.
• No other tool is used by more than half of respondents. 50% of users overall cite an IDE as essential, but among professionals this number is only 45%.

3. Text editor by brand popularity

The largest set of professional text editor users (75%) choose Sublime, Vim, Atom. 75% of all respondents indicated that they consider a text editor to be an essential tool. We looked at the distribution of primary text editors by brand and audience type:

Note: Throughout this post we use grouped bar charts to show point estimates for questions with single response options.

For each group (here, Hobbyists and Professionals), the length of the bars sum to 1. Also, note that the X axis is truncated to make differences at the lower end of the scale clearer.

Insights:

• Professionals are significantly more likely to be using Sublime Text than hobbyists (38% v. 31%), as well as TextMate (3% v. 1%).
• Atom is used at the same rate among professional users and hobbyist users (~13–14%).
• Hobbyists are substantially more likely to use NotePad++ than professionals (12% v. 4%).

4. IDE by brand popularity

51% of respondents consider an IDE essential to their development process. We asked these people which IDEs they use, allowing them to select as many as they wanted. IntelliJ is the most commonly used IDE by GitHub users.

Insights:

• Among professional IDE users, IntelliJ is the most common (37%), followed somewhat distantly by Eclipse (24%) and Visual Studio (20%).
• Among hobbyists, IDE users are more evenly split between IntelliJ (35%), Visual Studio (33%), and Eclipse (30%). The gap between professional and hobbyist use of Visual Studio is substantial (23 percentage points).

5. “Team Text Editor” vs. “Team IDE”

To a greater extent than other tool pairings, text editors and IDES are substitute goods: virtually everyone uses at least one (97%), but comparatively few consider both to be essential (29%).

Out of the 10 tool types we asked about in the closed answer options, only the use of IDEs and text editors was negatively correlated ($$ \rho $$ = -.4), people tend to choose a “Team.”

IDE users appear to have a general preference for GUIs, including when interacting with GitHub.

Insights:

• Team IDE is more likely than Team Text Editor use a GUI at least some of the time when using GitHub (66% vs. 55%).
• Twice as many members of Team IDE rely exclusively on GUIs when using GitHub as Team Text Editor members (6% vs 3%).
• Text editor users have older GitHub accounts, on average, than IDE users, but are no less likely to be new users (account age 90 days or less).

6. Native Git users are distinct

78% of respondents use Git today, but most used another VCS system prior to joining GitHub.

Insights:

• 35% of respondents are Git natives (started using version control with Git). The most common prior VCS is Subversion, with 47%. This pattern is consistent domestically and internationally.
• New account creators are more likely to be Git natives (48%), but less likely than tenured users to consider VCS essential to their development process (54% vs. 86%).
• Git natives are substantially younger than those who began on other systems. On average, they are 25 years old (human years). People migrating from other systems range from an average of 31 (SVN) up to 38 (Virtual Source Safe).

7. Exclusive GUI users are distinct

Hobbyists are more likely than professionals to rely exclusively on a GUI (6% vs. 2%). However, both groups are equally likely to report using a GUI at least some of the time (57–58%).

Similarities – GUI and exclusive CLI users are remarkably similar:

• They have identical account age distributions, identical human age distributions, and are equally likely to be professionals.
• Unsurprisingly, operating system strongly influences workflow. An equal proportion of MacOS and Windows developers rely on a GUI at least some of the time (60%), but developers on Linux systems are less likely to use a GUI (46%).

Differences – The 5% of users who rely exclusively on GUIs are distinct in a number of ways:

• 13% of exclusive GUI users are new to GitHub, compared to just 4% among those who use the command line at least some of the time.
• Exclusive GUI users have an average account age of 1.8 years, compared to 3.16 years among sometimes-CLI users.
• 74% of exclusive GUI users are on a Windows OS, compared to only 35% among sometimes-CLI users. Only 3% work on a Linux system, compared to 24% of other users.
• Exclusive GUI users are less likely to consider VCS essential to their workflow (70% v. 85%), and less likely to be in the United States (23% v. 33%).
• Exclusive GUI users are more likely to be IDE users (73% v. 50%), and less likely to be text editor users (61% v. 76%).
• Exclusive GUI users are more likely to be primarily using C# (18% v. 6%) and Java (26% v. 14%) than sometimes-CLI users.

I’m using the GitHub’s web flow most of the part just because I don’t know how to use it throughout the command line yet.

8. Git is the greatest challenge, but social challenges are the most persistent

We asked:

Q. When you first joined GitHub, what was your biggest challenge?

Insights:

• Learning Git is -by far- the most frequently cited challenge for both professional (44%) and hobbyist (43%) users. However, people who stick around on GitHub find ways to solve this problem; the greatest unsolved problems are social.
• Professionals are slightly more likely to cite social challenges (“Fear of making a mistake in front of an audience” or “Understanding social dynamics”) than hobbyists (29% v. 25%)
• Hobbyists are more likely to say that finding a project to work on was their biggest challenge (14% v. 9%).

Open text quotes:

fear of public ridicule, my shitty code/scripts, moving from subversion and git itself

For me, it was the fear of looking like an idiot in front of other people.

Git was my first real version control, so just understanding how it all works.

I feel that Git is much harder to use than, e.g. SVN.

It took a while to fully understand the Git model and workflow.

Merging is more complex than Perforce and the results are often unexpected.

9. Professionals solve their problems with support from human peers

We asked:

Q. How did you solve that challenge?

While professional users are technically more skilled and experienced than hobbyists in many ways, they are more likely to rely on human resources and low-tech tools like white boards to solve problems.

This isn’t always smooth- they are (slightly) more likely to cite social issues as being the biggest challenge. However, the factor that accounts for the lower rate of persistent challenges among professionals is their access to people who can help them solve their problems.

We asked:

Q. What was most helpful for learning how to use GitHub?

Insights:

• GitHub’s Docs and outside online resources like Stack Overflow predominate among the most helpful sources for learning GitHub, according to both hobbyist and professional respondents.
• Professionals are nearly twice as likely to learn GitHub from a friend or colleague than hobbyists (23% v. 12%). Professionals have greater access to human resources at work, and are less likely to rely on unofficial (i.e. non-GitHub) documentation sources like Stack Overflow.
• Even among hobbyists, few people cite community events or formal courses as a most helpful resource. We don’t know from this data whether they aren’t taking the courses at all, or whether the instruction they receive in them is unhelpful. This might be worth exploring more in future research efforts.

Open text quotes:

My colleagues helped me separate critiques of my code from critique of me personally.

Basically I had to just use Git with multiple other people, run into problems, have to use Github documentation and google to learn to fix them, and then teach my teammates how to fix them.

I had used Git loosely for version control while I was the only developer on a team. Now, I have moved to team collaboration and needing to do pull requests, rebasing, merging, etc. So I am having to get up to speed on those aspects of Git.

It was just early on in my career and I had trouble just figuring out the different “steps” that it takes to checkout a branch, make changes, merge them and push the code to the repo.

Conclusion

When we began this study we weren’t sure who would respond –who would take a 35-question survey and what would they say? While we asked about tools and workflows, we also asked about social challenges, which can be difficult to discuss. The data we’ve analyzed has been richer than we ever hoped for, and this is the first in a series of analyses.

As we reviewed the insights and spoke with members of GitHub’s product management and engineering teams, we saw a concluding story emerge. There appears to be a hierarchy of tools needs that lead into workflows.

Themes about human behaviors:

1. People desire and require experiences that build self-confidence.
2. People are hardwired for social connection, especially when learning or on the job.
3. Foundational tools can be organized into a hierarchy of needs, but code hosting services like GitHub are the least essential tools today. People can take their text editors and Git knowledge to a variety of tools.

—

Research is better when you get to discover new things about the world with someone who has a different perspective — someone who is willing to challenge you.