Good software engineers are critical for most modern companies, but many recruiters can’t tell if a programmer has mastered the craft.
I quickly analyzed the stack overflow survey from 2017 to find hidden patterns indicating programming skills. I discovered one, and it’s not what you think.
Assumptions & Data preparation
Before we start, I need to tell you about my preparations.
Correlation between salary and skill
There’s obviously no question in the survey asking how good you are at writing code. From what I can see, the question that best indicates your skill level is how much money you make.
So, to make my analysis, I assume that the higher your salary, the better you are. It’s not a bullet-proof assumption, but probably some positive correlation.
Filtering the data
To ensure that the data is consistent, I only looked at people who work full-time in the US.
Also, I remove the top 1% and bottom 1% to avoid outliers.
What I have left
I now have 3,398 respondents that earn between $30,000-$180,000 and the distribution looks almost like a Gaussian.
Let’s look at the analysis.
Here’s what I found
I decided to look at yes/no questions where the two groups are similar in size to make my life easy.
Let’s look at three questions that yielded exciting results.
Should a degree in computer science be a requirement?
Whether you need a degree or not is a question that I care about and an excellent place to start.
I divided the sample into two groups:
- Computer Science — People who studied computer science or software engineering (1634 respondents).
- Other Education — Those who studied something else or have no university education (1764 respondents).
Here’s what the salary distribution for the two groups looks like:
There’s an average salary difference of $3,751 in favor of computer science. The difference is statistically significant according to the T-test (0.0014), but only barely.
Education looks relevant, but let’s see if we can find better indicators.
What about contribution to open-source projects?
Now there’s a measurable difference.
People who contribute to open-source projects earn $12,299 more on average. That’s an almost four times larger difference compared to having a degree in computer science.
The surprise: Tabs vs spaces
A fierce debate in software development is whether to use spaces or tabs for indenting code. It sounds silly, but don’t say that to a programmer. If you want to learn more about this heated debate is, check out this article.
Fortunately for us, the stack overflow survey from 2017 included this very question, so let’s analyze. Here are the combatants:
- Uses only spaces — Programmers with a clear preference for spaces (1651 respondents).
- Only tabs or both — Everyone else (1660 respondents).
And here are the results:
We‘ve found an even more significant difference in average annual salary. This time it’s a whopping $14,235 in favor of spaces.
That’s a big surprise. Maybe one’s preference for spaces or tabs has to do with experience. Let’s see if we find the same pattern if we narrow down the respondents.
What about those who have worked as programmers for more than ten years?
Ok, how about less than five years?
The same pattern appears independent of how long the respondents have worked as programmers.
What does this mean for recruiters?
Conclusion
Let’s conclude that my research won’t be published in any scientific journal. The assumptions are shaky at best.
Anyway, here are some final thoughts:
- Don’t add spaces as a requirement in your job postings. I’m not sure why there’s such a significant correlation with salary.
- There are many things more important than degrees when hiring software developers.
I’ll leave you at that, thanks for reading!
