Yes, it’s getting harder to get an answer on Stackoverflow
Yesterday, I wrote about the declining answer rates on Stackoverflow and the discussion motivated me to dive into the numbers a bit more.
As a reminder, the entire analysis is based on the public Stackoverflow data set, which you can use on the free tier of Google Bigquery. My example queries are on GitHub, including short instructions on getting started.
Observation 1: Answer Rates are Declining
The chart above shows the Answer Rates on Stackoverflow for the top 20 programming languages (Java, Javascript, PHP, Python, C#, C++, Ruby, CSS, Objective C, Shell, R, Perl, Scala, Go, Swift, Matlab, Clojure, and Groovy) and the trend line for all questions on Stackoverflow.
Most data points are above the trend line, which shows that the communities for these twenty programming languages have higher answer rates than Stackoverflow in general, which is no surprise given their higher popularity that makes it easier to find someone who can answer your question.
Observation 2: It looks like people are getting answers faster
If the overall community was in decline, we would not just see that it is harder to get an answer overall, but also that it would take longer to get an answer in the first place.
Apparently, the opposite is the case: as time progresses, the average time to first answer (for those questions that have been answered) is falling, too. This trend is very likely just a statistical artifact, as we are working with a biased sample.
A question posted in July 2011 had more than five years time to wait for its first answer, which would not just drive up the rate of overall answers for old questions, but also the average time to first answer. A question posted in December 2016 can have answers that are only a few days old, at max.
While it looks like this observation is leading us astray, it actually makes it clear that the overall answer rate might not be a great indicator, but that we should look at the number of questions that have been answered in a given time.
Observation 3: The Rate of Questions that get Answered within One Day or Week is Falling, too
If we just count what percentage of questions get answered within 24 hours or 168 hours, which is the maximum amount of time you should grant strangers on the internet to do your home work, we get following picture:
Of course, if you give people seven times as much time, they are more likely to give you an answer, but not much more. And if you give them all the time in the world (or until you get a snapshot of the data, whichever comes first), the rate of answers is even higher.
Indeed, the rates are very closely connected, but more importantly, they are all falling over time.
This means: yes, it is getting harder to get an answer on Stackoverflow.
Why is that the case? Are people leaving the site? Unlikely, given that the absolute numbers of questions and answers are still rising. Are too many people asking questions and too few answering them? Also implausible, as the numbers of people asking questions (Users1) and people answering questions (Users2) are closely correlated.
Maybe, all the easy questions have been answered already. Maybe moderation (which always takes time) is resolving or deleting old unanswered questions or maybe, askers are even coming back to answer their own questions.
Bonus: Here are the top 3 Stackoverflow users who answer their own questions
- mpen with 295 questions
- b-clay-shannon with 276 questions
- prosseek with 213 questions
You are the real MVPs!
