Do people actually read release notes?
As part of my research into the best ways to write great release notes, I conducted a survey to try and gauge how many people actually read them. The results were really interesting – mainly because I accidentally stumbled upon a young sample set of respondents who were mostly aged 25 or younger.
Finding people to respond to the survey was difficult as most services cost money but in the end I found /r/SampleSize subreddit forum of more than 87,800 people on Reddit who are willing to complete surveys for free. The demographics of my respondents according to this survey, meant most of my respondents were 18–25-year-old, white, single American male students who use Android phones and Windows computers.
The second largest segment were aged 26–35, a third of respondents were female and the other top countries represented included Canada, the UK, Australia, Germany and the Netherlands.
My respondents = Generation Z
While my sample set might seem less than ideal, it’s actually a great representation of Generation Z, the digital natives who have literally been using smartphones and iPads since they were toddlers.
“Millennials and boomers are addicted to their phones. Generation Z simply hasn’t known a time without the device,” said MaryLeigh Bliss from Ypulse, a millennial and gen Z research and consulting agency based in New York. This generation also isn’t a bad representation of the average app user if you considering the following:
- The 18–24 age group spends the most number of hours (93.5 hrs) on apps each month (according to Statista).
- The 18–24 age group also spent more than any other age group on digital media (according to comscore)
- The 18–25 age group made up the largest segment of users (31%) who use entertainment apps (according to Google).
If we’re to consider everyone with access to the internet as the target audience for release notes, which is more than 4.1 billion people according Internet Live Stats, to achieve an industry standard confidence level of 95% with a 5.1% margin of error, my sample size needed to be 370 people (according to Survey Monkey’s sample size calculator). I had 372 respondents so I was just about on the money with that number (even if they were an unrepresentative portion of the overall audience).
What proportion read release notes?
Surprisingly, a majority of 83.6% respondents said that they did in fact read release notes or app updates. I had expected this to be far less. Prior to this survey if you had asked me to guess, I would have said 40% or less read them:
The next surprising statistic was that not only did the majority of respondents read release notes but that a third of them also read them regularly:
The majority (more than half) said they only read release notes sometimes, while the remainder either read them rarely or very rarely.
Why do people read release notes?
If people were reading release notes and app updates, I really wanted to know the reasons why. I allowed respondents to give multiple answers. The results looked something like this:
- I liked to know what’s new or changed — 320
- I find them interesting — 149
- I like to read technical details — 97
- I like the funny content — 86
- I work in tech and want to know what competitors are doing — 56
- To find out about early access features — 50
- No reason given — 46
What would make people read release notes more?
The answers to what would make people read release notes more were also quite surprising. The top answers were:
- Jokes and humour — 194 votes.
- Links to early access features — 134 votes.
- Less text — 117 votes.
The next most popular answers were creative writing (65 votes) and rewards like stickers and other swag (60 votes). Unsurprisingly not many people vote for release notes to contain more text.
Who do you think writes the best release notes?
I had my own personal opinions about who I think writes good release notes. Slack, Medium, Tumblr and Transit are all known for injecting some humour into their release notes but I was interested to know what the overall perception of which company writes the best release notes.
Perhaps it was a reflection of my Reddit sample set but the top voted company for release notes was Discord, the free chat service for teens and adults aged 13 or older. The other top votes were more mainstream choices:
- Discord — 13.7%
- Apple — 12%
- Google — 10.4%
- Slack — 10.4%
- Independent — 7.1%
Other popular votes went to Microsoft, Netflix and Medium. A handful of other votes also went to EA (specifically for the Sims), Paradox games, Tumblr, and Sync.
Conclusion
While this survey was by no means an exact science, I thought the results were pretty interesting but they showed that young people are actually reading release notes! As someone who has written and edited hundreds of release notes, it was nice to find out my hard work wasn’t for nothing — even if most of my respondents were only reading the updates for the latest game they’re playing at the time.
It wasn’t surprising that the majority of respondents said they read release notes to find out what is new in the software or app given that is their primary function but it was also particularly interesting to note the regularity that they were read with a majority of 86.8% of respondents stating they read them “sometimes” or “regularly”.
I wasn’t surprised to see so many people say links to early access features would make them read release notes but I didn’t expect to see 50% of respondents vote for jokes, humour and creative writing. I’ve shared my own thoughts and the opinions of others about the risks of trying to be funny in release notes in a previous blog (TL;DR — I think humour can work but only if you can also share some useful, technical content at the same time). However perhaps I was wrong, if this sample set represent the majority of app users today, then they represent the tech leaders and release notes readers of tomorrow.
