Our audit and audience obsession: How big questions led to fresh findings
By Tara Kimura and Haley Coppins
It’s an unfortunate truth that the word “audit” has some dry associations — spreadsheets, ID numbers, sums and averages come to mind. But, pair an audit with a user needs analysis, and we can say without exaggeration, a little magic happens. Fresh insights, powered by data, take root and bloom.
For the past year and a half, we’ve been working on a project that pairs a digital news user needs analysis with a series of audits. I (Tara) lead content design at CBC and I partnered with UX researcher Haley Coppins on this project. Following the model, established by Dmitry Shishkin formerly of BBC’s Global News, we identified the user needs of our audience. We then completed a content audit, measuring how well we were meeting those needs. Putting our own spin on the original model, we also conducted a product audit to understand how our news products are meeting our user needs and inform our design decisions.
Through our research, the user needs we identified for our audience are:
- Connect me to Canada (local, provincial and national news).
- Tell me the latest (breaking news, updates).
- Help me thrive (stories that have a direct impact on readers’ lives).
- Make me think (articles that explain complex issues or teach readers about new issues and perspectives).
- Divert me (a pause from the hard news, ie. arts and culture, sports, human interest, etc.).
These needs give us a good understanding of why people come to us specifically and what they’re seeking to stay informed and connected. Some people might come to us to satisfy one or two needs or they may come to us with the expectation that we’ll deliver on all five. Needs may change throughout the day, for example, with some people looking for updates in the morning and diversions later in the evening.
Using this framework, we can reconsider the way we tell and deliver our stories to our audience. It also allowed us to develop a common language, based on deep research, to be used across editorial and product teams so we’re all on the same page. And, most significantly, we uncovered some substantial and surprising findings.
Here’s how we did it and what we learned.
User interviews, diary study, survey: defining “news”
We took a mixed methods approach to uncover our user needs and focused our work around our audience who come to us routinely. We recruited participants via a newsletter campaign, our social channels and our help desk.
We dove deep with user interviews, tracked daily habits with a diary study and asked broader questions in a general survey. Our goal: explore how our audience defined “news” and why they chose us in a crowded media landscape. We also probed how news fit into their daily routines in addition to what they liked and disliked about the service we provide.
Not surprisingly, many participants told us they relied on CBC News to deliver trusted, unbiased reporting. They talked about the role we play as a connective thread across communities and provinces alike. They also told us they relied on us to shape the national conversation, bringing new perspectives and issues to the fore.
Eager to share these insights with our teams, we led co-creation workshops with both product and editorial teams to sift through the research, identify patterns and name our user needs together. For this work to take root, we knew we needed to all start from the same place of understanding and ownership. Naming the needs together was the first step in that process.
500+ stories, 2 auditors, 5 user needs, 1 massive spreadsheet
It’s true that we opened this piece saying that audits were dull which sets low expectations for this next bit. The thing is though, audits can be immensely satisfying — as it was in this case where we were aligning quantitative and qualitative in a tidy and deep framework.
Our approach was as follows:
- Gather all of the stories that appeared in our main news lineup over a two week period
- Read and bucket them according to need
- Cross correlate with analytics
We controlled the audit by limiting the work to the two of us. While this was no easy feat, with more than 500 stories to read and categorize, we felt it was important to prioritize consistency and accuracy. Where a story fulfilled more than one need, we recorded that. Where a story predominantly filled one need but still crossed over into other needs, we recorded that as well.
From this audit, we were able to look at representation on the top stories page, including frequency of stories featured from our sections (Entertainment, Health, etc.) and regions. Most importantly, we were able to see which user needs we featured the most.
The stories we featured the most frequently in our main news lineup fell into the Tell me the latest and Connect me to Canada categories. This was followed by stories we would classify as Divert me, Make me think and finally Help me thrive.
But when we looked at which story type was the most well read, we found that stories in the Help me thrive category performed the best by a significant margin. This was followed by Tell me the latest and Connect me to Canada.
So, while we confirmed that we’re delivering on our core service of connecting Canadians and keeping them informed of the latest news, we identified a user need, Help me thrive, we could be pushing harder to satisfy our audience.
This doesn’t necessarily mean our stories need to change but rather the way we focus them could be reconsidered. A story about inflation, for example, might focus on the Bank of Canada and fall into the Tell me the latest user need category. A story about the rising cost of groceries and gas, however, would fall into the Help me thrive category.
Matching design elements to user needs
As we were reconsidering the way we tell our stories, we also wanted to measure how we deliver them within our news app and website. Within our design team, we conducted a product audit across both platforms.
We catalogued all elements, from buttons and labels to swimlanes and widgets, along with user flows, and matched the elements to our user needs.
What was revealed was that we prioritize two of our user needs — Tell me the latest and Connect me to Canada — the most. For all other needs, the user has to put forth considerable effort to satisfy them.
A reader, for example, might consistently satisfy the Connect me to Canada user need by going to the fixed My Local swimlane. But a user hoping to satisfy the Help me thrive had less reliable routes to travel. They might find a story that fulfilled this need in the top stories section. But they were more likely to have to skim other sections or follow related links to find more of the same type of content.
Up next: Understanding our local news audiences
So what lies ahead for this project? The focus now turns to local news and the unique needs of that audience.
Editorially, our partners in news have embraced this model and if you’ve been a longtime reader of our site, you may have noticed some shifts in how we tell stories. Take a look at our top stories headlines and see how each story might fit into any of these user need categories.
The work of putting our audience at the centre of our focus and staying attuned to their needs is frankly work without a finish line as we expect these needs will evolve and change over time.
