Regularity, a metric newsrooms should look at to build user habits and brand loyalty
After recognizing the focus of engagement and loyalty as user habits, how do we measure and quantify user habits?
Common ground
More and more newsrooms have realized that building user habits is the key to converting subscribers and retaining them. There are lots of research and case studies about this insight. I’m highlighting two articles about a study from Medill Local News Initiative here.
Core Question
After shifting the focus of engagement and loyalty on user habits, how do we measure and quantify user habits?
The North Star Metric for frequency, engagement and loyalty in the news industry has been Pages per Session, but I have questioned that metric, as well as Bounce Rate, for a long time. Getting people to view more than one page in a user visit, which is known as Recirculation, may help with traffic and drive visitors to hit paywall limits, but it’s not developing user habits.
Certainly, newsrooms would not evaluate content performance based on a single metric, but The North Star Metric often directs efforts and priorities. Thus, in this earlier tweet, I suggested newsrooms look at Sessions per User, which indicates the number of times a reader coming back to a website or a specific page, instead of Pages per Session, which indicates how many pages a user sees in a visit.
However, I soon realized a key limit in Sessions Per User. It does not reflect a sense of routine. A user could visit a news website 10 times a day and then barely visit the site during the rest of the week. That will still lead to a high number of Sessions Per User.
You may wonder how that could be a realistic situation. Well, one case example would be the Election Day. Those election guides and result tracking pages likely have a high number of Sessions Per User because people keep going back to those pages on or near the Election Day. Lots of those visitors probably do not even visit once every day. Newsrooms could look at them and aim to duplicate that success in future elections. However, those pages with irregularly high interests become noise when we want to distill insights about how many people visit the site on a regular basis or what types of content drive visitors back to the site on a regular basis.
I tried to use existing metrics to measure user habits, but my attempt was unsuccessful. Let me illustrate my point in a case example here.
Let’s say we have Existing User A, B, C and D here, and we have their number of sessions by each of the days in a week period (Feb. 28 — March 6).
First, we look at User-level data. We calculated their Total Number of Sessions, Average Sessions per Day, Recency (Day) and Average Interval Between Sessions (Day).
In terms of user habit and a sense of routine, User D is the best user because the person visited the website once every day in the week. The second best user is User C because the person visited the website in three days of that week. However, none of the four metrics here reflect those insights.
Then, let’s look at Content-level data. To illustrate my point, let’s say the website only has these four users. User A and B only visited Page Alpha. User C and D only visited Page Beta. We calculated Total Session, Average Sessions per User, Average Sessions Per Day and Days Since Last Session.
Ideally, the insight we want to get is that Page Beta is much better than Page Alpha when it comes to developing user habits. However, none of the metrics here reflect that insight.
Among the existing metrics in Google Analytics, the closest we can get to the idea of user habit is Audience => Active Users. However, it only tells you the number of users who visited your site 1+time in the past 1 day, 7 days, 14 days and 28 days.
Proposed Solution
When it comes to user habits, we are not measuring Frequency. Instead, we are trying to measure Regularity. Here’s the formula for Regularity:
Regularity = Number of Active Time Units / Total Units in the Period.
There are three steps to take for setting up the needed data.
Step 1: Choose a time period to look at.
Like all kinds of analytics, we need to choose a time period to review content performance. This is very important to measuring Regularity.
In the case we illustrated above, the selected time period would be Feb. 28 — March 6.
Step 2: Choose a Time Unit and calculate the Total Number of Time Units in the selected time period.
Time Unit can be hour(s), day(s), week(s), month(s), quarter(s) and year(s).
For this illustrated case, we choose Time Unit as 1 Day.
Between Feb. 28 and March 6, the Total Number of Time Units is 7. (If we choose Time Unit as “1 Hour,” which is probably too much for average users’ news consumption routine, the Total Number of Time Units would be 24*7=168.)
Step 3: Define the meaning of an Active Time Unit and tally Active Time Units
Analysts can choose how to count a Time Unit as “active” based on other metrics. It could be having 1+ sessions or 3+ sessions during a Time Unit. The qualification requirement could also be having 1+ sessions plus an interactive event like tapping a specific button.
In this case example, we define Active Time Unit as a Time Unit with 1+ sessions.
Using the same data set, here’s the User-level analytics. The cells marked blue are Active Time Units. The higher Regularity reflects better user habit and a sense of *daily* routine. (We chose Time Unit as 1 Day, so it’s a daily routine). We are able to see User D has the best Regularity, followed by User C.
- Note: the numbers in the cells for each day is still the Number of Session because I want to show that it’s the same data set. I used blue to indicate Active Units.
- You can also directly mark (Active Time Units)/(Total Time Units) for each time window, which I did for the following content-level analytics.
Now, let’s look at the content-level analytics. We use the same assumption that User A and B only visited Page Alpha, and User C and D only visited Page Beta.
Because Page Alpha and Page Beta each have two users. Total Time Units Per User in the Period is 7. The Number of Total Time Units for each page is 14.
Then, we tally the number of Active Time Units and the number of Total Time Units from all users in each day. The Regularity for a content page is (Total Number of Active Time Units from All Users)/(Total Number of Time Units from All Users).
From this data set, we can clearly tell that Page Beta does a much better job of developing user habits and a daily routine (again, we chose Time Unit as 1 Day, so it’s daily routine) than Page Alpha because Page Beta has a higher Regularity.
Note: (If this website only has Page Alpha and Page Beta, the overall Regularity for the website would be (3+10)/(14+14)=46%.
I understand this process will take data scientists in newsrooms, which is a scarce resource. So, Google Analytics and Chartbeat, if you are reading this article, you can help news organizations simplify this process in your system.
The customer-level Regularity can let newsrooms keep track of the progress of developing user habits and brand loyalty. It also can be an important factor for the audience or subscriber segmentation.
The content-level Regularity can guide newsrooms to identify specific types of content that visitors or subscribers regularly consume or keep coming back for them throughout the period of time (my assumption would be content like local traffic update page, local event list and daily brief). Then, the newsrooms can choose whether they would want to feature them on places like their mobile apps or make other investments in them.
