Advanced User Research Techniques: Web Analytics (part 1)

There are a number of advanced user research techniques that can be used to get meaningful insight about users and their activities. 2of these techniques are:

  • Web Analytics
  • Statistical Analytics

In this 2 part blog series the above methods will be described. This post focuses on Web Analytics.

Web Analytics

Web analytics are a common method for organisations to track user behavioural patterns on websites. It monitors traffic and popularity trends by counting the number of website visitors and page views. This includes how long a user spends on a webpage, where they came from and ultimately if users are achieving their goals (Preece et al, 2015). More specifically:

  • Users/unique visitors
  • Page views
  • Unique page views
  • Pages/visit
  • Average visit duration
  • Bounce rate
  • Goal completions

It is useful to identify usage problems such as, the percentage of users dropping off on the last page of Amazons online checkout system. However, web analytics only identifies the “where”, “how many” and “what”, but does not explain the “why”. As in why a user behaved in a certain way. Khoo et al (2008) stresses the importance of triangulating web analytics metrics with other research. Coupling web analytics with qualitative research methods makes it is possible to get a more well rounded understanding of user behaviour.

Benefits of analytics include reducing unnecessary projects and opinion based decision making. Drawing from experience product owners and decision makers can often request to include functionality biased on personal preference. Analytics can validate these requests and be used to clarify and defuse differences in opinions, backed up by facts.

The most popular web analytics tool is Google Analytics which is used by over 50% of the 10,000 most popular websites (Epson, 2012). It can be useful to organisations for monitoring behavioural patterns and tracking if visitors came from referral sites, search engines, social networks, email marketing, pay-per-click networks and much more(Preece et al, 2015). At a much more granular level it can be used to measure conversion rates, how often a button was pressed and how long a visitor stays on a page.

While these basic traffic metrics are easy to track and provide a solid foundation for how a site is performing they can be difficult to use to evaluate the impact of UX changes (Rodden, 2015).

Heart Framework

Kerry Rodden and her team at Google have developed two methods for measuring the quality of user experience and goals of a product/project using analytics. These methods are the HEART framework and The Goals-Signals-Metrics Process.

The HEART framework is used to measure the quality of user experience. Rodden and team found that UX metrics usually fall into five categories.

Happiness is a measure of user attitudes, such as perceived ease of use and satisfaction, often collected via a survey.

Engagement can be measured using behavioural proxies such as depth of interaction, intensity and frequency. Examples of this can be retrieved through web analytics such as, visits per week on the New York Times website or tweets per day on Twitter.

Adoption relates to the number of new users of a feature or product. Examples of this include the number of new users following a Medium blog in the last month or the number of existing members using a new filter on Snapchat.

Retention measures the number of returning users. How many active users are coming back or not coming back (churn).

Task Success is a traditional behavioural metric used to measure the effectiveness and efficiency of completing a task.

4 of the above categories can easily be identified through Web Analytics. These metrics can be applied at a whole product or specific feature level. However, the use of all or some of the categories will depend on the scope and goal of the project/product. For example, enterprise software that employees are expected to use in a bank on a daily basis may not be concerned with retention or engagement but might be interested in task success rate.

A way of selecting which category to include from the HEART framework should be specific to a project or product. The Goals-Signal-Metric Process helps identify this. This process uses signals and metrics to examine if organisational goals are being met. These methods are useful for informing design by large scale data and help reflect the quality of the user experience of a website.

In the next post Statistical Analysis will be looked at as an Advanced User Research Technique.

References

Empson, R. (2012) Google Biz: Over 10M Websites Now Using Google Analytics. Retrieved from http://techcrunch.com/2012/04/12/google-analytics-officially-at-10M/.

Khoo, M., Pagano, J., Washington, A. L., Recker, M., Palmer, B. and Donahue, R. A. (2008) Using web metrics to analyse digital libraries. In Proceedings of Joint Conference on Digital Libraries, Pittsburgh, June 16–20.

Preece, J., Rogers Y., and Sharp, H. (2015) Interaction Design: Beyond Human Computer Interaction (4th ed.). John Wiley & Sons.

Rodden, K. (2015) How to choose the right UX metrics for your product, Retrieved December 5, 2016 from https://library.gv.com/how-to-choose-the-right-ux-metrics-for-your-product-5f46359ab5be#.3wkpklk2w.