CXL Scholarship — Conversion Optimization MiniDegree — Week 4 of 12
This article is the fourth in a series of 12 articles over 12 weeks that, I’ll be writing an article discussing my learnings in the CXL — Conversion Optimization Minidegree. I recently applied to the scholarship application program and was accepted. Part of that process is to write a weekly article discussing what I have learned in the previous week. I am a lifelong learner and every opportunity that I have to expand my knowledge in an area of passion is one I grab wholeheartedly.
Total time approximation to complete MiniDegree: 78h 59min
You can find more information here: https://cxl.com/institute/programs/conversion-optimization/
So this is week four and I’m now in the Conversion Research side of the course.
Conversion Research:
Instructor: Peep Laja
Every optimization project has to start with conversion research. This is diagnosing a website, figuring out where there may be gaps in potentially lost conversions. Then we can start making a strategic plan to optimize. I really enjoyed this lesson as it moves away from the “let’s try this” “let’s try that” by just guessing that can often happen in a team environment to a data-driven perspective. We want a mix of quantitative and qualitative assessments.
Peep mentions how user recordings are vital for conversion optimizers. I have found the same thing in my job. You get a really good idea of how people are interacting with your site. The challenge I have is to find the time to allocate yourself half a day as Peep recommends to watch the videos.
Conversion Research consists of:
- Experience based assessment
- Site walkthroughs
- Heuristic analysis
- Usability analysis
- Qualitative research
- Online surveys with recent customers
- On-site polls
- Phone interviews
- Live chat transcripts
- Customer support insight
- User testing
- Quantitative research
- Web analytics analysis
- Mouse tracking analysis
Using Analytics to find Conversion Opportunities
Instructor: Jeff Sauer
This course builds on a bit of what Mercer taught in the Google Analytics course. Metrics are only as good as the understanding we have of them. They have to be put into context. If you don’t have your Google Analytics setup to track events, then you may potentially have false bounce metrics.
Jeff goes into several examples of setting up conversion goals, understanding traffic quality, event tracking and auditing your analytics.
Google Tag Manager for beginners
Instructor: Chris Mercer
I’ve taken Mercer’s own Measurement Marketing classroom so I have to admit I skipped straight to the exam on this one as I knew the material already. I’ll come back to it and the Google Analytics one at a later date for writing purposes. Mercer is a fantastic instructor though as he takes things that are highly analytical and puts them in easy to understand formats.
Fast and Rigorous User Personas
Instructor: Stefania Mereu, Phd
The above courses I had completed over the last couple of weeks but this course I completed this weekend. It was different than I expected. My understanding of user personas was from looking at generalizations of our client base, with a bit of demographic info and creating a persona from that. In hindsight, I think I’ve seen a lot of bad personas based on the criteria below. This deep dive into data using Factor and Cluster Analysis was very interesting.
A user persona as per Alan Cooper is a prescriptive model of the user, what he wishes to accomplish and why.
Bad persona:
A bad persona is full of irrelevant details, ie. favourite colour or meal
Non-actionable details
Reflections of stereotypes ie soccer mom
The fast & rigorous Framework to create a persona:
Step 1 — collect data
Step 2 — identify groups
Step 3 — build archetype
This process helps with cross-functional communication so everyone is talking about the same thing.
Cluster analysis — Manual clustering — quantitative data.
Survey — easy to get data quickly and controlled
If you can’t do surveys, you can use Web Analytics.
The time to build personas varies depending on how accessible and how clean the data is. It takes approx 3–4 weeks to create personas.
Step 1 Collect Data
Personas are only as good as your data. Questions have to be good quality.
Good questions are relevant, actionable, and unbiased.
When doing survey data, use descriptive words for scales rather than just numbers to prevent accidental skewing of data if someone gets confused between a 1 and a 5.
Building a survey
Use a tool that allows you to use open-ended questions. You can get a lot of quotes and personality.
Survey tool options are:
1.Google Form
2. Survey Monkey
3.Qualtrix
Recruiting respondents:
Survey on own website:
HotJar — a tool to put a snippet on your website to survey respondents that land on your website
Survey the general population: Amazon Mechnical Turk
You typically need 300–1000 participants for your data.
If you can’t get 300–1000 participants, try the cluster analysis anyway.
Run a small version of the sample before running it for everyone, ie. a sample of 10–15 and analyze it for any potential mistakes, clarity. This is a good opportunity to be collaborative. Share incremental progress with your colleagues.
Step 2: Identify Groups
Simplify the data set — factor analysis (data reduction), cluster analysis
Simplify Your Data:
Download the survey:
Each row is a response (one unique participant)
Each column is a variable (the answer to a question)
Hadley Wickham refers to this as Tidy Data.
Enable yourself to make short, pithy statements that boil down to a few key issues.
Exploratory factor analysis (EFA) — Answers to questions reflect underlying factors. Extract meaning out of the questions into a few key factors.
Identify factors — an iterative process
Factor Analysis — helps you decide how much influence each factor has on each question.
Not all factors are 100% interpretable at first review. Sometimes they need to be grouped. Factor analysis allows you to identify questions and give weight around each factor.
Cluster Analysis
Now that you have the factor scores, you can look for clumps of persons that responded that were influenced by the factors by different degrees. A cluster consists of individuals who have assigned similar weights to different factors.
Identify Clusters
Once you have clusters, you can average cluster members across each of the factors scores to find your average cluster member. There will be some variation. Find the median or mean and build graphs. Then it becomes clearer how one cluster is different than the other.
There are several ways to do cluster analysis.
Things to do with cluster analysis:
- Normalize factor scores
- Check for outliers — make sure there aren’t individuals that differ a lot.
- Visualize the data
Step 3 — Building Archetypes
Start from user needs. Addressing the needs of the user, will allow you to find solutions.
After finishing collecting data, describe clusters, starting from factors. The factors represent the summary. Get the demographic data you have collected afterwards.
Choose an image that reflects some of the demographics of your persona.
Include some quotes from the open-ended questions.
Include word clouds.
Cluster Visualization
Make use of different visuals to communicate quantitative information.
Include the user journey.
Moving Forward with Personas
Personas can be used for several months to a year. They can change with value proposition changes.
Take notes when using personas. Make note of what works well. Note questions for future surveys. Personas can be used for hypotheses in other industries but should be tested. Don’t overgeneralize personas.
The next course I’ll be starting is Heuristic Analysis Frameworks for Conversion Optimization Audits. I’m stretched super thin these days as we’re in the middle of a big project with work and I’ve got family things going on as well. I’m trying to push through the material as quickly as I can so I can then go back when I’m done and spend more time on areas that I feel I need to. My brain is starting to feel full with all the material I’m learning :)
