Lean, rapid User Testing that doesn’t cost the earth but delivers great insights
User research can quickly become costly, in terms of recruiting, kit, staff time etc. I’m a big believer in Lean methods and we thought we’d really put it to the test. Just how can we do some user research that doesn’t cost the earth and take weeks to plan?
In one of our most recent pieces of user research, our UX and Product team wanted to understand how our existing customers would interact with a new concept. Our UX Team is new to the business and our Product team has recently grown substantially at interactive investor, so not only was this an opportunity to gain a massive amount of insight, but it was an opportunity for us to learn and show the business just what we can do.
Where we started
Firstly, we wanted to know what we were trying to achieve with the user testing and what outcome were we looking for. This meant a few sessions with stakeholders to understand and get a brief together.
Once we had this, we were able to very roughly tie together a concept together that we could present to the users. This consisted of a basic-level user journey. All of this was ‘dumb’ mocked up into a series of screenshots that we put into Invision and then linked together to create a clickable prototype.
Our UX Designer, Chris was tasked with writing a script — luckily for us, we had some loose formatted templates from previous research that reduced some of the overhead of creating these.
Once we had a script, it was clear our user testing was going to be in two clear sections: half the time would be spent on discussing with the users how they interact with our service; the other half would be spent going through our prototype and observing how they interacted with it.
Recruitment of users
Due to the nature of the test concept, we thought that existing customers would be the right fit for us.
We’re very lucky at interactive investor to have a very active, passionate and loyal customer base. This means that recruiting customers is as easy as asking customers if they would like to spend some time with us.
To recruit, we placed a survey entry into our website to explain we would be doing some research and if they were interested — they could be involved.
Chris then collated all this information and spent time with these users to plan and coordinate when would be the best time for them to come in.
Location, location, location
Our UX Team is Manchester based but the majority of our users for the research were down in London. Thankfully, we have an office in London with bookable meeting rooms.
One day would be spent down in London and one day would be spent in our teams home base in Manchester.
A few emails and phone calls later we were all set, rooms were booked and we had times planned for pretty much everyone with just a few final bits to iron out.
User Research testing kit
This bit can be costly; recording software, cameras, microphones, computers, screen mirroring and broadcasting of the research so others can view it. I get dizzy just thinking how fast these costs can initially spiral.
But, let’s look at what we already had.
Appear.in — We regularly use appear to video conference with our teams when they’re in different locations. Appear is conferencing software and offers screen mirroring and works on laptops and smartphones. We already pay for the premium account and quite recently they started offering an additional feature for recording the calls — this was potentially an option to use.
USB conference speaker — Because we use appear so much, we have a few of these lying around the office. We thought we could use this to record the audio.
Laptop — pretty much everyone on the team has a laptop these days, so this wasn’t an issue.
So, so far we have broadcasting software and a microphone. But we still need a few more things:
Camera — We could use the camera on the laptop but it wouldn’t capture both the researcher and the session participant so we’d need a webcam.
Stationery — We have some, but they’re used by the teams regularly, so we’d need some new bits for the sessions.
Rewards — As a thank you for their time, we’d need some thank you reward for our customers for attending the sessions.
One chat to my boss later and we had the OK to purchase and expense the other bits needed.
We went onto Amazon, ordered a very high-quality USB webcam, markers, sticky notes and I headed over to M&S to purchase a volume of vouchers. We then went to appear.in and paid the extra few pounds to unlock the recording feature.
So, our user research kit now consists of:
Broadcasting software — appear.in
Recording software — appear.in
Microphone — USB Conference Speaker
Camera — USB Webcam
Screen Sharing — appear.in
Prototype — Invision
Thank you vouchers — M&S Vouchers
We’d managed to pull all this together outside of other streams of work over three weeks, but in reality, it was probably about 20 hours’ work. Not bad, considering it can take weeks to set all this up, typically with one person working on it full time.
Conducting the research
As we were starting early down in London, Chris and I headed on a train from Manchester the day before the research, had some dinner and stopped overnight in a hotel. In the morning we headed to the office, did some audio visual set up and checks, sent out the web address for observation and were set up for the day:
Chris was set up in one room where we would conduct the research and I was set up in another, with my headphones, notepad, laptop and appear.in loaded up.
Everything worked fantastically. Not only could I observe the participants, but I could very clearly hear them too, observe their body language and see how they interacted with the prototype.
On day one, we had every single participant attend and did back to back interviews all day. We had staff from all over the company tune in to observe the sessions, dropping in and out as needed and I collected 27 pages of notes. Sessions were recorded and saved to my laptop and I kept them to re-review later in case we’d missed anything.
After a successful day Chris and I left the office, jumped on a train back to Manchester and headed home for the night.
Day 2, we set up in our Manchester office with a similar set up and another Product Manager took the task of writing the notes as I needed to head to our Leeds base.
We weren’t as lucky on day 2, with a few empty slots on the day — but this did mean we could spend the time doing all the other bits we had going on. We produced about another 10 pages of notes from day 2’s sessions.
Final result
We interviewed 8 customers and the research was hugely valuable. We were able to proof a concept with our customers, get a ton of qualitative feedback on the concept and our fledgling UX team spent time with our customers.
We wrote up and condensed our notes so that they made sense and wrote a summary and published them to the entire company.
We’ve used this feedback to revise the project and feedback into the direction of what we are going to be doing next too. We also had a few sentiments from customers that informed us of things we weren’t aware of either — so not only did we get insight into our research area, we got research and information on other bits too.
Final cost (in hours and money)
Total Time:
Around 40–50 hours, including prep, doing the research and the write up
Total Cost:
New kit for camera, stationery: £150
M&S Vouchers: £700
Appear upgrade: £60
Overnight stay for 2: £250
2 very tasty dinners and a couple of beers: £80
Trains to London: £280
Total: £1520
A very tidy and small amount considering that the cost of research can easily reach tens of thousands of pounds.
We also now have all the kit to conduct research next time, so the cost will be even less next time we do it.
User research doesn’t have to cost the earth, or take a millennia to plan. We’ll be using this kind of approach for our future research too, where our in-house team of excellent UX people conduct research.
