How we’re trying to do better at understanding what our users want
A huge number of people in the UK play the National Lottery on a regular basis. At the National Lottery Heritage Fund it is our job to ensure that a share of the money generated by those players goes to protecting, celebrating and sharing our national heritage. Over the next five years we expect to distribute over a billion pounds for this purpose, so it’s a big job and a big responsibility.
Because the money we grant comes from every kind of person in every part of the UK, it’s very important to us that absolutely anyone with a passion for heritage can understand what we do, and if appropriate, can submit an application to us through a straightforward and welcoming process.
We’ve recently taken some steps to double-down on our efforts to make our application experience easier and more painless. In this blog we’ll be posting updates on our steps and mis-steps, as we try various approaches to making things better for the people we serve.
We’re starting by listening and observing
We can only make our services great for the people we serve if we understand what it is that those people are trying to achieve when they interact with us.
Different people, though, come to us to achieve different things. Some people want to apply for funding right now, today. Others are checking us out for the first time ever, to see if we might support the sorts of ideas they are thinking about. Elected representatives and journalists come to us to find out what we’ve supported in their local areas. People who would like to work for us come to see what jobs we have. Far from just being people who ‘dole out grants’, there’s a wide range of services we offer to a wide range of different people. And we can only make them better if we can learn what it is that they need from us.
To learn about our users, we’re going to broadly pursue two parallel strands of activity, listening and observing. The rest of this post is about how these are different, and what they look like in action.
Listening
In order for us to listen to what people are telling us, they must first speak. Luckily, in this wired world, there are plenty of ways that people do talk to us. Here are a few of the most common ways that we hear directly from people:
- Phone calls and emails to helplines or individual colleagues
- Questions posed in social media
- Formal complaints
- Comments in face to face meetings
So, what does this kind of listening look like? Well, here’s an example from just last week.
Our funding colleagues were getting direct feedback from prospective applicants to say that they were getting confused about whether the funding guidance that they could see on our website was for the currently active rules for applying, or for historical rules that won’t apply to grants we make in the future. Specifically, people were getting confused because they were using Google to access our site, accessing our old guidance and assuming it was current.
We are grateful to the people who tell us about problems like this for a very special reason: it is a rule of thumb that if one person sends a complaint about a problem on a website, then odds on there are dozens or hundreds of people who have experienced that problem without us knowing.
Now, it might seem like there’s an easy solution to this particular problem: just delete the old guidance and old rules from our website to avoid confusion. However, there are plenty of people out there who have current grants that have to conform with the older rules, so we need to keep them accessible to that user group.
In the end we solved this by making some changes to the site that prevent Google’s search engine from finding and then listing the old guidance in its search results. This means that prospective applicants don’t accidentally stumble across the wrong thing. But we kept the old rules online, sat behind a login to a system that current grantees have access to. This means that they can get the information they need, without new applicants getting lost or confused, or worse still applying for funding using the wrong set of rules.
Observing
Here are two questions:
1 — Have you ever been to a website that had a problem and didn’t work like you wanted it to?
2 — Did you always send feedback to the people running the website, when this happened?
Unless you are a saint, or have only just started using the internet yesterday, the odds are that you have encountered plenty of problems that you dealt with without ever communicating directly with the website operators.
This is normal and nothing to be ashamed of — the vast majority of poor or failed experiences that happen online happen without generating complaints. In fact the same is believed to be true of offline or face to face customer experiences too.
Nevertheless, if we are going to ensure that our funding is available to as many people as possible, we have to know when people are having poor or confusing experiences, even if they don’t tell us. In this case we need to observe people, rather than talk to them.
These days there are quite a few ways of doing this, and a host of ethical questions to liven things up. But for today we’ll just talk about one of the oldest and least controversial approaches — studying anonymous data about what people are using the search box on our website for.
Searching in vain
Every time someone uses the search box on our website to search for something, we record the words that they type. This rapidly grows into a list that gives us insights into what people are looking for. We don’t know anything about who these users are, we only know what they type.
Recently we looked at the most popular search terms that people were typing in. Then, using our own website, we typed in the same queries to see what quality of result people were getting.
Many of the search results were good, they clearly worked. But some didn’t.
For example, we noticed that people were often searching for the word ‘deadlines’ — people clearly wanted to know if there were deadlines by which time they should submit funding applications. However they weren’t getting good results back because… we’ve abolished deadlines!
Fixing this was quick and easy: we changed the title of a particular page to include the word ‘deadlines’, and presto, people typing this term now get a more useful page that signposts people to the information they need. This problem isn’t entirely resolved, though: we know we can do an even better job of answering this deadlines question in future. However, this is still a nice, small case study about how we can learn what our users want, and improve our service, just by observing anonymous data created as they use the site.
In future posts we’ll be talking about other ways of both listening to people and observing user behaviour. If you’re interested in how a heritage funder does this, please keep reading.
