Diving into data before you hit the road

As field researchers, we’ve been brought up with the view that nothing beats talking to people and getting the real picture from the horse’s mouth. It’s also understandable that we defend this stance; the costs add up for field research and it can get expensive. It is often harder to articulate what we might get in return, compared to analysing data already available to us or conducting randomised controlled trials — techniques which can often seem to have lower barriers for higher gains. But it’s always struck me: instead of pitching qualitative and quantitative research tools against the other, wouldn’t it be more fruitful to combine them?

In my professional meanders, I’ve come across several cases where it makes sense to team up with data scientists to look at what data is available to us before we pick our research targets. Sadly, I can’t talk about the work I actually did, but let’s do a thought experiment with a potential case study.

Imagine you’ve been tasked to study customers for a large departmental store that has several shops in your city. The brief is general: discover their pain points and how we could improve their in-store experience, how we may achieve a continuity between their in-store experience with their digital experience. The goal is to eventually develop something that is on brand but fits in with the norms of each locality.

The tempting thing to do is to grab your recording or note-taking gear, walk into a shop, talk to customers for a few days and gather some insights. You might get fancy and produce some voxpop or a video of shadowing customers around the store as they do their shopping. In such a case, it’s easy for a qualitative researcher to default to our go-to habits. We also like to believe that we can take an unbiased view and it’s best to go in with a “blank slate” frame of mind. Instead, I’d like to argue there’s merit to holding back and looking at the possible customer profiles based on existing data before diving in.

For example, I would try and learn the following things first:

• Do all the shops across your city sell similar amounts of the same products? If not, what are certain standout sales patterns? Do we already know why?
• What do we know about the shopping demographic in each of these shops? Is there any other background data we can use to help us establish why the demographic is what it is? (If not, here’s one thing we can try and find out when we talk to real people.)
• Do we have any data that links people’s use of digital presence in these shops? For example, are there in-shop returns based on things bought online?

The reasoning for doing this kind of background work: it means that by the time you walk into the shop, you will be very well armed with the types of questions you may choose to ask. Rather than just gathering limited one-off stories, you’re gathering data that would allow you to link back to meaningful information at a business level or a product strategy level.

A single recording with a customer could then be weighted: is he or she an anomaly? Is a trend being seen at one location reproducible in others? You now have a way of evaluating the contextual data you get back. Having the data ahead of time would give you an idea of the scale and severity of issues, problems (even desires) that you unearth during the course of conducting effective interviews. With better awareness of which interviews are edge cases and which ones are more likely to be the norm, you can better compare similar interviews that may occur in different shops in different parts of a city. With clearer judgement, you’re in a much better position to decide what is interesting and what can potentially be useful rather than being led down a blind alley or a research rabbit hole.

So far I’m only speaking about what happens if you’re looking at shops across one city. What if our example departmental store has shops in several cities in a few countries? When this kind of study involves several locations, the background work using existing data becomes far more important. It is almost certain that you have limited time and budget on where you can visit, so you have to make informed choices up front.

Some example questions that we might have to work through in such a scenario:

• What are the sales patterns across different shops in different cities and countries?
• Which places provide the most interesting use cases? Obviously we might have to decide ahead of time what “interesting” means to us.
• Are there likely to be external forces which influence sales in one place over another? A popular baking show like the Great British Bake-off which airs over the UK summer might trigger sales of baking products that might not be reflected in other markets. At the same time somewhere else, perhaps a celebrity wearing a new outfit triggers a local fashion trend.

Again, digging up any data on these type of factors can provide a way to judge whether an issue, a problem or trend is universal or local, whether it is seasonal or if there is a long term pattern. When you’re working outside of your familiar context away from home, this kind of information provides you with a rough map or guide. Having a solid understanding of the context before you pack your bags means you can better confirm and dismiss your assumptions in the field, allowing you the room to uncover things you might not have expected — allowing you to identify what is an important insight when you see it.

Quantitative data can be great for showing trends, but qualitative methods still win at getting to the very deep wells of “why” and “how”. Qualitative research can also be powerful in unearthing potential trends that don’t yet show in data — which can help narrow possibilities in predictive modelling. On the other hand, data can help us focus our contextual research and other things besides. I’d go so far as saying that data from quantitative methods can be used in tandem while we undertake contextual research. If we keep an open mind, and look for ways to combine these approaches appropriately, it will help us build a greater, more complete story to make better strategic decisions.