Interviewing Data Analysts

In my role managing a team of data analysts, a common task is to interview candidates wishing to join the team. Such is the fashion of data nowadays, that separating the wheat from the chaff is always necessary, and this can only usually be done via a phone interview (I see brilliant resumes from awful candidates).

Assuming the candidate is capable of answering the Skype call (I’ve had some fail at this hurdle…) I like to start off with a very simple question:

Tell me a bit about yourself …

Here I’m looking for people capable of structuring and communicating the most intuitive of information: the history of themselves! Red flags are people talking for too long, people who can’t judge the level of detail and explanation needed to describe their past projects, and people who just say their name.

After that I hit them with a challenging scenario-based question:

The Head of TV Marketing comes you with a list of his TV spots. He wants you to tell him which ones are working and which aren’t. Talk me through your process, and ask me any questions you need to.

This is a multi-level question which seems to hit the heart of data analysis. Firstly, I’m looking for the candidate to challenge the definitions in the question. “How do you define working?” is a perfect riposte.

After that, I want to hear them ask for details on what data is available. I explain that they will have a list of TV spots, with the creative, the spot time, the TV station, and the country in which it was shown. At this point, many candidates get distracted by the qualities of the spots themselves, start talking about target demographics, but have lost track of what they’re trying to measure. Better candidates ask about more data, including visits to the website (we’re working for an online business), and possibly even mobile app installs, registrations, and sales.

By this point, a good proportion of the candidates have got lost — unfortunately failure to structure a new problem like this is unacceptable for me. Those that haven’t, tend to get stuck in the tricky task of working out how to assign website visits to the different spots. Many come up with some heuristic like comparing the times and attributing those within an hour or a day, which is good enough for me for an interview. But why not push them one step further:

What if two adverts are shown in the same country, on different channels, but within five minutes of each other?

I’m looking for them to suggest some sort of model for the visits per minute after a spot, and then learn the parameters of the model using historical examples, and the all-important gap between the spot times. I think I’ve only had one of hundreds of candidates answer this perfectly, and he’s one of my best analysts in my team today.

As for other data analysis questions, another favourite is to propose a pricing test for a subscription business. What are your most useful questions?