Avoiding hippos with a culture of peer review

By Douglas Cook, Skyscanner Growth

Ever been in a sign off meeting that felt a bit like this?

Scenario 1

Many of us have been there: the dreaded sign-off meeting. Months of hard work coming down to that one sign-off meeting, and very often that one individual in the room that matters the most- The HiPPO (Highest Paid Person’s Opinion).

Scenario 2

You are in a fast moving, highly autonomous team, and your team lead has passed on a critical project for you to work on yourself. You are up to your eyeballs in data to help in your decision making but could do with a second opinion to check your assumptions, but as there’s no process to check or approve your decision, it’s entirely up to you.

Neither scenario is ideal. Not only do I remember those dreaded sign off meetings, I also remember the way they used to often dictate projects right from the start; an idea didn’t even get past the agency pitch stage if you knew the marketing director wasn’t going to sign it off at the end, regardless of whether YOU thought it was right or not.

I also remember the pitfalls of the second scenario. Being data driven is great, but sometimes things aren’t always black and white and having someone else to help with that shade of grey can help. Or perhaps they are black and white but sometimes when you are so deep into your own work, it’s difficult to see the wood for the trees.

So how does it work in Skyscanner Growth?

Aside from a brief foray into the world of sign offs and approval meetings circa 2014, we have for the most part operated in the second model, with high levels of autonomy and accountability for all. Even where individual squads and tribes may have had particular processes, checks and measures in place, we are certainly more autonomous and independent than some businesses can become when it comes to getting activities live.

I have no doubt that our current modus operandi is the correct one, but as part of my squad’s role in enabling others, we have had questions posed to us about how we continue to ensure the quality of our activities are world class, while ensuring all squads get the level of support needed to continue their development as data-driven growth marketers…while still doing that in a ‘Skyscanner way’.

Improving our process in Growth

After looking at various options, and in particular taking a look at how our product colleagues currently review their own experiments, we liked the idea of peer review.

“Peer review is the evaluation of work by one or more people of similar competence to the producers of the work.” Wikipedia.

Aside from this video shared with me by a colleague…

…affectionately describing academic peer review as a ‘bloodbath’, it seemed like the best way to ensure the quality of output, while maintaining squad autonomy. There was no hierarchy when it came to review. Instead it came from those of similar competence at any level.

To roll this out, just throwing open the doors for anyone to critique everything didn’t seem like the leanest approach, so we assembled a crack team of data driven experts from across our Growth teams, data science, experimentation and product to start to review things on a small scale to see how we got on. After an intense period of upskilling to understand what peer review was, and ensure we were well versed in the latest techniques in data driven marketing, we went about meeting once a week to review a selection of activities.

Peer review process

It’s been an incredibly valuable two months since, not least because our test group have learnt a whole load about running better tests and experiments ourselves, then passing that knowledge on to others. The qualitative feedback we have had from squads has been very positive, and in campaigns going live we have seen clear improvements in the quality of how we test and validate our activity.

How do we scale?

Running a weekly meeting to review activities is not easily scalable, and risks become a quasi-sign off meeting that we are trying to avoid. So we are using Slack to encourage project owners to seek review at the appropriate points in their activity development.

We are using Slack to encourage project owners to seek review

How do others do it?

I’ll be honest, when we were looking for ideas as to how we do this ourselves, we didn’t come across many other examples of how Growth and Marketing teams working with high levels of autonomy ensure campaign quality without a sign-off meeting. Instead we had to look outside our domain for this. If anyone has tried peer review, or indeed any other alternative to that dreaded sign-off meeting themselves, we’d love your thoughts in the comments section below.

About the author

My name is Douglas Cook and I am Product Owner for the EMEA Growth Factory Squad here at Skyscanner. Our squad helps to develop and implement the processes, tools and techniques that are helping take us from a more traditional Marketing to Growth orientated business. Having started life as a ‘traditional’ marketer, I’ve loved being part of the Growth transformation, not just because of the impact on the business but also the development of my own skills and expertise over this time, which epitomises what makes Skyscanner such a great place to grow and learn.

Want to grow with us? We have great perks as well as producing a Growth environment. Remember if you see a job you like in a place you don’t already live and you have a successful interview, we will relocate you.

Douglas Cook, Skyscanner