How we stay user-focussed in the Boiler Room product team

Boiler Room has done very well curating the best music experiences for people — knowing what’s best is the way the company works. Boiler Room has managed to staff the team full of people that know exactly how to create the best underground music content — most of these people are trend-setters in their musical fields. The company is really good at getting the best underground music content made, by leading, not following. It’s no surprise that the early Boiler Room digital products were developed this same way, with the company ‘knowing’ how these should work and what they should do, and being built by freelancers, agencies, and whoever was available. It’s also no surprise that these weren’t the best products in the world ;) Luckily, as any successful business does, as Boiler Room grew, they adapted; Boiler Room knew they needed to build up a professional product team.

Enter the product team — about 18 months old now, we’re a small team of designers, developers, and PMs that have a lot of freedom to determine the future of Boiler Room products. One key choice we’ve made is to actively work to ensure that users are at the heart of our product decisions. These are a few of the ways we keep users at the heart of our product decisions and how well they’ve worked out so far.

Regular, in-person sessions every 2* weeks

Inspired by the Lean UX approach for regular fixed-date contact with users, we have regular days set (*we aim for every 2 weeks, in reality it’s slightly less than that) where we know that 5–8 users are coming into our office for us to show work in progress to, ask questions, and generally get an impartial, outside point of view on ideas we are working on.

Whilst we don’t always have a perfectly pruned research roadmap, the fixed-date means we always know we need to prep work, and run experiments that can get in front of real users, and validate things we are working on.

One of our participants giving his opinion on our prototypes

Our basic process is something like…

  1. create a Lean experiment / define a hypothesis
  2. find some people that fit our current requirements and invite them in (we’re extremely lucky to have an ultra-engaged audience)
  3. spend a week or so building up some artefacts to test with, and planning our research sessions
  4. run the sessions (in pairs, other people can observe)
  5. a quick turn-around of the synthesis, and create some output document stating our goals, methods, results, and whether our hypothesis was validated or not

How’s this been working out so far?

My (bias) view — great. Not only do these help us get impartial, real user views on ideas we have that drastically change the shape of our products, they also help remind the entire company that we are not our users, and that we are designing for a wide variety of real people with views different from our own sometimes. The output of these sessions, when shared around the company, inspire and shed valuable light on why we make certain product decisions. 10/10 would research again.

Should you do these?

Yes. If you can get users, or people close to your user groups, to come in easily (and they don’t cost £200 per hour as I have previously paid in another role -_-) then this is a fantastic way to just get a real sense-check every couple of weeks. I believe virtually any team can make this happen if they try hard enough, whether it’s every 2 weeks in-person, or every month using Skype, if you really want to have fixed date checkins with users you can.

These are the most resource-intensive of our research techniques but completely worth it. No matter what you think you know from support chats, inbound emails, or analytics, it isn’t until you sit with someone and skilfully probe them on a subject that you really get to understand their goals and motivations. The biggest investment, but the biggest reward.

Quantitative surveys ran on our site as-and-when, with results in hours

We have a pretty high number of users visit our site every day. Previously, we didn’t really gather much useful info from these visits, apart from general analytics data which is pretty much only ‘what’ and not ‘why’. We installed Hotjar, as a cool way to help us understand more about the ‘why’.

Hotjar does a bunch of things — auto-generated heat-maps, tracks user movement through the site, records videos of how users navigate your site — but thing we use most is the ability to quickly deploy simple surveys to get some rough user data to help shed light on any burning questions we have.

Picture the scene: there’s a meeting where a mixed bunch of non-product people are discussing a cool new idea. A lot of times these idea meetings can go something like so… ‘users will definitely do this, because they always like that!’ If you’re as lucky as I am, these wild assumptions spark energetic debate, and seem to go around and around in circles for 60–90 mins, with no real definitive outcome agreed that a product team can take away and run with.

Hotjar allows us to deploy a survey in 1 minute, leave up for a couple of hours, and suddenly we have a couple hundred responses that help us shed some light on the item in debate.

A basic poll in Hotjar

No this is not enough to be completely confident in an outcome, and no this is not the best way to research answers to assumptions… but this is minimal effort, very fast, and does shed some data on assumptions that can (and have done for Boiler Room) completely change opinions and help us move towards a better product solution we all agree on, faster.

How’s this been working out so far?

So-so. We get results in hours, we run surveys every couple of weeks. The results do help us make some basic decisions, and help inspire some more detailed research. These have a definite place for us at BR, but we would never rely on only this data.

Should you do these?

It’s fast, it’s cheap, but the quality of results is not super high. You won’t uncover insane insights with this, but you will collect data quickly that can help shape some decisions.

This is the fastest and most low-maintenance thing we have done. 20 mins to install, under a minute to run a survey, a few minutes to analyse results.

Recommended if you have a high traffic website to really quickly get results. If you put up a survey and it takes 12 hours to get results — then I would say it loses a lot of its power.

Foster a community of passionate users

This isn’t so much a research technique, more a tip that has helped us get user insights super-quickly. BR is lucky that our user base is so engaged and passionate about our brand. Started by our social marketing team, we now have the Boiler Room Insiders private Facebook group, made up of >6,000 passionate Boiler Room fans that are willing to give opinions pretty much instantly.

A research-related post in the insiders group

It’s very common for us to put a request out and get a great level of response within minutes. These awesome people give opinions on hot topics, beta test our apps, come in to meet us, and generally help us understand another perspective on everything Boiler Room. We know they skew ‘power users’, but it’s ok as long as we’re are aware of this.

This is obviously only useful in some areas of research, but has notably helped us understand which Smart TV platforms our audience owns, has found dozens of issues with our iOS app, and has given us hours of interesting chats to read between themselves.

Unfortunately this isn’t possible for every type of business, and is only really possible if you have a decent sized group of passionate fans, but if you have the opportunity at all to create and foster a group of passionate users, do it immediately — the value is instant.

Whilst it’s pretty known that passionate users will help out product teams, this has been the first time I’ve witnessed first-hand just how much of a benefit this has been in helping us create better products with fewer issues.

How’s this been working out so far?

Regarding investment… this is ongoing — with our community manager giving incentives and merch every once in a while, and slowly building out the group. Whatever the total investment as been so far, the return is >10x

Should you do these?

If you can, do it. Whether a Slack channel, Facebook group, whatever… if you can get a group of people willing to chat about your product all day long, then go for it!

This is just the start

This is a snapshot of a few ways we are changing our working to stay user-focussed in a company that did not originate that way.

I am well aware that this is not a perfect user-centred set of tools, but this has helped us immensely to keep the product centred around what users actually want.

Hopefully you can employ some of these tactics — I’d love to know if it helps your product process, and if so, how?