What does a Solutions Engineer do in BBC Experimentation?

Adrien is a Solutions Engineer working in the BBC Experimentation and Optimisation team. Designed as a centre of excellence, this team leverages its A/B testing expertise to help embed this discipline into products and support data-driven decision-making at the BBC.

What is your role as a Solutions Engineer in BBC Experimentation? And how does it compare to being a software engineer?

I am a BBC intrapreneur — my main mission is to drive change and evangelise A/B testing within the organisation through establishing myself as the technical expert in this field.

On a practical level, the job will sometimes be very close to a software developer-type role. You will often write code or peer-review technical implementations of other developers.

An important difference though lies in the kind of people I interact with. Within the same day, I might meet a product manager, a developer, a technical architect or a UX designer.

In this context, excellent communication and the ability to explain highly-technical concepts to a non-technical person, and to translate them into real business impact, is key.

What does a typical week as a Solutions Engineer in BBC Experimentation look like?

It’s difficult to describe what “typical” looks like as each day is different from the last — one of the things I enjoy most about the job! If I had to give an example, a week could include something like this:

Monday: Taking part in a strategy session with Optimizely (our A/B testing vendor) and BBC product stakeholder meetings to review the latest results, understand key goals and formulate how the next stage of A/B testing can help achieve this.

Tuesday: Presenting a headline testing prototype to internal stakeholders to bring A/B testing capabilities into editorial teams via a WYSIWYG tool.

Wednesday: Advising the BBC News Data Science team on how to structure an experiment to enable analysis by their key audience segments.

Thursday: Collaborating with teams based in the MediaCity Salford offices, such as Bitesize and Content Discovery, to offer technical support and drive best practices with the latest Optimizely features.

Friday: Fine-tuning the integration between Optimizely and our third-party analytics service to be able to enrich our Optimizely experiment data with our wealth of analytics data to better understand our audiences.

What qualities does an effective Solutions Engineer need to have?

If I had to pull out the 3 most important ones, I’d say:

1. Be solutions-driven (yes, it’s in the job title!)

You will spend a lot of time coming up with answers to questions that are completely new to you. For example:
• I want to run an A/B test on my Alexa Skill, which SDK should I use?
• How can I target my most disengaged audience?
• Why isn’t this metric firing?
• Should I implement Optimizely server-side or client-side?

Quoting a colleague, I am the tech wizard of the team:
Magic happens => problems disappear.

2. Forge an internal network of connections

BBC digital products can sometimes be quite independent from one another — in the sense that products such as BBC News, BBC iPlayer and BBC Sport each have their own organisational structures and success measures. To provide the best advice and guidance for A/B testing to support their objectives and product roadmaps, it’s critical for us to understand the team’s structure. For instance, a heavily engineering-led team will be more sensitive to performance discussions while another might want to start testing immediately.

In this context, forging an internal network of connections is extremely important: it helps us identify what the needs are, who makes the strategic decisions and how we can support this.

3. Be able to remain focused and productive in a fast-paced environment

One challenging aspect of the job is to be able to work on multiple things at the same time. Because of the number of products and teams we work with there are varying deadlines and priorities across different projects, so it’s critical to stay focused yet be flexible to adapt to changing business needs. A good SE should be able to deliver high quality work no matter the environment.

What are some guiding principles you follow when working with BBC teams or developing solutions?

Active listening is key when starting to work on a new project. During early conversations it’s extremely important to identify what kind of objectives and blockers exist.

I would also say that technically speaking, what matters most is to have robust and modularised code.

Finally, throughout the process, we work extremely collaboratively with teams. The objective is to support them in their A/B testing journey and really understand the intricacies of their product and their team before launching their first A/B test.

What do you love about your job?

The scope of our team is very large so every week I meet a wide range of people. Each product has different technical stacks, KPIs and objectives so in this respect I gain an overview on the breadth of the BBC online portfolio, along with the depth of technical expertise.

Another thing I love is that I often train teams about how to deploy their first A/B test. Bringing in new capabilities and empowering people is highly rewarding and I learn a lot from each person I talk to.

So, in general, I would say that what I value the most about my job is the fact that not only am I learning new technical skills but I am also developing a lot of soft skills by working in this role.

Is there anything you dislike?

The perception people have of A/B testing can be quite different from one team to the other. Some teams immediately see its value and actively adopt it, while others have different priorities and are harder to influence to instil experimentation as a tool in their toolkit. This means that although some teams will be fairly advanced in A/B testing practices, others prove challenging to drive adoption.

What’s unique about this role compared to working in a different organisation?

Working at the BBC is radically different to my previous experiences. Because it’s a public service organisation it’s vital to ensure quality and deliver on our public service values for all audiences for all of the content we produce. We’re driven by audience needs above all else which makes the work so rewarding.

Do you have any final advice for people with a similar role?

I feel incredibly lucky to have this job. It’s not often that you can work in a large organisation and have such a big impact on the business. So my first piece of advice would be to be an enabler. At the end of the day what matters most is how much you have supported and enabled the job of others.

Finally, don’t underestimate the creative part of the job. Even if the role is quite technical, you will often have to come up with new ideas and find original solutions.

Interested in joining the team? We’re currently recruiting. Apply for a Solutions Engineer role here, or search vacancies on the BBC Careers website for roles in Analytics, Experimentation and Data Science.