Setting your engineers up for success: how Skyscanner created greater clarity in our competencies and pathways to progression

By Emanuel Müller Ramos, Skyscanner Engineering Director

For many companies, this time of year is often appraisal or performance review season. Feedback is really important as we each seek to learn, progress and improve generally. However, making sure everyone is rated fairly and consistently can often be tricky if companies don’t have robust and clear frameworks in place. This is something we’ve worked hard to ensure we address here at Skyscanner, and today we’d like to share our journey of making competency frameworks, and performance expectations, really clear within our Engineering org.

Supporting our engineers’ growth is one of the most important responsibilities of our Engineering Managers. Part of this includes helping engineers understand the expectations of their role and level, while also having clarity as to what the next stage of someone’s career might look like. To do so, Skyscanner has a competency framework outlining skills and behaviours for each level, from entry level roles all the way to CEO.

However, while the competency framework was regularly shared and discussed, a few years ago we noticed that we’d occasionally find gaps in understanding. For instance:

• Some Engineering Managers having a gut feel that a engineer had a gap in their performance, but couldn’t relate that to the competency framework expectations
• Some Engineering Managers feeling uncomfortable to deliver feedback because they were not sure if they were being over-demanding or if their ask was within the expectations of that level
• Some Engineers feeling frustrated because they were not fully sure how a gap highlighted by their manager connected to the expectations in the competency framework

While cases like this were isolated, leaving it unchecked could have consequences on how we set our engineers up for success in their current role and in their career progression. In response, we decided to make the competency framework simpler to understand and created dedicated training around it.

Making the competency framework second nature for everyone

In the tech industry, most progression frameworks are organised in competency pillars. Example of pillars commonly seen in the industry are:

• Technical skills
• Delivery and impact
• Culture and behaviours
• Scope of the work
• etc

Normally, the expectations of these pillars are organised by level in a isolated manner:

Competency frameworks normally present the expectations for each role individually, one after the other. But this results in difficulties in identifying a difference in delivery expectation between levels — something that’s helpful to know when we think about personal progression

While in many ways presenting the information in this manner is valuable, it brings a few challenges. Indeed, as the description of each level can take one or multiple pages to be described, the connection between what changes between the competency pillars, and levels, might get lost. For instance, to clearly understand what changes in terms of the expectations around Technical Skills between a Graduate Engineer and a Senior Engineer, one needs to jump between multiple pages and carefully read their descriptions. When engineers rightly want to understand their progression path to the next level and beyond, that’s not the most helpful way to take this information in. We knew there had to be a easier way to do this.

Easy comparison

As engineers, looking for differences between pieces of code is part of our daily routine. For instance, when reviewing a Pull Request in GIT, we’re often looking at what changed from the previous change-set to the new one. Wouldn’t be great if we could do the same with the competency framework? This is exactly what we did!

We’ve built a tool which we call the ‘Levels Visualisation Guide’, which is basically a pivot of our competency framework, comparing and highlighting the differences in the competency pillars for each level.

In the same way Github allows you to compare two files side by side (source), the same was done with the competency framework

Let’s have a concrete look on how this works considering the Scope competency pillar at Skyscanner:

Example of side-by-side comparison of the Scope competency pillar. Same colours indicates the expectation is the same between two levels, while different colours highlight differences and unique expectations. The descriptions in the table give further clarification and examples of what the competency framework description means in practice.

Training all the engineering management community

After the Levels and Visualisations Guide was built, the next step was to make sure the message landed and all Engineering Managers were equipped to use it. Live training has been devised where the competency framework is presented using this format:

• Firstly, one of the Competency Pillars is presented and the differences between each level outlined (e.g. Scope)
• Then, the audience is split in breakout groups where an example is presented to them. Using the Levels Visualisation Guide, they need to assess the performance of the engineer for this very particular example
• After a few minutes, the audience regroups and shares their findings. Usually, the exercises include some pitfalls, which are revealed to the audience
• The same is repeated with the other competency pillars

Here is a concrete exercise:

Example of a practical exercise. On the left, a hypothetical scenario description for a given engineer. In the middle, the expectations for that level (Software Engineer) considering two Competency Pillars (Scope and Expertise). On the right, a list of common mistakes that engineers and engineering managers fall into when assessing performance.

Results and next steps

Since this initiative was piloted in 2021, training has rolled out to all Engineering Managers. The feedback and benefits outlined are very positive:

• During our regular Performance Calibration process, we observed Engineering Managers are significantly more confident on their assessments, using the Levels Visualisation Guide as an additional tool to help understand performance
• Feedback collected after training indicates an improvement of the understanding of the competency framework by 34%

This feedback also helped to surface areas of confusion and clarity in our Competency Framework, which has allowed us to iterate and improve training and tools again.

Now, we hope to expand this initiative even further, for the whole engineering community in Skyscanner, whether or not individuals are managers. This means that everyone will have a deeper understanding of roles, expectations and progression pathways. A pilot has been delivered to do so, and training is being made accessible offline also, so engineers can try it out at their own pace in case they can’t make the live session.

I like the idea! How can I do the same in my company?

As any agile project, consider starting small and iterating:

• Take a Competency Pillar which is very clear and known in your company. For instance, Expertise/Skills is normally less ambiguous than something such as Impact/Delivery
• Attempt to build a side by side comparison of each level for this pillar. In the beginning focus in levels up to Senior Engineer, leaving other levels such as Principal/Staff engineer for later
• Present this comparison to a pilot group to collect early feedback and buy-in
• As traction is gained, add more competency pillars, add the remaining levels and start expanding to other tribes

Let me know in the comments how this went, I would love to hear your feedback!

Emanuel Müller Ramos (aka Manu) is an Engineering Director at Skyscanner. Manu has been with us five years and leads the Flights Booking Tribe. He spends his time focussing on growing engineering teams and helping the engineering discipline to adopt a metrics-driven culture.