Engineering growth: assessing progress

Part three of a five part guide to assessing professional growth at Medium

As we continue to learn, we are now on our third iteration of this process. Please see here for the latest update.

We recently undertook a project to define professional development for members of Medium’s engineering team. As part of that, I wrote this document to explain how milestones are evaluated. It was originally published to Hatch, our internal version of Medium, on August 25, 2017. For more information about Medium’s practice of making internal documents public, see Hatching Inside Medium.

This is the actual document used by Medium to help understand how we assess progress, and has replaced the version on Hatch as the source of truth. If you take a job at Medium, this process is how the review panel will consider your growth during formal assessments.

Preface: Introduction
Part 1: Framework overview
Part 2: Tracks
Part 3: Assessing progress
Part 4: Appeals process
Part 5: Frequently asked questions

You can also read the rubric in full, and use the interactive tool.

At first glance, the growth rubric can be a little overwhelming. There are lots of tracks, and lots of examples. Engineers may be uncertain as to whether they qualify for a given milestone. It is important that everyone be clear on how we judge these milestones as they will directly affect an engineer’s level, and ultimately compensation. We must establish trust in the process and be as consistent as we can, judging people solely on the merits of their contributions.

This guide is designed to explain how to read the rubric, how to assess progress towards milestones, and how to discuss the framework.

Before that, some important points:

Subjectivity vs objectivity

Ideally, we would have a completely objective rubric, with simple Yes/No decisions made about clear, concrete tasks. Of course, this isn’t really possible.

Even in a team that is as mission-driven as Medium, if we are prescriptive about exactly what counts for credit, human nature says our people will do precisely those actions, and not focus any energy on tasks not defined. We can’t possibly define all the tasks that we want our people to do, and we cannot predict the tasks that we will need our people to do in the future. A fully objective rubric would be brittle, and would incentivize the growth of a team with homogenous skills.

There are also more intangible behaviours that, while incredibly valuable to the team, cannot adequately be captured by tasks alone. One of the central challenges of valuing knowledge work is that, though the output of building products in teams is actually quite measurable — does the product do what it’s supposed to do? — many of the skills necessary to do so aren’t easily measured. How do you capture communication objectively? How do you capture selflessness?

Of course, subjectivity invites the possibility, and likelihood, of bias. We have to be constantly vigilant and take steps to ensure that we’re applying the rubric evenly. This includes reviewing the overall distribution of level increases made during formal growth assessments, ensuring that the review panel that makes those assessments is broadly representative of the team, being as transparent as possible without compromising privacy, and providing an appeals process.

Developing a conversation

One of the key ways we hope to reduce the problem of bias is by moving away from all-or-nothing, twice-yearly assessments. We have observed that these are frequently opaque, frustrating and stressful for engineers, who see them as one of a limited number of high-stakes opportunities to demonstrate progress. Medium’s growth framework instead encourages and requires an ongoing conversation between an engineer and their group lead, where growth plans are created and discussed together at regularly scheduled check-ins. To facilitate this, we created an interactive growth tool that helps visualise how different kinds of progression will affect an engineer’s overall level.

It is important that the group lead establishes a long-term, trusting relationship with their group members, and takes the time to understand each one’s career goals and personality. Group leads should leverage this understanding to advise them on appropriate tracks for growth. From time to time, they may also have to guide engineers more firmly, either for business reasons, or because they recognise a latent capability that is not being realised. While engineers are in principle free to choose their own path, the realities of building a team and product will sometimes require compromise on their part. This is a healthy thing to discuss as part of career planning.

Impact and opportunity

Our tracks are framed as increasing in impact, whether that mean complexity, scope, or responsibility. However, impact is very strongly tied to opportunity. In many companies, it is much easier to progress, for example, if you’re directly making the company money. Those not in a position to do so are at a disadvantage.

Group leads at Medium should be working hard to ensure that members of their group are presented with opportunities to progress. To emphasise this, group leads are specifically incentivised to do so with the Career Development track, which aligns their goals with their group members’.

We believe that people management is an active endeavour. At the same time, we want our engineers to make their own opportunities. By providing many tracks for progress, we hope they take the initiative to lead in areas that interest them and which are underserved. Instigating Change is a core value of Medium, and engineers are strongly encouraged to do so on behalf of both themselves and the company.

Should they struggle with this, engineers are expected to advocate for themselves in the regular check-ins, and raise any concerns they have around a lack of opportunity to showcase and develop their skills.

Assessment

Each track has an overall description, and five milestones. Each milestone has a description, and a set of example behaviours and tasks.

The examples are only examples. It is not required that an engineer do the listed tasks, or exhibit all the listed behaviours. The examples and descriptions are intended to be read together to paint an overall picture of what we might expect from an engineer that has accomplished this milestone.

We do not wish to promote box-ticking, or to incentivise everyone to do the same work. Engineers should feel free to substitute tasks of equivalent complexity or scope, and group leads should help set expectations as to what qualifies, in consultation with domain experts, using the examples as a guide.

The 5 Cs

To achieve a milestone, the review panel must have a good faith belief that the engineer has been performing to the appropriate standard. In general, the engineer must have demonstrated a Conscious, Comfortable, Continuous, Consistent Competency, defined as follows:

• Conscious: having devoted intentional effort to this endeavour,
• Comfortable: without being overly stretched,
• Continuous: for a reasonable period of time,
• Consistent: reliably and evenly,
• Competency: meeting the criteria.

As a natural consequence, an engineer does not achieve a milestone the first time they demonstrate relevant behaviours or tasks.

Overall, if it is not Clear that an engineer is at a given milestone, they are at the previous milestone.

Some engineers may find that they exhibit some of the example behaviours at a later milestone without exhibiting all of the example behaviours at an earlier milestone. This is to be expected. At the earlier stages of an engineer’s career, they are limited to simple contributions. Engineers with more experience can typically contribute in higher leverage ways which supersede those simpler tasks, meaning they no longer do them. It is assumed they could do them, should they be required to.

However, there are certain fundamentals which engineers should not grow out of. It’s hard to imagine someone progressing very far along the Communication track if they aren’t Collaborating with empathy, even if they are facilitating the communication of entire teams and dozens of people. Writing tests for every patch, for example, is an important behaviour that every engineer should be exhibiting. If there are noticeable gaps at lower levels, engineers should use these as an opportunity for introspection, reflection, and improvement. At senior levels, role modelling becomes very important, and authority, respect, and credit gained from demonstrating the right behaviours is stronger than that merely based on tenure.

Cycle

In order to minimise the overhead in convening review panels, and on our people and finance teams, we will continue to make formal milestone and level changes at discrete six monthly intervals. However, in the intervening periods, it is expected that group leads will conduct informal progress check-ins with their group members about every six weeks. These check-ins should discuss the tracks along which the group member would like to improve, the kinds of projects and behaviours that would reasonably demonstrate improvement, and ongoing progress towards achieving them.

The aim should be that when the formal assessment is made, neither the group lead nor the engineer are surprised by the outcome. To ensure this, group leads should also regularly check-in with other engineering leaders to help understand whether significant enough progress is being made. Of course, the final assessment will still be subjective, but the provision of examples, good cross-team calibration on milestone requirements, and the regular discussion of progress should help alleviate stress around those decisions.

Formal assessment will be made by a review panel of senior engineers. Group leads are expected to advocate on behalf of their engineers, and make a proposal for milestone and level adjustments, if warranted. The review panel will consider the supplied evidence to support these adjustments, and may — based on their experience, and company knowledge — recommend lower or higher milestones than those suggested.

At this time, there is no plan to lower milestones established in a previous growth assessment, but we reserve the right to change this — with plenty of notice — in the future, if necessary.

Progress

Not every engineer will have an overall level change in every cycle, especially given that each milestone and level becomes successively harder to achieve. In situations where an overall level change is not warranted, we hope that formal recognition of improvement along some tracks may still be motivating for engineers, and will encourage them to keep up the effort. This is more likely for senior engineers, for whom overall level changes are typically much harder to achieve than for someone early on in their career.

Group leads should use the regular check-ins to effectively set expectations, so a senior engineer doesn’t wonder why they appear to be making no visible progress, even as they believe they are improving.