Evolving the One Medical Leveling Guide

Leveling guides have become a popular tool for engineering teams in recent years. While they have existed longer at a few larger companies, recently many smaller teams have been adopting and adapting them to create clear career growth paths and drive fair employee evaluations. Many people have written about the benefits of having a leveling guide / career ladder to serve as both a compass for engineer’s growth and also a bias-removing tool in performance and promotion decisions. Squeaky Vessel has a great collection and analysis of a number of company’s guides (Rent the Runway, Square, and Patreon are all good starting places.). While much is said on the value of having a leveling guide, what do you do once you have one, your team grows from 30–100, or you hire a new manager with good ideas to improve the guide? In this post I will share our process for evolving our guide to support a changing team and environment.

Any document that guides process at a company is bound to be imperfect, because it was written by humans about humans in an environment that is constantly evolving. In designing our leveling guide at One Medical, we wanted to reinforce the ‘living’ nature of this document — we expect it to change and want to encourage our engineers to actively participate in helping it evolve. This encouragement started from the moment we shared the guide with the team. In our communications (be they standups, team meetings, or one-on-ones) we emphasized that this is a tool, meant to make us better at achieving our mission, and that we will continuously hone it to ensure it is meeting that standard.

We started getting feedback, which was great! We then needed a method to decide which feedback to adopt, so we began collecting and discussing goals with the team on how we wanted the evolution of our leveling guide to work.

A few clear goals rose to the top. We wanted to ensure that we collected perspectives from anyone who had them, believing that a diversity of perspectives would help strengthen the guide — a Senior Manager and an Associate Engineer may read the same words completely differently. Similarly, we wanted to allow others to view and comment on the open suggestions to include additional perspectives.

We also considered that frequent changes to the document — and therefore the evaluation criteria for promotions — would feel too unstable. However, we wanted the guide to change and evolve, so needed to balance these considerations. We decided that a good balance would be to review changes in batch on a regular cadence. We also wanted to make the decisions clear to remove ambiguity around them, and that this regular review process could help.

After articulating requirements and some friendly debate, we decided on this list:

• Anyone can propose suggestions
• Suggestions are collected (similar to pull requests) and can be commented on
• The change suggestions should be addressed on a regular cadence with clear decisions
• The group approving them should be a combination of managers and individual contributors

Designing a Solution

Over the next few months we tweaked the original design some and ended up with this:

The columns here are:

• ID — so we can reference and move them around
• Cells — the referenced cells of our leveling guide
• Previous Text — what the cell said before
• Suggested Text — what the proposed change suggests
• Reasoning (for the change)
• Added by — who added this change; in case we need to ask followup questions
• Notes — for whatever miscellaneous notes are needed
• Status — limited to: Open, Closed — Rejected, and Closed — Accepted
• Implemented — housekeeping column to keep track of which ones have been updated

Communicating the Process

After creating this sheet, we sent an email to the engineering team and spoke about it at all-hands to encourage people to feel free to suggest changes to the document — again reiterating that it is a living document. Over the next few months, we accumulated 19 suggested changes from both engineers and managers. They ranged from simple formatting changes to more complex changes to the meanings of specific leveling criteria. Reading through the suggestions, it was clear that people had taken ownership of improving the guide, which was great to see!

Collecting and Reviewing Feedback

After about 9 months of using our leveling guide and collecting feedback, we moved on to the next step: accepting or rejecting suggested changes. (I’m not sure what the perfect cadence for these reviews is yet, but 9 months felt about right to balance update frequency with stability performance expectations. [Edit: after more time on this, we’ve landed on trying to do these every 12 months.)

To do this, we picked a team of engineers and managers who ranged across levels and teams, aiming to try to gather a diverse perspective, while attempting to keep the group size relatively small (we ended up with 9 people).

Given the size of this group, I wanted to make our priorities as clear as possible for the meeting. (Note: I switch from ‘We’ to ‘I’ framing for this part of the post because not everything needs to be done by committee and this part made sense to do individually to set up the group for success.) Prior to pulling this group together, I reviewed all of the changes and marked some as ‘easy to review’ where they were simple formatting or small wording tweaks. The remaining items I sorted into groups of related items and ordered them in a way I thought would allow us to make the most progress possible and address the highest priority issues first.

We started our 90-minute meeting reviewing the easy items, approving a number of them in quick succession. Then we were left with the harder-to-approve items such as debating how best to capture in words the role of mentorship for senior engineers. There is no magic bullet to make this part easy, we had to dig into each suggested change, consider the effects on the company and the rest of the guide, debate the merits of the change, and then decide to approve or reject the suggestion. Several times we decided to amend the existing suggestions where we all felt it was an opportunity to make an improved change.

We set the bar high, aiming for consensus to approve changes. We felt strongly that if something was a truism for software engineering, it should be possible to articulate it in a way we all agreed to. This felt risky going into the meeting, and I was worried that we might end up stuck, unable to approve anything, but this ended up not being an issue. Sometimes it took more discussion or modification of the suggested changes to bring the group to agreement, but those discussions led to better end-results. In one instance, for example, we ended up combining multiple suggestions from different people into a single change which addressed the tension of expecting senior engineers to understand our entire architecture while that architecture grows and scales beyond what is reasonable for an individual. We ended up moving from “Has great understanding of our entire architecture, systematically thinking through potential design impacts on other teams and the company.” to a better phrased alternative: “Has a broad understanding of our high-level architecture, …”

In full disclosure, we did not make it through the full list of items in this meeting, but we were able to make it far enough down that we felt we’d made decisions on the most important issues and could wait for the next cycle to address the remaining items. Each change has a cost, for individuals and the company, and as we made our way down the list of suggestions, we reached a point where the extra change cost wasn’t worth the benefits we’d get. Implementing many of the most timely and relevant changes while leaving some for the next round struck the right balance of change and stability for our team.

Making the Updates

Following the meeting, the remaining work was convert this work into a final artifact and communicate the changes clearly, with the rationale behind each, to the rest of the team. Doing this required us to make the changes in a new revision of the guide itself, write up documentation of what changed and why, and then send that out to the team across email and Slack. This last step of communication is critical to supporting all of the goals of our leveling guide from above and creates a feedback loop to reinforce that individuals have agency to improve the guide.

For any issues where we decided to decline a change suggestion, we included the decision in our decision spreadsheet and added notes describing why. I had invited many of the people who suggested changes to the meeting, which made this communication easier, otherwise I probably would have wanted to deliver any feedback on the rejected items to those who had suggested them.

From there, we’re on to the next revision.

Final Thoughts

This process will likely continue to evolve as we run more rounds of revisions, yet so far it has worked to solve the problem of encouraging engagement with the leveling guide. Engineers and engineering managers have voiced appreciation for having a clear way to participate and see how their feedback is handled in this process.

As I’m writing this, I realize that one thing I have not done well enough is to bake our communication about the leveling guide and how it works into our onboarding. It’s just a small piece of everything that’s going on day-to-day, so it doesn’t need to take center stage, but it is a critical piece of how people are evaluated for compensation and promotions. As such, it deserves thoughtful attention.

I hope learning about our process gives you a chance to consider how you might allow your team to engage with your leveling guide as a tool for your organization and its people to achieve success, both collectively and individually. If you have any questions, please feel free to reach out. You can find me at jammons@onemedical.com.

Thanks to Jason Herbst, my partner in designing and running this process and Kyle Munkittrick and Rachael Stedman for their amazing editing and feedback.