Measuring the leadership organization
How do we measure the effectiveness of leadership?
At the start of the year, when life still felt “normal,” our tech/product leadership team held an offsite to establish our goals for the year ahead:
- What would our theme be for 2020?
‘Innovate at Scale’ emerged as the top priority for us.
When we talk about innovating at scale, we essentially mean: how quickly can we validate our ideas? An example of this is our A/B testing framework. In 2019, we were able to run no more than two A/B tests at a time; today, we can run up to six due to the successful outcome of running those initial A/B tests.
- How then could the team deliver value for the company and our users more quickly and effectively, efficiently tackling any roadblocks that would inevitably appear?
We also realized how important it is to measure the effectiveness of our leadership organization, and ultimately the effectiveness of the organization itself. After all, setting goals without assessing how well we’re accomplishing them is not a worthwhile exercise. We asked ourselves what we’re doing to measure how we’re doing, and also what we might be doing better each day (our company motto).
Currently, we have a number of feedback loops to help in our process of ongoing self-assessment. We have regular retrospectives at the scrum and project team level; bi-weekly lean coffees, and a Kaizen team working with executive leaders at the company around sponsorship opportunities. We also have quarterly health checks for each feature team and those of the various disciplines, eg, Test Engineering and SRE (Site Reliability Engineering), which help determine how folks are feeling about the work they are doing.
What is a team health check?
Many technology organizations use the squad health check model to help scrum teams build self-awareness about what is and isn’t working. For example, when speed was an issue for us and this feedback surfaced often in our squad health checks, it made us realize the urgent need to focus on improving it. To help address this, the SRE team hired a release engineer to aid in automating release processes and set up CI pipelines that teams could then take over to develop further.
Here is a more recent example from a health check for one of our teams:
The importance of data gleaned from health checks
Our program management team gathers health check data on a quarterly basis from across teams and disciplines.
Upon sharing results with our teams, we then ask team members to vote on an area where they would like to see improvement. As an example, if the team feels that speed is still an issue, we might ask the team:
- Why is speed an issue?
- How is this impacting our ability to deliver value to our users?
- How can we solve this?
- How do we measure this?
- What is our MVP and how do we validate its outcome?
Now that you have a sense of the purpose for squad health check models, let’s talk about how to apply this to a leadership team.
What makes the leadership health check model different?
While health checks reveal important insights that help us get better every day, they can only be made useful if our team leaders get involved.
We have two health check models that help us to inform our leadership team, which includes leaders from engineering, product, uxd, data, and program management.
One health check to be completed is given to the leadership team, which is simultaneously evaluated by a check performed by the tech and product teams. Both models include questions around trust, psychological safety, effectiveness, values, and more. In addition, users have a choice to rank the leadership team’s effectiveness on a scale of 1 to 5, as well as being able to choose whether things are improving, getting worse, or staying the same.
Here’s how the process works, around the category of trust for example. The leadership received this question about the topic:
The tech/product organization, for the same category, received a similar question but in the following format:
What did we do next?
We analyzed the results to determine whether or not we were all aligned; is the leadership team aligned, based on how we all voted, and do the results from our model, from the questions given to the tech/product organization, help us validate if there is sufficient alignment with leadership? We also went through anonymous feedback — an important part of the feedback loop — for a better understanding of why members of the tech/product org voted the way they did, and also to assess possible trends.
What were our findings?
In most categories, based on the data, it was determined that the leadership organization was aligned with the greater tech/product organization. However, around the area of support, we felt more could be done. Based on specific feedback, we learned that, as we have grown, our team leads have been tasked with not only taking on management responsibilities but also setting technical and strategic directions for their teams. For many of the team leads, especially on the engineering side, our feedback loop reveals them as key contributors in delivering value to our users.
We also used this methodology as an opportunity to coach and mentor future leaders who would then be able to take on management responsibilities and introduced tech lead roles for various feature team groupings. The tech lead role was specifically designed to directly support the engineering team leads in executing the technical direction for the various products/features as well as the overall team.
A key takeaway
The trust example shows the effectiveness of our feedback process. We found this to be a useful tool and plan to use this as a feedback loop within the leadership team, as well as to receive feedback more widely from the tech/product organization.
We also learned that having a theme at the start of the year allows us to have a more narrow focus that can serve as a useful tool to ask the type of questions we asked at our leadership offsite in January.
Looking ahead, while as a company we are fortunate to have continued growth in these times, we are now faced with the challenge of growing and remaining effective while growing remotely at the same time.
Thus, the theme for 2021 may be along the lines of:
How might we build a scalable and effective remote team?
Looking for a new job opportunity? Become a part of our team! We are always looking for new Insiders.
