Improving the methodology: How we collect feedback to get better
In October 2021, we introduced the SpaceUP structure to our team internally (you could read more about how the missions are structured, planned and delivered in our previous articles). There were major changes for the whole Product and Engineering department in the way they work, and also with whom they work. Some teams split, others merged into what we call Spaceports, and some parts of products and features changed ownership.
We designed the new organizational structure because some aspects of the departments were, to be honest, not working well — there were friction points, lack of feedback, missing ownership, and other issues we knew needed to be addressed to scale and achieve our mission. So, which areas do you tackle first when you recognize you need to change? The major driving factors can be summed up in these key goals:
- improve planning, prioritization and assessment of features
- increase visibility of the work being done in Product and Engineering
- cascade company objectives down into the organization structure
- speed up delivery time and enhance quality; fail faster
- empower teammates to innovate and disrupt
- promote our colleagues’ sense of ownership and responsibility over their work and features, as well as for customers and users
- strengthen knowledge transfer between people and teams
- give proper focus to all parts of the product
- celebrate successes and get feedback on work from customers and users.
Since then our Product and Engineering team prepared and flew over 30 Missions which took Crews 6 weeks on average to land. The team grew by 60 colleagues in past 6 months as well. The new structure seems to have been adopted well, as shown by the number of Missions and their duration, and as word of mouth spreads on 1:1s upwards within the organization.
But did we really fix our problems? Does it really work as we intended, and is efficiency as a metric enough? We don’t know. Even before SpaceUP we were capable of delivering new features in a more or less timely manner and we hired new colleagues along the way.
SpaceUP as a Mission
When we look at SpaceUP through the lens of a Mission lifecycle, there was a long discovery phase at the beginning where the team compared different organization structures and created the first prototypes of the methodology. We iterated with a wider audience and tested if what we proposed made sense. To fly the Mission, we prepared the whole methodology including pitching, Ground Crew, landing sequences, and promotional materials, and landed the Mission on our Product and Engineering All-Hands Offsite. After three months under the new reorganization it was time to measure how we were doing.
We approached it similarly to measuring any other Mission. We needed to look at the usage, the conversion rate, and the error rate or user comfort. We needed to define the metrics on SpaceUP as well.
The Survey
We choose to first focus on likes, dislikes, and level of satisfaction with the overall SpaceUP methodology. Second, we examined things like individual rituals, processes, and Mission phases from the perspective of SpaceUP’s goals, including having more time to focus, feeling of ownership, and teams’ capacities. Finally, we addressed issues that were brought up during 1:1s, such as problems not being addressed, and lack of 1:1s structure. This last part was very similar to standardized HR questionnaires measuring employee engagement.
One of the objectives we set at the beginning was to be able to track how successful we are in reaching our goals. That influenced the wording of the survey, to be timeless and general enough that it won’t matter if a respondent fills it in now, or in half a year.
We collected all the questions and topics from key stakeholders, reviewed a few iterations of the questions and options and in the end implemented the survey in SurveyMonkey. To ensure the answers were anonymous and everyone could respond as open and honestly as possible, we distributed it in multiple Slack channels using a link rather than via email.
What We’ve Learned
Of the roughly 160 colleagues in Product and Engineering, 108 filled out the survey. They spent typically around 30 minutes on it, and 88% of those who started finished the whole survey.
Overall, 64% of respondents reported that they liked the SpaceUP methodology, especially that they had more focus to work on the product, felt more ownership and responsibility, more transparency and visibility of our work, and liked the concept of Missions. And, of course, they liked the visuals and theme of SpaceUP. Those who rated the whole SpaceUP methodology highly also tended to follow the rituals, processes, and Mission phases more carefully.
Those who didn’t like it cited problems in the Ground Crew such as too few people, being on Ground Crew for many weeks in a row, or that it’s hard to hand over knowledge to other colleagues. Some mentioned that they believed the Mission pitching felt too bureaucratic, low capacity of the team, and unclear terminology and methodology of SpaceUP.
We had 63% of colleagues flying at least one Mission and 60% spent at least one week at Ground Crew supporting customers and fixing bugs.
Even the understanding of the company’s vision and mission was rather high, with around 80% agreeing that they know and understand it. Not surprisingly, a solid understanding of the company’s vision and mission correlated with respondents being comfortable trying new things and driving innovation, and vice versa. In Spaceports without clear objectives, respondents avoided trying new approaches and taking risks.
On the other hand, our Circles (groups of people organized around a specific area of expertise) didn’t receive much improvement since the initial reorganization, and 56% of our colleagues found it the same as before. The most common comments were that there was not enough transparency about what different Circles focus on, and some of them were perceived as not so active.
In general, the feedback was great. However, the survey was quite long, so we will repeat it only once every 6 months. The respondents appreciated the survey and that the company is listening and taking responsibility.
The Changes in SpaceUP
After collecting and analyzing all the data, we shared it in two phases. First, the key problems were addressed on the Product and Engineering All-hands. Two weeks after the survey ended we introduced the team to the steps we will take to address these problems.
Second, we shared a slide deck with all the data explained, as well as the whole data set for those who wanted to deep dive into topics of interest.
Based on the feedback from our colleagues we focused on changing these areas:
- Making the Mission process less bureaucratic and asynchronous
- Doubling down on hiring (and our referral program, which just recently received third place for Referral Program of the Year by the Recruitment Academy Awards in Prague)
- Improving how Circles work, setting clear goals, and promoting them within the organization.
Wrap-Up
Yes, SpaceUP is working. But as with all products, we need to measure, learn, iterate and improve what we started. We were excited and overwhelmed by such a high response rate, comments that were on point, and the time that the team invested into this survey. Metrics aside, this tells us that our team cares and wants to participate in the changes. We will repeat the survey in the summer and see how the recent changes work.
Want to learn more about SpaceUP? Read our previous interview with Zenbase’s Tomas Rehor to see how we found inspiration for our new framework. Or, why not start at the beginning with Chapter One!
