My team at Kromatic and I originally published this post here on Grasshopper Herder, our lean startup blog. Kromatic’s mission is to help companies get to market fast and reduce the cost of innovation. Subscribe to our newsletter or speak to one of our coaches to see if we can help you reach your innovation goals.
When a member of your team recommends a book with the subtitle “Be a Kick-Ass Boss Without Losing Your Humanity,” you should probably read it. So I did!
Radical Candor is a best-selling book by Kim Scott about improving your relationships at work. My colleague allegedly didn’t notice the subtitle and suggested it because of the 2×2 on the cover (I love 2x2s), but I took the hint just in case.
One of several tactics that Radical Candor suggests to foster better team relationships is conducting regular 1-on-1 meetings with direct reports. Radical describes the process as:
“1:1s are your must-do meetings, your single best opportunity to listen, really listen, to the people on your team to make sure you understand their perspective on what’s working and what’s not working. These meetings also provide an opportunity to get to know your direct reports — to move up on the ‘care personally’ dimension on the Radical Candor framework. Remember: this is not a place to dump all of the criticism you’ve been saving up. That should come in those two- to three-minute impromptu conversations you’ve already been having!”
The benefits seemed clear. I once worked directly for a CEO in a small team where nine months passed before I was invited to a 1-on-1 meeting. As a result, I never really had the opportunity to bring up individual concerns or express dissatisfaction.
I could have been proactive and insisted on scheduling time, but I didn’t. Most of us don’t. By the time I had an opportunity to sit down with the CEO, I had decided to quit and used the opportunity to say so. It was too late.
Creating more avenues for feedback and discussion between a team lead and direct reports seemed like an obvious win.
However, the Kromatic team likes to apply lean thinking to all changes, even small ones. Especially if it’s going to affect the team.
Implementing 1-on-1 meetings the way Radical Candor suggests isn’t a small change. It would involve at least 6 person hours of time — almost a whole work day.
Instead of doing exactly what the book said, we decided to run experiments to see how 1-on-1 meetings would work for us. So here’s how we set up and ran our small-scale human resources experiment on one tactic from Radical Candor.
I’ll explain the process in detail. It may seem like a lot of effort for something so simple, but we have a lot of experience with similar experiments so it didn’t take a lot of time. We spent five minutes detailing each experiment, ten minutes collecting the data at the end, and another ten discussing things.
In fact, writing this blog post took me more time than the actual experiment.
What’s the Learning Goal?
“Employees set the agenda, you listen and help them clarify.”
We were already holding 1-on-1 meetings on a semi-regular basis, about once a month, but it was a two-way conversation rather than one where the direct report set the agenda. A strict implementation of a direct report-driven meeting seemed odd, because our team doesn’t really do direct reports.
And although we work with international partners on projects, our core team in San Francisco is small — only six people.
Technically everyone reports to me, but more often than not, we report to each other on different projects. If my colleague Nick Noreña is leading a project, it’s his responsibility to tell me what to do — not the other way around.
I have been known to step in and override if something is going off the rails, and we’re not immune to Hi.P.P.O issues. So we’re not a holacracy or anything like that. Regardless, we needed to modify the format to fit our culture.
We wanted to understand if the meeting format Radical Candor suggested would work on a peer-to-peer (P2P) basis. Each of us would set an agenda and hold a 1-on-1 meeting with someone on the team who would simply answer our questions.
Initial Experiment
We first defined an experiment on a Trello card as a checklist. We used an internal tool we built called Temello to implement templates in Trello. Here’s the original, unsanitized experiment design notes:
- Learning Goal: Will P2P 1-on-1 sessions create greater transparency and teamwork?
- Hypothesis/Assumptions: If we schedule 1-on-1 sessions with every team member, we will each identify one thing we’ve learned and one action item to improve our work together.
- Method: Real prototype
- Metric: # of learnings reported per team member, # of next steps reported per team member + qualitative feedback
- Fail Condition: <0.5 learnings and next steps reported per team member
- Wtf,s! (“What the f***, stop!”): Can’t get at least three 1-on-1s scheduled by 8/22
- Time Box: 1 week
- Plan: Ask everyone to set up their own schedule with team members
This format is a modified version of the Learn S.M.A.R.T template that we use to define experiments and research. (Note: There is an explicit warning on the template that it should be modified for your purposes.)
Let’s break this down step by step:
- Learning Goal: Will P2P 1-on-1 sessions create greater transparency and teamwork?
This is vague and high-level, but that’s OK. It’s a starting point that we like to phrase as a question. Our primary goal is to increase our ability to work as a team and create greater transparency between team members, which sounds fuzzy, but it’s important. We need to understand why people react the way they do and how we can better work together.
- Hypothesis/Assumptions: If we schedule a 1-on-1 session with every team member we will each identify one thing we’ve learned and one action item to improve our work together.
The hypothesis is stated as an IF ____ THEN _____ statement with a clear outcome. Every meeting should result in a lesson learned and a follow-up action.
- Method: Real prototype
This is an addition to the template, which allows us a quick shorthand to reference a known lean method such as a Comprehension Test or Concierge Test from the Real Startup Book.
“Real prototype” means that we were actually planning to meet. We weren’t going to do anything like a landing page/value proposition test by sending out meeting invites and seeing if our team members would respond “yes.”
That would have been silly since everyone knew about the experiment, and we have plenty of experience with team members accepting meeting invites. That would be over-experimentation.
- Metric: # of learnings reported per team member, # of next steps reported per team member + qualitative feedback
Here we are stating all the data we plan to collect. For example, on a landing page we might collect the number of unique visitors to the landing page and the number of people who sign up. For customer discovery interviews, we might take notes or make a recording.
Stating the metric is a way of making sure that we put a process or software in place to collect the data. For this experiment, we can get all the data from a quick group meeting or Slack thread.
A “learning” in this metric is just a piece of knowledge that the team member considered valuable enough to acknowledge and remember. A “next step” is some action to take based on one of those learnings.
- Fail Condition: <0.5 learnings and next steps reported per team member
Next we defined the fail condition rather than the success metric. Note that the hypothesis says we need to have at least one action and learning per meeting, and the fail condition is lower than that. That’s because perhaps some of the meetings don’t go well, we don’t follow the instructions, or some random disaster happens.
When we state the fail condition, we’re saying that if at least half of the meetings don’t result in at least one learning and action item, then this meeting format is not working and we ought to change something fundamental. There’s no question of “maybe let’s try the same thing again.” Something must change.
- Wtf,s!: Can’t get at least three 1-on-1s scheduled by 8/22
The polite way to say wtf,s! in a corporate setting is “Early Stop Condition.” Here, if we couldn’t get at least three meetings scheduled within two days, we knew that we were going to have a problem executing the experiment.
That’s not to say that the hypothesis is incorrect, but rather that we screwed up something in the experiment design.
- Time Box: 1 week
The time box refers to when the experiment should be concluded. We always try to keep this to within one week. If it’s longer, we’ll break it into smaller pieces.
- Plan: Ask everyone to set up their own schedule with team members
This is pretty vague. Normally we’d define this more clearly, but in this case we assigned each team member a separate Trello card to set up the 1-on-1 along with basic instructions on how to run them.
The Results
- Results: 3 meetings, 6 learnings, 3 actions (Data from Susan, Tristan, Kenny, Anteo, Megan)
Our results were very straightforward. We only managed to schedule three meetings out of five team members that were available that week, but that yielded two learnings and one action item per meeting.
One interesting piece of qualitative feedback was that the learnings were not one-sided.
“The 1-on-1s were really helpful in reducing some of the interpersonal friction I was experiencing.” — AQ
Even though only one person was asking questions, both parties learned something about the other person: how they liked to work, why they act the way they do, what would make them better collaborators, and so forth.
We also conducted a mini-retrospective. Again, here are the real notes:
- Make schedule for meetings clearer (some people scheduled for following week)
- Gather data more accurately (double counted meetings)
Essentially, we didn’t effectively gather data the first time because I was double counting meetings, so I had to redo the data collection. Also, we only held three meetings because of a flaw in the initial experiment design — it wasn’t clear to some team members that the meetings were supposed to take place that week.
Next Steps
- Next Steps: Continue. I will generate a pseudo-random schedule so that no one has more than one meeting a week.
We decided to continue the experiment and generate a more random schedule, because it was overwhelming when several people tried to schedule meetings with the same person at the same time.
So we defined a new experiment.
Increased Sample Size
- Learning Goal: Will p2p 1-on-1 sessions create greater transparency and teamwork?
- Hypothesis/Assumptions: If we schedule a 1-on-1 session with every team member we will each identify one thing we’ve learned and one action item to improve our work together.
- Method: Real prototype
- Metric: # of learnings reported per team member, # of next steps reported per team member + qualitative feedback
- Fail Condition: <0.5 learnings and next steps reported per team member
- Wtf,s!: Can’t get at least four 1-on-1s scheduled by September 8
- Time Box: 4 weeks
- Plan: ask everyone to set up their own schedule with team members
This experiment is largely the same with a couple small changes; most noticeably we wanted to have at least four 1-on-1 meetings each.
The time box was set to four weeks. That’s normally a no no for us, but running four more experiments would have been over-experimentation and not resulted in more significant learnings.
Also, doing a complete round of P2P meetings would have been 30 meetings. We didn’t want to spend the whole week in meetings, so we spread them out over a month.
The plan is still vague. Again, it’s not something we’d encourage for other teams, but our team had a good understanding of the experiment because we discussed it in person, and there were instructions in the schedule and Trello cards.
Results
- Results: KN 4 L, TK 4 L, SB 4 L 2 NS, AQ 2 L, NN 4 L
Our results were, frankly, not recorded well.
I used a shorthand that I can barely decipher a few months later. That’s not a good idea if the team needs to refer to the data in the future.
Here’s a translation: We ran the meetings with five team members because one team member was working part-time and couldn’t realistically contribute. From those 21 meetings we recorded 18 learnings and two next steps.
Our learnings were above the fail condition but the next steps were below. So what did we do?
- Next Steps: Keep rolling. Check in next month.
We ultimately decided to continue. Although we were below the fail condition, the qualitative feedback showed that the learnings were worth it, regardless of falling short on actionable next steps.
Next, Next Steps
We’ve continued our 1-on-1s, and it’s been a pleasure. Every time I go into one, I learn something, even if the conversations aren’t strictly about work.
For example, in Nick’s first 1-on-1 meeting with me where he drove the agenda, he asked questions about my background in philosophy. That threw me for a loop, but we ended up chatting for 25 minutes about Heraclitus, Lao Tzu, and religious studies. These topics turned out to be very relevant. Philosophy informs my approach to our work, especially discussions about moral hazards when it comes to which companies we choose to work with.
We don’t always make time to run small experiments like this. But when we do, it’s always valuable. So take the time!
Small, lean methods flex our brains and make sure that we’re constantly pushing an experimentation mindset as a habit. They’re not just a set of abstract principles we talk about and never practice. After all, if we aren’t learning something new or generating data, how can we improve?
Lessons Learned
1-on-1 meetings are hugely beneficial to team dynamics. (Try them on a P2P basis.) -Tweet This
We can run lean methods on internal operations such as HR, not just on landing pages. -Tweet This
Qualitative data is useful data, and it’s valid for making decisions. -Tweet This
Build lean thinking as a habit by practicing often. -Tweet This
My team at Kromatic and I originally published this post here on Grasshopper Herder, our lean startup blog. Kromatic’s mission is to help companies get to market fast and reduce the cost of innovation. Subscribe to our newsletter or speak to one of our coaches to see if we can help you reach your innovation goals.
