Built to Scale: CEO Workshop Recap — Hiring Your Senior Team with Corey Thomas

Pillar’s Built to Scale: CEO Workshops series brings together a select group of early-stage founders to learn from proven Boston CEOs. Each interactive workshop focuses on a core topic related to building a high-performing company.

Corey Thomas, CEO Rapid7

Hiring is one of the toughest challenges for both first-time CEOs and experienced CEOs. It takes a lifetime to master, and the stakes couldn’t be higher — success or failure is usually decided by the people you hire.

As CEO at Rapid7, Corey Thomas led the company’s growth from 50 to 1,200 people. He’s incredibly thoughtful about how to plan, how to hire, how to manage out, and how to approach go-to-market staffing for new products. So we jumped at the chance when Corey offered to share his depth with our team and a select group of founders at our recent Built to Scale: CEO Workshop, Hiring Senior Leaders.

Beyond his role at Rapid7, Corey’s commitment to the Boston ecosystem is clear — he’s one of the 17 Founding CEOs who co-founded our firm, Pillar, to help find and scale up the next generation of major companies in Boston. Corey was also previously a Group Project Manager at Microsoft Servers and Tools (SQL) and VP Marketing at Parallels, and graduated as a Baker Scholar from Harvard Business School.

Corey broke his talk into several parts:

How to Plan Before You Hire

The classic organization chart is “the worst thing in the world” when it comes to planning new hires. Too generic, lacks thoughtfulness.

So begin by thinking out ahead 2–3 years. Remind yourself what you want the business to accomplish. Honestly assess your people. Honestly assess yourself. Then ask: “What type of experience and talent would give us a materially higher chance of success?” That’s who you should hire next.

Sometimes that’s about attitude not experience. For example, when Rapid7 was at 200 people and wanted to transform from a product to a platform organization to reach the next level of growth, this kind of thinking helped Corey realize that the key element missing from his company was not in any one department. Rather, they needed more people with a mindset for “reckless experimentation” to help to find platform opportunities.

More recently, the greatest need they have is hiring more people focused on “operational excellence”. Again, this insight doesn’t come from an org chart, but from reflecting on greatest gaps between today and where they want to be in the future.

Hiring in Unknown Areas

Many CEOs are struggling with the challenge of how to identify and evaluate candidates who will lead areas of the business where the CEO does not have personal experience.

Quite often the next hire who will most help the company is a complement to the team, i.e. someone who fills in a gap. How do you hire someone whose job you don’t understand?

As an example, let’s say you’re finally ready to go into production, and you need a VP of Manufacturing. Perhaps no one in your company has any manufacturing experience, yet this is a make-or-break hire.

Corey took extra time to explain the process for filling a position that is new to you and your team:

  1. Resist the urge to hire quickly! Be particularly careful hiring anyone who could “break” your company. Allow up to 2 months just to scope the search.
  2. Start by finding other companies who are performing well, who have already gone through the same stage and evolution that you’re about to face. Resist the urge to find people who work at those companies now. You want the people who worked at those companies then.
  3. Identify people from that list who are NOT potential candidates. Meet them, and ask for their story — what actually mattered? What drove your success? Corey considers this a hugely important step and urges that CEOs not skip it, despite the time required. After each discussion, ask for one more contact, so you can keep learning from wider circles.
  4. Once you know what is important, write down an interview guide and run a good hiring process (more about this below).

Evaluating Performance Outside Your Expertise

After the hire, how do you manage a person whose job you don’t understand? For example, what if the company isn’t generating sales. How do you know if it is a bad VP Sales hire…or something else?

Corey’s answer may be counter-intuitive but is a great example of his trademark style of candor: “Start by simply acknowledging to the VP that you do not understand their job. Then just ask the VP, how he or she thinks you should assess performance?” You will learn a lot from that conversation.

The next step is to separate out the parts of their job that you do understand and see if that is going well. Think about it this way:

(Process + Behaviors) * Context = Outcomes

We all know outcomes are volatile so understanding what happened can be confusing. The context is the environment around you, the market, the timing, and other factors no one controls (creating a headwind or tailwind that acts as a force multiplier). But, process and behavior ought to be predictable. So ask them to explain the processes and behaviors they intend to follow and then watch to see if their actions match their words. e.g. If the manager says the process is two forecast meetings per week, do they actually hold two meetings?

If there is a gap in process or behavior, you will be able to see and manage accordingly. If not, and if the company’s results remain bad, then you probably have a larger strategic issue with your approach. At that point you need to get outside perspective. Bring on an adviser and let the manager know, “I’ll be using XYZ as a sounding board about this problem. You should meet them too.” Hopefully the adviser can help you figure out the problem.

How to Hire: Culture Comes First

Many people start by asking if a candidate has the skills to do the job, and if so, whether the culture can accept the person. However, “expertise is uncorrelated to organizational productivity”. Hiring a person with insufficient skill is a mistake that only affects one job, whereas a bad cultural fit brings down everyone in the company. On the upside, if you get culture right, culture is a productivity driver that boosts to morale for all.

So culture is what should be first in mind when you hire, then skills.

There are two culture questions you should emphasize during the hiring process to help protect the organization from “dangerous” hires:

  1. What is the candidate’s attitude about success? You are looking for people who share the aspirations of the organization and who have the same belief system how to attain success. For example, Rapid7 believes that success comes from curiosity and experimentation. One time Corey hired someone who instead believed that “repeatable standards are the most critical driver of success.” It was oil and water.
  2. How does this person behave under distress? This is vital because a start-up often faces distress, and you need a team that behaves constructively. Avoid people who “attack and blame” when they feel stress. That reflex will drive down productivity for all until it becomes “Kryptonite to a small team.”

Aggressive behavior under stress is particularly hard to screen out because it is invisible during the interview process, especially for senior people who have learned how to navigate interviews smoothly.

Therefore this should be a focus for your background checks. During the interview, you can ask a candidate “Give me a situation where things weren’t going well. What was your behavior? What would you do differently? Who were other people you worked with at that time?” Write down the people and then you have a list for potential reference checks.

You can also try this: open up about yourself first, and be brutally honest about how you handle stress. Then ask them “How do you handle stress?” They will typically be candid with you too. You can learn a lot just by whether the person has ANY answer, because that indicates an ability for self-awareness, which is key to working with others.

Commonality vs. Diversity

One CEO asked “How do I get the right mix of types of people?” Corey notes that in setting your culture, you need to be intentional about what you want to be common and what you want to be diverse. At Rapid7, the commonality is curiosity. That is a trait that can be shared across people from different backgrounds, race, gender, nationality, education. Conversely, look for aspects where you do want to encourage a healthy friction.

Interviewing Best Practices

Corey did not have time for a detailed discussion of what questions to ask during interviews. He gave two tips on the overall process:

First, write an interview guide for each position. Questions should include job expertise, context of their past jobs, work behaviors, and the traits that will help you assess fit with your company’s culture. By asking the same questions consistently across candidates, you develop a tuned ear.

Second, a good habit is to bring your existing team together for ten minutes before any candidate shows up for interviews. Talk through what you want from the position and what criteria are for hiring. Then get together again after the candidate leaves, and discuss how that person compared against the criteria. This extra work feels at first like it is slowing you down. But it will help bring the group onto the same page, so that when you do find the right person, you know it. Another benefit: as people compare questions they asked, the organization trains itself on how to interview.

All of this hiring takes work! Corey noted that start-ups frequently have concentrated bursts of hiring in the months just after a financing event. He estimates he would spend 50% of his time on hiring during these periods; in hindsight 70% would have been better.

Managing Out

As you plan for hiring, also plan for “managing out”.

This is not just about firing for poor attitude. Sooner or later every CEO makes a wrong assumption about the needs of a position. Or, you will define the right position, then hire the wrong person.

Or, you have the right person in the right job at first, and then the needs of the company change over time. The most gut-wrenching moment for a CEO is when you have an employee who did a good job but their role is no longer needed. You will feel a sense of loyalty. You still have to manage them out.

Corey likes how Reid Hoffman’s book The Alliance describes work in the 21st century. Reid argues that the old model of “employee for life” is dead. Companies now are more like sports teams, where a group of players come together for a season, after which it is normal and healthy for the lineup to change. Although a few people in a company are “foundational” and could stay for a decade, many are “transformational” and will stay just for a single tour of duty, and that’s the new normal.

So as you talk with employees, openly recognize that your company will evolve and that each stage may need different people. Then, when you manage out an employee who has played an effective role, you can start by thanking them for how they helped during that stage.

It is healthy to have regular check-ins with your team. Two helpful questions you can ask are: (1) Are you still enjoying this job? (2) Is this right for the organization going forward? Every person needs their job to stretch across a triangle of: FIT (skills and culture), OPPORTUNITIES and DESIRE. After a role changes, they may no longer have the right skills, see opportunities for growth, or continue to feel motivated. By having this kind of check-in, they may discover “I really love the ground floor, but we are past that. I’m ready for a change.” and manage themselves out gracefully.

Sometimes a person will ask to take on a new role they have never done. Ask them, “OK you want to try to do X, but is this a good risk for the company?” and also, “Is this new job really the best way for you to learn that skill?”

Ongoing Assessment Must Be Candid

Culture is set in three key ways: who you hire (use the interview guide for consistency), who you reward/promote and why, and who you manage out.

The most valuable feedback you can give in performance management is “what are the behaviors that are problematic?” If someone’s behavior violates your culture, warn them and if it does not change, manage them out ASAP. Also if people don’t turn out to have the skills you thought, you need to replace them ASAP.

There is no need to work yourself up to be all huffy and angry when managing someone out. Be polite but candid. It’s your job to make sure you have the right team at all times. They know that. As CEO you are going to be having tough moments like this on a regular basis. Invest in learning how to have difficult conversations. Then have them early. Don’t let people get surprised.

Corey says “If I can encourage you to do anything in a tough situation, it’s this: be explicit! State the issue openly. Write it down. Talk about it together.”

Hiring Your First Commercial Leader

Several CEOs at the workshop asked Corey about this situation. Suppose you are a technical founder CEO who raised seed to develop an MVP, and now you need a commercial person to bring it to market.

How to do this well?

All of Corey’s advice above about “Hiring in Unknown Areas” very much applies here, especially if you do not have any thesis yet about how the company should go to market. You will benefit particularly from finding expert advisers.

Once you do have a thesis, then think about your go-to-market effort just like running an experiment. We all understand experiments.

Start by considering your Value Proposition, and then define your Go-to-Market Hypothesis on four levels:

  1. The Customer Promise
  2. The Sales or Engagement Experience
  3. Your Own Salespeople vs. Distributors
  4. The Price Point

So, when you hire, hire someone who already has experience consistent with your particular plan across all four of those. Have them come in, tweak, and run the test.

Your first commercial leader should have a few unusual character traits:

  • Repeated career track record of being first to sell new products
  • Unfazed by failure; just sees it as input mechanism
  • Willing to tell your team their assumptions are all wrong, even if that will rub them the wrong way

In Corey’s experience, these sales leaders specialize in the early stage, and typically lose interest in the later stage when the process is well-understood and the job becomes more about operationally scaling up an organization.

So in a way, you need someone exceptional enough to be “almost a co-founder” and partner with you as you figure out product-market fit, but that co-founder may be temporary, as they may not want to stick around to scale.

Additional Tips on Go-to-Market Stage

You want to get yourself out of the loop sooner rather than later, because CEOs can sell in ways most people cannot and so those first few deals you close may not lead to a scalable process. If you do the selling personally, the rest of the company does not learn how to sell.

When Rapid7 launches a new product, they might assign four junior salespeople to see if they can use the “go-to-market hypothesis” to make sales. As less experienced people, any sales they close are most likely because of product and process fit, rather than just strength at personal selling.

In Corey’s view the goal of Go-to-Market stage is not to have the most sales! The goal is to have the most repeatable sales model. This means that revenue you earn opportunistically, for example by selling a custom product or by selling to off-target customers, is not productive for the company. So don’t let your sales target turn into the organization’s focus during Go-to-Market stage. The focus is to run a clean experiment and learn from that.


Thank you Corey, for sharing all your insights! Thank you also to Sarah Hodges and Vidhan Bhaiya for help preparing the recap.

As a reminder, if you are a start-up CEO, you can apply to attend one of our upcoming Built to Scale: CEO Workshops here.