The data driven way to build a technical organization

Ryszard Szopa
Inside Inovo
Published in
7 min readApr 19, 2019

Let’s say that you are hiring for your engineering team. You go through the usual motions. You put up an ad on the Internet, you receive some résumés. You sift through them to find the promising candidates. You invite them for interviews. Your engineers spend time interviewing them, and finally: you found a good candidate. They solved the interview questions with panache, and they are nice people. You know you won’t mind spending half of your waking life with them. Now the tricky part left is giving them an offer that they will accept.

S#!t tech interviewers say: “How would you simulate a fair coin using a biased one?”

There’re a few things that can go wrong.

It’s not rocket science that offering too much is not great. If the fact that you are overpaying wasn’t bad enough, you are also establishing a benchmark for your other employees. As the information about the new hire’s salary diffuses through your organization, people who feel to be at least as talented as them (yet paid worse) will come to you asking for a raise. If you refuse they will feel unhappy about it and might go look for another job. By the way, being overpaid doesn’t imply that someone is a bad employee whom you should let go. It’s possible to be very talented and overpaid at the same time.

Offering too little is also bad. The candidate may reject your offer, and then you wasted the time and money you put in the process. If they accept it, it can be even worse. They will somehow learn that they are underpaid and may feel cheated about it, even if you eventually give them the raise. What is more, undervalued employees are especially susceptible to poaching. To quote Jeff Bezos, “Your margin is my opportunity.” In that case you waste not only the effort that went into hiring, but also the effort put in the onboarding of the new employee.

Also, if all you have to back your salary offer is a gut feeling and what you are paying to your existing employees, salary negotiations tend not to be very productive. The candidate also has their own gut feelings, and if you are a smallish company, a similarly sized sample.

To make these issues less of a problem, you should establish a data driven salary policy. What do I mean by that?

First, create an engineering ladder. Most companies have vague notions of junior, mid, and senior software engineers, but that is not enough. You want to write down your requirements for each position. At MicroscopeIT, where I was CTO, we decided that we needed more granularity, and added 3 sublevels (so, junior 1, junior 2, junior 3, and similarly with mid and senior). Assign each position a salary, or a salary bracket. We went with a single number, because it was simpler. I would give a discretionary bonus if I felt someone deserved it.

The fabled engineering ladder. If you climb high enough, you’ll be able to reach the shelf with the one book that explains how to deliver every software project on time and under budget. Or so the legend goes.

You should consult your written requirements when making decisions about slotting a candidate to a position. At MicroscopeIT, we conducted 3 technical interviews, and the interviewers were not allowed to discuss the candidate until they wrote down their feedback, which included a position proposition. With 3 independent estimates, we tended to be quite accurate in our slotting.

You should document your whole hiring process in writing. Ask your interviewers to write down their feedback, ideally right after the interview. Create a template for them to follow. Prepare a note documenting the discussion of the hiring committee, and the decision. This is super helpful to train new interviewers, and to audit and improve your process. Keep the documentation confidential. Normally, only management and people directly involved in it should have access. It’s perfectly OK for an interviewer to be unenthusiastic about a candidate that is eventually hired. They should feel confident in voicing their opinion knowing that it won’t affect their relationship with the new hire.

Another use for your requirements is when people want to be promoted. Again, it won’t be a matter of opinion versus another opinion — you will have something to refer to. At MicroscopeIT, a promo candidate would have to prepare a promo package. This was a document in which they built the case that they were already performing at the target level (I didn’t want my organization to fall victim of the Peter principle). Having more steps in the ladder has the additional benefit that it keeps people motivated to keep progressing as engineers. There’s always a proximate goal in their sight. At the lower levels promotions should be smooth and relatively easy. For my junior programmers, I had a strong growth expectation. This is Google’s employee handbook euphemism for up or out. If a junior didn’t go for promo around every 6 months, it was a sign that they were not a good match for the company, and I might have to let them go. As people were climbing up the ladder, promotions became harder, but also the expected frequency went down. Becoming a senior engineer was like getting tenure in academia — you didn’t need to get promoted anymore.

A twist specific to MicroscopeIT’s promo process was that the candidate would be assigned a quest. This was a task whose accomplishment was a necessary condition to getting promoted. The quests varied from person to person. For example, someone would have to learn how to use a new technology. An engineer who was very shy had to prepare a presentation about a topic he was an expert in about for our weekly learning sessions. In another case, our “go to” person for some topic had to train someone else in his expertise area so that we had some redundancy. In yet another case, someone had to lead their first project. As a manager, I treated the quest as a very important tool to affect the direction in which my people were going as engineers.

Our quests didn’t involve brooding photogenically in the mountains, unfortunately.

A difficult question is how to assign salaries to positions. That is indeed tricky. You probably have some intuitions, and you can gauge the market by looking at how candidates react to your offers. Of course, this is a potentially expensive way to learn — amazing candidates are rare and far apart. You can try to compare notes with your competitors. However, they may not have the incentives to be completely sincere about their salaries. (Regardless of that I recommend that you socialize with your competition. It can be beneficial for your mental health — no one else understands your problems and struggles as well as they do. After switching to VC I became good friends with some of my former “frenemies.”) You can also look at salary brackets in job offers (if they are customary in your market), but in my experience, these tend to be very fuzzy.

What we did at MicroscopeIT was simply paying for a report about programmers’ salaries. We asked for (and got) an anonymized version of the raw data that went into the report, and did some data science on it. We filtered out people who didn’t match the profile of people we were hiring. Among those left, we identified cohorts that roughly matched our expectations for what a junior, mid and senior meant, in terms of years of experience. We decided on a target percentile, and set the salary for the highest sublevel of each position (junior 3, mid 3, senior 3) to match that percentile from the respective cohorts. For other sublevels, we did a linear approximation based on those numbers.

What was our target? At MicroscopeIT we worked on some rather complex stuff for our customers. To justify the premium we were asking for it, we needed to have very good people, so we set our target to the 90th percentile. This meant that only around one in ten companies would be offering the candidate a higher salary than us. Paired with rather difficult interviews, our hope was that this would translate to us getting people oscillating around the top 10%. Of course, setting a target percentile is a business decision, and it should match your business. You should adjust it to the kind of work that you expect your programmers to be doing, and the prices that you plan to give to your customers. For example, if your main value add is great product design, but the engineering work is relatively simple, it’s better to set a lower target. Spend that budget you just freed on product managers and UX people.

No, that’s not your budget for UX people, silly. But, how would you find the one biased coin in this jar of fair coins with as few tries as possible? Does knowing how the coin is biased change the complexity?

An interesting side effect was how this contributed positively to our employer branding. In a few cases, I could see how the candidates were getting excited as I was explaining our methodology to them. This kind of data driven people who care about their organization being data driven is exactly what I was looking for.

So, this is a few tricks that helped me as CTO of MicroscopeIT. A lot of them were inspired by how things were done at Google, where I worked for 5 years. Of course, I had to adapt them to a much smaller organization (and smaller budgets). I also left out the parts that I felt were overly byzantine or plain bad for morale. In this article, I am focusing on engineering, but you should be able to adapt the ideas to other functions. What is your organization’s policy? How would you make my process better?

--

--

Ryszard Szopa
Inside Inovo

Aspiring to be a gentleman and a scholar. Ex-Googler, ex-Affirmer. Trained to be a philosopher. Interested in AI, scalability and startups.