12 Observations from a Tech Due Diligence Survey
A personal view on the +800 results of a tech due diligence survey to early stage startups
Background and Distribution of Results
In a previous post, we ‘open sourced’ the technical due diligence of Point Nine Capital for early stage startups. We thought that a more transparent process would allow Founders to understand better our concerns and help them identify some of the potential challenges to come.
In that post we also released a toy project — a Typeform Tech Due Diligence Calculator and a Gitbook with the logic behind it, which we have left open for discussion.
We are overwhelmed by the results. In total, 869 people (at the time of editing, 909) took the time to fill in the survey and we owe them great thanks. We received much praise on the project and have had many requests to share the data collected.
As one respondent said:
“It’s sometimes painful to be honest with the survey, because everybody knows the right answer, but at the early stage it’s about trade-offs.
It will be interesting to see how the answers evolve over time.”
Here I’m trying to summarise some of my own observations from that survey. It’s not about getting the perfect answers, but rather trying to understand why some of them show those results.
If you want to see the data for yourself, here is a link to the Typeform report.
Observation 1: Most coding is still done in-house …
Maybe this doesn’t surprise you, but I expected more outsourcing for a couple of reasons:
- The lean startup methodology is the bible of founders today.
Everybody focuses on building an MVP as lean as possible to validate the opportunity. That means we more often see early stage products being built with third-parties like bubble.is, typeform, unbounce, etc. - It’s easier than ever before to access freelancers, and it’s harder than ever before to hire developers.
But it’s good to see that most of the tech founders are still in the role and sharing ownership with the early engineers.
2. … Even at the expense of speed of development?
Build or buy software decisions can have a big impact in an early stage company. This might sound dramatic but, we have experienced both scenarios:
- A company facing significant difficulty because of using an unreliable 3rd party API for a core function.
- A company with slower development speed because of technical debt in non-core systems. — like payments, invoicing, monitoring, etc.
In the survey, it looks like most teams have a some form of a guideline for those choices. Still, I’m surprised by the number of companies with custom invoicing and monitoring solutions.
50 companies even run their own payment system?!?
3. Unclear dichotomy between Reliability and Speed
Personally, I’m very aligned with this post http://coderlifestyle.com/software-development-speed-vs-quality-a-tech-shop-conundrum/
In my opinion, you have to make a conscious choice between speed and reliability. That way, you make the life of your developers easier. Once you choose and everybody is aligned, decisions are clear and they can run more freely.
If you don’t choose, then you need to add management processes to decide what comes as higher priority at any time. That means wasting time and effort in friction that reduces your speed.
In any case, that one was already controversial when we wrote the internal guidelines.
The results of the survey aren’t conclusive either! ;-)
4. Releasing continuously wins, but with limited tooling?
Half of the companies release continuously with high test and peer review coverage. A similar percentage of companies can roll back with a click, test features with flags or do staged releases.
I’m a bit surprised by those “low” percentages. In my opinion, the benefits of investing the time early to set those tools right are significant.
5. Investing into Monitoring …
Not surprisingly, monitoring tools have a significant degree of adoption — even at an early stage.
Almost every time I speak with the CTO of an early stage startup, she complains about the high price of New Relic. Still, nobody wants to live without it.
Other areas of monitoring are also getting a serious degree of attention.
6. … helps manage scaling?
Without proper monitoring, the task of scaling your infrastructure might become a lot harder. Therefore, to be honest, after the results in the monitoring questions, I expected a higher degree of awareness of some of the challenges to come in scaling your tech — and less about increasing the bill.
I guess the adoption of docker, among others technologies, helps us believe that, for a while, early scaling is about spinning servers / instances / containers.
7. In case something goes wrong, we are covered!
Good to hear that the source code and customer’s data are protected for a large number of the companies in the survey.
Interestingly, more than 10% of the companies don’t backup source code and can have trouble recovering customer’s data. Looks like a risky choice!
8. Customer-centric product roadmaps …
In our opinion, it’s pretty hard to wear the hats of CTO and product owner. Because many times there are trade-offs between what the CTO and CPO wants. In an ideal scenario, the CTO estimates how long does it take to develop something and the CPO prioritises based on that and customer’s feedback.
For the 25% of the companies who have the CTO as the product owner, that doesn’t mean you need a CPO, but it’s probably easier if the CEO or somebody else owns the product.
On the topic of product roadmaps, most companies have them written down and shared with the whole organisation. That can help them get contributions from the different areas of business exposed to customer feedback. Still, a few percentage of them have their customers involved — either in the form of VIP customers or sharing the roadmap openly with them.
9. … Without spending enough time with customers?
In our opinion, developers should be in touch with customers often to get first hand feedback. Not because they should set the roadmap, but because that will help to understand better why are they developing certain features. In my previous experiences, I’ve seen how powerful it is to sit together a few developers and customers in the same room for B2B companies. “Magically”, developers understand how/why some of the features they developer work/don’t work.
Surprisingly, in half of the companies the developers (not CTOs) never speak with customers. Even among CTOs, 25% of them never spoke with customers. But there is hope! Of the ones who speak with customers, 70% do it weekly.
10. On managing small tech teams
First of all, most of the teams from the survey are below 10 developers. That can explain why, for example, the majority of teams have worked together before. I guess that’s also a big driver for the amount of developers having shares in their companies.
On the more formal aspects of people management: it is hard to reach any conclusion in terms of the frequency of 1o1s, but it’s interesting that the majority of companies have their values written down.
11. In case you didn’t know, attracting tech talent is a challenge ;-)
50 CTOs aren’t able to convincingly pitch their company? Well, I hope this is because somebody was “trolling” the survey…
As expected, attracting candidates is a big challenge. Half of the companies get less than 10 applications when they advertise a position. Based on that, it becomes clear that building an attractive brand for developers is an asset.
I guess related with that, once companies attract a candidate, most of them try to finish the hiring process in a month. The survey also shows that developers aren’t in love with pre-screening tests.
Another surprise here is that only a third of companies do backdoor references. Because in our experience, that’s one of the best predictors for failed hires.
12. But retaining tech talent doesn’t seem an issue
Since the average size of the tech teams surveyed is small, it’s hard to reach a significant conclusion.
Among our respondents, it looks like they have healthy organisations because they attract developers from referrals (a key metric of team happiness). For the developers who leave the teams, we can guess is mostly because of wrong hires.
There are plenty of opinions in this post. I would love to hear yours.
Do you agree? Or disagree?Please reach out at @DecodingVC.
