Reflections on a failed startup

Nearly a year ago, we decided to shut down Ansaro. In the immediate aftermath, I wrote a long list of contributing reasons. Then I put it in a drawer for a year; it was too painful to think about. I recently came back to that list, to draw out some lessons, which I hope will be useful for myself and others.

Let’s set the stage… Ansaro’s mission was to improve hiring, via data science-driven SaaS. After 2 years, we’d had a team of a half dozen, $3M in funding, and a handful of Fortune 500 customers. Nonetheless, if product-market fit was a heartbeat, we never had a pulse.

How come it took us 2 years to figure that out? How could we have avoided it? My learnings fall into 3 buckets:

• Team beats problem beats solution
• Learning in a wicked environment
• People stuff

Team beats problem beats solution

(aka: a good team is most important, then identifying a good problem, then finding a solution for the problem)

Build a team that isn’t afraid to disagree

Cofounders unafraid to disagree with each other have an advantage. Willingness to disagree, even about core beliefs, increases the speed/likelihood a team will identify a failing product (mis-guided faith that a product should work can quickly become a core belief). Faster identification of failure means faster iteration, and more chances of survival.

I built a team around a specific solution I had conceived (and to which I was emotionally attached). Since the team was recruited to build that solution, there was a feeling that declaring “this solution isn’t working” meant “I don’t want to be on the team anymore.” Our team members took contrarian views in their areas of expertise, but the way I had constructed the team made it hard for them to push back against the solution I’d been promoting since Day 1. That made us slow to pivot, and slow = dead for startups.

Rather than recruiting cofounders who bought into my idea for a solution, I should have built a team based on more fundamental elements: (i) deep mutual respect, as demonstrated by the ability for team members to change each others minds and (ii) interest in the same general problem space (in our case: hiring), but without attachments to specific solutions.

Go after a problem that is acute and quickly measurable

The problem we set out to tackle, reducing turnover and improving new hire performance, was felt acutely by our buyers (CHROs / Heads of Talent). It was NOT felt acutely by our users (recruiters). Recruiters are measured against average time-to-hire, not new hire retention or performance. We were trying to improve outcomes about which users would pay lip service, but which didn’t impact their paychecks or promotions.

Moreover, we knew it would take at least 6 months for our customers to observe improvements in retention or new hire performance. While that timeline worked on paper — we had runway for at least 18 months — it left us waiting far too long for market feedback. When the ultimate feedback was bad, it was too late to do another pivot.

Control your solution’s own destiny

Our first solution was joining records from our customers’ Applicant Tracking System with records from their Performance Management System, to identify predictors of successful hires. This meant integrating with two enterprise systems holding sensitive data. Most of our customers’ systems were more than a decade old. We could only dream of modern APIs.[Note 1] Instead we pleaded for nightly file-based data dumps with “partnership” representatives from these ATS/PMS software vendors, representatives who had zero interest in partnering with us.

When we finally realized we couldn’t solve these technical dependencies quickly, we moved on to another solution: software for conducting better job interviews by using natural language processing to parse interview audio recordings. We built this system to avoid dependencies on any other systems. What we didn’t appreciate was that we’d traded away technical dependencies for process change dependencies. Conducting better interviews means convincing interviewers to interview differently! That’s a change management project, not a software pilot.

We needed an MVP that had both minimal technical and process change dependencies. Only then would it have been truly easy to trial.

Learning in a Wicked Environment

In a wicked learning environment, feedback is poor, misleading, or missing.[2] The early days of a startup is a classic wicked learning environment; relying on intuition is dangerous, but data is sparse. What to do?

Limit user research

Initially, I believed extensive user research was a badge of honor. In our first few months, I did 80+ prospective user interviews. I collected hundreds of pages of feedback on our prospective solution. And I found ~20 prospective users who told me they were interested in purchasing our soon-to-be product; “Call us back as soon as it’s live!” they told me.

Statisticians call this “overfitting to the data”. If you have the world’s worst product idea and talk to 100 potential customers, 10 of them will be nice enough to tell you they’re interested. With so much data, I semi-subconsciously parsed it selectively to convince myself and others of the story we wanted to hear.

We should have done no more than 5 interviews of potential users.[3] I would have heard “no” four times and “meh” once. And if that was all the data we had, we would have been forced to concede there wasn’t demand for our idea → come up with a new idea → do 5 more interviews …

Don’t confuse the user and the buyer

We spent a lot of time talking with our buyers: CHROs and Heads of Talent Acquisition. It felt good to have “big picture” conversations with senior executives. We knew they controlled the purse strings. And these developing relationships excited our team members and investors.

So we built a product for these buyers, not the recruiters who had to use it on a daily basis. And the product actually slowed our users down, and all the upside (the metrics we promised to improve) only mattered to their bosses. They were extremely unhappy. Ultimately, the CHROs, with whom we’d had so many “feel-good” conversations had to listen to their own people, who were begging them to stop using Ansaro.[4]

Give your (metrics) the benefit of the doubt

An MVP is meant to test the fundamental viability of a product, not whether a very rough v0.0 can achieve the metrics needed for a sustainable business. If the MVP works, metrics can be improved through product refinement. Because we didn’t know how much improvement could come from future product refinement, we worried about declaring a “false negative” on our MVP and thus avoided a precise metric goal.

A way to avoid this trap is to “give yourself the benefit of the doubt”: assume that product refinements can always triple the v0 KPI. Let’s put numbers on this. For Ansaro’s business model to be sustainable, we believed we needed to convert 10% of qualified leads (in-person sales pitches). After 100 pitches, we’d converted 2 to paying customers. So even with the benefit of the doubt (2 * 3 < 10), our business post-product refinements wasn’t going to be sustainable.

People Stuff

Accountability for Outputs, Not Inputs

When our CTO started, we failed to set clear near-term milestones. We gave him a big, nebulous project and a month to work on it. But without any interim milestones during that first month, we started focusing on facetime even though that ran totally contrary to our stated company values. This was enormously demotivating to our CTO, who turned out be incredibly hard-working — he just preferred working from home in the evenings (while my co-founder and I tended to stay later in the office).

An idea that seemed crazy to me then, but which resonates now, is limiting the team to a 40 hour work week during the MVP phase.[5] I estimate that of 100 MVPs, 90 will fail regardless of whether the team works 40 or 80 hour weeks; 8 will achieve Product-Market Fit (“PMF”) regardless of 40 or 80 hour weeks, and 2 will succeed at 80 hours but fail at 40 hours. If you work 40 hours and fail, you have to accept a 20% chance it’s a false negative (e.g. “if we’d worked more hours, we would have succeeded”). I think that’s a good trade-off because (a) it forces you to give up the fantasy you’ll achieve PMF if you just work a bit harder, and (b) you’ll have happier, more motivated team-mates.

Choose a mature tech stack

Why is this under people? Because choosing a stack should be based more on human considerations than pure technical considerations. We built mainly in Go, because it was lightning fast, massively concurrent, and had many other awesome technical attributes. But for every developer who is good at Go, there are 50 who are as good at Java or Python. When you look up a question on StackOverflow, you’ll get 3 answers for Go and 300 for Java or Python.

We never had the scale to realize the technical benefits of our stack. But the extra time required to hire people who knew our tech stack was significant.[6] So was the reduced velocity from developing with newer tools that lacked the community support of older tools.

Raise less money, later
We raised too much money, too early, which reduced our openness to changing paths. We didn’t set out to raise $3M; our target was $500K from only family and friends. But conversations with family and friends led to introductions to VCs. As investors flattered us, our confidence soared, and I started envisioning the awesome TechCrunch article about our seed round.

And then, one-by-one, those VCs told us they weren’t going to invest. Without ever explicitly discussing it, we’d moved our goal from “raise $500K from family and friends” to “raise many millions from blue-chip VCs”. We felt that new goal slipping away, and with that came a feeling of desperation. So when two VC firms relatively unknown to us offered us a term sheet, we quickly said yes. Big mistake. We never really understood these investors, and they never really understood us. It was like getting married right after your first date.

We should have stuck with our original plan. Raising a smaller amount of money would have forced a shorter timeline for testing our MVP (more iteration → higher likelihood of survival.) And raising from friends and family — who were investing in us as individuals, rather than in a specific business model or product — would have also enabled us to pivot more quickly.

I no longer believe in the saying “What doesn’t kill you, makes you stronger.” Just because something is hard, doesn’t make it worth doing. Starting Ansaro was hard. I learned a lot from it. But I could’ve learned just as much, or more, spending those years at a different wisely-chosen company, and avoided an incredibly high level of stress for myself and my family. I didn’t appreciate the learning opportunity cost to starting a company. Learning directly from the market is an intensely wicked environment, with highly uncertain learning returns (not to speak of financial returns). For me, I’ve realized I learn better at a growth-stage company, where I’m supported by others and have more access to data.

I was lucky to work with some truly great people at Ansaro, and for that I’m deeply thankful. One year out, I’m still not sure if I’ll try it again.

Footnotes

1. We were targeting large companies, which tend to use HR systems like IBM’s Kenexa, Oracle’s Taleo, or SAP SuccessFactors. These were nightmarish integrations. There are newer Applicant Tracking Systems like Greenhouse and Lever, which have good/decent APIs, but they are mainly used by smaller companies.
2. https://pdfs.semanticscholar.org/5c5d/33b858eaf38f6a14b3f042202f1f44e04326.pdf
3. I think it’s ok to do 10 or 20 or 50+ interviews to learn about an industry or problem space. My point is specific to limiting user testing interviews, in which the interviewer describes/introduces a potential solution to the interviewee and asks for their reaction. There is more risk of “gathering data until you get the data you want” with user testing interviews vs. general industry understanding interviews.
4. We also did a number of revenue-generating data science consulting projects that didn’t involve our main SaaS product. They were a distraction and a poor use of time; we failed to convert any consulting customers to SaaS customers.
5. The 40 hour argument may break down once you have Product Market Fit. At that point, there are or will soon be competitors, and the driver of success switches from having a fundamentally good idea to execution. And longer hours may be correlated with faster/better execution.
6. There’s an argument that a good engineer can succeed in any language, regardless of whether they know it when they take the job. I believe that. But we convinced ourselves at Ansaro that we didn’t have the luxury of a long ramp-time for new hires, so we would only hire people who already knew our stack. That was also probably a mistake.