If you’re running a B2B startup with a high priced product (say >$100K Annual Contract Value) then “pilots” / “trials” / “Proof of Concept” are an essential stage in your sales pipeline. If you’re at an early stage and your product is a component in your customers’, signing these PoCs is often your stated goal and how you validate progress — Putting your 3D camera in Microsoft’s next gaming console, your LIDAR in BMW’s next car or your encryption algorithm in Cisco’s next routing product is pretty much your objective for the next 2 years, and the PoC is the all-important stepping-stone. Investors are asking how many PoCs you will sign this year and with whom. You’re judging your business development or sales on the number of pilots, and setting “Deliver A Successful Pilot” as engineering’s OKR for the year.
…but sometimes you get that gut feeling that something may be fishy.
Pilots are a necessary evil.
Whether your product is a “tool” or a “component”, if it’s not yet market-proven, the customer is taking a significant risk. It ranges from wasting employee time and jeopardizing internal processes because a faulty tool was chosen, to putting a future product at risk and impacting future sales due to a faulty component. Even if your product was successful with another company, there is always a risk that the integration with the new customer’s product or process will not be. Therefor PoCs are justified stage in the procurement process and your opportunity to get alignment with the customer.
Unfortunately, they are also an opportunity for mismanagement and abuse.
Corporate Innovation and Flawed Incentives
The bible for modern economic thinking begins roughly this way:
In the beginning, god created the customers and the corporations. And god said “Let there be big skyscrapers with office workers, and big production facilities, and they shall produce goods for the customers”, and it was so. But the goods became stale, and the big corporations slow to innovate, and the customers were upset. Then god said “Let there be startups, with small teams in garages backed by venture money, that can ideate and innovate and fail fast, such that the corporations can adopt their innovations and pick the winners and incorporate their technology to keep their products fresh; and cull the losers sending them back to the chaos of creative destruction, so they can renew and reincarnate in a new startup.
And while this model generally works, even the god of economy is subject to the law of unintended consequences. These, in turn, lead to situations where a misalignment of incentives creates a conflict with your startup’s interests.
The 7 Pilots That Will Land You In Pilot Purgatory
The Innovation Pilot
Corporate Innovation has emerged as a “must-have” function for the 21st century corporation that wants to impress its shareholders. Many have an Innovation organization, run by a VP or SVP with dedicated staff and budget. Too often, these people are compensated by the number of projects (or really pilots) that they deliver, as opposed to the ones that actually make it into their companies’ products or processes. You will deliver the technology, the PoC may be successful and even impressive, but no one in the company intends to adopt your product at scale, at least not in the short term.
Dead giveaway: you’re dealing with people whose titles say “innovation”, the actual product people are not involved or not very engaged.
The Pet Project
Everyone likes playing with shiny new toys, whether or not they have the authority. So sure, they will try out your product, they will run a pilot with you, they may even get a bonus or a promotion at the end of the year. But they don’t have the authority to get your technology adopted, and nothing will happen post-pilot. You are working with someone who doesn’t really have the authority to buy your product, even if it works.
Dead giveaway: they can’t pay you a meaningful sum for the pilot (rule of thumb: $50K) and can’t tell you what the post-pilot plan is.
The Vanity Project
Think about Concept Cars. A car manufacturer will sink millions into building something just to show it a press event — with no intention to produce it any time soon. Big brands in different verticals do this all the time — and often by working with startups. The brand gets some brand value, market feedback and potentially some inspiration for its employees. But if you’re the startup in question, and you were assuming this was a stepping stone towards a scalable purchase — you may be left in the lurch.
Dead giveaway: Marketing people (rather than Product people) often drive vanity projects.
The Russian Doll
In industries with long product life-cycles or in highly regulated industries, the customer may require multiple phases of the PoC, for instance a pilot that just checks that your technology doesn’t break the current system, then one that checks that it doesn’t do worse than known solutions, and only after multiple stages you get to a pilot that actually proves the efficacy of your technology. You may find yourself wandering in pilot hell for years before you know if you actually have a sale.
Dead giveaway: the initial PoC doesn’t really demonstrate the value of your technology.
This is a nasty form of an Innovation Pilot, where the innovation people are trying to shove your product down the product / technology organization’s throat, but the latter are just not interested. Sometimes it’s a cultural issue. Sometimes it’s a personal one, and sometimes there are really good reasons — e.g. your technology works, but it will be too hard to plant it into an existing technology stack where tight integration between the layers requires close working relationships — closer than what can be expected from external vendors. You end up demonstrating the technology, but you can’t break into the tech stack, unless maybe through an acquisition.
Dead giveaway: innovation people are enthusiastic, but tech / product are skeptical. There is no post-pilot plan.
The Job Interview
You’re there to prove your product works, but what the customer is really interested is in your team’s skill-set. They’re not there to buy your product, they need a solution or access to a skill-set. Best case outcome — they will hire you as a professional services vendor, worst-case they will try to poach your team. The PoC is interesting for them as a demonstrator of what you know, not what you’ve built. The biggest catch: entrepreneurs’ egos are over-developed (proof: they think they can change the world). Therefore you and your co-founders may be feeling especially good about this pilot until it’s too late.
Dead Giveaway: There is a lot more interest in what you (as a team) know and what you can do, then what was already delivered, and maybe a little too much interest in your team members.
This may be the nastiest of the lot. What the “customer” is doing is really learning all they can about your product and trying to reverse-engineer it so that they can re-build it themselves. Sometimes it’s organizational culture (some big companies and whole nations are suspect). Sometimes it’s a particular individual. But the gist of it is they are highly unlikely to buy from you — what they really want is to learn how you do what you do.
Dead giveaway: Engineers are highly involved, ask a lot of “how” questions, but tell you very little about their product / tech roadmap.
A Brief Discussion of Ethics
Many a CEO said “I trust them, they are for real” before the pilot, and “how could they do that to me” in hindsight. But growing up, hopefully you’ve learned to recognize that life is not black-and-white. People are not necessarily “out to get you”. But they still might. Two factors affecting customer’s behavior here are ignorance and cognitive dissonance.
Someone who spent his entire career in a big corporation has a very different sense of urgency than a startup CRO. For them, wasting a year on a “failed” pilot is next to nothing. In fact what you see as failure they see as a necessary step. “Our company has deep pockets, we all learned something from that pilot, it was a valuable lesson and next year will do better.” The fact that this wasted year meant the end of the runway for your startup didn’t even cross their mind. As far as they’re concerned, if you decided to do it, you probably had a good reason to and “cheap VC money” to carry you forward.
This is where this crosses over to cognitive dissonance, allowing them to embark on a pilot even if they know that the likelihood of it turning into a relationship is low, with justifications ranging from “there is a chance that we will buy their product if it is really as great as they say it is — it may be small, but it’s not zero” to “if they are willing to take the chance, there’s probably some big investors backing them on it”.
Bottom line is — they don’t really feel an ethical dilemma.
So no, it’s not about the big, evil Goliath corporations vs. the small David startups. It is about human nature as it manifests itself in behavioral economics.
So What is a Startup CEO To Do?
Hopefully , The “dead giveaways” listed above will help you diagnose the situation and calibrate your expectations from your PoCs.
For a more systematic approach to measuring risk / reward, choosing the right relationships and learning when to say “no”, not chasing every shining object and negotiating win-wins, read the next post — “The CEOs Guide to Growing Your Company Through Pilots and PoCs”.
Nadav Gur is a serial entrepreneur who founded, ran and exited several companies in the AI, automotive and travel space. He is the principal at NG Vanguard Enterprises where he works with founding teams and investors to make them 10% better.