Five Questions to Help You Sort Through the AI Hype
AI has become the hot new buzzword, generically applied to any tech company trying to sound relevant or impress venture capitalists. Since the Future Labs began supporting artificial intelligence (AI) startups, we’ve been approached by hundreds of “AI” companies that don’t seem to have any AI at all.
While we’ve been through buzzword cycles before, the current AI hype presents a much tougher due diligence problem for investors and others trying to make sense of the space. Even for the most tech savvy individuals, the MIT Technology Review recently published a report about AI explaining that, “No one really knows how the most advanced algorithms do what they do.” If the computer scientists aren’t quite sure how advanced AI works, where does that leave the nontechnical angel, VC, or corporate innovation officer trying to understand what’s real and what’s fake and decide when to invest?
The good news is, for those looking for a baseline understanding of AI, you don’t need to know the specific intricacies of the technology to sort through most of the fakes. Here are five questions that will help you weed out 90 percent of the fake AI companies. (Disclaimer: Before making an investment, you should always have a technical expert perform due diligence on the company. This is something we do with applicants to our AI program.)
1. How is the tech being built?
This is a complicated question but the most important one. Is the tech built in-house, outsourced, or does it use off-the-shelf open source AI (wit.ai, or api.ai)? As an investor, customer, or potential employee, this is something you should know. It will determine where the company is headed, what problems will arise at scale, and what roles people will have internally.
Companies using off-the-shelf AI solutions versus those who have developed proprietary technology are really more of a service provider and unlikely to grow to scale. Think of them as a shop that customizes cars from the original manufacturers. From an investment standpoint, you want the company building the cars or, in this case the AI, not the company slapping on a custom paint job on something someone else built. While the custom shops can still build a business, like is the case with Zeno’s paradox, they will never catch up with the OEM.
The best case is in-house experts building the core technology in house. That’s expensive and hard to find, so understanding the problem, solution, market, value prop, and competitive landscape becomes infinitely more important.
2. Is the founding team technically proficient?
Hammering on the theme of technology as a core competence, ask who the founders are and the background of the startup’s earliest employees. Given the fairly niche nature of AI, top-tier AI researchers and practitioners want to work on AI problems alongside people they respect. Unlike more mature areas of technology, AI engineering talent is even more difficult to attract, and AI experts have their pick of where to work. The technical strength of the founding team will largely determine their success in recruiting additional AI talent as the company grows.
AI startups needs AI talent, which is in high demand, so AI experts choose where they work and are drawn to interesting AI problems and AI leaders.
3. Where’s the data?
For AI to work, it needs to be trained, which requires data — often an enormous amount of data. Furthermore, that data needs to be structured to make sense. And for the AI to learn accurately, the data should come from the real world, not from made-up sample sets. ImageNet is a good example of this. It serves a great purpose and is amazing for training data, but you also need real world images relevant to the problem you’re solving if you want a low error rate. Clarifai, a Future Labs alumnus, solved this problem in its early days by helping users with a big pain point, sorting their photos and applying labels to their own data, all to ensure Clarifai is the best at recognizing what’s in an image or video.
If someone pitches you an AI company but can’t tell you what kind of data they’re using to train the algorithm or where that data comes from, even if their AI engine is real, they’re essentially pitching you an amnesiatic brain. It might be able to “think,” but it has nothing to process and is a blank and useless slate.
GPs need LPs to invest, while algorithms need real world training data and lots of it to work in the real world. If the data is limited, there will either be a high error rate or the the computing power will be very expensive via reinforced learning.
4. Is this really an AI problem?
Can the task be accomplished easier with another technology? AI is so buzzworthy right now that everyone wants to apply it to everything, even when it’s completely unnecessary and creates unneeded complexity. AI is hard and should be used to solve difficult and real problems. Don’t try to use AI to fix something that isn’t broken or to solve a problem that doesn’t really exist. In a nutshell, don’t be the Juicero of AI.
If it doesn’t need AI, it shouldn’t be AI. To be an AI company, the business has to solve a big enough problem to get access to data that can be labeled and used for training.
5. Does it sound too good to be true?
Rocket AI launched at the NIPS conference last year with great fanfare. Their applied AI subtext was one thing, but their patent pending “Temporally Recurrent Optimal Learning” was a new buzzword most investors had yet to hear. Was it the next iteration of neural nets? Was it AGI? No, it was fake, a fantastic joke by Riva Melissa-Tez of Permutation Ventures, and everyone fell for it.
Fancy buzzwords do not make something real AI, and they certainly don’t make something worthy of investment. If it sounds too good to be true, it probably is.
Don’t fall in love with AI. If you haven’t heard the buzzwords, it’s not because it’s a new buzzword you don’t know yet. It’s most likely fake.
In my next post in this series, I’ll discuss questions to ask if you’re a CxO and you’re being approached by an AI company for a pilot.
Follow the Future Labs for more tips on navigating the startup landscape and how not to get fooled by the AI hype.
Steve Kuyan
Managing Director, Future Labs at NYU Tandon
Thank you to the Future Labs portfolio companies and the Future Labs team for their suggestions, edits, and most importantly condensing this down to an easily digestible length.
