The MMC Ventures AI Investment Framework: 17 success factors for the age of AI

Artificial intelligence — specifically, machine learning (ML) — is a powerful ‘enabling technology’ that represents a paradigm shift in software capability. (New to AI? Read our Primer and UK Market Map). But how do we, as investors, evaluate early stage software companies that put ML at the heart of their value proposition? Below, we introduce our ML Investment Framework.

Our Framework captures 17 success factors for early stage ML companies. Because sizeable returns stem from a company’s potential for value creation, effective value realisation, and defensibility, we group the success factors into these three categories. Using an alternative lens, the 17 factors span six competencies: strategy, technology, data, people, execution and capital. Informing, but not dictating, our discussions with ML companies, our Framework also provides a blueprint for supporting the ML companies in which we invest.

Interested in more venture capital insights on AI? Sign up for our blog posts.
AI entrepreneur? Get in touch.

The MMC Ventures Investment Framework for applied ML companies

Our Framework is focused on ‘applied’ ML companies — the 85% of startups developing ML-led solutions for specific sectors or functions. Success factors for developers of ‘horizontal’ ML technologies differ.

In addition to the usual factors we consider when meeting a startup, when evaluating ML companies there are additional factors we consider, and some traditional considerations on which we place more emphasis. Additional factors, such as the suitability of ML to solve a problem, and the scope for network effects through data, reflect particular characteristics of ML. Traditional points on which we place greater emphasis, such as quantifiability of ROI and commerciality of management teams, reflect the dynamics of the ML market we have observed in meetings with 90 UK ML startups. No company will be strong in all areas, and success factors differ in their relative importance.

1. Value creation

Six success factors for value creation in early stage applied AI companies.

To what extent does an ML company have the potential to create significant value? We consider six drivers of value creation. Three (value release, scope for disruption, and feasibility of alternatives) relate to business system impact. Three (suitability of ML, a path to better-than-human performance, and suitable data sets) are questions of technical feasibility.

I. Value release

How effectively, through predictive success or process automation, can a company unlock value in a business system by creating revenue, or cost savings, for customers? We assess a provider’s ability to create revenue for customers by:

• increasing uplift in conversion, yield, throughput, price or similar direct drivers of revenue;
• reducing churn by improving customer experience — through greater personalisation, better customer service, lower customer friction or enhanced brand loyalty; or
• creating new revenue opportunities — by identifying new customers, increasing up-sell or cross-sell opportunities, or enabling new market opportunities.

We also consider a provider’s potential to decrease costs for buyers by:

• reducing surplus spend, excess resourcing or core resource requirements through improved predictive efficiency, process efficiency or process automation; or
• reducing economic leakage through, for example, improved compliance.

II. Scope for disruption

Beyond its immediate impact, we consider whether an ML-led company has scope to disrupt (enable new categories of users to utilise a service) as well as optimise (streamline a process for existing users).

Consider AI-driven personal assistants. While AI assistants can undertake only a small range of tasks relative to their human counterparts, they can automate the scheduling of meetings for business users. With the salary of a human personal assistant averaging £25,000 per year, according to PayScale, personal assistants are unaffordable for many small and medium-sized businesses. AI assistants will make capabilities accessible to businesses of all sizes, creating value by expanding the addressable market.

Few businesses disrupt, and a business need not disrupt to be attractive. But businesses that disrupt may create outsized outcomes through scale.

III. Unattractive alternatives

Scope for value creation can be greater when the cost or availability of alternatives are prohibitive. In some attractive cases, there are no practical alternatives because ML makes possible the previously impossible. In most cases, alternatives can be found — with sufficient investment in human or other resources. Where alternatives to a company’s solution are particularly costly, scarce, inaccessible or non-scalable, scope for value creation is especially significant.

Human labour is frequently the direct, and most expensive, alternative to digitisation. It’s striking that three of the top four sectors on which most UK ML startups are focused — finance, IT and utilities — are those with the highest annual salaries. We see additional opportunities in Professional Services.

IV. Suitability of ML

To what extent is ML well suited to the business challenge at hand? ML is well suited to problems that are arduous, complex or inscrutable:

• arduous problems are those in which people are competent, and could codify a solution into a program, but it would be impractical to do so.
• for complex problems, people are competent but codifying that capability into a program is prohibitively difficult. Object recognition is a complex problem. People are very good at recognising pictures of cars, but we can’t codify an effective set of rules to do so.
• inscrutable problems are those in which people do not have competence. In these fields, we cannot label or organise data to underpin a predictive engine. Deep learning approaches to ML, with utilise neural networks, excel at inscrutable problems because neural networks can determine the parameters to optimise.

ML is poorly suited to unbounded problems and questions of causal inference.

• ML algorithms cannot draw on knowledge beyond the data provided to them. Anastassia Fedyk has highlighted the difficulty vividly with an example from the 1990s, when researchers at the University of Pittsburg evaluated ML algorithms for predicting mortality rates from pneumonia. “The algorithms recommended that hospitals send home pneumonia patients who were also asthma sufferers, estimating their risk of death from pneumonia to be lower. It turned out that the dataset that fed into the algorithms did not account for the fact that asthma sufferers had been immediately sent to intensive care, and had fared better only due to the additional attention.” For ML to be effective, problems need to be sufficiently self-contained.
• Second, ML is poorly suited to problems where causal inference is of primary interest. ML describes how elements of data relate to one another other, not the causal mechanisms of their relationship. ML is poorly suited to prediction problems when the future is not expected to be similar to the past, and where prior patterns are unlikely to reflect a new reality.

V. A path to performance

ML does not need to be 100% effective to be valuable. From a practical perspective, ML-led solutions need only offer near-human, or ideally better-than-human, levels of performance to enable automation and process scaling. When assessing the performance of ML-powered technologies, therefore, we look beyond the immediate term to assess whether there is a path to a level of performance — ideally better-than-human performance — to unlock value.

Human levels of performance can be lower than assumed. 94% of car accidents in the US are due to human error, according to data from the National Highway Traffic Safety Administration. Autonomous vehicles do not need to be 100% safe to be valuable; they need only offer a safety rate similar to, or better than, the 1.25 deaths per 100 million vehicle miles that human drivers in the US achieve. (In practice, of course, buyers’ trust in a technology is a further condition for its adoption — and the bar for acceptance will be higher in certain areas, including autonomous travel. We discuss this below).

VI: Suitable data

For ML to create value, it needs suitable data sets on which to be trained and deployed. We evaluate the extent to which a company can access suitable data. We gauge data suitability in the context of two stages of data manipulation required for ML:

• selection: data availability; the existence of gaps and duplication in data; quality of data labelling, existence of bias in data;
• processing: data fragmentation; data cleaning requirements; a need for data sampling; the need for data transformation, decomposition and aggregation.

We also gauge whether data sets will retain value. Data sets retain value if numerous new iterations of an algorithm can be tested, and improved, using historic data. This isn’t always the case. If a chatbot company improves its algorithm, the prompt it offers a user will differ from the prompt it will have offered in the past. If the chatbot’s prompt would differ, the response that follows from the user probably would too. With a vast set of user replies now decoupled, and potentially irrelevant, to the algorithm in question, the company’s historic data is of limited use. We can contrast this with a fraud prevention company. An algorithm can usefully be tested against historic customer activity data sets to see if the accuracy of the algorithm is improved.

2. Value realisation

Five success factors for value realisation in early stage applied AI companies.

Attractive companies have the potential to create value. But can that value be realised? In our experience, five factors are significant for ML-led companies. The first (management commerciality) relates to talent. The following three (a quantifiable ROI, buyer readiness and benign regulation) relate to business system adoption. The final consideration (deployment scalability) relates to go-to-market strategy.

I. Management Commerciality

Many founders of ML-led businesses have superb technical expertise. Commercial acumen, however, will play a great or greater role in the long-term success of their businesses. Sooner or later, most B2B software companies will need to build a direct sales team — the business won’t scale with the founder knocking on doors. Founders with a commercial mindset have a desire to build a big business, an urgency to go-to-market, and the ability to build strong sales teams. Investors can assist — my colleagues Jon Coker and Simon Menashy have great experience helping founders build and scale SaaS sales teams — but they can’t instil commercial drive where it is limited.

II. A Quantifiable ROI

Solutions with a quantifiable ROI usually enjoy greater adoption, shorter sales cycles, and less customer education in B2B markets. In the sales & marketing function, improvements in sales conversion can easily be assessed. In the finance vertical, increased trading profits are apparent. In the UK, most entrepreneurs are focusing their efforts in areas with a demonstrable ROI:

• More ML startups — 20% of the UK total — service the marketing & advertising function than any other; and
• The sector with most entrepreneurial activity is Finance (8% of early stage ML companies).

III. Buyer Readiness

Buyer readiness is usually assessed as a funnel: awareness, knowledge, liking, conviction and purchase. For ML-led companies:

• we add preparedness as a stage in the funnel. Preparedness assesses whether buyers possess suitable, accessible data sets for training and deployment, and the organisational buy-in to augment or disrupt existing workflows with ML-led solutions.
• within ‘liking’, we consider issues of trust and control. Trust is the ability to have confidence in the performance of the solution with limited human intervention. The bar for acceptance varies; travel in autonomous vehicles, and ML-led medical diagnosis, will require high levels of trust. Control describes the extent to which there is desire for human involvement in a system or process even if trust in the system is high. Value release, adoption or scalability may be limited in situations where high levels of human control need to be maintained.

IV. Benign Regulation

‘Deep learning’ approaches to ML, which use artificial neural networks, are often ‘black box’ in nature. “It’s not always clear what happens inside — you let the network organise itself, but that really means it does organise itself: it doesn’t necessarily tell you how it did it” (Nils Lenke, Nuance).

Accordingly, we consider whether an ML provider will face regulatory challenge regarding transparency. Is there a requirement to understand and explain the prediction or decision provided by an ML solution?

Entering a mortgage contract in the UK is regulated under the Financial Services and Markets Act (2000). Within The Mortgage Conduct of Business Rulebook, rule 11.6.2 states that a firm “must not enter into the transaction unless it can demonstrate that the…contract..is affordable for the customer.” If the lender has relied upon a black box deep learning algorithm to determine affordability, can it do so?

Regulatory risk can be overstated. It has been widely reported that the European Union’s new General Data Protection Regulation (GDPR), scheduled to become law across the EU in 2018, creates a ‘right to explanation’, whereby a user could ask for an explanation of an algorithmic decision made about them. The reality is less clear — and may imply more a ‘right to be informed’ regarding the general process for algorithmic decision-making and data sets involved. Nonetheless, the direction of policy-making is towards greater considerations of transparency and potential bias in systems. In the United States, in 2016 the White House Office of Science and Technology issued its “Preparing for the future of AI” report. “Researchers,” it concluded, “must learn how to design these systems so that their actions and decision-making are transparent and easily interpretable by humans.”

In certain business-to-business functions — sales, marketing, and business intelligence — explainability is unlikely to be a challenge. In others — including human resources, compliance and fraud — it may be, given legal or pragmatic considerations. Similarly, companies operating in certain sectors (financial services) will have greater need to deliver compliance than others. In evaluating ML companies, we will seek to understand the extent to which they may face regulatory headwinds, either now or in the future, and how they intend to respond.

V. Deployment scalability

The pace at which ML-led software companies scale can be inhibited by difficult deployment dynamics.

• Data integration requirements can be extensive. Amalgamating, integrating and cleansing disparate customer data sets, which often live in siloed depositories, can limit time to value.
• Resource requirements from the software company can be substantial, limiting potential for new customer acquisition and margins. Many of the ML-led company we speak with have one third of their teams involved in deployment. As one told us, given the personnel required for each customer “we couldn’t deal with more sales even if we had them.”

While deep customer relationships increase customer stickiness and up-sell, ML companies that minimise deployment requirements, or automate the data collection, data harmonisation and deployment involved, can scale faster.

3. Defensibility

Six success factors for defensibility in early stage applied AI companies.

How effectively can an ML-led company defend the value it creates over time against competitors? We see six keys to defensibility: distance from industry monoliths; domain complexity and associated expertise; the creation of a network effect through data; proprietary algorithms; the ability to attract high quality ML talent; and the use of capital as a weapon.

I. Distance from monoliths

Google, Amazon, IBM and Microsoft (GAIM) all offer cloud-based ML services in areas including generalised computer vision, speech and text processing. The capability and scope of these services will continually increase. Recently, Google extended its computer vision capabilities into video, releasing its video intelligence API which offers entity recognition, searching and cataloguing of video. Given, in particular, Google’s ownership of Deep Mind — effectively, Google’s ML research department — cutting edge technologies will filter down into broadly accessible services over time.

The performance and low cost of GAIM’s general ML services will commoditise all but the most sophisticated ‘best-of-breed’ competitors in equivalent areas.

Accordingly, ML companies with offerings in areas distant from GAIM’s core competencies will enjoy greater defensibility. Distance comes primarily through vertical focus, and secondarily through technical development in areas beyond generalised computer vision and language. In practice, this means function-specific or vertical-specific solutions remote from GAIM’s generic, horizontal solutions. At present, GAIM’s vertical ambitions are limited primarily to healthcare (Google, Microsoft and IBM) and transportation (Google) — although some will experiment with more verticalised offerings in the future, in sectors relevant to their data sets and business models.

Early stage ML companies have the opportunity to enhance, and then reinvent, myriad processes in sectors ranging from manufacturing through to law and agriculture. In these areas, GAIM lack the desire, data advantage and domain expertise to compete.

II. Domain complexity

A vertical or functional focus minimises competition from GAIM. But the dynamics of a company’s chosen domain can further broaden or narrow the ‘moat’ around the business.

• Complex domains include those which require extensive industry expertise, have elaborate regulation, or have particularly complex technical challenges such as extensive or fluid variables.

Tackling a complex domain places a greater burden on a business — the path across the moat is narrower. But those who can successfully reach the other side will enjoy a position of greater defensibility. The most attractive businesses we see operate in domains with a degree of complexity, and have the capability to deliver.

III. A network effect through data

Attractive ML businesses create network effects through their data, to develop lasting competitive advantage.

Companies with access to private, domain-specific data sets have unique training materials to improve their machine learning algorithms at the expense of competitors. A network effect develops: the more customers secured, the better the company’s product — and the more customers and proprietary data it can secure. A company powering fraud detection in the financial services industry will gain access to new, non-public data with every customer they acquire. In contrast, a company leveraging only public data, such as web data, can’t develop the same moat. It may achieve a competitive advantage by being a first mover or scaling faster, but ultimately its algorithms are replicable.

Notably, an ML-led company need only access and utilise — not own — a company’s private data to train their algorithms from it. Temporary access to data owned by incumbents is sufficient to neutralise much of the incumbent’s own data advantage.

The potential for network effects through access to private data has second-order consequences. Early stage ML companies may, sensibly, prioritise data access over short-term revenue. They may seed software without charge, or accept reduced revenue from initial customers, given the value of early customers’ data.

IV. Proprietary algorithms

While ever-better algorithms are available in open source libraries such as TensorFlow, valuable ML companies create intellectual property by developing enhanced, proprietary algorithms. A proprietary algorithm (in practice, often an ‘ensemble’ of various algorithms) may offer: greater accuracy; broader functionality; faster performance; lower fragility; greater explainability; or results from a smaller training data set.

Innovation comes by degree, from ‘know-how’ to novelty. ‘Know-how’ is the skilful implementation of existing algorithms to deliver improved results (fewer errors, faster performance). Novelty involves devising new approaches to problems and deploying them successfully.

The value of proprietary algorithms can be understated (‘all the value is in the data’). In many areas, including natural language processing, lack of data is no longer the bottleneck. In other areas, innovation around algorithms can enable, for example, performance from smaller data sets. More broadly: if access to data were all that mattered, demand for skilled ML talent would not be so high.

V. Machine learning talent

ML talent is scarce and expensive. In the UK, the number of open positions for even general data scientists grew 32% year-on-year during the first half of 2016, according to a study by Procorre, outstripping growth in supply. Further, among developers in the UK, ML specialists command the highest salaries:

Given fierce competition for talent, compelling companies must demonstrate an ability to attract and retain high quality ML personnel at acceptable cost.

Startups primarily complete with Google, Amazon, IBM and Microsoft (GAIM) for top ML talent. Startups cannot, and need not, compete with the scale, security and pay offered by GAIM. Effective startups emphasise to recruits their opportunity to impact product directly, the greater autonomy they can enjoy, the speed at which the company can iterate, recruits’ greater freedom to publish their learnings, the intellectual and technical challenges available, and the greater longer-term financial rewards available if the company is successful.

VI. Strong capitalisation

ML companies can have greater capital requirements given: the greater time required to develop a minimally viable product in this technically demanding field; long sales cycles associated with being a business-to-business company; the greater cost of machine learning talent relative to typical developers; and the requirement for extensive deployment resources, which increases personnel requirements and inhibits speed of scaling.

Effective ML companies respond by using adequate capitalisation as a weapon to strengthen competitive advantage. While tackling inefficiencies — for example, by automating as much of the deployment process as possible — strategic founders suitably capitalise their companies to withstand the journey to monetisation, offer competitive salaries to attract high quality machine learning talent, and maximise their pace of customer acquisition to secure access to non-public data sets.

We’re investing in the future

Early stage ML-led software companies have exciting potential to transform the corporate ecosystem and accrue value in the decade ahead. Enabling customers with foresight, and disrupting those without, they will be among the beneficiaries of the paradigm shift to ML.

Of the nearly 300 angel, seed, early stage and growth stage ML-led software providers in the UK, some will emerge as future industry leaders. From early in their lives, those that do will demonstrate several of the 17 success factors for effective value creation, value realisation and defensibility.

At MMC Ventures, ML is a core area of research, conviction and investment. If you’re an ambitious entrepreneur with some of the success factors we’re looking for, get in touch to see if we can accelerate your journey.

Interested in more venture capital insights on AI? Sign up for our blog posts.
AI entrepreneur? Get in touch.