Perspectives: What tooling will still be relevant in 6 months for enterprises building GenAI use cases?

What can we learn from the previous wave of AI and big data to help us anticipate which vendors are poised for success in the current market?

By: Julien Hobeika, Naza Metghalchi and Pierrick Cudonnec

The appeal of AI projects in the enterprise market is nothing new. It started almost a decade ago with the big data wave, when the growth of cloud, IoT, mobile and social, combined with the plummeting cost of storage, meant companies were sitting on a huge amount of data that they could leverage.

This data goldmine, coupled with the progress in ML algorithms, led to the creation of some of the most iconic businesses in the data and ML space, like Snowflake, Databricks and, in a second order of magnitude, companies like Dataiku, DBT, and C3.ai.

Now we’re seeing a similar trajectory with AI.

Interestingly, in the past, those that created the research discoveries didn’t necessarily win commercially — not directly at least. Case in point: Google laid the ground for big data technology with ‘The Unreasonable Effectiveness of Data’ and Dremel papers, and also led to the transformer architecture that now powers all LLMs with ‘Attention Is All You Need’.

With that in mind, how do you create long lasting value with this new technology, and build tooling that remains relevant? What can we learn from the previous wave of AI and big data to help us anticipate which vendors are poised for success in the current market?

Lessons from the big data wave

1. Data warehouses: owning the data, or a representation of it, can yield massive outcomes

Companies that store data are at a natural advantage due to switching costs. And it shows, with data warehouse players, like Snowflake and Databricks, accumulating $100bn+ in market cap.

Will this hold true for vector databases?

Most existing data warehouse players are now offering vector storage for unstructured data, but there’s still a huge amount of unstructured data that is not warehoused. This creates an opportunity for new entrants in the vector database space, like Pinecone and Vespa.ai.

However, there’s a greater risk of commoditization. Why? Because building a vector store is less computationally complex than previous data warehousing technology, leading to intense competition from incumbents.

2. End-to-end horizontal platforms gathered large customer spend

Enterprises spend more on platforms that help them build data projects end-to-end. Platforms like Databricks ($40bn+), Palantir (public, $35bn+), Dataiku ($4bn+), Alteryx ($4bn), C3.ai (public, $3bn+), and have seen their valuations soar.

These platforms won by abstracting away from the engineering and data layer, appealing to enterprises that lack the same access to the talent and organizational tech maturity found in tech companies.

Their success reveals a big opportunity in today’s market. The end-to-end platform’s key advantage in this rapidly evolving space will be its ability to help users navigate and exploit diverse and emerging approaches as they come to market, and the different tech stacks (whether LLM models or databases).

Those new approaches vary from data curation, embedding techniques and RAG techniques (e.g. generative RAG), to the use of small language models, and the size of context windows. The list is growing by the day and probably most of them will play a role.

Existing horizontal players will need to build or acquire these capabilities to keep up — M&A activities like MosaicML’s partnership with Databricks are always around the corner — but does that mean there’s no opportunity for a new horizontal player to emerge from this new wave?

As long as the previous AI paradigm based on structured data is relevant and the use cases enterprises are building are a combination of both AI paradigms, it’s gonna be challenging to build yet a new platform that covers both worlds.

But as enterprises mature their valuable use cases, they’re discovering some that would benefit from an unstructured data- and LLM-first approach. Chatbots are the obvious example. It’s an opportunity to create an end-to-end platform that focuses solely on use cases that don’t need the previous structured paradigm to deliver value — and that might become a long-lasting category alongside previous incumbents.

3. Modern data players have reached big valuations despite lagging revenue numbers (for now)

The previous wave introduced specialized, best-of-breed point solutions. Enterprises frequently opted to cherry-pick products from different categories like ELT (Fivetran, Airbyte), pipelining (DBT), orchestration (Airflow), observability (MonteCarlo, Sifflet), to tailor their tech stacks to specific needs.

Some argue that the VC renaissance as an asset class led to this explosion of new software categories and tools across the stack, and its dispersion. While these solutions may not generate the same level of revenue as the players in the end-to-end platform category, they can be similarly valued. Think Fivetran and DBT being valued at $5bn-ish each, similar to Dataiku or Alteryx while generating maybe ~5x less revenue.

The infamous MAD landscape (2023 edition)

Is there room for new $5bn+ players to create solutions that address the needs of the LLM paradigm which the current data stack can’t fulfill?

We think so. The advent of LLMs introduces new challenges and opportunities that the current stack hasn’t been designed to tackle.

Here are two examples we feel are interesting and under-represented in the number of companies created in the LLM space to date:

• Better control over leveraging unstructured data: Embedding all of your unstructured data in a vector database to do RAG or other training tasks has a lot of pitfalls when it comes to quality, relevance and compliance in production use cases. We’ve heard stories of banks applying LLMs to their mortgage scoring model and feeding them internal data, only to encounter policy “hallucinations” prohibiting loans to certain racial groups. We also believe that data building tools (DBT) will be of little help when it comes to “vector pipelining”. The specific embeddings you use, the number of vector spaces you embed into, and other factors will require additional tooling. This is an opportunity that companies like Deasie, Superlinked, and others are seizing.
• Model security: Because LLMs act like queryable databases, they introduce an all new field of security risks that didn’t exist in the previous stack, from data leakage to prompt injection. There is no serious security risk associated with predictive AI outputting a score, class, or value, but LLMs outputting words is like the results of an SQL query: you need to make sure those results are not leaking secrets. Lakera, Prompt Armor, and Giskard are examples of companies working in this space.

Enterprises are still iterating on and discovering all their valuable use cases. We confidently expect this list to drastically expand in the next six months. Stay tuned.

4. Models are not one size fits all. Enterprises want control over what they optimize for.

In the previous ML wave, no one created a commercially successful proprietary ML (even if Watson came close). Open-source ML trained by each enterprise on their own data became the norm.

Will history repeat itself and all companies end up training their own LLMs (as they did for predictive AI)?

OpenAI’s success leads us to think that history won’t repeat itself — they broke the historical status quo of “no proprietary AI model becoming widespread”. Predictive AI models were really easy and cheap to train or retrain, but today, unless training gets 10x cheaper, it’s much easier and cheaper to start experimenting with OpenAI or another pre-trained model.

If they don’t retrain from scratch, then they’ll use a pre-trained model. But who might win between open-weight models and proprietary?

For now, everyone looks at model benchmarks to evaluate quality. But general benchmarks have little to do with specific enterprise use cases performance. You might want to optimize for cost, speed or just quality, but with your own evaluation specific to a particular use case.

Will an enterprise get this level of optimization by building upon a slightly less performant, pre-trained, general open-weights model and fine tuning it on their own data, or by building upon the best proprietary models (for now, GPT4)? The story is currently unfolding in front of our eyes, and it might end up being very use case specific.

Current state of model performances (source: https://vectara.com/blog/hhem-flash-update-phi-2/)

We generally wouldn’t expect new market leaders to emerge in this category: it’s increasingly evident that enterprises display little loyalty and face minimal switching costs with these models. Their infrastructure is designed to seamlessly transition their endpoints to any model — be it open-source or proprietary — based on factors like use case, cost, and performance. The risk of commoditization is real, so you might want to avoid competing on that layer.

Final words: be future-proofed ahead of AI research breakthroughs

It’s hard to predict where models’ capabilities will be in six months. Some are underestimating — and others overestimating — future improvements. But one thing is sure: when building in the space, focus on challenges beyond current model performance.

Looking back at history, to maintain relevance and not become obsolete, you will need to create a tool that acts as a common building block for various AI use cases. Success will lie in continued dependence on the tool, no matter the approach, and more importantly, no matter the efficiency jumps made in AI research.

If you are a founder building in this space with this paradigm in mind, feel free to reach out at julien.hobeika@eqtventures.com