Europe’s shot for Artificial General Intelligence 🇪🇺🤖 — Why we invested in Aleph Alpha
We are excited to have led the € 23 million Series A financing round of Heidelberg-based Aleph Alpha, joined by our friends at Lakestar, UVC, and existing investors LEA Partners, 468 Capital, and Cavalry Ventures. The exceptional team around AI-serial-entrepreneur Jonas Andrulis and co-founder Samuel Weinbach research, develop and operationalise a new generation of huge and powerful AI like GPT-3, DALL-E or MuZero to maintain European sovereignty for Artificial General Intelligence (AGI).
Andrew Ng, one of the leading AI experts stated in 2017: “AI is the new electricity” — but is it really? Today, narrow AI — models that are trained to perform one very specific task like chess or solving equations on or above human level — have automated the development of Covid-19 vaccines, autonomous driving, the creation of music, perfumes, and a lot more. Yes, AI has led to mind boggling results but to fully gauge its potential, I’d like to put things into context:
🙃 AGI will turn humanity upside down
If humanity would have existed an equivalent of 1 day (to represent 300k years) and electricity for 1 minute (to represent 200 years), then narrow AI would be around for about 5 seconds (representing less than two decades). Imagining AI’s potential impact in this context, leaves no doubt that in just two, three or even ten times this period, it will eventually be way more revolutionary than electricity was at the time.
Perspectives on AGI range from dystopian singularity to utopian scenarios. On the one end celebrities such as Elon Musk equate it to summoning demons, while on the latter end of the spectrum people see it a means through which humans and machines fully merge causing mankind to evolve beyond comprehension. Either way, it is clear that regardless of the specific scenario, AGI is considered one of the most transformative platform technologies mankind will have ever seen.
🔐 Increasing entry barriers for AGI
Unlocking AI’s full potential and eventually transitioning into AGI, however, is not an easy task, as it relies heavily on computing power and the size of the underlying model. Although representing only one piece of the AGI puzzle, the graph below, from one of the most pioneering papers on generalizable language models by OpenAI (2020), shows that the performance (=validation loss on left scale; the closer to zero, the better) scales along a power-law distribution driven by computing power (=compute on x-axis) and size of the model (=parameters in color code).
In a nutshell, this means that moving closer towards large generalizable models/AGI requires significant computing power/resources, comprehensive training datasets and the ability to overcome unique engineering challenges. Combined with the need for world-class AI talent, the entry barriers for state-of-the-art AI research drastically increase which limits the number of players that can ultimately dominate the game. Bryan Catanzaro, VP of Allied Deep Learning Research at NVIDIA, framed the risk-reward context nicely: “These models are so adaptable and flexible and their capabilities have been so correlated with scale we may actually see them providing several billions of dollars worth of value from a single model, so in the next five years, spending a billion in compute to train those could make sense”
Considering these significant entry barriers, we expect that very few players will be able to research, develop and deploy such models ultimately leading to a “de-democratization” and monopolization of AGI, cutting out universities, startups, SMEs and even the majority of enterprises. Unsurprisingly, the geopolitical implications and potential dependence on foreign AGI gate keepers brought a range of economic leaders into the arena (example here). Today, OpenAI (with GPT-3) is spearheading AGI developments under the US flag, whereas the Beijing Academy of AI (BAAI) aspires to be the Chinese equivalent with their new model Wu Dao 2.0. But what about Europe?
🇪🇺🤖 Aleph Alpha safeguards European AI sovereignty
After selling his previous AI company (Pallas Ludens) to Apple in 2016 and spending more than 3 years as Senior AI R&D Engineering Manager with the firm, Jonas Andrulis noticed the previously mentioned dynamics and set out to found Aleph Alpha together with his co-founder Samuel Weinbach in 2019. Together they had the clear vision to establish a European alternative to OpenAI and BAAI, and establish a globally leading AI-research institution with European values at its core. Two years later, the Heidelberg based team managed to attract top AI talent including key personas from Eleuther.AI, an open-source community that has build one of the most efficient but best-performing language models in the world (see here for recent performance benchmarks with GPT-3: “GPT-Neo (author’s note: which is Eleuther.AI’s model) outperformed the closest comparable GPT-3 model on all NLP reasoning benchmarks”.
🏎 More than performance
To go beyond performance and better understand how GPT-3 is used in the wild, we spoke to several developers building applications on top of OpenAI’s API. On the positive side, everyone praised the incredible performance. Yet, that’s nothing new. On the negative side, however, we learned that all developers needed to build very similar integrations and software components such as language tuning (think of childish language versus business language), prompt optimization (how to ping the API and get what you actually want) or human-in-the-loop systems to actually power their applications. Clearly, this sounds like a call for out-of-the-box components. Moreover, the developers asked for a mix of better load balancing, multi-language capability, unbiasing or multi-modality, among others. After these conversations, it was crystal clear that today is still day 0 and that the market just started to evolve.
Aleph Alpha noticed these unmet needs, including the call for additional components on top of their API early on. As a result, they decided to build not only an independent and open multi-language alternative to the closed US and Chinese offerings but add functionality that makes the integration, (ethical) alignment and innovation based on large models easier, more transparent and robust. Knowing that developers seek more than performance, Aleph Alpha decided to pick a select few light-house customers with a strong pull-dynamic and broadly relevant use cases to gradually educate the market, better understand customer needs and ultimately identify reusable components. Once productised, these components can then be distributed to a wide range of developers through a plug&play library on top of Aleph Alpha’s API.
As a simplified analogy, I’d like to compare the application of GPT-3-like models with vehicles where the engine represents the GPT-3 model and where the chassis represents the tools and integration layer on top of the API. Looking at OpenAI’s initial go-to-market, we find them selling an engine without visibility on the chassis. Changing perspectives and looking at chassis production, we find some vehicle manufacturers building a race car, others creating a Limousine, others creating a Smart, whereas others creating a tractor. From this analogy, we can learn two things:
1) Not every vehicle needs a V12 twelve cylinder engine. A fuel-efficient four cylinder engine might be more suitable for an everyday city car. Similarly, not all applications need models as powerful as GPT-3. In addition to performance, Aleph Alpha considers additional criteria like cost or efficient load balancing and plans to offer a variety of models meeting the diverse requirements.
2) While the chassis and possible applications of the engine differ, we find very similar components like steering wheels, doors, wheels or seats across all of them. To Aleph Alpha, it seemed obvious to sell the engine and offer pre-build parts to reduce complexity and thereby accelerate vehicle production/ time to market.
🕸 Gathering the threads
With a clear go-to-market strategy, a powerful multi-modal (text, image, etc.) model, a proprietary multi-language dataset and cutting edge prototypes for the tools layer, the Aleph Alpha team is now training their large-scale models to make them available via their API. For the sake of the resource intensive model training, the subsequent inferencing (meaning that trained models are hosted to perform specific tasks), the further growth of a world-class team and the ramp-up of collaborations with open-source communities, universities and other partners, Aleph Alpha just raised a heavily oversubscribed €23 million Series A financing round.
For us at Earlybird, it is an honour to be a part of Aleph Alpha’s mission to establish themselves as the leading AI institution eventually achieving AGI. We are even more thrilled to be backing a rockstar team that shares our common European values and ethical standards and actively chooses to integrate these into their technology and approach to the market. So while some may accuse AI companies for creating a dystopian AGI future, we are confident that Aleph Alpha’s team will catapult humanity into the positive spectrum of future scenarios — providing a real alternative for public and private companies.
Team Aleph Alpha, welcome to the #EBVCgang! We’re excited about the journey ahead.
> To follow Aleph Alpha’s development even closer, you can find their Github channel here
> Ranging from administrative positions to engineering experts, Aleph Alpha is looking for talent! To join the incredible team on their journey, please find their open positions here
> If want to learn more about GPT-3 models and AGI, I highly suggest this video
Are you a founder, industry expert, VC or researcher interested in the field of AI? I’d be more than happy to learn about your work, so feel free to reach out via andre@earlybird.com.
