The Future of AI Computing: Introducing SEMRON

Productivity increase has been the core KPI to measure human progress for hundreds of years. But only recently have we seen a technology that brought about an absolute quantum leap in productivity: Deep Learning (DL) and Large Language Models (LLMs). Never before has any technology unlocked productivity gains of up to 70% across almost all professions… and all of that happened with a never-seen-before speed of technology adoption: 100m users of Chat GPT in only two months. It is hard to overstate the importance of this!

Computing Infrastructure Is “the New Oil”

One fundamental limitation of Deep Learning and LLMs is that modern computers are highly inefficient in performing these tasks, and our quick fix has been somewhat of a brute-force approach, throwing massive amounts of computing resources at the problem. It becomes increasingly evident that we are running into an enormous computing bottleneck that will generate a massive divide in global competitiveness. GPUs and general computing resources are already booked out for years to come, and big tech is investing dozens of billions every year to somehow keep up with exploding computing demand. Still, demand will outperform supply by far, and access to computing resources will become increasingly scarce, hugely affecting the productivity of businesses, individuals, and even entire nation-states.

Current Computing Approaches Are Not Sufficient

To make things worse, Moore’s law is reaching the limits of physics and further improvements of existing chips are increasingly costly. Even if we could double the number of transistors on a given footprint every year, we would still run into huge energy efficiency problems: AI computations require vast matrix multiplications of billions of parameters. Existing infrastructure solves these (very inefficiently) by chopping up the matrices into small junks and constantly moving data back and forth between the CPU and memory. This inefficiency is particularly problematic for LLMs, which require massive amounts of data processing. ChatGPT has recently been estimated to soon consume more energy than countries like Sweden, Norway or Argentina.

Hence, nothing short of a new computing paradigm is required to avoid humanity running into a dead end, where increasing conflict might emerge over access to computing infrastructure.

Developing a Novel Chip Is Like Playing 5-Dimensional Chess

Alternative computing concepts to solve the AI computing challenge have been around for years (e.g., neuromorphic or in-memory computing), and dozens of well-funded startups (Graphcore and Mythic, just to name a few…) have promised an efficiency panacea. However, so far, most of them have fallen massively short, and billions of dollars had to be written off in the quest to build the world’s next NVIDIA. Why?

The answer to this is complex, but in short, it can be said that the semiconductor industry is highly complex and hard to break into. Here are a few of the reasons why past and present ventures have struggled:

1. Interdependent Technical KPIs: Improving both computing performance and energy efficiency at the same time has proven to be a highly delicate balance. Further, adding a small desired footprint (for edge applications), high parameter density (for large modern LLMs), and acceptable costs, makes it an almost impossible equation to solve.
2. Manufacturing: Hundreds of billions have been invested into existing semiconductor manufacturing facilities and processes worldwide. These behave like ocean tankers that are hard to move. Hence, it is essential for a new semiconductor approach to fit seamlessly into the existing manufacturing landscape, or else upfront R&D expenditure would skyrocket. This has been a key hurdle for many novel computing companies out there.
3. Software Ecosystem & Integration: Lastly, innovative hardware is great, but the requirements for a state-of-the-art software ecosystem have often been underestimated (e.g., design tools, compilers, simulation suites, etc). Existing players (NVIDIA, Intel, ARM, etc.) have built extremely strong moats here. Additionally, chips never operate in isolation and tight integration with other chip components and network interfaces has been an additional hurdle.

SEMRON: A Paradigm Shift in AI Computing

Now the good news: SEMRON, a German DeepTech startup based in Dresden, has found an extremely compelling solution to the above challenges. They developed a novel and proprietary computing device that has the potential to become a real game-changer in AI infrastructure. The device (a processing unit called CapRAM) uses the working principle of a variable capacitor (=memcapacitor instead of resistance/transistors in classical computing) to perform computations via electromagnetic fields instead of currents. This enables high performance and parameter density while increasing energy efficiency by orders of magnitude.

So What Is the Big Deal About CapRAM?

The inherent working principle of CapRAM allows for 20–100x higher energy efficiency, which in itself is already great. But equally importantly, CapRAM can be produced within existing semiconductor manufacturing processes without the need for any novel materials in contrast to RRAM/resistive approaches. This both keeps development costs low and allows for another key feature of SEMRON chip: 3D scaling.

3D scaling means the possibility of stacking hundreds of computing layers on top of each other in one single chip. This unlocks extremely high parameter density and processing power on a very small footprint, bringing the power of a data center to the edge. Every mobile device (=smartphones, smart glasses, earpods, etc.) will be enabled by SEMRON to run even large GenAI models such as LLMs locally at low latency and ultra-low costs. Say hello to true AI democratisation!

Finally, starting with an AI chip for edge applications has another big advantage for SEMRON. It allows the company not to have to compete with NVIDIA (and their very dominant CUDA software ecosystem) on data center use cases from day one, which in the past has been a suicide mission for many new chip companies. Only in a second step, once SEMRON has established a strong foothold in the market and has watched large open-source initiatives weakening the NVIDIA CUDA dominance, will SEMRON go after chips for data centers and leverage its unique energy efficiency as well as orders of magnitudes lower costs.

Paving the Way For AI to Eat the World

In summary, SEMRON is at the forefront of a new era of AI computing and, in our view, is one of the most promising in-memory computing companies out there. With its proprietary CapRAM device, it offers a unique combination of off-the-charts technical KPIs that allow even the most powerful models to run locally on any consumer device. We are excited that SEMRON has a good chance to unlock affordable and ubiquitous AI, allowing for revolutionary computing features in every device: from smartphones and smart homes to self-driving cars and medical devices.

We are excited to join the €7.3m Seed round in SEMRON alongside Join Capital, OTB Ventures, and Hermann Hauser (Co-Founder of ARM via Onsight Ventures) together with Pre-Seed investors such as Hans Rohrer, former President at TSMC Europe, Dr. Wolfram Drescher (BlueWonder), and Andreas Werner and Sven Sieber (Gigahertz Ventures).

Put the company on your watchlist!