Paradigm shift at the very edge
Why we invested in Innatera Nanosystems
by Christian Reitberger and Julian Riebartsch
Over the last fifteen years, investing in semiconductor innovation has fallen out of favor in the VC community. On one hand, many generalist VCs decided to focus on business model innovations and gradually lost their semiconductor competencies. On the other hand, creating a winning fabless semi company grew increasingly expensive and in a rapidly consolidating market of tier one semi players, the number of potential trade buyers for the companies was declining year over year.
With that backdrop, it is particularly remarkable that one subsector in the semi space saw a Cambrian explosion of activity in the last five years — AI accelerator chipsets. Since Nvidia discovered that their GPUs could be applied (some would say mis-used) for deep neural network training and inference, semi entrepreneurs started to pursue a plethora of new architectural ideas and billions of euros were being invested in the US, Israel and in China to develop new AI accelerator chipsets.
And now we as btov Industrial Tech team are hopping on this bandwagon, too? Why — and is this not too late (and too expensive) for a European early stage fund?
The answer to these questions requires a deeper understanding of the various market segments and use cases in which AI accelerator chips can be utilized. Very roughly speaking, there are three market segments to distinguish: high performance computing data centers, mobile device applications — and at the very edge: applications close to the sensors whose signals are to be processed. These segments differ in terms of their specific required combination of “raw performance” and “performance per energy consumption” and their needs for low latency and the ability to learn at the edge.
While total energy consumption for processing in data centers is becoming more and more important, the overall energy budget exceeds the one of mobile devices by far. Typically, algorithms leverage the immense performance that can be offered for applications that do not require low latencies. Examples are dynamic pricing, route planning and optimization, or financial risk analyses, amongst many others. Mobile device applications face tighter energy bounds and can be much more latency sensitive. However, often times these use cases still require the execution of rather large deep neural networks to perform tasks like face detection, visual tracking, or language translation. The ASIC in question thus has to perform accordingly.
Lately, it has become imperative to shift some processing closer to or even into the sensor. Not only does this (pre-)processing reduce data transmission and thus bandwidth demands, but it mitigates information loss as well. Intricacies of and correlations in the sensor input can be lost during data conversion and transfer — not so if a pre-evaluation of the data is performed upfront. On top, this processing and communication behavior enables low energy consumption. Applications include intelligent speech processing in human-machine interfaces, vitals monitoring in wearable devices, target recognition in Radars and Lidars, and fault detection in industrial and automotive equipment. These have to be performed within a narrow power envelope of just a few mW not to significantly impact battery life of edge devices. Low response latency is a pivotal factor as well.
Admittedly, the strict demarcation of mobile device applications and applications at the very edge is a gross oversimplification and there are many nuances inbetween. We found the following illustration of the edge processing pipeline showing the typical power requirements of the different processing steps very useful:
And now we come to Innatera, our new portfolio company in that space. Taking inspiration from evolution’s millions of years of optimizing computing architectures for ultra-low power, ultra-low latency, Innatera’s neuromorphic chip mimics the brain’s mechanisms for pattern recognition. To be precise, it uses an analog mixed signal implementation of neurons and synapses that execute Spiking Neural Networks (SNNs). This means that the neurons fire asynchronously when the “potential” of the particular neuron exceeds a threshold in case an event occurs that influences this neuron (opposed to firing at each propagation cycle in classical feedforward neural networks).
This architecture inherently allows for energy efficient event-based data handling which utterly changes how sensory input is being processed. Its learning and inference behavior enables always-on pattern recognition functionalities, meaning it is continuously available to analyse input patterns while wasting practically no energy. By co-locating memory and processing the design provides better memory access and thus enables real-time processing with dramatically reduced response latency. Since its setup allows the chip and network to consider fine-grained spatio-temporal intricacies, the performance of the SNNs can be better than using traditional neural networks. In addition, this network, and thus the chip, usually has a smaller footprint than running similar analyses with, for example, convolutional neural networks on traditional von Neumann architectures. In the end, if accepted in real-life applications, these systems could even support mechanisms for true unsupervised online learning — the ability to adapt to changes in a task or situation as they occur at the edge.
In numbers, Innatera’s ASIC reaches 10,000x higher performance per watt while implementing 10–100 times more compact networks. At the same time sensor data processing can be 100x faster while consuming up to 500x less energy compared to conventional digital alternatives. Much more importantly though, this technology is able to herald a paradigm shift not only in terms of how computing is done but also on what data and where. The company is taking the complete processing chain from pre-processing through pattern recognition on “raw” sensory data to sensor fusion close to and even into the sensor.
In our humble opinion, there are not that many other players or technologies readily available in the market today that are actually able to perform this data processing shift reasonably. So no — it is not too late to support the potential leader in this emerging market.
What will it take to make Innatera successful? There are factors which are under our control: the iteratively released versions of the chip need to fulfill the performance promises which Innatera is making on real silicon. The software stack needs to allow users to quickly and without having to be SNN experts design applications, simulate their performance and enable rapid design iteration cycles.
But Innatera also needs external tailwind to make their vision come to life: they will need the openness of sensor makers and IDMs to try out something new and to work with a small company. Europe is particularly strong in advanced sensor development — and Europe still has a number of innovative large (and growing) semiconductor plays. One of us was asked a few years ago by the CEO of one of the large European IDMs why we did not fund more semiconductor innovations …well… this is the chance to show that it is appreciated and the big players really engage.
The other external factor will be the availability of growth capital if and when Innatera will have to raise its Series A round. Growth capital is available in Europe. However, most of it is still being spent on traditional SaaS investments and business model innovations. We will test the waters in a few quarters if the more enlightened representatives of the growth capital ecosystem will be willing to invest in some deeper tech real innovations, too, and convert lip service to investment action.
