Xnor.ai: Transforming AI at the Edge from Hype to Reality

Rouz Jazayeri
Feb 12 · 5 min read
Xnor.AI Co-Founders: Mohammad Rastegari (left) and Ali Farhadi (right)

When we started Catapult Ventures, we identified several themes that we wanted to invest behind. Agriculture robotics, high endurance drones for cargo and inspection applications, and new sensors to accelerate the shift to autonomy were just some of the areas that we were (and still are) excited about. But the very first investment thesis we discussed was the need for more efficient AI in order to bring true intelligence to the billions of smartphones, IOT devices, wearables, and endpoints in the world today.

Why is AI at the edge so important?

Let’s look at an example that we are all familiar with: smart home cameras from the likes of Ring, Nest, Arlo, etc. Until recently, all of these smart home cameras used IR sensors to detect motion and send notifications. While certainly acceptable for the first generation of smart home cameras, dependence on IR triggers a lot of false notifications: tree branches blowing in the wind, car headlights reflecting off of a driveway, or even a spider dropping in front of the camera. This was not a great user experience and also created unnecessary cloud storage costs.

The first wave of AI-enabled cameras incorporated smart algorithms for things like person detection and package detection. These cameras all sent the images to the cloud in order to properly process the image. This dependence on the cloud created a lot of inefficiencies: poor latency, high cloud costs, poor user experience, and complex privacy, reliability, and security issues. Our cameras certainly got smarter, but they still had a long way to go.

As we dove into the tech stack and identified the bottlenecks, it became pretty clear that these cameras (and billions of other IOT devices) needed two things: more efficient compute architectures and more efficient inference algorithms to run on these compute architectures. We’ve looked at (and will continue to look at) a lot of companies focusing on efficient AI compute architectures for the edge. But we decided to make an early investment in efficient software knowing that a software-focused company could ramp their business across multiple industries in a short period of time.

Developing efficient AI at the edge is not a novel concept. A lot of researchers have developed efficient algorithms using binary networks, reduction networks, and other simplifications, but there is an important nuance here: most of those optimization efforts sacrifice performance. The key is to build efficient algorithms that not only minimize compute resources, but also perform with minimal errors. This is where the challenging research problems remain. We quickly determined that these bottlenecks existed in many other verticals like manufacturing, security, automotive, retail, and consumer electronics, which gave us even more conviction to move forward on an investment.

XNOR.AI — World Class Efficient AI

Early Xnor.AI team showcasing their efficient object detection software running on an iPhone.

Enter Xnor.ai. We first met founders Ali Farhadi and Mohammad Rastegari during one of their visits to the bay area. We were immediately impressed with their technical acumen and it was apparent that they were two of the strongest efficient deep learning researchers we’d ever met.

We dove into their methodologies for training and inference, and were fascinated that they were able to create high performance, high accuracy inference by careful construction of CNN layers and smart algorithmic optimizations using single bit convolutions. These optimizations can be constructed with only single bit XNOR gates, a fundamental digital logic gate that every sophomore EE student learns about. (I know because I used to teach the digital logic design lab at Purdue.) The performance / watt was so good that they were able to run very advanced deep learning algorithms on low power, efficient Arm processors.

We were convinced that these were the founders we wanted to back. There was only one problem…despite having roughly half of our fund committed, we hadn’t done a first close yet. We hustled like crazy to finalize the paperwork and closing for our first LPs. Thankfully, Ali and Mohammad gave us some flexibility and a some extra time to get that done.

Two years later, it’s safe to say that we nailed the thesis. Customers from all types of industries were lining up to do business with Xnor.ai. Some customers wanted Xnor.ai to build customized models and datasets, other customers wanted Xnor.ai to optimize models that were already created. In both cases, Xnor.ai was able to show unbelievable efficiency improvements. I remember in one board meeting, the team showed us how they were able to take a customer’s models for a specific application and improve the efficiency by over 1,000X during a pilot engagement. 1,000X!!!

Apple recently acquired Xnor.aiand it goes without saying that we are incredibly proud of what Ali, Mohammad, Jon (CEO), and the entire Xnor.ai team have achieved. For Catapult, we still have a long way to go in order to achieve our goals, but this is an early, validating win. We formulated a thesis, we studied the market, we spoke to prospective customers, and then we scoured North America to find the best team. This is the blueprint we follow for a lot of thesis-driven investments. We don’t chase heat, we don’t get hung up on growth/traction/revenue numbers. We invest in founders with deep technical acumen and with a track record for execution in their research and early product development.

Looking back to our original investment memo, here is an excerpt that we sent to our LPs:

Finding new technologies that can help accelerate AI inference at the edge is a core thesis for Catapult Ventures. This is where XNOR.AI comes in. Current deep learning algorithms are very compute intensive and require tons of 32-bit matrix multiplication operations. The complexity of these operations requires significant GPU (or CPU) horsepower. XNOR takes these complicated deep learning algorithms and simplifies them down into single-bit (binary) operations, dramatically improving the efficiency of running deep neural nets without sacrificing performance/accuracy. This means that for the first time, AI inference can be run on low power microprocessors or even embedded microcontrollers. This is a very exciting opportunity and it will now allow a whole new level of intelligence in our smartphones, drones, IOT cameras and even across industrial IOT sensors, all places that typically haven’t had the ability to run AI inference locally.

So, what’s next on the horizon in terms of AI investing for Catapult? We’ll take a couple of months to formulate our thoughts, but it’s pretty clear that Xnor.ai was just scratching the surface. We need more efficient AI tightly coupled with, and maybe even co-designed with, more efficient compute. We need sensors with custom AI for a host of industries that are ready to add new levels of intelligence to their products, equipment, and services. And we need more AI tools to help the next wave of ML engineers accelerate their test-validate-iterate loops.

Until then, we celebrate and congratulate Ali and Mohammad, the next Chelo-kabob dinner is on us! J

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade