Chapter 13: Scaling Intelligence
TL;DR: We’re ecstatic to be introducing Remainder: the world’s most powerful ZKML prover.
In one of the first production implementations of our GKR prover, we achieve a 180x overhead for proving as compared to raw AI inference on the same hardware. For theory goodness and detailed benchmarks, check out “Scaling Intelligence: Verifiable Decision Forest Inference with Remainder”.
And! If you’d like to get an early preview of Remainder, we’re opening up sign-ups for the Modulus Early Access Program (or MEAP). Come experience the POWER of specialized ZK and build the future of verifiable AI with us!
Exactly 13 months ago, we wanted to answer a simple question: “What’s the concrete cost of proving that an AI result hasn’t been manipulated?”
To our surprise, this turned out to be a frustratingly tricky question to answer. After all, modern ZK proving systems are primarily built to support smaller, generic VM operations, each requiring its own unique conventions and optimizations.
Spurred on by our own desire for clarity, however, we eventually journeyed across six popular proving systems, diligently measuring the costs of provable AI in each (the results are chronicled in “The Cost of Intelligence”). Now that we were on the other side, the takeaway was clear:
As of Jan 2023, proving AI operations in ZK — the ultimate way to ensure tamper-resistance — is still hugely punishing… even optimistically, real-world overhead easily ranges from 10,000x to 100,000x as compared to non-verifiable AI outputs.
To put another way, even exotic silica wouldn’t alone make ZK practical for provable AI. The cost of ZK is simply too substantial. Bummer.
Looking at those graphs more closely, though, we noticed that the more a proving system acknowledged the architecture of the underlying compute, the better it seemed to perform. In other words, specialization appears to be a path towards a better future for ZK performance.
Now a year later, we are ecstatic to finally be introducing our custom prover — Remainder. We believe it is the world’s fastest ZK prover for AI inference. In fact, for the in-production implementation of Upshot’s NFT appraisal model, we currently demonstrate:
*180x overhead (as compared to naive AI inference on the same CPU hardware)
This is unprecedented. And yes, it’s live right now. Better yet, over the past few months, Remainder has already quietly logged hundreds of thousands of AI results on Ethereum. Want to learn more?
“Scaling Intelligence: Verifiable Decision Forest Inference with Remainder” is the first of a series of papers we’ll be releasing this year unpacking the theory behind our specialized prover.
In this first paper, we detail a couple highlights from the small armada of optimizations already powering Remainder. Among these include:
- A novel claim aggregation strategy which improves substantially on the interpolation method from [Tha13] in many cases.
- An expression of and refinement over nearly the entirety of the verifiable decision forest circuit from [ZFZS20a] within a structured GKR circuit.
- And a generalization of the linear-time prover techniques within [XZZ+19] to the dataparallel case, borrowing inspiration from a sumcheck technique within [WJB+17], which we dub ♎-🦒 in honor of the papers’ respective nicknames.
Taken together, we’ve stayed true to our original observation from “Cost” more than a year ago: Remainder double downs on the specialization premise, applying an optimized version of the GKR protocol to a verifiable decision forest circuit.
You’ll have to check out the benchmarking section in the paper for the juicy details. In the meantime, here’s a peek at some the beautiful graphs we’ve got in the paper:
Thank Yous and a Call-to-Action
Remainder is built on the spectacular work of talented academics and ZK practitioners — from the discovery of the sumcheck protocol itself, to GKR and every extension since. In particular, we’d love to highlight Zhang Zhenfei and Riad Wahby’s incredible advisorship. As well, our deepest thanks to Wei Dai and Justin Thaler, both of whom have been invaluable resources and incredible thought partners. Our gratitude also goes out to Vitalik Buterin, who gave us some clever ideas around nonlinearities that we’re giddy to implement next.
We are so grateful to be able to benefit from the intelligence and goodwill of so many from our community, and to those above as well the many more who helped us along the way — thank you!
Alrighty, now for the fun part:
That’s right, while we’re excited to share more about the large-scale use-cases we’ve already started to tackle with our partners, we know that we’ve only scratched the surface of what the community will be able to do with the world’s most powerful ZKML prover. That’s why we’re introducing the Modulus Early Access Program, or MEAP — a 4 week crash-course of Remainder goodness to an elite cohort of select builders. The program will take place starting in Q2 2024, and we will be leading anyone brave enough to join us through the theoretical underpinnings of practical GKR. That’s right, we’re going from the ground up, alongside a hearty dose of circuit-building and proving within Remainder itself!
MEAP1 won’t be for the faint of heart — it’s early days around here, and for our first cohort of MEAPers (that’s right! There will be more!), a strong math + programming background will be hugely helpful.
Whatever your context, however, we’d love for you to fill out our form nonetheless: https://itrrvgq8qar.typeform.com/to/RUAc0iTc. Even if you’re not a ZK native, we’re already organizing communities of the most talented devs and builders around ZKML intersections with DeFi, gaming, NFT/AI art, identity, and more. Sign up and join the movement around specialized ZK ;)
That’s all for today! Check out our new paper, stay tuned on X for more updates, and if you’re in town this week, make sure to come say hi to the team in Denver.
“Welcome to a world of pure imagination…
