Is Zero-Knowledge Machine Learning (zkML) Legit?
Sure there’s machine learning and all that, but what the heck is zero-knowledge machine learning (zkML)?
In our world dominated by AI and Sam Altmans, machine learning is no longer something new. Startups which claim to implement machine learning models are no longer garnering mainstream attention — the neighborhood postal office might be already using it to optimize delivery routes.
So what’s up with zkML, the new “variation” everyone is suddenly talking about? Is it the future of technology, or is it just another bombastic scam? In this article we will examine relevant concepts, how it actually works, and its implications for the future.
The Enigma of Zero Knowledge
Let’s start with the first part of the phrase. A zero-knowledge (ZK) proof is a cryptographic procedure wherein one party, known as the prover, can demonstrate to another party, the verifier, the truth of a given statement without divulging any additional information, other than the fact that the statement is true. It has three requirements:
- Completeness: If the statement is true, an honest verifier will be convinced by an honest prover.
- Soundness: If the statement is false, no dishonest prover can convince the honest verifier. The proof systems are truthful and do not allow cheating.
- Zero-knowledge: Unique characteristic of ZK proofs. If the statement is true, no verifier learns anything other than the fact that the statement is true.
For better understanding, imagine a ring-shaped cave with the entrance on one side and a magic door blocking the other side. Dave (the prover) has discovered the passcode that opens the magical door, and he wants to prove this to Ashley (the verifier) without explicitly giving out the code. This may be because Dave values his privacy and is unwilling to disclose the fact that he knows the passcode to the broader world.
The paths from the entrance are designated as A and B. Dave enters the cave, choosing either path A or B without Ashley’s knowledge. Ashley then randomly names path A or B for Dave to use on his return. If Dave indeed knows the passcode, he could effortlessly open the door and return via the specified path. If not, his choice is limited to the path he initially chose.
In this scenario, Dave’s chances of guessing correctly is 50% since Ashley’s designations are random. If the process is repeated numerous times, the likelihood of him predicting her designations diminishes significantly. Thus, if Dave consistently emerges from the path Ashley designates, Ashley can reasonably infer that Dave does indeed possess knowledge of the pas
A more formal definition of ZK proofs can be illustrated through computational models, namely, a system involving three Turing machines: the prover (P), the verifier (V), and the simulator (S). The interactive proof system (P,V) for a language L is considered zero-knowledge if, for any verifier V^, there exists a simulator S such that:
∀x ∈ L, z∈{0,1}, ViewV^ [P(x) ↔ V^(x,z)] = S(x,z)
Where x is an input of the language L, z is an auxiliary string, V^(x,z) is the verifier’s output with input x and z, ViewV^ [P(x) ↔ V^(x,z)] is the interaction between prover P and verifier V, and S(x,z) is the output of the simulator. Since the views of the interaction between P(x) and V(x,z) are indistinguishable from the views produced by S(x,z), we can be certain that the verifier cannot gain any additional information beyond the validity of the statement being proven.
One thing to note here is that the auxiliary string z represents prior knowledge. The verifier incorporates z in its challenge against the prover, while the simulator incorporates it in its attempt to mimic the interactions between the prover and verifier. Despite being accessible to both, z prevents the verifier from gaining information during the interaction with the prover.
Cryptographic Proofs: Guardians of Privacy
ZK proofs are beneficial for increasing privacy — only a small portion of key information is exchanged — and increasing scalability — proofs are verified at a faster rate compared to if the entire dataset is verified. The two most prominent cryptographic proofs as of now are zk-SNARKs and zk-STARKs.
zk-SNARKs (Zero Knowledge Succinct Non-interactive Argument of Knowledge) is introduced in a 2012 paper co-authored by Nir Bitansky, Ran Canetti, Alessandro Chiesa, and Eran Tromer. It enables one party, the prover, to convince another, the verifier, of the validity of a statement without revealing any details about the statement itself.
The key feature it offers is succinctness, which means that proofs are short and easily verifiable. Rather than re-running every transaction, nodes can use zk-SNARKS to verify the validity of a computation without replaying it and waste a significant amount of time and energy.
Moreover, by operating in a non-interactive fashion, it eliminates the need for continuous communication between two parties. This adds an extra layer of efficiency and convenience, making it particularly attractive for a wide range of applications. All of this, however, relies on a trusted centralized setup (specifically the CRS, or Common Reference String), raising concerns about security and the initial generation of parameters.
zk-STARKs (Zero Knowledge Scalable Transparent Argument of Knowledge) is introduced later in a 2018 paper by Eli Ben-Sasson, Iddo Bentov, Yinon Horesh, and Michael Riabzev. Similar to zk-SNARKs, zk-STARKs enable a prover to convince a verifier of a statement’s validity without divulging specific details.
Unlike zk-SNARKs, zk-STARKs work without the trusted setup of a CRS and utilize publicly verifiable randomness. They exhibit more scalability, with proving complexity scaling quasilinearly in response to computation complexity, as opposed to the linear scaling in zk-SNARKs. zk-STARKs prove to be more suitable for applications managing substantial transaction volumes, thanks to their effective handling of larger computations. Despite these advantages, they have a higher verification overhead compared to zk-SNARKs.
What are some real-life applications of these cryptographic proofs? In blockchain, zk-SNARKs, notably utilized in privacy-centric cryptocurrencies like Zcash, enable transaction validation without revealing sensitive details, hence maintaining blockchain integrity. Their succinct nature enhances scalability through efficient verification. zk-STARKs, with their trustless setup, find applications where establishing trust in a centralized authority is challenging.
ZK-rollups, a Layer 2 scaling solution which seeks to increase throughput on the Ethereum Mainnet, leverage both zk-SNARKs and zk-STARKs while moving computation and state-storage off-chain. After bundling (“rolling up”) transactions into batches and moving them off-chain, validity proofs need to be submitted to ensure the correctness of these changes. Those that utilize zk-SNARKs prioritize succinctness and privacy, while those with zk-STARKs tend to prioritize transparency and scalability.
Source: Messari
In healthcare, both ZK-SNARKs and ZK-STARKs facilitate secure computations on sensitive medical data without exposing individual records, ensuring privacy in collaborative research efforts. Supply chain management also employs them to authenticate products without disclosing sensitive information. Companies can prove product legitimacy without revealing proprietary details, hence building trust among stakeholders.
Operations of zkML
zkML, put simply, incorporates zero-knowledge proofs into the realm of machine learning. It solves two problems:
- It is common to want to keep the inputs and parameters of an ML model private. This is because these data may contain sensitive information such as personal financial or medical data.
- It is important for downstream systems, such as on-chain smart contracts, to be able to verify that the input was correctly processed to produce the public output.
zkML does so by accurately training models without exposing underlying inputs and parameters. It helps users keep their information private but also have the computer do the right thing. In essence, it creates a decentralized machine learning network anyone could participate in without revealing any unnecessary details of the models being used.
As depicted, whereas users of previous ML models have no idea whether their personal data is put to malicious use and whether the models are executed correctly, users of zkML models are certain that their data is in safe hands and that the models outputs are legitimate.
Practically speaking, all zkML models are built on top of a trained ML model. This involves converting the trained ML model into its zk-circuit representation and then adding in the cryptographic proofs (zk-SNARKs or zk-STARKs). Below are some (extremely) simplified steps to achieve this:
Step 1: Defining the model
Self-explanatory. The foundation of any machine learning endeavor is the definition of the model. Whether it’s a neural network for image recognition or a regression model for financial predictions, clarity in defining the architecture and parameters is paramount. Consideration should be given to the model’s layers, activation functions, and the nature of the task it aims to accomplish.
Step 2: Quantization for zk-SNARKs
Before delving into cryptographic intricacies, quantization is essential. This entails scaling the floating-point numbers representing model parameters and input data to fit within the constraints of cryptographic proofs. The quantization process, often achieved by multiplying values by constants, ensures compatibility with cryptographic operations.
Step 3: Commitment Mechanism
Privacy preservation starts with the commitment mechanism. Hashing and publicly committing to private inputs or model parameters set the stage for cryptographic operations. This commitment guarantees that the participant possesses specific information without revealing the actual data.
Step 4: Matrix-Vector Multiplication
Matrix-vector multiplication, a cornerstone of many machine learning operations, retains its significance in zkML. The circuit construction involves creating a zkML-specific matrix-vector multiplication operation that ensures computational accuracy while retaining privacy principles.
Step 5: ArgMax Computation for Classification
For classification tasks, determining the predicted class involves ArgMax computation. This step, within the zkML circuit, ensures that the position of the maximum value in a set of computed values is accurately determined.
Step 6: Integration with zk-SNARKs
All of our previous efforts coalesce in this crucial step. Now that the trained ML model has aligned with cryptographic standards, cryptographic proofs is brought in to do two important tasks: attesting to the correctness of computations and maintaining zero-knowledge privacy. Integration with these cryptographic proofs ensure that the entire ML model, from commitment to computation, aligns with the zkML standard.
Step 7: Model Training and Collaborative Learning
With the zkML model constructed, the training phase involves collaborative efforts without exposing individual data. Privacy-preserving collaborative learning becomes possible as participants can jointly train the model using their private information.
Numerous zkML projects have been launched in the past few months based on this architecture. EZKL by Jason Morton (named after the Biblical prophet Ezekiel), for example, aims to bring AI on-chain using a library that allows developers to create ZK proofs of ML models. EZKL allows users to convert ONNX (a popular open format representing machine learning models which emphasizes interoperability) models into Halo 2 circuits, a proof system that works without a trusted setup and which has efficient verification time.
Another prominent project is Modulus Labs by Daniel Schorr, which is working on two projects: RockyBot, a zero-knowledge secured fighting game where human players train AIs to battle one another, and Leela vs the World, the first ever on-chain AI game. Instead of merely implementing zero-knowledge concepts on Ethereum’s ZK-rollups, Modulus Labs aims to bring AI inference to the broader internet.
However, given how early the zkML market is, most applications still occur on the hackathon stage. Examples include AI Coliseum, an on-chain AI competition developed using EZKL, Hunter z Hunter, a photo scavenger hunt utilizing EZKL to validate outputs from an image classification model, and ZK Section 9, a model that provides minting and verification of AI-generated art.
So is zkML legitimate?
The integration of crypto and AI has been picking up steam, and zkML is a prime example of one of these intersections. By incorporating cryptographic proofs into the realm of machine learning, complex models can be trained and evaluated without exposing the underlying data.
In a world where the zero-knowledge proving market is projected to reach $75 million revenue in 2024 and has the potential to exceed $10 billion in revenue by 2030, zkML is bound to play a pivotal role. With over 24,000 Web3 startups in operation currently and more incoming, the concept of zero-knowledge is only going to appear more and more frequently.
Machine learning, another field which has gained significant traction in recent years, will also greatly benefit from it. While there are concerns that companies often prioritize personal interests over users’ privacy, zkML offers the perfect blend of effectiveness and security to combat such. In fact, several companies have been taking note, with Twitter recently open-sourcing their timeline ranking algorithm but not releasing the weights and OpenAi not releasing weights for GPT-3 and 4.
Most importantly, zkML allows organizations and individuals to jointly train machine learning models without the necessity of revealing sensitive data. This heightened privacy encourages broader data sharing, unlocking more opportunities for research, innovation, and technological advancements. As industries continue to navigate the delicate balance between innovation and safeguarding user privacy, zkML stands as a promising frontier which shapes a secure, transparent, and collaborative future.
In all, is zkML legit? As powerful as it is, the answer depends on the context. For databases sourcing through private biometric data, then yes, zkML offers a compelling solution which can potentially disrupt the industry. For an on-chain quantitative trading startup, then most likely no — branding the algorithm as “zero-knowledge” will not change any of its inner operations or infrastructure.
Humans have always had a tendency to clamor towards the latest trends throughout history, and zkML is no exception. Just like how numerous startups nowadays frantically claim heritage to “Artificial Intelligence” despite not having the faintest relationship with it, zkML is bound to arouse frenzy and stupor in the foreseeable future.
The future of technology or bombastic scam, let us wait and see.
