Exploring ZK frameworks: Mastermind game in 5 different ZK languages

At Veridise, we continuously explore new frontiers in zero-knowledge cryptography and frameworks. In addition to our security auditing and tool development, we occasionally conduct working groups to stay up-to-date on emerging technologies. One of our recent working groups focused on ZK languages and applications, as we were interested in comparing different ZK languages and frameworks.

We decided to implement the classic Mastermind board game across five distinct ZK languages/frameworks: Circom, Gnark, Noir, Halo2, and Arkworks. We set out to implement the game as a way to evaluate the capabilities and characteristics of these various ZK languages. In this blog post, we’ll share our experiences and insights.

You can find our Mastermind game code in all five languages/frameworks in this GitHub repository.

Mastermind game and research objectives:

For those unfamiliar with the game, here is a quick refresher on the rules of Mastermind (variations exist, but this rule set seems to be the most common):

• The Codemaker creates a secret code consisting of 4 colored pegs.
• The Codebreaker’s goal is then to guess the code within (usually) 10 attempts.
• The Codebreaker starts guessing. The Codemaker provides feedback for each guess:
  – 1 black peg indicates a correct color in the correct position, and 1 white peg indicates a correct color in the wrong position.
  – Feedback is unordered (i.e., the Codebreaker is not told which peg is in the correct position).

The goal of our working group was to gain more familiarity with the challenges of implementing a ZK application in each of the frameworks; we settled on implementing the Mastermind game in each of these languages, as the game is non-trivial, but not too time consuming to understand and implement. By implementing the same game across multiple languages, we primarily sought to gauge the complexity of learning the languages and related APIs and using them to implement the game. We measured this complexity empirically by measuring the differences in the implementation sizes in source lines of code (LoC) and non-empirically by summarizing what we liked or disliked about the languages. Overall, the aim of this exercise was to help us as ZK security analysts to better understand what pitfalls may befall developers using these languages.

Comparisons

Only source code lines are counted in the numbers above. Blank lines and comment lines are omitted. We used the cloc tool for the counting: https://github.com/AlDanial/cloc

Before we began hacking away, we first read some documentation and perused some example projects for each of our target languages. Of the five languages we surveyed, three of them are either Rust-inspired languages (Noir) or are Rust libraries (Halo2, Arkworks). While these have similar syntax due to their Rust origins, the other languages vary widely in syntax, as gnark is an embedded DSL written in Go and Circom is a very specialized DSL that has syntax similar to that of Verilog, a hardware description language used to design digital systems and circuits.

After performing this survey, we dove into our implementation. We crafted the Circom implementation together in a group call, and then our four person team each set out to implement Mastermind in one of the four remaining languages. After finishing our implementations, we reconvened to compare the implementations.

There was a clear divide in the implementation sizes of ZK DSLs, with Noir and Circom having the smallest implementations (77 lines and 90 lines, respectively), and with the embedded DSLs of gnark (101 LoC), Halo2 (156 LoC), and Arkworks (227 LoC) having larger implementations due to their less succinct syntax and larger required boilerplate. Notably, gnark has the most compact interface (as gnark Circuits only need to define a single function), which lends to it having the smallest implementation size among the embedded DSLs.

Next, let’s explore each framework in a bit more detail.

1. Circom: The Baseline

Circom served as the baseline for our comparison. This language is widely recognized and used in the ZK space. It’s also the most “well established” language of the bunch, with the initial version of Circom being almost 6 years old now, having launched in 2018. Veridise also has extensive expertise in Circom, as we’ve uncovered critical severity vulnerabilities in circom projects (e.g., in the circom-pairing library).

Circom is a unique domain-specific language (DSL) that forces developers to think in terms of arithmetic circuits, where circuits generate a set of output signals given a set of input signals. Modularity is achieved by “wiring” circuits together (i.e., connecting the output signals of one circuit to the input signals of another) to create larger and more sophisticated circuits. Unlike the other languages in this survey, Circom’s syntax isn’t inspired by a mainstream software language, but rather takes inspiration from Verilog, a hardware description language.

An example of Circom code used to validate a code in Mastermind. Compare this example to:

An example counter circuit written in Verilog, courtesy of https://www.chipverify.com/tutorials/verilog. We can see similar component instantiations (module in Verilog, template in Circom) signal declarations (input ... output ...), and signal operations (<= in Verilog, similar to <== in Circom) in comparison to our Circom example.

This allows Circom to present an intuitive interface for wiring arithmetic circuits (as it is inspired by languages used to describe digital circuits), but also lacks the ergonomic features of well established languages (e.g., Rust’s functional programming language features and tooling ecosystem).

Strengths:

• Structured DSL with reusable components
• Established in the ZK community with extensive documentation and examples

Weaknesses:

• Can be verbose due to lack of high-level language features
• Tooling and testing infrastructure is more primitive compared to other languages, which have better support for, e.g., unit testing

You can find our full implementation of Mastermind in Circom here.

Suggestion: Document Intent

Circuit implementations can sometimes become difficult to decipher as the developer works to implement their algorithms efficiently in an unfamiliar language with idiosyncrasies and limitations that differ from general-purpose programming languages. Adding documentation to explain the intent of the code, along with explanations of how the code achieves that intent, can help security analysts verify the correctness of the code and help future developers (or current developers who revisit the code in the future) understand the code and use it or update it appropriately.

2. Gnark: Customizability Meets Complexity

Gnark offers a highly customizable approach to ZK circuit design. Circuits in gnark are written as API calls in arbitrary Go code, giving developers significant flexibility. This customization even extends to the ability to fine-tune and customize the constraint system.

However, this flexibility comes at a cost. Since gnark is a library in a general-purpose programming language, developers (and security analysts) need to have a great understanding of gnark’s API and, to some extend, its internal workings, to ensure the correctness of the implemented application (this observation is even more applicable to Halo2 and Arkworks, which have even more complex APIs). This can make the process of reading and writing gnark circuits more tedious than would be the case for dedicated ZK DSLs.

This is the core structure that must be implemented for a circuit in gnark; furthermore, the frontend.API object contains the vast majority of methods required to implement arithmetic circuits.

One mitigating factor for gnark, however, is that the core APIs presented are smaller than that of other ZK libraries we experimented with, and hence the gnark implementation is the smallest (in terms of LoC) compared to that of other ZK languages.

Strengths:

• Highly customizable
• Highly flexible, as circuits are written as API calls in arbitrary Go code

Weaknesses:

• Requires more knowledge of APIs and internals to ensure correctness

You can find our full implementation of Mastermind in gnark here.

Suggestion: Modularize and Standardize

In embedded DSLs (which have fewer guard rails when compared to dedicated ZK DSLs), special care should be taken to keep code modular and to standardize how the codebase is organized across the entire project. It is much easier to review code that follows a small set of standard conventions.

3. Noir: High-Level Abstraction with Trade-offs

Noir is a relatively new ZK language, introduced in 2022. Noir has a Rust-like syntax and is designed to be backend agnostic, making it a high-level language that abstracts away many of the complexities associated with ZK circuit design. Noir’s ecosystem is well-developed and provides many examples (e.g., the awesome-noir GitHub repo), providing a robust foundation for developers.

Note the relative simplicity of generating a Poseidon hash in Noir compared to Arkworks below.

However, Noir’s high-level nature also limits fine-grained control over circuits, which may be a drawback for applications requiring precise optimization.

Strengths:

• Syntax is Rust-like and very compact compared to other ZK languages
• Language is high-level and backend agnostic
• Has a well-developed and well-supported ecosystem

Weaknesses:

• Limits fine-grained control over the circuits since the language is high-level
• Relatively new language, it may lack some maturity over other ZK frameworks

You can find our full implementation of Mastermind in Noir here.

4. Halo2: Efficiency at the Cost of Verbosity

Halo2 is a ZK library which emphasizes customizability and efficiency. It allows developers to create highly efficient proving systems tailored to specific applications. However, we found that Halo2’s verbosity and boilerplate code can be daunting, leading to long compilation times and a steeper learning curve.

A snippet from common.rs, which showcases the use of multiple different “chips”, each of which is used to perform different sets of operations. Finding, initializing, and using each chip correctly is one challenge of Halo2’s large API.

Despite these challenges, Halo2’s ability to produce optimized proving systems makes it an attractive choice for performance-critical applications.

Strengths:

• Highly customizable and capable of creating efficient proving systems
• Tailored towards applications that demand performance optimization

Weaknesses:

• Verbose and requires extensive boilerplate code
• Long compilation times

You can find our full implementation of Mastermind in Halo2 here.

5. Arkworks: The Power (and Complexity) of Rust-Based Modularity

Arkworks is a suite of Rust-based ZK libraries that offer a modular approach to building ZK applications and custom ZK components. Of the ZK technologies featured in our comparison, Arkworks deserves mention for its focus on modularity and the ability to compose ZK circuits from smaller, reusable components.

Arkworks’ strengths lie in its modularity and customizability, as common functionality can be augmented with custom components to create highly specialized ZK tech stacks. However, the complexity of Arkworks can be overwhelming for newcomers, and the steep learning curve may deter some developers.

An example of Arkworks’ complexity: while other languages may have relatively simple interfaces for hash functions, Arkworks’ requires custom configuration to compute a Poseidon hash.

Strengths:

• Rust-based, with a focus on modularity and composability.
• Supports a wide variety of proving systems and cryptographic primitives out of the box and is extensible to new ZK components as well.

Weaknesses:

• Steep learning curve and complexity.
• Requires familiarity with Rust and modular design principles.

You can find our full implementation of Mastermind in Arkworks here.

Suggestion: Separate Concerns

One advantage of DSLs like Circom is that the application logic is almost completely separate from other concerns, such as “what proof system am I using?”. In ZK libraries and embedded DSLs, strive to do this as much as possible; we often find it is much easier to review the application logic separately (to answer the question “are all values properly constrained?”), then review how the proof system is set up and used by the larger ecosystem. When these concerns become intertwined (as is all too easy to do in, e.g., Arkworks), reading the code becomes much more difficult, as the reader has to consider many different properties of the code at the same time.

Conclusion: Choosing the Right ZK Language

In general, we’ve found that customizability is often the enemy of comprehensibility, even if the language designers strive to avoid this tradeoff. Take Circom, for instance — while the specifics of the proof system or targeted curve are hidden from the source code, it allows the actual constraints and witness generation of the application to be more clearly understood.

Noir’s goals are similar to that of Circom, in the sense that its high-level, Rust-like syntax greatly simplifies development — even more so that in Circom, as Noir’s implementation was the smallest of all those we surveyed — and similarly hides ZK details of curves and proving systems from developers.

In contrast to Circom and Noir, Arkworks presents itself as much more of an “expert’s tool” — there are so many ways to customize your circuits that it’s difficult to even get started, as there is minimal guidance provided for setting up a “basic” circuit.

Halo2 is similar to Arkworks in that it aims to provide the developer with great power to customize and optimize their circuits to create high-performance ZK applications. But, as with Arkworks, Halo2 ends up being fairly verbose, and also tends towards having long circuit compilation times.

Among embedded DSLs, gnark is likely the easiest to get started with. It presents many opportunities for customization, but the “default” configuration presents a minimal, easy-to-use interface that makes writing your first circuit fairly straightforward. However, personally….Go is not my favorite language (interfaces are implemented implicitly? Who thought this was a good idea?), and I have found that it is a “easy to write, difficult to read” language where individual design decisions can quickly obfuscate the implementation.

Overall, we have found there is no one right ZK language or framework — just like how there’s no one right general purpose programming language (an unexciting conclusion, I know). So, when starting a new project, we recommend you try doing what we did — give a few different languages a shot! See what works best for you, your team, and your project.

GitHub repository

You can find our Mastermind game code in this GitHub repository, implemented using Noir, Circom, gnark, Halo2, and Arkworks.

Author: Ian Neal, R&D Engineer at Veridise

Working group team members: Alp Bassa, Ian Neal, Tim Hoffman, Tyler Diamond

Want to learn more about Veridise?

Twitter | Lens | LinkedIn | Github | Request Audit