zkSync Boojum Upgrade: Understand the zkSync Boojum Upgrade in One Article - Safer and More Efficient zkEVM
Original author: zkSync
Original Translation: Shenzhen TechFlow
TLDR
Upgrade: zkSync Era is transitioning to a new Boojum proof system without the need for regeneration.
Performance: Boojum demonstrates top-notch proof performance, complementing the zkSync Era sorter, which can already handle over 100 TPS.
Decentralization: Boojum provers only require 16 GB RAM, enabling future large-scale provers decentralization.
zkSync's mission is to advance personal freedom for everyone by building a trustless, secure, permissionless, cost-effective, user-friendly, scalable, and infinitely expandable blockchain network, making digital self-ownership universally accessible.
In order to fulfill this mission, the Alpha version of zkSync Era was opened to the public over three months ago, with amazing response. Network highlights:
Locked Total Value TVL of $577 million (source: L2Beats);
23,750,000 transactions in the past 30 days - the most among all L2s (source: L2Beats);
9,735 verified smart contracts.
In March 2023, the SNARK-based zkSync Era was born, utilizing a battle-tested circuit framework that has supported zkSync Lite on the mainnet for nearly three years. However, we know this won't be the final version of the zkSync Era proof system, so we designed the system to allow fundamental changes without regeneration. This means we can deploy significant cryptographic upgrades without disrupting developers and users.
Behind the scenes, we have been working on cryptographic upgrades for a long time. Today, we are excited to announce the first upgrade: zkSync Era is transitioning to a new STARK proof system called "Boojum".
Introducing Boojum
Boojum is an arithmetic and constraint library developed in Rust, which we use to implement the upgraded versions of zkSync Era and ZK Stack's ZK circuits.
What is Boojum?
Features of Boojum:
PLONK Arithmeticization: In the context of zero-knowledge protocols, arithmeticization is the process of converting general computations into mathematical form. Like the current proof system, the upgraded system still utilizes PLONK-style arithmeticization. By adopting this approach, writing ZK circuits becomes simpler compared to some alternative forms, making the system easier to develop, audit, maintain, and upgrade.
Powerful Commitment Scheme: The core of Boojum is the FRI commitment scheme, which is a crucial component that allows us to commit to bounded polynomials and subsequently effectively prove the claimed (polynomial's) opening belongs to a low-degree polynomial.
Efficiency of the System: While witness generation is sometimes overlooked when people talk about prover performance, in the current version of the proof system, we have achieved highly efficient GPU provers that are optimized to the extent where witness generation time is on par with proof generation time. Through Boojum, we provide witness generation with automatic parallelization (if dependency graph allows) while maintaining the ease of defining witness generation functions.
Extensibility: The basic constraint system abstraction is very concise but allows users to add custom gate types in various ways, for example by adding specialized polynomials or reusing so-called "generic columns". After defining simple geometric structures for their circuits, users are provided with an extension interface that enables the automatic generation of provers, verifiers, and recursive verifiers. This enables a highly efficient development process; if users change the circuit structure and choose to use different gate types, they simply need to call the interface again, which will regenerate the keys and ensure they use the correct provers and verifiers.
Single Stack: Through Boojum, all the above can be expressed using standard, idiomatic Rust, leveraging the expressiveness of its type system. The compute-intensive parts of the GPU prover are written in CUDA C++, but we provide Rust bindings for composition.
Boojum uses a prime field with a size of 2 ^ 64 - 2 ^ 32 + 1 (referred to as the "Goldilocks field") by default, and provides implementations of corresponding field binding primitives, such as the Poseidon 2 hash function, as well as implementations of more standard encryption primitives based on lookup tables, such as SHA 256, Keccak 256, and Blake 2 s.
It is important that in the final step, we will use an opaque pairing-based SNARK to wrap the STARK proof and verify this SNARK on Ethereum. This proof is much smaller and has lower verification cost, which reduces the cost of the proof system and therefore lowers the cost of the transaction itself.
Why choose Boojum?
When designing Boojum, there were two key factors behind the decisions: (1) world-class proof performance, and (2) reducing the hardware requirements for decentralization.
World-class performance
Current SNARK-based systems, although effective for the present, cannot scale to the high transaction volume and near real-time transactions in the ZK Stack. The future goal of Hyperchain as the zkSync Era is to support these systems in the coming years. Our vision for these systems is that proofs can be generated and verified cheaply and quickly, enabling fast finality and interoperability between Hyperchain and zkSync.
The performance of the proof system directly affects the fees users pay for their transactions, and over time, these costs need to approach zero. The current version of the proof system is efficient enough to build zkEVM and process millions of transactions in a few months, but through Boojum, we can do even better!
To measure the proof generation time of the network (as well as other key performance indicators), we collaborated with Celer, who has extensive experience in benchmarking and analyzing multiple proof systems. In the image below, you can see that Boojum performs well in most of the systems analyzed for performance. The results speak for themselves: our implementation demonstrates world-class proof performance and, to our knowledge, is currently the fastest proof system in production use.
For fair comparison, Celer conducted these benchmark tests on a CPU-based prover, while our mainnet system uses a GPU-based prover, which is faster.
Turning to STARK-based proof systems represents a significant performance boost and will contribute to ensuring finality and support increased activity levels for zkSync Era and other ZK Stack-based systems with low latency.
Reducing the requirements for decentralized hardware
These performance results are particularly impressive when one considers that this is not the only metric we optimize for — we aim to improve system performance while also reducing the hardware requirements to run it.
Currently used proof systems, including our existing one, have high hardware requirements. Our current proof system runs on a set of A 100 GPUs, each with 80 GB of RAM. The demand for expensive and powerful machines represents a significant barrier to our aspiration of user-driven decentralized proof generation. To achieve this goal, it is not enough to make proof generation permissionless; users should not need an expensive machine with hundreds of GBs of RAM to participate.
This is another area where we have made extraordinarily progress! Our GPU prover for Boojum requires only 16 GB of RAM, a low threshold that is an important step towards our envisioned future. CPU-based proving can also be done with as low as 64 GB of RAM, and we aim to further reduce it to 32 GB, making full use of modern multi-core processors.
Lastly, the zkSync Era sequencer, built on Rust, is already capable of processing over 100 transactions per second (TPS). The introduction of the new proof system not only improves performance but also reduces hardware requirements, making it an ideal complement to the sequencer. The performance gains brought by Boojum mean that the system can prove transactions faster, and the reduced hardware requirements improve accessibility to cheaper machines, enabling higher scalability.
The journey of Boojum to mainnet
The team has spent months developing this upgrade, and we are excited to now enter the phase of testing the system on the mainnet. We also want to share some stories from the journey so far.
Upgrading zkSync Era
Firstly, the design of zkSync Era allows us to upgrade each component over time, and the proof system is no exception.
Similar to Ethereum, we use the Merkle tree data structure to store information about the network state. Given that we are proving statements about the system state, this information is necessary for the proof system. For this Merkle tree (and the way the proof system interacts with it), a key design decision is to use a non-algebraic hash function, specifically Blake 2s. If we were only optimizing for the convenience of proof generation, we would use an algebraic hash function (such as Poseidon 2), but this choice would couple the observable state with the proof system parameters - for example, the choice of prime field. Then, any upgrade to the proof system would require regenerating the entire state.
From Design to Review: The Journey of Boojum
About a month ago, we began to gradually shape a complete end-to-end version of our new proof system implementation. Given the complexity of this change and the criticality of system correctness, we initiated a series of internal and external audits.
At the time, the zkEVM circuits and Boojum arithmetization library were still actively being developed, but we collaborated with external security auditors, focusing on early identification of soundness potential issues in our main circuits and Boojum components. We worked closely with them, providing them with full access to source code and documentation, and they conducted reviews and testing of the zkEVM circuits and Boojum tools using automated and manual testing approaches.
Boojum: From Review to Testing
This brings us to today, our next phased rollout: mainnet shadow mode! We are excited to announce that we are now running the new proof system in parallel with the existing proof system, even though Boojum is currently in a testing phase. We have begun generating and verifying "shadow proofs" for mainnet blocks.
These shadow proofs are not necessary for the mainnet version of zkSync Era - that will still be driven by the existing proof system. We are just verifying these shadow proofs to further test and optimize the system, but using actual production data from zkSync Era user activity.
Boojum: From Testing to Mainnet
As with everything we do, security is always a top consideration. We will only consider migration when we are fully satisfied with testing the new system and will share more details in the coming weeks and months. We also plan to conduct further audits and security assessments to bring this exciting upgrade closer to full deployment and gradually phase out the current proof system.
We believe that Boojum is the next stage of zkEVM that is more secure, scalable, and efficient.
