Alliance Dao: How to build Web3 products through ZKP?
Compilation of the original text: Deep Tide TechFlow
Compilation of the original text: Deep Tide TechFlow
Zero-knowledge proofs (ZKPs) are becoming a fundamentally transformative technology for the next decade. ZKPs have applications both inside and outside of Web3.
In Web3, ZKPs are solving two major bottlenecks - scalability and privacy:
In terms of scalability, several ZK Rollups, aka Validity Rollups, are being launched to scale Ethereum by 10-100x while improving user experience by reducing transaction costs.
In terms of privacy, ZKPs are expanding from the era of private transactions and transaction hybrids to more complex and useful areas such as private on-chain transactions, identities, and verified credentials.
There is a lot to be said about ZKPs, including our own vision for the future of the ZKP space and the startups needed to make that future a reality. However, there is still a certain education gap on how builders can benefit from ZKPs and where to start.
This article aims to fill this gap by bringing together important resources to guide developers in understanding how ZKPs work in practice and how to use ZKPs in their applications.
How do ZKPs work in practice?
ZKP is actually a technique for the prover to prove to the verifier that they know certain information without revealing that information.
In practice, at least in Web3, ZKPs are often used differently. Most applications do not use ZKPs to show ownership of proprietary data. Instead, ZKPs are used to increase trust through verifiability. We expect ZKPs to become the standard trust model between entities in the future. The reason is that the two main components of a ZKP - proof and verification - are being separated in a way that enables a unique interaction scheme between a trust-seeking entity and its users.
The main components of ZKP are proof generation and proof verification.
Proof generation involves running extensive computations to generate a proof of execution, which is used to remove the need for trust in the prover.
Instead, anyone can run a simple process on the proof to verify the integrity of the process run by the prover.
This mental model allows a business to run a process (often a complex one) and allows customers to trust the execution of said process without repeating the process.
Let's take an example:
Let's say you subscribe to OpenAI's paid plan to use one of their Large Language Models (LLMs), such as chatGPT. You have to trust OpenAI to actually run the specific model you requested, not replace it with a simpler, less efficient model. What if OpenAI could send you a small amount of data proving that it actually ran the specific model you requested? Furthermore, imagine what would happen if every proprietary SaaS product could provide such a guarantee to its consumers?
This trust minimization is the promise of ZKP. For example, in Web 2, ZKPs can guarantee fair credit evaluation or fair insurance claim processing by ensuring that all customers use the same algorithm. ZK technology has not reached that level yet, because running a ZKP process is still relatively expensive. However, we are seeing companies like Modulus Labs building a technology that uses ZKPs to prove AI inferences.
Technical Requirements for ZKPs
At the technical level, an efficient ZKP system needs to achieve the following goals at the same time:
Reduce the computational complexity and delay of the proof system, that is, allow the prover to efficiently generate the proof and pass it to the verifier in the shortest time.
Achieving small proof sizes.
Realize efficient verification, that is, minimize verification cost.
In addition to these primary goals, some secondary goals may be required depending on the use case, such as:
Preserving the privacy of data in privacy-focused applications means that the proof system can handle private inputs that are not revealed in the generated proofs.
Avoid trust settings where possible to simplify security assumptions.
Proof recursion is implemented to further reduce verification costs, that is, a single verification can verify multiple proofs and share the cost among different proofs.
Achieving all of these goals simultaneously is challenging. Depending on the use case, a ZKP system prioritizes some of these goals. For example, the SNARK proof system can produce concise proofs, but the proof complexity will increase. On the other hand, STRAK has an efficient prover, but the proof size may be 100 times larger than SNARK. zk researchers are constantly striving to push the frontier of technology, improving three indicators simultaneously by inventing new proof mechanisms.
Comparison of Different Proof Systems
For developers building ZKP-related products, an important question to consider is how to choose the underlying proof system, there are several ZKP proof implementations, and many more are in the research and development stage.
ZKP backend selection depends not only on technical aspects, but also on the target product. Take the Rollup selection proof system as an example. Key characteristics of Rollups, such as withdrawal times, transaction costs, and even degree of decentralization, will be largely determined by the ZKP proof architecture, as shown in the table below.
In Rollup, attestation happens on the business side, ie through Rollup operators. Existing ZK Rollups (zkRUs) such as Starknet and Zksync currently use centralized provers. Therefore, they can delegate proofs to dedicated provers, i.e. proof-as-a-service companies, to improve prover performance. Proof times can be reduced to minutes for Ethereum-compatible zkEVMs by specializing and utilizing optimized software/hardware. For example, the proof time for Polygon zkEVM is currently around 2 minutes. A few minutes of proof time, i.e. withdrawal delay, is acceptable for Rollup.
On the other hand, some use cases require proof that it happened on the client side, generating private transactions, such as Tornado Cash transactions. To ensure a reasonable user experience, proof times should not exceed a few seconds. Also, since users perform these calculations in the browser using wallets or on resource-constrained devices, it is important to choose a proof system with a fast prover. A good example is Zcash changing its proof system to Groth 16 in the 2018 Sapling upgrade, a major factor in improving the speed of shielded transactions.
comparative proof system
In general, it is difficult to obtain an accurate comparison of the performance of different proof systems, especially for proof and verification speeds, since they depend on the library implementation, the cryptographic curve chosen, and the hardware used.
The Mina team provides a nice high-level comparison in this post. There have also been efforts to create benchmarking tools for different ZK systems.
This table provides a good comparison of SNARK implementations and describes their progress in speed from Groth 16 to Plonk to Halo. Despite the progress, STARK still wins in proof speed, but at the cost of a larger proof size. The table also discusses two important features of proof systems: setup trustlessness and circuit programmability.
The pre-processing phase of circuit creation is discussed in the Setting up trustless section. Some proof techniques require the participation of multi-party computations to generate secret random numbers in the preprocessing stage. If the individual participants are honest, the generated random numbers are indeed secret and the preprocessing part is secure. This process is called "trust setup" because it trusts at least one of the preprocessing stage participants to be honest. Requiring a trust setting is considered a weakness. In this sense, STARKs and newer SNARK systems like Halo 2 have an advantage. However, some projects use trust settings as a tool to engage the community, such as Aztec and Manta.
The Programmability section discusses whether proof systems can prove arbitrary computations. SNARKs can generally be programmed for any computation. However, proof efficiency depends on the type of computation performed. For some types of STARK systems, it is not an easy case to adapt to different types of computations.
How to leverage ZKP for your product?
Building a product that can benefit from ZKP techniques is not easy and requires the right mental model.
This section attempts to provide a framework for developers to choose the best approach to integrate ZKPs into their products. Depending on product needs, ecosystem alignment, and performance requirements, there will be several tools from which developers can choose. Some developers will be able to reuse their existing code, while others will have to learn new domain-specific languages (DSLs) to create their applications.
performance-focused zk applications
Developers can use ZKPs to achieve higher throughput (TPS) or lower fees by off-chaining most application computations and only publishing proofs on-chain. In this case, there are several frameworks to choose from. Each framework provides a set of tools for compiling application code, generating ZK circuits, implementing ZK provers, and generating verifier code for the target ecosystem. We can divide these frameworks into two main categories: EVM-centric and non-EVM.
EVM-based ZK framework
This set of ZK frameworks is aligned with Ethereum, as Rollup is built on top of it. Transactions and applications are executed on Rollup's ZK Virtual Machine (zkVM). Proofs are generated by dedicated certifiers and published to L1 to be verified by smart contracts.
The first subset of this group implemented zkVM compatible with the EVM, hence the name zkEVM. The goal of these is to minimize friction by allowing Ethereum developers to use Solidity and familiar tools such as Hardhat and Foundry. They abstract ZK complexity by creating circuits and provers suitable for the EVM. Both Polygon zkEVM and Scroll are included in this bucket.
The second subset of this group is zkVM which is not natively compatible with the EVM. Despite the incompatibility, this group reduces friction by creating an intermediate layer that allows developers to use Solidity. Vitalik calls this type type-4 zkEVM. zkSync Era and Starknet are good examples of this group. The advantage of using Type-4 zkEVM is that it can provide higher throughput and lower cost than compatible EVM types. This makes them suitable for building high-throughput applications such as on-chain gaming or high-performance financial products such as order book DEXs.
Building applications for type-4 zkEVM requires more developer effort because there are restrictions on the Solidity code that can be used. Alternatively, developers may decide to learn another language, such as Cairo, to develop native applications against these frameworks.
Non-EVM zk framework
Another type of framework is those that do not target EVM architectures, as they target competitive L1 or general-purpose computing. Nonetheless, they can still be used to build application-specific zkRUs on Ethereum through specialized SDKs such as Sovereign.
There are two methods here:
Developers write code in a high-level language that targets a specific VM architecture, which is later compiled into a ZK circuit.
Developers use a domain-specific language (DSL), such as Circom, to generate ZK circuits directly.
The former method is more suitable for developers, but usually results in larger circuits and takes longer to prove.
Privacy-focused ZK applications
Developing privacy-focused applications using ZKPs is generally a more challenging task for developers. Developing a privacy-focused solution using ZKPs is less work than a scalability-focused solution, making for a steeper learning curve. Existing privacy applications mainly focus on payment privacy and do not allow much programmability. Combining privacy and programmability is a challenging task. Privacy-focused applications follow one of two implementation options:
1. Build on top of general-purpose L1
To enable private payment applications on L1, ZKP logic needs to be built as smart contracts. The application often uses ZKPs to create private capital pools. Users use these private pools as mixers, funding new wallets not linked to their original wallets. A famous example of this is Tornado Cash. For these applications, proof is performed by the user and verification happens on-chain. Therefore, it is crucial to use a ZKP system that has fast proofs, simple verification computations, and does not reveal any user information.
Since Universal Chain is not optimized for expensive cryptographic computations, verification costs are often expensive for mainstream users, limiting the adoption of these applications. The intuitive solution of moving a private affairs application to Rollup to reduce gas bills can present challenges. In this case, the private transaction proof needs to be included in the Rollup proof, that is, proof recursion, and the current general zk Rollup on Ethereum cannot achieve proof recursion.
2. Build a new L1/L2 that focuses on privacy
To keep costs down for private transactions and applications, developers have had to build a new privacy-focused L1 (e.g. Manta Network and Penumbra) or specialized Rollup (e.g. Aztec). Most privacy-focused chains are still unable to support general-purpose computing and focus on professional use. For example, Penumbra and Renegade focus on private transactions. Aleo is building a framework for proprietary applications by creating a special-purpose language, Leo, to compile programs written in high-level languages to generate corresponding ZK circuits. The interaction of the application is done off-chain, and only the proofs are published on-chain as private transactions. Aztec is moving in a similar direction, but as Ethereum L2. They recently announced a focus on creating generalized private Rollups that use Noir as the default smart contract language.
ZK Acceleration
After a developer has chosen the right ZK development framework for their application and selected an underlying proof system, the next step is to optimize the performance of the application and find ways to improve the user experience. This usually comes down to improving prover performance and latency. As discussed earlier, for Rollup, shorter proof times mean less delays in committing proofs to L1, and therefore less delays in withdrawals. For user-generated proofs, i.e. privacy applications, faster proofs mean shorter transaction generation times and better user experience.
As we discussed in previous articles, accelerating the proof process often requires software optimizations and specialized hardware. The competition for specialized hardware has intensified over the past few months, with multiple companies entering the race. In this section, we discuss the current state of ZK acceleration and the ways in which developers can benefit from this race.
Proof as a Service
Hardware Acceleration
Hardware Acceleration
With the rollout of several L1s and Rollups that need to efficiently generate zk proofs, the competition to generate these proofs and earn corresponding rewards will become intense. If these chains and L2 succeed in attracting a large number of users, proof generation could develop into an arms race similar to the Bitcoin mining competition. There are different approaches to ZKP acceleration, GPU vs FPGA vs ASIC. This article from the Amber Group has a good discussion of these different options and the challenges faced with each implementation option. However, in the long run, the company that produces the most efficient ASICs for proof-of-concept generation will have a significant economic advantage on zk-focused chains.
It is worth noting that there is an important difference between proof-of-ZK competition and Bitcoin mining that is worth highlighting. In Bitcoin, the mining process is based on a simple calculation, the SHA 256 hash. This calculation is fixed and not easily changed, so the focus is on chip design innovation and access to the most advanced semiconductor nodes. In the field of ZKPs, there is a significant split between different proof protocols. Even with the same proof backend, such as Plonk, the target circuit size can lead to differences in ASIC performance. This distinction between Bitcoin mining and ZKP generation could lead to multiple winners, each specializing in a different ZK backend.
There are multiple players entering the ZK specific chip space. Each participant focuses on one of two main operations that improve proof generation: multiscalar multiplications (MSMs) and number-theoretic transformations (NTTs). The last player to come out of stealth was Cysic, which announced its $6 million seed round during ETH Denver. Cysic focuses on accelerating MSMs by using FPGAs. The flexibility of FPGAs enables them to support different ZK systems. The approach is similar to Ulventanna, which announced a $15 million seed round in January. Other players in the ZK chip development space include Ingonyama, which released a library called Icicle that accelerates computations for MSM and NTT computations on GPUs, as well as Accseal, Snarkify, and Supranational. In addition to this list, there are other cryptic companies and research efforts by well-known players in the Web3 space. Examples of the latter include Jump Crypto's CycloneMSM implementation for accelerating MSM computations using FPGAs, and Jane Street's FPGA implementation for accelerating MSM and NTT.
Due to the increasing importance of ZKP acceleration, competitions (such as ZPrize) that require fair evaluation of different implementations are becoming important venues to move the field forward. The 2022 tournament has more than $4 million in prize money.
With the rollout of several L1s and Rollups that need to efficiently generate ZK proofs, the competition to generate these proofs and earn corresponding rewards will become intense. If these chains and L2 succeed in attracting a large number of users, proof generation could develop into an arms race similar to the Bitcoin mining competition. There are different approaches to ZKP acceleration, GPU vs FPGA vs ASIC. This article from the Amber Group has a good discussion of these different options and the challenges faced with each implementation option. However, in the long run, the company that produces the most efficient ASICs will have a significant economic advantage on a ZK-focused chain.
It is worth noting that there is an important difference between proof-of-ZK competition and Bitcoin mining that is worth highlighting. In Bitcoin, the mining process is based on a simple calculation, the SHA 256 hash. This calculation is fixed and not easily changed, so the focus is on chip design innovation and access to the most advanced semiconductor nodes. In the field of ZKPs, there is a significant split between different proof protocols. Even with the same proof backend, such as Plonk, the target circuit size can lead to differences in ASIC performance. This distinction between Bitcoin mining and ZKP generation could lead to multiple winners, each specializing in a different ZK backend.
There are multiple players entering the ZK specific chip space. Each participant focuses on one of two main operations that improve proof generation: multiscalar multiplications (MSMs) and number-theoretic transformations (NTTs). The last player to come out of stealth was Cysic, which announced its $6 million seed round during ETH Denver. Cysic focuses on accelerating MSMs by using FPGAs. The flexibility of FPGAs enables them to support different ZK systems. The approach is similar to Ulventanna, which announced a $15 million seed round in January. Other players in the ZK chip development space include Ingonyama, which released a library called Icicle that accelerates computations for MSM and NTT computations on GPUs, as well as Accseal, Snarkify, and Supranational. In addition to this list, there are other cryptic companies and research efforts by well-known players in the Web3 space. Examples of the latter include Jump Crypto's CycloneMSM implementation for accelerating MSM computations using FPGAs, and Jane Street's FPGA implementation for accelerating MSM and NTT.
Due to the increasing importance of ZKP acceleration, competitions (such as ZPrize) that require fair evaluation of different implementations are becoming important venues to move the field forward. The 2022 tournament has more than $4 million in prize money.
useful educational resources
In this section, we've put together some educational resources to help developers understand ZKP spaces. This is by no means an exhaustive list, as there is a lot of excellent content on the subject. A comprehensive list with all zk resources can be found here and here. It's a friendly way of setting up and letting developers know about the efforts in this space.
For those interested in understanding the basics of ZKPs and how they work, one of the first resources to check is ZK Whiteboard Sessions at ZK Hack. Dan Boneh's three introductory sessions in particular are high-level for anyone with some mathematical understanding. The remainder of the series deals with specific topics in the field.
in conclusion
in conclusion
Original link
