Foresight Ventures: Comprehensive analysis of EIP-4844, Ethereum's first step towards Full sharding

first level title

Overview

Ethereum

EIP-4844 ；

Proto-danksharding；

Danksharding；

KZG Commitment；

KZG Ceremony；

Dynamic state sharding；

Recently, the KZG Ceremony, which has exceeded the standard of community participation, has been swiped. What is this ceremony doing?

first level title

1. Sharding: A Long-term Solution for Ethereum Expansion

• Sharding expands from the perspective of data availability, while rollup mainly expands from the execution level to alleviate the congestion problem of the main network; I think sharding may be an attempt to break through the impossible triangle of the blockchain.

• Ethereum

• Due to the complexity and security considerations of the implementation process, the Ethereum development team divided sharding into multiple stages, including the recently mentioned proto-danksharding and danksharding. The whole process will be an update that will take several years;

• first level title

2. EIP-4844: Short-term high returns, a streamlined version of Sharding

EIP-4844 = Proto-Danksharding；

Proto comes from the name of an Ethereum researcher

secondary title

1. Overview of Proto-danksharding

Proto-Danksharding mainly introduces a new transaction type, namely blob-carrying transaction. So far, Rollup data is transmitted to layer 1 at a lower cost in the form of blobs and stored non-permanently. At the same time, the blob is much larger than the current calldata, which can better support the high TPS on the rollup.

About blobs:

• Each transaction can hang up to 2 blobs;

• Each block ideally contains 8 blobs, which is about 1 MB, and can contain up to 16 blobs, which is about 2 MB;

• Blob does not need to be permanently stored as a history log like calldata;

• Compared with danksharding, nodes still need to verify the complete DA.

2. Interpretation of Blob-carrying transaction

Ethereum

effect

cost

cost

Data blobs are originally designed to support rollup high-throughput transactions. Compared with calldata of the same size (using on-chain storage), the cost of blobs will be much lower (no permanent storage required). Therefore, the gas spent by rollup to maintain data availability will be significantly lower than before.

capacity

Each blob is about 125 kB in size (fact: the current average block size is only ~90 kB).

3. The value and challenges of Blob-carrying transaction

• value

The blob can be regarded as a kind of cache, and the transaction data submitted by rollup exists in the form of cache from then on. Reduce storage hardware requirements, provide additional data expansion for Ethereum and reduce gas costs.

• Challenge: Requirements for Ethereum node hardware performance

Ethereum's current average block size is only ~90 kB, but a blob is ~125 kB

According to the design of EIP-4844, each slot is normally about 1 MB, so the amount of data increased per year is: 1 MB/block * 5 block/min * 43200 min/month * 12 month/year = 2.47 TB per year

solution

• solution

From the short-term expansion effect, because each node still needs to store the full amount of historical data, while implementing EIP-4844, for blobs outside the period of time (the specific limit time has not been finalized, it may be 1 month or 1 year) to carry outautomatically delete；

From the perspective of long-term benefits of sharding, EIP-4444 is implemented, that is, nodes do not need to store the full amount of historical data, but only need to refer to the history expiry to store data after a specific time;

These two solutions alleviate the tradeoff of blob-carrying transaction in storage space to varying degrees.

4. KZG Commitment

KZG Commitment is a polynomial commitment scheme adopted in EIP-4844

Parsing KZG commitments

Ethereum

Referring to the schematic diagram in Dankrad Feist's speech, the KZG root is similar to the Merkle root, the difference is that the KZG root promises a polynomial, that is, all positions are on this polynomial. Based on the proto-danksharding scenario, the KZG root promises a bunch of data, any of which can be verified to belong to the whole.

This is why the KZG commitment is more friendly to the later implementation of DAS in terms of compatibility.

The process of KZG commitment is as follows:

• Prover: Provide proof and calculate the commitment of data. The prover cannot change the given polynomial, and the commitment used for proof is only valid for the current polynomial;

• Verifier: Receive and verify the commitment value sent by the prover to ensure that the prover provides a valid proof.

Advantages of KZG Commitment

cost

cost

• KZG commitment has the characteristics of fast verification, relatively low complexity and simplicity;

• There is no need to submit additional proofs, so the cost is lower and the bandwidth is saved;

• Safety

Safety

• Assuming a failure occurs, it will only affect the data in the blob corresponding to the commitment, without other far-reaching effects.

more compatible

• Looking at the overall solution of sharding, KZG commitment is compatible with the DAS solution, avoiding the cost of repeated development.

5. KZG Ceremony(trusted setup)

KZG Ceremony is a trust setup for KZG CommitmentEthereum

Recently, KZG Ceremony, which is enthusiastic about community participation, provides trust setup for the KZG commitment adopted by EIP-4844;

Ethereum

Referring to Vitalik's flow chart, anyone can contribute a secret as a participant and mix it with the previous result to generate a new result, and so on, to obtain the final SRS in the form of nesting dolls, and assist in completing the trust setup of the KZG commitment

trust setup

• A common multi-participant trust setup, namely powers-of-tau, is adopted in EIP-4844;

• Following the 1-of-N trusted model, no matter how many people participate in the generating setup process, as long as one person does not disclose his own generation method, the trusted initialization is valid;

necessity

• The trust setup of KZG commitment can be simply understood as: generate a parameter that needs to be relied upon each time the cryptographic protocol is executed, similar to zk-snark that requires trusted initialization;

• When Prover provides proof, KZG commitment C = f(s)g 1 . Among them, f is the evaluation function, and s is the final secret finally obtained by KZG trusted setup;

• It can be seen that the final secret is the core parameter for generating polynomial commitments, and as a credible process for obtaining this core parameter, this KZG Ceremony is very important for the realization of the entire sharding.

6. Changes brought by EIP-4844

Rollup

Referring to the schematic diagram given on ethresear, rollup needs to include state delta and versioned hash of KZG commitment in calldata for submission (zk-rollup also needs to submit zk proof)

It can be found that the difference is that the calldata only contains some small data, such as state delta, KZG commitment, and the transaction batch containing a large amount of transaction data is placed in the blob.

• Effectively reduce costs, it is very expensive to put it in calldata;

• safety

safety

• Data availability: Blobs are stored on the beacon chain, equivalent to layer 1 security;

• cost

cost

Proto-Danksharding introduces a new type of transaction. The addition of low-cost data format blob will undoubtedly further reduce the cost of rollup. Depending on the actual application and implementation progress on the chain, the cost of optimized rollup may be reduced by x 10 or even x 50;

At the same time, EIP-4844 introduced blob fee;

• Gas and blob will have adjustable gas price and limit respectively;

• The charging unit of blob is still gas, and the gas amount changes with the traffic, so as to maintain the target of 8 blobs per block on average (limiting the amount of additional data)

Implementation of Precompile

The data in the blob itself cannot be directly accessed, and the EVM can only obtain the commitment of the data blob. Therefore, rollup needs to provide precompile to verify the validity of the commitment.

The following analyzes the two precompile algorithms mentioned in EIP-4844

Point evaluation precompile (for those who are interested in mathematical principles, refer to the analysis of vitalik)

• Prove that multiple commitments point to the same data;

• Mainly for zk-rollup, rollup needs to provide 2 kinds of commitments: 1. KZG commitment; 2. zk-rollup's own commitment;

• For optimistic rollup, most of them have adopted the mechanism of multi-round fraud proof, and the amount of data required for final round fraud proof is small. Therefore, using point evaluation precompile can achieve lower cost

Blob verification precompile

• Prove that versioned hash and blob are effectively corresponding;

• Optimistic rollup needs access to the full amount of data when submitting a fraud proof, so first verify that the versioned hash and blob are legal, and then perform fraud proof verification

3. Danksharding: an important step towards full sharding

Danksharding is named after Ethereum researcher Dankrad Feist

1. Capacity expansion: further expand the cache space of Rollup

After proto-danksharding is implemented, each block can get an additional 1 MB of cache space on average due to the introduction of blobs in the new transaction format. After Danksharking is implemented, an additional 16 MB per block, with a maximum allowed of 32 MB.

2. Data availability: storage and verification strategies are more efficient

Compared with proto-danksharding, which requires all nodes to download the full amount of data, after Danksharking is implemented, Ethereum nodes only need to sample blobs. The data after sampling will be distributed in the nodes of the whole network and can form a completed data blob.

DAS: Efficient Sampling Inspection

• Through the entanglement code (erasure coding) to help the nodes of the whole network to find the probability of loss of the original data more easily when downloading part of the data, thereby improving security

3. Security: basically unchanged

Since each node no longer saves the full amount of historical data, from the perspective of data availability, backup, and sampling inspection, the security changes from at least one node storing full amount of data to multiple nodes storing part of the data, and finally restoring the complete data.

Although at first glance the dependence on a single point is far more secure than the dependence on multiple points, the number of nodes in the Ethereum network is large enough to fully guarantee the need for data backup, so the security will not change much.

4. New challenges: increased requirements for block builders

Ethereum

Here is a brief introduction to PBS (proposer/builder separation), refer to the application diagram of PBS in the darksharding scheme given by Dankrad:

• Split the role of the person in charge of producing blocks into proposer and builder. It was originally a proposal for anti-MEV design, in order to reduce the bandwidth requirements for block creation in the design of danksharding.

4. Other sharding solutions: dynamic sharding of Shardeum

Advantage

1. Dynamic state sharding

Advantage

The most intuitive advantage brought by Dynamic state sharding is linear scaling. Nodes connected to the network can be dynamically grouped by the sharding algorithm very efficiently and respond quickly, improving the TPS of the blockchain network. In the design of dynamic state sharding, each node will cover addresses of different ranges, and the coverage will have a redundant design to ensure efficient sharding and security.

In-scene implementation

Regardless of the complexity of the ecology, from the perspective of the implementation of the two technologies, dynamic state sharding is more difficult than static sharding. It can be seen that Shardeum's technical team has a deep accumulation in the technical level of sharding. The team's previous research and development on Shardus technology has also made a great contribution to the underlying technology of this public chain, even in the early stage of the project. Demonstrated the linear scaling brought by dynamic state sharding.

product

product

team

team

technology

technology

schedule

schedule

It will launch betanet on 2023-02-02, which is worthy of attention.

5. Thinking and prospect of sharding

• Sharding is a long-term plan for the expansion of Ethereum, and it is also a long-term plan with great value and far-reaching significance. In the process of implementing sharding, all existing solutions may be continuously iterated, including the proto-danksharding, danksharding, etc. mentioned now, which deserve continuous attention;

• Understanding the general direction of sharding is very important, but the technical solutions (PBS, DAS, multidimensional fee market) adopted by each proposal in the process of implementing full sharding are also worthy of attention, and I believe that many related excellent teams and projects will emerge ;

• Sharding is a general term for a kind of expansion technology, but there is not only one specific solution. It is necessary to realize that different public chains have sharding solutions suitable for their own scenarios. For example, some designs in danksharding are only suitable for the Ethereum network, and the security tradeoff requires a large number of nodes to offset;

• The reasonable combination of Sharding and other expansion solutions can improve scalability 1+ 1>2. The current Danksharding is not a self-contained expansion solution, but complements other implementations of the Ethereum ecosystem. For example, Danksharding and rollup work together to achieve better results for the expansion of Ethereum.