Overview:
In the market, DA (Data Availability) projects have sprung up like mushroom after rain, among which competitive projects include Celestia, EigenLayer, Avail, NearDA, and Covalent.
According to the analysis by the Huobi Research Institute, the core content of DA is not that difficult. The simplest standalone can be used for DA, and the most complex one, like Celestia, uses a sampling model for decentralization. In simple terms, DA is storage, and it is costly. If you don't strictly require Ethereum-level security services, choosing which DA to use is a balance between cost and security.
The most important principle when using the DA layer is: The greater the value of the service, the more secure the DA should be.
This research report from Huobi Research Institute analyzes the background, ecosystem, and future prospects of the Data Availability War, including a review and summary of various DA projects according to Vitalik Buterin's perspective. Based on a comprehensive analysis of the DA track, Huobi Research Institute believes that the future DA may be decentralized, and that there may be enough market demand for 7-8 major DAs.
1. Emergence of Data Availability Issues
1.1 What is DA
In simple terms, data availability refers to the action of block producers publishing all transaction data from a block to the network, so that validators can download it. If a block producer publishes complete data and allows validators to download it, we say that the data is available; if the producer withholds some data, making it unavailable for download by validators, we say that the data is unavailable.
1.2 Two Key Aspects of DA: Security and Cost
Therefore, based on the definition of DA, we can identify the following two aspects within DA:
First, ensuring the security of the validation mechanism. Second, reducing the cost of publishing data.
￮ Ensuring the security of the verification mechanism
To ensure the security of the verification process, current L2 sequencers widely publish the state and transaction data of L2 on the more secure Ethereum network, relying on Ethereum for settlement and data availability.
Therefore, the data availability layer is essentially the place where L2 publishes transaction data, and most mainstream L2 solutions use Ethereum as the data availability layer.
￮ Reducing the cost of data publishing
L2 simply executes data availability and settlement on Ethereum, which, although provides sufficient security, also incurs significant costs. This is the second issue that L2 faces, which is how to reduce the cost of data publishing.
2. Cost composition in DA & directions for cost reduction and efficiency improvement
To make L2 more affordable, the cost of publishing data needs to be reduced. So, how can we reduce the cost? There are mainly two methods:
￮ Reduce the cost of publishing data on L1, such as the upcoming EIP-4844 upgrade on Ethereum.
￮ Separate transaction execution and data availability from L1, similar to Rollup, to reduce costs. In other words, not using Ethereum as the data availability layer.
Therefore, various parties have made great efforts to reduce costs. Among the current DA solutions, Near DA has the lowest cost, approximately $0.0016/block. Celestia, EngenLayer, EIP-4844, etc. are ranked next.
3. DA in Vitalik's Eyes
3.1 Not using Ethereum's DA is not a true Layer 2 solution - Ethereum Foundation and Vitalik's counterattack
After Celestia's explosion, Vitalik hinted that "the layer 2 projects on Ethereum must use data availability on ETH." Subsequently, Dankrad Feist, a member of the Ethereum Foundation, also made a statement. He pointed out that not using ETH as the DA layer is not a Rollup, nor is it Ethereum Layer 2.
In this case, both Arbitrum Nova and Mantle will be "delisted" from the Layer 2 list because they only disclose transaction data outside of ETH (in a off-chain DA network called DAC).
At the same time, Dankrad also said that solutions like Plasmas and state channels that do not require on-chain data availability for security are still considered Layer 2, but Validium (ZKRollup without using ETH as the DA layer) is not considered Layer 2.
3.2 If not using Ethereum as the DA, it is Ethereum Validium - Vitalik's compromise
Later, in a reply on his Twitter stream, Vitalik mentioned, "being a validium is a correct choice for many apps, and using good distributed DA guarantee systems can be a good way to iIncrease the practical security of a validium"
At the same time, he believes that the core of rollup is an unconditional security guarantee: even if everyone is against you, you can still withdraw your assets. If data availability depends on external systems, this guarantee cannot be obtained.
3.3 About ENS and Data availability - -V God wants to consolidate the control of DA through ENS
The ENS domain name service will define a set of interaction logic. Users only need to enter a short domain name to automatically connect to the associated long address of the ENS smart contract, solving the pain points of complex and non-distinguishable EOA addresses. It should be said that this set of domain name services of ENS is more aimed at the future expansion market with larger user traffic, especially some user groups with mass adoption. Layer 2 is the future of Ethereum's expansion and absorption of heavy traffic.
Vitalik believes that if the domain name resolution solution of ENS cannot cover layer 2 and only stays at the Ethereum mainnet level, it is difficult to open up the imagination space. Based on this background, Vitalik explained the importance of ENS on Twitter, "it needs to be affordable!". ENS will naturally consider providing a complete set of data resolution solutions for layer 2, allowing users to directly perform domain name resolution and data retrieval on layer 2, reducing their dependence on each layer 2's biased decentralized gateway.
It is not difficult to see that in order for users to use ENS domain names on layer 2, the precondition is to call and verify the global data on the Ethereum mainnet. This means that to enjoy this set of services provided by ENS, orthodox Ethereum Data Availability capabilities must be adopted, and layer 2 solutions like OP Stack that take shortcuts and place DA on third-party DA platforms like Celestia will not be compatible with ENS. With this in mind, it is not difficult to understand Vitalik's intention. In short, Vitalik's deeper meaning is to use ENS to establish a set of interactive operational standards for the various platforms on layer 2, while consolidating DA control.
3
.4 V God talks about the return of Plasma
a. Vitalik mentioned in the article that Ethereum's layer 2 scaling solutions originally included various options such as Plasma, Rollup, Validium, Parallel. Vitalik expects the scaling direction to be balanced, with diversified layer 2 constructions that can adapt to various application scenarios. However, the current reality in the market is that the Rollup solution dominates and is increasingly self-contained.
b. Plasma is equivalent to a sidechain solution that regularly synchronizes Merkel state data with the mainnet. It is a scaling solution that relies on the mainnet for data and computation. In this way, layer 2 can efficiently scale using a highly centralized and complex ledger model, while also reusing the system capabilities of the mainnet validators. Vitalik's new article brings back Plasma and introduces a scaling solution called ZK+Plasma, which is once again a matter of layer 2 politics.
3.5 Summary
Summarizing Vitalik's various actions:
The demand for DA is too strong, but they don't want to give the market to Celestia. First, concerning security, they brought up the issue of ENS but found that the market didn't buy it. Many people still use the first-party DAs, so they consider using Validium. However, after a few days, they bring up the outdated Plasma again and intentionally guide the market towards exploring the direction of ZK+Plasma. Essentially, their goal is to continuously attract the DA market to Ethereum.
4. DA Solution and DA Project Planning
4.1 DA Solution
From the above content, we can see that there are many solutions for the DA layer. Broadly speaking, it can be divided into two major parts: on-chain and off-chain.
• Off-chain Solution
This refers to the approach where L2 still uses Ethereum as the DA layer and relies on Ethereum to reduce data availability cost. This means that Ethereum will be a real-time bulletin board in the future, where the data on the board will be deleted after a period of time. L2 must find a way to store backups of all data.
• On-chain Solution
This means no longer using Ethereum as the DA layer and instead finding a more economical way to achieve data availability. According to decentralized methods.The different types of off-chain solutions can be categorized into four: Validium, Data Availability Committee (DAC), Volition, and Generalized DA solutions.
4.2 Celestia
Celestia is the pioneer of modular public chains, developed based on the Cosmos SDK, focusing on data availability. It is a competitive DA-leading project that has already launched its mainnet.
• Technical Features
￮ Data Availability Sampling (DAS)
DAS allows light nodes to verify data availability without downloading the entire block. Light nodes, which cannot verify data availability by downloading only block headers, use a 2-dimensional Reed-Solomon encoding scheme to re-encode block data for DAS. The working principle of Data Availability Sampling (DAS) is to let light nodes randomly sample a small portion of block data in multiple rounds. As light nodes complete more rounds of block data sampling...Increase confidence in the availability of data. Once the light node reaches the predetermined confidence level (e.g. 99%), the data is considered available.
￮ Namespaced Merkle trees (NMT)
NMT enables the execution layer and settlement layer on Celestia to only download transactions related to them. Celestia divides the data in blocks into multiple namespaces, each corresponding to an application built on Celestia such as rollup. Each application only needs to download data relevant to itself to improve network efficiency.
• Celestia primarily earns revenue from applications in two ways:
￮ Paying blob space fees: Rollup uses $TIA to publish data to Celestia's blob space.
￮ Paying gas fees: Developers use $TIA as gas for Rollup.
Gas token, similar to ETH based on Ethereum Rollup .
• Development potential
• The project has been launched and has a high level of technical maturity
• Potential abundant airdrops, pledging from $TIA, for example, projects like Dymension and Altlayer will choose TIA validators for their airdrops. In the future, more Ethereum Layer 2 projects, modular blockchains, and Cosmos ecosystem projects are likely to take a similar airdrop approach.
• Rich ecosystem: collaborations with cross-chain bridges, settlement layer solutions, defi projects, games, sorters, and more.
• DA collaboration projects continue to increase, including Manta, Eclipse, Caldera, Snapchain. It is also collaborating with&+
EigenDA is a decentralized data availability (DA) service built on Ethereum using EigenLayer Restaking. It will be the first Active Validation Service (AVS) on EigenLayer. Unlike Celestia or Avail, EigenDA does not require bootstrapping a new set of validators. Ethereum validators can freely choose to join. - Technical features: - Enhances Ethereum's DA capabilities: Blob block data + KZGbsp;CommitmentEigenDA mobilizes the upgraded Blob block data after KanKun + KZG commitment. The Rollup chain can generate KZG commitment for the Blob Data and publish it to the EigenDA contract. EigenDA nodes ensure the DA capability of the subsequent chain, enhancing Ethereum's DA capability. The key is that the entire process of EigenDA revolves around the existing Ethereum infrastructure such as Blob and KZG, and the node validation work is also participated by Ethereum Validators.
￮ No autonomous consensus and P2P network
EigenDA nodes must redeposit ETH in the EigenLayer contract on Ethereum L1 (or more precisely, deposit ETH derivatives). EigenDA nodes are a subset of Ethereum validators. Afterwards, the DA buyer (e.g., rollup, also known as disperser) receives the data blob, encodes it with erasure coding, generates KZG commitment, and publishes and distributes it for node confirmation. Then, the disperser collects these signatures one by one, generates an aggregate signature, and publishes it to the EigenDA smart contract, which verifies the signatures.
￮ Adopting a custody proof method
EigenDA does not use data availability sampling to verify if nodes truly store data, but instead uses a delegated proof mechanism for implementation. Anyone can submit a proof to the EigenDA smart contract, which will then be verified by the contract. If the verification is successful, the lazy validator will be reduced.
• Development Potential
￮ Multiple cooperative projects competing with Celestia: Already integrated with various L2 projects like Celo, Mantle, Fluent, Offshore, OP stack, etc;
￮ Supported by diverse Eigenlayer ecosystem, including sorters, cross-chain bridges, oracles, etc.
4.4 Other DA Projects
4.4.1 Avail
Avail provides efficient sorting and recording of transactions, storage of data, and validation of data integrity. It supports blockchain compatible with the Ethereum Virtual Machine (EVM), allowing Rollup to directly publish data to Avail. The lightweight client network verification mechanism on Avail enables Rollup on Avail to verify states without relying on smart contracts and the underlying layer. In addition, due to its modular nature, developers can store data on Avail and choose other networks for settlement.
• Consensus Mechanism: Inherits the BABE and GRANDPA consensus mechanisms from Polkadot SDK
• Decentralization:
￮ Avail adopts Polkadot's NPoS (Nomination Proof-of-Stake), supporting up to 1000 validator nodes. NPoS also has an effective reward distribution that reduces the risk of centralization.
￮ Avail has the ability to sample data from its lightweight client's P2P network, providing an efficient and reliable backup mechanism for Avail to ensure data availability even in the event of failures.
• Validity Proof: Avail adopts KZG polynomial commitment
Current Status: Mainnet is not yet live
4.4.2 Near DA
On November 8, 2023, NEAR Foundation announced the launch of NEAR Data Availability (NEAR DA) layer, which provides powerful and cost-effective data availability for ETH rollup and Ethereum developers. The initial users include Madara, Caldera, Fluent, Vistara, Dymension RollApps, and Movement Labs from StarkNet.
• Security: Inherits the security of the Near network
• Cost advantage: 100 kB calldata on NEAR costs $0.0033
• Current Status: NEAR DA has integrated with Polygon CDK for developers to build Ethereum ZK Rollups.
4.4.3 Covalent
Covalent, a blockchain data query service platform, can standardize data from multiple blockchains. Its unified API allows developers to reuse queries in supported networks, solving the problem of difficult access to blockchain data.
After the Cancun upgrade, the Ethereum mainnet only keeps the states data submitted by L2 for one month, after which it will be discarded. Celestia, in order to maintain the decentralization of the network and the establishment of the DAS lightweight node mechanism, will also periodically discard the states data submitted by L2. However, Covalent launched the long-term DA service EWM (Ethereum Time Machine) at the end of last year, which permanently saves the L2 states data discarded by Ethereum. Covalent is responsible for reading L2 states data.
Moreover, Covalent indexes and structures these data, integrates them into its platform's on-chain data API service, and provides services and support for professional blockchain data websites, government regulatory departments, and AI research teams.
• Status: As of December 2023, Covalent supports over 210 blockchains and plans to support over 1000 blockchains by the end of 2024. According to a recent data availability report by Messari, Covalent has billions of data points and is currently the best service provider for applications that require general and widely applicable data.
4.4.4 zkPorter
zkPorter is an off-chain data availability solution optimized for decentralization, launched by Ethereum scaling solution zkSync. It combines the methods of zkRollup and sharding to handle data availability. zkPorter introduces an optional validator mechanism, where zkSync token holders can stake tokens to validate and sign blocks. Although the progress of zkPorter's product has not been disclosed to the public for a long time, the Layer 3 application chain GRVT on zkSync plans to launch its mainnet in Q1 this year. Based on the fact that GRVT stores data on zkPorter, it is speculated that zkPorter may be launched before that.
4.5 DA Project Comparison
Technical Aspect:
Celestia
Utilizes a data availability sampling scheme and employs a 2-dimensional RS erasure code to ensure data recoverability. Light nodes access block data through random sampling and submit data availability proofs optimistically.
Ethereum (Proto-Danksharding)
Also adopts a data availability sampling (DAS) scheme, with the same RS erasure code for data recoverability. However, Ethereum will use the KZG commitment scheme for data availability proofs. KZG (Kate Zaverucha Goldberg) polynomial commitment is a zero-knowledge proof system. Compared to Optimistic Rollup and ZK Rollup, Optimistic proofs have lower technical requirements, while KZG polynomial commitment has higher technical requirements but faster proof submission speed.
Avail
Also uses the Data Availability Sampling (DAS) scheme, the specific implementation is similar to Proto-Danksharding, which is an RS erasure code encoding scheme combined with KZG polynomial commitment.
EigenDA
The flagship product of Eigenlayer, its solution is derived from Proto-Danksharding like Ethereum, which uses the RS erasure code encoding scheme combined with KZG polynomial commitment.
Arbitrum Nova
The data availability implementation is similar toThe four projects mentioned above are completely different and are based on the Data Availability Committee (DAC) model, where an external committee is responsible for storing and providing transaction data. At least 6 out of 7 committee members submit BLS signatures to ensure the reliability of the data. Compared to the Data Availability Sampling (DAS) approach, the DAC model has lower costs but sacrifices decentralization and security. However, it does not compete with the aforementioned projects and only represents one solution for data availability.
zkPorter
combines ZK Rollup and sharding to address the data availability problem. It can support an arbitrary number of shards, and each shard can select and set its own data availability solution, providing greater flexibility for smart contract development.
Performance:
Summary
• The competitive projects among the above DA projects are Celestia, EigenLayer, Avail, and NearDA, while Covalent takes an unconventional approach and opens up unique demand for it in DA applications.
• Among all the data availability implementation solutions, data availability sampling with KZG polynomial commitment is the most mainstream solution. It can reduce node cost, improve proof efficiency, and ensure data availability.
• From a technical perspective, Ethereum Danksharding and Celestia are the most decentralized because they both use sampling techniques, which reduce the performance requirements of nodes while achieving high bandwidth. EigenDA is the next, also using sampling, but EigenDA parasitizes on Ethereum and its number of nodes is a subset of Ethereum. Other DA projects should not have used sampling, for example, the decentralization level of NearDA is equivalent to Near.Protocol's degree of decentralization.
• Celestia's chosen Optimistic Proof has a lower landing threshold compared to KZG polynomial commitment, with higher technical maturity. However, the future technical limit is not as good as KZG polynomial commitment. Compared to similar projects like Avail and EigenDA, Celestia's development progress is currently faster, and it will land on the mainnet earlier. However, Celestia will also face direct competition from Ethereum after the Cancun upgrade.
• As a Layer 2 developer, the tradeoff between DA orthodoxy and launch cost is inevitable. DA orthodoxy is relatively passive in the commercial market, suitable for comprehensive Layer 2 projects with more concern about security and consensus issues, and with certain brand foundation and market base. On the other hand, some emerging small Layer 2 chains, especially those based on OP Stack for quick launches, will try to minimize costs. For them, third-party DAs like Celestia are a better choice. Eigenlayer, however, is still unable to reduce the actual development cost of L2 for project parties.
• But for DA layer projects, with a single scenario and a B2B model, these projects cannot attract a large amount of liquidity like comprehensive public chains (such as Solana) by attracting DAPPs in ToC scenarios. If they fail to attract the adoption of many Rollup projects, their ecosystem construction will weaken. Moreover, Celestia seems to lack a very strongCapital pushers, although their technical narrative itself makes sense, if they cannot establish a strong foothold within the Ethereum ecosystem, their grand blueprint may well be a castle in the air.
5. DA Layer and Modularization of Blockchain
DA Layer has always existed, whether it be BTC, ETH, or Solana. For example, in the Bitcoin network, data is directly stored on the blocks. 15 years ago, Satoshi Nakamoto published the Bitcoin white paper and limited the block size to 1 MB, which also limited the amount of transaction data each block can accommodate. Later improvements such as SegWit, Taproot, and the Ordinal protocol were introduced to enhance the data availability of the BTC network.
In the pre-4844 Ethereum network, data transmitted from Layer 2 to Layer 1 is stored in Calldata. Although Rollup moves the computational security to L2, the storage still remains on L1. Due to limited L1 storage capacity, the capacity of Rollup on Ethereum is restricted. The Ethereum block size is approximately 150K - 250K, and even if all the space is allocated to Rollup, the storage space is still limited, which limits the throughput of L2. Therefore, Ethereum adopts Proto-Danksharding to enhance data availability by introducing new transaction types that include blobs.
The architecture of traditional monolithic blockchains usually consists of four functional layers:
￮ Execution Layer - The execution layer is mainly responsible for processing transactions and executing smart contracts. It includes transaction validation, execution, and state updates.
￮ Data Availability Layer - The data availability layer in modular blockchains is responsible for ensuring that data in the network can be accessed and verified. It usually includes functions such as data storage, transmission, and verification to ensure transparency and trust in the blockchain network.
￮ Consensus Layer - Responsible for the protocols between nodes to achieve consistency of data and transactions in the network. It uses specific consensus algorithms (e.g. proof of work) to ensure the security and validity of the blockchain network.
(PoW) or Proof of Stake (PoS) verifies transactions and creates new blocks.￮ Settlement Layer - responsible for completing the final settlement of transactions, ensuring the transfer of assets and recording them permanently on the blockchain, determining the final state of the blockchain.
Celestia first proposed the concept of modular blockchain, decoupling different functional layers from a monolithic blockchain, allowing the blockchain to focus on specific functions at each layer, enabling rational division of labor to maximize overall usability.
It is not to say that modular blockchain is necessarily superior to monolithic blockchain, but rather to consider the future development of blockchain with a modular mindset, which can open up more possibilities and speculations.
6. Speculations about the future
6.1 Viewing Blockchain Combination with a Modular Perspective
As mentioned earlier, looking at blockchain in a modular way, it can be divided into execution layer, data availability layer, consensus layer, and settlement layer.
For the Ethereum ecosystem, the number of possible modular blockchain solutions equals the number of solutions in the execution layer multiplied by the number of solutions in the settlement layer multiplied by the number of solutions in the DA (Decentralized Application) layer.
6.2 The DA Layer Will Be Decentralized
The DA layer can emerge rapidly, and the core concept is that DA is not as difficult as it seems. The simplest form of DA can be operated on a single machine, while the most complex form, like Celestia, involves decentralized sampling. Decentralization through sampling allows for greater bandwidth with more nodes (imagine a peer-to-peer movie downloading network), and this also exhibits network effects. Therefore, I don't think there will ultimately be too many forms of "decentralized sampling-based DAs." However, there are no restrictions on other forms of DAs, and there can be countless variations.
Here, we can make an analogy: DAs are like storage, which comes with high costs. If you don't strictly require Ethereum-level security services, choosing which DA to use becomes a balance between cost and security.
The principle is: the greater the value of the service, the more secure the DA should be.
Therefore, the future of DA might be decentralized. Even so, maybe 7-8 major DAs might be enough.
6.3 Ethereum may create a dedicated DA layer in ETH 3.0
Etheruem has upgraded from 1.0 to 2.0, dividing Ethereum into an execution layer and a consensus layer. The newly introduced Blobs will also be attached to the consensus layer of the beacon chain.
In the next Ethereum upgrade, with the advancement of technology and increased data availability, Ethereum might add a dedicated DA layer on top of the execution and consensus layers. Or, in the next upgrade, there might be a proprietary data storage chain similar to shard chains. Of course, these are all speculations, and we look forward to which direction it will develop.
-------------------
About Us
This speech is written by HTX Ventures,nbsp;HTX Ventures is the global investment department of Huobi HTX, integrating investment, incubation, and research to identify the most excellent and promising teams in the world. As a pioneer in the blockchain industry for ten years, HTX Ventures promotes the development of cutting-edge technology and emerging business models in the industry, providing comprehensive support to collaborative projects, including financing, resources, and strategic consulting, to establish a long-term blockchain ecosystem. Currently, HTX Ventures has supported over 200 projects across multiple blockchain tracks, and some high-quality projects have been launched on Huobi Exchange. At the same time, HTX Ventures is one of the most active fund-of-funds (FOF) investors, working together with IVC, Shima, Animoca, and other top global blockchain funds to build the blockchain ecosystem.
References
Long recommendation: Analysis of jianshu about Da https://twitter.com/jianshubiji
Long recommendation: The new narrative rising slowly in the post-Cancun upgrade era — DA War https://twitter.com/0xNing0x/status/1744350282509631590?s=20
Ethereum Rollups (STARKNET) end the battle, new narrative DA: https://news.marsbit.co/20240220083034666673.html
Priority warehouse research report: Modular blockchain Celestia https://medium.com/@first.vip/%E5%A4%B4%E7%AD%89%E4%BB%93%E7%A0%94%E6%8A%A5-%E6%A8%A1%E5%9D%97%E5%8C%96%E5%8C%BA%E5%9D%97%E9%93%BEcelestia-a2532da9d2be
EigenLayer in-depth research report: Ethereum's middleware protocol leading the staking narrative https://www.aicoin.com/article/360322.html
Long push: DA solution https://twitter.com/i/web/status/1700728653750087945
What is the re-pledge protocol EigenLayer flagship product EigenDA? https://www.blocktempo.com/what-is-eigenda-that-eigenlayer-will-launch/
A look at the five factors of modular design for DA: Celestia, Avail, and EigenDA - who is the winner? https://foresightnews.pro/article/detail/52019
Disclaimer
1. HTX Ventures has no affiliations that may affect the objectivity, independence, or fairness of the projects or other third parties mentioned in this report.
2. The information and data quoted in this report are sourced from compliance channels. The sources of information and data are deemed reliable by HTX Ventures, and necessary verification has been conducted regarding their authenticity, accuracy, and completeness. However, HTX Ventures does not guarantee their authenticity, accuracy, or completeness.
3. The content of the report is for reference only. The conclusions and opinions in the report do not constitute any investment advice for relevant digital assets. HTX Ventures shall not be liable for any loss caused by the use of the content of this report unless otherwise specified by laws and regulations. Readers should not solely base their investment decisions on this report, nor should they rely on this report to impair their independent judgment.
4. The information, opinions, and speculations contained in this report only reflect the judgment of the researchers on the day of finalizing the report and are subject to future changes based on industry fluctuations and data information.
Update, there is a possibility of updating viewpoints and judgments.5. The copyright of this report belongs to HTX Ventures only. If you need to quote the content of this report, please indicate the source. If you need to quote extensively, please inform in advance and use within the permitted range. Under no circumstances should the report be quoted, deleted, or modified in a way contrary to its original intention.
IOS
Android