Popular Science: Data Availability Issues

image description

-yes... it would take days to complete the transaction with that gas fee-

Ethercat is the first distributed application to achieve large-scale success. But it (and other applications) grabbed all the resources of the Ethereum network - causing the chain's mempool to grow at an unprecedented rate. Overnight, people started talking about Plasma and state channels as the solution to our scaling problems. But despite the discussion and excitement, the development of these solutions has struggled, leading some community members to doubt their viability. But now, as these solutions gradually come online, people realize that the real problem that has plagued developers for a long time is an obvious data availability problem.

secondary title

Data Availability Issues

As Vitalik explained before, the origin of the data availability problem is that malicious miners will try to publish a block with a block header but lose some or all of the data to the chain. This attack can result in:

• Trick the network into accepting an invalid block, and there is no way to prove the block's invalidity.

• Prevents the node from getting the current state.

• Prevent nodes from creating blocks or transferring funds because they lack the corresponding information to construct proofs.

But data availability is not just about hiding block data. Generally speaking, as long as some data is hidden from access by some actors (also known as a review mechanism), it can be called a data availability problem. As far as we know, this is not a problem on mainnet, but it is also very expensive. In fact, over the past 18 months, we've seen a 6.5x increase in the amount of state stored on each Ethereum node (using geth software with fast sync turned on).

Clearly, this is not sustainable for a truly decentralized web. As the size of the blockchain continues to grow, the number of computers capable of participating in the network and acting as nodes will continue to decrease. So how do we deal with this situation?

example

Suppose you go to a casino to play poker. At first, you go to the counter to exchange dollars for poker chips (think of this process as an on-chain transaction). Then, you start sitting at a table and playing poker for hours (these are called off-chain transactions) - sometimes you win money, other times you lose money. After winning a big hand, you tell the house that you want to cash out your chips.

But after you get up, someone gives you a sap in the back of your head - you wake up with a blurry memory and can't remember the details of the poker hand (that's "data not available"). When you're not there, the poker table decides to pretend the last hand didn't happen, and continue playing in the moments leading up to that hand -- i.e. cheating you out of the money you should have won.

secondary title

in practice

In Plasma, each participant must keep a complete transaction history and enough witness data to prove whether its cryptographic assets have been traded in each Plasma block. This makes each participant a node in the Plasma system but only stores its own transaction data. This requirement is made because on Plasma, anyone can collude with the operator of the chain and submit invalid transactions to steal other people's assets. The only way for participants to prevent this from happening is to ensure they have a complete and valid transaction history of all assets.

State channels have lower data requirements because all participants only agree on the current state, not on state updates (such as transactions). In this way, the contract can be settled through only one transaction without recalculating any transaction history. Also, since each state has an auto-incrementing nonce, and the smart contract does not consider it a valid state unless signed by both parties, participants only need to store the latest state.

solution

solution

Right now, the teams involved are doing everything they can to reduce the space taken up by data that needs to be maintained by clients or submitted to mainnet via things like ZK-SNARKS or RSA accumulators, which is a big improvement, but they can't fix it Data availability issues. In fact, we can't really solve this problem for a single client, because that would require that client to be online 100% of the time and never lose the data stored on it (sounds a lot like a blockchain, doesn't it?) .

However, given that hardware capable of doing so does not exist, it is widely believed that the solution to the data availability problem is an incentivized watchtower network (such as PISA) or similar constructs. These incentivized networks are actually a group of staked watchtowers that back up data for paying users and challenge suspicious transactions on behalf of users when they are unable to dispute (i.e. go offline). If they fail to contest the challenge within a certain amount of time, they lose their stake, and those stakes are awarded to a new watchtower in the network that participated in the challenge (assuming it actually submitted the challenge). This bug/reserve protocol has multiple layers so users can be sure they won't be tricked if they go offline or lose transaction/state history.

secondary title

SKALE's solution

SKALE's elastic sidechains solve the data availability problem through the block proposal process. After a validator creates a block proposal, it communicates it to other validators using the data availability protocol described below. The protocol ensures that block proposals are transmitted to a majority (>⅔) of validators.

The five-step protocol is described as follows:

1. Validator A, who sends a block, sends the block proposal and the hashes of all the transactions that make up proposal P to all of its peers.

2. After receiving P and the related hash value, each corresponding node matches the hash value with the transfer in the local transaction queue to be packaged to reconstruct P. For transfers not found in the queue, nodes will send a lookup request to validator A who sent the transfer. The sending validator A then sends the body of these transactions to the receiving validator, allowing the peer to reconstruct the block proposal and add it to its proposal storage database PD.

3. After that, the nodes send A a receipt with a threshold signature.

4. After collecting the signatures of the absolute majority (more than two-thirds) of nodes from the nodes (including A itself), A will create an absolute majority signature S. This signature will serve as a proof that a majority of verifiers own P.

5. A will broadcast this absolute majority signature S to all other verifiers.

Note: Each validator owns the BLS private key slice PKS[I]. The initial state of the key slice is performed using the Federated Feldman Distributed Key Generation (DKG) algorithm, which is run when elastic sidechains are created and whenever validators are randomly swapped. Check out our articles on the BLS and DKG to learn more!

Summarize

Summarize

All in all, if you're wondering what developers working on solutions at the execution layer have been up to over the past 18 months, it's likely that most of their time was initially spent solving this problem. While there is no perfect solution to all scaling problems yet, there is a lot of new and exciting work going on, and we are very confident about what we can achieve in the future!