Five questions about Ethereum: Where does the problem of client diversity come from?
Editor's Note: This article comes fromEthereum enthusiasts (ID: ethfans)Editor's Note: This article comes from
Ethereum enthusiasts (ID: ethfans)
Ethereum enthusiasts (ID: ethfans)
The recent 90th core dev meeting was devoted to one issue almost the entire time. I strongly encourage everyone to listen to this session for themselves.
In this session, Alexey raised the issue of client developer overload. While I think this discussion is an important start, we are too eager to find a solution to fully understand the problem. It is important that we take a moment to analyze the problem. In the analysis of the connotation of the question, "Five Whys" is one of the most simple and effective methods.
Without further ado, let's see what the first question is.
Geth: 75%
Parity & OpenEthereum: 20%
Nethermind: 1%
secondary title
First question: Why is the pressure on the Geth development team so great, even to the point of overload?
Through etherscan, we can see the statistics of the installed capacity of each client as follows:
The remaining 4% consists of some clients with less than 1% market share and is therefore ignored.
Importantly, more than 51% of the computing power is concentrated on the Geth client. Suppose that in the upcoming Berlin hard fork, Geth has a bug in the implementation of one of the EIPs (Ethereum Improvement Proposals). Even if there are no bugs in other implementations of this client, as long as a block encounters this bug, it will cause the Ethereum network to fork. Logically, this block is invalid, and other clients will treat it as an invalid block. However, more than 51% of the mining nodes are running the Geth client, so the entire network will be brought to the wrong fork chain.
Therefore, the answer to the first question is:
Because the clients of the Ethereum network lack sufficient diversity.
It's worth mentioning that client diversification won't suddenly make client development an easy job. But client diversity itself is still an area worth exploring, which will help us find ways to improve the efficiency of client development while reducing the burden on development teams. It cannot be denied that working on the Geth team alone is unlikely to solve this problem.
secondary title
Second question: Why does the Ethereum network lack client diversity?
When the Ethereum mainnet went live, we had multiple clients. Two of the main ones are Geth and CPPEthereum. Then came Parity, and CPPEthereum was eliminated.
Since then, no client other than Parity has gained a significant market share. Last year, Nethermind emerged as a rising star, but currently holds only 1% of the market. Recently, Parity's market share has dropped significantly due to some ups and downs and a bleak future for Parity. We believe that, ideally, the Ethereum network needs to have 3 or more clients, and the client share of each client should not be too high, and no client can occupy much more than 51% of the market share . While ideally client diversification should be achieved, we are used to client hegemony.
So, why do we need multiple clients?
Someone might immediately suggest that we figure out how to support and help clients who are lagging behind. I'm wary of this "mythical man-month" solution -- bringing more engineers to a difficult problem in software development rarely works, and I don't expect it to.
Instead, I think the focus should be on complexity.
Note: The Mythical Man-Month, mythical man-month, points out that a large number of people and a short period of time cannot shorten the progress of software development. Crowding doesn't help software production, it creates trouble and produces worse software. Adding manpower to a project that is behind schedule will only push it further behind schedule.
secondary title
Now, we are getting closer to the root of the problem.
It turns out that most of the difficulty comes from the networking protocol, the set of tools that ethereum client software uses to connect to each other and share information about the blockchain. The networking rules of Ethereum (defined in the devp2p codebase) ultimately affect and even determine the design and requirements of the Ethereum client.
Some networking tools specify unoptimized architectures and even require Ethereum clients to run unnecessary functions. Client developers need to work within these constraints.
secondary title
Question 4: Why does the network interconnection protocol increase the difficulty of client implementation?
I believe that the answer to this question can be basically divided into two parts.
state management
Overall Network Interconnection Requirements
As far as state management is concerned, an Ethereum client must be able to synchronize the complete state on the network and maintain a local copy of that state. Both are difficult to do. Synchronizing state requires millions of requests for clients and servers that read and process state requests and can saturate disk I/O. The newly synchronized state needs to be maintained and pruned so that the database can execute new blocks fast enough. From an engineering point of view, this is a serious challenge!
GetNodeData is the only networking tool we use to synchronize state, optimized for a specific state database format (commonly known as "the native layout"). The "flat" database layout popularized by Turbo Geth has great performance benefits in terms of state maintenance, but using this layout makes it harder to process GetNodeData requests.
Handles GetNodeData requests to access the state generated by the most recent block;
Handles requests to access the history of on-chain data, including block headers, block bodies, and receipts.
Question 5: Why...
solution
Looks like I only had to ask four "whys" to get to the root cause. The Ethereum protocol is not yet fully mature. When designing the Ethereum protocol, we were unaware of most of the problems found today, or were not a problem at the time because the state was small and the development history was short.
secondary title
solution
I've been following this question for the past year. It struck me that the root of many problems on Ethereum can actually be traced back to the network layer.
Perhaps the most obvious example is that disk I/O has historically been a bottleneck for clients. This bottleneck exists because clients tend to implement their state databases using a naive representation of a tree structure (Trie). The way the state database is constructed is determined by the GetNodeData network element.
To fix this, we need to overhaul various parts of the Ethereum consensus layer as well as the network layer. At present, maintenance work has started. A lot of work has been done under the name "Stateless Ethereum" that Alexey and I have co-led for 8 months. Some of the work we've done at least eases the burden on the Geth team, as they use the SNAP synchronization protocol they've developed over the years. There is still a part of the work that needs to be undertaken by talented people who understand the problem deeply and think of feasible solutions.
At present, such a huge DevP2P ETH protocol has not been fully deconstructed. We have a basic understanding of how to split this network into three separate private networks, but no one has done that directly yet.
