We run nearly 3,000 Validators on the Ethereum 2.0 testnet
Editor's Note: This article comes fromHashQuark Community (ID: HashQuarkPool), reprinted by Odaily with authorization.
Prysm is an excellent implementation of ETH2.0, and it is currently the most running implementation on the Medalla testnet. Prysm adopts the architecture of Beacon Chain Node + Validator Node, the former is responsible for synchronizing block data, and the latter is responsible for signing blocks and witnessing. Since the Validator Node can load multiple validators at the same time, in order to have a qualitative and quantitative understanding of the number of validators it can load and the relevant validator deployment steps, we specially arranged this test.
secondary title
test conclusion
We replicated the Medalla test network, built HashQuark's own ETH2.0 Beacon Chain, conducted two rounds of testing, a total of 14 test cases, and ran hundreds of thousands of Validators. The implementation of Prsym is excellent. For ordinary users who want to participate in Ethereum Staking with a small amount of ETH (dozens to hundreds of Validators), a 4-core 8G cloud server can run Beacon Chain Node and Validator smoothly. However, the technical problems encountered in the running process cannot be solved by ordinary users who are not technical personnel.
Next, we will conduct the next round of testing on the public test network Medalla to be closer to the main network environment. At present, we have nearly 3,000 validators running normally on Medalla, accounting for 5% of the entire network.
secondary title
test environment
For the cloud host deployment used in the following tests, we choose a general-purpose N model, and the CPU platform is Intel/Broadwell. The system uses Ubuntu 18.04.2 LTS. geth version is 1.9.19-stable and Prysm version is v1.0.0-alpha.24.
The first phase of preliminary attempts
secondary title
Test program
Using the most basic two ETH1.0 nodes + two ETH2.0 Beacon Chain Nodes + two Validator Node architectures to build a private network as a starting plan. The stable operation of the network for one day is the observation period.
secondary title
The table below gives an overview of our tests:
Table 1
secondary title
During the test, we collected the CPU, memory, disk IO, network bandwidth IO and other indicators of each instance server.
secondary title
image description
figure 1
figure 2
figure 2
Afterwards, we upgraded the configuration of Beacon Chain Node to 4-core 8G.
Afterwards, we deployed ETH2.0 nodes in different regions to observe whether the network interconnection in different regions would have a greater impact on each indicator, and the experimental results were normal.
secondary title
image description
image 3
In addition, in this stage of the test, the requirements for disk I/O performance and external network bandwidth are very low, which may be due to private network reasons. The specific corresponding performance requirements need to be tested in the official ETH2.0 test network .
Phase 2 Comparative Test
secondary title
The first stage mainly tested the pressure of different numbers of verifiers on server resources. In addition, the verifier's block generation and witness are also important indicators for verifiers. To this end we conducted a comparative test. In the same network, different numbers of validators are deployed in different regions for comparative testing.
secondary title
During the test, we will collect indicators such as CPU, memory, disk IO, network bandwidth IO, number of blocks that should be produced, actual number of blocks produced, number of blocks that should be witnessed, and actual number of blocks witnessed by each instance server.
secondary title
The following is our test case:
image description
Table 2
The ETH1.0 network consists of three 2-core 4G cloud servers, two in Hong Kong and one in Sao Paulo. The block time is set to 15s.
We have deployed Beacon Chain Node and Validator Node in Hong Kong, Singapore, Los Angeles, Frankfurt, Sao Paulo, and London. Beacon Chain Nodes and Validator Nodes in each region are connected through the intranet. The configuration and the corresponding number of validators are shown in the figure above.
Instance 1 and instance 2 both have 1k validators, the difference lies in the configuration of Validator Node, which is used to compare the impact of the number of validators with different configurations on the indicators.
Beacon Chain Nodes in each region are connected to other nodes.
secondary title
Testnet parameter selection
We first deployed the deposit contract on the ETH1.0 network, and after generating the required number of validator keys, we sent deposit transactions in batches. Prysm starts with the following parameters modified:
MinGenesisActiveValidatorCount is set to 8192, which is satisfied since 40k validators are included in our test;
Eth1FollowDistance is set to 20, that is, the deposit contract on the ETH1.0 network will be monitored by the ETH2.0 network after 5 minutes;
SecondsPerETH1Block is set to 15, which is consistent with the time setting of each block in the ETH1.0 network;
In fact, since we sent all deposit transactions in advance to meet the minimum number of validators set at the earliest start time, the entire ETH2.0 network was launched at 1599811207(2020-09-11T16:00:07+08:00).
secondary title
Statistics and test results
We additionally deployed a Beacon Chain Node to connect to the ETH2.0 private network to query data. Add the parameter --slots-per-archive-point=1 to persistently store the data of each block to speed up the query. Adding the parameter --rpc-max-page-size=1000 allows us to query more data each time, thereby reducing the number of requests and speeding up the overall speed.
We selected a period of time when the network was relatively stable, sampling from 75 epoch (approximately 2020-09-12 00:00:00 +0800) to 1200 epoch (2020-09-17 00:00:00 +0800) to obtain this segment The data of block generation and witness data of verifiers in different instances within a certain period of time are analyzed, and the following results are obtained:
Validator witnesses in different regions are slightly different:
image description
table 3
Here we define the witness rate divided by the number of witnesses included over a period of time divided by the number of witnesses assigned to it. It is not difficult to see that, in general, as the number of validators increases, the witness rate will decrease. But in Example 3, although the number of verifiers is only 2k, the witness rate is lower than the witness rate of 6k or even 10k.
We divide the proportion of each verifier according to the range, and get the following data:
image description
Since the number of verifiers in each instance is different, the conversion ratio will be more intuitive:
image description
table 5
To this end, we checked the logs of instance 3 and found that the connection between Beacon Chain Node and other nodes and ETH1.0 is not stable. It is guessed that this caused the abnormal witness rate, which needs to be checked later.
secondary title
In this round of testing, we observed the performance index data in the following table:
image description
Beacon Chain Node
table 6
Validator Node
In instance 1-5, the CPU usage of Beacon Chain Node is between 5% and 10%, and the CPU usage of Beacon Chain Node in instance 6 is about 12%. Memory shows a steady increase, between 12% and 17%, and there is no significant difference in disk IO and bandwidth.
In addition, for generating the verifier key file, we use a recommended python command linetool, the efficiency of this tool to generate keys is relatively low. It only occupies 1 core on a multi-core machine, and it takes about 2.5 hours to generate 2000 key pairs. On the other hand, the import key pair of Validator Node is also single-core execution, and the time to import 2000 key pairs is about 40 minutes.
secondary title
Through this round of testing, we observed in the private network that the increase in the number of validators will affect the block production rate of all validators on the node. For a single validator, in the best case and the worst case, the average Witness about 10 blocks less per day. In terms of block generation, we found no difference in our tests. Compared with CPU and bandwidth, the usage of memory and disk IO increases more obviously as the number of validators increases.
secondary title
Subsequent test plan and steps to be optimized
In this round of testing, the following aspects take up more preparation time:
Authenticator key pair generation
Deploy the deposit contract
Validator Node import key pair
In the follow-up plan, it is planned to optimize the above steps to improve the test efficiency.
In addition, in the follow-up test plan, considering the stability between networks in different regions and its impact on validator indicators, the following improvements can be considered:
Add multiple test instances in the same region to compare whether the difference is caused by the region;
Deploy multiple ETH1.0 nodes to enable Beacon Chain Node to connect to the ETH1.0 network smoothly and reduce the impact;
Add a separate comparison test in the same region, increase the number of verifiers, control variables, and simply compare the impact of the number of verifiers.In terms of statistical data, consider adding more dimensions, such as considering the distance of the witness being included, etc., you can refer to this article aboutarticle.
first level title
Summary of Test Questions
image description
Figure 4
Figure 4
Solution: Use the --grpc-max-msg-size parameter to increase the message size allowed by grpc when starting the Validator Node.
image description
Figure 5
Figure 5
Solution: We currently solve it by clearing the node's data resync. During the test, we found that as the number of Beacon Chain nodes increases, this problem no longer occurs.
Insufficient deposit amount misreportedissueimage description
Figure 6
