A Brief Analysis of Active Data Synchronization in Ethereum

I. Introduction

Straight to the point, Ethereum data synchronization uses nodes (peers) as data carriers to store and transmit data subjects mainly composed of Header, Body, and Reciept. Data synchronization transactions are managed through the Ethereum p2p communication protocol, and finally handed over to the executor (active synchronization will Handed over to Downloader, passive synchronization will be handed over to Fetcher) to perform the final data download task.

Then ask the data subject what data do we need to synchronize for synchronization? Does different data need to be classified and synchronized?

Know that Chuangyu Blockchain Security Lab has conducted a comprehensive and detailed analysis.

2. Data subject:

To perform data synchronization, we first need to clarify what the data subject we are synchronizing consists of. Generally speaking, there are two types. One type does not require nodes to send active synchronization requests, and the nodes will broadcast to the network by themselves when the data is packaged. There are three types of data in this type: Types - full block, block hash, and transaction Transaction. Today we focus on the next category of data.

The second type of data is the data that requires the node to actively send a synchronization request and then other nodes respond to the data to be synchronized. Except for the first type of data, other data that needs to be synchronized belong to the second type, so it can be said that its types are more complicated, and the most important There are also three kinds of block header, block body, transaction receipt Receipt (specific code ⬇⬇⬇), different from the first type, these three can be said to be a certain part of the complete block, which also shows In order to actively synchronize, it is hoped that the scope of data synchronization between nodes can be freely and controllable, and that necessary data can be synchronized first and temporary non-essential data can be given up.

⬆Header means the block header, which plays a very important role in the block. ParentHash will record the block hash of the previous block, uncleHash will record the uncle block hash, and coinbase will identify the miner's address. Root, Txhash, and ReceiptHash are the root nodes of the three prefix trees of state trie, tx trie, and Receipt.trie respectively. One RLP code hash is used to describe the state of the world, one is used to describe the transaction, and the other is used to describe the transaction receipt. Bloom is a Bloom filter in the block header to quickly determine whether the target hash is in a certain set. The above data in the Header are used by the block header to identify the block and the important data modules in the block using hash. The rest of the data have their own needs. It can be seen that a block header has basically described a block frame, which also paves the way for the data synchronization strategy mentioned later.

⬆Body is relatively concise, which contains a set of transaction objects and an array of block headers of uncle blocks.

⬆Receipt is an array generated after execution of all transaction objects in the block, which is used to record transaction processing information. After they are generated, they will be inserted into the Receipt trie one by one and the ReceiptHash in the block header will also be generated. A transaction receipt records three parts of information for transaction management and identification. The first part is the consensus part, PostState, Status, CumulativeGasUsed, Bloom, and Logs. Only these five data will be encoded in ReceiptRLP to form a receipt hash, and then the receipt hash will participate in the consensus The second part of the data is the transaction part, TxHash refers to the transaction hash corresponding to the transaction receipt, ContractAddress is the address of the new contract when the transaction is deployed, and GasUsed indicates the Gas of the transaction Usage, the third part is the block part, which records the BlockHash block hash, BlockNumber the current block number, and TransactionIndex the serial number of the transaction in the block. Receipt has an intuitive difference in the data synchronization strategy. The most intuitive difference between fullSync and fastSync mentioned below is whether to synchronize the receipt of the peer or to generate the receipt locally.

3. Data carrier

The data transmission carrier is a peer node. This data carrier peer, if you search carefully in the source code, you will find a peer.go in some corners. First, I will list the peer structure I found about the data synchronization module. . There are p2p/peer.go, eth/peer.go, les/peer.go, eth/downloader/peer.go. Ethereum's network also has a transport layer, a session layer, a presentation layer, and a protocol layer.

The peer in the p2p package acts as an underlying node model. The transport layer will discover adjacent peers based on the UDP protocol and maintain peer connections, and will also establish an information exchange channel between peers based on the TCP protocol. Peer management at the session layer mainly manages the interaction between nodes and upper-layer sub-protocols, while NodeTable management mainly manages the underlying node connection table built based on the UDP protocol. Therefore, peer nodes at the p2p layer need to be able to obtain and open sub-protocols, and also need to be able to ping other nodes, and of course to receive messages from other nodes. This is reflected in the three important functional modules in p2p/peer.go, pingLoop, readLoop, and startProtocols, which are also the three functions in the peer.run() method.

It can be seen that the above is the underlying peer structure, and the figure below is the peer structure of the sub-protocol layer. Different sub-protocol layers will have different peer structures, which is why there are eth/peer.go and les/peer.go after all Different sub-protocols apply to different data scenarios.

We can see that the data carrier has the bottom layer peer and the sub-protocol layer peer, and data synchronization needs to use the communication protocol to manage and connect the two layers of peers. The bottom layer realizes the peer's message distribution, message monitoring, and the sub-protocol layer peer executes data download and Synchronization, and the very important protocolManger is needed for management and communication between these two layers.

4. Data synchronization process

So how to start the protocolManager? We introduced the peer structure above. It mainly represents other remote nodes. In Ethereum, there is another type of structure Node that also represents nodes. It mainly represents the local node itself, which is slightly different, and the node wants to start the p2p network. It must rely on the local Node module, which uses the Node.Start() function to start two tasks, one is to start the Ethereum Service, and the Ethereum.start starts the protocolmanager; the other is to start the p2p.Server, create and refresh the K bucket , enable UDP port monitoring and TCP port monitoring at the same time, and process messages sent from remote nodes.

When the protocolManager is started, it needs to connect the underlying logic layer peer (p2p) and the top layer protocol layer peer (eth):

When the protocolManager is started, the node will first initialize and call NewProtocolManager. During the initialization process, the SubProtocol function will be called to obtain a message read and write channel from the dungeon p2p.peer, and then build a message processor Handle, Handle, which will not only accept The message requesting synchronization will also accept the message of node response. In the start method of protocolManger, it will start a regular synchronization coroutine syncer, which will call fecher or downloader executor to perform data synchronization according to the message notification.

• Data Synchronization Mode

• Active data synchronization means that local nodes spontaneously request block data from adjacent nodes, and the data entry is in eth/downloader/downloader and eth/handler

• Passive data synchronization means that the local node receives a data synchronization message (Message) from other nodes and then requests block data. The data entry is at eth/fetcher/fetcher.go

• Active synchronization process Let’s take active synchronization as an example. When the downloader executor is called, it will first call findAncestor to find the common ancestor of the local chain and the remote chain, and start synchronization from the common ancestor. It will configure 4 fetchers, namely fetcherHeader and fetcherBodies , fetcherReceipts, ProcessHeaders finally call processFullSyncContent and processFastSyncContent according to the synchronization mode, and finally call spawnSync to perform synchronization.

The above is the source code of the synchronization process. From the source code, it can be seen that the main download tasks are concentrated in the expandable fetchers constructed by him, which is a fetcher task queue. According to different data synchronization strategies, not only the structure of the fetcher task queue is different, but also the collaboration process between fetchers is also different.

5. Data Synchronization Strategy

The role of the fetcher task queue is summarized as ongoing, data filling, data assembly, and final data insertion. What kind of data is filled in, which data needs to be assembled, and when to insert data are all identified by an important data structure mode, which represents the data synchronization strategy, which is mainly divided into three types.

• lightSync: light node synchronization, when filling data, only block header data is filled, no data assembly is required, call insertHeaderchain to directly insert block header.

• fullSync: full-node synchronization. When filling data, you need to fill in the block header and block body, but do not fill in the transaction receipt, assemble the data into the result set Result, and then call importBlockResults to insert the data in the result set into the main chain, and The difference of lightSync is that there is no block body for light synchronization, so it will not execute and verify transactions, while fullSync full node synchronization will execute transactions and verify while inserting data, which is why its synchronization speed is slow, and then it will generate transaction receipts by itself Receipt, so he does not need Receipt to participate in filling and assembling.

• 6. Conclusion

6. Conclusion

Ethereum data synchronization is very complicated. It covers its network architecture, communication protocol management, cooperation of data transmission channels, arrangement of data synchronization strategies, and algorithms to speed up data synchronization rates. There are many details and logical relationships. And data synchronization is the cornerstone of the communication between blockchain nodes and the outside world, and there are more architectural easter eggs that are amazing. Looking forward to the next article to share with you.