In-depth understanding of Ethereum transaction processing mechanism

特邀专栏作者

2021-06-17 11:07 (UTC+8)

This article is about 5380 words, reading the full article takes about 8 minutes

From the perspective of source code, it briefly analyzes the logical design of the Ethereum transaction pool data structure, transaction fee setting, transaction construction, transaction entry into the pool, transaction signature, transaction verificati

AI Summary

Expand

foreword

first level title"foreword"、"Blockchain is a"decentralized"Distrust"A distributed ledger maintained collectively by way of"distributed"The blockchain is naturally indispensable for bookkeeping, and transactions naturally become the top priority. Knowing that Chuangyu Blockchain Security Lab will briefly analyze the logical design of the Ethereum transaction pool data structure, transaction fee setting, transaction construction, transaction pooling, transaction signature, transaction verification, etc. from the perspective of source code, and through the Fang transaction security mechanism design to study the public chain security mechanism design.

basic concept

first level title

basic concept

The schematic diagram of the transaction process is roughly as follows:
Flow Description:

First, the user initiates a transaction request through the network, signs the transaction with his own private key, and then broadcasts the transaction, and then adds the transaction to the transaction pool. The miners obtain the transaction information from the transaction pool, and then package it and generate Blocks are then generated through consensus, and finally the transaction blocks are broadcast to the entire network.
Data flow:
The data source of the transaction pool:
The whereabouts of the data in the transaction pool: obtained and verified by the miner (miner), used for mining, written into the block after successful mining and broadcasted, the transaction will be deleted from the transaction pool after it is written into the canonical chain, if the transaction is written After entering the fork, the transactions in the transaction pool will not be reduced, and then wait for repackaging.

data structure

first level title

data structure

The default configuration is as follows:

The TxPool data structure is as follows:

first level title

When analyzing transaction execution, we first need to look at some basic configurations, such as: whether there is a maximum upper limit/lower limit for transaction fees, transaction pool configuration, maximum number of transaction information retrieval, etc. Here we only look at some key points :

secondary title

02 Trading pool configuration

secondary title

03 Transaction Retrieval Quantity

Initialize the pool

The initialization of the transaction pool is realized through NewTxPool, and the specific code is as follows:

Here, the sanitize function is first called to verify the configuration parameters to avoid setting unreasonable gas prices.

Then initialize a transaction pool ( txpool ) with the default configuration:

Then initialize the local account and add the configured local account address to the transaction pool:

Then create a more gasprices-ordered transaction:

The specific implementation code is as follows:

Then call reset to update the transaction pool:

The specific implementation of reset is as follows:

Then start the reorg loop so that it can handle requests generated during log loading:

The specific implementation code of scheduleReorgLoop is shown below. This function is mainly used for the execution plan of reset and promoteExecutable.

Then subscribe to relevant transaction events and start the main loop:

The specific implementation code of the main loop loop is as follows. It is a goroutine of txPool and also the main event loop. It is mainly used to wait for and respond to external blockchain events and various reports and transaction eviction events:

first level title

build transaction

The transaction is initiated by the user, so that the assets are transferred from one party to the other, which is the so-called value transfer. Our most intuitive transaction construction is to transfer money through the wallet. Here we directly use the RPC of eth_sendTransaction as an example to analyze the construction process of the transaction , an example of an eth_sendTransaction request is as follows:
Example parameters:
from: DATA, 20 bytes - the source address of the sending transaction
to: DATA, 20 bytes - the target address of the transaction, optional when creating a new contract
gas: QUANTITY - available gas amount for transaction execution, optional integer, default value 90000, unused gas will be returned
gasPrice: QUANTITY - gas price, optional, default: To-Be-Determined
value: QUANTITY - the amount sent by the transaction, an optional integer

data: DATA - the signature and encoding parameters of the contract compilation or called method

nonce: QUANTITY - nonce, optional, the same nonce can be used to rewrite pending transactions

Example response:"Let's track the execution process of eth_sendTransaction, an RPC. Here, first check whether the account exists, then check whether the Nonce is empty, then call SingTx to perform the signature operation, and then call SubmitTransaction to submit the transaction:"The implementation code of SignTx is as follows, where SignTx will continue to be called for signature operations, so I won’t go into details here, and the follow-up

transaction signature

A fine-grained analysis will be performed:"After signing, return to SendTransaction to call SubmitTransaction to submit the signature. Here, it will first check whether the transaction fee is sufficient, and then call SendTx to send the transaction:"The specific implementation of SendTx is as follows. Here, AddLocal will be called to add transactions to the transaction pool. There will be no further details here.

This one analysis unit module:

Then check whether the receiving address is empty, if it is empty, create an address (usually appears when the contract is created), then print a log of complete TX details for subsequent manual investigation and analysis, and then return the hash value of the transaction:

first level title

Trade into the pool

We know that there are two sources of transactions: one is submitted locally, and the other is submitted remotely. The specific implementation codes of these two are AddLocals and AddRemotes respectively. These two functions are used when adding transactions to the transaction pool. This is achieved by calling addTxs:

The addTxs code looks like this:

First, the transaction will be filtered to check whether it is a known transaction (that is, added or broadcast), and then call the send function to verify the address derived from the signature (v, r, s) through the secp256k1 elliptic curve, if derived If it fails or is incorrectly signed, it returns an error:

Then add the transaction to the transaction pool (note: there are transaction locks here)

The specific implementation of addTxsLocked is as follows, it will queue valid transactions, and call the pool.add function to add transactions to the transaction queue:

The specific implementation of the add function is as follows:

Here, it will first check whether the current transaction has been known (that is, it has been broadcast or added to the pool), and if it is known, it will be discarded directly:

Then identify whether the transaction is submitted locally or remotely, and call the function validateTx to verify the transaction. If the verification fails, it will be discarded directly:

Then check whether the transaction pool is full, and if it is full, discard the underpriced transactions in the transaction queue. GlobalSlots and GlobalQueue are the maximum capacity of pending and queue:

Then judge whether there is a transaction with the same nonce value in the pending queue of the current transaction. If it exists, judge whether the gasprice set by the current exchange exceeds the set PriceBump percentage. If it exceeds, replace and overwrite the existing transaction. Otherwise, an error will be returned and the Gasprice of the replacement transaction is too low. , and throw it into the queue ( enqueueTx ):

After calling enqueueTx, it will be added to the transaction queue, and at the same time, check whether the from account is a local address, and if so, add it to the local address of the transaction pool:

The enqueueTx code is as follows, this function mainly inserts new transactions into the transaction queue:

Finally, it will go to the addTx function, where the requestPromoteExecutables function will be called to perform a transaction promotion request operation, which mainly puts the transaction from the queue to the pending:

transaction signature

The transaction signature is mainly realized by the function SignTx. First, check whether the wallet is closed, and then check whether the wallet account contains the account of the estrus transaction request, and then call SignTx for signature processing:

The specific implementation code of SignTx is as follows:

Use ECDSA (Elliptic Curve Encryption Algorithm) to sign in sign, and then return the signed result:

transaction verification

first level title

transaction verification
Transaction verification is the most important part of the entire transaction link. For users, transaction verification is an important means to ensure the safety of users' property. For the entire Ethereum, transaction verification is an important way to ensure the stable operation and sustainable development of Ethereum. , transaction verification mainly occurs in the following scenarios:
When the user completes the signature of a transaction, he needs to submit the transaction to the blockchain network, so that the transaction can be confirmed as soon as possible, and the node needs to verify the transaction before submitting the transaction to confirm the legitimacy of the transaction;
When a node receives a transaction broadcast by other nodes, the node needs to verify whether the transaction is legal, and the legal transaction will be added to the node's transaction pool;

When a node receives a block synchronized by other nodes, it also needs to verify the transactions contained in the block.

The transaction verification is completed by the validateTx function, and its logic code is as follows. Here, it will check whether eip2718 is enabled and the type of transaction, and then check the size of the transaction, the amount of transaction transfer, the gas of the transaction, the correctness of the transaction signature, and ensure the transaction Follow the Nonce order, whether the trader's assets are sufficient, and ensure the transaction gas price. The basic transaction fee is high:

first level title

transaction upgrade

Transaction upgrade mainly refers to putting the transaction into the pending list. The difference between this method and the add method is that the add function inserts the obtained new transaction into the pending, while PromoteExecutables puts the executable in the given account address list. The transaction is inserted into the pending from the queue list, and the invalid transaction is checked, and then the transaction pool update event is sent. The implementation code is as follows:

Here, a for loop is used to iterate all accounts and upgrade transactions. Here, firstly, all transactions in the queue whose nonce is lower than the current nonce of the account are deleted:

Then remove all transactions in the queue whose consumption is greater than the balance held by the account or whose gas is greater than the maximum gas limit:

Then add all executable transactions from the queue to pending, where the promoteTx method will be called to put the transactions ( Txs ) in the queue into pending:

The implementation code of promoteTx is as follows. This function first inserts the transaction into the pending queue. If the old transaction is better (the transaction Gasprice is greater than or equal to 110% of the original transaction value, the standard is related to the pricebump setting), then delete the current one. This transaction, if the current transaction is better than the old transaction deletes the old transaction, then updates the list:

Then return to the promoteExecutables function, and if the non-local account queue is less than the limit ( AccountQueue ), remove it:

Finally, record the removed entry and update the queuedGauge. If the transaction of this account in the queue is empty, delete this account:

transaction downgrade
Transaction downgrade means that when a new block appears, the packaged transaction will be downgraded from the padding to the queue, or when the Gas price of another transaction is higher, it will be downgraded from the padding to the queue. The downgrade operation The key implementation function is demoteUnexecutables, and the transaction downgrade mainly occurs in the following three situations:
The fork leads to a decrease in the Nonce value of Account: If the transaction number m on the original canonical chain A costs 20 and is already on the chain, but the transaction number m on the new canonical chain is not on the chain after the fork, resulting in the transaction number m recorded on the canonical chain The Nonce of the account is reduced, so transaction m must be rolled back to the transaction pool and placed in the queue;

The fork causes the nonce value of the pending transaction to be different from the nonce value of the state.

The code of demoteUnexecutables is as follows. Here, firstly, the latest Nonce value of each addr is obtained by traversing the pending list, and then the transaction whose Nonce is smaller than the Nonce value obtained by the previous query is deleted, and then the account balance is not enough to pay the transaction fee and some temporarily invalid transactions are returned. transaction, and put the temporarily invalid transaction in the queue. If there is a gap at this time, move the following transaction to the queue list. If the above downgrade is performed, if there is no transaction in a certain addr in the pending, the transaction will be deleted. Account to delete:

first level title

pool reset

We can reset the transaction pool through reset, the specific code of this method is as follows:

If the old head is not null, reorganization starts. At this time, if the head block of the old chain is greater than the height of the head block of the new chain, the old chain will roll back and recycle all rolled back transactions. If the head block of the new chain If it is larger than the head block of the old chain, the new chain will back off and recycle the transaction. When the new chain and the old chain reach the same height, they will back off at the same time until they find a common parent node, and then find out all the transactions stored in discard but not in included. value, and then reinsert these transactions into the pool:

Then set the latest world state, set the state of the new chain head block, and then put the transaction backed by the old chain into the transaction pool:

first level title"Summary at the end of the article"The blockchain is composed of blocks linked to each other in a chain structure. Each block consists of two parts: a block header and a block body. The block body stores transaction records, so

This article analyzes functional modules such as Ethereum transaction pool data structure, transaction fee setting, transaction construction, transaction signature, transaction pool entry, transaction verification, transaction upgrade, transaction downgrade, transaction pool reset, etc. from the perspective of source code. The process and security design of Ethereum transaction processing have been explored, but the construction of the public chain security system still has a long way to go and needs to be further explored.