Paradigm: Challenges and solutions to Ethereum state growth
Original author: Storm Slivkoff, Georgios Konstantopoulos
Original compilation: Luffy, Foresight News
Ethereum state growth and its relationship to gas limits are widely misunderstood. It is widely believed that state growth is Ethereum’s main scaling bottleneck. However, discussions about state growth are often hampered by terminological imprecision and a lack of detailed quantitative evidence.
Taking a data-driven approach can make the problem of state growth clearer. In this paper, we leverage a high-resolution dataset to understand the size and shape of state growth. In the process, we came to a surprising conclusion: modern consumer hardware can sustain its current state of growth for at least a decade. Furthermore, given continued improvements in software and hardware, this runway may be extended indefinitely.
We believe that Ethereum has a clear roadmap: 1) Completely eliminate state growth as a scaling bottleneck; 2) Increase Gas limits to a level that supports a global-scale decentralized financial system. The goal of this article series is to develop a scientific approach to understanding and developing a roadmap for this expansion.
This article is the first part of a series of articles about Ethereum expansion, mainly introducing state growth, part 2 is about historical growth, part 3 is about state access, and part 4 is about gas limits.
What is state growth?
The term state growth is often used to summarize any Ethereum scaling bottleneck, where the data size exceeds the capacity of the Ethereum node hardware. However, state growth should not be thought of in this singular way. There are many types of Ethereum data, each with a unique relationship to the node’s underlying hardware components. Therefore, it is crucial to use precise terminology to explain each different scaling bottleneck.
State is a set of data required to build and verify new Ethereum blocks. The state consists of contract bytecode, contract storage, account balance and account random number. History is the set of data a node needs to sync from the genesis block to the latest block. History consists of blocks and transactions. State and history are non-overlapping data sets. From these definitions, there are at least 3 different phenomena that put huge pressure on the nodes hardware:
State growth: the accumulation of new accounts, new contract bytecodes, and new contract storage.
Historical growth: the accumulation of new blocks and new transactions.
State access: A set of read and write operations used to build and verify blocks.
Each bottleneck has a unique relationship with the nodes hardware limitations. The four most relevant hardware limitations are:
Network IO is the amount of upload and download speed that a node must maintain in order to reach stable consensus with its peers.
Storage size is the amount of data a node must keep in permanent storage in order to build, verify, and distribute blocks.
Memory size is the amount of data a node must cache in memory in order to stay in sync with the end of the blockchain.
Storage IO is the amount of read and write operations per second that a node must perform in order to stay in sync with the end of the blockchain.
The relationship between these bottlenecks and hardware limitations is shown in Figure 1.
Figure 1: Ethereum expansion bottleneck
Starting at the top of the diagram, every time Ethereum executes a transaction, all resources used by that transaction are priced in Gas. Therefore, Ethereums gas limit is a one-dimensional quantity that rate-limits all forms of on-chain activity. Downstream of the gas limit are the block size and operations per block. The more bytes there are per block, the faster the history grows. The more IO operations per block, the greater the state access rate and (usually) the greater the state growth rate.
Therefore, the scaling bottleneck is related to the hardware constraints of the node, as follows:
To support large amounts of state growth, nodes must have sufficient storage and memory space. If the state becomes too large, it will either not fit in storage or frequently accessed portions of the state will not fit in memory, resulting in performance degradation.
To support massive historical growth, nodes must have sufficient network bandwidth to share large amounts of block data and sufficient storage capacity to store that data.
In order to support a large number of state accesses, the node must have a large amount of memory to cache the hot state and a large amount of storage IO to support sufficient read and write operations.
For state growth in particular, the main challenge is ensuring that the state size does not grow faster than continued improvements in consumer hardware. Node memory and storage are finite resources, so they will eventually reach a bottleneck unless the state stops growing or the hardware is regularly upgraded. Fortunately, memory and storage hardware have continued to improve over the years. Even so, accurate predictions of these improvements are uncertain, and their rapid growth should not be assumed to continue indefinitely.
Note that the data blobs introduced by the upcoming EIP-4844 will bring some changes to these extended relationships. After EIP-4844, it is expected that much less history will be accumulated on disk, and network IO may increase significantly when transferring large amounts of blob data.
In this article, we will focus primarily on state size and state growth rate, rather than memory size and state access patterns. We will investigate other topics in future work.
The composition of the Ethereum state
The next step in understanding state growth is to examine the total size of states and the size of each states contribution. Currently, the amount of Ethereum state data is approximately 245.5 GB. This number was measured using a reth node, but the per-node client numbers are roughly comparable, assheetshown. Accounts, contract bytecode and contract storage occupy 14.1%, 4.3% and 81.7% of the state respectively.
Figure 2 shows how much state size is occupied by various smart contract protocols. In the figure below, the size of each contract category represents the number of bytes occupied by its storage slot and bytecode.
Figure 2: Ethereum state distribution
The numbers in Figure 2 represent the total number of bytes that the node client must store on disk. This includes data used by indexes and other types of storage overhead. The average storage size per account and per storage slot is 133.6 bytes and 191.3 bytes respectively.
Here are some of the most important pieces of information in Figure 2:
Tokens are the largest contributors to state. The largest contributors to Ethereum’s state are ERC-20 and ERC-721 tokens, accounting for 27.2% and 21.6% of the state respectively. The reason tokens take up so much state is because each users balance for each token must be stored individually in its own 32-byte storage slot. Therefore, half of the Ethereum state size is proportional to the total number of Ethereum users and the total number of tokens held by each user.
At least 7.4% of Ethereum’s status is dormant. Some of the largest contracts in the Ethereum state are no longer active. These protocols were launched at a time when block space and state space were much cheaper than they are now, and include most protocols in the gaming, gambling, and scam categories, as well as many DEXs that are no longer active, including IDEX, Etherdelta, and Oasis. Together, these protocols make up at least 7.4% of Ethereum’s state. The true level of dormancy is higher as it also includes long-tail projects in ERC-20, ERC-721 and other categories.
L2 cross-chain bridges occupy less than 2% of the Ethereum state. By leveraging technologies such as compression, ZK proofs, and improved encoding, L2 transactions utilize state more efficiently than mainnet. Although L2 only accounts for 2% of the mainnet state, L2 has 5 times more total transactions per second than mainnet.
How fast is Ethereum state growing?
The most important aspect of state growth is the state growth rate over time. This ratio reveals the severity of state problems and their changing trends.
Figure 3 shows the state growth rate since Ethereum’s inception in 2015. These growth rates are calculated by summing contract bytecode and contract storage within each contract category.
Figure 3: Ethereum state growth over time
Here are some of the most important pieces of information from Figure 3:
Currently, status is growing at about 2.62 GB per month, down from a peak of 5.99 GB per month. These numbers predict that the total state size within 5 years will be between 396 GB and 606 GB. Although one might describe the current growth rate as 12.8% per year, the absolute growth rate has been declining while the state continues to grow, so simple exponential growth may not be an appropriate model.
The recent decline in state growth is primarily due to a decrease in NFT activity. Although one might expect a certain degree of correlation between different types of network activity, there is surprising independence between individual status contributors. For example, while overall state growth has declined over the past few years, ERC-20 state growth has actually increased every year since 2020.
State growth reached its lowest level since 2021. This drop is quite surprising, but it makes sense considering that status is primarily proportional to the balance of new tokens. If the state growth rate keeps decreasing, one might think that Ethereum has the ability to support higher gas limits. This may be true, but its important to remember that: 1) under the current gas pricing model, there is nothing to prevent a new surge in growth rate, and 2) status is not the only bottleneck downstream of gas limits.
What is an acceptable value for status growth?
We now know the 1) size, 2) composition, and 3) growth rate of Ethereum’s state. How do we determine the range of acceptable state growth values? The question is complex because it depends both on unpredictable market forces and on philosophical choices about what trade-offs Ethereum should make.
Lets start with the simplest model, assuming no improvements in future hardware, how long current state growth levels can be sustained on average consumer hardware. As shown in Figure 3, annual state growth in recent years has been between 31 GB/year and 72 GB/year. Currently, common consumer hardware has a maximum storage capacity of approximately 4 TB and a memory capacity of approximately 64 GB. From this we can create a simple storage and memory requirements prediction model:
Storage: Nodes currently need to store approximately 1 TB of state data in total. In practice, this means that many nodes are using disks that are at least 2 TB in size. For simplicity, lets ignore future historical growth and act as if we were in a post-EIP-4444 world. We can calculate the future running time as: (remaining storage capacity)/(state growth rate), such assheetshown. Therefore, the node storage hardware can support the current state growth rate for more than ten years without exhausting 2 TB of space. At current state growth levels, 4 TB is enough to support operations for nearly half a century.
Memory: Ethereum-on-arm users report that the minimum feasible memory to run an Ethereum node is approximately 16 GB. If we assume that memory requirements grow proportionally to state size, then 30 GB to 72 GB of state growth per year would translate into an additional memory requirement of 2 GB - 4.7 GB per year. So, at current gas rates, 32 GB RAM should be enough for 3 to 8 years. 64 GB of RAM should be enough for 10 to 23 years.
This is a simplified model with many assumptions. Possible expansion conditions for this model include 1) historical growth, 2) non-linear expansion of memory requirements, 3) reduced hardware costs, 4) increased gas limits, 5) opcode gas repricing, and 6) future Ethereum architecture improvements. Each of these factors can interact nonlinearly and evolve over time. We will explore these model extensions in future work.
It’s important to stress that long-term sustainability is a good thing. Even if modern hardware can support many years of operation, shortening the runtime should not be taken lightly. Any plan to accelerate state growth should include a significant buffer to account for unpredictable changes in the hardware or software environment.
How to solve the state growth problem?
To solve the state growth problem, many different solutions have been proposed. Three improvements to the Ethereum architecture stand out: Rollup, Verkle attempts, and state expiration. Taken together, these constitute a comprehensive roadmap for addressing short, medium and long-term state growth issues.
Short term: Rollups dont solve the state growth problem, but they do reduce the burden on the network. As shown in Figures 2 and 3, Rollup is able to use state more efficiently than Mainnet. Moving activity to L2 does require a certain amount of state to be stored on mainnet to support user exits. However, the state footprint of L2 transactions is much lower than that of mainnet transactions. Therefore, Rollup can increase the total activity in the ecosystem more sustainably. Rollup adoption is expected to grow with the upcoming launch of EIP-4844, where blobs will make Rollup even more affordable.
Mid-term: Verkle attempts to solve the problem of state growth for validator nodes, but not the problem of nodes needing to build new transactions. Verkle attempts are new data structures for Ethereum state. They support more efficient light clients and stateless nodes. These nodes will be able to validate new blocks without knowing the existing state values. This eliminates the problem of state growth for validator nodes. The construction of new transactions still requires storing and accessing state, but this is still more sustainable than what we have today, since transaction construction is a task that can be easily distributed across many machines. In terms of scope, the Verkle attempt represents a major undertaking that could take years to implement.
Long-term: State expiration solves the state growth problem for all nodes, but requires additional infrastructure. State expiration allows nodes to discard inactive portions of the state, as shown in Figure 2. Note that the term state hibernation may be a more appropriate name, as most existing proposals allow the restoration of expired states through proofs. Regarding concerns about expired state being lost over time, state can be reconstructed as long as history (block and transaction data) is available. Therefore, whatever solution is developed for EIP-4444s history preservation problem will also solve the state preservation problem. But if Verkle attempts to successfully achieve its goals, state expiration may not be necessary.
There are many more solutions to the state growth problem, others include state rent and sharding, but historically these may have had an impact on user experience or sanity. These solutions may need to be combined with others in order to achieve final solutions in the more distant future.
Summarize
Although state growth is a key challenge in scaling Ethereum, we believe it is a solvable problem. Our interpretation of the data suggests that Ethereum can sustain its current state-of-state growth levels for many years and provide a comfortable cushion for architectural upgrades.
We believe that an empirical approach is critical to designing Ethereum’s gas limits and guiding Ethereum toward its ultimate scaling solution. This article is just one step towards that goal. There are other types of data that transcend state, each of which places a burden on Ethereum nodes and Ethereum gas limits. We hope to explore these other bottlenecks in future work.
