secondary title

1 Introduction

world computerworld computer” becomes a reality, this “world computer” will serve as the operating system and deployment environment for decentralized applications.

Swarm can provide uninterrupted service and more effectively resist network outages or targeted DoS attacks. Swarm effectively promotes freedom of information as a permissionless publishing platform. Swarm is known for its unique privacy features such asbrowse anonymously、Deniable storage、andandA file format that does not leak metadataEtc., in response to the growing demand for network security.

Incentives built into Swarm are designed to optimize the allocation of bandwidth and storage resources so that they are economically self-sustaining. Swarm nodes track their respective bandwidth contributions in connection with each node, and BZZ is used to settle additional debts due to unequal consumption. Publishers in the Swarm must spend BZZ to purchase the right to write data to the Swarm, and prepay rent for some long-term storage.

Swarm's modular design consists of clearly separablelayercomposition (see Figure 1). Technically speaking,andandLayer 3 "High-level data access via API"secondary title

2 DISC: Distributed Immutable Storage of Blocks

DISC （Distributed Immutable Storage of Chunks）It is the underlying storage model of Swarm. It consists of nodes that store and provide data. In the collaboration between these nodes, assuming that each node pursues a strategy that maximizes the profit of its operators, the behavior of the network as a whole will show the following characteristics:

• Privacy protection and permissionless uploads and downloads
• Robust defenses that make it difficult to block or change access to content once published
• Automatic expansion as demand increases
• Integrity protected content
• Content that is no longer saved is eventually forgotten

Anyone with excess storage space and bandwidth can participate in DISC as a node operator and get rewarded accordingly. When an operator installs and runs the Swarm client software, a new node is created and becomes part of the Swarm network, essentially taking care of a small slice of the world's hard drives, the Swarm.

Next, we will further define DISC and explain why it produces the above properties.

2.1 Connections, topology and routing

The initial responsibility of DISC is to build and maintain a network of nodes so that all nodes can send messages between each other. This exchange of messages takes place over a persistent and secure communication channel that exists between nodes using a p2p network protocol (libp2p). Swarm expects nodes to establish Kademlia connections: when connected to a specific set of other nodes, the local decision made by the node to send the address will eventually lead to the transmission of the message to find the globally optimal route.

Kademlia assumes that each node is assigned a Swarm address different from its network address. By computing the value common to two Swarm addresses in the value of the prefix bits, we can define their proximity. The nodes closest to each other will form a fully connectedNeighborhood. Furthermore, each node is connected to multiple peer nodes from each discrete proximity class (see Figure 2).

(Note 1: Libp2p is a network framework for users to develop decentralized peer-to-peer applications.)

(Note 2: Kademlia is a P2P overlay network transmission protocol designed by Petar Maymounkov and David Mazières to build a distributed P2P computer network. It is a P2P information system based on XOR operations. It formulates the structure and specifications of the network A way for nodes to communicate and exchange information.)

The resulting topology guarantees that relaying moves the message at least one step closer to its intended destination with each message transition (see Figure 3).This technique enables messages to be routed between any two nodes, even if the two nodes do not maintain a direct connection between them. The upper limit of the number of hops required to deliver a message is the logarithm of the total number of nodes, ensuring that any two nodes can always be connected to each other, even in an extremely large network.

2.2 Blocks and Storage

The standard storage unit in Swarm is calledblock. pieceConsists of up to 4 kilobytes of data, with an address attached. Since the address of the block and the address of the node come from the same address space, their proximity can be calculated. Swarm's storage scheme states that each block is stored by a node at an address close to that of the block itself.

To facilitate data privacy, blocks can be encrypted after being padded to 4 kilobytes, making them indistinguishable from other random data without the key. Even for unencrypted blocks, node operators cannot easily determine what content each block came from. Since Swarm nodes cannot themselves choose which blocks of data to store and encrypt, this ambiguity of origin and the inability to leak metadata provide them with effective protection from liability related to what they store.

In order to insert a block into the Swarm, a node passesSynchronous push protocol (push-sync protocol)The block is forwarded until it reaches the neighborhood it belongs to. Then, the storage confirmation of the block will be sent back along the same path. To retrieve a block, simply use the retrieval protocol, routing a request with the block address to the relevant neighborhood. If any node on the way has the corresponding block locally, it will send it back in the response.

Nodes usepull-sync protocolContinuously syncs its block storage. This ensures that each neighborhood redundantly stores all blocks belonging to its neighborhood. This redundancy increases the resiliency of data transfer, maintaining block availability even if some nodes in a certain neighborhood are unreachable. The synchronization protocol also ensures that the stored content of the neighborhood remains consistent when nodes go offline and new nodes join the network.

2.3 Forwarding, Privacy and Caching

In Swarm, a message is routed by forwarding it recursively closer to its destination, and then passing a response back along the same route. This routing algorithm has two important properties:

• The person making the request is ambiguous.

• Automatically scale as demand increases.

The message sent by the node that initiates the request is the same as the message sent by the node that only forwards the request, at any level. This ambiguity allows the originator of the request to ensure that their privacy is not violated, thereby facilitating permissionless content publishing and private browsing.

Since nodes participating in routing retrieval requests may choose to store blocks forwarded by them, an auto-scalable distribution system is enabled. The bandwidth incentive mechanism discussed below for thisSpeculative caching (opportunistic caching)provide economic incentives.

2.4 Swarm Accounting Protocol

Swarm Accounting Protocol (Swarm Accounting Protocol, SWAP)Ensuring that node operators are cooperative in routing messages while protecting the network from indiscriminate use of bandwidth.

As nodes forward requests and responses, they keep track of their relative bandwidth consumption to each node. Within certain limits, services are exchanged between nodes. However, once the limit is reached, the indebted party can either choose to wait until its debt is amortized over time, or it can pay by sending checks, which can be cashed on the blockchain as BZZ (see Fig. 5).

This protocol ensures that those who download or upload small amounts of content can use Swarm for free, and those who are willing to wait, reciprocally serve each node until enoughcreditFinally, Swarm is also free to use. At the same time, it provides a faster experience for those who wish to pay when uploading or downloading larger volumes of content.

When helping each node forward messages, there is an economic incentive for each node, because each node that successfully routes a request to a node closer to the destination can obtain BZZ when the request is successfully delivered. If the node does not store data itself, it can request blocks of data from closer nodes for a small fee. Through such transactions, nodes can earn a small profit when processing requests. This means that there is an incentive for nodes to cache blocks becauseAfter buying a block once from a closer node, any subsequent requests for the same block will earn a pure profit.

2.5 Out of capacity and garbage collection

As new content is added to the Swarm, sooner or later the limited storage capacity of each node will be exhausted. At this point, nodes need a strategy to decide which blocks should be deleted to make way for new blocks.

The local storage of each Swarm node has two built-in subsystems, namely ""and""and"cache”。

A "reserve" is a fixed-size storage space dedicated to storing blocks belonging to a node's neighborhood. Whether a block is kept in the "reserve" depends on the "postage stamp". Contracts on the blockchain allow purchases via BZZ"postage batch". The owner of a "batch" has the right to issue a limited number of postmarks. These postmarks then act as a kind of trust token, indicating to the user the specific value of storing something in the Swarm. By using the magnitude of this value to Determining which blocks in the "reserve" are to be deleted first, so that the depositor's node can maximize the utility of DISC (see Figure 6). The value of each stamp will decrease over time, It's as if the storage rent is periodically deducted from the balance of the "batch"; as soon as the value of the postmark is insufficient, the relevant block is evicted from the "reserve" and placed in the "cache".

The role of the "cache" is to hold blocks that are not protected by the "reserve" due to insufficient "batch" values ​​or too far from the node address. When capacity is reached, the cache is periodically pruned, and blocks that have not been requested for the longest time are deleted. The popularity of a block can be predicted by the time when the request was last received, and blocks with more SWAP revenue will be reserved first. Combined with speculative caching, thisGarbage collectingThe strategy maximizes the operator's profit from bandwidth incentives, and at the network level, automatic expansion of popular content is realized.

2.6 Block Types

Above we defined a block as the standard unit of data in DISC. There are two basic block types in Swarm:andandsingle-owner chunks。

The address of a content-addressable block is based on thehash digest(See Figure 7). The integrity of the block data can be verified using the hash as the address of the block. Swarm uses theMerkle treeofBMT (Binary Merkle Tree) hashalgorithm.

secondary title

3 Functions of the Swarm API

In addition to blocks, Swarm also exposes APIs for implementing higher-level concepts such as files, hierarchical collections of files with various metadata, and even inter-node messaging. These APIs attempt to mirror those already in use on the web. More novel ideas and data structures can be drawn on top of these higher layers, opening a rich variety of possibilities for all who wish to benefit from the core offerings of privacy and decentralization that DISC provides.

3.1 Files and collections

Data larger than the 4 kilobytes allowed in a single block is split into multiple blocks. A group of homogeneous blocks is defined by a SwarmHash tree (hash-tree)means that this hash tree encodes the way the file is split into chunks during the upload process. This tree consists of a set ofleaf node chunksComposition, containing the data itself, is referenced by one or several layers of intermediate blocks, each of which contains references to its sub-blocks (see Figure 9).

The content address of the entire file is then determined by the hash digest of the root block, i.e. the hash tree spanning the entire fileMerkle root. In this way, the address of the file becomes itsChecksum (checksum), so that the integrity of the content can be verified. Representing a file as a balanced Merkle tree of blocks also provides efficient random access to the file, resulting in efficient range queries.

Swarm uses "manifests" to represent collections. The manifest encodes a generic map of string references, allowing it todirectory tree、ororrouting tableDo model building. These respectively enable Swarm to implement a file system, act as a database, and even provide virtual hosting for websites and dapps.

If we interpret the host part of a URL as a reference to a manifest, then the manifest provides URL-based addressing, the URL path is used as the key for lookups in the map represented by the manifest, and is only used to reach the file reference.

Manifests encode the mapping they represent in the form of a compact Merkle trie, and blocks serialize the nodes of the trie (see Figure 10). When finding a path, we only need to retrieve the corresponding blocks along the nodes of the branches we traverse. This ensures efficient lookup of files/records with latency and bandwidth logarithmic to the size of the collection.

The child node references in the middle block of the hash tree in the file, and the list prefix tree nodes in the collection, are aligned with the BMT hash segment in position. As a result, Swarm supports compact proofs that a particular piece of data is part of a file located at a given offset from a given URL, which is the basis for publicly provable database indexing and trustless aggregation.

3.2 Tracking updates: feeds and domain resolution

feed is a way to allowVariable resource display (impression of a mutable resource)Single owner block example for . A feed can represent a versioned revision of a mutable resource, a sequential update to a topic, or a continuum of messages published by a party on a communication channel.

The feed works by defining a single owner block identifier derived from the topic and index. When publishers and content consumers agree on how and when the index should be updated, specific references to updates to that feed can be constructed and found.

Similar to how DNS resolves a domain to the IP address of the host server, Swarm usesEthereum domain name resolution service ENS (Ehereum-Name Service)- A set of smart contracts on the blockchain that resolves to references to support human-readable domain names (e.g. Swarm.eth).

References registered with ENS can be updated whenever a web application or the website it represents gets a new Swarm reference due to an update. Alternatively, when a domain name references a feed, users can benefit from a human-readable domain name while also being able to update its content without having to interact with the blockchain and pay the associated transaction costs every time a change is made.

3.3 Message Passing

PSS (Postal Service on Swarm) is a direct inter-node messaging protocol in Swarm. It does this by encrypting the intended recipient's message and wrapping it with a subject in a content-addressable block. Since blocks are created in such a way that their content addresses fall within the recipient's neighborhood, delivery is naturally handled by the synchronous push protocol.

Also, for any third party, the message is indistinguishable from the randomly encrypted block, so it is also called **"Trojan" block**. A node expecting to receive a PSS message will attempt to decrypt and open all blocks arriving in its neighbourhood. After successfully decrypting and unwrapping the Trojan block to a legitimate recipient, the client node can send the message in clear text to an application subscribed to the topic using the PSS API.

PSS also provides asynchronous delivery, as blocks persist and are eventually synced to all neighboring nodes, even if those nodes come online later.

Since PSS allows users to receive messages from hitherto unknown individuals, it is an ideal communication primitive for sending anonymous messages to public individuals (eg registries), or for setting up secure communication channels via feeds to initiate a stream to a contact. Since PSS does not require any action by the recipient (such as polling), it can be the recommended primitive for push notifications.

3.4 Pin and restore

DISC will eventually forget about rarely accessed and unpaid content. By doing to the block "pinning"With this operation, nodes can ensure that they retain certain content locally. At the same time, this"The person who saves pinned content locally (pinner)” can participate in the passive or active restoration of content for the benefit of all users.

passive recoveryA recovery protocol is involved to notify pinners of missing blocks by sending recovery requests using PSS when retrieval fails. Pinners listen to recovery requests and respond by re-uploading missing chunks so downloaders can find them when they retry. This recovery feature also allows seeding of the original content directly from the publisher node, similar to the main mode of operation in some existing file sharing solutions (BitTorrent, IPFS).

Conversely, Swarm also providesactive recoverysecondary title

4 Conclusion

Swarm is a peer-to-peer network, and all its nodes jointly provide decentralized storage and communication services. Permissionless and private, Swarm addresses the needs of free speech, data sovereignty, and an open marketplace on the web, while keeping it secure with integrity protection, censorship resistance, and attack resistance. This article introduces the functions included in the initial mainnet launch of Bee1.0.

This is a milestone, and the journey is just beginning: join Swarm in its mission of empowering digital freedom.