A deep dive into the Filecoin retrieval market and its leading project, Saturn
Original source:Filecoin Network
Filecoin’s larger roadmap aims to transform cloud services into a permissionless marketplace where any provider can offer services. The network started in the storage market and launched on mainnet in October 2022. The Filecoin Virtual Machine (FVM) was recently introduced to bring smart contract functionality to the network. This allows users to program key services on the Filecoin network, including large-scale storage and, soon, retrieval.
This article will take an in-depth look at the retrieval market being developed by Filecoin and one of its leading projects, covering the following topics:
Filecoin Retrieval Market and Retrieval Market Working Group (RMWG)
Content Delivery Network (CDN) and the Role of the Saturn Project
The Saturn distributed CDN approach and its evolution to date
Future plans for the Saturn project
Filecoin Search Market and RMWG
Just like our previousarticleAs mentioned, Filecoin is committed to building open data services, which include three pillars (storage, retrieval and data computing). From 2020 to 2022, storage has been a focus for Filecoin—it has become the largest distributed storage network to date, storing more than 1170 PiB of data, with more than 200,000 users ranging from Opensea to the Internet Archive. Retrieval and data computing have also been under development since 2022, with working groups set up around the construction of these markets (open to any individual or entity). Working groups encourage modularity and are often composed of different teams responsible for solving different problems.
Key components of Filecoin’s open data service roadmap
Search Market Working Group (RMWG)’s core work is to build a distributed CDN (Content Delivery Network) for the Filecoin ecosystem. More than 15 teams (e.g.Magmo、Ken Labs, Protocol Labs, etc.) are working to solve technical challenges in this area, from enabling ultra-fast payments to data transfer protocol enhancements to cryptographic economic models for data retrieval. Below are the building blocks that the RMWG has been organizing since the first half of 2022, based on an anticipated retrieval process that can interact freely with the Filecoin storage market.
RMWG identifies building blocks for the functional search market
Even in its development phase, the RMWG project is already processing 160 million retrieval requests per day and more than 2 petabytes of data per month. Overall, these projects will seek to build a distributed CDN that can serve not only the web3 space, but also the web2 market.
CDN and Saturn project role
Content delivery networks are a critical component of the Internet infrastructure. Groups of servers work together to provide rapid delivery of Internet content, from static web pages to YouTube videos. Existing CDN providers include companies like Cloudflare, Akamai, and Fastfly. Businesses pay for these services, not end users, which means consistency, coverage and pricing are critical.
Today’s content delivery networks are highly centralized and dominated by large companies
Todays CDN market is highly concentrated and dominated by a few giants. Just 7 CDN providers meet more than 80% of the market demand. In the event of a network failure, such as the 2022 CloudFlare outage, there is considerable concentration risk and higher latency in regions far from the nearest data centers, such as Africa.
Small CDNs distributed in different regions can effectively solve these problems, but economies of scale hinder the challenge of small distributed CDNs for existing providers (capex can reach billions of dollars per year). Providing better quality distribution network content than existing providers will bring huge business opportunities. In 2022, the global CDN market will reach US$20 billion and is expected to reach approximately US$100 billion in 2032 (excluding new use cases based on web3 such as NFT). Thats where Project Saturn comes in.
Web3 CDN can potentially overcome this challenge by allowing anyone on the web to provide resources for content retrieval, as long as they meet minimum standards. This lowers barriers to entry by shifting the burden from one company to thousands (or more) of companies supporting the network, which is where the Saturn project comes from. The Saturn project is a distributed CDN network built on Filecoin, aiming to achieve reliable, efficient and economical Internet content retrieval. It is one of the key projects in the RMWG and will be launched publicly in November 2022. The Saturn project seeks to achieve the following goals:
CDN market democratization: allowing anyone to become a Saturn node operator in exchange for cryptographic rewards, nodes can be joined without permission, allowing multiple companies or individuals to contribute to the retrieval network (think franchising), thereby achieving wider and more distributed Mode.
High-performance retrieval: High-density nodes on all continents enable high-performance retrieval with sub-100ms TTFB, high network bandwidth, and low latency in all regions. While this is not currently the case, it could be possible with a wider geographical distribution of nodes.
No single point of failure: Unlike traditional CDN networks, there is no single point of failure.
Achieve data integrity and authenticity with content addressing: Project Saturn is the only distributed CDN natively compatible with content addressing.
Saturn project development status (as of August 2023)
The data below are accurate as of August 2023 unless otherwise stated.
Saturn aims to be a reliable alternative to traditional CDN networks, but its near-term goal is to efficiently serve the billions of content-addressed data requests received on Filecoin and IPFS every week. Currently, IPFS Gateway is achieving this goal as Saturns key benchmark for improving network capacity and performance.
Flowchart of how network participants in Saturn enable Filecoin and IPFS retrieval
To enable Filecoin and IPFS retrieval, Saturn’s approach involves four main network actors:
Node Operators: Node operators provide hardware and resources to the Saturn network by running Saturn nodes in various geographical locations around the world. They are rewarded based on the number of bytes provided to users in each pay period. Saturn nodes join the network by registering with the Saturn coordinator. The Saturn L1 network provides Saturn clients with a large, geographically distributed cache of content-addressed data.
Saturn Coordinator: The Saturn Coordinator manages the membership of node operators in the Saturn network and facilitates the payment process for these nodes. This is a key feature to democratize data retrieval while ensuring qualified players enter the market. Over time, our goal is to have the coordinator run entirely on the Filecoin Virtual Machine (FVM).
Client: A network user requests content from the Saturn network, the client is the device used to make the request. The client makes an HTTP request to the Saturn network and returnsCAR file, allowing the client to verify files step by step. When the Saturn L1 does not have a file in its cache, it takes a cache miss to wherever the file is stored in the IPFS network or the Filecoin network and returns it to the user.
Users: Users use the Saturn network as a CDN to accelerate content delivery to users. Saturn users can accelerate their content to a large number of Saturn nodes around the world, creating a superior experience for end users.
So far, the Saturn project has made significant progress. Since its public launch in November 2022, Saturn currently has a time-to-first-byte (TTFB) of 80 milliseconds (50th percentile), serving 30% of mirrored traffic from IPFS.io through the Bifrost Gateway, and A verifiable node reward payment system is launched on FVM.
It has also made significant progress in developing a geographically diverse network capable of handling high-volume requests and delivering content in a high-performance manner (fast time to first byte). Since its public release (only 8 months), Saturn has achieved:
More than 4,800 global locations (across 59 countries)
Capable of handling 478 million requests per day (July 2023)
IPFS content time to first byte (TTFB) is 3.80 milliseconds
1) More than 4,800 retrieval providers around the world (wider distribution than traditional CDN providers)
Currently, there are more than 4,800 search providers on Saturn working hard for network bandwidth, and there are only 662 nodes at the end of 2022, a month-on-month growth rate of 28.2%, which is an impressive number. In comparison, the Filecoin storage market has grown at 21% monthly in the first 6 months, and there are currently approximately 3,500 storage providers on the network (the largest in the web3 storage space).
This exceeds the current distribution of traditional CDN providers. The worlds largest CDN company, Akamai, has a 35% market share and more than 4,000 locations worldwide, followed by Alibaba with an estimated 2,800 locations (mostly in China).
Source: Saturn project data
This rate of growth is a testament to the convenience of being a retrieval provider on the Saturn network. Running a Saturn CDN node requires only 4 TB of storage space and Saturn open source software (much less resources than required as a storage provider on Filecoin). Saturn will allow more people to participate in Filecoin’s distributed data service market.
Source: Saturn Explorer
Participation varies across regions: North America has the most nodes (2200+), followed by Europe (1400+) and Asia (700+). Median TTFB is consistently low across continents, with Europe, Asia, Oceania, and South America having TTFB below 100 ms.
The distribution of these nodes is important because it keeps the distance between the client and the node as short as possible, resulting in low latency for the end user (overcoming the speed of light problem encountered by traditional CDN providers). Saturn’s permissionless and crypto-incentivized attributes allow for a more “resilient” supply to accommodate rapidly growing demand in developing regions such as Asia and Africa, which are currently facing these latency issues.
2) Saturn will process approximately 10.3 billion requests per month on average in 2023
Saturns network capacity is approximately 25+ terabits per second (approximately 10% of Cloudflares network capacity). In 2023, an average of 10.3 billion requests will be processed per month, providing 3.7 million terabytes of bandwidth per month. As of the end of July 2023, more than 478 million requests were processed every day, which was close to 50% of IPFS Gateways daily requests during the same period. Despite all the progress made so far, Saturns network capacity still has room to stabilize.
Source: Saturn project data
3) Time to first byte less than 80 milliseconds (2.5 times faster than IPFS Gateway)
In terms of speed, Saturn has achieved significant success; the median TTFB (time to first byte) is already below 80 milliseconds, which is 2.5 times the median IPFS Gateway TTFB. This coincides with several developments underway to optimize IPFS Gateway performance, such as Project Rhea, showing that Saturn is capable of achieving the same performance. Generally, a good TTFB should be below 100ms for static content and below 200-500ms for dynamic content. Today, Saturns TTFB has reached 80 milliseconds, making it the worlds fastest content-addressed CDN, and there is room for further growth as network density continues to increase.
Source: Saturn project data
Saturn future plans
Since its public launch, Saturn has made significant progress as an open source, community-operated CDN network. Going forward, the team hopes to continue improving TTFB speeds while improving performance correctness and improving latency. By the end of 2023, Saturn is expected to achieve further milestones, including serving 100% of IPFS.io traffic, implementing metering and billing on the user demand side, and launching a web application to help users who want to accelerate content through Saturn to onboard. .
Upcoming Milestones on the Saturn Project Roadmap
You canhereLearn about the Saturn Project and other projects in the RMWGlatest situation. Data in this article are accurate as of August 2023, unless otherwise stated.
Many thanks to HQ Han, Jonathan Victor, Alexander Kintsler and the Saturn project team for their efforts in publishing this article.
Disclaimer: The content of this article is for informational purposes only and does not constitute investment, financial, legal or other advice. This information is not an endorsement, offer or recommendation to use any particular service, product or application.
