Original Author: Yang
first level title
Original source: IOSG Ventures
0. Primer
first level title
1. Background introduction
With the growing ecology on the chain, such as DeFi transactions, lending, NFT casting, trading, etc., user behavior is directly and transparently recorded on the chain. The data of these on-chain behaviors corresponds to the flow of value on the chain, and the analysis of these data and the insights and insights derived from the analysis become extremely valuable. On-chain data analysis platforms, such as Nansen, Token Terminal, Dune Analytics, Footprint Analytics, flipsidecrypto, glassnode, Skew, etc., have launched products with slightly different focuses for individual and institutional users in response to these growing needs.
first level title
2. Introduction to the data architecture of the on-chain data analysis platform
Although the blockchain records all original transaction data, the data on the chain itself is open and transparent, but when we ask: what is the transaction volume of Uniswap in the past 24 hours? How many percent of the current BAYC holders also hold at least one Moonbirds?... etc., the original data on the chain cannot give us the answer, we need to use indexing, processing ), storage (storage) and a series of data ingestion (ingestion) processing, and then aggregate and calculate the corresponding data according to the questions asked, in order to get the answer to the question. It is very time-consuming and labor-intensive to directly query the blockchain to obtain the answer to the question. In order to allow the data on the chain to be retrieved quickly, the current mainstream data analysis platform on the chain will index the original data on the chain and go through a series of processing. Finally, it is stored in the data warehouse (data warehouse) that is responsible for updating and managing by the platform. When users track the transaction dynamics of smart money on Nansen, or view visual analysis on Dune Analytics, the user's query for the so-called "data on the chain" is actually querying the database controlled by the project party instead of the blockchain itself.
The data warehouse architecture of the on-chain data analysis platform is roughly as follows:
Data collection layer: The platform obtains the original on-chain data from the blockchain nodes, some platforms will accept data sources provided by third parties, and some platforms (such as Footprint Analytics) support users to upload off-chain data to assist in the final data analysis.
Data processing layer: Each platform performs data extraction, conversion and loading of raw data in the form of stream processing or batch processing. In stream processing, real-time raw data is continuously input and continuously processed, which usually means low data delay and higher timeliness of analysis results; while batch processing has slightly higher data delay and lower timeliness of analysis results, But it is more suitable for large-capacity data processing.
Data storage layer: The processed data will be stored in each data table of the data set according to the format predefined by the platform for subsequent use.
Data integration layer: the stored data will be aggregated. The calculation can be performed according to preset indicators (metrics computation), or it can be periodic (periodic) or triggered according to set conditions (event-driven aggregation).
Data analysis layer: The results of the calculation are reported and output in real time. For individual users, we mainly interact with the on-chain data analysis platform at the data analysis layer, such as the Business Intelligence report interface provided by Nansen, the numerous visual charts on Dune Analytics and Footprint Analytics, and the API interfaces provided by some platforms, etc. .
image description
(https://www.nansen.ai/post/nansen-and-google-cloud-empower-web3-investors-with-high-quality-real-time-market-intelligence)
image description
first level title
3. Comparison of data analysis platforms on mainstream chains
From the perspective of content and users, this section compares several mainstream on-chain data analysis platforms from the dimensions of data richness (number of blockchains covered), data granularity, data delay, platform ease of use, and query freedom, including Nansen, Token Terminal, Dune Analytics, Footprint Analytics.
secondary title
Nansen
Nansen should be one of the most familiar on-chain data analysis platforms. Compared to other platforms, its most outstanding feature is wallet profiler/wallet labeling. With the help of wallet tags and other on-chain data, valuable information can be extracted for users, such as Smart Money, which helps users track the real-time movements of giant whales and heavy DeFi players. Other popular products include Hot Contract, discovering emerging and popular DeFi and NFT contracts; NFT Paradise, viewing real-time NFT minting data and more.
[Blockchain coverage] Nansen now supports on-chain data analysis of 11 blockchains including Ethereum, Arbitrum, Avalanche, BSC, Celo, Fantom, Optimism, Polygon, Ronin, Terra, Solana
[Data granularity] Nansen normal version only provides curated data for users
[Data Delay] Streaming and batch processing. Some data analysis has enabled near real-time reporting
[Platform Ease of Use] Zero Threshold
[Query Freedom] The standard version of Nansen only provides a standard information template interface. In response to institutional customers' needs for custom on-chain data query and analysis, Nansen released Nansen Institutions products with the help of Google Cloud Platform's Blockchain Datasets, allowing professional/institutional users to write SQL Queries that meet customized needs.
secondary title
Token Terminal
Token Terminal is famous for providing accurate protocol revenue. Based on the agreement revenue, Token Terminal calculated the price-sales ratio (P/S), price-earnings ratio (P/E) and other data of each agreement. These data provide a valuation benchmark for each agreement to a certain extent.
[Overlay Blockchain] Token Terminal tracks the data of more than 130 protocols
[Data granularity] Token Terminal only provides curated data for users
[Data Delay] Batch processing. According to the recent communication between the IOSG team and Token Terminal, the data on the Token Terminal platform is currently delayed by about two days
[Platform Ease of Use] Zero Threshold
image description
Token Terminal protocol income data legend: the revenue ratio of the top ten blockchains and protocols in the past 365 days
secondary title
Dune Analytics
Dune Analytics is the first on-chain data analysis platform that allows users to independently query, and has the largest analyst group and user community. Dune Analytics provides highly granular raw on-chain data that analysts can freely leverage to write custom queries. Dune Analytics also opens Abstraction to the project side team, and the project side can create more suitable data tables for analysts to use according to the data content of their own agreements. However, there is a certain threshold for independent query. Analysts need to have the ability to write PostgreSQL to create data queries that meet their own needs. Moreover, the query latency is highly related to the analyst's SQL writing level and familiarity with the data tables provided by Dune Analytics.
[Blockchain Coverage] Dune Analytics provides on-chain data of 6 blockchains including Ethereum, BSC, Optimism, Polygon, Gnosis Chain, and Solana
【Data Granularity】Extremely fine
[Data Delay] Stream processing. Data delayed by about five minutes
[Platform Ease of Use] Dune Analytics puts forward certain SQL coding requirements for analysts
【Query Freedom】High
With highly granular raw data, analysts are free to create on-chain analytics in Dune Analytics. Such as daily StepN new shoe casting and historical accumulation data https://dune.com/queries/627689/1170627
secondary title
Footprint Analytics
Compared with Nansen, which has a low threshold of use but only provides a standardized information interface, Dune Analytics provides free query but requires analysts to have the ability to write PostgreSQL language. Footprint Analytics provides users with the best of both worlds solution, giving great query freedom At the same time, the threshold of use is lowered. How does it do it?
"The data on the chain is intricate, and analysts may need to write hundreds or thousands of lines of code to complete the calculation of an indicator. In order to solve the problem of high analysis threshold, Footprint cleans and integrates the data on the chain, giving the data business meaning, so that users do not need to SQL queries and coding can also analyze blockchain data. Anyone can build their own custom charts in minutes through a rich chart interface, decrypt on-chain data, and discover value trends behind projects.”
Footprint Analytics not only provides the original blockchain data, but also classifies the data on the chain. The most original data on the chain is Bronze data, the data that has been screened, cleaned and enhanced is Silver data, and the data that has further sorted out business significance is Gold data.
Gold and silver-level data that has been sorted out and has business logic and business significance can be directly used for analysis. With the help of gold and silver level data, Footprint Analytics provides users with a service to independently query the data on the chain by simply dragging and dropping the data table. Regardless of whether you can write SQL-like language code or not, you can quickly create a data analysis information interface that meets your customized needs, and visualize the required information through intuitive and interactive charts.
【Blockchain coverage】Footprint Analytics currently provides on-chain data of 17 blockchains, including Ethereum, Arbitrum, Avalanche, Boba, BSC, Celo, Fantom, Harmony, IOTEX, Moonbeam, Moonriver, Polygon, Thundercore, Solana, etc.
【Data granularity】Footprint Analytics not only provides users with extremely fine-grained raw data, but also provides users with curated data
[Data Delay] At present, Footprint Analytics performs batch processing on the collected raw data once a day, and the data delay is one day
[Platform Ease of Use] On the Footprint Analytics platform, users can freely analyze data on the chain without SQL query and coding. For analysts with SQL code capabilities, Footprint also provides raw data for analysts to use.
【Query Freedom】High
first level title
Image source: IOSG
4. A little imagination - data analysis on the decentralized chain
On-chain data analysis is so important, but today's users can only rely on centralized management "on-chain data" analysis platforms such as Nansen and Dune Analytics to assist investment decisions. On these platforms, users cannot verify that the data used has not been tampered with, and have to trust that the data sets provided by the platform are conclusive and real. "Don't Trust. Verify." has become an empty phrase in on-chain data analysis.
The wave of Web3 is coming, and the ecology on the chain is becoming more and more abundant. In the future, smart contracts and decentralized applications may not only need the original data on the chain and the data provided by the oracle machine as input information, but may also need to input data based on the original data on the chain. Can we still trust and use these centralized on-chain data analysis platforms for such purposes? The answer is probably no.
The IOSG team has recently seen that existing project teams have taken the first step on the road to realize data query and analysis on the decentralized chain. Due to the limited space, let's listen to the next chapter to break it down - the road to data analysis on the decentralized chain.
https://cloud.google.com/customers/nansen
https://www.nansen.ai/research/on-chain-forensics-demystifying-steth-depeg
https://docs.dune.com/data-tables/data-tables
https://docs.dune.com/dune-engine-v2-beta/query-engine
https://www.footprint.network/@Footprint/Footprint-Datasets-Data-Dictionary
https://www.youtube.com/watch?v=Pp9_wgYZB3I
Original link
IOS
Android