A brief introduction to the design of DeFi application architecture

text

secondary title

foreword

foreword

The difference between DeFi applications and traditional applications is quite large. The business models are different, the product models are also different, and even the technology stacks implemented are very different. Generally, traditional applications are also called Web2 applications, while DeFi applications can be classified as Web3.

Instead of talking about business models and product models, we only talk about technology stacks. The technology stacks currently involved in DeFi applications mainly include: Solidity, Subgraph, Price Oracle, Hardhat, Ethers, etc. Most of these technology stacks have not even been heard of by some of the big players at the P9 level in major Internet companies such as Ali, Tencent, and Byte. The popular microservice architecture, big data architecture, and cloud-native architecture in traditional application architectures are basically useless in DeFi applications.

I have been researching blockchain since the middle of 2017. After several years of deep cultivation in this industry, I have done several DeFi applications, and finally I have some foundation. Based on my experience, let me talk about the way I understand the architecture design of DeFi applications.

secondary title

Overall structure

Let’s take a look at the general architecture of a DeFi application system:

• Let me introduce the next few modules one by one:

• Blockchain: the underlying blockchain network, a DeFi application is generally deployed to multiple different blockchain networks

• Smart Contracts: Smart contracts are the core business realization of DeFi applications and also the soul

• Price Oracle: price oracle, used to provide price information, generally can be divided into off-chain oracle and on-chain oracle

• Keeper Services: Task triggers and executors of smart contracts, because smart contracts themselves do not have the ability to automatically trigger execution tasks, so external task triggers are required to assist

• Subgraph: Subgraph, also known as an indexer, mainly reassembles data on the chain into data that is convenient for front-end queries

• Graph Node: The environment in which Subgraph runs will synchronize block data on the chain to Subgraph for processing

• Wallet: wallet application, the most mainstream is MetaMask

• WebUI: The UI page displayed on the front end, generally using front-end frameworks such as Vue and React

SDK: Encapsulates the query of Subgraph, the call of the smart contract, the connection of the wallet, etc., to facilitate the call of the front-end UI

I believe that many people will have a doubt: why does a DeFi application system have so many different components?

First, the smart contract itself lacks an automatic execution mechanism. Because Web2 applications have timers, many tasks can be executed automatically with timers. However, the smart contract does not have a timer, so some tasks cannot be executed automatically, and an external program is required to trigger the execution of these tasks. Such an external program is called a Keeper. For example, the liquidator that executes the liquidation task of Compound is a kind of Keeper.

Most Keepers are off-chain centralized programs. In the past year, some decentralized Keeper networks have emerged one after another. The ones I know include keep3r.network, Chainlink Keepers, and KeeperDAO.

Secondly, also because of the limitations of the smart contract itself, it is impossible to actively initiate a network request to an external program to obtain data like a Web2 application. Otherwise, if the smart contract can send network requests to centralized exchanges such as Coinbase and Binance to obtain price data, then different nodes will not be able to reach a consensus because of different request times and different prices obtained. But smart contracts do need to obtain price information, so there is a Price Oracle.

Price Oracle can be mainly divided into two categories: off-chain oracle and on-chain oracle.

The off-chain oracle is represented by Chainlink. Its basic principle is to aggregate the data sources of multiple exchanges (including centralized and decentralized exchanges) at multiple levels and then update the final price data to the on-chain intelligence Contracts, DeFi applications can directly read the price information of smart contracts on the chain.The on-chain oracle is based on Uniswap’s TWAP (time-weighted average price), the principle of which I mentioned in the previous article"Analysis of Uniswap for DeFi Trading Products: Part 2 of V2"

It's already been discussed, so I won't repeat it.

In addition, the data stored in smart contracts is completely different from traditional Web2 databases. It is impossible to index and query data like MySQL and MongoDB databases, and it is also impossible to group, sort or combine data. And this demand is also a rigid demand. In order to meet this demand, The Graph, a blockchain data index protocol, appeared. Subgraph is the core component of the protocol, and Graph Node is its operating environment.

Specific to each different DeFi application, the actual architecture may be slightly different. For example, Uniswap does not require Keeper Services or Price Oracle. But most slightly more complex DeFi applications basically need these components, and therefore, this architecture has become the general architecture of most DeFi applications.

secondary title

Smart Contract Design

In the entire DeFi application system, the core and most complex is the smart contract, and the smart contract still needs to be open source, and its security is also crucial. Therefore, the design of the smart contract is naturally a very critical part.

Although the specific design varies from specific product to product, there are still some general design principles that help us design relatively elegant smart contract products. The so-called relative elegance, in my opinion, the most important thing is to meet several characteristics: security, functionality, and scalability.

Because all smart contracts need to be open source, security must be the first priority to avoid various security loopholes, especially to prevent flash loan attacks, re-entry attacks, permission loopholes, etc. Before the official launch of the main network, internal security audits and external audits should be fully conducted, as well as adequate testing, including internal testing and external open testing, and even a public reward, so that more professionals can come together to find hidden loopholes.

Satisfying functionality is a basic requirement. For example, a loan product that can deposit, withdraw, and borrow is a basic requirement that must be met; a derivative DEX that can magnify leveraged transactions is also a basic requirement that must be satisfied. But the extent to which it is satisfied may be limited by other constraints. For example, in order to prevent flash loan attacks, derivatives DEX may prohibit the same account from opening and closing positions in the same block.

Scalability is also a very important feature. After all, it is not enough to release only one version of an application system, and it is always necessary to iteratively add new functions. For example, for derivatives DEX, the first version may only implement the market price transaction function, the second version needs to add the price limit transaction function, and the third version needs to add the stop profit and stop loss function.

These features are not all positively correlated. For example, further improving security may reduce functionality and scalability. The choice is to reach a state of balance.

To achieve the goal, the design principles that need to be followed and the essential design ideas behind them are actually those that architects are familiar with, such as the single responsibility principle, the opening and closing principle, the dependency inversion principle, the interface isolation principle, etc., and the focus Architecture ideas such as separation, low coupling, high cohesion, and moderate design.

The core of architectural design is to simplify complex issues, which is even more important when applied to open source smart contracts.

secondary title

design practice

Let's take the DeFi application I am currently in charge of as an example to talk about some of my practice summary.

Let me briefly introduce the background first. A small number of friends know that I have recently joined a new team and am in charge of a DeFi product, to be precise, a derivative DEX. The transaction mode mainly adopts the AMM mode, but it is not the AMM that provides dual-currency liquidity like Uniswap, but the AMM that provides single-currency liquidity, which we call Elastic AMM. The specific business will not be launched, and I will talk about it in detail later. This time I will mainly talk about the considerations in architecture design.

The overall architecture of the entire application system is roughly the same as mentioned above, so I don't intend to repeat the overall architecture, but mainly want to talk about the design of smart contracts.

At the smart contract level, it is actually divided into the main agreement and the peripheral agreement. **And what I mainly want to talk about is the design of the main agreement. The following figure is the architecture of the main agreement:

The green ones only represent several roles participating in the system, and the other colors are sub-modules in the smart contract.

First of all, I adopted the idea of ​​layered architecture and divided the system into layers. The upper layer depends on the lower layer, but the lower layer cannot depend on the upper layer. Second, modules only depend on interfaces, not on specific implementations. In this way, low coupling between modules can be achieved.

As Trader and LP users, they all interact with the Router uniformly, and the Router is equivalent to playing the role of a routing gateway. Moreover, because the underlying layer has no dependencies on it, it is easy to upgrade and replace.

Each trading pair has an Amm contract and a Margin contract, and Amm and Margin are bound to each other. Therefore, it is natural to think of using the factory pattern to create different trading pairs. In the original design, there was actually only one factory contract, but in the actual implementation, it was finally found that the factory contract exceeded the maximum number of bytes of the smart contract, so the factory contract was split into three.

The responsibility of Amm is to be responsible for the underlying exchange transactions, while the responsibility of Margin is to manage the user's position. LiquidityERC20 is a liquidity token contract, inherited by Amm. Vault manages the actual assets in the main agreement, inherited by Margin. This module is the core of the entire protocol, and only implements the basic functions of the bottom layer, such as adding and subtracting margin, opening and closing positions, adding and removing liquidity, etc. Scalable functions can be implemented by upper-layer modules, such as the exchange support of ETH and WETH realized by Router, and then add upper-layer contracts to support price limit orders, stop-profit and stop-loss, etc.

secondary title

Summarize

Summarize

The overall complexity of the DeFi application system is actually far inferior to that of the Web2 application system, but because the technology stacks are almost completely different, and the product thinking is also quite different, and the DeFi applications are all open source, these have become threshold of this industry. Therefore, it is actually not easy for inexperienced people to get started now. Even high-end talents of Web2 still need some time to learn and accumulate after entering Web3.