Original title: "Defining the web3 stack"
Original Author: Nader Dabit
Compilation of the original text: Members of The Graph community
Want to build on web3? Nader Dabit points out the building blocks of the web3 technology stack in an introductory guide
In the next chapter of "Building on web3", Edge & Node Developer Relations Engineer Nader Dabit explains the web3 stack and how developers conceptualize the various layers of building on web3.
I transitioned to web3 in April 2021 after being a traditional full-stack developer for about 10 years. While diving into all these new technologies and ideas, the first thing I wondered was "what is a web3 stack?".
When building a traditional web or mobile app, I often rely on certain building blocks to get the job done:
- API/Application Server (REST or GraphQL)
- Authentication layer (managed or manual)
- database
- Client frameworks, platforms and libraries
- file storage
Using these core components, I can build most types of applications I want, or at least achieve most of my purposes. So, what does this look like in web3?
As it turns out, the answer to this question is not so straightforward because:
- This paradigm is completely different in many ways
- web3 tools, technology and ecosystem are not as mature as web2
It's also harder for me to understand how to get up, running and building a web3 application because I approach things the same way I do in the web2 world.
After working, researching, experimenting, and building over the past 8 months or so, I want to share with you what I've come to know.
What is web3?
Before we define the web3 stack, let's try to define web3:
Web3 is a protocol stack for fully decentralized applications.
With this decentralized technology stack, we can start to build decentralized applications, which have their own meaning and characteristics.
Some features of web3 are:
- Decentralized network infrastructure
- Ownership (data, content and platform)
- Local digital payments
- Self-sovereign identity
- Distributed, trustless, robust infrastructure
- Open, public, composable backend
While some applications built on decentralized technology stacks will replace their predecessors, the new vernaculars enabled by blockchains also enable new application paradigms.
Native digital payments and common back-end infrastructure -- such as machine learning, mobile devices, virtual reality, and other technological primitives, platforms, and building blocks -- enable entirely new types of applications to be built, some not yet imagined.
Does this mean everything will be replaced by web3? uncertain. While I think for certain types of applications, building on a decentralized tech stack is a better option - like almost any tech decision, it depends on what you're building.
Now let's start digging into the web3 stack, broken down into the following categories:
- Blockchain
- Blockchain development environment
- file storage
- P2P database
- API (index and query)
- identity
- Clients (frameworks and libraries)
blockchain
blockchain
There are countless blockchains that you can choose to build on. no one is"the best", instead you should consider the various trade-offs between them.
One thing that is often important to me when learning something new is to apply the Pareto principle to what I am learning. That is, what is the most efficient way to get results for that amount of time and effort. Following this idea, I can learn new things in the shortest amount of time while gaining the most traction and motivation.
In the blockchain space, learning Solidity and the EVM (or Ethereum Virtual Machine) is probably the best thing to do when getting started as a blockchain developer. Using this skill set (and tech stack), you can build not only for Ethereum, but for other Ethereum second layers, sidechains, and even other blockchains like Avalanche, Fantom, and Celo.
That said, Rust is starting to become more popular in the blockchain space, with top-notch Rust support from Solana, NEAR, Polkadot, and others. You probably can't go wrong learning either language, but for beginners, if someone asked me today, I'd say Solidity would still be the better choice.
Beyond these recommendations, here's a non-comprehensive sample of blockchains that have a solid mix of technology, utility, community, momentum, and future viability:
- Ethereum - the original smart contract platform
- ZKrollups:ZKSync, Starknet,Hermez - High throughput Ethereum layer 2, but not natively compatible with the EVM
- Optimistic rollups: Arbitrum & Optimism - Ethereum layer 2, compatible with EVM (read more about the difference between Optimistic rollups and ZKrollups here).
- Polygon - Ethereum sidechain
- Solana - high throughput, low transaction costs, fast block times, but harder to learn than EVM (Rust)
- NEAR - the underlying blockchain, smart contracts can be written in Rust or Assemblyscript
- Cosmos - an ecosystem of interoperable blockchains
- Polkadot - Blockchain-based computing platform that enables blockchains built on top of it to execute transactions between them, creating an internet of interconnected blockchains
- Fantom - EVM compatible underlying blockchain
- Avalanche - EVM compatible underlying blockchain
- Celo - EVM-compatible underlying blockchain designed to make it easy for anyone with a smartphone to send, receive and store cryptocurrencies
- Tezos - non-EVM compatible underlying blockchain, many NFT projects are using it
Blockchain development environment
For EVM development, there are several good development environments available:
- Hardhat (JavaScript) is a newer option, but it's gaining traction. Their documentation is great, the tooling and developer experience is great, and I personally use it all the time to build decentralized applications.
- Truffle (JavaScript) is a set of tools for building and developing applications on the EVM. It's mature, battle-tested, and well documented. It has been around for a while and many developers use it.
- Foundry is a new Solidity development environment from Paradigm that shows great promise. Its main features are the ability to write tests in Solidity, support for fuzzing, and speed (it's written in Rust). I wrote a separate introduction to it here.
- Brownie is a Python-based smart contract development and testing framework for Solidity/EVM development.
file storage
file storage
Where do we store pictures, videos and other files in web3? Storing such large files on-chain is usually very expensive, so we probably don't want to store them there.
Instead, we can use one of a handful of file storage protocols:
- IPFS--Peer-to-Peer File System Protocol
- Pros: It's reliable, has good documentation, has a huge ecosystem
- Disadvantage: If the data is not fixed, it will be lost.
- Arweave - Allows you to store data permanently for one transaction fee. I'm a fan of Arweave and wrote a blog post about it here.
- Filecoin - From Protocol Labs, the same team that built IPFS, a protocol designed to provide a persistent data storage system. It's nice that there are ways for developers to build on Filecoin, including web3.storage.
- Skynet - I haven't used this in production yet, but have tried it and it seems to work fine. The API here looks great. I have some questions like how long is the persistence of the data, and the interoperability of Skynet with other protocols.
P2P database
In addition to file storage and on-chain storage, you may also need to store data off-chain. You can use these types of solutions similar to how you would use databases in a traditional tech stack, but they are replicated across N nodes on a decentralized network and thus more reliable (at least in theory).
A few options are:
- Ceramic Network - a decentralized open source platform for creating, hosting and sharing data. Ceramic also has a nice identity protocol, which I'll touch on later. Probably my current favorite off-chain storage solution. Here's a pretty good demo.
- Textile ThreadDB - A multi-party database built on IPFS and Libp2p. If I understand correctly, it may be undergoing a big API change at the moment. I tried it and it shows some prospects, but documentation and DX need some improvement.
- GunDB - A decentralized, peer-to-peer database. Gun has been around for quite some time, and some pretty interesting applications have been built with it.
In terms of maturity, my take is that the ecosystem of off-chain storage solutions is not yet to the extent needed to establish more advanced use cases that some developers might want. Some of the challenges here are live data, conflict detection and resolution, write authorization, documentation, and general developer experience.
Combining off-chain data solutions with blockchain protocols is one of the last big hurdles we need to cross before we have a fully decentralized protocol stack capable of supporting any type of application.
API (Index and Query)
The way we interact with and build on top of blockchains is very different from databases in traditional tech stacks. With a blockchain, data is not stored in a format that can be efficiently or easily consumed directly from other applications or front-ends.
Blockchains are optimized for write operations. You often hear about innovations centered around transactions per second, block times, and transaction costs. Blockchain data is written in blocks over a period of time, making all but basic read operations impossible.
In most applications, you need functionality like relational data, sorting, filtering, full-text search, pagination and many other types of queries. In order to do this, the data needs to be indexed and organized for efficient retrieval.
Traditionally, this is what databases do in centralized tech stacks, but the indexing layer is missing in the web3 stack.
The Graph, a protocol for indexing and querying blockchain data, makes this process easier and provides a decentralized solution for doing so. Anyone can build and publish open GraphQL APIs, called subgraphs, that make blockchain data easily queryable.
Authentication
Authentication
In web3, authentication is a completely different paradigm. In web2, authentication is almost always based on the user's personal information. This information is usually collected through a form or OAuth provider, and the user is asked to surrender this information in exchange for access to the application.
Although
Although"wallet"The name serves its purpose, but I find the term confusing to people new to web3 as it relates to authentication and identity. I hope in the future we can figure out some other way to express what a wallet is, because it combines not only aspects of finance, but also identity and reputation.
As a developer, you will need to understand how to access and interact with users' wallets and addresses in various ways.
On a very basic level (and a very common requirement), you might want to request access to a user's wallet. To do this, you can usually access the user's wallet in a windows environment (web browser), or use a wallet adapter like WalletConnect or Solana.
For example, if they have an Ethereum wallet available, you will be able to access window.ethereum. The same is true for Solana (window.solana), Arweave (window.arweaveWallet) and a handful of other wallets. WalletConnect is great for mobile web and React Native because it allows users to authorize with their mobile wallet directly from the device.
If you wanted to handle the authentication yourself, you could have the user sign the transaction and then decode it somewhere to authenticate the user, but this usually requires a server. Here's an example using the EVM wallet, and here's an example using Solana/Phantom.
What about managing user profiles in a decentralized manner? Ceramic Network provides the most powerful suite of protocols and tools for managing decentralized identities. They recently published a blog post outlining some of their recent updates and giving some guidance on how all the tools work together. I'd like to start there, then explore their docs to gain an understanding of how to start building, and consider looking at my example project here using Ceramic self.id.
If you want to get a user's ENS text record, ensjs library provides a nice API to get user data:
SpruceID is also something that looks promising, but I haven't tried it yet.
client
client
As far as JavaScript frameworks go, you can really build with anything you want, since client-side blockchain SDKs are mostly framework-agnostic. That said, the vast majority of projects and examples are built in React. There are also libraries like Solana Wallet Adapter that provide additional tools for React, so I'd say learning or becoming familiar with React might be a smart move.
For the client SDK of Ethereum, there are web3.js and ethers.js. For me, Ethers is more accessible and has better documentation, although web3.js has been around longer.
other agreements
other agreements
Radicle is a decentralized code collaboration protocol built on top of Git. It can be thought of as a decentralized version of GitHub.
Livepeer is a decentralized video streaming network. It's mature and widely used, with over 70,000 GPUs live on the web.
Summarize
Summarize
This post will be a living document that I keep up with as I learn, experiment, and gather feedback from web3 developers.
If you have any feedback or ideas about what I've missed here, please share your thoughts with me. It's exciting to see all the activity happening around web3, as developers are eager to get involved. While the infrastructure is still evolving, the vision of building truly decentralized protocols and applications that allow people to coordinate without having to relinquish power and control to large corporations is important, and we are getting closer to realizing that vision .
IOS
Android