Join the PolkaWorld community and build Web 3.0 together!
Join the PolkaWorld community and build Web 3.0 together!
Let's start with an interesting fact. XCM is a "cross-consensus" message format, not just "cross-chain". This difference is a sign of what the format ultimately achieves, a format that not only communicates between chains, but also between smart contracts and modules, as well as through bridges and shards (such as Polkadot's Spree) to send various ideas .
mdnice editor
🤟 A format, not a protocol
In order to better understand XCM, it is important to understand its boundaries and its place in the Polkadot technology stack. XCM is a message format. It is not a messaging protocol. It cannot be used to actually "send" any messages between systems, it only serves to express what the receiver should do.
In addition to sending messages between chains, XCM is also useful in other contexts, for example, it can be used to conduct transactions on chains whose transaction format you did not know very well before. For chains where the business logic changes very little (such as Bitcoin), the transaction format—or the format wallets use to send instructions to the chain—tends to remain exactly the same, or at least compatible, indefinitely. Using highly evolvable meta-protocol-based chains, such as Polkadot and its constituent parachains, business logic can be upgraded across the network with a single transaction. This can change anything, including transaction formats, causing potential problems for wallet maintainers, especially for wallets that need to be kept offline (such as Parity Signer). Because XCM is well-versioned, abstract, and generic, it can be used as a means of providing wallets with a persistent transaction format for creating many common transactions.
mdnice editor
🥅 goal
XCM aims to be a language for the exchange of ideas between consensus systems. It should be generic enough to be correct and useful throughout the evolving ecosystem. It should be extensible, and since extensibility inevitably implies change, it should also be future-proof and forward-compatible. In the end, it should be efficient enough to run on-chain and also run in a metered environment.
This pervasiveness even extends to concepts such as paying for the execution of XCM messages. Since we know that XCM can be used in a variety of systems, from gas-metered smart contract platforms and community parachains, all the way to trusted interactions between system parachains and their relay chains, we don't want to bake elements like fee payments too deep and become irreversible in the agreement.
mdnice editor
😬 Why not just use native message format
Piggybacking a chain or smart contract in a native message/transaction format can be useful in some cases, but does have some big drawbacks that make it less useful for XCM's goals. First, there is a lack of compatibility between chains, so systems that intend to send messages to multiple destinations need to understand how to write messages for each destination. At this point, even a single destination may change its native transaction/message format over time. Smart contracts may be upgraded, and blockchains may introduce new features or alter existing ones, changing its transaction format.
Third, operations such as paying fees do not easily fit into a model that assumes fee payments have been negotiated like smart contract messages. In contrast, transaction envelopes provide some system for payment processing, but are also generally designed to contain a signature, which is meaningless when communicating between consensus systems.
mdnice editor
🎬 Some initial use cases
While XCM aims to be general, flexible, and future-proof, it must of course meet practical needs, especially for token transfers between chains. Optional fee payments (possibly using these tokens) are another way, like a common interface for exchange services, common throughout the DeFi world. Finally, it should be possible to do some platform-specific operations using the XCM language. For example, in a Substrate chain, it may be necessary to dispatch a remote call to one of its modules to access specialized functionality.
On top of that, there are many token transfer models that we wish to support: we may just want to simply control accounts on the remote chain, allow the local chain to have an address on the remote chain to receive funds and eventually transfer funds it controls to other accounts on the remote chain.
We might have two consensus systems, both of which are "native homes" for a particular token. Imagine a token like USDT or USDC that has instances on several different chains and is fully interchangeable. It should be possible to burn such tokens on one chain and mint corresponding tokens on another supported chain. In XCM parlance, we call it a teleport because the transfer of an asset is actually accomplished by destroying it on one side and creating a clone on the other.
Finally, there may be two chains that want to nominate a third chain, and an asset on one of those chains may be considered a native asset to be used as a reserve for that asset. Derivative forms of each on-chain asset will be fully supported, allowing the derivative asset to be exchanged for the underlying asset on the reserve chain backing it. This can be a case where two chains don't necessarily trust each other, but (at least as far as the asset in question is concerned) are willing to trust the asset's local chain. An example here is that we have several community parachains that want to send DOTs between each other. Each of them has a native form of DOT fully backed by a parachain-controlled DOT on the Statemint chain (DOT's local hub). While native forms of DOT are sent between chains, behind the scenes, "real" DOT moves between parachain accounts on Statemint.
Even this seemingly modest level of functionality has a relatively large number of configurations, their use may be desirable, and some interesting design is required to avoid overfitting.
Anatomy of XCM
At the heart of the XCM format lies XCVM. Contrary to what some may believe, this is not a (valid) Roman numeral (although if it were, it could mean 905). In fact, this stands for Cross Consensus Virtual Machine. It's a very high-level non-Turing-complete computer with instructions designed to be roughly at the same level as transactions.
A "message" in XCM is really just a program running on XCVM. It is one or more XCM directives. The program executes until it runs to the end or encounters an error, at which point it ends (I'm intentionally not explaining this right now) and stops.
XCVM includes a number of Registers, as well as access to the overall state of the consensus system that hosts it. Instructions may change a Register, they may change the state of the consensus system, or both.
enum Instruction {
TransferAsset {
assets: MultiAssets,
beneficiary: MultiLocation,
}
/* snip */
}
An example of such an instruction is TransferAsset, which is used to transfer an asset to some other address on a remote system. Need to tell which assets are to be transferred and to whom/where they are to be transferred. In Rust, it's declared like this:
The types used are very basic ideas in XCM: assets, represented by MultiAsset, locations within consensus, represented by MultiLocation. Origin Register is an optional MultiLocation (optional because it can be cleared completely if needed).
mdnice editor
📍 At the location of XCM
The MultiLocation type identifies any single location that exists in the consensus world. This is a fairly abstract idea that can represent everything from scalable multi-shard blockchains (like Polkadot) all the way down to low-level ERC-20 asset accounts on parachains that exist in consensus. In computer science terms, it's really just a global singleton data structure, regardless of size or complexity.
A MultiLocation always represents a location relative to the current location. You can think of it like a filesystem path, but there's no way to directly express the "root" of the filesystem tree. There's a simple reason for this: in Polkadot's world, blockchains can be merged into other blockchains, or forked from other blockchains. Blockchains can start life very independently and eventually be promoted as parachains in a larger consensus. If you do, the meaning of "root" will change overnight, potentially confusing XCM messages and anything else that uses MultiLocation. For simplicity, we completely ruled out this possibility.
Positions in XCM are hierarchical, with some places in the consensus completely encapsulated elsewhere in the consensus. Polkadot's parachains exist entirely within the overall Polkadot consensus, which we call internal positions. More strictly, we can say that whenever any change in one consensus system implies a change in another consensus system, then the former system is internal to the latter. For example, the Canvas smart contract lives inside the contract module that hosts it. UTXOs in Bitcoin are internal to the Bitcoin blockchain.
This means that XCM does not distinguish between "who" and "where" questions. From the perspective of something as fairly abstract as XCM, the distinction doesn't matter -- the two blur and become essentially the same thing.
When written down in text like this one, they are represented as some .. (or "parent", encapsulating the consensus system) components, followed by some join points, all separated by / . (This doesn't usually happen when we express them in a language like Rust, but it makes sense on paper because it resembles the familiar directory path in widespread use.) Connections in their encapsulated consensus Identifies an internal location system. If there is no parent node/junction point at all, then we just say the position is here.
For example:
For example:
../Parachain(1000): Evaluate in a parachain, this will identify our sibling parachain with index 1000. (In Rust, we would write ParentThen(Parachain(1000)).into())
../AccountId32(0x1234...cdef): Evaluated in the parachain, this will identify the 32-byte account 0x1234...cdef on the relay chain.
There are many different types of join points used to identify where you might find them on the chain in various ways, such as keys, indexes, binary blobs, and plural descriptions.
mdnice editor
💰 Assets in XCM
When working in XCM, it is often necessary to reference some kind of asset. This is because almost all existing public blockchains rely on some native digital asset to provide the backbone of their internal economic and security mechanisms. For proof-of-work blockchains like Bitcoin, the native asset (BTC) is used to reward miners who develop the blockchain and prevent double spending. For proof-of-stake blockchains such as Polkadot, native assets (DOTs) are used as a form of collateral that network administrators (called stakers) must bear in order to generate valid blocks and receive physical rewards.
Some blockchains manage multiple assets, for example Ethereum's ERC-20 framework allows many different assets to be managed on-chain. Some assets that manage non-fungibles, such as Ethereum's ETH, are non-fungible — unique instances; Crypto-kitties are an early example of such non-fungible tokens, or NFTs.
struct MultiAsset {
id: AssetId,
fun: Fungibility,
}
XCM is designed to handle all such assets effortlessly. For this purpose there is the data type MultiAsset and its associated types MultiAssets, WildMultiAsset and MultiAssetFilter. Let's look at MultiAsset in Rust:
plain
enum AssetId {
Concrete(MultiLocation),
Abstract(BinaryBlob),
}
So there are two fields that define our asset: id and fun, which is a good indication of how XCM handles assets. First, the overall asset identity must be provided. For fungible assets, this simply identifies the asset. For NFTs, this identifies an entire "class" of assets - different asset instances may be within this class.
enum Fungibility {
Fungible(NonZeroAmount),
NonFungible(AssetInstance),
}
Asset identities are represented in one of two ways; either concrete or abstract. Abstract isn't really used, but it allows asset IDs to be specified by name. This is convenient, but relies on the receiver to interpret the name in the way the sender expects, which may not always be easy. Concrete uses and uses locations in general to unambiguously identify assets. For native assets (e.g. DOT), the asset is often identified as the chain where the asset was minted (in this case the Polkadot relay chain, which would be where its parachains are .. ). Assets managed primarily within the chain module can be identified by including their indexed position within that module. For example, a Karura parachain might refer to an asset on a Statemine parachain at ../Parachain(1000)/PalletInstance(50)/GeneralIndex(42) .
Second, they must be either fungible or non-fungible. If they are fungible, there should be some associated non-zero quantity. If they are not replaceable, then there should be some indication of which instance they are rather than the quantity. (This is usually represented by an index, but XCM also allows various other data types, such as arrays and binary blobs.) This covers MultiAsset, but we sometimes use three other related types. MultiAssets is one of them, and really just means a set of MultiAsset items. Then we have WildMultiAsset; this is a wildcard that can be used to match one or more MultiAsset items. It really only supports two wildcards: All (matches all assets) and AllOf matches all assets with a specific identity (AssetId) and fungibility. Notably, for the latter, there is no need to specify the quantity (in the case of substitutables) or the instance (for non-substitutables), and all match.
Finally, there is MultiAssetFilter. This is the most commonly used, and is really just a combination of MultiAssets and WildMultiAsset, allowing wildcards or explicit (ie non-wildcard) lists of assets to be specified.
👉 Holding Register
In the Rust XCM API, we provide a number of conversions to make working with these data types as easy as possible. For example, when we are on the Polkadot relay chain, to specify a fungible MultiAsset (Planck, for those in the know) equal to 100 indivisible DOT asset units, then we would use (Here, 100).into ().
WithdrawAsset(MultiAssets),
Let's look at another XCM instruction: WithdrawAsset. On the surface, this is a bit like the first half of TransferAsset: it withdraws some assets from the originating account. But what does it matter to them? If they don't deposit it anywhere, then it must be a pretty useless operation. If we look at its Rust declaration, we find more clues about its usage:
So, this time there is only one parameter (MultiAssets type, which specifies which assets must be withdrawn from the Origin Register's ownership). But it doesn't specify where to put the assets.
enum Instruction {
DepositAsset {
assets: MultiAssetFilter,
max_assets: u32,
beneficiary: MultiLocation,
},
/* snip */
}
These temporarily held unused assets are called holding Registers. ("Holding" because they're in a temporary location that doesn't last indefinitely, and "Register" because it's a bit like a CPU Register, a place to store working data.) There are many instructions that operate on holding registers. A very simple directive is the DepositAsset directive. Let's take a look at it:
aha! Astute readers will notice that this looks a lot like the missing half of the TransferAsset directive. We have the assets parameter, which specifies which assets should be removed from holding registers to be deposited on-chain. max_assets lets the XCM author inform the recipient how many unique assets they intend to deposit. (This is helpful when calculating fees before knowing the contents of the Holding Register, since depositing assets can be an expensive operation.) Finally there is the beneficiary, which is the same parameter we encountered earlier in the TransferAsset operation. There are many directives that represent actions to be performed on a Holding Register, and DepositAsset is one of the simplest. Others are more complicated.
🤑 Fee payment in XCM
Fee payment in XCM is a fairly important use case. Most parachains in the Polkadot community will require their interlocutors to pay for any operation they wish, so as not to open the door to "transaction spam" and denial-of-service attacks. Exceptions also exist when chains have good reason to believe their interlocutors will behave well — this is the case when the Polkadot relay chain communicates with the Polkadot Statemint public interest chain. However, for the general case, fees are a good way to ensure that XCM messages and their transport protocols are not overused. Let's take a look at how fees are paid when XCM messages reach Polkadot.
As mentioned earlier, XCM does not include fees and fee payments as first-class citizens: unlike the Ethereum transaction model, for fee payments to be something that is not required in the protocol, it must be circumvented. Like Rust's zero-cost abstraction, fee payment has no significant design overhead in XCM.
For systems that do require payment, XCM provides the ability to use assets to purchase execution resources. Broadly speaking, this includes three parts:
First, some assets need to be provided.
Second, assets must be exchanged for computation time (weight in Substrate parlance).
Finally, the XCM operation will execute as directed.
The first part is managed by one of several XCM directives that provide assets. We already know one of these directives ( WithdrawAsset ), but there are several others that we'll see later. The resulting assets in the Holding Register will of course be used to pay for the costs associated with the execution of XCM. Any assets not used to pay fees will be deposited into a destination account. In our example, we assume that the XCM takes place on the Polkadot relay chain and that 1 DOT is traded (ie 10,000,000,000 indivisible units).
WithdrawAsset((Here, 10_000_000_000).into()),
Currently our XCM directive looks like this:
enum Instruction {
/* snip */
BuyExecution {
fees: MultiAsset,
weight: u64,
},
}
This brings us to the second part, exchanging (a portion of) these assets in exchange for computation time to pay us in XCM. For this we have the XCM instruction BuyExecution . Let's see what it looks like:
The first item fees is the amount that should be withdrawn from the Holding Register and used to pay the fee. Technically, this is only a maximum, as any unused balance is immediately refunded.
The final amount spent is up to the interpretation system - fees just cap it, and if the interpretation system needs to pay more for the desired execution, the BuyExecution instruction will cause an error. The second item specifies the amount of execution time to buy. This should generally not be less than the total weight of the XCM program.
WithdrawAsset((Here, 10_000_000_000).into()),
BuyExecution {
fees: (Here, 10_000_000_000).into(),
weight: 3_000_000,
},
In our example, we're assuming a weight of 1 million for all XCM instructions, so so far we have two projects (WithdrawAsset and BuyExecution) of 2 million, and one more to come. We'll just use all the DOT we have to pay those fees (which only makes sense if we trust the destination chain to not have crazy fees - assuming that's the case). At this point, let's look at our XCM:
The third part of our XCM is to deposit the remaining funds in the Holding Register. For this, we will just use the DepositAsset directive. We don't actually know how much is left in the Holding Register, but that doesn't matter since we can specify a wildcard for the assets that should be deposited. We put them in a sovereign account in Statemint (identified as Parachain(1000)).
WithdrawAsset((Here, 10_000_000_000).into()),
BuyExecution {
fees: (Here, 10_000_000_000).into(),
weight: 3_000_000,
},
DepositAsset {
assets: All.into(),
max_assets: 1,
beneficiary: Parachain(1000).into(),
},So our final XCM directive looks like this:
⛓ Use XCM to move assets between chains
Generally speaking, there are two ways for assets to move between chains, depending on whether the chains trust each other's security and logic.
mdnice editor
✨ Teleporting
For chains that trust each other (e.g. homogeneous shards under the same overall consensus and security umbrella), we can use Polkadot's framework called teleportation, which basically means destroying the asset on the sending side and minting it on the receiving side . This defense method is both simple and efficient - it requires only the coordination of two chains and involves only one action on each side. Unfortunately, if the receiving chain cannot 100% trust that the sending chain actually destroys the asset it is minting (and indeed does not mint assets outside of the asset's agreed rules), then the sending chain really has no reason to mint assets based on the message.
WithdrawAsset((Here, 10_000_000_000).into()),
InitiateTeleport {
assets: All.into(),
dest: Parachain(1000).into(),
xcm: Xcm(vec![
BuyExecution {
fees: (Parent, 10_000_000_000).into(),
weight: 3_000_000,
},
DepositAsset {
assets: All.into(),
max_assets: 1,
beneficiary: Parent.into(),
},
]),
}
Let's take a look at what it looks like for XCM to transfer (mostly) 1 DOT from the Polkadot relay chain to its sovereign account on Statemint. We assume that Polkadot has already paid the fee.
As you can see, this looks very similar to the direct withdraw-buy-deposit model we saw last time. The difference is the InitiateTeleport directive, which is inserted around the last two directives (BuyExecution and DepositAsset). Behind the scenes, the sending chain (Polkadot relay chain) is creating a brand new message when it executes the InitiateTeleport instruction; it takes the xcm field and puts it into a new XCMReceiveTeleportedAsset, then sends this XCM to the receiving chain (Statemint). Statemint believes the Polkadot relay chain has destroyed 1 DOT on its side before sending the message. (And it is!)
The beneficiary is declared as Parent.into() , astute readers may wonder what this refers to in the context of the Polkadot relay chain. The answer is "nothing", but there is nothing wrong here. Everything in the xcm parameter is written from the receiver's perspective, so while this is part of the overall XCM that is fed into the Polkadot relay chain, it's actually only executed on Statemint, so its context is followed by Statemint is gone.
ReceiveTeleportedAsset((Parent, 10_000_000_000).into()),
BuyExecution {
fees: (Parent, 10_000_000_000).into(),
weight: 3_000_000,
},
DepositAsset {
assets: All.into(),
max_assets: 1,
beneficiary: Parent.into(),
},
When Statemint finally gets the message, it looks like this:
The beneficiary is also designated as the Polkadot relay chain, so its sovereign account (on Statemint) is credited with newly minted 1 DOT minus fees. XCM may just easily specify an account for the beneficiary or something else. In fact, this 1 DOT can be moved using a subsequent TransferAsset sent from the relay chain.
mdnice editor
🏦 Reserves
Another way to transfer assets across chains is slightly more complicated. A third party called a reserve is used. The name comes from the bank's reserve system, where assets are "reserved" to lend credibility to certain published promises of value. For example, if we have reason to believe that each "derived" DOT issued on an independent parachain can be exchanged for exactly 1 "true" DOT (such as Statemint or DOT on the relay chain), then we can make the parachain DOTs are considered economically equivalent to real DOTs. (Most banks do fractional reserve banking, which means they keep less than face value in reserves. This is usually fine, but can quickly go wrong when too many people want to redeem.) So , a reserve is a place to store "real" assets, for transmission purposes, whose logic and security are trusted by both sender and receiver. Any corresponding assets on the sender and receiver will be derivatives, but they will be backed 100% by "real" reserve assets. Assuming the parachain is well behaved (i.e. it has no bugs and its governance has not decided to steal the reserve and run away), this will make the derivative DOTs roughly the same value as the underlying reserve DOTs. Reserve assets are held in sender/receiver sovereign accounts (i.e. accounts controlled by the sender or receiver chain) on the reserve chain, so unless there is a problem with the parachain, there is good reason to believe that these assets will be well received protection of.
Going back to the transfer mechanism, the sender will instruct the reserve to transfer an asset owned by the sender (and use it as a reserve for its own version of the same asset) into the receiver's sovereign account, and the reserve (instead of the sender party!) to notify the receiver of their new asset. This means that the sender and receiver do not need to trust each other's logic or security, but only the logic or security of the chain used as a reserve. However, it does mean that the three parties need to coordinate, which adds to the overall cost, time and complexity.
WithdrawAsset((Parent, 10_000_000_000).into()),
InitiateReserveWithdraw {
assets: All.into(),
dest: ParentThen(Parachain(1000)).into(),
xcm: Xcm(vec![
BuyExecution {
fees: (Parent, 10_000_000_000).into(),
weight: 3_000_000,
},
DepositReserveAsset {
assets: All.into(),
max_assets: 1,
dest: ParentThen(Parachain(2001)).into(),
xcm: Xcm(vec![
BuyExecution {
fees: (Parent, 10_000_000_000).into(),
weight: 3_000_000,
},
DepositAsset {
assets: All.into(),
max_assets: 1,
beneficiary: ParentThen(Parachain(2000)).into(),
},
]),
},
]),
},
Let's look at the required XCM. This time we will send 1 DOT from parachain 2000 to parachain 2001, which uses reserve-backed DOTs on parachain 1000. Again, we assume the fee has already been paid at the sender.
WithdrawAsset((Parent, 10_000_000_000).into()),
InitiateReserveWithdraw {
assets: All.into(),
dest: ParentThen(Parachain(1000)).into(),
xcm: /* snip */
}
Like I said before, it's going to be a little complicated. Let's walk through the process. The external part is responsible for withdrawing 1 DOT on the sender (parachain 2000) and withdrawing the corresponding 1 DOT on Statemint (parachain 1000) - for this it uses InitiateReserveWithdraw which does what it says .
/*snip*/
xcm: Xcm(vec![
BuyExecution {
fees: (Parent, 10_000_000_000).into(),
weight: 3_000_000,
},
DepositReserveAsset {
assets: All.into(),
max_assets: 1,
dest: ParentThen(Parachain(2001)).into(),
xcm: /* snip */
},
]),
/*snip*/
Now we hold 1 DOT in Statemint's Holding Register. Before we can do anything else, we need to buy some execution time on Statemint. The process also looks familiar:
/*snip*/
xcm: Xcm(vec![
BuyExecution {
fees: (Parent, 10_000_000_000).into(),
weight: 3_000_000,
},
DepositAsset {
assets: All.into(),
max_assets: 1,
beneficiary: ParentThen(Parachain(2000)).into(),
},
]),
/*snip*/
We use 1 DOT of our own to pay for fees, and we assume 1 million per XCM operation. After paying for this one operation, we deposit 1 DOT (minus fees, and we're lazy, so we just use All.into()) into the sovereign account of Parachain 2001, but this is done as a reserve asset, This means that we also ask Statemint to send a notification XCM to this receiving chain, informing it of the transfer and some instructions to be executed on the resulting derivative asset. The DepositReserveAsset directive doesn't always work; in order for it to work, dest must be a location where funds can reasonably be held on the reserve chain, and where the reserve chain can send XCM to. Brother parachains fit the bill exactly.
ReserveAssetDeposited((Parent, 10_000_000_000).into()),
BuyExecution {
fees: (Parent, 10_000_000_000).into(),
weight: 3_000_000,
},
DepositAsset {
assets: All.into(),
max_assets: 1,
beneficiary: ParentThen(Parachain(2000)).into(),
},
The last part defines the part of the message that arrives at the parachain 2001 . Just like starting the transfer operation, DepositReserveAsset composes and sends a new message, in this case ReserveAssetDeposited . It is this message, although containing the XCM program we defined, that reaches the receiving parachain. It looks like this:
(This assumes that Statemint didn't actually take any fees, and that the entire 1 DOT was passed. That's not particularly realistic, so the numbers for the assets line could be lower.) Much of this message should look familiar ; the only notable difference from the ReceiveTeleportedAsset message we saw in the previous section is the top-level directive ReserveAssetDeposited , which achieves a similar purpose, except that it says "the sending chain destroys the asset so you can mint an equivalent asset", which means It is "the sending chain received the assets and keeps them for you, so you can mint derivatives backed by the full assets". Either way, the destination chain mints them into the Holding Reserve, and we deposit them into the sender's sovereign account on the receiving chain. 🎉
🏁 Conclusion
“
The above is the content of this article. Hope it helps explain what XCM is, and the basics of how it's designed to work. In the next articles, we'll delve into XCVM's architecture, its execution model and its error handling, XCM's version control system; how format upgrades are managed in a well-connected, interdependent ecosystem; its query-response system; And how XCM works in Substrate. We'll also discuss some future directions for XCM, planned features, and the process of evolving it.
IOS
Android