Original author: Muhammad Yusuf, Delphi Digital

Original compilation: Wu Shuo Blockchain

Translator’s Note: This article details the working principles, privacy architecture, order matching, liquidity sources, MEV and censorship resistance of different types of dark pools. By introducing privacy-enhancing technologies such as zero-knowledge proofs, dark pools attempt to provide a high degree of security while protecting transaction privacy. Additionally, the article addresses issues related to regulatory compliance and transaction visibility, as well as exploring different business models in dark pools.

Before we dive into the inner workings of blockchain-based dark pools, let’s take a look at the ins and outs of dark pools, how they emerged, why they continue to operate, and how much they’ve changed the game.

Dark pools in the 1960s

In 1969, when computers were still big enough to take up a room or two and traders shouted orders on the trading floor, institutional investors needed a better way to buy and sell stocks without causing a ripple effect in the market. Jerome Pustilnik founded Instinet, pioneering electronic trading on Wall Street. Institutional investors can place orders and Instinet will match buy and sell orders and execute them. For a platform like Instinet to work, it requires a large number of orders from both supply and demand sides. So, what gives institutional investors a taste of the sweetness? Confidentiality. Instinet allows large companies to operate anonymously by hiding their identities and orders from surveillance by other players and the wider market. On the one hand, this prevents their transactions from affecting the market. On the other hand, it also means that the risk of being preempted by other traders is reduced.

As of 2022, more than 60 dark pools have been registered with the SEC. Some of them are run by exchanges, like the New York Stock Exchange (NYSE) and NASDAQ (NASDAQ); some are run by brokers, like Morgan Stanleys MS Pool and Goldman Sachs SigmaX; and some Are independently operated, such as Liquidnet and MatchNow.

The problem with having a dark pool run by an operator is that the operator may have more incentive to act improperly rather than operate in a compliant manner. The equation is simple: corruption profits > corruption costs. Operators can make more profit than the fines they have to pay. In 2018, the SEC fined Citigroup $12 million for misleading investors when operating a dark pool and leaking confidential order information to high-frequency traders, who executed a scheme against Citigroup clients. Profits from orders worth over $9 billion.

Users of traditional dark pools face the risk of being manipulated by operators. This is the heavy price they have to pay for participating in a broken trust system. Since 2011, dark pool operators have paid more than $340 million in fines to resolve charges. The price paid is relatively small compared to the profits they may accrue. Blockchain-based dark pools attempt to move away from the need to trust operators with potentially malicious behavior, but problems remain.

Mixers, pools, and everything in between

Blockchain is designed to be completely transparent. While it promotes accountability, it is also a double-edged sword. If you receive a salary on-chain, then anyone with your wallet address can see your salary over the years as well as your future salary.

DEXs (decentralized exchanges) and wallets are tracked by wallet tracking platforms and copy trading platforms, the latter of which even allow anonymous actors to snipe well-performing traders, making their trading strategies more complex. Large orders also run the risk of being snapped up by searchers in the public memory pool.

Before we dive into dark pools, let’s differentiate between mixers and dark pools. Mixers are a subset of dark pools. Mixers like Tornado Cash mix coins together to sever the connection between wallets and assets, making the source of funds difficult to trace. Dark pools, on the other hand, not only sever the connection between wallets and tokens, but also help users trade without revealing any information about other trading parties.

Now that we understand the origins and motivations behind traditional dark pools and differentiate between mixers and dark pools, let’s dive into blockchain-based dark pools that use Account/Balance models ) and PETs (privacy-enhancing technologies) such as zero-knowledge proofs, multi-party computation (MPC), and fully homomorphic encryption (FHE) to infuse privacy into its infrastructure.

As of this writing, dark pools like Portal Gate, Renegade, and Tristero are still under development. And privacy protocols like Panther and Railgun are already running to facilitate private DeFi (decentralized finance) transactions, including not only exchanges, but also operations such as staking, providing liquidity, and lending.

privacy architecture

balance model

Blockchain is a state machine. Status consists of accounts and transactions. As accounts transact, each state transition adds a new set of updates to the chain. The account balance will also be updated accordingly. The account balance model is a different way for the blockchain to track and manage user account balances. Blockchains such as Ethereum, Solana, and Polygon use the account model, while blockchains such as Bitcoin, Zcash, and Monero use the UTXO model. These models differ in how they manage and present blockchain state.

The account model, similar to a bank account, displays the current state as a set of addresses and their balances. Whenever you make a payment, you are sending the exact amount, there is no balance left on either party, and no new balance is created. But the UTXO (Unspent Transaction Output) model is not that simple. When you make a payment, your entire balance is transferred, burning your existing balance. The remainder of what you own is added to the new balance. The UTXO model displays the current state as a graph of spent and unspent outputs for all transactions.

To illustrate this, consider the following scenario:

Alices wallet contains two UTXOs, one with 6 coins and the other with 5 coins, while Bobs wallet contains only one UTXO with 5 coins. When Alice sends 5 coins to Bob, she merges her two UTXOs into a single transaction, making it difficult for an observer to determine the origin of the 5 coins she sent to Bob, and where the remaining balance in her UTXO goes. . This privacy-preserving mechanism effectively masks the flow of funds and ensures the privacy of transaction participants.

The newly generated UTXO has no known owner and requires more advanced on-chain analysis to associate it with a single user. The account model is fungible. It encourages address reuse, making it easy to track an individual users history.

In the UTXO model, double-spend is quite difficult. Each UTXO has a unique nullifier generated by itself. When a transaction is verified, that is, when the UTXO is spent, its nullifier is also spent, which prevents the transaction from being spent again.

Several versions of the UTXO model are already used in privacy-focused L1 and L2. The Record model used by Aleo is a variant of the UTXO model. Aztec and Polygon Miden use a mix of UTXO and account models.

The UTXO model used by Zcash largely inspired the UTXO model used in dark pools. Some protocols modify it to support multiple assets and accounts. Protocols like Portal Gate, Penumbra, Railgun, and Renegade use a note-based mechanism, where each UTXO is a note. These notes, like UTXOs, contain information about the asset, order and value, but are only visible to the owner/user. These notes are stored in the Merkle tree that stores the chain state, called the State Commitment Tree. These notes may be generated by transfers or settlements within the dark pool, or they may be generated by payments to the relayer. Renegade uses embedded encapsulated multi-party computation (MPC) to match UTXOs (orders) and update private state. Multiple notes can also be stored in a single note.

In the Panther protocol, each asset or account is represented as zAsset (privacy asset) or zNFT (privacy non-fungible token), and the UTXO model operates through a Merkle tree named Bus to process UTXO in batches. Each batch contains up to 64 UTXOs, optimizing users cost efficiency."Bus operator"Responsible for the process, while users incentivize them with rewards denominated in $ZKP. When executing a single UTXO, Panther uses a so-called Taxi Merkle tree. To support UTXOs across multiple chains, there is also a Merkle tree called Ferry.

proof system

This is a major component of Privacy Enhancing Technologies (PETs). Zero-knowledge proofs (ZKPs) are used to prove the ability to possess correct information without having to give it away. In the context of dark pools, ZKPs can be used to prove that users have sufficient funds and permissions to place orders, have them matched and settled, while hiding these details from dark pool operators, other traders and external third parties. arrive.

Proof systems are cryptographic constructs designed to generate these proofs. They come in various types, each producing different-sized proofs, requiring different time frames, and consuming different computational resources during generation and verification. Here we will take a deep dive into two important proof systems: Groth 16 and UltraPlonk, which have found use in the dark pools we discussed.

Panther, Penumbra, and Railgun utilize the Groth 16 proof system to generate their proofs. Groth 16 is known for generating consistently sized proofs, making it a preferred choice due to the impact proof size can have on transaction size, gas cost, and throughput. Portal Gate and Renegade, on the other hand, use the UltraPlonk proof system.

Both Groth 16 and UltraPlonK require a trusted set-up. In this setting, a group of trusted parties collaborate to establish a common set of parameters. These parameters are used to verify the proof in a way that the verifier does not have to trust the prover, and the prover does not have to trust the verifier. It is worth noting that a trusted setting implies an assumption of trust for the parties involved. Secure multi-party computation can be used to prevent any one party from gaining access to the underlying setup structure. Groth 16 relies on input from multiple parties to establish its trusted setup, while UltraPlonk only requires the participation of one party.

The inner workings of dark pools

Now, let’s take a look at the inner workings of dark pools, how they protect privacy when conducting transactions, provide liquidity, prevent MEV withdrawals, censorship-resistant features, and comply with regulations.

Wallet settings

In most cases, a new dedicated wallet will need to be created to interact with dark pools. Railgun has a third-party wallet provider called Railway Wallet, which allows the creation of a cryptographic EOA, allowing you to hold tokens/NFTs without exposing the amount and type, and can also be called privately through the Relay Adapt mechanism Any public smart contract.

In Renegade, a wallet is equivalent to a UTXO that has a private key and can sign transactions. You need to generate a new UTXO and submit it with a ZK proof. This proof proves that the new UTXO is actually new and that you have not spent the funds within it. Most operations such as depositing, placing orders, and trading will invalidate the wallet, so you will need to create a new wallet to ensure no double spending occurs. The user experience is the same as other Ethereum wallets.

To use Penumbra, you need to create a new wallet just like with other Cosmos chains. Afterwards, the wallet can be funded via another IBC transfer.

For both Panther and Portal Gate, a KYC process has been established. Ethereum wallets are available, but connected wallet users must verify their passports to obtain credentials to interact with the protocol. Business entities need to undergo a KYB process where they share their business details. Verification is performed off-chain by a registered compliance provider. These credentials expire periodically and must be revalidated.

Order discovery and matching

When it comes to order discovery and matching, common DEXs like Uniswap and Curve employ an automated market maker approach where orders are matched to liquidity pools. This approach is not optimal for privacy because when an order is matched to a public liquidity pool, the order details are publicly visible on the blockchain. This is not ideal for traders making large trades as it may reveal their intentions to other participants in the market who may take advantage of this information. Some dark pools will eventually match liquidity pools or use other external public contracts for order matching, and they use relays as a wall that separates traders from external liquidity sources where trader privacy must be protected. Next, we’ll explore how dark pools capture liquidity while protecting privacy.

Now, keep in mind that traders have deposited funds into wallets in dark pools. Now we will discuss what happens when an order is submitted. Some dark pools take a peer-to-peer order matching approach, while some take a peer-to-pool approach, using PETs such as zero-knowledge or multi-party computation, while their internal contracts interact with external public contracts. Taking a peer-to-peer approach or crossing orders between traders can help with better execution without slippage. Because if an order matches a liquidity pool, the price in the liquidity pool may change between the time the order is submitted and the order is executed. When orders cross between traders, the order will be executed at the price negotiated between them.

With Renegade’s peer-to-peer approach, when a trader submits an order, they also select a relay that can bid (match the order) on their behalf and pay a fee. Traders generate a ZK proof called VALID COMMITMENT, proving that they own their wallet and their orders. A handshake is then sent to the network to match other traders handshakes. The handshake is essentially an immutable list containing the VALID COMMITMENT ZK proof, hashed order details, nullifier and key pair. When a traders handshake matches another traders handshake, they perform multi-party calculations. Renegade uses collaborative ZK-SNARKs to prove that a traders order does indeed match an opponents order. This ensures that order details remain hidden from other parties while order matching is taking place. Once an order matches all required proofs, a new wallet is created to hold the exchanged tokens, which is then verified to ensure that the wallet is indeed new to prevent double spending. The user experience is the same as using any Ethereum wallet.

For Portal Gate, when a trader submits an order, an API node receives the order, encrypts it, generates a ZK proof, and then relays the order to Book. Book is a set of off-chain nodes that collect orders and run order matching algorithms in an FHE (Fully Homomorphic Encryption) environment. This means that encrypted orders do not need to be decrypted to be matched. Once an order is successfully matched, the total results for all order matches will be displayed without revealing the details of individual order matches. This also helps protect privacy.

For Railgun, when a trader places an exchange order through Railgun, a user named"Adapt Module"The smart contract performs multiple operations, namely obtaining the private balance specified by the trader in his wallet to be exchanged and lifting his privacy mask. The Adapt Module will also verify whether the order (UTXO) has been voided/spent. The 0x API then swaps assets between aggregated DEX liquidity, looking for the best exchange rate. The Adapt Module privacy-shields exchanged assets into private balances to ensure their activity or addresses are not leaked. This workflow can also be applied to other DeFi transaction types.

The Panther protocol operates similarly, where it uses a module called Zswap to connect Panthers MASP (Multi-Asset Privacy Pool) with other DeFi protocols. Rather than being so much a pool, MASP is more like a set of append-only Merkle trees, where each leaf is a UTXO commitment to the asset deposited in MASP. When a trader creates an order, Zswap aggregates quotes from other DeFi protocols for the trader to choose from. Once an order is placed, Zswap creates a cryptographic time-based escrow contract that can privately facilitate exchanges between users without revealing any details. Once the asset is exchanged, the user will receive an IOU (IOU) of the token in the form of ZAsset (ZNFT) from Panther’s MASP.

Penumbras swap module is also called Zswap, but it works on a completely different principle. When a trader creates an order, the assets in the traders private balance are burned and the encrypted order value is sent to the validator. There is also a fee to process your order. Users will receive a swap NFT to save a record of the event. NFTs are composed of a combination of various parameters, including transaction assets, fees, input amounts, and encryption keys. Validators aggregate order inputs from multiple traders into a batch and then execute these inputs against centralized liquidity positions. Once executed, outputs are delivered to traders in proportion to each traders input. In this case, individual data remains private, but aggregate data is exposed when validators aggregate orders for execution against centralized liquidity positions.

Looking for liquidity

Dark pools like Renegade and Portal Gate operate by anonymously crossing buy and sell orders directly between users, but there is not always enough liquidity to fill the demand for the orders. In a general DEX, you can almost guarantee that your order will have sufficient liquidity, but you need to give up privacy and price impact. Privacy - Anyone can view what you are selling or buying. Price Impact – Because guaranteed liquidity comes with a premium, you need to pay it along with your order. In dark pools, order information remains hidden before and after a trade.

Renegade follows this design philosophy and provides liquidity by matching order flow with reverse order flow without going through any intermediaries. In the event of insufficient liquidity, there is a mechanism called Indication of Interests where traders can disclose certain details of their orders to the relay that matches the order. Efficiently prove details such as price, size, asset, and order type with zero-knowledge. If a trader chooses to disclose these details, other traders will be able to view these orders pending on the order book, potentially resulting in these orders being executed more quickly.

The Panther Protocol is not just a dark pool. It also offers a range of other DeFi activities that can be performed privately. Users first deposit funds into the multi-asset privacy pool and represent the deposited assets as ZkAsset (ZkNFT). ZkAsset is essentially an IOU token, similar to stETH you get when staking on Ethereum. Plug-ins called DeFi Adapters can privately connect MASP with existing DeFi protocols. Zswap and Ztrade are adapters that connect MASP with DEXs such as Uniswap, Quickswap, and Curve.

Like Panther or Railgun, Penumbra is a gateway to a variety of DeFi activities. In Penumbra, when a user opens an LP position, they actually create their own mini-AMM. In this way, there will be tens of thousands of centralized liquidity positions, which are created by users and set the fees they want. However, this may result in liquidity fragmentation, but the Zswap/DEX engine will consider and synthesize all these individual liquidity positions into a single AMM. This will allow incoming trades to be routed throughout the liquidity graph.

Portal Gate uses an anonymous AMM DEX called Automaton as a backup for Book, that is, the order book run by the relayer is used to cross-dot users transactions. If the order book is unable to facilitate a trade due to lack of liquidity, the order will be routed to Automaton. Liquidity will be channeled in Automaton just like in any other AMM.

Railguns Railway DEX uses the 0x API created by 0x. This means that orders for Railway DEX are routed to find the best price executed in the 0x API DEX aggregator. Once a trade route is found, the wallet generates a proof to exchange using funds from the traders balance and blocks incoming tokens from the 0x API in the traders balance.

business model

Traditional finance (Tradfi) dark pools are largely seen as an elite club where only those with deep pockets can trade. They pay a membership fee to be able to place orders anonymously while also having access to associated reverse order flow from the other party. Trading fees are generally lower than traditional financial exchanges. Traditional financial dark pools can also act as brokers, buying securities from a seller at a lower price and then selling them to another buyer at a higher price.

Dark pools in the crypto space are moving away from “operator”-centric designs and playing more of a “facilitator” role. They are designed to generate revenue through two main components: protocol fees on successfully matched orders, and a portion of the fees traders pay to the relayers who process transactions. A portion of the relay fee may also go to the relay operator. Traders can also choose to run their own relay to gain better privacy and avoid paying relay fees.

Decentralized dark pools in the crypto space may face a “cold start” problem. Unlike DEXs that trade against liquidity pools, most dark pools let users trade with each other by matching buy and sell orders. It is important for a system like this to have traders placing orders on both sides of the market. It’s also important to note that dark pools in traditional finance are hosted by multi-billion dollar banks like Goldman Sachs and exchanges like the New York Stock Exchange. This way, providing order flow to its users is relatively easy. There were also independent dark pools, such as Liquidnet, which had estimated that at least 100 buy-side firms would be needed to run on their platform from that day forward to reach critical mass, but only 38 firms ended up coming online. Liquidnet is now among the top three dark pools in the world.

Centralized exchanges average 10x more trading volume than decentralized exchanges because they attract a wider user base, provide better user experience and compliance, thereby increasing supply to meet demand. Liquidity for OTC trading (over-the-counter) is relatively difficult to obtain.

Most OTC trading in the crypto space is conducted through OTC desks (over-the-counter trading desks) and escrow smart contracts, sometimes even through Telegram groups. This is a shocking phenomenon. People get scammed all the time. It is very convenient for scammers to impersonate others on Telegram and edit messages and transaction details. With dark pools, you may not be guaranteed instant settlement of illiquid tokens or NFTs, but it ensures a higher level of security and privacy. OTC desk clients also do not need to pay broker fees for OTC trading through dark pools.

Trading volume is critical for dark pools to operate effectively. It helps match orders efficiently and protects traders’ anonymity. The more anonymous funds there are in the pool and the more orders there are in the order book, the harder it is to connect depositors and traders to their assets. This is because there are more potential depositors to try to link the asset to. When protecting assets, it is also important to consider the type of token. Securing a common stablecoin like USDC or DAI will provide greater anonymity than protecting a meme token that is less well-known and has few depositors.

MEV and censorship resistance

Most DEXs running on public chains allow MEV to exist because they are open and transparent ledgers. Order details on DEXs like Uniswap or Curve are not confidential, which allows searchers and builders to rearrange transactions in bundles and blocks accordingly to extract MEV.

Privacy is important, but its a subjective issue. Tornado Cash excels at allowing people to conduct financial activities anonymously on-chain. Since it is open source and free for anyone to use, there are also criminals who use it to launder stolen funds. North Koreas hacking group Lazarus has so far stolen nearly $1 billion ($958 million). OFAC (Office of Foreign Assets Control) maintains a list of Specially Designated Nationals and Blocked Individuals. Malicious individuals and entities, such as Lazarus and even Tornado Cash’s smart contract addresses, have been placed on OFAC’s list for non-compliance. Protocols that are not OFAC compliant have a high likelihood of their transactions being excluded from on-chain blocks. Is this a serious anti-censorship issue? Yes. The problem is not limited to block producers. Validators and relayers can also choose to ignore transactions or blocks.

In dark pools, since every order generates ZKPs, order details are mostly hidden from all third parties, including validators, eliminating the need to disclose order matching or even settlement details. This architecture is MEV-resistant and censorship-resistant, or at least so long as the trading workflow remains premised on dark pool encryption. Opportunities to capture MEV may arise again in situations where a public contract is called or a shared sequencer is used.

Compliance

Whether you are operating within the protocol or as an individual user, the last thing you want to see is that the transaction is pending and not processed within the block, thereby jeopardizing the success of your state transition, or having the SEC come to investigate. tea party. For the founders and developers of Tornado Cash, these fears turned into a terrifying reality when they were arrested on money laundering charges. TC frontends are also illegal/deactivated in many countries and are not supported by some centralized exchanges. But the TC smart contract itself can still be used by people with certain technical knowledge.

To try to avoid this, dark pools and other privacy-focused protocols have found ways to let users decide for themselves whether to comply.

Renegade allows each trader to select a set of counterparties. Individual traders may only want some basic AML/sanctions checks on their counterparties, while institutions may want to run KYB/KYC checks on their counterparties. This counterparty selection logic using ZKPs for compliance checks can be set up before the MPC begins.

Portal Gate and Panther Protocol work with compliance oracles (oracles), which in turn work with traditional compliance service providers to perform KYC/KYB verification of user accounts off-chain. Once verified, users will receive a ZK certificate of compliance. This proof allows their wallet to be used with the dark pool protocol, but verification needs to be done regularly.

In Penumbra, users can demonstrate their compliance off-chain using a “transaction perspective”, selectively disclosing their activities, including funding sources. Likewise, in Railgun, users have view keys that allow users to view transactions and balances from block time X to Y.

Railgun uses the Proof of Innocence algorithm developed by Chainway and discussed in the Privacy Pools 2.0 paper. Proof of Innocence helps you prove that your transaction is legitimate without revealing any transaction details. The Merkel tree acts as a collection of all previous UTXOs (transactions, balances). With this feature, it is possible to determine that a specific balance is part of a certain set of transactions. But to prove that a specific balance is not part of a certain transaction set, you need to use a Sparse Merkle tree that records NULL values. This proof verifies that the balance is not part of a certain transaction set. In this case, recursive SNARKs are used to prove that the chain of proofs from the initial deposit to the final withdrawal is calculated accurately, as proof that the users balance is not part of a certain set of transactions.

lingering thoughts

Dark pool users rely on public prices as a benchmark for their traders, but the price of assets on public exchanges may not reflect the fluctuations that occur within dark pools. Prior to 2014, price discovery between dark pools and public exchanges was a one-sided relationship. The U.S. Financial Industry Regulatory Authority (FINRA) and the U.S. Securities and Exchange Commission (SEC) implemented an initiative requiring that post-settlement dark pool trading information be publicly released on a security-by-security basis with a delay of 2-4 weeks. It is unclear whether similar public disclosure standards will be applied to crypto dark pools in the future to promote better price discovery for all market participants.

A pure dark pool must accumulate order flow on both sides of the market to reach a critical point. We may see a lot of OTC trading move to dark pools as they will be more secure, private and cost-effective than OTC desks and Telegram groups. In the case of private DEXs interacting with public liquidity pools or external smart contracts, simulation of leaked information may lead to the exposure of certain transactions and traders.