Dry goods | A comprehensive overview of decentralized oracles
Editor's Note: This article comes fromEthereum enthusiasts (ID: Ethfans)Editor's Note: This article comes fromMediumEthereum enthusiasts (ID: Ethfans)
, translated by Anzi Clint, wuwei & A Jian; the original text comes fromFabric Ventures, by Julien Thevenard; reproduced by Odaily with permission.Joey KrugHere I would like to thank ourTomas Bertaniteam, and thanks toHugh Karp(Augur core developer),James Ryan Moreau(Founder of Oraclize),Gorka Irazoqui Apecechea(Founder of Nexus Mutual),Szymon Sypniewicz(Witnet Technical Community Leader),Przemysław Kowalczyk(Witnet Researcher),
(co-founder of Ramp) and
(Ramp founder) gave us great feedback.
Distributed ledgers and smart contracts can eliminate the friction caused by trust issues in current human interaction and collaboration, and bring unprecedented great innovations to society. But until then, if the above smart contracts cannot obtain off-chain input without additional trust, then the so-called innovation is only on paper.
Decentralized oracles are gateways through which smart contracts interact with the outside world, designed to work without relying too much on a single source of information. How to implement the decentralized oracle machine will also cause another wave of innovation.
This article aims to introduce the following points:
Why oracle services are a decentralized technology
Key components of the stack require trustless services
Overview of 8 projects working to address the aforementioned challenges
I. Why do we need oracles?
I. Why do we need oracles?
Smart contracts on permissionless chains (such as Ethereum, Dfinity) run in an adversary environment, and their security is guaranteed by only propagating deterministic transactions (that is, transactions verified by all nodes) in the network. The design idea of the smart contract is: accept input → execute logic → update the corresponding blockchain state, and this process is irreversible, there is no Ctlr+Z retracement in the blockchain world.
Allowing smart contracts to obtain data from outside the blockchain system is a double-edged sword. While greatly expanding blockchain application scenarios and empowering blockchains to interact with the outside world, it also introduces certain trust problems. Miners in a permissionless chain are not sure that they can verify all external inputs, so they can only indiscriminately perform any operation that meets the preconditions of the smart contract.
Before diving into the many technical details of trustless oracles, let’s look at five key use cases for smart contracts powered by oracles.
Financial product
Financial product
Financial productCDx、dYdX、MakerDAO、Vega Protocol、0x protocol 。
Some projects have the ability to even apply such oracles to financial products, including:
Smart Contract Insurance
Etherisc andFizzy With the blessing of trust-free and reliable information sources, certain insurance products can be implemented in the form of smart contracts through programming. The biggest cost in the insurance industry is fraud, so the importance of oracles is self-evident. Some current project examples of smart insurance include:
Nexus Mutualand
FlyingCarpetAutomatic compensation for flight delays has been realized.
A new type of programmable insurance that enables artificial intelligence and geodata.
freightFOAM freight
For example, it can lower dApp's trust threshold for the original data source, and rely on the decentralized network (in this scenario, a location oracle) to ensure the credibility of the location information declared by the source.
Collateralized Lending and StablecoinsMaker DAOofDAI ofReportIt is a stablecoin endorsed by Ethereum (will be upgraded to multi-asset endorsement soon), it uses multiple oracles in the design to
The price of Ethereum makes it possible to trigger the liquidation operation in time when the released stable currency does not have enough collateral endorsement, thereby protecting the entire system. These oracles are chosen and funded by Maker token holders.Ethlend andDharma also has a similar design.
prediction market
prediction marketAugur，Gnosis prediction market
) has brought human collaboration to an unprecedented new level, making full use of the wisdom of groups. These markets must rely on one or more oracles to determine off-chain events.
II. Ensuring oracle integrity
II. Ensuring oracle integrity
Blockchains are deliberately built to isolate the outside world and third parties that require additional trust. However, most events still happen off-chain, so we need to bridge the blockchain and the off-chain world without compromising censorship resistance. In fact, the trustlessness of a dApp depends on the weakest link between the on-chain and off-chain worlds, so a single source of information that can be corrupted is not enough.
Accessing multiple information sources can achieve higher security in terms of probability, but it will increase a lot of costs accordingly. The number of information sources required for a specific application scenario may be more or less. In practical applications, we should adopt a risk-based design idea to determine how many information sources are needed for different applications.
In general, it is necessary to balance the cost of the established oracle system according to the amount of funds involved, and find an oracle solution suitable for your own scenario in practical applications.
swiss cheese model
It is very difficult to ensure that the third party will not do evil all the time. Solving this trust problem in a centralized world requires multiple layers of protection: contracts, trusted companies, insurance, laws, etc. As long as at least one protective layer remains intact, the system can be considered to remain honest. However, if all protective layers are corrupted, the attack will work (Swiss cheese model).image description）-
- Adapted to the Swiss cheese model (
image source
The LIBOR scandal is a lesson in over-reliance on centralized oracles. The London Interbank Offered Rate, or LIBOR, is the commonly used rate reference in commercial and private lending, underpinning some $300 trillion in loans. However, according to the sources, the index was manipulated by certain swap traders as far back as 2005 or 2003. LIBOR is also an indicator of how healthy banks are, so manipulation of this rate also led to some institutions appearing healthier than they actually were during the 2007-2008 financial crisis.
There is no perfect system in the world, but we can apply the concept of multiple barriers to the decentralized oracle system, so as to improve the minimum trust mechanism as much as possible.
A. Multiple data sources
The easiest way to reduce data errors is to use oracles to aggregate multiple data sources. In this way, there are only two possibilities for collecting outrageous data: one, most of the data sources are corrupt, and two, the oracle itself is compromised (becoming a single point of failure problem).
B. Multiple Oracles
When increasing the number of oracles, the majority of oracles are probabilistically not malicious, so as long as the majority of the system remains honest, the system is secure. However, whether it is intentional manipulation or inadvertent (the source of information itself fails), the possibility that all oracles are delivering wrong information is not ruled out at this time.
C. Benefit sharing
C. Benefit sharing
It is very dangerous to allow anonymous participants in a decentralized network to act as oracles. In a decentralized system, once they do evil to obtain economic benefits, there are no legal provisions to recover those ill-gotten gains. Using the design of token engineering, nodes in the decentralized network can be forced to pledge a part of the margin/deposit, and these assets are usually the native currency in the system. When the node works well, it will get a certain reward, and if the node does evil, it will lose the pledged assets in a certain proportion. The above mechanism ensures that the oracle system has a positive incentive mechanism to promote participants to generate accurate data.
D. Trusted Execution Environment (TEE)
D. Trusted Execution Environment (TEE)
Intel's recent SGX (Software Guard eXtensions) and ARM's TrustZone both belong to the trusted execution environment. Since the two are relatively similar, let's take Intel's products as an example to introduce.
In short, SGX allows programs to execute in the CPU's enclave, giving user-level code hardware-level protection. First, enclaves protect applications (data, code, and control flow) from being corrupted by other processes. Second, the enclave protects the confidentiality of the application, that is, the data, code, and execution state of the program are theoretically opaque to the rest of the operating system, but the program can still read and write memory outside the enclave area.March-2018 & July-2018SGX seeks to protect programs in enclaves on malicious operating systems, and thus even protect applications from being compromised by system administrators on nodes. Running the oracle program and distributing data in the enclosure provides a strong guarantee that the oracle program will run safely, since it can be detected remotely that the system is running on a legitimate SGX system.However, it is worth noting that there have been two vulnerabilities since the release of SGX () have been discovered one after another, and there are still researches to explore
Some
The above-mentioned protective barriers cannot be considered foolproof when taken out individually, but when they are combined to work together, they can have a stronger protective effect. In the next section, we will introduce the main attack surfaces in the decentralized oracle system and illustrate the corresponding defense measures with examples.
III. Sybil Attacks and Other Decentralized System Weaknesses
III. Sybil Attacks and Other Decentralized System Weaknesses
It is not difficult to build an oracle system, what is difficult is the design of the trustless component.
Key risks: There is a conflict of interest between parties who rely on smart contracts to adjudicate matters, and anonymous oracles have no risk of litigation. When running a multi-oracle system, the oracles must reach a consensus because the smart contract only accepts a single input. Therefore, in order to defend against attacks, a mechanism needs to be designed to ensure that the oracle meets the following requirements:
unable to recognize each other
Cannot communicate with each other: an oracle with a considerable voting power (say 40%) can broadcast its own answer, and does not need to deliberately convince other nodes that its result is the mainstream answer. However, if other nodes already know that this node has such a large voting power, then this requirement is relatively meaningless.
The following attack strategies or vulnerabilities are valid for decentralized oracle networks.
Majority attack: There is a risk that a majority of the nodes in the network are controlled by a single entity or confederation. At this time, the network is still controlled by the majority, but it has been maliciously manipulated. Special attention should be paid to such risks in decentralized oracle network projects, and the power of nodes needs to be determined according to the reputation of nodes and the total number of nodes.
image description
-Majority Attack - Source: Narcos-
Mirroring: This is a special Sybil attack in a decentralized oracle network. In order to reduce the cost of node operation, the controller of a certain node can only collect data once through one node, and then pass the data to other nodes under his control under the chain. When the data he transmits is still true, this kind of attack is not very harmful, but if there is a problem with the transmitted data, it will greatly reduce the security of the entire system, because the mechanism of the system to refine the accuracy of data through multi-party queries is destroyed. broken.
Empty pay: When an oracle maliciously copies the answers of other oracles, we call it empty pay. This problem can be solved through the "commit-reveal" mechanism, that is, the answers submitted by the oracle machines are encrypted, and only when enough oracle machines have submitted the answers, all the answers will be decrypted.
Data confidentiality on the chain: If the data request is both sensitive and private, even if the data request is encrypted, the request information will be unconsciously revealed when the final data is reported. The solution to this problem is to force nodes to perform decryption in a TEE enclave environment and report a common answer to the blockchain that all users and nodes can see. For example, for a flight insurance, the user may not want others to know that he is flying from London to New York. Therefore, after knowing the specific flight, the oracle machine can only answer "is that flight delayed?" on the blockchain, or other questions that only need to be answered with yes/no without further disclosure of information.
IV. Projects working on decentralized oracles
IV. Projects working on decentralized oracles
All projects can be divided into two categories: providing oracle services through the network, and providing oracle services on the network.
1. Oracle as a service
ChainLink (already live)
Chainlink diagram
The security of any system is determined by its shortest board. Decentralization guarantees availability at all times, but there is a risk of invalid nodes delivering wrong data, for which ChainLink provides two solutions.image description)-
- On-chain aggregation (
image source
Initial Solution: On-Chain Aggregation
The contract aggregates the data of all nodes on the chain (can be publicly audited), and uses the commit-reveal method to prevent nodes from obtaining the results of other oracle machines and plagiarizing each other. The final result after aggregation will be determined after enough results are published on the chain, but this will bring a lot of computing costs: each node will initiate a transaction, each transaction needs to reach a consensus, and it needs to be in the ether The square deploys one or more aggregation contracts.
Off-chain aggregation can improve the economics of reaching consensus, but it cannot solve the above-mentioned problem of eating out money.
The off-chain aggregation scheme utilizes Schnorr signatures: each oracle participating in a task receives a [public key, private key] combination for that task. The oracle uses the private key to generate a partial signature for the (requested) data result (encrypted). A single partial signature cannot be used directly, but when a sufficient number of partial signatures are combined (as shown in the figure below), a collective signature will be formed, which is equivalent to the aggregation of data results by an on-chain transaction.image description)-
The key word here is a sufficient number of partial signatures, which also means that the scheme provides fault tolerance (when some nodes fail to respond). The downside of this approach is that even honest nodes are not rewarded if messages take too long to propagate - making nodes dependent on response time. This solution also solves the problem of plagiarism, because by the time the results are announced, it is no longer possible to submit new answers.
you can athereherewhite paper。
Witnet
Find out more about ChainLink. The diagrams for this article are taken from its
white paper
Witnet is a reputation-based decentralized oracle network: Whether a node running the Witnet software responds to a data request correctly determines whether it gains or loses a reputation value, and the correctness is determined by the consensus algorithm after analyzing the node's result set . Nodes that are inconsistent with the consensus will lose their reputation value (for example, they have an inconsistency due to the node going offline or trying to do evil), thereby distinguishing them from normal nodes. When the process of reaching consensus times out, as long as the final node result is consistent with the consensus, it will not be punished.
Oracle nodes, called witnesses, are randomly assigned tasks to mine blocks based on their network reputation, making them less susceptible to "majority attacks". Well-behaved nodes can quickly increase their reputation value and take on more responsibilities in the network; on the contrary, invalid and malicious nodes will quickly lose their reputation in the network, and then can no longer contribute to the network, and eventually become insignificant .
Because the reputation value in Witnet is so important, in addition to the transfer of reputation values between good and bad nodes (after completing a task), the system also designs a fixed reputation value redistribution mechanism among all nodes, The redistribution interval is once a block (90 seconds), so as to avoid: A. Centralization caused by the concentration of reputation values in the oldest honest nodes; income).
In each block production, the late fee function will be used to realize the redistribution of the reputation value: the reputation value of the effective node will be reduced by a logarithmic function each time, and the reduced reputation value will be shared by all nodes with good behavior. In other words, the reputation value of all nodes will be continuously reduced, and those nodes with the most reputation value will lose the most. Therefore, in order to remain at the top of the Witnet, good behavior must be maintained at all times.https://witnet.ioyou can athereFind out more, or at
Oraclize
here
Read the white paper.
Their solution is to use all TEE providers to reduce the vulnerability, a practice known as sandboxing. Oraclize utilizes products from IT vendors and manufacturers (including Amazon's EC2, Google's SafetyNet, Qualcomm's QSEE, Ledger's Nano S and Intel's SGX) as key components of their core services. These products are physically divided into independent groups and can be used collaboratively through software: Oraclize designed an ad-hoc-enabled custom program as a software access layer to support the access and interoperability of all TEEs. When a certain technical method is affected by a vulnerability (such as Intel's SGX Specter vulnerability), the overall data aggregation can still ignore the affected node and obtain correct data from other multiple TEEs. (This assumes that the vulnerability is based on a specific architecture, rather than a general vulnerability affecting all processors)hereTo ensure distributed trust and data correctness, Oraclize uses TLSNotary to sign the TLS data of https sites. There is a price for doing this: Oraclize can theoretically only transmit the data displayed by the website (it cannot be uploaded after off-chain preprocessing), but this is enough to cover commonly used scenarios. The main risk of the solution is that when most data sources are destroyed, there is no way to prevent the continued distribution of wrong data, but this risk will exist in most "decentralized" solutions. you can at
Town Crier
here
Learn more about Oraclize.
Behind this academic project is a team of five Cornell Ph.D. and undergraduate students.
Town Crier can act as a bridge between smart contracts (on any blockchain) and https-enabled websites, and it uses a TLS layer to ensure secure communication between the two to transmit authenticated data. This scheme is different from TLS-notary (only software layer security), which allows more customized data transmission. The data is decrypted and processed by those nodes running on Intel SGX (both have hardware and software security levels). Authorization data is finally sent from the enclave to the blockchain, only relying on SGX to ensure that the node software runs as expected.
In order to solve the order failure problem, their method is to perform data source aggregation and oracle deployment on multiple SGX platforms. The software is proven to be somewhat scalable and can support throughput of 15-65 transactions/second.AnnounceYou can find out more about Town Crier here. In the fourth developer meeting on November 1, 2018, CEO Sergey
News about ChainLink's acquisition of Town Crier.
2. Internal oracle service
2. Internal oracle service
A. Prediction Market
“Prediction markets are like betting markets. You bet on the outcome, and the price of the bet is determined by the probability of the outcome.”
Augur and Gnosis are two of the most well-known decentralized prediction markets, both launched in 2015. They build elegant prediction markets using collective intelligence and a less contentious decentralized network. For markets to function correctly, both have to rely on well-designed incentives. We will take Augur as an example for further analysis.
Augur is a trustless, decentralized oracle and prediction market platform that allows geographically dispersed individuals to speculate and report on the objective outcome of any event. For Augur, it is relatively easy to create a decentralized prediction market, the key lies in how to ensure the integrity of the market (guaranteeing honesty can become the most profitable choice for rational Augur nodes).
image descriptionsource）-
- Explanation of the life cycle of the prediction market (
source
Anyone can create prediction markets for upcoming events on the Augur platform. The creator sets the end time of the event and specifies the reporter of the event result. This result allows for queries from the community.
Below is a list of measures Augur has taken to protect honest participants:
Market creators need to hold validity bonds: when markets are imperfectly defined, or when outcomes are impossible to be objective and accurate, these bonds become invalid and zero.
Market creators hold no-result bonds: These bonds will expire when the creator-designated result reporter does not give a result within 3 days. At this point, anyone can submit provisional results for this event.
Once a provisional result is reported, any REP holder has the opportunity to challenge the provisional result (initiate a bid) within 7 days. The bidder whose bidding deposit accounts for the smallest proportion of the projected market value can win the bidding. Once the bidding fails, the pledged REP will be returned to the owner; then the entire market enters settlement.
If the first interim result bid is successful, the next steps depend on the amount of deposits in the current round. When the pledge is less than 2.5% of all REP (currently $1.5 million), the market will start the next round of bidding; otherwise (greater than 2.5%) enters the fork state.
The fork state will last for 60 days, during which time all markets will be temporarily frozen until they are migrated into a "New Augur World". If all markets have only two options (yes/no, selected/lost, win/lose, etc.), 3 worlds will be generated at this time, for example, there will be 3 possible truths: one truth is yes, one One is no, the last one is invalid. Markets are divided into corresponding worlds. Users can and can only migrate their REP tokens to one of these worlds. If the user completes the migration of REP tokens within 60 days (investing in one of the results), an additional 5% of REP can be obtained as risk compensation. The risk here is that when users migrate to the wrong world, their REPs get stuck (REPs are not interchangeable in A/B/C worlds). If users don't want to take the risk, they can migrate after 60 days, at which point the market has reached the correct consensus. Since parallel Augur worlds can co-exist (e.g. Ethereum/Ethereum Classic), the clear winner who wins most of the market activity will be the real Augur.
We hope readers realize the dangers of poorly defined prediction markets when they are created, and the weird economic consequences of having decentralized oracles controlling the broadcast of outcomes.
When the Augur oracle machine completes an Augur market prediction activity by reporting the results of real-life events, we can imagine that when the incentives are sufficient and Augur continues to grow, the results of the Augur market itself can be used as a reliable oracle machine. In other words, we can assume that the Augur market will eventually (probably) give a true outcome, and use that as input for another smart contract.
Ramp
More information on Augur can be found here.
B. Payment Services OpenBankProject Bank ledgers have historically not been easily accessible externally, but things are changing fast and more regulators are embracing open banking frameworks. For example, in Europe, the PSD2 directive requires European banks to open API access to bank accounts to authorized third-party providers, and Ramp is one of them.
-(Ismail Chaib andRamp Swapscoverage) -
*Ramp* is building financial infrastructure to secure trust-minimized communications between traditional financial institutions and public blockchains. In practice, they use PSD2 to provide p2p fiat and cryptocurrency atomic swap services (
). The process is as follows:
The seller creates a smart contract that locks a certain amount of digital assets in third-party custody. When the buyer sends legal currency as agreed, the digital asset will be released to the buyer, otherwise it will be returned to the seller.
The buyer pays the seller by wire transfer.
A payment oracle verifies the buyer's payment and sends a proof of payment (PoP) to the network. The oracle is able to access the accounts of buyers and sellers and see their incoming and outgoing payments.
The smart contract unlocks the asset in third-party custody and sends it to the buyer.
The data on a financial institution's ledger is extremely sensitive, so reporting such information presents an obvious challenge compared to reporting publicly available information such as commodity prices or sports results:
Data is provided by a single source of truth - eg: banks.
Transfer information is variable and financial institutions may reverse transactions.
The data is not publicly accessible or easily verifiable.
Given account information (identity information and transaction history) needs to be kept private.
Ledgers are accessed in different ways and often rely on user permission.
Two-way communication is not guaranteed to always work - there is two-way communication for smart contracts to be able to initiate transactions.
Because of privacy requirements, information obtained from banks cannot simply be propagated to the blockchain. Raw information needs to be desensitized to be suitable for blockchain transactions. Therefore, after the data needs to be processed off-chain, the answer is sent to the network in the form of a simple yes-or-no question (for example: is a certain transaction completed?).website。
Ramp is in the process of expanding its technology and getting it accepted by more regulators. Ramp is also exploring the use of secure enclaves as an additional layer of protection, allowing more authoritative third parties to act as oracles to verify banking transactions on the blockchain. These developments will remove the need for the greatest centralization: fiat gateways. If you want to know more about Ramp, please go to their
website
-EtheriscC. Decentralized Insurance
"We trust code more than profit-driven insurance companies, but without trusted oracles, nothing happens" Hugh Karp, NexusMutualEtherRiskDecentralized insurance products offered -Nexus MutualParametric insurance is a special kind of insurance with parameter triggers and payment conditions (pay $X if Y happens), such as a flight being delayed, an exchange being hacked. Obviously, these products are suitable for using smart contracts. They require oracle services to provide reliable data for these events, which can be obtained internally through economic incentives (using
defined data entry, or useFlyingcarpetvoting process), and may also be obtained through a decentralized oracle as a service (Oaas) provider.
It can be expected that we will see more and more insurance products handled by smart contracts, especially as there are already sophisticated IoT sensors, drones and satellites.
In some cases, declared results cannot be automatically detected and processed without locally coordinating input information. These cases require a local (incentivized) oracle as an independent claim agent, such as quantifying the damage of a car. Insurance companies can leverage these proxies, which can also easily add Mogg smart contracts as their potential employers. This approach is already on the roadmap of EtherRisk and Nexus Mutual, which use similar stake designs to punish evil and promote good. Fraud is usually the biggest cost in insurance, and it is impossible to expect a certain system to be absolutely free from fraud. However, if smart contracts can achieve the reliability comparable to centralized insurance, the saved success can be returned to customers.AragonorColonyEventually, these claims proxies could work in decentralized autonomous organizations, such as using
to coordinate their respective operations.
V. Conclusion
V. Conclusion
The opportunities and challenges posed by the immutable nature of decentralized networks can be summed up in one word: incentives. Through a well-designed economic incentive mechanism, the decentralized network can conduct unprecedented global collaboration and ensure that the behavior of rational participants is in line with the interests of the network. Removing reliance on trusted third parties is a challenge, as anything and anyone can theoretically go bad and the risk must be reduced to a tolerable level. It is true that the risk of short-term attacks is always present, but this should not diminish the long-term incentives for network defenders, who can reap huge profits from it.
Bitcoin miners as a whole are considered trusted "entities" hired and trusted to keep the Bitcoin ledger secure. There have been incidents where miners exploited loopholes to make short-term profits, but they are more willing to play a rational long-term game to ensure that their investment in ASIC mining machines and block rewards will not depreciate significantly. Going further, Ethereum has been criticized for being overly reliant on services like Infura or Metamask (which do become centralized through Consensys), but because these services have huge stakes in Ethereum, they are able to remain non-evil. The risk of these intermediate schemes is tolerable in the process of further decentralization.
