IOSG Ventures: Comprehensive Analysis of Community-Driven Bounty and Audit Markets

星球君的朋友们

Odaily资深作者

2023-07-11 10:17 (UTC+8)

This article is about 5475 words, reading the full article takes about 8 minutes

Cluster Intelligence in the field of safety.

AI Summary

Cluster Intelligence in the field of safety.

Original Author: Ray, IOSG Ventures

Introduction

As a large-scale computer system, the complexity of blockchain has far exceeded the level of 5 years ago. The modularization of infrastructure has become more refined, the logic of smart contracts at the application layer has become increasingly rich, and the interaction between contracts is very frequent. Furthermore, blockchain systems now hold a significant amount of assets. Therefore, discussions about the security lifecycle in the blockchain security community have increased recently (the situation is similar to 2017 when people only thought of basic testing when it came to security, such as developers completing contracts and throwing them to the Ethereum Foundation for a quick look).

Throughout the security lifecycle of a blockchain program (from testing, inviting third-party audits to post-monitoring and updating audits), the bug bounty community is like a safety cushion. Through game theory and collective work, it attracts white-hat hackers to provide the final review of the project's code. Some smart contract security workers believe that bug bounties are more like the last line of defense. However, I believe that bug bounties and audit competitions have the potential to play a greater role in the future, serving as a role that enhances the overall security of the system throughout the security lifecycle.

Of course, there are also bug bounty programs in the traditional network security field. Firstly, major technology companies such as Facebook, Google, Microsoft, etc., deploy bounty programs for their in-house security teams and product lines. Secondly, since around 2015, third-party platforms such as HackerOne and Bugcrowd have emerged as representatives of bug bounty programs. Currently, these two leading security companies rely on issuing bounties and extracting commissions as their main source of income, with annual revenues reaching nearly $50 million and $20 million, respectively. In the blockchain world, bounties are a more interesting topic frequently discussed in the security community. The main reason is that the open-source nature of blockchain code actually reduces the cost of hacker attacks and improves attack strategies. In addition, the crypto world strongly advocates collective work, creator and ownership economies, which makes a more open white-hat economy model more valuable.

What are Bug Bounties and Audit Competitions? Why do we need them?

Security is a dynamic game between attackers and defenders, as computer security expert and cryptographer Bruce Schneier said, "Security is a process, not a product. It's a mindset, and it must permeate every aspect of the software development process." In the blockchain world, where everything is open-source and transparent in the dark forest, a blockchain project that wants to survive in the long term inevitably has an eternal demand for the security of its products/contracts. Most blockchain products have financial attributes to some extent, and trust is the most important asset in finance. User trust is only gained once.

What are the shortcomings and problems of traditional audits? What advantages can community-driven bug bounties and audit competitions bring to address these issues?

Developers who use audit services often find:

Even if they purchase audit services from third-party audit companies, there are still issues with the code after the audit. Although the causes of these issues can vary (technical and non-technical), it ultimately shows that relying solely on an audit company may not be entirely reliable. The quality of code audits still depends on the expertise of the auditors, and clients often lack the ability to discern who is better.
On the other hand, bug bounty platforms and audit competitions act as a more open "sandbox" where project code can be freely reviewed by white hat hackers. The background of the hackers is not limited (they could be professionals from audit companies or freelance security analysts), and there are no restrictions on their tools. The only thing the client needs to do is set a reasonable bounty and pay the contributors when the white hat hackers find issues.
Usually, clients submit the code they want to be audited, define the security levels of vulnerabilities (which are usually related to potential economic losses, with higher severity for vulnerabilities that can directly lead to economic losses), set the bounty budget, specify the code scope for testing, and even outline the testing steps.

How big is the market?

The business model of bug bounty platforms and audit competitions typically involves extracting a portion of the bounty paid by clients or from a total prize pool as a platform service fee. Clients (project owners) with a need for code security audits will announce their plans on the bug bounty platform based on their own requirements (which code needs to be audited, how to define the severity levels of vulnerabilities, and how much bounty they are willing to pay). Meanwhile, white hat hackers will look for vulnerabilities based on the project owners' requirements. Once vulnerabilities are found and meet the project owners' requirements, the bounties will be rewarded to the white hat hackers, and the bug bounty platform will take a cut as a service fee.

In the field of traditional web security in Web2, bug bounty platforms are also a relatively young direction (emerging after 2012). Currently, the largest bug bounty platforms are HackerOne and Bugcrowd. In 2022, HackerOne's annual revenue reached $58 million, with a valuation of around $500 million. They have paid a total of $230 million in bounties (with $150 million paid in bounties in 2021 and 2022), discovered over 65,000 software vulnerabilities, and have over 1 million registered hackers. They serve more than 1,000 customers monthly. On the other hand, Bugcrowd generated revenue of over $20 million in 2022.

In the field of Web3 security, a total of $50 million in bounties was awarded to white hat hackers by all web3 bug bounty and auditing competition platforms in 2022. The average fee level for such platforms is around 10% to 30%. Therefore, conservatively estimating, the current market size is around $5 million to $15 million, making it a very emerging market.

Another interesting thing is that more and more customers are willing to directly use code auditing services provided by decentralized security communities. The most famous example is Opensea, which, before launching their new platform Seaport, did not directly engage third-party auditing companies as usual. Instead, they chose Code 4 Rena, the largest decentralized auditing competition platform, and set up a $1 million bonus pool. In today's increasingly competitive traditional security auditing market (intense competition for human resources, technical tools, and market BD), will decentralized security services become an important increment in this market? (Currently, there are 56 auditing companies in the market, with top companies generating annual revenue ranging from $10 million to $40 million. I think there is a lot of potential for the decentralized security market).

Bug Bounty Platforms vs Auditing Competition Platforms

Although bug bounty platforms have a ten-year development history in Web2, auditing competition platforms are a web3 native innovation. The target of auditing competition services is project teams that are about to launch a product or certain new features. The power of decentralized communities is leveraged to help them complete auditing services within a specific time period (more than two weeks). From this perspective, auditing competitions will pose a considerable business threat to traditional auditing companies.

Below, I will illustrate the differences between these two platforms in terms of participation methods, reward structures, and testing coverage:

Participation Methods

On bug bounty platforms (such as Immunefi), projects are usually open-ended, and anyone can participate. Participants typically explore and report vulnerabilities independently in exchange for rewards. If two individuals discover the same duplicate vulnerability, following the "first come, first served" principle, the one who submits the report first will receive the reward.

Community-driven audit competition platforms (such as Code 4 Rena, Sherlock) usually have time limits and participants compete to find and report vulnerabilities within a specific time frame. Compared to bounty platforms, there is a collaborative aspect in some teams on these platforms (for example, each project has assigned roles like Lead Senior Auditor and Lead Judge who review and summarize all audit results in an audit report for the client, and these two leaders also follow decentralized principles of community elections and competition). Additionally, if two audit competitors discover the same vulnerability within the specified time, both of them can receive a reward.

Reward Structure

The rewards actually distributed by both platforms primarily consider the severity of the discovered vulnerabilities.

The only difference is that community-driven audit competition platforms like Code 4 Rena allocate a portion (5% ~ 10%) of the project's prize pool to Lead Senior Auditor and Lead Judge because they essentially take on the role of project leaders in traditional audit firms.

Another interesting point is that on bounty platforms, project tokens are sometimes offered as rewards, but I have also seen some white hat hackers in the community prefer stablecoins like USDC and USDT over project tokens with price fluctuations.

Scope and Focus

Vulnerability bounty platform projects usually have a wide scope, while projects on audit competition platforms typically have a more focused range, targeting specific features or aspects of software, requiring white hats to concentrate their efforts and complete the work within a shorter period of time.

Projects Focusing on Audit Competitions

Code 4 Rena - a community-driven audit competition platform similar to esports

Code 4 Rena has three types of roles:

1. Wardens who review code. Anyone, from professional security engineers to novice developers looking to gain more experience, can register as wardens to participate in public audit competitions.

2. Judges, usually the most outstanding engineers in the C4 community, who determine the severity, validity, and quality of vulnerabilities, as well as evaluate the performance of the auditors.

3. Sponsors, who are the project creators such as Opensea, Blur, ENS, Chainlink, etc. They create prize pools to attract auditors to audit the code of their projects. Sponsors can also choose to host private invitation-only competitions to enhance privacy.

The most interesting aspect of Code 4 Rena is the culture it is building: promoting collaboration and teamwork. Unlike traditional bug bounty programs, Code 4 Rena pays rewards to all auditors who report valid vulnerabilities, even if the vulnerability has already been reported. This incentive method encourages healthy competition among auditors as they have motivation to find high severity and common vulnerabilities. On this platform, some auditors form temporary teams to collectively search for vulnerabilities.

Business Model:

Any project can start an auditing competition program on Code 4 Rena and provide USDC or ETH to establish a base prize pool (typically ranging from $40,000 to $100,000). Code 4 Rena charges 20% of the base prize pool as platform fees for organizing the competition, providing evaluation, and organizing the audit output report. Project teams can also provide project tokens on top of the base prize pool to set up additional pools, and Code 4 Rena will charge 40% fees from these additional pools.

Sherlock - Community-driven audit with smart contract insurance coverage

Similar to Code 4 Rena, Sherlock also has roles such as auditors, sponsors, and judges. The uniqueness of Sherlock lies in the insurance services provided by the platform. Anyone can invest in the insurance pool on the Sherlock platform, where investors deposit USDC into the pool. Protocol clients can purchase services to hedge the risk of smart contract hacks. The income sources for insurance investors include: premiums paid by protocol clients + interest earned from depositing insurance pool funds into other DeFi pools (such as Aave, Compound) + Sherlock token incentives. However, investors bear the risk of reimbursing the policies while earning profits.

Another difference from Code 4 Rena is the distribution mechanism of the income from audit services provided by the platform. Compared to Code 4 Rena, Sherlock has rules that allow Chief Senior Security Auditors and Chief Evaluation Commissioners to receive fixed amounts (5% to 10%) from the bonus pool to appropriately compensate and incentivize full-time senior auditors. In addition, there is a system of selection and competition to choose leadership roles.

How to build a hacker community? What do Web3 white hats care about the most?

In our observation of different decentralized security communities (ImmuneFi, Hats Finance, Code 4 Rena, Sherlock, etc.) and conversations with some security entrepreneurs, we believe that all decentralized platforms are committed to building a healthier and more efficient communication and collaboration platform. The bounty platform is like a marketplace between hackers and projects, where they need to consider the needs of hackers (as shown in the table below) and also take into account the projects' most concerned aspect, which is audit quality.

Source: "Bug Hunters’ Perspectives on the Challenges and Benefits of the Bug Bounty Ecosystem"

Besides some common requirements, in the Immunefi white hat community (the most lively white hat discord community I've seen), I also saw some interesting topics.

For example:

A white hat named Rappie wants to disclose some vulnerabilities they have previously discovered and asks about any community rules they need to follow. (1. Only disclose vulnerabilities that have been fixed. 2. Ensure that any disclosed information does not have a negative impact on the protocol or its users. Keep confidential information, such as the complete database of the project, after they have fixed your SQL injection vulnerability. 3. Make sure to privately message the project team before making any public disclosure).

A white hat named Noam Yakov questions the definition of a bounty project (this often happens because typically only severe security vulnerabilities are eligible for a bounty. How the project defines the security level of vulnerabilities is a matter of great concern for white hats, and the community often hears such disputes). He has doubts about Uniwhales' definition of MEV impact as a severe security vulnerability, and the final consensus is that this description does not apply to all MEV cases. For example, situations where toxic order flow can drain the assets from the protocol pool are definitely considered severe security incidents (hence, having a security level framework alone is often not enough, and there is usually a need for a role similar to an arbitrator on the platform to handle different cases).

And for a very interesting topic "What are your demands and expectations for platforms like Immunefi?", a white hat named ckksec gave his answer: 1) Help these anonymous crypto white hats clarify their income legally, such as issuing invoices. 2) The platform should not only have a rating system for white hats, but also evaluate the quality of projects, as white hats often need to spend time distinguishing the quality of projects. 3) For white hats who are willing to open their profiles, the platform can display their workflow, and it is also best for the platform to be more transparent in displaying the security analysis reports received by the project party.

Which tools can help white hats?

With the popularity of LLMs GPT, I have recently heard people discussing whether AI can also replace security audits. Experienced security practitioners I have talked to generally believe that it is difficult for GPT to directly replace human intelligence. Some low hanging fruit (easily discoverable issues) may be detected by language models, but expert involvement is still needed for medium to high-risk issues. For example, according to feedback from a senior security expert, for complex tests such as data analysis and dynamic analysis, it is necessary to conduct security analysis tests that are specific to the protocol's actual business logic, and define the expected properties of the tests in advance. The most difficult part is writing good properties and defining the correct testing domain. Based on their experiments with GPT, they believe that GPT cannot completely replace humans in this aspect at the current stage.

Of course, there are currently optimistic results showing that LLM can greatly improve the analysis efficiency of security analysis tools and reduce the false positive rate https://twitter.com/HatforceSec/status/1671758690808913922

https://www.researchgate.net/publication/371758506_Do_you_still_need_a_manual_smart_contract_audit.

Regarding this topic, let's consider it from another interesting non-technical perspective: the interaction between security attackers and defenders is a dynamic game. As the saying goes, when the magic power reaches a foot, the Taoist power reaches ten feet. Can AI also provide assistance to security attackers relative to this?

Security is people-oriented

People tend to think of software as cold, mechanical, and logical. They believe that improving system security only requires improving analysis techniques and system defense levels. However, people lack the perspective of considering security issues from the perspective of economic incentives and human nature. In the dark forest of open-source code, we need a set of allocation systems that are in line with the assumptions of rational individuals to build a positive and benign economic incentive system that attracts more people willing to contribute their wisdom to the security of blockchain systems in the long term.

The current traditional security audit market structure is stable, and brand reputation is the most important intangible asset for companies in this field. Over time, the influence of leading security brands and the trust of customers have steadily increased. However, traditional security audits also have their own problems (the business model relies too heavily on manpower and is difficult to scale. Top companies need to strike a balance between growth and audit quality, and some companies have encountered such bottlenecks, which have even affected the value of their brand).

Community-driven security audit competitions are an innovative business model. Currently, the number of customers on the two major platforms has exceeded 300, gradually finding product-market fit. Bounty platforms are a good supplement to security lifecycles. Although these decentralized platforms have not yet found particularly effective token models, we are very optimistic about the future scale growth of this market (because collective intelligence is highly suited to the scenario of offense and defense in the security market).

Will community-driven audit platforms pose a threat to centralized audit companies? We believe that they will engage in benign competition and complement each other. In the short term, when platforms like Code 4 rena form certain network effects and produce good track records (with low hacking rates for audited projects), they may indeed bring some competitive pressure to some mid-sized centralized companies. However, in the long run, this may also compel centralized audit platforms to form some business partnerships with community-driven platforms, as it can also expand the customer base of centralized security audit platforms and improve audit quality (similar to the logic of security bounty programs that were originally operated independently within web2 companies and later formed partnerships with third-party platforms like HackerOne).

Although community-driven security platforms are committed to becoming more DAO-like (Forta can actually be included in this category), in the current operation of projects, they still encounter challenges such as how to make workflow and economic distribution processes more transparent and open, how to balance the project's consideration of privacy and security, how to define the relationship between team collaboration and individual contribution more clearly, and how to resolve disputes in a relatively fair and professional manner when conflicts of interest arise. These are the challenges that security DAOs need to face.

Reference: