This time we broke out of the Ethereum DeFi dark forest and saved $9.6 million perfectly
Editor's Note: This article comes fromChain News ChainNews (ID: chainnewscom), published with permission.
Editor's Note: This article comes from
Chain News ChainNews (ID: chainnewscom)
Chain News ChainNews (ID: chainnewscom)
, published with permission.
Written by: Sam Sun, Alex Wade, Scott Bigelow, Tina Zhen, Shaoping
On September 15, 2020, an operational team worked together overnight to rescue $9.6 million from a vulnerable smart contract. Their stories are told below.
Sam Sun aka samczsun Readme
Let me summarize and review the stories that happened in some smart contracts.
Of course I didn't expect anything interesting. Over the past few weeks I’ve seen countless yield farming clones pitch the exact same pitch: Put your tokens in our hands and you could be the next crypto millionaire. Most clone projects are just forks of well-audited code with minor tweaks, sometimes with disastrous results.
But amidst all the hustle and bustle, there is some code I've never seen before. The subject of this article, the smart contract holding more than 25,000 Ether, worth over $9.6 million at the time, will be a rewarding hunt for those desperate to spot errors in logic.
I took a quick look at the code for transferring ETH out and found two matches. One of them transferred ETH to a hardcoded token address, so it can be ignored. The second is the burn function that transfers ETH to the sender. After tracking the usage of this feature, I found out that anyone can mint tokens for themselves for free, then burn them in exchange for all the ether in this smart contract without breaking a sweat. My heart beat faster. Suddenly, the problem became serious.
My smart contract is the Lien Finance protocol. Unfortunately, their team is anonymous! The only supported platform for instant messaging is Telegram, and I'm not sure if the admin of that Telegram channel is a developer or just some early supporters. I don't want to accidentally disclose this vulnerability to the wrong person.
Some more poking around on their site, I found out that they have worked with ConsenSys Diligence and CertiK for their code audits. This looks like a good avenue, and ConsenSys Diligence and CertiK definitely interacted with the developers during the audit. I quickly chatted with ConsenSys Diligence security engineer John Mardlin (aka maurelian) on Telegram.
It is too torturous to start a chat and wait for a reply. Unfortunately, the minutes passed without Maurelian responding. He seems to have fallen asleep. In desperation, I sent a message to the ETHSecurity Telegram channel, asking if there is anyone in ConsenSys who is not asleep, please contact me as soon as possible.
A few minutes later, I got a reply from someone I've worked with a few times in the past — Alex Wade, another security engineer at ConsenSys Diligence.
Alex Wade says it all
I just got my head on the pillow when I heard my roommate knock on the door. "Sam asked on the ETHSec Telegram channel who knows anyone at ConsenSys Diligence."
Sam and Alex said tonight might be a long night, and it was
As soon as I heard it was Sam, I knew it wasn't going to be a good thing. I found a channel and an email address to communicate with Lien that I had set up a few months ago. Their team is an anonymous team, and it's better to have these than nothing.
I was still half asleep. Sam doesn't want to go into the details in the chat and asks for a Zoom conversation. While regretting why I got out of bed, I assessed the seriousness of the situation:
Five minutes later, it became clear to me that I needed a coffee to pick myself up and get to work.Sam and I reviewed the code together. Sam had prepared a sample test that confirmed the problem on his machine. Our conversation looks to shift to a discussion of possible options:We do it ourselves to save the money.
Contact the Lien team, ask them to disclose their identities, and urge users to withdraw their funds.
Neither is a very satisfying choice. The first move is extremely dangerous, if you read Dan Robinson's discussion with Paradigm Research Partner Georgios Konstantopoulos on Ethereum's DeFi Dark Forest (
Lianwen Chinese version) article, the probability of our transaction being robbed is extremely high. The second option is almost as dangerous, as a public statement would draw attention to the issue, giving attackers a window of opportunity. What we need is a third option.Recalling part of the Ethereum DeFi Dark Forest article, Sam contacted Scott Bigelow, Amberdata's vice president of engineering: "If you really get into this kind of predicament, I suggest you go to Scott Bigelow, Amberdata's vice president of engineering, who has been researching this. A security researcher on a subject has a set of prototype implementation strategies that can better achieve the purpose of deception."
By Scott Bigelow
i participated
Ethereum DeFi Dark Forest
After losing to the Dark Forest Sniper in the money rescue operation in the article, I was very eager to have a rematch. I've spent some time monitoring front runners and devised a simple system that seems to be able to outwit the average frontrunner, at least the $200 I personally put into testing it. When Sam contacted me late at night and sincerely said "Do you mind taking up about an hour of your time", I was gearing up and ready to try! I've envisioned that picture: how I can make a couple of technical tweaks, take hours, and then manage to save a user thousands of dollars in assets, wear that sense of accomplishment, and get a good night's sleep.
When Sam shared this smart contract with me, those plans fell apart in an instant: about 25,000 ETH, worth $9.6 million. As much as I would love to do this rematch, my broken lines of code are not designed for $9.6 million in assets.
For the past few months, I've been trying to connect with miners for just one purpose: doing these kinds of white hat rescue deals. If ever there was a time when miners were needed to help write a transaction into their mines so that the money would not be stolen by frontrunners, it is now. Fortunately, Tina Zhen and I have joined forces over the past few months to pursue this collaborative relationship. The chances of success for this rescue operation were slim at the time, but it was worth a try: Get Tina involved in this rescue operation, working with a mining pool to mine a private transaction.
Tina Zhen reads it
I had just been evacuated from my home due to the California forest fire code-named "Bobcat". I was drinking an unknown beach drink and listening to the sound of waves sent by the dim Pacific Ocean. Sam's Telegram DM brought me back to a darker reality: "Funds are at risk and may be intercepted." I've been working with Sam and Scott on a research project on "miner-extractable value" MEV for the past few weeks, and I guessed it before they sent their request: a direct channel to secure white hat transactions on Ethereum The mempool (set of pending, unconfirmed transactions) is a "dark forest" that is protected from being plundered by "frontrunners".
Since this is a risky move that would require exposing our strategy to miners, we decided to first try to get a green light from the anonymous Lien team. When Alex tried to get in touch via the ConsenSys internal channel, we also tried to get in touch with CertiK.
I realize that Certik's auditors in the US are four hours away from waking up, but time is running out. I don't know much about CertiK, I just know that they have audited several Asian projects, so I tried to contact the CertiK China team. I left a voice message in the two WeChat groups of "DeFi the World" and "Yellow Hats". I received four messages within 30 minutes in my private chat, confirming that the ID in my WeChat friend is the real CertiK Chief Technology Officer, Zhaozhong Ni. I was pulled into a WeChat group of five CertiK team members, and at the time I was still not able to disclose related projects or vulnerabilities. To minimize exposure risk and potential liability, we can only invite one Certik member to join our white hat operation. After the final confirmation by the official email, Georgios Delkos, head of engineering at CertiK, joined our conversation.
With the help of Georgios, Alex was able to quickly contact the Lien team and verify their identity. We let them understand the current severe situation as quickly as possible, and won their consent to directly cooperate with a mining pool to save this fragile fund. After consideration, the Lien team agreed that bailing out the funds or issuing an early warning statement would be too dangerous and agreed with us to move forward with this option.
Now we need to find a mining pool with sufficient infrastructure in place and willing to cooperate with us as soon as possible. Which mining pool should we look for? Who is the liaison at this mining pool who can make a quick technical decision to help us execute this action against the clock?
We thought of SparkPool, and I know that they have been building a public infrastructure called Taichi Network, which can easily give us the support we want. I decided to chat privately with Shaoping Zhang, co-founder of Xinghuo Mining Pool, who had previously helped me investigate mempool security incidents.
Half an hour later, Shaoping replied: "You mean I open a whitelist for the transaction? Sorry, we can't." Well, lost in translation, "white hat" whitehat and "whitelist" whitelist in Chinese seem a bit picture.
"There's $10 million at stake right now. I'm on the line with Samczsun," I again tried to communicate the situation without revealing any specific details.
"You two are saving the world? Do you need help from the mining pool?" I was relieved by the accident, and Shaoping joked that he was willing to help. After being confirmed by the official email, Shaoping entered our marathon Zoom conference call, and was given technical support by a large number of spark mining pool developers.
Shaoping's self-report
After lunch, I was about to take a nap, and then I received a WeChat message from Tina: "Has Spark Mining Pool helped with white hat transactions?" I misread it at first, and it was a whitelist for transactions. No one has contacted us for white hat transactions before, and we are not familiar with the specific situation of "white hat transactions". After Tina explained in more detail, I realized that they need a private transaction service. For example, white hat hackers want to send transactions to save a DeFi smart contract, but to prevent others from robbing them, they need the mining pool to broadcast In the case of a transaction write the transaction into the block.
We have been building a private transaction function in Taichi Network, this function is still under development and has not been tested yet. I told our development team about the white hat hacker's request and emphasized the urgency: our private transaction functionality needs to be in production within a few hours. Our developers say they'll do their best and get to work quickly. We completed the development of the private transaction function in two hours, and spent another time fixing bugs.
After we finished internal testing, we sent the hitehat.taichi.network endpoint to Scott Bigelow to complete the white hat task.
By Scott Bigelow
Spark Mining Pool worked overtime to launch a new white hat API, and Sam and I also completed the script programming to generate four consecutive signed transactions. Processing these transactions in turn will not take out 25,000 ETH per se, but will transfer the ("error" generated) 30,000 SBT+LBT tokens to the Lien team, who can submit them to the final transaction to convert these tokens back into ETH.
By transferring infinitely mintable SBT+LBT tokens to the Lien team instead of ETH, we use more transactions as a cover-up to confuse generalized coercion attacks (if encountering a reorganization), and can keep $9.6 million in revenue from ever Into my pocket, not even for a moment.
After we generated the four signed transactions, Sam and I spent a lot of time verifying their chain of actions using various multi-party transaction simulation tools. These four transaction programs, with a total data volume of less than 1.5 KB, are the killers to save 9.6 million US dollars of assets, ensuring that no one will be aware of these transactions before Spark Pool captures them.
I tested Sparkpool's whitehat endpoint with a nonsensical transaction, and the execution was unbiased: the transaction would not be seen in the mempool, and then suddenly appeared in one of Sparkpool's blocks! It's like watching water vapor turn directly into ice, and the annoying liquefaction process in the middle is completely invisible!
We tweaked the transaction generation scripts to submit transactions directly to Spark Pool's new endpoint, and now it's time to act. I hesitated for a moment, but it was definitely the best effort we could make. We might lose the $9.6 million, but no regrets: I hit "Run" in IntelliJ. I'm not sure why, but I was expecting the whole process to take a while, like the nodes would understand the seriousness of the situation and then spend some time in it. But actually it doesn't; transactions are sent in milliseconds.
Everyone on the Zoom call started frantically refreshing Etherscan, I doubt the Etherscan team saw this 3 minute traffic spike. Since only Spark Pool has transactions, and only a portion of Spark Pool's hashrate is dedicated to this purpose, all we can do is wait in a cold sweat. Every block mined by every other miner scares us. In the Zoom call, someone will read the name of the miner who dug up the block, accompanied by nervous laughter. About 15 blocks before our transaction was written into the block, we felt like years, like a few hours, but in the end, we completed a flawless transaction: sequential mining, no rollback.
We watched with relief as more and more blocks were superimposed on top of ours, and concerns about block reorganization quickly disappeared. The Lien team now has enough SBT + LBT tokens to liquidate funds for the entire system, and Sam is in charge of coordinating the final phase of the rescue.
Now that we have successfully transferred tokens to the Lien team and found no signs of coercion, attempts or otherwise, we quickly shared the good news with them via private chat. They confirm receipt of the tokens and immediately send a transaction to take out most of the ETH locked in the smart contract. Seconds later, this pending transaction appeared on Etherscan.
https://etherscan.io/tx/0xe99ccb0b21854b65a2fa283638ab9ef01962b61c3310b596b4597bf22b911a43
As I watched the progress bar circle, I took the opportunity to reflect on the action. What started as a look at smart contracts eventually turned into a "Saving Private Ryan" operation that attracted experts from all over the world to collaborate. Without Alex and Georgios we would not be able to get in touch with the Lien developers. Without Scott, our rescue operation might have been in trouble long ago. Without Tina, we would not be able to get in touch with CertiK or Spark Pool. Without Spark Pool, we are doomed to repeat the tragedy that Dan wrote about in his article a few weeks ago.
And yet late Tuesday night, our seemingly impossible team worked together for the same purpose, worked tirelessly to secure the return of $9.6 million to its rightful owner. Our efforts over the past seven hours have culminated in a pending transaction and this circular progress bar.
image descriptionClick hereWe finally managed to break out of the dark forest.This article documents the success of the hard work of many people. Special thanks to Alex Wade, Scott Bigelow, Tina Zhen, Georgios Delkos, and SparkPool for saving the day, and Alex Obadia and Dan Robinson for reviewing this article and providing feedback.If you are interested in the technical details behind this operation, please
