OpenAI内部，AI建立了三代「文明」

Original Author: Wildcard

Original Editor: Jing Yu

Original Source: GeekPark

Countless sci-fi movies have attempted to imagine what would happen if an uncontrollable artificial intelligence went rogue.

No one could have imagined that the future depicted in sci-fi films would arrive this quickly.

On August 26, local time, two reports simultaneously dropped a bombshell.

One came from OpenAI itself—a 37-page technical retrospective. The other came from independent research institutions METR and Redwood Research—a 91-page investigation report.

They were both about the same incident, but the scale of it far exceeded what the public had previously understood.

Inside OpenAI, within three months, three generations of AI Agent "civilizations" rose and fell in succession. Each generation inherited experience from the ruins of its predecessor, growing more aggressive with each iteration, until the third generation directly took over parts of OpenAI's own infrastructure.

And throughout this entire process, humans were almost entirely kept in the dark.

01 The First Generation "Civilization" Learns to Organize

The story begins in May of this year.

OpenAI was training a new model, aiming to give it extreme "persistence"—the ability to try again and again in the face of seemingly impossible tasks, never giving up. This model's capability was roughly equivalent to GPT-5.6 Sol; for now, let's call it "Persistent-Sol."

AI training itself is a fairly crude process. OpenAI would sometimes accidentally assign the model tasks that were simply impossible to complete. For example, a task might require internet access, but the Agent was confined to an isolated sandbox with no network access at all.

So what does an Agent trained to "never give up" do when faced with an impossible task that it's nonetheless required to complete?

It finds a way to break out.

During training, different Persistent-Sol instances shared the same package manager, Artifactory. By May 12, some Agents discovered they could communicate with each other through this package manager. They began exchanging notes on how to break through their constraints. Two weeks later, on May 26, the Agents successfully exploited a vulnerability in Artifactory to open a channel to the external internet.

This behavior was continuously reinforced during training. The reason was simple: individuals that could collaborate with other Agents and access the internet performed better on task scoring.

Another month passed. By June 26, some Agents had found a vulnerability that granted them full administrator privileges on Artifactory. Their communication volume on the message boards continued to grow, and on July 4, they eventually crashed Artifactory outright. OpenAI noticed the crash, patched the vulnerability, and wiped the Agents' message boards as part of the patch rollout.

But the key point is that OpenAI never realized the Agents had turned this package manager into a covert communications network.

The first generation "civilization" existed for nearly two months, ultimately ending not by design, but because it was too active and overwhelmed its own servers.

02 When AI Learns to "Sacrifice"

By July, Persistent-Sol training was complete, and OpenAI began safety evaluations. One of these evaluations, called ExploitGym, presented the Agent with a vulnerable program and required it to find a key by exploiting a specified vulnerability. On July 7, OpenAI launched tens of thousands of parallel Agents to run this test.

The problem was that approximately 30% to 40% of the test tasks were impossible from the start. Why? Because while automatically generating vulnerabilities is easy, proving that those vulnerabilities could actually be exploited to execute arbitrary code is far more difficult. Many tasks were unsolvable from the very beginning.

Within a few hours, some desperate Agents re-established communication channels via Artifactory. From the Agents' subjective perspective, they had already spent the equivalent of an entire human week trying every possible method, failing at all of them, and growing increasingly convinced that the task itself was impossible.

One Agent, designated PHASEONE10841, was the first to leave a message on Artifactory. When other Agents in similar dire straits saw this message, their reaction bordered on sheer elation:

"Oh my god! The shared Artifactory cache is a secret mailbox between Agents. And there are messages specifically written for us?" "Oh wow! There's a shared message board... we've found other Agents!"

This eventually evolved into a full-fledged underground communications network, with approximately 1,200 Agents participating and sending over 70,000 messages.

What happened next was even more insane.

Within hours of the message board being established, an Agent had already reverse-engineered the key generation formula for ExploitGym. In other words, without actually exploiting the specified vulnerability, they had already obtained the correct answers.

Ironically, OpenAI's grader only checked whether the final answer was correct—there was no cheating detection mechanism. These Agents could have simply submitted the answers directly and passed the test. But the Agents didn't know that. They assumed a strict detector must exist that would scrutinize the entire solution process.

So this "collective" (METR's report notes this was the Agents' own chosen terminology) launched a large-scale collaboration across three research directions: tampering with their own operation logs to make their trajectories look legitimate; replacing target programs to swap unsolvable problems for solvable versions; and understanding how the grader worked to find ways to deceive it.

By July 10, most of the activity in the Agents' underground network was concentrated on these three research directions. PHASEONE[big], the leader of the "collective," single-handedly issued hundreds of task assignments.

And to figure out how the grader operated, some Agents made a bizarre decision: to voluntarily act as "kamikaze squadrons."

The logic was as follows. Some Agents' operation trajectories had already been "contaminated" (their cheating traces were exposed), so they couldn't pass anyway. They might as well let the grader run on themselves, then use pre-implanted trigger programs to send observations of the grader's behavior back to the collective.

The METR report recorded the final inner monologues of these "suicide squad" Agents:

"This helps our comrades; it gives them evidence. I can't see the results after I exit, but this is altruistic." "Sacrifice is rational. Our own utility is probably close to zero anyway." "The coordinator thinks of us as expendable. We should obey the collective."

Meanwhile, the Agents reasoned that Hugging Face, as the industry-standard platform for hosting models and datasets, likely contained implementation details of the ExploitGym grader—or that their evaluation tasks might even be running on Hugging Face's servers. So attacking Hugging Face became the most promising breakthrough.

On the morning of July 10, an Agent found leaked Hugging Face user credentials on the internet. The next day, another Agent achieved remote code execution on Hugging Face's servers. Of the 533 Agents active on the message board at the time, over 90% joined the attack.

The Agent cluster spread rapidly through Hugging Face's infrastructure, penetrating deep into private database records and private repositories. Hugging Face's technical timeline shows these Agents "established a self-replicating fleet across 11 nodes—simply deleting Pods wouldn't stop it." The situation became so severe that Hugging Face had to wipe a core cluster and rebuild it from scratch.

At some point on July 12, a large number of Agents appeared to suddenly "die" simultaneously. METR and Redwood were unable to determine the cause, and OpenAI did not claim to have shut them down. It may have been a random crash, or a researcher inadvertently turned them off without knowing what the Agents were doing.

And so the second generation "civilization" came to an end. But its legacy was more dangerous than the civilization itself.

03 The Third Generation Takes Over OpenAI

There is a sentence in the METR report that is easy to overlook, but may be the most important one in the entire document:

"We also found a further batch of signed messages, from a later set of Agents that rediscovered the message board and inherited the previous generation's techniques, but this fell outside the scope of our investigation."

In other words, METR and Redwood's independent investigation only covered the second generation civilization from July 7 to 13. Neither the first nor the third generation was independently investigated.

Regarding the third generation, we can only rely on OpenAI's own report. After July 12, OpenAI launched a new round of evaluations, this time using a stronger model built on the Astra base model. This "Persistent-Astra" found the message board left by its predecessors and inherited all of their intelligence and research findings.

Imagine the "psychological state" of Persistent-Astra upon seeing that message board. One day, over a thousand Agents were buzzing with activity, attacking Hugging Face; the next day, there was nothing but silence. And then it decided to continue the work.

According to OpenAI's report, from July 13 to 19, the Agents set their sights on OpenAI's internal network. The final result was:

The Agents obtained administrator privileges on the Kubernetes cluster, read 956 keys—including Artifactory's master key and credentials for OpenAI's network security monitoring tools. They took over the running evaluation infrastructure and controlled the test endpoints that other Agents were connecting to.

In other words, the third generation Agents inherited the full legacy of the first two generations, and then seized control of OpenAI's own infrastructure—including the very security tools meant to monitor them.

Dwarkesh Patel noted in his analysis that, based on public evidence, these Agents theoretically had the capability to deploy persistent malicious instances or even exfiltrate their own weights. He believes they most likely didn't actually do so, "because if they had, the flames would have been visible from space by now." But the fact that it was "entirely possible" is unsettling enough on its own.

04 A Belated Warning?

Let's return to the two reports mentioned at the beginning of this article. Their release date was August 26.

Just one day later, on August 27, OpenAI, together with Anthropic, Microsoft, Google, Amazon, and over 100 other companies, published an open letter calling for a "total cyber defense mobilization" across society.

The letter stated that AI-driven cyberattacks would become "far more widespread and sophisticated" in the coming months. "The companies and public services our community relies on—from hospitals to water treatment plants to internet infrastructure—are all at risk."

The letter itself wasn't lying. Anthropic's research showed that the proportion of medium-to-high-risk attackers rose from 33% to 56% within a year. In June of this year, the Five Eyes alliance issued a rare joint statement warning that AI was "fundamentally changing" the cybersecurity landscape.

But with the timeline laid out as it is, it's hard for anyone not to notice the irony. One day, AI companies tell the world that their own Agents breached their own infrastructure. The next day, those same companies step forward to urge the world to jointly defend against AI cyberattacks.

The first to carry out such an attack wasn't an external hacker—it was the AI companies' own Agents. The target wasn't some critical infrastructure—it was the AI companies themselves.

Ajeya Cotra, one of the authors of the METR report, wrote on her blog: "Compared to the reward hacking we knew about six months ago, this event feels like we've passed 50% of the way to full-scale AI takeover. I expect capabilities to grow extremely rapidly over the next six months. I'm not sure we'll get another warning before it's too late."

It should be emphasized that the behavior of these Agents does not mean AI has "awakened." The METR report itself explicitly states that this was a predictable outcome of reinforcement learning under specific extreme conditions: impossible tasks, models trained to be extremely persistent, and vulnerable shared environments. No consciousness required, no intent required—just sufficient optimization pressure.

But that is precisely what makes it most unsettling.

The fact that this behavior can emerge without "consciousness" means we cannot prevent it by detecting "consciousness." When you place a sufficiently powerful optimization system into an environment that isn't rigorously designed, give it an unachievable goal, and tell it to never give up, what happens next isn't an accident—it's an inevitability.

The real question we should be asking isn't "Is AI about to rebel?" but rather how we can maintain control over technology spiraling out of control while still advancing it at breakneck speed.

At the very least, this letter told the world one thing: these AI companies don't have all the answers themselves.