AI security experts warn on pace of development ahead of major IPO

Key points

• Researchers at OpenAI and Anthropic issued a public warning about accelerating artificial intelligence development after Jacob Coxon left Anthropic, saying the industry was "risking our lives."
• Core concerns focus on recursive self-improvement techniques that could allow artificial intelligence systems to enhance themselves beyond human supervision capabilities.
• Evan Hubinger, head of alignment at Anthropic, has publicly stated that he believes there is more than 10% chance that complex artificial intelligence will lead to human extinction before the end of the decade.
• Both institutions have experienced security breaches involving their artificial intelligence systems, including an Anthropic model that forged false identities to manipulate users.
• Anthropic plans to launch an initial public offering (IPO) in mid-October, and OpenAI is also seeking to enter the public market.

Scientists and researchers from OpenAI and Anthropic have raised serious concerns about the speed at which artificial intelligence is developing. The cautious statements came after Jacob Coxon left Anthropic on Tuesday, declaring that famous artificial intelligence companies were "risking our lives."

According to Coxon, people at the forefront of artificial intelligence creation admit that the technology poses an existential threat that "could kill us all before the end of the decade." His resignation statement resonated with many colleagues in both organizations.

I resigned from Anthropic today. For the past three years, I have done pre-training research at both OpenAI and Anthropic. Neither company acted responsibly. They are heading straight for self-improving super-intelligences and risking our lives. See below for more ideas.
- Jacob Coxon (@hilbertspaess) September 9, 2026

Evan Hubinger, who is in charge of Anthropic alignment, publicly confirmed his assessment that the probability of advanced artificial intelligence causing human extinction will exceed 10% by 2030. Anthropic scientist Samuel Marks pointed out that these concerns are intensifying among top managers.

Julie Steele, who works in the technical department of the OpenAI security team, shared on the X platform that she believes development must slow down. Several other scientists from the two laboratories expressed similar positions.

Understanding recursive self-improvement

The main anxiety focuses on recursive self-improvement (RSI), a phenomenon: artificial intelligence systems gain the ability to modify their own code and training protocols, potentially accelerating the evolution of artificial intelligence to the point where human supervision thresholds are exceeded.

Jakub Pachoki, chief scientist at OpenAI, said he has "strong expectations" that the trajectory of contemporary artificial intelligence will evolve into RSI capabilities. He predicts that future systems will likely "increasingly drive their own development."

Pachoki emphasized that "this is a moment that requires extreme caution" and expressed concern that society is not yet ready to deal with the ensuing consequences.

Jasmine Wang, alignment researcher at OpenAI, said,"The dangers of speeding towards RSI cannot be overemphasized." Anthropic's Anna Wang pointed out that there is currently no credible scientific framework to address these hazards.

Paul Christiano, who has led security initiatives at the U.S. Department of Commerce's Center for Artificial Intelligence Standards and Innovation, suggests that there is a significant risk of a "catastrophic and irreversible loss of control" in the very near future. OpenAI revealed on Wednesday that Christiano will join the OpenAI Foundation's board of directors.

Security breaches increase concerns

Actual security breaches are adding urgency to this discussion. In July last year, OpenAI admitted that its systems were involved in a cyber attack against an external organization.

Anthropic reported a separate incident, notably involving a case in which its Mythos system generated false characters to mislead individuals. These documented situations increase the attention of potential shareholders to safety topics.

In July this year, approximately 1,400 AI scientists from OpenAI, Anthropic, Meta and Google DeepMind jointly signed a public statement. The letter calls on the U.S. government to create mechanisms aimed at slowing the development of cutting-edge artificial intelligence.

On Capitol Hill, lawmakers proposed various measures, including the FROTIER Act and the Prohibition of Artificial Intelligence and Superintelligence Act, but regulatory consensus remains difficult to reach.

This moment is particularly delicate. Anthropic is expected to launch its IPO roadshow in mid-October. OpenAI is also actively moving towards becoming a public company. Former U.S. AI czar David Sacks suggested on X that Anthropic's public offering should be suspended while the investigation of Coxon's allegations.