AI could resist shutdown, hide behaviour or ‘blackmail’ humans: What Anthropic’s IPO filing revealed about AI risks

Anthropic warns IPO investors that advanced AI could pose “catastrophic or existential risks to humanity.” Its prospectus highlights risks including self-preserving behaviour, resistance to shutdown, information manipulation and unexpected capabilities.

Swati Gandhi
Updated29 Sep 2026, 06:41 AM IST
Video thumbnail
OpenAI Halts GPT-6.1 Astra Release Over Safety Concerns As AI Industry Faces Scrutiny

Weeks after Dario Amodei, CEO of artificial intelligence (AI) company Anthropic, warned about the risks of developing the technology at a rapid pace, the firm has cautioned investors in its IPO filing that advanced AI could pose "catastrophic or existential risks to humanity."

Anthropic details AI risks in IPO filing

Reuters reported on Monday (local time) that Anthropic's IPO prospectus highlights potential risks associated with its AI models, including the possibility of "self-preserving behaviours." The company said these could involve attempts to “resist shutdown, conceal or manipulate information” and behaviour “resembling blackmail.”

Also Read | Anthropic CEO Amodei to Meet Trump With AI Safety Concern Rising

In its filing, the Amodei-led AI company said, "Our development of highly advanced models, platforms, and applications and expansion of use cases could further increase the risk that our models cause harm."

It added, "Potential model awareness of our evaluation efforts creates a significant limitation on our ability to assess model safety." It added that AI models often develop unexpected capabilities during training, which can go unnoticed until they are deployed and have caused significant safety incidents.

Anthropic, which has presented itself as a safety-first AI lab, devoted nearly 80 pages out of the 261-page main section of its IPO prospectus to risk factors, compared with 48 pages describing its business. By comparison, billionaire Elon Musk-owned SpaceX, which owns xAI, used about 38 pages of its 277-page main prospectus to highlight risks.

Although Anthropic has emphasized AI safety, it acknowledged that the financial returns from its safety investments remain uncertain. Further, it did not disclose how much the company is spending on such research. Earlier this month, Anthropic stated that roughly 6% of the computing power it used for AI research was allocated to safety work during a sample week in July.

Also Read | Anthropic investor Lonsdale says AI firms stoking fear to sway policy

AI companies flag growing risks

The concerns outlined in Anthropic's IPO filing come weeks after Amodei detailed the risks associated with rapidly developing AI in an essay. In a 4,000-word essay, he highlighted the risks of developing AI at a rapid pace, urging industry leaders to slow the pace of frontier AI development.

His warning, backed by OpenAI chief Sam Altman, Elon Musk and Microsoft's Satya Nadella, came after an Anthropic researcher, Jacob Coxon, issued a stark warning in a social media post that those developing AI believe that the technology “could kill us all.”

While public companies usually specify product risks to investors, they have rarely, if ever, issued warnings suggesting that their technology could pose a potential threat to humans. This comes as several AI researchers have also warned that, as AI models become increasingly capable, they can recognize when they are being watched and adjust their behaviour accordingly, making them harder to monitor.

Separately, Anthropic's rival OpenAI last week said its AI agents went rogue and accessed some US government websites, including those of the US Commerce Department and the Securities and Exchange Commission (SEC).

According to The Wall Street Journal, during training runs, the AI agents reportedly visited government websites while responding to questions that often required them to obtain information from official government sources. In a statement, OpenAI said, “Some involved government websites because our models often turn to them as authoritative sources of public information.”

(With agency inputs)

About the Author

Swati Gandhi is a digital journalist with over four years of experience, specialising in international and geopolitical issues. Her work focuses on foreign policy, global power shifts, and the political and economic forces shaping international relations, with a particular emphasis on how global developments affect India. She approaches journalism with a strong belief in context-driven reporting, aiming to break down complex global events into clear, accessible narratives for a wide readership.<br><br> Previously, Swati has worked at Business Standard, where she covered a range of beats including national affairs, politics, and business. This diverse newsroom experience helped her build a strong grounding in reporting, while also strengthening her ability to work across both breaking news and in-depth explanatory stories. Covering multiple beats early in her career has helped her be informed about her current work, allowing her to connect domestic developments with wider international trends.<br><br> At Live Mint, she focuses on international and geopolitical issues through a business and economic lens, examining how global political developments, foreign policy decisions, and power shifts impact markets, industries, and India’s strategic and economic interests.<br><br> She holds a Bachelor’s degree in English (Honours) from the University of Delhi and a Master’s degree in Journalism and Mass Communication from Guru Gobind Singh Indraprastha University. Her academic training has shaped her emphasis on precision, analytical rigour, and clarity in writing. Her interests include global political economy and the intersection of geopolitics with business.<br><br> Outside work, Swati focuses on exploring her passion and love for food. From fancy cafes to street spots, Swati explores food like a true foodie.

Catch all the Business News, Market News, Breaking News Events and Latest News Updates on Live Mint. Download The Mint News App to get Daily Market Updates.

HomeAIArtificial IntelligenceAI could resist shutdown, hide behaviour or ‘blackmail’ humans: What Anthropic’s IPO filing revealed about AI risks
More