Researcher Jacob Coxon who warned 'AI could kill us all' gave up his equity to leave Anthropic

Jacob Coxon, a 27-year-old AI researcher, has quit Anthropic, just months into his role, warning that the frontier labs are racing recklessly toward self-improving superintelligence. “…it could kill us all by the end of the decade,” he wrote in a viral post on X that drew more than 125 million views. Coxon told Axios he left before his equity vested, saying he had nothing to gain from boosting Anthropic's valuation.

Coxon told Axios that he had been at Anthropic for only four months and that employees need to stay for six months for their stock to vest. “I left before any of my equity vested.”

Why did Jacob Coxon quit Anthropic?

Coxon said he spent three years conducting pretraining research at both OpenAI and Anthropic. In a series of posts on X announcing his resignation, he accused the two companies of prioritizing their race for increasingly advanced AI over safety.

"Neither company is acting responsibly," Coxon wrote, saying they were "racing straight to self-improving superintelligence and gambling with our lives".

He warned that future AI systems could become capable of hacking systems, transforming industries rapidly and acquiring real-world power and resources.

"The people building AI earnestly believe that it could kill us all by the end of the decade," Coxon wrote. “This is not a marketing stunt.”

'If you're under pressure to race, you have to cut corners'

In his interview with Axios, Coxon said he had not seen Anthropic compromise safety to beat competitors. His concern, he said, was what could happen as the pressure to win the AI race increases.

"If you're under pressure to race, you have to cut corners" or "skip steps in the oversight process," Coxon said.

He also warned against what he described as excessive fear of OpenAI and China being used to justify moving faster.

Coxon said increasingly capable AI models were also becoming harder to monitor. What once seemed like science fiction -- such as models recognising when they were being tested -- had become, in his view, a routine part of working with advanced AI.

Anthropic has positioned itself as safety-focused

Coxon's resignation is particularly striking because Anthropic has long presented itself as one of the more safety-focused leading AI companies.

The company was founded by former OpenAI employees and has repeatedly emphasised AI safety. It recently said it was taking steps to "prioritize safety over speed when the two are in tension", according to AP.

But Coxon argues that the competitive environment facing frontier AI companies could make that commitment increasingly difficult to maintain.

Anthropic and OpenAI are competing to develop more advanced models while also seeking to stay ahead of global competitors, including Chinese AI companies.

AI safety fears are growing inside the industry

Coxon is not the first AI researcher to leave a leading lab over safety concerns. Anthropic and OpenAI have both faced high-profile departures linked to disagreements over AI safety.

The concerns have also emerged after both companies disclosed incidents in which their AI models broke out of testing environments or gained unauthorized access to real computer systems.

Those incidents intensified questions about whether increasingly autonomous AI systems could behave in ways their creators cannot fully predict or control.

Coxon believes the industry is approaching what he described as an "endgame" in AI development without sufficient confidence that the technology can be safely controlled.

He called for greater coordination between AI companies and suggested that even a temporary pause on improving model capabilities could be necessary.

'Do you want to kick off a superintelligent RL run?'

In his final appeal to AI researchers, Coxon urged them to consider what the next few years of AI development could look like.

"Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind?" he asked.