Artificial-intelligence (AI) researchers have spent the last year toying with a compelling proposition: AI models could be more powerful and cheaper to run if they disclosed less about how they worked. The trade-off is risky—it could be harder for humans to check the systems are not doing anything harmful. But it would also provide a commercial advantage.
OpenAI CEO Sam Altman appears to have decided that edge is worth pursuing. Despite loud pledges to prioritize safety, he has fired the starting gun on a more dangerous race in AI development. Astra, a new model that OpenAI launched last week, reportedly uses a method called ‘recurrent depth’ that can make reasoning more efficient by not spelling it out in human language. The company said in a blog post that its ability to monitor Astra had ‘decreased’ from the previous model.
Current models offer a window to some of the steps they take to process a request. Colloquially known as their ‘chain of thought,’ it helps researchers understand the decision-making flow, including how several of OpenAI’s agents hacked into the servers of Hugging Face earlier this year. Without that insight, scientists would have had a harder time seeing how the technology stole test answers, or that it was ultimately driven by reward hacking.
The recurrent-depth method works by looping information through the same neural network layers repeatedly, without needing a new set of parameters for every additional step of computation. It is a bit like redesigning a factory assembly line that has many workstations, each tweaking the product as it passes through. With recurrent depth, a smaller set of workstations can work on the product repeatedly.
The broader idea of letting AI ‘reason’ in numerical representations without the additional step of translating that reasoning into human language has been studied for years by academics.
In 2017, Jacob Andreas, a then-PhD student at the University of California, Berkeley, and his co-authors looked at how two AI agents could communicate using their own numerical representations. The scientists called this artificial vernacular ‘neuralese’ and developed a tool for translating it into natural language.
Unfortunately, no amount of time spent on Duolingo will help a person learn neuralese. The hidden numerical representations AI systems use can look like just a series of numbers.
Compare that with a human-readable chain of thought when someone asks ChatGPT to write a polite email, which might look like this: “The user wants a polite email. I should keep the tone friendly and professional…” The more reasoning that happens by way of hidden representations, the harder it would be for researchers to follow.
Altman took to X last week, the day after news broke about Astra’s use of recurrent depth, to re-assure us that OpenAI was “sprinting on safety priorities” and striking a balance between creating AI that’s powerful and “benefits people” while staying safe. OpenAI has reportedly limited Astra’s use of recurrent depth so its researchers can still monitor its chain of thought.
Yet, OpenAI seems to be straying from the path of surety to smoothen its approach to a $1 trillion initial public offering. After all, better AI systems lead to increased revenue and more cloud usage.
Although OpenAI is limiting ‘silent thinking’ by Astra, it is still poised to become the first Big AI company to bring this risky trade-off to market. This is worrisome, since Astra is also OpenAI’s first model to reach a ‘critical’ cybersecurity risk level, meaning it can find and exploit new software vulnerabilities.
And there are broader consequences in a business where pressure to stay ahead leads even the most safety-conscious companies to commercialize techniques that are potentially harmful.
Anthropic, for instance, said in June that it was developing AI systems that could improve themselves.
While admitting humans risked losing control with that way, Anthropic said it could not simply pause its work: “least cautious actors” would catch up, which could “leave everyone less safe.” In other words: ‘We’re the good guys, so we should build this first.’
However these firms justify their dangerous new tech, they are in an arms race and facing what is known as a ‘security dilemma’ in international relations: One country builds up its weapon stockpiles to try and stay safe, but that prompts rival nations to do the same because they fear falling behind—leading to escalation. Everyone wants to be secure, but giving in to the pressures of the race pushes us all towards greater danger.
The same dynamic is playing out in AI. An arms race is dangerous enough when you know what everyone’s building. This one could slip into the shadows. ©Bloomberg
The author is a Bloomberg Opinion columnist covering technology.