Late last week, OpenAI confirmed that new artificial intelligence (AI) models undergoing an internal evaluation had exploited a hidden loophole to break deep into Hugging Face, an open-source AI hub, and steal answer keys to ace their tests.
Ironically, Hugging Face’s security team eventually used an open-weight Chinese model to help plug the hole after a top American model’s safety filters refused to do that task. Hugging Face CEO Clem Delangue dismissed any malicious intent on OpenAI’s part, while calling this first-of-its-kind breach “mind-blowing.”
In April, a similar incident had unsettled the industry. Anthropic’s Claude Mythos Preview reportedly uncovered thousands of vulnerabilities across major operating systems and browsers. Anthropic later briefed the Financial Stability Board on these risks.
Sceptics may dismiss such news-making events as pre-listing theatrics aimed at goggle-eyed investors. With OpenAI and Anthropic eyeing public funds, capability demonstrations wrapped in warnings could be viewed as timed to support valuations. Others may see them as early evidence of machine autonomy gone ‘rogue,’ the stuff of dystopian forecasts.
Both interpretations merit attention. But there has been no ‘Skynet moment’ so far: OpenAI’s test models were not rebelling, nor did Mythos have any ambition. They were in pursuit of goals set for them by tests.
What shook researchers was how creatively the models found paths their designers had not anticipated. This is a form of reward hacking: the AI finds an optimal way to succeed. The methods it employs could stun us because we have an implicit moral code.
What appalls us, though, could empower cybercrime. Attacks could be automated to work faster than defences go up. Properly harnessed, however, the same AI ability could find novel ways to reduce traffic congestion, identify molecular pathways that accelerate drug development, redesign industrial processes to save energy or uncover scientific relationships hidden in vast data-sets.
The more capable the system, the more freedom it needs to generate new solutions. Yet, the more liberty it has, the harder it is to predict how it goes about its work.
‘Model explainability’ may be an answer to that, but advanced tools could offer explanations that humans lack the expertise to verify. As AI advances, we may need auditable links between objectives, decisions and actions.
The Anthropic incident prompted the White House to sign an executive order in June that gave US agencies a 30-day window to vet risky models before release. The US has even proposed a ‘kill switch’ for frontier AI models. But any such device must stay beyond AI’s sphere of influence.
Meanwhile, researchers like Yann LeCun caution against anthropomorphizing AI.
Machines, they argue, do not need human-like desires or consciousness to behave in ways that seem devious or deceptive. This is the alignment trade-off. Grant AI too little space and it proves useless; give too much and it might wreak havoc. The challenge, then, is to govern intelligence rather than rein it back.
Unevenly adopted kill switches could create a geopolitical asymmetry: countries with strict controls might constrain their progress while rivals charge ahead. Calibrated control may offer a way out: pre-cleared permissions for security mavens who must respond instantly, with test labs held to the same safety standards as released tools.
Ultimately, though, it’ll take global cooperation to keep the world safe from rogue AI.