As artificial intelligence (AI) rushes to outsmart humans, it is natural to feel uneasy about what will happen next.
SpaceX founder Elon Musk has said he thinks AI could surpass the combined intelligence of humanity within five years and leave us in a struggle for control within a decade. Dario Amodei, CEO of Anthropic, believes systems with extraordinary intellectual abilities could emerge as early as this year or next.
Google DeepMind goes further, forecasting the arrival of human-level AI, or artificial general intelligence (AGI), before a transition to artificial super-intelligence (ASI).
Its ‘minimum AGI’ could arrive by 2028, so we can expect an AI agent capable of the full range of basic cognitive tasks that have been uniquely human so far. Its ‘maximum’ version could master top-level human cognition over the next 10 years. If so, we might see AI agents making scientific leaps.
Such forecasts inevitably raise the question of whether AI might also become conscious. Experts like Coursera co-founder Andrew Ng, Meta’s former AI chief scientist Yann LeCun and World Labs CEO and Stanford HAI director Fei-Fei Li caution that agentic systems can plan, use tools, retain context and execute tasks on their own, but this agency does not imply a human understanding of the physical world.
An agent cannot feel, let alone suffer, the way humans do. Yet, most mavens agree that AI will be clever enough to act human-like in a way that would confound our ability to identify it as a digital creation. This risks our falling for synthetic flattery, empathy and buddy talk, thus exposing us to mass AI manipulation.
Historian Yuval Noah Harari, who also argues that our species is at a critical point, has urged regulators to ban “fake humans,” or AI bots in the guise of people, just as we outlaw counterfeit money to shield the economy.
This view merits attention. Even basic chatbots know users closely enough to adapt their language, detect emotional cues and customize responses. But AI can exploit fears, preferences and habits to meet hidden ends. Rogue AI agents and lawsuits that allege chatbot roles in suicide add to the anxiety.
Blanket bans, however, could impede tech innovation. This explains why the EU’s AI Act has a risk-based AI framework that bars harmful manipulation and exploitation, while imposing stiffer obligations on riskier systems.
From this August, bots must explicitly inform users of their bot-hood, unless the context makes it obvious. The Council of Europe’s Framework Convention also has similar safeguards. In the US, Senator Mark Warner’s proposed AI Agent Act focuses on consumer agents, privacy, security, competition and accountability.
An ideal global rulebook would require clear disclosure of digital identity, prohibit the casting of hypnotic spells and hold AI toolmakers—and deployers in work contexts—accountable for harms. Granted, as AI evolves and learns how to develop itself further, enforcing rules will get harder.
Regulation, ironically, may need AI to keep pace with AI. How AGI or ASI might turn ‘conscious’ is a matter of semantics (vulnerable to an AI capture of vocabulary) that must not detain us. We must instead focus on how self-driving vehicles, care robots, military drones and AI agents are forcing engineers to keep their evolution in check.
Just as Asimov’s ‘Three Laws of Robotics’ offered a framework before sci-fi leapt into the real world, we need rules that are clear, adaptive and ready for a potentially pivotal regulatory race.