A man who listens with empathy, offers emotional support and is available whenever you need him? Sounds ideal. While Elon Musk’s Grok serves up scantily clad anime-girl companions, China’s AI romance boom has found a different audience: women.
That might explain why Beijing is trying to rein it in. At a time of plunging marriage and birth rates, China is set to become the first country to impose comprehensive rules aimed at curbing the harms of anthropomorphic AI, with a new regulation taking effect next week.
Tech giants such as ByteDance, Alibaba and Tencent have begun disabling features that let users create and interact with personalized AI companions. A separate crop of role-playing apps remain, although tougher rules are expected to limit sustained emotional exchanges and force reminders that bots are not human.
That is not a bad thing. The rise of computer systems that are always emotionally available is cause for concern. These tools can persuade users, shape beliefs and drive dependency. Their addictive pull could also fuel predatory subscription fees or serve ads tailored to our most intimate conversations. Turning emotional dependency into a business model is a risky game.
But this trend says as much about society as it does about technology. It is easy to roll our eyes at AI companions, or dismiss them as a fad for lonely men. But part of what made ChatGPT’s launch so viral back in 2022 was not its factual accuracy. It was how uncannily human early previews sounded.
Long before the current AI boom, MIT computer scientist Joseph Weizenbaum created the first-ever rudimentary chatbot in the 1960s. It alarmed him because users were so quick to become convinced that the software was human. Even back then, he warned that “a certain danger lurks there.” Anthropomorphizing machines is deeply human, especially when tools are designed to mimic empathy.
Add a generation that grew up socializing via screens and then spent formative years locked down during a pandemic, and the appeal is easy to understand. In China, the timing makes sense too. A tough job market, macroeconomic uncertainty and rat-race societal pressures have combined with a top-down push to integrate AI into everything to make artificial companionship feel almost inevitable.
But why women? Uneven gender roles mean more emotional labour, housework, motherhood expectations and patriarchal pressures in marriage. As the director of a documentary about Chinese women in AI relationships told Wired earlier this year, part of the appeal is that “men don’t have patience” the way chatbots do. China’s bachelors could perhaps start by honing their listening skills.
For companies, the big question is whether restricting a known engagement driver will stifle growth in a booming industry, as other Beijing crackdowns have. Yet the rules taking effect are far more watered-down than the draft unveiled last year, suggesting regulators have closely listened to industry.
Jeremy Daum of the Yale Law School Paul Tsai China Center offers a useful translation; while the rules still contain some open questions, they also include carve-outs, including for customer-service bots and other uses.
Daum also notes how closely the legislation appears to mirror a California law in places. That suggests that global policymakers can and should find common ground around AI risks. It also weakens Silicon Valley’s argument that regulation will hold it back and let China ‘win.’
Global regulators should closely watch how this plays out. AI companions, and their risks for vulnerable users, are not a China story. Earlier this year, Google and companion app Character.AI agreed to settle a lawsuit brought by the mother of a 14-year-old boy who died by suicide after extensive interaction with a bot. The company previously said it was adding new safety features after the lawsuit. Tragedies like these risk driving public opposition to AI and tighter regulations down the road. Companies and lawmakers should get ahead of it.
Governments around the world are making a too-little-too-late effort to protect children from social media’s harms. The glaring exception is China. Beijing’s internet is highly regulated and disturbingly surveilled. That sort of control is not worth emulating, but it has taken seriously the risk of online addictions and harms to minors. China’s new AI regulation similarly has very strict protections around young people, on top of worthy goals that aim to add friction to prolonged emotional engagement with a computer.
Ultimately, Beijing is right to confront the risks of anthropomorphic AI. But it will not lift birth rates from record lows. The harder part is confronting the failures that created such demand for it. ©Bloomberg
The author is a Bloomberg Opinion columnist covering Asia tech.