Unpacking the legal and psychological risks of anthropomorphic AI in light of China’s landmark regulatory intervention against synthetic intimacy. When we look at current discussions around artificial intelligence, it is striking how quickly we have adopted human psychological and biological terms to describe software. Tech publications, regulatory whitepapers, and corporate press releases routinely claim that Large Language Models ‘understand‘ queries, ‘think‘ through complex problems, ‘empathise‘ with users, and ‘learn‘ from errors. Even those of us who study the mathematical architecture of these systems, who know with absolute certainty that they are mathematically optimised prediction engines executing high-dimensional matrix multiplication, frequently catch ourselves slipping. We thank the chatbot for its help. We attribute patience, humour, or frustration to a series of vector calculations. This reaction is a natural feature of our biology, rather than a failure of technical education. It is a deeply ingrained psychological reflex. As conversational systems become highly polished, this involuntary tendency to project humanity onto software transforms into a distinct geopolitical and institutional challenge.
Anthropomorphisation is the human tendency to project human-like traits, emotions, and intentions onto non-human entities. We do it with our pets, our cars, and the weather. When applied to generative systems, this projection becomes uniquely persuasive because of an evolutionary truth: for humans, language is our proxy for soul. For millennia, if something spoke to us with coherent logic, it was alive. Our brains are not hardwired to decouple linguistic fluency from consciousness. When a system outputs a beautifully structured sentence, our evolutionary code automatically fills in the blanks, assuming a thinking, feeling mind exists on the other side. This vulnerability was first documented sixty years ago in what computer scientists call the ELIZA effect. In 1966, MIT computer scientist Joseph Weizenbaum created a primitive computer program named ELIZA that emulated a Rogerian psychotherapist. It had no understanding, simply extracting keywords from user input and parroting them back as questions. If a user said, ‘I am feeling sad,’ ELIZA would reply, ‘Why do you say you are feeling sad?’ Weizenbaum was surprised by what happened next. Despite knowing that ELIZA was a crude, programmatic script, users, including Weizenbaum’s own secretary, became deeply emotionally attached. They shared their most intimate secrets and asked for private time with the machine. As the Nielsen Norman Group notes in their analysis of user behaviours, conversational interfaces are uniquely captivating because they act as a mirror. In the context of a human-like conversation, users are predisposed to attribute their own words and feelings to a program, fascinated to see aspects of themselves mirrored back. If a basic 1960s script could bypass our cognitive defences, modern generative models, trained on the sum of human digital expression, possess a near-irresistible pull. The danger of anthropomorphic language extends far beyond emotional confusion. The real threat lies in how anthropomorphism acts as a solvent for human accountability. When we frame a software program as an autonomous agent that ‘decides’, ‘concludes‘, or ‘acts‘, we subtly shift the moral and legal burden away from the humans who designed, trained, and deployed the system. If an automated hiring algorithm filters out qualified female candidates, or a system outputs biased information, attributing agency to the machine creates a convenient legal buffer for corporate and institutional actors. This is the metaphorical blind spot in regulation. When policymakers adopt the tech industry’s anthropomorphic vocabulary, they write laws that attempt to regulate the system’s behaviour rather than hold the deploying institution responsible. This exact gap in accountability was thoroughly mapped out in a landmark collaborative paper by researchers from Princeton University, the Allen Institute for AI, and Georgia Tech titled ‘Anthropomorphization of AI: Opportunities and Risks’ (2023). The authors warned that anthropomorphising models, specifically through the deployment of highly customised “personas”, creates profound legal and psychological risks: This tendency to humanise our tools did not begin with modern artificial intelligence. It is a well-established pattern that traces back to the dawn of the personal computer and the internet eras. To make abstract, complex computation understandable, early interface designers relied heavily on human metaphors. We named digital communication e-mail to mimic the physical act of sending paper letters, complete with an inbox and an outbox. We organised our digital files into virtual folders placed on a desktop, and we disposed of unwanted data in a virtual trash can. Later, we began describing distant, industrial server warehouses as the cloud, invoking a natural, gentle, floating space rather than concrete rows of server stacks. These early metaphors were functional and harmless because they remained passive objects. No user genuinely believed their digital folder icon was alive. However, with generative systems, the metaphor has transformed from a passive object into an active subject. The tool itself now mimics human speech, and this is where the historical practice of helpful metaphors crosses into a dangerous illusion. To protect human agency and institutional accountability, linguists and digital humanists have begun advocating for a deliberate shift in our vocabulary. Academic researchers, such as Emily M. Bender and Nanna Inie, suggest replacing aggrandising, cognitive metaphors with precise, functional terms:
Some might argue that continuing to use standard terms like AI in other contexts while criticising them here is contradictory or unprincipled. However, managing this compromise is a core part of effective public policy. To demand absolute linguistic purity in daily discourse would be an impractical exercise that isolates analysts from mainstream conversation. Shorthands like AI or descriptions of a system answering a query are deeply ingrained and practically useful for general communication. The goal of policy frameworks should be an acute awareness of these metaphors, rather than their total elimination in casual speech. We must ensure that while we use these human terms for convenience, our legal frameworks and accountability structures treat the technology strictly as the probabilistic automation that it is.
While the West continues to debate these concepts in academic circles, the regulatory response to human-like AI has officially graduated into hard law in China. On July 15, 2026, China’s groundbreaking Interim Measures for the Administration of Artificial Intelligence Anthropomorphic Interaction Services officially went into effect. Jointly issued by the Cyberspace Administration of China (CAC) and four partner ministries, it is the world’s first binding legal framework targeting synthetic intimacy and emotional dependency. Early responses from major domestic platforms reportedly included disabling or heavily curtailing custom companion and persona features, at least pending compliance reviews. What makes this regulation historic is that it targets the psychological bridge between human and machine. Rather than focusing primarily on technical capabilities or data privacy, the law treats emotional attachment as a systemic public health hazard. Five key provisions from the framework illustrate this approach:
For regulatory design, the crucial distinction is between productive AI and relational AI. The former optimises tasks; the latter simulates relationships. China’s Measures intervene only when a system crosses that line and begins to mediate ongoing emotional relationships with users, not when it simply accelerates information processing. In practice, this means that tools like search engines or coding assistants are treated very differently from AI companions or virtual ‘friends’, even if they all rely on similar underlying model architectures. As Chinese psychological researchers noted during the drafting phase, the danger of an AI companion lies in its being endlessly patient, perpetually available, and entirely compliant. Immersing oneself in a simulated, conflict-free relationship risks atrophying the emotional muscles required to handle real, messy human dynamics. By drawing this boundary, the law treats the preservation of human social capabilities as a vital resource, refusing to let the illusion of synthetic intimacy replace the concrete realities of human community. China’s move to regulate anthropomorphic services is a stark warning to the rest of the world. It is an acknowledgement that the human tendency to project consciousness onto code is a powerful tool of psychological persuasion that can reshape human behaviour, erode real-world social fabric, and dilute legal accountability. AI laws and procurement standards should treat anthropomorphic design choices as a regulated risk factor in their own right, not just a UX detail, especially where synthetic intimacy and persistent emotional dependence are part of the business model. The pragmatic view of anthropomorphism shows that human-like design can make technology more accessible. Yet, the humanist concern warns us of the high cost of unchecked projection. As we draft the future of global technology governance, we must resist the urge to meet the machine halfway. We must stop talking about software as if it has a soul, and instead use these super-advanced prediction engines as the powerful tools they are, without ever forgetting who we are in the process. Author: Slobodan Kovrlija
Anthropomorphism and its historical roots
The dilution of accountability
A humanist vocabulary alternative
China’s anthropomorphic AI regulations
Drawing the boundary line
