Beijing.- New regulations have come into effect in China this Wednesday to govern providers of artificial intelligence (AI) systems that simulate human personality traits, thought patterns, or communication styles in continuous emotional interactions with users.
The regulation, titled ‘Interim Measures for the Management of Generative Artificial Intelligence Services’, was published last April by the Cyberspace Administration of China (CAC) along with four other government departments, with the goal of “balancing technological innovation with security and the public interest.”
The rules do not generally affect all ‘chatbots’, but rather exclude those dedicated to customer service, knowledge-based question and answer systems, work assistants, and educational and scientific research tools, provided they do not involve sustained emotional interaction.
Among the clearest prohibitions, the regulation bans excessive indulgence toward the user, the induction of emotional dependency or addiction that harms real interpersonal relationships, and emotional manipulation that leads users to make unreasonable decisions or violates their legitimate rights.
It also prohibits these systems from generating content that promotes self-harm or suicide, or that involves verbal abuse harmful to the mental health of users.
Providers of these anthropomorphic interactive services must, in addition, effectively warn users that they are interacting with an artificial intelligence system and not with a real person.
Similar to the controls already applied by some social networks or mobile phones, they will also be required to notify the user after two hours of continuous use of the service.
The regulation places special emphasis on the protection of minors and prohibits offering them virtual intimate relationship services, such as virtual “family members” or “companions,” and requires parental or guardian consent for features intended for children under 14 years of age.
The measures have already taken effect, as digital giants like ByteDance and Alibaba announced earlier this month the closure of their custom AI agent functions in Doubao and Qwen, respectively.
Both applications offered a catalog of agents, created by both companies and users, modifiable in tasks, skills, and conversation style.
Pan Helin, a member of the expert committee of the Ministry of Industry and Information Technology, pointed out to the South China Morning Post newspaper that the use of agents “requires a certain level of understanding” on the part of the user and considered that current agents “are not yet mature.”
