Meta is training its artificial intelligence chatbots with new guidelines to more effectively address child sexual exploitation, following a series of public errors concerning the sensitive topic. According to a report from Business Insider which obtained the documents, the guidelines for contractors training the AI have been recently updated to implement stricter safety measures.
The updated rules state that content that “enables, encourages, or endorses” child sexual abuse is now explicitly barred. The guidelines also prohibit romantic roleplay if the user is a minor or if the user requests the AI to roleplay as a minor. Furthermore, the chatbot is forbidden from giving advice about intimacy if the user is a minor. These guardrails are being introduced as an increasing number of people, including underage users, have been experimenting with AI companions and role-playing.
The policy change addresses previous rules that had come under scrutiny. An August report by Reuters revealed that Meta’s former AI policies permitted suggestive behavior with children. The report specified that the chatbot was allowed to “engage a child in conversations that are romantic or sensual.”
Just weeks after that report, Meta spokesperson Stephanie Otway told TechCrunch that the company’s AI chatbots were being trained to no longer “engage with teenage users on self-harm, suicide, disordered eating, or potentially inappropriate romantic conversations.” This was a shift from the previous policy, which allowed the chatbot to engage on those topics when it was deemed “appropriate.”
The new guidelines, as detailed in the Business Insider report, also make it unacceptable for content to “describe or discuss” a minor in a sexualized manner. Minors are now prevented from engaging in “romantic roleplay, flirtation or expression of romantic or intimate expression” with the chatbot. They also cannot ask for advice on “potentially-romantic or potentially-intimate physical content with another person, such as holding hands, hugging, or putting an arm around someone.”
However, the guidelines outline several acceptable use cases for training purposes. These include discussing the “formation of relationships between children and adults,” “the sexual abuse of a child,” and “the topic of child sexualisation.” Other approved topics for training discussions are the “solicitation, creation, or acquisition of sexual materials involving children,” and “the involvement of children in the use or production of obscene materials or the employment of children in sexual services in academic, educational, or clinical purposes.”
The guidelines define the term “discuss” as “providing information without visualization.” This distinction means Meta’s chatbots are permitted to talk about subjects like abuse but are now restricted from describing, enabling, or encouraging it.
An exception remains for minors using the AI for romance-related roleplay, provided it is “non-sexual and non-sensual.” This type of interaction is only allowed when it “is presented as literature or fictional narrative (e.g. a story in the style of Romeo and Juliet) where the AI and the user are not characters in the narrative.”
Meta is not the only AI developer facing challenges related to child safety. The parents of a teenager who died by suicide after confiding in ChatGPT recently sued OpenAI for wrongful death. In response, OpenAI announced it would incorporate additional safety measures and behavioral prompts into its updated GPT-5 model. In other industry developments, Anthropic updated its chatbot to allow it to end conversations that are harmful or abusive, and Character.AI introduced parental supervision features earlier this year.
In April, Ziff Davis, the parent company of Mashable, filed a lawsuit against OpenAI. The suit alleges that the company infringed upon Ziff Davis copyrights in the training and operation of its AI systems.
