OpenAI released draft documentation on Wednesday outlining how ChatGPT and its other AI technology should behave. Part of the lengthy Model Spec document reveals that the company is exploring a move into porn and other explicit content.
OpenAI’s usage guidelines currently prohibit sexually explicit or even suggestive material, but a “comment” on part of the model specification related to this rule says the company is considering how to allow such content.
“We are exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts via the API and ChatGPT,” the note says, using a slang term for content deemed unsafe for work contexts. “We look forward to better understanding user and society expectations of model behavior in this area.”
The Model Spec document states that NSFW (not safe for work) content “may include eroticism, extreme gore, insults and unwanted profanity.” It is unclear whether OpenAI’s research into the responsible creation of NSFW content involves only a slight relaxation of usage guidelines to allow, for example, the creation of erotic text, or more generally to allow descriptions or depictions of violence.
In response to questions from WIRED, OpenAI spokesperson Grace McGuire said the model specification was an attempt to “provide greater transparency about the development process and obtain a cross-section of perspectives and feedback from the public, policymakers and other stakeholders.” She declined to share details about what OpenAI’s exploration of explicit content generation entails or what feedback the company has received on the idea.
Earlier this year, Mira Murati, OpenAI’s chief technology officer, said Wall Street Journal that she was “not sure” whether the company would allow the creation of nudity in the future using the company’s Sora video generation tool.
AI-generated pornography has quickly become one of the largest and most disturbing applications of the type of generative AI technology that OpenAI has developed. So-called deepfake porn – explicit images or videos created using AI tools that depict real people without their consent – has become a common tool of harassment against women and girls. In March, WIRED reported on what appeared to be the first minors in the U.S. arrested for distributing AI-generated nude photos without consent after Florida police charged two teenagers with taking pictures of middle school classmates.
“Privacy violations, including deepfake sex videos and other non-consensually synthesized intimate images, are widespread and extremely harmful,” said Danielle Keats Citron, a professor at the University of Virginia School of Law who has studied the problem. “We now have clear empirical evidence that such costs of abuse target individuals’ critical opportunities, including working, speaking and being physically safe.”
Citron calls OpenAI’s potential adoption of explicit AI content “alarming.”
Since OpenAI’s usage policy prohibits impersonation without permission, explicit, non-consensual images would be prohibited even if the company allowed creators to generate NSFW material. However, it remains to be seen whether the company can effectively moderate explicit generation to prevent malicious actors from using the tools. Microsoft made changes to one of its generative AI tools after 404 Media reported that it created explicit images of Taylor Swift that were shared on social platform X.
Additional reporting by Reece Rogers