ByteDance has turned off AI-image generation capabilities in one of its AI apps after 4chan users appeared to generate pornographic content.
ByteDance quietly launched its generative AI chatbot “Cici AI” sometime between the end of 2023 and January 2024. The chatbot is currently in its testing stage and is not meant to be available in the US. But 404 Media reports that 4chan users are using VPNs to access the app.
The chatbot app was developed by ByteDance subsidiary Spring Pte Ltd. It is partially powered by OpenAI’s ChatGPT via a Microsoft Azure license. Like ChatGPT, Cici AI functions as a chatbot and is able to generate AI images from text prompts.
According to 404 Media, AI-generated porn started to appear on Telegram and 4chan earlier this month, with users claiming that they made it using Cici. These images circulated in the same forums in which Taylor Swift deepfakes were previously found in.
404 Media also found that the images generated were “far more graphic” than the ones generated in the past by Microsoft Designer users.
While Cici AI appears unable to produce celebrity deepfakes, 404 Media was able to replicate explicit images by using prompts previously suggested to have worked by 4chan and Telegram users. It was only after approaching ByteDance for comment that the malfunction seemed to be fixed.
Correction: The headline of this article was updated to more accurately reflect the nature of the content created by 4chan users.
