When AI-generated p*rn message board posters began circulating comedian Bobbi Althoff’s deepfake videos, the clips reached a relatively muted audience, garnering 178,000 views over the past six months.
Then someone uploaded one of the videos to X. The fake one, which shows the 26-year-old naked and masturbating, was copied and reposted so many times that Althoff’s name became popular on the platform. In just nine hours, the clip received more than 4.5 million views – 25 times the views of p*rn sites, according to one industry analyst.
X, formerly Twitter, was one of the first social platforms to set clear rules against AI-generated fakes, with executives saying 2020 that they have recognized the threat of misleading “synthetic media” and that they are “committed to getting it right”.
But under owner Elon Musk, X has become one of the most powerful and prominent distribution channels for non-consensual deepfake p*rnography. The platform not only helps fake photos and videos go viral in a low-moderation environment, but can also end up rewarding spreaders of fake fake content who can use manipulated p*rnography to make money.
“Twitter is 4chan 2,” said Genevieve Oh, an analyst who studies deepfakes, referring to the dangerous, no-rules message board known for hosting not only deepfake p*rnography, but antisemitic memes and tributes to mass shooters. “It encourages future malevolent figures to coordinate towards demeaning more popular women with synthetic footage and images,” she said.
There is no federal law regulating deepfakes, although some states, such as Georgia and Virginia, ban non-consensual AI-generated p*rnography.
x prohibitions “non-consensual nudity,” but enforcement has been limited as the company, at Musk’s direction, has fired thousands of employees and gutted the “trust and safety” team that traditionally removed such images.
Musk mocked the need for content moderation. The day before Althoff’s video went viral, he shared message from X’s chatbot, Grokcalling content moderation a “digital belt of innocence” and a “hot pile of horse manure” carried out only by “digital tyrants”.
“Let’s give a big middle finger to content moderation and embrace the chaos of the internet!” the post said.
X did not respond to requests for comment.
X’s failure to stop deepfakes was highlighted last month when artificial intelligence generated images of pop star Taylor Swift having sex went viral on the platform, with tens of millions of views. Without enough moderators, the company took an unusual step blocking searches for Swift’s name.
But Althoff’s case shows that the company is grappling with the problem. One of the most popular posts directing viewers to the video remained online after more than 30 hours.
Another post, which promised to “send Bobbi Althoff’s full leaks to anyone who likes and comments,” was online for 8 hours — X removed it after The Washington Post sought comment on the fake posts. By the time it was removed, the video post had been viewed more than 5 million times.
Althoff, a content creator first known for her carefree nature TikTok parenting and pregnancy videos, has gained millions of followers on social media over the past year for a podcast in which he awkwardly interviews celebrities, including Drake and Shaquille O’Neal.
Althoff representatives did not respond to requests for comment. She took to Instagram on Wednesday, sharing a screenshot of her name on X’s trending list with a comment saying it was “100% not me and definitely AI generated.”
“I was like, ‘What… is this?'” she said in the video. “I felt like it was a mistake or something. … I didn’t realize that people actually believed it was me.”
Her name was found on more than 17,000 posts, according to a screenshot of X’s trending data. These topics were once filtered by a “curation team” that removed offensive trends. Under Musk, X fired them,also.
X is the only mainstream social platform that allows p*rnography, increasing the challenge for the remaining moderators who are responsible for deciding between genuinely explicit content and non-consensual spoofing.
But the company also encourages virality by offering to pay accounts with high viewership share of advertising revenue. Many of the accounts sharing the Althoff clip had blue ticks, which meant they were eligible for payout.
Many of the X posters who shared Althoff’s video sought to encourage their engagement by calling it an actual “leaked” sex scene or offering to send the video to anyone who shared or interacted with their tweet.
Deepfakes are made by using artificial intelligence to digitally superimpose someone’s face onto another body. They have been used for years to harass, shame and humiliate women and girls – including Hollywood actresses, online creators, members of Congress and high school teenagers whose photos were taken from social networks and artificially “naked”.
Deepfake forums, as well as platforms such as Telegram, have become common places for making photos and videos. Some users even ask for money to add a certain face to explicit scenes.
The creator of one fake Althoff video offered to sell a 20-minute version of it on a deepfake forum for $10, with payment via PayPal, according to a listing reviewed by The Post. (The preview video on the list has been viewed 60,000 times in the past four months.)
To gain attention outside of message boards, some deepfakes have moved their content to X, where they hope to sell more clips or attract a larger audience. Some of Swift and Althoff’s spoofs were also posted on platforms like Instagram and Reddit, but gained only a fraction of the audience there and were quickly removed.
To replace X’s moderators, Musk has often pointed to “Community Notes,” where volunteers can suggest comments that—with enough upvotes—will appear on specific tweets. But many of the posts featuring the fake Althoff video don’t include such notes, and some of the notes didn’t appear for several hours after the video went viral. Notes also do not prevent the clip from being viewed, shared, or saved.
One post, as of Wednesday afternoon, included a community note saying the video was generated by artificial intelligence and was being spread “knowingly to bait Twitter engagement and revenue.” The author of the original post — who suggested followers could find the “leaked” video in the “hidden” replies to the tweet — later wrote a comment: “Bobbi Althoff if you see this, I apologize.”
But the account didn’t remove the original post, and many X users went on to share it with their followers. After 24 hours, the original post had more than 20 million views and had been “liked” 29,000 times.
Will Oremus contributed to this report.
