Instagram is flooded with hundreds of AI-generated influencers who are stealing videos from real models and adult content creators, giving them AI-generated faces, and monetizing their bodies with links to dating sites, Patreon, OnlyFans competitors, and various AI apps.
The practice, first reported by 404 Media in April, has since exploded in popularity, showing that Instagram is unable or unwilling to stop the flood of AI-generated content on its platform and protect the human creators on Instagram who say they are now competing with AI content in a way that is impacting their ability to make a living.
According to our review of more than 1,000 AI-generated Instagram accounts, Discord channels where the people who make this content share tips and discuss strategy, and several guides that explain how to make money by “AI pimping,” it is now trivially easy to make these accounts and monetize them using an assortment of off-the-shelf AI tools and apps. Some of these apps are hosted on the Apple App and Google Play Stores. Our investigation shows that what was once a niche problem on the platform has industrialized in scale, and it shows what social media may become in the near future: a space where AI-generated content eclipses that of humans.
Elaina St James, an adult content creator who promotes her work on Instagram, said she and other adult content creators are now directly competing with these AI rip-off accounts, many of which use photographs and videos stolen from adult content creators and Instagram models. She said that while there may be other changes to Instagram’s algorithm that could have contributed to this, since the explosion of AI-generated influencer accounts on Instagram her “reach went down tremendously,” from a typical 1 million to 5 million views a month to not cracking a million in the last 10 months, and sometimes coming in under 500,000 views.
“This is probably one of the reasons my views are going down,” St James told us in an interview. “It’s because I’m competing with something that’s unnatural.”
Alexios Mantzarlis, the director of the security, trust, and safety initiative at Cornell Tech and formerly principal of trust and safety intelligence at Google, compiled a list of around 900 accounts that 404 Media reviewed in its investigation. Mantzarlis, who stumbled on one of these accounts while casually using Instagram, said he started researching the AI-generated influencer accounts because it might show us where AI-generated content is taking social media and the internet more broadly, where he sees a “a rising blended unreality.” Mantzarlis believes he could have easily found 900 more accounts and that the only reason he didn’t get more is that Instagram restricted the account he used to scrape the platform.
“It felt like a possible sign of what social media is going to look like in five years,” Mantzarlis said in an interview. “Because this may be coming to other parts of the internet, not just the attractive-people niche on Instagram. This is probably a sign that it’s going to be pretty bad.”
The Accounts
Out of more than 1,000 AI-generated Instagram influencer accounts we reviewed, 100 included at least some deepfake content which took existing videos, usually from models and adult entertainment performers, and replaced their face with an AI-generated face to make those videos seem like new, original content consistent with the other AI-generated images and videos shared by the AI-generated influencer. The other 900 accounts shared images that in some cases were trained on real photographs and in some cases made to look like celebrities, but were entirely AI-generated, not edited photographs or videos.
Out of those 100 accounts that shared deepfake or face-swapped videos, 60 self-identify as being AI-generated, writing in their bios that they are a “virtual model & influencer” or stating “all photos crafted with AI and apps.” The other 40 do not include any disclaimer stating that they are AI-generated.
One of the bigger accounts in the latter category is “Chloe Johnson,” who has a verified account on Instagram and 171,000 followers. This account was deleted by Meta within the past few weeks.