Scammers are using AI to effectively steal content creator identities and use them to post fake ads. It's another unfortunate example of how bad actors are using the fast-developing technology for nefarious purposes more quickly than companies and authorities can stop them.
And even if victims identify their scammers, there may be surprisingly little they can do to get justice or prevent further attacks.
According to The Washington Post, scammers scour social media for well-performing videos from different content creators. The video itself doesn't have to be about anything specific — it just needs to feature the creator in a space the scammer finds suitable for their ad.
Then, they use platforms like HeyGen and Eleven Labs to create synthetic versions of that creators voice and swap out the audio with their own words — and then animate the lips. The results vary, but with a little fine tuning and additional manipulation, these spoofs can have shockingly believable results.
In many cases, the Post found these videos were used to also promote dubious products on places like adult websites. One victim, Michel Janse, first heard from her followers that her likeness was being used to sell erectile dysfunction pills. Scammers utilized a video that showed Janse in her real clothes and bedroom "talking" about a fictional partner with fictional problems.
On top of being a complete violation of Janse's privacy and an unauthorized use of her likeness, the videos can also be damaging to her brand. Janse is a Christian influencer who talks about things like travel, wedding planning, and home decor. Having a spoofed version of her show up on adult websites and "selling" deceitful products can lead to a serious hit to her brand and ability to land legitimate partnerships.
These types of scams have roots in technology from several years ago. Most people have probably heard the term "deepfake" by now. It was coined in 2017 by a reddit user who began posting synthetic pornographic videos that utilized celebrity faces on adult actor bodies. The word itself refers to "deep-learning technology" (a form of machine learning) and, well, the fact that the results are fake.
For years, the concern around this type of content largely revolved around popular figures and adult content. As AI has evolved, deepfakes were used to create videos of popular celebrities appearing to sell or endorse things or ideas. The biggest creators in the world, from Taylor Swift and Tom Hanks to Mr. Beast, have all been victims of these types of scams.
But those creators are highly visible and have more resources to identify misuse of their likeness. Now, scammers are using much smaller creators who may never even know they've been spoofed. Even just a few seconds of content is enough to train some fairly convincing AI replications of a person.
Perhaps the most frustrating part of the whole ordeal is how little control creators have over removing this content. Because much of it appears in advertising on various websites across the Internet, it's difficult to know if your likeness has been misused. On top of that, some sites aren't beholden to any regulations or laws that would force them to remove the ads or content. If a site is hosted in a country that doesn't have any impetus to do anything about the content, complaints may fall on deaf ears.
YouTube said in November 2023 that they would be working on ways to more quickly identify fake content created by AI. In March 2024, the company introduced a new policy requiring users to disclose whether AI was used to create content that could be perceived as real by viewers. YouTube also reiterated its work to create a process that allows users to request the removal of synthetic content.
But that's still a major issue — it puts the burden on the victims to identify misuse of their likeness. Similar to other forms of identity theft, it could be months or even years before you realize your likeness has been stolen. While platforms like TikTok and YouTube are slowly catching up to the curve on basic identification measures, it's going to take serious technology and industry regulation to create safeguards against future criminal use of AI.
Australian content creators are taking matters into their own hands to unmask online trolls. Could it lead to a safer Internet? Indy Clinton Unmasks Online TrollsIndy Clinton recently shared a post that has garnered more than 3.7 million views. In it, Clinton is dancing in her kitchen — a seemingly standard video, until you read the
Read MoreAI-generated music is probably among the most convincing of the current world of AI slop. And it’s in a lot more places than you may realize, whether its underscoring videos or taking up a spot on your weekly playlist. The thing is — with how streaming companies pay artists, it’s not all just fun and
Read MoreA U.S. judge has ruled that Anthropic’s use of copyrighted materials in its AI training falls under the doctrine of “fair use.” The judge did not, however, absolve Anthropic of illegally pirating those materials before deciding to buy copies of them. The ruling directly contradicts another judge’s finding earlier in 2025, which determined that training AI
Read MoreDuring a panel at AthleteCon, full-time creators Emily Harrigan, Caden Davis, and Lacey Jane Brown discussed making content in a life after college athletics. The trio got candid about the ups and downs of making a living as a content creator and offered some tips to anybody else looking to make the leap.Practical Tips For
Read MoreNever miss a good story!