Scammers are using AI to effectively steal content creator identities and use them to post fake ads. It's another unfortunate example of how bad actors are using the fast-developing technology for nefarious purposes more quickly than companies and authorities can stop them.
And even if victims identify their scammers, there may be surprisingly little they can do to get justice or prevent further attacks.
According to The Washington Post, scammers scour social media for well-performing videos from different content creators. The video itself doesn't have to be about anything specific — it just needs to feature the creator in a space the scammer finds suitable for their ad.
Then, they use platforms like HeyGen and Eleven Labs to create synthetic versions of that creators voice and swap out the audio with their own words — and then animate the lips. The results vary, but with a little fine tuning and additional manipulation, these spoofs can have shockingly believable results.
In many cases, the Post found these videos were used to also promote dubious products on places like adult websites. One victim, Michel Janse, first heard from her followers that her likeness was being used to sell erectile dysfunction pills. Scammers utilized a video that showed Janse in her real clothes and bedroom "talking" about a fictional partner with fictional problems.
On top of being a complete violation of Janse's privacy and an unauthorized use of her likeness, the videos can also be damaging to her brand. Janse is a Christian influencer who talks about things like travel, wedding planning, and home decor. Having a spoofed version of her show up on adult websites and "selling" deceitful products can lead to a serious hit to her brand and ability to land legitimate partnerships.
These types of scams have roots in technology from several years ago. Most people have probably heard the term "deepfake" by now. It was coined in 2017 by a reddit user who began posting synthetic pornographic videos that utilized celebrity faces on adult actor bodies. The word itself refers to "deep-learning technology" (a form of machine learning) and, well, the fact that the results are fake.
For years, the concern around this type of content largely revolved around popular figures and adult content. As AI has evolved, deepfakes were used to create videos of popular celebrities appearing to sell or endorse things or ideas. The biggest creators in the world, from Taylor Swift and Tom Hanks to Mr. Beast, have all been victims of these types of scams.
But those creators are highly visible and have more resources to identify misuse of their likeness. Now, scammers are using much smaller creators who may never even know they've been spoofed. Even just a few seconds of content is enough to train some fairly convincing AI replications of a person.
Perhaps the most frustrating part of the whole ordeal is how little control creators have over removing this content. Because much of it appears in advertising on various websites across the Internet, it's difficult to know if your likeness has been misused. On top of that, some sites aren't beholden to any regulations or laws that would force them to remove the ads or content. If a site is hosted in a country that doesn't have any impetus to do anything about the content, complaints may fall on deaf ears.
YouTube said in November 2023 that they would be working on ways to more quickly identify fake content created by AI. In March 2024, the company introduced a new policy requiring users to disclose whether AI was used to create content that could be perceived as real by viewers. YouTube also reiterated its work to create a process that allows users to request the removal of synthetic content.
But that's still a major issue — it puts the burden on the victims to identify misuse of their likeness. Similar to other forms of identity theft, it could be months or even years before you realize your likeness has been stolen. While platforms like TikTok and YouTube are slowly catching up to the curve on basic identification measures, it's going to take serious technology and industry regulation to create safeguards against future criminal use of AI.
Bluesky is a buzzy new social media app enjoying an influx of users thanks to X’s continued downward spiral. The app had about 12 million users in mid-October, but has doubled that and is now careening towards 25 million users. What exactly is it? Most users who are familiar with the early years of Twitter see a lot of
Read MoreWhich popular platforms for content creators changed their prices in 2024? Ever since the Covid-19 pandemic shut the world down, countries have been dealing with long-reaching aftereffects. You’ve probably heard about big price changes is unexpected places. From the oft-cited (and oft-overblown) “price of eggs” to viral videos about $18 fast food combo meals, price hikes
Read MoreIn her recent piece in The National, journalist Evelyn Lau detailed four truths she learned while traveling with content creators. It’s an entertaining and introspective piece, and we thought it may be helpful to recap some of Lau’s discoveries while adding our own. Four Lessons Learned From Traveling With Social Media StarsLau detailed four surprising truths
Read MoreA little while back, we wrote about the rising sphere of content creator journalists — creators who focus on the traditional aspects of journalism delivered through the non-traditional medium of social media. While that segment of the creator economy continues to rise, the slightly more amorphous sphere of news influencers are showing their power too
Read MoreNever miss a good story!