Synthesised Voices Aren't a New Technology (the Hype Might Be, They Call It "Hey Hi" Now)
The eminent Jeff Geerling, whose voice had gotten 'cloned' with neither his knowledge nor permission, brought attention to this issue, even in Free software circles (photo source)
Synthesising voices of people is not only possible; it is very old technology. It is decades old. Nuance is the harder part (not the company). I played with it in the mid-90s. We didn't call it "generative" or "hey hi" (AI), but going by today's low standards we might as well have (or could have).
Companies including Adobe and GAFAM refined those things over time, taking advantage of offline compute time and real-time computing capacity (on the client side). It was more progressive than innovative and its evolution basically was expected all along (costs of storage and speed went down). Over time, given large enough training sets (data) and computation capacity, the capability/capacity to 'comprehend' (index) and synthesise voices would improve. There were some - but not many - use cases for it. Recently, unethical applications were adopted, initially for entertainment purposes and then for political stunts or even making people "say" indecent things (related to models that can superimpose them on naked people or synthesise naked bodies for them by extrapolation). It should be noted that this was done decades ago; here in the faculty where I studied some colleagues made a start-up about 20 years ago, but they found no use case for it and could not get Hollywood (or whatever) interested enough; even two decades ago synthesising realistic faces with animated expressions was more "for fun" than "for practical purposes".
Regarding inauthentic pornography or forged videos of high-profile people...
Bad people or rogue actors who do such stuff always existed. Back in the early 90s there were many fake "nudes" of famous people all around the Web (it's as old as the Web and probably predates the Web). Some were not generated by a computer but drawn; some were painted digitally using programs (those had already existed for decades by then). So the moral panic isn't something unprecedented. The issue was well covered in newspapers even in the 90s. Celebrities were mortified, seeing fake nudes of theirs (no, not really theirs; those were fabricated but terms like "deepfake" didn't exist). They demanded some kind of action.
Here we are in 2024 and the issue is discussed as if that's a new problem. An associate said "the recent bill veto of S.B. 1047 in California by the governor is relevant" because it shows the battle is on and some FOSS bloggers/vloggers are already being imitated ("They stole my voice with AI," Jeff Geerling wrote [1, 2]).
"It's not different than trademark or copyright in many ways," the associate noted. "It's a hot issue and ought not be ignored. It's related to copyright in that "AI" companies illegally scrape and recycle video and audio material from the net which they then use to falsify videos. See what recently happened to Jeff Geerling on that. He only got remediation due to a large following." (See the links above; we already included them in Daily Links at the time they were new)
I insisted that it is "not a very new issue, but it got much bigger owing to hype and perception it'll be a gold mine (for litigation lawyers, yes!)"
"It's not new at all," the associate responded, "but it is nonetheless a hot topic."
"Lawyers fold quickly to whomever has more money. The new phase of the problem is in its relatively early stage and thus treatable. There have, of course, been sell outs like the late James Earl Jones to name the latest one of a small number."
I was told that many articles did exist about this in the distant past, "but not about the damage it causes people." Even some dead people, who are resurrected [sic] by a synthesised voice and made to say things that upset their estate (George Carlin for instance).
In some cases, not only is this done without authorisation; it's done in spite of a very clear and very explicit lack of consent ("Scarlett Johansson was 'shocked, angered' by ChatGPT voice that sounded like her").
In Geerling's case, "Elecrow responded, apologized for AI voice cloning," to quote Geerling. But without being able to create strong backlash, this wouldn't end this way.
The way I see it, Elecrow didn't really stand a chance making money from Geerling (those grifters will perish along with the expensive bubble) because a person reading some text is cheap and adopting the voice of someone well known isn't a gold mine, it's primarily or at least reducible to hype. I did voiceover for videos when I was about 16 (for a company I was in) and I wasn't paid for. It seems improbable that someone will pay money to a company like Elecrow for the "service" of reading some text like Geerling. Funny? Maybe. Fun? Maybe. But it's more like a game, banter. It's pastime.
"It's hype," the associate said, "but the damage is real and with the easy availability of the software, the problem will not go away until curb stomped".
Also, "voiceover is not the same thing," he said, as "in voiceover you are paid and you are an active participant. In the voice cloning, your likeness is simply copied and misused."
Writing a "sky is falling" commentary might only exacerbate things as it helps the hype.
"It's not hype," the associate insisted though. "There is a real problem of companies falsifying the sound and appearance of various people including actors. The Scarlett Johansson incident was a *big* deal."
I still consider this an extension of the "hey hi" (AI) hype, especially the ripoff/plagiarism aspects. The associate agreed "it is *very* closely related to plagiarism. The Jeff Geerling incident was a low blow against him."
We agree on some things, but we disagree on the severity.
So far, with a month left until the US election, we haven't seen those things being used more ruthlessly than 4 years ago or 8 years ago (contrary to some doomsday predictions). In Social Control Media people could always spread fakes and many people did. For instance, Donald Trump falsely claimed that Taylor Swift had endorsed him, but she responded by saying that she endorses VP Harris, choosing to actively endorse his opponent instead of staying silent. So his fake basically backfired on him! Maybe the magnitude of this issue will increase in the coming weeks. Let's wait and see. █
Photo credit: How I make my YouTube videos - 100K Office/Studio Tour