Op-Ed: Deepfake Audio Brings Sinatra, Beatles & Presley Back To Life – Up To A Point
Press photo of The Beatles during the Magical Mystery Tour. – Parlophone Music Sweden (CC BY 3.0)
Deepfake audio, songs created by artificial intelligence, is still in its infancy, but it is already getting interesting. A pretty commendable deepfake Sinatra has garnered a lot of attention, and he’s very close to Ole Blue Eyes vocally. Deepfakes are often called “in the style” of artists. This is a rough description.
The AI machine learns songs from millions of tracks by a number of artists. The “style” is configured using these tracks. Interestingly, the AI finds some good hooks, and even some bizarre lyrics, some of which even word-smith John Lennon wouldn’t have bothered.
The sound varies, a lot. Some of them are very “digital garage”. Doesn’t sound balanced, things are buried in the signal. It’s often enough, however, to be worth hearing just to find out what it can do.
What he can do is put together some really interesting stuff and some good hooks. It’s not as two-dimensional as the idea might sound. It creates hybrid songs, and some of them really deserve a little respect as musical ideas.
Open AI Jukebox is one of the main drivers of deepfakes. You can make your own, but there is a lot to learn about how these things are put together. Jukebox is very open about the limits of deepfakes, and their site is worth seeing in terms of what it can and can’t do.
I have to say I love the amount of effort put into making deepfakes work. It’s quite a technical feat. The amount of data required for any music track is appalling in normal formats, let alone synthesizing new tracks from millions of other tracks.
Be warned – There is a lot of back-end to this idea. You have to check out how it’s done to really understand what it can do and why. There are obviously many more technologies and developments to come.
Comparing deepfakes to the revolting, stagnant, stereotypical slop that now claims to be pop music, all is in favor of deepfakes. (Admit it, bozos; you’re dead and buried, but you’re still making noise.) Deepfakes are relatively honest attempts at making music, not just recycling high-fructose recipes, supposedly musical nothings. .
Copyright problem? Maybe, maybe not.
There is a naturally sensitive question as to whether you can copyright a built-in voice to tamper with the audio. In theory, yes. In practice, maybe not. A voiceprint can be copyrighted, in theory. Whether or not you can stick with an AI that will have a few tone differences or pre-programmed is another matter.
Music copyright is a different, and sometimes quite nasty, ball game. Using similar structures may violate copyright, but it could be easy to find and fix. A fairly similar AI could do a “Copyscape for Music” quite easily. It would end the endless copyright fights just by reading and comparing the two pieces of music.
You can run your music through a Musical Copyscape and see if anything else affects your copyright. Easy and above all fast. Unlike the senseless rage assaults on people sharing a few files, you can fix copyright instantly, without the melodrama.
What to do with deepfake audio
There are many possibilities. If you’re a bit of a music lover like me, someone who sometimes makes unprovoked musical noises, there are a lot of them.
Musicians should also note that they can use things like Jukebox to review their own music. The Beatles deepfakes are quite interesting, and I’m pretty sure Lennon-McCartney would have been quick to see the value of having a few more options. (They both said, on a regular basis, that songwriting can be a chore. This way of adding options to songs might eliminate some of those problems.)
i thought i would do a deepfake The dark side of the moon. i would call her Spoon Smug Snide. Somewhere between a sneer and a genuine interest in seeing what any sort of intelligence could do with the undeclared musical strengths of most of Floyd’s albums. A track called Big pig in the pie, for example, could fit together Animals with Dark side, perhaps groping for this ultra-mundane lived sound. “Obscured by clods“, may be?
On the more practical side of deepfake audio, any music track is a series of decisions. Most musicians don’t have the time, patience, or space to make other deeply insightful options. As a creative asset, deepfake audio can be the AI version of George Martin that so many people need.
In any event – Don’t ignore it. It is potentially extremely useful and interesting. These are AI’s first steps in composing music with character rather than just sheet music to work with. I see a lot more happening, and that’s mostly good.
Check out this YouTube playlist of deepfakes and deepfake fixes. You will at least hear a few things that you weren’t expecting to hear, and that’s a really good sign for the future.