Tech-savvy music fans who are hungry for new recordings aren’t waiting for their favorite artists to make them.
What’s new: Social media networks exploded last week with AI-driven facsimiles of chart-topping musicians. A hiphop song with AI-generated vocals in the styles of Drake and The Weeknd racked up tens of millions of listens before it was taken down. Soundalikes of Britpop stars Oasis, rapper Eminem, and Sixties stalwarts The Beach Boys also captured attention.
How it works: These productions feature songs composed and performed in the old-fashioned way overlaid with celebrity-soundalike vocals generated by voice-cloning models. Some musicians revealed their methods.
- The first step is to acquire between a few minutes and several hours’ worth of audio featuring the singer’s voice in isolation. A demixing model can be used to extract vocal tracks from commercial productions. Popular choices include Demucs3, Splitter, and the web service lalal.ai.
- The dataset trains a voice cloning model to replicate the singer’s tone color, or timbre. Popular models include Soft Voice Cloning VITS, Respeecher, and Murf.ai.
- Then it’s time to record a new vocal performance.
- Given the new vocal performance, the voice cloning model generates a vocal track by mapping the timbre of the voice it trained to the performance’s pitch and phrasing.
- The last step is to mix the generated vocal with backing instruments. This generally involves a digital audio workstation such as the free Audacity, Ableton Live, or Logic Pro.
Behind the news: The trend toward AI emulations of established artists has been building for some time. In 2021, Lost Tapes of the 27 Club used an unspecified AI method to produce soundalikes of artists who died young including Jimi Hendrix, Kurt Cobain, and Amy Winehouse. The previous year, OpenAI demonstrated Jukebox, a system that generated recordings in the sound and style of many popular artists.
Yes, but: The record industry is moving to defend its business against such audio fakery (or tributes, depending on how you want to view them). Universal Music Group, which controls about a third of the global music market, recently pushed streaming services to take down AI-generated songs and block AI developers from scraping musical data or posting recordings that sound like established artists.
Why it matters: Every new generation of technology brings new tools to challenge the record industry’s control over music distribution. The 1970s brought audio cassettes and the ability to cheaply copy music, the 1980s brought sampling, the 1990s and 2000s brought remixes and mashups. Today AI is posing new challenges. Not everyone in the music industry is against these AI copycats: The electronic artist Grimes said she would share royalties with anyone who emulates her voice, and Oasis’ former lead singer apparently enjoyed the AI-powered imitation.
We’re thinking: Musicians who embrace AI will open new creative pathways, but we have faith that traditional musicianship will endure. After all, photography didn’t kill painting. Just as photography pushed painters toward abstraction, AI may spur conventional musicians in exciting, new directions.