Bored with your Spotify playlists? Let this robot singer/songwriter take you on a trip “Into Your Mind.”
What’s Goin’ On: A music-composing, marimba-playing robot named Shimon has learned to write and sing its own lyrics, IEEE Spectrum reports. Shimon performs its debut single with a flesh-and-blood backup band in this video. An album is scheduled to drop on Spotify on April 10.
(A)I Write the Songs: Two of the robot’s creators, Georgia Tech professor Gil Weinberg and grad student Richard Savery, treat the machine as though it were a human collaborator.
- Shimon’s language model was trained on 50,000 lyrics from jazz, rock, and hip hop songs. Given a keyword, it generates thousands of phrases that are related to the word itself, its synonyms, or its antonyms. It picks a song’s worth of phrases, emphasizing rhymes and thematic similarity.
- After it has completed the lyrics, the humans weave melodies it generates with their own ideas to produce music that suits the words.
- Shimon’s voice is based on a model developed by the University of Pomeu Fabra and trained on human voices in a wide variety of pitches.
- Weinberg aims to combine Shimon’s musical improvisations with its lyrical skills to transform the machine into a freestyle rapper.
Robot Rock: Shimon’s music-video debut showcases several other innovations created by Weinberg and his colleagues, who have been working on this project for nearly a decade.
- The robot’s vaguely serpentine head automatically moves to the rhythm while its mouth and eyebrows emote along with the lyrics.
- The system originally was designed to improvise on marimba in response to notes played by human musicians. New hardware enables its arms to play more precisely in time and with greater control over loud and soft accents.
- Shimon’s human bandmate Jason Barnes lost one hand in a 2012 accident. He performs on drums using a robotic prosthesis that holds two sticks. Electrical activity from Barnes’ muscle controls one stick, while the second stick moves autonomously in response to the music.
Unchained Melody: Machine learning is challenging timeless assumptions about human creativity. Shimon’s output is reminiscent of progressive rock masters. Could it conquer the pop charts by working with the producers of Taylor Swift, Beyonce, or Drake?
Wish You Were Here: The Covid-19 pandemic forced Weinberg to postpone plans to take Shimon on tour. Will his next project be building robot fans?