AI Generation Comes To Music, Letting You Compose New Tunes From Text

By now, you might have come across artwork churned out by DALL-E or Stable Diffusion, two of the most currently well-known AI art generators.

 
Still, art isn’t just paintings and drawings; it’s also music. And if you’re more musically-inclined, or don’t have a musical bone in your body (it goes both ways), Riffusion could be the tool to turn to.

The musical answer to art generators like DALL-E was set up by two bandmates, Seth Forsgren and Hayk Matriros, who sought an algorithm to create tunes instead of artistic pieces.

The model churns out spectrograms, visual representations of melodies, which can be turned into audio clips. Instead of teaching the model with images of objects and places in the real world, Forsgren and Matriros trained the open-source Stable Diffusion on spectrograms paired with text prompts.
 
On the Riffusion website, there is a list of prompts you can cycle through, or you could feed it your imaginative idea if you want.
 
Below is the spectrogram of “bubblegum eurodance,” which sounds like a distorted Madonna singing to ethereal techno beats. Click here to have a listen for yourself.

According to PC Mag, Forsgren notes that the language used to “sing” in audio bites is “a bit otherworldly,” as the AI has not learned to string together proper lyrics in any language. Another thing to note is that most clips are under a minute long, and the generator cannot give you a full studio-length song if that’s what you were after.
 
Ultimately, the generator is a bit of fun for all the musicians out there looking to experiment with new sounds or for whoever is looking for inspiration to go out and create music of their own. As with its image-creating counterparts, though, the tool may be tone-deaf when it comes to protecting artists’ rights.
 
 
 
[via PC Mag and TechCrunch, screenshots via Riffusion]