Music Writing with Artificial Intelligence

 To write music with AI, the process is the same as with other image generation models, because a piece of music, is just a spectrogram. Yes, that music player visualization that everyone had in the 90's.

The amazing team at Riffusion, have created an awesomely good music generation software that uses spectrogram analysis to generate new pieces.

As the Riffusion Githob page says:


Riffusion v1 Model

Riffusion is a latent text-to-image diffusion model capable of generating spectrogram images given any text input. These spectrograms can be converted into audio clips.

model created by Seth Forsgren and Hayk Martiros

Riffusion web app. 🤗's Stable Diffusion blog.


By generating new images of sines and waves based on a catalog of training examples, the model achieves frequen

@software{Forsgren_Martiros_2022, author = {Forsgren, Seth* and Martiros, Hayk*}, title = {{Riffusion - Stable diffusion for real-time music generation}}, url = {}, year = {2022} }

Popular Posts

Odin Ideas

Data Science

Service Design