Interactive scores

Multimedia Scenarios in interactive scores are represented by temporal objects, temporal relations, and interactive objects. Examples of temporal objects are sounds, videos and light controls. Temporal objects can be triggered by interactive objects (usually launched by the user) and several temporal objects can be executed simultaneously. A temporal object may contain other temporal objects: this hierarchy allows us to control the start or end of a temporal object by controlling the start or end of its parent. Hierarchy is ever-present in all kinds of music: Music pieces are often hierarchized by movements, parts, motives, measures, among other segmentations.^[8][9]

Computer Accompaniment (Carnegie Mellon University)

The Computer Music Project at CMU develops computer music and interactive performance technology to enhance human musical experience and creativity. This interdisciplinary effort draws on music theory, cognitive science, artificial intelligence and machine learning, human computer interaction, real-time systems, computer graphics and animation, multimedia, programming languages, and signal processing.^[10]

ChucK

Developed at Princeton University by Ge Wang and Perry Cook, ChucK is a text-based, cross-platform language that allows real-time synthesis, composition, performance, and analysis of music.^[11] It is used by SLOrk (Stanford Laptop Orchestra)^[12] and PLOrk (Princeton Laptop Orchestra).

Jukedeck

Jukedeck was a website that let people use artificial intelligence to generate original, royalty-free music for use in videos.^[13][14] The team started building the music generation technology in 2010,^[15] formed a company around it in 2012,^[16] and launched the website publicly in 2015.^[14] The technology used was originally a rule-based algorithmic composition system,^[17] which was later replaced with artificial neural networks.^[13] The website was used to create over 1 million pieces of music, and brands that used it included Coca-Cola, Google, UKTV, and the Natural History Museum, London.^[18] In 2019, the company was acquired by ByteDance.^[19][20][21]

MorpheuS

MorpheuS^[22] is a research project by Dorien Herremans and Elaine Chew at Queen Mary University of London, funded by a Marie Skłodowská-Curie EU project. The system uses an optimization approach based on a variable neighborhood search algorithm to morph existing template pieces into novel pieces with a set level of tonal tension that changes dynamically throughout the piece. This optimization approach allows for the integration of a pattern detection technique in order to enforce long term structure and recurring themes in the generated music. Pieces composed by MorpheuS have been performed at concerts in both Stanford and London.

AIVA

Created in February 2016, in Luxembourg, AIVA is a program that produces soundtracks for any type of media. The algorithms behind AIVA are based on deep learning architectures^[23] AIVA has also been used to compose a Rock track called On the Edge,^[24] as well as a pop tune Love Sick^[25] in collaboration with singer Taryn Southern,^[26] for the creation of her 2018 album "I am AI".

Google Magenta

Google's Magenta team has published several AI music applications and technical papers since their launch in 2016.^[27] In 2017 they released the NSynth algorithm and dataset,^[28] and an open source hardware musical instrument, designed to facilitate musicians in using the algorithm.^[29] The instrument was used by notable artists such as Grimes and YACHT in their albums.^[30][31] In 2018, they released a piano improvisation app called Piano Genie. This was later followed by Magenta Studio, a suite of 5 MIDI plugins that allow music producers to elaborate on existing music in their DAW.^[32] In 2023, their machine learning team published a technical paper on Github that described MusicLM, a private text-to-music generator developed.^[33][34]

Riffusion

This section is an excerpt from Riffusion.[edit]

Generated spectrogram from the prompt "bossa nova with electric guitar" (top), and the resulting audio after conversion (bottom)

Riffusion is a neural network, designed by Seth Forsgren and Hayk Martiros, that generates music using images of sound rather than audio.^[35] It was created as a fine-tuning of Stable Diffusion, an existing open-source model for generating images from text prompts, on spectrograms.^[35] This results in a model which uses text prompts to generate image files, which can be put through an inverse Fourier transform and converted into audio files.^[36] While these files are only several seconds long, the model can also use latent space between outputs to interpolate different files together.^[35][37] This is accomplished using a functionality of the Stable Diffusion model known as img2img.^[38]

The resulting music has been described as "de otro mundo" (otherworldly),^[39] although unlikely to replace man-made music.^[39] The model was made available on December 15, 2022, with the code also freely available on GitHub.^[36] It is one of many models derived from Stable Diffusion.^[38]

Riffusion is classified within a subset of AI text-to-music generators. In December 2022, Mubert^[40] similarly used Stable Diffusion to turn descriptive text into music loops. In January 2023, Google published a paper on their own text-to-music generator called MusicLM.^[41][42]