Pushing boundaries and amplifying human creativity by using AI in music

The mechanization of music is not a recent phenomenon.

While few artists defend synthetic variations of generative AI like Suno or Udio, some have chosen to experiment with artificial intelligence to enhance their creativity and push the boundaries of their art.

The audacity of the French pianist and composer, Édouard Ferlet , is a perfect example of what AI can bring to artists and their creative processes.

Köln Variations, Edouard Ferlet

Last April, Édouard Ferlet gave a unique concert at the Printemps de Bourges festival : Köln Variations . On this occasion, he tackled a monument of jazz: The Köln Concert, the best-selling piano album in the world, born from a live improvisation by Keith Jarrett in Cologne in January 1975.

The world's best-selling piano album is an improvised album: in the age of artificial intelligence, this gives hope for humanity!

By daring to collaborate in a duo with an AI on the repertoire of a great piano master, Édouard Ferlet reinvents improvisation in concert, and also plunges the spectator into a fascinating abyss: the free improvisation of a concert itself improvised, in the form of a dialogue with an AI that seems to improvise but produces outputs calculated on the last hundred notes of the pianist! Édouard Ferlet came to Plaine Images to decipher this avant-garde work.

With a background in classical and jazz music, as well as computer music, Édouard Ferlet has been working on robotic devices for about ten years. Even before integrating AI into his work, he developed Pianoïd, a device created to combine his love of the acoustic piano with his interest in robotics and computer science. This is a good opportunity to remember that player pianos reached their peak at the beginning of the 20th century!

Pianoïd device, Edouart Ferlet (YouTube screenshot)

Pianoïd system , conceived over ten years by Édouard Ferlet, consists of two acoustic pianos: a grand piano played by the artist, and a motorized upright piano (Disklavier). The latter is operated using the MIDI protocol, a standard music file format that transcribes crucial information such as the note name, velocity, length, timing, etc.

Initially, Pianoïd operated using an algorithmic system and employed effects known as arpeggiators. According to Édouard Ferlet, making an acoustic piano play arpeggios already possessed a certain magic, as the mechanics of the motorized piano produced interesting organic variations while offering a captivating spectacle. The possibilities brought by artificial intelligence have unleashed new forms of creation… and intensified the dialogue with the piano!

A new era of prosperity for assisted creation?

Édouard Ferlet has therefore integrated artificial intelligence into his system, to propose the Köln Variations at Printemps de Bourges 2025. To do this, he has surrounded himself with scientific partners such as IRCAM and Sony CSL , as well as the musician Benoît Carré .

His approach focuses particularly on processing the MIDI system rather than audio. Tech giants' R&D is currently more focused on audio processing, as these are the profitable applications: mastering, dubbing or replacing voices, etc. But it's from the MIDI system that you can, for example, generate a musical score, and thus activate a motorized piano.

To understand how AI works, Édouard compares it to a fill-in-the-blank text system. AI operates on a predictive model, which guesses the missing notes from very specific data sets that had to be compiled:

• A MIDI dataset containing a large number of royalty-free classical compositions
• A dataset of Keith Jarrett tracks converted from audio to MIDI, for fine-tuning
• This increased data allows the AI ​​to understand that a Keith Jarrett piece in C major at a slow tempo can also exist in E-flat at a fast tempo, for example. These datasets contain a multitude of variations that enable the AI ​​to understand all the different ways of playing a piece, specific to jazz.
• A final dataset including pieces recorded by Edouard Ferlet, so that the AI ​​can model his playing style.

The rigorous and patient training of the AI ​​enabled Édouard Ferlet to improvise in dialogue with the piano variations on the work of Keith Jarrett.

In practice? The AI ​​is programmed to predict the continuation of the human game by listening to the last hundred notes played by Édouard. It starts playing when he stops, inspires him with a response, and so on.

Human-Computer Interaction and Inverted Virtuosity

The human-computer interaction projects that Édouard Ferlet has been pursuing for about ten years raise fascinating questions related to robotic gestures:

I've played piano almost daily for 40 years, constantly grappling with the challenge of virtuosity. A composer's job also involves knowing how to write things that are playable! With a computer, I can now compose unplayable pieces. Now I'm wondering how I can compose for the piano, things I won't be able to play myself.

This is what Édouard Ferlet calls reverse virtuosity, where the machine can perform "unimaginable, impossible" movements, supplanting human capabilities.

The AI ​​project of Köln Variations raises other fundamental questions about the artistic gesture: the nature of the robot is artificial. Devoid of heart, emotion, or soul, the AI ​​plays in a way that is the antithesis of that of the pianist. In his musical dialogue with the piano, Édouard Ferlet therefore experimented with playing “like the robot,” in an insensitive and detached manner. This exercise allowed him to touch the very essence of musical notation and the MIDI note, and enriched his practice.

Finally, according to Édouard Ferlet, the “AI Variations” project is still in its early stages. For the moment, the AI's proposals are far from conclusive. The Köln Variations contain only a few minutes of actual AI and are primarily an artistic demonstration of the possibilities of this interactive collaboration.

For Édouard, AI is a “distorting mirror”, whose main interest lies in the reflection it provokes: “it allows the artist to move within his boundaries”.

The revolutionary educational potential of AI

Beyond the artistic aspect, AI experiments on the Pianoïd system open up an unexpected range of possibilities in the field of pedagogy: AI is a formidable tool for synthesis, which cleanly works with raw material.

In the history of art, learning begins with imitation. Édouard envisions a future “Piano ChatGPT,” capable of responding to specific requests, such as playing a piece in the style of Bach or practicing a particular scale. This artificial intelligence would become a neutral and impartial tool, adapting seamlessly to the student's progress and musical preferences. These are topics we've already addressed with EuraCreative through incubated projects focused on AI-powered music learning !

AI could also be useful to composers, automating repetitive tasks such as creating scores. The key to precisely meeting their needs is the ability to analyze the MIDI signal of an audio file. Even more impressive, one could imagine an AI that analyzes a video of a pianist's hands to identify the notes played by the left and right hands, thus simplifying notation. A vast field of possibilities opens up for composers!

The normalization of relations between generative AI companies and major corporations

And what about the music industry? Things have changed drastically in the last few weeks. A year and a half ago, Universal Music Group was among the major labels suing the generative music startup Udio . In a turn of events (which some saw coming), the two parties have finally reached an agreement! A strange ending indeed? This lawsuit now looks very much like a strategic pressure tactic by UMG (Udio made no secret of the fact that it was training its generative model on a corpus of copyrighted tracks), and has turned into a commercial opportunity. Aware of the irresistible rise of AI in content creation, the major label is trying to reframe its use within a legal and profitable framework.

This commercial opportunity remains to be seen. An uncomfortable question arises: who exactly demanded AI-generated music? The artists? The fans? The record labels? It seems this value proposition materialized spontaneously, without any addressable consumer market. While the Udio and Suno platforms have entertained millions of internet users, they haven't yet created any significant economic value. The principle behind these musical AIs is to create tracks based on the most salient patterns in billions of training data points—that is, existing tracks created and produced by humans. The generated tracks, however melodious and "plausible" they may be, are devoid of artistic substance.

UMG is therefore making a bet on the future of music consumption, but this strategic shift is interesting for more than one reason:

• The major label has set a precedent that could be a landmark in the industry's history, proving the necessity of licensing for generative AI. By accepting the deal, Udio refutes the fair use , which they themselves had raised during the litigation. Now, the startup is positioning itself as a responsibly trained AI platform, and UMG can use this shift as validation of the need to seek permission from rights holders.
• It also intends to control creation , since the tracks created will remain on the platform, and will be limited to artists who have given their consent (the famous opt-in ).
• With this consent, UMG also wants to guarantee the remuneration of artists and songwriters: they will be paid for training data and for AI-generated results.

And the competition is following suit! Major label Warner and the music-generating AI startup Suno have also just signed a "legal settlement." This puts an end to the lawsuit filed by Warner for copyright infringement. Similar to the agreement between UMG and Udio, the settlement allows Warner artists who wish to participate to offer their voices and compositions as material for Suno's AI services. The startup, however, remains in a legal dispute with Sony Music and has not yet signed an agreement with UMG.

There are still major disagreements surrounding intellectual property

Bucking the trend of normalizing relations between major corporations and AI startups, the lawsuit filed against OpenAI (ChatGPT) by the German copyright management organization, GEMA , suggests a more nuanced approach. The lawsuit resulted in a court ruling that will likely set a precedent, as the Munich court determined that the American giant had infringed upon "exploitation rights protected by copyright." OpenAI disagreed with the decision and is considering "its next steps."

Currently, the law allows generative AI service providers to use copyrighted works for their data mining operations, provided they have accessed them legally and the author has not exercised their right to object ( opt-out ). The French copyright management society, SACEM , therefore quickly opted for a opt-out for its members, legally protecting them against the use of their creations. Any data mining of members' works will require their prior authorization.

The paradox of these measures is that management bodies cannot, as things stand, prove that AI services have not complied with this clause, as the datasets remain confidential.

And what is your perspective on this major change?