Pushing boundaries and unlocking human creativity through the use of AI in music

The mechanisation of music is nothing new

Whilst few artists are keen to champion the synthetic variations produced by generative AIs such as Suno or Udio, some have chosen to experiment with artificial intelligence to enhance their creativity and push the boundaries of their art.

The bold approach of French pianist and composer Édouard Ferlet is a perfect example of the best that AI can offer artists and their creative processes.

Last April, Édouard Ferlet gave a unique concert at the Printemps de Bourges: Köln Variations. On this occasion, he tackled a jazz classic: The Köln Concert, the world’s best-selling piano album, born from a live improvisation by Keith Jarrett in Cologne in January 1975.

The world’s best-selling piano album is an improvised album: in the age of artificial intelligence, this gives hope for humanity!

By daring to collaborate as a duo with an AI on the repertoire of a great master of the piano, Édouard Ferlet reinvents concert improvisation, whilst also plunging the audience into a fascinating abyss: the free improvisation of a concert itself improvised, in the form of a dialogue with an AI that appears to improvise but produces outputs calculated based on the pianist’s last hundred notes! Édouard Ferlet came to La Plaine Images to explain this avant-garde work.

With a background in classical and jazz music as well as a background in computer music, Édouard Ferlet has been working on robotic devices for around ten years: even before integrating AI into his work, he developed Pianoïd, a device created to reconcile his love of the acoustic piano with his interest in robotics and computer science. This is a reminder that mechanical pianos reached their peak in the early 20th century!

The Pianoïd device, conceived over a period of ten years by Édouard Ferlet, consists of two acoustic pianos: a grand piano played by the artist, and a motorised upright piano (Disklavier). The latter is operated via the MIDI protocol, a standard music file format that transmits crucial information such as the note name, velocity, duration, timing, etc.

Initially, Pianoïd operated on the basis of an algorithmic system and utilised familiar effects such as arpeggiators. According to Édouard Ferlet, having an acoustic piano play arpeggios already possessed a certain magic, as the mechanics of the motorised piano produced interesting organic variations whilst offering a spellbinding spectacle. The possibilities brought about by artificial intelligence have opened up new forms of creativity… and intensified the dialogue with the piano!

A new era of prosperity for assisted creation?

Édouard Ferlet has therefore integrated artificial intelligence into his system to present the Köln Variations at the Printemps de Bourges 2025. To achieve this, he has teamed up with scientific partners IRCAM and Sony CSL, as well as musician Benoît Carré.

His approach focuses specifically on processing the MIDI system rather than audio. R&D by tech giants currently focuses more on audio processing, as these are the processes that yield tangible benefits: mastering, dubbing or voice replacement, etc. But it is from the MIDI system that one can, for example, generate a score, and thus activate a motorised piano.

To understand how AI works, Édouard compares it to a fill-in-the-blank exercise. The AI operates on a predictive model, which guesses the missing notes based on a very specific dataset that had to be compiled:

• A MIDI dataset comprising a large number of copyright-free classical compositions
• A dataset of Keith Jarrett’s pieces converted from audio to MIDI, for fine-tuning
• An expansion of this data so that the AI understands that a Keith Jarrett piece in D major at a slow tempo can also exist in E flat at a fast tempo, for example. These datasets contain a variety of variations that allow the AI to understand all the different ways of playing a piece, specific to jazz.
• A final dataset comprising pieces recorded by Édouard Ferlet, so that the AI can model his playing style.

The rigorous and patient training of the AI has enabled Édouard Ferlet to improvise variations on Keith Jarrett’s work in dialogue with the piano.

In practice? The AI is programmed to predict the next move in the human performance by listening to the last hundred notes played by Edouard. It starts playing when he stops, inspires a response from him, and so on.

Human-Machine Interaction and Reverse Virtuosity

The human-machine interaction projects that Édouard Ferlet has been pursuing for the past ten years raise fascinating questions relating to robotic movement:

I have been playing the piano almost daily for 40 years, constantly grappling with the problem of virtuosity. Part of a composer’s job is also knowing how to write music that is playable! With the machine, I can now compose things that are unplayable. I now find myself wondering how I will be able to compose for the piano pieces that I myself cannot play.

This is what Édouard Ferlet calls ‘reverse virtuosity’, where the machine can perform ‘unimaginable, unplayable’ movements, surpassing human capabilities.

The AI project behind the Köln Variations raises other fundamental questions about the artistic act: the robot’s nature is artificial. Devoid of heart, emotion or feeling, the AI’s playing style is the polar opposite of that of a pianist. In his musical dialogue with the piano, Édouard Ferlet therefore experimented with playing “like the robot”, in an unfeeling and cold manner. This exercise allowed him to touch upon the very essence of musical composition and the MIDI note, and has enriched his practice.

Finally, as Édouard Ferlet admits, the “AI Variations” project is still in its early stages. For the time being, the AI’s suggestions are far from conclusive. The Köln Variations contain only a few minutes of actual AI output and are primarily an artistic demonstration of the possibilities offered by this interactive collaboration.

For Édouard, AI is a “distorting mirror”, whose main interest lies in the reflection it provokes: “it allows the artist to push the boundaries”.

The revolutionary educational potential of AI

Beyond the artistic aspect, AI experiments on the Pianoïd system open up unexpected possibilities in the field of education: AI is a formidable tool for synthesis, working effectively with raw material.

In the history of art, learning begins with imitation. Édouard envisions a future “ChatGPT for the piano”, which could respond to specific requests, such as playing a piece in the style of Bach, or practising a particular scale. This artificial intelligence would become a neutral and impartial tool, finely adapting to the student’s progress or musical preferences. These are topics we have already addressed at EuraCreative through projects incubated on the theme of AI-enhanced music learning!

AI could also be of use to composers, by automating repetitive tasks such as the creation of sheet music. The key to meeting these needs precisely lies in the ability to analyse the MIDI signal from an audio file. Going even further, one could imagine an AI that analyses a video of the pianist’s hands to identify the notes played by the left and right hands, thereby simplifying the notation. A vast field of possibilities is opening up for composers!

The normalisation of relations between generative AI companies and the major labels

And what about the music industry? Things have changed drastically in recent weeks. A year and a half ago, Universal Music Group was one of the major labels taking legal action against the generative music start-up Udio. In a turnaround (which some saw coming), the two warring parties have finally signed an agreement! A strange epilogue? This legal action now looks very much like a strategic pressure tactic for UMG (Udio made no secret of the fact that it was training its generative model on a corpus of copyright-protected tracks), and is turning into a commercial windfall. Aware of the unstoppable rise of AI in content creation, the major label is attempting to bring its use within a legal and profitable framework.

This commercial windfall remains to be confirmed, however. An uncomfortable question arises in this regard: who, exactly, has been asking for AI-generated music? The artists? The fans? The labels? It seems that this value proposition has materialised spontaneously, without any identifiable consumer market. Whilst the Udio and Suno platforms have entertained millions of internet users, they have not yet created any economic value of note. The principle behind these music AIs is to create tracks based on the most prominent patterns in billions of training data points, i.e. existing tracks created and produced by humans. The generated tracks, however melodious and “plausible” they may be, are devoid of artistic substance.

UMG is therefore placing a bet on the future of music consumption, but this strategic shift is interesting for more than one reason:

• The major label is setting a precedent that could prove landmark in the industry’s history, by demonstrating the need for licences for generative AI. By accepting the deal, Udio refutes the fair use argument, which they themselves had brandished during the dispute. Going forward, the start-up is positioning itself as a responsibly trained AI platform, and UMG can use this shift as validation of the need to seek permission from rights holders.
• It also intends to control the creative process, as the tracks created will remain on the platform and will be limited to artists who have given their consent (the so-called ‘opt-in’).
• Through this consent, UMG also aims to guarantee remuneration for artists and songwriters: they will be paid for the training data and for the AI-generated results.

And the competition is following suit! Major label Warner and the generative music AI start-up Suno have also just signed a “legal settlement”. This brings an end to the lawsuit brought by Warner for copyright infringement. As with the agreement between UMG and Udio, the settlement offers willing Warner artists the opportunity to provide their voices and compositions as material for Suno’s AI services. The start-up remains in legal dispute with Sony Music, however, and has still not signed an agreement with UMG.

More friction over intellectual property

Contrary to the trend towards normalised relations between major tech firms and AI start-ups, the complaint against OpenAI (ChatGPT) by the German copyright management organisation, GEMA, tends to cast a shadow over the truce… The complaint resulted in a court ruling that is likely to set a legal precedent, as the Munich court found that the US giant had infringed “copyright-protected exploitation rights”. OpenAI has stated its disagreement with this ruling and is considering “possible next steps”.

Currently, the law allows providers of generative AI services to use protected works for their data mining operations, provided they have accessed them legally and the author has not exercised their right to object (opt-out). The French copyright management society, SACEM, therefore quickly opted for a default opt-out for its members, legally protecting them against the use of their creations. Any data mining of members’ works will require their prior authorisation.

The paradox of these measures is that, as things stand, the collecting societies cannot prove that AI services have not breached this clause, as the datasets remain confidential.

And you, what is your view on this major shift?