Robot rhythms: the startups using AI to shake up the music business

Mention the words “AI” and “music” in the same sentence and you may attract some tuts and eye rolls, as people wax lyrical about the good old days. If robots are making our call centre jobs redundant, then at least let us have our Sunday jam sessions.

But music, like anything else, has to move on: “In a world of personalisation and on-demand services, music is one of very few remaining static artefacts,” says Ken Lythgoe, head of business development at creative AI technology company MXX. The company has created what it says is the world’s first AI tech that allows individual users to instantly edit music to fit their own video footage, complete with rises and fades.

AI doesn’t need to be the enemy of music, says Lythgoe: “There are two types of AI – the AI that is here to replace us and AI that is here to empower us – we are definitely in the empowerment camp. We are not about computers replacing musicians or editors; we are firm believers in the creative process.”

“Businesses and individuals want to alter their experience with music, depending on their needs,” says Lythgoe. “They want a world where music can be adapted to perfectly fit certain experiences – such as gym workouts and runs, gaming, user-generated content and virtual or augmented-reality experiences.”

MXX was founded in 2015 by AI specialist and composer Joe Lyske and Philip Walsh, former financial services CEO and founder of investment vehicle and AI incubator, Time Machine Capital. Lyske’s research included devising a tool that generated film scores that fooled 50% of professional film composers into thinking they were listening to a fellow professional.

MXX’s tech listens to music and creates metadata of its understanding of it. This data includes where it can edit in and out of sections, as well as what the sections might mean to a human, such as “building tension”, “climax”, “chorus” and “verse”. When the user provides the brief for the music they want, the AI can edit the original track to fit the brief.

Lythgoe explains: “Just as a silent film pianist would adapt the music to the mood of the audience or speed of the projector, we can have our favourite music adapt to our car journey time, the pace of our run or gym workout or the way the cat falls off the sofa in our social media post.”

The technology’s target users are advertising agencies, game creators, production companies, music supervisors, labels and sync agencies, a market that, says Lythgoe, is crying out for tech like this: “There are 400,000 hours of video uploaded to YouTube every 60 seconds alone. There aren’t enough music editing professionals in the world to cope with that kind of content.”

AI-generated music does not take jobs, says Lythgoe: “In fact editors say the opposite. They often have to edit dozens of versions of different music to adverts for free, only being able to charge for the final version. This tech allows them to set up a template to sell to their customers, that the customer can play with all day till they have found the track they are looking for.”

Valerio Velardo is co-founder and CEO of Berlin-based Melodrive, which is using AI to automatically generate soundtracks for virtual reality and video games in real time. A classically trained musician, he has a PhD in music and AI, and a degree in astrophysics. Melodrive (which has five staff) essentially composes an infinite stream of original, “emotionally variable” music in real time. Velardo explains: “In a video game, if a player encounters more than 10 enemies, the music can move from being ‘almost angry’ to ‘completely crazy’.”

Far from putting musicians and composers out of business, Velardo says tech like Melodrive is “more of a sparring partner. We are just building tools to be more creative with and this is no different from the evolution of the piano from the harpsichord, to things like [digital audio workstation] Cubase.”

The UK’s AI sector could be worth £232bn to the UK economy by 2030, according to a report (pdf) from PwC, and back in 2018 the government announced a £1bn deal to put the UK at the forefront of the AI industry. UK Music’s Measuring Music 2018 report puts the UK music industry’s gross value added (GVA) at £4.5bn.

“AI and music is certainly a growth area and likely to continue to be so,” says Craig Hamilton, research fellow at the Birmingham Centre for Media and Cultural Research: “A number of startups have attracted investment over the last couple of years – such as Amper and AI Music – and there is also a lot of activity within many of the larger tech and entertainment firms, notably Sony, Google, Spotify, IBM and Facebook. This should not be surprising – AI systems rely on large amounts of data to build and train models and on powerful computational, connected systems to process data and deploy the results.”

Since the turn of the century, music consumption has increasingly shifted to systems capable of generating and analysing data on both user activity and musical works, says Hamilton, “and there has also been an exponential growth in the scale and power of computational processing systems.” In that sense, he adds: “AI and music together does have a certain air of inevitability.”