Yahoo News Imogen Heap on AI, making her own voice model, and a new era of musical collaboration
When AI voice models began making headlines around the world, Imogen Heap started getting calls. The award-winning singer-songwriter, producer, polymath, and entrepreneur is renowned for using cutting-edge tech in her music, and, as AI hype hit a fever pitch, Heap kept fielding the same question: ‘can we make a model of your voice?’
“They knew that I'm interested in technology, and they knew my answer probably wouldn’t be a flat no,” recalls Heap. Initially excited by the creative possibilities, Heap says those offers were quickly followed by a host of caveats and clauses, terms and conditions. “I began to feel like I had no control over the whole thing. Everybody kept saying how hard it is to create an AI voice model… but I just thought, it can’t be that hard.”
Rebuffing offers from AI startups, Heap instead worked with an audio engineer to cook up her very own vocal model. Together, the pair found an open-source model and began feeding it recordings from Heap’s decades-long career. “You know what? It came out pretty good,” Heap says with audible pride. “After that I was feeling more empowered, like I had a leg to stand on.”
The result is ai.mogen; a bespoke vocal model that can turn any singer, or instrument, into a passable simulacrum of Imogen Heap. However, ai.mogen – or Mogen as Heap refers to it – is far more than an idle experiment or a tech demo, it’s set to be a pillar of the artist’s return to music-making after a decade out of the limelight.
Heap’s first release using the model is a remix of Slovakian pop star Karin Ann’s single, false gold, produced in collaboration with creative agency Juxtapose Studio. It’s a dreamy and richly layered production, but on first listen you might struggle to hear Heap’s distinctive vocals in the mix. That is, until you realise that her voice is everywhere; woven into the very fabric of the track.
"It was the weirdest thing, but it sounded amazing. It was my voice trying to sing the kick and snare, the bassline, the keys"
Working with over twenty instrumental and vocal stems, Heap says she fed the whole thing through the ai.mogen vocal model. “It was the weirdest thing, but it sounded amazing,” she enthuses. “It was my voice trying to sing the kick and snare, the bassline, the keys. My voice became a kind of aura surrounding everything and it really decided the direction I wanted to take the remix.”
That kind of experimentation is only the smallest taste of what we can expect from Heap in the near future. She teases a forthcoming single, titled What Have You Done To Me?, that clocks in at 14 minutes long and is chock full of what she describes as “absolutely mental stuff”.
“It has a lot of the technology that I've been quietly working on,” Heap confides. “I've been developing tech for 15 years now and it feels as much a part of me as the music does – but, with this song, I hope that it will all manifest together in a much more cohesive way.”
READ MORE
10 ways to use generative AI in your music production
Few artists are so closely associated with music technology as Imogen Heap. From the iconic vocoder harmonies on her breakout single, Hide And Seek, to her much-discussed Mi.Mu gloves which let performers control music through physical gesture, to more recent experiments with web3 monetisation and community engagement – time and again, she’s been ahead of the technological curve.
Similarly, Heap’s interest in AI predates the current craze. While ai.mogen now boasts a vocal component, the project in fact got its start over five years ago as a fan-powered chatbot.
Heap’s fans have long enjoyed an unusually close connection with the singer, welcome to reach out on social media, or more recently through her subscription-based community, to discuss anything and everything. While this tight-knit network is clearly something Heap treasures, the need to repeatedly answer simple questions left her with no energy for more meaningful discussions.
“Basically, I don't want to repeat myself,” she says simply. “I’ve only got so many seconds on this earth. With this chatbot, you type questions, and, if it can't answer, then it gives you a direct ticket to chat with me when I'm next online.”
INTERVIEW
While ai.mogen might have begun as a functional tool, it’s clear that Heap hopes it will grow into something much more profound. “I've had hundreds and hundreds of conversations with fans about all kinds of amazing and deep things, including my sister's death, Covid, war, and so many other things. I love the idea that we are developing an AI that understands empathy, kindness, and family, that understands how we think about death and dating, and God knows what else.”
Asked whether she’ll ever use ai.mogen’s text generation capabilities to write song lyrics, her answer is characteristically open-minded. “I mean… yeah? Perhaps I wouldn’t use her because there are other services out there that do a much better job right now. If someone wanted to generate something in the style of my lyrics that would be fine, though I would like to be credited at some point."
Heap also has plans to expand ai.mogen’s capabilities to eventually serve as a songwriting and production assistant. “Every single scrap of unused or used audio that I ever create goes into a folder,” she says. “We’re preparing to semantically describe all of it so that, in the future, I can come into my studio and Mogen will say ‘may I suggest this thing that you created in 1998 and as good a place to start?’”
On the other hand, AI song generators like Suno and Udio, which can spit out hundreds of full-length songs per minute, are quickly dismissed as “boring”. “I'm not worried about them,” she says with a verbal shrug. “I live and breathe music, and I know that the human story is what people want. I'm not excited about AI making stuff that sounds like something a human could do - I want to hear things that humans have never imagined.”
While Heap is unreservedly enthusiastic about the creative potential that AI offers, she is also quick to acknowledge the many troubling issues that society needs to work through to harness the technology – most pertinent among them is AI’s profound impact on the music industry.
“It does worry me,” she responds when asked about the proliferation of unauthorised voice models. “I will eventually release ai.mogen so that everyone can use it, but I don't want my voice to say hateful things. So, I need to find a way to do it on acceptable terms.”
READ MORE
Rogue voice models are troubling, but Heap’s biggest concerns, and sharpest critiques, are reserved for the large industry players who are quietly negotiating with a new generation of AI startups. “AI companies are going straight to the music rights holders,” she points out. “They know they won’t get sued by individual artists because most musicians are penniless anyway – but big labels have big lawyers. So, we’re seeing labels doing deals en masse with commercial AI voice shops, and the artists are either unaware or don't feel empowered to say no. It’s an exact replica of what happened with DSPs.”
Heap says governments need to step in and fill the current policy void, but she’s also not waiting around for this to happen. Instead, she’s forging ahead with a solution of her own. “I’m in the middle of creating an app that enables musicians to train their own vocal models with privacy and security,” says Heap. “The hope is that we can educate and protect people and help them feel a bit more in control of their voice.”
Alongside the ability to train their own voice models, Heap is also aiming to implement what she describes as a ‘lineage’ system to help keep track of how vocal models are used.
"I’m in the middle of creating an app that enables musicians to train their own vocal models with privacy and security"
“Models that generate pictures, videos, lyrics – you should be able to look under the hood and see the pieces that made it possible. We’re working with a company as we train Mogen, so that, if someone uses it, then you will see a collaboration between myself and that other vocalist. If someone trains their own voice and then wants to make a new model that blends Mogen and their own model, then you would see that like a family tree of voices.”
Collaboration is a subject Heap returns to time and again as she discusses the benefits of AI voice models. Her voice and music have been sampled, using traditional means, by countless artists over the decades and, in recent years, her work has gained renewed interest from hip-hop luminaries like A$AP Rocky and Clams Casino.
“There are probably lots of people who don’t ask because they think I’ll say no, but I almost always say yes,” she says about other artists sampling her work. “Over the years there have been so many people that I've wanted to experiment and collaborate with, but I just never had the time to do it all – now it feels like a whole piece of me can be there to collaborate in a virtual, remote kind of way.”
"This new wave of AI is helping musicians realise that they actually do need a way to link their identity to their voice"
The key, Heap says, is to ensure that artists are properly acknowledged for their contributions. In 2019 she founded Creative Passport, a platform to help artists verify and manage their online identity, network with other creators, interface with the wider music industry, and licence their music without the backing of a record label. While the endeavour got off to a slow start, Heap says the arrival of AI makes the project more important than ever.
“It's necessary for so many reasons,” she emphasises. “This new wave of AI is helping musicians realise that they actually do need a way to link their identity to their voice, and to have their own body that can interface with the industry. I feel this real sense of urgency because I know that deals are happening right now – but, if we can add this layer of permission, attribution, IP, and ID then it will be the start of many beautiful things.”
There is an infectious optimism to Heap's worldview – whether she’s speaking about the music industry or the species writ large. The teething troubles of AI’s meteoric rise are readily acknowledged but always paired with possible solutions, creative potentials, and long-term benefits.
"AI is like a whole new species. We cannot predict where it's going, and we cannot control it in the near future"
“AI is like a whole new species.” She says flatly. “We cannot predict where it's going, and we cannot control it in the near future. But I believe that we will come to a peaceful place in 30- or 40-years’ time. Out of curiosity, industry, and innovation, we’ve obviously ended up making mistakes, but I think we need these technologies to help us unravel all of the terrible things that we've done. AI is a thread we can pull to bring us to a more present state, past the chatter and narcissistic rubbish, past the hatred and the black and white thinking.”
One thing is for certain, in the very near term we can expect much more from Heap. While she may not be ready to put firm dates on the myriad projects she has in the works, the word ‘soon’ crops up again and again in conversation. With new tunes, new tech, and a renewed sense of purpose, Heap seems poised to once again make a bold impact after her 10-year hiatus from the music industry. As she herself puts it: “The future is fucking exciting.”
