Announcing AudioShake —An AI to Open up Music for New Uses
Today we launch AudioShake, a powerful AI that can take a recording from any point in time and deconstruct it into separated vocal and instrument tracks (often called “stems”).
Musicians today lose out on many opportunities for their music because of missing or lost stems. This happens across the spectrum in the music industry — from damaged or lost master tracks of world-famous artists, to the indie band who recorded on their computer and only passed along the final song (the “mix-down”) to the label.
But a song is much more than what we hear on the radio. There are the instrumentals we sing along to in karaoke, or that provide the background mood music in commercials, games, and movies. Re-mixes that bring together iconic vocals from different songs. And samples that begin as catchy beats in one song and decades later are imagined and re-imagined again by new artists.
To make these kinds of uses happen, you need stems — the isolated instruments of a track. As an artist or label, if all you have is the radio version of your song, you’re missing out on far more opportunities to earn money from your music. Far too often, sync licensing opportunities are left on the table, and catalog that could be revitalized is left gasping for air. The impact is not just on contemporary music, but music from 100 years ago.
The decades-long problem of sound separation
Researchers have worked for decades on sound separation and music information retrieval (MIR), but recent advances in deep learning have allowed the field to grow in leaps and bounds, with particularly exciting research published over the past year.
As AI researchers, music lovers, and slightly-bad musicians, we wanted to see if we could harness the power of AI to help create new opportunities for artists.
Over the past two years, we’ve built what we believe is the best-performing sound separation model in the world. It’s able to break songs into stems for vocals, drums, bass, guitars, “other,” and instrumentals — with more instruments to come.
We’re particularly proud of our work on songs from the mono-track era (think artists like Elvis, Sinatra, Nina Simone, and the early Motown era) and hope the creation of stems for these types of older songs will open up new possibilities to introduce these artists and their songs to a younger audience.
We’re also excited to see the new creative uses that will emerge from stem creation — from remixes, mash-ups, and reissues, to superfan karaoke boxsets of the greatest catalogs in the world. Imagine the ability to sing karoake to the original Led Zeppelin IV, or play Jimmy Page’s electric guitar parts yourself along with the original band.
AudioShake AI can break songs into vocal, drums, bass, guitar, and other — as well as instantly create instrumentals. Here’s an example of 5-stem separation:
How it works
AudioShake trains on tens of thousands of high-quality stems to learn what each instrument sounds like. Our model learns how to identify drums in a rock song, for example, or the differences in vocals in rap versus classic rock. This then allows us to pull apart songs our model has never seen before.
All of this happens very quickly — a song can be processed 150x faster than real-time, with further possibilities for optimization.
Is it perfect every time? Of course not. Perfection lies in the master tracks. But when you don’t have access to good masters, AudioShake can get you very close to the real thing. Occasionally you may hear a distortion, or a bit of bleed from an instrument into the vocals. On some challenging older tracks, we might have crystal clear separation, but we’ve actually separated too much, and a sense of presence is lost. But most of the time, our tracks are at a quality level where they can be used commercially.
Work with us!
The opportunity to work with people from so many different areas of the music business has been one of the most exciting parts of our work on AudioShake. We work directly with rightsholders and licensees to ensure that musicians are compensated for their work.
Are you an artist or label wanting to explore new uses for your music? We’d love to hear from you.
A producer excited about the re-mix possibilities, or an audio engineer struggling with removing bleed on a re-mastering project? Hit us up.
A tech company building a future in which we’re all transformed into instruments in a virtual world powered by bitcoin? Hmm…tell us more.
Find us at audioshake.ai or on Twitter @audioshakeAI
