Announcing Audioshake —An AI to Open up Music for New Uses

Audioshake
Jan 26 · 4 min read
Image for post
Image for post

Today we launch Audioshake, a powerful AI that can take a recording from any point in time and deconstruct it into separated vocal and instrument tracks (often called “stems”).

Musicians today lose out on many opportunities for their music because of missing or lost stems. This happens across the spectrum in the music industry — from damaged or lost master tracks of world-famous artists, to the indie band who recorded on their computer and only passed along the final song (the “mix-down”) to the label.

But a song is much more than what we hear on the radio. There are the instrumentals we sing along to in karaoke, or that provide the background mood music in commercials, games, and movies. Re-mixes that bring together iconic vocals from different songs. And samples that begin as catchy beats in one song and decades later are imagined and re-imagined again by new artists.

To make these kinds of uses happen, you need stems — the isolated instruments of a track. As an artist or label, if all you have is the radio version of your song, you’re missing out on far more opportunities to earn money from your music. Far too often, sync licensing opportunities are left on the table, and catalog that could be revitalized is left gasping for air. The impact is not just on contemporary music, but music from 100 years ago.

The decades-long problem of sound separation

Researchers have worked for decades on sound separation and music information retrieval (MIR), but recent advances in deep learning have allowed the field to grow in leaps and bounds, with particularly exciting research published over the past year.

As AI researchers, music lovers, and slightly-bad musicians, we wanted to see if we could harness the power of AI to help create new opportunities for artists.

Over the past two years, we’ve built what we believe is the best-performing sound separation model in the world. Its able to break songs into stems for vocals, drums, bass, guitars, “other,” and instrumentals — with more instruments to come.

We’re particularly proud of our work on songs from the mono-track era (think artists like Elvis, Sinatra, Nina Simone, and the early Motown era) and hope the creation of stems for these types of older songs will open up new possibilities to introduce these artists and their songs to a younger audience.

We’re also excited to see the new creative uses that will emerge from stem creation — from remixes, mash-ups, and reissues, to superfan karaoke boxsets of the greatest catalogs in the world. Imagine the ability to sing karoake to the original Led Zeppelin IV, or play Jimmy Page’s electric guitar parts yourself along with the original band.

Audioshake AI can break songs into vocal, drums, bass, guitar, and other — as well as instantly create instrumentals. Here’s an example of 5-stem separation:

To a Place by The Broken Orchestra

How it works

Audioshake trains on tens of thousands of high-quality stems to learn what each instrument sounds like. Our model learns how to identify drums in a rock song, for example, or the differences in vocals in rap versus classic rock. This then allows us to pull apart songs our model has never seen before.

All of this happens very quickly — a song can be processed 150x faster than real-time, with further possibilities for optimization.

Is it perfect every time? Of course not. Perfection lies in the master tracks. But when you don’t have access to good masters, Audioshake can get you very close to the real thing. Occasionally you may hear a distortion, or a bit of bleed from an instrument into the vocals. On some challenging older tracks, we might have crystal clear separation, but we’ve actually separated too much, and a sense of presence is lost. But most of the time, our tracks are at a quality level where they can be used commercially.

Work with us!

The opportunity to work with people from so many different areas of the music business has been one of the most exciting parts of our work on Audioshake. We work directly with rightsholders and licensees to ensure that musicians are compensated for their work.

Are you an artist or label wanting to explore new uses for your music? We’d love to hear from you.

A producer excited about the re-mix possibilities, or an audio engineer struggling with removing bleed on a re-mastering project? Hit us up.

A tech company building a future in which we’re all transformed into instruments in a virtual world powered by bitcoin? Hmm…tell us more.

Find us at audioshake.ai or on Twitter @audioshakeAI

The Startup

Medium's largest active publication, followed by +773K people. Follow to join our community.

Audioshake

Written by

Audioshake uses sound separation to help break songs into their stems, opening up new opportunities for musicians to make money for their work.

The Startup

Medium's largest active publication, followed by +773K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store