👨🏼‍💻 Audio Extraction & Compose & Audio Source Separation — Audio Editor Kit

Introduction

Hello everyone,

In this article, we will develop an audio edit app with you. In this application, we will develop the Audio Editor Kit’s ability to extract audio from video, combine different audio files, and separate different audio sources (vocals, instruments) from music.

Audio Sources to Separate:

1. Vocals
2. Music
3. Drums
4. Guitar
5. Bass Guitar
6. Violin
7. Piano

These sources can be changed in the future in this application these are the sources we can separate.

Use Case Example

1. The user has a video of The White Stripes — Seven Nation Army.
2. The user will extract the audio of this video to create .wav file.
3. The user will select sources to separate from the audio file. In this example; vocals, drums, and guitar.
4. 3 different audio files will be created for selected sources.
5. On compose page. Users will see these files and can listen to them. Users will choose sources among the created files to compose them. In this example; vocals and drums, not choosing a guitar.
6. A new audio file will be created of The White Stripes only consisting of vocals and drums without any music any guitar or other instruments.

What we will learn

1. Audio extraction.
2. Audio source separation.
3. Audio composing.
4. Exporting Audio

Requirements

• JDK version: 1.8 or later
• Android Studio version: 3x or later
• minSdkVersion: 21 or later
• Huawei Phone with EMUI 5.0 or later | Non-Huawei phones with Android 5.0 or later

HMS Core Integration

First of all, we need to create a project in android studio and AppGallery Connect. Then enable Audio Editor Kit. If you are new to these processes you can follow these steps:

1. Creating a Project and App in AppGallery Connect
2. Creating an Android Studio Project
3. Generating a Signing Certificate
4. Generating a Signing Certificate Fingerprint
5. Configuring the Signing Certificate Fingerprint in AppGallery Connect
6. Enabling Required Services in AppGallery Connect
7. Adding Configurations
8. Configuring the Signing Information for Your Project
9. Synchronizing the Project

Dependencies and Preparations

We need to add dependencies to the app-level gradle file.

Audio Editor Kit SDK

Then required permissions must be added to the AndroidManifest.xml file. In this case, to save the converted audio file and play it from phone storage we need to get storage access permissions. Also, the Audio Source Separation feature can be used in on-local and on-cloud ways. We will use it on the cloud so we also need internet permission. Sources might differ depending on whether it is on-cloud or on-local. Usually, on-cloud has more source options than on-local.

Required Permissions

Permission Request

The next step is requesting storage permissions from the user.

Requesting Permissions

You can check ManagePermissions class in the sample project I shared to have better use of requesting permission or if you have any problems in this step.

API Key

Also setting the API key is essential. You can get it from agconnect-services.json file.

Setting ApiKey

Audio Extraction— Video to Audio

We will create a simple page for this section. The page contains chosen video source from the device option and an “Extract” button to start the process.

Extraction Page Layout

Extraction Page

Now we can start to integrate audio extraction. Function “extractAudio” can be called from the HAEAudioExpansion instance as shown below. This function takes 5 parameters: context, video source path, output path, name of an extracted audio file, and callback object to help you track the progress and steps of the extraction process.

Selecting Video From Device

Audio Extraction

After extraction is complete, we can use the extracted audio file to separate its’ sources on the Separation page.

Audio Source Separation

After extraction user will be navigated to the separation page. On this page, there is a list of sources that can be separated from the audio file. We will gather that sources list from the HaeAudioSeparationFile instance. All the sources or a few of them can be separated during the process. HaeAudioSeparationFile takes a list of sources to set which audio sources separate. The separation process has 4 parameters; audio path, export path, created file name, and callback object.

Layout code and view assignments:

Separation Page Layout

Separation Page

Creating HAEAudioSeparationFile Instance

Gathering Audio Sources

Audio Source Separation

The separation process can take several minutes. After the process, we can navigate to the third page which is Compose page.

Composing Audio Files

This page design is similar to the separation page. But the list that contains audio sources only contains separated sources on the previous page. Users can choose these sources and compose them into new audio files. To compose these audio files we need a HuaweiAudioEditor instance. We will add all the paths to compose to this instance with the appendAudioAsset function. To create the final file we need to use Exporting Audio feature of the Audio Editor Kit.

Compose Page Layout

Compose Page

Compose Audio Files

Exporting Composed Audio File

Final App Part 1

Final App Part 2

Tips & Tricks

1. Don’t forget to enable Audio Editor Kit from AGConnect, add sha256 to AGConnect and add agconnect-services.json to the app folder of your project.
2. Use getInstruments method to gather supported audio sources.
3. If you want to see all features of the Audio Editor Kit, you can use its UI SDK. This SDK is easy to implement and it launches a page with all features of the Audio Editor Kit.

Conclusion

We develop a simple audio editor app and used just a few features of the Audio Editor Kit. You can check the links below to see a wide range of features of the Audio Editor Kit. I’m sharing the project repo in reference. If you have any problem you can use it.

Thank you for reading. See you in my other articles…;)

References

AudioEditorKit

Audio Editor Project Repo