Simultaneous Internal Audio/Video Recording on Android
Capturing internal audio in android has always been a challenge and often required root privilege which most devices don’t have and thanks Google they haven’t added this feature officially into core android base for a long time, Of course, some manufacturers (Samsung etc.) had a custom audio capture application on their phones, but pure android did not have this feature.
Unlike internal audio capture, many devices had screen recording feature as default for a long time, from the early android versions to the latest.
luckily Google added internal audio capture and its related APIs into android starting from Android 10 version
for screen recording feature, android has default built-in utility called “screenrecord” that can be executed through ADB (a.k.a Android Device Bridge) like the following way:
adb shell screenrecord /sdcard/screenrecord.mp4
but the problem is default screenrecord utility does not capture internal audio (even in latest android R, correct me if i’m wrong) it has only ability to capture outer sound via microphone
combining both internal audio(mp3, aac) and screen recording (mp4) is not straightforward currently (of course there are 3rd party application that can do this)
in this article, i will show you how to do this with only using public android APIs (no 3rd party library, no challenging/custom mod required)
To record/capture internal audio and screen and combine them into mp4 video, we first need service (background service) which can run even after app is closed
so we define our service:
since our service will capture screen and internal audio, we also need additional attribute(android:foregroundServiceType=”mediaProjection”) when describing our service in AndroidManifest.xml
Next, for our app to work, we need to define required permissions via AndroidManifest.xml
for dynamically obtaining required permissions, we also define verifyPermission() function inside MainActivity (see dynamic permissions of Android M)
We then need to create a thread that will run in our service and do our heavy operations (recording data, encoding etc.)
inside RecorderThread, we need to setup audio record object with correct parameters in order to record/capture internal audio (with the help of AudioPlaybackCapture APIs)
after then we also need to setup required video encoder and recorder objects in order to capture device screen
we now have completed the setup for both internal audio record and screen capture, and also since audio record APIs returns audio in pcm audio format and screen recording APIs return data in chunks, we need to encode them in proper audio/video formats and then merge/combine them into single mp4 file, that’s why we also defined encoders
for data flow, audio record captures internal audio as pcm data and write that data into some sort of queue of encoder(implementation detail) via queueInputBuffer and audio codec APIs reads from queue with dequeueInputBuffer
same flow also applies to video record and video codec pair as well
after encoding audio/video data in proper formats (aac, mpeg4) we now can merge them into single mp4 file, we do this with the help of MediaMuxer APIs
done you can now record screen with internal audio, the recorded mp4 file will be on /storage/emulated/0/MyRecord.mp4 (you can change it through here)
Note: i used a workaround here for faster demonstrating it on my android 10 device, but it MAY crash on upper android versions (android 11,12), this workaround simply resolves android’s latest “scoped storage” mess, if you experience a crash related to file read write permission denial, you can solve it through here (even if this issue is not related with screen recording/audio recording APIs, I will also update code myself to address this issue ASAP i hope)
Beware: this app can only work on android 10(as i tested) and up versions, here also the full implementation on Github https://github.com/sirmordred/Android-AudioVideoRecorder
if you like thumbs up and share, if you have any question, you can write comment and also open issue on Github repo
thank you for reading