Recording internal audio and device screen while executing Appium Test on Android

Appium is an open source test automation framework for use with native, hybrid and mobile web apps.
It drives iOS, Android, and Windows apps using the WebDriver protocol and it is used heavily in application testing field, especially android mobile application testing, it allows developers to simulate clicks, scrolls etc. simply user interactions in an automated fashion to test various functionalities of application

image source: https://academy.brightest.be/courses/mobile-test-automation-with-appium/

long story short, while i was using appium in my job and also in an android development, i needed to test an app which has some audio functionalities, like producing audio song file which audio effects has been applied into, when i started to write automated test for my app, i need to somehow record the audio played by my app, but as far as i know, and as i searched it on documents of appium and net, appium currently does not have ability to record internal audio on android while executing test (see https://github.com/appium/appium/issues/13177)

if you did not read my previous article (simultaneous internal audio/video recording on android) check out here, i wondered if i can somehow implement this functionality into appium itself, so i started to deep dive into appium and see how it works

since internal audio/video recording is possible on android in an application level, i need to apply my internal audio/video recorder into appium on android application level

luckily i found out that appium uses a helper android application called “io.appium.settings” while executing tests, so i targeted that application

it is an open-source application that works on android during test execution, and it provides some internal device data for appium

i managed to implement my recorder code into that appium android application but i realized it needs much more work than implementing it

firstly i need to talk (start/stop recording) with application via only adb commands (adb stands for android device bridge, and appium has ability to execute adb commands) because there is no any other way than that (except hacky/tricky solutions)

so i used getIntent() method inside onCreate() method of Settings activity to capture adb commands and talk with my recorder service:

and inside handleRecording() method, i parsed the intent:

With adb commands, we can open applications, start activities/fragments and also services, we can also broadcast messages into running broadcast receivers on the phone through intents

we can also include data as strings into broadcast message commands, and it can be received and accessed through intent’s extra objects (see broadcast intent extras)

so we now can start recording via the following command:

adb shell am start -n "io.appium.settings/io.appium.settings.Settings" -a io.appium.settings.recording.ACTION_START --es filename abc.mp4

to stop we can use the following command as well:

adb shell am start -n "io.appium.settings/io.appium.settings.Settings" -a io.appium.settings.recording.ACTION_STOP

one of the challenging issues that need to be solved is that appium executing test in an automated fashion and since our recorder service needs some permissions that needs to be granted by user, we need to grant them via adb command persistently and so we don’t interrupt automated test execution

luckily pm utility of adb (adb shell pm) and appops utility does exactly what we want

we can grant required permissions persistently with adb via the following commands:

adb shell pm grant io.appium.settings android.permission.RECORD_AUDIO

another obstacle is media projection permission dialog:

and to overcome this and avoid the need of user interaction, we will use appops utility (see here for more info)

adb shell appops set io.appium.settings PROJECT_MEDIA allow

perfect, we can now start internal audio/video recording seamlessly

now you can record internal audio/video during execution of appium test like the following way:

python example(java may also be used since appium supports it as well):

test_recording_filename = "MyExample.mp4"
# Start recording
driver.execute_script('mobile: shell', {
'command': 'am',
'args': ['start', '-n', 'io.appium.settings/io.appium.settings.Settings', '-a', 'io.appium.settings.recording.ACTION_START', '--es', 'filename', test_recording_filename],
'includeStderr': True,
'timeout': 5000
})

# Run your test, click/scroll etc.
el = driver.find_element_by_class_name('UIATextField')
el.click()

# Stop recording
driver.execute_script('mobile: shell', {
'command': 'am',
'args': ['start', '-n', 'io.appium.settings/io.appium.settings.Settings', '-a', 'io.appium.settings.recording.ACTION_STOP'],
'includeStderr': True,
'timeout': 5000
})

# to obtain recorded mp4 file, issue the following command:
file_base64 = driver.pull_file('/storage/emulated/0/Android/data/io.appium.settings/files/' + test_recording_filename)

# Release driver instance
driver.quit()

another sample snippet can be found on here https://gist.github.com/sirmordred/4c505ddce96f7513bfbe49d38ef041be

Note that recorder mp4 file’s path always starts with /storage/emulated/0/Android/data/io.appium.settings/files/ the path is the best out of my options (it has read/write access for both app and adb, also compatible across android versions)

(EDIT: Pull Request is now merged into official Appium repository, you can use this feature on Appium started with Version 4.1.0) You need to build my custom io.appium.settings application through here from source and install into your device in order to use recording features, i also opened PR into official appium repository

Keep in mind that the above 2 adb commands need to be issued first in order to use the functionality (dont worry, it is persistent across reboots)

BEWARE: this functionality can only work on android 10 and upper versions because of internal audio capturing API limitations

thanks for reading