Multimedia Operations for Android using FFmpeg

FFmpeg is a free and open-source software project consisting of a large suite of libraries and programs for handling video, audio, and other multimedia files and streams. The FFmpeg core itself designed for the command-line-based processing of multimedia files.

FFmpeg is a very fast multimedia file converter that can also grab from a live audio/video source. It can also convert between arbitrary sample rates and resize video on the fly with a high quality poly-phase filter.

Multimedia FFmpeg process diagram

FFmpeg calls the libavformat library (containing demuxers) to read input files and get packets containing encoded data from them. When there are multiple input files, FFmpeg tries to keep them synchronized by tracking lowest timestamp on any active input stream.

Encoded packets are then passed to the decoder. The decoder produces uncompressed frames, which can be processed further by filtering. After filtering, the frames are passed to the encoder, which encodes them and outputs encoded packets. Finally those are passed to the muxer, which writes the encoded packets to the output file.

Features of FFmpeg

• Enabled network capabilities
• Multi-threading
• Supports zlib and Media-codec system libraries
• Camera access on supported devices
• Supports API Level 21+
• Supports target SDK version 30

Let’s move to the FFmpeg library setup.

Update your build configuration

To integrate FFmpeg in your android application you can use pre-compiled libraries like FFmpeg, which provide multiple predefined multimedia operations(Queries) and which is easy to integrate by adding FFmpeg dependency in app module gradle file and sync project.

FFmpeg provide predefined multimedia operations(Queries) like :

• Cut video using time,
• Convert image to video,
• Add water mark on video,
• Add text on video,
• Combine image and video,
• Combine images,
• Combine videos,
• Compress a video,
• Extract frames from video,
• Merge GIFs
• and many more queries…

Let’s take one example of query: Cut a small part from video using time

val query : Array<String> = “-i” , “input video path.MP4” , “-ss” , “startTime(00=hrs:00=Min:00=Sec:00=m.Sec)” , “to”, “endTime(00=hrs:00=Min:00=Sec:00=m.Sec)” , “-r” , “$FRAME_RATE” , “-preset” , “ultrafast” , “output video path.MP4”

Add an easy way to execute the In-build command(Query):

val startCutTimeString = "00:01:00" (HH:MM:SS)
val endCutTimeString = "00:02:00" (HH:MM:SS)
val query:Array<String> = FFmpegQueryExtension.cutVideo(inputPath, startCutTimeString, endCutTimeString, outputPath)
CallBackOfQuery.callQuery(this, query, object : FFmpegCallBack {
         override fun statisticsProcess(statistics: Statistics){}
         override fun process(logMessage: LogMessage) {}
         override fun success() {}
         override fun cancel() {}
         override fun failed() {}
})

same for other queries and also you can apply your own query like as below.

Add an easy way to execute the your command(Query):

val query:Array<String> = "-i, input,....,...., outout" <- (your query)
CallBackOfQuery.callQuery(this, query, object : FFmpegCallBack {
         override fun statisticsProcess(statistics: Statistics){}
         override fun process(logMessage: LogMessage) {}
         override fun success() {}
         override fun cancel() {}
         override fun failed() {}
})

No extra code required when we use pre-defined queries.

FFmpeg supports latest architectures :

• arm-v7a, arm-v7a-neon, arm64-v8a, x86 and x86_64

Basically, a given Android device might have an arm or an x86 cpu, these are just different architecture types from different manufacturers. Arm cpus are most common, but x86 is unusual. When you compile code, the output depends on the architecture target. When you build an app, you specify one of the architectures and then the app will only work on that type of the device. If you want to support all devices, you can compile multiple APKs to distribute.

Within a given architecture type there are multiple versions.
armeabi-v7a is the older target, for 32 bit arm cpus, almost all arm
devices support this target. arm64-v8a is the more recent 64 bit target.
I think most new devices are 64 bit, but not sure. arm64-v8a devices can run code compiled against armeabi-v7a, it’s backwards compatible.
For more details : Click here

Conclusion

FFmpeg framework provides a great variety of trans-coding processes. However, Source code structure is very complex, which requires a deep understanding of multimedia processes and programming skills. Future works aims to explore live streaming feature with FFmpeg.

You can check out the fully working example on our GitHub Repository.

If you are facing any bugs/issue or want any feature then feel free to create an issue on GitHub.

If this article and package helps you then please show some love by staring ⭐️ our package and giving a few claps 👏.