The power of FFmpeg on Android
Back to the basics
What is FFmpeg?
FFmpeg is a multimedia framework able to decode, encode, transcode, mux, demux, stream, filter and play almost any type of content. It is used for professional image and video processing, including for various TV companies.
FFmpeg compiles and runs in a wide variety of environments such as Linux, Mac OS X, Microsoft Windows under different machine architectures, and configurations.
This post focuses on how to use the basic tools of FFmpeg to exploit your ability and create fun image and video content on Android.
First things first, we need an environment ready to run our FFmpeg commands. Which can be found and answered by the large FFmpeg community. There are two options to do so
- Building your own library:
2. Using any compiled source provided by the community:
- I highly recommend FFmpeg Android java library if you are going to use FFmpeg on Android for the first time.
Getting started on Your first App
You only need to keep in mind two steps before running your first FFmpeg command:
- Add a validation to check if the device supports the current version of FFmpeg.
2. Add an easy way to execute the commands:
With that, we are able to create the first FFmpeg command!
Command- line Tools
The FFmpeg libraries have different types of tools (Ffmpeg, Ffserver, Ffplay, Ffprobe, etc), however, in this post, we are going to focus on the FFmpeg capabilities.
“ Ffmpeg: converts audio, image or video between different formats. It can also capture and encode in real-time from various hardware and software sources.”
Getting started with commands
It’s very important that you start creating a command from a basic line and then increase the complexity to accomplish what you want to do.
FFmpeg-all is going to be, say our Bible of FFmpeg, however, it’s very difficult to read if you don’t know what you are you looking for. Therefore, I will disclose the most important arguments to create your app.
In FFmpeg, we are going to work with a variety of inputs- images, videos, GIFs, sounds, etc, and we are going to create a new output source with all our changes. All inputs will be defined by an argument -i.
Our first command of FFmpeg will be:
String command = “ -i beach.jpg output.jpg“
Where our input is:
And the output is awesome!:
Want to take this image to the next level? Let’s overlay 2 images. To do this we need to introduce a new argument called -vf or -filter_complex (both are the same). With this argument we can overlay, scale, concatenate images, change colors, set the transparency of an image, etc.
The new command looks like this:
String command = “ -y -i beach.jpg -i wolox.png -filter_complex [0][1]overlay=(W-w)/2:(H-h)/2 output.jpg”
As you can see, with the overlay feature, we can add the wolox logo over the beach image, generating a new image (output.jpg).
This command comes with new info (detailed below):
- -y Allows you to override the existing output.jpg file without any confirmation of the user. This is very useful when you are testing a command.
- [0][1] To stay organized we can make a reference to every input by the order number of the input itself starting with 0. Therefore, the beach would be 0 and the Wolox logo would be 1.
- Overlay Is the instruction for FFmpeg command to overlay 2 sources. In this case, we want to overlay the input [1] (the Wolox logo) over the input [0] (The beach)
- (W-w)/2:(H-h)/2 If you read the official documentation of an overlay, it states that we can make a reference to the position of the input and the overlay with the capacity to change it. W and H are the width and height of the beach and w/h the same for the overlay. So with this command, we are able to center the Wolox logo within the beach image.
Then we can do something like this
String command = “ -y -i beach.jpg -i wolox.png -filter_complex [1]scale=iw*2:ih*2,transpose=1,geq=r=’r(X,Y)’:a=’0.5*alpha(X,Y)’[s1];[0][s1]overlay=(W-w)/2:(H-h)/2 output.jpg”
Although the command gets longer, if you look carefully and use the documentation you will find that is very easy to understand.
- [1] makes a reference to the second input (the Wolox logo)
- Scale the Wolox logo and rotate 90° clockwise
- Make it half transparent (with geq=r=’r(X,Y)’:a=’0.5*alpha(X,Y)’) . If you read the documentation you see that we need to set r= ‘r(X, Y)’ to change from YCbCr color space (by default) to RGB color space
- After all these changes to the source [1], we create a new source called [s1]
- Finally, we will do the same as before and overlay input [0] (the beach) with the new source of the Wolox logo[s1]
Now let’s take your creativity to the next level. A GIF, for example, everyone loves GIFs nowadays right?
To create a GIF you will need two inputs:
- A background (an image, solid color, video, or other GIF).
- An overlay that can also be another image or GIF.
String command = “ -y -ignore_loop 0 -i beach-pixel.gif -i chicken.gif -filter_complex [1]scale=iw*1.5:ih*1.5[s1];[0][s1]overlay=100:H-320:shortest=1 output.gif ”
Beach-pixel.gif
In the above GIFS, we are able to control the time of the loop of the output file. Here is how:
- ignore_loop 0 We indicate in the background GIF to avoid looping a certain number of times (if defined in his metadata) and making it loop infinitely.
- shortest=1 we selected the duration of the output GIF to the shortest one. In this case, the background loops an infinite amount of times, so the chicken GIF will define the duration of the output file.
FFmpeg is a powerful tool, with a vast capability. Because it is available in Android we can begin thinking of new ways to create content for apps.
Although FFmpeg may seem complex and overwhelming at first glance, the large community of users is extremely helpful.
It’s your turn, give FFmpeg on Android a go!
