Object Recognition with OpenCV on Android

This article is for a person who has some knowledge on Android and OpenCV. We will look at how to use the OpenCV library to recognize objects on Android using feature extraction.

  1. Download and setup Android Studio

I am using Android Studio and you can follow this link to download and install Android studio and SDK tools but if you are a die hard eclipse fan you also can follow this tutorial( no hard feelings ;) )

2. Setting up OpenCV library inside Android Studio

You have to download and import OpenCV library to android studio and there is a stackoverflow answer which you can follow to setup everything. If you are using Eclipse use this link.

Now you are ready to mingle with me ;). The algorithm we are going to use is ORB(Oriented FAST and Rotated BRIEF). As an OpenCV enthusiast, the most important thing about the ORB is that it came from “OpenCV Labs”. This algorithm was brought up by Ethan Rublee, Vincent Rabaud, Kurt Konolige and Gary R. Bradski in their paper ORB: An efficient alternative to SIFT or SURF in 2011. It is a good alternative to SIFT and SURF in computation cost, matching performance and mainly the patents. Yes, SIFT and SURF are patented and you are supposed to pay them for its use. But ORB is not . You still can use SIFT and SURF but you have to compile them separately as they are not contained in the latest version of the OpenCV library and if you are going to make a commercial application you have to pay and get them.

Here is a link for the other existing algorithms in OpenCV for object detection

Now you know about ORB and it’s history, lets start coding.

3. Create a new project on Android Studio

This step is trivial therefore I am giving this link so you could follow that and create a new project. Please do read what is on that page it is very informative.

4.Creating the XML for the UI

If you are not new to Android you know that very well that you need to create a XML file for the UI. These files are created inside the layout folder.

You can create a XML file using the above code. First segment is the Camera View, second segment (TextView) is quite not necessary but that is there for testing purposes ( just in case if you want to get some data on the screen rather than the log)

5. Write about the Main Activity

You can create the Main Activity copying the above code. But I would like to point out few important factors in the above code. After all we are software engineers not code monkeys ;)

First you would notice is that MainActivity is extending CameraBridgeViewBase.CvCameraViewListener2 . This interface would enforce us to implement few methods which are related to the camera. The method which is important to us is the onCameraFrame(CvCameraViewFrame inputFrame). This would receive the video as frames and you can do all the image processing inside this method and return a Mat with that image :)

There are three main variables

detector is used to detector features and the descriptor will compute the descriptors and the matcher would match the descriptors.

Later in the code you can find there are distance values given which are to set the level of threshold that you want the feature matching has to happen, after that in the code best matches have been sorted out and drawn using Features2d.drawMatches() method. That image has been returned to the camera view.

6. Image preprocessing

It is important to have the input image and the image that you receive from the camera has the same dimensions. You definitely can change those values in the code but for now I am changing that manually before I insert the image to the assets folder in the project. if there is a mismatch between your image’s dimensions, image’s type you will get few errors. I got them and I solved them using the following two links.

Stackoverflow Link1

Stackoverflow Link2

Then you can create an assets folder on android and then you can copy paste the image to that folder. You can follow this link to create an assets folder inside your project.

7. AndroidManifest.XML file

This is the file which manages permissions and controls which layout loaded first etc.You don’t have to copy paste this, but make sure to add camera permission and also the it’s features in this file so the application can use the camera on the phone.

7. Sample

Sorry about the background music, I will upload a better video later :)

8. Git Hub Link : https://github.com/akshika47/OpencvAndroid

Please do leave your comments and feedback :)

Akshika Wijesundara

Written by

PhD Candidate in Computer Science | Founder & Chairperson of SEF| Ex-Google Student Ambassador

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade