Tensorflow object detection api for Indian Driving Dataset(IDD) in windows 10
This blog explains how to build a custom object detection model using tensorflow object detection api considering huge data of IDD.
Overview
The IDD dataset consists of images obtained from a front facing camera attached to a car. It consists of 10,000 images, finely annotated with 34 classes collected from 182 drive sequences on Indian roads. The car was driven around Hyderabad, Bangalore cities and their outskirts. The images are mostly of 1080p resolution, but there is also some images with 720p and other resolutions. This was sponsored by IIT Hyderabad and Intel.
There are five data sets available in the official website. You might have to sign up to get access to it. I have used the dataset named Dataset Name: IDD- Detection (22.8 GB). I have explained the whole folder structure and code in my git. For the sake of brevity I will keep it short here. The given input data is image and xml containing information about it.
The link I followed to figure out custom object detection using tensorflow is this. The object detection api of tensorflow works in tensorflow 1.15.0. I had hard time figuring out the version of tensorflow which will work for object detection api. Now with the recent release of tensorflow 2.x many people might think that the new code for object detection api has been released too. But that’s definitely not the case. To save your struggle find this link, which clearly says that it will take some months to develop the new modules.
We are collaborating with TF team to migrate to 2.0 now, but this is a huge effort which may take months. — pkulzc (a contributor of code for object detection api)
If you have already figured out how to make custom object detection work for IDD dataset then you can skip to result session where I have uploaded a video showing results. If not lets get going. For reference please download the folder named try_tensorflow_object_detection_api from here. It will keep you aligned while creating folders and sub folders.
These are the list of things you need to do.
1. Install tensorflow 1.15.0
Open anaconda prompt from windows and run below command
2. Create a new folder of name of your choice.
My folder name is try_tensorflow_object_detection_api
3. Download tensorflow models.
You can either download the whole repo from git and place it under current working directory(CWD), which is try_tensorflow_object_detection_api or git clone in CWD using git bash.
4. Install Protobuf and add all necessary paths to environment
The Tensorflow Object Detection API uses Protobufs to configure model and training parameters. Navigate to this link and download “ protoc-3.11.4-win64.zip” from it. Create folder named Google Protobuff.
Now extract the content of downloaded zip file to Google Protobuff and add the bin folder inside it to environment variables.
Now navigate to try_tensorflow_object_detection_api/models/research folder and run the below command.
5. Create folders and sub folders for training
Create a folder named workspace and a folder training_demo in it. training_demo is the folder which will contain all files realted to our training. Create folders with name annotation, image, pre-trained-model, training, scripts. Create two folders named train, test in image folder and create a folder named preprocessing in side scripts folder.
6. Copy all train images to train images\train fold
7. Create a label map
create a file named label_map.pbtxt in annotations folder
and paste below content to it and save.
item {
id: 1
name: 'truck'
}item {
id: 2
name: 'vehicle fallback'
}item {
id: 3
name: 'car'
}item {
id: 4
name: 'motorcycle'
}item {
id: 5
name: 'rider'
}item {
id: 6
name: 'person'
}item {
id: 7
name: 'bus'
}item {
id: 8
name: 'bicycle'
}item {
id: 9
name: 'autorickshaw'
}item {
id: 10
name: 'animal'
}item {
id: 11
name: 'traffic sign'
}item {
id: 12
name: 'train'
}item {
id: 13
name: 'traffic light'
}item {
id: 14
name: 'caravan'
}item {
id: 15
name: 'trailer'
}
8. Place train.csv file in place and create record file for tesnsorflow
You should have a train.csv in below format. How I processed IDD data set to get that csv is a different discussion. We will talk about it at the end. For now let’s continue the flow. We need to convert it to a record file(file understood by tensorflow and for faster execution) for which you would need this piece of code. Place this piece of code in scripts\preprocessing folder and execute below command.
9. Configuring a Training Pipeline
The model we will use is ssd_inception_v2_coco which is a pretrained model given by as it provides a good trade-off between performance and speed. How ever there are many other networks which can be used too which can be found here. We will download 2 things here. 1st: the ssd_inception_v2_cocopre-trained model and its config file. Place the config file inside training folder and extract the content of pre-trained model weights in pre-trained-model folder.
10. Parameter tuning in configuration file
Open the config file you just placed in training folder and change below parameters.
model {
ssd {
num_classes: 15 # number of classes. For us its 15
box_coder {
faster_rcnn_box_coder {
################################################################feature_extractor {
type: ‘ssd_inception_v2’ # the name pre-trained model. For us its ssd_inception_v2 as we have downloaded that model.
min_depth: 16
depth_multiplier: 1.0
###############################################################train_config: {
batch_size: 32 #For us its 32, as my memory crashed in higher value
optimizer {
rms_prop_optimizer: {
################################################################fine_tune_checkpoint: “pre-trained-model/model.ckpt” #Path to extracted files of pre-trained model. We had downloaded the pretrained model and placed the extracted the content in pre-trained-model folder.
################################################################# never decay). Remove the below line to train indefinitely.
num_steps: 20000 # number of epochs
data_augmentation_options {
random_horizontal_flip {
################################################################train_input_reader: {
tf_record_input_reader {
input_path: "annotations/train.record" #path to train record
}
label_map_path: "annotations/label_map.pbtxt" #path to labelmap
}
11. Train the model
Copy the train file from object detection folder and paste it in training_demo folder and run python command.
12. Detecting objects using the model.
Copy the file named export_inference_graph.py from git and and paste it in training_demo folder and run below command.
This will create a folder named inference_graph in same directory and save the model as frozen_inference_graph.pb in inference_graph.
13. Run on sample image
You can simply run the file Object_detection_image.py and see the results.
To test on your image change the variable name IMAGE_NAME.
There are two files Object_detection_video.py and Object_detection_webcam.py which works on videos and webcam respectively. I will suggest you should those too. Its fun.
14. Results
I have taken live feed from my mobile by connecting it to my PC as a webcam. First two from below 3 images are screenshots of videos.
I processed an youtube video and uploaded the results too. Source video https://www.youtube.com/watch?v=BqOLehUdqkU . I thank the video maker deeply.
15. Processing IDD dataset to create train.csv
If you remember I promised I will explain this at the end. I have another repo in git where I applied faster RCNN to same IDD dataset. Please take a look there.
If you run below two jupyter notebook from there you will get desired train.csv in format compatible with tensorflow Object detection API.
1. 2. Road map to problem solving and Data preparation.ipynb
2. create_csv_for_object_detection_api.ipynb
