Object Tracking In Surveillance Video

Today I would like to share with you about the simple tutorial to use the OpenCV object tracker to trace the object of interest appeared in the real time surveillance video.

Background

A lot of companies invest millions in security and surveillance technology as well as hire security personnel to oversee the surroundings of their buildings. However, paying attention to the screen is usually tiresome and boring, and may result in human error when any suspicious person appears.

Building a object tracker to support the surveillance system is easy, straightforward and computationally cheap. In this tutorial, we will be using the openCV KCF (Kernelized Correlation Filter) tracker to achieve the tracking task on the video clips.

openCV KCF Class

We understood that video is actually a sequence of frames captured over time. When you are recording using any mobile devices, there is usually a setting to configure the number of frames per seconds (fps) taken by the camera. The concept of the object tracker is to detect the motion of the object over the time and predict the bounding box of the object detected based on the region of interest during the initialization. Open KCF is a tracking framework which processing speed is enhanced by optimizing the circulant matrix of the possible detected sample in the current frame.

Refer to the following link to understand the KCF Tracker

OpenCV: cv::TrackerKCF Class Reference

After the a quick introduction about the object tracker, let us start about the tutorial today!

Reading from WebCam

First of all, we will read the real time frames from the surveillance camera.

import cv2
camera_ip = “rtsp://username:password@IP/port”
stream = cv2.VideoCapture(0) #Read from the local camera

All the real time object tracker is required to be initialized at the beginning stage to highlight the region of interest. Therefore, a single frame of the surveillance camera is picked and fed into the object tracker.

if not stream.isOpened():
   print("Could not open camera stream...")
ok, frame = stream.read()

Assume that we want to trace the face of a person appeared in the surveillance camera, we can use a simple face detection pretrained model to find the bounding block of the frame in the frame. In this tutorial, we will use Medipipe Face Detection model to find the bounding box of the face in the frames:

import mediapipe as mp
bbox = None
if ok:
  mp_face_detection = mp.solutions.face_detection
  with mp_face_detection.FaceDetection(model_selection=1,min_detection_condence=0.5) as face_detection:
  results = face_detection.process(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
  if results.detections:
    row, column,channel = frame.shape
    y = int(results.detections[0].location_data.relative_bounding_box.ymin*row)
    h = int(results.detections[0].location_data.relative_bounding_box.height*row)
    x = int(results.detections[0].location_data.relative_bounding_box.xmin*column)
    w = int(results.detections[0].location_data.relative_bounding_box.width*column)
    bbox = (x, y, w, h)

The steps shown in the snippet on top is to detect the bounding box of the face appears in the frame. If there is no face detected in the frame, the value of the bounding box remain in None and the object tracker will not be initialized in next steps. Besides, we only pick one face detected in the frame if there are more faces found by the face detection model.

if bbox:
   #define openCV KCF tracker
   tracker = cv2.TrackerKCF_create()
   initialized = tracker.init(frame,bbox)
   while initialized:
     try:
       success,nextframe = stream.read()
       if not success:
         raise Exception('Stream frame cannot be read')
       detected,nextBBox = tracker.update(nextframe)
       if detected:
       # Tracking success
         p1 = (int(bbox[0]), int(bbox[1]))
         p2 = (int(bbox[0] + bbox[2]), int(bbox[1] + bbox[3]))
         cv2.rectangle(nextframe, p1, p2, (255,0,0), 2, 1)
       else :
       # Tracking failure
         cv2.putText(nextframe, "Tracking failure detected", (10,80),cv2.FONT_HERSHEY_SIMPLEX, 0.75,(0,0,255),2)
       # Display result
       cv2.imshow("Tracking", nextframe)
       # Exit if ESC pressed
       k = cv2.waitKey(1) & 0xff
       if k == 27:
         raise Exception('Key pressed to stop')
     except:
       stream.release()

The following snippets illustrate how an object tracker is defined and use it to trace the face detected by Medipipe Face Detection model. Take note that the video streaming will be terminated if the tracker cannot trace the face detected anymore. As the tutorial is just giving you an overview on the object tracker, you could enhance the tutorial for your own use

Conclusion

In the digital age, computer vision is becoming increasingly important. The real-time object tracking technology will definitely be helpful in many ways, not just in the surveillance and security technology. I hope you found the tutorial helpful, and please do not hesitate to comment if you have any suggestions for improvement.