How to do intrusion detection?

Introduction

Alerting system are quite important for safety and surveillance . As the number of critical nodes increases doing manual surveillance becomes difficult. CCTV is an important way to partially automated way to monitor but as number of CCTV units in an area increases, monitoring all of them becomes more and more difficult. As such we require fully automated system.

Intrusion or unwanted entry detection is an application where we observe an area (selected by operator) and the software returns an alert whenever there’s an unwanted entry in that area. Intrusion detection is also baseline approach for zone monitoring. Intrusion detection systems are very important in critical areas such as server room, Electric room, children playing area unattended places such as swimming pool etc. Whenever there’s person entry, an alert is generated, and it is pushed to desired alerting format like whatsapp, direct-call or an email.

Everyone loves to have live alert like this minion

Approach

First step in automated intrusion detection is to detect object of interest. In most of the case its a person. With the advancement of deep learning,models based on neural network become more and more generalised. CNN based models can easily be trained based on object type.

A person detection model can be trained by collecting person data and annotating it. Once annotated a neural network based model can be trained.
More detailed steps are available on my blog(coming soon)

person detection in action

Second step is to choose area to monitor. It can be any polygon in the ground plane . It is known as ROI(region of interest).
OpenCV provides polyline drawing function which can be used to easily draw a ROI . A simple version of that is cv2 rectangle function which can be used to select rectangular ROI.

But if you want to work on jupyter and want to visualise ROI in jupyter notebook you can refer my blog.

How to select ROI in Jupyter Notebook ?

limitation :- An image is 2D projection of 3D coordinates. The detection will be also run in this 2D image, thus the output will also be an 2D-coordinates([x1,y1,x2,y2]). This may lead to slightly incorrect detection. To avoid this problem, we can create ROI that’s continuous w.r.t to camera.

On the Left a ROI which is non continuous from camera position , thus there’s possible area intersection when person is in foreground and ROI is in background …On the Right ROI is continuous from the camera position, thus overlap and intersection both happens whenever there’s a person in the view.

Third and the last step is to compute intersection between ROI and person detection(bbox). Intersection can be computed in two ways. First by Evaluating position of detection point w.r.t. ROI and second by computing non-zero intersection between two polygons(ROI and detection). We are going to use both ways sequentially to improve goodness of model in production. Following is the sequence to compute intersection

• Check if the ground points([x3,y3],[x4,y4]) of person detection are within the ROI

both (x3,y3) and (x4,y4) are considered as ground point as they are in same plane as ROI

to check if a point lie within a polygon we are going to use shapely. Shapely is a python library for manipulation and analysis of geometric objects in the Cartesian plane.

Polygon1 and Polygon2 are intersecting with common intersection area as P

• If above condition is True then check for the intersection, which have intersection area above threshold.

Following is the github notebook for computing intersection

intersection functions

** intersection between two polygon can be computed using shapely or bitwise and operation in opencv but by time profiling i found out shapely is much faster than opencv.

time comparison shapely vs opencv for intersection detection

Once intersection is computed we can take input from opencv video input. After deciding on ROI we can push feed to person detection model. If person detection with ROI is above threshold, then a alert is generated.

Output of above Code.

you can find whole working code here(coming soon).

This post is guided and edited by my colleague Anuj Khandelwal