Real-time Face Recognition on Home Security Cameras using Python and Jetson Nano (Diary) - part 1
We have just begun to scratch the surface of what all Artificial Intelligence can achieve and among these things face recognition is one of the most intriguing applications with major implications, both positive and negative.
I have been learning Artificial Intelligence for over a year now. Even though I have made many small and big projects on Deep Learning, Computer Vision, Machine Learning and Data Science since I started, none of them involves working with actual hardware.
Hardware projects take time, you need to learn a lot of things and they also require investment in the hardware. Maybe for these reasons I had been avoiding doing one. But last month while doing another one of the Deep Learning courses I realized that there is a limit to how much I can learn with the current method and hence I decided to do a full end to end project that also involved the physical deployment of the system.
We have security cameras installed at our house so I decided to make a project using the data from the cameras and the first thing that came to my mind was a Face Recognition System.
As this is going to be a long and big project. I decided to write my daily work as a blog. So, this blog will act as my project journal. I plan on summarizing my work in another blog once this project is complete.
Currently, I don’t even know if this project is feasible or not.
“Process is important than the result.”
This might end up as a complete failure but I still believe that it will be a great learning experience.
I will keep updating this blog as I progress in my project.
Project Description
The objective of this project is to build a face recognition and threat alert system using the video feed from home security cameras. I will be using Nvidia Jetson Nano for deployment and python for programming. The major libraries that will be used include OpenCV, TensorFlow, face_recognition and dlib.
Hardware to be used: Nvidia Jetson Nano (4GB)
I want to follow an industrial project workflow for this, so, I have decided to divide the project into the following steps:
- Budgeting
- Feasibility Analysis
- Exploring Face Recognition techniques
- Data Collection
- Data processing
- Modelling
- Evaluation
- Testing
- Improving
- Deployment
- Monitoring
These might change as I move forward in the project and new things come up.
Budgeting
The major equipment required for the project include:
1. Nvidia Jetson Nano (4GB)
2. Case for Jetson Nano
3. Power supply for Jetson Nano (5V — 4A)
4. > 64 GB memory card
Total Cost = around ₹ 13000 (175 $)
Feasibility Analysis
I identified the following divisions for feasibility analysis:
a) Is it possible to obtain data? If so, How?
b) Given the data obtained, is it possible to use it to build a face recognition system?
c) If it is possible to build a face recognition system with the data, is it possible to run it in real-time on Jetson Nano?
Obtaining Data
Before I could start anything, the first and the most important thing was to get data (video) to work on.
The challenge here was that collecting data from the security cameras which operate on their independent circuits wasn’t the same as getting it simply from a webcam or from a USB camera directly connected to the computer.
Till now, I had only worked on cameras connected directly to a PC. I didn’t know how to obtain a video feed from a camera that is on a different network. So, I discussed it with my friends who had more knowledge of working with external hardware and scoured through the internet.
Initially, I thought of ways such as using HDMI or ethernet with a suitable converter to obtain the feed but since the live feed from the cameras could also be viewed on mobile devices I thought there must be a way to get video input to our program using a similar way.
That is when I came across a youtube video on IP cameras. An IP camera is a camera that transmits and receives video over a network connection. I decided to go with this method as it was the easiest and the most common way to obtain feed.
Detailed work:
July 1 — July 3:
Explored camera connections, How they are deployed across the house? Which wire goes where? Which wires do I need?
Looked for Jetson Nano buying options. Decided to delay buying it till I have done sufficient feasibility assessment.
July 4:
Explored how to view camera feed on the laptop (without programming) thinking that it might give a hint on how to get input in python. Used HDMI to view on a TV monitor. The camera supplier came to my rescue here. He told me that the IP was required to view it on the laptop.
At this point, I was just randomly doing things and didn’t know if any of this would actually help.
July 5 — July 6:
Explored youtube on how to get the IP of the security camera. Came across 3 ways:
1. Router login
2. Wire Shark
3. Camera company’s discovery tool (Dahua config tool in my case)
I used wire shark to get the IP of the cameras’ network.
Once I obtained the IP, I was able to login into the camera configuration page and view its feed on my laptop.
While searching the internet to find a way to obtain the IP of a Dahua camera, I came across a method that used the RTSP protocol to get IP camera video feed in a python program.
This day I completed the first task of getting video feed as input to the python program.
Section Summary (Obtaining data from an IP Camera):
The following command summarizes this section and can be used to get input feed from a Dahua IP camera into a python program using OpenCV:
cam = cv2.VideoCapture(RTSP URL)
RTSP URL signature:
rtsp://<username>:<password>@<ip>:<port>/cam/realmonitor?channel=<channel>&subtype=<subtype>
<username> — Camera login username.
<password> — Camera login password.
<ip> — the IP address of the camera or the system.
<port> — RTSP port (default:554).
<channel> — the channel number. 1 to n depending upon the number of connected cameras.
<subtype> — the type of stream. 0 for main stream, 1 for extra stream 1, 2 for extra stream 2.
Example:
rtsp://admin:admin123@192.168.1.12:554/cam/realmonitor?channel=1&subtype=0
Dahua Remote Access article gives a detailed explanation.
If you are using cameras from other companies this article on RTSP stream URLs from (nearly) Every Manufacturer is a good place to get help.
Data-Driven Feasibility Analysis
Under this sub-section of the feasibility analysis, we study the data from home security cameras to discover problems that might hinder our project.
I decided to skip this part and assume that the data was good because at this point even after visualizing the data there was no way for me to know if the data was right for the project or not.
Hence, I decided to move forward with the assumption that the data from the security cameras was sufficiently good to make a face recognition system.
Real-time Operation Analysis
The target here is to analyse whether it is possible to run our face recognition system on an embedded system or not? (Is the technique sufficiently optimizable?)
Thorough research on face recognition techniques is required before we can decide on real-time feasibility. Hence, we will revisit this section after carrying out step 3, which is, “ Exploring face recognition techniques”.
The article continues here: Real-time Face Recognition on Home Security Cameras using Python and Jetson Nano (Diary) — part 2.
