Real-Time Face Recognition Using Pinecone DB, OpenCV, and Face Recognition Library (Part 2) : Face recognition and search with OpenCV video feed
In Part 2 of our tutorial, we will extend our real-time face recognition system to work with a webcam feed. This part will show you how to capture video from a webcam, process the video frames for face recognition, and query the Pinecone database for matches. In you directly started reading this part without setting up the image database in Pinecone, use this link for Part 1.
Step 1: Import Libraries and Set Up Environment
First, we need to import the necessary libraries and set up the environment. This includes importing cv2 for video capture and image processing, numpy for numerical operations, os for handling environment variables, face_recognition for face detection and encoding, and Pinecone for interacting with the Pinecone database.
import cv2
import numpy as np
import os
import face_recognition
from pinecone import Pinecone
import warnings
warnings.filterwarnings('ignore')Step 2: Initialize Pinecone and Connect to Index
Next, we initialize the Pinecone client using the API key stored in the environment variables. We then connect to the existing index that we created in Part 1.
pc = Pinecone(api_key=os.environ.get("PINECONE_API_KEY"))
# Set the index name
index_name = "img"
# Connect to the created index
index = pc.Index(index_name)Step 3: Capture Video from Webcam
We use OpenCV to capture video from the webcam. The cv2.VideoCapture(0) command starts capturing video from the default webcam.
cap = cv2.VideoCapture(0)
name = '' # We initialize the variable name to empty.Step 4: Process Video Frames for Face Recognition
In this step, we continuously capture frames from the webcam, resize them for faster processing, and convert them from BGR to RGB format. We then use the face_recognition library to locate faces and encode them. This encoding will be further used to query our database when ‘c’ is pressed.
while True:
success, img = cap.read()
if success:
imgS = cv2.resize(img, (0, 0), None, 0.25, 0.25)
imgS = cv2.cvtColor(imgS, cv2.COLOR_BGR2RGB)
faces_cur_frame = face_recognition.face_locations(imgS)
curr_frame_encodings = face_recognition.face_encodings(imgS, faces_cur_frame)
for face_encoding in curr_frame_encodings:
encode_query = face_encoding
y1, x2, y2, x1 = faces_cur_frame[0]
y1, x2, y2, x1 = (y1 * 4)-10, (x2 * 4)+10, (y2 * 4)+10, (x1 * 4)-10
cv2.rectangle(img, (x1, y1), (x2, y2), (0, 255, 0), 2)
cv2.rectangle(img, (x1, y2 - 35), (x2, y2), (0, 255, 0), cv2.FILLED)
cv2.putText(img, name, (x1 + 6, y2 - 6), cv2.FONT_HERSHEY_COMPLEX, 1, (255, 255, 255), 2)
cv2.imshow('Webcam',img)
k=cv2.waitKey(1)Step 5: Query Pinecone for Face Recognition
When the ‘c’ key is pressed, the current frame’s face encodings are used to query the Pinecone database for matches. The closest match is retrieved from Pinecone and name from the metadata is displayed on the video feed.
if k==ord('c'):
result = index.query(
top_k=1,
vector=encode_query.tolist(),
include_metadata=True,
)
if result["matches"][0]:
name = result["matches"][0]["metadata"]["name"]Step 6: Exit the Program
If the ‘q’ key is pressed, the program will exit, releasing the webcam and closing all OpenCV windows.
if k == ord('q'):
break
cap.release()
cv2.destroyAllWindows()Complete Code and Results:
This is the commplete code with correct indents:
import cv2
import numpy as np
import os
import face_recognition
from pinecone import Pinecone
import warnings
warnings.filterwarnings('ignore')
pc = Pinecone(api_key=os.environ.get("PINECONE_API_KEY"))
# Set the index name
index_name = "img"
# Connect to the created index
index = pc.Index(index_name)
cap = cv2.VideoCapture(0)
name='' # We initialize the variable name to empty.
while True:
success, img = cap.read()
if success:
imgS = cv2.resize(img, (0, 0), None, 0.25, 0.25)
imgS = cv2.cvtColor(imgS, cv2.COLOR_BGR2RGB)
faces_cur_frame = face_recognition.face_locations(imgS)
curr_frame_encodings = face_recognition.face_encodings(imgS, faces_cur_frame)
for face_encoding in curr_frame_encodings:
encode_query = face_encoding
y1, x2, y2, x1 = faces_cur_frame[0]
y1, x2, y2, x1 = (y1 * 4)-10, (x2 * 4)+10, (y2 * 4)+10, (x1 * 4)-10
cv2.rectangle(img, (x1, y1), (x2, y2), (0, 255, 0), 2)
cv2.rectangle(img, (x1, y2 - 35), (x2, y2), (0, 255, 0), cv2.FILLED)
cv2.putText(img, name, (x1 + 6, y2 - 6), cv2.FONT_HERSHEY_COMPLEX, 1, (255, 255, 255), 2)
cv2.imshow('Webcam',img)
k=cv2.waitKey(1)
if k==ord('c'):
result = index.query(
top_k=1,
vector=encode_query.tolist(),
include_metadata=True,
)
if result["matches"][0]:
name = result["matches"][0]["metadata"]["name"]
if k==ord('q'):
break
cap.release()
cv2.destroyAllWindows()Result:
This concludes Part 2 of the tutorial. By now, you should have a fully functional real-time face recognition system using your webcam, with face embeddings stored and queried from Pinecone. Leave your comments in case you face any bugs or errors in the code and I will fix them asap.
Stay tuned for more!
