Demystifying Clearview AI: Downloading the Code and Data

Demystifying Clearview AI Blog Series (Code and Data)

Published in

Analytics Vidhya

4 min readJan 2, 2021

With Conda and Git installed, you can easily download and run the ResNet and DenseNet deep learning models used in CCTView directly from your computer on real-life CCTV footage.

You can also quickly use a Jupyter notebook to experiment with different models, vehicle matching algorithms, or perform your own image processing.

The repository github.com/samdbrice/cctview contains fully functional demos of the models, code, and other data referenced in the series.

Cloning and Instaling from GitHub

To run the example models, code, and notebooks clone the repository from GitHub, then create a new Conda environment based on the included environment.yml.

Requirements:

# (1) Clone the repository and data...
git lfs install && git clone https://github.com/samdbrice/cctview.git 
# (2) Install and activate environment
cd cctview
conda env create -n cctview -f environment.yml
conda activate cctview

Note: Git Large File Storage (LFS) is required to download the models state files and the inlinedata files.

Running the Example Script

After installing and activating the included environment, you can run the code/example.py script from your machine with pythonw.

Note: pythonw will automatically be on your path. It's a wrapper for the python command designed to get matplotlib working correctly cross-platform.

pythonw code/example.py

The example script uses images from the examples directory to demonstrate the three core modules within the cctview code package:

Playing with Jupyter Notebook

After installing and activating the included environment, you can start a Jupyter Notebook server by running:

jupyter notebook

From within Jupyter navigate to the notebooks the directory then open the Working With Sample Data.ipynb file.

Detect Module — Residual Network

Object detection is implemented using the ImageAI RetinaNet type model.

The underlying model is loaded with weights from models/resnet50_coco_best_v2.0.1.h5 based on a 50-layer ResNet trained on the COCO dataset.

Extract Module — Dense Convolutional Network

Features from the objects detected using the RetinaNet model are extracted using the second-to-last layer of a DenseNet CNN.

PyTorch is used to load weights from models/VeRI_densenet_ft50.pth based on a Densenet201 architecture, tuned using the VeRI dataset.

Match Module — Euclidian Distance Metric

As discussed earlier in the series, an ideal matching and ReID algorithm would be based on Track-to-Track ranking. The Match module demonstrates a rudimentary matching implementation based on the minimal Euclidean distance between two points.

The features extracted into features.json represent points based on Cartesian coordinates in n-dimensional Euclidian space, the distance formula is:

The matcher calculates distances from all objects within a base frame to all objects within a list of target frames. It also groups images for each base object and their candidates into a new directory called matches.

Raw Footage, Detections, and Features Data

The codebase contains a small subset of the preprocessed data zipped inline within the data directory.

data
├── [8.6M]  253-134-S.zip
├── [ 20M]  689-640-S.zip
├── [ 43M]  731-679-N.zip
├── [ 51M]  732-680-N.zip
├── [8.1M]  733-681-N.zip
└── [ 22M]  735-683-N.zip

The full dataset for each FDR camera, including detections and feature extractions (approximately 10 GB) are stored within the following external repositories:

Raw CCTV Footage

The repository github.com/samdbrice/cctview-data-videos — hourly separately contains raw hourly CCTV footage from Tuesday, May 5th, 2020, from 1 PM to midnight.