Aleksey Tikhonov
Feb 5, 2018 · 4 min read

TLDR: I used the fresh published Detectron from Facebook to automatically count the number of people in the canteen’s queue.

A while ago I discovered the URL of the camera in our campus’ canteen.
The camera is located in such way the spectator could check the length of the current queue. There are a couple of thousands of colleagues working in our main campus so it’s useful to understand when it’s time to visit the canteen.
Hence I decided to gather some dataset and build the tool for the automatic queue length measurement to optimize my lunch time.

The camera gives low-quality jpegs, 480x360 pixels, one screenshot per minute. I gathered several weeks of data using the cron-activated curl and tried to process it. The quality of the picture was so low that I couldn’t always count the number of people in the queue even manually. The other problem was picture fluctuations depending on the weather, daytime and light sources:

Image for post
Image for post

Initially, I tried to solve the problem by a bunch of dirty heuristics. I collected the set of “empty” (background) pictures, including screenshots taken during non-working time and weekends. Then I selected the small subset of background pictures with the greedy method — maximizing the difference by any pair in the subset and minimizing the distance from any random screenshot to the best of candidates from the subset. Finally, I made the frame processing in such way: I subtracted the closest background candidate from the given frame and calculated the mismatched area in pixels.

Based on a small set manual markup I made the formula which gives the approximation of the queue length based on the mismatched area.
The results were too noisy to use them on the daily basis:

Image for post
Image for post

Then, just two weeks ago Facebook AI Research released the sources of Detectron, the packet that implements several state-of-art object detection algorithms and I decided to give it a try. Based on the first runs with the default settings, I found its quality amazing. It doesn’t always found all the objects, but it’s good enough (even just out of the box) to make some approximations:

Image for post
Image for post

So I tuned parameters and thresholds a bit and played with the models (all Detectron models are available under the Creative Commons Attribution-ShareAlike 3.0 license). Finally, I made the script to process the given frame and return a list of found objects with probabilities:

— — file: 0203–0950.jpg
dining table 0.71
chair 0.63
chair 0.96
person 0.67
chair 0.64
chair 0.57
bench 0.55
bench 0.68
chair 0.65
person 0.93

Using this script I calculated some historical graphs of the canteen load:

Image for post
Image for post
Image for post
Image for post

Also, I noticed some interesting objects detections during the history data processing. Here it is the wordle of different objects with sizes proportional to the logarithm of the frequency:

Image for post
Image for post

Some of them were clearly the false-positive detections (i.e., the elephant, the frisbee or the toothbrush), but some of the others were very precise spots, actually. These are examples of the book (left) and the cell phone (right) detected in the hands of some person in the queue:

Image for post
Image for post

Overall, Detectron seems to be useful in the tasks of such kind.

Altsoph’s blog

Random notes on people and machines

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store