QUEUE-O-METER
--
TLDR: I used the fresh published Detectron from Facebook to automatically count the number of people in the canteen’s queue.
A while ago I discovered the URL of the camera in our campus’ canteen.
The camera is located in such way the spectator could check the length of the current queue. There are a couple of thousands of colleagues working in our main campus so it’s useful to understand when it’s time to visit the canteen.
Hence I decided to gather some dataset and build the tool for the automatic queue length measurement to optimize my lunch time.
The camera gives low-quality jpegs, 480x360 pixels, one screenshot per minute. I gathered several weeks of data using the cron-activated curl and tried to process it. The quality of the picture was so low that I couldn’t always count the number of people in the queue even manually. The other problem was picture fluctuations depending on the weather, daytime and light sources:
Initially, I tried to solve the problem by a bunch of dirty heuristics. I collected the set of “empty” (background) pictures, including screenshots taken during non-working time and weekends. Then I selected the small subset of background pictures with the greedy method — maximizing the difference by any pair in the subset and minimizing the distance from any random screenshot to the best of candidates from the subset. Finally, I made the frame processing in such way: I subtracted the closest background candidate from the given frame and calculated the mismatched area in pixels.
Based on a small set manual markup I made the formula which gives the approximation of the queue length based on the mismatched area.
The results were too noisy to use them on the daily basis:
Then, just two weeks ago Facebook AI Research released the sources of Detectron, the packet that implements several state-of-art object detection algorithms and I decided to give it a try. Based on the first runs with the default settings, I found its quality amazing. It doesn’t always found all the objects, but it’s good enough (even just out of the box) to make some approximations:
So I tuned parameters and thresholds a bit and played with the models (all Detectron models are available under the Creative Commons Attribution-ShareAlike 3.0 license). Finally, I made the script to process the given frame and return a list of found objects with probabilities:
— — file: 0203–0950.jpg
dining table 0.71
chair 0.63
chair 0.96
person 0.67
chair 0.64
chair 0.57
bench 0.55
bench 0.68
chair 0.65
person 0.93
Using this script I calculated some historical graphs of the canteen load:
Also, I noticed some interesting objects detections during the history data processing. Here it is the wordle of different objects with sizes proportional to the logarithm of the frequency:
Some of them were clearly the false-positive detections (i.e., the elephant, the frisbee or the toothbrush), but some of the others were very precise spots, actually. These are examples of the book (left) and the cell phone (right) detected in the hands of some person in the queue:
Overall, Detectron seems to be useful in the tasks of such kind.
