Using ML to Analyze the Office Best Scene (Emotion Detection)

Daniel Rojas Ugalde
Jan 5 · 4 min read

The Office is my favorite show. I always have a great laugh and I have even learned a thing or two from Michael Scott's leadership style. The show portrays a lot of funny situations and it’s always a blast to watch it. Looks like it has been removed from Netflix this January 1st, I wanted to write a little post about it.

My favorite clip ever is the intro from Stress Relief (Fire Drill). I find it almost impossible to watch it and not laugh. It has brightened my day a lot of times even on some pretty bad days. If you have not already, I recommend you watch it right now. Based on some conversations with my mentor (thanks Oscar!) this is the best episode on TV ever.

I wanted to leverage Machine Learning to analyze this piece of art. I wanted to understand the different facial expressions and maybe find the magic touch that makes everything worked. Instead of building something from scratch, I leveraged a very cool library I found from justinshenk. Please check this library in Github and feel free to star it. The library is very easy to use and abstracts all the complexities.

There is a very simple example in the documentation that can help you analyze a video and get some very cool outputs. The following line is the one you might want to play a little bit with. I disabled display and zip images to make it run faster. You can also fine-tune the frequency of frames you want to analyze. By default is 1, it will analyze every single frame. In this case I run the test with 50.

raw_data = video.analyze(detector, display=False,zip_images=False,frequency=50)

The library will generate an output with all the images and the bounding boxes. It’s very useful if you want to review specific frames.

It will also generate a plot of different emotions. It has some emotions by default. You could eventually adjust the model and train it for additional emotions.

Given that the plot above might be a little confusing. I selected some specific emotions that I thought would be interesting to analyze.

If you watched the clip, this plot might make sense to you. Based on the context it’s reasonable that a lot of the emotions on the scene are sad, surprise, and angry. I also saw a peak of happiness which does not make sense, let’s take a closer look.

Looking at the frames, I found this image. Even though it’s easy for a human to tell that she is not happy, by the teeth and the expression it’s also possible that a machine can be confused. This can happen from time to time, this is the kind of data you might want to get rid of.

The library outputs a data frame with all the results. This is very useful if you need to do additional calculations. I applied a simple mean. We can see that Angry and Sad are the top emotions. Makes sense based on what is going on.

angry       0.248986
disgust 0.008261
fear 0.133043
happy 0.136667
sad 0.234493
surprise 0.041449
neutral 0.197246

You can find the code here.

If you have any questions/comments, feel free to reach me on Twitter.

The Startup

Get smarter at building your thing. Join The Startup’s +799K followers.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store