“The time has come,” the walrus said, “to talk of many things: Of shoes and ships — and sealing wax — of cabbages and kings” — Lewis Carroll, Alice in Wonderland
If you spend a day watching Periscope streams, you’ll certainly see many things, and they’ll probably vary as widely as the walrus’ thoughts. Five alarm fires, Baltimore protest aftershocks, crazy speeders on dashcam in Dubai, beautiful beaches, and so much more. Here at Dextro we fell down the Periscope rabbit hole and quickly found ourselves looking for a flashlight. Live streams generate tremendous amounts of unstructured, raw pixel data in real time; we felt compelled to shed light on the contents and utility of this data.
Though Periscope and Meerkat operate in a nascent market, both live streaming apps have become wildly popular, in the US and internationally. This boon of diverse content sharing opens a window into life around the world, but at times it is difficult to comprehend. For example, people stream videos in many languages, and as a result, text searches of captions have limited value. (None of us on staff read Turkish, but seeing all the cool streams coming out of Turkey has motivated us to learn!) Real time scenes change fairly quickly, making a video’s text-based caption obsolete. (It might start on a selfie captioned “Test,” but then the camera might turn to start streaming their dog’s tricks.)
We quickly realized that the only way to make sense of all this data was via computer vision to analyze the contents of the livestreamed video itself, in real time.
With these principles in mind, Dextro’s development team built Stream, a way to explore everything happening on Periscope. Stream analyzes every public video posted on Periscope in real-time and categorizes content into trending themes. Pulsing bubbles that grow and shrink as streaming trends do show the most prevalent content in real time. While you stand still at your computer, you can catch a glimpse of what’s happening around the world as users post content throughout the day. For the more analytically minded, you can view our aggregate data that depicts popularity of stream topics over the past 24 hours and the past week. At any point on the page, you can click on a trend and see a collection of links to current streams featuring that topic. By applying computer vision to Periscope, Stream solves several major challenges related to cataloging and analyzing video — the rapidly changing nature of live streams, and the infrequent and inconsistent application of user-generated captions — to enhance the rapidly expanding streaming experience. With Stream, our goal is to analyze and access an untapped data source and illuminate the value of its visual content.
Our dev team thought using computer vision to analyze live streams would be an interesting problem to solve, one applicable to myriad use cases. Using Dextro’s computer vision system, we can see what people stream most often, and how it changes throughout the day and across the globe. A few examples come to mind. When Periscope first launched, fridge videos became a surprising new meme. Our data analysis helped monitor that trend and track where the meme was still popular, or not! (Turns out, not.)We also wanted to find a way to better search and to discover new threads.
Stream is the next milestone in the advancement of Dextro’s technology. As a computer vision company, we analyze videos and photos to allow our clients to understand the contents of their datasets without knowing any machine learning themselves. Dextro was the first company to develop and launch video analysis as a service in 2014; since then, companies have used Dextro to automatically understand or curate firehoses of countless hours of pre-recorded videos, moving past spotty or nonexistent metadata to focus on actual visual contents.
Today, we are concurrently launching Dextro’s new livestream API endpoint, which powers Stream. Any company or developer can now tap into the same technology that makes Stream possible to analyze live streamed content themselves. To make it happen, our development team solved key computer vision and GPU optimization problems to pull off the concurrent processing of every single live Periscope stream, upwards of 50,000 per day, in real time with imperceptible delay, demonstrating the scale and performance of Dextro’s system; you can now access all of it with just a REST API call.
Want to learn more? Check out our website.
