Live Video Goes Interactive
Hi there! My name is Sarah and I work at a startup called Muxy. Most people I talk to in my daily life haven’t heard of us and don’t know what we do (#startuplife in a nutshell). As a result, I’ve gotten pretty good at explaining what Muxy does to outsiders, which is what I will attempt to do in this post. Specifically, I will be talking through everything you need to know to understand interactive live video applications, which we at Muxy are calling ILV apps for short.
A Brief(ish) History of Interactivity in Live Video
The concept of adding some kind of interactivity to video has been around for a while and, depending on your definition of interactivity, already exists in a variety of forms. A ticker of relevant tweets on cable TV is a way for the viewing audience to interact with the live broadcast, albeit in a heavily moderated way. American Idol is interactive live TV, with viewers casting votes via text to influence the outcome of the show. I remember seeing SMS-based polls during commercial breaks on MTV a decade ago, with the poll results being shown during the next commercial break.
The advent of smartphones added a whole new universe of possibilities to the idea of live broadcast interactivity. An SMS-based system relies on an audience to send one or two word text messages at most a few times during a program. Smartphones, though, provide the opportunity for companies and networks to build apps that connect directly with broadcasts, enabling a nearly infinite amount and variety of approved interactions. However, we haven’t seen too much of this happening — which I have been told by engineers is because mobile app development is difficult and costly, especially for an unproven idea that might amount to nothing. Undertaking a project like that is a big, expensive risk with an uncertain payoff that TV networks and other traditional live content producers haven’t been interested in taking.
The story of interactivity in live content might have ended there, but the live content market has experienced some radical changes in recent years. The advent of internet-based live streaming platforms like Periscope, Facebook Live, YouTube Live, Beam, Mixer, and Twitch has created a new landscape of decidedly nontraditional independent live content creators who stream live video of themselves doing everything from playing video games to creating art to living their daily lives. Unlike their TV network predecessors, these new live content platforms have been willing to experiment and take risks on new kinds of interactive content… which brings us to the launch of Twitch Extensions.
Twitch envisioned Extensions as a new frontier of broadcaster/audience interaction, giving developers the opportunity to run their own code on top of Twitch’s website to provide both viewers and streamers with new ways to interact and communicate that couldn’t be provided by the existing chat system. Extensions on Twitch officially launched on August 31, 2017 with extensions that included OP.gg for League of Legends by OP.gg, Innkeeper: Interactive Hearth Overlay by Curse, and MasterOverwatch by Master Network — as well as Overlay and Leaderboard from the team here at Muxy.
Since their launch, Extensions have emerged as a major differentiating feature for Twitch. Extensions have become a big draw for the platform, attracting viewers and broadcasters excited to engage with each other in creative ways as well as third party companies who want to find a home for their live content that has a fresh angle. Adding interactivity to live video is enticing to marketers and advertisers, who are always looking for new ways to grow their brand engagement in a world where ad block usage is constantly increasing. In addition to the branded extensions that have been growing in popularity and number, there are still plenty of developers building their own extensions that enhance popular streaming games or are minigames in themselves.
At this point, Twitch Extensions are the only ILV app platform in mainstream use. With the remarkable success that the program has had, however, we anticipate that this will change in the near future. The concept of ILV apps is here to stay and it will only become more prevalent.
The Concept
The basic concept of an ILV app is relatively simple: it is a piece of interactive, clickable software that runs on top of a live video stream. When I say “on top”, I mean that in the literal sense: overlay ILV apps literally appear layered on top of the video player.
Although it may sound like adding an additional layer on top of a video feed distracts viewers and obscures parts of the video,when it comes to Twitch, Extensions are in fact a valuable tool for reducing visual clutter. The typical Twitch stream layout includes any number of brightly colored popups and panels all over the video feed, which usually includes its own busy video game interface, and it can be hard to figure out where to look. A well-designed Extension can help reduce on-screen chaos and confusion by consolidating static on-screen graphics and information to a new interactive interface that a viewer can explore at their leisure.
ILV apps can provide the interactivity smartphones have taught our minds to crave in a way that complements the on-screen content rather than competing with it.
Another way we have seen well-executed Extensions reduce viewer distraction is by adding available information right in the feed that would normally require use of a second screen like another browser tab or a smarphone. One of the analogies we use all the time here at Muxy is that ILV apps are essentially on-screen smartphone apps. ILV apps can provide the interactivity smartphones have taught our minds to crave in a way that complements the on-screen content rather than competing with it.
What Muxy Has Built
If all of this seems somewhat abstract, that’s because it is — so far. Now, let’s walk through some of the ILV apps Muxy has built to further explain and illustrate the power of the concept.
Heat Map
Heat Map is one of the apps contained in Muxy’s Omni Overlay, one of the first Twitch Extensions our company built. All the apps in the Omni Overlay were designed as demos to get people excited about what extensions were capable of doing, and Heat Map was very successful at that task. Each dot of “heat” in the screenshot above represents one person clicking on that location on the video player at home. In this particular example, the viewers of Twitch broadcaster GiantWaffle worked together to click and make a giant heart.
Although making shapes is cool, my favorite use of this tech is when broadcasters tell viewers to click to indicate the location of an on-screen item (think crowdsourcing a game of “Where’s Waldo”).
NBA G League
Another category of ILV app we’ve found particularly interesting is live sports. With Muxy’s NBA G League extension, we worked closely with the NBA and Twitch to create an interface that provides viewers with a wealth of information during G League broadcasts on Twitch. Viewers can explore in-depth team and player statistics, make predictions about game outcomes, and even participate in a micro-fantasy system where they “boost” a player they think will play well and earn points when their chosen player scores baskets, rebounds, or assists.
Muxy has also developed a similar Twitch Extension for the Overwatch League (OWL), a competitive e-sports league owned by Blizzard.
The Game Awards
One of Muxy’s earliest bespoke ILV app projects, the prediction polling game we built for The Game Awards, allowed Twitch viewers to vote on who they thought would win awards at The Game Awards 2017. Channels co-streaming the event competed against each other to see who was best at predicting winners, with the channel that won earning a special Twitch emote usable in chat across the site.
We found that the interactivity and competitive spirit fostered by this extension had a profound effect on viewer retention and engagement, with an astonishing 70% interaction rate with the Extension among Twitch viewers. You can read more in one of Twitch’s blog posts, but perhaps the best indicator of this project’s popularity and success was Muxy’s subsequent licensing of the same type of Extension to the Crunchy Roll Anime Awards and the SXSW Gaming Awards for their event streams.
The Future Is Bright
Although building custom ILV apps for clients will continue to be part of Muxy’s business model in the future, the Muxy team is thinking even bigger with our next project. We love ILV apps and we want lots of other people with great ideas to start building them, which is why our team of world-class software engineers have created Muxy’s SDK. Our SDK lets any developer borrow Muxy’s existing back end architecture to build ILV apps effortlessly, reducing the required development time for new application to weeks or days rather than months.
Especially now that Twitch has launched bits-based Extension monetization, the ILV app landscape is primed for an explosion of new and exciting ways to make live video interactive and fun. Watching a new technology evolve is always interesting, and it’s totally possible that the best application for extension technology hasn’t even been imagined yet. When you use Muxy’s SDK, you don’t have to be an experienced back end developer to build a great extension. Personally, I can’t wait to see what innovative ideas will come to life now that ILV app development is accessible to a wider audience.
