Why Instagram’s new Hyperlapse app makes me glad I own Facebook stock

For those not aware, Instagram released a new app called Hyperlapse that allows you to take really long videos, stabilize them, and speed them up. It basically allows you to turn ordinary videos into timelapse videos. Or at least that seems to be the primary way that they are marketing it. For an example of the app in action, check out the clip below:

As seems to be par for the course at this point, I’m sure most people hear this news and think, “Oh great, another app,” and, “Why not just include this feature inside Instagram?” The answer to both these questions, in my opinion, is that this app is going to be a much bigger deal than people think. To explain why, I have to delve into the technology the app uses just a little bit. Once I learned more about how it works I was actually pretty surprised, as I alluded to earlier, that they are marketing this as a timelapse app. This is because the most amazing feature of the app has very little to do with timelapses. That feature is the clever new image stabilization they developed for it.

Before this technique there were basically three ways of getting stable video: 1. Expensive hardware like a steadicam, 2. hardware like optical image stabilization that can really only handle very minor shakiness, and 3. software stabilization like Adobe’s Warp Stabilizer or Final Cut’s image stabilization. The problems with the first two are obvious, and the problem with the third is that this software is very computationally expensive (i.e. it’s hard for computers and eats up a lot of their resources), it takes a long time to process the video, and the results aren’t that great anyway. The more shakiness in the original video, the more distortion and weird effects are going to be in the final product. Basically, it can only handle a little more than optical image stabilization for creating desirable looking video.

So what is this magical new technology? Well, it’s actually very simple, and could have been implented years ago! Like so many innovations, it was not discovered by some super genius, it was discovered as a consequence of the constraints facing the team building the feature. They were working with smartphones which both limited the computational power available and the time which users were willing to dedicate to an app. No one’s going to wait around for a half hour to wait for their footage to be stabilized. But smartphones do have something that most cameras do not: gyroscopes. Gyroscopes measure the position of the camera at all times. Think of it like a marble covered in ink inside a bowl. Move the bowl and the marble moves in the opposite direction. Move the marble in a different direction and the marble moves in the opposite of that direction. If you look at the ink path made by the ball it’s very easy to determine how the bowl moved. To extend the analogy, the old methods of stabilizing video would be like determining the position of the bowl constantly filming it, tracking it using complicated algorithms that can recognize objects in videos, set track points, monitor how those points move, etc. Whereas Hyperlapse’s approach is just looking at those ink paths left by the ball inside the bowl. Much easier. Hyperlapse looks at the data generated by the internal gyroscope then goes frame by frame through the video and just shifts the images up, down, left, right, solely based on the data generated by the gyroscope. If the gyroscope knows that at X time the camera moved vertically .003 inches. All Hyperlapse has to do is move the image taken at time X down .oo3 inches to match that image up with the previous image. Now, this is an oversimplification and I’m sure there are some pretty complicated alogorithms at work, but the proof is in the pudding: the feature seems to work great on smartphones.

So why is this important? After all, how many people really care about stable video? Well, first I don’t think anyone actually likes their videos to be shaky, but this is also a big deal because stable shots are very important for filming and are currently very expensive to achieve. The very best method is still to use an expensive steadicam rig. This provides a barrier to entry for smaller filmmakers and independent video content producers like the people on youtube. The beauty of Hyperlapse’s solution is that it relies on very cheap, and very small, hardware: gyroscopes. If this method can work on a smarthphone, that means that it could work on any digital camera that has a gyroscope and a CPU (pretty sure all digital cameras have CPUs at this point). Basically all Canon, Nikon, Red, Sony, etc. have to do is put gyroscopes into their cameras and videographers will have the ability to take steadicam-like shots basically free. Plus, who wants their video to be all shaky?

However, there is an even bigger reason why I believe that Hyperlapse is a big deal. The advent of smartphones (and the digital cameras they house) is that the sheer amount of footage we have accumulated has gone up an order of magnitude. We’re all becoming digital horders with hundreds, thousands of minutes of footage we never even watch, and this problem will only grow the more wearable computers and cameras become. For example, assuming no macro-level disruptions to the economy and the world, in 5 to 10 years (maybe sooner) we will have unobtrusive head mounted cameras (probably in some form of eye-wear that does not look like Google Glass) capable of constantly recording the wearer’s POV. That will lead to yet another order of magnitude increase in the amount of video footage generated by individuals. How are we going to deal with all of this video? There will have to be a solution to this problem. One part of this is developing programs and algorithms that go through your footage automatically searching for “good” footage, i.e. footage you will want to see again, for example, by using speech recognition to look for markers like people laughing, multiple people speaking at the same time (a conversation), utilizing GPS and other contextual data to determine if someone is in a new location or in transit, etc.

Hyperlapse is going to be positioned extremely well to be the solution, or part of the solution, to these problems. First off, it enables you to view massive amounts of footage, quickly and in an aesthetically pleasing manner. For example, in a week I’m heading off to Iceland. Imagine it’s 5–10 years from now and Hyperlapse is now a mature and feature-rich application. I’m also wearing Apple iGlass, normal looking glasses that house a discreet (nearly invisible) camera which is constantly recording and storing the video wirelessly on my smartphone (which at this point has over 200 gigs of storage). Maybe the glasses don’t constantly record, but, hey, I’m going on vacation, time to open up the Hyperlapse app and set my glasses to constantly record. Or maybe Google Now/Siri are integrated into Hyperlapse so that the app knows that I am taking a flight (either because I did a search with the flight info or because it scraped the info from my email). Getting the app into “vacation-mode” either manually or automatically should not be a problem. Hyperlapse looks at GPS data and knows that I just drove to the airport: the vacation has begun. It uses the microphone embedded in either my phone or my glasses to determine that nothing really happened on the ride there. No one said much, no one laughed, etc. Nothing happened, but a trip to the airport can be a fun thing to include in a travel video, so Hyperlapse takes that video, stabilizes it and speeds it up by 10 times. While on the plane I have a funny conversation with my travel companion. Hyperlapse recognizes laughter, assumes that this is a funny conversation, uses speech recognition to ascertain when the conversation began (i.e. when people started talking), stabilizes the video and doesn’t speed it up so as to preserve the funny conversation, then adds this video the previous video. The plane lands and again GPS is used to determine that I’m not in any of my usual locations, I’m in Iceland. Clearly the vacation has begun. Hyperlapse can go into “vacation-mode” making sure to err on the side of caution and preserve as much video as possible—hyperlapsing all the video where nothing much happens (e.g. while the user is traveling for long periods of time on a road), preserving and stabilizing video that the algorithm marks as important. In addition, if I see something awe-inspiring that I want to make extra sure I have recorded, I just press a button on my glasses or phone (or I might have to open the app itself, but this seems like a clunky solution) and the Hyperlapse knows to include to add this video to the previous ones. When I have access to wi-fi the app can automatically start offloading video (along with the data necessary for Hyperlapse to know what to do with the video) to the cloud so as to free up space locally on my portable devices.

Proceeding in this manner, one can imagine going on vacation and returning home to find a totally complete, edited, and aesthetically pleasing video of their trip waiting for them on the Hyperlapse app! This could even happen with the user doing nothing on their end. I believe that would be an enormously valuable tool, and not unimportantly, a tool I would gladly pay for (unlike Instagram, Facebook, etc.). Maybe they give me a free 5 minute video, but I have to pay for the full half hour video. An arrangement like that would be extremely compelling. Taking it another step further, at some point the Hyperlapse videos should retain all the original data from video. In other words, just because the video has been Hyperlapsed doesn’t mean you can’t still go back and stretch out any portion of the video that you want to see at regular speed. This could be especially useful as a memory supplement. What was I doing exactly a year ago? Well let me see, I call up the Hyperlapse app, tell it to go to exactly a year ago today and start showing me video. If anything is marked as important, for whatever reason, it can start with that, or just start hyperlapsing through my day at 10 times, 100 times, 1000 times speed. When I see something of interest I can tell the app to stop and show me the video in normal speed. In seconds I can know exactly what I was doing at any time in the past! There could also be benefits to law enforcement for having so much video so accessible.

UPDATE: Some user-generated examples of Hyperlapses: http://www.theverge.com/2014/8/28/6075803/25-of-the-most-amazing-hyperlapse-videos-weve-seen-so-far