How Can AI Make Old Videos Look Smoother?

Video frame interpolation has come a long way and AI is taking it in some strange directions

It was back in 2015 and I was creating some data visualizations for a meeting with my PhD supervisory committee. This was towards the end of my graduate studies in astrophysics and I had run some supercomputer simulations of a massive star forming inside an interstellar cloud of gas.

I figured the best way to show off the results of these simulations was to create various animations of the data. My analysis code had chewed through gigabytes of simulation data and dumped image files at specific time intervals. So I used ffmpeg to stitch them together into an animation:

Video of an astrophysics simulation showing the evolution of a protostellar disk around a forming star. Author’s own work. See Klassen et al. (2016).

I always wanted higher frame rates, so I could have smoother and more beautiful animation, but I was limited by the data I had from my simulation.

Sometimes I would spend a few hours seeing if I could perhaps generate intermediate frames to make my movies appear smoother when I compiled them. My attempts at the time were clumsy and ultimately abandoned, but today there are many techniques that work very well. In fact, most modern TVs include the option (on by default) to artificially enhance the frame rate of the video.