Member-only story
Deep Video Inpainting
Removing unwanted objects from videos with deep neural networks. Problem set up and state-of-the-art review.
While the technology for wiping out memories, as in the famous movie “Eternal sunshine of the spotless mind”, is not there (yet), we are making good progress with images and videos. Wouldn’t it be cool to remove this random guy with a beer, ruining a stunning sunset, from a holiday video?
Video-inpainting enables us to mask unwanted parts of the video. Some time ago this job would have taken animators and graphic designers hundreds of hours of manual video editing — frame by frame.
But with the advent of machine learning we can often achieve results if not better, but certainly very close to what artists are capable of — replacing unwanted objects with content that would seamlessly fit into the video.
Just check out a small snippet below, isn’t it impressive?
To me the above animation looks incredible! Notice, for example, how high frequency textures of the grass in foreground of the dancing girl are preserved. To see how we got to these results, below I set…