Member-only story

Deep Video Inpainting

Removing unwanted objects from videos with deep neural networks. Problem set up and state-of-the-art review.

Aliaksei Mikhailiuk
Towards Data Science
14 min readMar 23, 2022

--

Image by Author.

While the technology for wiping out memories, as in the famous movie Eternal sunshine of the spotless mind”, is not there (yet), we are making good progress with images and videos. Wouldn’t it be cool to remove this random guy with a beer, ruining a stunning sunset, from a holiday video?

Video-inpainting enables us to mask unwanted parts of the video. Some time ago this job would have taken animators and graphic designers hundreds of hours of manual video editing — frame by frame.

But with the advent of machine learning we can often achieve results if not better, but certainly very close to what artists are capable of — replacing unwanted objects with content that would seamlessly fit into the video.

Just check out a small snippet below, isn’t it impressive?

Video produced with Flow Guided Video Completion method (FGVC). GIF by Chen Gao.

To me the above animation looks incredible! Notice, for example, how high frequency textures of the grass in foreground of the dancing girl are preserved. To see how we got to these results, below I set…

--

--

Towards Data Science
Towards Data Science

Published in Towards Data Science

Your home for data science and AI. The world’s leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.

Aliaksei Mikhailiuk
Aliaksei Mikhailiuk

Written by Aliaksei Mikhailiuk

Tech Lead Manager at Snap. Ex-AI Team Lead at Huawei, PhD from University of Cambridge https://www.linkedin.com/in/aliakseimikhailiuk/

Responses (1)