Bringing Old Photos Back to Life

Content provided by Bo Zhang, the co-author of the paper Bringing Old Photos Back to Life.

We propose to restore old photos that suffer from severe degradation through a deep learning approach. Unlike conventional restoration tasks that can be solved through supervised learning, the degradation in real photos is complex and the domain gap between synthetic images and real old photos makes the network fail to generalize. Therefore, we propose a novel triplet domain translation network by leveraging real photos along with massive synthetic image pairs. Besides, to address multiple degradations mixed in one old photo, we design a global branch with a partial nonlocal block targeting to the structured defects, such as scratches and dust spots, and a local branch targeting to the unstructured defects, such as noises and blurriness. The proposed method outperforms state-of-the-art methods in terms of visual quality for old photos restoration.

What’s New: Unlike conventional restoration tasks that can be solved through supervised learning, the degradation in real photos is complex and the domain gap between synthetic images and real old photos makes the network fail to generalize. Therefore, we propose a novel triplet domain translation network by leveraging real photos along with massive synthetic image pairs. The domain gap between synthesis data and real old photos is minimized in the latent space so that the learned network generalizes well to real photos.

How It Works: In order to solve the gap between synthetic data and real old photos, we propose a novel triplet domain translation network. Specifically, we train two variational autoencoders (VAEs) to respectively transform old photos and clean photos into two latent spaces. And the translation between these two latent spaces is learned with synthetic paired data. This translation generalizes well to real photos because the domain gap is closed in the compact latent space. Besides, to address multiple degradations mixed in one old photo, we design a global branch with a partial nonlocal block targeting to the structured defects, such as scratches and dust spots, and a local branch targeting to the unstructured defects, such as noises and blurriness. Two branches are fused in the latent space, leading to improved capability to restore old photos from multiple defects.

Key Insights:

• Old photos contain far more complex degradation that is hard to be modelled realistically and there always exists a domain gap between synthetic and real photos. To solve the domain gap issue, we propose a novel triplet domain translation network by leveraging real photos along with massive synthetic image pairs.
• The defects of old photos are a compound of multiple degradations, thus essentially requiring different strategies for restoration. To solve this mixed degradation issue, we propose to enhance the latent restoration network by incorporating a global branch.
• The proposed method outperforms supervised methods and unsupervised translation methods on restoring real photos.

The paper Bringing Old Photos Back to Life is on arXiv. Click here to visit the project website.

Meet the authors Ziyu Wan, Bo Zhang, Dongdong Chen, Pan Zhang, Dong Chen, Jing Liao, Fang Wen from City University of Hong Kong, Microsoft Research Asia, Microsoft Cloud + AI, University of Science and Technology of China.

Microsoft Research Asia (MSRA), Microsoft’s fundamental research arm in the Asia Pacific region and the company’s largest research institute outside the United States, was founded in 1998 in Beijing. Through collaboration with the best talents from Asia and across the globe, MSRA has grown into a world-class research lab, conducting both basic and applied research.

Share Your Research With Synced

Share My Research is Synced’s new column that welcomes scholars to share their own research breakthroughs with over 1.5M global AI enthusiasts. Beyond technological advances, Share My Research also calls for interesting stories behind the research and exciting research ideas. Share your research with us by clicking here.

We know you don’t want to miss any story. Subscribe to our popular Synced Global AI Weekly to get weekly AI updates.

Need a comprehensive review of the past, present and future of modern AI research development? Trends of AI Technology Development Report is out!

2018 Fortune Global 500 Public Company AI Adaptivity Report is out!
Purchase a Kindle-formatted report on Amazon.
Apply for Insight Partner Program to get a complimentary full PDF report.