Breakthrough Colourization Technique Enables Instance-Aware Treatment of Multiple Objects
Image colourization is a practical task that automatically maps grey-scale images to plausible colour images. Such techniques enable the addition of natural colour to old family photos or historic images and can bring old films back to life for modern audiences. Although previous work in this area has shown impressive results, most approaches tend to struggle when the input images contain multiple objects, as this presents multiple plausible choices for object colourization and increases complexity and consistency challenges. To address this, researchers from National Tsing Hua University and Virginia Tech have introduced a novel deep learning framework for instance-aware colourization.
The research team proposes that colourization performance can be improved dramatically at the instance level for a few reasons. Learning to colourize instances is easier compared to existing methods that learn to colourize an entire image, which involves handling complex background clutter. Learning object-level representations from localized objects can also help avoid colour confusion with backgrounds.
The proposed method borrows the idea of instance-aware image synthesis and manipulation, providing a clear figure-ground separation and facilitating visual appearance synthesization and manipulation. The proposed instance-aware image colourization method can further advance the process by handling complex scenes with multiple instances and producing spatially coherent colourization results by tackling potentially overlapping instances simultaneously.
Given a grey-scale image X as input, the proposed method uses an off-the-shelf pre-trained model to detect object bounding boxes (Bi), so as to crop out instance Xi via Bi, then uses an instance colourization network trained end-to-end for instance and full-image colourization. To make the instance colours compatible with the predicted background colours, the researchers used a fusion module to fuse all the instance feature maps with the extracted full-image feature map to ensure globally consistent colourization results are obtained.
The results show that the proposed instance-aware model performs favourably when compared with existing state-of-the-art models, highlighting the potential for instance-aware colourization models to improve image colourization performance for complex images with multiple objects.
The paper Instance-aware Image Colorization is on arXiv.
Author: Hecate He | Editor: Michael Sarazen
We know you don’t want to miss any story. Subscribe to our popular Synced Global AI Weekly to get weekly AI updates.
Share Your Research With Synced Review
Share My Research is Synced’s new column that welcomes scholars to share their own research breakthroughs with over 1.5M global AI enthusiasts. Beyond technological advances, Share My Research also calls for interesting stories behind the research and exciting research ideas. Share your research with us by clicking here.
Need a comprehensive review of the past, present and future of modern AI research development? Trends of AI Technology Development Report is out!
2018 Fortune Global 500 Public Company AI Adaptivity Report is out!
Purchase a Kindle-formatted report on Amazon.
Apply for Insight Partner Program to get a complimentary full PDF report.
