Artificial Intelligence & Historical Media
AI Recreates the Past
Synthetic media and how neural networks preserve history.
How AI makes sense of images
Before color photography took off in the 1960s, most photographs existed in black and white. Today, we can appreciate photography from past decades and centuries, but our appreciation for them is limited to the range of colors in which they were developed. More so, photographs don’t last forever: if not preserved properly, their images fade, and with it our clarity of their history and memory blurs. Of course, history is not black and white, and nor should personal and global histories fade from memory, especially not colorful moments in American history.
Technical advances in artificial intelligence and image processing have created new possibilities for restoring dimensions of color and visuality to historical, photographic artifacts. One such class of model is called the Generative Adversarial Network (GAN), made up of two interacting neural networks: the first is called a “generator,” which transforms the input image into the output image; and the second is called a “discriminator,” which evaluates how well the generator created real-looking images. As the neural networks interact, they generate increasingly more realistic images.
The “Hello World” of GANs is mapping images of numbers to their corresponding digits, from 0 to 9.
GAN models have endless applications: you can color landscapes, create cartoon characters, even synthetically age human faces.
These deep learning models also enable the production of what’s called “synthetic media,” media that’s part real and part fake. Synthetic media is the product of taking a real image and transforming it into something virtually new, that didn’t exist before. This affects photography and film, but also music, advertisement, and even journalism!
DeOldify’s neural networks
Advances in deep learning means more deep fakes around social media, but also expand the possibilities for preserving historical artifacts, like photography and film media. A project called DeOldify has become known around Twitter for colorizing and restoring old images and film footage by applying the latest advances in neural networks and deep learning techniques.
DeOldify works on top of a NoGAN model, a technique that applies the colorizing capabilities of GAN models and additional implementations to speed up processing time. While before it would take the software several minutes to colorize and restore photographs, with NoGAN the process takes only a few seconds! The project even offers pre-trained weights for the neural network, making it possible for anyone to train and apply the model to colorize and restore their own photos.
In the words of DeOldify’s creator, Jason Antic:
I basically created a deep learning model to colorize old black and white photos. While this isn’t the first deep learning model to colorize photos, it does a few new things that make it significantly better than previous efforts. Namely, the output is much more colorful and convincing, and this comes from setting up training of the colorizing model to involve a second model — the “critic” — that basically is there to “criticize” the colorizations and teach the “generator” to produce better images. This design is called a GAN — Generative Adversarial Network.
Because the “critic” model is also a neural network, it can pick up on a lot of the nuances of what makes something look “realistic” that simpler methods just can’t. The key here also is that I as a programmer simply cannot comprehend how to explicitly code something to evaluate “realism” — I just don’t know what all that entails. So that’s what a neural network is here to learn for me!
(Source: HackerNoon)
Using neural networks to recreate photographic realism changes our relationship with the past. People living fifty or a hundred years ago didn’t perceive the world in greyscale, they experienced it in color. But the photographs they took that captured those experienced moments did so in black and white. DeOldify leverages the power of neural networks to paint an approximate portrait of the colorful realities of our past.
Little surprise, then, that earlier this year Jason Antic partnered with MyHeritage, a platform that enables people to document their family trees with historical records and their DNA. DeOldify’s algorithms and MyHeritage’s platform allows anyone to upload old family photographs and media, colorizing and restoring these historical artifacts. This makes our perception and experience of history more personal and less distant, especially when viewing restored and colorized pictures of our grandparents and their lived histories.
Testing the image colorizer
We tested a few examples, using DeOldify’s Google-hosted Python notebook, which works as a lightweight interface for running the algorithms: the code takes care of the technical setup, and also pulls data for you to train the neural network. All you have to do is run the model on an image of your choosing. The earlier photo from the Summer of Love was colorized through this hosted code, and so were the pictures below.
Above is a before and after of a 1926 photograph by French photographer Eugène Atget. Depicting the storefront of Paris’ famous Bon Marché, the photo was printed with silver gelatin, which deteriorates and fades over time. DeOldify both preserves the photograph and colorizes a range of details: from color of mannequin clothing, their texture, even the reflection on the storefront vitrine! The neural network is sensitive enough to detect and preserve the glare, and enhance the reflected background details.
Here’s a more colorful restoration of a late ’70s photo depicting two New Yorkers sporting punk fashion in East Village. In the original photo, the viewer can tell that both the girl and boy are fair-haired and wear black clothing. After the restoration, some details gain more depth! Notice their blonde highlights, how the boy’s trousers appear navy blue, and even the weathered details of the granite stoop. DeOldify was also able to restore color to the graffiti and faded posters in the background. A photograph already full of life and emotion provides a deeper window into the past with color.
These are just a few ways that we can apply techniques in neural networks, particularly GANs. The future of AI will open new doors for humanity, including many into our past.
If you enjoyed this article, check out the photos on Twitter that people have colorized with DeOldify: twitter.com/deoldify
Follow BloomAI on Medium for the latest on artificial intelligence, neural networks, and the future of synthetic media: medium.com/bloom-ai-blog
And visit our website if you want to see our own neural networks in action: gobloom.ai
