Bringing DeOldify to life using Clouderizer
Transforming the black & white images from the past with GANs.
A little about the project ( Credits : Jantic)
The goal of this model is to colourise the black and white images based on its interpretation of how the scenario might have been while clicking the picture.
Overview of GANs:
Generative adversarial networks (GANs) are deep neural net architectures comprised of two nets namely Generator and Discriminator(aka Critic), trained one against the other (thus the word “adversarial”).
Typically, the generator is of main interest — the discriminator is an adaptive loss function that gets discarded once the generator has been trained.
We can think of GANs as a competition between 2 networks. Generative model tries to fool the discriminator while discriminator acts as a detective trying to catch the forgers. Since we don’t have a predefined function to determine which images are real/fake we use the discriminator model. It is usually dispensable after generator model is trained.
The standard objective function for adversarial loss:
Steps to start working with the model:(With Screenshots of the Process)
1. Go to the Clouderizer Community.(Or click here!)
2. Clone the DeOldify project.
Note: At this point, you will be asked to signup if you are not already a user.
3. In this section, you can change/add new packages while setting up the project but in case of DeOldify we have already configured it for you.
4. After Cloning, you can directly start the project using Colab(or other backend systems like GCP, AWS, Kaggle or any Ubuntu machine).
Note : Connect your account to Google drive to start the project. We save the project files in your Google Drive and pull from there when the project starts(we care about your privacy!).
5. To try Clouderizer serving(Completely Optional!), you need to enable it in the Project Configuration → Remote Access → Enable Clouderizer Serve.
Once you enable it and restart the project, you will find app.py in code/ folder. This is the flask server that we are using for model inference.
There are a few steps we need to follow in remote terminal to get it working:
1. We need to copy app.py from code/ to serve/ folder. cp -R ~/clouderizer/deoldify/code/app.py ~/clouderizer/deoldify/serve/ .
2. We are running a tmux session(don’t worry about it) for the serve part already. Runtmux attach -t se to look at the flask server output. The model will take ~5 mins to load(GANs!) and you can see “model loading is done” message in the tmux session itself. That’s it upload images and get output.
To tinker with DeOldify click on the Jupyter button once the project is running and go to the code/ folder. This is a workspace/IDE which contains all the files & folders used in the project.
In DeOldify, we have 2 important Jupyter notebooks to look out for: ColorizeTraining.ipynb → for training the model.
ColorizeVisualization.ipynb → for model inference.
We have put in the weights colorize_gen_192.h5 which are used in ColorizeVisualization notebook(for those who don’t want to train the model).
We can get the training data from ImageNet and provide appropriate path in ColorizeTraining. But we have added a small dataset(~1 GB) chose randomly from coco datasets which will help in the training process.
Try images from https://www.reddit.com/r/TheWayWeWere/ for inference.
To know the technical details & explanation about the code : here’s my part 2 explaining GANs with code.
Conclusion:
Clouderizer takes care of the DevOps and let’s you focus on your project.
We would love to hear what you think! You can post all the issues you faced while with this project in the comments section.
