Creating a Web App to Colorize Images and YouTube Videos

A step-by-step tutorial to build and deploy a DL web app on Hugging Face with Streamlit

Photo by Greyson Joralemon on Unsplash

Have you ever found hidden gems on GitHub? More and more scientific papers are making their code and models available in open source, but they don’t offer interfaces so that a lambda user can make use of their discovery.

In this article, I’ll take the example of colorizing black and white images with a GitHub repo that contains two pre-trained models for this task and use them to illustrate how to deploy a model for free online with Streamlit and Hugging Face.

After following this step-by-step tutorial, you’ll be able to deploy any machine learning model of your choice for free, with a pleasant interface, as shown in the GIF below.

Final application of this tutorial — GIF by author

The GitHub code for this tutorial is available here. The Web application is available here.

Table of Contents:

• Part 1: Create the Hugging Face Space to host the application
• Part 2: Build the Web Application with Streamlit
• Part 3: Add Multiple Pages to the Application

Part 1: Create the Hugging Face Space to host the application

Let’s get started! Head to Hugging Face and create a new space.

Screenshot of the Hugging Face interface for creating a Space — Image by author

• Choose the Space name of your choice
• Choose the License of your choice
• Select the Streamlit Space SDK as shown in the screenshot above
• Select the Space hardware of your choice (keep the default one for free access)

When done, click on the “Create Space” button. You should arrive on a page similar to the one below.

Screenshot of the initial Space interface — Image by author

For the moment, our application is empty, but it’s up to us to complete it! Let’s move on to the second part of this tutorial

Part 2: Build the Web Application with Streamlit

This part of the tutorial is split into the following subsections.

• Creating app.py file
• Interface premises
• User Inputs for models & YouTube videos
• Colorizing the video

1. Creating app.py file

To build our application, we start by cloning the repository created by Hugging Face. To do this, copy the git command given by Hugging Face, which will vary according to your username and the name of your Space. If you don’t have git installed on your computer you can also upload the files manually.

git clone https://huggingface.co/spaces/Username/space_name

Once copied locally, you can begin development. I’m going to develop the application using PyCharm, but any other IDE will do. However, I’d advise you to create a virtual environment so that you can more easily retrieve the list of packages needed for the application to facilitate deployment. If you’re not familiar with those, I’d suggest you take a look at this article before continuing the tutorial.

A Data Scientists Guide to Python Virtual Environments

I told you about GitHub’s hidden gems at the beginning of this article. Mine is a repo from Richard Zhang where he shared his work and that of his co-authors on automatic image colorization using CNN [1] [2]. The aim of this article isn’t to go into the details of how these models work. If you’re interested, I suggest you read the two articles referenced at the end of this article or check the GitHub repo where model weights and code snippets for using them are available.

Our streamlit application is divided into 3 pages.

1. Colorizing videos
2. Colorizing YouTube videos
3. Colorizing images

The page structure is the same for all of them. Only a few details change, depending on the input formats. In this tutorial, we’ll focus on the colorization page for a YouTube video and how to handle multiple pages. The other pages are available on my GitHub repo.

Start by creating a Python file. Let’s call it app.py, which will facilitate deployment.

2. Interface premises

Next, we need to present our application and make it attractive. What better way to do this than with a nice animation?

Application overview — GIF by author

Such animations are available with the Lottie library, which parses Adobe After Effect animations. I’ve chosen the color palette for my application, but you can find an infinite number of others here. Here’s the code to create a similar interface.

pip install streamlit
pip install streamlit-lottie

import streamlit as st
import requests
from streamlit_lottie import st_lottie

# Define a function that we can use to load lottie files from a link.
@st.cache_data()
def load_lottieurl(url: str):
    r = requests.get(url)
    if r.status_code != 200:
        return None
    return r.json()

st.set_page_config(page_title="Image & Video Colorizer", page_icon="🎨", layout="wide")

col1, col2 = st.columns([1, 3])
with col1:
    lottie = load_lottieurl("https://assets5.lottiefiles.com/packages/lf20_RHdEuzVfEL.json")
    st_lottie(lottie)

with col2:
    st.write("""
    ## B&W Videos Colorizer
    ##### Input a YouTube black and white video link and get a colorized version of it.
    ###### ➠ This space is using CPU Basic so it might take a while to colorize a video.
    ###### ➠ If you want more models and GPU available please support this space by donating.""")

• Import the necessary libraries
• Create a function to load lottie’s files and cache them
• Set page config with title, icon, and layout
• Divide the page into two columns with different widths and put the text

If you want to see the result live, you can run streamlit locally:

streamlit run app.py

Or you can view your application directly on Hugging Face. To do this, you need to save the necessary libraries and push your files.

pip freeze > requirements.txt

git add .
git commit -m "Add requirements.txt and other files"
git push

And voila! You can see your application on Hugging Face now.

3. User Inputs for models & YouTube videos

Now let’s take a look at how to make the user choose a model and enter a link to a YouTube video to be colorized.

To give the user a choice, we’ll use a st.selectbox.

model = st.selectbox(
        "Select Model (Both models have their pros and cons, I recommend trying both and keeping the best for your task)",
        ["ECCV16", "SIGGRAPH17"], index=0)

The modelvariable takes the value “ECCV16” or “SIGGRAPH17”. Depending on the user’s choice, one model or another is loaded.

@st.cache_resource()
def change_model(current_model, model):
    if current_model != model:
        if model == "ECCV16":
            loaded_model = eccv16(pretrained=True).eval()
        elif model == "SIGGRAPH17":
            loaded_model = siggraph17(pretrained=True).eval()
        return loaded_model
    else:
        raise Exception("Model is the same as the current one.")

loaded_model = change_model(current_model, model)
st.write(f"Model is now {model}")

Model Selection — GIF by the author

Now let’s create a st.text_input for the user to enter a YouTube link and a st.button to start the colorization.

link = st.text_input("YouTube Link (The longer the video, the longer the processing time)")
if st.button("Colorize"):
    # Add code to handle user inputs

Model Selection + YouTube link — GIF by author

4. Colorizing the video

My hidden gem can colorize images but not videos. How can we adapt the model? Well, we’ll need to download the YouTube video entered by the user, colorize it frame by frame and render a new video.

• Download YouTube video

To download the video, we’ll use the Pytube library.

pip install pytube

from pytube import YouTube

# [...]

@st.cache_data()
def download_video(link):
    yt = YouTube(link)
    video = yt.streams.filter(progressive=True, file_extension='mp4').order_by('resolution').desc().first().download(filename="video.mp4")
    return video

This function will download the YouTube video entered by the user and save it locally in a file named video.mp4.

Depending on the length of the video, it may take a long time to colorize. So, let’s start by displaying on the left column the video entered by the user using st.video.

if st.button("Colorize"):
    yt_video = download_video(link)
    col1, col2 = st.columns([0.5, 0.5])
    with col1:
        st.markdown('<p style="text-align: center;">Before</p>', unsafe_allow_html=True)
        st.video(yt_video)

• Colorizing it frame by frame

In the right-hand column, we’ll display a st.progress and a st.spinner to give the user an indication, while in the background we colorize the video frame by frame. To manage video files, we’ll use OpenCV and MoviePy.

pip install moviepy
pip install opencv-python

import cv2
import moviepy.editor as mp

# [...]

with col2:
    st.markdown('<p style="text-align: center;">After</p>', unsafe_allow_html=True)
    with st.spinner("Colorizing frames..."):
        # Colorize video frames and store them in a list
        output_frames = []

        audio = mp.AudioFileClip("video.mp4")
        video = cv2.VideoCapture("video.mp4")

        total_frames = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
        fps = video.get(cv2.CAP_PROP_FPS)

        progress_bar = st.progress(0)  # Create a progress bar
        start_time = time.time()
        time_text = st.text("Time Remaining: ")  # Initialize text value

        for _ in tqdm(range(total_frames), unit='frame', desc="Progress"):
            ret, frame = video.read()
            if not ret:
                break

            colorized_frame = colorize_frame(frame, loaded_model)
            output_frames.append((colorized_frame * 255).astype(np.uint8))

            elapsed_time = time.time() - start_time
            frames_completed = len(output_frames)
            frames_remaining = total_frames - frames_completed
            time_remaining = (frames_remaining / frames_completed) * elapsed_time

            progress_bar.progress(frames_completed / total_frames)  # Update progress bar

            if frames_completed < total_frames:
                time_text.text(f"Time Remaining: {format_time(time_remaining)}")  # Update text value
            else:
                time_text.empty()  # Remove text value
                progress_bar.empty()

Basically, we read the original video frame by frame, colorize each frame with the template loaded by the user and store the new colorized frame in output_frames. At the same time, we update the progress bar with the remaining time to inform the user.

• Render a new video

Once this part is complete, we have a list containing all the colorized frames. All that remains is to render the new video at the same framerate as the original and add the sound. Finally, we can make the colorized video available for download via the st.download_button.

with st.spinner("Merging frames to video..."):
    frame_size = output_frames[0].shape[:2]
    output_filename = "output.mp4"
    fourcc = cv2.VideoWriter_fourcc(*"mp4v")  # Codec for MP4 video
    out = cv2.VideoWriter(output_filename, fourcc, fps, (frame_size[1], frame_size[0]))

    # Display the colorized video using st.video
    for frame in output_frames:
        frame_bgr = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)

        out.write(frame_bgr)

    out.release()

    # Convert the output video to a format compatible with Streamlit
    converted_filename = "converted_output.mp4"
    clip = mp.VideoFileClip(output_filename)
    clip = clip.set_audio(audio)

    clip.write_videofile(converted_filename, codec="libx264")

    # Display the converted video using st.video()
    st.video(converted_filename)
    st.balloons()

    # Add a download button for the colorized video
    st.download_button(
        label="Download Colorized Video",
        data=open(converted_filename, "rb").read(),
        file_name="colorized_video.mp4"
    )

    # Close and delete the temporary file after processing
    video.release()

Here’s the final interface of our page when colorizing a video.

Final interface for the colorization of YouTube videos — GIF by author

This part of the tutorial is finished. We’ve built a single-page web application with Streamlit and Hugging Face to colorize YouTube videos. In the next part, we’ll see how to handle multiple pages.

The full code of this part is available here.

Part 3: Add Multiple Pages to the Application

For the moment, our project file structure looks like this.

Project file structure after Part 2 — Image by author

• One application file app.py
• One txt file requirements.txt containing the libraries needed for the app

To have several pages within the same application, create a new subdirectory called pages.

Directory ‘pages’ created — Image by author

Within it, you can create independent Python files that will exist as independent pages. For this example, I’ve renamed our app.py to page_1.py and copied it into the pages folder as page_2.py.

Application not working anymore — Image by author

However, as you can see in the screenshot above, natively, streamlit doesn’t understand that the main application file isn’t app.py anymore. Take a look at README.md :

---
title: Tutorial Medium
emoji: 🐢
colorFrom: blue
colorTo: blue
sdk: streamlit
sdk_version: 1.21.0
app_file: app.py
pinned: false
---

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

This makes sense, as app_file is still set to app.py. To get our application working again, change this parameter in the file according to your main application file, and voila!

Basic multipage web application on Hugging Face with Streamlit — Image by author

‘page 1’ and ‘page 2’ are not very cool names for our application pages. Let’s add emojis and numbers to sort them. In my final application, my main file is named the following way:

• 01_📼_Upload_Video_File.py

And the two other pages within the pages folders are named that way :

• pages/02_🎥_Input_Youtube_Link.py
• pages/03_🖼️_Input_Images.py

The final result looks like the following.

Pleasing naming of application pages — Image by author

Bonus: Customize Streamlit’s theme and file upload size

If like me you’re not satisfied with the default theme, you can customize it.

• Create a .streamlit folder at the root of the project directory
• Create a config.toml file and add the following pieces of information:

[theme]
primaryColor="#F63366"
backgroundColor="#FFFFFF"
secondaryBackgroundColor="#F0F2F6"
textColor="#262730"
font="sans serif"
[server]
maxUploadSize=1028

This allows you to configure certain aspects of your application. More information is in the documentation here. In our case, I’ve customized the theme, the default font as well as the maximum file upload size. I’ve set it to 1GB which is reasonable for this application.

Conclusion

In this tutorial, we’ve seen several aspects of model deployment and some advanced features of streamlit such as caching and building a multi-page web application.

If you want to try out the colorization app check it out here. Keep in mind that it is hosted on the free tier of Hugging Face spaces. Therefore, colorizing a video will take a lot of time. However, you can colorize batches of images in an instant.

It’s now your turn to find hidden gems on GitHub or in a conference of your choice and deploy the model on Hugging Face with Streamlit.

Have fun learning 🧐!

The GitHub code for this tutorial is available here.

Want to connect?

• 📖 Follow me on Medium
• 💌 Subscribe to get an email whenever I publish
• ☕Fancy a coffee in Paris to discuss AI ?
• 🤓 Connect with me on LinkedIn

I’ve also written:

Monet Paintings Reimagined: Training a DCGAN on Impressionist Art

Identify Green Spaces in Your City With Folium and Open Data

Unsupervised Sentiment Analysis With Real-World Data: 500,000 Tweets on Elon Musk

References

[1] R. Zhang, P. Isola, and A. A. Efros, “Colorful Image Colorization” (2016), ECCV.

[2] R. Zhang, J.-Y. Zhu, P. Isola, X. Geng, A. S. Lin, T. Yu, and A. A. Efros, “Real-Time User-Guided Image Colorization with Learned Deep Priors” (2017), ACM Transactions on Graphics (TOG), vol. 9, no. 4.