An Exploration Into Papers With Code

Add A New Dimension To Your Photos Using Python

Breathe New Life Into Your Photos With A 3D Ken Burns Effect

Dylan Roy
Dylan Roy
Nov 19, 2020 · 6 min read
Image by Author

Some Backstory

When looking through the available options I stumbled across a paper with some code that would take your photos, and add a transition that applies a 3D Ken Burns Effect to your photos. The Ken Burns Effect gets the 3D aspect for free with video obviously, but is less compelling with photos due to that missing dimension. So in the following write up I am hoping to set you up to be able to try, and add this extra dimension to your photos as I was able to.

Concepts

Ken Burns Effect A zooming and panning across photographs gives the feeling of motion. It’s named after an American documentarian who often used this effect. The writers of this paper have added the third dimension to this that would normally be missing from photos.

CC BY-SA 3.0, via Wikimedia Commons

Depth Estimation — Describes the method for obtaining a measure of the distance of the objects in a scene. Doing this effectively will result in a convincing transition that will scale each object with the parallax.

Inpainting — This describes the method for reconstructing the part of the image that doesn’t exist due to the changing perspective of this transition. In images, and videos this technique also refers to repairing damaged, deteriorating, or missing parts of an artwork are filled in to present a complete image.

DIY

I won’t get into the details into how this is achieved as the authors do this in their paper, Github Repo, and Youtube video which all have been provided at the end in the resources section.

I am going to step you through the approach that we take to create these 3D scenes, but if you want to just blindly try it yourself at the end of this section I added all of the code that we will be going through to a Google Colab that you can copy, and use immediately.

Before stepping through this yourself I suggest opening a new Google Colab, because in the code samples I go through I use some Colab specific functionality to help with ease of use.

Setup Environment and Install Requirements

!git clone https://github.com/sniklaus/3d-ken-burns.git# Move into the downloaded repository
%cd 3d-ken-burns
# Make a directory for the videos
!mkdir videos
video_dir = "./videos/"
# Install dependencies
!pip install moviepy gevent

Upload Photos

from google.colab import filesuploads = files.upload()for filename in uploads.keys():
!mv ./$filename ./images/$filename

Convert Your Photos

python autozoom.py --in {input_image}.jpg --out {output_video}.mp4

Though since we want to automate what we can, we can take advantage of bash, and iterate through the directory then execute on each photo.

!for image in ./images/*; do python autozoom.py --in $image --out ./videos/$(basename $image | cut -f1 -d '.').mp4; done

View Your Video

import os
from base64 import b64encode
from IPython.display import HTML
import ipywidgets as widgets
def video(path):
mp4 = open(path,'rb').read()
data_url = "data:video/mp4;base64," + b64encode(mp4).decode()
return HTML(f'<video width=600 controls loop> <source src="{data_url}" type="video/mp4"></video>')
files_list = os.listdir(video_dir)
video_list = widgets.Dropdown(
options=files_list,
value="" if not files_list else files_list[0],
description='Video:',
disabled=False,
)
display(video_list)

Convert Mp4 to Animated GIF

import imageio
import os, sys
class TargetFormat(object):
GIF = ".gif"
MP4 = ".mp4"
AVI = ".avi"
def convertFile(inputpath, targetFormat):
"""Reference: https://gist.github.com/michaelosthege/cd3e0c3c556b70a79deba6855deb2cc8"""
outputpath = os.path.splitext(inputpath)[0] + targetFormat
print("converting\r\n\t{0}\r\nto\r\n\t{1}".format(inputpath, outputpath))
reader = imageio.get_reader(inputpath)
fps = reader.get_meta_data()['fps']
writer = imageio.get_writer(outputpath, fps=fps)
for i,im in enumerate(reader):
sys.stdout.write("\rframe {0}".format(i))
sys.stdout.flush()
writer.append_data(im)
print("\r\nFinalizing...")
writer.close()
print("Done.")
for file in [x for x in os.listdir(video_dir) if x.endswith(".mp4")]:
convertFile(f"{video_dir}{file}", TargetFormat.GIF)

Download All Videos

for file in os.listdir(video_dir):
files.download(f"{video_dir}{file}")

If all has gone well then your photos have been properly converted as one of mine I took here when enjoying Bolonga when times we different.

Image by Author

For those that want to skip putting a Colab notebook together themselves here is the complete implementation for you to make a copy of, and start converting your photo albums.

Key Takeaways

Other Posts By Dylan

Resources

Robot Academy

Automate the things that you don’t want to spend time doing.

Dylan Roy

Written by

Dylan Roy

Engineer & Side Project Junkie sharing what I am working on so readers like you can benefit from my experiences. Subscribe here for even more (dylanroy.com)

Robot Academy

From posting twitter messages from Slack to finding the best deals on Amazon. We help you learn to start automating, and to grow your army of robot minions now!

Dylan Roy

Written by

Dylan Roy

Engineer & Side Project Junkie sharing what I am working on so readers like you can benefit from my experiences. Subscribe here for even more (dylanroy.com)

Robot Academy

From posting twitter messages from Slack to finding the best deals on Amazon. We help you learn to start automating, and to grow your army of robot minions now!

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store