How to make the most from old WW2 footage

My grandmother was born in Kowel, Poland in 1928. During WW2 the town was around 90% destroyed, and today very few of the structures from that time period remain intact. I’ve got a background in art and design and have always liked a challenge. For the last few years I’ve been on a quest to recreate Kowel from scratch using digital resources. In this way part of the history of the city will be preserved, and along the way I’ll be able to put my various skills to use and learn new ones as well.

If you Google “Battle of Kowel video” you’ll find a grainy old video recording taken by the Germans. Among the many shots of the city, there is one decent panorama. Decent, in one respect that it covers quite a lot of the city. Not decent in the fact that it looks like blurry mush:

Image for post
Image for post
A still frame from the “Battle of Kowel” footage found on YouTube

When I saw that panorama I immediately thought that there must be a few ways to try to make what’s pictured there clearer, perhaps clear enough to be able to make on a modern map.

The first thing I want to do is employ a technique called “focus stacking”. Focus stacking is an old technique that takes a series of aligned images and ‘stacks’ their pixel values to create a higher quality image. Often used by astronomers to improve the resolution of their telescopes, I though it would be a great candidate for converting this short sequence into a higher quality (though still quite poor) image.

Once you’ve extracted the video, you’ll want to export all the frames to your local workstation. I used code (taken from this question on Stack overflow question: https://stackoverflow.com/questions/33311153/python-extracting-and-saving-video-frames)

You’ve got a bunch of frames you don’t need, so feel free to delete all the images that aren’t part of your target image. In my case I ended up retaining 40 frames. The next thing you want to do is align the frames. In order to do that open terminal in the directory with your images and run the following code:

align_image_stack -a aligned -C *.jpg

Source: http://montessorimuddle.org/2013/06/21/focus-stacking-combining-images-for-a-sharper-view/

Now you’ll have a bunch of images that are ready to be stacked. Just run this code:

enfuse -o stack_image_result.tif — exposure-weight=0 — saturation-weight=0 — contrast-weight=1 — hard-mask — contrast-window-size=9 aligned000?.tif

Source: http://montessorimuddle.org/2013/06/21/focus-stacking-combining-images-for-a-sharper-view/

Take a look at the image named ‘stack_image_result.tif’. If you compare it to the original frame, it should look better. For my Battle of Kowel panorama here is the result:

Image for post
Image for post
A side-by-side comparison of a single video frame, and the resulting stacked image.

A single glance at the image on the right gives us a much clearer idea of the area in question. We are able to more easily make out buildings and other features. However this is still not enough information to determine the precise location from which this image was taken of the city of Kowel.

Next, I turn to the amazing tool called Deoldify. This powerful model using deep learning to colorize images of practically anything. As I lack a GPU I decide to use the custom-created Google Colab for Deoldify. My hope is that the tool will be able to suss out some additional details (or identify patterns) that I am simply unable to do on my own.

Once you have the Colab notebook open just run the entire thing by clicking on: Runtime > Run all

When everything has run through (and the model has been loaded) scroll down to the section titled “Colorize!!”. Add your image URL in the field: source_url:

If you don’t have an easy way to host your image and you want a quick-and-dirty solution just upload it as a hidden image to your Imgur account and add “.jpg” to the end of the imgur sharing link. Next, hit shift+enter in colab to let Deoldify do it’s magic. My final result looks like this:

Image for post
Image for post
The stacked image after it’s fed through Deoldify on Google Colab
Image for post
Image for post
Update: I fed the photo through this app to improve the contrast and bring out features
(center of image) These appear to be 2 vehicles in the road, perhaps a pair of tanks

I’m still not 100% satisfied, yet. So I will now try to convert this image into a 3D model. The opensource Blender app will be my tool of choice. But before I can build something from the image, I’ll need to use fspy to help me set up a proper scene in Blender. FSpy will take any 2-D image and create the correct camera setup so that you can model in 3D using the image as a blueprint. As you draw shapes they will follow the perspective from the original scene. The first thing is to install Fspy. Once you have FSpy installed just drag and drop your image into the GUI of FSpy:

Image for post
Image for post
Using FSpy to quickly align your Blender Camera with the target images perspective

Locate a feature that has lines going toward vanishing points and for each of the 2 sets of lines align the pivot points so that the lines match up with the feature. Once you are done just click: file > save as and save your new FSpy file somewhere convenient.

Now open Blender. Install the FSpy Blender plug-in (not the software installed above). To install the FSpy importer follow the tutorial. Now, in Blender delete the default cube and click: file > import > FSpy. Select your FSpy file (created above). Now your Blender GUI will feature a background image with the camera aligned to the perspective as you defined it in FSpy.

I tried modelling right away, but soon realized that it would be much more prudent to first create a large plane across the ground of the image and then extrude the basic shapes of the structures from it. After some time your model should look like this:

Image for post
Image for post
A very rough model of a portion of the town of Kowel sometime in 1944.

One great thing is that now you can see what the area would’ve looked like from an aerial photograph. Move your camera into position, and now you may be able to compare your rendering to a real-world image of the city in Google Earth.

Image for post
Image for post
A simulated ‘aerial photograph’ based on the original photo.

It turns out that so much of Kowel was destroyed that using the above image to try to identify the location of the photo without having any historic knowledge of the city would be a challenge. However I happen to have spent around 50 hours this holidy season researching this town. It featured a vantage point that served as a popular destination from post card photographers from the period. In fact the landmark is featured in one of the frames of the video:

Image for post
Image for post
The Church was called “Pomnikowy” and stood here until around 1944.

It’s very likely that the panorama is from the area of the church, but as the church was demolished how do we know where it stood? Well, thankfully there’s a service called Mapster that offers a whole bunch of maps. It turns out that a Kowel map exists that shows just this church. Here is a cropped version of the map (which you may download):

Image for post
Image for post
One of the many churches on the 1943 map of Kowel, Poland (now Kovel, Ukraine).

The fact is that at the time of the start of WW2 in 1939 Kowel featured several churches. The only reason I know that this specific church was located at this very spot is that I’ve researched it’s location, and managed to identify this spot as the likely location. Today the former site of the church is a football pitch:

Image for post
Image for post
Kovel, Ukraine as it appears in OpenStreetMap

I am quite certain that the image was taken from the vantage point of the church as it’s one of the best spots from which to record the city at that time. However we still don’t know from which direction the photo is being taken. For some insight we may consult the Polana digital archive. Here we can find an image of the church (which happens to be in the public domain):

Image for post
Image for post
The future football pitch circa 1938

It looks like we can learn something interesting from this photo. Time for some analysis. In the map above it’s clear that there’s a large empty area next to the church, which at that time was a park. If we look at the various elements we can make a quick decision as to which part of the city featured the park and which part we are looking at in the grainy photo of the church during the Battle of Kowel:

Image for post
Image for post
Determining from which vantage point footage of the church was taken

In the next article we will go into alternative approaches and go deeper into this topic.

After some time I decided to look for more images of the church. Among those that I was able to find, I got luck with this one:

Image for post
Image for post
Translation: Memorial Church in Kowel in Wolyn (Poland). Entrance off of May 3rd street — Interior unfinished due to lack of funds. circa 1935.

This is a very wonderful image as it lets us discover two wonderful things, the precise position of the church, and proof that the name of the street is in fact May 3rd, as predicted based on other data. Now that we know what the church looks like from the street, and which street the main entrance is toward, we can place the church correctly on the map. Here’s my rough 3D sketch of where the church was at the time of the battle of Kowel:

Image for post
Image for post
Clockwise from left: (1) Camera, 3D sketch (object), colorized (via Deoldify) 1944 aerial photo. (2) The rough model of the demolished church matched up with the photo (3) taken during the battle of Kowel, 1944.

Now I have a really good idea of the most likely location of the photographer when they filmed the church still standing in 1944. The next step is to try to recreate the building in the foreground and to line it up so that the scene is completed. You can view a vectorized sketch (created in Inkscape) that brings out some of the details of the buildings in the frame, in preperation for layout in Blender.

Written by

Marcin Kraszewski lives and works in Warsaw, Poland. He has been running the SOAI.world website since September of 2018. He works in IT as a developer.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store