How to make the most from old WW2 footage
My grandmother was born in Kowel, Poland in 1928. During WW2 the town was around 90% destroyed, and today very few of the structures from that time period remain intact. I’ve got a background in art and design and have always liked a challenge. For the last few years I’ve been on a quest to recreate Kowel from scratch using digital resources. In this way part of the history of the city will be preserved, and along the way I’ll be able to put my various skills to use and learn new ones as well.
If you Google “Battle of Kowel video” you’ll find a grainy old video recording taken by the Germans. Among the many shots of the city, there is one decent panorama. Decent, in one respect that it covers quite a lot of the city. Not decent in the fact that it looks like blurry mush:
When I saw that panorama I immediately thought that there must be a few ways to try to make what’s pictured there clearer, perhaps clear enough to be able to make on a modern map.
The first thing I want to do is employ a technique called “focus stacking”. Focus stacking is an old technique that takes a series of aligned images and ‘stacks’ their pixel values to create a higher quality image. Often used by astronomers to improve the resolution of their telescopes, I though it would be a great candidate for converting this short sequence into a higher quality (though still quite poor) image.
Once you’ve extracted the video, you’ll want to export all the frames to your local workstation. I used code (taken from this question on Stack overflow question: https://stackoverflow.com/questions/33311153/python-extracting-and-saving-video-frames)
You’ve got a bunch of frames you don’t need, so feel free to delete all the images that aren’t part of your target image. In my case I ended up retaining 40 frames. The next thing you want to do is align the frames. In order to do that open terminal in the directory with your images and run the following code:
align_image_stack -a aligned -C *.jpg
Source: http://montessorimuddle.org/2013/06/21/focus-stacking-combining-images-for-a-sharper-view/
Now you’ll have a bunch of images that are ready to be stacked. Just run this code:
enfuse -o stack_image_result.tif — exposure-weight=0 — saturation-weight=0 — contrast-weight=1 — hard-mask — contrast-window-size=9 aligned000?.tif
Source: http://montessorimuddle.org/2013/06/21/focus-stacking-combining-images-for-a-sharper-view/
Take a look at the image named ‘stack_image_result.tif’. If you compare it to the original frame, it should look better. For my Battle of Kowel panorama here is the result:
A single glance at the image on the right gives us a much clearer idea of the area in question. We are able to more easily make out buildings and other features. However this is still not enough information to determine the precise location from which this image was taken of the city of Kowel.
Next, I turn to the amazing tool called Deoldify. This powerful model using deep learning to colorize images of practically anything. As I lack a GPU I decide to use the custom-created Google Colab for Deoldify. My hope is that the tool will be able to suss out some additional details (or identify patterns) that I am simply unable to do on my own.
Once you have the Colab notebook open just run the entire thing by clicking on: Runtime > Run all
When everything has run through (and the model has been loaded) scroll down to the section titled “Colorize!!”. Add your image URL in the field: source_url:
If you don’t have an easy way to host your image and you want a quick-and-dirty solution just upload it as a hidden image to your Imgur account and add “.jpg” to the end of the imgur sharing link. Next, hit shift+enter in colab to let Deoldify do it’s magic. My final result looks like this:
I’m still not 100% satisfied, yet. So I will now try to convert this image into a 3D model. The opensource Blender app will be my tool of choice. But before I can build something from the image, I’ll need to use fspy to help me set up a proper scene in Blender. FSpy will take any 2-D image and create the correct camera setup so that you can model in 3D using the image as a blueprint. As you draw shapes they will follow the perspective from the original scene. The first thing is to install Fspy. Once you have FSpy installed just drag and drop your image into the GUI of FSpy:
Locate a feature that has lines going toward vanishing points and for each of the 2 sets of lines align the pivot points so that the lines match up with the feature. Once you are done just click: file > save as and save your new FSpy file somewhere convenient.
Now open Blender. Install the FSpy Blender plug-in (not the software installed above). To install the FSpy importer follow the tutorial. Now, in Blender delete the default cube and click: file > import > FSpy. Select your FSpy file (created above). Now your Blender GUI will feature a background image with the camera aligned to the perspective as you defined it in FSpy.
I tried modelling right away, but soon realized that it would be much more prudent to first create a large plane across the ground of the image and then extrude the basic shapes of the structures from it. After some time your model should look like this:
One great thing is that now you can see what the area would’ve looked like from an aerial photograph. Move your camera into position, and now you may be able to compare your rendering to a real-world image of the city in Google Earth.
It turns out that so much of Kowel was destroyed that using the above image to try to identify the location of the photo without having any historic knowledge of the city would be a challenge. However I happen to have spent around 50 hours this holidy season researching this town. It featured a vantage point that served as a popular destination from post card photographers from the period. In fact the landmark is featured in one of the frames of the video:
It’s very likely that the panorama is from the area of the church, but as the church was demolished how do we know where it stood? Well, thankfully there’s a service called Mapster that offers a whole bunch of maps. It turns out that a Kowel map exists that shows just this church. Here is a cropped version of the map (which you may download):
The fact is that at the time of the start of WW2 in 1939 Kowel featured several churches. The only reason I know that this specific church was located at this very spot is that I’ve researched it’s location, and managed to identify this spot as the likely location. Today the former site of the church is a football pitch:
I am quite certain that the image was taken from the vantage point of the church as it’s one of the best spots from which to record the city at that time. However we still don’t know from which direction the photo is being taken. For some insight we may consult the Polana digital archive. Here we can find an image of the church (which happens to be in the public domain):
It looks like we can learn something interesting from this photo. Time for some analysis. In the map above it’s clear that there’s a large empty area next to the church, which at that time was a park. If we look at the various elements we can make a quick decision as to which part of the city featured the park and which part we are looking at in the grainy photo of the church during the Battle of Kowel:
In the next article we will go into alternative approaches and go deeper into this topic.
After some time I decided to look for more images of the church. Among those that I was able to find, I got luck with this one:
This is a very wonderful image as it lets us discover two wonderful things, the precise position of the church, and proof that the name of the street is in fact May 3rd, as predicted based on other data. Now that we know what the church looks like from the street, and which street the main entrance is toward, we can place the church correctly on the map. Here’s my rough 3D sketch of where the church was at the time of the battle of Kowel:
Now I have a really good idea of the most likely location of the photographer when they filmed the church still standing in 1944. The next step is to try to recreate the building in the foreground and to line it up so that the scene is completed. You can view a vectorized sketch (created in Inkscape) that brings out some of the details of the buildings in the frame, in preperation for layout in Blender.