Carving out the Jack-o’-Lantern: A Guide to Reusing Halloween Phots

I have this nice picture from Halloween, and want to know if I can reuse it with a new portrait of my family. The first step would be to extract the Halloween elements. Luckily, in this case, they’re all orange. Let’s try the most basic method, which would be to use binary thresholding at different values to see if we can extract the light parts from the dark. The first step would be to convert the image to grayscale:

from skimage.io import imread
from skimage.color import rgb2gray
import matplotlib.pyplot as plt

halloween = imread('halloween.JPG')
halloween_gray = rgb2gray(halloween)
plt.figure(figsize=(10, 10))
plt.imshow(halloween_gray, cmap='gray')
plt.axis('off')
plt.show()

Now let’s apply different binarization threshold levels to see how they affect the image:

import numpy as np

rows = 3
cols = 3
fig, ax = plt.subplots(rows, cols, figsize=(12, 12))  # Increase figure size

for i, thresh in enumerate(np.linspace(0.4, 0.6, num=9)):
    halloween_bw = halloween_gray<thresh
    ax[int(i/cols), i%cols].imshow(halloween_bw, cmap='gray')
    ax[int(i/cols), i%cols].set_title(r'$t=%s$' % round(thresh, 2))
    ax[int(i/cols), i%cols].axis('off')

plt.tight_layout()
plt.show()

halloween_bw = halloween_gray<0.5

Interesting results, but would there be way to automate this process instead of visually checking the results at the different thresholds? Yes, let’s use Otsu’s method. It tries to find the best separation between what could be considered ‘light’ pixels, and what could be considered ‘dark’ pixels by determining the ideal threshold value.

from skimage.filters import threshold_otsu

thresh = threshold_otsu(halloween_gray)
halloween_binary_otsu2  = halloween_gray < thresh
print(f"Threshold used: {thresh}")

fig, ax = plt.subplots(figsize=(10,10))
ax.imshow(halloween_binary_otsu2, cmap=plt.cm.gray)
ax.axis('off')
plt.show()

Pretty good and fast results, although it doesn’t bring us closer to our goal of extracting the orange elements. Notice that the orange jack-o’-lanterns in the front are very black while the orange leaves behind them are very white. So this method will probably not work for our use case. Otsu’s method assumes that there is only 1 foreground object and 1 background.

Let’s try color image segmentation using the RGB color channels. We’ll manually adjust the red channel to be more than 150 out of 255 (true orange is 255, but we’ll allow for certain shades) and the blue channel to be less than 100 (true orange has 0 blue, but again, we allow for certain shades). The green should be around 165, so we’ll ignore the green channel for now.

orange_mask = (halloween[:, :, 0] > 150) & (halloween[:, :, 2] < 100)
orange = np.zeros_like(halloween)
orange[orange_mask] = halloween[orange_mask]

fig, ax = plt.subplots(figsize=(10,10))
ax.imshow(orange)
plt.axis('off')
plt.show()

Now we’re able to get the orange pumpkins and the orange leaves, however, there are patches in the leaves and pumpkin that are still missing. Is there another way we can extract the essence of orange without relying on combinations of red, green, and blue?

Yes. Another option we have is moving from RGB into the HSV space, or hue, saturation, and value. Notice that I used the HSV color map for visualizing the hue, while keeping grayscale for the saturation and value.

from skimage.color import rgb2hsv
halloween_hsv = rgb2hsv(halloween)

fig, ax = plt.subplots(1, 3, figsize=(20,10))
ax[0].imshow(halloween_hsv[:,:,0], cmap='hsv')
ax[0].set_title('Hue')
ax[0].axis('off')
ax[1].imshow(halloween_hsv[:,:,1], cmap='gray')
ax[1].set_title('Saturation')
ax[1].axis('off')
ax[2].imshow(halloween_hsv[:,:,2], cmap='gray')
ax[2].set_title('Value')
ax[2].axis('off')

plt.show()

Now let’s try to isolate the orange objects in the picture. Here, we set the hue value to be less than 0.35 (out of 1.0), or more than 0.65. Essentially removing the blue colors in the color wheel. We also maximize the saturation to be above 0.58. Below this value, the skin colors start to be included.

orange_mask = (((halloween_hsv[:, :, 0] < 0.35) |
                (halloween_hsv[:, :, 0] > 0.65)) &
               (halloween_hsv[:, :, 1] > 0.58))
orange = np.zeros_like(halloween)
orange[orange_mask] = halloween[orange_mask]

plt.figure(figsize=(10,10))
plt.imshow(orange)
plt.axis('off')
plt.show()

Very nice! We got all the orange leaves and the orange pumpkins! There are still some orange stripes from the baby’s clothes, but this would need to removed using non-color-based methods.

In conclusion, we’ve detailed various methods of image segmentation and color extraction, focusing on reusing elements from a Halloween-themed photograph. From simple binary thresholding to Otsu’s method, RGB color segmentation, and ultimately to HSV color space usage, we successfully isolated key orange elements in the image. Despite a few remaining unwanted elements, these techniques form a strong foundation for extracting and reusing specific elements from images, illustrating how image processing can repurpose photographs into versatile creative assets.