Segmenting the Bear Part 1

Let’s explore some image segmentation strategies

Threshold

In the first part of this article, we will isolate the subject from the background using thresholding. There are two ways to perform thresholding, manually with trial-and-error and the automatic Otsu method. For these examples, we will be using a large teddy bear.

Trial and Error

Trial and error method is straightforward. We manually choose a threshold until we are satisfied with the separation of subject and background. Below, we show an example of a bear in a park with a threshold of 0.6.

from skimage.color import rgb2gray
bear_gray = rgb2gray(bear)
imshow(orange_bag_gray)
th = 0.6
bear_bw = bear_gray>th
imshow(bear_bw)

Otsu Method

Manually assigning the threshold is effective; however, if we have to process numerous photos, manually selecting a threshold is time consuming. This brings us to the second method, Otsu method.

The method is named after Otsu Nobuyuki. The algorithm works by returning a single intensity threshold that best separates the image into two classes (subject and background)

Below, we will show the numpy and skimage implementations

freq, bins = histogram(orange_bag_gray)
plt.step(bins, freq*1.0/freq.sum())
plt.xlabel('intensity value')
plt.ylabel('fraction of pixels');
total = sum(freq)
sumB = 0 # Sum of Background
wB = 0 # Weight of Background
maximum = 0
sum1 = sum(np.arange(0, 256) * freq) #Total Sum
levels = []
for i in np.arange(len(bins)):
    wB = wB + freq[i]
    wF = total - wB # Weight of Foreground
    if wB == 0 or wF == 0:
        break
    sumB = sumB + (i-1)*freq[i]
    mF = (sum1 - sumB) / wF # Mean of Foreground
    mB = sumB / wB # Mean of Background
    between = wB * wF * (mB - mF) * (mB - mF)
    levels.append(between)
    if between >= maximum:
        level = i
        maximum = between
fig, ax1 = plt.subplots()
ax1.step(bins, freq*1.0/freq.sum(),'b')
ax1.set_xlabel('intensity value')
# Make the y-axis label, ticks and tick labels match the line color.
ax1.set_ylabel('fraction of pixels', color='b')
ax1.tick_params('y', colors='b')
ax2 = ax1.twinx()
ax2.plot(bins[0:255],levels,'r')
ax2.set_ylabel('inter-class variance', color='r')
ax2.tick_params('y', colors='r')
fig.tight_layout()
plt.show()

bear_bw = bear_gray<(level*1.0/255)
imshow(bear_bw)

Otsu implementation with skimage.filters

from skimage.filters import threshold_otsu
thresh = threshold_otsu(orange_bag_gray)
orange_bag_binary_otsu2  = orange_bag_gray < thresh
imshow(orange_bag_binary_otsu2)

As noticed in the example, threshold has its limitations, it only works with distinct differentiation in images. The threshold still kept most of the background, while removing the leaves.

Color Image Segmentation

Images can separated by using their colors. Using the same bear photo, we will segment the different objects in the photo — the bear, bench and leaves.

We will first observe the three color channels — Red, Green, and Blue.

fig, ax = plt.subplots(1, 3, figsize=(12,4))
ax[0].imshow(bear[:,:,0], cmap='gray')
ax[0].set_title('Red')
ax[1].imshow(bear[:,:,1], cmap='gray')
ax[1].set_title('Green')
ax[2].imshow(bear[:,:,2], cmap='gray')
ax[2].set_title('Blue');

Next, we will explore the Hue, Saturation, and Value spaces.

bear_hsv = rgb2hsv(bear)
fig, ax = plt.subplots(1, 3, figsize=(12,4))
ax[0].imshow(bear_hsv[:,:,0], cmap='gray')
ax[0].set_title('Hue')
ax[1].imshow(bear_hsv[:,:,1], cmap='gray')
ax[1].set_title('Saturation')
ax[2].imshow(bear_hsv[:,:,2], cmap='gray')
ax[2].set_title('Value');

Now that we know the information on the 6 different channels, we can create a mask with the code below that can segment the bear apart from the background.

lower_mask = bear_hsv[:,:,0] > 0.0
upper_mask = bear_hsv[:,:,0] < 0.15
mask3 = bear_hsv[:,:,2] > 0.8
mask = upper_mask*lower_mask*mask3
plt.imshow(mask)

As shown above, the color segmentation works better in isolating the bear from the image. Nevertheless, it’s not perfect, and can still be improved.

Thank you for reading. Don’t forget to check part 2 of image segmentation.