Segmenting the Bear Part 2
In this article, we will explore Chromaticity Segmentation.
From part 1 of image segmentation, we observed the threshold and color segmentation have their limitations. Especially with colors that are obscure, such as the fur of the teddy bear.
Chromaticity segmentation relies on the RG chromacity space of an image. It is a two dimensional color space that does not contain color intensity information.
We begin parametric segmentation by computing the RG chromacity of the photograph. The plots below shows all the colors in the photo on the RG chromacity space.
bear_R = bear[:,:,0]*1.0/(bear.sum(axis=2)+1)
bear_G = bear[:,:,1]*1.0/(bear.sum(axis=2)+1)plt.figure(figsize=(10,10))
plt.scatter(bear_R.flatten(),bear_G.flatten())
plt.xlim(0,1)
plt.ylim(0,1)plt.figure(figsize=(5,5))
plt.hist2d(bear_R.flatten(), bear_G.flatten(), bins=100,cmap='binary')
plt.xlim(0,1)
plt.ylim(0,1)
We then take a sample patch of the colors we want to segment. Here we take a part of the bear’s neck and mouth.
patch = bear[200:300,300:400,:]
imshow(patch)
Similarly, we take the plot of all colors in the region patch. As you may have noticed, only a fraction of all the colors are present. This will help us select the colors to segment from the whole image.
patch_R = patch[:,:,0]*1.0/patch.sum(axis=2)
patch_G = patch[:,:,1]*1.0/patch.sum(axis=2)plt.figure(figsize=(5,5))
plt.scatter(patch_R.flatten(),patch_G.flatten())
plt.xlim(0,1)
plt.ylim(0,1)plt.figure(figsize=(5,5))
plt.hist2d(patch_R.flatten(), patch_G.flatten(), bins=100,cmap='binary')
plt.xlim(0,1)
plt.ylim(0,1)
To find the color space similar to the patch, we first fit a gaussian probability distribution over the region patch color space. We computed this by getting the mean and standard deviation of the patch color space. This distribution will determine whether a color on the main image matches the patch.
std_patch_R = np.std(patch_R.flatten())
mean_patch_R = np.mean(patch_R.flatten())std_patch_G = np.std(patch_G.flatten())
mean_patch_G = np.mean(patch_G.flatten())def gaussian(p,mean,std):
return np.exp(-(p-mean)**2/(2*std**2))*(1/(std*((2*np.pi)**0.5)))x = np.linspace(0,1)
y = gaussian(x,mean_patch_R,std_patch_R)
plt.plot(x,y)r_test = np.tile(np.linspace(0,1,64),(64,1))
plt.imshow(r_test)g_test = r_test.transpose()
plt.imshow(g_test)test_R = gaussian(r_test,mean_patch_R,std_patch_R)
imshow(test_R)test_G = gaussian(g_test,mean_patch_G,std_patch_G)
imshow(test_G)prob_test=test_R * test_G
plt.imshow(prob_test)
Now that we have our distributions and probabilities, we can mask the bear image.
prob_R = gaussian(bags_R,mean_patch_R,std_patch_R)
plt.imshow(prob_R)prob_G = gaussian(bags_G,mean_patch_G,std_patch_G)
plt.imshow(prob_G)prob=prob_R * prob_G
plt.imshow(prob)
mask = prob > 70red = bags[:,:,0]*mask
green = bags[:,:,1]*mask
blue = bags[:,:,2]*mask
bags_masked = np.dstack((red,green,blue))
imshow(bags_masked)
Tada! The bear is segmented better than in part 1. Although it is not perfect, it is still a big improvement from the previous methods explained. We can also combine several methodologies together to get the perfect image segmentation.
On to the next topic. Thank you for reading.
