Multi-channel support- Week3

GSoC23 caMicroscope

In this article, I will share the progress for Week 3 for my contribution to Multi-channel support for caMicroscope as a part of Google Summer of Code 2023.

Feedback Overview

During Week 2, I played around with different datasets and different approaches to convert multi-channel images into RGB channel. These were the feedback, I received from my mentor(Birmingham Ryan).

1. In the tif files, 44153.tif, convert the first image into red, the second image into green and the third image into blue and then try to combine all the separate channels into the single final image.
2. Study and read https://www.mdpi.com/2072-4292/9/4/323.
3. For the Hyperspectral approach pick more than the first 10 bands and normalize to red, repeat the next 10 for green and the last 10 bands into blue.
4. Linear approach colour conversion(Weights concept) — testing

Multiple Tif image conversion into RGB

Implementation

In the below code, I implemented and converted the first tif into the Red channel, the second tif into the Green channel and the last tif into the Blue channel.

def combine():
    image = load_data(os.path.join("data","44153.tif"))
    image.seek(0)
    red_ch = assign_red_channel(image)
    image.seek(1)
    green_ch = assign_green_channel(image)
    image.seek(2)
    blue_ch = assign_blue_channel(image)
    final_image = np.empty((1020, 954, 3), dtype=np.uint8)
    final_image[:, :, 0] = red_ch[:, :, 0]
    final_image[:, :, 1] = green_ch[:, :, 1]
    final_image[:, :, 2] = blue_ch[:, :, 2]
    
    save_image(final_image)

Combined Image

Also, a linear approach i.e., assigning the Weights to convert RGBA to RGB was performed and the results were not promising.

Hyperspectral band conversion to RGB using PCA

Moving to the next dataset, where we have the Hyperspectral Images with 30 bands. We again perform the conversion of the first 10 bands into

Implementation

Approach- 1

Combination of bands

red_channel = ballon_arr[:, :, :10]
green_channel = ballon_arr[:, :, 10:20]
blue_channel = ballon_arr[:, :, 20:30]

red_channel = ((red_channel - red_channel.min()) / (red_channel.max() - red_channel.min())) * 255
green_channel = ((green_channel - green_channel.min()) / (green_channel.max() - green_channel.min())) * 255
blue_channel = ((blue_channel - blue_channel.min()) / (blue_channel.max() - blue_channel.min())) * 255

rgb_image = np.stack((red_channel, green_channel, blue_channel), axis=2).astype(np.uint8)

plt.figure(figsize=(15,10))
for i in range(1,11):
    plt.subplot(2,5,i)
    plt.imshow(rgb_image[:,:,:,i-1])

Approach- 2

Auto thresholding

threshold = threshold_otsu(reshaped_image)
binary_image = reshaped_image > threshold

binary_image = np.zeros_like(reshaped_image, dtype=bool)
for i in range(30):
    threshold = threshold_otsu(reshaped_image[:, i])
    binary_image[:, i] = reshaped_image[:, i] > threshold

Group similar matrix

sim = np.full_like(reshaped_image,0.8)
for i in range(30):
    sim[:, i] = np.sum(reshaped_image[:, i][binary_image[:, i]])

PCA- Dimensionality Reduction

pca = PCA(n_components=3)
pca.fit(norm)

rgb = pca.components_
rgb_image = np.dot(norm, rgb.T)
rgb_image = rgb_image.reshape(512, 512, 3)

Conclusion

With this progress, the algorithm for the Multi-channel support is completed. The progress is on track as mentioned in the proposal, and in the coming weeks, API endpoints for the Image support will be integrated as a backed service.

LinkedIn: https://www.linkedin.com/in/jaintarun75/

GitHub: https://github.com/lucifertrj/

Twitter: https://twitter.com/TRJ_0751