Multichannel Image Support- Week2

In this article, I will share my second-week progress in Google Summer of Code 2023 at caMicroscope.

Getting started with tif files and fluorescence microscopy images.

Since there is no available dataset from the caMicroscope side, I had to work on external data which consisted of Multi-channel images. Since caMicroscope is mainly about Digital pathology and fluorescence microscopy field related.

Fluorescence microscopy images utilize fluorescent dyes or proteins to label specific molecules or structures within cells, allowing researchers to visualize and study their localization and dynamics with exceptional precision. The vibrant and contrasting colours in fluorescence microscopy images provide valuable insights into cellular processes, enabling scientists to unravel intricate details and gain a deeper understanding of biological phenomena.

Implementation

For implementation, I started working with three different datasets.

Dataset-1

1st dataset was taken from the Cell Image library: http://www.cellimagelibrary.org/images/44153. In this particular dataset, we have three different images stored inside a single tif file. Each image is a grayscale image.

from PIL import Image

tif_image = Image.open("44153.tif")
rgb_image = tif_image.convert("RGB")
rgb_array = np.array(rgb_image)

rgb_image = Image.fromarray(rgb_array)
output_path = "rgb_image.tif"
rgb_image.save(output_path)

I had frequent meetings with the Mentor for feedback, we usually meet twice a week and discuss if the method approached is valid or not. The above approach was not the exact output that we required. I have mentioned the feedback at the end of the article, and changes will be implemented in Week 3.

Dataset-2

The second dataset had 4 image channels mostly RGBA, where A is the alpha channel. The dataset used in Drosophila melanogaster Kc167 cells were stained for DNA (to label nuclei) and actin (a cytoskeletal protein, to show the cell body). Automatic cytometry requires that cells be segmented, i.e., that the pixels belonging to each cell be identified. https://bbbc.broadinstitute.org/BBBC007.

There are 4 sub-categories in Kc167 cells: A9, f113, f9620 and f96 (17).

import os
import skimage.io as io
import skimage.transform as trans
import numpy as np
import matplotlib.pyplot as plt
import SimpleITK as stk

import warnings
warnings.filterwarnings('ignore')


def choose_image_files(image_path,outline_path):
    img = stk.ReadImage(image_path)
    x = stk.GetArrayFromImage(img)

    outline_img = stk.ReadImage(outline_path)
    out_x = stk.GetArrayFromImage(outline_img)
    return x,out_x

def plot_tiff(image,outline):
    plt.figure(figsize=(12,10))

    plt.subplot(1,2,1)
    plt.imshow(image)
    plt.title(f"Image: shape{image.shape}")

    plt.subplot(1,2,2)
    plt.imshow(outline)
    plt.title(f"Outline: shape{outline.shape}")

img,out = choose_image_files('BBBC007_v1_images/f113/AS_09125_040701150004_A02f00d1.tif','BBBC007_v1_outlines/f113/AS_09125_040701150004_A02f00d1.tif')
plot_tiff(img,out)

f113 output

The same procedure is followed for A9, f9620 and f96 (17). For the above method, I tried to implement Weight based channel conversion.

"""
weight_range = np.linspace(0, 1, num=101)
ideal_weights = None
lower_weights = float('inf')

for i in product(weight_range, repeat=3):
    weights = np.append(i, 0)
    channels = np.dot(out, np.array(weights)[:, None])
    error = np.mean(np.abs(channels[:, :, :3] - channels[:, :, :3]))
    if error < lower_weights:
        ideal_weights = weights
        lower_weights = error
"""

weights = [0.9, 0.5, 0.1, 0]
matrix_mul = np.dot(out[:, :, :3], np.array(weights[:3]))

three_channel = matrix_mul
plt.imshow(three_channel,cmap="gray")

Weight-based conversion RGBA to RGB

Working with Multispectral and Hyperspectral Imaging

Multispectral imaging involves capturing images at a limited number of discrete wavelengths or narrow bands, typically ranging from a few to a few dozen. These specific bands are carefully selected to target particular features or properties of the objects being imaged. Multispectral imaging provides valuable information about the composition, morphology, or particular characteristics of the imaged objects.

On the other hand, hyperspectral imaging takes this concept further by capturing images at hundreds or even thousands of contiguous narrow spectral bands. This creates a highly detailed spectral signature for each pixel in the image, enabling a more comprehensive analysis of the imaged scene. Hyperspectral imaging allows for the detection and characterization of subtle variations in materials, as well as the identification and discrimination of objects with similar appearances but different spectral properties.

Implementation

The third dataset used in the second week was Hyperspectral (HS) images containing detailed spectral information that has proven crucial in applications like remote sensing, surveillance, and astronomy.

Dataset-3

import numpy as np
import matplotlib.pyplot as plt
import scipy.io as sio
import os

import warnings
warnings.filterwarnings('ignore')

BX = sio.loadmat("data/X/balloons_ms")
ballon_arr = BX['msi']

plt.figure(figsize=(14,14))
for i in range(25):
    plt.subplot(5,5,i+1)
    plt.imshow(ballon_arr[:,:,i])

Multispectral-30 band

Now that we have 30 bands, our main goal is to convert this combine these 30 bands into an RGB format.

image_data = np.clip(ballon_arr, a_min=0, a_max=None)
image_data = np.interp(image_data, (image_data.min(), image_data.max()), (0, 1))
composite_image = np.stack((image_data[:, :, 10], image_data[:, :, 20], image_data[:, :, 30]), axis=-1)
#composite_image.shape
plt.imshow(composite_image)

Combined RGB colour image

We obtained the RGB image but the original image was quite different. Let me plot it.

BY = sio.loadmat("data/Y/balloons_ms")
ballon_y = BY['RGB']
plt.imshow(ballon_y)

Original RGB Image

Feedback- To be carried out in Week 3

1. In the tif files, 44153.tif, convert the first image into red, the second image into green and the third image into blue and then try to combine all the separate channels into the single final image.
2. Study and read https://www.mdpi.com/2072-4292/9/4/323.
3. For the Hyperspectral approach pick more than the first 10 bands and normalize to red, repeat the next 10 for green and the last 10 bands into blue.
4. Linear approach colour conversion(Weights concept) — testing

LinkedIn: https://www.linkedin.com/in/jaintarun75/

GitHub: https://github.com/lucifertrj/

Twitter: https://twitter.com/TRJ_0751