Removing Moving Objects From Images in Go

Koray Göçmen
Oct 15 · 6 min read
Image for post
Image for post

Median filtering can be used to join multiple images to get a single image with all moving objects removed. The simple idea is to compare the pixels at the same location on all images and get the median of RGBA values of the pixels to form a new composite image with all moving objects removed.

The image set provided has to be taken by a camera that did not move during the shoot, since the pixels at the same location on all images have to correspond to the same physical location. Otherwise, the median filter will blend all images together to create a meaningless composite image. You can read more about median filtering on Nikolas Moya’s medium article. (I took the test frames I used from this article as well): Simple algorithm to remove moving objects from pictures

The following code is the definition of the image struct I am going to be using, a few utility functions to read an image from a file to my structs and functions to write the struct back to a proper image file. This code is exactly the same code I used in my 3 part article “Writing an image manipulation library in Go”. You can find more information about these functions and structs here: https://koraygocmen.medium.com/writing-an-image-manipulation-library-in-go-part-1-d4e20dada8b

// Pixel is a single pixel in 2d array
type Pixel struct {
R int
G int
B int
A int
}

// Image is the main object that holds information about the
// image file. Also is a wrapper around the decoded image
// from the standard image library.
type Image struct {
Pixels [][]Pixel
Width int
Height int
_Rect image.Rectangle
_Image image.Image
}

// set pixel value with key name and new value
func (pix *Pixel) set(keyName string, val int) Pixel {
switch keyName {
case "R":
pix.R = val
case "G":
pix.G = val
case "B":
pix.B = val
case "A":
pix.A = val
}
return *pix
}

// rgbaToPixel alpha-premultiplied red, green, blue and alpha values
// to 8 bit red, green, blue and alpha values.
func rgbaToPixel(r uint32, g uint32, b uint32, a uint32) Pixel {
return Pixel{
R: int(r / 257),
G: int(g / 257),
B: int(b / 257),
A: int(a / 257),
}
}

// newImage reads an image from the given file path and return a
// new `Image` struct.
func newImage(filePath string) (*Image, error) {
s := strings.Split(filePath, ".")
imgType := s[len(s)-1]

switch imgType {
case "jpeg", "jpg":
image.RegisterFormat("jpeg", "jpeg", jpeg.Decode, jpeg.DecodeConfig)
case "png":
image.RegisterFormat("png", "png", png.Decode, png.DecodeConfig)
default:
return nil, errors.New("unknown image type")
}

imgReader, err := os.Open(filePath)
if err != nil {
fmt.Println("error opening")
return nil, err
}

img, _, err := image.Decode(imgReader)
if err != nil {
fmt.Println("error decoding")
return nil, err
}

bounds := img.Bounds()
width, height := bounds.Max.X, bounds.Max.Y

var pixels [][]Pixel
for y := 0; y < height; y++ {
var row []Pixel
for x := 0; x < width; x++ {
pixel := rgbaToPixel(img.At(x, y).RGBA())
row = append(row, pixel)
}
pixels = append(pixels, row)
}

return &Image{
Pixels: pixels,
Width: width,
Height: height,
_Rect: img.Bounds(),
_Image: img,
}, nil
}

// writeToFile writes iamges to the given filepath.
// Returns an error if it occurs.
func (img *Image) writeToFile(outputPath string) error {
cimg := image.NewRGBA(img._Rect)
draw.Draw(cimg, img._Rect, img._Image, image.Point{}, draw.Over)

for y := 0; y < img.Height; y++ {
for x := 0; x < img.Width; x++ {
rowIndex, colIndex := y, x
pixel := img.Pixels[rowIndex][colIndex]
cimg.Set(x, y, color.RGBA{
uint8(pixel.R),
uint8(pixel.G),
uint8(pixel.B),
uint8(pixel.A),
})
}
}

s := strings.Split(outputPath, ".")
imgType := s[len(s)-1]

switch imgType {
case "jpeg", "jpg", "png":
fd, err := os.Create(outputPath)
if err != nil {
return err
}

switch imgType {
case "jpeg", "jpg":
jpeg.Encode(fd, cimg, nil)
case "png":
png.Encode(fd, cimg)
}
default:
return errors.New("unknown image type")
}

return nil
}

The following code finds the median RGB values of an array of pixels provided and creates a new pixel with the median values.

// medianPixel finds the median r, g, b values from the given
// pixel array and creates a new pixel from that median values
func medianPixel(pixels []Pixel) Pixel {
var (
rValues []int
gValues []int
bValues []int
)

for _, pix := range pixels {
rValues = append(rValues, pix.R)
gValues = append(gValues, pix.G)
bValues = append(bValues, pix.B)
}

sort.Ints(rValues)
sort.Ints(gValues)
sort.Ints(bValues)

rMedian := rValues[int(len(rValues)/2)]
gMedian := gValues[int(len(gValues)/2)]
bMedian := bValues[int(len(bValues)/2)]

return Pixel{rMedian, gMedian, bMedian, 0}
}

The `medianFilter` function is the main function that creates an image with each pixel generated from the `medianPixel` function. It starts by reading all images from a provided array of image paths and creating image objects for each of them. These images have to have the same height and width for this function to work. It then iterates through all pixels locations and creates an image with all pixels generated from `medianPixel` function.

This function actually manipulates all rows in parallel. By using sync.WaitGroup and go keyword, I am able to do things concurrently. I actually wrote a blog post about concurrency in go. https://koraygocmen.medium.com/concurrency-and-mutex-locks-in-go-9c403a676cf7

// medianFilter iterates the given filepaths and generates new image
// objects. It then checks to see if all the heights and the widths
// of the images are matching. If they are, each pixel of every image is
// iterated and a median filter is applied to given images. Returns the
// output image object and an error if there is any.
func medianFilter(filePaths []string) (*Image, error) {
var images []*Image
for _, filePath := range filePaths {
img, err := newImage(filePath)
if err != nil {
return nil, err
}
images = append(images, img)
}

if len(images) < 5 {
return nil, errors.New("not enough images to perform noise reduction")
}

outputImage := images[0]

heigth := outputImage.Height
width := outputImage.Width
for _, img := range images {
if heigth != img.Height || width != img.Width {
return nil, errors.New("at least one image has a different width or height")
}
}

var wg sync.WaitGroup

for rowIndex := 0; rowIndex < heigth; rowIndex++ {
wg.Add(1)

go (func(rowIndex int) {
for colIndex := 0; colIndex < width; colIndex++ {
var pixels []Pixel
for _, img := range images {
pixels = append(pixels, img.Pixels[rowIndex][colIndex])
}

medPixel := medianPixel(pixels)
outputImage.Pixels[rowIndex][colIndex].set("R", medPixel.R)
outputImage.Pixels[rowIndex][colIndex].set("G", medPixel.G)
outputImage.Pixels[rowIndex][colIndex].set("B", medPixel.B)
}
wg.Done()
})(rowIndex)
}

wg.Wait()
return outputImage, nil
}

Finally putting everything together under one exported package function that reads an array of image filepaths, performs median filtering and writes the output image into a new file specified in `outputPath`.

// RemoveMovingObjs iterates the given filepaths and generates new image
// image that does not have the moving objects in the given images.
func RemoveMovingObjs(filepaths []string, outputPath string) error {
img, err := medianFilter(filepaths)
if err != nil {
return err
}
img.writeToFile(outputPath)
return nil
}

Using image filtering we are able to turn this:

Image for post
Image for post
20 frames of this gif was used to create the input images

Into this:

Image for post
Image for post
Median filtered image

That’s pretty much it. I wasn’t expecting something this cool to be that easy. This algorithm can be used to build a lot of interesting things. Maybe I would build an app around this some time.

The Startup

Medium's largest active publication, followed by +721K people. Follow to join our community.

Koray Göçmen

Written by

University of Toronto, Computer Engineering, architected and implemented reliable infrastructures and worked as the lead developer for multiple startups.

The Startup

Medium's largest active publication, followed by +721K people. Follow to join our community.

Koray Göçmen

Written by

University of Toronto, Computer Engineering, architected and implemented reliable infrastructures and worked as the lead developer for multiple startups.

The Startup

Medium's largest active publication, followed by +721K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store