OpenCV Decorator for easy Image and Video Testing

In Computer Vision, when we test our image processing models like VGG-16, Yolo, Res-Net101 or a custom made model, we face a repetitive task of looking at image using cv2.imshow() function. Let me elaborate, in computer vision for doing Proof-Of-Concept first we synthesize our model. Then for the testing part, we put our model in a function that takes a single image as an argument and returns the bounding boxes or some kind of identifiers drawn onto the image itself for debugging. And finally, we see that debugged image using cv2.imshow() function.

If you are new to OpenCV, go through this link: https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_gui/py_image_display/py_image_display.html

If the processing done by the model(function) on a single image works fine then we move forward to test it on video stream. And this is the part where Decorator comes into play. We can use a decorator function like I’ve made, which will add functionality to our model-function to take a video stream as input and show debugged output by just adding a decorator(“@decorator”) line above our model-function!!

To explain it more clearly, I’ve shown a simple example of a function that returns a grayscale image when a color image is passed in. Now we want to modify our function such that it will input a color video and return a black-and-white video and showing the result using cv2.imshow() at the same time… My decorator will do it for you in no time!!

Defining a basic image processing function

Example of image processing function processFrame which will return the processed frame (here doing BGR to Grayscale conversion but we usually do operations like passing the frame in “Res-Net-101” ):-

def processFrame(frame):
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    return [frame,gray]

Testing function on single image

#reading image from disk
inputFrame= cv2.imread('testColorImage.jpg')
#processing the input image
img= processFrame(inputFrame)
#showing image that we got after processing
cv2.imshow('inputImage',inputFrame)
cv2.imshow('processedImage',img)
cv2.waitKey(0)
cv2.destroyAllWindows()

Testing function on video feed

* Traditional approach is to put our function inside below code and copy paste rest of the code every time we test new functions :-

import numpy as np
import cv2
#videoPath is 0
cap = cv2.VideoCapture(0)
while(True):
    # Capture frame-by-frame
    ret, frame = cap.read()
    # Our operations on the frame come here
    inputFrame,outputFrame= processFrame(frame)
    # Display the resulting frame
    cv2.imshow('inframe',inputFrame)
    cv2.imshow('outframe',outputFrame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()

* Decorator’s Approach — This is what we can do if we use decorator:-

import numpy as np
import cv2
@testFuncOnVideo 
processFrame(frame):
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    return [frame,gray]
processFrame(videoPath=0)

above code with decorator will convert this function which takes only a single frame, now will process video feed coming from videoPath source and also show the output of the processed video.

We just need to import testFuncOnVideo function and use it as a decorator whenever needed. The code for this function is below, I’ve just modified the Traditional approach and tried to return a wrapper which will mutate out function from processing image to processing video.

def testFuncOnVideo(function):
    #defining a wrapper function
    def coverFunction(*args,**kwargs):
        try:
            cap = cv2.VideoCapture(kwargs['videoPath'])
            while(True):
                # Capture frame-by-frame
                ret, frame = cap.read()
                if frame is not None:
                    # Our operations on the frame comes here
                    output= function(frame)
                    #output should be a list
                    if type(output)==type(list()):
                        for i,image in enumerate(output):
                            if image is not None:
                                cv2.imshow('frame'+str(i),image)
                    if cv2.waitKey(1) & 0xFF == ord('q'):
                        break
            # When everything done, release the capture
            cap.release()
            cv2.destroyAllWindows()
        except KeyboardInterrupt:
            pass
    return coverFunction

So, just like this example, we can find other suitable places in the code part where we see the repetition and implement a decorator. But it should be clear what it does actually. Above code where we used @ operator, that code is equivalent to:-

@testFuncOnVideo 
processFrame(frame):
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    return [frame,gray]
||           ||
processFrame(frame):
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    return [frame,gray]
processFrame = testFuncOnVideo(processFrame)

To read more about decorators:-https://www.geeksforgeeks.org/decorators-in-python

Summary

• While coding we come across a situation where we need to add features which are not crucial to the function and in this scenario decorators work perfectly without affecting function’s original structure
• I found a sweet spot in OpenCV where we can take the help of the decorator to make our debugging and testing life easier by using @testFuncOnVideo. One point to remember is that the host function should return a list of images.