Image Processing Using FPGAs
Image processing is the new gateway for numerous applications like Face recognition, Driver-less vehicles, Vehicle and object identifications, etc. With this increase in the application, the hardware should be improved for increasing the latency and that due is filled by Field Programmable Gate Arrays(FPGAs).
Introduction
A Field Programmable Gate Array (FPGA) is increasingly used in image processing applications especially for real-time applications like Vehicle Number Plate Recognition, where latency and power are important parameters.
A Field Programmable Gate Array embedded with a camera is able to perform image processing rapidly as the image is streamed from the sensors, with the camera providing an output data stream, rather than a sequence of Images. Unfortunately, uploading software over FPGA often gives disappointing results as many image processing algorithms are for serial processing rather than parallel computing. This lead to novel algorithms and hardware computational architecture , both for image processing operation level and also at the application level.
Approaches
Programming an FPGA to accelerate complex algorithm is difficult, one of the four approaches which can be used:
- Custom hardware design of the algorithm using a hardware description language optimized for performance and resources.
- Implementing the algorithm by instantiating a set of application-specific intellectual property cores (from a library).
- Using high-level synthesis to convert a C-based representation of the algorithm to synthesizable hardware.
- Mapping the algorithm onto a parallel set of programmable soft-core processors.
Researches
One of the articles by Siddiqui, describes a design of an efficient 16-bit integer soft microprocessor, IPPro which is capable of operating at 337 MHz.The design uses 32 element memory for storing pixels and intermediate stage results and other 32-element kernel memory for storing filter coefficients and other parameters and constants. The exploitation of both data-level parallelism and task-level parallelism is demonstrated through the mapping of a K-means clustering algorithm onto the architecture, showing good scalability of processing speed with multiple cores. For the traffic sign recognition, it is divided into two IPPro cores and ARM processors. The colour conversion and morphological filtering stages mapped to the IPPro. In this, we can process parallelly and accelerate the tasks without making the hardware design more complex.
Garcia studied on image processing which takes a random frame , difficult to feed in FPGA. This demonstrates for many image sizes the difficult mapping is inefficient in terms of power utilization.
Shi et al worked on assembling the output window to allow arbitrary imahe width reconfiguration takes very few clock , rather than the conventional approach of reconfiguration results demonstrate better resource utilisation, higher throughput, and lower power than their earlier approach.
Window operation to an image outputs smaller width than the input. Prior work only considered border management using direct form filter.
In Computer Vision Segmenting from the background is important Chen et al. Address scalability issues with increasing image size by using super-pixels — small blocks of adjacent pixels that are treated as a single unit. As each super-pixel is considered to be either object or background, this means that fewerJ. Imaging 3 of 4 models need to be maintained (less memory) and fewer elements need to be classified (reduced computation time),they accelerated the computationally expensive steps of Gaussian filtering and calculating the mean and variance within each super-pixel with hardware, with the rest of the algorithm being realised on the on-chip CPU. The resulting system gave close to state-of-the-art classification accuracy.
Badawi and Bilal used k-means clustering to segment objects within video sequences. To reduce the computations the centre pixel is given an average value, this gives better performance than the conventional k-means algorithm , but with far less computations and power requirement.Zhou used generalized Laplacian of Gaussian for detecting nuclie for histopathology. Their FPGA gave modest acceleration over a software implemt on a high end computer.
Given a processed segmented image , a required task is to measure feature vectors of each connected component for analysis. Bailey and Khaiber claimed minimum latency and relatively few required resources. They used zig-zag scans instead of the traditional style. Real time application for image processing is for reducing band width for image transmission.. This eliminates the end-of-row processing for label resolution by integrating it directly within the reverse scan. The result is true single-pixel-per-clock-cycle processing, with no overheads at the end of each row or frame.
Finally, Wang et al introduced a new image compression codec which works efficiently with streamed image and minimized the perceptual distortion within the reconstructed image. This provides a visually lossless 4:1 compression.These relate to a perceptual model of contrast masking, allowing just noticeable distortion (JND) thresholds to be defined
Summing Up
The whole study indicates the diverse uses of FPGAs in the field of image processing with different approaches and fine tuning with the efficient results with latency as well as power requirements.The range of image processing techniques include filtering, segmentation, clustering, and compression. Applications include traffic sign recognition for autonomous driving, histopathology, and video compression.
References
Image Processing Using FPGAs Donald G. Bailey Department of Mechanical and Electrical Engineering, School of Food and Advanced Technology, Massey University, Palmerston North 4442, New Zealand; D.G.Bailey@massey.ac.nz Received: 6 May 2019; Accepted: 7 May 2019; Published: 10 May 2019.
Present in the Journal Of Imaging.
BY- Prithviraj Chauhan
