How high-speed global shutter image sensors reduce the burden on AI-based vision systems

Sajjad Hussain
Nov 5 · 4 min read
Image for post
Image for post

Vision sensors are becoming more and more important for data collection. Originally simple image sensors were developed for photography applications, and today image sensors are used to provide high-quality input to artificial intelligence (AI) and machine learning systems. These systems have become sophisticated decision-making entities that utilize new and innovative processor architectures.

Edge data collection

Although edge data acquisition devices are mainly analog in nature, the unique features of image sensors are:

Their output is time-division multiplexed on continuous dynamic optical input.They need to be able to maintain the integrity of the converted light input during output to provide image outputThe image output provided has the best quality and supports important processing

These requirements and subsequent results may have a significant impact on the accuracy of decisions made by the vision system, and this result defines the safety, reliability and profitability of the entire automation system.

Vision system based on machine learning

The emergence of machine learning has promoted the innovation of image sensors, and its performance level has been improved to support various applications. Visual input is high-fidelity data-what you see is the information entered into the system. Today, AI algorithms can detect, recognize and classify these inputs and generate accurate decision outputs. The reliability of these outputs depends on the quality of the inputs and the accuracy of their algorithms, as well as the neural networks that process these algorithms.

Vision systems based on machine learning and deep learning mainly use convolutional neural network (CNN) algorithms to create powerful automatic recognition expert systems. In these systems, increasing the depth of the CNN layer will improve the accuracy of inference, but more layers will also adversely affect the time it takes for these networks to learn during the training phase and the delay for the system to complete inference (don’t forget Combination will also affect the results and power consumption). Similarly, high-quality image output enables the vision system to carry a minimal set of CNN layers, but can also produce highly accurate inferences. While obtaining a smart system that can be quickly deployed at low cost and small size, it also achieves high performance and low power consumption, which brings significant benefits.

Image for post
Image for post
Typical Convolutional Neural Network (CNN)

Deep learning algorithms such as CNN are extremely resource-intensive. Today, there are various processing engines, including CPUs, GPUs, FPGAs, dedicated accelerators and the latest microcontrollers. Designing a CNN-based vision system also requires a powerful optimization library support. Covers from proprietary (such as MVTec’s HALCON & MERLIC, MATLAB’s deep learning toolbox or Cognex’s ViDi) to standard tools (such as OpenCV) and software and hardware integration functions. These choices are directly related to the time to market of the product. Resource-intensive processors usually require larger form factors, such as power consumption additional components for heat sinks, or only require larger free space to dissipate power through convection. An image sensor that provides high-quality output eliminates the need for expensive processors, expensive third-party libraries, and/or the creation of new libraries, as well as the expensive tools required to optimally combine hardware and software resources. In other words, these sensors greatly reduce the total cost of ownership (TCO) and increase the adoption rate in various applications and markets.

Image sensor input to machine learning system

There are quite high requirements for the image sensor output passed to the CNN layer, Global shutter can capture the scene and preserve the scene to minimize motion artifacts.High global shutter efficiency to ensure that the scene retained in each pixel will not be damaged by light input outside the light path of that pixel.Sufficiently large pixel size to support good image quality even under challenging light conditions.Low total noise in image output to ensure high integrity input

Low power consumption in running and standby states, to meet the typical challenges of camera systems where convective heat transfer is the norm.

Image for post
Image for post
Semiconductor AR0234CS 2.3 million pixel CMOS image sensor

Semiconductor AR0234CS 2.3 million pixel CMOS image sensor has a high data rate MIPI interface, which is very suitable for AI-based vision systems. Coupled with its high frame rate, low power consumption, full frame rate, and full resolution, vision system developers can allocate most of their time and power budget to the processor.

God Hardware

For hardware discussion

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store