How high-speed global shutter image sensors reduce the burden on AI-based vision systems
Vision sensors are becoming more and more important for data collection. Originally simple image sensors were developed for photography applications, and today image sensors are used to provide high-quality input to artificial intelligence (AI) and machine learning systems. These systems have become sophisticated decision-making entities that utilize new and innovative processor architectures.
Edge data collection
Although edge data acquisition devices are mainly analog in nature, the unique features of image sensors are:
Their output is time-division multiplexed on continuous dynamic optical input.They need to be able to maintain the integrity of the converted light input during output to provide image outputThe image output provided has the best quality and supports important processing
These requirements and subsequent results may have a significant impact on the accuracy of decisions made by the vision system, and this result defines the safety, reliability and profitability of the entire automation system.
Vision system based on machine learning
The emergence of machine learning has promoted the innovation of image sensors, and its performance level has been improved to support various applications. Visual input is high-fidelity data-what you see is the information entered into the system. Today, AI algorithms can detect, recognize and classify these inputs and generate accurate decision outputs. The reliability of these outputs depends on the quality of the inputs and the accuracy of their algorithms, as well as the neural networks that process these algorithms.
Vision systems based on machine learning and deep learning mainly use convolutional neural network (CNN) algorithms to create powerful automatic recognition expert systems. In these systems, increasing the depth of the CNN layer will improve the accuracy of inference, but more layers will also adversely affect the time it takes for these networks to learn during the training phase and the delay for the system to complete inference (don’t forget Combination will also affect the results and power consumption). Similarly, high-quality image output enables the vision system to carry a minimal set of CNN layers, but can also produce highly accurate inferences. While obtaining a smart system that can be quickly deployed at low cost and small size, it also achieves high performance and low power consumption, which brings significant benefits.
Deep learning algorithms such as CNN are extremely resource-intensive. Today, there are various processing engines, including CPUs, GPUs, FPGAs, dedicated accelerators and the latest microcontrollers. Designing a CNN-based vision system also requires a powerful optimization library support. Covers from proprietary (such as MVTec’s HALCON & MERLIC, MATLAB’s deep learning toolbox or Cognex’s ViDi) to standard tools (such as OpenCV) and software and hardware integration functions. These choices are directly related to the time to market of the product. Resource-intensive processors usually require larger form factors, such as power consumption additional components for heat sinks, or only require larger free space to dissipate power through convection. An image sensor that provides high-quality output eliminates the need for expensive processors, expensive third-party libraries, and/or the creation of new libraries, as well as the expensive tools required to optimally combine hardware and software resources. In other words, these sensors greatly reduce the total cost of ownership (TCO) and increase the adoption rate in various applications and markets.
Image sensor input to machine learning system
There are quite high requirements for the image sensor output passed to the CNN layer, Global shutter can capture the scene and preserve the scene to minimize motion artifacts.High global shutter efficiency to ensure that the scene retained in each pixel will not be damaged by light input outside the light path of that pixel.Sufficiently large pixel size to support good image quality even under challenging light conditions.Low total noise in image output to ensure high integrity input
Low power consumption in running and standby states, to meet the typical challenges of camera systems where convective heat transfer is the norm.
Semiconductor AR0234CS 2.3 million pixel CMOS image sensor has a high data rate MIPI interface, which is very suitable for AI-based vision systems. Coupled with its high frame rate, low power consumption, full frame rate, and full resolution, vision system developers can allocate most of their time and power budget to the processor.