A beginner’s guide to 3D machine vision cameras
Originally posted here: https://info.zivid.com/blog/a-beginners-guide-to-3d-machine-vision-cameras
What is a 3D vision camera? What considerations need to be made to choose the right 3D camera? How does it work once you get it?
This guide is created to help you answer the questions and provide the steps necessary to build an application using 3D machine vision technology.
What is a 3D machine vision camera?
Search for the term 3D cameras on, for instance, Amazon, and chances are you will see a big list of 360-degree consumer cameras for video and VR experiences.
For industrial use, though, 3D cameras refer to a machine vision sensor, a depth camera, specifically designed to present the environment with accurate depth information in three dimensions.
We won’t cover lidars or cameras used in vehicles here, but rather focus on the category of 3D cameras often integrated into manufacturing processing, production lines, or together with robots and industrial applications.
Why do we need a 3D camera in industrial automation applications?
2D cameras have been widely used in applications such as barcode reading, object tracking, and presence detection. As applications become more advanced and require accurate depth information, the need for 3D cameras grow. (Read this article for more information: Why 3D machine vision? What’s wrong with 2D machine vision?)
3D vision technology allows us to design automated task applications like bin-picking. We can capture objects like shiny parts or challenging environments where lighting is limited or excessive.
3D cameras and machine vision technology play a vital role as eyes for machines. Similar to a human, the information from these robot eyes can be processed by the brain (the computer). 3D cameras provide a digital representation, a point cloud, of the world and give the necessary depth information to our machine vision algorithms. With the two components, 3D sensor and vision algorithms, we can identify the size of an object, the color, and the distance between two points, just like we do with our own eyes.
The purpose of the 3D machine vision system is to enable an application to perform specific tasks faster, smarter, and more accurate. From a certain distance, large objects are easier to detect and recognize, and they require less accurate or lower resolution depth cameras. From the same distance, detecting a small object require much higher 3D cameras to produce high-quality point clouds from the sensor. This is why industrial 3D machine vision cameras are typically used in advanced bin-picking, piece picking in logistics, machine tending in manufacturing, and inspection applications.
With more manufacturing industries adopting robotic automation, the 3D machine vision market is expected to grow continuously in the future.
A 3D camera enabled bin-picking example
If you take an application like bin-picking, where we need to pick lots of small, randomly placed objects from a tote or a bin, a complete machine vision system typically requires a high-performance 3D camera to provide the point cloud quality needed for reliable detection and picking.
In a bin-picking solution, the computer processes the point cloud from the 3D camera to find accurate depth information for the next gripping or suction position. The 3D cameras are often mounted stationary above the target. For flexibility, more and more cameras are attached on or near the gripper on a robot. You can read more about the pros and cons of stationary and robot arm mounting here.
What are the key considerations when choosing a 3D machine vision camera?
The following aspects need to be considered when you compare 3D machine vision cameras from different vendors.
1) Type of Application
When you understand the scope of your application and what you want to achieve by adding a 3D vision sensor to your automation system, the selection process will be easier. If you’re unsure about the feasibility of application and the vision system, you should talk to the vendor as they often perform image studies for you.
Since the 3D camera is used to enable the detection, picking, and placing of objects, we can say that very simplified, you need a higher accuracy (precision and trueness) 3D camera for short distances and small objects.
If you have for instance an application that needs to separate similar types of groceries, color detection should be a requirement, and you need a 3D machine vision camera with native RGB colors.
Here you can read more about the typical requirements needed for
- bin-picking (single product in a bin/tote)
- piece picking (multiple SKUs or products in a bin/tote)
- machine tending
With a clear goal of using 3D machine vision technology in your application, you can start listing your requirements and decide what to prioritize. Which element matters most to you? Is it resolution, processing speed, field of view, or the ability to handle various materials?
As every use case is unique, a 3D machine vision camera that works for others may not work for you. Having your own checklist will save your time and effort when comparing different 3D camera solutions.
2) Accuracy
To compare the accuracy between 3D cameras, you need to understand the definition of different terms that are used to measure accuracy.
According to ISO 5725, the definition of accuracy is the combination for precision and trueness:
- Precision: Describing random errors, a measure of statistical variability.
- Trueness: Describing systematic errors, a measure of statistical bias.
- Accuracy: Describing the combination of random and systematic errors. Sum of Precision and Trueness.
Figure 1. ISO 5725
As you can see in Figure 1, accuracy includes both precision and trueness factors. Keep in mind that the accuracy numbers in documents will vary depending on conditions such as working distance, ambient temperature, ambient light, and camera settings.
Figure 2. 3D Camera Accuracy Test
3) Ease of use
There are three elements to make your development process more manageable: hardware, software, and documentation. From a hardware perspective, a 3D camera should be flexible enough to be installed on the robot, or stationary depending on your use case. In case of any twisting, bending, and pull force, the cable management needs to be designed for machine vision applications.
Machine vision software is an essential part of your application development and therefore, it needs to be tested before you choose your 3D camera. You may want to check if it is easy to set up, supports your programming language and have any built-in features for advanced image calibration and analysis. It is also recommended to read release notes to see bug fixes, new features, and the frequency of new releases.
Well-organized documentation will help you get started with your 3D camera and solve any issues quickly in the future. Scan through datasheets, training materials, examples and what kinds of support you can expect in the development phase.
4) Safety
If you are building a cobot (collaborative robot) application, keep in mind that your camera is eye-safe and less annoying for humans. A non-laser based camera can be advantageous as it prevents you from being exposed to laser light either from direct exposure or from reflections from shiny objects. Plus, you don’t have to wear laser safety glasses or hire an officer to control the laser hazards.
How can I get started with a 3D camera?
Once you have ordered and received a 3D camera, you are likely to take the following steps.
1. Install a software package
A 3D camera usually comes with a machine vision software package. You can download it to your PC to view images in the point cloud environment.
2. Mount your 3D camera
You can decide where to set up your camera depending on your use case and working environment. Test different positions to remove reflections and get high-resolution results.
3. Connect the 3D camera to your PC
Though it sounds easy, it is vital that you use the correct cable and connect it to your PC properly. This is because a poor setup might cause connectivity issues from potential data transfer errors that could result in loss of connection or corrupted 3D images.
4. Capture 3D images
Once you complete the initial setup, explore built-in features and settings available in your machine vision software. Try different angles and filters to get the best result out of a 3D image. As an example, check out what you can expect from our Zivid Studio software tool in this 3-min video:
5. Start developing with SDK
To start working on your application, you need to use the SDK of the camera to customize settings and access to tools, utilities, libraries and application examples.
And what’s Next?
We hope that this guide helps you find the right 3D machine vision camera and understand what to expect once you have received the camera. If you want to learn about our 3D camera offerings, please check out the resources below.
Sometimes, it’s best to see how the camera works and ask questions in person. You can book an online demo meeting with one of our engineers to learn more about our 3D camera solution.
