Visual Inspection AI | A differentiated service for Manufacturing Industry| Google Cloud

Background:

Industry 4.0 is revolutionising the way companies manufacture, improve and deliver their products. Manufacturers are integrating cutting-edge new technologies, including Internet of Things (IoT), Edge Computing , AI and machine learning into their production facilities and throughout their operations.

Production quality and Yield are two of the industry’s top performance metrics. Poor production quality control results in significant operational and financial costs in the form of low yield, increased inventory, high recalls, claims, and repairs. In fact, the American Society for Quality estimates that for many organizations, cost of quality is as high as 15–20% of annual sales revenue, or billions of dollars annually for larger manufacturers.

Therefore, it’s extremely important for manufacturers to embrace the new age technology advancements and look at new ways to improve Product quality and Yield.

What is Visual Inspection AI?

Visual Inspection (VI) is the process of examining a component or piece of equipment using one’s naked eye to look for flaws.

Visual Inspection AI is a computer science discipline that trains machines to make sense of images and visual data the same way people do. Visual inspection AI uses the best of Deep Learning technologies to emulate Vision capabilities of human beings, and build a cognitive understanding of image attributes to point out anomalies in them, just as a person would but with much greater speeds and accuracies.

What are the problems with current Visual Inspection approaches?

• Manual: Most visual inspection jobs are done manually. This is time consuming, highly dependent of the skills of the operator, prone to errors, and also costly.
• Machines: Another approach is using Specialized inspection machines that use Ultrasonics, Radiography etc. These are extremely expensive and purpose-built to solve for very specific problems only and are not extensible across variety of usecases.
• AI: Generic Computer Vision models used for inspection are not purpose-built to solve for manufacturing usecases. These are models that are trained on a dataset for general purpose Vision AI usecases around face , object , sentiments, colors, places detection etc and therefore require extremely large datasets to train for Visual inspection usecases from scratch and even then struggle to deliver the required accuracies.

Introducing Google Cloud Visual Inspection AI (VIAI)

Google teams faced the same challenges as highlighted above, in its manufacturing of pixel phones and invested years in solving for the quality related challenges. Visual Inspection AI — a state-of-the-art deep learning based Visual inspection AI platform, is a culmination of above effort, that provides a purpose-built AI based quality inspection platform specifically designed to solve for the manufacturing problems.

Following are some of the key highlights of Visual Inspection AI Service:

• Superior computer vision and AI technology: VIAI customers improved accuracy by up to 10x compared with general purpose ML approaches. It can detect the tiniest defects by supporting ultra-high resolution images (up to 100M pixels) using Computer vision technology that Forrester ranked as the leader in the industry.
• Run autonomously on-premises: Manufacturers can run inspection models at the network edge or on-premises. The inspection can run either in Google Cloud or fully autonomous on your factory shop floor.
• Short time-to-value: Customers can deploy in weeks, rather than months with traditional machine learning (ML) solutions. Built for process and quality engineers, no computer vision or ML experience required.
• Get started quickly, with little effort: VIAI can build accurate models with up to 300x fewer human-labelled images than general purpose ML platforms. Legacy solutions require thousands of labelled images.
• Goes beyond anomaly detection: Unlike competing solutions that use simple anomaly detection, VIAI’s deep learning allows customers to train models that detect, classify, and precisely locate multiple defect types in a single image.
• Highly scalable deployment: Manufacturers can flexibly deploy and manage the lifecycle of ML models, scaling the solution across production lines and factories.

Like a lot other google services, Visual Inspection AI service is now available as a public service and its the same service that inspects millions of pixel phones as shown in the image above.

Which use cases does Visual Inspection AI solve for?

Google Cloud’s Visual Inspection AI platform solves for 3 distinct Manufacturing use cases :

1) Image Anomaly Detection:

Anomaly detection model detects anomalies at an image level. The training needs to include images with and without anomalies and the model will be able to distinguish between the same. Anomaly detection can be used across a variety of industrial use-cases like defective parts identification, parts wear and tear detection, deformation, packaging and labelling anomalies etc

2) Cosmetic inspection:

Cosmetic inspection locates even tiniest and most complex defects (dents, scratches, cracks, deformations, etc.) on any kind of surface. It has the capability to detect multiple defects types(scratch, dent)as well as identify multiple occurrences of the defects types which is very unique.

Cosmetic inspection can be used for a variety of use-cases like Paint shop surface inspection, body shop welding seam inspection, press shop inspection (scratch, dents, cracks, staining), product surface check (glue spill, mesh deformation, scratches, bubbles,) etc.

3) Assembly inspection

Assembly inspection detects even the most subtle defects at various stages of the assembly process (wrong, misplaced, missing, rotated, or deformed components).Assembly inspection can be used in the assembly areas of industrial production to check for packaging inspection, board assembly inspection etc

Wait! How is VIAI different from Google Cloud Vision AI or AutoML Vision?

Vision AI Vs Visual Inspection

Vision API offers powerful pre-trained Vision model hosted and trained by Google. It is used for generic Vision AI services like detecting objects, faces, sentiments, colors, reading text etc via REST API services. Its not purpose-built to solve for the manufacturing usecases and it also does provide a platform like Visual inspection to allow for training model using your own datasets.

AutoML Vision Vs Visual Inspection

AutoML Vision is a platform that allows for training of your own custom machine learning models. However, AutoML Image/Video service is best for creating generic Vision models for usecases like face, objection, places, sentiment detection etc.

Visual Inspection, on the other hand, while similar to AutoML platform, leverages industrial datasets and models, pioneered by Google, to solve for the three specific manufacturing usecases stated above. It’s a domain based industry solution as compared general purpose AutoML vision service.

OK! So, how does one go about building a Visual Inspection service?

VIAI platform provides 4 easy steps in building an VIAI service through an intuitive UI and does not require any code.

1. Image collection : The first step is collecting images and saving those them in Cloud storage. These images are for defective as well as non-defective items.
2. Image Labelling: Each image has to be annotated to identify and label the items as defective or correct. For Cosmetic Inspection, this could include annotating multiple defects types and occurrences.
3. Model Training: Once the labelling is complete, the model training phase commences. The training is broken down into several steps. The initial training uses an aspect of AI called active learning which helps suggest items to be labelled.The second step uses these newly labelled items to improve the accuracy.
4. Model deployment: On completion of the training, a new VIAI model is created and stored in the registry. The model is ready to be exported as a container to the plant or factory for deployment and live predictions.

Let’s go and build a model!

We will be building a Cosmetic Visual Inspection model to identify scratches and dents on the surface of the Pixel phone as shown below.

Step 1 : Image collection: Navigate to the Visual Inspection service in Google Console and click on Create Datasets.

Specify a dataset name →Select Cosmetic Inspection as Objective, Bounding Box as annotation type → Click Create. This will create a new dataset collection.

Next, upload the training images to the dataset by uploading it directly from your computer or via loading the images into GCS, creating a CSV and specifying the same in the import file path below.

On completion, all the images will be uploaded into the dataset.

Step 2 : Image Labelling (Annotations)

On the dataset screen, select Defects tab beside the browse option.

Click — Add Defect Type and add 2 types of defects: Dent and Scratch. The pixel phone surface has these two types of defects which we will annotate and map to the corresponding defect types.

Once complete, you will see them listed with different colors on the left hand side as shown below.

We will now begin with image annotation.Click on an image to open annotation view.Click “Add Bounding Box” icon from the palette.

Draw polygon around the defect on the pixel phone and map it to the configured defect types.

Repeat it for all the training images.

Now, we are ready with the training dataset, so let the model training begin!

Step -3 Model Training:

Click — Start Training button on the right side. This will kick start the training process in the background and depending on the number of images and complexity, it will take between 24 to 48+ hours to train the model.

On completion of training, a new custom cosmetic model for inspecting your pixel phone is created!

Click on the Model options on the left menu. Drill down inside the Model title you will see the model.

Evaluation: Click on the model name and it will take you to the evaluation page which contains details of the model evaluation metrics like Precision, Recall, Confusion matrix etc thereby providing a view of accuracy of the model.

Model Testing:

Now that the model is ready, lets test it before deploying it to production.

Click on the “Test & Use’ tab beside Evaluation to get to the following page.

To test the Visual inspection model, a Solution Artifact needs to be created. Solution artifact is the actual container containing our trained model.

Click on the “Create Solution Architect” option. It provides 2 types of provisioning.

1. Online Testing : This option is to create an online version of the model which you can use to test upto 20 images at no cost. Fill in details as shown below and click continue.

2. Deployment:

This is to create a deployment artifact for production. It takes in additional details like the number of camera streams and number of month of usage to derive cost of using this model. This is a fixed cost that allows for any number of images to be processed via the model as opposed to be charged on the number of images processed.

The deployment model will be typically created after the online-testing model created above is tested and good to go.

Fill in the appropriate details and click continue.

On completion of the above activity, the platform will create both the artifacts as shown below.

Click on the “Create Test” option on the screen.

Choose the online-testing model in the dropdown and provide the sample test images CSV and an output path for the model to write the output. Click create to run the batch.

The batch process will pick up each of the test image provided as an input, process it through the cosmetic inspection model and store the output jsonl file and the images into the output location specified.

To make it more intuitive, the platform has integrated an output page that shows all the defects identified in the image with its annotations.

To see this — Click on the path generated in the ‘Storage’ column and the following page appears showcasing the results of model inference on the test image, annotated with the defects types specified in right side.

Click on the “Preview Images” to change the image and review all the test images for the accuracy of cosmetic inspection model and see how the tiniest of the scratches and dents have been properly identified by the model- Bravo!

Step -4 Model Deployment:

The VIAI model can be deployed both on on Google Cloud or on-prem.

On-prem deployment:

Lets look at on-prem deployments as it is the most commonly requested option by customers.

Hardware Requirements:

Before the model can be deployed on-prem, the required hardware provisions needs to be done.

1. Edge server : We recommend Nexcom X300 Edge industrial PC with GPU as an optimum hardware specs for edge analytics. It supports all key camera stream protocols like Genicam, RTSP etc

2. Camera: Following is the proposed specification of the camera and it needs to support one of the above protocols.

3. Lighting: Lighting is key to get a consistent quality and a clear image of the source for the overall success of the model output.

End-to-End Reference Architecture:

VIAI Edge Set Up:

With the required hardware in place, VIAI provides an Edge Server software that can be installed on the industrial PC. The software stack is represented by the green block above. The software provisions all the required components on the edge through an automated script and it also provision the Google Cloud back-end services.

Our Cosmetic Model is now ready for prime-time. The model we generated through the “Create solution artifact” Deployment option above is stored in the Google Cloud Container registry. Using configuration, the Edge server pulls the container from the registry and deploys it on to the K8s Cluster on Edge server.

The images captured by the cameras from the production line can be processed through the model hosted on the VIAI server and the inference can be used to automated the Quality inspection.

At a periodic intervals, the inference details and training images can be sent over to the cloud for re-training the models and the refined model can be deploy back on to the edge server to constantly tune and improve on the model performance.

Hurray! You have successfully created and deployed a Cosmetic Inspection Model!!

Conclusion:

Advancements in Artificial Intelligence and Edge Computing have led to a whole new ways of looking at traditional challenges in the Quality and inspection areas of Manufacturing. By employing cutting-edge Deep Learning technologies in Visual inspection, industries can get much higher precision, scale, throughput at a lower cost and with little to no human interventions.

Google is a pioneer in building Artificial Intelligence based technology solutions like Visual Inspection. For more details, pls visit → https://cloud.google.com/solutions/visual-inspection-ai

Happy Edge Analytics!