How to detect defects on images

Building a visual quality control system to check your products on a few clicks with Convolutional Neural Networks and Ximilar platform.

Photo by FWStudio

Building visual inspection system is the common problem in lot of factories and Machine Learning approach is scalable solution. Not only your production process can be automated, it can also create more high quality products …

That is why this blog post will be focused on recognizing defects on the images with Ximilar platform. We are going to show how easy is to build an image quality control model. The visual quality control can be built on other types of images like:

• any material/surface as wood, metal, concrete, leather, plastic, rubber, fabrics, …
• any products/objects as screws, automotive parts, toys, food products, …

Some of the images are from MVTec Anomaly Detection Dataset [1].

Who could benefit from visual inspections:

• factories, assembly lines
• insurance companies
• companies interested in industry 4.0

Let’s start with data …

We will play with Severstal dataset[2] published on Kaggle. This dataset contains flat sheet steel images. You can download the full dataset through the Kaggle page.

The image from Severstal dataset. Let’s say that the defect is located on the right side of the image!

Here, we are going to use cropped data to 256x256. Thanks to iafoss there is a kernel with already prepared cropped images, you can download it here[6].

We cut the images from the long flat to the short 256x256 and label them with OK vs DEFECT labels.

First, we need to login to app.ximilar.com then we need to select Recognition service on the dashboard page. Our task is to recognize OK patches vs DEFECT ones. In production the images will be always divided into this kind of patches, so we will be able to tell approximate location. We will talk about how to detect the exact DEFECT location in another blog post. We are going to create a Categorization Task with two labels OK and DEFECT.

Through the quick actions in the overview of Image Recognition service, click on the NEW TASK button.

Next step is to create a label “OK” and “DEFECT” and upload images:

Create two labels and add images to them with drag and drop.

Now just click the train button and wait. It can take from several minutes to hours in order to train the model. This depends on your pricing plan, training queue in front of your task and complexity of the task… Once it is trained you can inspect the model by click on the DETAIL button in the model list table.

Through the Classify in the left menu/panel, you can test your trained visual inspection model. If you are satisfied with the result you can deploy it to your application through the API. Otherwise iterate with the process of uploading more images and retraining the model. Some models could require hundreds or thousands of images. We store the last 5 models for you so you can A/B test them. For more information about API read our documentation. We are also providing SDK library for python.

Finding the exact location of the defect …

Finding the exact location of the DEFECT can be achieved with multiple approaches:

• Train object detection model to find bounding boxes of defects.

• Train pixel wised segmentation model

• Train unsupervised anomaly model (Auto-Encoder architecture) only on positive “OK” samples, the anomalies/defects are images with large reconstruction error.
• Train Anomaly GAN or f-AnoGAN (Generative Adversarial Network)
• Train simple CNN model for classification (OK vs DEFECT) and perform GRAD-CAM

We will explain all of these approaches in another blog post.

One interesting approach when we are working with a simple classification model is to perform GRAD-CAM method on the image. For this, we will need to write a little bit of code to extend our classification model not only to classify to two categories OK/DEFECT but also to find regions of the defects. Thanks to sicara project tf-explain[4] this is quite easy. The visualization with GRAD-CAM can look like this:

What part of the image leads model to tell that the product is a DEFECT.

GRAD-CAM method is currently the most famous one but other methods can be used also as Occlusion Sensitivity or GRAD-CAM++[5].

To sum up …

With Ximilar platform you can easily train accurate visual quality control models for your business. The models can be run in the cloud as an API endpoint. We are able to export the model for offline usage, so your factory doesn’t need to send the data over the internet. Do you need any advanced custom solution which utilizes computer vision and machine learning? Then contact us at info@ximilar.com!

In future blog posts, I’m gonna show you how you can create a more advanced system for visual inspection with Flows. We are building also machine learning services for other domains like Fashion. Check out our demo! If you are interested in visual quality control, please read the comprehensive article about it.

Flows — Evolution in building complex AI vision systems

Sources

[1] Paul Bergmann, Michael Fauser, David Sattlegger, Carsten Steger. MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection; in: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019

[2] https://www.kaggle.com/c/severstal-steel-defect-detection

[3] https://arxiv.org/abs/1610.02391

[4] https://github.com/sicara/tf-explain

[5] https://arxiv.org/abs/1710.11063

[6] https://www.kaggle.com/iafoss/severstal-fast-ai-256x256-crops