If I had a manufacturing company and wanted to automate my final quality control, what would be the first step I take?

First and foremost, it is important that you, as managing director, production manager or head of quality assurance, think about what exactly the application is, i.e. at which point the quality control is still carried out manually. We don’t need to know much more at the beginning. Subsequently, we define what the process has looked like so far. For this purpose, we have a conversation where we answer the following questions: what are the requirements of quality control? What defects need to be detected? Is there already an error list? What cycle time is necessary? How fast do the products go through the inspection? How many parts do I produce? Are these always the same or very different parts? If we realize when answering these questions that there is a high potential for automation, then you have already taken the first step to work with us.

What happens next? What happens until the actual automation?

Once we have come together — via whichever channel — a first conversation takes place in which we talk about the use case. We discuss the questions mentioned earlier in detail and define together how the production volume, the error rate, the diversity and the distribution of the errors as well as the system requirements look like. In addition, you have to decide whether your quality control should be fully automated or whether you prefer a stand-alone solution. At this point it is helpful to already see pictures of good and defective products in order to get to know your specific use case.

Once all questions have been answered, we will always visit the factory. Every factory looks different and needs a solution that is individually tailored to the respective processes. That’s why we take a close look on site at how the process is running so far and what needs to be done.

If we want to work together, we start by using the already existing pictures or by taking the first pictures. To do this, we take good and defective products with us and consider which camera and which light are suitable. For this purpose, we also have a large network of machine vision experts who contribute their expertise.

Based on the products and the pictures of the products, we carry out a first feasibility study. The aim of the feasibility study, on the one hand, is to find out which hardware is suitable for your specific case and, on the other hand, whether your use case can be solved well with our software. In this stage, we are already able to train a first deep learning model and deliver results. If these initial results are positive, we proceed to a proof-of-concept phase. We order the required hardware and install it at your factory as a stand-alone solution. In this phase we do not interfere in the production process, instead the system runs independently, and we take pictures of the products on site. The pictures already have the quality they will have later on and we collect them in such a quantity that we are able to train a deep learning model.

Then we start to create the deep learning model. Our team of Data Scientists in our office in Berlin trains a model that is optimized for this particular application. Depending on the application, this process takes 1 to 2 weeks. The software is then placed on our AI-Box — a mini-computer designed for the use of our software — and tested in the factory. An expert from your company will also be present during the test to evaluate our classification. If our system says that this product is defective and belongs into defect category 3, for instance, he evaluates this assessment and agrees with it, or says no, this is actually defect category 2. This feedback makes the model much better and more accurate. We repeat this process until we have a model that operates within an accuracy range of over 99 percent.

And then comes the final factory acceptance test. We’ll make an appointment on site to test the model with new images it has never seen before. If everything went well from the first meeting to the factory visit, the feasibility study and the proof-of-concept, this is the starting signal for industrialization. In this step, we work together with system integrators, machine builders and machine vision experts who take care of the hardware and machine control, among other things. Hence, we offer you complete automation. After this last step, the system can be put into operation and runs automatically.

How long does it usually take from the first contact with deevio to the actual automation of the quality control?

There are two use cases: if the company already has a machine vision system and, therefore, already has pictures of the products, we can be very fast. We conduct a feasibility study within a week and carry out the proof-of-concept in 3 to 4 months, because the customer naturally wants to test for a correspondingly long time. Afterwards, we can go directly into industrialization. This takes another 1 to 2 months. All in all, you have the running system in 4 to 5 months. If you don’t have any hardware yet and you come into contact with the topic for the first time, we have to order the hardware first, which takes about a month, and then test and calibrate it. In total, it takes 6 to 7 months until your quality control is automated.

How does the transition from manual to automated quality control take place in my production and what do I have to keep in mind?

Above all, we should consider relatively early in the project what the final automation should look like in the end. There are two possibilities: one would be to fully automate. If that is the case, we work with an automation company to fully automate the visual inspections. The second option is to support the people who are responsible for quality control. To do this, we put together a system that does not replace the position but supports it. This system makes a recommendation, but the person working in quality control gets to decide. In this case, the production process does only change slightly. You can imagine it that way: I have the camera next to me, a screen is connected to it, a photo is triggered and I see the picture with the classification good or bad on the screen.

The Path to Automated Industrial Quality Control was originally published in deevioberlin on Medium, where people are continuing the conversation by highlighting and responding to this story.

Damian Heimel from deevio explains in which cases traditional machine vision reaches its limits and why it is not too late for companies that have relied on such a system and are dissatisfied with the result to switch to machine vision with deep learning.

Can you start by explaining what traditional machine vision means and how it works?

Machine vision means image processing in the industrial sector. It’s all about automating visual inspections, especially within the production line. For this purpose, cameras are integrated into the production line, images are taken and appropriate software is created. Until now, software development has been rule-based. You can imagine it this way: You have the image in front of you and define very specific rules for it. For example, that a scratch always goes from top left to bottom right or that the distance should always be 55 mm. Machine vision is already being used very frequently within the production line, which works very well. There are probably hardly any production lines left where this is not the case. Here, attention is normally paid to smaller defects in parts of the product at several points on the production line, and less frequently to entire products.

Then why are there so many companies that are sceptical about machine vision? Where does the dissatisfaction come from?

This is strongly dependent on the application. We would continue to recommend our customers to use rule-based machine vision for tasks such as measuring distances. However, there are some aspects that are difficult to cover with traditional image processing methods. These include, for example, the detection of scratches or dents.

Many companies have already invested in machine vision systems with very good hardware, but with a software that is not able to cover the variability of errors mentioned above. These companies have noticed that the systems show disproportionately high pseudo error rates, some of which are 30–50 percent. One reason for this is that the machine vision systems are often very sensitive and thus react strongly to small deviations. This can be an incoming beam of light or someone who accidentally hits the machine and thereby changes the camera angle. The resulting pseudo defect rate means that the inspection costs are not reduced as expected and the expensive systems are basically worthless. Instead, despite the investment in the system, the products have to be tested several times to prevent defects, which ultimately drives up the costs of the products. Therefore, many companies are sceptical about traditional machine vision systems.

What can these companies that have already invested in a machine vision system do?

We at deevio can help these companies. What we can do is that we train a properly functioning deep learning model with the images from the already installed system, which have a very good image quality. We install the deep learning model on one of our AI boxes consisting of graphics card and minicomputer and integrate it into the existing system. As our software learns, it is more flexible and can handle some use cases better. This allows us to reduce the pseudo error rate from 50 to up to 1 percent. Hence, we are able to retrofit the already existing system.

Why should a company invest in your solution when it already has a machine vision system?

On the one hand, because it is a relatively simple and uncomplicated process for the company. The hardware remains the same. Only the software is replaced. On the other hand, because the company can save the already spent investment costs and, in the end, has a working machine vision system that actually reduces the inspection costs.

The Limits of Traditional Machine Vision was originally published in deevioberlin on Medium, where people are continuing the conversation by highlighting and responding to this story.

With deevio you have decided to focus on a specific problem in a specific area. Why did you choose industrial quality control?

Europe — and especially Germany — has a large industrial base. As we wanted to use this base, we took a closer look at how production processes work today, how new technologies are used, but also where technologies are not yet used and which processes are still carried out manually. To get an overall picture, we went to factories to see for ourselves on site. We noticed that, despite the existence of new phenomena such as predictive maintenance and digital twin, reality in 2019 looks different and many processes are still manual.

One of the processes we have observed in various factories is manual quality control. We have seen time and again that there are people whose job it is to decide whether a product is good or bad. Considering the quality standards that German companies — in particular small and medium-sized businesses — have, we were surprised by this observation. After all, manual quality control entails several problems.

Can you describe these problems? What are the disadvantages of manual quality control?

On the one hand, manual quality control is extremely inefficient. If, for example, I have to judge a metal part according to its quality, a scratch looks different on a Monday morning than it does on a Friday night. I get tired, the lighting conditions are different, I may be particularly happy or sad and, thus, distracted. The resulting inconsistencies in the assessment of quality are a huge problem.

In addition, it is very difficult for smaller companies in structurally weak areas to find people who still want to do this job. Here in Berlin, in Hamburg, Munich or Cologne, this is not yet so serious, but when we look at the countryside in Saxony or Thuringia for example, things are different. Companies can no longer find anyone who wants to work in quality control because it is very strenuous. It’s impossible to concentrate for 8 hours on metal parts. One overlooks mistakes at some point. Once a company has found new employees, the training is also very time-consuming, as the large number of possible defects can only be seen by a trained eye.

We further noticed that there is no documentation of the decision-making processes. If you say that a metal part is broken, two hours later I can no longer understand why you said that. This lack of traceability is another problem and important information is lost which, if properly documented, could be used to continuously improve production processes.

All these aspects indicated that there is a huge potential for new solutions in industrial quality control, especially at the end of the production line. While the processes within the production line are often already automated, at the end of the line there are almost always people who carry out the visual inspection and decide whether a product is good or bad.

Quality control with camera technology and image processing has been in place for some time now. Why is there still manual quality control?

Visual inspections have indeed been carried out with machine vision for several years and the technology works very well in certain applications. Traditional machine vision reaches its limits, however, when there is a high variability of defects, e.g. when scratches on surfaces take on different shapes and colours or occur at different locations on the product. Such a variance can only be covered to a limited extent with traditional machine vision, since the algorithms are written rule-based and, therefore, rules for a scratch, such as the exact location, must be defined in advance. It is precisely with these defects on surfaces that we can see that machine vision systems have a high pseudo error rate. This means that the systems are set “too sharp” and mistakenly indicate good products as defective, which significantly increases the amount of rejects and leads to high follow-up costs in production. Despite the high acquisition costs of such systems, they are ultimately ineffective and worsen the condition rather than drive the desired automation of quality control.

The Status Quo in Industrial Quality Control was originally published in deevioberlin on Medium, where people are continuing the conversation by highlighting and responding to this story.

In today’s world, Artificial Intelligence is surrounding us everywhere. Could you give a short definition and explain why there is such a hype about AI?

AI is a very general term. It refers to decisions made by machines. What is fairly new in the realm of AI, and has been particularly successful in the last 10 years, is the concept of Machine Learning: the machine is not just making a decision, but it is learning through the decisions it makes. There are several ways to implement Machine Learning but one of the most successful ones is Deep Learning.

Deep Learning refers to algorithms trying to replicate the structure of the brain. All of this has happened very recently because of the incredible breakthrough in computing power.

And in which contexts and industries can AI and Deep Learning be applied?

Anytime you have an important breakthrough in technology, you try to apply it everywhere. The same has been happening here. Every market and every industry is trying to figure out how this technology can be helpful. The consumer industry is being the one driving this process in a lot of areas. For instance, when we speak into our phone and the phone translates what we say into text, that’s Deep Learning. And what we observe in this respect is that, when you start doing this, it’s not very accurate. You have to repeat the words and the sentences, and it makes a lot of mistakes. But the more you use it, the more accurate it becomes because it learns how you speak. This is probably the most prolific example of Deep Learning because everybody uses it. And that’s why it gets better and better so fast. Therefore, the companies with the best software in this field are the ones that have the most users giving them the data so they can train the software. This makes companies like Google, Apple, Microsoft and Facebook the key players on this market.

Who are the driving forces when we talk about AI on a global level? You already mentioned companies like Google and Facebook. So, it’s all about the data?

Let’s break it down. The fundamental science behind AI and Deep Learning is old. Then you have the technological implementation — the computer power — which is very recent. And the most important part is the data. So, the questions are: Who has the data? How do you use the data? And what kind of regulations, if any, are on the data? If you were to look at it globally for the consumer market, most of the data is where most of the people are. Which means that the countries where you expect Deep Learning to be more successful are the countries that are most populated. If you tried to measure it in terms of how broadly Deep Learning has been applied or in terms of how many patents a country has on the technology or how many papers have been written on the topic, China is way beyond everybody else because they have billions of people they can get the data from. This is the biggest advantage that China has in the consumer sector right now. And this tremendous advantage in demographic is coupled with a strong government policy allowing companies to do whatever they want with people’s data. The combination of a lot of people and no data privacy means that these machines can learn extremely fast.

So, what role does Europe play in the AI revolution, if any?

I think it depends on which market we focus on. In terms of population, we cannot compete with Asia. So clearly, in consumer-centric applications Europe cannot be the leader. Moreover, privacy is very important to Europeans and, especially in Germany, we are leading in privacy protection, which is a good thing in my opinion. Where I think we do have a chance to be the leader is in the manufacturing world. Europe was the first continent to be industrialized, so it’s really in our DNA to manage repetitive processes and to make the same thing millions of times all the same. We have the discipline for that. And most important: We know how to track it. If we now think in terms of data in the manufacturing industry, data that is pertinent to the production process, to quality control, to the output and to the incoming materials, then Europe is probably the continent where most of this data is stored. With the automotive industry in Germany leading the way, Europe has a tradition of process optimisation. Process optimisation relies on statistics and statistics relies on data. Which means that here in Europe we have a lot of data in the manufacturing world. Whereas China leads the consumer market because they have the consumer-related data, Europe could lead the manufacturing world because we have the manufacturing data. When it comes to applying Deep Learning, Machine Learning and AI in the manufacturing industry, Europe is the place to be. Take deevio for example. We do end-of-line quality control with Deep Learning and we think we are the best in the world at doing this. And we decided to do this in Berlin.

Why is Germany and especially Berlin an interesting location for you?

I was really impressed today when I saw this map of Saxony, which is not well-known for being a high-tech region, showing all the automotive companies situated there.

The density of factories in the Dresden-Leipzig-Chemnitz triangle is amazing. And that’s the same everywhere in Germany. That’s why in Germany: this is where the data is located. And why Berlin? Because once you have the data, you need people who can do something with it. People who can transform the raw data into actionable insights. And thanks to companies like Zalando and Spotify that are successful in the consumer sector, Berlin has these people: the data scientists. One of the reasons why we decided to start deevio in Berlin is to have access to this pool of talent because there are not many regions outside of China and the US where you can find experienced data scientists who are really top-notch. The combination of having the data and the data scientists in the same country is very powerful.

Is Europe a Leader or a Follower in the AI Revolution? was originally published in deevioberlin on Medium, where people are continuing the conversation by highlighting and responding to this story.