Featured Product
This Week in Quality Digest Live
Quality Insider Features
Laurie Flynn
Researchers find ways to lower US healthcare administration costs by analyzing other countries’ approaches
Nilanjana Dasgupta
When educators connect STEM courses with social good and community support, they advance diversity and talent
Matthias Gouthier
Digital technologies are dazzling, but so are the challenges—especially for customer service
Leon Chao
A millennial's take on real-world applications
Angie Basiouny
Three Wharton experts don’t hold out hope for change

More Features

Quality Insider News
Oak Ridge Lab is developing technology to coordinate solar microgrids
Monitors and replenishes all CNC machine sumps with optimized top-off ratios
System can pick parts, measure with robotic calipers and gauges, record data, and place and sort parts
The startup is developing a device that can collect tire debris, the second-largest cause of plastic pollution on the planet
Discovery breaks barrier to fast charging
US Dept. of Commerce issues seven grand challenges
1-nanometer axis resolution and less than +/– 0.002 mm line-form accuracy
New calibration software increases efficiency

More News

Anthony Tarantino

Quality Insider

Using Computer Vision AI to Automate Inspection

Removing the human factor can increase visual inspection accuracy

Published: Wednesday, March 16, 2022 - 13:03

This article is an extract from Smart Manufacturing, the Lean Six Sigma Way, Wiley, available May 2022

The U.S. and EU economies are facing a major labor shortage in manufacturing. A pre-Covid survey projected the need for more than four million new manufacturing jobs by 2030, but with a shortfall of more than two million. Given the growing labor shortage caused by Covid-19, the problem can be expected to worsen.2 The shortage in quality assurance will be even more critical as these positions require dedication and advanced skills beyond what many factory workers have. Even with the most dedicated and skilled inspectors, unacceptable error rates are common occurrences as people get distracted and fatigued.

Harish Jose writes that the Federal Aviation Administration (FAA) defines visual inspection as “the process of using the unaided eye, alone or in conjunction with various aids, as the sensing mechanism from which judgments may be made about the condition of a unit to be inspected.”4

Joseph Juran, a pioneer in the field quality assurance, argues in his Quality Handbook that 100 percent manual visual inspection can be expected to yield no more than 87 percent accuracy, and that it would require 300 percent manual visual inspection to yield 99.7 percent accuracy.3 Other studies argue that visual inspection error rates can be as low as 3 to 10 percent under optimal conditions, such as with skilled and experienced inspectors working in a comfortable environment with good lighting. On the other hand, error rates can be as high as 20 to 30 percent under suboptimal conditions.12

According to Jose, it has been found that as defect rates go down, inspection accuracy suffers, and as defect rates go up, inspection accuracy improves. Therefore, if you have high quality levels it is less likely that you will find defects.

There are several factors that affect physical inspection. In Harish’s Notebook, the author lists 41 factors affecting physical inspection. They include task (e.g. defect rate), individual (e.g. time in job), social (e.g. communication), environmental (e.g. lighting), and organizational (e.g. training).4 Of the 41 factors listed, 24 are human factors. In the age of smart manufacturing and smart technology, this list will be fundamentally simplified by eliminating the human factors.

So why is it so important to reduce factors? The answer is simple. Each factor presents its own unique risk, so eliminating factors eliminates risk. The example I liked to use is from my Wiley book, Essentials of Risk Management in Finance, and in my risk management classes at Santa Clara University. It is not manufacturing based, but simple to comprehend.

Consider the risk of the first solo nonstop flight over the Atlantic Ocean in May 1927. Chasing the $25,000 Orteig prize for a non-stop flight from New York to Paris, several aviators made the attempt... and died. Charles Lindbergh succeeded by eliminating as many factors as possible. Unlike the others and although the prize didn’t require it, he flew solo, eliminating the co-pilot as a factor; he flew a simple one-engine plane, eliminating multiple engines as a factor; he did not take a radio, eliminating the weight it would have added; finally, he oversaw all phases of the construction of his plane, eliminating many of the risks associated with the plane’s design.5

Automated visual inspection using smart technologies eliminates all human factors and focuses on a much shorter and more easily managed list of factors. With fewer factors and with the advantages of high-speed cameras, deep-learning artificial intelligence (AI), and edge computing, the quality and consistency rates of product inspections can approach 100 percent.

Automated Visual Inspection Factors

Lighting

Camera Location

Camera Specifications

How Fast is the Inspected Object Moving?

Location of the Defect

Complexity of the Defect

Level of Accuracy Required to Accept/Reject Parts

Figure 1: A list of typical factors found in automated visual inspections.

Some industries have automated visual inspection, as human inspection was not practical or too dangerous, and the direct and indirect costs of defects were high. The direct costs include warranty, rework, and replacement. The indirect costs can be much higher with lawsuits, government intervention, and major quality issues threatening brand acceptance.

Figure 2 below, from Jamshed Khan’s writing on Nanonets, shows a bottling line where automated visual inspection is essential to operations.6 No person or group of people can be expected to notice and remove cracked bottles running down a production line at high speed.


Figure 2: On this bottling line, no person or group of people can be expected to notice and remove cracked bottles as they fly by. Credit: Jamshed Khan, Nanonets.

In the bottling line example, cameras used in computer vision take 100 or more frames per second, deep learning analyzes the imagery and flags defects in one second, and an application programming interface (API) sends a signal to a robotic arm to immediately remove the defective item. While edge computing is used for all the immediate action, data sent to the cloud for analysis is used for continuous improvement.

Also from Khan’s article, Figure 3 shows the automated visual inspection adoption rates for various industries. The chart indicates adoption rates in manufacturing factories are low when compared to other industries (less than 10 percent).

 


Figure 3: Automated vision inspection adoption rates in various industries. Credit: Jamshed Khan, Nanonets.

This begs a question: If AI-based computer vision has so many advantages over human/manual visual inspection, why are adoption rates so low? After all, machine vision and machine learning have been used for years. The answer lies in the rapid advances in deep-learning technology that reduce the physical labor required to accurately inspect objects.

In Smart Manufacturing, the Lean Six Sigma Way, Steven Herman defines artificial intelligence as “software performing tasks traditionally requiring human intelligence to complete. Machine learning is a subset of artificial intelligence wherein software ‘learns’ or improves through data and/or experience. Deep learning is a subset of machine learning, usually distinguished by two characteristics: (1) presence of three or more layers and (2) automatic derivation of features.”1

Before deep learning came on the scene, computer vision typically used image-processing algorithms and methods. This required extracting image features, such as edges, colors, and corners of objects. This in turned required human intervention and labor. As a result, model reliability and accuracy depended on the features extracted and the methods used in the feature extraction. Haritha Thilakarathne, writing on NaadiSpeaks, describes the problems this presents: “The difficulty with this approach of feature extraction in image classification is that you have to choose which features to look for in each given image. When the number of classes of the classification goes up or the image clarity goes down, it’s really hard to come up with traditional computer vision algorithms.”7

According to Khan, deep learning uses neural networks, which contain thousands of layers that are good at mimicking human intelligence, in order to distinguish between parts, anomalies, and characters while tolerating natural variations in complex patterns (a major advantage over earlier technologies). As a result, deep learning gets closer to merging the adaptability of humans conducting visual inspection with the speed and robustness of computerized systems conducting visual inspection.

Below are examples of computer vision using deep learning algorithms for visual inspection in manufacturing.


Figure 4: How parts are classified on a printed board assembly (PCBA). Credit: Radiant Vision Systems 8


Figure 5: Verification of model numbers on an automotive part. Credit: Machine Vision Experts 9

 


Figure 6: Discovery of fabric defects. Credit: Machine Vision Experts 9


Figure 7: Detecting plastic bottle cap defects. Credit: MobiDev10

Conclusion

Deep learning teaches machines to learn by example, something that comes naturally to people. With hardware and software costs continuing to drop, manufacturing is given amazing new abilities, including distinguishing trends, recognizing images, and making intelligent decisions and predictions.

Automated visual inspection using deep learning has proven that it can overcome the limitations of human inspection and do so at lower costs and faster times than traditional manual methods. Examples of successful applications can be found in virtually every industry and in all stages of manufacturing and distribution.

The number of computer vision solution providers has grown rapidly. AI Startups published a list in August 2021 of its top 90 computer vision startups11 around the globe, including Australia, Bangladesh, Belarus, Canada, Chile, China, France, Germany, India, Israel, Netherlands, Russia, Singapore, Sweden, Switzerland, Taiwan, Turkey, the United Kingdom, Ukraine, and the United States. The list is far from being complete, as I know at least another dozen startups in Silicon Valley focused on computer vision. With so many organizations working to improve automated inspection technology, solution capabilities are guaranteed to grow quickly and make computer vision affordable for even the smallest of manufacturers.

References

1. Tarantino, Anthony. Smart Manufacturing, the Lean Six Sigma Way. Wiley, 2022.
2. Deloitte. “2018 skills gap in manufacturing study.” Perspectives (accessed August 22, 2021).
3. De Feo, Joseph. Juran’s Quality Handbook: The Complete Guide to Performance Excellence. McGraw-Hill, 2016.
4. Jose, Harish. “100% Visual Inspection—Being Human.” Harish’s Notebook (accessed August 22, 2021).
5. Tarantino, Anthony. (2011) Essentials of Risk Management in Finance. Wiley, 2010.
6. Khan, Jamshed. (May 2021). “Everything you need to know about Visual Inspection with AI.” Nanonets (accessed August 21, 2020).
7. Thilakarathne, Haritha. “Deep Learning Vs. Traditional Computer Vision.” NaadiSpeaks. August 12, 2018 (accessed August 22, 2021).
8. “Applications.” Radiant Vision Systems (accessed August 22, 2021).
9. “Top 10 Deep Learning application types in industrial vision systems.” Machine Vision Experts (accessed August 22, 2021).
10. Krasnokutsky, Evgeniy. “AI Visual Inspection For Defect Detection.” MobiDev. August 26, 2021.
11. “Top 107 Computer Vision startups.” AI Startups. February 3, 2022.
12. Solving for the Limits of Human Visual Inspection.” Creative Electron.

Discuss

About The Author

Anthony Tarantino’s picture

Anthony Tarantino

Anthony G. Tarantino, Ph.D., CPIM, CPM, began his career with 25 years in manufacturing supply chain operations. Over the next 15 years he led dozens of supply chain, lean/continuous improvement, and risk management projects for KPMG Consulting/BearingPoint, IBM, and Cisco. While at Cisco, he became a Lean Six Sigma Master Black Belt. Over the last five years, he has supported multiple smart manufacturing startups focused on applying computer vision to help companies improve efficiencies, quality, and safety. He has been an adjunct faculty member at Santa Clara University for 12 years, developing lean/continuous improvement training and certification programs for students and corporate clients. The author of more than 20 articles, along with five books for Wiley and Sons, his latest book, Smart Manufacturing, the Lean Six Sigma Way, will be released in May 2022.

Comments

Visual Inspection Effectiveness

Dear Sir, I would encourage you to read further on human visual inspection effectiveness by obtaining the text, "Improving the Effectiveness of Visual Inspection" published by American Foundry Society in 2018. I would further suggest that your thoughts be refined by the several peer-reviewed journal articles on the subject I have written for AFS Transactions, available in the on-line library of American Foundry Society (www.afsinc.org). A recent paper, "Quality Management Perspectives on the Use of Machine Vision and AI-Guided Inspections," published in 2021, would be of direct relevance to your column.