陶显, 侯伟, 徐德. 基于深度学习的表面缺陷检测方法综述. 自动化学报, 2021, 47(5): 1017−1034 doi: 10.16383/j.aas.c190811 引用本文: 陶显, 侯伟, 徐德. 基于深度学习的表面缺陷检测方法综述. 自动化学报, 2021, 47 (5): 1017−1034 doi: 10.16383/j.aas.c190811 Tao Xian, Hou Wei, Xu De. A survey of surface defect detection methods based on deep learning. Acta Automatica Sinica, 2021, 47(5): 1017−1034 doi: 10.16383/j.aas.c190811 Citation: Tao Xian, Hou Wei, Xu De. A survey of surface defect detection methods based on deep learning. Acta Automatica Sinica, 2021, 47 (5): 1017−1034 doi: 10.16383/j.aas.c190811 陶显, 侯伟, 徐德. 基于深度学习的表面缺陷检测方法综述. 自动化学报, 2021, 47(5): 1017−1034 doi: 10.16383/j.aas.c190811 引用本文: 陶显, 侯伟, 徐德. 基于深度学习的表面缺陷检测方法综述. 自动化学报, 2021, 47 (5): 1017−1034 doi: 10.16383/j.aas.c190811 Tao Xian, Hou Wei, Xu De. A survey of surface defect detection methods based on deep learning. Acta Automatica Sinica, 2021, 47(5): 1017−1034 doi: 10.16383/j.aas.c190811 Citation: Tao Xian, Hou Wei, Xu De. A survey of surface defect detection methods based on deep learning. Acta Automatica Sinica, 2021, 47 (5): 1017−1034 doi: 10.16383/j.aas.c190811 作者简介:

陶显：中国科学院自动化研究所副研究员. 2016年获得中国科学院自动化研究所博士学位. 主要研究方向为机器视觉, 缺陷检测和深度学习. 本文通信作者.E-mail: [email protected]

侯伟：中国科学院大学人工智能学院博士研究生. 2009年和2014年分别获得兰州大学学士和硕士学位. 主要研究方向为缺陷检测, 计算机视觉, 图像处理和机器学习.E-mail: [email protected]

徐德：中国科学院自动化研究所研究员. 1985年和1990年分别获得山东工业大学学士和硕士学位. 2001年获得浙江大学博士学位. 主要研究方向为机器人视觉测量, 视觉控制, 智能控制, 视觉定位, 显微视觉, 微装配.E-mail: [email protected]

More Information Author Bio: TAO Xian Associate professor at the Institute of Automation, Chinese Academy of Sciences. He received his Ph.D. degree at the Institute of Automation, Chinese Academy of Sciences in 2016. His research interest covers machine vision, defect detection, and deep learning. Corresponding author of this paper

HOU Wei Ph.D. candidate at the School of Artificial Intelligence, University of Chinese Academy of Science. He received his bachelor degree and master degree from Lanzhou University in 2009 and 2014, respectively. His research interest covers defect detection, computer vision, image processing, and machine learning

XU De Professor at the Institute of Automation, Chinese Academy of Sciences. He received his bachelor degree and master degree from Shandong University of Technology in 1985 and 1990, respectively, and received his Ph.D. degree from Zhejiang University in 2001. His research interest covers robotics and automation such as visual measurement, visual control, intelligent control, visual positioning, microscopic vision, and microassembly

近年来, 基于深度学习的表面缺陷检测技术广泛应用在各种工业场景中. 本文对近年来基于深度学习的表面缺陷检测方法进行了梳理, 根据数据标签的不同将其分为全监督学习模型方法、无监督学习模型方法和其他方法三大类, 并对各种典型方法进一步细分归类和对比分析, 总结了每种方法的优缺点和应用场景. 本文探讨了表面缺陷检测中三个关键问题, 介绍了工业表面缺陷常用数据集. 最后, 对表面缺陷检测的未来发展趋势进行了展望.

深度学习 /  表面缺陷检测 /  机器视觉 /  卷积神经网络 Abstract:

In recent years, surface defect detection techniques based on deep learning have been widely used in various industrial scenarios. This paper reviews the latest works on deep learning based surface defect detection methods. They are classified into three categories: full-supervised learning model method, unsupervised learning model method and other methods. The typical methods are further subdivided and compared. The advantages and disadvantages of these methods and their application scenarios are summarized. This paper analyzes three key issues in surface defect detection and introduces common data sets for industrial surface defects. Finally, the future development trend of surface defect detection is predicted.

Key words: Deep learning /  surface defect detection /  machine vision /  convolutional neural network (CNN)  代表子方法 优点 缺点 直接分类 结构经典, 也是其他分类网络子方法
的基础, 可参考诸多现成网络 缺陷在图像中需要占一定比例, 否则其特征容易被池化掉, 同时一般
一幅图像中只容许存在一种类别的缺陷 (多标签分类除外) 定位 ROI 后分类 获取 ROI 的缺陷信息 需借助其他方法获取 ROI 多类别分类 一定程度上解决样本不平衡问题 网络采用二级训练 滑动窗口 在大图中实现缺陷的粗定位 滑动窗口尺寸需要准确选择, 且只能获得较粗位置, 遍历滑动速度慢 热力图 得到较为精准的缺陷区域 缺陷精确定位效果依赖网络分类性能 多任务学习 联合其他网络同时获取缺陷精确位置
和类别, 也能减少所需训练样本数目 网络结构相对复杂, 在添加分割分支时, 需要逐像素的标签 做特征提取器 获取有效的缺陷特征 依赖其他分类器才能获得最终分类结果 对比项目 传统基于图像处理的方法 深度学习方法 方法 1) 结构法: 边缘、骨架、形态学等 基于卷积神经网络 CNN 2) 统计法: 直方图、局部二值化特征 LBP、纹理特征、灰度共生矩阵 GLCM 等 3) 滤波法: 空间滤波、频域滤波 (傅里叶、gabor、小波) 等 4) 模型法: 随机场模型、反散射模型和分形体等 本质 人工设计特征 + 分类器 (或规则) 从大量数据中自动学习特征 所需条件 相对苛刻的成像环境要求, 缺陷和非缺陷区域之间的高对比度, 少噪声 足够的学习数据和高性能运算单元 适应性 差 (成像环境变化或缺陷类型变化时往往需要更改阈值或重新设计算法) 相对强 (能够应对一定的工业检测环境变化) 方法 应用场景 数据集名称 链接 分类 钢材表面 NEU-CLS ^{[ 51 ]} http://faculty.neu.edu.cn/yunhyan/NEU_surface_defect_database.html 太阳能板 elpv-dataset ^{[ 109 ]} https://github.com/zae-bayern/elpv-dataset 金属表面 KolektorSDD ^{[ 40 ]} http://www.vicos.si/Downloads/KolektorSDD 木材表面 wood defect database ^{[ 137 ]} http://www.ee.oulu.fi/olli/Projects/Lumber.Grading.html 定位 钢材表面 NEU-DET ^{[ 51 ]} http://faculty.neu.edu.cn/yunhyan/NEU_surface_defect_database.html 铸件X射线图像 GDXray Casting ^{[ 138 ]} https://domingomery.ing.puc.cl/material/gdxray/ 分割 磁瓦表面 Magnetic-tile-defect-datasets ^{[ 67 ]} https://github.com/abin24/Magnetic-tile-defect-datasets. 钢轨表面 RSDDs dataset ^{[ 139 ]} http://icn.bjtu.edu.cn/Visint/resources/RSDDs.aspx 地面裂纹 Crack_Dataset ^{[ 140 ]} https://drive.google.com/drive/folders/1cplcUBmgHfD82YQTWnn1dssK2Z_xRpjx 桥梁裂缝 Bridge Cracks ^{[ 141 ]} https://github.com/maweifei/BridgeCrack_Image_Data 孪生网络 PCB 板 PCB Dataset ^{[ 90 ]} https://github.com/tangsanli5201/DeepPCB 无监督学习 多种材质缺陷 MVTec AD ^{[ 142 ]} http://www.mvtec.com/company/research/datasets 扫描隧道显微镜成像
SEM 材料表面 NanoTWICE ^{[ 143 ]} http://www.mi.imati.cnr.it/ettore/NanoTWICE/ 弱监督学习 纹理缺陷 DAGM 2007 ^{[ 144 ]} https://hci.iwr.uni-heidelberg.de/node/3616 中国产业信息网. 随着现代工业自动化技术日趋成熟到2020年全球机器视觉行业市场规模将达到125亿美元2025年将超过192亿美元. [Online], available: http://www.chyxx.com/industry/201901/705852.html , October 20, 2019

China Industry Information Network. With the maturity of modern industrial automation technology, the global machine vision industry market will reach US$ 12.5 billion by 2020, and it will exceed US$ 19.2 billion by 2025. [Online], available: http://www.chyxx.com/industry/201901/705852.html , October 20, 2019 Nagata F, Tokuno K, Nakashima K, Otsuka A, Ikeda T, Ochi H, et al. Fusion method of convolutional neural network and support vector machine for high accuracy anomaly detection. In: Proceedings of the 2019 IEEE International Conference on Mechatronics and Automation (ICMA). Tianjin, China: IEEE, 2019. 970−975 Youkachen S, Ruchanurucks M, Phatrapomnant T, Kaneko H. Defect segmentation of hot-rolled steel strip surface by using convolutional auto-encoder and conventional image processing. In: Proceedings of the 10th International Conference of Information and Communication Technology for Embedded Systems (IC-ICTES). Bangkok, Thailand: IEEE, 2019. 1−5 罗菁, 董婷婷, 宋丹, 修春波. 表面缺陷检测综述. 计算机科学与探索, 2014, 8 (9): 1041−1048 doi: 10.3778/j.issn.1673-9418.1405007

Luo Jing, Dong Ting-Ting, Song Dan, Xiu Chun-Bo. A review on surface defect detection. Journal of Frontiers of Computer Science and Technology, 2014, 8 (9): 1041−1048 doi: 10.3778/j.issn.1673-9418.1405007

地址：北京中关村东路95号 邮政编码：100190 E-mail： [email protected]

电话：010-82544677 (日常咨询和稿件处理)， 010-82544653(费用管理、寄刊)

北京仁和汇智信息技术有限公司 开发 技术支持： [email protected]