陈晋音, 吴长安, 郑海斌, 王巍, 温浩. 基于通用逆扰动的对抗攻击防御方法. 自动化学报, 2023, 49(10): 2172−2187 doi: 10.16383/j.aas.c201077 引用本文: 陈晋音, 吴长安, 郑海斌, 王巍, 温浩. 基于通用逆扰动的对抗攻击防御方法. 自动化学报, 2023, 49 (10): 2172−2187 doi: 10.16383/j.aas.c201077 Chen Jin-Yin, Wu Chang-An, Zheng Hai-Bin, Wang Wei, Wen Hao. Universal inverse perturbation defense against adversarial attacks. Acta Automatica Sinica, 2023, 49(10): 2172−2187 doi: 10.16383/j.aas.c201077 Citation: Chen Jin-Yin, Wu Chang-An, Zheng Hai-Bin, Wang Wei, Wen Hao. Universal inverse perturbation defense against adversarial attacks. Acta Automatica Sinica, 2023, 49 (10): 2172−2187 doi: 10.16383/j.aas.c201077 陈晋音, 吴长安, 郑海斌, 王巍, 温浩. 基于通用逆扰动的对抗攻击防御方法. 自动化学报, 2023, 49(10): 2172−2187 doi: 10.16383/j.aas.c201077 引用本文: 陈晋音, 吴长安, 郑海斌, 王巍, 温浩. 基于通用逆扰动的对抗攻击防御方法. 自动化学报, 2023, 49 (10): 2172−2187 doi: 10.16383/j.aas.c201077 Chen Jin-Yin, Wu Chang-An, Zheng Hai-Bin, Wang Wei, Wen Hao. Universal inverse perturbation defense against adversarial attacks. Acta Automatica Sinica, 2023, 49(10): 2172−2187 doi: 10.16383/j.aas.c201077 Citation: Chen Jin-Yin, Wu Chang-An, Zheng Hai-Bin, Wang Wei, Wen Hao. Universal inverse perturbation defense against adversarial attacks. Acta Automatica Sinica, 2023, 49 (10): 2172−2187 doi: 10.16383/j.aas.c201077 作者简介:

陈晋音：浙江工业大学网络空间安全研究院和信息工程学院教授. 2009年获得浙江工业大学博士学位. 主要研究方向为人工智能安全, 图数据挖掘和进化计算. 本文通信作者.E-mail: [email protected]

吴长安：浙江工业大学硕士研究生. 主要研究方向为深度学习, 计算机视觉, 对抗攻击和防御. E-mail: [email protected]

郑海斌：浙江工业大学信息工程学院博士研究生. 主要研究方向为深度学习, 人工智能安全, 对抗攻击和防御, 图像识别. E-mail: [email protected]

王巍：中国电子科技集团公司第三十六研究所研究员. 主要研究方向为无线通信分析, 网络安全. E-mail: [email protected]

温浩：重庆中科云从科技有限公司高级工程师. 主要研究方向为量子通信, 计算机通信网络与大规模人工智能计算. E-mail: [email protected]

More Information Author Bio: CHEN Jin-Yin Professor at the Institute of Cyberspace Security and the College of Information Engineering, Zhejiang University of Technology. She received her Ph.D. degree from Zhejiang University of Technology in 2009. Her research interest covers artificial intelligence security, graph data mining, and evolutionary computing. Corresponding author of this paper

WU Chang-An Master student at the College of Information Engineering, Zhejiang University of Technology. His research interest covers deep learning, computer vision, adversarial attack and defense

ZHENG Hai-Bin Ph.D. candidate at the College of Information Engineering, Zhejiang University of Technology. His research interest covers deep learning, artificial intelligence security, adversarial attack and defense, and image recognition

WANG Wei Researcher at the 36th Research Institute of China Electronics Technology Group Corporation. His research interest covers wireless communication analysis and network security

WEN Hao Senior engineer at Chongqing Zhongke Yuncong Technology Co., Ltd.. His research interest covers guantum communication, computer communication networks, and large-scale artificial intelligence computing

现有研究表明深度学习模型容易受到精心设计的对抗样本攻击, 从而导致模型给出错误的推理结果, 引发潜在的安全威胁. 已有较多有效的防御方法, 其中大多数针对特定攻击方法具有较好防御效果, 但由于实际应用中无法预知攻击者可能采用的攻击策略, 因此提出不依赖攻击方法的通用防御方法是一个挑战. 为此, 提出一种基于通用逆扰动(Universal inverse perturbation, UIP)的对抗样本防御方法, 通过学习原始数据集中的类相关主要特征, 生成通用逆扰动, 且UIP对数据样本和攻击方法都具有通用性, 即一个UIP可以实现对不同攻击方法作用于整个数据集得到的所有对抗样本进行防御. 此外, UIP通过强化良性样本的类相关重要特征实现对良性样本精度的无影响, 且生成UIP无需对抗样本的先验知识. 通过大量实验验证, 表明UIP在不同数据集、不同模型中对各类攻击方法都具备显著的防御效果, 且提升了模型对正常样本的分类性能. 深度学习 /  通用逆扰动 /  对抗样本 / Abstract: Existing studies have shown that deep learning models are vulnerable to carefully crafted adversarial sample, leading to wrong decision by the model, which will cause potential security threats. Many effective defense methods have been proposed, most of which have good defense effects against specific attack methods. However, since the possible strategies of attackers cannot be predicted in practical applications, it is a challenge to propose a general defense that does not rely on attack methods. This paper proposes a defense method based on universal inverse perturbation (UIP), which generates universal inverse perturbation by learning important features of classes in the original data. UIP is universal to data and attack methods, that is, one UIP can realize defense against all samples obtained by different attack methods acting on the entire data set. In addition, UIP can guarantee the accuracy of benign samples by enhancing the important characteristics of the benign samples, and the generation of UIP does not require prior knowledge of adversarial samples. Extensive experiments are carried out to testify that UIP has a significant defense effect against various attack methods in different data sets and different models, and the model＇s classification performance for normal samples is improved as well. Key words: Deep learning /  universal inverse perturbation (UIP) /  adversarial example /  general defense  DSRMNISTFMNISTCIFAR-10ImageNetAlexNetLeNetM_CNNAlexNetF_CNNVGG19VGG19 良性样本识别准确率92.3495.7190.4589.0187.4279.5589.00FGSM ^{[ 8 ]} 73.3185.2177.3579.1580.0578.1343.61BIM ^{[ 18 ]} 99.30 93.7399.11 95.28 97.61 85.32 72.90 MI-FGSM ^{[ 9 ]} 69.6590.3298.9988.3585.7556.9344.76PGD ^{[ 17 ]} 99.3195.93 99.19 97.8095.8381.0573.13C&W ^{[ 19 ]} 99.3496.0492.1096.4494.4480.6746.67L-BFGS ^{[ 6 ]} 98.5870.1267.7966.3571.7568.6931.36JSMA ^{[ 10 ]} 64.3355.5976.6172.3169.5160.0437.54DeepFool ^{[ 20 ]} 98.98 97.98 94.5293.5491.6383.1362.54UAP ^{[ 15 ]} 97.4697.0999.3997.8596.5583.0772.66Boundary ^{[ 12 ]} 93.6394.3895.7292.6791.8876.2168.45ZOO ^{[ 11 ]} 77.3875.4376.3968.3665.4261.5854.18AGNA ^{[ 21 ]} 75.6976.4081.6064.8072.1462.1055.70AUNA ^{[ 21 ]} 74.2073.6578.5365.7562.2062.7052.40SPNA ^{[ 21 ]} 92.1088.3589.1777.5874.2672.9060.30 第1组 第2组 第3组 第4组良性样本类标置信度 (良性样本 + UIP)类标置信度 对抗样本类标置信度 (对抗样本 + UIP) 类标置信度 01.00001.00050.539000.980411.00011.00080.490610.984821.00021.00010.501520.984131.00031.00070.502930.954941.00041.00090.514640.976151.00051.00030.502050.944261.00061.00040.521260.976071.00071.00030.522570.896081.00081.00060.522880.942091.00091.00070.507690.9796 MNISTFMNISTCIFAR-10ImageNetAlexNetLeNetM_CNNAlexNetF_CNNVGG19VGG19 　　平均ASR (%)95.4699.6997.8898.7797.5987.6381.79DSR (%)resize178.2474.3281.8279.8477.2469.3847.83resize278.5464.9478.6479.3469.6564.2643.26rotate76.6680.5484.7477.6361.4672.4942.49Distil-D83.5182.0880.4985.2482.5575.1757.13Ens-D87.1988.0385.2487.7183.2177.4658.34D-GAN72.4068.2670.3179.5475.0473.0551.04GN22.6030.2627.5627.9622.6023.3513.85DAE84.5485.2585.6886.9480.2175.8559.31APE-GAN83.4080.7182.3684.1079.4572.1557.88 UIPD 88.92 86.89 87.45 87.77 83.91 78.23 59.91 Rconfresize10.92310.96310.94240.89330.93840.67420.4442resize20.89310.91840.9642 0.9731 0.94730.73710.4341rotate0.90420.89140.92740.95350.81440.68140.4152Distil-D0.92210.90530.91620.93400.92780.67410.4528Ens-D0.96230.91730.96860.92100.93310.79940.5029D-GAN0.87390.84190.88290.90120.89810.78390.4290GN0.14450.17420.24520.16310.18350.12550.0759DAE0.94700.93460.96330.94200.93240.77820.5090APE-GAN0.89640.92700.94250.88970.90150.63010.4749 UIPD 0.9788 0.9463 0.9842 0.9642 0.9531 0.8141 0.5141 MNISTFMNISTCIFAR-10ImageNetAlexNetLeNetM_CNNAlexNetF_CNNVGG19VGG19 　　平均ASR (%)93.2896.3294.6595.2093.5888.1083.39DSR (%)resize178.6570.6279.0974.3766.5465.3138.28resize263.1467.9477.1466.9863.0962.6341.60rotate76.6272.1971.8466.7564.4265.6042.67Distil-D82.3782.2280.4982.4783.2871.1445.39Ens-D86.9783.0385.2483.4182.5074.2947.85D-GAN82.4380.3486.1379.3580.4770.0843.10GN20.1621.8025.3019.6718.6321.4013.56DAE83.6684.1786.8882.4083.6674.3051.61APE-GAN82.4685.0185.1481.8082.5073.8049.28 UIPD 87.92 85.22 87.54 83.70 83.91 75.38 52.91 Rconfresize10.85130.86140.84600.79630.8324 0.6010 0.3742resize20.78140.88100.86550.82900.84750.63200.3800rotate0.85190.83740.83190.81000.80400.64620.4058Distil-D0.91410.89130.90330.91350.92000.78210.4528Ens-D0.95150.92800.87200.89400.90110.81550.4788D-GAN0.85390.87890.88290.87330.88200.74500.4390GN0.16300.19200.21520.17610.19710.14500.0619DAE0.91200.92900.95100.94200.93240.77820.5090APE-GAN0.89640.92700.94250.88970.90150.63010.4749 UIPD 0.9210 0.9340 0.9520 0.9512 0.9781 0.8051 0.5290 MNISTFMNISTCIFAR-10ImageNetAlexNetLeNetM_CNNAlexNetF_CNNVGG19VGG19 良性样本　　92.34　　95.71　　90.45　　89.01　　87.42　　79.55　　89.00resize192.27 (−0.07)95.66 (−0.05)90.47 (+0.02)88.97 (−0.04)87.38 (−0.04)79.49 (−0.06)88.98 (−0.02)resize292.26 (−0.80)95.68 (−0.30)90.29 (−0.16)88.71 (−0.30)87.38 (−0.04)79.48 (−0.07)87.61 (−1.39)rotate92.31 (−0.03)95.68 (−0.03)90.39 (−0.06)88.95 (−0.06)87.40 (0.02)79.53 (−0.02)88.82 (−0.18)Distil-D90.00 (−2.34)95.70 (−0.01)90.02 (−0.43)88.89 (−0.12)86.72 (−0.70)76.97 (−2.58)87.85 (−1.15)Ens-D 94.35 (+2.01) 96.15 (+0.44) 92.38 (+1.93) 89.13 (+0.12)87.45 (+0.03) 80.13 (+0.58) 89.05 (+0.05)D-GAN92.08 (−0.26)95.18 (−0.53)90.04 (−0.41)88.60 (−0.41)87.13 (−0.29)78.80 (−0.75)87.83 (−1.17)GN22.54 (−69.80)25.31 (−70.40)33.58 (−56.87)35.71 (−53.30)28.92 (−58.59)23.65 (−55.90)17.13 (−71.87)DAE91.57 (−0.77)95.28 (−0.43)89.91 (−0.54)88.13 (−0.88)86.80 (−0.62)79.46 (−0.09)87.10 (−1.90)APE-GAN92.30 (−0.04)95.68 (−0.03)90.42 (−0.03)89.00 (−0.01)87.28 (−0.14)79.49 (−0.06)88.88 (−0.12) UIPD 92.37 (+0.03)95.96 (+0.25)90.51 (+0.06) 89.15 (+0.14) 87.48 (+0.06) 79.61 (+0.06) 89.15 (+0.15) 第1组 第2组 第3组 第4组良性样本类标置信度 (良性样本 + UIP)类标置信度 对抗样本类标置信度 (对抗样本 + UIP) 类标置信度 01.00001.00060.453100.941511.00011.00030.471410.894521.00021.00060.564120.913131.00031.00010.510330.942541.00041.00020.483140.877351.00051.00070.542250.902661.00061.00050.486460.878771.00071.00050.514470.830981.00081.00040.478188.942491.00091.00070.496190.8872 第1组 第2组 第3组 第4组良性样本类标置信度 (良性样本 + UIP)类标置信度 对抗样本类标置信度 (对抗样本 + UIP) 类标置信度 飞机1.000飞机1.000船0.4914飞机0.9331汽车1.000汽车1.000卡车0.5212汽车0.9131鸟1.000鸟1.000马0.5031鸟0.8913猫1.000猫1.000狗0.5041猫0.9043鹿1.000鹿1.000鸟0.5010鹿0.8831狗1.000狗1.000马0.5347狗0.9141青蛙1.000青蛙1.000猫0.5314青蛙0.8863马1.000马1.000狗0.4814马0.8947船1.000船1.000飞机0.5142船0.9251卡车1.000卡车1.000飞机0.4761卡车0.9529 第1组 第2组 第3组 第4组良性样本类标置信度 (良性样本 + UIP)类标置信度 对抗样本类标置信度 (对抗样本 + UIP) 类标置信度 导弹0.9425导弹0.9445军装0.5134导弹0.8942步枪0.9475步枪0.9525航空母舰0.4981步枪0.7342军装0.9825军装0.9925防弹背心0.5014军装0.8245皮套0.9652皮套0.9692军装0.4831皮套0.8074航空母舰0.9926航空母舰0.9926灯塔0.4788航空母舰0.8142航天飞机0.9652航天飞机0.9652导弹0.5101航天飞机0.7912防弹背心0.9256防弹背心0.9159步枪0.4698防弹背心0.8141灯塔0.9413灯塔0.9782客机0.5194灯塔0.7861客机0.9515客机0.9634坦克0.4983客机0.7134坦克0.9823坦克0.9782灯塔0.5310坦克0.7613 袁文浩, 孙文珠, 夏斌, 欧世峰. 利用深度卷积神经网络提高未知噪声下的语音增强性能. 自动化学报, 2018, 44 (4): 751-759 doi: 10.16383/j.aas.2018.c170001

Yuan Wen-Hao, Sun Wen-Zhu, Xia Bin, Ou Shi-Feng. Improving speech enhancement in unseen noise using deep convolutional neural network. Acta Automatica Sinica , 2018, 44 (4): 751-759 doi: 10.16383/j.aas.2018.c170001

地址：北京中关村东路95号 邮政编码：100190 E-mail： [email protected]

电话：010-82544677 (日常咨询和稿件处理)， 010-82544653(费用管理、寄刊)

北京仁和汇智信息技术有限公司 开发 技术支持： [email protected]