关键词 : 深度神经网络 知识产权保护 所有权验证 鲁棒水印 投稿时间：2023-03-07 修订日期：2023-06-21 基金项目 : 国家自然科学基金(No. 61972207, No. U1836208, No. U1836110, No. 61672290)，国家社会科学基金重大项目(No. 17ZDA092)，中国国家重点研发计划(No. 2018YFB1003205)，大气环境与装备技术协同创新中心基金以及江苏省高等学校重点学科建设基金 The Transformer model has emerged as the most powerful deep learning model in natural language processing (NLP), achieving remarkable performance in various NLP tasks such as machine translation and natural language generation. However, the high performance of these models also makes them increasingly susceptible to Intellectual Property Rights (IPR) infringement, especially for large models with high training costs. While ownership verifica-tion methods exist for other models such as Convolutional Neural Networks (CNN) and Generative Adversarial Networks (GAN), there is still a lack of adequate protection for Transformer models. Therefore, this paper has pro-posed a novel and robust watermarking scheme that can effectively verify the ownership of Transformer models in both black-box and white-box settings. The proposed scheme employs Extra Attention as a white-box watermark carrier, which enhances the watermark's robustness, making it less susceptible to various attacks, including ambigu-ity attacks. Ambiguity attacks are particularly challenging for normal watermarking schemes to withstand, as they involve adding the attacker's watermark to the original watermark, causing ownership obfuscation. Meanwhile, it greatly improves the concealment of the watermark by selecting embedding methods in multiple locations. Fur-thermore, this paper proposes a backdoor addition scheme based on Hybrid Triggers, which achieves verification of model ownership without accessing the model source code in a black-box manner by combining loss functions of multiple triggers, with good stealth and resistance to removal. This scheme has good crypticity and removal re-sistance, making it difficult for attackers to remove the watermark. This paper also investigates a new form of am-biguity attack and demonstrates that the proposed watermarking scheme outperforms existing deep neural network watermarking schemes in the face of such attacks. The experimental results show that the proposed watermarking scheme is highly effective in protecting Transformer models from IPR infringement. Overall, the watermarking method proposed in this paper addresses the limitations of previous works, provides more robust watermarking for the Transformer, and enhances the intellectual property protection of the model. Key words : deep neural network intellectual property protection ownership verification robust watermark 京公网安备11010802043679号
主办：中国科学院信息工程研究所、中国科技出版传媒股份有限公司