基于生成式对抗网络的图像超分辨率重建算法

link管理

链接快照平台

输入网页链接，自动生成快照
标签化管理网页链接

相关文章推荐

1548603420 · 如何找到附近的Remitly网点并进行快速汇 ...· 1 周前 ·

瘦瘦的山羊 · 最全！关于北京摇号新政，大家关心的这35个问 ...· 5 月前 ·

完美的生姜 · 最高检发布“3·15”消费者权益保护检察公益 ...· 7 月前 ·

风流倜傥的单杠 · 基于Python opencv ...· 7 月前 ·

犯傻的饺子 · 处方_全文在线阅读_plantinvain小 ...· 1 年前 ·

摘要： 针对传统图像超分辨率重建算法存在网络训练困难与生成图像存在伪影的问题，提出一种利用生成式对抗网络的超分辨率重建算法。去除生成式对抗网络的批量归一化层降低计算复杂度，将其中的残差块替换为密集残差块构成生成网络，使用VGG19网络作为判别网络的基础框架，以全局平均池化代替全连接层防止过拟合，引入纹理损失函数、感知损失函数、对抗损失函数和内容损失函数构成生成器的总目标函数，利用纹理损失增强局部信息匹配度，采用激活层前的特征信息计算感知损失获取更多细节特征，使用WGAN-GP理论优化网络模型的对抗损失加速收敛，运用内容损失提升图像低频信息的准确性。实验结果表明，该算法重建图像的平均峰值信噪比为27.97 dB，平均结构相似性为0.777，与SRGAN和EDSR等算法相比，其在未延长较多运行时间的情况下，重建结果的纹理细节更清晰且亮度信息更准确，更符合视觉感官评价要求。

超分辨率重建, 生成式对抗网络, 密集卷积网络, 纹理损失, 梯度惩罚Wasserstein生成式对抗网络

Abstract: The existing image Super-Resolution (SR) reconstruction algorithms have difficulty in network training and cause artifacts in the generated images.To address the problem,this paper proposes a SR reconstruction algorithm based on Generative Adversarial Networks (GAN).The Batch Normalization(BN)layer of the Generative Adversarial Networks(SRGAN)is removed to reduce the computational complexity,and the Residual Block (RB) is replaced by Residual Dense Block (RDB) to form the generative network.VGG19 network is used as the basic framework of the discrimination network.The Global Average Pooling (GAP)is used to replace the full connection layer to prevent over fitting.Texture loss function, perceptual loss function, adversarial loss function and content loss function are introduced to form the overall objective function of the generator. Texture loss is used to enhance the matching degree of local information, and the feature information in front of the activation layer is used to calculate the perceptual loss to obtain more detailed features. Wasserstein Generative Adversarial Nets-Gradient Penalty(WGAN-GP) theory is used to optimize the adversarial loss of the network model to accelerate the convergence, and content loss is used to improve the accuracy of low-frequency information of the image.Experimental results show that the average Peak Signal to Noise Ratio (PSNR) of the image reconstructed by the proposed algorithm is 27.97 dB,and its average Structural Similarity(SSIM)is 0.777.Compared with SRGAN and EDSR algorithms,the proposed algorithm improves the texture details and brightness of the reconstructed image,making it more in line with the requirements of visual sensory evaluation without prolonging much running time.

Key words: Super-Resolution(SR) reconstruction, Generative Adversarial Networks(GAN), Dense Convolutional Networks(DenseNet), texture loss, Wasserstein Generative Adversarial Nets-Gradient Penalty (WGAN-GP) [1] KEYS R.Cubic convolution interpolation for digital image processing[J].IEEE Transactions on Acoustics,Speech,and Signal Processing,1981,29(6):1153-1160.
[2] ZHANG Lei,WU Xiaolin.An edge-guided image interpolation algorithm via directional filtering and data fusion[J].IEEE Transactions on Image Processing,2006,15(8):2226-2238.
[3] FATTAL R.Image upsampling via imposed edge statistics[J].ACM Transactions on Graphics,2007,26(3):95-103.
[4] JUNG S W,KIM T H,KO S J.A novel multiple image deblurring technique using fuzzy projection onto convex sets[J].IEEE Signal Processing Letters,2009,16(3):192-195.
[5] NAYAK R,HARSHAVARDHAN S,PATRA D.Morphology based iterative back-projection for super-resolution reconstruction of image[C]//Proceedings of the 2nd International Conference on Emerging Technology Trends in Electronics,Communication and Networking.Washington D.C.,USA:IEEE Press,2014:1-6.
[6] SUN Dong,GAO Qingwei,LU Yixiang,et al.A novel image denoising algorithm using linear Bayesian MAP estimation based on sparse representation[J].Signal Processing,2014,100(7):132-145.
[7] DONG C,LOY C C,HE K M,et al.Image super-resolution using deep convolutional networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[8] KIM J,LEE J K,LEE K M.Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2016:1646-1654.
[9] LEDIG C,THEIS L,HUSZAR F,et al.Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:4681-4690.
[10] LAI W S,HUANG J B,AHUJA N,et al.Deep laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:624-632.
[11] TAI Ying,YANG Jian,LIU Xiaoming.Image super-resolution via deep recursive residual network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:3147-3155.
[12] JOHNSON J,ALAHI A,LI F F.Perceptual losses for real-time style transfer and super-resolution[C]//Proceedings of ECCV'16.Berlin,Germany:Springer,2016:694-711.
[13] SAJJADI M S M,SCHOLKOPF B,HIRSCH M.EnhanceNet:single image super-resolution through automated texture synthesis[C]//Proceedings of 2017 IEEE International Conference on Computer Vision.Washington D.C.,USA:IEEE Press,2017:4491-4500.
[14] GOODFELLOW I,ABADIE J P,MIRZA M,et al.Generative adversarial nets[EB/OL].[2019-11-25].https://www.researchgate.net/publication/319770355_Generative_Adversarial_Nets.
[15] LIM B,SON S,KIM H,et al.Enhanced deep residual networks for single image super-resolution[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C.,USA:IEEE Press,2017:136-144.
[16] WANG Xintao,YU Ke,WU Shixiang,et al.ESRGAN:enhanced super-resolution generative adversarial net-works[C]//Proceedings of 2019 European Conference on Computer Vision.Berlin,Germany:Springer,2019:63-79.
[17] RATLIFF L J,BURDEN S A,SASTRY S S.Characterization and computation of local Nash equilibria in continuous games[C]//Proceedings of 2013 Annual Allerton Conference on Communication.Washington D.C.,USA:IEEE Press,2013:917-924.
[18] HE Kaiming,ZHANG Xiangyu,REN Shaoqing,et al.Identity mappings in deep residual networks[C]//Proceedings of ECCV'16.Berlin,Germany:Springer,2016:630-645.
[19] SRIVASTAVA R K,GREFF K,SCHMIDHUBER J.Training very deep networks[EB/OL].[2019-11-25].https://www.researchgate.net/publication/280329704_Training_Very_Deep_Networks.
[20] HUANG Gao,SUN Yu,LIU Zhuang,et al.Deep networks with stochastic depth[C]//Proceedings of ECCV'16.Berlin,Germany:Springer,2016:646-661.
[21] HUANG G,LIU Z,LAURENS V D M,et al.Densely connected convolutional networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:4700-4708.
[22] JEGOU S,DROZDZAL M,VAZQUEZ D,et al.The one hundred layers Tiramisu:fully convolutional DenseNets for semantic segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops.Washington D.C.,USA:IEEE Press,2017:11-19.
[23] PARK S,JEONG Y,KIM H S.Multi-resolution DenseNet based acoustic models for reverberant speech recognition[J].Phonetics and Speech Sciences,2018:10(1):33-38.
[24] ARJOVSKY M,CHINTALA S,BOTTOU L.Wasserstein GAN[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,38(2):295-307.
[25] GULRAJANI I,AHMED F,ARJOVSKY M,et al.Improved training of Wasserstein GANs[EB/OL].[2019-11-25].https://www.researchgate.net/publication/315765101_Imp roved_Training_of_Wasserstein_GANs.
[26] NAH S,KIM T H,LEE K M.Deep multi-scale convolu-tional neural network for dynamic scene deblurring[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2017:3883-3891.
[27] ZHANG Yulun,TIAN Yapeng,KONG Yu,et al.Residual dense network for image super-resolution[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2018:2472-2481.
[28] ZHANG Yulun,LI Kunpeng,LI Kai,et al.Image super-resolution using very deep residual channel attention networks[C]//Proceedings of ECCV'18.Berlin,Germany:Springer,2018:294-310.
[29] SZEGEDY C,LIU W,JIA Y Q,et al.Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Washington D.C.,USA:IEEE Press,2015:1-9. 范文卓, 吴涛, 许俊平, 李庆庆, 张建林, 李美惠, 魏宇星. 基于多分辨率特征融合的任意尺度图像超分辨率重建 [J]. 计算机工程, 2023, 49(9): 217-225. 丁子轩, 俞雷, 张娟, 李想, 王新宇. 基于深度残差自适应注意力网络的图像超分辨率重建 [J]. 计算机工程, 2023, 49(5): 231-238. 李培育, 张雅丽. 基于改进SRGAN模型的人脸图像超分辨率重建 [J]. 计算机工程, 2023, 49(4): 199-205. 黎浩民, 李光平. 基于稀疏神经网络的图像超分辨率重建算法 [J]. 计算机工程, 2022, 48(7): 247-253. 柳聪, 屈丹, 司念文, 魏紫薇. 基于深度可分离卷积的轻量级图像超分辨率重建 [J]. 计算机工程, 2022, 48(6): 228-234. 楼鑫杰, 李小薪, 刘志勇. 基于反馈机制的图像超分辨率重建算法 [J]. 计算机工程, 2022, 48(2): 261-267. 张法正, 杨娟, 汪荣贵, 薛丽霞. 基于动态自适应层叠网络的轻量化图像超分辨率重建 [J]. 计算机工程, 2022, 48(12): 196-202. 黄山, 贾俊. 基于改进循环生成式对抗网络的图像去雾方法 [J]. 计算机工程, 2022, 48(12): 218-223,231. 陈乔松, 蒲柳, 张羽, 孙开伟, 邓欣, 王进. 结合整体注意力与分形稠密特征的图像超分辨率重建 [J]. 计算机工程, 2022, 48(11): 207-214,223. 姜红涛, 孙京, 谢成, 赖少川, 沈焕锋. 内外部梯度联合约束的图像超分辨率重建方法 [J]. 计算机工程, 2022, 48(1): 220-227,235.

地址：上海市嘉定区澄浏公路63号(201808) Tel：021-67092217(费用/发票)（稿件问题请直接联系栏目编辑） E-mail：[email protected]

沪公网安备31011402007040号

本系统由北京玛格泰克科技发展有限公司设计开发