尹梦晓^{1, 2,},
林振峰¹,
杨锋^{1, 2,,}
1.广西大学计算机与电子信息学院 南宁 530004
2.广西多媒体通信与网络技术重点实验室 南宁 530004
基金项目:国家自然科学基金(61762007, 61861004)，广西自然科学基金(2017GXNSFAA198269, 2017GXNSFAA198267)

详细信息

作者简介:尹梦晓：女，1978年生，博士，副教授，CCF会员，研究方向为计算机图形学与虚拟现实、数字几何处理、图像与视频编辑
林振峰：男，1996年生，硕士生，研究方向为图像生成、图像转换
杨锋：男，1979年生，博士，副教授，CCF会员，研究方向为人工智能、网络信息安全、大数据与高性能计算、精准医学

通讯作者:杨锋　yf@gxu.edu.cn

中图分类号:TN911.73; TP391

计量

文章访问数:435
HTML全文浏览量:218
PDF下载量:59
被引次数:0

出版历程

收稿日期:2020-08-04
修回日期:2021-01-04
网络出版日期:2021-01-10
刊出日期:2021-08-10

Adaptive Multi-scale Information Fusion Based on Dynamic Receptive Field for Image-to-image Translation

Mengxiao YIN^{1, 2,},
Zhenfeng LIN¹,
Feng YANG^{1, 2,,}
1. School of Computer and Electronics Information, Guangxi University, Nanning 530004, China
2. Guangxi Key Laboratory of Multimedia Communications and Network Technology, Guangxi University, Nanning 530004, China
Funds:The National Natural Science Foundation of China (61762007, 61861004), The Natural Science Foundation of Guangxi (2017GXNSFAA198269, 2017GXNSFAA198267)


摘要
摘要:为提高图像转换模型生成图像的质量，该文针对转换模型中的生成器进行改进，同时探究多样化的图像转换，拓展转换模型的生成能力。在生成器的改进方面，利用选择性(卷积)核模块(SKBlock)的动态感受野机制获取和融合生成器中每个上采样特征的多尺度信息，借助特征的多尺度信息和动态感受野构造选择性(卷积)核的生成式对抗网络(SK-GAN)。与传统生成器相比，SK-GAN以动态感受野获取多尺度信息的生成结构提高了生成图像的质量。在多样化图像转换方面，基于SK-GAN在草图合成真实图像任务提出带引导图像的选择性(卷积)核的生成式对抗网络(GSK-GAN)。该模型利用引导图像指导源图像的转换，通过引导图像编码器提取引导图像特征，然后由参数生成器(PG)和特征转换层(FT)将引导图像特征的信息传递至生成器。此外，该文还提出双分支引导图像编码器以提高转换模型的编辑能力，以及利用引导图像的隐变量分布实现随机样式的图像生成。实验表明，改进后的生成器有助于提高生成图像质量，SK-GAN在多个数据集中获得合理的生成结果。GSK-GAN不仅保证了生成图像的质量，还能生成更多样式的图像。
关键词:图像转换/
多尺度信息/
动态感受野/
自适应特征选择
Abstract:In order to improve the quality of the generated images by the image translation model, the generator in the translation model to obtain high-quality generated images is improved, the diversified image translation is explored and the generation ability of the translation model is expanded. In terms of generator improvement, the dynamic receptive field mechanism of Selective Kernel Block (SKBlock) is used to obtain and fuse the multi-scale information of each up sampling feature in the generator. With the help of multi-scale information of features and dynamic receptive field, the Selective Kernel Generative Adversarial Network (SK-GAN) is constructed. Compared with the traditional generator, SK-GAN improves the quality of the generated image by using dynamic receptive field to obtain multi-scale information. In terms of diversified image translation, the Selective Kernel Generative Adversarial Network with Guide (GSK-GAN) is proposed based on SK-GAN in sketch synthesis realistic image task. GSK-GAN uses the guided image to guide the source image translation and extracts the guide image features through the guided image encoder. Then transmits information of the guided image features to the generator by Parameter Generator (PG) and Feature Transformation (FT). In addition, a dual branch guided image encoder is proposed to improve the editing ability of the translation model. The random style image generation is realized by using the latent variable distribution of the guide image. The experimental results show that the improved generator is helpful to improve the quality of the generated images, and SK-GAN can obtain reasonable results in multiple datasets. GSK-GAN no only ensures the quality of the generated images, but also generates more styles of images
Key words:Image translation/
Multi-scale information/
Dynamic receptive field/
Adaptive feature selection



PDF全文下载地址:

https://jeit.ac.cn/article/exportPdf?id=e548863a-ee35-43cc-b18e-c6e2ff21fdf3