沈丽丽,王 丹,徐 珂
AuthorsHTML:沈丽丽，王 丹，徐 珂
AuthorsListE:Shen Lili,Wang Dan,Xu Ke
AuthorsHTMLE:Shen Lili，Wang Dan，Xu Ke
Unit:天津大学电气自动化与信息工程学院，天津 300072
Unit_EngLish:School of Electrical and Information Engineering，Tianjin University，Tianjin 300072，China
Abstract_Chinese:近年来，深度学习在无参考图像质量评价领域的应用获得了广泛的关注．然而，复杂的双目视觉机制和立 体图像的多维特性使得立体图像的质量评估更具有挑战性．基于人类视觉皮层的交互作用，提出一个多级表示网络 用于无参考立体图像质量评价．为了克服缺乏具有主观质量分数的大规模训练数据问题，将图像的主观平均意见分 数分配给图像中的所有小图像块．同时，考虑到大量来自同质区域的图像块会混淆网络训练和质量分数预测过程， 采用基于方差的阈值指标来过滤部分同质图像块．多级表示网络由初级子网和高级子网两部分组成，在初级子网 中，将视差图加入网络构成三通道卷积神经网络，分别提取并交互立体视图和视差图的初级特征；在初级子网的输 出端，将特征分别级联得到基于视差的交互式双目特征和无视差交互的单眼特征；高级子网采用二通道卷积神经网 络，进一步提取高级融合特征信息，更好地模拟人类视觉系统的信息处理机制．最终，所提出的多级表示网络不仅 提取了立体图像的单眼特征，还提取了基于视差/深度信息的双目特征．在公开的 LIVE 3D Phase Ⅰ和 LIVE 3D Phase Ⅱ数据库上进行测试，实验结果表明该网络在对称和非对称立体图像数据库上较其他方法均取得了较好的结 果，能够与人类的主观感知保持良好的一致性．
Abstract_English:No-reference stereoscopic image quality assessment by deep learning has attracted lot of attention. However，complex binocular vision mechanisms and multi-dimensional characteristics of stereoscopic images make the assessment more challenging. A multi-level representation network is proposed to solve these problems based on the interactions in the human visual cortex. All patches of an image were assigned the subjective mean opinion score to overcome the problem of inadequate large-scale training data with subjective quality scores. Meanwhile，considering that patches from homogeneous regions might confuse the process of network training and quality score prediction，a threshold index based on variance was used to filter some homogeneous patches. The multi-level representation net\u0002work consisted of a primary and an advanced subnetwork. A disparity map was added to the network to form a three\u0002channel convolutional neural network in the primary subnetwork where the primary features of the stereoscopic view and disparity map were extracted and interacted. At the output of the primary subnetwork，the features were concate\u0002nated to obtain a disparity-based interactive binocular feature map and a no-disparity-based monocular feature map； thus，the advanced subnetwork used a two-channel convolutional neural network，which could further extract advanced fusion features and better simulate the information processing mechanism of the human visual system. The performance of the network was evaluated over LIVE 3D Phase Ⅰ and Phase Ⅱ databases. The experimental re-sults demonstrate that the multi-level representation network is superior to other state-of-the-art stereoscopic images quality assessment algorithms and can keep a high degree of consistency with human subjective perception.
Keyword_Chinese:立体图像质量评价；多级表示网络；视差；人类视觉系统
Keywords_English:stereoscopic image quality assessment；multi-level representation network；disparity；human visual system

PDF全文下载地址:http://xbzrb.tju.edu.cn/#/digest?ArticleID=6612