高逸飞1,
刘琲贝1,
廖广军3
1.华南理工大学电子与信息学院 广州 510641
2.中新国际联合研究院 广州 511356
3.广东警官学院 广州 510230
基金项目:国家重点研发计划项目(2019QY2202)，广州市开发区国际合作项目(201902010028)，中新国际联合研究院项目(206-A017023, 206-A018001)，广东省自然科学基金博士科研启动项目(2017A030310320)，中央高校基本科研业务费专项资金(2019MS025)，广东省教育厅特色创新类项目(2017KTSCX132)
详细信息
作者简介:胡永健：男，1962年生，教授，博士生导师，研究方向为多媒体信息安全、图像处理、人工智能及其应用
高逸飞：男，1996年生，硕士生，研究方向为多媒体信息安全、图像处理和机器学习
刘琲贝：女，1980年生，讲师，研究方向为多媒体信息安全、图像处理和机器学习
廖广军：男，1981年生，副教授，研究方向为多媒体信息安全、图像处理和机器学习
通讯作者:胡永健 eeyjhu@scut.edu.cn
1) DeepFake Detection Challenge: < 中图分类号:TN911.73
计量
文章访问数:1697
HTML全文浏览量:661
PDF下载量:214
被引次数:0
出版历程
收稿日期:2020-01-17
修回日期:2020-07-10
网络出版日期:2020-07-22
刊出日期:2021-01-15
Deepfake Videos Detection Based on Image Segmentation with Deep Neural Networks
Yongjian HU1, 2,,,Yifei GAO1,
Beibei LIU1,
Guangjun LIAO3
1. School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510641, China
2. Sino-Singapore International Joint Research Institute, Guangzhou 511356, China
3. Guangdong Police College, Guangzhou 510230, China
Funds:The National Key R & D Program (2019QY2202), The International Cooperation Project of Guangzhou Development Zone (201902010028), The Sino Singapore International Joint Research Institute Project (206-A017023, 206-A018001), The Doctoral Research Project of Natural Science Foundation of Guangdong Province (2017A030310320), The Special Fund for Basic Scientific Research of Central University (2019MS025), The Department of Education of Guangdong Province Characteristic Innovation Project (2017KTSCX132)
摘要
摘要:随着深度学习技术的快速发展，利用深度神经网络模型伪造出的深度假脸(deepfake)视频越来越逼真，假脸视频造成的威胁也越来越大。文献中已出现一些基于卷积神经网络的换脸视频检测算法，他们在库内获得较好的检测效果，但跨库检测性能急剧下降，存在泛化能力不足的问题。该文从假脸篡改的机制出发，将视频换脸视为特殊的拼接篡改问题，利用流行的神经分割网络首先预测篡改区域，得到预测掩膜概率图，去噪并二值化，然后根据换脸主要发生在人脸区域的前提，提出一种计算人脸交并比的新方法，并进一步根据换脸处理的先验知识改进人脸交并比的计算，将其作为篡改检测的分类准则。所提出方法分别在3个不同的基础分割网络上实现，并在TIMIT, FaceForensics++, FFW数据库上进行了实验，与文献中流行的同类方法相比，在保持库内检测的高准确率同时，跨库检测的平均错误率显著下降。在近期发布的合成质量较高的DFD数据库上也获得了很好的检测性能，充分证明了所提出方法的有效性和通用性。
关键词:假脸视频/
图像分割网络/
人脸交并比/
信任机制/
泛化能力
Abstract:With the rapid development of deep learning technology, videos with changed faces generated by deep neural networks (i.e., Deepfake videos) become more and more indistinguishable. As a result, the threat raised by Deepfake videos becomes greater and greater. In literature, there are some convolutional neural networks-based detection algorithms for fake face videos. Although those algorithms perform well when the training set and the testing set are from the same dataset, their performance could deteriorate dramatically in cross-dataset scenario where the training and the testing sets are from different sources. Motivated by the fabrication course of fake face videos, this article attempts to solve the problem of fake faces detection with the way of image splicing detection. A neural network borrowed from image segmentation is adopted for predicting the tampered face area from which a tampering mask is obtained through denoising and thresholding the probability map. Using the prior knowledge of face tampering that the changing of face mainly happens in face region, a new way is proposed to determine the Face-Intersection over Union (Face-IoU) and to further improve the ratio calculation method. The Face-Intersection over Union with Penalty (Face-IoUP) is used as the classification criterion for deepfake video detection. The proposed method is impletmented using three basic image segmentation neural networks separately and is tested them on datasets of TIMIT, FaceForensics++, Fake Face in the Wild(FFW). Compared with current methods in literature, the HTER (Half Total Error Rate) in cross-dataset test decreases significantly while the detection accuracy in intra-dataset test keeps high. For the Deep Fake Detection(DFD) dataset with higher synthesis quality, the proposed method still performs very well. Experimental results validate the proposed method and demonstrate its good generality.
Key words:Deepfake videos/
Image segmentation networks/
Face-Intersection over Union(Face-IoU)/
Confidence mechanism/
Generalization
注释:
1) 1) DeepFake Detection Challenge: <
PDF全文下载地址:
https://jeit.ac.cn/article/exportPdf?id=7764062e-82cf-47e1-8c64-f00216fcf3af
