李宏伟1, 3,,
1.中国地质大学数理学院 ??武汉 ??430074
2.湖北经济学院信息管理与统计学院 ??武汉 ??430205
3.中国地质大学地球内部多尺度成像湖北省重点实验室 ??武汉 ??430074
基金项目:湖北省教育厅科学技术研究计划重点项目(D20182203)
详细信息
作者简介:沈卉卉：女，1980 年生，博士生，副教授，研究方向为机器学习与数据处理
李宏伟：男，1965 年生，教授，博士生导师，主要研究方向为信息处理与智能计算
通讯作者:李宏伟 hwli@cug.edu.cn
中图分类号:TP182; TP183计量
文章访问数:1153
HTML全文浏览量:287
PDF下载量:21
被引次数:0
出版历程
收稿日期:2017-09-18
修回日期:2018-05-24
网络出版日期:2018-07-12
刊出日期:2018-09-01
An Improved Algorithm of Product of Experts System Based on Restricted Boltzmann Machine
Huihui SHEN1, 2, 3,Hongwei LI1, 3,,
1. Institute of Geophysics & Geomatics, China University of Geosciences, Wuhan 430074, China
2. School of Statistics & Information Management, Hubei University of Economics, Wuhan 430205, China
3. Hubei Subsurface Multi-scale Imaging Key Laboratory, China University of Geosciences, Wuhan 430074, China
Funds:The Science and Technology Research Program Key Project of Hubei Provincial Education Department (D20182203)
摘要
摘要:深度学习在高维特征向量的信息提取和分类中具有很强的能力，但深度学习训练时间也比较长，超参数搜索空间大，从而导致超参数寻优较困难。针对此问题，该文提出一种基于受限玻尔兹曼机(RBM)专家乘积系统的改进方法。先将专家乘积系统原理与RBM算法相结合，采用全是真实概率值的参数更新方式会引起模型识别效果不理想和带来密度问题，为此将其更新方式进行改进；为加快网络收敛和提高模型识别能力，采取在RBM预训练阶段和微调阶段引入不同组合方式动量项的一种改进算法。通过对MNIST数据库中的0~9的手写数字体的识别和CMU-PIE数据库的人脸识别实验，提出的算法减少了学习时间，提高了超参数寻优的效率，进而构建的深层网络能获得较好的分类效果。试验结果表明，提出的改进算法在处理高维大量的数据时，计算效率有较大提高，其算法有效。
关键词:深度学习/
专家乘积/
神经网络/
受限玻尔兹曼机/
动量
Abstract:Deep learning has a strong ability in the high-dimensional feature vector information extraction and classification. But the training time of deep learning is so long that the optimal hyper-parameters combination can not be found in a short time. To solve these problems, a method of product of experts system based on Restricted Boltzmann Machine (RBM) is proposed. The product of experts theory is combined with the RBM algorithm and the parameter updating way is all adopted the probability value, which leads to the undesirable recognition effect and slightly worse density models, so the parameter updating way is improved. An improved algorithm with momentum terms in different combinations is used not only in the RBM pre-training phase but also in the fine-tuning stage for both classification accuracy enhancement and training time decreasing. Through the recognition experiments on the MNIST database and CMU-PIE face database, the proposed algorithm reduces the training time, and improves the efficiency of hyper-parameters optimization, and then the deep belief network can achieve better classification performance. The result shows that the improved algorithm can improve both accuracy and computation efficiency in dealing with high-dimensional and large amounts of data, the new method is effective.
Key words:Deep learning/
Product Of Experts (POE)/
Neural network/
Restricted Boltzmann Machine (RBM)/
Momentum
PDF全文下载地址:
https://jeit.ac.cn/article/exportPdf?id=c9892cf1-4648-4aec-8164-186f71d0da63
