王成龙1,
蒋富1,,,
顾欣2,
牟玥玥1,
刘伟荣1
1.中南大学计算机学院 长沙 410083
2.中南大学自动化学院 长沙 410083
基金项目:国家自然科学基金(61873353, 61672539)
详细信息
作者简介:彭军：女，1967年生，教授，研究方向为智能交通网络通信与安全控制、移动云服务、机器学习、智慧新能源管理
王成龙：男，1996年生，博士生，研究方向为车联网、智能交通网络通信、机器学习
蒋富：男，1983年生，副教授，研究方向为无线通信与移动网络、绿色认知无线传感器网络、协同通信、网络空间安全
顾欣：女，1993年生，博士生，研究方向为无线网络优化、车联网通信技术
牟玥玥：女，1996年生，硕士生，研究方向为图像处理、无线通信
刘伟荣：男，1976年生，教授，研究方向为物联网、协同通信、CPS系统、云机器人情景感知
通讯作者:蒋富 jiangfu0912@csu.edu.cn
中图分类号:TN929.52计量
文章访问数:1417
HTML全文浏览量:461
PDF下载量:69
被引次数:0
出版历程
收稿日期:2019-08-12
修回日期:2019-11-04
网络出版日期:2019-11-12
刊出日期:2020-01-21
A Fast Deep Q-learning Network Edge Cloud Migration Strategy for Vehicular Service
Jun PENG1,Chenglong WANG1,
Fu JIANG1,,,
Xin GU2,
Yueyue MU1,
Weirong LIU1
1. School of Computer Science and Engineering, Central South University, Changsha 410083, China
2. School of Automation, Central South University, Changsha 410083, China
Funds:The National Natural Science Foundation of China(61873353, 61672539)
摘要
摘要:智能网联交通系统中车载用户的高速移动，不可避免地造成了数据在边缘服务器之间频繁迁移，产生了额外的通信回传时延，对边缘服务器的实时计算服务带来了巨大的挑战。为此，该文提出一种基于车辆运动轨迹的快速深度Q学习网络(DQN-TP)边云迁移策略，实现数据迁移的离线评估和在线决策。车载决策神经网络实时获取接入的边缘服务器网络状态和通信回传时延，根据车辆的运动轨迹进行虚拟机或任务迁移的决策，同时将实时的决策信息和获取的边缘服务器网络状态信息发送到云端的经验回放池中；评估神经网络在云端读取经验回放池中的相关信息进行网络参数的优化训练，定时更新车载决策神经网络的权值，实现在线决策的优化。最后仿真验证了所提算法与虚拟机迁移算法和任务迁移算法相比能有效地降低时延。
关键词:智能网联交通系统/
虚拟机迁移/
强化学习/
深度Q学习网络
Abstract:The high-speed movement of vehicles inevitably leads to frequent data migration between edge servers and increases communication delay, which brings great challenges to the real-time computing service of edge servers. To solve this problem, a real-time reinforcement learning method based on Deep Q-learning Networks according to vehicle motion Trajectory Process (DQN-TP) is proposed. The proposed algorithm separates the decision-making process from the training process by using two neural networks. The decision neural network obtains the network state in real time according to the vehicle’s movement track and chooses the migration method in the virtual machine migration and task migration. At the same time, the decision neural network uploads the decision records to the memory replay pool in the cloud. The evaluation neural network in the cloud trains with the records in the memory replay pool and periodically updates the parameters to the on-board decision neural network. In this way, training and decision-making can be carried out simultaneously. At last, a large number of simulation experiments show that the proposed algorithm can effectively reduce the latency compared with the existing methods of task migration and virtual machine migration.
Key words:Intelligent transportation system/
Virtual machine migration/
Reinforcement learning/
Deep Q-learning Networks(DQN)
PDF全文下载地址:
https://jeit.ac.cn/article/exportPdf?id=f6accdeb-3716-4303-8862-cdca1b938b53
