贺兰钦,,
谭颀,
陈前斌
1.重庆邮电大学通信与信息工程学院 重庆 400065
2.重庆邮电大学移动通信技术重点实验室 重庆 400065
基金项目:国家自然科学基金(62071078)，重庆市教委科学技术研究项目(KJZD-M20180601)，重庆市重大主题专项项目 (cstc2019jscx-zdztzxX0006)
详细信息
作者简介:唐伦：男，1973年生，教授，博士，主要研究方向为下一代无线通信网络、异构蜂窝网络、软件定义无线网络等
贺兰钦：男，1995年生，硕士生，研究方向为5G网络切片，机器学习算法
谭颀：女，1995年生，硕士生，研究方向为5G网络切片、资源分配、随机优化理论
陈前斌：男，1967年生，教授，博士生导师，主要研究方向为个人通信、多媒体信息处理与传输、异构蜂窝网络等
通讯作者:贺兰钦 719097886@qq.com
中图分类号:TN929.5计量
文章访问数:591
HTML全文浏览量:213
PDF下载量:71
被引次数:0
出版历程
收稿日期:2019-11-15
修回日期:2020-11-02
网络出版日期:2020-12-09
刊出日期:2021-02-23
Virtual Network Function Migration Optimization Algorithm Based on Deep Deterministic Policy Gradient
Lun TANG,Lanqin HE,,
Qi TAN,
Qianbin CHEN
1. School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
2. Key Laboratory of Mobile Communications Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
Funds:The National Natural Science Foundation of China (62071078), The Science and Technology Research Program of Chongqing Municipal Education Commission (KJZD-M20180601), The Major Theme Special Projects of Chongqing (cstc2019jscx-zdztzxX0006)
摘要
摘要:针对NFV/SDN架构下，服务功能链(SFC)的资源需求动态变化引起的虚拟网络功能(VNF)迁移优化问题，该文提出一种基于深度强化学习的VNF迁移优化算法。首先，在底层CPU、带宽资源和SFC端到端时延约束下，建立基于马尔可夫决策过程(MDP)的随机优化模型，该模型通过迁移VNF来联合优化网络能耗和SFC端到端时延。其次，由于状态空间和动作空间是连续值集合，提出一种基于深度确定性策略梯度(DDPG)的VNF智能迁移算法，从而得到近似最优的VNF迁移策略。仿真结果表明，该算法可以实现网络能耗和SFC端到端时延的折中，并提高物理网络的资源利用率。
关键词:虚拟网络功能/
深度强化学习/
SFC端到端时延/
网络能耗
Abstract:To solve the problem of Virtual Network Function (VNF) migration optimization, which is caused by the dynamic change of resource requirements of Service Function Chain (SFC) under Network Function Virtualization/ Software Defined Network (NFV/SDN) architecture, a VNF migration optimization algorithm is proposed based on deep reinforcement learning. Firstly, based on the underlying CPU, bandwidth resources and SFC end-to-end delay constraints, a Markov Decision Process (MDP) based stochastic optimization model is established. This model is used to optimize jointly network energy consumption and SFC end-to-end delay by migrating VNF. Secondly, since the state space and action space of this paper are continuous value sets, a VNF intelligent migration algorithm based on Deep Deterministic Policy Gradient (DDPG) is proposed to obtain an approximate optimal VNF migration strategy. The simulation results show that the algorithm can achieve the compromise between network energy consumption and SFC end-to-end delay, and improve the resource utilization of the physical network.
Key words:Virtual Network Function (VNF)/
Deep reinforcement learning/
SFC end-to-end delay/
Network energy consumption
PDF全文下载地址:
https://jeit.ac.cn/article/exportPdf?id=b20bd8fd-af90-4198-9cf7-99ac94e33cd9
