方禹,李学华,潘春雨,云翔

• 1:北京信息科技大学现代测控技术教育部重点实验室
• 2:北京信息科技大学-佰才邦技术智慧物联联合实验室

摘要(Abstract)：

为满足时延敏感型业务的需求，同时解决工业互联网设备能耗受限、边缘服务器资源有限等问题，将时延和能耗作为优化目标，采用改进的深度强化学习算法进行工业互联网场景的资源分配。进一步，通过两个不同参数的神经网络互相监督，解决传统深度学习算法单一神经网络的估计值偏大问题，获得更优结果。仿真结果表明，与全卸载计算、全本地计算、随机卸载计算和传统Q学习算法相比，所提策略在分别改变终端数量、服务器计算能力、任务数据量时，均能得到更优性能。

关键词(KeyWords)： 移动边缘计算;工业互联网;资源分配;智能优化;深度强化学习

Abstract:

Keywords:

基金项目(Foundation): 北京市自然科学基金-市教委联合资助项目(KZ201911232046);; 北京市自然科学基金-海淀原始创新联合基金项目(L212026)

作者(Author): 方禹,李学华,潘春雨,云翔

参考文献(References)：

• [1] 国务院.国务院关于印发2030年前碳达峰行动方案的通知[EB/OL].(2021-10-26)[2022-04-12].http://www.gov.cn/zhengce/content/2021-10/26/content_5644984.htm.
• [2] Bozorgchenani A,Mashhadi F,Tarchi D,et al.Multi-objective computation sharing in energy and delay constrained mobile edge computing environments[J].IEEE Transactions on Mobile Computing,2021,20(10):2992-3005.
• [3] 闫文卿.面向智慧健康领域的移动边缘计算服务优化研究[D].南京：南京邮电大学，2021.
• [4] 王鸿璋.基于移动边缘计算的视频业务QoS保障研究[D].北京：北京邮电大学，2021.
• [5] Li Q,Ma X,Zhou A,et al.User-oriented edge node grouping in mobile edge computing[J/OL].IEEE Transactions on Mobile Computing(2021-12-31)[2022-04-12].https://ieeexplore.ieee.org/document/9667295.
• [6] Hammer J,Moll P,Hellwagner H.Transparent access to 5G edge computing services[C]//2019 IEEE International Parallel and Distributed Processing Symposium Workshops,Rio de Janeiro,Brazil:IEEE,2019:895-898.
• [7] Liao J X,Wu X W.Resource allocation and task scheduling scheme in priority-based hierarchical edge computing system[C]//2020 19th International Symposium on Distributed Computing and Applications for Business Engineering and Science,Xuzhou,China:IEEE,2020:46-49.
• [8] Zhang S H,Cui G F,Long Y T,et al.Joint computing and communication resource allocation for satellite communication networks with edge computing[J].China Communications,2021,18(7):236-252.
• [9] García A A,May D,Nutting E.Garbage collection for edge computing[C]//2020 IEEE/ACM Symposium on Edge Computing,San Jose,CA,USA:IEEE,2020:319-319.
• [10] Deng S G,Zhao H L,Fang W J,et al.Edge intelligence:the confluence of edge computing and artificial intelligence[J].IEEE Internet of Things Journal,2020,7(8):7457-7469.
• [11] Xu F M,Yang F,Zhao C L,et al.Deep reinforcement learning based joint edge resource management in maritime network[J].China Communications,2020,17(5):211-222.
• [12] Wang X F,Han Y W,Leung V C M,et al.Convergence of edge computing and deep learning:a comprehensive survey[J].IEEE Communications Surveys & Tutorials,2020,22(2):869-904.
• [13] Guim F,Metsch T,Moustafa H,et al.Autonomous lifecycle management for resource-efficient workload orchestration for green edge computing[J].IEEE Transactions on Green Communications and Networking,2022,6(1):571-582.
• [14] Wang J J,Jiang C X,Zhang K,et al.Distributed Q-learning aided heterogeneous network association for energy-efficient IIoT[J].IEEE Transactions on Industrial Informatics,2020,16(4):2756-2764.
• [15] Lai X L,Hu Q,Wang W X,et al.Adaptive resource allocation method based on deep Q network for industrial Internet of things[J].IEEE Access,2020,8:27426-27434.
• [16] Chen Y,Liu Z Y,Zhang Y C,et al.Deep reinforcement learning-based dynamic resource management for mobile edge computing in industrial Internet of things[J].IEEE Transactions on Industrial Informatics,2021,17(7):4925-4934.
• [17] NATH S,Wu J X.Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems[J].Intelligent and Converged Networks,2020,1(2):181-198.
• [18] Wang D Y,Tian X Q,Cui H R,et al.Reinforcement learning-based joint task offloading and migration schemes optimization in mobility-aware MEC network[J].China Communications,2020,17(8):31-44.
• [19] 张一帆.移动边缘计算网络中的任务卸载和资源分配策略研究[D].北京：北京邮电大学，2021.
• [20] Zhao P T,Tian H,Qin C,et al.Energy-saving offloading by jointly allocating radio and computational resources for mobile edge computing[J].IEEE Access,2017,5:11255-11268.
• [21] Wen Y G,Zhang W W,Luo H Y.Energy-optimal Mobile application execution:taming resource-poor mobile devices with cloud clones[C]//2012 IEEE INFOCOM,Orlando,FL,USA:IEEE,2012:2716-2720.
• [22] 王得全.面向工业物联网的计算卸载模块的设计与实现[D].北京：北京邮电大学，2021.
• [23] Chen X,Jiao L,Li W Z,et al.Efficient multi-user computation offloading for mobile-edge cloud computing[J].IEEE/ACM Transactions on Networking,2016,24(5):2795-2808.
• [24] Ye H,LI G Y,JUANG B H F.Deep reinforcement learning based resource allocation for V2V communications[J].IEEE Transactions on Vehicular Technology,2019,68(4):3163-3173.
• [25] Li M S,Gao J,Zhao L,et al.Deep reinforcement learning for collaborative edge computing in vehicular networks[J].IEEE Transactions on Cognitive Communications and Networking,2020,6(4):1122-1135.