Abstract

Deep reinforcement learning (DRL) has recently revolutionized the resolution of decision-making and automated control problems. In the context of networking, there is a growing trend in the research community to apply DRL algorithms to optimization problems such as routing. However, existing proposals fail to achieve good results, often under-performing traditional routing techniques. We argue that the reason behind this poor performance is that they use straightforward representations of networks. In this paper, we propose a DRL-based solution for routing in optical transport networks (OTNs). Contrary to previous works, we propose a more elaborate representation of the network state that reduces the level of knowledge abstraction required for DRL agents and easily captures the singularities of network topologies. Our evaluation results show that using our novel representation, DRL agents achieve better performance and learn how to route traffic in OTNs significantly faster compared to state-of-the-art representations. Additionally, we reverse engineered the routing strategy learned by our DRL agent, and as a result, we found a routing algorithm that outperforms well-known traditional routing heuristics.

© 2019 Optical Society of America

Techniques for applying reinforcement learning to routing and wavelength assignment problems in optical fiber communication networks

Josh W. Nevin, Sam Nallaperuma, Nikita A. Shevchenko, Zacharaya Shabka, Georgios Zervas, and Seb J. Savory
J. Opt. Commun. Netw. 14(9) 733-748 (2022)

Experimental evaluation of a latency-aware routing and spectrum assignment mechanism based on deep reinforcement learning

C. Hernández-Chulde, R. Casellas, R. Martínez, R. Vilalta, and R. Muñoz
J. Opt. Commun. Netw. 15(11) 925-937 (2023)

Deep reinforcement learning for proactive spectrum defragmentation in elastic optical networks

Ehsan Etezadi, Carlos Natalino, Renzo Diaz, Anders Lindgren, Stefan Melin, Lena Wosinska, Paolo Monti, and Marija Furdek
J. Opt. Commun. Netw. 15(10) E86-E96 (2023)

Reconfiguring multicast sessions in elastic optical networks adaptively with graph-aware deep reinforcement learning

Xiaojian Tian, Baojia Li, Rentao Gu, and Zuqing Zhu
J. Opt. Commun. Netw. 13(11) 253-265 (2021)

Leveraging double-agent-based deep reinforcement learning to global optimization of elastic optical networks with enhanced survivability

Xiao Luo, Chen Shi, Liqian Wang, Xue Chen, Yang Li, and Tao Yang
Opt. Express 27(6) 7896-7911 (2019)