Research article Special Issues

Intelligent multicast routing method based on multi-agent deep reinforcement learning in SDWN


  • Received: 17 June 2023 Revised: 11 August 2023 Accepted: 17 August 2023 Published: 04 September 2023
  • Multicast communication technology is widely applied in wireless environments with a high device density. Traditional wireless network architectures have difficulty flexibly obtaining and maintaining global network state information and cannot quickly respond to network state changes, thus affecting the throughput, delay, and other QoS requirements of existing multicasting solutions. Therefore, this paper proposes a new multicast routing method based on multiagent deep reinforcement learning (MADRL-MR) in a software-defined wireless networking (SDWN) environment. First, SDWN technology is adopted to flexibly configure the network and obtain network state information in the form of traffic matrices representing global network links information, such as link bandwidth, delay, and packet loss rate. Second, the multicast routing problem is divided into multiple subproblems, which are solved through multiagent cooperation. To enable each agent to accurately understand the current network state and the status of multicast tree construction, the state space of each agent is designed based on the traffic and multicast tree status matrices, and the set of AP nodes in the network is used as the action space. A novel single-hop action strategy is designed, along with a reward function based on the four states that may occur during tree construction: progress, invalid, loop, and termination. Finally, a decentralized training approach is combined with transfer learning to enable each agent to quickly adapt to the dynamic changes of network link information and accelerate convergence. Simulation experiments show that MADRL-MR outperforms existing algorithms in terms of throughput, delay, packet loss rate, etc., and can establish more intelligent multicast routes. Code and model are available at https://github.com/GuetYe/MADRL-MR_code.

    Citation: Hongwen Hu, Miao Ye, Chenwei Zhao, Qiuxiang Jiang, Xingsi Xue. Intelligent multicast routing method based on multi-agent deep reinforcement learning in SDWN[J]. Mathematical Biosciences and Engineering, 2023, 20(9): 17158-17196. doi: 10.3934/mbe.2023765

    Related Papers:

    [1] Ruiping Yuan, Jiangtao Dou, Juntao Li, Wei Wang, Yingfan Jiang . Multi-robot task allocation in e-commerce RMFS based on deep reinforcement learning. Mathematical Biosciences and Engineering, 2023, 20(2): 1903-1918. doi: 10.3934/mbe.2023087
    [2] Yangjie Sun, Xiaoxi Che, Nan Zhang . 3D human pose detection using nano sensor and multi-agent deep reinforcement learning. Mathematical Biosciences and Engineering, 2023, 20(3): 4970-4987. doi: 10.3934/mbe.2023230
    [3] Jin Zhang, Nan Ma, Zhixuan Wu, Cheng Wang, Yongqiang Yao . Intelligent control of self-driving vehicles based on adaptive sampling supervised actor-critic and human driving experience. Mathematical Biosciences and Engineering, 2024, 21(5): 6077-6096. doi: 10.3934/mbe.2024267
    [4] Siqi Chen, Ran Su . An autonomous agent for negotiation with multiple communication channels using parametrized deep Q-network. Mathematical Biosciences and Engineering, 2022, 19(8): 7933-7951. doi: 10.3934/mbe.2022371
    [5] Shixuan Yao, Xiaochen Liu, Yinghui Zhang, Ze Cui . An approach to solving optimal control problems of nonlinear systems by introducing detail-reward mechanism in deep reinforcement learning. Mathematical Biosciences and Engineering, 2022, 19(9): 9258-9290. doi: 10.3934/mbe.2022430
    [6] Siqi Chen, Yang Yang, Ran Su . Deep reinforcement learning with emergent communication for coalitional negotiation games. Mathematical Biosciences and Engineering, 2022, 19(5): 4592-4609. doi: 10.3934/mbe.2022212
    [7] Jia Mian Tan, Haoran Liao, Wei Liu, Changjun Fan, Jincai Huang, Zhong Liu, Junchi Yan . Hyperparameter optimization: Classics, acceleration, online, multi-objective, and tools. Mathematical Biosciences and Engineering, 2024, 21(6): 6289-6335. doi: 10.3934/mbe.2024275
    [8] Jingxu Xiao, Chaowen Chang, Yingying Ma, Chenli Yang, Lu Yuan . Secure multi-path routing for Internet of Things based on trust evaluation. Mathematical Biosciences and Engineering, 2024, 21(2): 3335-3363. doi: 10.3934/mbe.2024148
    [9] Koji Oshima, Daisuke Yamamoto, Atsuhiro Yumoto, Song-Ju Kim, Yusuke Ito, Mikio Hasegawa . Online machine learning algorithms to optimize performances of complex wireless communication systems. Mathematical Biosciences and Engineering, 2022, 19(2): 2056-2094. doi: 10.3934/mbe.2022097
    [10] Jose Guadalupe Beltran-Hernandez, Jose Ruiz-Pinales, Pedro Lopez-Rodriguez, Jose Luis Lopez-Ramirez, Juan Gabriel Avina-Cervantes . Multi-Stroke handwriting character recognition based on sEMG using convolutional-recurrent neural networks. Mathematical Biosciences and Engineering, 2020, 17(5): 5432-5448. doi: 10.3934/mbe.2020293
  • Multicast communication technology is widely applied in wireless environments with a high device density. Traditional wireless network architectures have difficulty flexibly obtaining and maintaining global network state information and cannot quickly respond to network state changes, thus affecting the throughput, delay, and other QoS requirements of existing multicasting solutions. Therefore, this paper proposes a new multicast routing method based on multiagent deep reinforcement learning (MADRL-MR) in a software-defined wireless networking (SDWN) environment. First, SDWN technology is adopted to flexibly configure the network and obtain network state information in the form of traffic matrices representing global network links information, such as link bandwidth, delay, and packet loss rate. Second, the multicast routing problem is divided into multiple subproblems, which are solved through multiagent cooperation. To enable each agent to accurately understand the current network state and the status of multicast tree construction, the state space of each agent is designed based on the traffic and multicast tree status matrices, and the set of AP nodes in the network is used as the action space. A novel single-hop action strategy is designed, along with a reward function based on the four states that may occur during tree construction: progress, invalid, loop, and termination. Finally, a decentralized training approach is combined with transfer learning to enable each agent to quickly adapt to the dynamic changes of network link information and accelerate convergence. Simulation experiments show that MADRL-MR outperforms existing algorithms in terms of throughput, delay, packet loss rate, etc., and can establish more intelligent multicast routes. Code and model are available at https://github.com/GuetYe/MADRL-MR_code.



    Let C be the complex plane. Denote by CN the N-dimensional complex Euclidean space with the inner product z,w=Nj=1zj¯wj; by |z|2=z,z; by H(CN) the set of all holomorphic functions on CN; and by I the identity operator on CN.

    The Fock space F2(CN) is a Hilbert space of all holomorphic functions fH(CN) with the inner product

    f,g=1(2π)NCNf(z)¯g(z)e12|z|2dν(z),

    where ν(z) denotes Lebesgue measure on CN. To simplify notation, we will often use F2 instead of F2(CN), and we will denote by f the corresponding norm of f. The reproducing kernel functions of the Fock space are given by

    Kw(z)=ez,w2,zCN,

    which means that if fF2, then f(z)=f,Kz for all zCN. It is easy to see that Kw=e|w|2/4. Therefore, the following evaluation holds:

    |f(z)|e|z|24f

    for fF2 and zCN. If kw is the normalization of Kw, then

    kw(z)=ez,w2|w|24,zCN.

    Indeed, F2 is used to describe systems with varying numbers of particles in the states of quantum harmonic oscillators. On the other hand, the reproducing kernels in F2 are used to describe the coherent states in quantum physics. See [17] for more about the Fock space, and see [1,7,11] for the studies of some operators on the Fock space.

    For a given holomorphic mapping φ:CNCN and uH(CN), the weighted composition operator, usually denoted by Wu,φ, on or between some subspaces of H(CN) is defined by

    Wu,φf(z)=u(z)f(φ(z)).

    When u=1, it is the composition operator, usually denoted by Cφ. While φ(z)=z, it is the multiplication operator, usually denoted by Mu.

    Forelli in [8] proved that the isometries on Hardy space Hp defined on the open unit disk (for p2) are certain weighted composition operators, which can be regarded as the earliest presence of the weighted composition operators. Weighted composition operators have also been used in descriptions of adjoints of composition operators (see [4]). An elementary problem is to provide function-theoretic characterizations for which the symbols u and φ induce a bounded or compact weighted composition operator on various holomorphic function spaces. There have been many studies of the weighted composition operators and composition operators on holomorphic function spaces. For instance, several authors have recently worked on the composition operators and weighted composition operators on Fock space. For the one-variable case, Ueki [13] characterized the boundedness and compactness of weighted composition operators on Fock space. As a further work of [13], Le [10] found the easier criteria for the boundedness and compactness of weighted composition operators. Recently, Bhuia in [2] characterized a class of C-normal weighted composition operators on Fock space.

    For the several-variable case, Carswell et al. [3] studied the boundedness and compactness of composition operators. From [3], we see that the one-variable case composition operator Cφ is bounded on Fock space if and only if φ(z)=az+b, where |a|1, and if |a|=1, then b=0. Let A:CNCN be a linear operator. Zhao [14,15,16] characterized the unitary, invertible, and normal weighted composition operator Wu,φ on Fock space, when φ(z)=Az+b and u=kc. Interestingly enough, Zhao [15] proved that for φ(z)=Az+b and u(z)=Kc(z), weighted composition operator Wu,φ is bounded on Fock space if and only if A1 and Aζ,b+Ac=0 whenever |Aζ|=|ζ| for ζCN.

    Motivated by the above-mentioned interesting works, for the special symbols φ(z)=Az+b and u=Kc, here we study the adjoint, self-adjointness, and hyponormality of weighted composition operators on Fock space. Such properties of the abstract or concrete operators (for example, Toeplitz operators, Hankel operators, and composition operators) have been extensively studied on some other holomorphic function spaces. This paper can be regarded as a continuation of the weighted composition operators on Fock space.

    In this section, we characterize the adjoints of weighted composition operators Wu,φ on Fock space, where φ(z)=Az+b and u=Kc.

    We first have the following result:

    Lemma 2.1. Let A, B:CNCN be linear operators with A1 and B1, φ(z)=Az+a, ψ(z)=Bz+b for a,bCN, and the operators Cφ and Cψ be bounded on F2. Then

    CφCψ=WKa,BAz+b,

    where A is the adjoint operator of A.

    Proof. From Lemma 2 in [3], it follows that

    CφCψ=MKaCAzCBz+b=MKaC(Bz+b)Az=MKaCBAz+b=WKa,BAz+b,

    from which the result follows. The proof is complete.

    In Lemma 2.1, we prove that the product of the adjoint of a composition operator and another composition operator is expressed as a weighted composition operator. Next, we will see that in some sense, the converse of Lemma 2.1 is also true. Namely, we will prove that if φ(z)=Az+b, where A:CNCN is a linear operator with A<1, and u=Kc, then the operator Wu,φ on F2 can be written as the product of the adjoint of a composition operator and another composition operator.

    Lemma 2.2. Let A:CNCN be a linear operator with A<1. If A and c satisfy the condition Aζ,c=0 whenever |Aζ|=|ζ|, then there exists a positive integer n such that the operator Wu,φ on F2 defined by φ(z)=Az+b and u(z)=Kc(z) is expressed as

    Wu,φ=Cn+1nAz+cCnn+1z+b.

    Proof. From Theorem 2 in [3], we see that the operator CAz+c is bounded on F2. Since A<1, there exists a large enough positive integer n such that

    (1+1n)A1.

    Also, by Theorem 2 in [3], the operator Cn+1nAz+c is bounded on F2, which implies that the operator Cn+1nAz+c is also bounded on F2. Since |nn+1Iζ|=|ζ| if and only if ζ=0, nn+1Iζ,b=0 whenever |nn+1Iζ|=|ζ|. By Theorem 2 in [3], the operator Cnn+1Iz+b is bounded on F2. Then, it follows from Lemma 2.1 that

    Cn+1nAz+cCnn+1Iz+b=WKc,Az+b.

    The proof is complete.

    Now, we can obtain the adjoint for some weighted composition operators.

    Theorem 2.1. Let φ(z)=Az+b, u(z)=Kc(z), and A and c satisfy Aζ,c=0 whenever |Aζ|=|ζ|. Then it holds that

    Wu,φ=WKb,Az+c.

    Proof. In Lemma 2.2, we have

    Wu,φ=Cn+1nAz+cCnn+1Iz+b. (2.1)

    It follows from (2.1) that

    Wu,φ=Cnn+1Iz+bCn+1nAz+c. (2.2)

    Therefore, from (2.2) and Lemma 2.1, the desired result follows. The proof is complete.

    By using the kernel functions, we can obtain the following result:

    Lemma 2.3. Let the operator Wu,φ be a bounded operator on F2. Then it holds that

    Wu,φKw=¯u(w)Kφ(w).

    Proof. Let f be an arbitrary function in F2. We see that

    Wu,φKw,f=Kw,Wu,φf=¯Wu,φf,Kw=¯u(w)f(φ(w))=¯u(w)Kφ(w),f.

    From this, we deduce that Wu,φKw=¯u(w)Kφ(w). The proof is complete.

    Here, we characterize the self-adjoint weighted composition operators.

    Theorem 2.2. Let A:CNCN be a linear operator, b,cCN, φ(z)=Az+b, u(z)=Kc(z), and the operator Wu,φ be bounded on F2. Then the operator Wu,φ is self-adjoint on F2 if and only if A:CNCN is self-adjoint and b=c.

    Proof. In Lemma 2.3, we have

    Wu,φKw(z)=¯u(w)Kφ(w)=¯Kc(w)ez,φ(w)2=ec,w2ez,Aw+b2. (2.3)

    On the other hand,

    Wu,φKw(z)=u(z)Kw(φ(z))=ez,c2eAz+b,w2. (2.4)

    It is clear that operator Wu,φ is self-adjoint on F2 if and only if

    Wu,φKw=Wu,φKw.

    From (2.3) and (2.4), it follows that

    ec,w2ez,Aw+b2=ez,c2eAz+b,w2. (2.5)

    Letting z=0 in (2.5), we obtain that ec,w2=eb,w2 which implies that

    c,wb,w=4kπi, (2.6)

    where kN. Also, letting w=0 in (2.6), we see that k=0. This shows that c,wb,w=0, that is, c,w=b,w. From this, we deduce that b=c. Therefore, (2.5) becomes ez,Aw2=eAz,w2. From this, we obtain that z,Aw=Az,w, which implies that Az,w=Az,w. This shows that A=A, that is, A:CNCN is self-adjoint.

    Now, assume that A is a self-adjoint operator on CN and b=c. A direct calculation shows that (2.5) holds. Then Wu,φ is a self-adjoint operator on F2. The proof is complete.

    In [14], Zhao proved that the operator Wu,φ on F2 is unitary if and only if there exist an unitary operator A:CNCN, a vector bCN, and a constant α with |α|=1 such that φ(z)=Azb and u(z)=αKA1b(z). Without loss of generality, here we characterize the self-adjoint unitary operator Wu,φ on F2 for the case α=1 and obtain the following result from Theorem 2.2.

    Corollary 2.1. Let A:CNCN be a unitary operator and bCN such that φ(z)=Azb and u(z)=KA1b(z). Then the operator Wu,φ is self-adjoint on F2 if and only if A:CNCN is self-adjoint and Ab+b=0.

    First, we recall the definition of hyponormal operators. An operator T on a Hilbert space H is said to be hyponormal if AxAx for all vectors xH. T is called co-hyponormal if T is hyponormal. In 1950, Halmos, in his attempt to solve the invariant subspace problem, extended the notion of normal operators to two new classes, one of which is now known as the hyponormal operator (see [9]). Clearly, every normal operator is hyponormal. From the proof in [6], it follows that T is hyponormal if and only if there exists a linear operator C with C1 such that T=CT. In some sense, this result can help people realize the characterizations of the hyponormality of some operators. For example, Sadraoui in [12] used this result to characterize the hyponormality of composition operators defined by the linear fractional symbols on Hardy space. On the other hand, some scholars studied the hyponormality of composition operators on Hardy space by using the fact that the operator Cφ on Hardy space is hyponormal if and only if

    Cφf2Cφf2

    for all f in Hardy space. For example, Dennis in [5] used the fact to study the hyponormality of composition operators on Hardy space. In particular, this inequality for norms is used when f is a reproducing kernel function Kw for any wCN. Actually, to the best of our knowledge, there are few studies on the hyponormality of weighted composition operators. Here, we consider this property of weighted composition operators on Fock space.

    First, we have the following result, which can be proved by using the reproducing kernel functions.

    Lemma 3.1. Let wCN and the operator Wu,φ be bounded on F2. Then

    Wu,φKw2=Wu,φWu,φKw(w).

    Proof. From the inner product, we have

    Wu,φKw2=Wu,φKw,Wu,φKw=Wu,φWu,φKw,Kw=Wu,φWu,φKw(w).

    The proof is complete.

    Theorem 3.1. Let A:CNCN be a linear operator, φ(z)=Az+b, u=kc, and the operator Wu,φ be bounded on F2. If the operator Wu,φ is hyponormal on F2, then Abb=Acc and |b||c|.

    Proof. From a direct calculation, we have

    Wu,φKw(z)=u(z)Kw(φ(z))=kc(z)Kw(Az+b)=ez,c2|c|24eAz+b,w2=ez,Aw+c+b,w2|c|24=eb,w2|c|24KAw+c(z). (3.1)

    From (3.1), it follows that

    Wu,φWu,φKw(z)=eb,w2|c|24Wu,φKAw+c(z)=eb,w2|c|24¯u(Aw+c)Kφ(Aw+c)(z)=eb,w2+c,Aw+c2+z,AAw+Ac+b2|c|22=eb+Ac,w2+z,AAw2+z,Ac+b2. (3.2)

    On the other hand, we also have

    Wu,φWu,φKw(z)=¯u(w)Wu,φKφ(w)(z)=¯u(w)u(z)Kφ(w)(φ(z))=ec,w2+z,c2+Az+b,Aw+b2|c|22=ec+Ab,w2+|b|22+z,AAw2+z,c+Ab2|c|22. (3.3)

    From Lemma 3.1, (3.2), and (3.3), it follows that

    Wu,φKw2=Wu,φWu,φKw(w)=ec+Ab,w2+|b|22+|Aw|22+w,c+Ab2|c|22

    and

    Wu,φKw2=Wu,φWu,φKw(w)=eb+Ac,w2+|Aw|22+w,Ac+b2.

    Then, we have

    Wu,φKw2Wu,φKw2=e|Aw|22(ec+Ab,w2+|b|22+w,c+Ab2|c|22eb+Ac,w2+w,Ac+b2),

    which shows that

    Wu,φKw2Wu,φKw20

    for all wCN if and only if

    ec+Ab,w2+|b|22+w,c+Ab2|c|22eb+Ac,w2+w,Ac+b2. (3.4)

    It is clear that (3.4) holds if and only if

    c+Ab,w+|b|2+w,c+Ab|c|2b+Ac,w+w,Ac+b. (3.5)

    From (3.5), we see that (3.4) holds if and only if

    AbAc+cb,w+w,AbAc+cb+|b|2|c|20. (3.6)

    Therefore, we deduce that (3.4) holds for all wCN if and only if |b||c| and Abb=Acc. The proof is complete.

    If b=c=0 in Theorem 3.1, then Wu,φ is reduced into the composition operator CAz. For this case, Theorem 3.1 does not provide any useful information on the operator A:CNCN when CAz is hyponormal on F2. However, we have the following result, which completely characterizes the hyponormal composition operators:

    Theorem 3.2. Let A:CNCN be a linear operator such that CAz is bounded on F2. Then the operator CAz is hyponormal on F2 if and only if A:CNCN is co-hyponormal.

    Proof. Assume that A:CNCN is co-hyponormal. Then there exists an operator B:CNCN with B1 such that A=BA. We therefore have

    CAz=CAz=CABz=CBzCAz.

    Next, we want to show that CBz=1. By Theorem 4 in [3], we have

    CBz=e14(|w0|2|Bw0|2), (3.7)

    where w0 is any solution to (IBB)w=0. From this, we obtain that w0=BBw0, and then

    |Bw0|2=Bw0,Bw0=w0,BBw0=w0,w0=|w0|2. (3.8)

    Thus, by considering (3.7) and (3.8), we see that CBz=1. It follows that the operator CAz is hyponormal on F2.

    Now, assume that the operator CAz is hyponormal on F2. Then there exists a linear operator C on F2 with C1 such that CAz=CCAz. By Lemma 2 in [3], we have CAz=CAz. This shows that CCAz is a composition operator. This result shows that there exists a holomorphic mapping φ:CNCN such that C=Cφ. So Az=A(φ(z)) for all zCN, which implies that there exists a linear operator B:CNCN such that φ(z)=Bz, and then C=CBz. Therefore, A=AB, that is, A=BA. Since C1, this shows that the operator C=CBz is bounded on F2. From Lemma 2.3 in [15], we obtain that B1, which also shows that B1 since B=B. We prove that A:CNCN is co-hyponormal. The proof is complete.

    Remark 3.1. In the paper, we only obtain a necessary condition for the hyponormality of weighted composition operators on Fock space. We hope that the readers can continuously consider the problem in Fock space.

    In this paper, I give a proper description of the adjoint Wu,φ on Fock space for the special symbol functions u(z)=Kc(z) and φ(z)=Az+b. However, it is difficult to give a proper description of the general symbols. On the other hand, I consider the hyponormal weighted composition operators on Fock space and completely characterize hyponormal composition operators on this space. I hope that people are interested in the research in this paper.

    The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

    This study was supported by Sichuan Science and Technology Program (2024NSFSC0416).

    The author declares that he has no competing interests.



    [1] K. A. Farhan, F. Abdel-Fattah, F. Altarawneh, M. Lafi, Survey paper on multicast routing in mobile ad-hoc networks, in 2019 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT), IEEE, Conference Proceedings, (2019), 449–452. https://doi.org/10.1109/JEEIT.2019.8717477
    [2] L. Derdouri, C. Pham, E. M. El Amine Zouaoui, N. Zeghib, Performance analysis of self-organised multicast group in multi-radio multi-channel wireless mesh networks, IET Commun., 14 (2020), 693–702. https://doi.org/10.1049/iet-com.2018.6276 doi: 10.1049/iet-com.2018.6276
    [3] P. M. Ruiz, A. F. Gómez-Skarmeta, Approximating optimal multicast trees in wireless multihop networks, in 10th IEEE Symposium on Computers and Communications (ISCC'05), IEEE, Conference Proceedings, (2005), 686–691. https://doi.org/10.1109/ISCC.2005.34
    [4] I. F. Akyildiz, X. Wang, W. Wang, Wireless mesh networks: a survey, Comput. Networks, 47 (2015), 445–487. https://doi.org/10.1109/MCOM.2005.1509968 doi: 10.1109/MCOM.2005.1509968
    [5] S. Costanzo, L. Galluccio, G. Morabito, S. Palazzo, Software defined wireless network (SDWN): An evolvable architecture for W-PANs, in 2015 IEEE 1st International Forum on Research and Technologies for Society and Industry Leveraging a better tomorrow (RTSI), IEEE, Conference Proceedings, (2015), 23–28. https://doi.org/10.1109/RTSI.2015.7325066
    [6] K. Benzekki, A. El Fergougui, A. Elbelrhiti Elalaoui, Software defined networking (SDN): a survey, Secur. Commun. Netw., 9 (2016), 5803–5833. https://doi.org/10.1002/sec.1737 doi: 10.1002/sec.1737
    [7] S. Babu, P. Mithun, B. Manoj, A novel framework for resource discovery and self-configuration in software defined wireless mesh networks, IEEE Trans. Network Serv. Manage., 17 (2019), 132–146. https://doi.org/10.1109/TNSM.2019.2922107 doi: 10.1109/TNSM.2019.2922107
    [8] L. Kou, G. Markowsky, L. Berman, A fast algorithm for Steiner trees, Acta Inf., 15 (1981), 141–145. https://doi.org/10.1007/BF00288961 doi: 10.1007/BF00288961
    [9] H. Takahashi, An approximate solution for steiner problem in graphs, Math. Japonica, 24 (1980), 573–577.
    [10] V. J. Rayward-Smith, The computation of nearly minimal steiner trees in graphs, Int. J. Math. Educ. Sci. Technol., 14 (1983), 15–23. https://doi.org/10.1080/0020739830140103 doi: 10.1080/0020739830140103
    [11] Y. R. Chen, A. Rezapour, W. G. Tzeng, S. C. Tsai, RL-routing: An SDN routing algorithm based on deep reinforcement learning, IEEE Trans. Network Sci. Eng., 7 (2020), 3185–3199. https://doi.org/10.1109/TNSE.2020.3017751 doi: 10.1109/TNSE.2020.3017751
    [12] D. M. Casas-Velasco, O. M. C. Rendon, N. L. da Fonseca, Intelligent routing based on reinforcement learning for software-defined networking, IEEE Trans. Network Serv. Manage., 18 (2020), 870–881. https://doi.org/10.1109/TNSM.2020.3036911 doi: 10.1109/TNSM.2020.3036911
    [13] D. M. Casas-Velasco, O. M. C. Rendon, N. L. da Fonseca, DRSIR: A deep reinforcement learning approach for routing in software-defined networking, IEEE Trans. Network Serv. Manage., 19 (2021), 4807–4820. https://doi.org/10.1109/TNSM.2021.3132491 doi: 10.1109/TNSM.2021.3132491
    [14] J. Zhang, M. Ye, Z. Guo, C. Y. Yen, H. J. Chao, CFR-RL: Traffic engineering with reinforcement learning in SDN, IEEE J. Sel. Areas Commun., 38 (2020), 2249–2259. https://doi.org/10.1109/JSAC.2020.3000371 doi: 10.1109/JSAC.2020.3000371
    [15] Y. Hou, Y. S. Ong, L. Feng, J. M. Zurada, An evolutionary transfer reinforcement learning framework for multiagent systems, IEEE Trans. Evol. Comput., 21 (2017), 601–615. https://doi.org/10.1109/TEVC.2017.2664665 doi: 10.1109/TEVC.2017.2664665
    [16] Y. Yu, P. Qiu, An improved algorithm for Steiner trees, J. Commun., 23 (2002), 35–40.
    [17] L. Zhou, Y. M. Sun, A delay-constrained steiner tree algorithm using MPH, J. Comput. Res. Dev., 45 (2008), 810–816.
    [18] X. Wang, Steiner tree heuristic algorithm based on weighted node, J. Comput. Appl., 34 (2014), 3414–3416.
    [19] L. Farzinvash, Online multicast tree construction with bandwidth and delay constraints in multi-channel multi-radio wireless mesh networks, Telecommun. Syst., 72 (2019), 413–429. https://doi.org/10.1007/s11235-019-00576-6 doi: 10.1007/s11235-019-00576-6
    [20] M. W. Przewozniczek, K. Walkowiak, A. Sen, M. Komarnicki, P. Lechowicz, The transformation of the k-shortest steiner trees search problem into binary dynamic problem for effective evolutionary methods application, Inf. Sci., 479 (2019), 1–19. https://doi.org/10.1016/j.ins.2018.11.015 doi: 10.1016/j.ins.2018.11.015
    [21] K. Walkowiak, A. Kasprzak, M. Wozniak, Algorithms for calculation of candidate trees for efficient multicasting in elastic optical networks, in 2015 17th International Conference on Transparent Optical Networks (ICTON), IEEE, Conference Proceedings, (2015), 1–4. https://doi.org/10.1109/ICTON.2015.7193692
    [22] L. Martins, D. Santos, T. Gomes, R. Girao-Silva, Determining the minimum cost steiner tree for delay constrained problems, IEEE Access, 9 (2021), 144927–144939. https://doi.org/10.1109/ACCESS.2021.3122024 doi: 10.1109/ACCESS.2021.3122024
    [23] X. Zhang, Y. Wang, G. Geng, J. Yu, Delay-optimized multicast tree packing in software-defined networks, IEEE Trans. Serv. Comput., 16 (2021), 261–275. https://doi.org/10.1109/TSC.2021.3106264 doi: 10.1109/TSC.2021.3106264
    [24] M. Hu, J. Li, C. Cai, T. Deng, W. Xu, Y. Dong, Software defined multicast for large-scale multi-layer leo satellite networks, IEEE Trans. Netw. Serv. Manage., 19 (2022), 2119–2130. https://doi.org/10.1109/TNSM.2022.3151552 doi: 10.1109/TNSM.2022.3151552
    [25] V. Annapurna, C. V. Raj, Improving QoS performance of ATM and MPLS using multicast routing and ACO optimization, in 2022 2nd International Conference on Intelligent Technologies (CONIT), IEEE, Conference Proceedings, (2022), 1–6. https://doi.org/10.1109/CONIT55038.2022.9848211
    [26] X. Zhang, X. Shen, Z. Yu, A novel hybrid ant colony optimization for a multicast routing problem, Algorithms, 12 (2019), 18. https://doi.org/10.3390/a12010018 doi: 10.3390/a12010018
    [27] L. Zhang, Y. Huang, W. Chen, W. Guo, G. Liu, X-architecture steiner tree algorithm with limited routing length inside obstacle, in 2021 11th International Conference on Information Technology in Medicine and Education (ITME), (2021), 152–156. https://doi.org/10.1109/ITME53901.2021.00040
    [28] S. Nath, S. Gupta, S. Biswas, R. Banerjee, J. K. Sing, S. K. Sarkar, Gpso hybrid algorithm for rectilinear steiner tree optimization, in 2020 IEEE VLSI DEVICE CIRCUIT AND SYSTEM (VLSI DCS), IEEE, Conference Proceedings, (2020), 365–369. https://doi.org/10.1109/VLSIDCS47293.2020.9179861
    [29] Q. Zhang, S. Yang, M. Liu, J. Liu, L. Jiang, A new crossover mechanism for genetic algorithms for Steiner tree optimization, IEEE Trans. Cybern., 52 (2020), 3147–3158. https://doi.org/10.1109/TCYB.2020.3005047 doi: 10.1109/TCYB.2020.3005047
    [30] H. J. Heo, N. Kim, B. D. Lee, Multicast tree generation technique using reinforcement learning in sdn environments, in 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), IEEE, Conference Proceedings, (2018), 77–81. https://doi.org/10.1109/SmartWorld.2018.00048
    [31] A. E. Araqi, B. Mahboobi, Joint channel assignment and multicast routing in multi-channel multi-radio wireless mesh networks based on q-learning, in 2019 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing (PACRIM), IEEE, Conference Proceedings, (2019), 1–6. https://doi.org/10.1109/PACRIM47961.2019.8985111
    [32] T. N. Tran, T. V. Nguyen, K. Shim, D. B. Da Costa, B. An, A new deep Q-network design for QoS multicast routing in cognitive radio MANETs, IEEE Access, 9 (2021), 152841–152856. https://doi.org/10.1109/ACCESS.2021.3126844 doi: 10.1109/ACCESS.2021.3126844
    [33] J. Chae, N. Kim, Multicast Tree Generation using Meta Reinforcement Learning in SDN-based Smart Network Platforms, KSII Trans. Internet Inf. Syst., 15 (2021).
    [34] C. Zhao, M. Ye, X. Xue, J. Lv, Q. Jiang, Y. Wang, DRL-M4MR: An intelligent multicast routing approach based on DQN deep reinforcement learning in SDN, Phys. Commun., 55 (2022), 101919. https://doi.org/10.1016/j.phycom.2022.101919 doi: 10.1016/j.phycom.2022.101919
    [35] J. Yang, J. Zhang, H. Wang, Urban traffic control in software defined internet of things via a multi-agent deep reinforcement learning approach, IEEE Trans. Intell. Transp. Syst., 22 (2020), 3742–3754. https://doi.org/10.1109/TITS.2020.3023788 doi: 10.1109/TITS.2020.3023788
    [36] A. Suzuki, R. Kawahara, S. Harada, Cooperative Multi-agent deep reinforcement learning for dynamic virtual network allocation with traffic fluctuations, IEEE Trans. Netw. Serv. Manage., 19 (2022), 1982–2000. https://doi.org/10.1109/TNSM.2022.3149243 doi: 10.1109/TNSM.2022.3149243
    [37] T. Wu, P. Zhou, B. Wang, A. Li, X. Tang, Z. Xu, et al., Joint traffic control and multi-channel reassignment for core backbone network in SDN-IoT: a multi-agent deep reinforcement learning approach, IEEE Trans. Network Sci. Eng., 8 (2020), 231–245. https://doi.org/10.1109/TNSE.2020.3036456 doi: 10.1109/TNSE.2020.3036456
    [38] S. S. Bhavanasi, L. Pappone, F. Esposito, Routing with Graph Convolutional Networks and Multi-Agent Deep Reinforcement Learning, in 2022 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN), IEEE, Conference Proceedings, (2022), 72–77. https://doi.org/10.1109/NFV-SDN56302.2022.9974607
    [39] D. K. Dake, J. D. Gadze, G. S. Klogo, H. Nunoo-Mensah, Multi-agent reinforcement learning framework in SND-IoT for transient load detection and prevention, Technologies, 9 (2021), 44. https://doi.org/10.3390/technologies9030044 doi: 10.3390/technologies9030044
    [40] L. Torrey, M. Taylor, Teaching on a budget: Agents advising agents in reinforcement learning, in Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems, Conference Proceedings, (2013), 1053–1060.
    [41] E. Parisotto, J. L. Ba, R. Salakhutdinov, Actor-mimic: Deep multitask and transfer reinforcement learning, preprint, arXiv: 1511.06342.
    [42] F. L. Da Silva, A. H. R. Costa, A survey on transfer learning for multiagent reinforcement learning systems, J. Artif. Intell. Res., 64 (2019), 645–703. https://doi.org/10.1613/jair.1.11396 doi: 10.1613/jair.1.11396
    [43] Y. Li, Z. P. Cai, H. Xu, LLMP: exploiting LLDP for latency measurement in software-defined data center networks, J. Comput. Sci. Technol., 33 (2018), 277–285. https://doi.org/10.1007/s11390-018-1819-2 doi: 10.1007/s11390-018-1819-2
    [44] L. Al Shalabi, Z. Shaaban, Normalization as a preprocessing engine for data mining and the approach of preference matrix, in 2006 International Conference on Dependability of Computer Systems, IEEE, Conference Proceedings, (2006), 207–214. https://doi.org/10.1109/DEPCOS-RELCOMEX.2006.38
    [45] O. Ashour, M. St-Hilaire, T. Kunz, M. Wang, A survey of applying reinforcement learning techniques to multicast routing, in 2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), IEEE, Conference Proceedings, (2019), 1145–1151. https://doi.org/10.1109/UEMCON47517.2019.8993014
    [46] V. Konda, J. Tsitsiklis, Actor-critic algorithms, in Advances in Neural Information Processing Systems, 12 (1999).
    [47] A. Feriani, E. Hossain, Single and multi-agent deep reinforcement learning for AI-enabled wireless networks: A tutorial, IEEE Commun. Surv. Tutorials, 23 (2021), 1226–1252. https://doi.org/10.1109/COMST.2021.3063822 doi: 10.1109/COMST.2021.3063822
    [48] J. Heydari, V. Ganapathy, M. Shah, Dynamic task offloading in multi-agent mobile edge computing networks, in 2019 IEEE Global Communications Conference (GLOBECOM), IEEE, Conference Proceedings, (2019), 1–6. https://doi.org/10.1109/GLOBECOM38437.2019.9013115
    [49] X. Liu, J. Yu, Z. Feng, Y. Gao, Multi-agent reinforcement learning for resource allocation in IoT networks with edge computing, China Commun., 17 (2020), 220–236. https://doi.org/10.23919/JCC.2020.09.017 doi: 10.23919/JCC.2020.09.017
    [50] J. Cui, Y. Liu, A. Nallanathan, The application of multi-agent reinforcement learning in UAV networks, in 2019 IEEE International Conference on Communications Workshops (ICC Workshops), IEEE, Conference Proceedings, (2019), 1–6. https://doi.org/10.1109/ICCW.2019.8756984
    [51] Z. Zhu, K. Lin, A. K. Jain, J. Zhou, Transfer learning in deep reinforcement learning: A survey, preprint, arXiv: 2009.07888.
    [52] Mininet-WIFI, Access date: March 16, Available from: https://mininet-wifi.github.io/.
    [53] Ryu, Access date: March 16, Available from: https://ryu-sdn.org/.
    [54] Iperf, Access date: March 16, Available from: https://iperf.fr.
  • Reader Comments
  • © 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Metrics

Article views(1701) PDF downloads(162) Cited by(2)

/

DownLoad:  Full-Size Img  PowerPoint
Return
Return

Catalog