Multicast communication technology is widely applied in wireless environments with a high device density. Traditional wireless network architectures have difficulty flexibly obtaining and maintaining global network state information and cannot quickly respond to network state changes, thus affecting the throughput, delay, and other QoS requirements of existing multicasting solutions. Therefore, this paper proposes a new multicast routing method based on multiagent deep reinforcement learning (MADRL-MR) in a software-defined wireless networking (SDWN) environment. First, SDWN technology is adopted to flexibly configure the network and obtain network state information in the form of traffic matrices representing global network links information, such as link bandwidth, delay, and packet loss rate. Second, the multicast routing problem is divided into multiple subproblems, which are solved through multiagent cooperation. To enable each agent to accurately understand the current network state and the status of multicast tree construction, the state space of each agent is designed based on the traffic and multicast tree status matrices, and the set of AP nodes in the network is used as the action space. A novel single-hop action strategy is designed, along with a reward function based on the four states that may occur during tree construction: progress, invalid, loop, and termination. Finally, a decentralized training approach is combined with transfer learning to enable each agent to quickly adapt to the dynamic changes of network link information and accelerate convergence. Simulation experiments show that MADRL-MR outperforms existing algorithms in terms of throughput, delay, packet loss rate, etc., and can establish more intelligent multicast routes. Code and model are available at https://github.com/GuetYe/MADRL-MR_code.
Citation: Hongwen Hu, Miao Ye, Chenwei Zhao, Qiuxiang Jiang, Xingsi Xue. Intelligent multicast routing method based on multi-agent deep reinforcement learning in SDWN[J]. Mathematical Biosciences and Engineering, 2023, 20(9): 17158-17196. doi: 10.3934/mbe.2023765
[1] | Ruiping Yuan, Jiangtao Dou, Juntao Li, Wei Wang, Yingfan Jiang . Multi-robot task allocation in e-commerce RMFS based on deep reinforcement learning. Mathematical Biosciences and Engineering, 2023, 20(2): 1903-1918. doi: 10.3934/mbe.2023087 |
[2] | Yangjie Sun, Xiaoxi Che, Nan Zhang . 3D human pose detection using nano sensor and multi-agent deep reinforcement learning. Mathematical Biosciences and Engineering, 2023, 20(3): 4970-4987. doi: 10.3934/mbe.2023230 |
[3] | Jin Zhang, Nan Ma, Zhixuan Wu, Cheng Wang, Yongqiang Yao . Intelligent control of self-driving vehicles based on adaptive sampling supervised actor-critic and human driving experience. Mathematical Biosciences and Engineering, 2024, 21(5): 6077-6096. doi: 10.3934/mbe.2024267 |
[4] | Siqi Chen, Ran Su . An autonomous agent for negotiation with multiple communication channels using parametrized deep Q-network. Mathematical Biosciences and Engineering, 2022, 19(8): 7933-7951. doi: 10.3934/mbe.2022371 |
[5] | Shixuan Yao, Xiaochen Liu, Yinghui Zhang, Ze Cui . An approach to solving optimal control problems of nonlinear systems by introducing detail-reward mechanism in deep reinforcement learning. Mathematical Biosciences and Engineering, 2022, 19(9): 9258-9290. doi: 10.3934/mbe.2022430 |
[6] | Siqi Chen, Yang Yang, Ran Su . Deep reinforcement learning with emergent communication for coalitional negotiation games. Mathematical Biosciences and Engineering, 2022, 19(5): 4592-4609. doi: 10.3934/mbe.2022212 |
[7] | Jia Mian Tan, Haoran Liao, Wei Liu, Changjun Fan, Jincai Huang, Zhong Liu, Junchi Yan . Hyperparameter optimization: Classics, acceleration, online, multi-objective, and tools. Mathematical Biosciences and Engineering, 2024, 21(6): 6289-6335. doi: 10.3934/mbe.2024275 |
[8] | Jingxu Xiao, Chaowen Chang, Yingying Ma, Chenli Yang, Lu Yuan . Secure multi-path routing for Internet of Things based on trust evaluation. Mathematical Biosciences and Engineering, 2024, 21(2): 3335-3363. doi: 10.3934/mbe.2024148 |
[9] | Koji Oshima, Daisuke Yamamoto, Atsuhiro Yumoto, Song-Ju Kim, Yusuke Ito, Mikio Hasegawa . Online machine learning algorithms to optimize performances of complex wireless communication systems. Mathematical Biosciences and Engineering, 2022, 19(2): 2056-2094. doi: 10.3934/mbe.2022097 |
[10] | Jose Guadalupe Beltran-Hernandez, Jose Ruiz-Pinales, Pedro Lopez-Rodriguez, Jose Luis Lopez-Ramirez, Juan Gabriel Avina-Cervantes . Multi-Stroke handwriting character recognition based on sEMG using convolutional-recurrent neural networks. Mathematical Biosciences and Engineering, 2020, 17(5): 5432-5448. doi: 10.3934/mbe.2020293 |
Multicast communication technology is widely applied in wireless environments with a high device density. Traditional wireless network architectures have difficulty flexibly obtaining and maintaining global network state information and cannot quickly respond to network state changes, thus affecting the throughput, delay, and other QoS requirements of existing multicasting solutions. Therefore, this paper proposes a new multicast routing method based on multiagent deep reinforcement learning (MADRL-MR) in a software-defined wireless networking (SDWN) environment. First, SDWN technology is adopted to flexibly configure the network and obtain network state information in the form of traffic matrices representing global network links information, such as link bandwidth, delay, and packet loss rate. Second, the multicast routing problem is divided into multiple subproblems, which are solved through multiagent cooperation. To enable each agent to accurately understand the current network state and the status of multicast tree construction, the state space of each agent is designed based on the traffic and multicast tree status matrices, and the set of AP nodes in the network is used as the action space. A novel single-hop action strategy is designed, along with a reward function based on the four states that may occur during tree construction: progress, invalid, loop, and termination. Finally, a decentralized training approach is combined with transfer learning to enable each agent to quickly adapt to the dynamic changes of network link information and accelerate convergence. Simulation experiments show that MADRL-MR outperforms existing algorithms in terms of throughput, delay, packet loss rate, etc., and can establish more intelligent multicast routes. Code and model are available at https://github.com/GuetYe/MADRL-MR_code.
Let C be the complex plane. Denote by CN the N-dimensional complex Euclidean space with the inner product ⟨z,w⟩=∑Nj=1zj¯wj; by |z|2=⟨z,z⟩; by H(CN) the set of all holomorphic functions on CN; and by I the identity operator on CN.
The Fock space F2(CN) is a Hilbert space of all holomorphic functions f∈H(CN) with the inner product
⟨f,g⟩=1(2π)N∫CNf(z)¯g(z)e−12|z|2dν(z), |
where ν(z) denotes Lebesgue measure on CN. To simplify notation, we will often use F2 instead of F2(CN), and we will denote by ‖f‖ the corresponding norm of f. The reproducing kernel functions of the Fock space are given by
Kw(z)=e⟨z,w⟩2,z∈CN, |
which means that if f∈F2, then f(z)=⟨f,Kz⟩ for all z∈CN. It is easy to see that ‖Kw‖=e|w|2/4. Therefore, the following evaluation holds:
|f(z)|≤e|z|24‖f‖ |
for f∈F2 and z∈CN. If kw is the normalization of Kw, then
kw(z)=e⟨z,w⟩2−|w|24,z∈CN. |
Indeed, F2 is used to describe systems with varying numbers of particles in the states of quantum harmonic oscillators. On the other hand, the reproducing kernels in F2 are used to describe the coherent states in quantum physics. See [17] for more about the Fock space, and see [1,7,11] for the studies of some operators on the Fock space.
For a given holomorphic mapping φ:CN→CN and u∈H(CN), the weighted composition operator, usually denoted by Wu,φ, on or between some subspaces of H(CN) is defined by
Wu,φf(z)=u(z)f(φ(z)). |
When u=1, it is the composition operator, usually denoted by Cφ. While φ(z)=z, it is the multiplication operator, usually denoted by Mu.
Forelli in [8] proved that the isometries on Hardy space Hp defined on the open unit disk (for p≠2) are certain weighted composition operators, which can be regarded as the earliest presence of the weighted composition operators. Weighted composition operators have also been used in descriptions of adjoints of composition operators (see [4]). An elementary problem is to provide function-theoretic characterizations for which the symbols u and φ induce a bounded or compact weighted composition operator on various holomorphic function spaces. There have been many studies of the weighted composition operators and composition operators on holomorphic function spaces. For instance, several authors have recently worked on the composition operators and weighted composition operators on Fock space. For the one-variable case, Ueki [13] characterized the boundedness and compactness of weighted composition operators on Fock space. As a further work of [13], Le [10] found the easier criteria for the boundedness and compactness of weighted composition operators. Recently, Bhuia in [2] characterized a class of C-normal weighted composition operators on Fock space.
For the several-variable case, Carswell et al. [3] studied the boundedness and compactness of composition operators. From [3], we see that the one-variable case composition operator Cφ is bounded on Fock space if and only if φ(z)=az+b, where |a|≤1, and if |a|=1, then b=0. Let A:CN→CN be a linear operator. Zhao [14,15,16] characterized the unitary, invertible, and normal weighted composition operator Wu,φ on Fock space, when φ(z)=Az+b and u=kc. Interestingly enough, Zhao [15] proved that for φ(z)=Az+b and u(z)=Kc(z), weighted composition operator Wu,φ is bounded on Fock space if and only if ‖A‖≤1 and ⟨Aζ,b+Ac⟩=0 whenever |Aζ|=|ζ| for ζ∈CN.
Motivated by the above-mentioned interesting works, for the special symbols φ(z)=Az+b and u=Kc, here we study the adjoint, self-adjointness, and hyponormality of weighted composition operators on Fock space. Such properties of the abstract or concrete operators (for example, Toeplitz operators, Hankel operators, and composition operators) have been extensively studied on some other holomorphic function spaces. This paper can be regarded as a continuation of the weighted composition operators on Fock space.
In this section, we characterize the adjoints of weighted composition operators Wu,φ on Fock space, where φ(z)=Az+b and u=Kc.
We first have the following result:
Lemma 2.1. Let A, B:CN→CN be linear operators with ‖A‖≤1 and ‖B‖≤1, φ(z)=Az+a, ψ(z)=Bz+b for a,b∈CN, and the operators Cφ and Cψ be bounded on F2. Then
C∗φCψ=WKa,BA∗z+b, |
where A∗ is the adjoint operator of A.
Proof. From Lemma 2 in [3], it follows that
C∗φCψ=MKaCA∗zCBz+b=MKaC(Bz+b)∘A∗z=MKaCBA∗z+b=WKa,BA∗z+b, |
from which the result follows. The proof is complete.
In Lemma 2.1, we prove that the product of the adjoint of a composition operator and another composition operator is expressed as a weighted composition operator. Next, we will see that in some sense, the converse of Lemma 2.1 is also true. Namely, we will prove that if φ(z)=Az+b, where A:CN→CN is a linear operator with ‖A‖<1, and u=Kc, then the operator Wu,φ on F2 can be written as the product of the adjoint of a composition operator and another composition operator.
Lemma 2.2. Let A:CN→CN be a linear operator with ‖A‖<1. If A and c satisfy the condition ⟨A∗ζ,c⟩=0 whenever |A∗ζ|=|ζ|, then there exists a positive integer n such that the operator Wu,φ on F2 defined by φ(z)=Az+b and u(z)=Kc(z) is expressed as
Wu,φ=C∗n+1nA∗z+cCnn+1z+b. |
Proof. From Theorem 2 in [3], we see that the operator CA∗z+c is bounded on F2. Since ‖A∗‖<1, there exists a large enough positive integer n such that
‖(1+1n)A∗‖≤1. |
Also, by Theorem 2 in [3], the operator Cn+1nA∗z+c is bounded on F2, which implies that the operator C∗n+1nA∗z+c is also bounded on F2. Since |nn+1Iζ|=|ζ| if and only if ζ=0, ⟨nn+1Iζ,b⟩=0 whenever |nn+1Iζ|=|ζ|. By Theorem 2 in [3], the operator Cnn+1Iz+b is bounded on F2. Then, it follows from Lemma 2.1 that
C∗n+1nA∗z+cCnn+1Iz+b=WKc,Az+b. |
The proof is complete.
Now, we can obtain the adjoint for some weighted composition operators.
Theorem 2.1. Let φ(z)=Az+b, u(z)=Kc(z), and A and c satisfy ⟨A∗ζ,c⟩=0 whenever |A∗ζ|=|ζ|. Then it holds that
W∗u,φ=WKb,A∗z+c. |
Proof. In Lemma 2.2, we have
Wu,φ=C∗n+1nA∗z+cCnn+1Iz+b. | (2.1) |
It follows from (2.1) that
W∗u,φ=C∗nn+1Iz+bCn+1nA∗z+c. | (2.2) |
Therefore, from (2.2) and Lemma 2.1, the desired result follows. The proof is complete.
By using the kernel functions, we can obtain the following result:
Lemma 2.3. Let the operator Wu,φ be a bounded operator on F2. Then it holds that
W∗u,φKw=¯u(w)Kφ(w). |
Proof. Let f be an arbitrary function in F2. We see that
⟨W∗u,φKw,f⟩=⟨Kw,Wu,φf⟩=¯⟨Wu,φf,Kw⟩=¯u(w)f(φ(w))=¯u(w)⟨Kφ(w),f⟩. |
From this, we deduce that W∗u,φKw=¯u(w)Kφ(w). The proof is complete.
Here, we characterize the self-adjoint weighted composition operators.
Theorem 2.2. Let A:CN→CN be a linear operator, b,c∈CN, φ(z)=Az+b, u(z)=Kc(z), and the operator Wu,φ be bounded on F2. Then the operator Wu,φ is self-adjoint on F2 if and only if A:CN→CN is self-adjoint and b=c.
Proof. In Lemma 2.3, we have
W∗u,φKw(z)=¯u(w)Kφ(w)=¯Kc(w)e⟨z,φ(w)⟩2=e⟨c,w⟩2e⟨z,Aw+b⟩2. | (2.3) |
On the other hand,
Wu,φKw(z)=u(z)Kw(φ(z))=e⟨z,c⟩2e⟨Az+b,w⟩2. | (2.4) |
It is clear that operator Wu,φ is self-adjoint on F2 if and only if
W∗u,φKw=Wu,φKw. |
From (2.3) and (2.4), it follows that
e⟨c,w⟩2e⟨z,Aw+b⟩2=e⟨z,c⟩2e⟨Az+b,w⟩2. | (2.5) |
Letting z=0 in (2.5), we obtain that e⟨c,w⟩2=e⟨b,w⟩2 which implies that
⟨c,w⟩−⟨b,w⟩=4kπi, | (2.6) |
where k∈N. Also, letting w=0 in (2.6), we see that k=0. This shows that ⟨c,w⟩−⟨b,w⟩=0, that is, ⟨c,w⟩=⟨b,w⟩. From this, we deduce that b=c. Therefore, (2.5) becomes e⟨z,Aw⟩2=e⟨Az,w⟩2. From this, we obtain that ⟨z,Aw⟩=⟨Az,w⟩, which implies that ⟨A∗z,w⟩=⟨Az,w⟩. This shows that A=A∗, that is, A:CN→CN is self-adjoint.
Now, assume that A is a self-adjoint operator on CN and b=c. A direct calculation shows that (2.5) holds. Then Wu,φ is a self-adjoint operator on F2. The proof is complete.
In [14], Zhao proved that the operator Wu,φ on F2 is unitary if and only if there exist an unitary operator A:CN→CN, a vector b∈CN, and a constant α with |α|=1 such that φ(z)=Az−b and u(z)=αKA−1b(z). Without loss of generality, here we characterize the self-adjoint unitary operator Wu,φ on F2 for the case α=1 and obtain the following result from Theorem 2.2.
Corollary 2.1. Let A:CN→CN be a unitary operator and b∈CN such that φ(z)=Az−b and u(z)=KA−1b(z). Then the operator Wu,φ is self-adjoint on F2 if and only if A:CN→CN is self-adjoint and Ab+b=0.
First, we recall the definition of hyponormal operators. An operator T on a Hilbert space H is said to be hyponormal if ‖Ax‖≥‖A∗x‖ for all vectors x∈H. T is called co-hyponormal if T∗ is hyponormal. In 1950, Halmos, in his attempt to solve the invariant subspace problem, extended the notion of normal operators to two new classes, one of which is now known as the hyponormal operator (see [9]). Clearly, every normal operator is hyponormal. From the proof in [6], it follows that T is hyponormal if and only if there exists a linear operator C with ‖C‖≤1 such that T∗=CT. In some sense, this result can help people realize the characterizations of the hyponormality of some operators. For example, Sadraoui in [12] used this result to characterize the hyponormality of composition operators defined by the linear fractional symbols on Hardy space. On the other hand, some scholars studied the hyponormality of composition operators on Hardy space by using the fact that the operator Cφ on Hardy space is hyponormal if and only if
‖Cφf‖2≥‖C∗φf‖2 |
for all f in Hardy space. For example, Dennis in [5] used the fact to study the hyponormality of composition operators on Hardy space. In particular, this inequality for norms is used when f is a reproducing kernel function Kw for any w∈CN. Actually, to the best of our knowledge, there are few studies on the hyponormality of weighted composition operators. Here, we consider this property of weighted composition operators on Fock space.
First, we have the following result, which can be proved by using the reproducing kernel functions.
Lemma 3.1. Let w∈CN and the operator Wu,φ be bounded on F2. Then
‖Wu,φKw‖2=W∗u,φWu,φKw(w). |
Proof. From the inner product, we have
‖Wu,φKw‖2=⟨Wu,φKw,Wu,φKw⟩=⟨W∗u,φWu,φKw,Kw⟩=W∗u,φWu,φKw(w). |
The proof is complete.
Theorem 3.1. Let A:CN→CN be a linear operator, φ(z)=Az+b, u=kc, and the operator Wu,φ be bounded on F2. If the operator Wu,φ is hyponormal on F2, then A∗b−b=Ac−c and |b|≤|c|.
Proof. From a direct calculation, we have
Wu,φKw(z)=u(z)Kw(φ(z))=kc(z)Kw(Az+b)=e⟨z,c⟩2−|c|24e⟨Az+b,w⟩2=e⟨z,A∗w+c⟩+⟨b,w⟩2−|c|24=e⟨b,w⟩2−|c|24KA∗w+c(z). | (3.1) |
From (3.1), it follows that
W∗u,φWu,φKw(z)=e⟨b,w⟩2−|c|24W∗u,φKA∗w+c(z)=e⟨b,w⟩2−|c|24¯u(A∗w+c)Kφ(A∗w+c)(z)=e⟨b,w⟩2+⟨c,A∗w+c⟩2+⟨z,AA∗w+Ac+b⟩2−|c|22=e⟨b+Ac,w⟩2+⟨z,AA∗w⟩2+⟨z,Ac+b⟩2. | (3.2) |
On the other hand, we also have
Wu,φW∗u,φKw(z)=¯u(w)Wu,φKφ(w)(z)=¯u(w)u(z)Kφ(w)(φ(z))=e⟨c,w⟩2+⟨z,c⟩2+⟨Az+b,Aw+b⟩2−|c|22=e⟨c+A∗b,w⟩2+|b|22+⟨z,AA∗w⟩2+⟨z,c+A∗b⟩2−|c|22. | (3.3) |
From Lemma 3.1, (3.2), and (3.3), it follows that
‖W∗u,φKw‖2=Wu,φW∗u,φKw(w)=e⟨c+A∗b,w⟩2+|b|22+|A∗w|22+⟨w,c+A∗b⟩2−|c|22 |
and
‖Wu,φKw‖2=W∗u,φWu,φKw(w)=e⟨b+Ac,w⟩2+|A∗w|22+⟨w,Ac+b⟩2. |
Then, we have
‖W∗u,φKw‖2−‖Wu,φKw‖2=e|A∗w|22(e⟨c+A∗b,w⟩2+|b|22+⟨w,c+A∗b⟩2−|c|22−e⟨b+Ac,w⟩2+⟨w,Ac+b⟩2), |
which shows that
‖W∗u,φKw‖2−‖Wu,φKw‖2≤0 |
for all w∈CN if and only if
e⟨c+A∗b,w⟩2+|b|22+⟨w,c+A∗b⟩2−|c|22≤e⟨b+Ac,w⟩2+⟨w,Ac+b⟩2. | (3.4) |
It is clear that (3.4) holds if and only if
⟨c+A∗b,w⟩+|b|2+⟨w,c+A∗b⟩−|c|2≤⟨b+Ac,w⟩+⟨w,Ac+b⟩. | (3.5) |
From (3.5), we see that (3.4) holds if and only if
⟨A∗b−Ac+c−b,w⟩+⟨w,A∗b−Ac+c−b⟩+|b|2−|c|2≤0. | (3.6) |
Therefore, we deduce that (3.4) holds for all w∈CN if and only if |b|≤|c| and A∗b−b=Ac−c. The proof is complete.
If b=c=0 in Theorem 3.1, then Wu,φ is reduced into the composition operator CAz. For this case, Theorem 3.1 does not provide any useful information on the operator A:CN→CN when CAz is hyponormal on F2. However, we have the following result, which completely characterizes the hyponormal composition operators:
Theorem 3.2. Let A:CN→CN be a linear operator such that CAz is bounded on F2. Then the operator CAz is hyponormal on F2 if and only if A:CN→CN is co-hyponormal.
Proof. Assume that A:CN→CN is co-hyponormal. Then there exists an operator B:CN→CN with ‖B‖≤1 such that A=BA∗. We therefore have
C∗Az=CA∗z=CAB∗z=CB∗zCAz. |
Next, we want to show that ‖CB∗z‖=1. By Theorem 4 in [3], we have
‖CB∗z‖=e14(|w0|2−|B∗w0|2), | (3.7) |
where w0 is any solution to (I−BB∗)w=0. From this, we obtain that w0=BB∗w0, and then
|B∗w0|2=⟨B∗w0,B∗w0⟩=⟨w0,BB∗w0⟩=⟨w0,w0⟩=|w0|2. | (3.8) |
Thus, by considering (3.7) and (3.8), we see that ‖CB∗z‖=1. It follows that the operator CAz is hyponormal on F2.
Now, assume that the operator CAz is hyponormal on F2. Then there exists a linear operator C on F2 with ‖C‖≤1 such that C∗Az=CCAz. By Lemma 2 in [3], we have C∗Az=CA∗z. This shows that CCAz is a composition operator. This result shows that there exists a holomorphic mapping φ:CN→CN such that C=Cφ. So A∗z=A(φ(z)) for all z∈CN, which implies that there exists a linear operator B:CN→CN such that φ(z)=B∗z, and then C=CB∗z. Therefore, A∗=AB∗, that is, A=BA∗. Since ‖C‖≤1, this shows that the operator C=CB∗z is bounded on F2. From Lemma 2.3 in [15], we obtain that ‖B∗‖≤1, which also shows that ‖B‖≤1 since ‖B∗‖=‖B‖. We prove that A:CN→CN is co-hyponormal. The proof is complete.
Remark 3.1. In the paper, we only obtain a necessary condition for the hyponormality of weighted composition operators on Fock space. We hope that the readers can continuously consider the problem in Fock space.
In this paper, I give a proper description of the adjoint W∗u,φ on Fock space for the special symbol functions u(z)=Kc(z) and φ(z)=Az+b. However, it is difficult to give a proper description of the general symbols. On the other hand, I consider the hyponormal weighted composition operators on Fock space and completely characterize hyponormal composition operators on this space. I hope that people are interested in the research in this paper.
The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.
This study was supported by Sichuan Science and Technology Program (2024NSFSC0416).
The author declares that he has no competing interests.
[1] | K. A. Farhan, F. Abdel-Fattah, F. Altarawneh, M. Lafi, Survey paper on multicast routing in mobile ad-hoc networks, in 2019 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT), IEEE, Conference Proceedings, (2019), 449–452. https://doi.org/10.1109/JEEIT.2019.8717477 |
[2] |
L. Derdouri, C. Pham, E. M. El Amine Zouaoui, N. Zeghib, Performance analysis of self-organised multicast group in multi-radio multi-channel wireless mesh networks, IET Commun., 14 (2020), 693–702. https://doi.org/10.1049/iet-com.2018.6276 doi: 10.1049/iet-com.2018.6276
![]() |
[3] | P. M. Ruiz, A. F. Gómez-Skarmeta, Approximating optimal multicast trees in wireless multihop networks, in 10th IEEE Symposium on Computers and Communications (ISCC'05), IEEE, Conference Proceedings, (2005), 686–691. https://doi.org/10.1109/ISCC.2005.34 |
[4] |
I. F. Akyildiz, X. Wang, W. Wang, Wireless mesh networks: a survey, Comput. Networks, 47 (2015), 445–487. https://doi.org/10.1109/MCOM.2005.1509968 doi: 10.1109/MCOM.2005.1509968
![]() |
[5] | S. Costanzo, L. Galluccio, G. Morabito, S. Palazzo, Software defined wireless network (SDWN): An evolvable architecture for W-PANs, in 2015 IEEE 1st International Forum on Research and Technologies for Society and Industry Leveraging a better tomorrow (RTSI), IEEE, Conference Proceedings, (2015), 23–28. https://doi.org/10.1109/RTSI.2015.7325066 |
[6] |
K. Benzekki, A. El Fergougui, A. Elbelrhiti Elalaoui, Software defined networking (SDN): a survey, Secur. Commun. Netw., 9 (2016), 5803–5833. https://doi.org/10.1002/sec.1737 doi: 10.1002/sec.1737
![]() |
[7] |
S. Babu, P. Mithun, B. Manoj, A novel framework for resource discovery and self-configuration in software defined wireless mesh networks, IEEE Trans. Network Serv. Manage., 17 (2019), 132–146. https://doi.org/10.1109/TNSM.2019.2922107 doi: 10.1109/TNSM.2019.2922107
![]() |
[8] |
L. Kou, G. Markowsky, L. Berman, A fast algorithm for Steiner trees, Acta Inf., 15 (1981), 141–145. https://doi.org/10.1007/BF00288961 doi: 10.1007/BF00288961
![]() |
[9] | H. Takahashi, An approximate solution for steiner problem in graphs, Math. Japonica, 24 (1980), 573–577. |
[10] |
V. J. Rayward-Smith, The computation of nearly minimal steiner trees in graphs, Int. J. Math. Educ. Sci. Technol., 14 (1983), 15–23. https://doi.org/10.1080/0020739830140103 doi: 10.1080/0020739830140103
![]() |
[11] |
Y. R. Chen, A. Rezapour, W. G. Tzeng, S. C. Tsai, RL-routing: An SDN routing algorithm based on deep reinforcement learning, IEEE Trans. Network Sci. Eng., 7 (2020), 3185–3199. https://doi.org/10.1109/TNSE.2020.3017751 doi: 10.1109/TNSE.2020.3017751
![]() |
[12] |
D. M. Casas-Velasco, O. M. C. Rendon, N. L. da Fonseca, Intelligent routing based on reinforcement learning for software-defined networking, IEEE Trans. Network Serv. Manage., 18 (2020), 870–881. https://doi.org/10.1109/TNSM.2020.3036911 doi: 10.1109/TNSM.2020.3036911
![]() |
[13] |
D. M. Casas-Velasco, O. M. C. Rendon, N. L. da Fonseca, DRSIR: A deep reinforcement learning approach for routing in software-defined networking, IEEE Trans. Network Serv. Manage., 19 (2021), 4807–4820. https://doi.org/10.1109/TNSM.2021.3132491 doi: 10.1109/TNSM.2021.3132491
![]() |
[14] |
J. Zhang, M. Ye, Z. Guo, C. Y. Yen, H. J. Chao, CFR-RL: Traffic engineering with reinforcement learning in SDN, IEEE J. Sel. Areas Commun., 38 (2020), 2249–2259. https://doi.org/10.1109/JSAC.2020.3000371 doi: 10.1109/JSAC.2020.3000371
![]() |
[15] |
Y. Hou, Y. S. Ong, L. Feng, J. M. Zurada, An evolutionary transfer reinforcement learning framework for multiagent systems, IEEE Trans. Evol. Comput., 21 (2017), 601–615. https://doi.org/10.1109/TEVC.2017.2664665 doi: 10.1109/TEVC.2017.2664665
![]() |
[16] | Y. Yu, P. Qiu, An improved algorithm for Steiner trees, J. Commun., 23 (2002), 35–40. |
[17] | L. Zhou, Y. M. Sun, A delay-constrained steiner tree algorithm using MPH, J. Comput. Res. Dev., 45 (2008), 810–816. |
[18] | X. Wang, Steiner tree heuristic algorithm based on weighted node, J. Comput. Appl., 34 (2014), 3414–3416. |
[19] |
L. Farzinvash, Online multicast tree construction with bandwidth and delay constraints in multi-channel multi-radio wireless mesh networks, Telecommun. Syst., 72 (2019), 413–429. https://doi.org/10.1007/s11235-019-00576-6 doi: 10.1007/s11235-019-00576-6
![]() |
[20] |
M. W. Przewozniczek, K. Walkowiak, A. Sen, M. Komarnicki, P. Lechowicz, The transformation of the k-shortest steiner trees search problem into binary dynamic problem for effective evolutionary methods application, Inf. Sci., 479 (2019), 1–19. https://doi.org/10.1016/j.ins.2018.11.015 doi: 10.1016/j.ins.2018.11.015
![]() |
[21] | K. Walkowiak, A. Kasprzak, M. Wozniak, Algorithms for calculation of candidate trees for efficient multicasting in elastic optical networks, in 2015 17th International Conference on Transparent Optical Networks (ICTON), IEEE, Conference Proceedings, (2015), 1–4. https://doi.org/10.1109/ICTON.2015.7193692 |
[22] |
L. Martins, D. Santos, T. Gomes, R. Girao-Silva, Determining the minimum cost steiner tree for delay constrained problems, IEEE Access, 9 (2021), 144927–144939. https://doi.org/10.1109/ACCESS.2021.3122024 doi: 10.1109/ACCESS.2021.3122024
![]() |
[23] |
X. Zhang, Y. Wang, G. Geng, J. Yu, Delay-optimized multicast tree packing in software-defined networks, IEEE Trans. Serv. Comput., 16 (2021), 261–275. https://doi.org/10.1109/TSC.2021.3106264 doi: 10.1109/TSC.2021.3106264
![]() |
[24] |
M. Hu, J. Li, C. Cai, T. Deng, W. Xu, Y. Dong, Software defined multicast for large-scale multi-layer leo satellite networks, IEEE Trans. Netw. Serv. Manage., 19 (2022), 2119–2130. https://doi.org/10.1109/TNSM.2022.3151552 doi: 10.1109/TNSM.2022.3151552
![]() |
[25] | V. Annapurna, C. V. Raj, Improving QoS performance of ATM and MPLS using multicast routing and ACO optimization, in 2022 2nd International Conference on Intelligent Technologies (CONIT), IEEE, Conference Proceedings, (2022), 1–6. https://doi.org/10.1109/CONIT55038.2022.9848211 |
[26] |
X. Zhang, X. Shen, Z. Yu, A novel hybrid ant colony optimization for a multicast routing problem, Algorithms, 12 (2019), 18. https://doi.org/10.3390/a12010018 doi: 10.3390/a12010018
![]() |
[27] | L. Zhang, Y. Huang, W. Chen, W. Guo, G. Liu, X-architecture steiner tree algorithm with limited routing length inside obstacle, in 2021 11th International Conference on Information Technology in Medicine and Education (ITME), (2021), 152–156. https://doi.org/10.1109/ITME53901.2021.00040 |
[28] | S. Nath, S. Gupta, S. Biswas, R. Banerjee, J. K. Sing, S. K. Sarkar, Gpso hybrid algorithm for rectilinear steiner tree optimization, in 2020 IEEE VLSI DEVICE CIRCUIT AND SYSTEM (VLSI DCS), IEEE, Conference Proceedings, (2020), 365–369. https://doi.org/10.1109/VLSIDCS47293.2020.9179861 |
[29] |
Q. Zhang, S. Yang, M. Liu, J. Liu, L. Jiang, A new crossover mechanism for genetic algorithms for Steiner tree optimization, IEEE Trans. Cybern., 52 (2020), 3147–3158. https://doi.org/10.1109/TCYB.2020.3005047 doi: 10.1109/TCYB.2020.3005047
![]() |
[30] | H. J. Heo, N. Kim, B. D. Lee, Multicast tree generation technique using reinforcement learning in sdn environments, in 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), IEEE, Conference Proceedings, (2018), 77–81. https://doi.org/10.1109/SmartWorld.2018.00048 |
[31] | A. E. Araqi, B. Mahboobi, Joint channel assignment and multicast routing in multi-channel multi-radio wireless mesh networks based on q-learning, in 2019 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing (PACRIM), IEEE, Conference Proceedings, (2019), 1–6. https://doi.org/10.1109/PACRIM47961.2019.8985111 |
[32] |
T. N. Tran, T. V. Nguyen, K. Shim, D. B. Da Costa, B. An, A new deep Q-network design for QoS multicast routing in cognitive radio MANETs, IEEE Access, 9 (2021), 152841–152856. https://doi.org/10.1109/ACCESS.2021.3126844 doi: 10.1109/ACCESS.2021.3126844
![]() |
[33] | J. Chae, N. Kim, Multicast Tree Generation using Meta Reinforcement Learning in SDN-based Smart Network Platforms, KSII Trans. Internet Inf. Syst., 15 (2021). |
[34] |
C. Zhao, M. Ye, X. Xue, J. Lv, Q. Jiang, Y. Wang, DRL-M4MR: An intelligent multicast routing approach based on DQN deep reinforcement learning in SDN, Phys. Commun., 55 (2022), 101919. https://doi.org/10.1016/j.phycom.2022.101919 doi: 10.1016/j.phycom.2022.101919
![]() |
[35] |
J. Yang, J. Zhang, H. Wang, Urban traffic control in software defined internet of things via a multi-agent deep reinforcement learning approach, IEEE Trans. Intell. Transp. Syst., 22 (2020), 3742–3754. https://doi.org/10.1109/TITS.2020.3023788 doi: 10.1109/TITS.2020.3023788
![]() |
[36] |
A. Suzuki, R. Kawahara, S. Harada, Cooperative Multi-agent deep reinforcement learning for dynamic virtual network allocation with traffic fluctuations, IEEE Trans. Netw. Serv. Manage., 19 (2022), 1982–2000. https://doi.org/10.1109/TNSM.2022.3149243 doi: 10.1109/TNSM.2022.3149243
![]() |
[37] |
T. Wu, P. Zhou, B. Wang, A. Li, X. Tang, Z. Xu, et al., Joint traffic control and multi-channel reassignment for core backbone network in SDN-IoT: a multi-agent deep reinforcement learning approach, IEEE Trans. Network Sci. Eng., 8 (2020), 231–245. https://doi.org/10.1109/TNSE.2020.3036456 doi: 10.1109/TNSE.2020.3036456
![]() |
[38] | S. S. Bhavanasi, L. Pappone, F. Esposito, Routing with Graph Convolutional Networks and Multi-Agent Deep Reinforcement Learning, in 2022 IEEE Conference on Network Function Virtualization and Software Defined Networks (NFV-SDN), IEEE, Conference Proceedings, (2022), 72–77. https://doi.org/10.1109/NFV-SDN56302.2022.9974607 |
[39] |
D. K. Dake, J. D. Gadze, G. S. Klogo, H. Nunoo-Mensah, Multi-agent reinforcement learning framework in SND-IoT for transient load detection and prevention, Technologies, 9 (2021), 44. https://doi.org/10.3390/technologies9030044 doi: 10.3390/technologies9030044
![]() |
[40] | L. Torrey, M. Taylor, Teaching on a budget: Agents advising agents in reinforcement learning, in Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems, Conference Proceedings, (2013), 1053–1060. |
[41] | E. Parisotto, J. L. Ba, R. Salakhutdinov, Actor-mimic: Deep multitask and transfer reinforcement learning, preprint, arXiv: 1511.06342. |
[42] |
F. L. Da Silva, A. H. R. Costa, A survey on transfer learning for multiagent reinforcement learning systems, J. Artif. Intell. Res., 64 (2019), 645–703. https://doi.org/10.1613/jair.1.11396 doi: 10.1613/jair.1.11396
![]() |
[43] |
Y. Li, Z. P. Cai, H. Xu, LLMP: exploiting LLDP for latency measurement in software-defined data center networks, J. Comput. Sci. Technol., 33 (2018), 277–285. https://doi.org/10.1007/s11390-018-1819-2 doi: 10.1007/s11390-018-1819-2
![]() |
[44] | L. Al Shalabi, Z. Shaaban, Normalization as a preprocessing engine for data mining and the approach of preference matrix, in 2006 International Conference on Dependability of Computer Systems, IEEE, Conference Proceedings, (2006), 207–214. https://doi.org/10.1109/DEPCOS-RELCOMEX.2006.38 |
[45] | O. Ashour, M. St-Hilaire, T. Kunz, M. Wang, A survey of applying reinforcement learning techniques to multicast routing, in 2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), IEEE, Conference Proceedings, (2019), 1145–1151. https://doi.org/10.1109/UEMCON47517.2019.8993014 |
[46] | V. Konda, J. Tsitsiklis, Actor-critic algorithms, in Advances in Neural Information Processing Systems, 12 (1999). |
[47] |
A. Feriani, E. Hossain, Single and multi-agent deep reinforcement learning for AI-enabled wireless networks: A tutorial, IEEE Commun. Surv. Tutorials, 23 (2021), 1226–1252. https://doi.org/10.1109/COMST.2021.3063822 doi: 10.1109/COMST.2021.3063822
![]() |
[48] | J. Heydari, V. Ganapathy, M. Shah, Dynamic task offloading in multi-agent mobile edge computing networks, in 2019 IEEE Global Communications Conference (GLOBECOM), IEEE, Conference Proceedings, (2019), 1–6. https://doi.org/10.1109/GLOBECOM38437.2019.9013115 |
[49] |
X. Liu, J. Yu, Z. Feng, Y. Gao, Multi-agent reinforcement learning for resource allocation in IoT networks with edge computing, China Commun., 17 (2020), 220–236. https://doi.org/10.23919/JCC.2020.09.017 doi: 10.23919/JCC.2020.09.017
![]() |
[50] | J. Cui, Y. Liu, A. Nallanathan, The application of multi-agent reinforcement learning in UAV networks, in 2019 IEEE International Conference on Communications Workshops (ICC Workshops), IEEE, Conference Proceedings, (2019), 1–6. https://doi.org/10.1109/ICCW.2019.8756984 |
[51] | Z. Zhu, K. Lin, A. K. Jain, J. Zhou, Transfer learning in deep reinforcement learning: A survey, preprint, arXiv: 2009.07888. |
[52] | Mininet-WIFI, Access date: March 16, Available from: https://mininet-wifi.github.io/. |
[53] | Ryu, Access date: March 16, Available from: https://ryu-sdn.org/. |
[54] | Iperf, Access date: March 16, Available from: https://iperf.fr. |