Fixed-time optimal consensus for nonlinear strict-feedback multi-agent systems based on reinforcement learning and neural network observers

Kaile Zhang; Zhanheng Chen; Zhiyong Yu; Haijun Jiang; Kaile Zhang; Zhanheng Chen; Zhiyong Yu; Haijun Jiang

doi:10.3934/math.20251330

AIMS Mathematics

2025, Volume 10, Issue 12: 30271-30306. doi: 10.3934/math.20251330

Previous Article Next Article

Research article Special Issues

Fixed-time optimal consensus for nonlinear strict-feedback multi-agent systems based on reinforcement learning and neural network observers

1.
College of Mathematics and Statistics, Yili Normal University, Yining 835000, China
2.
Institute of Applied Mathematics, Yili Normal University, Yining 835000, China
3.
College of Mathematics and System Sciences, Xinjiang University, Urumqi 830017, China

Received: 15 October 2025 Revised: 05 December 2025 Accepted: 17 December 2025 Published: 24 December 2025
MSC : 93A14, 93D50, 90C39, 93E11

This paper investigates the fixed-time consensus control problem for strict-feedback multi-agent systems based on reinforcement learning. First, under the observer–critic–actor framework, neural networks are applied to the observer to address the issue of unmeasurable system states and nonlinear functions. Furthermore, based on the backstepping method, a reinforcement learning algorithm is constructed to obtain the optimal control input, which is then evaluated and optimized by the critic–actor network to derive an approximate optimal control input. Second, by constructing a Lyapunov function and utilizing the boundedness of the critic–actor network matrix trace along with Lyapunov stability theory, the fixed-time consensus of the system is proven. Finally, the effectiveness of the algorithm is verified through numerical simulations.
- neural network,
- reinforcement learning,
- fixed-time consensus,
- backstepping method,
- critic–actor network
Citation: Kaile Zhang, Zhanheng Chen, Zhiyong Yu, Haijun Jiang. Fixed-time optimal consensus for nonlinear strict-feedback multi-agent systems based on reinforcement learning and neural network observers[J]. AIMS Mathematics, 2025, 10(12): 30271-30306. doi: 10.3934/math.20251330

Related Papers:

Abstract

This paper investigates the fixed-time consensus control problem for strict-feedback multi-agent systems based on reinforcement learning. First, under the observer–critic–actor framework, neural networks are applied to the observer to address the issue of unmeasurable system states and nonlinear functions. Furthermore, based on the backstepping method, a reinforcement learning algorithm is constructed to obtain the optimal control input, which is then evaluated and optimized by the critic–actor network to derive an approximate optimal control input. Second, by constructing a Lyapunov function and utilizing the boundedness of the critic–actor network matrix trace along with Lyapunov stability theory, the fixed-time consensus of the system is proven. Finally, the effectiveness of the algorithm is verified through numerical simulations.

References

[1]	J. Zhang, H. Xia, G. Ma, Fixed-time coordinated attitude control for spacecraft formation flying via event-triggered and delayed communication, Adv. Space Res., 73 (2024), 6140–6146. https://doi.org/10.1016/j.asr.2024.03.020 doi: 10.1016/j.asr.2024.03.020
[2]	X. Dong, H. Zhang, X. Xie, Z. Ming, Data-driven distributed $H_\infty$ current sharing consensus optimal control of DC microgrids via reinforcement learning, Adv. Space Res., 71 (2024), 2824–2834. https://doi.org/10.1109/TCSI.2024.3366942 doi: 10.1109/TCSI.2024.3366942
[3]	A. Coppola, D. G. Lui, A. Petrillo, S. Santini, Cooperative driving of heterogeneous uncertain nonlinear connected and autonomous vehicles via distributed switching robust pid-like control, Inf. Sci., 625 (2023), 277–298. https://doi.org/10.1016/j.ins.2023.01.045 doi: 10.1016/j.ins.2023.01.045
[4]	M. H. Moradi, S. Razini, S. M. Hosseinian, State of art of multiagent systems in power engineering: A review, Renew. Sust. Energ. Rev., 58 (2016), 814–824. https://doi.org/10.1016/j.rser.2015.12.339 doi: 10.1016/j.rser.2015.12.339
[5]	W. Wu, Y. Li, S. Tong, Neural network output-feedback consensus fault-tolerant control for nonlinear multiagent systems with intermittent actuator faults, IEEE T. Neural Networ., 34 (2024), 4728–4740. https://doi.org/10.1109/TNNLS.2021.3117364 doi: 10.1109/TNNLS.2021.3117364
[6]	W. Su, C. Mu, S. Zhu, Event-triggered leader-follower bipartite consensus control for nonlinear multi-agent systems under DoS attacks, Sci. China Inf. Sci., 68 (2024), 132206. https://doi.org/10.1007/s11432-024-4148-7 doi: 10.1007/s11432-024-4148-7
[7]	X. Wu, X. He, L. Wu, Finite-time adaptive fault-tolerant control for a class of nonlinear systems with actuator faults, J. Control Autom. Electr. Syst., 31 (2020), 1075–1086. https://doi.org/10.1007/s40313-020-00618-4 doi: 10.1007/s40313-020-00618-4
[8]	Y. Li, G. Lu, K. Li, Fuzzy adaptive event-triggered consensus control for nonlinear multiagent systems with output constraints and DoS attacks, IEEE T. Cybernetics, 55 (2025), 2–13. https://doi.org/10.1109/TCYB.2024.3456821 doi: 10.1109/TCYB.2024.3456821
[9]	T. Yu, J. Cao, K. Lu, Finite-time synchronization control of networked chaotic complex-valued systems with adaptive coupling, IEEE T. Netw. Sci. Eng., 9 (2022), 2510–2520. https://doi.org/10.1109/TNSE.2022.3164773 doi: 10.1109/TNSE.2022.3164773
[10]	X. Chen, J. Sun, Y. Wang, Finite-time control for attitude tracking of spacecraft with actuator faults, Robotics, 10 (2020), 128.
[11]	C. Huang, Z. Liu, C. L. P. Chen, Y. Zhang, Adaptive fixed-time neural control for uncertain nonlinear multiagent systems, IEEE T. Neural Networ., 34 (2022), 10346–10358. https://doi.org/10.1109/TNNLS.2022.3165836 doi: 10.1109/TNNLS.2022.3165836
[12]	F. Wang, G. Lai, Fixed-time control design for nonlinear uncertain systems via adaptive method, Syst. Control Lett., 140 (2020), 104704. https://doi.org/10.1016/j.sysconle.2020.104704 doi: 10.1016/j.sysconle.2020.104704
[13]	A. Polyakov, Nonlinear feedback design for fixed-time stabilization of linear control systems, IEEE Trans. Automat. Contr., 57 (2012), 2106–2110. https://doi.org/10.1109/TAC.2011.2179869 doi: 10.1109/TAC.2011.2179869
[14]	P. Wang, C. Yu, M. Lv, J. Cao, Adaptive fixed-time optimal formation control for uncertain nonlinear multiagent systems using reinforcement learning, IEEE T. Netw. Sci. Eng., 12 (2024), 1729–1743. https://doi.org/10.1109/TNSE.2023.3330266 doi: 10.1109/TNSE.2023.3330266
[15]	S. Du, J. Han, H. Y. Sun, H. G. Han, J. Qiao, Fully distributed practical fixed-time consensus of second-order nonlinear multiagent systems under actuator faults, J. Franklin I., 362 (2025), 108009. https://doi.org/10.1016/j.jfranklin.2025.108009 doi: 10.1016/j.jfranklin.2025.108009
[16]	S. Wang, X. Zhan, J. Wu, B. Wu, L. Chen, Fixed-time consensus algorithm for second-order multi-agent systems with actuator faults based on non-singular terminal sliding mode, Circ. Syst. Signal Pr., 44 (2025), 1–22. https://doi.org/10.1007/s00034-025-03039-3 doi: 10.1007/s00034-025-03039-3
[17]	L. Shi, W. Guo, L. Wang, W. Sun, H. Jahanshahi, Fixed-time consensus control for nonlinear multi-agent systems via fully distributed edge-based adaptive protocols, Int. J. Control, 97 (2024), 1576–1585. https://doi.org/10.1080/00207179.2023.2215068 doi: 10.1080/00207179.2023.2215068
[18]	A. Liu, S. He, H. Yang, Observer-based double event-triggered fast finite-time Cconsensus control for heterogeneous nonlinear multi-agent systems, Nonlinear Dyn., 113 (2025), 1–19. https://doi.org/10.1007/s11071-025-11423-6 doi: 10.1007/s11071-025-11423-6
[19]	W. T. Miller, R. S. Sutton, P. J. Werbos, A menu of designs for reinforcement learning over time, IEEE T. Neural Networ., 3 (1995), 67–95.
[20]	T. Chen, J. Yuan, H. Yang, Event-triggered adaptive neural network backstepping sliding mode control of fractional-order multi-agent systems with input delay, J. Vib. Control, 28 (2022), 1008–1021.
[21]	Y. Wu, G. Zhang, L. B. Wu, Event-triggered adaptive fault-tolerant control for non-affine uncertain systems with output tracking errors constraints, IEEE T. Fuzzy Syst., 30 (2021), 1750–1761. https://doi.org/10.1109/TFUZZ.2021.3065538 doi: 10.1109/TFUZZ.2021.3065538
[22]	J. Wu, F. He, H. Shen, S. Ding, Z. G. Wu, Adaptive NN fixed-time fault-tolerant control for uncertain stochastic system with deferred output constraint via self-triggered mechanism, IEEE T. Cybernetics, 53 (2022), 5892–5903. https://doi.org/10.1109/TCYB.2022.3205765 doi: 10.1109/TCYB.2022.3205765
[23]	Y. Wang, Y. Yang, L. Wu, Adaptive fault-tolerant consensus control of multi-agent systems with event-triggered inputs, Inf. Sci., 650 (2023), 119594. https://doi.org/10.1016/j.ins.2023.119594 doi: 10.1016/j.ins.2023.119594
[24]	J. Wu, H. Lu, W. Wang, Adaptive deep neural network optimized backstepping control for a class of nonlinear strict-feedback systems, In: 2024 12th International Conference on Intelligent Control and Information Processing (ICICIP), 2024,172–181. https://doi.org/10.1109/ICICIP60808.2024.10477826
[25]	B. Zhu, H. Liang, B. Niu, H. Wang, N. Zhao, X. Zhao, Observer-based reinforcement learning for optimal fault-tolerant consensus control of nonlinear multiagent systems via a dynamic event-triggered mechanism, Inf. Sci., 689 (2025), 121350. https://doi.org/10.1016/j.ins.2024.121350 doi: 10.1016/j.ins.2024.121350
[26]	D. Li, J. Dong, Fuzzy control based on reinforcement learning and subsystem error derivatives for strict-feedback systems with an observer, IEEE T. Fuzzy Syst., 31 (2022), 2509–2521. https://doi.org/10.1109/TFUZZ.2022.3227993 doi: 10.1109/TFUZZ.2022.3227993
[27]	J. Liu, Y. Dong, G. Zhou, Security consensus control for multi-agent systems under DoS attacks via reinforcement learning method, J. Franklin I., 361 (2024), 164–176. https://doi.org/10.1016/j.jfranklin.2023.11.032 doi: 10.1016/j.jfranklin.2023.11.032
[28]	X. Song, P. Sun, C. K. Ahn, S. Song, Reinforcement learning-based event-triggered predefined-time optimal fuzzy control for nonlinear constrained systems, IEEE Trans. Fuzzy Syst., 32 (2024), 4646–4659. https://doi.org/10.1109/TFUZZ.2024.3408847 doi: 10.1109/TFUZZ.2024.3408847
[29]	Y. Sun, G. Lu, K. Peng, L. Wu, Finite-time adaptive optimal control of uncertain strict-feedback nonlinear systems based on fuzzy observer and reinforcement learning, Int. J. Syst. Sci., 55 (2023), 1553–1570. https://doi.org/10.1080/00207721.2024.2312882 doi: 10.1080/00207721.2024.2312882
[30]	W. Qi, C. Zhang, G. Zong, S. F. Su, M. Chadli, Finite-time event-triggered stabilization for discrete-time fuzzy markov jump singularly perturbed systems, IEEE T. Cybernetics, 53 (2023), 4511–4520. https://doi.org/10.1109/TCYB.2022.3207430 doi: 10.1109/TCYB.2022.3207430
[31]	B. Zhu, N. Xu, G. Zong, X. Zhao, Adaptive optimized backstepping tracking control for full-state constrained nonlinear strict-feedback systems without using barrier Lyapunov function method, Optim. Contr. Appl. Met., 45 (2024), 2051–2075. https://doi.org/10.1002/oca.3136 doi: 10.1002/oca.3136
[32]	R. Hao, H. Wang, W. Zheng, Finite-time dynamic event-triggered full-state constraints consensus control for multiagent systems with switching topologies and mismatched disturbances, J. Intell. Robot. Syst., 110 (2024), 147. https://doi.org/10.1007/s10846-024-02168-2 doi: 10.1007/s10846-024-02168-2
[33]	Y. J. Liu, S. Tong, Barrier Lyapunov functions-based adaptive control for a class of nonlinear pure-feedback systems with full state constraints, Automatica, 64 (2016), 70–75. https://doi.org/10.1016/j.automatica.2015.10.034 doi: 10.1016/j.automatica.2015.10.034

Reader Comments

Your name:*

Email:*
© 2025 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)