Weekly aircraft maintenance scheduling is a path-dependent sequential decision problem in which dynamic task arrivals, inherited backlog, and uncertain repair efficiency reshape future maintenance capacity. We propose a feasibility-aware proximal policy optimization (PPO) rolling framework that learns aircraft-level task allocation and proactive maintenance preferences and converts them into executable weekly schedules through deterministic priority conversion, eligibility logic, capacity projection, integerization, and a common physical executor. A 50-week evaluation with 10 paired seeds compared the method with FCFS, Greedy-Urgency, an exact one-week receding-horizon MILP, and a genetic algorithm rolling heuristic. In the N = 20 base case, the proposed PPO improved task pool service by 2.05 percentage points and reduced average backlog by 86.89 relative to Greedy-Urgency; relative to FCFS, it increased aircraft availability by 11.09 percentage points, reduced cost by 258.60, and lowered constraint pressure by 34.60 percentage points. It also achieved higher long-horizon service and lower backlog than the exact MILP and GA baselines with approximately 5 ms online decision time. Component studies showed that proactive maintenance improves long-horizon service and backlog control, while action masking improves availability, cost, constraint pressure, and proposal feasibility. The architecture maintained a consistent operating profile across repair efficiency variation and fleet sizes N = 10-30, with online decision times of 3.463-7.173 ms. The resulting framework provides transparent and computationally efficient decision support for weekly aircraft maintenance under uncertain execution capacity.
Citation: Shihong Li, Pengliang Che, Mengjin Qu, Qing Li. A reinforcement learning-based rolling optimization method for weekly aircraft maintenance scheduling under repair efficiency uncertainty[J]. Journal of Industrial and Management Optimization, 2026, 22(9): 4481-4513. doi: 10.3934/jimo.2026156
Weekly aircraft maintenance scheduling is a path-dependent sequential decision problem in which dynamic task arrivals, inherited backlog, and uncertain repair efficiency reshape future maintenance capacity. We propose a feasibility-aware proximal policy optimization (PPO) rolling framework that learns aircraft-level task allocation and proactive maintenance preferences and converts them into executable weekly schedules through deterministic priority conversion, eligibility logic, capacity projection, integerization, and a common physical executor. A 50-week evaluation with 10 paired seeds compared the method with FCFS, Greedy-Urgency, an exact one-week receding-horizon MILP, and a genetic algorithm rolling heuristic. In the N = 20 base case, the proposed PPO improved task pool service by 2.05 percentage points and reduced average backlog by 86.89 relative to Greedy-Urgency; relative to FCFS, it increased aircraft availability by 11.09 percentage points, reduced cost by 258.60, and lowered constraint pressure by 34.60 percentage points. It also achieved higher long-horizon service and lower backlog than the exact MILP and GA baselines with approximately 5 ms online decision time. Component studies showed that proactive maintenance improves long-horizon service and backlog control, while action masking improves availability, cost, constraint pressure, and proposal feasibility. The architecture maintained a consistent operating profile across repair efficiency variation and fleet sizes N = 10-30, with online decision times of 3.463-7.173 ms. The resulting framework provides transparent and computationally efficient decision support for weekly aircraft maintenance under uncertain execution capacity.
| [1] |
I. Tseremoglou, B. F. Santos, Condition-based maintenance scheduling of an aircraft fleet under partial observability: A deep reinforcement learning approach, Reliab. Eng. Syst. Saf., 241 (2024), 109582. https://doi.org/10.1016/j.ress.2023.109582 doi: 10.1016/j.ress.2023.109582
|
| [2] |
J. Lee, M. Mitici, Deep reinforcement learning for predictive aircraft maintenance using probabilistic remaining-useful-life prognostics, Reliab. Eng. Syst. Saf., 230 (2023), 108908. https://doi.org/10.1016/j.ress.2022.108908 doi: 10.1016/j.ress.2022.108908
|
| [3] |
P. Andrade, C. Silva, B. Ribeiro, B. F. Santos, Aircraft maintenance check scheduling using reinforcement learning, Aerospace, 8 (2021), 113. https://doi.org/10.3390/aerospace8040113 doi: 10.3390/aerospace8040113
|
| [4] | S. Widmer, S. Shaukat, C. L. Wu, Aircraft line maintenance scheduling using simulation and reinforcement learning, Proc. 2023 Winter Simul. Conf. (WSC), 2023,600-611. https://doi.org/10.1109/WSC60868.2023.10408075 |
| [5] |
I. Tseremoglou, P. J. V. Kessel, B. F. Santos, A comparative study of optimization models for condition-based maintenance scheduling of an aircraft fleet, Aerospace, 10 (2023), 120. https://doi.org/10.3390/aerospace10020120 doi: 10.3390/aerospace10020120
|
| [6] | J. P. Sprong, X. Jiang, H. Polinder, A deployment of prognostics to optimize aircraft maintenance: A literature review, Annu. Conf. PHM Soc., 11 (2019). https://doi.org/10.36001/phmconf.2019.v11i1.776 |
| [7] | I. Tseremoglou, M. T. Bieber, W. J. C. Verhagen, B. F. Santos, F. C. Freeman, P. J. V. Kessel, The impact of prognostic uncertainty on condition-based maintenance scheduling: An integrated approach, AIAA Aviation 2022 Forum, 2022. https://doi.org/10.2514/6.2022-3967 |
| [8] |
I. D. Pater, M. Mitici, Predictive maintenance for multi-component systems of repairables with remaining-useful-life prognostics and a limited stock of spare components, Reliab. Eng. Syst. Saf., 214 (2021), 107761. https://doi.org/10.1016/j.ress.2021.107761 doi: 10.1016/j.ress.2021.107761
|
| [9] |
I. D. Pater, A. Reijns, M. Mitici, Alarm-based predictive maintenance scheduling for aircraft engines with imperfect remaining useful life prognostics, Reliab. Eng. Syst. Saf., 221 (2022), 108341. https://doi.org/10.1016/j.ress.2022.108341 doi: 10.1016/j.ress.2022.108341
|
| [10] |
M. T. Bieber, W. J. C. Verhagen, B. F. Santos, An adaptive framework for remaining useful life predictions of aircraft systems, PHM Soc. Eur. Conf., 6 (2021), 60-70. https://doi.org/10.36001/phme.2021.v6i1.2868 doi: 10.36001/phme.2021.v6i1.2868
|
| [11] |
O. Ogunfowora, H. Najjaran, Reinforcement and deep reinforcement learning-based solutions for machine maintenance planning, scheduling policies, and optimization, J. Manuf. Syst., 70 (2023), 244-263. https://doi.org/10.1016/j.jmsy.2023.07.014 doi: 10.1016/j.jmsy.2023.07.014
|
| [12] | R. Lamprecht, F. Wurst, M. F. Huber, Reinforcement learning based condition-oriented maintenance scheduling for flow line systems, Proc. 2021 IEEE 19th Int. Conf. Ind. Inform. (INDIN), 2021. https://doi.org/10.1109/INDIN45523.2021.9557373 |
| [13] |
K. Vos, Z. Peng, E. Lee, W. Wang, Aircraft fleet availability optimisation: A reinforcement learning approach, Aeronaut. J., 127 (2023), 2204-2218. https://doi.org/10.1017/aer.2023.104 doi: 10.1017/aer.2023.104
|
| [14] | V. Mattila, K. Virtanen, Scheduling fighter aircraft maintenance with reinforcement learning, Proc. 2011 Winter Simul. Conf. (WSC), 2011. https://doi.org/10.1109/WSC.2011.6147962 |
| [15] |
R. S. Sutton, A. G. Barto, Reinforcement Learning: An Introduction, IEEE Trans. Neural Netw. Learn. Syst., 9 (2018), 1054. https://doi.org/10.1109/TNN.1998.712192 doi: 10.1109/TNN.1998.712192
|
| [16] |
T. A. Feo, J. F. Bard, Flight scheduling and maintenance base planning, Manage. Sci., 35 (1989), 1415-1432. https://doi.org/10.1287/mnsc.35.12.1415 doi: 10.1287/mnsc.35.12.1415
|
| [17] |
R. Gopalan, K. T. Talluri, The aircraft maintenance routing problem, Oper. Res., 46 (1998), 260-271. https://doi.org/10.1287/opre.46.2.260 doi: 10.1287/opre.46.2.260
|
| [18] |
C. Barnhart, P. Belobaba, A. R. Odoni, Applications of operations research in the air transport industry, Transp. Sci., 37 (2003), 368-391. https://doi.org/10.1287/trsc.37.4.368.23276 doi: 10.1287/trsc.37.4.368.23276
|
| [19] |
C. Barnhart, A. Cohn, Airline schedule planning: Accomplishments and opportunities, Manuf. Serv. Oper. Manag., 6 (2004), 3-22. https://doi.org/10.1287/msom.1030.0018 doi: 10.1287/msom.1030.0018
|
| [20] |
B. Rexing, C. Barnhart, T. Kniker, A. Jarrah, N. Krishnamurthy, Airline fleet assignment with time windows, Transp. Sci., 34 (2000), 1-20. https://doi.org/10.1287/trsc.34.1.1.12277 doi: 10.1287/trsc.34.1.1.12277
|
| [21] |
Q. Deng, B. F. Santos, R. Curran, A practical dynamic programming based methodology for aircraft maintenance check scheduling optimization, Eur. J. Oper. Res., 281 (2020), 256-273. https://doi.org/10.1016/j.ejor.2019.08.025 doi: 10.1016/j.ejor.2019.08.025
|
| [22] |
C. Lagos, F. Delgado, M. Klapp, Dynamic optimization for airline maintenance operations, Transp. Sci., 54 (2020), 998-1015. https://doi.org/10.1287/trsc.2020.0984 doi: 10.1287/trsc.2020.0984
|
| [23] |
P. J. V. Kessel, F. C. Freeman, B. F. Santos, Airline maintenance task rescheduling in a disruptive environment, Eur. J. Oper. Res., 308 (2023), 605-621. https://doi.org/10.1016/j.ejor.2022.11.017 doi: 10.1016/j.ejor.2022.11.017
|
| [24] |
Y. Qin, Z. X. Wang, F. T. S. Chan, S. H. Chung, T. Qu, A mathematical model and algorithms for the aircraft hangar maintenance scheduling problem, Appl. Math. Model., 67 (2019), 491-509. https://doi.org/10.1016/j.apm.2018.11.008 doi: 10.1016/j.apm.2018.11.008
|
| [25] |
D. T. Sanchez, B. Boyacı, K. G. Zografos, An optimisation framework for airline fleet maintenance scheduling with tail assignment considerations, Transp. Res. Part B Methodol., 133 (2020), 142-164. https://doi.org/10.1016/j.trb.2019.12.008 doi: 10.1016/j.trb.2019.12.008
|
| [26] |
H. Shahmoradi-Moghadam, N. Safaei, S. J. Sadjadi, Robust maintenance scheduling of aircraft fleet: A hybrid simulation-optimization approach, IEEE Access, 9 (2021), 17854-17865. https://doi.org/10.1109/ACCESS.2021.3053714 doi: 10.1109/ACCESS.2021.3053714
|
| [27] |
V. Mattila, K. Virtanen, Maintenance scheduling of a fleet of fighter aircraft through multi-objective simulation-optimization, Simulation, 90 (2014), 1023-1040. https://doi.org/10.1177/0037549714540008 doi: 10.1177/0037549714540008
|
| [28] |
Z. Liang, F. Xiao, X. Qian, L. Zhou, X. Jin, X. Lu, et al., A column generation-based heuristic for aircraft recovery problem with airport capacity constraints and maintenance flexibility, Transp. Res. Part B Methodol., 113 (2018), 70-90. https://doi.org/10.1016/j.trb.2018.05.007 doi: 10.1016/j.trb.2018.05.007
|
| [29] |
J. Lee, K. Lee, I. Moon, A reinforcement learning approach for multi-fleet aircraft recovery under airline disruption, Appl. Soft Comput., 129 (2022), 109556. https://doi.org/10.1016/j.asoc.2022.109556 doi: 10.1016/j.asoc.2022.109556
|
| [30] |
X. Jia, T. Guo, Y. Mei, M. Zhang, F. Xiao, X. Gu, et al., A self-adapting memetic algorithm with reinforcement learning for multi-objective aircraft recovery problems, Appl. Soft Comput., 201 (2026), 115550. https://doi.org/10.1016/j.asoc.2026.115550 doi: 10.1016/j.asoc.2026.115550
|
| [31] |
K. Ali, D. Rudinskas, V. Leonavičiūtė, Optimizing aircraft maintenance tasks allocation using mixed integer linear programming, Aviation, 29 (2025), 164-173. https://doi.org/10.3846/aviation.2025.24535 doi: 10.3846/aviation.2025.24535
|
| [32] |
A. C. D. Castillo, A. K. Parlikad, Dynamic fleet management: Integrating predictive and preventive maintenance with operation workload balance to minimise cost, Reliab. Eng. Syst. Saf., 249 (2024), 110243. https://doi.org/10.1016/j.ress.2024.110243 doi: 10.1016/j.ress.2024.110243
|
| [33] |
S. Alaswad, Y. Xiang, A review on condition-based maintenance optimization models for stochastically deteriorating system, Reliab. Eng. Syst. Saf., 157 (2017), 54-63. https://doi.org/10.1016/j.ress.2016.08.009 doi: 10.1016/j.ress.2016.08.009
|
| [34] |
K. Nguyen, P. Do, K. Huynh, C. Berenguer, A. Grall, Joint optimization of monitoring quality and replacement decisions in condition-based maintenance, Reliab. Eng. Syst. Saf., 189 (2019), 177-195. https://doi.org/10.1016/j.ress.2019.04.034 doi: 10.1016/j.ress.2019.04.034
|
| [35] |
X. Liu, Q. Sun, Z. S. Ye, M. Yildirim, Optimal multi-type inspection policy for systems with imperfect online monitoring, Reliab. Eng. Syst. Saf., 207 (2021), 107335. https://doi.org/10.1016/j.ress.2020.107335 doi: 10.1016/j.ress.2020.107335
|
| [36] |
C. Song, C. Zhang, A. Shafieezadeh, R. Xiao, Value of information analysis in non-stationary stochastic decision environments: A reliability-assisted POMDP approach, Reliab. Eng. Syst. Saf., 217 (2022), 108034. https://doi.org/10.1016/j.ress.2021.108034 doi: 10.1016/j.ress.2021.108034
|
| [37] |
M. L. Ruiz-Rodríguez, S. Kubler, J. Robert, Y. L. Traon, Dynamic maintenance scheduling approach under uncertainty: Comparison between reinforcement learning, genetic algorithm simheuristic, dispatching rules, Expert Syst. Appl., 248 (2024), 123404. https://doi.org/10.1016/j.eswa.2024.123404 doi: 10.1016/j.eswa.2024.123404
|
| [38] |
X. Liu, L. Wang, X. Li, M. Qian, X. Su, C. Lu, et al., Scheduling for carrier aircraft hangar maintenance in wave missions involving chance constraints, Asia-Pac. J. Oper. Res., 43 (2026), 2540014. https://doi.org/10.1142/S0217595925400147 doi: 10.1142/S0217595925400147
|
| [39] |
J. Chen, X. Zhou, Reinforcement learning based maintenance scheduling of flexible multi-machine manufacturing systems with varying interactive degradation, Reliab. Eng. Syst. Saf., 260 (2025), 111018. https://doi.org/10.1016/j.ress.2025.111018 doi: 10.1016/j.ress.2025.111018
|
| [40] |
Y. Zhao, C. Smidts, Reinforcement learning for adaptive maintenance policy optimization under imperfect knowledge of the system degradation model and partial observability of system states, Reliab. Eng. Syst. Saf., 224 (2022), 108541. https://doi.org/10.1016/j.ress.2022.108541 doi: 10.1016/j.ress.2022.108541
|
| [41] |
N. Zhang, W. Si, Deep reinforcement learning for condition-based maintenance planning of multi-component systems under dependent competing risks, Reliab. Eng. Syst. Saf., 203 (2020), 107094. https://doi.org/10.1016/j.ress.2020.107094 doi: 10.1016/j.ress.2020.107094
|
| [42] |
K. G. Papakonstantinou, M. Shinozuka, Planning structural inspection and maintenance policies via dynamic programming and Markov processes. Part II: POMDP implementation, Reliab. Eng. Syst. Saf., 130 (2014), 214-224. https://doi.org/10.1016/j.ress.2014.04.006 doi: 10.1016/j.ress.2014.04.006
|
| [43] |
C. P. Andriotis, K. G. Papakonstantinou, Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints, Reliab. Eng. Syst. Saf., 212 (2021), 107551. https://doi.org/10.1016/j.ress.2021.107551 doi: 10.1016/j.ress.2021.107551
|
| [44] |
D. Zhang, Y. Hu, S. Zhang, Y. Zhang, Distributed hierarchical reinforcement learning for dynamic maintenance scheduling of large-scale airline fleets, Reliab. Eng. Syst. Saf., 271 (2026), 112249. https://doi.org/10.1016/j.ress.2026.112249 doi: 10.1016/j.ress.2026.112249
|
| [45] |
C. Li, W. Han, H. Li, J. Liu, X. Wang, Y. Zhang, et al., Deep reinforcement learning for carrier-based aircraft flight deck operations scheduling problem, Neural Netw., 200 (2026), 108776. https://doi.org/10.1016/j.neunet.2026.108776 doi: 10.1016/j.neunet.2026.108776
|
| [46] | X. Wang, X. Yu, Z. Wang, Z. Si, G. Wu, X. Su, et al., Machine learning-driven combinatorial optimization: A systematic review, Arch. Comput. Methods Eng., 2026. https://doi.org/10.1007/s11831-026-10679-4 |
 jimo-22-09-156-s001.pdf |
 |
Figures(8) / Tables(14)
Shihong Li, Pengliang Che, Mengjin Qu, Qing Li. A reinforcement learning-based rolling optimization method for weekly aircraft maintenance scheduling under repair efficiency uncertainty[J]. Journal of Industrial and Management Optimization, 2026, 22(9): 4481-4513. doi: 10.3934/jimo.2026156
DownLoad: