This paper addresses the energy-efficient distributed flexible job shop scheduling problem (EEDFJSP) with the dual objectives of minimizing makespan and total energy consumption (TEC). In this problem, jobs are assigned to multiple factories, each containing several machines that can operate at variable speed levels, leading to a trade-off between productivity and energy efficiency. To solve this complex multi-objective optimization problem, a Q-learning-based hyper-heuristic algorithm (QL-HH) is proposed. The algorithm incorporates a hybrid initialization strategy combining four heuristic rules (NR2, longest remaining workload, minimum completion time, and speed-level heuristics) to generate a high-quality and diverse initial population. A Q-learning framework with an improved dynamic ε-greedy mechanism is developed to adaptively select eight low-level heuristic (LLH) operators according to the current state of each individual, which is characterized by normalized objective values and crowding distance. Additionally, critical path-based speed adjustment operators are designed to reduce energy consumption without increasing makespan. Extensive experiments are conducted on 23 benchmark instances, including high-flexibility and low-flexibility datasets derived from classical FJSP instances. The results demonstrate that QL-HH consistently outperforms four state-of-the-art algorithms (QMOEA/D-AWA, MOEA/D, NSGA-Ⅱ, and KBEA) in terms of hypervolume (HV) and inverted generational distance (IGD), with statistical significance confirmed by Wilcoxon and Friedman tests.
Citation: Haojie Li, Qifang Luo, Yongquan Zhou. Energy-efficient distributed flexible job shop scheduling with variable processing speeds: a Q-learning hyper-heuristic algorithm[J]. Journal of Industrial and Management Optimization, 2026, 22(8): 3926-3967. doi: 10.3934/jimo.2026140
This paper addresses the energy-efficient distributed flexible job shop scheduling problem (EEDFJSP) with the dual objectives of minimizing makespan and total energy consumption (TEC). In this problem, jobs are assigned to multiple factories, each containing several machines that can operate at variable speed levels, leading to a trade-off between productivity and energy efficiency. To solve this complex multi-objective optimization problem, a Q-learning-based hyper-heuristic algorithm (QL-HH) is proposed. The algorithm incorporates a hybrid initialization strategy combining four heuristic rules (NR2, longest remaining workload, minimum completion time, and speed-level heuristics) to generate a high-quality and diverse initial population. A Q-learning framework with an improved dynamic ε-greedy mechanism is developed to adaptively select eight low-level heuristic (LLH) operators according to the current state of each individual, which is characterized by normalized objective values and crowding distance. Additionally, critical path-based speed adjustment operators are designed to reduce energy consumption without increasing makespan. Extensive experiments are conducted on 23 benchmark instances, including high-flexibility and low-flexibility datasets derived from classical FJSP instances. The results demonstrate that QL-HH consistently outperforms four state-of-the-art algorithms (QMOEA/D-AWA, MOEA/D, NSGA-Ⅱ, and KBEA) in terms of hypervolume (HV) and inverted generational distance (IGD), with statistical significance confirmed by Wilcoxon and Friedman tests.
| [1] | S. Kumar, P. Kumar, Challenges and Opportunities for Lean 4.0 in Indian SMEs: A Case Study of Jharkhand, in Industry 4.0 Technologies: Sustainable Manufacturing Supply Chains, Springer Nature Singapore, Singapore, 2024, 77–98. https://doi.org/10.1007/978-981-99-4894-9_6 |
| [2] |
C. Wang, J. Chen, B. Xu, S. Liu, A Discrete Improved Gray Wolf Optimization Algorithm for Dynamic Distributed Flexible Job Shop Scheduling Considering Random Job Arrivals and Machine Breakdowns, Processes, 13 (2025), 1987. https://doi.org/10.3390/pr13071987 doi: 10.3390/pr13071987
|
| [3] |
J. Xie, X. Li, L. Gao, L. Gui, A hybrid genetic tabu search algorithm for distributed flexible job shop scheduling problems, J. Manuf. Syst., 71 (2023), 82–94. https://doi.org/10.1016/j.jmsy.2023.09.002 doi: 10.1016/j.jmsy.2023.09.002
|
| [4] |
S. Cao, R. Li, W. Gong, C. Lu, Inverse model and adaptive neighborhood search based cooperative optimizer for energy-efficient distributed flexible job shop scheduling, Swarm Evol. Comput., 83 (2023), 101419. https://doi.org/10.1016/j.swevo.2023.101419 doi: 10.1016/j.swevo.2023.101419
|
| [5] |
S. Zhang, T. Hou, Q. Qu, A. Glowacz, S. M. Alqhtani, M. Irfan, et al., An Improved Mayfly Method to Solve Distributed Flexible Job Shop Scheduling Problem under Dual Resource Constraints, Sustainability, 14 (2022), 12120. https://doi.org/10.3390/su141912120 doi: 10.3390/su141912120
|
| [6] |
T. Jamrus, C. F. Chien, M. Gen, K. Sethanan, Hybrid Particle Swarm Optimization Combined With Genetic Operators for Flexible Job-Shop Scheduling Under Uncertain Processing Time for Semiconductor Manufacturing, IEEE Trans. Semicond. Manuf., 31 (2018), 32–41. https://doi.org/10.1109/TSM.2017.2758380 doi: 10.1109/TSM.2017.2758380
|
| [7] | C. Y. Zhang, P. G. Li, Y. Q. Rao, S. Z. Li, A new hybrid GA/SA algorithm for the job shop scheduling problem, in Evolutionary Computation in Combinatorial Optimization, Proceedings, Springer-Verlag Berlin, Berlin, 2005,246–259. https://doi.org/10.1007/978-3-540-31996-2_23 |
| [8] |
M. A. Fernandez Perez, F. M. P. Raupp, A Newton-based heuristic algorithm for multi-objective flexible job-shop scheduling problem, J. Intell. Manuf., 27 (2016), 409–416. https://doi.org/10.1007/s10845-014-0872-0 doi: 10.1007/s10845-014-0872-0
|
| [9] |
M. Ziaee, A heuristic algorithm for solving flexible job shop scheduling problem, Int. J. Adv. Manuf. Technol., 71 (2014), 519–528. https://doi.org/10.1007/s00170-013-5510-z doi: 10.1007/s00170-013-5510-z
|
| [10] |
S. J. Wang, B. H. Zhou, L. F. Xi, A filtered-beam-search-based heuristic algorithm for flexible job-shop scheduling problem, Int. J. Prod. Res., 46 (2008), 3027–3058. https://doi.org/10.1080/00207540600988105 doi: 10.1080/00207540600988105
|
| [11] |
J. Li, Y. Han, K. Gao, X. Xiao, P. Duan, Bi-Population Balancing Multi-Objective Algorithm for Fuzzy Flexible Job Shop With Energy and Transportation, IEEE Trans. Autom. Sci. Eng., 21 (2024), 4686–4702. https://doi.org/10.1109/TASE.2023.3300922 doi: 10.1109/TASE.2023.3300922
|
| [12] |
J. Jose Palacios, I. Gonzalez-Rodriguez, C. R. Vela, J. Puente, Robust multiobjective optimisation for fuzzy job shop problems, Appl. Soft. Comput., 56 (2017), 604–616. https://doi.org/10.1016/j.asoc.2016.07.004 doi: 10.1016/j.asoc.2016.07.004
|
| [13] |
Z. Liu, J. Wang, C. Zhang, H. Chu, G. Ding, L. Zhang, A hybrid genetic-particle swarm algorithm based on multilevel neighbourhood structure for flexible job shop scheduling problem, Comput. Oper. Res., 135 (2021), 105431. https://doi.org/10.1016/j.cor.2021.105431 doi: 10.1016/j.cor.2021.105431
|
| [14] |
G. Zhang, L. Gao, Y. Shi, An effective genetic algorithm for the flexible job-shop scheduling problem, Expert Syst. Appl., 38 (2011), 3563–3573. https://doi.org/10.1016/j.eswa.2010.08.145 doi: 10.1016/j.eswa.2010.08.145
|
| [15] |
M. Frutos, A. Carolina Olivera, F. Tohme, A memetic algorithm based on a NSGAII scheme for the flexible job-shop scheduling problem, Ann. Oper. Res., 181 (2010), 745–765. https://doi.org/10.1007/s10479-010-0751-9 doi: 10.1007/s10479-010-0751-9
|
| [16] |
E. Jiang, L. Wang, Multi-objective optimization based on decomposition for flexible job shop scheduling under time-of-use electricity prices, Knowledge-Based Syst., 204 (2020), 106177. https://doi.org/10.1016/j.knosys.2020.106177 doi: 10.1016/j.knosys.2020.106177
|
| [17] |
J. H. Drake, A. Kheiri, E. Ozcan, E. K. Burke, Recent advances in selection hyper-heuristics, Eur. J. Oper. Res., 285 (2020), 405–428. https://doi.org/10.1016/j.ejor.2019.07.073 doi: 10.1016/j.ejor.2019.07.073
|
| [18] |
B. Dong, L. Jiao, J. Wu, A two-phase knowledge based hyper-heuristic scheduling algorithm in cellular system, Knowledge-Based Syst., 88 (2015), 244–252. https://doi.org/10.1016/j.knosys.2015.07.028 doi: 10.1016/j.knosys.2015.07.028
|
| [19] |
S. S. Choong, L. P. Wong, C. P. Lim, Automatic design of hyper-heuristic based on reinforcement learning, Inf. Sci., 436 (2018), 89–107. https://doi.org/10.1016/j.ins.2018.01.005 doi: 10.1016/j.ins.2018.01.005
|
| [20] |
Y. Fu, Z. Zhang, K. Gao, Q. Pan, H. F. Rahman, Integrated distributed flexible job shop scheduling and vehicle routing problem via Q-learning-based evolutionary algorithms, Inf. Sci., 713 (2025), 122169. https://doi.org/10.1016/j.ins.2025.122169 doi: 10.1016/j.ins.2025.122169
|
| [21] |
H. B. Song, J. Lin, A genetic programming hyper-heuristic for the distributed assembly permutation flow-shop scheduling problem with sequence dependent setup times, Swarm and Evolutionary Computation, 60 (2021), 100807. https://doi.org/10.1016/j.swevo.2020.100807 doi: 10.1016/j.swevo.2020.100807
|
| [22] |
I. Golcuk, F. B. Ozsoydan, Q-learning and hyper-heuristic based algorithm recommendation for changing environments, Eng. Appl. Artif. Intell., 102 (2021), 104284. https://doi.org/10.1016/j.engappai.2021.104284 doi: 10.1016/j.engappai.2021.104284
|
| [23] |
J. Lin, X. Wang, R. Niu, Y. He, A Q-Learning-Based Hyper-Heuristic for Capacitated Electric Vehicle Routing Problem, IEEE Trans. Intell. Transp. Syst., 26 (2025), 15746–15757. https://doi.org/10.1109/TITS.2025.3594393 doi: 10.1109/TITS.2025.3594393
|
| [24] |
J. Lin, Y. Y. Li, H. B. Song, Semiconductor final testing scheduling using Q-learning based hyper-heuristic, Expert Syst. Appl., 187 (2022), 115978. https://doi.org/10.1016/j.eswa.2021.115978 doi: 10.1016/j.eswa.2021.115978
|
| [25] |
G. Gong, Q. Deng, X. Gong, W. Liu, Q. Ren, A new double flexible job-shop scheduling problem integrating processing time, green production, and human factor indicators, J. Clean Prod., 174 (2018), 560–576. https://doi.org/10.1016/j.jclepro.2017.10.188 doi: 10.1016/j.jclepro.2017.10.188
|
| [26] |
M. S. Zadeh, Y. Katebi, A. Doniavi, A heuristic model for dynamic flexible job shop scheduling problem considering variable processing times, Int. J. Prod. Res., 57 (2019), 3020–3035. https://doi.org/10.1080/00207543.2018.1524165 doi: 10.1080/00207543.2018.1524165
|
| [27] |
H. Wang, B. R. Sarker, J. Li, J. Li, Adaptive scheduling for assembly job shop with uncertain assembly times based on dual Q-learning, Int. J. Prod. Res., 59 (2021), 5867–5883. https://doi.org/10.1080/00207543.2020.1794075 doi: 10.1080/00207543.2020.1794075
|
| [28] |
K. Geng, L. Liu, S. Wu, A reinforcement learning based memetic algorithm for energy-efficient distributed two-stage flexible job shop scheduling problem, Sci. Rep., 14 (2024), 30816. https://doi.org/10.1038/s41598-024-81064-z doi: 10.1038/s41598-024-81064-z
|
| [29] |
J. Lin, Z. J. Wang, X. Li, A backtracking search hyper-heuristic for the distributed assembly flow-shop scheduling problem, Swarm Evol. Comput., 36 (2017), 124–135. https://doi.org/10.1016/j.swevo.2017.04.007 doi: 10.1016/j.swevo.2017.04.007
|
| [30] |
B. H. Abed-alguni, N. A. Alawad, Distributed Grey Wolf Optimizer for scheduling of workflow applications in cloud environments, Appl. Soft. Comput., 102 (2021), 107113. https://doi.org/10.1016/j.asoc.2021.107113 doi: 10.1016/j.asoc.2021.107113
|
| [31] |
L. De Giovanni, F. Pezzella, An Improved Genetic Algorithm for the Distributed and Flexible Job-shop Scheduling problem, Eur. J. Oper. Res., 200 (2010), 395–408. https://doi.org/10.1016/j.ejor.2009.01.008 doi: 10.1016/j.ejor.2009.01.008
|
| [32] |
H. C. Chang, T. K. Liu, Optimisation of distributed manufacturing flexible job shop scheduling by using hybrid genetic algorithms, J. Intell. Manuf., 28 (2017), 1973–1986. https://doi.org/10.1007/s10845-015-1084-y doi: 10.1007/s10845-015-1084-y
|
| [33] | B. Marzouki, O. B. Driss, K. Ghedira, Decentralized Tabu Searches in Multi Agent system for Distributed and Flexible Job shop Scheduling Problem, in 2017 Ieee/Acs 14th International Conference on Computer Systems and Applications (aiccsa), IEEE, New York, 2017, 1019–1026. https://doi.org/10.1109/AICCSA.2017.133 |
| [34] |
R. Wu, E. Luo, X. Li, H. Tang, Y. Li, Hybrid artificial bee colony algorithm with Q-learning for distributed heterogeneous flexible job shop scheduling problem considering machine preventive maintenance, Swarm Evol. Comput., 98 (2025), 102134. https://doi.org/10.1016/j.swevo.2025.102134 doi: 10.1016/j.swevo.2025.102134
|
| [35] |
Q. Zhang, W. Shao, Z. Shao, D. Pi, J. Gao, Deep reinforcement learning driven trajectory-based meta-heuristic for distributed heterogeneous flexible job shop scheduling problem, Swarm Evol. Comput., 91 (2024), 101753. https://doi.org/10.1016/j.swevo.2024.101753 doi: 10.1016/j.swevo.2024.101753
|
| [36] |
F. Zhao, Z. Fu, L. Wang, H. Sang, A Heterogeneous Graph Reinforcement Learning Framework With Question-Aware Neighborhood Aggregation and Interoption Prompt Attention for Dynamic Flexible Job Shop Scheduling Problem, IEEE Trans. Ind. Inform., 22 (2026), 2863–2874. https://doi.org/10.1109/TII.2025.3646962 doi: 10.1109/TII.2025.3646962
|
| [37] |
E. Jiang, L. Wang, Z. Peng, Solving energy-efficient distributed job shop scheduling via multi-objective evolutionary algorithm with decomposition, Swarm Evol. Comput., 58 (2020), 100745. https://doi.org/10.1016/j.swevo.2020.100745 doi: 10.1016/j.swevo.2020.100745
|
| [38] |
R. Li, W. Gong, L. Wang, C. Lu, X. Zhuang, Surprisingly Popular-Based Adaptive Memetic Algorithm for Energy-Efficient Distributed Flexible Job Shop Scheduling, IEEE T. Cybern., 53 (2023), 8013–8023. https://doi.org/10.1109/TCYB.2023.3280175 doi: 10.1109/TCYB.2023.3280175
|
| [39] |
F. Zhao, M. Li, N. Zhu, T. Xu, Jonrinaldi, Multi-objective fitness landscape-based estimation of distribution algorithm for distributed heterogeneous flexible job shop scheduling problem, Appl. Soft. Comput., 171 (2025), 112780. https://doi.org/10.1016/j.asoc.2025.112780 doi: 10.1016/j.asoc.2025.112780
|
| [40] | F. C. Wu, B. Qian, R. Hu, Z. Q. Zhang, B. Wang, A Q-Learning-Based Hyper-Heuristic Evolutionary Algorithm for the Distributed Flexible Job-Shop Scheduling Problem, in Advanced Intelligent Computing Technology and Applications, Icic 2023, Pt I, Springer-Verlag Singapore Pte Ltd, Singapore, 2023,251–261. https://doi.org/10.1007/978-981-99-4755-3_22 |
| [41] |
Z. Wang, M. He, H. Chen, Y. Hu, Y. Xia, A Q-learning-based evolutionary algorithm for solving the low-carbon multi-objective flexible job shop scheduling problem, Comput. Oper. Res., 185 (2026), 107266. https://doi.org/10.1016/j.cor.2025.107266 doi: 10.1016/j.cor.2025.107266
|
| [42] |
Z. Q. Zhang, Z. M. Wu, B. Qian, R. Hu, A reward-shaping dueling distributed multi-agent deep reinforcement learning framework for dynamic flexible job shop scheduling with random job arrivals, Expert Syst. Appl., 297 (2026), 128951. https://doi.org/10.1016/j.eswa.2025.128951 doi: 10.1016/j.eswa.2025.128951
|
| [43] |
Y. Li, Y. He, Y. Wang, F. Tao, J. W. Sutherland, An optimization method for energy-conscious production in flexible machining job shops with dynamic job arrivals and machine breakdowns, J. Clean Prod., 254 (2020), 120009. https://doi.org/10.1016/j.jclepro.2020.120009 doi: 10.1016/j.jclepro.2020.120009
|
| [44] | M. Hassanchokami, A. Vital-Soto, J. Olivares-Aguila, The Role of Environmental Factors in the Flexible Job-Shop Scheduling Problem: A Literature Review, in Ifac Papersonline, Elsevier, Amsterdam, 2022,175–180. https://doi.org/10.1016/j.ifacol.2022.09.386 |
| [45] |
Z. Pan, L. Wang, J. Wang, Q. Zhang, A Bi-Learning Evolutionary Algorithm for Transportation-Constrained and Distributed Energy-Efficient Flexible Scheduling, IEEE Trans. Evol. Comput., 29 (2025), 232–246. https://doi.org/10.1109/TEVC.2024.3354850 doi: 10.1109/TEVC.2024.3354850
|
| [46] |
F. Yu, C. Lu, J. Zhou, L. Yin, K. Wang, A knowledge-guided bi-population evolutionary algorithm for energy-efficient scheduling of distributed flexible job shop problem, Eng. Appl. Artif. Intell., 128 (2024), 107458. https://doi.org/10.1016/j.engappai.2023.107458 doi: 10.1016/j.engappai.2023.107458
|
| [47] |
Z. Pan, D. Lei, L. Wang, A Knowledge-Based Two-Population Optimization Algorithm for Distributed Energy-Efficient Parallel Machines Scheduling, IEEE T. Cybern., 52 (2022), 5051–5063. https://doi.org/10.1109/TCYB.2020.3026571 doi: 10.1109/TCYB.2020.3026571
|
| [48] |
J. Wang, Y. Liu, S. Ren, C. Wang, W. Wang, Evolutionary game based real-time scheduling for energy-efficient distributed and flexible job shop, J. Clean Prod., 293 (2021), 126093. https://doi.org/10.1016/j.jclepro.2021.126093 doi: 10.1016/j.jclepro.2021.126093
|
| [49] |
L. Chen, C. Yang, D. Zhu, T. Li, A population diffusion algorithm for energy-efficient distributed flexible job shop scheduling problem, Appl. Soft. Comput., 195 (2026), 115056. https://doi.org/10.1016/j.asoc.2026.115056 doi: 10.1016/j.asoc.2026.115056
|
| [50] |
Z. Q. Zhang, X. P. Zhu, Y. X. Xu, B. Qian, B. Yang, R. Hu, MEDHEA: Multidimensional estimation of distribution based hyper-heuristic evolutionary algorithm for energy-efficient distributed assembly no-wait flow-shop scheduling problem, Expert Syst. Appl., 271 (2025), 126526. https://doi.org/10.1016/j.eswa.2025.126526 doi: 10.1016/j.eswa.2025.126526
|
| [51] |
B. Naderi, R. Ruiz, The distributed permutation flowshop scheduling problem, Comput. Oper. Res., 37 (2010), 754–768. https://doi.org/10.1016/j.cor.2009.06.019 doi: 10.1016/j.cor.2009.06.019
|
| [52] |
Z. Zhang, Y. Fu, K. Gao, Q. Pan, M. Huang, A learning-driven multi-objective cooperative artificial bee colony algorithm for distributed flexible job shop scheduling problems with preventive maintenance and transportation operations, Comput. Ind. Eng., 196 (2024), 110484. https://doi.org/10.1016/j.cie.2024.110484 doi: 10.1016/j.cie.2024.110484
|
| [53] |
K. Z. Gao, P. N. Suganthan, M. F. Tasgetiren, Q. K. Pan, Q. Q. Sun, Effective ensembles of heuristics for scheduling flexible job shop problem with new job insertion, Comput. Ind. Eng., 90 (2015), 107–117. https://doi.org/10.1016/j.cie.2015.09.005 doi: 10.1016/j.cie.2015.09.005
|
| [54] |
J. Deng, J. Zhang, S. Yang, A cooperative Q-learning-based memetic algorithm for distributed assembly heterogeneous flexible flowshop scheduling, Expert Syst. Appl., 288 (2025), 128198. https://doi.org/10.1016/j.eswa.2025.128198 doi: 10.1016/j.eswa.2025.128198
|
| [55] |
K. Lei, P. Guo, W. Zhao, Y. Wang, L. Qian, X. Meng, et al., A multi-action deep reinforcement learning framework for flexible Job-shop scheduling problem, Expert Syst. Appl., 205 (2022), 117796. https://doi.org/10.1016/j.eswa.2022.117796 doi: 10.1016/j.eswa.2022.117796
|
| [56] |
P. Brandimarte, Routing and scheduling in a flexible job shop by tabu search, Ann Oper Res, 41 (1993), 157–183. https://doi.org/10.1007/BF02023073 doi: 10.1007/BF02023073
|
| [57] |
J. Hurink, B. Jurisch, M. Thole, Tabu search for the job-shop scheduling problem with multi-purpose machines, OR Spektrum, 15 (1994), 205–215. https://doi.org/10.1007/BF01719451 doi: 10.1007/BF01719451
|
| [58] |
F. Xiong, H. Liu, Logic-based benders decomposition methods for the distributed flexible job shop scheduling problem, Eur. J. Oper. Res., 329 (2026), 778–797. https://doi.org/10.1016/j.ejor.2025.08.039 doi: 10.1016/j.ejor.2025.08.039
|
| [59] |
S. Du, W. Zhou, D. Wu, M. Fei, An effective discrete monarch butterfly optimization algorithm for distributed blocking flow shop scheduling with an assembly machine, Expert Syst. Appl., 225 (2023), 120113. https://doi.org/10.1016/j.eswa.2023.120113 doi: 10.1016/j.eswa.2023.120113
|
| [60] |
J. Wang, H. Han, L. Wang, A Feedback Learning-Based Memetic Algorithm for Energy-Aware Distributed Flexible Job-Shop Scheduling With Transportation Constraints, IEEE Trans. Evol. Comput., 29 (2025), 1085–1099. https://doi.org/10.1109/TEVC.2024.3388527 doi: 10.1109/TEVC.2024.3388527
|
| [61] |
K. Deb, A. Pratap, S. Agarwal, T. Meyarivan, A fast and elitist multiobjective genetic algorithm: NSGA-Ⅱ, IEEE Trans. Evol. Comput., 6 (2002), 182–197. https://doi.org/10.1109/4235.996017 doi: 10.1109/4235.996017
|
Figures(16) / Tables(20)
Haojie Li, Qifang Luo, Yongquan Zhou. Energy-efficient distributed flexible job shop scheduling with variable processing speeds: a Q-learning hyper-heuristic algorithm[J]. Journal of Industrial and Management Optimization, 2026, 22(8): 3926-3967. doi: 10.3934/jimo.2026140
DownLoad: