Research article

Energy-aware dynamic scheduling in hybrid flow shops via integrated multi-objective evolutionary optimization and Actor–Critic reinforcement learning

  • Published: 10 November 2026
  • 90B35, 90C29

  • Dynamic hybrid flow shops must simultaneously balance production efficiency, volatile energy prices, carbon emissions, and stochastic shop-floor disruptions. Optimization-based schedulers cannot react to real-time events, whereas purely learning-based dispatchers converge slowly and struggle to satisfy operational constraints. In this study, we proposed a Hybrid Push–Pull Reinforcement Learning (HP2-RL) framework for energy-aware dynamic scheduling in hybrid flow shops with parallel heterogeneous machines, finite buffers, and dynamic lot splitting. An upper-layer MOEA/D-DE optimizer with Tchebycheff decomposition handled seven objectives (makespan, tardiness, energy cost, carbon emissions, WIP dwell time, machine idle time, and material-delay penalty) and generated Pareto-oriented scheduling templates; a lower-layer Actor–Critic agent performed event-driven dispatching and lot splitting guided by the selected template; and a closed-loop feedback mechanism triggered rolling-horizon re-optimization every 4 hours or when KPI deviations exceeded 15%. Simulation results over 30 independent runs showed that HP2-RL reduced makespan by 14.5% relative to rule-based dispatching (1240.5 min) and total energy cost by 11.5% relative to static MOEA/D-DE (, 450), achieved the lowest carbon footprint (1,241 kg), and converged within approximately 300 training episodes. Under machine failures, recovery took 58.6 minutes versus 115 minutes for rule-based dispatching. Ablation experiments and Wilcoxon signed-rank tests (p < 0.05) confirmed that each component contributes significantly, and an inference time below 3.8 ms per decision supports real-time deployment.

    Citation: Kuo-Feng Huang. Energy-aware dynamic scheduling in hybrid flow shops via integrated multi-objective evolutionary optimization and Actor–Critic reinforcement learning[J]. Journal of Industrial and Management Optimization, 2026, 22(12): 6339-6373. doi: 10.3934/jimo.2026213

    Related Papers:

  • Dynamic hybrid flow shops must simultaneously balance production efficiency, volatile energy prices, carbon emissions, and stochastic shop-floor disruptions. Optimization-based schedulers cannot react to real-time events, whereas purely learning-based dispatchers converge slowly and struggle to satisfy operational constraints. In this study, we proposed a Hybrid Push–Pull Reinforcement Learning (HP2-RL) framework for energy-aware dynamic scheduling in hybrid flow shops with parallel heterogeneous machines, finite buffers, and dynamic lot splitting. An upper-layer MOEA/D-DE optimizer with Tchebycheff decomposition handled seven objectives (makespan, tardiness, energy cost, carbon emissions, WIP dwell time, machine idle time, and material-delay penalty) and generated Pareto-oriented scheduling templates; a lower-layer Actor–Critic agent performed event-driven dispatching and lot splitting guided by the selected template; and a closed-loop feedback mechanism triggered rolling-horizon re-optimization every 4 hours or when KPI deviations exceeded 15%. Simulation results over 30 independent runs showed that HP2-RL reduced makespan by 14.5% relative to rule-based dispatching (1240.5 min) and total energy cost by 11.5% relative to static MOEA/D-DE (, 450), achieved the lowest carbon footprint (1,241 kg), and converged within approximately 300 training episodes. Under machine failures, recovery took 58.6 minutes versus 115 minutes for rule-based dispatching. Ablation experiments and Wilcoxon signed-rank tests (p < 0.05) confirmed that each component contributes significantly, and an inference time below 3.8 ms per decision supports real-time deployment.



    加载中


    [1]

    F. Tao, J. F. Cheng, Q. L. Qi, M. Zhang, F. Y. Sui, Digital twin-driven product design, manufacturing and service with big data, Int. J. Adv. Manuf. Technol., 94 (2017), 3563–3576. https://doi.org/10.1007/s00170-017-0233-1

    doi: 10.1007/s00170-017-0233-1
    [2]

    L. R. Lu, Q. W. Deng, Y. Y. Hu, Q. Luo, S. C. Gao, J. X. Zhang, Deep reinforcement learning for dynamic scheduling in distributed heterogeneous flexible job shop with integrated inventory allocation and product delivery, Eng. Appl. Artif. Intell., 162 (2025), 112681. https://doi.org/10.1016/j.engappai.2025.112681

    doi: 10.1016/j.engappai.2025.112681
    [3]

    Y. J. Qu, X. G. Ming, Z. W. Liu, X. Y. Zhang, Z. T. Hou, Smart manufacturing systems: state of the art and future trends, Int. J. Adv. Manuf. Technol., 103 (2019), 3751–3768. https://doi.org/10.1007/s00170-019-03754-7

    doi: 10.1007/s00170-019-03754-7
    [4]

    M. Füchtenhans, C. H. Glock, Energy-efficient job shop scheduling considering processing speed and incentive-based programs, IFAC-PapersOnLine, 55 (2022), 192–197. https://doi.org/10.1016/j.ifacol.2022.09.389

    doi: 10.1016/j.ifacol.2022.09.389
    [5]

    G. Mouzon, M. B. Yildirim, J. Twomey, Operational methods for minimization of energy consumption of manufacturing equipment, Int. J. Prod. Res., 45 (2007), 4247–4271. https://doi.org/10.1080/00207540701450013

    doi: 10.1080/00207540701450013
    [6]

    R. Li, L. Wang, W. Gong, J. F. Chen, Z. X. Pan, Y. Wu, et al., Evolutionary computation and reinforcement learning integrated algorithm for distributed heterogeneous flowshop scheduling, Eng. Appl. Artif. Intell., 135 (2024), 108775. https://doi.org/10.1016/j.engappai.2024.108775

    doi: 10.1016/j.engappai.2024.108775
    [7]

    Y. Tang, L. Shen, S. Han, Low-carbon flexible job shop scheduling problem based on deep reinforcement learning, Sustainability, 16 (2024), 4544. https://doi.org/10.3390/su16114544

    doi: 10.3390/su16114544
    [8]

    M. L. Pinedo, Scheduling: Theory, algorithms, and systems, Springer Cham, 2022. https://doi.org/10.1007/978-3-031-05921-6

    [9]

    D. Ouelhadj, S. Petrovic, A survey of dynamic scheduling in manufacturing systems, J. Sched., 12 (2009), 417–431. https://doi.org/10.1007/s10951-008-0090-8

    doi: 10.1007/s10951-008-0090-8
    [10]

    R. Ruiz, J. A. Vázquez-Rodríguez, The hybrid flow shop scheduling problem, Eur. J. Oper. Res., 205 (2010), 1–18. https://doi.org/10.1016/j.ejor.2009.09.024

    doi: 10.1016/j.ejor.2009.09.024
    [11]

    P. Godinho, F. G. Branco, Adaptive policies for multi-mode project scheduling under uncertainty, Eur. J. Oper. Res., 216 (2012), 553–562. https://doi.org/10.1016/j.ejor.2011.08.010

    doi: 10.1016/j.ejor.2011.08.010
    [12]

    R. L. Haupt, S. E. Haupt, Practical genetic algorithms, Wiley, 2003. https://doi.org/10.1002/0471671746

    [13]

    M. F. Ashby, Materials and the environment, ScienceDirect, 2013. https://doi.org/10.1016/C2010-0-66554-0

    [14]

    Y. Dong, H. Zhang, C. Wang, X. Zhou, Soft actor-critic DRL algorithm for interval optimal dispatch of integrated energy systems with uncertainty in demand response and renewable energy, Eng. Appl. Artif. Intell., 127 (2024), 107230. https://doi.org/10.1016/j.engappai.2023.107230

    doi: 10.1016/j.engappai.2023.107230
    [15]

    A. Mencaroni, P. Leyman, B. Raa, S. D. Vuyst, D. Claeys, Towards net-zero manufacturing: Carbon-aware scheduling for GHG emissions reduction, J. Clean. Prod., 529 (2025), 146787. https://doi.org/10.1016/j.jclepro.2025.146787

    doi: 10.1016/j.jclepro.2025.146787
    [16]

    K. Gao, Y. Huang, A. Sadollah, L. Wang, A review of energy-efficient scheduling in intelligent production systems, Complex Intell. Syst., 6 (2020), 237–249. https://doi.org/10.1007/s40747-019-00122-6

    doi: 10.1007/s40747-019-00122-6
    [17]

    L. Glomb, F. Liers, F. Rösel, A rolling-horizon approach for multi-period optimization, Eur. J. Oper. Res., 300 (2022), 189–206. https://doi.org/10.1016/j.ejor.2021.07.043

    doi: 10.1016/j.ejor.2021.07.043
    [18]

    K. Deb, A. Pratap, S. Agarwal, T. Meyarivan, A fast and elitist multiobjective genetic algorithm: NSGA-Ⅱ, IEEE Trans. Evol. Comput., 6 (2002), 182–197. https://doi.org/10.1109/4235.996017

    doi: 10.1109/4235.996017
    [19]

    Q. Zhang, H. Li, MOEA/D: A multiobjective evolutionary algorithm based on decomposition, IEEE Trans. Evol. Comput., 11 (2007), 712–731. https://doi.org/10.1109/TEVC.2007.892759

    doi: 10.1109/TEVC.2007.892759
    [20]

    C. A. C. Coello, G. B. Lamont, D. A. V. Veldhuizen, Evolutionary algorithms for solving multi-objective problems, Springer, 2007. https://doi.org/10.1007/978-0-387-36797-2

    [21]

    K. Tamssaouet, S. Dauzère-Pérès, S. Knopp, A. Bitar, C. Yugma, Multiobjective optimization for complex flexible job-shop scheduling problems, Eur. J. Oper. Res., 296 (2022), 87–100. https://doi.org/10.1016/j.ejor.2021.03.069

    doi: 10.1016/j.ejor.2021.03.069
    [22]

    G. Fu, T. Yu, W. Li, Q. Deng, B. Yang, A decomposition-based multiobjective optimization evolutionary algorithm with adaptive weight generation strategy, Math. Probl. Eng., 2021 (2021), 2764558. https://doi.org/10.1155/2021/2764558

    doi: 10.1155/2021/2764558
    [23]

    W. Chen, J. Wang, G. Yu, Y. Hu, Energy-efficient hybrid flow-shop scheduling under time-of-use and ladder electricity tariffs, Appl. Sci., 12 (2022), 6456. https://doi.org/10.3390/app12136456

    doi: 10.3390/app12136456
    [24]

    D. Schweisgut, A. Benoit, Y. Robert, H. Meyerhenke, Carbon-aware workflow scheduling with fixed mapping and deadline constraint, Proc. Int. Conf. Parallel Process., 2025,627–637. https://doi.org/10.1145/3754598.3754673

    doi: 10.1145/3754598.3754673
    [25]

    M. S. Cavallieri, E. Viles, J. R. Montoya-Torresb, Sustainable production scheduling: insights from real-life applications, Prod. Plan. Control., 36 (2025), 1595–1622. https://doi.org/10.1080/09537287.2024.2389879

    doi: 10.1080/09537287.2024.2389879
    [26]

    B. Wang, Q. Li, Rolling horizon procedure for large-scale job-shop scheduling problems, IEEE Int. Conf. Autom. Logist., 2007, 18–21. https://doi.org/10.1109/ICAL.2007.4338679

    doi: 10.1109/ICAL.2007.4338679
    [27]

    R. Yu, D. Oron, Single-machine scheduling with fixed energy recharging times to minimize the number of late jobs and the number of just-in-time jobs: A parameterized complexity analysis, Eur. J. Oper. Res., 324 (2025), 40–48. https://doi.org/10.1016/j.ejor.2025.01.007

    doi: 10.1016/j.ejor.2025.01.007
    [28]

    B. Kocot, P. Czarnul, J. Proficz, Energy-aware scheduling for high-performance computing systems: A survey, Energies, 16 (2023), 890. https://doi.org/10.3390/en16020890

    doi: 10.3390/en16020890
    [29]

    R. S. Sutton, A. G. Barto, Reinforcement learning: An introduction, MIT Press, 2018.

    [30]

    Z. Wang, M. Gombolay, Learning scheduling policies for multi-robot coordination with graph attention networks, IEEE Robot. Autom. Lett., 5 (2020), 4509–4516. https://doi.org/10.1109/LRA.2020.3002198

    doi: 10.1109/LRA.2020.3002198
    [31]

    Z. Luo, C. Jiang, L. Liu, X. Zheng, H. Ma, F. Dong, Deep-reinforcement-learning-based production scheduling in industrial internet of things, IEEE Internet Things J., 10 (2023), 19725–19739. https://doi.org/10.1109/JIOT.2023.3283056

    doi: 10.1109/JIOT.2023.3283056
    [32]

    M. Monaci, V. Agasucci, G. Grani, An actor-critic algorithm with policy gradients to solve the job shop scheduling problem using deep double recurrent agents, Eur. J. Oper. Res., 312 (2024), 910–926. https://doi.org/10.1016/j.ejor.2023.07.037

    doi: 10.1016/j.ejor.2023.07.037
    [33]

    P. Stöckermann, S. Feudel, A. Immordino, N. Hayen, T. Altenmüller, M. Gebser, et al., Reinforcement learning based dispatching solutions in semiconductor manufacturing: a literature review on validation and deployment, Prod. Manuf. Res., 13 (2025), 2582472. https://doi.org/10.1080/21693277.2025.2582472

    doi: 10.1080/21693277.2025.2582472
    [34]

    J. Jang, D. Klabjan, H. Liu, N. S. Patel, X. Li, B. Ananthanarayanan, et al., Scalable multi-agent reinforcement learning for factory-wide dynamic scheduling in semiconductor manufacturing, Eng. Appl. Artif. Intell., 161 (2025), 112168. https://doi.org/10.1016/j.engappai.2025.112168

    doi: 10.1016/j.engappai.2025.112168
    [35]

    F. Eberle, F. Fischer, J. Matuschke, N. Megow, On index policies for stochastic minsum scheduling, Oper. Res. Lett., 47 (2019), 213–218. https://doi.org/10.1016/j.orl.2019.03.007

    doi: 10.1016/j.orl.2019.03.007
    [36]

    B. Wang, H. Wang, Q. Yan, E. Ma, Multi-agent reinforcement learning-aided evolutionary algorithm for a many-objective distributed hybrid flow shop scheduling problem, Swarm Evol. Comput., 97 (2025), 101991. https://doi.org/10.1016/j.swevo.2025.101991

    doi: 10.1016/j.swevo.2025.101991
    [37]

    Z. Pan, D. Lei, L. Wang, A knowledge-based two-population optimization algorithm for distributed energy-efficient parallel machines scheduling, IEEE Trans. Cybern., 52 (2022), 5051–5063. https://doi.org/10.1109/TCYB.2020.3026571

    doi: 10.1109/TCYB.2020.3026571
    [38]

    H. Wang, B. R. Sarker, J. Li, J. Li, Adaptive scheduling for assembly job shop with uncertain assembly times based on dual Q-learning, Int. J. Prod. Res., 59 (2021), 5867–5883. https://doi.org/10.1080/00207543.2020.1794075

    doi: 10.1080/00207543.2020.1794075
    [39]

    F. Zhao, Z. Fu, L. Wang, H. Sang, A heterogeneous graph reinforcement learning framework with question-aware neighborhood aggregation and interoption prompt attention for dynamic flexible job shop scheduling problem, IEEE Trans. Ind. Inform., 22 (2026), 2863–2874. https://doi.org/10.1109/TII.2025.3646962

    doi: 10.1109/TII.2025.3646962
    [40]

    S. Chen, X. Wang, Y. Wang, X. Gu, A knowledge-driven many-objective algorithm for energy-efficient distributed heterogeneous hybrid flowshop scheduling with lot-streaming, Swarm Evol. Comput., 91 (2024), 101771. https://doi.org/10.1016/j.swevo.2024.101771

    doi: 10.1016/j.swevo.2024.101771
  • Reader Comments
  • © 2026 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)

Metrics

Article views(124) PDF downloads(5) Cited by(0)

Article outline

Figures and Tables

Figures(13)  /  Tables(7)

Other Articles By Authors

/

DownLoad:  Full-Size Img  PowerPoint
Return
Return

Catalog