Multi-objective optimization method for charging and discharging of electric vehicles via Q-learning-based grey wolf algorithm

Zhi Zhang; Taijun Guo; Yefeng Liu; Xinfu Pang; Yi Zhang; Zhi Zhang; Taijun Guo; Yefeng Liu; Xinfu Pang; Yi Zhang

doi:10.3934/electreng.2025021

AIMS Electronics and Electrical Engineering

2025, Volume 9, Issue 4: 448-475. doi: 10.3934/electreng.2025021

Previous Article Next Article

Research article Special Issues

Multi-objective optimization method for charging and discharging of electric vehicles via Q-learning-based grey wolf algorithm

1.
School of Mechanical Engineering and Automation, Shenyang Institute of Technology, Fushun 113122, China
2.
School of Automation and Electrical Engineering, Linyi University, Linyi 276005, China
3.
Key Laboratory of Energy Saving and Controlling in Power System of Liaoning Province, Shenyang Institute of Engineering, Shenyang 110136, China
4.
Liaoning Urban Construction Technical College, Shenyang 110122, China

Received: 09 April 2025 Revised: 03 August 2025 Accepted: 15 August 2025 Published: 01 September 2025

With the widespread adoption of electric vehicles (EVs), their large-scale integration into the power grid may lead to uncoordinated charging issues, resulting in a significant increase in peak power loads. This may adversely affect the safe, stable operation and economic efficiency of the power grid. To address these challenges, this paper proposed a multi-objective optimization method for EV charging and discharging via Q-learning-based grey wolf optimizer (GWO). The proposed method follows a systematic approach: First, the system architecture was established, defining the roles of EVs and flexible devices in the charging and discharging scheduling process. Second, a multi-objective optimization dispatch model was formulated, with the objectives of minimizing system operation cost, load mean square error, and load peak-to-valley difference. The model was subject to constraints related to power balance and the operational limits of various devices. Third, an advanced multi-objective grey wolf optimizer with integrated Q-learning was developed to address the dispatch model. This optimizer aimed to achieve the Pareto front. Parameters were optimized via orthogonal experiments. The optimizer's effectiveness was measured using metrics assessing distribution and convergence. To address the problem, a case study was used to assess the performance of the proposed Q-GWO algorithm compared to benchmark algorithms, analyzing optimization outcomes and algorithm strengths. Case studies demonstrated 17.3% cost reduction and 31.5% lower peak-valley difference versus benchmark algorithms. Q-GWO's convergence speed (validated via IGD/HV metrics) outperformed GWO by 15.2%, proving efficacy for real-world V2G dispatch. Additionally, the TOPSIS multi-criteria decision analysis approach was employed to identify an optimal solution.
- grey wolf algorithm,
- multi-objective optimization,
- Q-learning,
- time-of-use electricity price,
- vehicle-to-grid (V2G)
Citation: Zhi Zhang, Taijun Guo, Yefeng Liu, Xinfu Pang, Yi Zhang. Multi-objective optimization method for charging and discharging of electric vehicles via Q-learning-based grey wolf algorithm[J]. AIMS Electronics and Electrical Engineering, 2025, 9(4): 448-475. doi: 10.3934/electreng.2025021

Related Papers:

Abstract

With the widespread adoption of electric vehicles (EVs), their large-scale integration into the power grid may lead to uncoordinated charging issues, resulting in a significant increase in peak power loads. This may adversely affect the safe, stable operation and economic efficiency of the power grid. To address these challenges, this paper proposed a multi-objective optimization method for EV charging and discharging via Q-learning-based grey wolf optimizer (GWO). The proposed method follows a systematic approach: First, the system architecture was established, defining the roles of EVs and flexible devices in the charging and discharging scheduling process. Second, a multi-objective optimization dispatch model was formulated, with the objectives of minimizing system operation cost, load mean square error, and load peak-to-valley difference. The model was subject to constraints related to power balance and the operational limits of various devices. Third, an advanced multi-objective grey wolf optimizer with integrated Q-learning was developed to address the dispatch model. This optimizer aimed to achieve the Pareto front. Parameters were optimized via orthogonal experiments. The optimizer's effectiveness was measured using metrics assessing distribution and convergence. To address the problem, a case study was used to assess the performance of the proposed Q-GWO algorithm compared to benchmark algorithms, analyzing optimization outcomes and algorithm strengths. Case studies demonstrated 17.3% cost reduction and 31.5% lower peak-valley difference versus benchmark algorithms. Q-GWO's convergence speed (validated via IGD/HV metrics) outperformed GWO by 15.2%, proving efficacy for real-world V2G dispatch. Additionally, the TOPSIS multi-criteria decision analysis approach was employed to identify an optimal solution.

References

[1]	Shaheen HI, Rashed GI, Yang B, Yang J (2024) Optimal electric vehicle charging and discharging scheduling using metaheuristic algorithms: V2G approach for cost reduction and grid support. J Energy Storage 90: 111816. https://doi.org/10.1016/j.est.2024.111816 doi: 10.1016/j.est.2024.111816
[2]	Yin W, Jia L, Ji J (2024) Energy optimal scheduling strategy considering V2G characteristics of electric vehicle. Energy 294: 130967. https://doi.org/10.1016/j.energy.2024.130967 doi: 10.1016/j.energy.2024.130967
[3]	Huang Z, Guo Z, Ma P, Wang M, Long Y, Zhang M (2022) Economic-environmental scheduling of microgrid considering V2G-enabled electric vehicles integration. Sustainable Energy, Grids and Networks 32: 100872. https://doi.org/10.1016/j.segan.2022.100872 doi: 10.1016/j.segan.2022.100872
[4]	Jia R, Pan H, Zhang S, Hu Y (2024) Charging scheduling strategy for electric vehicles in residential areas based on offline reinforcement learning. J Energy Storage 103: 114319. https://doi.org/10.1016/j.est.2024.114319 doi: 10.1016/j.est.2024.114319
[5]	Mosammam ZM, Ahmadi P, Houshfar E (2024) Multi-objective optimization-driven machine learning for charging and V2G pattern for plug-in hybrid vehicles: Balancing battery aging and power management. J Power Sources 608: 234639. https://doi.org/10.1016/j.jpowsour.2024.234639 doi: 10.1016/j.jpowsour.2024.234639
[6]	Khezri R, Steen D, Wikner E, Tuan LA (2024) Optimal V2G scheduling of an EV with calendar and cycle aging of battery: An MILP approach. IEEE T Transp Electr 10(4): 10497‒10507. https://doi.org/10.1109/TTE.2024.3384293 doi: 10.1109/TTE.2024.3384293
[7]	Wang Y, Shen X, Xu Y (2023) Joint planning of active distribution network and EV charging stations considering vehicle-to-grid functionality and reactive power support. CSEE J Power Energy Syst 10(5): 2100‒2113. https://doi.org/10.17775/CSEEJPES.2023.03930 doi: 10.17775/CSEEJPES.2023.03930
[8]	Zhang S, JAMES JQ (2021) Electric vehicle dynamic wireless charging system: Optimal placement and vehicle-to-grid scheduling. IEEE Internet of Things Journal 9(8): 6047‒6057. https://doi.org/10.1109/JIOT.2021.3109956 doi: 10.1109/JIOT.2021.3109956
[9]	Saner CB, Trivedi A, Srinivasan D (2022) A cooperative hierarchical multi-agent system for EV charging scheduling in presence of multiple charging stations. IEEE T Smart Grid 13(3): 2218‒2233. https://doi.org/10.1109/TSG.2022.3140927 doi: 10.1109/TSG.2022.3140927
[10]	Yu H, Tu J, Lei X, Shao Z, Jian L (2024) A cost-effective and high-efficient EV shared fast charging scheme with hierarchical coordinated operation strategy for addressing difficult-to-charge issue in old residential communities. Sustain Cities Soc 101: 105090. https://doi.org/10.1016/j.scs.2023.105090 doi: 10.1016/j.scs.2023.105090
[11]	Kandpal B, Pareek P, Verma A (2022) A robust day-ahead scheduling strategy for EV charging stations in unbalanced distribution grid. Energy 249: 123737. https://doi.org/10.1016/j.energy.2022.123737 doi: 10.1016/j.energy.2022.123737
[12]	Chai YT, Che HS, Tan C, Tan WN, Yip SC, Gan MT (2023) A two-stage optimization method for Vehicle to Grid coordination considering building and Electric Vehicle user expectations. Int J Electr Power Energy Syst 148: 108984. https://doi.org/10.1016/j.ijepes.2023.108984 doi: 10.1016/j.ijepes.2023.108984
[13]	Yin W, Wen T, Zhang C (2023) Cooperative optimal scheduling strategy of electric vehicles based on dynamic electricity price mechanism. Energy 263: 125627. https://doi.org/10.1016/j.energy.2022.125627 doi: 10.1016/j.energy.2022.125627
[14]	Triviño-Cabrera A, Aguado JA, de la Torre S (2019) Joint routing and scheduling for electric vehicles in smart grids with V2G. Energy 175: 113‒122. https://doi.org/10.1016/j.energy.2019.02.184 doi: 10.1016/j.energy.2019.02.184
[15]	Wang Z, Zhang L, Tang W, Ma Z, Huang J (2024) Equilibrium configuration strategy of vehicle-to-grid-based electric vehicle charging stations in low-carbon resilient distribution networks. Applied Energy 361: 122931. https://doi.org/10.1016/j.apenergy.2024.122931 doi: 10.1016/j.apenergy.2024.122931
[16]	Hou H, Wang Y, Chen Y, Zhao B, Zhang L, Xie C (2022) Long-time scale vehicle-to-grid scheduling strategy considering psychological effect based on Weber-Fechner law. Int J Electr Power Energy Syst 136: 107709. https://doi.org/10.1016/j.ijepes.2021.107709 doi: 10.1016/j.ijepes.2021.107709
[17]	Yu Y, Zhong Q, Sun L, Han Y, Zhang Q, Jing X, et al. (2025) A Self-adaptive two stage iterative greedy algorithm based job scales for energy-efficient distributed permutation flowshop scheduling problem. Swarm Evol Comput 92: 101777. https://doi.org/10.1016/j.swevo.2024.101777 doi: 10.1016/j.swevo.2024.101777
[18]	Cheng L, Tang Q, Zhang L, Yu C (2022) Scheduling flexible manufacturing cell with no-idle flow-lines and job-shop via Q-learning-based genetic algorithm. Comput Ind Eng 169: 108293. https://doi.org/10.1016/j.cie.2022.108293 doi: 10.1016/j.cie.2022.108293
[19]	Zhao Y, Ding Y, Pei Y (2024) Adaptive Optimization in Evolutionary Reinforcement Learning Using Evolutionary Mutation Rates. IEEE Access 12: 165384‒165394. https://doi.org/10.1109/ACCESS.2024.3493198 doi: 10.1109/ACCESS.2024.3493198
[20]	Wang J, Lei D, Cai J (2022) An adaptive artificial bee colony with reinforcement learning for distributed three-stage assembly scheduling with maintenance. Applied Soft Computing 117: 108371. https://doi.org/10.1016/j.asoc.2021.108371 doi: 10.1016/j.asoc.2021.108371
[21]	Huang JP, Gao L, Li XY (2024) A hierarchical multi-action deep reinforcement learning method for dynamic distributed job-shop scheduling problem with job arrivals. IEEE T Autom Sci Eng 22: 2501‒2513. https://doi.org/10.1109/TASE.2024.3380644 doi: 10.1109/TASE.2024.3380644
[22]	Karimi-Mamaghan M, Mohammadi M, Pasdeloup B, Meyer P (2023) Learning to select operators in meta-heuristics: An integration of Q-learning into the iterated greedy algorithm for the permutation flowshop scheduling problem. Eur J Oper Res 304(3): 1296‒1330. https://doi.org/10.1016/j.ejor.2022.03.054 doi: 10.1016/j.ejor.2022.03.054
[23]	Ma Y, Lu Y, Yin Y, Lei Y (2024) Pricing strategy of V2G demand response for industrial and commercial enterprises based on cooperative game. Int J Electr Power Energy Syst 160: 110051. https://doi.org/10.1016/j.ijepes.2024.110051 doi: 10.1016/j.ijepes.2024.110051
[24]	Shariatzadeh M, Antunes CH, Lopes MA (2024) Charging scheduling in a workplace parking lot: Bi-objective optimization approaches through predictive analytics of electric vehicle users' charging behavior. Sustainable Energy, Grids and Networks 39: 101463. https://doi.org/10.1016/j.segan.2024.101463 doi: 10.1016/j.segan.2024.101463
[25]	Triviño A, López A, Yuste AJ, Cuevas JC (2024) Decentralized EV charging and discharging scheduling algorithm based on Type-Ⅱ fuzzy-logic controllers. J Energy Storage 93: 112054. https://doi.org/10.1016/j.est.2024.112054 doi: 10.1016/j.est.2024.112054

Reader Comments

Your name:*

Email:*
© 2025 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)