The conjugate gradient (CG) method is an optimization technique known for its rapid convergence; it has blossomed into significant developments and applications. Numerous variations of CG methods have emerged to enhance computational efficiency and address real-world challenges. This work presents a new conjugate gradient method for solving nonlinear unconstrained optimization problems by introducing a new conjugate gradient parameter. To improve the convergence properties, we have proposed a new inexact line search technique that fits in with the suggested approach and can also be useful for other gradient descent methods. The existence of a steplength that meets the new line search conditions is established. The generated descent direction and the convergence properties of the suggested approach are studied under the new line search conditions, where the global convergence is proven under mild assumptions. The proposed approach is evaluated on various test functions, and a comparison with recent similar algorithms is carried out. Furthermore, the proposed algorithm is applied for restoring images with different noise levels.
Citation: Asma Maiza, Raouf Ziadi, Mohammed A. Saleh, Abdulgader Z. Almaymuni. An improved conjugate gradient algorithm by adapting a new line search technique[J]. Electronic Research Archive, 2025, 33(4): 2148-2171. doi: 10.3934/era.2025094
| [1] | C. W. Chukwu, Fatmawati . Modelling fractional-order dynamics of COVID-19 with environmental transmission and vaccination: A case study of Indonesia. AIMS Mathematics, 2022, 7(3): 4416-4438. doi: 10.3934/math.2022246 |
| [2] | Moh. Mashum Mujur Ihsanjaya, Nanang Susyanto . A mathematical model for policy of vaccinating recovered people in controlling the spread of COVID-19 outbreak. AIMS Mathematics, 2023, 8(6): 14508-14521. doi: 10.3934/math.2023741 |
| [3] | Salah Boulaaras, Ziad Ur Rehman, Farah Aini Abdullah, Rashid Jan, Mohamed Abdalla, Asif Jan . Coronavirus dynamics, infections and preventive interventions using fractional-calculus analysis. AIMS Mathematics, 2023, 8(4): 8680-8701. doi: 10.3934/math.2023436 |
| [4] | Huda Alsaud, Muhammad Owais Kulachi, Aqeel Ahmad, Mustafa Inc, Muhammad Taimoor . Investigation of SEIR model with vaccinated effects using sustainable fractional approach for low immune individuals. AIMS Mathematics, 2024, 9(4): 10208-10234. doi: 10.3934/math.2024499 |
| [5] | Mdi Begum Jeelani, Abeer S Alnahdi, Rahim Ud Din, Hussam Alrabaiah, Azeem Sultana . Mathematical model to investigate transmission dynamics of COVID-19 with vaccinated class. AIMS Mathematics, 2023, 8(12): 29932-29955. doi: 10.3934/math.20231531 |
| [6] | Xiaoying Pan, Longkun Tang . A new model for COVID-19 in the post-pandemic era. AIMS Mathematics, 2024, 9(8): 21255-21272. doi: 10.3934/math.20241032 |
| [7] | Qinyue Zheng, Xinwei Wang, Qiuwei Pan, Lei Wang . Optimal strategy for a dose-escalation vaccination against COVID-19 in refugee camps. AIMS Mathematics, 2022, 7(5): 9288-9310. doi: 10.3934/math.2022515 |
| [8] | Peter Joseph Witbooi, Sibaliwe Maku Vyambwera, Mozart Umba Nsuami . Control and elimination in an SEIR model for the disease dynamics of COVID-19 with vaccination. AIMS Mathematics, 2023, 8(4): 8144-8161. doi: 10.3934/math.2023411 |
| [9] | Maria M. Martignoni, Proton Rahman, Amy Hurford . Rotational worker vaccination provides indirect protection to vulnerable groups in regions with low COVID-19 prevalence. AIMS Mathematics, 2022, 7(3): 3988-4003. doi: 10.3934/math.2022220 |
| [10] | Khalaf M. Alanazi . The asymptotic spreading speeds of COVID-19 with the effect of delay and quarantine. AIMS Mathematics, 2024, 9(7): 19397-19413. doi: 10.3934/math.2024945 |
The conjugate gradient (CG) method is an optimization technique known for its rapid convergence; it has blossomed into significant developments and applications. Numerous variations of CG methods have emerged to enhance computational efficiency and address real-world challenges. This work presents a new conjugate gradient method for solving nonlinear unconstrained optimization problems by introducing a new conjugate gradient parameter. To improve the convergence properties, we have proposed a new inexact line search technique that fits in with the suggested approach and can also be useful for other gradient descent methods. The existence of a steplength that meets the new line search conditions is established. The generated descent direction and the convergence properties of the suggested approach are studied under the new line search conditions, where the global convergence is proven under mild assumptions. The proposed approach is evaluated on various test functions, and a comparison with recent similar algorithms is carried out. Furthermore, the proposed algorithm is applied for restoring images with different noise levels.
| [1] |
R. Ziadi, A. Bencherif-Madani, A perturbed quasi-Newton algorithm for bound-constrained global optimization, J. Comput. Math., 43 (2025), 143–173. https://doi.org/10.4208/jcm.2307-m2023-0016 doi: 10.4208/jcm.2307-m2023-0016
|
| [2] |
R. Ziadi, A. Bencherif-Madani, A mixed algorithm for smooth global optimization, J. Math. Model., 11 (2023), 207–228. https://doi.org/10.22124/JMM.2022.23133.2061 doi: 10.22124/JMM.2022.23133.2061
|
| [3] |
O. O. O. Yousif, R. Ziadi, M. A. Saleh, A. Z. Almaymuni, Another updated parameter for the Hestenes-Stiefel conjugate gradient method, Int. J. Anal. Appl., 23 (2025), 10. https://doi.org/10.28924/2291-8639-23-2025-10 doi: 10.28924/2291-8639-23-2025-10
|
| [4] |
C. Souli, R. Ziadi, A. Bencherif-Madani, H. M. Khudhur, A hybrid CG algorithm for nonlinear unconstrained optimization with application in image restoration, J. Math. Model., 12 (2024), 301–317. https://doi.org/10.22124/JMM.2024.26151.2317 doi: 10.22124/JMM.2024.26151.2317
|
| [5] |
Y. Chen, H. Huang, K. Huang, M. Roohi, C. Tang, A selective chaos-driven encryption technique for protecting medical images, Phys. Scr., 100 (2025), 0152a3. https://doi.org/10.1088/1402-4896/ad9fad doi: 10.1088/1402-4896/ad9fad
|
| [6] |
K. Xiong, S. Wang, The online random Fourier features conjugate gradient algorithm, IEEE Signal Process. Lett., 26 (2019), 740–744. https://doi.org/10.1109/LSP.2019.2907480 doi: 10.1109/LSP.2019.2907480
|
| [7] | M. A. Saleh, Enhancing deep learning optimizers for detecting malware using line search method under strong Wolfe conditions, in 2023 3rd International Conference on Computing and Information Technology (ICCIT), (2023), 222–226. https://doi.org/10.1109/ICCIT58132.2023.10273908 |
| [8] |
M. Zhang, X. Wang, X. Chen, A. Zhang, The kernel conjugate gradient algorithms, IEEE Trans. Signal Process., 66 (2018), 4377–4387. https://doi.org/10.1109/TSP.2018.2853109 doi: 10.1109/TSP.2018.2853109
|
| [9] |
K. Xiong, H. H. Lu, S. Wang, Kernel correntropy conjugate gradient algorithms based on half-quadratic optimization, IEEE Trans. Cyber., 55 (2020), 5497–5510. https://doi.org/10.1109/TCYB.2019.2959834 doi: 10.1109/TCYB.2019.2959834
|
| [10] |
R. Ziadi, R. Ellaia, A. Bencherif-Madani, Global optimization through a stochastic perturbation of the PolakC Ribière conjugate gradient method, J. Comput. Appl. Math., 317 (2017), 672–684. https://doi.org/10.1016/j.cam.2016.12.021 doi: 10.1016/j.cam.2016.12.021
|
| [11] |
A. Mehamdia, Y. Chaib, T. Bechouat, Two modified conjugate gradient methods for unconstrained optimization and applications, RAIRO Oper. Res., 57 (2023), 333–353. https://doi.org/10.1051/ro/2023010 doi: 10.1051/ro/2023010
|
| [12] | M. R. Hestenes, E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Natl. Bur. Stand., 49 (1952), 409–436. |
| [13] | R. Fletcher, Practical Methods of Optimization, Wiley, New York, 1997. |
| [14] |
B. T. Polyak, The conjugate gradient method in extreme problems, USSR Comput. Math. Math. Phys., 9 (1969), 94–112. https://doi.org/10.1016/0041-5553(69)90035-4 doi: 10.1016/0041-5553(69)90035-4
|
| [15] | E. Polak, G. Ribiere, Note sur la convergence de méthodes de directions conjuguées, Rev. Française Inf. Rech. Opr., 16 (1969), 35–43. |
| [16] |
Y. Liu, C. Storey, Efficient generalized conjugate gradient algorithms. Part 1: Theory, J. Optim. Theory Appl., 69 (1992), 129–137. https://doi.org/10.1007/BF00940464 doi: 10.1007/BF00940464
|
| [17] |
Y. H. Dai, Y. Yuan, An efficient hybrid conjugate gradient method for unconstrained optimization, Ann. Oper. Res., 103 (2001), 33–47. https://doi.org/10.1023/A:1012930416777 doi: 10.1023/A:1012930416777
|
| [18] |
D. Ahmed, C. Storey, Efficient hybrid conjugate gradient techniques, J. Optim. Theory Appl., 64 (1990), 379–397. https://doi.org/10.1007/BF00939455 doi: 10.1007/BF00939455
|
| [19] |
J. K. Liu, S. J. Li, New hybrid conjugate gradient method for unconstrained optimization, Appl. Math. Comput., 245 (2014), 36–43. https://doi.org/10.1016/j.amc.2014.07.096 doi: 10.1016/j.amc.2014.07.096
|
| [20] |
N. Andrei, Another nonlinear conjugate gradient algorithm for unconstrained optimization, Optim. Meth. Soft., 24 (2008), 89–104. https://doi.org/10.1080/10556780802393326 doi: 10.1080/10556780802393326
|
| [21] |
X. Y. Zheng, X. L. Dong, J. R. Shi, W. Yang, Further comment on another hybrid conjugate gradient algorithm for unconstrained optimization by Andrei, Numer. Algorithms, 84 (2019), 603–608. https://doi.org/10.1007/s11075-019-00771-1 doi: 10.1007/s11075-019-00771-1
|
| [22] |
S. S. Djordjevic New hybrid conjugate gradient method as a convex combination of LS and CD methods, Filomat, 31 (2017), 1813–1825. https://doi.org/10.1007/s10473-019-0117-6 doi: 10.1007/s10473-019-0117-6
|
| [23] |
C. Souli, R. Ziadi, I. E. Lakhdari, A. Leulmi, An efficient hybrid conjugate gradient method for unconstrained optimization and image restoration problems, Iran. J. Numer. Anal. Optim., 15 (2024), 99–123. https://doi.org/10.1109/LSP.2019.290748010.22067/ijnao.2024.88087.1449 doi: 10.1109/LSP.2019.290748010.22067/ijnao.2024.88087.1449
|
| [24] |
M. Rivaie, M. Mamat, L. W. June, I. Mohd, A new class of nonlinear conjugate gradient coefficients with global convergence properties, Appl. Math. Comput., 218 (2012), 11323–11332. https://doi.org/10.1016/j.amc.2012.05.030 doi: 10.1016/j.amc.2012.05.030
|
| [25] |
Z. X. Wei, S. W. Yao, L. Y. Liu, The convergence properties of some new conjugate gradient methods, Appl. Math. Comput., 183 (2006), 1341–1350. https://doi.org/10.1016/j.amc.2006.05.150 doi: 10.1016/j.amc.2006.05.150
|
| [26] |
L. Zhang, An improved Wei-Yao-Liu nonlinear conjugate gradient method for optimization computation, J. Appl. Math. Comput., 215 (2009), 2269–2274. https://doi.org/10.1016/j.amc.2009.08.016 doi: 10.1016/j.amc.2009.08.016
|
| [27] |
I. M. Sulaiman, N. A. Bakar, M. Mamat, B. A. Hassan, M. Malik, M. A. Ahmed, A new hybrid conjugate gradient algorithm for optimization models and its application to regression analysis, J. Electr. Eng. Comput. Sci., 23 (2021), 1100–1109. https://doi.org/10.11591/ijeecs.v23.i2.pp1100-1109 doi: 10.11591/ijeecs.v23.i2.pp1100-1109
|
| [28] |
S. Yao, Z. Wei, H. Huang, A note about WYLs conjugate gradient method and its applications, Appl. Math. Comput., 191 (2007), 381–388. https://doi.org/10.1016/j.amc.2007.02.094 doi: 10.1016/j.amc.2007.02.094
|
| [29] |
I. Bongartz, A. R. Conn, N. Gould, P. L. Toint, CUTEr: A constrained and unconstrained testing environment, ACM Trans. Math. Softw. 29 (2003), 373–394. https://doi.org/10.1145/962437.96243 doi: 10.1145/962437.96243
|
| [30] | N. Andrei, An unconstrained optimization test functions, Adv. Mode. Optim., 10 (2008), 147–161. |
| [31] |
M. Hamoda, M. Mamat, M. Rivaie, Z. Salleh, A conjugate gradient method with strong Wolfe-Powell line search for unconstrained optimization, Appl. Math. Sci., 10 (2016), 13–16. http://dx.doi.org/10.12988/ams.2016.56449 doi: 10.12988/ams.2016.56449
|
| [32] |
I. E. Livieris, P. Pintelas, Globally convergent modified Perrys conjugate gradient method, Appl. Math. Comput., 218 (2012), 9197–9207. https://doi.org/10.1016/j.amc.2012.02.076 doi: 10.1016/j.amc.2012.02.076
|
| [33] |
W. W. Hager, H. Zhang, A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim., 16 (2005), 170–192. https://doi.org/10.1137/030601880 doi: 10.1137/030601880
|
| [34] |
E. D. Dolan, J. J. Mor, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), 201–213. https://doi.org/10.1007/s101070100263 doi: 10.1007/s101070100263
|
| 1. | Xin Liu, Kerim Fouli, Rui Kang, Martin Maier, Network-Coding-Based Energy Management for Next-Generation Passive Optical Networks, 2012, 30, 0733-8724, 864, 10.1109/JLT.2011.2182633 | |
| 2. | Konstantin Miller, Thorsten Biermann, Hagen Woesner, Holger Karl, 2010, Network Coding in Passive Optical Networks, 978-1-4244-7189-8, 1, 10.1109/NETCOD.2010.5487683 | |
| 3. | Z. Dulinski, M. Kantor, W. Krzysztofek, R. Stankiewicz, P. Cholda, 2010, Optimal Choice of Peers Based on BGP Information, 978-1-4244-6402-9, 1, 10.1109/ICC.2010.5502269 | |
| 4. | Zbigniew Dulinski, Rafal Stankiewicz, Piotr Wydrych, Miroslaw Kantor, Piotr Cholda, 2011, Cost-Driven Peer Rating Algorithm, 978-1-61284-232-5, 1, 10.1109/icc.2011.5962879 | |
| 5. | Dirk Haage, Ralph Holz, Heiko Niedermayer, Pavel Laskov, 2009, Chapter 24, 978-3-540-92665-8, 279, 10.1007/978-3-540-92666-5_24 | |
| 6. | Rafal Stankiewicz, Piotr Cholda, Jerzy Domzal, Robert Wojcik, Tobias Hosfeld, Thomas Zinner, Simon Oechsner, Frank Lehrieder, 2011, Influence of traffic management solutions on Quality of Experience for prevailing overlay applications, 978-1-4577-0915-9, 1, 10.1109/NGI.2011.5985866 | |
| 7. | Piotr Cholda, Jerzy Domzal, Robert Wojcik, Ratal Stankiewicz, Frank Lehrieder, Tobias HoBfeld, Simon Oechsner, Vlad Singeorzan, 2010, Performance evaluation of P2P caches: Flash-crowd case, 978-1-4244-8173-6, 108, 10.1109/ATNAC.2010.5680247 | |
| 8. | Matthias Wichtlhuber, Robert Reinecke, David Hausheer, An SDN-Based CDN/ISP Collaboration Architecture for Managing High-Volume Flows, 2015, 12, 1932-4537, 48, 10.1109/TNSM.2015.2404792 |
Figures(10) / Tables(4)
Asma Maiza, Raouf Ziadi, Mohammed A. Saleh, Abdulgader Z. Almaymuni. An improved conjugate gradient algorithm by adapting a new line search technique[J]. Electronic Research Archive, 2025, 33(4): 2148-2171. doi: 10.3934/era.2025094
DownLoad: