A subspace minimization Riemannian conjugate gradient method with nonmonotone Wolfe line search and a restart strategy

Shajie Xing; Huangyue Chen; Chunming Tang; Shajie Xing; Huangyue Chen; Chunming Tang

doi:10.3934/math.2026977

AIMS Mathematics

2026, Volume 11, Issue 8: 24194-24224. doi: 10.3934/math.2026977

Previous Article Next Article

Research article

A subspace minimization Riemannian conjugate gradient method with nonmonotone Wolfe line search and a restart strategy

1.
School of Physical Science and Technology, Guangxi University, Nanning 530004, China
2.
School of Mathematics & Center for Applied Mathematics of Guangxi, Guangxi University, Nanning 530004, China

Received: 25 May 2026 Revised: 16 July 2026 Accepted: 24 July 2026 Published: 07 August 2026
MSC : 65K05, 90C30

This paper proposes a subspace minimization Riemannian conjugate gradient (CG) method combined with nonmonotone Wolfe line search and a restart strategy. First, an improved nonmonotone version of the Riemannian Wolfe line search is proposed to address a numerical drawback of the Riemannian Wolfe line search. Second, the search direction is determined by minimizing a quadratic approximation model of the objective function in a two-dimensional subspace, with the CG parameters dynamically adjusted via four decision criteria based on the local curvature information and the inner product between the current gradient and the previous search direction. This direction satisfies the sufficient descent condition. In addition, to accelerate convergence, a quantity measuring the degree of local quadratic approximation of the objective function is introduced as a restart criterion, thereby establishing the complete framework of the proposed method. Under mild conditions, the method is proven to achieve global convergence and R-linear convergence. Finally, comparative experiments on seven test problems demonstrate that the proposed method outperforms several existing competitive approaches in terms of the number of iterations and CPU time.
- conjugate gradient method,
- Riemannian optimization,
- subspace technique,
- nonmonotone Wolfe line search,
- restart strategy
Citation: Shajie Xing, Huangyue Chen, Chunming Tang. A subspace minimization Riemannian conjugate gradient method with nonmonotone Wolfe line search and a restart strategy[J]. AIMS Mathematics, 2026, 11(8): 24194-24224. doi: 10.3934/math.2026977

Related Papers:

Abstract

This paper proposes a subspace minimization Riemannian conjugate gradient (CG) method combined with nonmonotone Wolfe line search and a restart strategy. First, an improved nonmonotone version of the Riemannian Wolfe line search is proposed to address a numerical drawback of the Riemannian Wolfe line search. Second, the search direction is determined by minimizing a quadratic approximation model of the objective function in a two-dimensional subspace, with the CG parameters dynamically adjusted via four decision criteria based on the local curvature information and the inner product between the current gradient and the previous search direction. This direction satisfies the sufficient descent condition. In addition, to accelerate convergence, a quantity measuring the degree of local quadratic approximation of the objective function is introduced as a restart criterion, thereby establishing the complete framework of the proposed method. Under mild conditions, the method is proven to achieve global convergence and R-linear convergence. Finally, comparative experiments on seven test problems demonstrate that the proposed method outperforms several existing competitive approaches in terms of the number of iterations and CPU time.

References

[1]	Y. Saad, Numerical methods for large eigenvalue problems, SIAM, 2011. https://doi.org/10.1137/1.9781611970739
[2]	Z. Wen, C. Yang, X. Liu, Y. Zhang, Trace-penalty minimization for large-scale eigenspace computation, J. Sci. Comput., 66 (2016), 1175–1203. https://doi.org/10.1007/s10915-015-0061-0 doi: 10.1007/s10915-015-0061-0
[3]	B. Vandereycken, Low-rank matrix completion by Riemannian optimization, SIAM J. Optim., 23 (2013), 1214–1236. https://doi.org/10.1137/110845768 doi: 10.1137/110845768
[4]	D. Kressner, M. Steinlechner, B. Vandereycken, Low-rank tensor completion by Riemannian optimization, BIT Numer. Math., 54 (2014), 447–468. https://doi.org/10.1007/s10543-013-0455-z doi: 10.1007/s10543-013-0455-z
[5]	P. A. Absil, K. A. Gallivan, Joint diagonalization on the oblique manifold for independent component analysis, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, 2014. https://doi.org/10.1109/ICASSP.2006.1661433
[6]	P. Comon, G. H. Golub, Tracking a few extreme singular values and vectors in signal processing, Proc. IEEE, 78 (1990), 1327–1343. https://doi.org/10.1109/5.58320 doi: 10.1109/5.58320
[7]	P. H. Schönemann, A generalized solution of the orthogonal procrustes problem, Psychometrika, 31 (1966), 1–10. https://doi.org/10.1007/BF02289451 doi: 10.1007/BF02289451
[8]	L. Eldén, H. Park, A procrustes problem on the Stiefel manifold, Numer. Math., 82 (1999), 599–619. https://doi.org/10.1007/s002110050432 doi: 10.1007/s002110050432
[9]	M. R. Hestenes, E. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Natl. Bur. Stand., 49 (1952), 409–436. https://doi.org/10.6028/JRES.049.044 doi: 10.6028/JRES.049.044
[10]	R. Fletcher, C. M. Reeves, Function minimization by conjugate gradients, Comput. J., 7 (1964), 149–154. https://doi.org/10.1093/comjnl/7.2.149 doi: 10.1093/comjnl/7.2.149
[11]	E. Polak, G. Ribiere, Note sur la convergence de méthodes de directions conjuguées, Rev. Fr. Inf. Rech. Opér. Sér. Rouge, 3 (1969), 35–43. https://doi.org/10.1051/m2an/196903R100351 doi: 10.1051/m2an/196903R100351
[12]	B. T. Polak, The conjugate gradient method in extremal problems, USSR Comput. Math. Math. Phys., 9 (1969), 94–112. https://doi.org/10.1016/0041-5553(69)90035-4 doi: 10.1016/0041-5553(69)90035-4
[13]	Y. H. Dai, Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim., 10 (1999), 177–182. https://doi.org/10.1137/S1052623497318992 doi: 10.1137/S1052623497318992
[14]	Y. H. Dai, Y. Yuan, An efficient hybrid conjugate gradient method for unconstrained optimization, Ann. Oper. Res., 103 (2001), 33–47. https://doi.org/10.1023/A:1012930416777 doi: 10.1023/A:1012930416777
[15]	L. Zhang, W. Zhou, D. H. Li, A descent modified Polak–Ribière–Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal., 26 (2006), 629–640. https://doi.org/10.1093/imanum/drl016 doi: 10.1093/imanum/drl016
[16]	Z. Wan, Z. Yang, Y. Wang, New spectral PRP conjugate gradient method for unconstrained optimization, Appl. Math. Lett., 24 (2011), 16–22. https://doi.org/10.1016/j.aml.2010.08.002 doi: 10.1016/j.aml.2010.08.002
[17]	Y. Xiao, H. Song, Z. Wang, A modified conjugate gradient algorithm with cyclic Barzilai–Borwein steplength for unconstrained optimization, J. Comput. Appl. Math., 236 (2012), 3101–3110. https://doi.org/10.1016/j.cam.2012.01.032 doi: 10.1016/j.cam.2012.01.032
[18]	J. Liu, Y. Feng, L. Zou, A spectral conjugate gradient method for solving large-scale unconstrained optimization, Comput. Math. Appl., 77 (2019), 731–739. https://doi.org/10.1016/j.camwa.2018.10.002 doi: 10.1016/j.camwa.2018.10.002
[19]	X. Jiang, H. Yang, J. Yin, W. Liao, A three-term conjugate gradient algorithm with restart procedure to solve image restoration problems, J. Comput. Appl. Math., 424 (2023), 115020. https://doi.org/10.1016/j.cam.2022.115020 doi: 10.1016/j.cam.2022.115020
[20]	Y. X. Yuan, J. Stoer, A subspace study on conjugate gradient algorithms, Z. Angew. Math. Mech., 75 (1995), 69–77. https://doi.org/10.1002/zamm.19950750118 doi: 10.1002/zamm.19950750118
[21]	N. Andrei, An accelerated subspace minimization three-term conjugate gradient algorithm for unconstrained optimization, Numer. Algorithms, 65 (2014), 859–874. https://doi.org/10.1007/s11075-013-9718-7 doi: 10.1007/s11075-013-9718-7
[22]	Y. Dai, C. Kou, A Barzilai-Borwein conjugate gradient method, Sci. China Math., 59 (2016), 1511–1524. https://doi.org/10.1007/s11425-016-0279-2 doi: 10.1007/s11425-016-0279-2
[23]	J. Barzilai, J. M. Borwein, Two-point step size gradient methods, IMA J. Numer. Anal., 8 (1988), 141–148. https://doi.org/10.1093/imanum/8.1.141 doi: 10.1093/imanum/8.1.141
[24]	M. Li, H. Liu, Z. Liu, A new subspace minimization conjugate gradient method with nonmonotone line search for unconstrained optimization, Numer. Algorithms, 79 (2018), 195–219. https://doi.org/10.1007/s11075-017-0434-6 doi: 10.1007/s11075-017-0434-6
[25]	H. Liu, Z. Liu, An efficient Barzilai–Borwein conjugate gradient method for unconstrained optimization, J. Optim. Theory Appl., 180 (2019), 879–906. https://doi.org/10.1007/s10957-018-1393-3 doi: 10.1007/s10957-018-1393-3
[26]	Z. Liu, Y. Ni, H. Liu, W. Sun, A new subspace minimization conjugate gradient method for unconstrained minimization, J. Optim. Theory Appl., 200 (2024), 820–851. https://doi.org/10.1007/s10957-023-02325-x doi: 10.1007/s10957-023-02325-x
[27]	A. Lichnewsky, Une methode de gradient conjugue sur des varietes application a certains problemes de valeurs propres non lineaires, Numer. Funct. Anal. Optim., 1 (1979), 515–560. https://doi.org/10.1080/01630567908816032 doi: 10.1080/01630567908816032
[28]	S. T. Smith, Optimization techniques on Riemannian manifolds, Fields Inst. Commun., 1994.
[29]	P. A. Absil, R. Mahony, R. Sepulchre, Optimization algorithms on matrix manifolds, Princeton University Press, 2009. https://doi.org/10.1515/9781400830244
[30]	H. Sato, Riemannian conjugate gradient methods: general framework and specific algorithms with convergence analyses, SIAM J. Optim., 32 (2022), 2690–2717. https://doi.org/10.1137/21M1464178 doi: 10.1137/21M1464178
[31]	H. Sato, T. Iwai, A new, globally convergent Riemannian conjugate gradient method, Optimization, 64 (2015), 1011–1031. https://doi.org/10.1080/02331934.2013.836650 doi: 10.1080/02331934.2013.836650
[32]	H. Sato, A Dai–Yuan-type Riemannian conjugate gradient method with the weak Wolfe conditions, Comput. Optim. Appl., 64 (2016), 101–118. https://doi.org/10.1007/s10589-015-9801-1 doi: 10.1007/s10589-015-9801-1
[33]	H. Sakai, H. Iiduka, Hybrid Riemannian conjugate gradient methods with global convergence properties, Comput. Optim. Appl., 77 (2020), 811–830. https://doi.org/10.1007/s10589-020-00224-9 doi: 10.1007/s10589-020-00224-9
[34]	H. Sakai, H. Iiduka, Sufficient descent Riemannian conjugate gradient methods, J. Optim. Theory Appl., 190 (2021), 130–150. https://doi.org/10.1007/s10957-021-01874-3 doi: 10.1007/s10957-021-01874-3
[35]	C. Tang, X. Rong, J. Jian, S. Xing, A hybrid Riemannian conjugate gradient method for nonconvex optimization problems, J. Appl. Math. Comput., 69 (2023), 823–852. https://doi.org/10.1007/s12190-022-01772-5 doi: 10.1007/s12190-022-01772-5
[36]	N. Salihu, P. Kumam, S. Salisu, K. Khammahawong, Some hybrid Riemannian conjugate gradient methods with restart strategy, Franklin Open, 10 (2025), 100240. https://doi.org/10.1016/j.fraope.2025.100240 doi: 10.1016/j.fraope.2025.100240
[37]	C. Tang, W. Tan, S. Xing, H. Zheng, A class of spectral conjugate gradient methods for Riemannian optimization, Numer. Algorithms, 94 (2023), 131–147. https://doi.org/10.1007/s11075-022-01495-5 doi: 10.1007/s11075-022-01495-5
[38]	C. Tang, W. Tan, Y. Zhang, Z. Liu, An accelerated spectral CG based algorithm for optimization techniques on Riemannian manifolds and its comparative evaluation, J. Comput. Appl. Math., 462 (2025), 116482. https://doi.org/10.1016/j.cam.2024.116482 doi: 10.1016/j.cam.2024.116482
[39]	S. Nasiru, P. Kumam, S. Salisu, L. Wang, T. Seangwattana, Spectral conjugate gradient for Riemannian optimization: application to the Gough–Stewart platform, Int. J. Dyn. Control, 13 (2025), 288. https://doi.org/10.1007/s40435-025-01788-2 doi: 10.1007/s40435-025-01788-2
[40]	Y. Wang, H. Shao, H. Sun, T. Wu, An adaptive restart Riemannian spectral Dai-Kou conjugate gradient method with quasi-Newton update, J. Appl. Math. Comput., 72 (2026), 128. https://doi.org/10.1007/s12190-026-02773-4 doi: 10.1007/s12190-026-02773-4
[41]	N. Salihu, P. Kumam, S. Salisu, K. Sitthithakerngkiet, On new spectral conjugate gradient methods for Riemannian optimization using retraction and scaled vector transport, Oper. Res. Forum, 7 (2026), 7. https://doi.org/10.1007/s43069-025-00594-y doi: 10.1007/s43069-025-00594-y
[42]	N. Salihu, P. Kumam, L. Wang, S. Salisu, Some new three-term conjugate gradient methods for Riemannian optimization with application to the Gough-Stewart platform, Math. Model. Numer. Simul. Appl., 5 (2025), 2. https://doi.org/10.53391/2791-8564.1001 doi: 10.53391/2791-8564.1001
[43]	Y. Wang, H. Shao, H. Sun, M. Jiang, The global convergence of a spectral three-term Riemannian conjugate gradient method with convex combination, Oper. Res. Lett., 65 (2026), 107406. https://doi.org/10.1016/j.orl.2026.107406 doi: 10.1016/j.orl.2026.107406
[44]	W. Ring, B. Wirth, Optimization methods on Riemannian manifolds and their application to shape space, SIAM J. Optim., 22 (2012), 596–627. https://doi.org/10.1137/11082885x doi: 10.1137/11082885x
[45]	H. Sato, Riemannian optimization and its applications, Springer, 2021. https://doi.org/10.1007/978-3-030-62391-3
[46]	N. Boumal, An introduction to optimization on smooth manifolds, Cambridge University Press, 2023. https://doi.org/10.1017/9781009166164
[47]	Y. H. Dai, C. X. Kou, A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search, SIAM J. Optim., 23 (2013), 296–320. https://doi.org/10.1137/100813026 doi: 10.1137/100813026
[48]	H. Zhang, W. W. Hager, A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim., 14 (2004), 1043–1056. https://doi.org/10.1137/S1052623403428208 doi: 10.1137/S1052623403428208
[49]	Y. Narushima, S. Nakayama, M. Takemura, H. Yabe, Memoryless quasi-Newton methods based on the spectral-scaling Broyden family for Riemannian optimization, J. Optim. Theory Appl., 197 (2023), 639–664. https://doi.org/10.1007/s10957-023-02183-7 doi: 10.1007/s10957-023-02183-7
[50]	W. Huang, K. A. Gallivan, P. A. Absil, A Broyden class of quasi-Newton methods for Riemannian optimization, SIAM J. Optim., 25 (2015), 1660–1685. https://doi.org/10.1137/140955483 doi: 10.1137/140955483
[51]	W. Huang, P. A. Absil, K. A. Gallivan, A Riemannian BFGS method without differentiated retraction for nonconvex optimization problems, SIAM J. Optim., 28 (2018), 470–495. https://doi.org/10.1137/17M1127582 doi: 10.1137/17M1127582
[52]	X. Li, X. Wang, M. K. Lal, A nonmonotone trust region method for unconstrained optimization problems on Riemannian manifolds, J. Optim. Theory Appl., 188 (2021), 547–570. https://doi.org/10.1007/s10957-020-01796-6 doi: 10.1007/s10957-020-01796-6
[53]	W. Huang, P. A. Absil, K. A. Gallivan, A Riemannian symmetric rank-one trust-region method, Math. Program., 150 (2015), 179–216. https://doi.org/10.1007/s10107-014-0765-1 doi: 10.1007/s10107-014-0765-1
[54]	W. Huang, Optimization algorithms on Riemannian manifolds with applications, Ph.D. Thesis, The Florida State University, 2013.
[55]	M. Moakher, A differential geometric approach to the geometric mean of symmetric positive-definite matrices, SIAM J. Matrix Anal. Appl., 26 (2005), 735–747. https://doi.org/10.1137/S0895479803436937 doi: 10.1137/S0895479803436937
[56]	S. Xing, H. Chen, C. Tang, A class of generalized Barzilai-Borwein methods for Riemannian optimization, Appl. Numer. Math., 224 (2026), 106–118. https://doi.org/10.1016/j.apnum.2026.01.019 doi: 10.1016/j.apnum.2026.01.019
[57]	R. Ferreira, J. Xavier, J. P. Costeira, V. Barroso, Newton method for Riemannian centroid computation in naturally reductive homogeneous spaces, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, 2006. https://doi.org/10.1109/ICASSP.2006.1660751
[58]	B. Iannazzo, M. Porcelli, The Riemannian Barzilai–Borwein method with nonmonotone line search and the matrix geometric mean computation, IMA J. Numer. Anal., 38 (2018), 495–517. https://doi.org/10.1093/imanum/drx015 doi: 10.1093/imanum/drx015
[59]	E. D. Dolan, J. J. Moré, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), 201–213. https://doi.org/10.1007/s101070100263 doi: 10.1007/s101070100263

Reader Comments

Your name:*

Email:*
© 2026 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)