Nonlinear least-squares (NLS) problems arise in many scientific and engineering applications, motivating the development of efficient and reliable iterative methods, particularly for large-scale problems. Conjugate gradient (CG) methods are attractive because of their low memory requirements, but their performance depends strongly on the choice of conjugacy parameters and the use of curvature information. In this paper, we have proposed a structured family of the Dai-Liao conjugate gradient (SFDL) method for solving NLS problems. The proposed method combines a structured secant relation based on higher-order information from the residual functions with an adaptively selected Dai-Liao parameter derived from a spectral approximation of the inverse Hessian. A safeguard mechanism ensures the required curvature condition and the sufficient descent property of the search direction. Under standard assumptions, the global convergence of the proposed method was established. Numerical experiments on classical and large-scale NLS test problems showed that the SFDL method is competitive with existing structured CG methods in terms of iteration counts and computational time. Furthermore, experiments on three classical Moré-Garbow-Hillstrom data-fitting problems demonstrated its robustness and accuracy on both well-conditioned and severely ill-conditioned problems.
Citation: Bashir Danladi Garba, Mansur Hassan, Nooraini Zainuddin, Hadil Alhazmi, Abdulhadi Aminu, Nazreen Waeleh, Rabiu Bashir Yunus. A structured family of Dai-Liao conjugate gradient methods with an adaptive parameter for nonlinear least squares[J]. AIMS Mathematics, 2026, 11(11): 38083-38108. doi: 10.3934/math.20261471
Nonlinear least-squares (NLS) problems arise in many scientific and engineering applications, motivating the development of efficient and reliable iterative methods, particularly for large-scale problems. Conjugate gradient (CG) methods are attractive because of their low memory requirements, but their performance depends strongly on the choice of conjugacy parameters and the use of curvature information. In this paper, we have proposed a structured family of the Dai-Liao conjugate gradient (SFDL) method for solving NLS problems. The proposed method combines a structured secant relation based on higher-order information from the residual functions with an adaptively selected Dai-Liao parameter derived from a spectral approximation of the inverse Hessian. A safeguard mechanism ensures the required curvature condition and the sufficient descent property of the search direction. Under standard assumptions, the global convergence of the proposed method was established. Numerical experiments on classical and large-scale NLS test problems showed that the SFDL method is competitive with existing structured CG methods in terms of iteration counts and computational time. Furthermore, experiments on three classical Moré-Garbow-Hillstrom data-fitting problems demonstrated its robustness and accuracy on both well-conditioned and severely ill-conditioned problems.
| [1] |
M. M. Yahaya, P. Kumam, A. M. Awwal, P. Chaipunya, S. Aji, S. Salisu, A new generalized quasi-Newton algorithm based on structured diagonal Hessian approximation for solving nonlinear least-squares problems with application to 3DOF planar robot arm manipulator, IEEE Access, 10 (2022), 10816–10826. https://doi.org/10.1109/ACCESS.2022.3144875 doi: 10.1109/ACCESS.2022.3144875
|
| [2] |
L. Meng, Y. Hao, Q. Yuan, Y. Zhan, D. Zhang, Safe human-centric shared control for consumer service robots in smart home environments via observer-based ADP, IEEE Trans. Consum. Electron., 2026. https://doi.org/10.1109/TCE.2026.3718874 |
| [3] |
S. Henn, A Levenberg-Marquardt scheme for nonlinear image registration, BIT Numer. Math., 43 (2003), 743–759. https://doi.org/10.1023/B:BITN.0000009940.58397.98 doi: 10.1023/B:BITN.0000009940.58397.98
|
| [4] |
G. Rojas-Dueñas, J. R. Riba, M. Moreno-Eguilaz, Nonlinear least squares optimization for parametric identification of DC-DC converters, IEEE Trans. Power Electron., 36 (2021), 654–661. https://doi.org/10.1109/TPEL.2020.3003075 doi: 10.1109/TPEL.2020.3003075
|
| [5] |
M. M. Yahaya, P. Kumam, P. Chaipunya, A. M. Awwal, L. Wang, On diagonally structured scheme for nonlinear least squares and data-fitting problems, RAIRO Oper. Res., 58 (2024), 2887–2905. https://doi.org/10.1051/ro/2024102 doi: 10.1051/ro/2024102
|
| [6] |
R. M. Passi, Use of nonlinear least squares in meteorological applications, J. Appl. Meteorol. Climatol., 16 (1977), 827–832. |
| [7] |
D. Zhang, L. Meng, L. Liang, C. Qin, D. Liu, Dynamic event-triggered control for human-machine cooperative systems based on dynamic authority allocation, IEEE Trans. Syst. Man Cybern. Syst., 56 (2026), 3733–3744. https://doi.org/10.1109/TSMC.2026.3661716 doi: 10.1109/TSMC.2026.3661716
|
| [8] |
D. Zhang, Y. Hao, Q. Yuan, C. Qin, Dynamic event-triggered approximate optimal consensus control for unknown nonlinear multi-agent systems via adaptive dynamic programming, ISA Trans., 172 (2026), 21–32. https://doi.org/10.1016/j.isatra.2026.03.010 doi: 10.1016/j.isatra.2026.03.010
|
| [9] |
H. Wu, C. Fan, D. Zhang, An innovative neural network architecture designed for industrial fault diagnosis with hierarchical adaptive attention mechanism, Process Saf. Environ. Prot., 205 (2026), 108162. https://doi.org/10.1016/j.psep.2025.108162 doi: 10.1016/j.psep.2025.108162
|
| [10] |
L. Liang, P. Xiang, H. Huang, N. Zhang, P. Qi, Z. Yin, et al., Sec-GNN-driven joint multidimensional anti-eavesdropping optimization for secure UAV-satellite communications, IEEE Internet Things J., 13 (2026), 28694–28706. https://doi.org/10.1109/JIOT.2026.3685201 doi: 10.1109/JIOT.2026.3685201
|
| [11] |
D. Zhang, Q. Yuan, L. Meng, R. Xia, W. Liu, C. Qin, Reinforcement learning for single-agent to multi-agent systems: from basic theory to industrial application progress, a survey, Artif. Intell. Rev., 59 (2026), 46. https://doi.org/10.1007/s10462-025-11439-9 doi: 10.1007/s10462-025-11439-9
|
| [12] |
D. Zhang, C. Fan, Z. Li, C. Qin, J. Dan, Scale-spectrum-state collaborative fusion network for real-time extreme small object detection in intelligent sports analytics, Expert Syst. Appl., 333 (2026), 133831. https://doi.org/10.1016/j.eswa.2026.133831 doi: 10.1016/j.eswa.2026.133831
|
| [13] |
J. Nocedal, S. J. Wright, Numerical optimization, New York: Springer, 2006. https://doi.org/10.1007/978-0-387-40065-5 |
| [14] |
H. O. Hartley, The modified Gauss-Newton method for the fitting of non-linear regression functions by least squares, Technometrics, 3 (1961), 269–280. https://doi.org/10.1080/00401706.1961.10489945 doi: 10.1080/00401706.1961.10489945
|
| [15] |
D. W. Marquardt, An algorithm for least-squares estimation of nonlinear parameters, J. Soc. Ind. Appl. Math., 11 (1963), 431–441. https://doi.org/10.1137/0111030 doi: 10.1137/0111030
|
| [16] |
J. E. Dennis Jr., D. M. Gay, R. E. Welsch, An adaptive nonlinear least-squares algorithm, ACM Trans. Math. Softw., 7 (1981), 348–368. https://doi.org/10.1145/355958.355965 doi: 10.1145/355958.355965
|
| [17] |
J. Z. Zhang, Y. Xue, K. Zhang, A structured secant method based on a new quasi-Newton equation for nonlinear least squares problems, BIT Numer. Math., 43 (2003), 217–229. https://doi.org/10.1023/A:1023665409152 doi: 10.1023/A:1023665409152
|
| [18] |
M. Nosrati, K. Amini, A new structured spectral conjugate gradient method for nonlinear least squares problems, Numer. Algorithms, 97 (2024), 897–914. https://doi.org/10.1007/s11075-023-01729-0 doi: 10.1007/s11075-023-01729-0
|
| [19] |
M. Kobayashi, Y. Narushima, H. Yabe, Nonlinear conjugate gradient methods with structured secant condition for nonlinear least squares problems, J. Comput. Appl. Math., 234 (2010), 375–397. https://doi.org/10.1016/j.cam.2009.12.031 doi: 10.1016/j.cam.2009.12.031
|
| [20] |
R. Dehghani, N. Mahdavi-Amiri, Scaled nonlinear conjugate gradient methods for nonlinear least squares problems, Numer. Algorithms, 82 (2019), 1–20. https://doi.org/10.1007/s11075-018-0591-2 doi: 10.1007/s11075-018-0591-2
|
| [21] |
R. B. Yunus, A. Ben Ghorbal, N. Zainuddin, S. M. Ibrahim, An accelerated diagonally structured CG algorithm for nonlinear least squares and inverse kinematics, Mathematics, 13 (2025), 2766. https://doi.org/10.3390/math13172766 doi: 10.3390/math13172766
|
| [22] |
R. B. Yunus, A. R. El-Saeed, N. Zainuddin, H. Daud, A structured RMIL conjugate gradient-based strategy for nonlinear least squares with applications in image restoration problems, AIMS Math., 10 (2025), 14893–14916. https://doi.org/10.3934/math.2025668 doi: 10.3934/math.2025668
|
| [23] |
R. B. Yunus, N. Zainuddin, H. Daud, R. Kannan, M. M. Yahaya, A structured Liu-Storey-type conjugate gradient approach for nonlinear least-squares with application to a 3DOF robot model, J. Adv. Res. Appl. Sci. Eng. Technol., 64 (2025), 187–200. https://doi.org/10.37934/araset.64.187200 doi: 10.37934/araset.64.187200
|
| [24] |
R. B. Yunus, N. Zainuddin, H. Daud, R. Kannan, S. A. Abdul Karim, M. M. Yahaya, A modified structured spectral HS method for nonlinear least squares problems and applications in robot arm control, Mathematics, 11 (2023), 3215. https://doi.org/10.3390/math11143215 doi: 10.3390/math11143215
|
| [25] |
S. Babaie-Kafaki, A survey on the Dai-Liao family of nonlinear conjugate gradient methods, RAIRO Oper. Res., 57 (2023), 43–58. https://doi.org/10.1051/ro/2022213 doi: 10.1051/ro/2022213
|
| [26] |
Y. H. Dai, L. Z. Liao, New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim., 43 (2001), 87–101. https://doi.org/10.1007/s002450010019 doi: 10.1007/s002450010019
|
| [27] |
J. Barzilai, J. M. Borwein, Two-point step size gradient methods, IMA J. Numer. Anal., 8 (1988), 141–148. https://doi.org/10.1093/imanum/8.1.141 doi: 10.1093/imanum/8.1.141
|
| [28] |
Y. H. Dai, C. X. Kou, A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search, SIAM J. Optim., 23 (2013), 296–320. https://doi.org/10.1137/100813026 doi: 10.1137/100813026
|
| [29] |
J. J. Moré, B. S. Garbow, K. E. Hillstrom, Testing unconstrained optimization software, ACM Trans. Math. Softw., 7 (1981), 17–41. https://doi.org/10.1145/355934.355936 doi: 10.1145/355934.355936
|
| [30] |
N. I. M. Gould, D. Orban, P. L. Toint, CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization, Comput. Optim. Appl., 60 (2015), 545–557. https://doi.org/10.1007/s10589-014-9687-3 doi: 10.1007/s10589-014-9687-3
|
| [31] |
W. Hock, K. Schittkowski, Test examples for nonlinear programming codes, Berlin, Heidelberg: Springer-Verlag, 1981. https://doi.org/10.1007/978-3-642-48320-2 |
| [32] |
M. Raydan, The Barzilai and Borwein gradient method for the large-scale unconstrained minimization problem, SIAM J. Optim., 7 (1997), 26–33. https://doi.org/10.1137/S1052623494266365 doi: 10.1137/S1052623494266365
|
| [33] |
B. Roth, F. Freudenstein, Synthesis of path-generating mechanisms by numerical methods, J. Eng. Ind., 85 (1963), 298–304. https://doi.org/10.1115/1.3669870 doi: 10.1115/1.3669870
|
| [34] |
C. G. Broyden, A class of methods for solving nonlinear simultaneous equations, Math. Comp., 19 (1965), 577–593. https://doi.org/10.1090/S0025-5718-1965-0198670-6 doi: 10.1090/S0025-5718-1965-0198670-6
|
| [35] |
H. H. Rosenbrock, An automatic method for finding the greatest or least value of a function, Comput. J., 3 (1960), 175–184. https://doi.org/10.1093/comjnl/3.3.175 doi: 10.1093/comjnl/3.3.175
|
| [36] |
D. M. Himmelblau, Applied nonlinear programming, New York: McGraw-Hill, 1972. |
| [37] |
M. J. D. Powell, A Fortran subroutine for solving systems of nonlinear algebraic equations, In: Numerical methods for nonlinear algebraic equations, New York: Gordon and Breach, 1970. |
| [38] |
Wood, J. Non-linear least squares methods for parameter estimation with applications in reactor physics, J. Inst. Nucl. Eng., 14, (1973), 106–110,113–121. |
| [39] |
E. D. Dolan, J. J. Moré, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), 201–213. https://doi.org/10.1007/s101070100263 doi: 10.1007/s101070100263
|
| [40] |
C. N. L. Eu, H. G. Harno, K. H. Lim, Nonlinear least squares problems using approximate greatest descent method, 2022 International Conference on Computer and Drone Applications, 2022,101–106. https://doi.org/10.1109/ICONDA56696.2022.10000382 |
Performance profile based on the number of iterations
Performance profile based on feval
Performance profile based on the number of gradient evaluations
Performance profile based on CPU time
Performance profile based on the function value
A nonlinear Gaussian fitting problem in which the red curve denotes the Gaussian model with $ n = 3 $ unknown parameters fitted to $ m = 15 $ observed data points
A nonlinear fitting of the Osborne Ⅱ function, where the red curve represents the fitted Osborne model with $ n = 11 $ parameters (variables) and $ m = 65 $ observed data points
A nonlinear fitting of the Meyer function, where the red curve represents the fitted Meyer model with $ n = 3 $ parameters and $ m = 16 $ observed data points