Research article Special Issues

A structured family of Dai-Liao conjugate gradient methods with an adaptive parameter for nonlinear least squares

  • Published: 16 November 2026
  • MSC : 65K05, 90C52, 90C53, 90C56

  • Nonlinear least-squares (NLS) problems arise in many scientific and engineering applications, motivating the development of efficient and reliable iterative methods, particularly for large-scale problems. Conjugate gradient (CG) methods are attractive because of their low memory requirements, but their performance depends strongly on the choice of conjugacy parameters and the use of curvature information. In this paper, we have proposed a structured family of the Dai-Liao conjugate gradient (SFDL) method for solving NLS problems. The proposed method combines a structured secant relation based on higher-order information from the residual functions with an adaptively selected Dai-Liao parameter derived from a spectral approximation of the inverse Hessian. A safeguard mechanism ensures the required curvature condition and the sufficient descent property of the search direction. Under standard assumptions, the global convergence of the proposed method was established. Numerical experiments on classical and large-scale NLS test problems showed that the SFDL method is competitive with existing structured CG methods in terms of iteration counts and computational time. Furthermore, experiments on three classical Moré-Garbow-Hillstrom data-fitting problems demonstrated its robustness and accuracy on both well-conditioned and severely ill-conditioned problems.

    Citation: Bashir Danladi Garba, Mansur Hassan, Nooraini Zainuddin, Hadil Alhazmi, Abdulhadi Aminu, Nazreen Waeleh, Rabiu Bashir Yunus. A structured family of Dai-Liao conjugate gradient methods with an adaptive parameter for nonlinear least squares[J]. AIMS Mathematics, 2026, 11(11): 38083-38108. doi: 10.3934/math.20261471

    Related Papers:

  • Nonlinear least-squares (NLS) problems arise in many scientific and engineering applications, motivating the development of efficient and reliable iterative methods, particularly for large-scale problems. Conjugate gradient (CG) methods are attractive because of their low memory requirements, but their performance depends strongly on the choice of conjugacy parameters and the use of curvature information. In this paper, we have proposed a structured family of the Dai-Liao conjugate gradient (SFDL) method for solving NLS problems. The proposed method combines a structured secant relation based on higher-order information from the residual functions with an adaptively selected Dai-Liao parameter derived from a spectral approximation of the inverse Hessian. A safeguard mechanism ensures the required curvature condition and the sufficient descent property of the search direction. Under standard assumptions, the global convergence of the proposed method was established. Numerical experiments on classical and large-scale NLS test problems showed that the SFDL method is competitive with existing structured CG methods in terms of iteration counts and computational time. Furthermore, experiments on three classical Moré-Garbow-Hillstrom data-fitting problems demonstrated its robustness and accuracy on both well-conditioned and severely ill-conditioned problems.



    加载中


    [1]

    M. M. Yahaya, P. Kumam, A. M. Awwal, P. Chaipunya, S. Aji, S. Salisu, A new generalized quasi-Newton algorithm based on structured diagonal Hessian approximation for solving nonlinear least-squares problems with application to 3DOF planar robot arm manipulator, IEEE Access, 10 (2022), 10816–10826. https://doi.org/10.1109/ACCESS.2022.3144875

    doi: 10.1109/ACCESS.2022.3144875
    [2]

    L. Meng, Y. Hao, Q. Yuan, Y. Zhan, D. Zhang, Safe human-centric shared control for consumer service robots in smart home environments via observer-based ADP, IEEE Trans. Consum. Electron., 2026. https://doi.org/10.1109/TCE.2026.3718874

    [3]

    S. Henn, A Levenberg-Marquardt scheme for nonlinear image registration, BIT Numer. Math., 43 (2003), 743–759. https://doi.org/10.1023/B:BITN.0000009940.58397.98

    doi: 10.1023/B:BITN.0000009940.58397.98
    [4]

    G. Rojas-Dueñas, J. R. Riba, M. Moreno-Eguilaz, Nonlinear least squares optimization for parametric identification of DC-DC converters, IEEE Trans. Power Electron., 36 (2021), 654–661. https://doi.org/10.1109/TPEL.2020.3003075

    doi: 10.1109/TPEL.2020.3003075
    [5]

    M. M. Yahaya, P. Kumam, P. Chaipunya, A. M. Awwal, L. Wang, On diagonally structured scheme for nonlinear least squares and data-fitting problems, RAIRO Oper. Res., 58 (2024), 2887–2905. https://doi.org/10.1051/ro/2024102

    doi: 10.1051/ro/2024102
    [6]

    R. M. Passi, Use of nonlinear least squares in meteorological applications, J. Appl. Meteorol. Climatol., 16 (1977), 827–832.

    [7]

    D. Zhang, L. Meng, L. Liang, C. Qin, D. Liu, Dynamic event-triggered control for human-machine cooperative systems based on dynamic authority allocation, IEEE Trans. Syst. Man Cybern. Syst., 56 (2026), 3733–3744. https://doi.org/10.1109/TSMC.2026.3661716

    doi: 10.1109/TSMC.2026.3661716
    [8]

    D. Zhang, Y. Hao, Q. Yuan, C. Qin, Dynamic event-triggered approximate optimal consensus control for unknown nonlinear multi-agent systems via adaptive dynamic programming, ISA Trans., 172 (2026), 21–32. https://doi.org/10.1016/j.isatra.2026.03.010

    doi: 10.1016/j.isatra.2026.03.010
    [9]

    H. Wu, C. Fan, D. Zhang, An innovative neural network architecture designed for industrial fault diagnosis with hierarchical adaptive attention mechanism, Process Saf. Environ. Prot., 205 (2026), 108162. https://doi.org/10.1016/j.psep.2025.108162

    doi: 10.1016/j.psep.2025.108162
    [10]

    L. Liang, P. Xiang, H. Huang, N. Zhang, P. Qi, Z. Yin, et al., Sec-GNN-driven joint multidimensional anti-eavesdropping optimization for secure UAV-satellite communications, IEEE Internet Things J., 13 (2026), 28694–28706. https://doi.org/10.1109/JIOT.2026.3685201

    doi: 10.1109/JIOT.2026.3685201
    [11]

    D. Zhang, Q. Yuan, L. Meng, R. Xia, W. Liu, C. Qin, Reinforcement learning for single-agent to multi-agent systems: from basic theory to industrial application progress, a survey, Artif. Intell. Rev., 59 (2026), 46. https://doi.org/10.1007/s10462-025-11439-9

    doi: 10.1007/s10462-025-11439-9
    [12]

    D. Zhang, C. Fan, Z. Li, C. Qin, J. Dan, Scale-spectrum-state collaborative fusion network for real-time extreme small object detection in intelligent sports analytics, Expert Syst. Appl., 333 (2026), 133831. https://doi.org/10.1016/j.eswa.2026.133831

    doi: 10.1016/j.eswa.2026.133831
    [13]

    J. Nocedal, S. J. Wright, Numerical optimization, New York: Springer, 2006. https://doi.org/10.1007/978-0-387-40065-5

    [14]

    H. O. Hartley, The modified Gauss-Newton method for the fitting of non-linear regression functions by least squares, Technometrics, 3 (1961), 269–280. https://doi.org/10.1080/00401706.1961.10489945

    doi: 10.1080/00401706.1961.10489945
    [15]

    D. W. Marquardt, An algorithm for least-squares estimation of nonlinear parameters, J. Soc. Ind. Appl. Math., 11 (1963), 431–441. https://doi.org/10.1137/0111030

    doi: 10.1137/0111030
    [16]

    J. E. Dennis Jr., D. M. Gay, R. E. Welsch, An adaptive nonlinear least-squares algorithm, ACM Trans. Math. Softw., 7 (1981), 348–368. https://doi.org/10.1145/355958.355965

    doi: 10.1145/355958.355965
    [17]

    J. Z. Zhang, Y. Xue, K. Zhang, A structured secant method based on a new quasi-Newton equation for nonlinear least squares problems, BIT Numer. Math., 43 (2003), 217–229. https://doi.org/10.1023/A:1023665409152

    doi: 10.1023/A:1023665409152
    [18]

    M. Nosrati, K. Amini, A new structured spectral conjugate gradient method for nonlinear least squares problems, Numer. Algorithms, 97 (2024), 897–914. https://doi.org/10.1007/s11075-023-01729-0

    doi: 10.1007/s11075-023-01729-0
    [19]

    M. Kobayashi, Y. Narushima, H. Yabe, Nonlinear conjugate gradient methods with structured secant condition for nonlinear least squares problems, J. Comput. Appl. Math., 234 (2010), 375–397. https://doi.org/10.1016/j.cam.2009.12.031

    doi: 10.1016/j.cam.2009.12.031
    [20]

    R. Dehghani, N. Mahdavi-Amiri, Scaled nonlinear conjugate gradient methods for nonlinear least squares problems, Numer. Algorithms, 82 (2019), 1–20. https://doi.org/10.1007/s11075-018-0591-2

    doi: 10.1007/s11075-018-0591-2
    [21]

    R. B. Yunus, A. Ben Ghorbal, N. Zainuddin, S. M. Ibrahim, An accelerated diagonally structured CG algorithm for nonlinear least squares and inverse kinematics, Mathematics, 13 (2025), 2766. https://doi.org/10.3390/math13172766

    doi: 10.3390/math13172766
    [22]

    R. B. Yunus, A. R. El-Saeed, N. Zainuddin, H. Daud, A structured RMIL conjugate gradient-based strategy for nonlinear least squares with applications in image restoration problems, AIMS Math., 10 (2025), 14893–14916. https://doi.org/10.3934/math.2025668

    doi: 10.3934/math.2025668
    [23]

    R. B. Yunus, N. Zainuddin, H. Daud, R. Kannan, M. M. Yahaya, A structured Liu-Storey-type conjugate gradient approach for nonlinear least-squares with application to a 3DOF robot model, J. Adv. Res. Appl. Sci. Eng. Technol., 64 (2025), 187–200. https://doi.org/10.37934/araset.64.187200

    doi: 10.37934/araset.64.187200
    [24]

    R. B. Yunus, N. Zainuddin, H. Daud, R. Kannan, S. A. Abdul Karim, M. M. Yahaya, A modified structured spectral HS method for nonlinear least squares problems and applications in robot arm control, Mathematics, 11 (2023), 3215. https://doi.org/10.3390/math11143215

    doi: 10.3390/math11143215
    [25]

    S. Babaie-Kafaki, A survey on the Dai-Liao family of nonlinear conjugate gradient methods, RAIRO Oper. Res., 57 (2023), 43–58. https://doi.org/10.1051/ro/2022213

    doi: 10.1051/ro/2022213
    [26]

    Y. H. Dai, L. Z. Liao, New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim., 43 (2001), 87–101. https://doi.org/10.1007/s002450010019

    doi: 10.1007/s002450010019
    [27]

    J. Barzilai, J. M. Borwein, Two-point step size gradient methods, IMA J. Numer. Anal., 8 (1988), 141–148. https://doi.org/10.1093/imanum/8.1.141

    doi: 10.1093/imanum/8.1.141
    [28]

    Y. H. Dai, C. X. Kou, A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search, SIAM J. Optim., 23 (2013), 296–320. https://doi.org/10.1137/100813026

    doi: 10.1137/100813026
    [29]

    J. J. Moré, B. S. Garbow, K. E. Hillstrom, Testing unconstrained optimization software, ACM Trans. Math. Softw., 7 (1981), 17–41. https://doi.org/10.1145/355934.355936

    doi: 10.1145/355934.355936
    [30]

    N. I. M. Gould, D. Orban, P. L. Toint, CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization, Comput. Optim. Appl., 60 (2015), 545–557. https://doi.org/10.1007/s10589-014-9687-3

    doi: 10.1007/s10589-014-9687-3
    [31]

    W. Hock, K. Schittkowski, Test examples for nonlinear programming codes, Berlin, Heidelberg: Springer-Verlag, 1981. https://doi.org/10.1007/978-3-642-48320-2

    [32]

    M. Raydan, The Barzilai and Borwein gradient method for the large-scale unconstrained minimization problem, SIAM J. Optim., 7 (1997), 26–33. https://doi.org/10.1137/S1052623494266365

    doi: 10.1137/S1052623494266365
    [33]

    B. Roth, F. Freudenstein, Synthesis of path-generating mechanisms by numerical methods, J. Eng. Ind., 85 (1963), 298–304. https://doi.org/10.1115/1.3669870

    doi: 10.1115/1.3669870
    [34]

    C. G. Broyden, A class of methods for solving nonlinear simultaneous equations, Math. Comp., 19 (1965), 577–593. https://doi.org/10.1090/S0025-5718-1965-0198670-6

    doi: 10.1090/S0025-5718-1965-0198670-6
    [35]

    H. H. Rosenbrock, An automatic method for finding the greatest or least value of a function, Comput. J., 3 (1960), 175–184. https://doi.org/10.1093/comjnl/3.3.175

    doi: 10.1093/comjnl/3.3.175
    [36]

    D. M. Himmelblau, Applied nonlinear programming, New York: McGraw-Hill, 1972.

    [37]

    M. J. D. Powell, A Fortran subroutine for solving systems of nonlinear algebraic equations, In: Numerical methods for nonlinear algebraic equations, New York: Gordon and Breach, 1970.

    [38]

    Wood, J. Non-linear least squares methods for parameter estimation with applications in reactor physics, J. Inst. Nucl. Eng., 14, (1973), 106–110,113–121.

    [39]

    E. D. Dolan, J. J. Moré, Benchmarking optimization software with performance profiles, Math. Program., 91 (2002), 201–213. https://doi.org/10.1007/s101070100263

    doi: 10.1007/s101070100263
    [40]

    C. N. L. Eu, H. G. Harno, K. H. Lim, Nonlinear least squares problems using approximate greatest descent method, 2022 International Conference on Computer and Drone Applications, 2022,101–106. https://doi.org/10.1109/ICONDA56696.2022.10000382

  • Reader Comments
  • © 2026 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)

Metrics

Article views(33) PDF downloads(10) Cited by(0)

Article outline

Figures and Tables

Figures(8)  /  Tables(2)

/

DownLoad:  Full-Size Img  PowerPoint
Return
Return

Catalog