Runge-Kutta pairs for scalar autonomous Neural ODEs

Ibraheem Alolyan; Theodore E. Simos; Charalampos Tsitouras; Ibraheem Alolyan; Theodore E. Simos; Charalampos Tsitouras

doi:10.3934/math.2026117

AIMS Mathematics

2026, Volume 11, Issue 1: 2935-2953. doi: 10.3934/math.2026117

Previous Article Next Article

Research article

Runge-Kutta pairs for scalar autonomous Neural ODEs

1.
Mathematics Dept., College of Science, King Saud University, P.O. Box 2455, Riyadh, 11451, Saudi Arabia
2.
Center for Applied Mathematics and Bioinformatics, Gulf University for Science and Technology, West Mishref, 32093, Kuwait; ORCID: https://orcid.org/0000-0002-9220-6924
3.
Section of Mathematics, Dept. of Civil Engineering, Democritus Univ. of Thrace, Xanthi 67100, Greece
4.
General Department, National & Kapodistrian University of Athens, 34400 Euripus Campus, Greece; ORCID: https://orcid.org/0000-0001-6801-8117

Received: 28 September 2025 Revised: 17 January 2026 Accepted: 20 January 2026 Published: 29 January 2026
MSC : 65L05, 65L06, 05C05, 90C26, 68T07

Runge–Kutta (RK) pairs remain among the most effective tools for the numerical integration of ordinary differential equations; however in the scalar autonomous setting, their structure allows further efficiency. In this case, the system of order conditions simplifies, with fewer equations needing to be satisfied, which in turn enables the construction of embedded pairs of higher accuracy than in the general case. Notably, one may design pairs of order 7(5) requiring only eight stages per step, whereas conventional RK pairs with the same number of stages are limited to order 6(5). In the present work, we derived the modified set of equations of condition up to seventh order, and by means of differential evolution techniques, constructed a new embedded pair of order 7(5) specifically adapted to scalar autonomous problems. The performance of the method was assessed in the context of system identification through neural ODEs, where it was used to approximate governing dynamics from data. The logistic growth and saturating cubic models were employed as a representative test cases, illustrating both the efficiency advantages of the proposed scheme. Numerical experiments confirmed that the new pair provides a valuable bridge between high–order RK methodology and modern machine learning approaches to dynamical systems.
- order conditions,
- integer partitions,
- evolutionary optimization,
- neural ODEs,
- system identification
Citation: Ibraheem Alolyan, Theodore E. Simos, Charalampos Tsitouras. Runge-Kutta pairs for scalar autonomous Neural ODEs[J]. AIMS Mathematics, 2026, 11(1): 2935-2953. doi: 10.3934/math.2026117

Related Papers:

Abstract

Runge–Kutta (RK) pairs remain among the most effective tools for the numerical integration of ordinary differential equations; however in the scalar autonomous setting, their structure allows further efficiency. In this case, the system of order conditions simplifies, with fewer equations needing to be satisfied, which in turn enables the construction of embedded pairs of higher accuracy than in the general case. Notably, one may design pairs of order 7(5) requiring only eight stages per step, whereas conventional RK pairs with the same number of stages are limited to order 6(5). In the present work, we derived the modified set of equations of condition up to seventh order, and by means of differential evolution techniques, constructed a new embedded pair of order 7(5) specifically adapted to scalar autonomous problems. The performance of the method was assessed in the context of system identification through neural ODEs, where it was used to approximate governing dynamics from data. The logistic growth and saturating cubic models were employed as a representative test cases, illustrating both the efficiency advantages of the proposed scheme. Numerical experiments confirmed that the new pair provides a valuable bridge between high–order RK methodology and modern machine learning approaches to dynamical systems.

References

[1]	R. W. Brankin, I. Gladwell, J. R. Dormand, P. J. Prince, W. L. Seward, Algorithm 670: A Runge-Kutta-Nyström code, ACM T. Math. Software, 15 (1989), 31–40. https://doi.org/10.1145/62038.69650 doi: 10.1145/62038.69650
[2]	J. C. Butcher, Implicit Runge-Kutta processes, Math. Comput., 18 (1964), 50–64. https://doi.org/10.1090/S0025-5718-1964-0159424-9 doi: 10.1090/S0025-5718-1964-0159424-9
[3]	J. C. Butcher, On Runge-Kutta Processes of high order, J. Aust. Math. Soc., 4 (1964), 179–194. https://doi.org/10.1017/S1446788700023387 doi: 10.1017/S1446788700023387
[4]	J. C. Butcher, The numerical analysis of ordinary differential equations: Runge-Kutta and general linear methods, Wiley: Chichester, UK, 1987.
[5]	M. Caldana, J. S. Hesthaven, Neural ordinary differential equations for model order reduction for stiff systems, Int. J. Numer. Meth. Eng., 126 (2025), e70060. https://doi.org/10.1002/nme.70060 doi: 10.1002/nme.70060
[6]	T. Q. Chen, Y. Rubanova, J. Bettencourt, D. Duvenaud, Neural ordinary differential equations, Adv. Neural Inform. Process. Syst., 2018.
[7]	J. R. Dormand, P. J. Prince, A family of Embedded Runge-Kutta formulae, J. Comput. Appl. Math., 6 (1980), 19–26. https://doi.org/10.1016/0771-050X(80)90013-3 doi: 10.1016/0771-050X(80)90013-3
[8]	E. Dupont, A. Doucet, Y. W. Teh, Augmented neural ODEs, Adv. Neural Inform. Process. Syst., 2019.
[9]	W. H. Enright, J. D. Pryce, Two FORTRAN packages for assessing initial value methods, ACM T. Math. Software, 13 (1987), 1–27.
[10]	I. T. Famelis, C. Tsitouras, Symbolic derivation of order conditions for hybrid Numerov-type methods solving $y'' = f(x, y)$, J. Comput. Appl. Math., 218 (2008), 543–555. https://doi.org/10.1016/j.cam.2007.09.017 doi: 10.1016/j.cam.2007.09.017
[11]	E. Fehlberg, Low order classical Runge-Kutta formulas with step-size control and their application to some heat transfer problems, NASA Technical Report TR R-315, 1969.
[12]	E. Hairer, S. P. Nørsett, G. Wanner, Solving ordinary differential equations I, Springer: Berlin, Germany, 1987. https://doi.org/10.1007/978-3-662-12607-3
[13]	H. Jerbi, S. B. Aoun, M. Omri, T. E. Simos, C. Tsitouras, A neural network type approach for constructing Runge-Kutta pairs of orders six and five that perform best on problems with oscillatory solutions, Mathematics, 10 (2022), 827. https://doi.org/10.3390/math10050827 doi: 10.3390/math10050827
[14]	H. Jerbi, S. Maali, S. B. Aoun, A. N. Aledaily, V. Jeyamani, T. E. Simos, et al., On High-Order Runge-Kutta pairs for linear inhomogeneous problems, Axioms, 14 (2025), 245. https://doi.org/10.3390/axioms14040245 doi: 10.3390/axioms14040245
[15]	J. D. Lambert, Numerical methods for ordinary differential systems, Wiley: Chichester, UK, 1991.
[16]	Y. Lu, A. Zhong, Q. Li, B. Dong, Beyond finite layer neural networks: Bridging deep architectures and numerical differential equations, In: Proc. 35th Int. Conf. Machine Learning, Stockholm, Sweden, PMLR, 80 (2018), 3276–3285.
[17]	S. Massaroli, M. Poli, D. Park, J. Yamashita, A. Asama, J. Park, Dissipative Hamiltonian neural networks, Adv. Neural Inform. Process. Syst., 33 (2020), 33.
[18]	Wolfram Research, NDSolve Mathematica function, 2025. Available from: https://reference.wolfram.com/language/tutorial/NDSolveExplicitRungeKutta.html.
[19]	G. Papageorgiou, C. Tsitouras, Runge-Kutta pairs for scalar autonomous initial value problems, Int. J. Comput. Math., 80 (2003), 201–209. https://doi.org/10.1080/00207160304669 doi: 10.1080/00207160304669
[20]	M. Poli, S. Massaroli, A. Yamashita, H. Asama, J. Park, Graph neural ordinary differential equations, arXiv preprint, 2019. https://doi.org/10.48550/arXiv.1911.07532
[21]	P. J. Prince, J. R. Dormand, High order embedded Runge-Kutta formulae, J. Comput. Appl. Math., 7 (1981), 67–75. https://doi.org/10.1016/0771-050X(81)90010-3 doi: 10.1016/0771-050X(81)90010-3
[22]	J. Riordan, An introduction to combinatorial analysis, Wiley: New York, NY, USA, 1958.
[23]	L. F. Shampine, Some practical Runge-Kutta formulas, Math. Comput., 48 (1985), 135–150. https://doi.org/10.1090/S0025-5718-1986-0815836-3 doi: 10.1090/S0025-5718-1986-0815836-3
[24]	R. Storn, K. Price, Differential evolution–-a simple and efficient heuristic for global optimization over continuous spaces, J. Global Optim., 11 (1997), 341–359. https://doi.org/10.1023/A:1008202821328 doi: 10.1023/A:1008202821328
[25]	T. E. Simos, C. Tsitouras, 6th order Runge-Kutta pairs for scalar autonomous IVP, Appl. Comput. Math., 19 (2020), 412–421.
[26]	C. Tsitouras, Runge–Kutta pairs of order 5(4) satisfying only the first column simplifying assumption, Comput. Math. Appl., 62 (2011), 770–775. https://doi.org/10.1016/j.camwa.2011.06.002 doi: 10.1016/j.camwa.2011.06.002
[27]	C. Tsitouras, S. N. Papakostas, Cheap error estimation for Runge-Kutta methods, SIAM J. Sci. Comput., 20 (1999), 2067–2088. https://doi.org/10.1137/S1064827596302230 doi: 10.1137/S1064827596302230
[28]	J. H. Verner, Numerically optimal Runge-Kutta pairs with interpolants, Numer. Algorithms, 53 (2010), 383–396. https://doi.org/10.1007/s11075-009-9290-3 doi: 10.1007/s11075-009-9290-3
[29]	Wolfram Research, Inc., Mathematica, Version 13.3, Champaign, IL, 2023.

Reader Comments

Your name:*

Email:*
© 2026 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)