Export file:


  • RIS(for EndNote,Reference Manager,ProCite)
  • BibTex
  • Text


  • Citation Only
  • Citation and Abstract

A machine learning framework for data driven acceleration of computations of differential equations

Seminar for Applied Mathematics (SAM), D-Math, ETH Zürich, Rämistrasse 101, Zürich-8092,Switzerland

We propose a machine learning framework to accelerate numerical computations oftime-dependent ODEs and PDEs. Our method is based on recasting (generalizations of) existingnumerical methods as artificial neural networks, with a set of trainable parameters. These parametersare determined in an offline training process by (approximately) minimizing suitable (possibly non-convex)loss functions by (stochastic) gradient descent methods. The proposed algorithm is designed tobe always consistent with the underlying differential equation. Numerical experiments involving bothlinear and non-linear ODE and PDE model problems demonstrate a significant gain in computational efficiency over standard numerical methods.
  Article Metrics

Keywords machine learning; deep learning; differential equations; non-convex optimization; time-dependent problems

Citation: Siddhartha Mishra. A machine learning framework for data driven acceleration of computations of differential equations. Mathematics in Engineering, 2019, 1(1): 118-146. doi: 10.3934/Mine.2018.1.118


  • 1. Barron AR (1993) Universal approximation bounds for superpositions of a sigmoidal function. IEEE T Inform Theory 39: 930–945.    
  • 2. Beck C, Weinan E and Jentzen A (2017) Machine learning approximation algorithms for high dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations. Technical Report 2017-49, Seminar for Applied Mathematics, ETH Zürich.
  • 3. Bijl H, Lucor D, Mishra S, et al. (2014) Uncertainty quantification in computational fluid dynamics. Lecture notes in computational science and engineering 92, Springer.
  • 4. Borzi A and Schulz V (2012) Computational optimization of systems governed by partial differential equations, SIAM.
  • 5. Brenner SC and Scott LR (2008) The mathematical theory of finite element methods. Texts in applied mathematics 15, Springer.
  • 6. Dafermos CM (2005) Hyperbolic Conservation Laws in Continuum Physics (2nd Ed.), Springer Verlag.
  • 7. Weinan E and Yu B (2018) The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems. Commun Math Stat 6: 1–12.    
  • 8. Fjordholm US, Mishra S and Tadmor E (2012) Arbitrarily high-order order accurate essentially non-oscillatory entropy stable schemes for systems of conservation laws. SIAM J Numer Anal 50: 544–573.    
  • 9. Ghanem R, Higdon D and Owhadi H (2016) Handbook of uncertainty quantification, Springer.
  • 10. Godlewski E and Raviart PA (1991) Hyperbolic Systems of Conservation Laws. Mathematiques et Applications, Ellipses Publ., Paris.
  • 11. Goodfellow I, Bengio Y and Courville A (2016) Deep learning. MIT Press. Available from: http://www.deeplearningbook.org.
  • 12. Hairer E and Wanner G (1991) Solving ordinary differential equations. Springer Series in computational mathematics, 14, Springer.
  • 13. Hornik K, Stinchcombe M, and White H (1989) Multilayer feedforward networks are universal approximators. Neural networks 2: 359–366.    
  • 14. Kingma DP and Ba JL (2015) Adam: a Method for Stochastic Optimization. International Conference on Learning Representations, 1–13.
  • 15. LeCun Y, Bengio Y and Hinton G (2015) Deep learning. Nature 521: 436–444.    
  • 16. LeVeque RJ (2007) Finite difference methods for ordinary and partial differential equations, steady state and time dependent problems, SIAM.
  • 17. Mishra S, Schwab C and Šukys J (2012) Multi-level Monte Carlo finite volume methods for nonlinear systems of conservation laws in multi-dimensions. J Comput Phys 231: 3365–3388.    
  • 18. Miyanawala TP and Jaiman RK (2017) An efficient deep learning technique for the Navier-Stokes equations: application to unsteady wake flow dynamics. Preprint, arXiv :1710.09099v2.
  • 19. Poon H and Domingos P (2011) Sum-product Networks: A new deep architecture. International conference on computer vision (ICCV), 689–690.
  • 20. Raissi M and Karniadakis GE (2018) Hidden physics models: machine learning of nonlinear partial differential equations. J Comput Phys 357: 125–141.    
  • 21. Ray D and Hesthaven JS (2018) An artificial neural network as a troubled cell indicator. J Comput Phys to appear.
  • 22. Ruder S (2017) An overview of gradient descent optimization algorithms. Preprint, arXiv.1609.04747v2.
  • 23. Schwab C and Zech J (2017) Deep learning in high dimension. Technical Report 2017-57, Seminar for Applied Mathematics, ETH Zürich.
  • 24. Stuart AM (2010) Inverse problems: a Bayesian perspective. Acta Numerica 19: 451–559.    
  • 25. Tadmor E (2003) Entropy stability theory for difference approximations of nonlinear conservation laws and related time-dependent problems. Acta Numerica, 451–512.
  • 26. Tompson J, Schlachter K, Sprechmann P, et al. (2017) Accelarating Eulerian fluid simulation with convolutional networks. Preprint, arXiv:1607.03597v6.
  • 27. Trefethen LN (2000) Spectral methods in MATLAB, SIAM.
  • 28. Troltzsch F (2010) Optimal control of partial differential equations. AMS.
  • 29. Quateroni A, Manzoni A and Negri F (2015) Reduced basis methods for partial differential equations: an introduction, Springer Verlag.
  • 30. Yarotsky D (2017) Error bounds for approximations with deep ReLU networks. Neural Networks 94: 103–114    


This article has been cited by

  • 1. Kjetil O. Lye, Siddhartha Mishra, Deep Ray, Deep learning observables in computational fluid dynamics, Journal of Computational Physics, 2020, 109339, 10.1016/j.jcp.2020.109339
  • 2. Martin Hutzenthaler, Arnulf Jentzen, Thomas Kruse, Tuan Anh Nguyen, A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations, SN Partial Differential Equations and Applications, 2020, 1, 2, 10.1007/s42985-019-0006-9
  • 3. KJETIL O. LYE, SIDDHARTHA MISHRA, ROBERTO MOLINARO, A multi-level procedure for enhancing accuracy of machine learning algorithms, European Journal of Applied Mathematics, 2020, 1, 10.1017/S0956792520000224

Reader Comments

your name: *   your email: *  

© 2019 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution Licese (http://creativecommons.org/licenses/by/4.0)

Download full text in PDF

Export Citation

Copyright © AIMS Press All Rights Reserved