PhyICNet: Physics-informed interactive learning convolutional recurrent network for spatiotemporal dynamics

Ruohan Cao; Jin Su; Jinqian Feng; Qin Guo; Ruohan Cao; Jin Su; Jinqian Feng; Qin Guo

doi:10.3934/era.2024310

Electronic Research Archive

2024, Volume 32, Issue 12: 6641-6659. doi: 10.3934/era.2024310

Previous Article Next Article

Research article

PhyICNet: Physics-informed interactive learning convolutional recurrent network for spatiotemporal dynamics

1.
School of Science, Xi'an Polytechnic University, Xi'an 710048, China
2.
Xi'an International Science and Technology Cooperation Base for Big Data Analysis and Algorithms, Xi'an 710048, China

Received: 03 September 2024 Revised: 31 October 2024 Accepted: 03 December 2024 Published: 09 December 2024

The numerical solution of spatiotemporal partial differential equations (PDEs) using the deep learning method has attracted considerable attention in quantum mechanics, fluid mechanics, and many other natural sciences. In this paper, we propose an interactive temporal physics-informed neural network architecture based on ConvLSTM for solving spatiotemporal PDEs, in which the information feedback mechanism in learning is introduced between the current input and the previous state of network. Numerical experiments on four kinds of classical spatiotemporal PDEs tasks show that the extended models have superiority in accuracy, long-range learning ability, and robustness. Our key takeaway is that the proposed network architecture is capable of learning information correlation of the PDEs model with spatiotemporal data through the input state interaction process. Furthermore, our method also has a natural advantage in carrying out physical information and boundary conditions, which could improve interpretability and reduce the bias of numerical solutions.
- partial differential equations,
- physics-informed deep learning,
- data-driven modeling,
- interactive learning,
- neural network
Citation: Ruohan Cao, Jin Su, Jinqian Feng, Qin Guo. PhyICNet: Physics-informed interactive learning convolutional recurrent network for spatiotemporal dynamics[J]. Electronic Research Archive, 2024, 32(12): 6641-6659. doi: 10.3934/era.2024310

Related Papers:

Abstract

The numerical solution of spatiotemporal partial differential equations (PDEs) using the deep learning method has attracted considerable attention in quantum mechanics, fluid mechanics, and many other natural sciences. In this paper, we propose an interactive temporal physics-informed neural network architecture based on ConvLSTM for solving spatiotemporal PDEs, in which the information feedback mechanism in learning is introduced between the current input and the previous state of network. Numerical experiments on four kinds of classical spatiotemporal PDEs tasks show that the extended models have superiority in accuracy, long-range learning ability, and robustness. Our key takeaway is that the proposed network architecture is capable of learning information correlation of the PDEs model with spatiotemporal data through the input state interaction process. Furthermore, our method also has a natural advantage in carrying out physical information and boundary conditions, which could improve interpretability and reduce the bias of numerical solutions.

References

[1]	T. J. R. Hughes, The finite element method. Linear static and dynamic finite element analysis, Comput. Methods Appl. Mech. Eng., 65 (1987), 191–193. https://doi.org/10.1016/0045-7825(87)90013-2 doi: 10.1016/0045-7825(87)90013-2
[2]	M. W. M. G. Dissanayake, N. Phan-Thien, Neural-network-based approximations for solving partial differential equations, Commun. Numer. Methods Eng., 10 (1994), 195–201. https://doi.org/10.1002/cnm.1640100303 doi: 10.1002/cnm.1640100303
[3]	I. E. Lagaris, A. Likas, D. I. Fotiadis, Artificial neural networks for solving ordinary and partial differential equations, IEEE Trans. Neural Networks, 9 (1998), 987–1000. https://doi.org/10.1109/72.712178 doi: 10.1109/72.712178
[4]	T. J. R. Hughes, J. A. Cottrell, Y. Bazilevs, Isogeometric analysis: CAD, finite elements, NURBS, exact geometry and mesh refinement, Comput. Methods Appl. Mech. Eng., 194 (2005), 4135–4195. https://doi.org/10.1016/j.cma.2004.10.008 doi: 10.1016/j.cma.2004.10.008
[5]	J. Han, A. Jentzen, W. E, Solving high-dimensional partial differential equations using deep learning, PNAS, 115 (2018), 8505–8510. https://doi.org/10.1073/pnas.1718942115 doi: 10.1073/pnas.1718942115
[6]	M. Raissi, P. Perdikaris, G. E. Karniadakis, Numerical gaussian processes for time-dependent and nonlinear partial differential equations, SIAM J. Sci. Comput., 40 (2018), A172–A198. https://doi.org/10.1137/17M1120762 doi: 10.1137/17M1120762
[7]	M. Raissi, P. Perdikaris, G. E. Karniadakis, Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys., 378 (2019), 686–707. https://doi.org/10.1016/j.jcp.2018.10.045 doi: 10.1016/j.jcp.2018.10.045
[8]	C. Song, T. Alkhalifah, U. B. Waheed, Solving the frequency-domain acoustic VTI wave equation using physics-informed neural networks, Geophys. J. Int., 225 (2020), 846–859. https://doi.org/10.1093/gji/ggab010 doi: 10.1093/gji/ggab010
[9]	F. Sahli Costabal, Y. Yang, P. Perdikaris, D. E. Hurtado, E. Kuhl, Physics-informed neural networks for cardiac activation mapping, Front. Phys., 8 (2020), 42. https://doi.org/10.3389/fphy.2020.00042 doi: 10.3389/fphy.2020.00042
[10]	J. D. Willard, X. Jia, S. Xu, M. S. Steinbach, V. Kumar, Integrating physics-based modeling with machine learning: A Survey, preprint, arXiv: 2003.04919, 2020. https://doi.org/10.48550/arXiv.2003.04919
[11]	G. E. Karniadakis, I. G. Kevrekidis, L. Lu, P. Perdikaris, S. Wang, L. Yang, Physics-informed machine learning, Nat. Rev. Phys., 3 (2021), 422–440. https://doi.org/10.1038/s42254-021-00314-5 doi: 10.1038/s42254-021-00314-5
[12]	M. Rashtbehesht, C. Huber, K. Shukla, G. Karniadakis, Physics-informed deep learning for wave propagation and full waveform inversions, J. Geophys. Res.: Solid Earth, 2021 (2021). https://doi.org/10.1029/2021JB023120 doi: 10.1029/2021JB023120
[13]	S. Liu, B. B. Kappes, B. Amin-ahmadi, O. Benafan, X. Zhang, A. P. Stebner, Physics-informed machine learning for composition - process - property design: Shape memory alloy demonstration, Appl. Mater. Today, 22 (2021), 100898. https://doi.org/10.1016/j.apmt.2020.100898 doi: 10.1016/j.apmt.2020.100898
[14]	M. Raissi, G. E. Karniadakis, Hidden physics models: Machine learning of nonlinear partial differential equations, J. Comput. Phys., 357 (2018), 125–141. https://doi.org/10.1016/j.jcp.2017.11.039 doi: 10.1016/j.jcp.2017.11.039
[15]	E. Samaniego, C. Anitescu, S. Goswami, V. M. Nguyen-Thanh, H. Guo, K. Hamdia, et al., An energy approach to the solution of partial differential equations in computational mechanics via machine learning: Concepts, implementation and applications, Comput. Methods Appl. Mech. Eng., 362 (2020), 112790. https://doi.org/10.1016/j.cma.2019.112790 doi: 10.1016/j.cma.2019.112790
[16]	C. Rao, H. Sun, Y. Liu, Physics-informed deep learning for computational elastodynamics without labeled data, J. Eng. Mech., 147 (2021), 04021043. https://doi.org/10.1061/(ASCE)EM.1943-7889.0001947 doi: 10.1061/(ASCE)EM.1943-7889.0001947
[17]	B. Wu, O. Hennigh, J. Kautz, S. Choudhry, W. Byeon, Physics informed RNN-DCT networks for time-dependent partial differential equations, preprint, arXiv: 2202.12358, 2022. https://doi.org/10.48550/arXiv.2202.12358
[18]	A. Sherstinsky, Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network, Physica D, 404 (2020), 132306. https://doi.org/10.1016/j.physd.2019.132306 doi: 10.1016/j.physd.2019.132306
[19]	R. Schmidt, Recurrent neural networks (RNNs): A gentle introduction and overview, preprint, arXiv: 1912.05911, 2019. https://doi.org/10.48550/arXiv.1912.05911
[20]	Y. Kim, Convolutional neural networks for sentence classification, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, 2014. https://doi.org/10.3115/v1/D14-1181
[21]	Y. Zhu, N. Zabaras, Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification, J. Comput. Phys., 366 (2018), 243–266. https://doi.org/10.1016/j.jcp.2018.04.018 doi: 10.1016/j.jcp.2018.04.018
[22]	C. Rao, H. Sun, Y. Liu, Hard encoding of physics for learning spatiotemporal dynamics, preprint, arXiv: 2105.00557, 2021. https://doi.org/10.48550/arXiv.2105.00557
[23]	Y. Zhu, N. Zabaras, P. S. Koutsourelakis, P. Perdikaris, Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data, J. Comput. Phys., 394 (2019), 56–81. https://doi.org/10.1016/j.jcp.2019.05.024 doi: 10.1016/j.jcp.2019.05.024
[24]	L. Sun, H. Gao, S. Pan, J. Wang, Surrogate modeling for fluid flows based on physics-constrained deep learning without simulation data, Comput. Methods Appl. Mech. Eng., 351 (2019), 112732. https://doi.org/10.1016/j.cma.2019.112732 doi: 10.1016/j.cma.2019.112732
[25]	H. Gao, L. Sun, J. X. Wang, PhyGeoNet: Physics-informed geometry-adaptive convolutional neural networks for solving parameterized steady-state PDEs on irregular domain, J. Comput. Phys., 428 (2020), 110079. https://doi.org/10.1016/j.jcp.2020.110079 doi: 10.1016/j.jcp.2020.110079
[26]	P. Ren, C. Rao, Y. Liu, Z. Ma, Q. Wang, J. X. Wang, et al., PhySR: Physics-informed deep super-resolution for spatiotemporal data, J. Comput. Phys., 492 (2023), 112438. https://doi.org/10.1016/j.jcp.2023.112438 doi: 10.1016/j.jcp.2023.112438
[27]	A. D. Jagtap, G. E. Karniadakis, Extended physics-informed neural networks (XPINNs): A generalized space-time domain decomposition based deep learning framework for nonlinear partial differential equations, Commun. Comput. Phys., 28 (2020), 2002–2041. https://doi.org/10.4208/cicp.OA-2020-0164 doi: 10.4208/cicp.OA-2020-0164
[28]	P. Ren, N. Erichson, S. Subramanian, O. San, Z. Lukic, M. Mahoney, SuperBench: A super-resolution benchmark dataset for scientific machine learning, preprint, arXiv: 2306.14070, 2023. https://doi.org/10.48550/arXiv.2306.14070
[29]	L. Wang, Z. Zhou, Z. Yan, Data-driven vortex solitons and parameter discovery of 2D generalized nonlinear Schrödinger equations with a PT-symmetric optical lattice, Comput. Math. Appl., 140 (2023), 17–23. https://doi.org/10.1016/j.camwa.2023.03.015 doi: 10.1016/j.camwa.2023.03.015
[30]	M. Sadr, T. Tohme, K. Youcef-Toumi, Data-driven discovery of PDEs via the adjoint method, preprint, arXiv: 2401.17177, 2024. https://doi.org/10.48550/arXiv.2401.17177
[31]	F. J. Aguilar-Canto, C. Brito-Loeza, H. Calvo, Model discovery of compartmental models with graph-supported neural networks, Appl. Math. Comput., 464 (2024), 128392. https://doi.org/10.1016/j.amc.2023.128392 doi: 10.1016/j.amc.2023.128392
[32]	C. Rao, P. Ren, Y. Liu, H. Sun, Discovering nonlinear PDEs from scarce data with physics-encoded learning, preprint, arXiv: 2201.12354, 2022. https://doi.org/10.48550/arXiv.2201.12354
[33]	Y. Hu, T. Zhao, S. Xu, L. Lin, Z. Xu, Neural-PDE: a RNN based neural network for solving time dependent PDEs, Commun. Inf. Syst., 22 (2020), 223–245. https://doi.org/10.4310/CIS.2022.v22.n2.a3 doi: 10.4310/CIS.2022.v22.n2.a3
[34]	P. Ren, C. Rao, Y. Liu, J. X. Wang, H. Sun, PhyCRNet: Physics-informed convolutional-recurrent network for solving spatiotemporal PDEs, Comput. Methods Appl. Mech. Eng., 389 (2022), 114399. https://doi.org/10.1016/j.cma.2021.114399 doi: 10.1016/j.cma.2021.114399
[35]	L. Jiang, L. Wang, X. Chu, Y. Xiao, H. Zhang, PhyGNNet: Solving spatiotemporal PDEs with physics-informed graph neural network, in Proceedings of the 2023 2nd Asia Conference on Algorithms, Computing and Machine Learning, 2022. https://doi.org/10.1145/3590003.3590029
[36]	X. Meng, Z. Li, D. Zhang, G. E. Karniadakis, PPINN: Parareal physics-informed neural network for time-dependent PDEs, Comput. Methods Appl. Mech. Eng., 370 (2020), 113250. https://doi.org/10.1016/j.cma.2020.113250 doi: 10.1016/j.cma.2020.113250
[37]	A. Mavi, A. C. Bekar, E. Haghighat, E. Madenci, An unsupervised latent/output physics-informed convolutional-LSTM network for solving partial differential equations using peridynamic differential operator, Comput. Methods Appl. Mech. Eng., 407 (2023), 115944. https://doi.org/10.1016/j.cma.2023.115944 doi: 10.1016/j.cma.2023.115944
[38]	P. Saha, S. Dash, S. Mukhopadhyay, Physics-incorporated convolutional recurrent neural networks for source identification and forecasting of dynamical systems, Neural Networks, 144 (2021), 359–371. https://doi.org/10.1016/j.neunet.2021.08.033 doi: 10.1016/j.neunet.2021.08.033
[39]	P. R. Kakka, Sequence to sequence AE-ConvLSTM network for modelling the dynamics of PDE systems, preprint, arXiv: 2208.07315, 2022. https://doi.org/10.48550/arXiv.2208.07315
[40]	B. Krause, L. Lu, I. Murray, S. Renals, Multiplicative LSTM for sequence modelling, preprint, arXiv: 1609.07959, 2017. https://doi.org/10.48550/arXiv.1609.07959
[41]	G. Melis, T. Kočišký, P. Blunsom, Mogrifier LSTM, preprint, arXiv: 1909.01792, 2020. https://doi.org/10.48550/arXiv.1909.01792
[42]	G. Larsson, M. Maire, G. Shakhnarovich, FractalNet: Ultra-deep neural networks without residuals, preprint, arXiv: 1605.07648, 2017. https://doi.org/10.48550/arXiv.1605.07648
[43]	Y. Lu, A. Zhong, Q. Li, B. Dong, Beyond finite layer neural networks: bridging deep architectures and numerical differential equations, preprint, arXiv: 1710.10121, 2020. https://doi.org/10.48550/arXiv.1710.10121
[44]	L. Ruthotto, E. Haber, Deep neural networks motivated by partial differential equations, J. Math. Imaging Vision, 61 (2019), 787–805. https://doi.org/10.1007/s10851-019-00903-1 doi: 10.1007/s10851-019-00903-1
[45]	Z. Long, Y. Lu, B. Dong, PDE-Net 2.0: Learning PDEs from data with a numeric-symbolic hybrid deep network, J. Comput. Phys., 399 (2019), 108925. https://doi.org/10.1016/j.jcp.2019.108925 doi: 10.1016/j.jcp.2019.108925
[46]	M. Raissi, Deep hidden physics models: deep learning of nonlinear partial differential equations, J. Mach. Learn. Res., 19 (2018), 932–955. https://doi.org/10.5555/3291125.3291150 doi: 10.5555/3291125.3291150
[47]	D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, preprint, arXiv: 1412.6980, 2017. https://doi.org/10.48550/arXiv.1412.6980
[48]	A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, et al., Automatic differentiation in PyTorch, in NeurIPS Autodiff Workshop, 2017.

Reader Comments

Your name:*

Email:*
© 2024 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)