Reinforcement learning in optimization problems. Applications to geophysical data inversion

Paolo Dell'Aversana; Paolo Dell'Aversana

doi:10.3934/geosci.2022027

AIMS Geosciences

2022, Volume 8, Issue 3: 488-502. doi: 10.3934/geosci.2022027

Previous Article Next Article

Research article Special Issues

Reinforcement learning in optimization problems. Applications to geophysical data inversion

Paolo Dell'Aversana ^,

Eni S.p.A., San Donato Milanese, Milan, Italy

Received: 06 May 2022 Revised: 18 July 2022 Accepted: 31 July 2022 Published: 08 August 2022

In this paper, we introduce a novel inversion methodology that combines the benefits offered by Reinforcement-Learning techniques with the advantages of the Epsilon-Greedy method for an expanded exploration of the model space. Among the various Reinforcement Learning approaches, we applied the set of algorithms included in the category of the Q-Learning methods. We show that the Temporal Difference algorithm offers an effective iterative approach that allows finding an optimal solution in geophysical inverse problems. Furthermore, the Epsilon-Greedy method properly coupled with the Reinforcement Learning workflow, allows expanding the exploration of the model-space, minimizing the misfit between observed and predicted responses and limiting the problem of local minima of the cost function. In order to prove the feasibility of our methodology, we tested it using synthetic geo-electric data and a seismic refraction data set available in the public domain.

Keywords:

Citation: Paolo Dell'Aversana. Reinforcement learning in optimization problems. Applications to geophysical data inversion[J]. AIMS Geosciences, 2022, 8(3): 488-502. doi: 10.3934/geosci.2022027

Related Papers:

[1]	Zongcheng Li, Jin Li . Linear barycentric rational collocation method for solving a class of generalized Boussinesq equations. AIMS Mathematics, 2023, 8(8): 18141-18162. doi: 10.3934/math.2023921
[2]	Jin Li . Barycentric rational collocation method for semi-infinite domain problems. AIMS Mathematics, 2023, 8(4): 8756-8771. doi: 10.3934/math.2023439
[3]	Jin Li . Barycentric rational collocation method for fractional reaction-diffusion equation. AIMS Mathematics, 2023, 8(4): 9009-9026. doi: 10.3934/math.2023451
[4]	Haoran Sun, Siyu Huang, Mingyang Zhou, Yilun Li, Zhifeng Weng . A numerical investigation of nonlinear Schrödinger equation using barycentric interpolation collocation method. AIMS Mathematics, 2023, 8(1): 361-381. doi: 10.3934/math.2023017
[5]	Kareem T. Elgindy, Hareth M. Refat . A direct integral pseudospectral method for solving a class of infinite-horizon optimal control problems using Gegenbauer polynomials and certain parametric maps. AIMS Mathematics, 2023, 8(2): 3561-3605. doi: 10.3934/math.2023181
[6]	Qasem M. Tawhari . Mathematical analysis of time-fractional nonlinear Kuramoto-Sivashinsky equation. AIMS Mathematics, 2025, 10(4): 9237-9255. doi: 10.3934/math.2025424
[7]	Yunmei Zhao, Yinghui He, Huizhang Yang . The two variable (φ/φ, 1/φ)-expansion method for solving the time-fractional partial differential equations. AIMS Mathematics, 2020, 5(5): 4121-4135. doi: 10.3934/math.2020264
[8]	Yangfang Deng, Zhifeng Weng . Barycentric interpolation collocation method based on Crank-Nicolson scheme for the Allen-Cahn equation. AIMS Mathematics, 2021, 6(4): 3857-3873. doi: 10.3934/math.2021229
[9]	M. Mossa Al-Sawalha, Safyan Mukhtar, Albandari W. Alrowaily, Saleh Alshammari, Sherif. M. E. Ismaeel, S. A. El-Tantawy . Analytical solutions to time-space fractional Kuramoto-Sivashinsky Model using the integrated Bäcklund transformation and Riccati-Bernoulli sub-ODE method. AIMS Mathematics, 2024, 9(5): 12357-12374. doi: 10.3934/math.2024604
[10]	Sunyoung Bu . A collocation methods based on the quadratic quadrature technique for fractional differential equations. AIMS Mathematics, 2022, 7(1): 804-820. doi: 10.3934/math.2022048

Abstract

1. Introduction

Lots of physical phenomena can be expressed by non-linear partial differential equations (PDE), including, inter alia, dissipative and dispersive PDE. In this paper, we consider the Kuramoto-Sivashinsky (KS) equation

$\begin{eqnarray} \frac{\partial \phi}{\partial t}+\gamma \frac{\partial^4 \phi}{\partial s^4}+\frac{\partial^2 \phi}{\partial s^2}+ \phi\frac{\partial \phi}{\partial s} = \varphi(s, t)\; 0 \le s\le 1, \; 0 \le t \le T, \; \gamma > 0, \end{eqnarray}$

(1.1)

$\begin{eqnarray} &&\phi(0, t) = 0, \; \phi(1, t) = 0, \; \phi_{ss}(0, t) = 0, \phi_{ss}(1, t) = 0, \; 0 < t < T, \end{eqnarray}$

(1.2)

$\begin{eqnarray} &&\phi(s, 0) = \varphi(s), \; 0\le s \le 1, \end{eqnarray}$

(1.3)

where $\gamma \in R$ is the constant.

The KS equation plays an important role in physics such as in diffusion, convection and so on. Lots of attention has been paid by researchers in recent years. An $H_1$ -Galerkin mixed finite element method for the KS equation was proposed in ^[1], lattice Boltzmann models for the Kuramoto-Sivashinsky equation were studied in ^[2], Backward difference formulae (BDF) methods for the KS equation were investigate in ^[3]. Stability regions and results for the Korteweg-de Vries-Burgers and Kuramoto-Sivashinsky equations were given in ^[4,5], respectively. In ^[6], an improvised quintic B-spline extrapolated collocation technique was used to solve the KS equation, and the stability of the technique was analyzed using the von Neumann scheme, which was found to be unconditionally stable. In ^[7], a septic Hermite collocation method (SHCM) was proposed to simulate the KS equation, and the nonlinear terms of the KS equation were linearized using the quasi-linearization process. In ^[8], a semidiscrete approach was presented to solve the variable-order (VO) time fractional 2D KS equation, and the differentiation operational matrices and the collocation technique were used to get a linear system of algebraic equations. In ^[9] the discrete Legendre polynomials (LPs) and the collocation scheme for nonlinear space-time fractional KdV-Burgers-Kuramoto equation were presented.

In order to avoid the Runge's phenomenon, barycentric interpolation ^[10,11,12] was developed. In recent years, linear rational interpolation (LRI) was proposed by Floater ^[13,14,15], and error of linear rational interpolation was also proved. The barycentric interpolation collocation method (BICM) has been developed by Wang et al.^{[22,23,24,25]}, and the algorithm of BICM has been used for linear/non-linear problems ^[21]. Volterra integro-differential equation (VIDE)^[16,20], heat equation (HE) ^[17], biharmonic equation (BE) ^[18], the Kolmogorov-Petrovskii-Piskunov (KPP) equation ^[19], fractional differential equations ^[20], fractional reaction-diffusion equation ^[28], semi-infinite domain problems ^[27] and biharmonic equation ^[26], plane elastic problems ^[29] have been studied by the linear barycentric interpolation collocation method (LBICM), and their convergence rates also have been proved.

In order to solve the KS equation efficiently, the LBRIM is presented. Because the nonlinear part of the KS equation cannot be solved directly, three kinds of linearization methods, including direct linearization, partial linearization and Newton linearization, are presented. Then, the nonlinear part of the KS equation is translated into the linear part, three kinds of iterative schemes are presented, and matrix equation of the linearization schemes are constructed. The convergence rate of the LBRCM for the KS equation is also given. At last, two numerical examples are presented to validate the theoretical analysis.

2. Linearization for KS equation

In the following, the KS equation is changed into the linear equation by the linearization scheme, including direct linearization, partial linearization and Newton linearization.

2.1. Direct linearization

For the Kuramoto-Sivashinskyr equation with the initial value of nonlinear term $\phi\frac{\partial \phi}{\partial s}$ is changed to $\phi_{0}\frac{\partial \phi_{0}}{\partial s}$ ,

$\begin{eqnarray} \frac{\partial \phi}{\partial t}+\gamma \frac{\partial^4 \phi}{\partial s^4}+\frac{\partial^2 \phi}{\partial s^2}+ \phi_{0}\frac{\partial \phi_{0}}{\partial s} = \varphi(s, t), \end{eqnarray}$

(2.1)

and then we get the linear scheme as

$\begin{eqnarray} \frac{\partial \phi_{n}}{\partial t}+\gamma \frac{\partial^4 \phi_{n}}{\partial s^4}+\frac{\partial^2 \phi_{n}}{\partial s^2} = -\phi_{n-1}\frac{\partial \phi_{n-1}}{\partial s}+\varphi(s, t), a \le s\le b, 0 \le t \le T. \end{eqnarray}$

(2.2)

2.2. Partial linearization

By the partial linearization, nonlinear term $\phi\frac{\partial \phi}{\partial s}$ is changed to $\phi_{0}\frac{\partial \phi}{\partial s}$ ,

(2.3)

and then we have

$\begin{eqnarray} \frac{\partial \phi_{n}}{\partial t}+\gamma \frac{\partial^4 \phi_{n}}{\partial s^4}+\frac{\partial^2 \phi_{n}}{\partial s^2}+\phi_{n-1}\frac{\partial \phi_{n}}{\partial s} = \varphi(s, t), a \le s\le b, 0 \le t \le T. \end{eqnarray}$

(2.4)

2.3. Newton linearization

For the initial value $\phi\frac{\partial \phi}{\partial s} = \phi_{0}\frac{\partial \phi_{0}}{\partial s}+(\frac{\partial \phi_{0}}{\partial s}+\phi_{0}\frac{\partial \phi_{0}}{\partial s})(\phi-\phi_{0})$ , we have

$\begin{eqnarray} \frac{\partial \phi}{\partial t}+\gamma \frac{\partial^4 \phi}{\partial s^4}+\frac{\partial^2 \phi}{\partial s^2}+ \phi\frac{\partial \phi_{0}}{\partial s}+ \phi_{0}\frac{\partial \phi_{0}}{\partial s}\phi = \varphi(s, t)+\phi_{0}\frac{\partial \phi_{0}}{\partial s}\phi_{0}, \end{eqnarray}$

(2.5)

and then we have

$\begin{eqnarray} \frac{\partial \phi_{n}}{\partial t}+\gamma \frac{\partial^4 \phi_{n}}{\partial s^4}+\frac{\partial^2 \phi_{n}}{\partial s^2}+\phi_{n}\frac{\partial \phi_{n-1}}{\partial s}+\phi_{n-1}\frac{\partial \phi_{n-1}}{\partial s}\phi_{n} = \varphi(s, t)+\phi_{n-1}\frac{\partial \phi_{n-1}}{\partial s}\phi_{n-1}, \end{eqnarray}$

(2.6)

where $n = 1, 2, \cdots.$

3. Differentiation matrices of KS equation

Interval $[a, b]$ is divided into $a = s_{0} < s_{1} < s_{2} < \cdots < s_{m-1} < s_{m} = b$ , for uniform partition with $h_{s} = \frac{b-a}{m}$ and nonuniform partition to be the second kind of Chebychev point. Time $[0, T]$ is divided into $0 = t_{0} < t_{1} < t_{2} < \cdots < t_{n-1} < t_{n} = T$ and $h_{t} = \frac{T}{n}$ for uniform partition. Then, we take $\phi_{nm}(s, t)$ to approximate $\phi(s, t)$ as

$\begin{eqnarray} \phi_{nm}(s, t) = \sum\limits_{i = 0}^{m} \sum\limits_{j = 0}^{n} r_{i}(s) r_{j}(t) \phi_{ij} \end{eqnarray}$

(3.1)

where $\phi_{ij} = \phi(s_{i}, t_{j})$ ,

$\begin{eqnarray} r_{i}(s) = \frac{ \frac{w_{i}}{s-s_{i}} }{ \sum\limits_{j = 0}^{m} \frac{w_{j}}{s-s_{j}}}, \quad r_{j}(t) = \frac{ \frac{w_{j}}{t-t_{j}} }{ \sum\limits_{i = 0}^{n} \frac{w_{i}}{t-t_{i}}} \end{eqnarray}$

(3.2)

is the barycentric interpolation basis ^[26], and

$\begin{eqnarray} w_{i} = \sum\limits_{k \in J_{i}}(-1)^{k} \prod\limits_{j = k, j \neq i}^{k+d_{s}} \frac{1}{s_{i}-s_{j}}, \quad w_{j} = \sum\limits_{k \in J_{j}}(-1)^{k} \prod\limits_{i = k, k \neq j}^{k+d_{t}} \frac{1}{t_{j}-t_{i}} \end{eqnarray}$

(3.3)

where $J_{i} = \{k \in I, i-d_{s} \le k \le i\}, I = \{0, 1, \cdots, m-d_{s} \}$ . See ^[26]. We get the barycentric rational interpolation.

For the case

$\begin{eqnarray} w_{i} = \frac{1}{\prod\nolimits_{i \neq k}\left(s_{i}-s_{k}\right)}, w_{j} = \frac{1}{\prod\nolimits_{j \neq k}\left(t_{j}-t_{k}\right)}, \end{eqnarray}$

(3.4)

we get the barycentric Lagrange interpolation.

So,

$\begin{eqnarray} r_{j}^{\prime }\left(s_{i}\right) = \frac{w_{j} / w_{i}}{s_{i}-s_{j}}, \quad j \neq i, \quad r_{i}^{\prime }\left(s_{i}\right) = -\sum\limits_{j \neq i} r_{j}^{\prime }\left(s_{i}\right), \end{eqnarray}$

(3.5)

$\begin{eqnarray} r_{j}^{(k)}\left(s_{i}\right) = k \left(r_{i}^{(k-1)}\left(s_{i}\right) r_{i}^{\prime}\left(s_{j}\right) - \frac{r_{i}^{(k-1)}\left(s_{j}\right) }{s_{i}-s_{j}}\right), \quad j \neq i, \end{eqnarray}$

(3.6)

$\begin{eqnarray} r_{i}^{(k)}\left(s_{i}\right) = -\sum\limits_{j \neq i} r_{j}^{(k)}\left(s_{i}\right). \end{eqnarray}$

(3.7)

Then, we have

$\begin{eqnarray} \mathrm{{D}}^{(0, 1)}_{ij}& = & r_{i}^{\prime }\left(t_{j}\right), \end{eqnarray}$

(3.8)

$\begin{eqnarray} \mathrm{{D}}^{(1, 0)}_{ij}& = & r_{i}^{\prime }\left(s_{j}\right), \end{eqnarray}$

(3.9)

$\begin{eqnarray} \mathrm{{D}}^{(k, 0)}_{ij}& = & r_{i}^{(k) }\left(s_{j}\right), k = 2, 3, \cdots. \end{eqnarray}$

(3.10)

3.1. Matrix equation of direct linearization

Combining (3.1) and (2.2), we have

$\begin{eqnarray} \left[\mathcal{{I}}_{m} \otimes \mathcal{{D}}^{(0, 1)} + \mathcal{{D}}^{(2, 0)}\otimes\mathcal{{I}}_{n}+\gamma \mathcal{{D}}^{(4, 0)}\otimes\mathcal{{I}}_{n} \right] \mathcal{\phi}_{n} = {\Psi}-diag(\mathcal{\phi}_{n-1})\mathcal{{D}}^{(1, 0)}\otimes\mathcal{{I}}_{n}\cdot\mathcal{\phi}_{n-1}, \end{eqnarray}$

(3.11)

and then we have

$\begin{eqnarray} \mathcal{L} \mathcal{\phi}_{n} = {\Psi}_{n-1} \end{eqnarray}$

(3.12)

where

$\mathcal{L} = \mathcal{{I}}_{m} \otimes \mathcal{{D}}^{(0, 1)} + \mathcal{{D}}^{(2, 0)}\otimes\mathcal{{I}}_{n}+\gamma \mathcal{{D}}^{(4, 0)}\otimes\mathcal{{I}}_{n},$

${\Psi}_{n-1} = {\Psi}-diag(\mathcal{\phi}_{n-1})\mathcal{{D}}^{(1, 0)}\otimes\mathcal{{I}}_{n}\cdot\mathcal{\phi}_{n-1}$

and $\otimes$ is the Kronecher product ^[17].

3.2. Matrix equation of partial linearization

Combining (3.1) and (2.4), we have

$\begin{eqnarray} \left[\mathcal{{I}}_{m} \otimes \mathcal{{D}}^{(0, 1)} + \mathcal{{D}}^{(2, 0)}\otimes\mathcal{{I}}_{n}+\gamma \mathcal{{D}}^{(4, 0)}\otimes\mathcal{{I}}_{n}+diag(\mathcal{\phi}_{n-1})\mathcal{{D}}^{(1, 0)}\otimes\mathcal{{I}}_{n} \right] \mathcal{\phi}_{n} = {\Psi}, \end{eqnarray}$

(3.13)

$n = 1, 2, \cdots$ , and then we have

$\begin{eqnarray} \mathcal{L} \mathcal{\phi} = {\Psi} \end{eqnarray}$

(3.14)

where $\mathcal{L} = \mathcal{{I}}_{m} \otimes \mathcal{{D}}^{(0, 1)} + \mathcal{{D}}^{(2, 0)}\otimes\mathcal{{I}}_{n}+\gamma \mathcal{{D}}^{(4, 0)}\otimes\mathcal{{I}}_{n}+diag(\mathcal{\phi}_{n-1})\mathcal{{D}}^{(1, 0)}\otimes\mathcal{{I}}_{n}$ .

3.3. Matrix equation of Newton linearization

Combining (3.1) and (2.6), we have

$\begin{eqnarray} &&\left[\mathcal{{I}}_{m} \otimes \mathcal{{D}}^{(0, 1)} + \mathcal{{D}}^{(2, 0)}\otimes\mathcal{{I}}_{n}+\gamma \mathcal{{D}}^{(4, 0)}\otimes\mathcal{{I}}_{n}+diag(\mathcal{\phi}_{n-1})\mathcal{{D}}^{(1, 0)}\otimes\mathcal{{I}}_{n} \right] \mathcal{\phi}_{n} \\&& = {\Psi} + [diag(\mathcal{\phi}_{n})-diag(\mathcal{\phi}_{n-1})]\mathcal{{D}}^{(1, 0)}\otimes\mathcal{{I}}_{n}\cdot\mathcal{\phi}_{n-1}, \end{eqnarray}$

(3.15)

and then we get

$\begin{eqnarray} \mathcal{L} \mathcal{\phi} = {\Psi}_{n-1} \end{eqnarray}$

(3.16)

where

$\mathcal{L} = \mathcal{{I}}_{m} \otimes \mathcal{{D}}^{(0, 1)} + \mathcal{{D}}^{(2, 0)}\otimes\mathcal{{I}}_{n}+\gamma \mathcal{{D}}^{(4, 0)}\otimes\mathcal{{I}}_{n}+diag(\mathcal{\phi}_{n-1})\mathcal{{D}}^{(1, 0)}\otimes\mathcal{{I}}_{n},$

and

${\Psi}_{n-1} = {\Psi} + [diag(\mathcal{\phi}_{n})-diag(\mathcal{\phi}_{n-1})]\mathcal{{D}}^{(1, 0)}\otimes\mathcal{{I}}_{n}\cdot\mathcal{\phi}_{n-1}.$

4. Convergence rate of KS equation

In this part, an error estimate of the KS equation is given with $r_n(s) = \sum\limits_{i = 0}^{n} r_{i}(s) \phi_{i}$ to replace $\phi(s)$ , where $r_{i}(s)$ is defined as (3.2), and $\phi_{i} = \phi(s_{i})$ . We also define

$\begin{eqnarray} e(s): = \phi(s)-r_n(s) = (s-s_{i}) \cdots(s-s_{i+d})\phi\left[s_{i}, s_{i+1}, \ldots, s_{i+d}, s\right]. \end{eqnarray}$

(4.1)

Then, we have the following.

Lemma 1. For $e(s)$ defined by (4.1) and $\phi(s)\in C^{d+2}[a, b]$ , there is

$\begin{equation} \left|e^{(k)}(s)\right| \leq C h^{d-k+1}, k = 0, 1, \cdots. \end{equation}$

(4.2)

For KS equation, rational interpolation function of $\phi(s, t)$ is defined as $r_{mn}(s, t)$

$\begin{equation} r_{mn}(s, t) = \frac{ \sum\limits_{i = 0}^{m+d_{s}} \sum\limits_{j = 0}^{n+d_{t}} \frac{w_{i, j}}{\left(s-s_{i}\right)\left(t-t_{j}\right)} {\phi}_{i, j}}{ \sum\limits_{i = 0}^{m+d_{s}} \sum\limits_{j = 0}^{n+d_{t}} \frac{w_{i, j}}{\left(s-s_{i}\right)\left(t-t_{j}\right)}} \end{equation}$

(4.3)

where

$\begin{equation} w_{i, j} = (-1)^{i-d_{s}+j-d_{t}} \sum\limits_{k_{1} \in J_{i}}\prod\limits_{h_{1} = k_{1}, h_{1} \neq j}^{k_{1}+d_{s}}\frac{1}{\left|s_{i}-s_{h_{1}}\right|} \sum\limits_{k_{2} \in J_{i}} \prod\limits_{h_{2} = k_{2}, h_{2} \neq j}^{k_{2}+d_{t}} \frac{1}{\left|t_{j}-t_{h_{2}}\right|}. \end{equation}$

(4.4)

We define $e(s, t)$ to be the error of $\phi(s, t)$ as

$\begin{eqnarray} e(s, t) :& = & \phi(s, t)-r_{mn}(s, t) \\ & = & \left(s-s_{i}\right) \cdots\left(s-s_{i+d_{s}}\right)\phi\left[s_{i}, s_{i+1}, \ldots, s_{i+d_1}, s, t\right] \\ &+&\left(t-t_{j}\right) \cdots\left(t-t_{j+d_{t}}\right)\phi\left[s, t_{j}, t_{j+1}, \ldots, t_{j+d_2}, t\right]. \end{eqnarray}$

(4.5)

With similar analysis of Lemma 1, we have the following

Theorem 1. For $e(s, t)$ defined as (4.5) and $\phi(s, t) \in C^{d_{s}+2}[a, b]\times C^{d_{t}+2}[0, T]$ , we have

$\begin{equation} \left|e^{(k_{1}, k_{2})}\left(s, t\right)\right| \leq C (h_{s}^{d_{s}-k_{1}+1}+h_{t}^{d_{t}-k_{2}+1}), k_{1}, k_{2} = 0, 1, \cdots. \end{equation}$

(4.6)

We take the direct linearization of the KS equation as an example prove the convergence rate. Let $\phi(s_{m}, t_{n})$ be the approximate function of $\phi(s, t)$ and $L$ be a bounded operator. There holds

$\begin{eqnarray} L \phi(s_{m}, t_{n}) = \varphi(s_{m}, t_{n}), \end{eqnarray}$

(4.7)

and

$\begin{eqnarray} \lim\limits_{m, n\rightarrow \infty} \phi(s_{m}, t_{n}) = \phi(s, t). \end{eqnarray}$

(4.8)

Then, we get the following

Theorem 2. For $\phi(s_{m}, t_{n}):L\phi(s_{m}, t_{n}) = \varphi(s, t)$ and $L$ defined as (4.7), there

$|\phi(s, t)-\phi(s_{m}, t_{n})|\le C(h^{d_{s}-3}+\tau^{d_{t}}).$

Proof. As

$\begin{eqnarray} \begin{array}{ll} &L \phi(s, t)-L \phi(s_{m}, t_{n}) \\[0.5cm]& = \frac{\partial \phi}{\partial t}+\gamma \frac{\partial^4 \phi}{\partial s^4}+\frac{\partial^2 \phi}{\partial s^2}- \phi_{0}\frac{\partial \phi_{0}}{\partial s}-\varphi(s, t) \\[0.5cm]&- \left[\frac{\partial \phi(s_{m}, t_{n})}{\partial t}+\gamma \frac{\partial^4 \phi(s_{m}, t_{n})}{\partial s^4}+\frac{\partial^2 \phi(s_{m}, t_{n})}{\partial s^2}+ \phi_{0}(s_{m}, t_{n})\frac{\partial \phi_{0}(s_{m}, t_{n})}{\partial s}-\varphi(s, t)\right] \\[0.5cm]& = \frac{\partial \phi}{\partial t}-\frac{\partial \phi}{\partial t}(s_{m}, t_{n}) +\gamma \left[\frac{\partial^4 \phi}{\partial s^4}- \frac{\partial^4 \phi}{\partial s^4}(s_{m}, t_{n})\right] \\[0.5cm]&+ \frac{\partial^2 \phi}{\partial s^2}-\frac{\partial^2 \phi}{\partial s^2}(s_{m}, t_{n})+\left[\phi_{0}\frac{\partial \phi_{0}}{\partial s}- \phi_{0}(s_{m}, t_{n})\frac{\partial \phi_{0}}{\partial s}(s_{m}, t_{n}) \right] \\[0.5cm]&: = E_{1}(s, t)+E_{2}(s, t)+E_{3}(s, t)+E_{4}(s, t). \end{array} \end{eqnarray}$

(4.9)

Here,

$E_{1}(s, t) = \frac{\partial \phi}{\partial t}-\frac{\partial \phi}{\partial t}(s_{m}, t_{n}),$

$E_{2}(s, t) = \gamma \left[\frac{\partial^4 \phi}{\partial s^4}- \frac{\partial^4 \phi}{\partial s^4}(s_{m}, t_{n})\right],$

$E_{3}(s, t) = \frac{\partial^2 \phi}{\partial s^2}-\frac{\partial^2 \phi}{\partial s^2}(s_{m}, t_{n}),$

$E_{4}(s, t) = \phi_{0}\frac{\partial \phi_{0}}{\partial s}- \phi_{0}(s_{m}, t_{n})\frac{\partial \phi_{0}}{\partial s}(s_{m}, t_{n}) .$

With $E_{2}(s, t)$ , we have

$\begin{equation} \begin{array} {ll} E_{2}(s, t)& = \gamma \left[\frac{\partial^4 \phi}{\partial s^4}- \frac{\partial^4 \phi}{\partial s^4}(s_{m}, t_{n})\right] \\& = \gamma \left[\frac{\partial^4 \phi}{\partial s^4}- \frac{\partial^4 \phi}{\partial s^4}(s_{m}, t)+ \frac{\partial^4 \phi}{\partial s^4}(s_{m}, t)- \frac{\partial^4 \phi}{\partial s^4}(s_{m}, t_{n})\right] \\& = \frac{ \sum\limits_{i = 0}^{m-d_{s}} (-1)^{i}\frac{\partial^4 \phi}{\partial s^4}[s_{i}, s_{i+1}, \ldots, s_{i+d_1}, s_{m}, t]} { \sum\limits_{i = 0}^{m-d_{s}} \lambda_{i}(s)} \\ \nonumber&+ \frac{ \sum\limits_{j = 0}^{n-d_{t}}(-1)^{j}\frac{\partial^4 \phi}{\partial s^4}[ t_{j}, t_{j+1}, \ldots, t_{j+d_2}, s_{m}, t_{n}]} { \sum\limits_{j = 0}^{n-d_{t}}\lambda_{j}(t)} \\& = \frac{\partial^4 e}{\partial s^4}(s_{m}, t)+\frac{\partial^4 e}{\partial s^4}(s_{m}, t_{n}). \end{array} \end{equation}$

For $E_{2}(s, t)$ we get

$\begin{equation} |E_{2}(s, t)|\le \left|\frac{\partial^4 e}{\partial s^4}(s_{m}, x)+\frac{\partial^4 e}{\partial s^4}(s_{m}, t_{n})\right|\le C (h^{d_{s}-3}+\tau^{d_{t}+1}). \end{equation}$

(4.10)

Then, we have

$\begin{equation} |E_{1}(s, t)|\le \left|\frac{\partial e}{\partial t}(s_{m}, t)+\frac{\partial e}{\partial t}(s_{m}, t_{n})\right|\le C (h^{d_{s}+1}+\tau^{d_{t}}). \end{equation}$

(4.11)

Similarly, for $E_{3}(s, t)$ we have

$\begin{equation} \begin{array}{ll} E_{3}(s, t) = \frac{\partial^2 \phi}{\partial s^2}(s, t)- \frac{\partial^2 \phi}{\partial s^2}(s_{m}, t_{n}) = \frac{\partial^2 e}{\partial s^2}(s, t_{n})+ \frac{\partial^2 e}{\partial s^2}(s_{m}, t_{n}), \end{array} \end{equation}$

(4.12)

and

$\begin{equation} |E_{3}(s, t)|\le \left| \frac{\partial^2 e}{\partial s^2}(s, t_{n})+ \frac{\partial^2 e}{\partial s^2}(s_{m}, t_{n}) \right|\le C (h^{d_{s}-1}+\tau^{d_{t}+1}). \end{equation}$

(4.13)

For $E_{4}(s, t)$ we get

$\begin{equation} \begin{array}{ll} |E_{4}(s, t)|& = \left|\phi_{0}\frac{\partial \phi}{\partial s}- \phi_{0}(s_{m}, t_{n})\frac{\partial \phi}{\partial s}(s_{m}, t_{n})\right| \\ &\le \left|\frac{\partial e}{\partial t}(s_{m}, t)+\frac{\partial e}{\partial t}(s_{m}, t_{n})\right|\le C (h^{d_{s}+1}+\tau^{d_{t}}). \end{array} \end{equation}$

(4.14)

Combining (4.9) and (4.11)–(4.14) together, the proof of Theorem 2 is completed. □

5. Numerical examples

All the examples are carried on a computer with Intel(R) Core(TM) i5-8265U CPU @ 1.60 GHz 1.80 GHz operating system, 16 G radon access running memory and a 512 G solid state disk memory. All simulation experiments were realized by the software Matlab (Version: R2016a). In this part, two examples are presented to test the theorem.

Example 1. Consider the KS equation

$\frac{\partial \phi}{\partial t}+\gamma \frac{\partial^4 \phi}{\partial s^4}+\frac{\partial^2 \phi}{\partial s^2}+ \phi\frac{\partial \phi}{\partial s} = \varphi(s, t)$

with the condition is

$\phi(0, t) = 0, \phi(1, t) = 0,$

and

$\phi(s, 0) = \sin(2\pi s).$

$\phi_{ss}(0, t) = 0, \phi_{ss}(1, t) = 0,$

and

$\varphi(s, t) = e^{-t}\sin(2\pi s)[2\pi e^{-t}\cos(2\pi s)-1+16\pi^4-4\pi^2].$

The solution of the KS equation is

$\phi(s, t) = e^{-t}\sin(2\pi s).$

In Figures 1–3, errors of unform partition with direct linearization, partial linearization, Newton linearization for the KS equation are presented. In Figures 4–6, errors of non-uniform partition with direct linearization, partial linearization, Newton linearization for the KS equation are presented.

Figure 1. Errors of nonuniform partition by direct linearization with

$m = n = 19$ .

DownLoad: Full-Size Img PowerPoint

Figure 2. Errors of nonuniform partition by partial linearization with

$m = n = 19$ .

DownLoad: Full-Size Img PowerPoint

Figure 3. Errors of nonuniform partition by Newton linearization with

$m = n = 19$ .

DownLoad: Full-Size Img PowerPoint

Figure 4. Errors of uniform partition by direct linearization with

$m = n = 19$ .

DownLoad: Full-Size Img PowerPoint

Figure 5. Errors of uniform partition by partial linearization with

$m = n = 19$ .

DownLoad: Full-Size Img PowerPoint

Figure 6. Errors of uniform partition by Newton linearization with

$m = n = 19$ .

DownLoad: Full-Size Img PowerPoint

In Tables 1 and 2, errors of LBCM and LBRCM for the KS equation with boundary condition dealt with by the method of substitution and method of addition are given. From Table 1, we know that the accuracy of LBCM is higher than LBRCM, and from Table 2 the accuracy of the method of additional is higher than the method of substitution.

Table 1. Errors of LBCM for KS equation with

$m = n = 17$ .

	Method of substitution		Method of additional
Linearization	Uniform partition	Nonuniform partition	Uniform partition	Nonuniform partition
direct	1.3278e-07	5.6616e-10	1.7050e-08	4.6293e-10
partial	5.5563e-07	2.6381e-09	1.1492e-07	5.0974e-10
Newton	6.6705e-07	4.8875e-10	8.8609e-08	2.5867e-11

| Show Table

DownLoad: CSV

Table 2. Errors of LBRCM for KS equation with

$m = n = 17, d_{s} = d_{t} = 12$ .

	Method of substitution		Method of additional
Linearization	Uniform partition	Nonuniform partition	Uniform partition	Nonuniform partition
direct	4.4575e-06	3.2280e-08	4.1010e-08	2.2749e-09
partial	4.4573e-06	3.2245e-08	5.4191e-07	1.5951e-07
Newton	4.4560e-06	3.2215e-08	1.2972e-06	3.5137e-07

| Show Table

DownLoad: CSV

In , we choose the Newton linearization to solve the KS equation, and the error of LBRCM for uniform and nonuniform partitions are presented with $t = 0.3, 0.9, 2, 4, 8, 16$ .

Table 3. Errors of Newton linearization for

$t$ .

	Uniform partition		Nonuniform partition
$t$	$(8, 8) d_{s}=d_{t}=7$	$(16, 16) d_{s}=d_{t}=15$	$(8, 8) d_{s}=d_{t}=7$	$(16, 16) d_{s}=d_{t}=15$
0.3	1.5449e-01	1.3163e-06	6.2692e-02	2.4769e-08
0.9	1.4211e-01	1.1737e-06	6.1721e-02	2.3846e-08
2	1.2162e-01	1.0785e-06	5.8680e-02	2.3685e-08
4	9.1544e-02	9.4383e-07	5.3241e-02	2.3353e-08
8	5.1798e-02	7.2283e-07	4.3721e-02	2.2440e-08
16	1.6540e-02	4.1712e-07	2.9435e-02	1.9220e-08

| Show Table

DownLoad: CSV

The errors of LBRCM of uniform and Chebyshev partitions are presented with $(m, n, d_{s}, d_{t}) = (8, 8, 7, 7), (16, 16, 15, 15)$ . From the table, comparing $(m, n) = (8, 8)$ with $(m, n) = (16, 16)$ , the accuracy was higher when the number became bigger.

In the following table, we take Newton linearization to present numerical results. From and , with errors of Newton linearization for uniform partition $d_{t} = 6;t = 1$ are given and convergence rate is $O(h^{d_{s}})$ . From , with space variable $s, d_{s} = 6$ , and there is superconvergence rate $O(h^{d_{s}-1})$ at $t = 1$ .

Table 4. Errors of Newton linearization for uniform partition

$d_{t} = 6$ .

$m, n$	$d_{s}=2$	$h^{\alpha}$	$d_{s}=3$	$h^{\alpha}$	$d_{s}=4$	$h^{\alpha}$
8, 8	4.1317e-01		3.2652e-03		3.3180e-01
16, 16	1.8608e-01	1.1508	3.1257e-02	-	3.3919e-02	3.2902
32, 32	9.5437e-02	0.9633	1.0198e-02	1.6159	3.3873e-03	3.3239
64, 64	4.7221e-02	1.0151	2.6490e-03	1.9448	3.5472e-04	3.2554

| Show Table

DownLoad: CSV

Table 5. Errors of Newton linearization for uniform partition

$d_{s} = 6$ .

$m, n$	$d_{t}=2$	$\tau^{\alpha}$	$d_{t}=3$	$\tau^{\alpha}$	$d_{t}=4$	$\tau^{\alpha}$
8, 8	1.3997e-01		1.4004e-01		1.4008e-01
16, 16	5.4923e-03	4.6716	5.4957e-03	4.6714	5.4973e-03	4.6714
32, 32	1.2850e-04	5.4176	1.2883e-04	5.4148	1.2891e-04	5.4143
64, 64	2.9976e-06	5.4218	3.0728e-06	5.3898	3.0798e-06	5.3874

| Show Table

DownLoad: CSV

For and , the errors of Chebyshev partition for Newton linearization with $s$ and $t$ are presented. For $d_{t} = 6$ , the convergence rate is $O(h^{d_{s}})$ in Table 6, while in Table 7, there are also superconvergence phenomena.

Table 6. Errors of Newton linearization for Chebyshev partition

$d_{t} = 6$ .

$m, n$	$d_{s}=2$	$h^{\alpha}$	$d_{s}=3$	$h^{\alpha}$	$d_{s}=4$	$h^{\alpha}$
8, 8	5.4754e-01		2.9399e-02		8.5922e-02
16, 16	1.0318e-01	2.4078	4.6815e-03	2.6507	1.2658e-03	6.0849
32, 32	9.6912e-02	0.0904	8.0675e-04	2.5368	1.9577e-05	6.0148
64, 64	4.8014e-01	-	1.7672e-03	-	2.2716e-05	-

| Show Table

DownLoad: CSV

Table 7. Errors of Newton linearization for Chebyshev partition

$d_{s} = 6$ .

$m, n$	$d_{t}=2$	$\tau^{\alpha}$	$d_{t}=3$	$\tau^{\alpha}$	$d_{t}=4$	$\tau^{\alpha}$
8, 8	6.1344e-02		6.1386e-02		6.1415e-02
16, 16	8.1492e-05	9.5561	8.1163e-05	9.5629	8.0977e-05	9.5669
32, 32	1.4204e-07	9.1642	1.4183e-07	9.1606	1.5487e-07	9.0303
64, 64	6.3190e-06	-	3.8960e-06	-	1.4861e-06	-

| Show Table

DownLoad: CSV

Example 2. Consider the KS equation

$\frac{\partial \phi}{\partial t}+\gamma \frac{\partial^4 \phi}{\partial s^4}+\frac{\partial^2 \phi}{\partial s^2}+ \phi\frac{\partial \phi}{\partial s} = 0,$

with the analytic solution

$\phi(s, t) = c+\frac{15\sqrt{11}}{19\sqrt{19}}\left[-3\tanh \frac{\sqrt{11}}{2\sqrt{19}}(s-ct+s_{0}) +\tanh^3 \frac{\sqrt{11}}{2\sqrt{19}}(s-ct+s_{0})\right],$

and boundary condition

$\phi(-10, t) = c+\frac{15\sqrt{11}}{19\sqrt{19}}\left[-3\tanh \frac{\sqrt{11}}{2\sqrt{19}}(-10-ct+s_{0}) +\tanh^3 \frac{\sqrt{11}}{2\sqrt{19}}(-10-ct+s_{0})\right],$

$\phi(10, t) = c+\frac{15\sqrt{11}}{19\sqrt{19}}\left[-3\tanh \frac{\sqrt{11}}{2\sqrt{19}}(10-ct+s_{0}) +\tanh^3 \frac{\sqrt{11}}{2\sqrt{19}}(10-ct+s_{0})\right],$

and initial condition

$\phi(s, 0) = c+\frac{15\sqrt{11}}{19\sqrt{19}}\left[-3\tanh \frac{\sqrt{11}}{2\sqrt{19}}(s+s_{0}) +\tanh^3 \frac{\sqrt{11}}{2\sqrt{19}}(s+s_{0})\right],$

with $c = 2, x_{0} = 10$ .

In –, errors of direct linearization, partial linearization, Newton linearization with $m = n = 19$ KS equation are presented, respectively.

Figure 7. Errors of direct linearization with

$m = n = 19$ .

DownLoad: Full-Size Img PowerPoint

Figure 8. Errors of partial linearization with

$m = n = 19$ .

DownLoad: Full-Size Img PowerPoint

Figure 9. Errors of Newton linearization with

$m = n = 19$ .

DownLoad: Full-Size Img PowerPoint

In the following table, direct linearization is chosen to present numerical results. From and , errors of direct linearization for uniform partition $d_{t} = 7$ with different $d_{s}$ are given and the convergence rate is $O(h^{d_{s}}-1)$ . From , with space variable $s, d_{s} = 7$ , and there are also superconvergence phenomena.

Table 8. Errors of direct linearization for uniform partition for

$d_{t} = 7$ .

$m, n$	$d_{s}=2$	$h^{\alpha}$	$d_{s}=3$	$h^{\alpha}$	$d_{s}=4$	$h^{\alpha}$
8, 8	1.3587e+00		8.9361e-01		6.3703e-01
16, 16	2.1617e-01	2.6520	2.7467e-01	1.7019	2.5682e-01	1.3106
32, 32	6.7743e-02	1.6740	6.8822e-02	1.9967	4.7078e-02	2.4476
64, 64	2.5175e-02	1.4281	1.3216e-02	2.3806	4.3739e-03	3.4281

| Show Table

DownLoad: CSV

Table 9. Errors of direct linearization for uniform partition for

$d_{s} = 7$ .

$m, n$	$d_{t}=2$	$\tau^{\alpha}$	$d_{t}=3$	$\tau^{\alpha}$	$d_{t}=4$	$\tau^{\alpha}$
8, 8	3.6253e-01		3.6380e-01		3.6446e-01
16, 16	1.8147e-01	0.9984	1.8124e-01	1.0052	1.8121e-01	1.0081
32, 32	6.4076e-02	1.5019	6.4158e-02	1.4982	6.4141e-02	1.4983
64, 64	8.9037e-04	6.1692	8.9840e-04	6.1581	8.9863e-04	6.1574

| Show Table

DownLoad: CSV

For and , the errors of Chebyshev partition for direct linearization with $s$ and $t$ are presented. For $d_{t} = 7$ , the convergence rate is $O(h^{d_{s}})$ in Table 10, while in Table 11, there are also superconvergence phenomena.

Table 10. Errors of direct linearization for Chebyshev partition for

$d_{t} = 7$ .

$m, n$	$d_{s}=2$	$h^{\alpha}$	$d_{s}=3$	$h^{\alpha}$	$d_{s}=4$	$h^{\alpha}$
8, 8	6.5990e-01		4.0742e-01		3.6175e-01
16, 16	1.1154e-01	2.5646	1.7539e-01	1.2160	2.1752e-01	0.7338
32, 32	4.3052e-02	1.3735	8.6654e-03	4.3391	1.2511e-03	7.4418
64, 64	3.9204e-02	0.1351	2.3776e-03	1.8658	3.5682e-04	1.8099

| Show Table

DownLoad: CSV

Table 11. Errors of direct linearization for Chebyshev partition for

$d_{s} = 7$ .

$m, n$	$d_{t}=2$	$\tau^{\alpha}$	$d_{t}=3$	$\tau^{\alpha}$	$d_{t}=4$	$\tau^{\alpha}$
8, 8	4.3760e-01		4.3745e-01		4.3739e-01
16, 16	1.1801e-01	1.8908	1.1801e-01	1.8902	1.1801e-01	1.8900
32, 32	9.9842e-04	6.8850	9.9854e-04	6.8849	9.9801e-04	6.8857
64, 64	2.5749e-06	8.5990	2.5052e-06	8.6388	4.8401e-06	7.6879

| Show Table

DownLoad: CSV

6. Conclusions

In this paper, LBRCM is used to solve the (1+1) dimensional SK equation. Three kinds of linearization methods are taken to translate the nonlinear part into a linear part. Matrix equations of the discrete SK equation are obtained from corresponding linearization schemes. The convergence rate of LBRCM is also presented. In the future work, LBRCM can be developed for the (2+1) dimensional SK equation and other partial differential equations classes, including Kolmogorov-Petrovskii-Piskunov (KPP) equation and, fractional reaction-diffusion equation and so on.

Acknowledgments

The work of Jin Li was supported by the Natural Science Foundation of Shandong Province (Grant No. ZR2022MA003).

The authors also gratefully acknowledges the helpful comments and suggestions of the reviewers, which have improved the presentation.

Conflict of interest

The author declares no conflict of interest.

References

[1]	Boyd SP, Vandenberghe L (2004) Convex Optimization, Cambridge University Press, 129.
[2]	Tarantola A (2005) Inverse Problem Theory and Methods for Model Parameter Estimation, SIAM. https://doi.org/10.1137/1.9780898717921
[3]	Horst R, Tuy H (1996) Global Optimization: Deterministic Approaches, Springer.
[4]	Neumaier A (2004) Complete Search in Continuous Global Optimization and Constraint Satisfaction. Acta Numerica 13: 271–369. https://doi.org/10.1017/S0962492904000194 doi: 10.1017/S0962492904000194
[5]	Raschka S, Mirjalili V (2017) Python Machine Learning: Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow, PACKT Books.
[6]	Russell S, Norvig P (2016) Artificial Intelligence: A Modern approach, Pearson Education, Inc.
[7]	Ravichandiran S (2020) Deep Reinforcement Learning with Python, Packt Publishing.
[8]	Duan Y, Chen X, Houthooft R, et al. (2016) Benchmarking deep reinforcement learning for continuous control. ICML 48: 1329–1338. https://arXiv.org/abs/1604.06778
[9]	Ernst D, Geurts P, Wehenkel L (2005) Tree-based batch mode reinforcement learning. J Mach Learn Res 6: 503–556.
[10]	Geramifard A, Dann C, Klein RH, et al. (2015) RLPy: A Value-Function-Based Reinforcement Learning Framework for Education and Research. J Mach Learn Res 16: 1573–1578.
[11]	Lample G, Chaplot DS (2017) Playing FPS Games with Deep Reinforcement Learning. AAAI 2140–2146. https://doi.org/10.48550/arXiv.1609.05521 doi: 10.48550/arXiv.1609.05521
[12]	Littman ML (1994) Markov games as a framework for multi-agent reinforcement learning. Proc Eleventh Int Conf Mach Learn, 157–163. https://doi.org/10.1016/b978-1-55860-335-6.50027-1 doi: 10.1016/b978-1-55860-335-6.50027-1
[13]	Nagabandi A, Kahn G, Fearing RS, et al. (2018) Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning. ICRA, 7559–7566. https://doi.org/10.1109/ICRA.2018.8463189 doi: 10.1109/ICRA.2018.8463189
[14]	Ribeiro C, Szepesvári C (1996) Q-learning combined with spreading: Convergence and results. Proc ISRF-IEE Int Conf Intell Cognit Syst, 32–36.
[15]	Rücker C, Günther T, Wagner FM (2017) pyGIMLi: an open-source library for modelling and inversion in geophysics. Comput Geosci 109: 106–123. https://doi.org/10.1016/j.cageo.2017.07.011 doi: 10.1016/j.cageo.2017.07.011

This article has been cited by:

Jin Li, Yongling Cheng, Barycentric rational interpolation method for solving 3 dimensional convection–diffusion equation, 2024, 304, 00219045, 106106, 10.1016/j.jat.2024.106106

Reader Comments

Your name:*

Email:*
© 2022 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)