A feedback design for numerical solution to optimal control problems based on Hamilton-Jacobi-Bellman equation

1.   Introduction

Let $\mathbb R, \; \mathbb Z$, and $\mathbb N$ be the sets of all real numbers, integers, and natural numbers, respectively. Here and below, for $a, b\in \mathbb N$ with $a < b$, we use the notation $\mathbb Z(a, b): = \{a, a+1, \cdots, b\}$.

In recent years, difference equations as mathematical models to describe a variety of practical problems in the fields of economy, biology, disease prevention and control, environmental protection, etc., have attracted great attention and there have been a lot of outstanding works in theory and actual application; see, for example, ^[1,2].

In the paper, we consider the following discrete Dirichlet boundary value problem involving the mean curvature operator

where $\phi(s) = \frac{s}{\sqrt{1+ \kappa s^2}}, \kappa > 0$ is a constant ^[3], $T$ is a given positive integer, $\triangle$ is the forward difference operator defined by $\triangle u(k) = u(k+1)-u(k), \, \triangle^2u(k) = \triangle(\triangle u(k)), \, q(k)\geq 0$ for all $k\in\mathbb Z(1, T), \, \lambda$ is a real positive parameter, and $f(k, \cdot)\in C(\mathbb R, \mathbb R)$ for each $k\in\mathbb Z(1, T).$

When $q(k) = 0$ for all $k\in\mathbb Z(1, T)$, the problem (1.1) is the discrete analogy of one-dimensional prescribed mean curvature equations with Dirichlet boundary conditions

The existence of (positive) solutions of the above problem has been studied by using the variational method, the standard techniques from bifurcation theory, fixed point index, and Kresnoselskii's fixed point theorem (see ^[4,5,6] and the references therein).

Note that when $\kappa = 0$, the problem (1.1) degenerates into the classical second-order difference equation boundary value problem

The existence of (positive) solutions to the problem (1.2) has been well known with various qualitative assumptions of nonlinearity $f$ (^[7,8,9]).

When $\kappa = 1$, the problem (1.1) is the ordinary discrete mean curvature problem

which has aroused the interest of many scholars recently ^[10,11,12]. Their approaches are variational. The existence of (positive) solutions to the problem (1.3) depends on the behavior at zero or infinity of the potential $F(k, u) = \int_{0}^{u}f(k, s)ds$. In ^[10,11], the oscillating behavior of $F$ at $+\infty$ played an important role in obtaining an infinite number of positive solutions to the problem (1.3). Chen and Zhou ^[12] obtained at least three positive solutions of the problem (1.3) with $q(k) = 0$ for all $k\in\mathbb Z(1, T),$ where the primitive $F$ on the nonlinear datum satisfies that $\limsup_{s\rightarrow +\infty}{\frac{F(k, s)}{|s|}} < \alpha$ ($\alpha$ is a positive constant). Meanwhile, the case that the potential $F$ satisfies $\liminf_{s\rightarrow \infty}{\frac{F(k, s)}{|s|}} > \beta$ ($\beta$ is a positive constant) has been also discussed. They obtained at least two nontrivial solutions based on a two-critical-point theorem (Theorem 2.1) established in ^[13]. It is an important tool in obtaining at least two positive solutions of the Laplacian or the algebraic boundary value problems (see, for instance, ^[15,16]). However, it is relatively less used to look for positive solutions to the mean curvature boundary value problems.

Inspired by this, we will apply the two-critical-point theorem to establish the existence of at least two positive solutions for the problem (1.1) in this paper. As a special case of our main theorem, the existence of two positive solutions for the problem (1.1) with $\kappa = 1$ and all $q(k) = 0$ is obtained in Remark 4. The result improves Theorem 2 ^[12], where the author establishes the existence of only two nontrivial solutions without providing sign information for them under stronger hypotheses on the potential $F$. In Theorem 2 ^[12], the bilateral limit assumption on $\frac{F(k, s)}{|s|}$ at $\infty$ ensures that the energy functional of that problem is anticoercive, which consequently guarantees the establishment of the Palais-Smale condition—a pivotal requirement for applying critical point theorems. In our paper, the bilateral limit on the potential $F$ is weakened to a unilateral limit at $+\infty$, and then the energy functional of the problem (1.1) loses its anticoercivity. However, with the help of some inequality techniques, it is proven that the energy functional still satisfies the Palais-Smale condition. Moreover, in our main result (Theorem 3.1), the existence of at least two positive solutions is established without any asymptotic condition of the potential $F$ at $0$ and with no requiring that $f(k, 0) > 0$ for any $k\in \mathbb Z(1, T)$. In fact, the algebraic conditions in Theorem 3.1 are more general than the conditions that the potential $F$ is subquadratic at $0$ and superlinear at $+\infty$ (see Corollary 3.1).

In 2003, Guo and Yu ^[17] first used the variational method to study the periodic solutions of second-order difference equations. Since then, for nonlinear difference systems, many scholars have used the variational method to study the existence of various solutions, such as periodic solutions, subharmonic solutions, and homoclinic solutions ^[18,19,20]. For general references on difference equations and their applications, we refer the reader to monographs ^[21,22] and the references therein.

This paper is organized as follows: In section 2, some definitions and results are collected. Some estimations of equivalence of the norm are provided. Moreover, Lemma 2.1 is presented to guarantee us to obtain positive solutions rather than nontrivial solutions to the problem (1.1). Section 3 is devoted to our main result. Lemma 3.1 is given to ensure the Palais-Smale condition of the functional on this basis. Some consequences of our main result are presented together with three examples.

2.   Preliminaries

2.1.   Relevant notations and inequalities

Consider the $T$ dimensional Banach space

endowed with the norm $\|\cdot\|$ as

Moreover, the space $S$ can also be equipped with the following equivalent norms:

respectively. It is easy to know that for all $u\in S$,

By Theorem 12.6.1 in ^[21], we have for all $u\in S$,

where $\lambda_j = 4\sin^2{j\pi/2(T+1)}$ for all $j\in \mathbb Z(1, T).$ By virtue of Lemma 2.2 in ^[23], we get for all $u\in S$,

Note that $v(0) = v(T+1) = 0$ for all $v\in S$, we have

for all $u, v\in S$.

2.2.   Variational framework

Let

for all $s\in \mathbb R.$

Remark 2.1. In fact, the nonnegative solution of the following problem

is the nonnegative solution of the problem (1.1). To obtain positive solutions to the problem (1.1), we only need to search for positive solutions to the problem (2.5) now.

Consider two functionals $\Phi$ and $\Psi$ defined respectively on $S$ by

and

$F^+(k, s) = \int_0^sf(k, t^+)dt$ for all $(k, s)\in \mathbb Z(1, T)\times\mathbb R$. Obviously,

for all $k\in\mathbb Z(1, T),$ and $\Phi, \Psi\in C^1(S, \mathbb R)$, which means that $\Phi$ and $\Psi$ are two continuously Gâteaux-differentiable functionals defined on $S$. By (2.4), we have that for any $u, v\in S$,

and

We define $I_\lambda:S\rightarrow \mathbb R$ by

Clearly, $I_\lambda\in C^1(S, \mathbb R)$ and for any $u, v\in S$,

Remark 2.2. For any $u, v\in S$, $I_\lambda'(u)(v) = 0$ if and only if

for all $u\in S$ and $k\in \mathbb Z(1, T)$. In other words, a critical point of $I_\lambda$ on $S$ corresponds to a solution to the problem (2.5).

For the problem (1.1), we are interested in the existence of positive solutions rather than nontrivial ones. To the end, we present an assumption on $f(k, 0)$ as follows.

$(H_1)$ $f(k, 0)\ge 0, \quad \forall k\in \mathbb Z(1, T)$.

Lemma 2.1. If $(H_1)$ holds, then any nonzero critical point of the functional $I_\lambda$ on $S$ is a positive solution to the problem (1.1).

Proof. In fact, taking Remarks 2.1 and 2.2 into account, it is sufficient to verify that any nontrivial solution $u$ of the problem (2.5) is positive, that is, $u(k) > 0$ for all $k\in\mathbb Z(1, T).$

Notice that $\phi$ is a strictly monotonically increasing and odd function on $\mathbb R$. By standard computation, we can conclude that

for all $k\in\mathbb Z(1, T+1)$. By a direct computation, we get

for all $k\in\mathbb Z(1, T)$. For any nontrivial solution $u$ of the problem (2.5), combining Eqs (2.4), (2.8) and (2.9) with assumption $(H_1)$, we obtain that

Hence $\triangle u^-(0) = \triangle u^-(1) = \cdots = \triangle u^-(T) = 0$. Recalling that $u(0) = u(T+1) = 0$ for any $u\in S$, we have $u^-(k) = 0$ for all $k\in\mathbb Z(1, T)$. Thus $u$ is nonnegative.

Next, we further prove that $u$ is positive. Otherwise, there exists some $k_0\in\mathbb Z(1, T)$ such that $u(k_0) = 0$, then

Clearly, $\phi(\triangle u(k_0))\le \phi(\triangle u(k_0-1))$. Because $\phi$ is a strictly monotonically increasing homomorphism, we get $\triangle u(k_0)\le \triangle u(k_0-1)$. Hence, $u(k_0+1)+u(k_0-1)\le 0$. As a result, $u(k_0\pm 1) = 0$. Then iterating the process, we have that $u(k) = 0$ for all $k\in\mathbb Z(1, T)$. In short, $u$ is zero somewhere in $\mathbb Z(1, T)$. Then it is zero identically. This contradicts the nontriviality of $u$. The proof is ended. □

Let $(X, \|\cdot\|)$ be a real Banach space and $\varphi\in C^1(X, \mathbb R)$. $\varphi$ is said to satisfy the Palais-Smale condition ((PS) condition), if any sequence $\{u_n\}\subset X$ for which $\{\varphi(u_n)\}$ is bounded and $\varphi^{\prime}(u_n)\rightarrow 0$ as $n\rightarrow \infty$ possesses a convergent subsequence in $X$.

The following two-critical-point theorem is given by Bonanno and D'Aguì in 2016 ^[13].

Theorem 2.1. ^[13] Let $X$ be a real Banach space, and let $\Phi, \Psi:X\rightarrow \mathbb R$ be two functionals of class $C^1$ such that $\inf_{X}\Phi = \Phi(0) = \Psi(0) = 0.$ Assume that there are $r\in\mathbb R$ and $\tilde{u}\in X$, with $0 < \Phi(\tilde{u}) < r$, such that

for each

the functional $I_\lambda = \Phi-\lambda\Psi$ satisfies the (PS) condition, and it is unbounded from below. Then for each $\lambda\in\Lambda$, the functional $I_\lambda$ admits at least two nonzero critical points $u_{\lambda, 1}, \, u_{\lambda, 2}$ such that $I_\lambda(u_{\lambda, 1}) < 0 < I_\lambda(u_{\lambda, 2})$.

3.   Result and discussion

3.1.   The Palais-Smale condition

Let

where $F(k, s) = \int_0^sf(k, t)dt$ for all $(k, s)\in \mathbb Z(1, T)\times\mathbb R$. Here and below, when $L_\infty = 0$, we think $\frac1{L_\infty} = \infty$. Before we come to a conclusion, let's give a lemma on the (PS) condition.

Lemma 3.1. If $L_\infty > 0$ and $(H_1)$ hold, then $I_\lambda$ satisfies the $(PS)$ condition, and it is unbounded from below for all $\lambda\in \left(\frac{\sqrt{(T+1)\lambda_T}+Q}{\sqrt{ \kappa}L_\infty}, \, +\infty\right)$, where $Q: = (\sum_{k = 1}^{T} |q(k)|^2)^{\frac{1}{2}}$.

Proof. Because $S$ is finite dimensional, it is sufficient to show that any (PS) sequence of $I_\lambda$ is bounded on $S$. Let $\{u_n\}\subset S$ be a sequence such that $\{I_\lambda(u_n)\}$ is bounded and $I_\lambda^{\prime}(u_n)\rightarrow 0$ as $n\rightarrow \infty$.

First, we claim that $\{u^-_n\}$ is bounded. According to (2.8) and (2.9), we have for all $n\in \mathbb N$ and $k\in\mathbb Z(1, T+1)$,

and

By (3.1) and (3.2), we can estimate the derivative of $\Phi$ at $u_n$ in the direction of $-u_n^-$ that

To put it simply,

And, from the condition $(H_1)$, it follows that

Combining (3.3) and (3.4), we have

for all $\lambda > 0$ and $n\in \mathbb N$. Moreover, by the definition of the functional $\Phi$, we obtain, for all $u\in S$,

which implies that

for all $u\in S$. Thus, it is clear that for all $u\in S$,

So combining (3.5) and the above inequality, we have

Also, from $\lim_{n \to \infty}I^{\prime}_\lambda(u_n) = 0$, it follows that

As a result, we obtain

By standard computation, the above equation is transformed into

which means that $\lim_{n \to \infty}\|u_n^-\| = 0$. Hence our claim is proved. So there is $M > 0$ such that $0\le u_n^-(k)\le M$ for all $k\in \mathbb Z(1, T)$ and $n\in \mathbb N.$

Next, we prove that $\{u_n \}$ is bounded. If $\{u_n \}$ is not bounded, we may assume, going if necessary to a subsequence, that $\|u_n\|\rightarrow \infty (n\rightarrow \infty)$. Taking $L_\infty > 0$ into account, we fix $\lambda > \frac{\sqrt{(T+1)\lambda_T}+Q}{\sqrt{ \kappa}L_\infty}$ and fix $l = l(\lambda)$ such that for all $k\in \mathbb Z(1, T)$,

then for all $k\in \mathbb Z(1, T)$, there is a $\delta_k > 0$ such that

Meanwhile, for all $k\in \mathbb Z(1, T)$ and $s\in [-M, \delta_k]$,

Hence, for all $k\in \mathbb Z(1, T)$,

On account of (2.1), we obtain that for all $n\in \mathbb N$,

where $\eta = \sum_{k = 1}^{T}\eta(k)$. Using the Cauchy-Schwarz inequality, we get that for all $n\in \mathbb N$,

Therefore, from the previous two inequalities and (2.2), it follows that for all $n\in \mathbb N$,

When $\|u_n\|\rightarrow +\infty (n\rightarrow \infty)$, by $\frac{\sqrt{(T+1)\lambda_T}+Q}{\sqrt{ \kappa}}-\lambda l < 0$, we have $\lim_{n\rightarrow \infty}I_\lambda(u_n) = -\infty$. This leads to a contradiction. Hence $\{u_n\}$ is bound and $I_\lambda$ satisfies the (PS) condition.

Finally, we prove that $I_\lambda$ is unbounded from below. Let $\{u_n\}$ be such that $u_n = u_n^+$ for any $n\in\mathbb N$ and $\|u_n^+\| \to\infty \, (n\to \infty)$. Arguing as before, we obtain that

So $\lim_{n\rightarrow \infty}I_\lambda(u_n) = -\infty$. Our conclusion follows. □

3.2.   Main result

Below, our main result is presented.

Theorem 3.1. Suppose that $(H_1)$ holds and there exist two positive constants $c$ and $d$ with $d < c$ such that

and

Then for each

the problem (1.1) possesses at least two positive solutions, where $q: = \sum_{k = 1}^{T}q(k)$.

Proof. By Lemma 2.1, it is enough to prove that $I_\lambda$ has at least two nonzero critical points. We apply Theorem 2.1 by putting $X = S, \, I_\lambda = \Phi-\lambda\Psi$ where $\Phi, \, \Psi$ are the functions introduced in (2.6) and (2.7).

Clearly, $\inf_{S}\Phi = \Phi(0) = \Psi(0) = 0$. Also notice that $F^+(k, s) = F(k, s)$ for all $(k, s)\in \mathbb Z(1, T)\times [0, +\infty)$. From the conditions $(H_1), \, (H_2)$ and $(H_3)$, it follows that $L_\infty > 0$ and $\Lambda ^+$ is non-degenerate. Then Lemma 3.1 ensures that the function $I_\lambda$ satisfies the (PS) condition and it is unbounded from below for $\lambda\in \Lambda^+$.

For fixed $\lambda\in \Lambda^+$, there exists $c > 0$ such that $\lambda \le \frac{-\frac{1}{ \kappa}+\sqrt{\frac{4c^2}{ \kappa (T+1)}+\frac{1}{ \kappa^2}} } {\sum_{k = 1}^{T} \max_{s\in [0, c]} F(k, s) }.$ Put $r: = -\frac{1}{ \kappa}+\sqrt{\frac{4c^2}{ \kappa (T+1)}+\frac{1}{ \kappa^2}}$. We claim that

If $\Phi(u)\le r$, by (3.6), we get that $\|u\|^2\le \kappa \Phi^2(u)+2\Phi(u)\le \kappa r^2+2r = \frac{4c^2}{T+1}$. Thus, from the inequality (2.3), it is clear that

The assertion is verified. Therefore, we obtain

Now, we look for $\tilde{u}\in S$ in Theorem 2.1. Define $\tilde{u} \in \mathbb R^{T+2}$ as $\tilde{u}(k) = d\, (0 < d < c)$ for all $k\in \mathbb Z(1, T)$ and $\tilde{u}(0) = \tilde{u}(T+1) = 0$. Clearly, $\tilde{u}\in S$. It is easy to see that

and hence we get

Therefore, from (3.7), (3.9), and $(H_3)$, it follows that

and $\Lambda^+\subset \Lambda$. Moreover, since $0 < d < c$, and again by virtue of the condition $(H_3)$, we obtain that

On account of (3.8) and (3.10), we get that $0 < \Phi(\tilde{u}) < r$. Thus, (2.10) holds. Hence Theorem 2.1 ensures that $I_\lambda$ admits two nonzero critical points. This completes the proof. □

Remark 3.1. Indeed, condition $(H_1)$ implicitly covers both $f(k, 0) > 0$ and $f(k, 0) = 0$ cases. Theorem 3.1 holds true under either sub-case of condition $(H_1)$.

Remark 3.2. If all $f(k, \cdot)$ are nonnegative on $[0, c](c > 0)$, then $\max_{s\in [0, c]}F(k, s) = F(k, c)$ for all $k\in \mathbb Z(1, T)$. According to Theorem 3.1, it is enough to assume that there is a positive constant $d$ with $d < c$ such that

Remark 3.3. When $\kappa = 1$ and $q(k) = 0$ for all $k\in \mathbb Z(1, T)$ ($Q = 0$ and $q = 0$), the conditions in Remark 3.2 are more general than those of Theorem 2 in ^[12], where $f(k, s)$ is required to be positive for all $(k, s)\in \mathbb Z(1, T)\times [-c, c]$ rather than nonnegative, and the potential $F$ is assumed to possess asymptotic behavior not only at $+\infty$ but also at $-\infty$. Here, the two solutions we obtain are positive, while the solutions in Theorem 2 of ^[12] are only nontrivial. Obviously, our main result improves Theorem 2 in ^[12].

Now, we present two particular cases of Theorem 3.1.

Corollary 3.1. Assume that $(H_1)$ holds,

and

for all $k\in\mathbb Z(1, T)$. Then for each $\lambda\in(0, \lambda^*)$, where

the problem (1.1) admits at least two positive solutions.

Proof. Let $\lambda\in(0, \lambda^*)$ and $c > 0$ such that

Taking (3.11) into account, we get that $\limsup_{s\rightarrow 0^+} \frac{F(k, s)}{\sqrt{1+ \kappa s^2}-1 } = +\infty$. As a result, there is $d > 0$ with $d < c$ such that $\frac{ \kappa}{2+q}\frac{\sum_{k = 1}^{T} F(k, d)}{\sqrt{1+ \kappa d^2}-1} > \frac{1}{\lambda}$. Note that $L_\infty = +\infty$, so Theorem 3.1 ensures the conclusion. □

Corollary 3.2. Assume that $f(k, \cdot) = f$ for all $k\in\mathbb Z(1, T)$ and $f$ is a continuous function such that

and

Then, for each

the problem (1.1) admits at least two positive solutions.

Proof. In fact, Corollary 3.2 is a consequence of Corollary 3.1. It is easy to see that (3.13) implies $f(0)\geq 0$, and (3.11) and (3.12) can be derived from (3.13) and (3.14), respectively. □

Example 3.1. For each $\lambda \in \left(0, \frac{-\frac{1}{ \kappa}+\sqrt{\frac{4}{ \kappa (T+1)}+\frac{1}{ \kappa^2}}} {(3e-1)T }\right)$, the problem

admits at least two positive solutions. Here, the nonlinearity $f(t) = e^{\sqrt[3]{t}} -1$ satisfies the conditions of Corollary 3.2 and

The image of $f(t)$ is shown in Figure 1.

Example 3.2. For each $\lambda \in \left(0, \frac{-\frac{1}{ \kappa}+\sqrt{\frac{4}{ \kappa (T+1)}+\frac{1}{ \kappa^2}}} { T(T+1) } \right)$, the problem

admits at least two positive solutions. It is easy to see that $f(k, t) = k\sqrt{t}(\cos t +2)$ satisfies the assumptions of Corollary 3.1 and

Figure 2 displays the functional plot of $f(k, t)$ for $k\in (0,100)$.

Remark 3.4. We observe that Theorem 2 in ^[12] cannot be applied in the previous two examples with $\kappa = 1$ and $q(k) = 0$ for all $k\in \mathbb Z(1, T)$, since $f(k, 0) > 0$ is assumed there.

Example 3.3. Consider the boundary value problem (1.1) with $\kappa = 1, q(k) = 0$ and

for all $k\in\mathbb Z(1, T)$. Clearly, if $t\in (0, \frac{4}{3})$, then $f(t) > 0$; and if $t\in(\frac{4}{3}, 2\pi)$, then $f(t) < 0$. Such a relationship is visually apparent in Figure 3. A straightforward calculation yields

for all $k\in\mathbb Z(1, T)$.

Obviously, $L_{\infty}(k) = \liminf_{s\rightarrow +\infty}\frac{F(k, s)}{s} = 2\pi-1$ for all $k\in\mathbb Z(1, T)$, then

Letting $T = 3, c = 5$, and $d = 1$, we obtain

and

It follows that

and

From $f(k, 0) = f(0) = 2 > 0$ and the preceding relations, all conditions of Theorem 3.1 hold. Thus for each $\lambda\in \left(\frac{\sqrt{(T+1)\lambda_T}}{L_{\infty}}, \frac{-1+\sqrt{\frac{4c^2}{T+1}+1}}{\sum_{k = 1}^{T} \max_{s\in [0, c]} F(k, s) } \right)\approx (0.687, 1.034)$, the boundary value problem (1.1) has at least two positive solutions.

Remark 3.5. The potential function $F$ in the above example lacks superlinearity at $+\infty$, failing to satisfy the conditions of Corollaries 3.1 and 3.2. Nevertheless, Theorem 3.1 still guarantees the existence of two positive solutions for the problem (1.1) in Example 3.3.

4.   Conclusions

In this paper, we consider a generalized difference mean curvature problem, which includes the conventional one and the classical second-order difference equation boundary value problem. The main novelties of this research are as follows:

(1) The existence of two positive solutions rather than nontrivial solutions for our problem is established based on a two-zero critical points theorem and some inequality techniques.

(2) Under the assumption of the unilateral limit of $\frac{F(k, s)}{|s|}$ at $+\infty$ on the potential $F(k, s) = \int_0^sf(k, t)dt$ instead of the bilateral limit at $\infty$, it is proved that without anticoercivity the energy functional associated with our problem still satisfies the Palais-Smale condition that plays a key role in the critical point theorem.

(3) Our principal result can be applied to cases with nonlinear terms where $f(k, 0) > 0$ but also to cases where the nonlinear terms satisfy $f(k, 0) = 0$.

(4) It is also worth mentioning that the algebraic conditions in our main result are more general than the subquadraticity at $0$ and the superlinearity at $+\infty$ for the potential $F$.

In fact, due to the previous (1)–(3), our main result extends Theorem 2 in ^[12].

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

This work is supported by the National Nature Science Foundation of China (No.10901071, No.11501054, and No.12301186).

Conflict of interest

The authors declare there is no conflicts of interest.