Nash equilibrium strategies for non-zero-sum differential games of SDEs with time-varying coefficient and infinite Markov jumps

1.   Introduction

Applications of Markov jump stochastic systems have been found in a variety of fields, such as robotics, economics, and fault detection. For Markov jump stochastic systems, a sample of problems can be found in the literature, such as stability and stabilization, see ^{[1,2,3,4,5,6]}, reinforcement learning-based optimization, see ^[7,8], $H_2$ optimal control, see ^[9,10], $H_\infty$ control, see ^{[11,12,13,14]}, $H_2/H_\infty$ control, see ^[15,16], and game problem, see ^[17,18,19]. Recalling some existing results, most of them take values in finite state space for Markov chains, while few are based on the assumption that Markov chains are valued in countable state space. Thus, this is a significant research topic.

From an applicable point of view, countable Markov chain may be better suited to describe sudden changes in many practical scenarios such as modern queueing theory, solar thermal receivers, and so on ^[20]. From a theoretical point of view, in terms of stability, stochastic systems with finite or infinite Markov jumps are fundamentally different. The essential root lies in the fact that the causal and anticausal Lyapunov operators of infinite Markov jump systems are no longer adjoint. As a special hybrid system, infinite Markov jump stochastic systems contain two kinds of mixed dynamic forms. One is called mode, which is described by a Markov process with countable discrete states. The other, called the state, is described by stochastic differential equations (SDEs) for each mode. To be specific, ^[21] clarified the relationship among four kinds of stability for stochastic systems with countable Markov chains. Further, ^[22] took into account the effect of time delay and parametric uncertainties and also summarized the relationship among the above four stabilities. On this basis, some controller synthesis problems have been solved in ^[23,24,25]. Therefore, it is of significance both in theory and in practice to consider stochastic systems with countable Markov chains.

The differential games are widely applied in many fields, such as engineering, finance, and biology. For a special case where the state equations are linear and the payoff functionals are quadratic, ^[26] considered the linear quadratic (LQ) stochastic zero-sum differential game for the Markov jump system driven by Brownian motion and obtained a linear feedback saddle point characterized by the set of coupled Riccati differential equations. Under a more general functional, both open-loop solvability and closed-loop solvability are discussed in ^[27], and the solvability of associated Riccati equations under the uniform convexity-concavity condition has been studied. Besides, the differential equation characterizations of the lower and upper value functions of the game under a rather general setup are obtained in ^[28]. Further, ^[29] extended the differential games for Markov jump-diffusion models to the leader-follower Stackelberg game framework. For non-zero-sum differential games, games of regime-switching diffusions with mean-field interactions were concerned in ^[30]. However, for such systems with countable Markov chains, there are few results reported on the Nash game problem. In ^[31], although a unified treatment approach for the three control design problems, that is, $H_{2}$, $H_{\infty}$, and $H_{2}/H_{\infty}$ control, is presented via Nash equilibrium solution, a countable Markov chain is not involved. In ^[32], the Nash game problem is studied, while the effect of the control term on noise is neglected, which is focused on revealing the relationship between Nash equilibrium strategies and $H_2/H_\infty$ control. This is basically different from the starting point of our research that emphasizes a unified treatment for the three control problems. In ^[33], although the finite horizon $H_2/H_\infty$ controller design was investigated for a system considering countable Markov chain, the Nash equilibrium points and finite horizon $H_2/H_\infty$ controller design were not equivalent for system (2.1); see Remark 4.3. Hence, it is necessary to investigate the non-zero-sum Nash differential games for SDEs involving time-varying coefficient and infinite Markov jumps, which is the main motivation of the paper.

In this paper, we will discuss the non-zero-sum Nash differential games for SDEs involving time-varying coefficients and infinite Markov jumps, in which the Markov chain takes values in countable state space. The contribution of our paper rests on four aspects: First of all, since finite and infinite Markov jump systems have the essential difference on stability, the countable dimension Banach spaces are introduced, and their elements are linear and bounded operators. Next, with the tool of stochastic analysis, the Nash equilibrium strategies can be obtained by coupled generalized differential Riccati equations (CGDREs), which are a countable coupled Riccati equations, and this makes the equations more difficult to handle than those in ^[31,32]. Specifically, for the existence of Nash equilibrium strategies, a necessary and sufficient condition is given based on the pseudo inverse matrix. Once more, to demonstrate the above game result's theoretical value, a unified treatment for the three control problems is presented with some corresponding parameters. Last but not least, to overcome the difficulty of solving the CGDREs analytically, the discretization method and backward recursive algorithm are applied to solve CGDREs approximately.

The structure of this article is as follows: Preliminary discussions are included in Section 2. We show in Section 3 that Nash equilibrium strategies can be obtained by the CGDREs. Based on this result, in Section 4, a unified treatment is presented for the three control problems. In Section 5, one numerical example is given. Section 6 describes the conclusions.

The following symbols are used. $\mathcal{R}^{n} (\mathcal{R}^{l \times m})$ is $n$-dimensional real Euclidean space (the linear space of all $l$ by $m$ real matrices). $A'$ and $A^\dagger$ stand for the transpose and pseudo-inverse of the matrix $A$, respectively. The totality of $\mathcal{P}$-null sets is denoted by $\mathcal{N}$, $\mathcal{F}_{\varsigma}: = \sigma(w(s), 0 \leq s \leq \varsigma)\vee\sigma(\varpi(s), 0 \leq s \leq \varsigma)\vee\mathcal{N}.$ $l^{2}([0, T];\mathcal{R}^{l}) = \{e\in{R}^{l}|e$ is $\mathcal{F}_{\varsigma}-$measurable and $\int^{T}_{0}E\|e(\varsigma)\|^{2}d\varsigma < \infty\}$. $\mathcal{D}: = \{1, 2, \ldots\}$.

2.   Preliminaries

Consider the following linear SDEs with time-varying coefficients and infinite Markov jumps:

where $x(0) = x_0\in \mathcal{R}^n$, $\varpi(0) = \varpi_0\in \mathcal{D}$, $x(\varsigma)\in\mathcal{R}^{n}$ is on behalf of the system state, $z(\varsigma)\in\mathcal{R}^{n_{z}}$ stands for the measurement output, and for two different players, $\eta(\varsigma)\in\mathcal{R}^{n_{\eta}}$ and $\sigma(\varsigma)\in\mathcal{R}^{n_{\sigma}}$ are the control processes, respectively. $w(\varsigma)$ represents a standard one-dimensional Brownian motion. $\{\varpi_\varsigma\}_{\varsigma\in [0, T]}$ takes values in the countable state space $\mathcal{D}$, which is a right continuous and homogeneous Markov process. $\mathbf{P} = [p_{\aleph j}(\varsigma)]$ is the transition probability matrix of $\{\varpi_\varsigma\}_{\varsigma\in [0, T]}$ with $p_{\aleph j}(\varsigma) = P(\varpi_{s+\varsigma} = j|\varpi_{s} = \aleph)$, which is assumed to be stationary. The infinitesimal matrix of $\{\varpi_ \varsigma \}_{ \varsigma \in [0, T]}$ is defined as $\Phi = (\phi_{\aleph j})_{ \aleph, j\in\mathcal{D}}$, where $\phi_{\aleph j} = \underset{\varsigma\rightarrow 0}{\lim} \frac{p_{\aleph j}(\varsigma)-p_{\aleph j}(0)}{\varsigma}, \ \ p_{\aleph j}(0) = \delta_{(\aleph-j)}, \ \ \aleph, j\in\mathcal{D}.$ It should be noted that when $\aleph\neq j$, $\phi_{\aleph j}\geq 0$, and for $\aleph\in\mathcal{D}$ and some $c > 0$, $0\leq -\phi_{\aleph\aleph} = \sum_{j\in\mathcal{D}, j\neq \aleph}\phi_{\aleph j} < c < \infty$. Added to that, $\mathbf{P}$ is nondegenerate, and for $s\in [0, T]$, $\pi_{s}(\aleph): = P(\varpi_{s} = \aleph) > 0$, $\aleph\in\mathcal{D}$.

$\mathbb{A}_{1}^{m \times n}$ ($\mathbb{A}_{\infty}^{m \times n}$) represents the real Banach space of $\{A|A = (A(1), A(2), \cdots), A(\aleph) \in \mathcal{R}^{m \times n}\}$ with the norm $\|A\|_{1} = \sum^{\infty}_{\aleph = 1}\|A(\aleph)\| < \infty$ ($\|A\|_{\infty} = \sup_{\aleph\in\mathcal{D}}\|A(\aleph)\| < \infty$). When $m = n$, $\mathbb{A}_{1}^{m \times n}$ ($\mathbb{A}_{\infty}^{m \times n}$) will be expressed as $\mathbb{A}_{1}^{n}$ ($\mathbb{A}_{\infty}^{n}$). For $A \in \mathbb{A}_{1}^{n+}$($\mathbb{A}_{\infty}^{n+}$), $A \geq 0$ if and only if $A(\aleph) \geq 0$ for all $\aleph \in \mathcal{D}$. $A\in \mathbb{\widetilde{A}}_{1}^{n+}$($\mathbb{\widetilde{A}}_{\infty}^{n+}$) means $A > 0$. By $C^1([0, T], \mathbb{A}_{\infty}^{n+})$ ($C_b([0, T], \mathbb{A}_{\infty}^{n+})$), we denote all continuously differentiable (bounded) mappings $g$, and by $C^1_b([0, T], \mathbb{A}_{\infty}^{n+})$, we denote all bounded mappings $g(\varsigma)$ and $\frac{dg(\varsigma)}{d\varsigma}$.

Given two parameters with $\alpha > 0$ and $\beta\geq0$, the relevant cost functionals are defined as follows:

We will look for the optimal Nash equilibrium strategies $(\eta^*(\cdot), \sigma^*(\cdot))$ to minimize cost functionals (2.2) and (2.3) subject to system (2.1).

Definition 2.1. For all admissible $(\eta(\cdot), \sigma(\cdot))\in l^{2}([0, T];\mathcal{R}^{n_\eta})\times l^{2}([0, T];\mathcal{R}^{n_{\sigma}})$, if

then $(\eta^*(\cdot), \sigma^*(\cdot))\in l^{2}(N_T; \mathcal{R}^{n_{\eta}})\times l^{2}(N_T; \mathcal{R}^{n_{\sigma}})$ is called the Nash equilibrium strategy.

Lemma 2.1 (^[33]). For system (2.1) with $D_l(\varsigma, \varpi_\varsigma) = 0, E_l(\varsigma, \varpi_\varsigma) = 0, l = 1, 2$, if $G(\varsigma, \varpi_\varsigma)\in C^1_b([0, T], \mathbb{A}_{\infty}^{n+})$, then we yield

for $(x_0, \aleph)\in \mathcal{R}^n\times \mathcal{D}$.

Lemma 2.2 (^[36]). Let matrices $T_1$, $T_2$, $T_3$, and $F$ be given appropriate sizes. Then the matrix equation $T_1YT_2 = T_3$ admits a solution $Y$ if and only if $T_1T_1^{\dagger}T_3T_2^{\dagger}T_2 = T_3.$ Moreover, the solution can be represented by $Y = T_1^{\dagger}T_3T_2^{\dagger}+F-T_1^{\dagger}T_1FT_2T_2^{\dagger}$.

3.   Nash equilibrium strategies

The purpose of this section is to get the necessary and sufficient conditions for the existence of Nash equilibrium strategies. For this end, suppose that the feedback strategies can take the following form ^[37]:

Theorem 3.1. Under hypothetical conditions that $C_m(\varsigma, \varpi_\varsigma)\in C_b([0, T], \mathbb{A}_{\infty}^{n})$, $D_m(\varsigma, \varpi_\varsigma)\in C_b([0, T], \mathbb{A}_{\infty}^{n\times n_\eta})$, $E_m(\varsigma, \varpi_\varsigma)\in C_b([0, T], \mathbb{A}_{\infty}^{n\times n_{\sigma}})$, $m = 1, 2$, $A(\varsigma, \varpi_\varsigma)\in C_b([0, T], \mathbb{A}_{\infty}^{n_z\times n})$ in (2.1), for Nash game problem (2.4) and (2.5), a unique Nash equilibrium strategy

exists iff the following CGDREs:

admit solutions $(G_{1}(\varsigma, \aleph), G_{2}(\varsigma, \aleph))$ for $(\varsigma, \aleph)\in [0, T]\times\mathcal{D}$, where

Proof. Sufficiency: Because CGDREs (3.2) and (3.3) have solutions $G_{1}(\varsigma, \aleph)\leq0$, $G_{2}(\varsigma, \aleph)\geq0,$ $(\varsigma, \aleph)\in [0, T]\times\mathcal{D}$, we can infer from $\Gamma_2(\varsigma, \varpi_\varsigma) = -M_{2}(\varsigma, \varpi_\varsigma)^{\dagger}H_{2}(\varsigma, \varpi_\varsigma)'$ that $\eta(\varsigma)$ can be substituted by $\eta^{*}(\varsigma) = \Gamma_2(\varsigma, \varpi_\varsigma)x(\varsigma)$ in system (2.1)

By now applying the method of completing the square, it can be obtained from Lemma 2.1 and Eq (3.2) that

where $\sigma^*(\varsigma) = \Gamma_1(\varsigma, \varpi_\varsigma)x(\varsigma) = -M_1(\varsigma, \varpi_\varsigma)^{\dagger}H_{1}(\varsigma, \varpi_\varsigma)'x(\varsigma)$, $\pi_{0}(\aleph) = G_1(\varpi_0 = \aleph)$, $\aleph\in\mathcal{D}$. This means the Nash equilibrium strategies inequality (2.4) of Definition 2.1 is valid.

In order to illustrate the other Nash equilibrium strategies inequality (2.5), put $\sigma^*(\varsigma) = \Gamma_1(\varsigma, \varpi_\varsigma)x(\varsigma)$ into system (2.1), and we can obtain

Then, with such restrictive conditions attached to (2.5), we can compute

To move forward a single step, a combination of Lemma 2.1 and system (3.6) causes

Via Eq (3.3), it can be received from completing the square that

where $\eta^{*}(\varsigma) = \Gamma_2(\varsigma, \varpi_\varsigma)x(\varsigma) = -M_2(\varsigma, \varpi_\varsigma)^{\dagger}H_{2}(\varsigma, \varpi_\varsigma)'x(\varsigma)$. So far, there exist Nash equilibrium strategies $(\eta^*(\cdot), \sigma^*(\cdot))$ for system (2.1).

Necessity: For Nash game problems (2.4) and (2.5), make use of the definition of (2.4). We discover that $\sigma^*(\varsigma) = \Gamma_1(\varsigma, \varpi_\varsigma)x(\varsigma)$ solves the following LQ optimal control problem:

In fact, it is an indefinite problem on account of

Of course, the above problem is well-posed, as it should be. Subsequent work will state that the well-posed indefinite LQ optimal control problem (3.10) leads to the following CGDREs:

admit a solution $\widetilde{G}_{1}(\varsigma, \aleph)$ for $(\varsigma, \aleph)\in [0, T]\times\mathcal{D}$, where $\widetilde{M}_{1}(\varsigma, \aleph), \widetilde{H}_{1}(\varsigma, \aleph), \widetilde{\Gamma}_{1}(\varsigma, \aleph)$ can be obtained by replacing $G_1(\varsigma, \aleph)$ with $\widetilde{G}_1(\varsigma, \aleph)$ in ${M}_{1}(\varsigma, \aleph), {H}_{1}(\varsigma, \aleph), {\Gamma}_{1}(\varsigma, \aleph)$. In this connection, define the value function as

Since the problem (3.10) is well-posed, similar to ^[36], by a simple adaptation, (3.12) has the form

where $\widetilde{G}_1(\varsigma, \aleph)$ is a symmetric matrix. Further, for $\aleph\in \mathcal{D}$, applying the dynamic programming method and considering (3.13), we can yield that

To minimize the above equation, the following condition is required:

and (3.15) is equivalent to

At present, let $T_1 = \widetilde{M}_1(\varsigma, \aleph)$, $T_2 = I_{n_{\sigma}}$, $T_3 = -\widetilde{H}_{1}(\varsigma, \aleph)'$, and via Lemma 2, we can gain

and

with $F = 0$. Plugging (3.17) into (3.14) can be calculated. The Eq (3.11) has solution $\widetilde{G}_{1}(\varsigma, \aleph)$, $(\varsigma, \aleph)\in [0, T]\times\mathcal{D}$. Besides, we can generalize Lemma 3 in ^[38] to the infinite Markov jump case, which is processed in a similar manner. What follows is $\widetilde{M}_1(\varsigma, \aleph)\geq0$. So far, we can get that the CGDREs (3.11) admit a solution $\widetilde{G}_{1}(\varsigma, \aleph)$ for $(\varsigma, \aleph)\in [0, T]\times\mathcal{D}$: then it can be obtained that the solution of the indefinite LQ optimal control problem (3.10) is $\sigma^*(\varsigma) = -\widetilde{M}_1(\varsigma, \aleph)^{\dagger}\widetilde{H}_{1}(\varsigma, \aleph)'x(\varsigma)$ with $\widetilde{\Gamma}_{1}(\varsigma, \aleph) = -\widetilde{M}_1(\varsigma, \aleph)^{\dagger}\widetilde{H}_{1}(\varsigma, \aleph)'$.

The same can be seen with inequality (2.5) by Definition 2.1, and $\eta^{*}(\varsigma) = \Gamma_2(\varsigma, \varpi_\varsigma)x(\varsigma)$ is the solution of the following indefinite LQ optimal control problem:

There is an analogous method for proving the following CGDREs

has solution $\widetilde{G}_{2}(\varsigma, \aleph)$, $(\varsigma, \aleph)\in [0, T]\times\mathcal{D}$, where $\widetilde{M}_{2}(\varsigma, \aleph), \widetilde{H}_{2}(\varsigma, \aleph), \widetilde{\Gamma}_{2}(\varsigma, \aleph)$ can be gained by replacing $G_2(\varsigma, \aleph)$ with $\widetilde{G}_2(\varsigma, \aleph)$ in ${M}_{2}(\varsigma, \aleph), {H}_{2}(\varsigma, \aleph), {\Gamma}_{2}(\varsigma, \aleph)$, and the solution of indefinite LQ optimal problem (3.18) is $\eta^{*}(\varsigma) = -\widetilde{M}_2(\varsigma, \aleph)^{\dagger}\widetilde{H}_{2}(\varsigma, \aleph)'x(\varsigma)$ with $\widetilde{\Gamma}_{2}(\varsigma, \aleph) = -\widetilde{M}_2(\varsigma, \aleph)^{\dagger}\widetilde{H}_{2}(\varsigma, \aleph)'$. To sum up, a mixture of (3.11) and (3.19) decides $\widetilde{G}_{1}(\varsigma, \aleph) = G_{1}(\varsigma, \aleph), \widetilde{G}_{2}(\varsigma, \aleph) = G_{2}(\varsigma, \aleph)$. That shows the CGDREs admit solutions $(G_{1}(\varsigma, \aleph), G_{2}(\varsigma, \aleph))$ for $(\varsigma, \aleph)\in [0, T]\times\mathcal{D}$. This ends the proof. □

Remark 3.1. Based on Theorem 3.1, one can glean that the key to obtaining Nash equilibrium strategies for system (2.1) is solving the CGDREs (3.2) and (3.3). Since the CGDREs here are a countably infinite set of equations, compared with ^[32] and the discrete-time case of ^[23,31], it is harder and more complex to solve. Actually, a discretization method is presented in ^[33], which can apply to calculate (3.2) and (3.3).

Remark 3.2. Indeed, the solvability of (3.2) and (3.3) is very crucial. However, it is worth noting that even for LQ problems with no Markov jumps, the related Riccati equations remain unsolved, and their solvability can only be guaranteed under certain specific conditions, for example, LQ optimal control in ^[34], LQ zero-sum game in ^[35], and LQ non-zero-sum game in ^[30]. The main difficulty lies in that the Riccati equations are highly coupled. In future research, we will focus on the discussion about the existence of solutions to the Riccati equations.

4.   A unified control design for $H_{2}$, $H_{\infty}$ and $H_{2}/H_{\infty}$

From the previous studies, the existence of Nash equilibrium strategies for system (2.1) has been discussed. This section will focus on a unified treatment for the three control problems with the different values of $\alpha$ and $\beta$ or regarding $\sigma(\varsigma)$ as an exogenous disturbance.

4.1.   $H_2$ control

Let $\alpha\rightarrow \infty, \beta = 0$ in (2.2) and (2.3) we can obtain the following LQ optimal control problem

subject to

The weighting matrices of state and control in the cost function (4.1) of the above LQ optimal control problem are $A(\varsigma, \varpi_\varsigma)'A(\varsigma, \varpi_\varsigma)$ and $I_{n_{\eta}}$, respectively. Moreover, it can be computed by Theorem 3.1 that $M_1(\varsigma, \aleph)^{\dagger}\rightarrow0, H_{1}(\varsigma, \aleph) = 0, \Gamma_1(\varsigma, \aleph)\rightarrow0, H_{2}(\varsigma, \aleph)\rightarrow G_2(\varsigma, \aleph)D_1(\varsigma, \aleph)+C_2(\varsigma, \aleph)'G_2(\varsigma, \aleph)D_2(\varsigma, \aleph)$, and $G_2(\varsigma, \aleph)$ is the solution of the following CGDREs

Via Theorem 3.1, we can further obtain that optimal control is $\eta^{*}(\varsigma) = \Gamma_2(\varsigma, \varpi_\varsigma)x(\varsigma)$ with

for $\varpi_t = \aleph$, and the optimal value function is

Remark 4.1. Significantly different from (3.3), the positive definiteness of $M_{2}(\varsigma, \aleph)$ can be guaranteed; in other words, $M_2(\varsigma, \aleph)^{\dagger} = M_2(\varsigma, \aleph)^{-1}$. In reality, the main reason is that, taking advantage of (4.4), it can easily prove $G_2(\varsigma, \aleph)\geq0$ for $(\varsigma, \aleph)\in [0, T]\times\mathcal{D}$.

4.2.   $H_{\infty}$ control

Set $\alpha = \beta$ in (2.2) and (2.3), then it is clear that

Furthermore, the Eq (3.3) is expressed in terms of the following form:

and it is equivalent to

At present, we plug $G_{2}(\varsigma, \aleph) = -G_{1}(\varsigma, \aleph)$ into (4.6), and have

Taking notice of $\Gamma_1(\varsigma, \aleph) = -M_{1}(\varsigma, \aleph)^{\dagger}H_{1}(\varsigma, \aleph)'$, which makes (4.7) the same as (3.2). On the other hand, it should be noted that on the grounds of the definition of $H_{\infty}$ control, $\|L_{T}\| < \gamma$ is the premise ^[33]. Hence, under the condition of $\|L_{T}\| < \gamma$, following the line of Lemma 8.1.2 in ^[39], it can be deduced that $M_{1}(\varsigma, \aleph) > 0$. Keep in mind that $M_{1}(\varsigma, \aleph) > 0$ leads to $M_{1}(\varsigma, \aleph)^{\dagger} = M_{1}(\varsigma, \aleph)^{-1}$. At this point, taking advantage of Theorem 3.1, we can obtain that the $H_{\infty}$ optimal controller is $\eta^{*}(\varsigma) = \underline{\Gamma}_2(\varsigma, \aleph)x(\varsigma)$, and $\sigma^*(\varsigma)$ is the corresponding worst-case disturbance, where $G(\varsigma, \aleph)$ is the solution of (4.7) and satisfies the following CGDREs:

where

Remark 4.2. Through the above analysis, to ensure the existence of an $H_{\infty}$ optimal controller, the premise is $M_{1}(\varsigma, \aleph) > 0$. Besides, the solution to the CGDREs (4.8) is $G_{1}(\varsigma, \aleph)\leq0$; the reason for $G_{1}(\varsigma, \aleph)\leq0$ is

4.3.   $H_{2}/H_{\infty}$ control

If we set $\beta = 0$ in (2.2) and (2.3), then we have the following new cost functionals:

In light of the definition of $H_{2}/H_{\infty}$ control in ^[33], it can be concluded that

Consequently, it can be deduced from $M_{1}(\varsigma, \aleph) > 0$ and Theorem 3.1 that the following CGDREs

admit solutions $G_{1}(\varsigma, \aleph)\leq0$, $G_{2}(\varsigma, \aleph)\geq0$ for $(\varsigma, \aleph)\in [0, T]\times\mathcal{D}$. As a matter of fact, as we described in Remark 4.1 and Remark 4.2, we have $G_{1}(\varsigma, \aleph)\leq0$, $G_{2}(\varsigma, \aleph)\geq0$. In the meantime, $\eta^*(\varsigma) = \Gamma_2(\varsigma, \aleph)x(\varsigma)$, $\sigma^*(\varsigma) = \Gamma_1(\varsigma, \aleph)x(\varsigma)$ is the $H_2/H_\infty$ optimal controller.

Remark 4.3. It is important to note that the CGDREs (4.11)–(4.14) are the same as (5.2)–(5.5) in Theorem 5.1 of ^[33]. By contrast, the two group equations between Theorem 3.1 and Theorem 5.1 are not equivalent, which indicates that although we deal with the $H_2/H_\infty$ control by making use of Theorem 3.1 for system (2.1), the equivalence between Nash equilibrium points and $H_2/H_\infty$ control is not valid. This is fundamentally different from the discussion in ^[40].

Remark 4.4. The main reason for the inequivalence between Nash equilibrium points and $H_2/H_\infty$ control is that the conditions of $M_1(\varsigma, \aleph) > 0$ in (4.11) and $M_2(\varsigma, \aleph) > 0$ in (4.13) are not satisfied for Nash equilibrium points. In fact, it is important to notice that the root cause is whether the diffusion term contains disturbance.

5.   Numerical example

This part concentrates on a numerical example, which states the validity of the proposed method.

Example 5.1. Consider the linear SDEs with time-varying coefficients and infinite Markov jumps (2.1) with

Set $T = 2$; it should be noted that a homogeneous Poisson process can be regarded as an infinite Markov process. Thus let $\{\varpi_\varsigma\}_{\varsigma\in[0, T]}$ be a homogeneous Poisson process with parameter $\psi > 0$, and the infinitesimal matrix of $\{\varpi_ \varsigma\}_{\varsigma \in[0, T]}$ is $\Phi = (\phi_{\aleph j})_{ \varsigma, j\in \mathcal{D}}$ with $-\phi_{\aleph \aleph} = \phi_{\aleph, \aleph+1} = \psi$ and $\phi_{\aleph j} = 0$, $\aleph \in \mathcal{D}$, $j\in\mathcal{D}/\{\aleph, \aleph+1\}$.

Observe that the discretization method and backward recursive algorithm are the key to solving corresponding CGDREs as described in Remark 3.1. In view of the discussion in the previous section, first of all, it turns the LQ optimal controller designing into solving Eq (4.3). Further, we can compute the following approximate solutions:

The LQ optimal control and optimal value function can be immediately obtained by (4.4). And then, let $\alpha = 1$; the $H_{\infty}$ optimal controller design problem is translated to the solvability of Eq (4.8). Additionally, the following approximate solutions can be given:

We can get the $H_{\infty}$ optimal control quickly. And finally, when $\beta = 0$, the finite horizon $H_2/H_\infty$ optimal control can be transformed into the existence of the solution to the CGDREs (4.11)–(4.14). Go a step further; we can figure out the approximate solutions below:

The corresponding finite-horizon $H_2/H_\infty$ optimal controller can be derived naturally.

6.   Conclusions

In this note, we studied the non-zero-sum Nash differential games for SDEs involving time-varying coefficients and infinite Markov jumps. By means of a pseudo-inverse matrix, necessary and sufficient condition for the existence of Nash equilibrium strategies is given by the solvability of CGDREs. As an application, by the Nash game approach, we present a unified treatment for $H_{2}$, $H_{\infty}$, and $H_{2}/H_{\infty}$ control with some corresponding parameters. At last, the theoretical results are used to solve a numerical example. There are several interesting problems that deserve further investigation, in particular, how to generalize our result to the infinite horizon Nash game problem.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

This research was funded by the Natural Science Foundation of Qingdao under Grant 23-2-1-7-zyyd-jch, the National Natural Science Foundation of China under Grant 62273212, the Natural Science Foundation of Shandong Province under grant ZR202103030362, the Science Research Project of the Hebei Education Department under Grant QN2022077, the S$&$T Program of Hebei under Grant 236Z1603G, and the Hebei Natural Science Foundation under Grant F2022203097.

Conflict of interest

The authors declare there are no conflicts of interest.