Efficacy and safety of non-invasive brain stimulation techniques for the treatment of nicotine addiction: A systematic review of randomized controlled trials

1.   Introduction

Ever since the studies of epidemiology conducted by Kermack et al. ^[1] mathematical models have become important tools for studying the spread and control of infectious diseases. For some deterministic epidemic models, many researchers have studied the dynamical behaviors ^{[1,2,3,4,5,6]}. Moreover, Chen et al. ^[9] proposed that the coefficient of deterministic epidemic system may be influenced by stochastic perturbation. Therefore, stochastic models have been studied to find out impacts of stochastic fluctuation upon the infectious diseases ^{[11,11,24,29,30]}. For example, the studies conducted by Zhao et al. ^[11] on the asymptotic behavior of a stochastic SIS epidemic model with vaccination. Cai et al. ^[11] proves the existence of stationary distribution of a stochastic SIRS epidemic model.

In fact, some parameters of the epidemic models may be influenced by the sudden changes such as color noise which deems to be a random switching from one environment regime to another under the influence of such factors as temperature or rainfall ^[15,17]. For the first equation of the SIV epidemic model, the parameters are determined as follows:

we assume that there are $n$ regimes and the system obeys

when it is in regime 1, while it obeys another stochastic model

in regime 2 and so on. Therefore, the system obeys

in regime $i (1 \leq i \leq n)$. To best reveal the way the environment noise affects the epidemic systems, we propose the application of continuous-time Markov chain to define this phenomenon ^[18,19,20]. It is significant to study the effect of random switching of environmental regimes on the spread dynamic of diseases.

As it is known that SIV epidemic model has drawn much attention. For instance, Liu et al. ^[26] discussed the existence and uniqueness of the global positive solution of the system, and Lin et al. ^[23] considered the asymptotic stability of the stochastic SIV epidemic model and the existence of a stationary distribution. The above work done just focused on the studies of dynamic behaviors of SIV system with quite few studies focusing on the optimal control for stochastic SIV models with Markov chains. The study of optimal control of stochastic SIV epidemic model, therefore, becomes significant.

In addition, vaccination and treatment have become the most effective strategies in the control of the epidemic transmission. We give priority to vaccinations to susceptible people. Vaccination duration and effective treatment make it possible to curb the spread of diseases and cut the cost of vaccines. For instance, Measles is a highly contagious disease that may result in serious illness in the early childhood stage, and therefore the advance immunization is necessary ^[25]. The global Measles and Rubella Strategic Plan 2012–2020, aims to reduce global measles mortality and to achieve measles elimination by the end of the 2020 (http://www.measlesinitiative.org/). The most effective approach to achieve this plan is vaccination. In view of this above, it is meaningful to carry out the research of a stochastic SIV epidemic model with vaccination and saturated treatment to analyze the near-optimal control with Markov chains.

According to the paper, the objective function is determined and constructed on the basis of an optimal control problem with two control variables of vaccination and treatment. And the optimal solution of the model is provided according to the objective function through analysis in the relevant optimal theories. Aiming at the stochastic SIV model with Markov chain, we carried out thorough studies on the minimization of the number of infected individuals through the application of prophylactic vaccination and saturated treatment. We also define the sufficient and necessary conditions for the near-optimal control problems. The innovative ideas of the study are as follows:

● Provision of an innovative stochastic SIV epidemic model with vaccination, treatment and Markov chains;

● Discussion of sufficient and necessary conditions of the near-optimal control for a stochastic SIV model.

The remaining part of of the paper is so organized as: Section 2 preliminaries and introduction of stochastic SIV model; Section 3 proof of the necessary condition for near-optimal control; Section 4, institution of sufficient conditions for near-optimal control; Section 5 numerical simulations display to confirm the results; and, Section 6, conclusions.

2.   Preliminaries

In this section, we first introduce the following notations and definitions before we formulate the model.

We define $(\Omega, \mathcal{F}, \mathcal{F}_{0\leq t\leq T}, \mathbb{P})$ be a complete filtered probability space on a finite time horizon $[0, T]$. We assume $(\mathcal{F}_{t})_{0\leq t\leq T}$ is the natural filtration and $\mathcal{F} = \mathcal{F}_{T}$, $\mathcal{U}_{ad}$ is the set of all admissible controls, $f_{x}$ denotes the partial derivative of $f$ with respect to $x$, $|\cdot|$ denotes the norm of an Euclidean space, $\mathcal{X}_{S}$ denotes the indicator function of a set $S$, $X + Y$ is the set $\{x+y:\ x\in X, \ y\in Y \}$ for any sets $X$ and $Y$, $C$ is the different parameters in the paper.

We establish the optimal problems on the basis of a stochastic SIV model was put forward by Safan and Rihan et al. ^[14] as follows:

with initial conditions $S(0)\geq 0, I(0)\geq 0,$ and $V(0)\geq 0$. The population size is denoted by $N(t)$ with $N(t) = S(t)+I(t)+V(t)$. The SIV model assume that recovered individuals may lose immunity and move into the susceptible class again. Note that $0\leq p\leq 1$ and ${\beta S(t)I(t)}$ is the incidence rate. Vaccinated individuals can either die with rate $\mu$ or obtain infected with force of infection $(1-e)\beta I$ where $e$ measures the efficacy of the vaccine induced protection against infection. If $e = 1$, then the vaccine is perfectly effective in preventing infection, while $e = 0$ means that the vaccine has no effect. The following flux diagram (Figure 1) illustrating the transmission of susceptible, infected and vaccinated. S, I and V represent susceptible, infected and vaccinated respectively. The continuous line represents nonlinear transmission rate between different compartments.

We assume that stochastic perturbations in the environment will mainly affect the parameter $\beta$, as in Tornatore et al. ^[27], so that $\beta \to \beta +\sigma \dot{B}(t)$, where $B(t)$ is a standard Brownian motion with the intensity $\sigma^2 > 0$. All parameters in model (2.1) are assumed to be positive due to the physical meaning and more precisely are listed in Table.1. By this way, the stochastic version corresponding to system (2.1) be expressed by Liu et al. ^[26] as follows:

According to ^[28], we know that the switching of the Markov chain is memoryless and the waiting time for the next switch is subjected to an exponential distribution. In addition, we assume that there are $N$ regimes and the switching between them is governed by a Markov chain on the state space $S = \{1, 2, ..., N\}$. Then as the sample path of Markov chain $\xi(t)$ is right continuous step function almost surely, its jump points have no accumulation point. Thus, there exists a stopping sequence $\{\tau_{k}\}_{k\geq0}$ for every $w\in \Omega$, and a finite random constant $\bar{k} = \bar{k}(w)$ such that $0 = \tau_{0} < \tau_{1}... < \tau_{\bar{k}} = \tau$. Then when $k > \bar{k}$, we get that $\tau_{k} > \tau$ and $\xi(t)$ is a random constant in every intervals. That is to say, $\xi(t) = \xi(\tau_{k})$ for $\tau_{k}\leq t\leq\tau_{k+1}$.

Let $\mathbb{R}_{+}^{n} = \{x\in\mathbb{R}^{n}:x_{i} > 0, i = 1, 2, ..., n\}$, and $|x| = \sqrt{\sum_{i = 1}^{n}x_{i}^{2}}$. Then Markov process $(x(t), \xi(t))\in\mathbb{R}_{+}^{n}\times \mathbb{S}$, making up a diffusion component $x(t)$ and a jump component $\xi(t)$, can be described as follows:

where $B(t)$ is a d-dimensional Brownian motion, and $f(\cdot, \cdot):\mathbb{R}^{n}\times \mathbb{S}\longrightarrow\mathbb{R}^{n}$, $g(\cdot, \cdot):\mathbb{R}^{n}\times \mathbb{S}\longrightarrow\mathbb{R}^{n\times d}$.

If for any twice continuously differentiable function $V(x, k)\in C^{2}(\mathbb{R}^{n}\times \mathbb{S})$, define linear operator $\mathcal{L}$ by

where $a(x, i) = g(x, i)g^{\top}(x, i)$ with the superscript $T$ stands for the transpose of a matrix or vector.

Hypothesis. The following basic hypothesis need to be satisfied

(H1) For all $0 \leq t \leq T$, the partial derivatives $L_{x_{1}(t)}(t, x(t); u(t))$, $L_{x_{2}(t)}(t, x(t); u(t))$, and $L_{x_{3}(t)}(t, x(t); u(t))$ are continuous, and there is a constant $C$ such that

(H2) Let $x(t)$, $x'(t)\in\mathbb{R}_{+}^{3}$ and $u_{i}(t)$, $u_{i}'(t) \in \mathcal{U}_{ad}, i = 1, 2$, then for any $0 \leq t \leq T$, the function $L(t, x(t); u(t))$ is differentiable in $u(t)$, and there exists a constant $C$ such that

(H3) The control set $\mathcal{U}_{ad}$ is convex.

(H4) Assume that

holds.

(H5) Assume that

holds. We introduce the treatment control $u_{2}(t)$ into the above model (2.2). With the help of vaccination or treatment, the number of susceptible individuals increased, using $S_u$ represent it. In fact, every community should have proper treatment ability. If investment in treatment is too large, the community will pay unnecessary expenses. If it is too small, the community will have the risk of disease outbreaks. Therefore, it is very important to determine the right treatment ability. Following Wang's argument in ^[22], the advantage of using a saturated type treatment function is that it generates near-linear treatment results. We assume that the treatment function is a function of $u_{2}(t)$ and $I(t)$. It is defined as $T(u_{2}(t), I(t)) = \frac{mu_{2}(t) I(t)}{1+\eta I(t)}$, where $m > 0, \eta \geq 0$. Our new model is as follows:

where $m$ is the cure rate, $\eta$ in the treatment function measures the extent of delayed treatment given to infected people. Let $U \subseteq \mathbb{R}$ be a bounded nonempty closed set. The control $u(t)\in U$ is called admissible, if it is an $\mathcal{F}_{t}$-adapted process with values in $U$. The set of all admissible controls is denoted by $\mathcal{U}_{ad}$. In this optimal problem, we assume a restriction on the control variable such that $0 \leq u_{1}(t) \leq 0.9$ ^[12], because it is impossible for all susceptible individuals to be vaccinated at one time. $u_{2}(t) = 0$ represents no treatment, and $u_{2}(t) = 1$ represents totally effective treatment.

For the sake of showing such sudden environmental abrupt shift in different regimes, we will introduce the colored noise (i.e. the Markov chain) into the SIV epidemic model (2.6). Let $\xi(t)$ be a right-continuous Markov chain in a finite state space $S = \{1, 2, ..., N\}$ with generator $\alpha = (q_{ij})_{N\times N}$ given by

where $\Delta t > 0$ and $q_{ij}$ is the transition rate from state $i$ to state $j$ and $q_{ij}\geq 0$ if $j\neq i$ while $q_{ii} = -\sum_{j\neq i}q_{ij}$.

Then we can obtain the novel SIV epidemic model as follows:

To simplify equations, we represent $x_{i}(t) = (x_{1}(t), x_{2}(t), x_{3}(t))^{\top} = (S(t), I(t), V(t))^{\top}$, $u(t) = (u_{1}(t), u_{2}(t))^{\top}$ and the model (2.8) can be rewritten as

with the objective function

where $x(t) = \{ x(t): 0 \leq t\leq T \}$ is the solution of model (2.9) on the filtered space $(\Omega, \mathcal{F}, (\mathcal{F}_t)_{0 \leq t \leq T}, \mathbb{P})$. For any $u(t) \in \mathcal{U}_{ad}$, model (2.9) has a unique $\mathcal{F}_t$-adapted solution $x(t)$ and $(x(t), u(t))$ is called an admissible pair. The control problem is to find an admissible control which minimizes or nearly minimizes the objective function $J(0, x_{0};u(\cdot))$ over all $u(\cdot)\in \mathcal{U}_{ad}$. The objective function $J(0, x_{0};u(\cdot))$ represent a function that $t = 0, x(t) = x_{0}$ with control variable $u(\cdot)$. The value function is as follows:

Our goal is to minimize the total number of the infected and susceptible individuals by using minimal control efforts. Refering to Lashari et al. ^[13], let the objective function in this paper is

where $\tau_{1}$ and $\tau_{2}$ are positive constants,

Definition 2.1. (Optimal Control) ^[7]. If $u^{*}(\cdot)$ or an admissible pair$(x^*(t), u^*(t))$ attains the minimum of $J(0, x_{0};u(\cdot))$, then an admissible control $u^{*}(\cdot)$ is called optimal.

Definition 2.2. ($\varepsilon$-Optimal Control) ^[7]. For a given $\varepsilon > 0$, an admissible pair$(x^*(t), u^*(t))$ or an admissible control $u^{\varepsilon}(\cdot)$ is called $\varepsilon$-optimal, if

Definition 2.3. (Near-optimal Control) ^[7]. A family of admissible controls $\{u^{\varepsilon}(\cdot)\}$ parameterized by $\varepsilon > 0$ and any element $(x^*(t), u^*(t))$, or any element $u^{\varepsilon}(\cdot)$ in the family are called near-optimal, if

holds for a sufficiently small $\varepsilon > 0$, where $\delta$ is a function of $\varepsilon$ satisfying $\delta(\varepsilon)\rightarrow 0$ as $\varepsilon \rightarrow 0$. The estimate $\delta(\varepsilon)$ is called an error bound. If $\delta(\varepsilon) = c\varepsilon^{\kappa}$ for some $c, \kappa > 0$, then $u^{\varepsilon}(\cdot)$ is called near-optimal with order $\varepsilon^{\kappa}$.

Definition 2.4. ^[11]. Let $\Xi \subset \mathbb{R}^{n}$ be a region and $u_1:\Xi \longrightarrow \mathbb{R}^{n}$ be a locally Lipschitz continuous function. The generalized gradient of $\phi$ at $\xi \in \Xi$ is defined as

Our next goal is to derive a set of necessary conditions for near-optimal controls.

3.   Necessary conditions for near-optimal controls

In this section, we will introduce two lemmas at first, then we will obtain the necessary condition for the near-optimal control of the model (2.8).

3.1.   Some priori estimates of the susceptible, infected and recovered

Lemma 3.1. For all $\theta \geq 0$ and $0 < \kappa < 1$ satisfying $\kappa\theta < 1$, and $u(t), u'(t) < /italic > < italic > \in \mathcal{U}_{ad}$, along with the corresponding trajectories $x(t), x'(t)$, there exists a constant $C = C(\theta, \kappa)$ such that

Proof. We assume $\theta \geq 1$ and $\forall r > 0$, use the elementary inequality, and get

If $0 < \kappa < 1$, then $\theta \geq 1$, we apply Cauchy-Schwartz's inequality and get

Similarly, we can get the following estimates for $|x_{i}(t)-x'_{i}(t)|^{2\theta} (i = 2, 3)$

Then, summing up (3.2) and (3.4), we may obtain that

Now by considering $0 \leq \theta < 1$, with the help of Cauchy-Schwartz's inequality, we have

Therefore, the result is true by making use of Gronwall's inequality.

The proof is completed.

Lemma 3.2. Let Hypotheses $(H3)$ and $(H4)$ hold. For all $0 < \kappa < 1$ and $0 < \theta < 2$ satisfying $(1 + \kappa)\theta < 2$, and $u(t), u'(t)\in \mathcal{U}_{ad}$, along with the corresponding trajectories $x(t), x'(t)$, and the solution $(p(t), q(t)), (p'(t), q'(t))$ of corresponding adjoint equation, there exists a constant $C = C(\kappa, \theta) > 0$ such that

Proof. Let $\widehat{p}_{i}(t) \equiv p_{i}(t)-p'_{i}(t), \widehat{q}_{j}(t) \equiv q_{j}(t)-q'_{j}(t)(i, j = 1, 2, 3)$. From the adjoint equation (4.5), we have

where

We assume the solution of the following stochastic differential equation is $\phi(t) = (\phi_{1}(t), \phi_{2}(t), \phi_{3}(t))^{\top}$.

where $sgn(\cdot)$ is a symbolic function.

According to Hypothesis $(H1)$, the definition of $L$ and Lemma 4.2, the above equation admits a unique solution. Meanwhile, Cauchy-Schwartz's inequality indicates that

where $\theta_{1} > 2$ and $\frac{1}{\theta_{1}} + \frac{1}{\theta} = 1$.

In order to obtain the above inequality (3.6), we define the function

where $\bar{w} = (\bar{w}_{1}, \bar{w}_{2}, \cdots, \bar{w}_{N})^{\top}$, $|\bar{w}| = \sqrt{\bar{w}_{1}^{2}+\cdots+\bar{w}_{N}^{2}}$ and $\bar{w_{k}}(k\in\mathbb{S})$ are to be determined later and the reason for $|\bar{w}|$ being is to make $\bar{w_{k}}+|\bar{w}|$ non-negative. Using Itô's formula ^[4] yields

We define a vector $\Xi = (\Xi_{1}, \Xi_{2}, \cdots, \Xi_{N})^{\top}$ with

Due to the generator matrix $\alpha$ is irreducible and Lemma 2.3 in ^[5], for $\Xi_{k}$ there exists a solution

for the following poisson systems:

Thus, we obtain that

According to Hypotheses $(H4)$ and $(H5)$, we get that

Integrating both side of (3.16) from 0 to $T$ and taking expectations, we get that

Substituting (3.9) into (3.17),

We proceed to estimate the right side of (3.18). From Hypothesis (H3) and Lemma 4.1, it follows that

Application of Cauchy-Schwartz's inequality, we obtain

Noting that $\frac{2\theta}{1-\theta} < 1$, $1-\frac{1}{\theta} > \frac{{\kappa\theta}}{2}$ and $d(u(t), u'(t)) < 1$, using Lemmas 4.2 and 3.1, we derive that

Applying the same method, we can get the same estimate, i.e.

Combining (3.18) with the above two estimates, the desired result then holds immediately.

3.2.   Necessary conditions for near-optimal controls

Theorem 3.1. Let Hypotheses $(H1)$ and $(H2)$ hold. $(p^{\varepsilon}(t), q^{\varepsilon}(t))$ is the solution of the adjoint equation (4.5) under the control $u^{\varepsilon}(t)$. Then, there exists a constant $C$ such that for any $\theta\in[0, 1), \varepsilon > 0$ and any $\varepsilon$-optimal pair $(x^{\varepsilon}(t), u^{\varepsilon}(t))$, it holds that

Proof. The crucial step of the proof is to indicate that $H_{u(t)}(t, x^{\varepsilon}, u^{\varepsilon}(t), p^{\varepsilon}(t), q^{\varepsilon}(t))$ is very small and use $\varepsilon$ to estimate it. Let us fix an $\varepsilon > 0$. Define a new metric $d$ as follows:

and

where the cost functional

This implies that $(\widetilde{x}^{\varepsilon}(t), \widetilde{u}^{\varepsilon}(t))$ is optimal for state equations (2.9) with the cost functional (2.11). Then, for a.e. $t\in[0, 1], \theta\in[0, 1)$, a stochastic maximum principle yields

where $(\widetilde{p}^{\varepsilon}(t), \widetilde{q}^{\varepsilon}(t))$ is the solution of (4.5) with $\widetilde{u}^{\varepsilon}(t)$.

For simplicity, let

By virtue of (3.27) and elementary inequality, there exists a constant $C$ such that

From Hypothesis (H1) and the definition of $L$, Lemma 3.1 and 3.2, we get

Thus, we get the result, i.e.

4.   Sufficient conditions for near-optimal controls

4.1.   Some priori estimates of the susceptible, infected and recovered

Lemma 4.1. For any $\theta \geq 0$ and $u(t)\in \mathcal{U}_{ad}$, we have

where $C$ is a constant that depends only on $\theta$.

Proof. First, we assume $\theta \geq 1$. The first equation of (2.9) can be rewritten as

Similarly,

where $\mathbb{E}^{\mathcal{F}_{t}}[x]$ represent the conditional expectation of $x$ with respect to $\mathcal{F}_{t}$. Using the elementary inequality, we can get

then we get

Summing up (4.2), (4.3) and (4.4), we can get

Let $t\in[T-\varepsilon, T]$ with $\varepsilon = \frac{1}{2C}$, we get

By Gronwall's inequality, we can get that

when $0 < \theta < 1$.

Then, we can get

Similarly, we have

The proof is complete.

Next, we will draw into the adjoint equation ^[11] as follows:

where

Lemma 4.2. For any $u(t)$, $u'(t)\in \mathcal{U}_{ad}$, we have

where $C$ is a constant.

Proof. Integrating the first equation of both sides (4.5) from $t$ to $T$, we obtain that

Squaring the equation above and making use of the second moments, taking note of $x(t)\in\alpha$ for all $t \geq 0$, we have

Similarly,

Summing up (4.8), (4.9) and (4.10), we can get

If $t\in[T-\epsilon, T]$ with $\epsilon = \frac{1}{2 C}$, we have

From the above inequality and Gronwall's inequality shows that

Apply the same way to (4.8), (4.9) and (4.10) in $[T-\epsilon, T]$, therefore, we see that (4.12) holds for all $t\in[T-2\epsilon, T]$ by a finite number of iterations, Eq (4.7) can be recapped as follows:

According to the above equation, we have

Furthermore, using the same method in (4.14) yields similar results of the above inequality (4.15), which combines with (4.15) and shows that

Taking expectation of equation (4.16) and using Burkholder-Davis-Gundy inequality (see Theorem 1.7.3 ^[4]), we get

By Gronwall's inequality in the above inequality to get our result (4.6).

The proof is complete We define a metric on the admissible control domain $\mathcal{U}_{ad}[0, T]$ as follows:

where $mes$ represents Lebesgue measure. Since $U$ is closed, it can be shown similar to ^[7] that $\mathcal{U}_{ad}$ is a complete metric space under $d$.

Lemma 4.3. (Ekeland's principle) ^[16]. Let $(Q, d)$ be a complete metric space and $F(\cdot): Q \rightarrow \mathbb{R}$ be a lower-semicontinuous and bounded from below. For any $\varepsilon > 0$, we assume that $u^{\varepsilon}(\cdot)\in Q$ satisfies

Then there is $u^{\lambda}(\cdot)\in Q$ such that for all $\lambda > 0$ and $u(\cdot) \in Q$,

4.2.   Sufficient conditions for near-optimal controls

We define the Hamiltonian function ^[11] $H(t, x(t), u(t), p(t), q(t)): [0, T]\times\mathbb{R}^{3}_{+}\times\mathcal{U}_{ad}\times\mathbb{R}^{3}_{+}\times\mathbb{R}^{3}_{+}\rightarrow \mathbb{R}$ as follows:

with

where $f_{i} (i = 1, 2, 3)$ and $\sigma_{i4} (i = 1, 2, 3)$ are defined in (2.9), and $L(x(t), u(t))$ is defined in (2.11).

Theorem 4.1. Suppose $(H1), (H2)$ and $(H3)$ hold. Let $(x^{\varepsilon}(t), u^{\varepsilon}(t))$ be an admissible pair and $(p^{\varepsilon}(t), q^{\varepsilon}(t))$ be the solutions of adjoint equation (4.5) corresponding to $(x^{\varepsilon}(t), u^{\varepsilon}(t))$. Assume $H(t, x(t), u(t), p(t), q(t))$ is convex, a.s. For any $\varepsilon > 0$,

then

Proof. To estimate the term $H_{u(t)}(t, x^{\varepsilon}(t), u^{\varepsilon}(t), p^{\varepsilon}(t), q^{\varepsilon}(t))$, we define a new metric $\widetilde{d}$ on $\mathcal{U}_{ad}$: for any $\varepsilon > 0$, any $u(t)$, $u'(t)\in\mathcal{U}_{ad}$,

where

It is easy to find that $\widetilde{d}$ is a complete metric as a weighted $L^{1}$ norm.

According to Eq (2.11) and the definition of the Hamiltonian function $H(t, x(t), u(t), p(t), q(t))$, we have

with

Due to the convexity of $H(t, x^{\varepsilon}(t), u^{\varepsilon}(t), p^{\varepsilon}(t), q^{\varepsilon}(t))$, we have

Similarly,

Now, we define $V$ function:

where $\bar{w} = (\bar{w}_{1}, \bar{w}_{2}, \cdots, \bar{w}_{N})^{\top}$, $|\bar{w}| = \sqrt{\bar{w}_{1}^{2}+\cdots+\bar{w}_{N}^{2}}$ and $\bar{w_{k}}(k\in\mathbb{S})$ are to be determined later and the reason for $|\bar{w}|$ being is to make $\bar{w_{k}}+|\bar{w}|$ non-negative. Using Itô's formula ^[4] yields

We set a vector $\Xi = (\Xi_{1}, \Xi_{2}, \cdots, \Xi_{N})^{\top}$ with

Because of the generator matrix $\alpha$ is irreducible and Lemma 2.3 in ^[5], for $\Xi_{k}$ there exists a solution

for the following poisson systems:

Thus, we have

From Hypotheses $(H4)$ and $(H5)$, we get that

Integrating both side of (4.33) from 0 to $T$ and taking expectations, we obtain that

Hence,

Substitute (4.25), and (4.34) into (4.24), we have

According to the definition of Hamiltonian function (4.19), we have

Define a function $F(\cdot): \mathcal{U}_{ad}\rightarrow \mathbb{R}$

By means of Hypothesis (H2) and the definition of $L$, we learn that $F(\cdot)$ is continuous on $\mathcal{U}_{ad}$ concerning the metric $\widetilde{d}$. Hence, from (4.36) and Lemma 4.3, if exists a $\widetilde{u}^{\varepsilon}(t) \in\mathcal{U}_{ad}$, then

This shows that

Using Lemma 4.3 in ^[16], we can get

Since the Hamiltonian function $H$ is differentiable in $u(t)$ with regard to Hypothesis $(H2)$, (4.40) shows that if exists a $\lambda_{1}^{\varepsilon}(t)\in [-\varepsilon^{\frac{1}{2}}y^{\varepsilon}(t), \varepsilon^{\frac{1}{2}}y^{\varepsilon}(t)],$ then

Consequently, from (4.41) and Hypothesis $(H2)$ and the definition of $L$, we get

From the definition of $\widetilde{d}$ of Lemma 4.2, we can obtain the conclusion according to (4.23), (4.42) and Holder's inequality.

In the above sections, we have give the sufficient and necessary conditions of the near-optimality control of SIV epidemic model. Our next goal is to illustrate the theoretical results through the numerical solution. In the following, we will give some figures to show the results.

5.   Numerical examples

Applying Milstein's method referred to ^[25], we have the corresponding diffuse equation of state equation (2.9) and adjoint equation (4.5) as follows:

where $\zeta_i^2 (i = 1, 2, ...)$ are not interdependent Gaussian random variables $N(0, 1)$. At below, we will present numerical simulation of the SIV model, let us assume that the Markov chain $\xi(t)$ is on the state space $S = \{1, 2\}$ with the generator $\alpha = \left(\begin{array}{cc} -2 & 2 \\ 7 & -7 \\ \end{array} \right)$ and the following setting:

When $\xi(t) = 1$,

$p(1) = 0.5, b(1) = 4.0, \beta(1) = 0.02, \mu(1) = 0.04, \eta(1) = 1.03, m(1) = 0.01, \alpha(1) = 0.001, \alpha(1) = 0.8, \sigma = 0.035;$

When $\xi(t) = 2$,

$p(2) = 0.6, b(2) = 5.0, \beta(2) = 0.04, \mu(2) = 0.05, \eta(2) = 1.05, m(2) = 0.02, \alpha(2) = 0.002, \alpha(2) = 0.9, \sigma = 0.036.$

5.1.   Vaccination control and treatment control case

We compare the results of calculations with and without control. In Figure 2, the three solution curves represent susceptible, infected and vaccinated individuals at different intervals of time. As we can see that with the change of time, the susceptible people with control drops faster than that without control. And the number of infected individuals with optimal vaccination and treatment control drops faster than that without control. The application of vaccination and treatment control give rise to the number of individuals vaccinated.

5.2.   Near-optimal control case

As it is indicated in Figure 3 that all of $p_{1}, p_{2}$, and $p_{3}$ tend to reach zero at last, which shows a minimum value of cost function.

6.   Conclusion

In this paper, we study the near-optimal control for stochastic SIV epidemic model with vaccination and saturated treatment using Markovian switching. We obtain the sufficient and necessary conditions for near-optimality. According to the Hamiltonian function to approximate the cost function, we reached the estimate of the error boundary of near-optimality. On the basis of the proposed and relative sections, it is advised tom take immediate effective measures to control the spread of epidemic diseases as this has great influence on infectious diseases. An illustrative numerical simulations example is presented to interpret the influence of vaccination and treatment control on dynamic behavior. Vaccination is another effective way to prevent individuals from getting infected and can be also incorporated into the optimal control problem. Some interesting questions deserve further investigation. It is worth studying more realistic but complex models, such as Lévy noise and time delays ^[21]. We leave this for further consideration.

Acknowledgments

The research is supported by the Natural Science Foundation of China (Grant numbers 11661064).

Conflict of interest

The authors declare there is no conflict of interest.