Bang-bang control for uncertain random continuous-time switched systems

1.   Introduction

A switched system is a type of hybrid system that consists of a group of discrete-time subsystems or continuous-time subsystems and a switching rule. The switching rule orchestrates the switching sequence between different subsystems to achieve the expected performance. This class of systems has been successfully used to model different practical systems such as manufacturing systems ^[1], communications systems ^[2], aerospace systems ^[3], power systems ^[4], and economics systems ^[5]. Optimal control for switched systems is not only to seek the optimal continuous input, but also to find the optimal switching rule to optimize a certain performance. Complexity is increased due to analyzing the switching in the interior of the system. Since the 1980s, lots of contributions ^{[6,7,8,9,10,11]} have been focused on the switched systems. For example, a Riccati recursion was used to solve the optimal control problems of nonlinear switched systems ^[9]. In reference ^[10], a novel perturbation observer with a nested parameter adaptive law was designed to study the optimal control problem for a class of switched continuous-time nonlinear systems.

All of these studies on switched systems are concerned with deterministic models. However, indeterminacy is more likely to appear in realistic switched systems. The indeterminacy is classified as objective indeterminacy and subjective indeterminacy according to whether the sample data is large enough. When the sample data is enough, the indeterminacy is characterized as objective indeterminacy. We define it as a random variable or a stochastic process because large amounts of historical data help to estimate probability distribution. In stochastic environments, optimal control problems have been widely studied ^{[12,13,14,15,16,17,18,19]}. For example, Patrinos et al. ^[15] studied constrained stochastic optimal control problems for Markovian switched systems and developed appropriate notions of invariance and stability for such systems. Reference ^[16] provided necessary conditions of optimality, in the form of a maximum principle, for optimal control problems of switched systems. A quadratic optimal control problem for discrete-time switched linear stochastic systems with nonautonomous subsystems perturbed by Gaussian random noises was proposed in reference ^[17]. A deterministic switching sequence and a continuous feedback law were implemented to minimize the expectation of a finite-horizon quadratic cost function. Fan et al. ^[18] solved the problem of stochastically weakly locally exponential stability of switched stochastic systems with the state-dependent delay. Ding and Zhu ^[19] proposed a novel intermittent static output feedback control method to solve switched positive systems with stochastic interval delay.

The other is characterized as subjective indeterminacy when the sample data is not sufficient to estimate their distributions. Subjective indeterminacy is regarded as an uncertain variable or an uncertain process ^[20]. Uncertainty theory ^[20,21] was found by Liu and developed into an important mathematical tool for dealing with subjective uncertainty. With the development of uncertainty theory, Zhu ^[22] studied optimal control problems with subjective indeterminacy. Later, numbers of works on optimal control problems have been made in uncertain environment, such as uncertain bang-bang control problems ^[23], uncertain optimal control problems with jump ^[24], and uncertain differential game problems ^[25].

However, in practice, the system may be affected not only by randomness but also by uncertainty. Obviously, the complex system cannot be dealed with simply by probability theory or uncertainty theory. Chance theory ^[26] was introduced to model complicated systems including the above two indeterminacy. After that, Liu ^[27] studied uncertain stochastic programming and proposed its mathematical properties. Then, some researchers conducted important studies in this field. For example, Chen and Zhu ^[28] presented the linear quadratic optimal control issue in uncertain random environments. The mean-square stability based on uncertain time-delayed stochastic systems driven by G-Brownian motions was investigated ^[29]. Gao and Zhang ^[30] defined the state of each component as an uncertain random variable and analyzed the importance of each component in a multistate uncertain stochastic output system. Uncertain random discrete-time noncausal systems and optimal control problems were investigated under the framework of chance theory with chance expectation ^[31], and in the field of engineering, reference ^[32] considered the human-machine system as an uncertain stochastic system and established an evaluation method of operator's simple emergency-stop action. The linear and nonlinear event-triggered extended state observers based on uncertain stochastic systems were designed in ^[33].

The main contribution of this paper is to study the optimal control problems with uncertain random continuous-time switched systems. In terms of theory, first, compared with the random switched system ^[17] and uncertain switched system ^[23], it is the first time to use both the random differential equation and uncertain differential equation to describe uncertain random switched system. Second, in order to solve the optimal control problem of the uncertain random switched system, based on chance theory and the dynamic method, this paper first proposes the equation of optimality for this system which is different from random environment ^[17] or uncertain environment ^[23]. If the dynamic system degenerates into a stochastic dynamic system or an uncertain dynamic system, the equation of optimality is consistent with that in a random environment (Hamilton-Jacobi-Bellman equation) or an uncertain environment (an equation of optimality presented in ^[23]). So, it is an extension of the equation of optimality from uncertain environment to uncertain random environment. Third, the equation of optimality is employed to obtain bang-bang control for a kind of problem. In order to implement optimal control, the Brent algorithm and genetic algorithm are applied and the results are compared. Finally, an optimal cash holding model under uncertain random environment is established, which is closely related to the real economic scenario, and the uncertainty and randomness are considered in the cash holding model for enterprise's cash holding problems, such as the fluctuation of the economic cycle, the change of the market interest rate, and the transaction cost. By adding these practical factors into the model, the model is more practical.

The organization of this paper is as follows. In Section 2, some basic concepts and theorems are reviewed. Section 3 introduces an optimal control model for uncertain random switched systems and provides the associated equation of optimality. In Section 4, a bang-bang control model is formulated and its value function and optimal control strategy are derived analytically. The genetic algorithm and Brent algorithm are used to solve a numerical example in Section 5. Finally, a cash holding problem is analyzed by applying the above model and an optimal solution is obtained.

Throughout this paper, $\boldsymbol{A}^T$ denotes the transpose of a matrix $\boldsymbol{A}$ and $tr[\boldsymbol{A}]$ is trace. $Pr(\cdot), M(\cdot), Ch(\cdot)$ represent probability measure, uncertain measure, and chance measure, respectively. $E_{Pr}$ denotes the expected value of a random variable in the sense of probability measure, $E_M$ denotes the expected value of an uncertain variable in the sense of uncertain measure, and $E_{Ch}$ denotes the expected value of an uncertain random variable in the sense of chance measure. $O(t)$ represents the high-order infinitesimal of $t$. The $[\alpha, \beta]^n$ implies the Cartesian product $[\alpha, \beta]^n = [\alpha, \beta] \times [\alpha, \beta]\times \cdots \times [\alpha, \beta]$.

2.   Preliminaries

In this section, several fundamental concepts in uncertainty theory ^[20,21], chance theory ^[26,27], and switched systems ^[6,8] are recalled.

2.1.   Uncertain variables

Probability theory is a branch of mathematics concerned with the analysis of frequency and is used to study the behavior of random phenomena. Uncertainty theory ^[20,21] is a branch of mathematics considered with the analysis of belief degree and it is used to study uncertain phenomena.

Definition 2.1. (Liu ^[21]) Let $\Gamma$ be a nonempty set and $\mathcal{L}$ be a $\sigma-$ algebra over $\Gamma$. Each element $A_i \in \mathcal{L}$ is referred to an event. A set function $\mathcal{M}$ defined on the $\sigma-$ algebra $\mathcal{L}$ is called an uncertain measure if it satisfies (i) $\mathcal{M}\{\Gamma\} = 1$; (ii) $\mathcal{M}\{A\}+\mathcal{M}\{A^c\} = 1$ for any $A \in \mathcal{L}$; (iii) $\mathcal{M} \bigg\{\bigcup\limits_{i = 1}^{\infty} A_i \bigg\} \leq \sum\limits_{i = 1}^{\infty} \mathcal{M}\{ A_i\}$ for every countable sequence of events ${A_i} \in \mathcal{L}$.

Then, the triplet $(\Gamma, \mathcal{L}, \mathcal{M})$ is called an uncertainty space. Let $(\Gamma_k, \mathcal{L}_k, \mathcal{M}_k)$ be uncertain spaces for $k = 1, 2, \cdots$. Then, the product uncertainty measure $\mathcal{M}$ is an uncertain measure defined on the product $\sigma$-algebra $\mathcal{L}_1 \times \mathcal{L}_2 \times \cdots$ satisfying $\mathcal{M} \bigg\{ \prod\limits_{k = 1}^\infty A_k\bigg\} = \bigwedge\limits_{k = 1}^\infty \mathcal{M}_k \{ A_k\}$, where $A_k$ are arbitrary events from $\mathcal{L}_k$ for $k = 1, 2, \cdots$, respectively.

Remark 2.1. The difference between probability theory and uncertainty theory does not lie in whether the measures are additive or not, but how the product measures are defined. The product probability measure is the multiplication of the probability measures of the individual events, i.e., $Pr\bigg\{ \prod\limits_{k = 1}^r A_k\bigg\} = \prod\limits_{k = 1}^r Pr (A_k)$, while the product uncertain measure is the minimum of the uncertain measures of the individual events as above.

An uncertain variable $\xi$ is a measurable function from an uncertain space $(\Gamma, \mathcal{L}, \mathcal{M})$ to the real numbers set $R$, and an uncertain vector is a measurable function from an uncertainty space to $R^n$.

Definition 2.2. (Liu ^[21]) The uncertainty distribution $\Phi: R \rightarrow [0, 1]$ of an uncertain variable $\xi$ is defined by

for any $x\in R$.

Definition 2.3. (Liu ^[21]) Let $T$ be a totally ordered set (e.g., time) and let $(\Gamma, \mathcal{L}, \mathcal{M})$ be an uncertainty space. An uncertain process is a function $X_t(\gamma)$ from $T \times (\Gamma, \mathcal{L}, \mathcal{M})$ to the set of real numbers such that $\{ X_t \in B\}$ is an event for any Borel set $B$ of real numbers at each time $t$.

The above definition says $X_t$ is an uncertain process if, and only if, it is an uncertain variable at each time $t$.

Definition 2.4. (Liu ^[21]) An uncertain process $X_t$ is said to have stationary increments if its increments are identically distributed uncertain variables whenever the time intervals have the same length, i.e., for any given $t > 0$, the increments $X_{s+t} - X_s$ are identically distributed uncertain variables for all $s > 0$. An uncertain process is said to be a stationary independent increment process if it has not only stationary increments but also independent increments.

Definition 2.5. (Liu ^[21]) Let $C_t$ be an indeterminate process if the following three conditions are satisfied:

(i) $C_0 = 0$ and nearly all sample paths are Lipschitz continuous;

(ii) $C_t$ has stationary and independent increments;

(iii) The uncertainty distribution of each increment $C_{s+t} - C_s$ is a normal uncertainty variable with expectation $0$ and variance $t^2$, and the uncertainty distribution is

We call the uncertain process $C_t$ the canonical Liu process.

Definition 2.6. (Liu ^[21]) Let $C_t$ be a canonical Liu process, $g_1$ and $g_2$ be given real functions, then

is called a differential equation driven by a canonical Liu process. Its solution satisfies the Eq (2.1) and is an uncertain process.

2.2.   Uncertain random variables

In a complicated system, there may be both uncertainty and randomness. The chance theory proposed by Liu ^[26] can deal with this uncertain random phenomenon. The chance space consists of uncertain space and probability space, described by $(\Gamma, \mathcal{L}, \mathcal{M}) \times (\Omega, \mathcal{A}, Pr) = (\Gamma \times \Omega, \mathcal{L} \times \mathcal{A}, \mathcal{M} \times Pr)$. The chance measure is denoted by $Ch(\cdot)$ as follows.

Definition 2.7. (Liu ^[26]) Assuming that there is a chance space $(\Gamma, \mathcal{L}, \mathcal{M}) \times (\Omega, \mathcal{A}, Pr)$, chance measure defining event $\vartheta \in \mathcal{L} \times \mathcal{A}$ is

Chance measure satisfies properties of normality, self-duality, and monotonicity which were certified in ^[26].

Definition 2.8. (Liu ^[26]) Let $\vartheta$ be a measurable function from the chance space $(\Gamma, \mathcal{L}, \mathcal{M}) \times (\Omega, \mathcal{A}, Pr)$ to the set $R$ of real numbers. For Borel set $B$, the set

is an event, then the $\vartheta$ is an uncertain random variable.

Definition 2.9. (Liu ^[26]) Suppose that $\vartheta$ is an uncertain random variable in the chance space $(\Gamma, \mathcal{L}, \mathcal{M}) \times (\Omega, \mathcal{A}, Pr)$, then the function

is said to be the chance distribution of $\vartheta$ for any real number $z\in R$.

Definition 2.10. (Liu ^[26]) Let $\vartheta$ be an uncertain random variable. If at least one of the following two integrals exists, then

is thought to be the expected value of the uncertain random variable $\vartheta$.

Lemma 2.1. (Chen ^[45]) Let $\boldsymbol{\eta} = (\eta_1, \eta_2, \cdots, \eta_{n_1})^T$ be a random vector where $\eta_1, \eta_2, \cdots, \eta_{n_1}$ are independent and identically distribution normal random variables with expected value $0$ and variance $t^2 (t > 0)$, and let $\boldsymbol{\theta} = (\theta_1, \theta_2, \cdots, \theta_{n_2})$ be an uncertain vector where $\theta_1, \theta_2, \cdots, \theta_{n_2}$ are independent and identically distribution normal uncertain variables with expected value $0$ and variance $t^2$. Then, for any real vectors $\boldsymbol{a_1} \in \mathbb{R}^{n_1}, \boldsymbol{a_2} \in \mathbb{R}^{n_2}$ and matrices $\boldsymbol{B_1} \in \mathbb{R}^{n_1 \times n_1}, \boldsymbol{B_2} \in \mathbb{R}^{n_2 \times n_2}, \boldsymbol{B_3} \in \mathbb{R}^{n_1 \times n_2}$, we have

2.3.   Switched systems

Definition 2.11. (Switched system ^[8]) Switched systems consisting of several subsystems can be described as

In order to control a switched system, one needs to choose not only a continuous input but also a switching sequence.

Definition 2.12. A switching sequence in $t \in [t_0, t_f]$ regulates the sequence of active subsystems and is defined as

where $0 \leq K < \infty, t_0 \leq t_1 \leq t_2 \leq \cdots \leq t_K \leq t_f$, and $i_k \in I \ for \ k = 0, 1, \cdots, K$.

Note here $(t_k, i_k)$ indicates that at instant $t_k$, the system switches from subsystem $i_{k-1}$ to subsystem $i_k$. That is, during the time interval $[t_k, t_{k+1})$ ($[t_K, t_f]$ if $k = K$), subsystem $i_k$ is active.

Since many practical problems only involve optimizations in which a prespecified sequence of active subsystem $(i_0, i_1, \cdots, i_k)$ is given (for example, in the speeding up of an automobile power train which only requires switchings from gear 1–4), we concentrate on such problems. The switching instants $t^*_i |_{i = 1}^{K}$ and optimal control $u^*(t)$ are what we need to find out. So, optimal control for switched systems is not only to seek the optimal continuous input, but also to find the optimal switching rule to optimize a certain performance.

3.   Problem formulation

3.1.   Uncertain random continuous-time switched systems

Let us consider the following uncertain random continuous-time switched system:

where (ⅰ) $\boldsymbol{X}_{s}\in\mathbb{R}^{m_1}, \boldsymbol{Y}_{s}\in \mathbb{R}^{m_2}$ are the state vectors of the system at time $s$ with the initial conditions $\boldsymbol{X}_{0} = \boldsymbol{x}_{0}, \boldsymbol{Y}_{0} = \boldsymbol{y}_{0}$. (ⅱ) $\boldsymbol{u}_{s}\in\boldsymbol{U}_{s}\subset\mathbb{R}^{n_1}$ and $\boldsymbol{v}_{s}\in\boldsymbol{V}_{s}\subset\mathbb{R}^{n_2}$ are the control vectors; (ⅲ) The $\boldsymbol{p}_{i}: \mathbb{R}^{m_1}\times\mathbb{R}^{n_1}\times\mathbb{R}^{n_2}\times[\boldsymbol{t}_{i-1}, \boldsymbol{t}_{i}]\rightarrow \mathbb{R}^{m_1}$ and $\boldsymbol{p}_{i}^{'} : \mathbb{R}^{m_2}\times\mathbb{R}^{n_1}\times\mathbb{R}^{n_2}\times[\boldsymbol{t}_{i-1}, \boldsymbol{t}_{i}]\rightarrow \mathbb{R}^{m_2}$ are given vector-valued functions, and $\boldsymbol{Q}_{1} : \mathbb{R}^{m_1}\times\mathbb{R}^{n_1}\times\mathbb{R}^{n_2}\times[\boldsymbol{t}_{i-1}, \boldsymbol{t}_{i}]\rightarrow \mathbb{R}^{m_1 \times p_1}$ and $\boldsymbol{Q}_{2} : \mathbb{R}^{m_2}\times\mathbb{R}^{n_1}\times\mathbb{R}^{n_2}\times[\boldsymbol{t}_{i-1}, \boldsymbol{t}_{i}]\rightarrow \mathbb{R}^{m_2 \times p_2}$ are matrix-valued functions. (ⅳ) $\boldsymbol{C}_{s} = {(C_{s1}, C_{s2}, \cdots, C_{sp_1})}^{T}$, where $C_{s1}, C_{s2}, \cdots, C_{sp_1}$ are independent Liu processes, and $\boldsymbol{W}_{s} = {(W_{s1}, W_{s2}, \cdots, W_{sp_2})}^{T}$, where $W_{s1}, W_{s2}, \cdots, W_{sp_2}$ are independent standard Wiener processes. (ⅴ) $t_i|_{i = 1}^K$ are the switching instants and $t_i \leq t_{i+1} (i = 1, 2, \cdots, K)$, $t_0 = 0, t_{K+1} = T$ represent the initial and terminal instants, respectively.

For the control vectors $\boldsymbol{u}_{s}$ and $\boldsymbol{v}_{s}$, $d\boldsymbol{X}_{s}$ is defined by an uncertain differential equation, and $d\boldsymbol{Y}_{s}$ is defined by a random differential equation. Subjective uncertainty is described by the uncertain differential equation driven by the Liu process, while objective randomness is described by the random differential equation driven by the normal Wiener process. When both random and uncertain phenomena exist in a control system, we can describe such uncertain random phenomena by system (3.1).

Remark 3.1. $X_s, Y_s$ cannot be combined into one state vector. The reasons are described as follows. Note that the stochastic differential equation and uncertain differential equation are derived from the stochastic integral equation and uncertain integral equation, respectively.

The stochastic integral equation is determined by the Ito integral of the stochastic process with respect to the standard Wiener process. The Ito integral is introduced as follows: Let $Y_t$ be a stochastic process and let $W_t$ be a standard Wiener process. For any partition of closed interval $[a, b]$ with $a = t_1, t_2, \cdots, t_{k+1} = b$, the mesh is written as $\triangle = max \lim\limits_{1 \leq i \leq k} |t_{i+1}-t_i|$. Then, the Ito integral of $Y_t$ with respect to $W_t$ is

provided that the limit exists in the mean square and is finite.

The uncertain integral of an uncertain process with respect to the Liu process is introduced as follows. Let $X_t$ be an uncertain process and $C_t$ be a Liu process. For any partition of closed interval $[a, b]$ with $a = t_1, t_2, \cdots, t_{k+1} = b$, the mesh is written as $\triangle = max \lim\limits_{1 \leq i \leq k} |t_{i+1}-t_i|$. Then, the uncertain integral of $X_t$ with respect to $C_t$ is

provided that the limit exists almost surely and is finite.

Note that the Ito integral is an integral of the stochastic process concerning the standard Wiener process, and the corresponding limit exists in the mean square. However, the uncertain integral of the uncertain process concerning the Liu process requires the uncertain process to be integrable and the corresponding limit exists almost surely. For the uncertain stochastic integral introduced as follows,

where the symbols have the same mean as that in Eqs (3.2) and (3.3). Up to now, there is no convergence to make the corresponding limit exist. So, we can't combine them into one equation of state.

3.2.   Optimal control model

Based on an uncertain random switched system (3.1), an optimal control model is provided as

In the above model, the terminal-reward function is $S$: $\mathbb{R}^{m_1}\times\mathbb{R}^{m_2}\times[0, T]\rightarrow \mathbb{R}$, and $f$ is the objective function: $\mathbb{R}^{m_1}\times\mathbb{R}^{m_2}\times\mathbb{R}^{n_1}\times\mathbb{R}^{n_2}\times[0, T]\rightarrow \mathbb{R}$. The $f$ and $S$ are twice differentiable functions. $J(\boldsymbol{x}_{0}, \boldsymbol{y}_{0}, 0)$ is the expected optimal value in time interval $[0, T]$ with initial conditions $\boldsymbol{X}_0 = \boldsymbol{x}_0, \boldsymbol{Y}_0 = \boldsymbol{Y}_0$. The goal of such a problem requires finding not only the optimal control values $u_s^*, v_s^*$ but also the optimal switching instants $t_i^*|_{i = 1}^K$ in order to make the value function $J(x_0, y_0, 0)$ optimal.

Use $J(\boldsymbol{x}, \boldsymbol{y}, t)$ to denote the optimal value in $[t, T]$ with the condition that at time t we are in states $\boldsymbol{X}_{t} = \boldsymbol{x}, \boldsymbol{Y}_{t} = \boldsymbol{y},$

Remark 3.2. There are not only close relations but also essential differences among the uncertain optimal control problem, stochastic optimal control problem, and uncertain stochastic optimal control problem. The differences bring difficulties to the study of the uncertain stochastic optimal control problem. The first difficulty is how to describe an uncertain random dynamic including both objective randomness and human uncertainty and applying to the switched systems. The second difficulty is how to solve such kind of optimal control problem.

In order to deal with the difficulties, both stochastic differential equations and uncertain differential equations are used to describe an uncertain stochastic dynamic switched system. Inspired by previous research results on the stochastic optimal control problem and uncertain optimal control problem, the dynamic programming method and chance theory are adopted to solve the optimal control problem of the uncertain random switched system. Then, we deduced the equation of optimality to solve the optimal control problem.

Theorem 3.1. (Equation of optimality): Let $J(\boldsymbol{x}, \boldsymbol{y}, t)$ be twice differentiable on $\mathbb{R}^{m_1} \times \mathbb{R}^{m_2} \times [t_{i-1}, t_{i}).$ Then, we have

where $J_t(\boldsymbol{x}, \boldsymbol{y}, t)$ is the partial derivative of function $J(\boldsymbol{x}, \boldsymbol{y}, t)$ in $t$, $\triangledown_x J(\boldsymbol{x}, \boldsymbol{y}, t)$ and $\triangledown_y J(\boldsymbol{x}, \boldsymbol{y}, t)$ are the gradients of function $J(\boldsymbol{x}, \boldsymbol{y}, t)$ in $x$ and $y$, respectively, and $\triangledown_{yy} J(\boldsymbol{x}, \boldsymbol{y}, t)$ is the Hessian matrix of function $J(\boldsymbol{x}, \boldsymbol{y}, t)$ in $y$.

The proof of Theorem 3.1 is presented in the Appendix. The equation of optimality (3.7) is a useful tool for solving optimal control problems in continuous-time uncertain random switched systems. Furthermore, it provides an important condition for the existence of an extremum. If a solution to the equation exists, then the solution to the Eq (3.5) can be obtained by solving the optimal Eq (3.7).

Remark 3.3. If the dynamic system of model (3.5) degenerates into a stochastic dynamic system or an uncertain dynamic system, the equation of optimality is consistent with that in a random environment (HJB equation) or an uncertain environment (an equation of optimality presented in ^[23]). So, it is an extension of the equation of optimality from uncertain environment or random environment to uncertain random environment.

4.   Bang-bang control model for uncertain random switched systems

If the optimal control takes a value on its constraint boundary, it is called bang-bang control. Bang-bang control is an important integrated control method in engineering field ^[34] and servo systems ^[35], which has the characteristics of real-time control, fast response speed, and strong robustness. Bang-bang control was first studied in the 1950s arising from the time optimal control of deterministic systems ^[36], and then extended to various systems such as the time optimal control problem of distributed parameter systems ^[37].

The study for bang-bang control of stochastic systems or uncertain systems has been around for a long time and can be found in ^[38,39]. With the development of chance theory, we would like to further investigate bang-bang control in uncertain random environment on the basis of the above studies.

Bang-bang control problems with a linear objective function subject to a linear uncertain random switched system as follows are studied.

where the $\boldsymbol{\Psi}_{1s}, \boldsymbol{\Psi}_{2s}, \boldsymbol{\Psi}_{3s}, \boldsymbol{\Psi}_{4s}, \boldsymbol{H}_{is}, \boldsymbol{P}_{is}, \boldsymbol{\tilde{P}}_{is}, \boldsymbol{k}_{is}, \boldsymbol{Q}_{1s}, \boldsymbol{R}_{1s}, \boldsymbol{\tilde{R}}_{1s}, \boldsymbol{l}_{1s}, \boldsymbol{H}_{is}^{'}, \boldsymbol{P}_{is}^{'}, \boldsymbol{\tilde{P}}_{is}^{'}, \boldsymbol{k}_{is}^{'}, \boldsymbol{Q}_{2s}, \boldsymbol{R}_{2s}, \boldsymbol{\tilde{R}}_{2s}, \boldsymbol{l}_{2s}$ are functions of time $s$ with the suitable number of dimensions, and $\boldsymbol{\varphi}_{1T} \in \mathbb{R}^{m_1}, \boldsymbol{\varphi}_{2T} \in \mathbb{R}^{m_2}$ are constant vectors.

In order to implement the optimal control of model (4.1), we break it down into two stages. Stage(a) is an uncertain random optimal control problem under a given switching sequence. In this stage, $J(x_0, y_0, 0)$ is written as $J({x}_0, {y}_0, 0, {t}_{1}, \cdots, {t}_{K})$ because ${t}_{1}, \cdots, {t}_{K}$ are fixed. Stage(b) is an optimization problem by obtaining switching instants $t_i^*|_{i = 1}^K$. Applying optimality equation to stage(a), we have the following conclusion.

Theorem 4.1. Assume $J(\boldsymbol{x}, \boldsymbol{y}, t)$ is a twice differentiable function on $\mathbb{R}^{m_1} \times \mathbb{R}^{m_2} \times [t_{i-1}, t_i)\quad (i = 1, 2, \cdots, K+1),$ the optimal control inputs $\boldsymbol{u}_{t}^{(i)*} = ({u}_{1}^{(i)*}(t), {u}_{2}^{(i)*}(t), \cdots, {u}_{{n}_1}^{(i)*}(t))^T$, and $\boldsymbol{v}_{t}^{(i)*} = ({v}_{1}^{(i)*}(t), {v}_{2}^{(i)*}(t), \cdots, {v}_{{n}_2}^{(i)*}(t))^T$ of model (4.1) (${t}_{1}, \cdots, {t}_{K}$ are fixed) are the bang-bang control provided as

where $(\boldsymbol{\Psi}_{3t}^{T} + \boldsymbol{q}_{i}^{T}(t) \boldsymbol{P}_{it}+\boldsymbol{r}_{i}^{T}(t) \boldsymbol{P}_{it}^{'})_e$ is the eth element of vector $\boldsymbol{\Psi}_{3t}^{T} + \boldsymbol{q}_{i}^{T}(t) \boldsymbol{P}_{it}+\boldsymbol{r}_{i}^{T}(t) \boldsymbol{P}_{it}^{'}$, $(\boldsymbol{\Psi}_{4t}^{T} + \boldsymbol{q}_{i}^{T}(t) \boldsymbol{\tilde{P}}_{it}+\boldsymbol{r}_{i}^{T}(t) \boldsymbol{\tilde{P}}_{it}^{'})_f$ is the fth element of vector $\boldsymbol{\Psi}_{4t}^{T} + \boldsymbol{q}_{i}^{T}(t) \boldsymbol{\tilde{P}}_{it}+\boldsymbol{r}_{i}^{T}(t) \boldsymbol{\tilde{P}}_{it}^{'}$, for $e = 1, 2, \cdots, n_1, f = 1, 2, \cdots, n_2, i = 1, 2, \cdots, K+1$, and functions $\boldsymbol{q}_{i}(t) \in \mathbb{R}^{m_1}$ and $\boldsymbol{r}_{i}(t) \in \mathbb{R}^{m_2}$ satisfy the following equations:

The optimal value is

Proof. With the application of the equation of optimality (3.7), when $t \in [t_K, T]$, we get

Since

we assume $J(\boldsymbol{x}, \boldsymbol{y}, t) = \boldsymbol{q}_{K+1}^{T}(t) \boldsymbol{x} + \boldsymbol{r}_{K+1}^{T}(t) \boldsymbol{y} + \omega _{K+1}(t) \ (t \in [t_K, T])$ and $\boldsymbol{q}_{K+1}(T) = \boldsymbol{\varphi}_{1T}, \boldsymbol{r}_{K+1}(T) = \boldsymbol{\varphi}_{2T}, \omega _{K+1}(T) = 0$, so

Substituting Eq (4.8) into Eq (4.7), it holds that

We make

Let $\boldsymbol{u}_{t}^{(i)*} = ({u}_{1}^{(i)*}(t), {u}_{2}^{(i)*}(t), \cdots, {u}_{{n}_1}^{(i)*}(t))^T$ and $\boldsymbol{v}_{t}^{(i)*} = ({v}_{1}^{(i)*}(t), {v}_{2}^{(i)*}(t), \cdots, {v}_{{n}_2}^{(i)*}(t))^T$ be the solution of the preceding equations, $(\boldsymbol{\Psi}_{3t}^{T} + \boldsymbol{q}_{it}^{T} \boldsymbol{P}_{it}+\boldsymbol{r}_{it}^{T} \boldsymbol{P}_{it}^{'})_e$ be the eth element of vector $\boldsymbol{\Psi}_{3t}^{T} + \boldsymbol{q}_{it}^{T} \boldsymbol{P}_{it}+\boldsymbol{r}_{it}^{T} \boldsymbol{P}_{it}^{'}$, $(\boldsymbol{\Psi}_{4t}^{T} + \boldsymbol{q}_{it}^{T} \boldsymbol{\tilde{P}}_{it}+\boldsymbol{r}_{it}^{T} \boldsymbol{\tilde{P}}_{it}^{'})_f$ be the fth element of vector $\boldsymbol{\Psi}_{4t}^{T} + \boldsymbol{q}_{it}^{T} \boldsymbol{\tilde{P}}_{it}+\boldsymbol{r}_{it}^{T} \boldsymbol{\tilde{P}}_{it}^{'}$, for $e = 1, 2, \cdots, n_1, f = 1, 2, \cdots, n_2$, the optimal control inputs can be obtained when $t\in[t_K, T]$.

for $e = 1, 2, \cdots, n_1, f = 1, 2, \cdots, n_2$, and

Comparing both sides of Eq (4.9), we deduce the following equations where $\boldsymbol{q}_{K+1}(t), \boldsymbol{r}_{K+1}(T), \omega _{K+1}(T)$ should be satisfied as

Integrate $\frac{d\boldsymbol{\omega}_{K+1}(t)}{dt}$ from $t$ to $T$, $t \in [t_K, T]$, and we obtain

Furthermore,

where $\boldsymbol{q}_{K+1}(t)$ and $\boldsymbol{r}_{K+1}(t)$ satisfy the Riccati differential equation and boundary conditions (4.12) and (4.13).

When $t \in [t_{i-1}, t_i)$ for $i \leq K$, we assume

By the same methods as the above steps, we get

where $\boldsymbol{q}_{i}, \boldsymbol{r}_{i}, \boldsymbol{\omega}_{i}$ satisfy

Hence, the optimal value of model (4.1) (where $t_1, \cdots, t_K$ are fixed) is

□

The theorem is proved.

The Bellman optimality principle can be reduced to a basic recursive relationship, which continuously transfers the decision-making process and transforms a multistep optimal control problem into a series of one-step optimal control problems. In uncertain random continuous-time switched systems, the dynamic process of the system can still be decomposed into some single problems to optimize despite the uncertainty and randomness. By applying equation of optimalitity to get the optimal control and optimal value of the model, we divide it into $K$ time periods for optimization(see it in the proof of Theorem 4.1). Riccati differential equations should be solved at each time period one by one and backward.

Suppose that there are only two switched subsystems, then the optimal control model is presented below:

According to Theorem 4.1, we rewrite the cost function $J(\boldsymbol{x}_0, \boldsymbol{y}_0, 0, \boldsymbol{t}_{1})$ in the following form as

Denote $\tilde{J}(t_1) = J(\boldsymbol{x}_0, \boldsymbol{y}_0, 0, t_1)$. The next step is to minimize the objective function $\tilde{J}(t_1)$, where $t_1$ is the switching instant between two subsystems.

The above problem can be reformulated as minimizing a single-variable real function subject to constraint bounds, which can be addressed through nonderivative optimization techniques such as stochastic search, golden sectioning, or genetic algorithms.

Brent algorithm effectively integrates the strengths of golden section search and parabolic interpolation ^[40]. By incorporating parabolic interpolation into the golden section method for selecting the optimization iteration step size, Brent algorithm intelligently constrains the magnitude of the step size and ensures that the new interval contains the optimal solution.

Genetic algorithm serves as an optimization approach that imitates the biological evolution process, frequently employed to tackle intricate search and optimization challenges. This algorithm explores optimal solutions within the solution space by emulating fundamental principles of biological evolution, including genetic variation, mutation, and fitness.

Therefore, Brent algorithm and genetic algorithm are employed to solve the optimization problem in Stage(b). The performance is illustrated by the numerical example in the next section.

5.   Numerical example

Considering the following optimal control problem for uncertain random switched system:

where the states $\boldsymbol{X}_{s} = (X_{1s}, X_{2s})^T, \boldsymbol{Y}_{s} = (Y_{1s}, Y_{2s})^T$ with initial conditions $\boldsymbol{X}_{0} = \boldsymbol{x}_{0} = (\frac{1}{2}, \frac{1}{2})^T, \boldsymbol{Y}_{0} = \boldsymbol{y}_{0} = (1, \frac{1}{2})^T.$ The $\boldsymbol{\Psi}_{1s} = \boldsymbol{\Psi}_{2s} = (1, 1)^T, \boldsymbol{l}_{1s} = (\frac{1}{2}, \frac{1}{2})^T, \boldsymbol{l}_{2s} = (1, \frac{1}{2})^T, \boldsymbol{\varphi}_{11} = (1, 2)^T, \boldsymbol{\varphi}_{21} = (2, 2)^T$.

The first switching subsystem:

The second switching subsystem:

Stage(a): Fix $t_1$ and formulate $\tilde{J}(t_1)$ according to Theorem 4.1.

Denote $q_i(s) = (q_{i1}(s), q_{i2}(s))^T, r_i(s) = (r_{i1}(s), r_{i2}(s))^T$, and we know $q_2(1) = \boldsymbol{\varphi}_{11} = (1, 2)^T, r_2(1) = \boldsymbol{\varphi}_{21} = (2, 2)^T.$ It follows from Eqs (4.4) and (4.5) that

which has the solution

Hence,

It also follows from Eqs (4.4) and (4.5) that

and the solutions are

Hence,

According to the above results, we get

Stage (b): Find the optimal switching instant $t_{1}^{*}$.

Through the Brent algorithm, we choose the termination condition $tol < 0.01$ and obtain the optimal switching moment $t_1^* = 0.606$. The optimal cost is $-13.807$. Through the genetic algorithm, we initialize the population size to 100, the probability of intersection to 0.8, the probability of mutation to 0.3, the number of iterations to 100, and find the optimal switching moment $t_1^* = 0.604$. The optimal cost is $-13.807$, too. The optimization process of the two methods is shown in Figures 1 and 2.

From the results in Figures 1 and 2, we can see that there is no difference between the results of the Brent algorithm and genetic algorithm in solving this problem. According to the Eqs (5.5) and (5.9), we can get the optimal control inputs:

The optimal control inputs $u_s^{*}$ and $v_s^{*}$ are shown in Figures 3 and 4. From the figures, we can see that the optimal switching moment is 0.6, and there are two switching functions in each of the two switching subsystems that control the positive and negative of $u$ and $v$, respectively, where $u$ changes at 0.79 and $v$ at 0.24.

where

where

The distributions of variables $C_s, \int_{0}^{s} C_t dt, W_s, \int_{0}^{s} W_t dt$ are $\Phi_1(x), \Phi_2(x), \Phi_3(x), \Phi_4(x)$, respectively, where

We may get the sample points by $\Phi_1(\zeta_s) = \alpha, \Phi_2(\eta_s) = \alpha, \Phi_3(\kappa_s) = \alpha, \Phi_4(\nu_s) = \alpha$. The sample trajectories $\boldsymbol{X}_{s} = (X_{1s}, X_{2s})^T, \boldsymbol{Y}_{s} = (Y_{1s}, Y_{2s})^T$ can be given by

When assuming $\alpha$ as 0.1, we obtain the trajectories of $\boldsymbol{X}_{s} = (X_{1s}, X_{2s})^T$ and $\boldsymbol{Y}_{s} = (Y_{1s}, Y_{2s})^T$ in Figures 5 and 6.

Next, we derive the trajectory equations for $\boldsymbol{X}_{s} = (X_{1s}, X_{2s})^T, \boldsymbol{Y}_{s} = (Y_{1s}, Y_{2s})^T$ associated with the optimal control inputs $u_s^{*}$ and $v_s^{*}$. It follows from Eq (5.1) that:

When assuming $\alpha$ as 0.9, we obtain the trajectories of $\boldsymbol{X}_{s} = (X_{1s}, X_{2s})^T$ and $\boldsymbol{Y}_{s} = (Y_{1s}, Y_{2s})^T$ as in Figures 7 and 8.

By comparing Figure 5 with Figure 7, it can be concluded that the states of $X_{1s}$ and $X_{2s}$ show opposite trends over time. It means that when $\alpha$ takes different values, $C_s\ or\ (\int_{0}^{s} C_t dt)$ has more influence on the trajectories of $\boldsymbol{X}_{s} = (X_{1s}, X_{2s})^T$. Comparing Figure 6 with Figure 8, it can be seen that the trajectories of $Y_{1s}$ and $Y_{2s}$ over time are much higher when $\alpha$ is taken 0.9 than when $\alpha$ is taken 0.1. It shows that when $\alpha$ takes different values, $W_s\ or\ (\int_{0}^{s} W_t dt)$ has more influence on the trajectories of $\boldsymbol{Y}_{s} = (Y_{1s}, Y_{2s})^T$. In conclusion, uncertainty and randomness have effects on $X_s$ and $Y_s$, respectively. Therefore, it is necessary to consider both uncertainty and randomness in a system.

6.   Cash holding problem with a safe area constraint

The cash holding problems have always been an important topic in the management of corporate liquid assets. For enterprises, cash requirements are often highly volatile and unpredictable. However, based on historical experience and actual needs, control ranges determining the upper and lower limits of cash holdings can be calculated by companies. Miller and Orr ^[41] consider a cash inventory model in a random model. Reference ^[42] examines the cash holding problem in an uncertain environment. According to reference ^[43], the economic environment can be categorized into high volatility period and low volatility period, and different economic environments have an impact on the interest rates of both securities and cash. In addition, in the process of securities trading, we often need to consider the impact of transaction costs. Reference ^[44] discusses the three components of transaction costs: explicit cost, implicit cost, and the cost of missed trading opportunities. It is also pointed out that different kinds of market mechanisms and economic environments also affect the transaction costs in a country's capital market. Therefore, in this section, we will model an uncertain random switched system described by uncertain differential equations and stochastic differential equations affected by two economic environments, which, unlike general models, not only incorporates two switched subsystems, but also has to satisfy the constraint of cash in the safe area. Bellman's principle of optimality provides an effective framework that enables us to consider the optimal choice at each decision stage in this dynamically changing uncertain stochastic environment to achieve the optimal objective (e.g., maximizing the cash holding) over the entire time horizon. Based on this model, we can predict in advance the optimal switching moment in different economic environments that maximize the objective function, so that people can adjust their cash holding strategy if expectations are not met.

We denote the lower limit on cash holdings as $L$ and the upper limit as $H$. When the cash holding level exceeds the upper limit $H$, cash is used to purchase securities to reduce the level of cash holdings. In contrast, when the level of cash holdings is below the lower limit $L$, securities are sold for cash to increase the cash holdings level. If the cash holding level stays within the range $[L, H]$, then we do not need to change and keep the current cash level. When $s\in [t_{i-1}, t_i]\ (i = 1, 2)$, let $X_s$ and $Y_s$ denote the firm's cash assets and risky assets at time $s$, respectively, with initial conditions $X_0 = x_0$ and $Y_0 = y_0$. $\mu_i(s)$ denotes the interest rate on cash in the $i$th economic environment, and $\tilde{\mu_i}(s)$ denotes the interest rate on securities in the $i$th economic environment. $h_s$ denotes the sale price of the security, and when $h_s$ is negative it denotes the purchased price of the security with $-H_2 \leq h_s \leq H_1, H_1 \geq 0, H_2 \geq 0$. The $l_{1s}dW_s$ is the fluctuation of cash balance and the $l_{2s}dC_s$ is the fluctuation of securities balance. The $\theta_i$ denotes the transaction cost of the security in the $i$th economic environment. Thus, the cash holding problem is to find the optimal switching moment that maximizes $X_s+Y_s$ over a finite time range $[0, T]$ as well as the optimal cash holding level. The optimal cash holding problem for an uncertain random switched system in a finite time range of $[0, T]$ is proposed below.

Because $|h_s|$ is an absolute value function of the control input, we can rewrite the $|h_s|$ as the difference between two nonnegative variables

where $u_s$ denotes the fraction of cash transformed to risky assets at time $s$, and $v_s$ denotes the fraction of risky assets transformed to cash assets at time $s$. In order to make $h_s = v_s$ when $u_s$ is strictly positive, and $h_s = -u_s$ when $v_s$ is nonnegative, we also apply the quadratic constraint as

so that one of $u_s$ and $v_s$ must be 0. The reason is that cash holdings cannot be both greater than $H$ and less than $L$, which means $u_s$ and $v_s$ cannot occur simultaneously. Based on Eqs (6.2) and (6.3), we may rewrite $|h_s|$ as

Thus, the following is an optimal control problem for a linear uncertain random switched system:

According to Eq (6.3), we can solve Eq (6.5) into two cases.

Case 1. For $X(s) > H (s\in [t_{i-1}, t_i], i = 1, 2)$, we need to convert cash into risk assets at this time. At time $s$, the transformed cash holdings are $X(s) - u(s) - \theta_i u(s)$. The level of cash holdings must be within the safety area $[L, H]$, therefore, $\frac{X(s)-H}{1+\theta_i} \leq u_s \leq \frac{X(s)-L}{1+\theta_i}$. We note that $U = [\frac{X(s)-H}{1+\theta_i}, \frac{X(s)-L}{1+\theta_i}]$. The optimal cash holding problem for uncertain random switched systems in the time range of $[0, T]$ is proposed as

where $t_0 = 0, t_2 = T$. By comparing (6.6) with (5.1), $\varphi_{1T} = \varphi_{2T} = -1$ and ${\Psi}_{1s} = {\Psi}_{2s} = 0$ can be obtained. The first switched subsystem: $H_{1s} = \mu_1(s), P_{1s} = -1 - \theta_1, H_{1s}^{'} = \tilde{\mu}_1(s), P_{1s}^{'} = 1$. The second switched subsystem: $H_{2s} = \mu_2(s), P_{2s} = -1 - \theta_2, H_{2s}^{'} = \tilde{\mu}_2(s), P_{2s}^{'} = 1$. All other values are $0$.

According to Eq (4.2), we can get

Then, the optimal control is provided as

Furthermore, the optimal cash holding is

Moreover, for $t = 0$ with the initial conditions $X_0 = x_0$ and $Y_0 = y_0$, by Theorem 4.1, the cost function is written as

Then the optimization algorithm is used to solve the optimal switching moment $t_1$ for function $J(\boldsymbol{x}_0, \boldsymbol{y}_0, 0, \boldsymbol{t}_{1})$, where $t_1$ is the optimal switching moment in different economic environments that maximizes the objective function.

Function (6.9) indicates that in the $i$th economic environment, if the expected value of $1+\theta_i$ units cash assets is greater than the expected value of $1$ unit risky assets, then the optimal cash holding is $H$. Otherwise, the optimal cash holding level is $L$.

Case 2. For $X(s) < L(s\in [t_{i-1}, t_i], i = 1, 2)$, we need to transform risky assets into cash assets at this time. At time $s$, the transformed cash holdings are $X(s) + v_s - \theta_i v_s$. The number of cash holdings must be within the safety area $[L, H]$, therefore, $\frac{L-X(s)}{1-\theta_i} \leq v_s \leq \frac{H-X(t)}{1-\theta_i}$. We note that $V = [\frac{L-X(s)}{1-\theta_i}, \frac{H-X(s)}{1-\theta_i}]$. The optimal control model is provided as

where $t_0 = 0, t_2 = T$. By comparing (6.11) with (5.1), $\varphi_{1T} = \varphi_{2T} = -1$ and ${\Psi}_{1s} = {\Psi}_{2s} = 0$ can be obtained. The first switched subsystem: $H_{1s} = \mu_1(s), \tilde P_{1s} = 1 - \theta_1, H_{1s}^{'} = \tilde{\mu}_1(s), \tilde P_{1s}^{'} = -1$. The second switched subsystem: $H_{2s} = \mu_2(s), \tilde P_{2s} = 1 - \theta_2, H_{2s}^{'} = \tilde{\mu}_2(s), \tilde P_{2s}^{'} = -1$. All other values are $0$.

According to Eq (4.3), we can get

Then, the optimal control is provided as

Furthermore, the optimal cash holding is

Similarly, in Case 2, for $t = 0$ with the initial conditions $X_0 = x_0$ and $Y_0 = y_0$, by Theorem 4.1, the cost function is written as

and the optimal switching moment $t_1$ can be found by the optimization algorithm to maximize the objective function so that we can adjust our cash holding strategy if expectations are not met.

Equation (6.14) indicates that in the $i$th economy environment, if the expected value of $1-\theta_i$ units cash assets is greater than the expected value of $1$ unit risky assets, then the optimal cash holding is $H$. In contrast, the optimal cash holding level is $L$.

7.   Conclusions

In this paper, the optimal control problem of uncertain random continuous-time switched systems is presented. In order to solve this problem, the optimality equation is generalized to uncertain random switched systems. Using the optimality equation, we obtain the analytical solution and optimal bang-bang control for a kind of control problem when the corresponding performances are linear. The two-stage algorithm is applied to implement optimal control. Then, a numerical example is provided to illustrate the effectiveness of the proposed method. Finally, the optimal control model and optimality equation for uncertain stochastic switching systems are applied to derive the optimal cash holdings under different economic cycles. In order to consider the influence of some external extreme events or noises on switched systems, in future studies, expected value models and optimistic value models of the optimal control problem for uncertain switched systems with jump will be studied.

Author contributions

Yang Chang: Conceptualization, Methodology, Software, Formal Analysis, Writing-Original Draft; Guangyang Liu: Visualization, Writing-Original Draft; Hongyan Yan: Validation, Supervision, Writing-Review & Editing. All authors have read and approved the final version of the manuscript for publication.

Use of Generative-AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Conflict of interest

The authors declare that there is no conflict of interest.

Appendix

The proof of Theorem 3.1.

Proof. For any $\triangle t$ with $t+\triangle t \in [t_{i-1}, t_i)$, denote $\boldsymbol{X}_{t+\triangle t} = \boldsymbol{x} + \triangle \boldsymbol{X}_{t}, \boldsymbol{Y}_{t+\triangle t} = \boldsymbol{y} + \triangle \boldsymbol{Y}_{t}.$ By using Taylor series expansion, we have

Denote

Then, the expansion (.1) may be rewritten as

Since

we have

Denote

Then, we can simplify Eq (.1) as

It follows from the principle of optimality in ^[45] that

By principle of optimality in ^[45] and Lemma 2.1, it holds that

So, Eq (.2) may be simply expressed as

The equation subtracts $J(\boldsymbol{x}, \boldsymbol{y}, t)$ at the same time, and we can get

Both sides of Eq (.3) are divided by $\triangle t$ at the same time and make $\triangle t \rightarrow 0$, thus, we can obtain the conclusion. The theorem is proved. □