Adaptive learning nonsynchronous control of nonlinear hidden Markov jump systems with limited mode information

1.   Introduction

During these past years, a significant research trend is to investigate the intelligence-based strategies instead of conventional control approaches for control theory and engineering in many areas. For instance, much success has been achieved in robotic systems ^[1,2], power systems ^[3,4], traffic systems ^[5] and so on ^[6,7]. One of the primary purposes of intelligent control methods is to deal with uncertain or unknown nonlinear complex dynamics. Among the various methods, artificial neural network (NN) has emerged as an effective tool owing to its capability of approximating any continuous nonlinearities ^[8]. As a consequence, significant research results working on NN-based control approaches have been reported in the literature ^{[9,10,11,12,13,14,15]}. These remarkable investigations have proven that the integration of the controller with NN can well improve the dynamical control performance of complex nonlinear systems ^[16,17]. So far, there are two general and main learning strategies for the NN: the gradient-descent learning ^[18,19] and the adaptive learning methods ^[20,21]. Especially, the adaptive learning strategy has the main advantages of online characteristics and guaranteed accuracy, while the convergence of NN can be effectively ensured during its learning processes. Meanwhile, it is noticed that the NN approximation method would still lead to certain approximation error to some extent, such that only uniformly boundedness can be achieved for most closed-loop control systems. In order to overcome this problem, some notable methods have been developed for compensating the NN approximation errors, which include sliding mode control ^[22,23], $H_{\infty}$ control ^[24] and so on.

On another active research front, as a significant sort of hybrid systems, Markov jump systems (MJSs) have been extensively investigated in many practical scenarios, which is mainly due to their notable features in describing jumping system structures or parameters ^{[25,26,27,28]}. Generally speaking, the system mode jumping for subsystems of the MJSs is conducted according to the transition probability rates. As is well known, the transition probabilities of jumping modes are supposed to be prior known conditions for the MJSs ^[29,30]. Nevertheless, these assumptions are always difficult to be satisfied in most realistic applications, especially for certain limited or vulnerable networked control environments ^[31,32]. In this circumstance, some initial research interests have been paid to the hidden Markov jump systems (HMJSs), which implies that only limited mode information can be acquired for analysis and synthesis procedures ^[33,34]. Obviously, it is more challenging for the mode-dependent designs that depend upon system mode information, although mode-dependent controllers can lead to less conservatism. Hence, a recently developed mode-observation based control strategy is reported to release the limitation on mismatched modes between the true system modes and the observed system modes ^[35,36]. The key idea is to utilize the estimated system modes instead of the true system modes, such that a conditional probability based strategy can be developed. More precisely, a nonsynchronous feature would be encountered for the mode-dependent controllers under this context. As a result, the nonsynchronous controllers for HMJSs are reasonable and important. Moreover, it should be pointed out that the mode transition probabilities could also be further restricted with partly unknown cases. However, to the authors' best knowledge, there are few existing results on the nonsynchronous control issues of HMJSs with limited mode information, let alone that with unknown nonlinear dynamics.

Motivated by the aforementioned insight, this paper presents the nonsynchronous control methodology of HMJS with unknown nonlinearities and external disturbances. In comparison with most reported works, our paper contributes to the literature with two main aspects and novelties. Firstly, a NN-based control strategy associated with the adaptive online learning law is introduced to handle the unmodeled nonlinear dynamics and external disturbances of HMJS. Especially, the mode-dependent nonlinearities in HMJS are taken into account for more applicability due to the mode jumping. Secondly, a novel nonsynchronous mode-dependent controller is applied to cope with the influences of limited mode information in HMJS by considering that the mode information between the HMJS and the corresponding controller could be different. Furthermore, the limited mode information is considered as the partly accessible transition rates, which are more general cases for HMJS in practical applications. Finally, the optimal $H_{\infty}$ performance index is employed with the developed nonsynchronous controller for optimizing the NN approximation error and external disturbance, such that a desired control performance can be achieved.

The rest of the article is organized as follows: in Section 2, some necessary preliminaries of HMJS are given and the nonsynchronous controller along with the online learning NN are introduced. Section 3 constructs the mode-dependent Lyapunov function and presents sufficient conditions for guaranteeing a desired control performance. Based on these derived conditions, the nonsynchronous controller and NN learning law design procedures are presented. In Section 4, an illustrative example of a robotic manipulator is provided to demonstrate the usefulness of the established theoretical findings. Finally, Section 5 draws the conclusions of this paper with further research perspectives.

$\bf{Notation}$: $\mathbb{R}^{n}$ stands for $n$-Euclidean space. Matrix $P \succ 0$ means that $P$ is positive definite, $tr(P)$ represents the trace of $P$ and * represents the ellipsis parts in symmetric block matrices. $(\mathbb{O}, \mathbb{F}, \mathbb{P})$ corresponds a probability space. $\mathcal{E}$ denotes mathematical expectation. All matrices are set to have compatible dimensions.

2.   Problem formulation and preliminaries

Consider the following nonlinear HMJS:

where $x(t) = [x_{1}(t), x_{2}(t), \ldots, x_{n}(t)]^{T}\in \mathbb{R}^{n}$ denotes the system state, $f(\sigma _{t}, x(t)):\mathbb{R}\times \mathbb{R}^{n}\rightarrow \mathbb{R}^{n}$ represents an unknown nonlinear function, $u(t)\in \mathbb{R} ^{m}$ stands for the control input and $d(t)\in \mathbb{R}^{l}$ is an unknown disturbance; $\sigma _{t}\in \mathcal{S} = \{1, \ldots, \mathcal{N}\}$ corresponds the continuous-time discrete-state Markov process in $(\mathbb{O}, \mathbb{F}, \mathbb{P})$, whose transition probability matrix is defined by $\Pi = (\pi _{ij})_{\mathcal{N} \times \mathcal{N}}$ with

and $\pi _{ii} = -\underset{j = 1, {\text{ }}i\neq j}{\overset{\mathcal{N}}{\sum\limits }} \pi _{ij}.$ Consequently, $A(\sigma _{t})$ and $B(\sigma _{t})$ are known system matrices with fixed system mode $\sigma _{t}$. Furthermore, the infinitesimal operator $\mathcal{L}$ for function $V(t, i)$ can be defined as follows:

In addition, the transition probability rates are further assumed to be partly accessible to describe limited mode information, i.e.,

such that we can denote $\mathcal{R}^{i} = \mathcal{R}_{k}^{i}\cup \mathcal{R} _{uk}^{i}$, where $\mathcal{R}^{i}$ represents the $i$th row of $\Pi$ and

Without loss of generality, denote $\mathcal{R}^{i}$ as $\mathcal{R}^{i} = \{\pi_{k1}, \pi_{k2}, \ldots, \pi_{k\mathcal{L}}\}$ with $0 \leq \mathcal{L}\leq \mathcal{N}$ ^[37].

Remark 1. It is worth mentioning that partly accessible transition rates are more general cases for MJSs with all known transition rates, especially for HMJSs. When $\mathcal{R}_{uk}^{i} = \varnothing$, all accessible transition rates can be derived accordingly.

In order to deal with the unknown nonlinearity and disturbance, a dynamical NN $\mathcal{W}\varphi (x(t))$ is adopted to reconstruct approximation. Then, $f(\sigma _{t}, x(t))+d(t)$ can be described by the following:

where $e(t)$ implies the approximate error, $\mathcal{W}^{\ast }(\sigma _{t}) = \arg \min_{ \mathcal{W}(\sigma _{t}, t)\in \Omega _{\mathcal{W}}}\left\{ \sup_{x\in \Omega _{x}}\Vert e(t)\Vert \right\}$ denotes the learned NN weights for the optimized minimization value of $e(t)$.

Remark 2. It is noteworthy that the NNs have the ability to approximate unknown nonlinearities with high accuracy when the corresponding weights are well chosen. As a result, if the adaptive learning laws can be carefully designed, the desired nonlinearities reconstruction can be obtained accordingly.

As a result, the approximation of $f(\sigma _{t}, x(t))+d(t)$ can be described as follows:

where $\mathcal{\hat{W}}(\sigma _{t}, t)$ represents the estimation of $\mathcal{W}^{\ast }(\sigma _{t}).$

Subsequently, the following nonsynchronous mode-dependent controller with NN is designed based on observed system mode information

where $K(r_{t})$ denotes controller gains to be determined and $r_{t}\in \mathcal{T} = \{1, \ldots, \mathcal{M}\}$ denotes the stochastic process with the following conditional probability:

Remark 3. It is noted that the controller only utilizes the observed mode information of true system modes with conditional probability. As a result, the nonsynchronous feature should be considered during the controller design. Meanwhile, the integration of dynamical NN in the controller design can further improve the applicability in certain practical applications.

By applying the above controller, the closed-loop dynamics can be represented as follows:

where $\mathcal{\tilde{W}}(\sigma _{t}, t) = \mathcal{W}^{\ast }(\sigma _{t})-\mathcal{\hat{W}}(\sigma _{t}, t)$.

For simplicity, denote $\sigma _{t} = i$ and $r_{t} = \rho$, respectively. Then, we can rewrite system (2.8) by the following:

Before proceeding further, the following definitions are introduced to deal with the NN approximation error:

Definition 1. System (2.8) is said to achieve stochastic stability in the mean-square sense if it holds that

for any initial conditions.

Definition 2. System (2.8) is said to achieve $H_{\infty }$ performance in the mean-square sense if there exists a constant $\gamma > 0$ such that

for zero initial conditions.

To this end, an important lemma is also employed for deriving the main results.

Lemma 1. ^[38] Given real matrices $\mathcal{A}$, $\mathcal{B}$, $\mathcal{C}$, $\mathcal{X}$, $\mathcal{W}_{1}$, $\mathcal{W}_{2}$ with appropriate dimensions, if there exists a matrix $\mathcal{P} \succ 0$ it satisfies that

then there exist a matrix $\mathcal{Z} \succ 0$ and a positive scalar $\mu > 0$ such that

3.   Main results

This section derives sufficient conditions for the resulting closed-loop HMJS and designs the nonsynchronous mode-dependent controller along with the NN updating laws by proven details.

Theorem 1. System (2.8) can achieve the mean-square $H_{\infty }$ performance with given controller gain $K(\rho)$, $\rho \in \mathcal{T}$ and parameter $\gamma > 0$, if there exist mode-dependent matrices $P(i) \succ 0$ and $Q^{T}(i) = Q(i)$, $i \in \mathcal{S}$, such that the following linear matrix inequalities hold, where

Based on the above condition, the online NN learning law can be designed by

where $\Omega \succ 0$ is a constant matrix.

Proof 1. For given mode $i$, select the mode-dependent Lyapunov function candidate as follows:

Taking the infinitesimal operator along with evolution of $V(t, i)$ yields

Subsequently, it can be verified that

Moreover, based on the trace property of matrices, it holds that

Hence, it can be obtained by the online learning law that

which implies that when

holds, then it can lead to

Therefore, the desired $H_{\infty}$ performance can be satisfied according to Definition 2.

On the other hand, there must exist a mode-dependent matrix $Q(i)\succ 0$ such that

and this means that

As a result, it can be further deduced that

By invoking the fact that

it can be verified that the established conditions in Theorem 1 can ensure the $H_{\infty}$ performance and the proof is therefore completed.

Remark 4. It is worth mentioning that the online NN learning law in Theorem 1 is developed according to the Lyapunov function with a desired $H_{\infty}$ performance. As such, with the desired nonlinearity approximation accuracy, the effect of unknown nonlinear function and external disturbance can be well solved.

Remark 5. Since the transition rates of the jumping process play a significant role for the controller design of HMJSs, the partly accessible modes are considered for more generalization ability. Moreover, another mode-dependent weighting matrix is adopted to deal with the transition rate relations, which can bring more mode-dependent controller design flexibility.

Remark 6. Theorem 1 gives sufficient conditions in terms of linear matrix inequalities, which can be conveniently solved with feasible solutions. With the help of matrix manipulation technique, Theorem 2 will calculate the mode-dependent controller gains.

Theorem 2. System (2.8) can achieve the mean-square $H_{\infty }$ performance with parameter $\gamma > 0$, if there exist mode-dependent matrices $\tilde{P}(i)\succ 0$, $\tilde{Q}^{T}(i) = \tilde{Q}(i)$, $i \in \mathcal{S}$, $\tilde{K}(\rho)\succ$, $\rho \in \mathcal{T}$ and matrix $Z \succ 0$, such that the following linear matrix inequalities hold, where

Based on the above conditions, the nonsynchronous mode-dependent controller gains can be calculated by

and the online NN learning law is designed by

where $\Omega \succ 0$ is a constant matrix.

Proof 2. Denote

and perform congruent transformation to $\Theta (i) < 0$ with $\tilde{ P}(i)$. Then, by applying Lemma 1, the rest of proof can be directly obtained by Theorem 1.

Based on the above results, the following controller design algorithm can be presented:

Algorithm 1. Require: Input: $A(i)$, $B(i)$, and $\gamma$, Output: $K(\rho)$

Ensure:

1: Given system parameters $A(i)$, $B(i)$, and $\gamma$

2: Solving the linear matrix inequalities in Theorem 2

3: Setting the learning law with $\dot{\hat{W}}(i, t)=\Omega P(i)x(t)\varphi ^{T}(x(t))$

4: Obtaining the matrices with $\tilde{K}(\rho)$ and $Z$

5: return $K(\rho)=\tilde{K}(\rho)Z^{-1}$

Remark 7. The above established linear matrix inequalities are strict convex optimization conditions, which can be effectively solved by Matlab LMI toolbox once the system parameters are given. It can be found that the feasibility and complexity is related to the limited mode information, such that the computational efficiency should be considered by selecting appropriate models.

Remark 8. The optimized minimization value of $\gamma$ can be further obtained by solving the convex optimization problem:

4.   Illustrative example

This section performs the validation simulation via a robotic manipulator example, where different loads modeled by hidden Markov chain are considered.

Choose the following single-link robotic manipulator model as depicted in Figure 1 ^[39]:

where $\vartheta (t)$ and $\theta (t)$ denote the angular position and velocity, respectively. $J(i)$ represents the moment of inertia$,$ $D(i)$ stands for the coefficient of viscous friction, $M(i)$ denotes the mass of payload, $g$ denotes the gravity acceleration and $L$ denotes the manipulator length.

In practical applications of robotic manipulators, the manipulator parameters are always kept varying, especially for the cases in unstructured working environments. One typical example is a transportation manipulator performing carrying up and laying down tasks. Since the loads are often different, the actual robotic manipulator parameters of inertias and mass centers are jumping according to the different loads. Under this context, it is reasonable and necessary to model the robotic manipulator as MJS or HMJS.

Defining $x(t) = [\vartheta (t), \theta (t)]^{T}$ and considering the disturbance $d(t)$, it can be derived that

where

In the simulation, it is assumed that there are three operation modes for the robotic manipulator, and the corresponding parameters are set as

such that the state-space system parameters can be obtained by

and the nonlinear functions $f(i, x(t))$ and $d(t)$ are assumed to be unknown.

Consequently, it is supposed that $\sigma _{t} = i$ has three jumping modes and its corresponding transition probability matrix is given by

Furthermore, the conditional probability for mode observation is set as

such that a nonsynchronous mode information can be utilized in the controller.

Based on these established results in Theorem 2 with $\Omega = I$, $\gamma = 2$, $\mu = 1$, the controller gains are obtained as follows:

Setting initial conditions as $x(t) = [5, 2]^{T}$ and the parameters of NN as

Figure 2 depicts the system jumping modes and controller observed jumping modes. The closed-loop system trajectories are shown in Figure 3 and the NN learning errors can been seen in Figure 4, where the approximation errors can converge well. Therefore, the simulation results can support our developed theoretical results. In addition, Figure 5 illustrates the advantages of using the NN-based control method for unknown nonlinear dynamics, where a better control performance can be acquired.

Furthermore, in order to illustrate the effect of limited mode information on optimal $H_{\infty}$ performance index, Table 1 is presented.

Based on the above optimization results, one can observe that the minimized $H_{\infty}$ performance $\gamma$ can be obtained by linear matrix inequality conditions in Theorem 2. On the other hand, a prescribed $H_{\infty}$ performance can also be set in the controller design processes. Moreover, it can be found that a better optimal $H_{\infty}$ control performance can be obtained by more mode information, which implies a tradeoff during the mode-dependent design procedure with limited modes.

5.   Conclusions

This paper addresses the adaptive NN learning control issue of nonlinear HMJSs with limited transition probability information. More precisely, a nonsynchronous mode-dependent controller is designed and its NN adaptive learning law is developed accordingly. In order to deal with the approximation error for unknown nonlinearities and disturbances, the $H_{\infty}$ control performance is employed. By constructing a stochastic Lyapunov function, sufficient criteria are established and the controller gains are determined by convex optimization. The simulation results of a robotic manipulator verify that our developed control scheme is effective for HMJSs. Our future work would consider the hidden semi-Markov jump systems, whose transition probabilities are more applicable for certain practical implementations. In addition, as the mode information would be time-varying and inaccessible in the hidden semi-Markov jump systems, it is more desirable to develop the nonsynchronous mode-dependent control strategies.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant 61703038, 61627808, 62173326, the Fundamental Research Funds for the Central Universities under Grant FRF-GF-20-11B, CAS Project for Young Scientists in Basic Research YSBR-041, and Major program of the National Natural Science Foundation of China T2293720/T2293724.

Conflict of interest

The authors declare no potential conflict of interest.

Data availability statement

Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.