
Citation: Ravi Agarwal, Snezhana Hristova, Donal O’Regan, Radoslava Terzieva. Stability properties of neural networks with non-instantaneous impulses[J]. Mathematical Biosciences and Engineering, 2019, 16(3): 1210-1227. doi: 10.3934/mbe.2019058
[1] | Guodong Li, Ying Zhang, Yajuan Guan, Wenjie Li . Stability analysis of multi-point boundary conditions for fractional differential equation with non-instantaneous integral impulse. Mathematical Biosciences and Engineering, 2023, 20(4): 7020-7041. doi: 10.3934/mbe.2023303 |
[2] | Luyao Li, Licheng Fang, Huan Liang, Tengda Wei . Observer-based event-triggered impulsive control of delayed reaction-diffusion neural networks. Mathematical Biosciences and Engineering, 2025, 22(7): 1634-1652. doi: 10.3934/mbe.2025060 |
[3] | Liping Wu, Zhongyi Xiang . A study of integrated pest management models with instantaneous and non-instantaneous impulse effects. Mathematical Biosciences and Engineering, 2024, 21(2): 3063-3094. doi: 10.3934/mbe.2024136 |
[4] | Trayan Stamov, Gani Stamov, Ivanka Stamova, Ekaterina Gospodinova . Lyapunov approach to manifolds stability for impulsive Cohen–Grossberg-type conformable neural network models. Mathematical Biosciences and Engineering, 2023, 20(8): 15431-15455. doi: 10.3934/mbe.2023689 |
[5] | Yang Wang, Yongyang Liu, Yansheng Liu . Total controllability of non-autonomous second-order measure evolution systems with state-dependent delay and non-instantaneous impulses. Mathematical Biosciences and Engineering, 2023, 20(2): 2061-2080. doi: 10.3934/mbe.2023095 |
[6] | Jian Meng, Bin Zhang, Tengda Wei, Xinyi He, Xiaodi Li . Robust finite-time stability of nonlinear systems involving hybrid impulses with application to sliding-mode control. Mathematical Biosciences and Engineering, 2023, 20(2): 4198-4218. doi: 10.3934/mbe.2023196 |
[7] | Ganesh Kumar Thakur, Sudesh Kumar Garg, Tej Singh, M. Syed Ali, Tarun Kumar Arora . Non-fragile synchronization of BAM neural networks with randomly occurring controller gain fluctuation. Mathematical Biosciences and Engineering, 2023, 20(4): 7302-7315. doi: 10.3934/mbe.2023317 |
[8] | Haitao Zhu, Xinrui Ji, Jianquan Lu . Impulsive strategies in nonlinear dynamical systems: A brief overview. Mathematical Biosciences and Engineering, 2023, 20(2): 4274-4321. doi: 10.3934/mbe.2023200 |
[9] | Yanjie Hong, Wanbiao Ma . Sufficient and necessary conditions for global attractivity and stability of a class of discrete Hopfield-type neural networks with time delays. Mathematical Biosciences and Engineering, 2019, 16(5): 4936-4946. doi: 10.3934/mbe.2019249 |
[10] | Biwen Li, Yujie Liu . Average-delay impulsive control for synchronization of uncertain chaotic neural networks with variable delay impulses. Mathematical Biosciences and Engineering, 2025, 22(6): 1382-1398. doi: 10.3934/mbe.2025052 |
Modeling of activity represents, manipulates and communicates real-world objects of daily life [11]. One of the basic tools in mathematical models for representing the biological processes of the human brain are neural networks. These are also commonly referred to as parallel distributed processing models or simply neural nets. The first mathematical model of a neural network is attributed to McCulloch and Pitts [17] who in 1943 showed that any logical function could be configured by a parallel distributed processing system of interconnected digital neurons. A very concise summary of the foundations of neural networks was provided in the book [4]. Various models of neural networks are used to consider specific tasks and in order to specify a particular neural network model it suffices to define its architectural, computational, and adaptive dynamics. The computational dynamics specifies the network initial state and a rule for its updates in time, providing that the network topology and configuration are fixed. At the beginning of the computational mode, the states of input neurons are assigned to the network input and the remaining neurons find themselves in the initial state. All potential network inputs and states form the input and state space of neural network, respectively. In dynamic (recurrent, feedback) systems, the current output signals depend, in general, on current and past input signals. There are two equivalent classes of dynamic systems: continuous-time and discrete-time systems. Continuous-time dynamic systems operate with signals which are functions of a continuous variable, that is continuous functions of time and they are described by means of differential equations.
The main objective of this paper is to study stability of continuous-time neural networks without assuming the smoothness, monotonicity and unboundedness of the activation function. The model introduced is a generalization of some additive neural networks. These type of models can simulate biological neural networks and artificial intelligence systems, from the view point of reality, and it should also naturally take into account evolutionary processes of the biological systems as well as disturbances of external influence.
In controlling nonlinear systems, the Lyapunov second method provides a way to analyze the stability of the system without explicitly solving the differential equations. Stability results concerning neural networks can be found in [10,15]. One type of stability, very useful in real world problems, is the so called Lipschitz stability. Dannan and Elaydi [5] introduced the notion of Lipschitz stability for ordinary differential equations and as noted in [5] this type of stability is important only for nonlinear problems, since it coincides with uniform stability in linear systems.
Sometimes neural networks are subjected to certain impulsive state displacements at fixed moments of time. Stability problems of neural networks under the presence of instantaneous impulses was considered in [2,3,6]. Finite-time stability of systems with instantaneous impulses was studied in [16] and for delays see [13,14]. In the case when the time acting of these state displacements is not negligibly small, this phenomenon is modeled by so called non-instantaneous impulses.
In [19] the speed of the neural impulse is discussed and the reason why modeling with instantaneous impulses is not adequate is considered. As a result, we will consider non-instantaneous impulses in modeling neural impulses.
In this paper, we study stability properties of neural networks with non-instantaneous impulses, i.e. perturbations which start their actions abruptly at some points and then continue on given finite intervals. We study the general case when
− the self-regulating parameters of all units are variable in time;
− the functions of the connection between two neurons in the network are variable in time bounded functions;
− the activation functions are non-Lipschitz (usually in the literature only the case of Lipschitz activation functions is studied).
In this paper, the investigation is based on the Lyapunov method. Some Lipschitz stability sufficient criteria are provided by using Lyapunov functions (not only quadratic ones). The results are illustrated on several particular neural networks.
We will consider the model proposed by Hopfield [7] and known as Hopfield's graded response neural network in the case when the neurons are subject to a certain impulsive state displacement at fixed moments and the duration of this displacement is not negligible small (they are known as non-instantaneous impulses). The model is based on an analog circuit consisting of capacitors, resistors and amplifiers. The input-output characteristics of the amplifiers in the circuit are modeled by some functions known as activation functions. In the literature these functions are assumed to satisfy a wide variety of assumptions, mainly they are Lipschitz functions. A neuron in the circuit is modeled by an elementary sub-circuit composed of an amplifier, a reverse amplifier, a capacitor and a resistor. The neural network is itself constructed from such elementary neurons interconnected by resistors which are analogous to biological synapses. The biological counterpart of inhibitory synapses is realized by reverse amplifiers. Usually there are two sets of inputs to the neurons. The first set of inputs are the currents I1,I2,…,In which are external inputs and often in the literature are considered as constants. The second set consists of those arising from feedback connections from other op-amps including possible self-connections. Resistances are always positive and a negative resistance to a neuron can be implemented by choosing an inverted output from certain amplifiers.
In this paper, we will assume two increasing sequences of points {ti}∞i=1 and {si}∞i=1 are given such that 0<si≤ti<si+1, i=1,2,…, and limk→∞sk=∞.
Let t0∈R+=[0,∞) be a given arbitrary point. Without loss of generality we will assume that 0≤t0<s1.
Remark 1. In the case tk=sk, k=1,2,… we obtain the case of instantaneous impulses and the so called fixed points of impulses (see, for example [8,12]).
If for the natural number k: tk≠sk then the interval (sk,tk] is called the interval of non-instantaneous impulses (see [1]).
Consider the general model of Hopfield's graded response neural network with non-instantaneous impulses (NNNI)
x′i(t)=−ci(t)xi(t)+n∑j=1aij(t)fj(xj(t))+Ii(t) for t∈∪∞k=0(tk,sk+1], i=1,2,…nxi(t)=Φk,i(t,xi(sk−0)) for t∈(sk,tk], k=1,2,…xi(t0)=x0i, | (2.1) |
where n represents the number of neurons in the network, xi(t) corresponds to the membrane potential of i-th neuron at time t, x(t)=(x1(t),x2(t),…,xn(t))T∈Rn, xi(sk−0)=limt↑skxi(t), k=1,2,…,i=1,2,…,n, ci(t)>0,i=1,2,…,n, is the rate with which the unit self-regulates or resets its potential when isolated from other units and inputs of the i-th unit at time t, aij(t), i,j=1,2,…,n, correspond to the synaptic connection weights of the i-th neuron to the j-th neuron at time t, fj(xj) denotes a measure of response or activation of j-th neuron to its membrane potential, f(x)=(f1(x1),f2(x2),…,fn(xn))T and I=(I1,I2,…,In)T is an external bias vector, the intervals (sk,tk),k=1,2,…, are intervals of acting the state displacements which are impulsive activated at fixed moments sk, the functions Φk,i(t,xi), k=1,2,… are the impulsive functions giving the impulsive perturbation of the i-th neuron on the interval (sk,tk].
In applications, when Hopfield-type neural networks are used for associative memories, the equilibrium states of the network serve as stored patterns and their stability implies that the stored patterns can be retrieved in the presence of perturbations of the network which can be non-instantaneous impulsive state displacements. When Hopfield networks are used for the solution of optimization problems, the equilibrium states denote the possible optimal solutions and their stability will guarantee the convergence to the solutions beginning from an appropriate initial state.
Definition 1. A vector x∗∈Rn, x∗=(x∗1,x∗2,…,x∗n)T is an equilibrium of NNNI (2.1), iff the equalities
0=−ci(t)x∗i+n∑j=1aij(t)fj(x∗j)+Ii(t) for t∈∪∞k=0(tk,sk+1], i=1,2,…,n | (2.2) |
and
x∗i=Φk,i(t,x∗i) for t∈(sk,tk],k=1,2,…, i=1,2,…,n | (2.3) |
hold.
Remark 2. The problem (2.1) is studied in [6] in the special case of constant self-regulating parameters ci(t)≡ci, constant connection of the i-th neuron to the j-th neuron, i.e. aij(t)≡aij, i,j=1,2,…,n, constant external bias vectors I(t)≡I, instantaneous impulses tk=sk and Lipschitz activation functions. Several types of sufficient conditions for existence of a constant vector satisfying equality (2.2) for t≥t0 are given in [6].
We assume the following:
Assumption A1. The NNNI (2.1) has an equilibrium x∗∈Rn.
If assumption A1 is satisfied, then we can shift the equilibrium point x∗ of system (2.1) to the origin. The transformation y(t)=x(t)−x∗ is used to put system (2.1) in the following form:
y′i(t)=−ci(t)yi(t)+n∑j=1aij(t)Fj(yj(t)) for t∈∪∞k=0(tk,sk+1], i=1,2,…nyi(t)=ϕk,i(t,yi(sk−0)) for t∈(sk,tk], k=1,2,…yi(t0)=y0i, | (2.4) |
where Fj(u)=fj(u+x∗j)−fj(x∗j),j=1,2,…,n and ϕk,i(t,u)=Φk,i(t,u+x∗i)−Φk,i(t,x∗i), i=1,2,…,n, k=1,2,…, y0i=x0i−x∗i with u∈R.
Note that Fi(0)≡0 and Φj,i(t,0)=0 for j=1,2,…, i=1,2,…,n, and t∈[sj,tj] and if the point x∗∈Rn is an equilibrium of NNNI (2.1), then the point y∗=0 is an equilibrium of NNNI (2.4). This allows us to study the stability properties of the zero equilibrium of NNNI (2.4). Also, without loss of generality we can assume t0=0.
We will discuss the equilibrium of some NNNI with various activation functions. It will be useful for stability analysis.
EXAMPLE 1. Let n=1, Φk∈C([sk,tk]×R,R), k=1,2,…, f∈C(R,R) and consider the scalar NNNI
x′(t)=−c(t)x(t)+a(t)f(x(t))+I(t) for t∈∪∞k=0(tk,sk+1],x(t)=Φk(t,x(sk−0)) for t∈(sk,tk], k=1,2,…x(0)=x0, | (2.5) |
where c∈C(∪∞k=0(tk,sk+1],(0,∞)), I∈C(∪∞k=0(tk,sk+1],R).
Case 1.. Let I(t)=0.5πc(t), Φk(t,u)=tku+(1−tk)π2,t∈[sk,tk], and the activation function be the cosine function f(u)=cos(u) (see [18]).
The point x∗=0.5π is an equilibrium point of NNNI(2.5) because
−c(t)0.5π+a(t)cos(0.5π)+0.5πc(t)=0, t∈∪∞k=0(tk,sk+1] |
and 0.5π=Φk(t,0.5π),t∈[sk,tk],k=1,2,….
The transformation y(t)=x(t)−0.5π will shift the model (2.5) to the following model with zero equilibrium:
y′(t)=−c(t)y(t)+a(t)cos(y(t)+π2) for t∈∪∞k=0(tk,sk+1],y(t)=tky(sk−0) for t∈(sk,tk], k=1,2,…y(0)=y0, | (2.6) |
with y0=x0−π2.
Case 2. Let I(t)=0.5c(t), Φk(t,u)=tku+0.5−t2k,t∈[sk,tk], and the activation function be the Probit function f(u)=√2 erf−1(2u−1) with erf(u)=2√π∫u0e−t2dt is the error function or the activation function be the Logit function f(u)=log(u1−u).
The point x∗=0.5 is an equilibrium point of NNNI(2.5) because −c(t)0.5+a(t)f(0.5)+0.5c(t)=0 for all t∈∪∞k=0[tk,sk+1] and Φk(t,0.5)=0.5 for t∈[sk,tk].
The transformation y(t)=x(t)−0.5 will shift the model (2.5) to the following model with zero equilibrium:
y′(t)=−c(t)y(t)+a(t)√2 erf−1(2y(t)) for t∈∪∞k=0(tk,sk+1],y(t)=tky(sk−0) for t∈(sk,tk], k=1,2,…y(0)=y0, | (2.7) |
with y0=x0−0.5.
Case 3. If I(t)=0, Φk(t,u)=ξk(t)u for t∈[sk,tk],u∈R with ξk∈C([sk,tk],R) and any activation function such that f(0)=0 then the point x∗=0 is an equilibrium point of NNNI(2.5).
First we give some results for Lipschitz stability of the general type of nonlinear non-instantaneous impulsive differential equations (NIDE).
Consider the general type of initial value problem (IVP) for NIDE:
x′(t)=H(t,x(t)) for t∈∪∞k=0(tk,sk+1],x(t)=Ψk(t,x(sk−0)) for t∈(sk,tk], k=1,2,…x(t0)=x0, | (3.1) |
where x,x0∈Rn,H:∪∞k=0[tk,sk+1]×Rn→Rn,Ψk:[sk,tk]×Rn→Rn,(k=1,2,3,…).
Remark 3. The model MNNI (2.1) is a special case of NIDE (3.1).
A detailed description of the solutions of IVP for NIDE (3.1) and some qualitative properties of the solutions of (3.1) are given in the book [1].
Let J⊂R+ be a given interval. We introduce the following classes of functions
NPC(J)={u:J→Rn:u∈C(J/∪∞k=1{sk},Rn):aaaaaaaaau(sk)=u(sk−0)=limt↑sku(t)<∞, u(sk+0)=limt↓sku(t)<∞, k:sk∈J},NPC1(J)={u:J→Rn:u∈NPC(J), u∈C1(J⋂([0,s1]∪∞k=1[tk,sk+1],Rn):aaaaaaaaau′(sk)=u′(sk−0)=limt↑sku′(t)<∞, k:sk∈J}. |
Remark 4. According to the above description any solution of (3.1) might have a discontinuity at any point sk,k=1,2,….
Let J⊂R+, 0∈J, ρ>0. We introduce the following sets:
M(J)={a∈C[J,R+]:a(0)=0, a(r) is strictly increasing in J, and a−1(αr)≤rqa(α) for some function qa: qa(α)≥1, if α≥1},K(J)={a∈C[J,R+]:a(0)=0, a(r) is strictly increasing in J, and a(r)≤Kar for some constant Ka>0},Sρ={x∈Rn: ||x||≤ρ}. |
Remark 5. The function a(u)=K1u, K1>0 is from the class K(R+) with Ka=K1.
The function a(u)=K2√u, K2≥1 is from the class M([0,1]) with qa(α)=α2.
The function a(u)=K3u, K3≥1 is from the class M(R+) with qa(α)=K3α.
We will use the class Λ of Lyapunov-like functions, defined and used for impulsive differential equations in [12].
Definition 2. Let J⊂R+ be a given interval, and Δ⊂Rn be a given set. We will say that the function V(t,x):J×Δ→R+, belongs to the class Λ(J,Δ) if
− The function V(t,x) is a continuous on J/{sk∈J}×Δ and it is locally Lipschitz with respect to its second argument;
− For each sk∈J and x∈Δ there exist finite limits
V(sk,x)=V(sk−0,x)=limt↑skV(t,x) and V(sk+0,x)=limt↓skV(t,x). |
Lipschitz stability of systems of differential equations with non-instantaneous impulses is defined and studied in [9]:
Definition 3. (Lipschitz Stability) [9] The equilibrium x∗ of NNNI (2.1) is said to be
− uniformly Lipschitz stable if there exists M≥1 and δ>0 such that for any t0≥0 and x0∈Rn the inequality |x0−x∗|<δ implies |x(t;t0,x0)−x∗|≤M|x0−x∗| for t≥t0;
− globally uniformly Lipschitz stable if there exists M≥1 such that for any t0≥0 and x0∈Rn the inequality |x0−x∗|<∞ implies |x(t;t0,x0)−x∗|≤M|x0−x∗| for t≥t0.
Remark 6. In Definitions 1.1-1.4 [5] for Lipschitz stability the constants M>0. Note we use the constant M≥1 in Definition 3 since our inequality has to hold at t=t0.
Remark 7. Note the Lipschitz stability property of the equilibrium of NNNI (2.1) implies the corresponding stability property of the zero solution of (2.4) and vice versa.
We will use the following scalar comparison differential equation with non-instantaneous impulses:
u′=g(t,u) for t∈∪∞k=0(tk,sk+1],u(t)=ψk(t,u(sk−0)) for t∈(sk,tk],k=0,1,2,…,u(t0)=u0 | (3.2) |
where u,u0∈R,g:∪∞k=0[tk,sk]×R→R,ψk:[sk,tk+1]×R→R (k=0,1,2,3,…).
Note the Lipschitz stability of the scalar non-instantaneous impulsive equation (3.2) is similarly defined as in Definition 3.
We introduce the following conditions:
(H1). The function H∈C([0,s1]⋃∞k=1[tk,sk+1]×Rn,Rn), H(t,0)=0, and for any k=1,2,… the functions Ψk∈C([sk,tk]×Rn,Rn), Ψk(t,0)=0.
(H2). The function g∈C([0,s1]⋃∞k=1[tk,sk+1]×R+,R), g(t,0)=0, and for any k=1,2,… the functions ψk∈C([sk,tk]×R+,R+) are nondecreasing with respect to their second argument and ψk(t,0)=0.
EXAMPLE 2. Consider the scalar ordinary differential equation
u′=cos(t)u(t) for t≥t0, u(t0)=u0, | (3.3) |
with a solution u(t)=u0esin(t)−sin(t0) for t≥t0. Therefore, |u(t)|≤|u0|e2 for t≥t0, i.e. the solution is globally uniformly Lipschitz stable. (see Figure 1 for the graph of the solution with initial value u0=−0.2 and the graph of the solution with initial value u0=2 and M=1.1).
Now consider the scalar differential equation with non-instantaneous impulses:
u′=cos(t)u(t) for t∈∪∞k=0(tk,sk+1],u(t)=ψk(t,u(sk−0)) for t∈(sk,tk],k=1,2,…,u(t0)=u0, | (3.4) |
where tk=2k+1,k=0,1,2,…, sk=2k,k=1,2,….
Case 1. Let ψk(t,u)=tu. The solution is a nondecreasing function approaching infinity and the zero solution is not globally uniformly Lipschitz stable (see Figure 2 for the graph of the solution with initial value u0=1 and the graph of the solution with initial u0=0.2).
Case 2. Let ψk(t,u)=ut. The zero solution is uniformly Lipschitz stable with M=1.2 and δ=2 (see Figure 3) but it is not globally uniformly Lipschitz stable.
Theorem 1. [9] Let the following conditions be satisfied:
1. Conditions (H1), (H2) are fulfilled.
2. There exists a function V(t,x)∈Λ(R+,Rn) and
(i) the inequalities
b(||x||)≤V(t,x)≤a(||x||),x∈Sρ,t∈R+ |
hold, where b∈K([0,ρ]), a∈M([0,ρ]), ρ>0;
(ii) the inequality D+V(t,x)≤g(t,V(t,x)),t∈[0,s1]⋃∞k=1(tk,sk),x∈Sρ holds where D+V(t,x)=limsuph→0+1h{V(t,x)−V(t−h,x−hH(t,x))};
(iii) for any k=1,2,… the inequality
V(t,Ψk(t,y))≤ψk(t,V(sk−0,y)),t∈(sk,tk+1],y∈Sρ |
holds.
3. The zero solution of (3.2) is uniformly Lipschitz stable (globally uniformly Lipschitz stable).
Then the zero solution of (3.1) is uniformly Lipschitz stable (globally uniformly Lipschitz stable).
Corollary 1. Let (H1) be satisfied and the inequalities
n∑i=1xiHi(t,x)≤0,t∈[0,s1]∞⋃k=0[tk,sk],x∈Rn:x=(x1,x2,…,xn)T | (3.5) |
and
n∑i=1(Ψk,i(t,x))2≤n∑i=1x2i,x∈Rn,t∈(sk,tk+1],k=1,2,… | (3.6) |
hold.
Then the zero solution of (3.1) is globally uniformly Lipschitz stable.
Proof. For V(t,x)=0.5∑ni=1x2i, x=(x1,x2,…,xn)T from (3.5) it follows that condition 2(ⅱ) of Theorem 1 is satisfied with g(t,u)≡0 and from (3.6) it follows that 2(ⅲ) of Theorem 1 is fulfilled with ψk(t,u)=u. Then the comparison scalar equation is the following equation:
u′=0 for t∈∪∞k=0(tk,sk+1],u(t)=u(sk−0) for t∈(sk,tk],k=1,2,…,u(t0)=u0. | (3.7) |
The solution of the IVP (3.7) is u(t)=u0 for t≥t0 and therefore the zero solution of (3.7) is globally uniformly Lipschitz stable with M=1. According to Theorem 1 the zero solution of (3.1) is globally uniformly Lipschitz stable.
We will study stability properties of several different types of NNNI (2.1).
We will introduce the following assumptions:
Assumption A2. The neuron activation functions are locally Lipschitz, i.e. there exist positive numbers Li, i=1,2,…,n and λ>0 such that |fi(u)−fi(v)|≤Li|u−v|, i=1,2,…,n for u,v∈R: |u−x∗i|≤λ,|v−x∗i|≤λ where the point x∗∈Rn is the equilibrium from assumption A1.
Assumption A3. There exist positive numbers Mi,j, i,j=1,2,…,n such that |ai,j(t)|≤Mi,j for t∈[0,s1]⋃∪∞k=1(tk,sk+1].
Assumption A4. There exists a number B>0 such that ci(t)≥B for t∈[0,s1]⋃∪∞k=1(tk,sk+1], i=1,2,…,n and the inequality
B≥0.5maxi=1,2,…,nn∑j=1Mij Lj+0.5n∑i=1maxj=1,2…,nMijLj | (4.1) |
holds.
Assumption A5. For any k=1,2,… and x∈Rn: ||x−x∗||≤λ the inequalities
n∑i=1(Φk,i(t,xi)−Φk,i(t,x∗i))2≤n∑i=1(xi−x∗i)2, t∈[sk,tk] |
hold where x∗ is the equilibrium point from assumption A1 and λ is the number defined in A2.
Remark 8. Assumption A5 and the inequality (4.1) are connected with the strength of the self-regulating negative feedback of each neuron which dominates its own contribution to the entire network. Such phenomena are observed in Lotka-Volterra systems modeling interacting population systems.
Theorem 2. Let assumptions A1 - A5 be satisfied. Then the equilibrium x∗ of NNNI (2.1) is globally uniformly Lipschitz stable.
Proof. From the definition of the functions Fj, assumption A2 and the inequality 2ab≤a2+b2, a,b∈R we get the inequalities
n∑i=1xiHi(t,x)=−n∑i=1ci(t)x2i+n∑i=1xin∑j=1aij(t)Fj(xj)≤−n∑i=1ci(t)x2i+n∑i=1n∑j=1Mij|xi| |fj(xj+x∗j)−fj(x∗j)|≤−n∑i=1ci(t)x2i+n∑i=1n∑j=1Mij|xi| Lj|xj|≤−n∑i=1ci(t)x2i+0.5n∑i=1n∑j=1Mij Ljx2i+0.5n∑i=1n∑j=1Mij Ljx2j≤(−B+0.5maxi=1,2,…,nn∑j=1Mij Lj+0.5n∑i=1maxj=1,2…,nMijLj)n∑j=1x2j. | (4.2) |
From assumption A5 it follows that the inequality (3.5) holds.
From assumption A5 it follows that the inequality (3.6) holds for Ψk,i(t,yi)=Φk,i(t,yi+x∗i)−Φk,i(t,x∗i).
According to Corollary 1 the zero solution of (2.1) globally uniformly Lipschitz stable. This proves the claim.
EXAMPLE 3. Let n=3, t0=0, sk=(k−0.5k)π, tk=kπ,k=1,2,…. Consider the NNNI
x′i(t)=−|csc(t)|xi(t)+3∑j=1aij(t)cos(xj(t))+0.5π|csc(t)| for t∈∪∞k=0(tk,sk+1], i=1,2,3xi(t)=Φk,i(t,xi(sk−0)) for t∈(sk,tk], k=1,2,…xi(t0)=x0i | (4.3) |
with Φk,i(t,u)=sinkt u+(1−sinkt)0.5π and A=aij(t) is given by
A(t)=(−0.1sint0.40.3−t25t2+10.3t5t+1t10t+1−0.2cost−0.1sint). |
The point x∗=(0.5π,0.5π,0.5π) is the equilibrium point of NNNI (4.3).
Note assumptions A2, A4 are satisfied with L=1 and B=0.9. Next assumption A5 is fulfilled because
3∑i=1(Φk,i(t,xi)−Φk,i(t,x∗i))2=3∑i=1(sinkt xi+(1−sinkt)0.5π−0.5π)2=3∑i=1(sinkt(xi−0.5π))2≤3∑i=1(xi−0.5π)2, t∈[sk,tk] |
Also, |aij|≤Mij, i,j=1,2,3, t≥0 where M={Mij}, is given by
M=(0.10.40.30.20.30.20.10.20.1) |
and ∑3i=1maxjMij+maxi∑3j=1Mij=(0.4+0.3+0.2)+max(0.4,0.9,0.6)=0.9+0.9=1.8, i.e. inequality (4.1) holds. Therefore, according to Theorem 2 the equilibrium of NNNI(4.3) is globally uniformly Lipschitz stable.
Consider the system (4.3) without any impulses. The right side part is not defined for all t≥t0 because of the function csc(t). The appropriate perturbing of the model by non-instantaneous impulses can induce existence of the solution and stability properties of the equilibrium.
There are many types of activation functions which are not Lipschitz. In this case, we assume:
Assumption A6. There exists a function ξ∈C([0,s1]⋃∪∞k=1(tk,sk+1],R) such that for any point x: x−x∗∈S(λ) and t∈[0,s1]⋃∪∞k=1(tk,sk+1] the inequality
(xi−x∗i)(n∑j=1ai,j(t)(fj(xj)−fj(x∗j)))≤ξ(t)n∑j=1(xj−x∗j)2, i=1,2,…,n | (4.4) |
holds where x∗ is the equilibrium from condition A1.
Theorem 3. Let the following conditions be satisfied:
1. Assumptions A1 and A6 are satisfied.
2. The functions ci∈C(∪∞k=0(tk,sk+1],R+), i=1,,…,n.
3. The impulsive functions Φk,i∈C([sk,tk]×R,R) and there exists a positive number λ such that for k=1,2,… and x∈Rn:x−x∗∈S(λ) the inequality
n∑i=1(Φk,i(t,xi)−Φk,i(t,x∗i))2≤μk(t)n∑i=1(xi−x∗i)2,t∈[sk,tk] |
holds where x∗ is the equilibrium from condition 1.
4. There exists a function p∈C1([0,∞),R+):0<α≤p(t)≤β,t≥0 such that p(t)≤p(sk) for t∈[sk,tk] and the zero solution of the scalar non-instantaneous impulsive equation (3.2) with g(t,u)=2γ(t)u and ψk(t,u)=μk(t)u is uniformly Lipschitz stable (globally uniformly Lipschitz stable) where
γ(t)=−mini=1,2,…,nci(t)+nξ(t)+0.5p′(t)p(t),t∈∪∞k=0(tk,sk+1]. |
Then the equilibrium x∗ of NNNI (2.1) is uniformly Lipschitz stable (globally uniformly Lipschitz stable).
Proof. Consider the Lyapunov function V(t,x)=p(t)∑ni=1x2i, x=(x1,x2,…,xn). Then condition 2(ⅰ) of Theorem 1 is satisfied with b(u)=α√u and a(u)=K√u, K=max{1,β}. From Remark 5 it follows that b∈K([0,λ]) and a∈M([0,λ]). From assumption A6 it follows that the function F in NNNI (2.4) satisfies the inequality
ui(n∑j=1ai,j(t)Fj(uj))≤ξ(t)n∑i=1u2i(t), t∈∪∞k=0(tk,sk+1], u∈S(λ). | (4.5) |
From inequality (4.5) with ui=yi−x∗i∈S(λ) we obtain for the Dini derivative among the system (2.4):
D+V(t,y)=p′(t)n∑i=1y2i+2p(t)n∑i=1yi(y′i)=p′(t)n∑i=1y2i+2p(t)n∑i=1(−ci(t)y2i+yin∑j=1ai,j(t)Fj(yj))≤p′(t)n∑i=1y2i−2p(t)n∑i=1(mini=1,2,…,nci(t))y2i+2p(t)n∑i=1yin∑j=1ai,j(t)Fj(yj)≤p′(t)n∑i=1y2i−2p(t)n∑i=1(mini=1,2,…,nci(t))y2i+2p(t)n∑i=1ξ(t)n∑j=1y2j=2p(t)(−mini=1,2,…,nci(t)+nξ(t)−0.5p′(t)p(t))n∑i=1y2i(t)≤2γ(t)p(t)n∑i=1y2i(t)=2γ(t)V(t,y), t∈∪∞k=0(tk,sk+1]. | (4.6) |
Therefore, Condition 2(ⅱ) of Theorem 1 is satisfied with g(t,u)=2γ(t)u.
Then for t∈(sk,tk], y∈Sλ the Lyapunov function satisfies
V(t,ϕk(t,yi))=p(t)n∑i=1ϕ2k,i(t,y)≤μk(t)p(t)n∑i=1y2i≤μk(t)p(si)n∑i=1y2i=μk(t)V(sk−0,y). |
Therefore, Condition 2(ⅲ) of Theorem 1 is satisfied with ψk(t,u)=μk(t)u.
According to Theorem 1 the zero solution of NNNI (2.4) is uniformly Lipschitz stable (globally uniformly Lipschitz stable)
This proves the claim of Theorem 3.
Remark 9. The conditions in Theorem 2 and Theorem 3 differ. The inequality for the impulsive functions in condition 3 of Theorem 3 is weaker than Assumption A5. At the same time if Assumption A2 is satisfied then Assumption A6 follows. However the inequality in condition 4 of Theorem 3 is more restrictive than the inequality in Assumption A3. Both Theorems are applicable in different cases.
We will illustrate the difference between the applications of Theorem 2 and Theorem 3 on an example.
EXAMPLE 4. To simplify the calculations we consider the simplest neural network of two units, i.e. n=2. Let t0∈[0,1), sk=k, tk=k+0.5, k=1,2,… and consider the nonlinear NNNI
x′1(t)=−e−tx1(t)−0.5t+1f1(x1(t))+1t+1f2(x2(t))+0.5e−t−1t+1x′2(t)=−1t+1x2(t)−1t+1f2(x2(t))+1t+1 for t∈∪∞k=0(tk,sk+1],x1(t)=1tx1(sk−0)+0.5t−1tx2(t)=1tx2(sk−0) for t∈(sk,tk], k=1,2,…. | (4.7) |
where the activation functions are the Probit function f1(u)=√2 erf−1(2u−1) and the cosine function f2(u)=cos(u), c1(t)=e−t, c2(t)=1t+1, I1(t)=0.5e−t−1t+1, I2(t)=1t+1, Φ1,k(t,u)=1tu−0.5t−1t, Φ2,k(t,u)=ut and
A(t)=(−0.5t+11t+10−1t+1 ). |
Then the equilibrium is x∗1=0.5,x∗2=0.
Theorem 3 is not applicable because assumptions A4 and A2 for c1(t),c2(t) and f1(u) are not satisfied.
Now using (u−0.5)f1(u)≥(u−0.5)2, u∈R (see Figure 4) it follows that
−0.5t+1(u−0.5)f1(u)≤−0.5t+1(u−0.5)2≤0.5t+1(u−0.5)2, u∈R. | (4.8) |
According to inequalities (4.8) and |cos(u)−1|≤|u|, u∈R (see Figure 5) we obtain
(x1−0.5)(a1,1(t)(f1(x1)−f1(0.5))+a1,2(t)(f2(x2)−f2(0)))=(x1−0.5)(a1,1(t)f1(x1)+a1,2(t)(f2(x2)−1))=−0.5t+1(x1−0.5)f1(x1)+1t+1(x1−0.5)(f2(x2)−1))≤0.5t+1(x1−0.5)2+1t+1(x1−0.5)(cos(x2)−1)≤0.5t+1(x1−0.5)2+1t+1 |(x1−0.5)||x2|≤1t+1(x1−0.5)2+0.5t+1x22≤1t+1((x1−0.5)2+x22). | (4.9) |
From inequality u(cos(u)−1)≤u2 (see Figure 6) it follows that
x2(a2,1(t)(f1(x1)−f1(0.5))+a2,2(t)(f2(x2)−f2(0)))=−1t+1x2(cos(x2)−1)≤0.5t+1x22≤1t+1((x1−0.5)2+x22). | (4.10) |
Therefore, assumption A6 is satisfied with ξ(t)=1t+1.
Consider the function p(t)=1.1+cos(t), 0.1<p(t)≤1.1 for t≥0. The inequality p(t)≤p(sk) holds for t∈[sk,tk].
Also, applying the inequality e−t≤11+t (see Figure 7) we obtain the inequality
γ(t)=−mini=1,2ci(t)+2ξ(t)+0.5p′(t)p(t)=−e−t+211+t+0.5sin(t)1.1+cos(t)≤1+0.5sin(t)1.1+cos(t)≤1.05, t∈(tk,sk], k=1,2,…. | (4.11) |
Condition 3 of Theorem 3 is satisfied with μk(t)=1t2 because ∑2i=1(Φk,i(t,xi)−Φk,i(t,x∗i))2=1t2((x1−0.5)2+x22), t∈[sk,tk].
Then the scalar non-instantaneous impulsive equation (3.2) will be reduced to
u′=2.1u for t∈[t0,1]∪∞k=1(k+0.5,k+1],u(t)=1t2u(sk−0)) for t∈(k,k+0.5],k=1,2,…,u(t0)=u0. | (4.12) |
The zero solution of (4.12) is uniformly Lipschitz stable (see Figure 8) with M=11 and δ=1. According to Theorem 3 the equilibrium of (4.7) is uniformly Lipschitz stable.
Several sufficient conditions are given for stability of the equilibrium point of neural networks with non-instantaneous impulses. We study the general case when
− the self-regulating parameters of all units are variable in time;
− the functions of the connection between two neurons in the network are variable in time bounded functions;
− the activation functions are non-Lipschitz (usually in the literature only the case of Lipschitz activation functions is studied).
The investigation is based on applying the Lyapunov approach and various types of Lyapunov functions are used. These piecewise continuous Lyapunov functions are used as an apparatus to study stability properties of neural networks with non-instantaneous for different types of activation functions. Many examples with different types of activation functions such as cos function, Logit function, continuous Tan-Sigmoid function, are provided to illustrate the practical application of our results.
All authors declare no conflicts of interest in this paper.
[1] | R. Agarwal, S. Hristova and D. O'Regan, Non-Instantaneous Impulses in Differential Equations, Springer, 2017. |
[2] | H. Akca, R. Alassar, V. Covachev, Z. Covacheva and E. A. Al-Zahrani, Continuous-time additive Hopfield-type neural networks with impulses, J. Math. Anal. Appl., 290 (2004), 436–451. |
[3] | H. Akca, R. Alassar, Y. M. Shebadeh and V. Covachev, Neural networks: Modelling with impulsive differential equations, Proc. Dynamical Syst. Appl., (2004), 32–47. |
[4] | N. T. Carnevale and M. L. Hines, The NEURON Book, Cambridge, UK, Cambridge University Press, 2009. |
[5] | F. M. Dannan and S. Elaydi, Lipschitz stability of nonlinear systems of differential equations, J. Math. Anal. Appl., 113, (1986), 562–577. |
[6] | K. Gopalsamy, Stability of artificial neural networks with impulses, Appl. Math. Comput., 154 (2004), 783–813. |
[7] | J. J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, Proc. Nat. Acad. Sci. USA, 79 (1982), 2554–2558. |
[8] | S. Hristova, Qualitative investigations and approximate methods for impulsive equations, Nova Sci. Publ. Inc., New York, 2009. |
[9] | S. Hristova and R. Terzieva, Lipschitz stability of differential equations with non-instantaneous impulses, Adv. Differ. Equ., 2016, 322. |
[10] | Y. Huang, H. Zhang and Z. Wang, Dynamical stability analysis of multiple equilibrium points in time-varying delayed recurrent neural networks with discontinuous activation functions, Neurocomputing, 91 (2012), 21–28. |
[11] | R. D. King, S. M. Garrett and G. M. Coghill, On the use of qualitative reasoning to simulate and identify metabolic pathways, Bioinformatics, 21 (2005), 2017–2026. |
[12] | V. Lakshmikantham, D. D. Bainov and P. S. Simeonov, Theory of Impulsive Differential Equations, World Scientific, Singapore, 1989. |
[13] | X. Li and J.Wu, Stability of nonlinear differential systems with state-dependent delayed impulses, Automatica, 64 (2016), 63–69. |
[14] | X. Li and S. Song, Stabilization of Delay Systems: Delay-Dependent Impulsive Control, IEEE Transactions on Automatic Control, 62 (2017), 406–411. |
[15] | C. Li and G. Feng, Delay-interval-dependent stability of recurrent neural networks with timevarying delay, Neurocomputing, 72 (2009), 1179–1183. |
[16] | X. Li, D. W. C. Ho and J. Cao, Finite-time stability and settling-time estimation of nonlinear impulsive systems, Automatica, 99 (2019), 361–368. |
[17] | W. McCulloch and W. H. Pitts, A logical calculus of the ideas immanent in nervous activity, Bill. Math. Bioph., 5 (1943), 115–133. |
[18] | A. Rahimi and B. Recht, Weighted sums of random kitchen sinks: Replacing minimization with randomization in learning, Adv. Neural Information Processing Syst, 21 (2008), 1313–1320. |
[19] | R. I. Watson Sr., The great psychologists, J.B. Lippincott Co., New York, 1978. |
1. | Antonia Dobreva, 2021, 2321, 0094-243X, 030006, 10.1063/5.0040098 | |
2. | Ravi Agarwal, Snezhana Hristova, Donal O’Regan, Lyapunov Functions and Lipschitz Stability for Riemann–Liouville Non-Instantaneous Impulsive Fractional Differential Equations, 2021, 13, 2073-8994, 730, 10.3390/sym13040730 | |
3. | Snezhana Hristova, Stepan Tersian, Radoslava Terzieva, Lipschitz Stability in Time for Riemann–Liouville Fractional Differential Equations, 2021, 5, 2504-3110, 37, 10.3390/fractalfract5020037 | |
4. | Mohammed Said Souid, Krasimira Ivanova, 2022, 2459, 0094-243X, 030034, 10.1063/5.0083544 | |
5. | Hao Deng, Chuandong Li, Yinuo Wang, Asymptotic stability of non‐instantaneous impulsive systems and T‐S fuzzy non‐instantaneous impulsive control for nonlinear systems, 2023, 1751-8644, 10.1049/cth2.12448 |