Mixed-coexistence of periodic orbits and chaotic attractors in an inertial neural system with a nonmonotonic activation function

1.   Introduction

The coexistence of many attractors for a given set of parameters, called as multistability, is one of the most interesting phenomena in dynamical systems ^[1]. It can be found in the different scientific fields, such as biology, physics, chemistry, and even economics. The long-term behavior of a multistable system depends crucially on external perturbations and initial conditions. A slight change of parameters will cause the final state of system dynamics to exhibit a facile switching between the different stable states. This provides a great flexibility in system function ^[2].

In neural systems, multistability is represented by the coexistence of some firing patterns, such as silence, spiking, bursting, and chaos ^[3]. Perceptual illusion of visual and auditory senses can be illuminated in terms of multistability ^[4]. Parkinson's disease may be treated by switching neuron activities from the synchrony to asynchrony state, which is a bistability ^[5]. In cortical neurons, sleep and wakefulness are reflected as different spiking and bursting ^[6]. The R15 neuron in Aplysia was found to exhibit the coexistence of periodic spiking, bursting, and chaos ^[7]. Further, some theoretical models were constructed to study the neural multistability ^[8]. For example, the integrate- and-fire and Hodgkin-Huxley neuron models displayed multiple types of spike trains ^[9,10]. Buric et al. found that the delayed Hindmarsh-Rose/FitzHugh-Nagumo neural system had the coexistence of a rest state and periodic activity ^[11,12]. Song et al. ^[13] found that the delay-coupled neural oscillator system with inhibitory-to-inhibitory connection exhibited the coexistence with multiple equilibria and multi-periodicities. Recently, some coexistence with equilibria and oscillations ^[14] and coexistence with synchronous/asynchronous oscillations ^[15] were exhibited in the coupled neural network system constituted of individual sub-networks.

It is evident that the brain system generates chaotic behaviors played an important role in information processing ^[16]. In addition, chaotic behavior in artificial neural networks has a valuable application in optimization, associative memory, and cryptography ^[17]. There are some, though not too many, neural network systems with multiple chaotic attractors. For example, the three-neuron Hopfield network with self- and neighbor-connection presented two coexisting chaotic attractors with symmetrical pattern. For the different connection weights, the neural system exhibited two coexisting chaotic attractors ^[18,19] and two coexisting transient chaos ^[20]. The dynamic behavior of the single neuron system with sinusoidal signal input was studied by Li and Chen ^[21], in which the neural system displayed two coexisting chaotic attractors and evolved into a connected chaotic attractor.

Recently, chaotic coexistence with some mixed-type of stable steady states have drawn an increasing attention and have been reported in various dynamical systems, including Lorenz system ^[22], Jerk circuit system ^[23,24], Duffing–Holmes system ^[25], Chua's circuit system ^[26], Shinriki's circuit system ^[27], a radio-physical oscillator system ^[28], electrochemical reaction system ^[29], etc. However, to the best of our knowledge, there are few studies on a neural network system having mixed coexistence with periodic orbits and chaotic attractors. Cheng ^[30] proposed a ring of discrete-time neural networks with self-feedback and non-monotonic activation function. The coexistence of multiple equilibria and chaotic dynamics was obtained. For continuous-time neural networks, Li et al. ^[31] presented a Hopfield neural network, which can display the coexistence of a periodic orbit and a chaotic attractor.

Motivated by this consideration, in this paper a simple inertial two-neural system with a non-monotonic activation function is proposed. Numerical simulations illustrate that the proposed neural system displays the mixed-type of coexistence with multiple periodic orbits and chaotic attractors. Increasing the neighbor-connection weight, a pair of chaotic behaviors is obtained by the period-doubling bifurcation of the periodic orbit, which comprise the two-coexisting attractors. Furthermore, in this evolutionary process, multiple pairs of periodic orbits are presented in sequence by the Hopf bifurcation of the nontrivial equilibria, and they evolve into chaotic behavior via the period-doubling bifurcation. The neural system exhibits the mixed coexistence of periodic orbits and chaotic attractors, such as the bistable period orbits, multistable periodic orbits, and multistable chaotic attractors with multi-periodic orbits.

The rest of this paper is organized as follows. In the next section, the simple two-neuron network system with a non-monotonic activation function is proposed. Some basic properties are analyzed, including the number of equilibria and their parameter regions. The system exhibits multiple equilibria through the static bifurcation of trivial and nontrivial equilibria. Employing some numerical simulations, we find that the neural system generates two coexisting single-scroll chaotic attractors by the period-doubling bifurcation. Furthermore, with the increase of the neighbor- connection weight, the coexisting single-scroll attractors evolve into a double-scroll chaotic attractor by the attractor merging crisis, which is main focus of Section 3. In Section 4, the detailed bifurcation diagrams and corresponding attractor basins are proposed to illustrate the dynamical evolution from two coexisting sinks; to two, four, and six coexisting periodic orbits; and then to coexisting single-scroll chaotic attractors. The neural system exhibits the mixed coexistence of periodic orbits and chaotic attractors. Finally, transient chaos is observed in neural system by phase portraits and time histories. Conclusions and discussions are given in Section 6.

2.   Model and its equilibrium

Modeling the biological neural system has attracted considerable attention in engineer, physics, and mathematics as Hopfield proposed a simplified network system. However, to avoid the difficulty of theoretical analysis, Schieve et al. ^[32] presented the concept of the effective neuron to simplify the larger Hopfield network. The previous researches illustrate that few-neuron network system exhibits the similar complex behaviors as larger networks. In fact, two-neuron coupled systems are sometimes viewed as the nonlinear dynamical system with two modules, where each module represents the mean activity of spatially localized neural populations.

In inertial neural model, inertial term is introduced from the equivalent RLC (resistance–inductance–capacitance) circuit system. Babcock and Westervelt ^[33] presented a neural model by combining inertial term into a single effective neuron. They presented chaos activity in a two-neuron coupled system. Wheeler and Schieve ^[34] discussed equilibrium and chaos in a two-neuron model with inertial terms. Liu et al. ^[35,36] obtained the dynamical behaviors in the inertial neural model systems. Recently, an inertial two-neural system was established to show the stable coexistence ^[37] and find some mixed types of periodic orbits, such as period-1, period-2, period-3, and even quasi-period orbits ^[38]. Based on the above mentioned references, in this paper, we found the inertial coupled system with two neurons, which is described by the following differential equation. It should be noticed that the presented neural system model can be used to reconstruct the neural activity of one neuron or neural population. Further, the kinds of neural activity can be simulated by the electric circuit, which is an important tool to solve some complicated problems by using neural computation properties.

where x and y represent the neural activations; k₁, k₂ > 0 denote the damping factors; and c₁ and c₂ are the connection strengths.

The neural activation function considered in the paper is chosen as $f(u) = u\exp (- {u^2}/2)$, which is called as Crespi function ^[21,39]. It is a bound, differentiable and nonmonotonic function. The graph is plotted in Figure 1. Furthermore, system (1) is symmetric with respect to the original point via the coordinate transformation (x, y) → (− x, − y). It should be noticed that the activation function has an essential role in neural dynamics. Different activation functions may induce the different neural computation properties. To obtain many kinds of the stable patterns and chaotic dynamics, some nonmonotonic activation functions are introduced, which is demand for the neural network application. In fact, associative memory storage is defined by the coexistence of multiple stable equilibria and periodic orbits. Further, chaos behavior in neural systems plays an important role in optimization problem and associative memory.

Figure 1.  Graph of (a) the neural activation function f(u) and (b) the derivative function fʹ(u).

DownLoad: Full-Size Img PowerPoint

Employing $x = {x_1}, {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \dot x = {x_2}$ and $y = {y_1}, {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \dot y = {y_2}$, we obtain the following equivalent system:

It is obvious that system (2) has the trivial equilibrium E₀ (0, 0, 0, 0). Furthermore, the nontrivial equilibrium E$(x_1^{}, {\kern 1pt} {\kern 1pt} y_1^{}, 0, {\kern 1pt} {\kern 1pt} {\kern 1pt} 0)$ is satisfied with the following equations through the equations ${\dot x_1} = 0, {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\dot y_1} = 0, {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\dot x_2} = 0,$ and ${\kern 1pt} {\dot y_2} = 0$, that is

Solutions of Eq (3), i.e., the equilibria of system (2), depend only on parameters c₁ and c₂ but are independent of k₁ and k₂. The curves determined by Eq (3) are called as the nullclines of dynamical system. The equilibria are the intersection points of two nullcline curves. For the different parameter values, the neural system has the different number of equilibria, as shown in Figure 2.

Figure 2.  Intersection points of two curves given by Eq (3), showing the number of equilibria with (a) c₂ = 0.1, (b) c₂ = 1, (c) c₂ = 3.5, and (d) c₂ = 5 for the fixed parameter c₁ = 5 in system (2) ^[40].

DownLoad: Full-Size Img PowerPoint

Inserting $f(u) = u\exp (- {u^2}/2)$ into Eqs (3) and eliminating y₁, we have ^[40]

In fact, Eq (4) is a complicated transcendental equation that cannot be solved in a theoretical form. However, for the given system parameters c₁ and c₂, the number and value can be obtained by numerical methods. Figure 3(a) shows the parameter regions in the (c₁, c₂) plane, where the system has a different number of equilibria, including trivial and nontrivial equilibria. Furthermore, it follows from bifurcation theory that the critical value of the equilibrium's number corresponds to the typical static bifurcation. To this end, by linearizing system (2) at the equilibrium E$(x_1^{}, {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} y_1^{}, 0, {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} 0)$, one has the characteristic equation

Figure 3.  (a) Static bifurcation curves dividing the parameter (c₁, c₂) plane into different regions, with one equilibrium in the blue region, three in the green, and seven in the red, where (b) is an enlargement of (a). Region boundaries are the static bifurcation curves of the trivial and nontrivial equilibria.

DownLoad: Full-Size Img PowerPoint

where ${p_0} = f'(x_1^{}){\kern 1pt} {\kern 1pt}, {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {q_0} = f'(y_1^{})$. System (2) exhibits a static bifurcation if an eigenvalue passes through the imaginary axis along the real axis. Therefore, letting λ = 0 in Eq (5), we have

which is the critical value of the static bifurcation for the trivial and nontrivial equilibria. Actually, the simplified static bifurcation for the trivial equilibrium is c₁c₂ = 1 by inserting x₁ = 0 and y₁ = 0 into Eq (6). Figure 3(b) shows all static bifurcations for the trivial and nontrivial equilibria. It implies that the boundary curve of the parameter region with one and three equilibria is the static bifurcation curve of the trivial equilibrium, i.e., c₁c₂ = 1. The two bifurcation curves for the nontrivial equilibria are the boundaries between the regions having three and seven equilibria, which are terminated at the location of the cusp bifurcation point, a codimension-2 bifurcation ^[40].

3.   Chaotic coexistence and its attractor merging crisis

In this section, we illustrate chaos and its coexistence in neural system (2). The Matlab ode45 method is used in the numerical simulation to solve differential equations. The result shows that the two-coexisting chaos can evolve into a double-scroll chaotic attractor. So, we firstly fix the system parameters as k₁ = 1, k₂ = 1.6, c₁ = 5, and c₂ = 14. System (2) exhibits the two-coexisting chaotic behaviors with a single-scroll attractor. The three-dimensional projections of phase portraits on x₁ – x₂ – y₁ and y₁ – x₂ – y₂ are shown in Figure 4, where the right attractor (in red) is constructed with the initial condition (1, 1, 0, 0) and the left (in blue) with (−1, −1, 0, 0).

Figure 4.  Three-dimensional projections of two coexisting single-scroll chaotic attractors on (a) x₁ – x₂ – y₁ and (b) y₁ – x₂ – y₂ with k₁ = 1, k₂ = 1.6, c₁ = 5, and c₂ = 14, where the right attractor (red) is constructed with initial value (1, 1, 0, 0) and the left (blue) with (−1, −1, 0, 0).

DownLoad: Full-Size Img PowerPoint

The prominent property of a chaotic attractor is sensitive dependence on initial condition. When a system exhibits chaotic behavior, a slight difference in the initial trajectory will cause a distinct difference in its final trajectory. Figure 5 illustrates the sensitivity by comparing the different trajectories with initial conditions (1, 1, 0, 0) (in red) and (1.01, 1, 0, 0) (in blue). It follows that the two time histories are almost the same at the beginning, but after some time the difference between them starts growing rapidly. This means that system (2) has a sensitive dependence on initial conditions, i.e. the trajectory is a chaos solution.

Figure 5.  Sensitive dependence on initial conditions, where red lines are the time histories for initial conditions (1, 1, 0, 0) and blue are those for (1.01, 1, 0, 0).

DownLoad: Full-Size Img PowerPoint

The Lyapunov exponent is used to characterize the degree of exponential divergence/convergence of trajectories arising from the nearby initial conditions. For a dynamical system, two trajectories (denoted as ${X_0}$ and $X_0^\delta = {X_0} + \delta {X_0}$) starting from the nearby initial conditions have the distance $d({X_0}, t) = \left\| {\delta {X_0}({X_0}, t)} \right\|$ by the Euclidean norm with time evolution. Then the mean rate of divergence of two close trajectories is given by

which is the Lyapunov exponent of a dynamical system. To calculate the Lyapunov exponent by numerical simulation, we just compute the system trajectories in a long finite-time and obtain the finite-time Lyapunov exponent. In fact, when the Lyapunov exponent in finite-time shows the convergence with time evolution, the calculated quantities can be regarded as the Lyapunov exponent of a dynamical system. The Lyapunov exponents of neural system (2) with initial value (1, 1, 0, 0) are LE₁ = 0.071, LE₂ = 0, LE₃ = −1.289, and LE₄ = −1.381, as shown in Figure 6. This implies that the attractor exhibits chaotic motion since the largest Lyapunov exponent is positive. Actually, from the analysis in Section 2, system (2) has seven equilibria. The corresponding characteristic equation has either a positive eigenvalue or a pair of complex conjugate eigenvalues with positive real parts, which is displayed in Table 1. It follows that all equilibria are unstable.

Figure 6.  Lyapunov exponents of system (2) with the fixed parameters k₁ = 1, k₂ = 1.6, c₁ = 5, c₂ = 14, and initial value (1, 1, 0, 0).

DownLoad: Full-Size Img PowerPoint

Table 1.  Equilibria and eigenvalues with fixed parameters k₁ = 1, k₂ = 1.6, c₁ = 5, and c₂ = 14.

Symbol	Equilibria	Eigenvalues	Types
E0	(0, 0, 0, 0)	λ1 = 2.1453, λ2, 3 = –0.6465 ± 2.9830i, λ4 = –3.4521	saddle-focus
E1, 2	(±0.2123, ±2.9072, 0, 0)	λ1, 2 = 0.3964 ± 1.2866i, λ3, 4 = –1.6964 ± 1.2647i	focus-focus
E3, 4	(±2.5266, ±1.4538, 0, 0)	λ1 = 0.7213, λ2, 3= –0.6380± 1.7259i, λ4 = –2.0452	saddle-focus
E5, 6	(±2.8187, ±0.7429, 0, 0)	λ1, 2 = 0.1605 ± 1.1070i, λ3, 4 = –1.4605 ± 1.0739i	focus-focus

---

 | Show Table

DownLoad: CSV

For the fixed parameters k₁ = 1, k₂ = 1.6, and c₁ = 5, the bifurcation diagrams with respect to c₂∈(11, 16) are shown in Figure 7 with initial values (1, 1, 0, 0) (in red) and (–1, –1, 0, 0) (in blue), where the Poincaré section x₁ = 0 is used for a clear presentation. It is found that system (2) evolves into chaos behavior employing the period-doubling bifurcation. Actually, for c₂ = 11, system (2) just exhibits two coexisting period-1 orbits. The system trajectory just converges into two points in the Poincaré section. With increasing c₂, the period-1 behaviors evolve into the period-2 orbits by the period-doubling bifurcation. A further increase of c₂ causes the double-periodic behaviors to lose their stability and give rise to orbits with period 4, 8, and etc.

Figure 7.  Bifurcation diagrams generated by Poincaré section x₁ = 0 with initial value (±1, ±1, 0, 0) for c₂∈(11, 16) showing the dynamic evolution from period to chaos by period-doubling bifurcation as c₂ varies, where the other parameters are fixed as k₁ = 1, k₂ = 1.6, and c₁ = 5.

DownLoad: Full-Size Img PowerPoint

Finally, the system exhibits two coexisting chaotic behaviors, as shown in Figure 8(a) for c₂ = 14. Interestingly, with further c₂ evolution, the two coexisting chaotic behaviors generate into a single two-scroll chaotic attractor via the attractor merging crisis, see Figure 8(b) for c₂ = 16. The three-dimensional projections of the phase portraits on x₁–x₂–y₁ and y₁–x₂–y₂ are shown in Figure 8c, d, respectively. It follows from Figure 9 that the maximum Lyapunov exponent changes from zero to a positive number as c₂ increases, which is in very good agreement with the above dynamical evolution, from period to chaos via the period-doubling bifurcation.

Figure 8.  Dynamic evolution from the two coexisting chaotic behaviors (a) c₂ = 14 to the single two-scroll chaos (b) c₂ = 16, where (c) and (d) are the three-dimensional projections of the two-scroll chaotic attractor. The other system parameters are fixed as k₁ = 1, k₂ = 1.6, and c₁ = 5.

DownLoad: Full-Size Img PowerPoint

Figure 9.  Lyapunov exponent of system (2) with respect to c₂, showing the dynamical evolution from period to chaos for k₁ = 1, k₂ = 1.6, and c₁ = 5.

DownLoad: Full-Size Img PowerPoint

4.   Coexisting attractors

In this section, we will exhibit the global evolution of system dynamics and find the multiple attractors' coexistence. To this end, the coupling weight c₂ is chosen as the bifurcation parameter. For the different coupling weight c₁, we show the bifurcation diagrams to find the dynamical mechanism of equilibrium and periodic orbit. To get a clear view, we just illustrate one branch for each Hopf bifurcation. The other parameters are fixed as k₁ = 1 and k₂ = 1.6. It follows from Figure 10 that system (2) exhibits multiple equilibria coexistence by the pitchfork/saddle-node bifurcation of the trivial/nontrivial equilibrium. The coexisting periodic orbit is obtained by the Hopf bifurcation (HB) of the nontrivial equilibrium.

Figure 10.  Bifurcation diagrams with (a) c₁ = 2, (b) c₁ = 3, (c) c₁ = 3.5, and (d) c₁ = 4 for the fixed parameters are k₁ = 1, k₂ = 1.6. It shows the dynamic evolution of equilibrium and periodic orbit, where PB, SN, HB, and PD denote the pitchfork bifurcation, saddle-node bifurcation, Hopf bifurcation and period-doubling bifurcation, respectively.

DownLoad: Full-Size Img PowerPoint

In Figure 10(a), the neural system just has the pitchfork bifurcation (PB) of the trivial equilibrium, which induces two coexisting nontrivial equilibria. The two-coexisting periodic orbit is obtained by the Hopf bifurcation of the nontrivial equilibrium labeled as HB in Figure 10(a). When the coupling weight is fixed as c₁ = 3, the nontrivial equilibrium exhibits the saddle-node bifurcation (labeled as SN in Figure 10(b)), which induces another two stable nontrivial equilibria. The neural system exhibits four coexisting equilibria in some parameter regions. Further, when the coupling weight increases to c₁ = 3.5, system (2) exhibits two periodic-doubling bifurcations from the periodic orbit emerged by the Hopf bifurcation of the nontrivial equilibrium, labeled as PD in Figure 10(c). It induces two coexisting period-2 orbits in system (2). Moreover, it follows from Figure 10(d) that the neural system exhibits four coexisting periodic orbits when the coupling weight is fixed as c₁ = 4. In fact, the nontrivial equilibrium emerged by the pitchfork bifurcation of the trivial equilibrium presents the Hopf bifurcation, which induces the two-coexisting periodic orbit. Further, another two periodic orbits will be presented by the Hopf bifurcation of the nontrivial equilibrium emerged by the saddle-node bifurcation of the nontrivial equilibrium. Due to the period-doubling bifurcation of the periodic orbit, the neural system exhibits the mixed coexistence with period-1 and period-2 orbits.

In fact, with the coupling weight c₁ increasing to c₁ = 5, the period-doubling bifurcation will induce the periodic orbit into chaotic behaviors. System (2) presents the multistable chaotic attractors with multi-periodic orbits. To illustrate the multiple period-doubling bifurcation of the periodic orbit, we present the bifurcation diagrams employing the Poincaré method, where x₂ = 0 is the Poincaré section in this case. It follows from Figure 11 that the bifurcation sequences of the periodic orbits are labeled with different colors (red, yellow, blue, and pink) for the different initial values. The equilibrium is shown in green. Further, The enlarged details of the periodic orbits are exhibited in Figure 12. The phase portraits are shown in Figure 13. The parameters, initial values, and dynamics properties are listed in Table 2.

Figure 11.  Bifurcation diagrams corresponding to periodic orbits (blue) and equilibrium points (red), showing the stable coexistence of multi-periodic orbits with a pair of chaos attractors.

DownLoad: Full-Size Img PowerPoint

Figure 12.  Enlarged details of the periodic bifurcation diagrams, showing the period-doubling evolutions for the different initial values (a) (1, 1, 0, 0), (b) (−1, −1, 0, 0), (c) (1.5, 2, 0, 0), and (d) (−1.5, −2, 0, 0).

DownLoad: Full-Size Img PowerPoint

Figure 13.  Phase portraits with different initial values, showing the multiple coexisting attractors via period-doubling bifurcation, where the parameters are fixed as k₁ = 1, k₂ = 1.6, and c₁ = 5. Corresponding initial conditions and parameter c₂ are given in Table 2.

DownLoad: Full-Size Img PowerPoint

Table 2.  Coexisting attractors with varying c₂ for k₁ = 1, k₂ = 1.6, and c₁ = 5.

c2	Dynamical property	Initial conditions	Figure
2 4.5 4.6 4.95 5.1 5.5 5.75 6.0 8.85 9.99   11.2 11.3 12.8 13.0 14.5	pair of period-1 orbits pair of period-1 orbits with pair of period-1 orbits pair of period-1 orbits with pair of period-2 orbits pair of period-1 orbits with pair of period-4 orbits pair of period-2 orbits with pair of chaotic attractors pair of period-2 orbits with pair of period-2 orbits pair of period-2 orbits with pair of period-1 orbits pair of period-2 orbits pair of period-4 orbits with pair of period-1 orbits pair of period-4 orbits with two pairs of period-1 orbits pair of chaotic attractors with pair of period-1 orbits pair of chaotic attractors with pair of period-2 orbits pair of period-4 orbits with pair of period-2 orbits pair of period-4 orbits with pair of period-4 orbits pair of period-2 orbits with pair of chaotic attractor	(±1, ±1, 0, 0) (±1, ±1, 0, 0), (±1.5, ±2, 0, 0) (±1, ±1, 0, 0), (±1.5, ±2, 0, 0) (±1, ±1, 0, 0), (±1.5, ±2, 0, 0) (±1, ±1, 0, 0), (±1.5, ±2, 0, 0) (±1, ±1, 0, 0), (±1.5, ±2, 0, 0) (±3, ±1, 0, 0), (±1, $\mp$1, 0, 0) (±3, ±1, 0, 0) (±3, ±1, 0, 0), (±1, ±1, 0, 0) (±3, ±1, 0, 0), (±1, ±1, 0, 0) (±3, $\mp$1, 0, 0) (±3, ±1, 0, 0), (±1, ±1, 0, 0) (±3, ±1, 0, 0), (±1, ±1, 0, 0) (±3, ±1, 0, 0), (±1, ±1, 0, 0) (±3, ±1, 0, 0), (±1, ±1, 0, 0) (±3, ±1, 0, 0), (±1, ±1, 0, 0)	13(a) 13(b) 13(c) 13(d) 13(e) 13(f) 13(g) 13(h) 13(i) 13(j)   13(k) 13(l) 13(m) 13(n) 13(o)

---

 | Show Table

DownLoad: CSV

Actually, neural system (2) exhibits only a stable trivial equilibrium E₀ for c₂ = 0. Employing its pitchfork bifurcation, a symmetric pair of stable nontrivial equilibria, i.e. E₁ and E₂, is obtained. With increasing c₂ and passing through the critical value c₂ ≈ 0.86, the system exhibits a pair of period-1 orbits employing the Hopf bifurcation (HB) of the nontrivial equilibria E₁ and E₂. The phase portraits with initial values (±1, ±1, 0, 0) are shown in Figure 13(a) for the fixed parameter c₂ = 2. It follows from Figures 12(a) and 12(b) that the pair of periodic orbits evolves into chaos attractor through the period-doubling bifurcation, and then degenerates into the period orbit by the reversal period-doubling bifurcation with c₂ increasing. The neural system has many types of multi-periodic orbits with period-2, -4, -8, … chaos … period -8, -4, -2. The corresponding phase portraits are shown in Figure 13 in yellow and red.

A further increase of c₂ causes system (2) to exhibit another pair of period-1 orbits employing the Hopf bifurcation (HB) of the nontrivial equilibria E₃ and E₄, which are labeled (3) and (4) and rendered in pink and blue in Figure 11. This implies that system (2) exhibits four coexisting attractors with two pairs of period-1 orbits. The phase portraits are shown in Figure 13(b) with c₂ = 4.5 for the initial values (±1, ±1, 0, 0) and (±1.5, ±2, 0, 0). Moreover, with the same evolution as mentioned above, the new pair of period-1 orbits evolves into the chaos attractors and then regains the period-1 orbits employing the forward and inverse period-doubling bifurcation. The bifurcation diagrams are magnified in Figure 12c, d. In this evolution, the system has many types of multi-periodic orbits. The phase portraits of the period-2, -4 … chaos … period-2, -1 orbits are displayed in Figure 13(c–g), denoted by the pink (positive initial values) and blue (negative ones) trajectories.

The attractor basins, defined as the set of initial values whose trajectories converge on the respective attractor, are shown in Figure 14 for varying c₂, where the (x₁, y₁) plane, i.e., x₂ = y₂ = 0 cross-section is selected. The attractor basins are all originally symmetric. For the regime of four coexisting attractors, with c₂ = 5.1, the attractor basins are shown in Figure 14(a), where the yellow and red regions are the attractor basins of the periodic orbits that emerged via the Hopf bifurcation of E₁ and E₂, and the pink and blue regions are those for the second pair of periodic orbits. Furthermore, it follows from Figures 14(b, c) that the sizes of the pink and blue regions increase and are then eroded by the yellow and red ones. The second sequence of the periodic orbits will vanish. In fact, the period-1 orbits disappear by the reversal via homoclinic bifurcation (HomoB) of the nontrivial equilibrium. The system then exhibits only a pair of period-2 orbits, as shown in Figure 13(h).

Figure 14.  Attractor basins in y₁ = y₂ = 0 cross-section with the increasing parameter (a) c₂ = 5.1, (b) c₂ = 5.55, (c) c₂ = 5.75, (d) c₂ = 8.85, (e) c₂ = 9.99, and (f) c₂ = 14.5 for the fixed system k₁ = 1, k₂ = 1.6, and c₁ = 5.

DownLoad: Full-Size Img PowerPoint

The third pair of period-1 orbits is obtained when c₂ increases and passes through the critical value c₂ ≈ 8.71. The neural system represents four coexisting attractors, including a pair of period-4 orbits and a pair of period-1 orbits. The phase portraits are shown in Figure 13(i), where the red and yellow trajectories are the period-4 orbits with initial values (±1, ±1, 0, 0) and the pink and light green trajectories are period-1 orbits with (±3, ±1, 0, 0). It follows from Figure 14(d) that the attractor basins of the period-4 orbits rendered in blue and light blue are just divided into some small regions by the period-1 basins rendered in red and yellow. Furthermore, the new pair of period-1 orbits is presented employing the Hopf bifurcation of the nontrivial equilibria E₅ and E₆ with increasing c₂. This implies that the neural system presents a coexistence with six period orbits, i.e., a pair of period-4 and two pairs of period-1 orbits. The corresponding attractor basins are exhibited in Figure 14(e), where the new regions shown in deep yellow and light green are presented in the plane of initial values. The phase portraits are shown in Figure 13(j). Furthermore, it follows from Figure 13(k) that the period-1 orbits that emerged near c₂ ≈ 8.71 will disappear and that the neural system regains the four coexisting orbits, i.e., a pair of period-4 and period-1 orbits. The attractor basins are exhibited in Figure 14(f). Interestingly, the new period-1 orbits also evolve into the chaotic behaviors via the period-doubling bifurcation. The phase portraits are verified by the dynamic evolution, as shown in Figure 13(j–o). Finally, the neural system (2) exhibits two chaotic attractors coexisting with a pair of period-2 orbits.

In a word, the neural system (2) exhibits three sequences of dynamical coexistence with the periodic orbits and chaotic attractors when the neighbor-connection weight c₂ increases. The first two-coexisting chaotic behavior was obtained near c₂ ≈ 5.1 and vanished c₂ ≈ 6.0 by the forward and reversal period-doubling bifurcation. The second pair of chaos coexistence emerged c₂ ≈ 11.2 can evolve into the period-2 orbit near c₂ ≈ 14.5 by the reversal period-doubling. The third pair of chaos coexistence can be illustrated when the neighbor-connection weight was fixed as c₂ = 14.5, which evolves into the double-scroll chaotic behavior by the attractor merging crisis.

5.   Transient chaos

By varying c₂ in some regions, system (2) exhibits the transient chaotic attractor. Transient chaos is a novel phenomenon in high-dimensional nonlinear dynamical systems, such as the Hénon map system, Lorenz system, Hopfield neural networks, the Duffing oscillator, Chua's system, etc. In a dynamical system exhibiting transient chaos, the system trajectory exhibits a seemingly chaotic attractor for a longer time, but eventually evolves into a final non-chaotic state, such as a periodic orbit or equilibrium ^[41].

Numerical simulations show that system (2) exhibits transient chaos, as shown in Figures 15 and 16 for c₂ = 16.5. The Matlab ode45 method is used in numerical simulation. It should be noticed that the qualitative results of the transient chaotic attractor are the same with the different numerical methods. The chaos motion is presented in blue and the corresponding final period orbit is depicted in red. It follows from Figure 15 that the transient chaos motion evolves into a periodic orbit with developing time. Figure 16 shows the time histories, from which it can be seen that chaos motion is annihilated near t ≈ 700, and system (2) displays a periodic orbit from then on. It is worth noting that the time that the system takes to evolve from chaotic to periodic will be longer if the parameter is chosen near the critical value of the chaos motion.

Figure 15.  Phase portraits of transient chaos (blue) and final periodic orbit (red) in (a) x₁ – x₂ – y₁ and (b) y₁ – x₂ – y₂, with the fixed parameters k₁ = 1, k₂ = 1.6, c₁ = 5, and c₂ = 16.5 for the initial condition (1, 1, 0, 0).

DownLoad: Full-Size Img PowerPoint

Figure 16.  Time histories showing that transient chaos is preserved for time t∈(0,700), and then the final periodic orbit is obtained.

DownLoad: Full-Size Img PowerPoint

6.   Conclusion

Imitating the properties of a biological nerve system to build artificial neural network model plays an important role in the fields of the neural network applications. In the biological neural systems, experimental researches and theoretical analyses are shown that the multiple coexistence of stable steady states or chaotic attractors are very important and universal phenomenon. Further, multiple coexistence is in strong demand for neural network applications. In fact, associative memory storage in neural networks is defined by the multistability, which is the coexistence of equilibrium, periodic orbits, and chaotic attractors. Chaotic dynamics in neural systems plays an important role in the application of optimization problem and associative memory. In this paper, a simple inertial two-neural network system with a non-monotonic activation function was established. The complex dynamics including the bistable equilibria, bistable period orbits, multistable periodic orbits, and multistable chaotic attractors with multi-periodic orbits were presented by employing theoretical analyses and numerical simulations. The neural system exhibited the stable coexistence with the same-type and mixed-type attractors. Further, from a realistic point of view for the biological neural system, the noise is inevitable. The final activity of the multistable system is influenced by the external perturbations. The fluctuation between the different stable states maybe play a great flexibility in the system functions.

To exhibit the global evolution of system dynamics and find the multiple attractors' coexistence, we presented the bifurcation diagrams including equilibrium points and periodic orbits. The phase portraits and attractor basins were exhibited to verify the coexisting attractors. In fact, increasing the neighbor-connection weight, the first pair of period orbits was obtained by the Hopf bifurcation, which evolved into a multiperiodic orbit and then chaos motion by employing the forward period-doubling bifurcation. Furthermore, the chaotic attractor degenerated into a multiperiodic orbit via the reversal period-doubling bifurcation. The second and third pairs of period orbits were obtained by the Hopf bifurcation of another nontrivial equilibria, which can also be evolved into chaos attractors via the period-doubling bifurcation. The difference is that the second pair of period orbits regained periodic orbits via the reversal period-doubling bifurcation, and then disappeared via homoclinic bifurcation. The third pair evolved into a pair of single-scroll chaotic attractors, which transformed into double-scroll chaotic attractors via the attractor merging crisis. Additionally, the transient chaos was displayed using phase portraits and time histories.

Acknowledgments

This research was supported by the National Natural Science Foundation of China under Grant Nos. 11672177, 11772229, 11572224, and 11672185.

Conflict of Interest

The authors declare no conflict of interest.