1. Introduction

It remains a mystery how neural networks composed of neurons with fluctuating characteristics can reliably transmit information. In an attempt to solve this mystery, many approaches have been presented, including spike-coding metrics ^[1], spatiotemporal coding models ^{[2,3,4,5,6,7,8]}, and synchronous action models ^{[9,10,11,12,13]}. From a communication viewpoint, we previously showed that a signal can be transmitted in a multiplex communication manner within an artificial synchronous neural network ^[14,15]. We have also visualized information flow/communication in the brain ^[16,17]. Furthermore, we have shown that spike waves, which spread and propagate from stimulated neurons, are received by afferent neurons as random-like sequences in natural asynchronous neuronal networks ^[18]. These networks can be well-simulated by 2D mesh asynchronous neural networks composed of an integrate-and-fire model without leakage ^[19,20]. The theme of our research combines wet-lab experiments of cultured neuronal networks with computer simulations. Importantly, communication within the neural networks of the brain may be the basis for higher cognitive functions.

In the current study, we show using a 9 × 9 2D mesh neural network simulation that 9 to 1 multiplex communication is possible at a success rate of 99%. This study corresponds with our previous publication ^[21] on the ability of remote receiving neurons to identify two transmitting neuron groups stimulated in a cultured neuronal network, i.e., 2 to 1 communication. Section 2 explains the integrate-and-fire model without leakage. Furthermore, we show that spikes spread from transmitting neuron groups, propagate as spike waves, and are received by receiving neurons. In section 3, we show using the back propagation neural network (BPN) method that receiving neurons can classify which neuron group transmitted the spike waves. The simulation results and discussion are provided in sections 4 and 5, respectively.

---

2. Simulation

We performed a computer simulation to observe spike propagation. We designed a 2D network with a 9 × 9 mesh of neurons with connections that were randomly generated and uniformly distributed between +1 and -1/3 in each experiment; thus, the number of positive weights was three times the number of negative weights in the mesh. Each neuron has connections to and from eight neighboring neurons, except for peripheral ones (Figure 1).

The neuron model used was previously reported ^[20] (Figure 2), where thresholding effect to small variation of action potentials is modeled by fluctuations of neuron characteristics of accepting period and output delay. Neuron n accumulates weighted inputs during the accepting period, A_nk. If the weighted input sum is positive and the neuron is not in its refractory period, the neuron produces a spike after the output delay time, D_nk, in the k-th firing. Some of the parameters used in the simulation are provided below.

A basic accepting period, a_n, which is intrinsic to neuron n is randomly generated in each network within a range width from 18 (1.8 ms) to 22 (2.2 ms) and a 0.1 ms sampling bin (a_n ∈ {18, 19, … 22}). The true instantaneous accepting period, A_nk, of neuron n for the k-th firing is randomly given within the fluctuation range R_A(a_n) around a_n, i.e., A_nk ∈ R_A(a_n), where

R_A(a_n)={a_n − 1, a_n, a_n + 1}.

A_nk=a_n − 1, a_n, or a_n +1 with probability p_a, 1-2 × p_a, and p_a, respectively, and p_a=0.2.

We define the refractory period of a neuron as an accepting period plus the output delay. The output delay is comparably smaller than the accepting period; thus, we can regard the accepting period roughly equals the refractory period when the output delay is negligible. In the present simulation, the intrinsic accepting period, a_n, was set around 2.0 ms to raise the simulation speed; however, this value may be several times smaller than that of a typical true accepting (refractory) period.

Next, we defined the instantaneous variance of A_nk by

${F_{AC}} = {\text{ Expectation }}\left[ {{{\left( {{A_{nk}} - {a_n}} \right)}^2}} \right]{\text{ }}\left( {{\text{instantaneous variance of accepting period}}} \right)$

(1)

F_AC=Expectation [(A_nk − a_n)²] (instantaneous variance of accepting period) (1)

Similarly, the output delay is set with the following equation:

d_nis an intrinsic output delay time of neuron n, and randomly selected from {2, 3, … 8}. Though our network model is a regular 2D mesh type, the real distances between neurons will be various, and this d_n allows for this condition.

$\begin{gathered} {R_D}\left( {{d_n}} \right){\text{ }} = {\text{ }}\{ {d_n} - 1,{d_n},{d_n} + {\text{ }}1\} . \hfill \\ {D_{nk}} \in {R_D}\left( {{d_n}} \right),{\text{ and }}{p_d} = {\text{ }}0.2,{\text{ which is the same probability as }}{p_a}. \hfill \\ {F_{OD}} = {\text{ Expectation }}[{({D_{nk}} - {d_n})^2}]{\text{ }}\left( {{\text{instantaneous variance of output delay}}} \right) \hfill \\ \end{gathered}$

(2)

The stimulation of class c is represented by the spatiotemporal pattern, S_c, on 3 neurons in a transmitting neuron group, c:

${S_c} = {\text{ }}\left\{ {\left( {{N_c}_1,{t_c}_1} \right),{\text{ }}\left( {{N_c}_2,{t_c}_2} \right),{\text{ }}\left( {{N_c}_3,{t_c}_3} \right)} \right\};c = {\text{ }}1,{\text{ }}2,{\text{ }} \ldots {\text{ }}9,$

(3)

where N_c1 N_c2, N_c3 are stimulated neuron number, and t_c1, t_c2, t_c3 are stimulated time.

For example (see Figure 3),

$\begin{gathered} {S_1} = \left\{ {\left( {3,{\text{ }}1} \right),{\text{ }}\left( {37,{\text{ }}1} \right),{\text{ }}\left( {51,{\text{ }}1} \right)} \right\},{S_2} = \left\{ {\left( {1,{\text{ }}1} \right),{\text{ }}\left( {43,{\text{ }}1} \right),{\text{ }}\left( {48,{\text{ }}1} \right)} \right\},\hfill \\ \qquad \qquad \qquad \cdot \cdot \cdot ,{S_9} = \left\{ {{\text{ }}\left( {13,{\text{ }}1} \right),{\text{ }}\left( {34,{\text{ }}1} \right),{\text{ }}\left( {55,{\text{ }}1} \right)} \right\}. \hfill \\ \end{gathered}$

(4)

One of S_c’s is selected in a trial and all neurons in S_c are stimulated at the same time t=1 [bin].

Though the number of neurons Q in the transmitting neuron group is not always 3, Q=3 is used for simplicity in the following examples. The same holds true for the number of receiving neuron groups (i.e., M=3) in the following examples.

An example of the spike waves in a simulation of a 9 × 9 neural network was previously shown ^[20]; thus, the extended result for a 25 × 25 neural network is shown in Figure 4, which includes “spike waves” that propagate from stimulated transmitting neurons. This figure illustrates the aim of our full-scale research.

---

3. Classification by BPN

A trial denotes the process from when the neurons are stimulated by 1 of 9 spatiotemporal pattern classes to the final determination of which pattern class was used for the stimulation. It is not possible to identify the kind of information that is transmitted within the waves by only observing wave propagation. The BPN method has moderately strong pattern recognition ability ^[22]; thus, the BPN method can be applied to show the feasibility of multiplex communication in neural networks. We used this method as the receiving/recognizing scheme for communication in the neural network.

The logical composition of the simulation is shown in Figure 5. There are several neuron settings. One of 9 kinds (classes, c) of stimulation is applied to the target 9 × 9 neural network per trial. In each stimulation, Q=1, 2, or 3 neurons fire as specified by (3).

In the network, the receiving neuron groups, M, are defined as 1, 2, or 3. Each group is composed of 2 × 2 blocks of neurons. The total number of receiving neurons is 4M=4, 8, or 12. An example of the geometric arrangement of the 9 × 9 network with (4) is shown in Figure 3. One of the receiving neurons is assigned as a reference receiving neuron prior to the simulations. The time sequences for spike occurrence in the network are shown in Figure 6.

We used the first four waves at the reference receiving neuron for the identification. Since the wave arrival times are not known beforehand at each neuron, we set the reference neuron. There, we encoded 3 intervals between adjacent spikes into [−1, 1] as shown in Figure 7. If the spike interval exceeds the minimum refractory period, Tr, then the reference receiving neuron is encoded according to the following equations.

The encoded value for the interval _k between the k-th and (k + 1)-th spikes is defined by (5):

$f({\Delta _k}){\text{ }} = \max {\text{ }}[3 - 2({\Delta _k}/Tr),- 1],$

(5)

where _k=t_k+1 − t_k and t_k=time of the k-th spike.

Now, we have already known the four wave arrival times to the reference neuron, and we can utilize them to encode the spikes at another receiving neuron n. A spike near the k-th spike at the t_k of the reference neuron is encoded by the following equations (Figure 8):

If a j-th spike is received by neuron n at time t_jⁿ ∈ [t_k − Tr/2, t_k + Tr/2], ∃j=1, 2, … k=1, 2, 3, 4, then

${f^n}_k = \max {\text{ }}[1 - 2|{t_k} - {t^n}_j|/Tr,{\text{ }}0],$

(6)

${g^n}_k = {\text{sign }}[{t_k} - {t^n}_j].$

(7)

Otherwise,

${f^n}_k = {g^n}_k = {\text{ }}0;k = {\text{ }}1,{\text{ }}2,{\text{ }}3,{\text{ }}4.$

(8)

Practically, receiving neurons are spatially close; thus, their firings are similar and do not often fit (8), i.e., j=k in most cases.

The refractory period (roughly equal to the accepting period) fluctuates time to time in our model; thus, Tr is defined as the minimum of these value. In the current simulation, the minimum instantaneous accepting period was 18 − 1 (=17), the output delay was 2 − 1 (=1), and the total Tr was 17 + 1=18 (1.8 ms). The difference, f_kⁿ, of the arrival time at another receiving neuron n from the reference receiving neuron as well as its sign, g_kⁿ, (+1, −1; precede or delay) were obtained. The 3 wave arrival interval lengths, f ( _k), at the reference neuron were also sent to the BP N; thus, the total number of input neurons to the BPN is 8(4M − 1) + 3. The number of middle layer neurons was 45, and the learning rate of all computational units was 0.2. The reference receiving neuron and other receiving neurons can receive spikes at any time, which is defined as asynchronous reception.

---

4. Classification Results

The correct rate of classification by the BPN depends on how much the neuron characteristics fluctuate, how many kinds of spatiotemporal patterns are given in a trial, and how many receiving neurons are employed. After many trials, the correct rates of pattern classification by the BPN after learning are shown in Figure 9. Each data point was obtained from more than 50 randomly generated networks and more than 250 learning cycles within each network after the learning converged. The fluctuation sizes are F_OD=F_AC=0.4. Parameters Q (number of neurons in a transmitting neuron group)=1 (9), 3 (27), and M (number of receiving neuron groups)=1 (4), 2 (8), 3 (12). The numbers in parentheses are the total number of neurons transmitting or receiving. The maximum correct rate of 0.987 was obtained for Q=3 and M=3. In this case, the average number of learning cycles needed to reach convergence was 72.5.

It appears that a larger number of neurons involved in communication, e.g., 3 neurons in a transmitting neuron group and 3 receiving groups, results in smooth and stable transmission, whereas fewer neurons make communication more difficult. Increasing M is more effective than increasing Q. In addition, larger fluctuations in neuron characteristics, a larger number of kinds of patterns, and a wider range of connections (e.g., 24 neighbors) decrease the correct classification rate of the BPN. Results of extensive simulations are in process and will be reported in subsequent papers. In these simulations, i5 PC’s are employed with Basic program. Computation time varies according to various conditions such as M, Q, F_AC, F_OD, mesh size, and also need many runs to obtain statistical averages. It ranges from several minutes to a few months as well as parallel usages of several PC’s.

---

5. Discussion and Conclusion

The coding and communication mechanisms of neural networks are yet to be determined. In a simulation of a 2D mesh neural network, we showed that 9 to 1 communication is possible with classification by the BPN, irrespective of fluctuations in neuron characteristics. Although the specific classification algorithms are different, this result corresponds to our wet-lab experiment on 2 to 1 communication in cultured neuronal networks, which showed discrimination with the dynamic time warping (DTW) significance test ^[21]. In this previous study, some (receiving for test) neuron groups were able to discriminate which of the two (transmitting) neuron groups was stimulated, while others failed to make the discrimination. In the current simulation, neural networks could identify even 9 multiplexed signal sources despite various randomly generated weights. This ability was likely due to the strong learning and discrimination abilities of the BPN and combination of neuron groups.

These experiments support the hypothesis that the spatiotemporal firing pattern is transmitted through the neural network as spike waves in a broadcast manner. At the receiving side, neurons may decode the spatiotemporal pattern of the wave and respond according to the transmitted pattern class. In other words, each neuron acts as a local minor relaying media and is relatively insensitive to the communication function of the whole network. For example, some spike losses will not cause a loss in communication. In this process, a refractory period helps to regulate and stabilize the spike waves. The wave proceeds to a new area that has not fired recently, and as a result, the wave front formed often appears as a synfire chain or synchronous and coherent firing. In other words, spatially close neurons often fire similarly and together. Synchronistic group activity, rather than single neuron activity, is effective for stable communication, and is often observed in neuronal networks ^[23,24,25].

This communication process is similar to sound wave propagation in the air (Figure 10). With this type of communication, we can identify the source and type of sounds. The communication process used in neural networks is also partially similar to diversity communication, in which plural antennae are spatially separated at base stations to stabilize mobile communication ^[26].

We confirmed that each neuron can perform 3 types of tasks, including signal emission, relay, and reception. However, within our model neurons mainly function as a relaying media for multiplex communication. In this study, we showed that 9 to 1 transmission is possible in a 9 × 9 2D neural network. Currently, we are assessing wider networks with more natural recognition filters, i.e., Laplacian Gaussian functions, instead of the BPN method which is inconsistent with natural functioning. We believe these communication functions may explain the physiological basis of higher order cognitive functions.

Rather than high precision processors, many low precision processers are used in the AlphaGo of Google AI, which showed strong power for AI ^[27]. In our case, although each neuron has a low precision processing function, e.g., fluctuating characteristics, the overall neural network communicates well.

The features of this paper are summarized as follows:

(1) To our knowledge this paper is the first attempt to simulate multiplex communication in a neural network. Our work shows a signal transmission principle in neural networks which provides a possible solution to the mystery of the manner of reliable neural communication.

(2) The simulation corresponds to our wet lab experiment ^[21] of two to one communication in cultured neuronal network.

(3) The simulation showed quantitatively that grouping firing of neurons is effective for stable information transmission, which is often observed in the naturally occuring neuronal networks.

---

Acknowledgements

This study was supported in part by the Grant-in-Aid for Scientific Research of Exploratory Research JP21656100, JP25630176, JP16K12524 and Scientific Research (A) JP22246054 of Japan Society for the Promotion of Science.

---

Conflict of Interest

The authors declare that there is no conflict of interest regarding the publication of this paper.

---