Single dendritic neural classification with an effective spherical search-based whale learning algorithm

1.   Introduction

Artificial neural networks have been studied for more than half a century ^[1]. Among which, deep learning ^[2], over the last two decades, has dominated the area of artificial intelligence research as a result of the advancement of computer technology. Speech recognition has benefited greatly from deep learning ^[3]. Visual classification and recognition ^[4], unmanned vehicles ^[5,6] as well as protein structure prediction ^[7] also achieve fundamental technical evolution with the aid of deep learning. Nowadays, the popular generative adversarial networks ^[8], neural network architecture search ^[9], migration learning ^[10], federation learning ^[11], multitask learning ^[12], meta-learning ^[5] and attention models ^[13] are all built based on neural networks.

Traditional neural networks usually contain one or two hidden layers, where each "neuron" is a very simple computational unit ^[14]. A neuron receives input signals from others, which are amplified by the connection rights, and when they reach a neuron, if the total amount exceeds a certain threshold, the current neuron is "activated" and passes its output signals outwards. In fact, each neuron is a very simple computational equation, and a so-called neural network is a mathematical system obtained by nesting and iterating many such computational equations. The basic computational units of a neural network are continuously differentiable. In the past, the sigmoid function was used as an activation function of the neuron model, which was continuously differentiable. Recently, the rectified linear units (ReLU) is used as an activation function in deep neural networks, which is also continuously differentiable ^[15]. Thus, it is relatively easy to calculate the "gradient" of the whole system. Therefore, the BP algorithm is used to train the neural network by gradient descent optimization ^[16]. Before the work of Hinton et al. ^[17], it was not possible to train a neural network with more than five layers of the meridians because the neural network suffered from "gradient disappearance" during training. When the BP algorithm back-propagates the output layer of the neural network to the part far from it, a "zero" adjustment may be derived, so that the part of the network far from the output layer cannot be adjusted according to the output error, thus making the training fail. This is a huge technical hurdle to develop deep neural networks based on traditional neural networks, and Hinton et al. demonstrate that deep neural networks may be trained using "joint fine-tuning after layer-by-layer training" to avoid gradient disappearance ^[17].

The complexity of a machine learning model is related to its capacity, which has a significant impact on its learning ability ^[18,19]. If the complexity of a learning model can be increased, then its learning ability can also be improved. There are two ways to do this for neural network models: make the models "deeper" or "wider" ^[20,21,22]. From the perspective of improving the complexity of the models, "deepening" is more effective because, in simple terms, only the amount of computational units is increased through "widening" and then that of basis functions; whereas through "deepening", the amount of not only basis functions but also layers of function nesting is increased, so the general function expression ability will be stronger. Therefore, in order to improve the complexity, the network should be "deepened" ^[23].

However, instead of building neural networks in increasing depth, some researchers have reconsidered the problem of modeling neurons from scratch^[24]. The McCulloch-Pitts neuron has long been criticized for being overly simplified, using only weight to represent the strength of the connection between two neurons ^[25,26]. The complexity of a genuine biological neuron allows it to exhibit both temporal and spatial characteristics. Some spiking neuron models have been presented, which were motivated by the capacity of neurons to process temporal information ^{[27,28,29,30]}. As basic units, due to their rich temporal dynamics, the new generations of neural units created as a consequence of the integration of spiking neuron models into deep learning ^[31] offer a great lot of promise for handling challenging recognition and prediction tasks. On the other hand, the construction of neuronal models based on their spatial functionality and large tree-like structures have also received a lot of research ^[32]. However, it is traditionally believed that this requires multiple layers of McCulloch-Pitts neurons ^[33], and that individual neurons with large dendritic structures remain a challenge to model ^[34,35].

Evolutionary optimization and brain-like inspired deep learning are at the core of artificial intelligence, both of which are inspired by the knowledge of processing and natural evolution in the brain to obtain optimal solutions and advance artificial intelligence ^[36]. Evolutionary computation is influenced by the natural selection mechanism of "superiority and inferiority" during biological evolution based on the law of transmission of genetic information, which is simulated by iterative programs that are used to solve the problems as an environment and seek optimal solutions through a natural evolution in a population of possible solutions. In 2005, Julie Greensmith ^[37] proposed that hazard theory doctrines and dendritic cell work mechanisms inspired immune algorithms that could be used to exploit prior knowledge to construct new algorithms that were fault-tolerant, immune, robust and evolutionary, and that optimal solutions closer to the dynamics in real-world problems should be sought. With the deepening of brain-like perception and cognition, the understanding and study of brain biomechanics as well as the sparsity, learning, selectivity, directionality, knowledge and diversity of brain neurons, deep learning networks are now being built to simulate these properties ^[32]. Thereby, the feedback connections between dendrites and dendrites, dendrites and synapses, layers and layers as well as neurons and neurons are less studied and utilized ^[32].

A neuron contains $10^4$ dendritic interconnections on average, which leads to a broad range of neuronal morphology. The nonlinearity of dendrites explains the dynamics of various neurons, which mechanistically demonstrates the role of biological neurons in various circuits with behavioral significance ^[38]. Peripheral evidence has provided strong support for the robustness of dendritic computation, such as auditory coincidence detection ^[39], direction selectivity of the local computation of dendritic branches ^[40] and logical operations ^[41]. Most recently, studies have even shown that complex exclusive OR (XOR) logical computations can be performed through dendrites. Several dendritic neuron models have been proposed ^[42,43,44]. Logic ANDs and ORs models were employed as synapses inside the dendritic tree for interaction in Koch and Segev's logical neuron model ^[45]. In a two-layer model of a single neuron given by Poirazi et al. ^[46], synaptic subunits experienced nonlinear information processing in the sigma style. Legenstein and Maass ^[47] offered a mathematical model that supported the system's capacity to be self-organized in mixing various input qualities. These models, albeit used to take advantage of synapses' nonlinear properties, are unable to resolve complicated issues, notably those involving nonlinear separation ^[47]. Recently, it was shown that $k$-tree neuron models, which depicted dendritic structures as entire binary trees, excel at various challenging machine learning tasks including creating and expanding MNIST ^[48]. However, as the authors have pointed out ^[48], the model is biologically infeasible for presenting an unstructured input from mathematical representations and one-dimensional computational activities. By articulating synaptic processing in terms of sigmoid functions and dendritic interactions inside dendrites, which are performed by multiplicative operations, the development of a new physiologically realistic dendritic neuron model (DNM) has been made ^[49,50,51].

DNM has proved its capacity to solve difficult challenges as a single neuron model ^{[50,52,53,54]}. Its huge dendritic connection structure is considered, so as to construct individual neuron learning models under the width structure, make full use of the nonlinear information processing mechanism on the synapses and achieve an automatic deletion ^[55] on the neuron structure. The intelligent algorithm of multi-objective evolutionary learning ^[56,57] is used to complete the neurons in the idiosyncratic function module, hyper-parameters, network structure and learning algorithm auto-evolution on itself. Evolutionary computation is combined with deep networks ^[58], in which weight learning is used for weight optimization of neural networks, dynamic structure learning for neural network structure search, and super-parametric learning for the optimization of neural network parameters. The properties of the standard problem of deep learning networks are used to maintain the diversity of populations, deep learning networks are used to replace the expensive process of fitness evaluation, and deep neural networks are used to learn the solution selection mechanism of evolutionary computation. Six evolutionary algorithms are presented to learn a novel dendritic neuron model with successful applications to classification, fitting and prediction problems ^[50,59]. In addition, the authors use the "orthogonal experiment method" to systematically study the algorithm parameters and obtain optimal parameter combinations. With continuous research on evolutionary algorithms, the necessity of combining evolutionary computation with deep networks is becoming more and more prominent. A training dendritic neuron model learning method is proposed based on the differential evolutionary algorithm of scale-free networks, and the trained model shows very a high prediction accuracy in real PV power prediction problems, through which successful applications to classification and fitting problems are also achieved ^[60]. The classification and fitting issues have been satisfactorily addressed using the trained model. The experimental findings demonstrate that the learning method suggested in the research, which is based on the globally optimum differential evolution algorithm, may be utilized to significantly enhance the overall performance of the dendritic neuron model, are also successfully applied to complex function optimization and many practical problems to train dendritic neuron models ^[61]. Gao extended the original dendritic neural network to a full-complex dendritic neuron model (CDNM), deriving its error backpropagation algorithm in the complex domain and comprehensively discussing its initialization conditions, with nine different analytic functions as its activation functions ^[54]. The effectiveness of CDNM is successfully verified for complex XOR, non-minimum phase balance problems, and complex wind energy prediction problems. In addition, the combination of evolutionary computation and deep networks has shown outstanding results in dealing with practical problems ^[58].

Although various evolutionary algorithms have been proposed to learn DNMs, creating a cutting-edge DNM learning algorithm is necessary and challenging. Mirjalili introduced the whale optimization algorithm (WOA) ^[62] in 2016 to optimize the solution of the objective function by imitating the foraging behavior of whale schools in the ocean. WOA offers benefits such as straightforward operation, fewer parameters to specify, and strong performance in finding the optimum. Nevertheless, it has flaws including weak convergence precision and a tendency to quickly enter local optimum. To address these shortcomings, the hybrid algorithm is also quite hot recently, which compensates for their shortcomings through other algorithms. Depending on the spherical search algorithm (SS) ^[63,64] and WOA, we propose a novel hybrid algorithm, namely SSWOA, to learn DNM. In SSWOA, the SS algorithm is used to improve the low convergence problem of WOA, which tends to fall into local optima. The local exploitation of the search space is the main goal of the WOA algorithm. WOA optimization can speed up convergence while preventing a local optimum by modifying non-convergence factors, establishing adaptive position weights, and setting adaptive thresholds. To our best knowledge, no hybrid algorithm was employed to understand the single dendritic neuron framework.

The contributions made in this study are listed as follows: 1) We innovatively employed a dynamic mixing of two separate algorithms via an intricate co-evolutionary process to test if the hybrid algorithm's performance can be improved further while learning single dendritic neuron models. 2) The approach outperforms the original evolutionary algorithm, a derivative of the WOA algorithm, as well as several other methods to train neural models, according to extensive tests conducted in roughly three directions. The SSWOA algorithm may be used to more effectively resolve challenging practical problems from the standpoint of useful functions.

The remainder of the article is structured as follows: the details of DNM is elaborated in Section 2. SSWOA for learning DNM are discussed in Section 3. Section 4 provides comparison findings on 11 common classification tasks. Finally, Section 5 provides a summary and future projections.

2.   Model architecture

There are $10^ {11}$ neurons in the brain, with a total of nearly $10^ {15}$ neural connections among the neurons. There is an average of $10^4$ interconnections for a neuron upon dendrites, causing a vast range of different neuronal morphology. Nonlinearity of dendrites accounts for the dynamics of various neurons, which mechanistically explains the role of biological neurons in various circuits with behavioral significance ^[38]. Each neuron is capable of unequally handling thousands of different synaptic inputs. Moreover, the large dendritic structure of dendrites ^[32] can largely help explain why biological neural networks are able to handle quite complex tasks but consume little energy. Not only the postsynaptic signals are nonlinearly integrated in dendrites, but irrelevant background information is also filtered out there ^[65]. A physiologically valid single-neuron model was presented in light of the dendrites' capacity for nonlinear information processing, that is, DNM ^[50,51].

DNM include four layers, i.e., the synaptic layer, dendritic layer, membrane layer, and somatic layer. The method by which dendritic neurons in living organisms acquire information is seen in Figure 1(a). A biological model of dendrites and extracellular inputs from other neurons is shown in Figure 1(b). In contrast, specifically, information from other neurons (i.e., stimuli) is collected in the synaptic layer. Information collected from other neurons (i.e., stimuli) is collected in the synapses and then transferred to the dendritic layer, where all of the data is multiplicatively processed, and the outcomes are subsequently sent to the membrane layer for accumulative processing. All of the results are then added together and sent to the soma layer, where the outcome is processed using another sigmoid function.

2.1.   Dendritic layer

Synapses enable neurons to receive signals from axons or dendrites of other synapses by joining them to other dendrites or axons. Each input signal has a unique weight or threshold that is connected to a branch of the dendrite on the axon. The relationship between the $i$th ($i = 1, 2, ..., N$) synapse and the $j$th ($j = 1, 2, ..., M$) synaptic layer, which receives input signals, is shown by the formula:

In this equation, $x_i$ is the $i$th dendritic input, $x_i$ is 0 or 1, $Y_{ij}$ is the $i$th synaptic input to the $j$th output, and $k$ is a parameter that describes the strength of presynaptic and postsynaptic connections among neurons. The connection parameters $w_{ij}$ and $\theta_{ij}$ ^[50] indicate the weighting as well as threshold of synapses, respectively. A learning method will be used to teach these two sorts of parameters.

After a synapse is activated through the sigmoid function, there are four connectivity scenarios ^[49], as is shown in Figure 2, replying on different scores of $w_{ij}$ and $\theta_{ij}$: direct-connecting state is also known as being excited to connect (), the opposite-connecting state is also known as an inhibitory connection (), constant-1 connection () and constant-0 connection (). Furthermore, these four states can be separated into the six situations depicted in Figure 3. Presynaptic neural input is represented by the horizontal axis, while synaptic layer output is represented by the vertical axis.

1) As is shown in Figure 3a(1), when 0$\leq w_{ij} < \theta_{ij}$, for example, if $w_{ij} = 1.0$ and $\theta_{ij} = 1.5$, then it indicates that there is never a connection. Regardless of its input value, the result is a constant score of 0. As is shown in Figure 3a(2), when $w_{ij} < 0 < \theta_{ij}$, for example, if $w_{ij} = -1.0$ and $\theta_{ij} = 0.5$, the output score is 0 regardless of the input.

2) As is shown in Figure 3(b), when $w_{ij} < \theta_{ij} < 0$, a backward connection is established, for example, if $w_{ij} = -1.0$ and $\theta_{ij} = -0.5$, it indicates that regardless of whenever the input goes from 0 to 1, the potential is always inversely proportional to the input.

3) As is shown in Figure 3(c), when $0 < \theta_{ij} < w_{ij}$, there is a direct link created, which is a fascinating relationship. For example, if $w_{ij} = 1.0$ and $\theta_{ij} = 0.5$, the input is proportional to the output anytime when the input is changed from 0 to 1.

4) As is shown in Figure 3d(1), when $\theta_{ij} < 0 < w_{ij}$, for example, if $w_{ij} = 1.0$ and $\theta_{ij} = -0.5$, it signifies that the output is a linked constant score of 1, which is independent of the input value. As is shown in Figure 3 d(2), when $\theta_{ij} < w_{ij} < 0$, for example, if $w_{ij} = -1.0$ and $\theta_{ij} = -1.5$, 1 is the output value, which is unrelated to the input.

2.2.   Dendrite layer

Impulses are sent from the synaptic layer to the dendritic layer, which receives them. Because synapses are nonlinear, that is, nonlinear signals are processed at the dendritic layer using the multiplication operation, which is the same as logical AND, and can be explained as follows:

2.3.   Membrane layer

Each dendrite sends signals to the membrane layer, which adds the results, before sending the processed information to the soma bodies. A logical OR operation, which can be expressed as the term for this summing process:

2.4.   Soma layer

After layer-by-layer signal processing is finished, the soma layer receives membrane layer signals, and the final result is determined using a sigmoid function, which is expressed as follows:

where $\theta_{s}$ has a score range of $[0, 1]$ and $k_{s}$ is a distance score that represents the distance between the dendrites and the soma.

2.5.   Neuronal pruning mechanism

Nodes and weights are eliminated through a pruning mechanism, which is the most important feature of DNM. Some dendrites and synapses can be unnecessary since the DNM has a complete connectivity pattern at first. The DNM can be pruned during training by recognizing superfluous synapses and duplicated dendrites. For many practical challenges, a simpler and unique neural structure can be generated. The neural pruning process consists of two parts: dendritic pruning and synaptic pruning. For synaptic pruning, the synaptic output is always when the synaptic layer receiving the axonal input is in a constant-1 connection. Because of the multiplication used in the dendritic layer, any arbitrary score multiplied by 1 will equal itself in the dendritic layer, and none of the results will change. There is clearly a constant-1 connection in which the synaptic input has the least effect on the output of the dendritic layer. Thus, those synapses with a constant-1 connectivity have such little impact on the dendritic layer's output that it is possible to disregard them entirely. Therefore, throughout the computation we may completely disregard these synaptic layers. For pruning of dendrites, the output is always 0 when the synaptic layer receiving the input signal is in a constant-0 connection, regardless of the input value. When one synaptic output is 0, the output of all other synapses is also 0 due to the multiplication operation. With a synapse's output being 0, the entire dendritic branch's outcome will also be 0. They should be removed because they barely affect the outcome of the adjacent soma. Thus, such dendritic layers ought to be eliminated.

DNM can also be used to implement neuronal pruning. Figure 4 shows the simulation for synaptic and dendritic pruning. First, the DNM has with two dendrites and four synapses, as shown in Figure 4(a). From the Figure 4(b) and (c), it can be seen that synaptic and dendritic pruning has occurred. According to the pruning mechanism of the neuron, the first synaptic layer needs to be discarded because it belongs to the case of constant-1 connection. The fourth synapse on the second dendrite belongs to the constant-0 connection case, so this dendrite should also be eliminated. Finally, the morphology of the neuron after DNM pruning is shown in Figure 4(d).

3.   Learning algorithm SSWOA

To train DNMs, an amount of training methods have been developed. The stochastic gradient descent method BP has been widely employed ^[51,52,66], but it suffers from the local trapping problems, which influences the performance of models with high inductive power. An effective global search optimization method, biogeography-based optimization (BBO) ^[50] outperforms ant colony optimization (ACO) ^[59], genetic algorithm (GA), evolutionary strategy (ES), particle swarm optimization (PSO), and BP by a wide margin, showing to be a promising DNM training method. In addition, states of matter search algorithm (SMS) ^[67] has also shown its superior performance in training DNMs. Recently, DNM classification ^[68] and wind speed prediction ^[69] have both employed differential evolution (DE) and one of its variations, LSHADE ^[70]. The learning capacity of DNMs has been significantly enhanced by the most recent state-of-the-art information feedback-based differential evolution algorithm (IFDE) ^[71]. Given that the learning algorithm significantly influences how well DNMs perform, single algorithms have been used to train DNMs for a long time, which motivates us to further enhance performance by utilizing several algorithms to address a single algorithm's faults. Therefore, the major objective of this work is to provide the best learning method for DNMs.

The mathematical optimization challenge of minimizing the objective function can be used to represent the learning problem of DNMs, which is represented as:

where the SSWOA is the mean square error (MSE) between the computed target value and the output of the DNM for individual $X_i$. $P$ stands for the quantity of training samples, while $T_p$ and $O_p$ stand for the $p$th target value and the DNM's actual output, respectively. Individual solutions represented by the following are among the learnable parameters in the DNM's synaptic layer:

where $M$ is the number of DNM dendrites and $n$ is the number of external inputs. As a result, the dimensionality of the DNM learning problem as it has been defined is $D = 2nM$.

3.1.   Characteristics of whale optimization and spherical search

The SS algorithm ^[63] is a vast, undirected search method that maintains a reasonable balance between the expansion and exploration of the search space, which can be used to compute spherical boundaries and develop innovative experimental answers for spherical boundary surfaces for non-linear global optimization problems with constrained boundaries. SS algorithm starts with half of the better solutions concentrating on development and the other half of the worse solutions concentrating on exploration, in contrast to the majority of algorithms, which usually begin with comprehensive exploration and then begin development, leading to increased diversity in the better one half of the solutions and increased adaptability in the other half of the worse solutions. Nonetheless, the search mechanism of SS algorithms leads to relatively poor exploration and often settles for local optima.

WOA ^[62] is a multi-application optimization strategy that mimics the preference behavior of humpback whales. Nonetheless, the ability to leverage the whale optimization technique is heavily dependent on the distance between the search unit and the current best solution, and a good enough current optimal solution is required. In this work, the benefits of the SS algorithm are combined with the WOA approach to generate a hybrid whale algorithmo-based search algorithm (SSWOA). One aspect of the SSWOA remains the computation of the search direction, thus the SS algorithm is used to approximately establish the orientation of the ideal solution, and then the whale optimization method is used to discover the optimal solution, thereby improving its performance.

The standard WOA has disadvantages such as slow convergence, the tendency to fall into local optima and low convergence accuracy, which will affect the overall performance. Hence, in this work, two improvements are innovatively made to the WOA: a non-linear convergence coefficient and a co-evolutionary control strategy.

3.2.   Spherical search optimization algorithm

Proposed by Abhishek Kumar ^[63] in 2019, a group-based meta-heuristic approach is the spherical search (SS) method, which can contribute to solving nonlinear restricted global optimization problems. The modified SS algorithm solution is shown in Figure 5 as a two-dimensional search space. The surface of the spherical boundary is used in the SS method to generate the suitable next solution for each individual solution. The method concentrates on global exploration while the spherical boundary is small, while it specializes on local advancement when the spherical boundary is large. As a result, the SS algorithm may in certain cases aid in achieving a balance between the advancement and exploration of the search area. The solution initialization in the SS algorithm is denoted as follows:

where $X_{hj}$ and $X_{lj}$ denote the $j$th element's upper and lower boundaries, respectively. Within the limit, $rand(0, 1\rbrack$ generates random numbers from a uniform distribution in $(0, 1]$.

The SS method uses the following formula to provide the appropriate test for the $i$th solution:

The projection matrix $P_i$ determines the score of $\epsilon$ on the spherical edge of (D-1). For a given solution, $\phi$ corresponds to multiple different $P_i^{k}$ values and gives several $\epsilon$ values. The range of the step length control vector $C_i^{k}$ is $[0.5, 0.7]$. The search direction is $\varphi$, which is produced by utilizing two random solutions as target locations. With "towards-rand" having stronger exploration skills and "towards-best" having better local exploitation capabilities, the SS method uses both of these features to determine the search direction. As a result, in the exploration space, the balance between the advancement and exploration is maintained. Through the instilling of the variety in a superior solution as well as inferior options, adaptation is promoted.

The direction of the search is $\varphi$. It is necessary to calculate the search direction as follows, using $r1$ and $r2$:

The desired point is $\phi$. In Eq (3.5), from the present set of solutions, $r1$ and $r2$ are chosen at random in the population.

The search direction, $\varphi$ for the $k$th iteration of the $i$th solution in "towards-rand" is determined as:

The search direction, $\varphi$, for the $k$th iteration of the $i$th solution is determined in "towards-best" as:

The population of $i$ solutions is updated and filled with a new set in the subsequent iteration using a greedy selection strategy that adheres to the following guidelines: If the test solution's score $f(\epsilon)$ is less than that of the solution to the goal function $f(\phi)$, therefore, $y_i$ substitutes $x_i$, expressed as:

3.3.   Whale optimization algorithm

In 2016, Seyedali Mirjalili ^[62] introduced the whale optimization algorithm as a population-based search strategy, as well as a program that mimics the hunting behavior of humpback whales via bubble nets. Three parts make up the algorithm: encircling the target, unleashing a bubble net assault, and hunting for prey. Figure 6 provides a rough idea of the bubble net search mechanism implemented in WOA: Figure 6(a) is the shrink-wrap mechanism and Figure 6(b) is the spiral update position. Considering that the intended prey is the best option for the existing population, other whales in the population, after the selection of prey, would update their locations based on the current position of the prey. Suppose that the ideal solution in the current population is the target prey, other whales in the population, after the selection of prey, would update their locations based on the current position of the prey, with the mathematical model as follows:

where $t$ stands for the current iteration count, and $\vec{X_{best}^{t}}$ represents the optimum solution's position in the present group vector. $\vec{X_{i}^{t}}$ shows where the whale is at the moment, $\vec{A}\cdot \vec{D}$ indicates enclosing measure, and $\vec{A}$ and $\vec{C}$ relate to the coefficient vectors which are represented as:

where $\vec{a}$ decreases linearly in the iterative procedure from 2 to 0, $\vec{r_{1}}$ and $\vec{r_{2}}$ are random vectors produced using a uniform distribution in the [0, 1] range. Equation (3.12) states that the search unit (whale) modifies its location in accordance with the location of the ideal solution (prey). By changing the values of the $A$ and $C$ vectors, the whale's area surrounding its prey may be managed.

According to Eq (3.13), reducing the score of a variable in Eq (3.11) results in the contraction and enclosing behavior of the whale:

where $t$ denotes the number of permissible iterations and $MaxIter$ the maximum number of allowed iterations. For a spiral route replica, the distance between the search unit $\vec{X_{i}^{t}}$ and the best response so far $\vec{X_{best}^{t}}$ is computed first, followed by the spiral equation as in Eq (3.14). Following that, is used to establish the position of the nearest search unit:

where $\vec{D}$ is the distance between the $i$th Whale and its prey, $b$ is a constant defining the logarithmic spiral form, and l is a random value between [-1, 1]. This is the best response so far. The whale must hunt following the spiral pattern the prey takes as it shrinks the area around it. So, according to the WOA algorithm, there is a 0.5 chance that the whale would use two different hunting strategies. Here is the mathematical model:

3.4.   Nonlinear convergence factor

Global search and local advancement are two actions performed by swarm intelligence algorithms. If the two are not appropriately timed, the algorithm will enter local optimization or suffer from poor performance and slower convergence. Population will remain diverse if the global search strength is strong enough to prevent the algorithm from settling into the local optimum. The speed of the algorithm's convergence will increase whether local advancement can be ensured by the local advancement strength. The WOA algorithm illustrates that the global search and local advancement capabilities of WOA are dependent on parameter $A$. According to Eqs (3.10), (3.14), and (3.15), the parameter "$\vec{A}$" is reliant on the convergence factor "$\vec{A}$" in the phase of comparing WOA. If the convergence factor $\vec{a}$ is larger, the algorithm's ability for global exploration will be strengthened, preventing it from settling into the local optimum. On the other hand, if the convergence factor $\vec{a}$ is less, the algorithm will be more capable of global exploration. The algorithm's capacity for local growth will be enhanced, and will have a faster convergence rate. As the amount of iterations grows in WOA, $\vec{a}$ decreases linearly. Based on the literature, the following formula must be employed to ensure global exploration and local advancement capabilities, as well as to increase the algorithm's convergence rate:

Equation (3.16) should be used when $\vec{a}$ is between 2 and 0, while Eq (3.17) should be used when $\vec{a}$ is between -1 and -2. Among them, $t$ which indicates the current iteration count and $maxIter$, which stands for the maximum iteration count. The first phases $\vec{a}$ will improve the capability of global investigation and quicken the velocity of algorithm convergence when the effects of the convergence factor $\vec{a}$. Later on, it will be more biased, with a focus on local advancement.

3.5.   Co-evolutionary control strategy

WOA's predatory behavior targeting humpback whale bubble nets employs two strategies: spiral renewal and contraction circular processes. To choose a foraging strategy, the randomly generated $P$ in $[0, 1]$ is compared to the algorithm's assumed threshold of 0.5 by modeling the two processes (contracting and enclosing prey and spiral paths) as illustrated in the calculation. Repeated testing on the data set given in the image has made it clearly evident that WOA's convergence time is unnecessarily slow, making the local optimization issue easy to introduce. The possibility of altering the threshold in WOA has been demonstrated in ^[72]. The score of the threshold will have an effect on the algorithm's findings while attempting to find the best answer. If the score is too high, the algorithm's optimization effect will be very little, but if the score is too small, the algorithm's capacity to evolve locally will be hindered. The feasibility of adaptive parameters is supported by references ^[72,73]. WOA's original 0.5 threshold is replaced with the adaptive threshold. It will change when the algorithm is adjusted repeatedly to discover the best mechanism for both local and global advancement. Based on the above considerations, the threshold is modeled in the following way:

Among them, $\gamma$ represents the threshold in WOA, while t and maxIter stand for the current and maximum iterations, respectively. $\alpha$ and $\beta$ denote the control parameters, respectively. The updating rule of individuals changes to the following equation when $\alpha = 3$ and $\beta = 2$.

3.6.   Adaptive position weights

The position vector of the randomly chosen whales, $vec{a}$ and $vec{A}$, is the primary factor that may affect the WOA location update process in addition to the WOA algorithm's location update process. i.e., the population $\vec{X_{best}^{t}}$'s position vector that was randomly chosen ^[74]. Moreover, taking into consideration the numerous interactions between randomly selected whales and their prey at various times during the algorithm's iterative phase, adaptive parameters are utilized as weight coefficients and paired with the aforementioned threshold to rewrite the locations of WOA as follows:

where $\omega$ is the prey adaptive weight, which is chosen between [0, 1]. If $\omega$ increases as the number of iterations increases, so will the optimal solution of the prey chosen in each iteration, i.e., for the present population. In Eqs (3.10), (3.16), and (3.17), the weight coefficient increases as the number of iterations increases. While updating the location of the whale in Eqs (3.23)–(3.25), one may investigate the best solution around the prey using the algorithm's rapid convergence and optimization accuracy Eq (3.25), thereby improving the local advancement ability. The whole implementation of SSWOA to learn DNM is summarized in Algorithm 1.

4.   Experimental results

4.1.   Experimental setup

To comprehensively study of learning performance of SSWOA on DNM, a number of representative optimization techniques have been compared, which are summarized within three categories: the variants of SSWOA (including its components), some other evolutionary algorithms, and some other machine learning techniques. These include the latest variant of the WOA algorithm, i.e., the spatial bound whale optimization algorithm (SBWOA) ^[75] proposed in 2021, the cellular automata-based whale optimization algorithm (CAWOA) ^[76] proposed in 2020, the nonlinear adaptive weight and golden sine operator-based whale optimization algorithm (NGS-WOA) ^[77] proposed in 2020, the hybrid improved whale optimization algorithm (HIWOA) ^[78] proposed in 2019, IFDE ^[71] proposed in 2021, which is currently regarded as the most effective method for training DNM, BBO ^[50], SMS ^[67], the most effective version of SS, i.e., SASS ^[63], BP, K-nearest neighbor (KNN), and support vector machines (SVM). All experiments were conducted using MATLAB (R2019b) and implemented on a machine with a 32 GB RAM and Intel(R) Core i7-10700K 5.1 GHz processor. To better verify the difference between the hybrid SSWOA algorithm and its peer, we conduct the experiment from two aspects, namely the benchmark function in IEEE CEC2017 ^[79], and the artificial dendritic neural model training problem (DNM). The following evaluation tools were used to assess the performance of SSWOA.

4.2.   Verifying the performance of SSWOA on IEEE CEC2017

To assess the performance of the SSWOA approach, single-peaked functions (F1–F3), basic multi-peaked functions (F4–F10), hybrid functions (F11–F20), and combination functions (F21–F30) taken from IEEE CEC2017 are used. The studies involved a comparison of SSWOA with IFDE, SS, SMS, BBO, WOA, CAWOA, HIWOA, NGSWOA, and SBWOA. Instead, the commonly used parameters are set as follows: the number of function evaluations with a population size of 100, the maximum number of function evaluations of $10^{4} \times D$, where $D$ is the number of dimensions. Each algorithm to obtain statistics was executed 51 times for each function separately.

1) $W/T/L$: $T$ represents the amount of functions where SSWOA performs similarly to other algorithms. $L$ represents the amount of functions for which SSWOA performs noticeably worse. $W$ represents the amount of functions for which SSWOA surpasses the other algorithms.

2) The convergence curve represents the most recent optimum history recorded at each repeat. The function evaluation numbers are shown on the $x$-axis, while the average error score is shown on the $y$-axis.

3) The box-and-whisker graph includes boxes and peaks. Maximum values are indicated by the line above the blue box, and minimum values are indicated by the line below the blue box. The box's upper and lower edges correspond to the first and third quartiles, respectively. The red "+" sign represents the extreme extremes, while the red line represents the median. Furthermore, the greater the discrepancy between the highest and minimum numbers, the more unstable the method's performance.

4.3.   Verifying the performance of SSWOA for learning DNM

Additionally, 11 frequently used classification problems from the UCI Machine Learning Repository were used to verify the performance of the proposed SSWOA for learning DNM, including the BreastEW, Australia, CongressEW, Exactly, Heart, German, KrVsKpEW, Ionosphere, SpectEW, TIC-tac-toe, and Vote datasets. Additionally, the number of attributes (features other than class labels), the total sample size and the learning space dimension of the DNM for each problem (i.e., $D$) are summarized in Table 1.

Samples including missing values were removed for a fair comparison or because of the classifier used. The samples for each dataset were randomly divided. On the same computer, each method for these comparisons was ran 30 times separately with the parameters set to recommended levels in the relevant literature ^[80]. The amount of function evaluations for each of the algorithms under comparison was made to be the same in order to provide a generally fair performance comparison, namely, $MaxFES = 3000$. 70% of the samples were used for training, with the remaining 30% as test samples.

First, we normalize all the values to prevent small numerical attributes from being dominated by large numerical attributes. Therefore, the variables for which input is performed are normalized. Specifically, all attributes are normalized to the range $[0, 1]$ based on a min-max normalization rule. The equation can be expressed as follows:

Furthermore, since three hyper-parameters of DNM need to be predetermined, Taguchi's orthogonal experimental design technique was used to determine the ideal arrangement of hyper-parameter settings. This contained dendrite number $M$, firing parameter $Q$, and distance parameter $K$ ^[50,81]. As Taguchi's method is not a full factor analysis, but tests only partial orthogonal combinations between factors and levels, with the promise of minimal experimental testing and the most accurate statistical calculations of the implementation-related parameters. In DNM, each factor (i.e., hyper-parameter) was set to five levels as the following: $M \in \{1, 3, 5, 10, 20\}$, $Q\in \{0.1, 0.3, 0.5, 0.7, 0.9\}$, and $K \in \{1, 5, 10, 15, 20\}$. In order to completely find the most promising set of parameters, a thorough factor analysis would thus need $5^{3} = 125$ trials, which is clearly time-consuming. Taguchi technique can better go about this task of counting by using only an orthogonal array $L_{25}(53)$ containing 25 combinations, as shown in Table 2. Table 3 shows the optimal settings of the three hyper-parameters for each classification problem.

In our studies, we employed a wide range of performance criteria in order to further precisely assess our model. True positives (TP) are cases that were both expected to be positive and were in fact positive. The term "true negatives" (TN) refers to cases that are both predicted and actual negative. False positive instances (FP) are those that are projected to be positive but are actually negative. False negatives (FN) are instances where a negative forecast was made but the result was actually positive. Accuracy (Acc), which is calculated using the formula below, is the proportion of accurate predictions to all samples.

The true positive rate is defined as the ratio of genuine positive cases in the positive class predicted by the classifier to all positive instances (TPR), also known as sensitivity. This ratio is determined by the following formula:

Rate of False Positives (FPR) FPR is the ratio of real negative instances to all negative instances in the positive class that the classifier predicted. The following formula yields:

The true negative rate (TNR), also known as specificity, is the ratio of genuine negative cases predicted by the classifier to all actual negative instances. It is calculated using the following formula:

The proportion of individuals who are labeled as negative but are actually positive is known as the false negative rate (FNR). The following formula yields:

The bigger the FPR, the more negative categories are really in the expected positive categories (horizontal axis of ROC FPR: 1-TNR, 1-Specificity). TPR sensitivity (positive category coverage), the higher the TPR, the more positive categories are predicted in the positive category. The ideal goal with $TPR = 1$ and $FPR = 0$, which corresponds to the $(0, 1)$ point on the graph; hence, the closer the ROC curve is to the $(0, 1)$ point, the larger the divergence from the 45-degree diagonal, and the higher the sensitivity and specificity values. As the ROC curve is generally above the straight line $y = x$, its value is between 0.5 and 1. The reason for using AUC as an evaluation metric is that the ROC curve usually does not give a clear indication of which classifier is more effective. It is given by the following equation.

5.   Performance comparison

5.1.   Algorithm performance comparison

Tables 4–6 summarize the classification findings for the mean test error and standard deviation for all comparison methods (std). We evaluated and compared several methods in dimensions 30, 50, and 100 at IEEE CEC2017, as well as examined the performance change of SSWOA from medium to large dimensions. It is important to note that F2 was left in place despite the function's unreliable performance in CEC2017, particularly at high dimensions due to the guaranteed data integrity. Compared to IFDE, SS, SMS, BBO, WOA, CAWOA, HIWOA, NGSWOA, and SBWOA, the number of wins for SSWOA in Table 4 is 4, 19, 27, 21, 29, 29, 29, 29, 29, 29, 29, 29, in Table 5 are 4, 17, 24, 17, 28, 28, 29, 28, 28, 28, and in Table 6 are 4, 25, 27, 16, 27, 28, 27, 27, 27, 27, respectively. It is shown that SSWOA outperforms all algorithms except IFDE in the 30, 50 and 100 dimensions of CEC2017, indicating that SSWOA performs well in the medium, high and large dimensions of the function.

Figures 7 and 8 provide the experimental findings to further demonstrate the advantages of SSWOA. It is clear from the convergence plots that SSWOA's convergence ability and speed are superior to those of comparable methods, with the exception of IFDE. It is evident from Figure 7 that the SSWOA convergence curve in F20 at $D = 30$ is very different from other algorithms.

To demonstrate the benefits of SSWOA's performance in training DNM models for classification, several comparison algorithms were adopted simultaneously to train DNM models and classify datasets. The comparison results are presented in Table 7, where the criteria for comparison contain accuracy and standard deviation, and the data of optimal classification results is shown in bold. Although some are inferior to the non-algorithmic categories in terms of results, four results stand out in comparison to the algorithm and outperform the other algorithms. This also demonstrates the effectiveness and applicability of the SSWOA algorithm.

To better verify whether the hybrid algorithm SSWOA is the optimal choice for learning DNM, the Wilcoxon symbolic ranking test ^[82] was conducted to identify the differences between each set of data, and to determine if there were any significant differences between each pair of learning algorithms, and to figure out whether the differences between algorithms were at the significance level of $\alpha = 0.05$. The $p$-values obtained, which are all below than the significance threshold, are included in Table 8 as well. Taking Tables 7 and 8 into account, it can prove that the hybrid algorithm SSWOA outperforms other comparative algorithms in terms of classification accuracy.

5.2.   Algorithm training model performance comparison

In order to precisely visualize the performance of SSWOA, we decided to employ two different graphs, namely the box-and-whisker plots graphs (Figures 9 and 10) and the convergence graph, i.e., Figure 11. Figures 9 and 10 depict two typical problems, namely the CongressEW, Exactly. This box-and-line diagram includes two evaluation metrics, MSE and accuracy. The box-and-whisker plot includes the maximum, minimum, first quartile, and median. Figure 10 summarizes the maximum, minimum, first quartile, median, third quartile and extreme values. The distribution of solutions shown in Figure 10 indicates that the difference between the lowest and highest numbers shows how consistently the algorithm performs. Additionally, the reduced height in terms of MSE suggests that the algorithm might provide a superior answer during training.

SSWOA has the best median and lowest values for the Exactly and CongressEW issues, as can be observed in Figures 9 and 10, which indicates that SSWOA exhibits the optimal effect in training DNM. It is inevitable that some of the evolutionary algorithms may experience problems with excesses in the training process. Nevertheless, SSWOA presents the best classification accuracy for all problems tested compared to its peers.

Figure 11 introduces the convergence plots for all compared algorithms. It can be noticed from the graphs that SSWOA has a reasonable convergence rate compared to other learning algorithms. Therefore, combining Figures 10–11 and the time displayed by Table 9, we can conclude that SSWOA is a hybrid algorithm that combines high classification accuracy, good robustness and high convergence. SSWOA is exactly the DNM's learning system that shows the greatest promise.

In addition, we compared the ROC curves and AUC values of the models trained by both evolutionary and non-evolutionary algorithms, as shown in Figure 12. From the Congress dataset we can observe that the AUC values of SSWOA are slightly larger than those of the evolutionary algorithms IFDE, SASS, which is much better than WOA and its variants. Also, in evolutionary algorithms, the AUC values of SSWOA are also slightly larger than those of non-evolutionary algorithms. This is even more evident in the dataset Exactly, where SSWOA has a larger computed area under the ROC curve than other evolutionary and non-evolutionary algorithms. This is also true for other important metrics. Based on these experimental results, we can easily conclude that the hybrid algorithm has better classification performance than other single algorithms. To a certain extent, SSWOA improves the classification accuracy, precision and has good robustness and high convergence on the basis of one single algorithm.

5.3.   Morphology and logical circuits realization

In training the DNM with the evolutionary algorithm, it is important to determine the appropriate values of $w_{ij}$ and $q_{ij}$. As described in Section 2, $w_{ij}$ represents the state that the synapse is in, i.e., excitation or inhibition, while $q_{ij}$ represents a threshold value regarding the presence or absence of the synapse. Then, in Figure 13, we represent synaptic weights obtained from the DNM model of synapses on both Exactly and CongressEW datasets, suggesting that dendrites impose a strong influence on excitation types and synaptic plasticity.

These state transitions are based on mechanisms previously used in morphology and include here four types of connected states, i.e., direct, inverse, constant 1 and constant 0. Once the pruning operation is complete, this morphological representation can be converted to a logic circuit on the hardware. The ultimate morphology may be acquired after dendritic and synaptic pruning. The precise information is shown in Figure 14. A complex and large unpruned morphology can be seen in the Exactly dataset, whereas only four dendritic layers were retained after pruning and up to seven attributes in the dataset were also removed from the data by the learned DNM, with only $X1$–$X5$, $X7$, $X9$ and $X11$ needing to be involved in information processing. Figure 15 shows the neuromorphology of the DNM on the CongressEW dataset. Attribute $X1$ has been removed, which is functionally equivalent to a feature extraction, and more than half of the attributes have been removed. This drastic simplification produces a short and clear logic circuit.

It is also easy and convenient to implement inside hardware. It is important to note that typical neural networks take a significant amount of design work to build their complete models, whether they are implemented digitally, analogously, or on FPGA-based circuits ^[83,84], Even silicon photonic chips are required for the hardware implementation of some particular neural networks ^[85]. Compared to these efforts, the hardware implementation of DNMs is relatively simple, as only logic circuits are utilized, which brings DNMs greater potential and broader prospects in real-world applications.

6.   Conclusions and future works

A hybrid algorithm (SSWOA) was innovatively proposed in the paper to train the single dendritic neuron model. SSWOA was constructed by fusing the benefits of the SS algorithm and the WOA approach. The SS algorithm was employed to roughly determine the direction of the best option, followed by the advancement capabilities of the whale optimization algorithm were used to find the optimal solution, thus improving the capability of the search algorithm. The specific solution is to use SS to exploit exploration in the direction-Rand and WOA to exploit in the other portion towards-best, allowing faster convergence in the best region.

Twelve common classification tasks were used in the experiments, and the findings show that SSWOA can perform much better than BP and ten other non-BP advanced learning methods. The findings also verified that the performance of the hybrid algorithm can be further enhanced when learning single dendritic neuron models.

However, the proposed SSWOA has the following limitations: 1) it is less impressive in terms of time consumption and outcomes compared to other machine learning techniques, 2) the DNM learned by SSWOA still suffers from the curse of dimensionality, thus limiting its applicability on large-scale applications, e.g., image recognition of ImageNet, videos information processing, etc.

The future work of this study include: 1) There is no one global optimization procedure that is the optimal for all optimization problems, according to the renowned "no free lunch" theory. In training DNMs, a variety of heuristic optimization strategies have shown good results, although they still have drawbacks. It is still unclear whether we need to take into account the effectiveness of other heuristic optimization methods, e.g., the farmland fertility algorithm ^[86,87,88], African vultures optimization algorithm ^[89], mountain gazelle optimizer ^[90], artificial gorilla troops optimizer ^[91], that co-evolve to ultimately improve the DNM.

2) DNM can identify neuromorphology for a particular task, implying that for a neuron with organized dendrites, it is possible for DNM to determine the amount of dendrites required, the location of the presynaptic axon terminals connecting the dendrites, and the nature of the connections, i.e., excitatory or inhibitory. If we can utilize this local spatial information for visual orientation detection, as Taylor ^[92] suggests, we will have a better understanding of how a neuron is structured when executing a particular task.

3) Given the spatial dendritic structure, the promising single-neuron model DNM should be able to serve as the foundation for multi-layer deep learning models, enhancing its capacity to resolve challenging real-world problems.

4) More bio-experimental data, such as auditory coincidence detection data, may be obtained with DNM since it can conduct neural morphology detection. The exact rules and methods that biologists and neurologists may utilize as teacher signals while examining neurons, synapses, and dendrites.

Acknowledgments

This research was partially supported by the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (Nos. 19KJB520059, 19KJB520058, 19KJB520057), and the project of Talent Development of Taizhou University (No. QD2016061).

Conflict of interest

The authors declare there is no conflict of interest.