
McCulloch-Pitts neuron-based neural networks have been the mainstream deep learning methods, achieving breakthrough in various real-world applications. However, McCulloch-Pitts neuron is also under longtime criticism of being overly simplistic. To alleviate this issue, the dendritic neuron model (DNM), which employs non-linear information processing capabilities of dendrites, has been widely used for prediction and classification tasks. In this study, we innovatively propose a hybrid approach to co-evolve DNM in contrast to back propagation (BP) techniques, which are sensitive to initial circumstances and readily fall into local minima. The whale optimization algorithm is improved by spherical search learning to perform co-evolution through dynamic hybridizing. Eleven classification datasets were selected from the well-known UCI Machine Learning Repository. Its efficiency in our model was verified by statistical analysis of convergence speed and Wilcoxon sign-rank tests, with receiver operating characteristic curves and the calculation of area under the curve. In terms of classification accuracy, the proposed co-evolution method beats 10 existing cutting-edge non-BP methods and BP, suggesting that well-learned DNMs are computationally significantly more potent than conventional McCulloch-Pitts types and can be employed as the building blocks for the next-generation deep learning methods.
Citation: Hang Yu, Jiarui Shi, Jin Qian, Shi Wang, Sheng Li. Single dendritic neural classification with an effective spherical search-based whale learning algorithm[J]. Mathematical Biosciences and Engineering, 2023, 20(4): 7594-7632. doi: 10.3934/mbe.2023328
[1] | Jing Zhang, Haoliang Zhang, Ding Lang, Yuguang Xu, Hong-an Li, Xuewen Li . Research on rainy day traffic sign recognition algorithm based on PMRNet. Mathematical Biosciences and Engineering, 2023, 20(7): 12240-12262. doi: 10.3934/mbe.2023545 |
[2] | Sung Woong Cho, Sunwoo Hwang, Hyung Ju Hwang . The monotone traveling wave solution of a bistable three-species competition system via unconstrained neural networks. Mathematical Biosciences and Engineering, 2023, 20(4): 7154-7170. doi: 10.3934/mbe.2023309 |
[3] | Shuai Cao, Biao Song . Visual attentional-driven deep learning method for flower recognition. Mathematical Biosciences and Engineering, 2021, 18(3): 1981-1991. doi: 10.3934/mbe.2021103 |
[4] | Weibin Jiang, Xuelin Ye, Ruiqi Chen, Feng Su, Mengru Lin, Yuhanxiao Ma, Yanxiang Zhu, Shizhen Huang . Wearable on-device deep learning system for hand gesture recognition based on FPGA accelerator. Mathematical Biosciences and Engineering, 2021, 18(1): 132-153. doi: 10.3934/mbe.2021007 |
[5] | Boyang Wang, Wenyu Zhang . ACRnet: Adaptive Cross-transfer Residual neural network for chest X-ray images discrimination of the cardiothoracic diseases. Mathematical Biosciences and Engineering, 2022, 19(7): 6841-6859. doi: 10.3934/mbe.2022322 |
[6] | Xiao Ma, Xuemei Luo . Finger vein recognition method based on ant colony optimization and improved EfficientNetV2. Mathematical Biosciences and Engineering, 2023, 20(6): 11081-11100. doi: 10.3934/mbe.2023490 |
[7] | Qingwei Wang, Xiaolong Zhang, Xiaofeng Li . Facial feature point recognition method for human motion image using GNN. Mathematical Biosciences and Engineering, 2022, 19(4): 3803-3819. doi: 10.3934/mbe.2022175 |
[8] | Ting Yao, Farong Gao, Qizhong Zhang, Yuliang Ma . Multi-feature gait recognition with DNN based on sEMG signals. Mathematical Biosciences and Engineering, 2021, 18(4): 3521-3542. doi: 10.3934/mbe.2021177 |
[9] | Jing Wang, Jiaohua Qin, Xuyu Xiang, Yun Tan, Nan Pan . CAPTCHA recognition based on deep convolutional neural network. Mathematical Biosciences and Engineering, 2019, 16(5): 5851-5861. doi: 10.3934/mbe.2019292 |
[10] | Jinhua Zeng, Xiulian Qiu, Shaopei Shi . Image processing effects on the deep face recognition system. Mathematical Biosciences and Engineering, 2021, 18(2): 1187-1200. doi: 10.3934/mbe.2021064 |
McCulloch-Pitts neuron-based neural networks have been the mainstream deep learning methods, achieving breakthrough in various real-world applications. However, McCulloch-Pitts neuron is also under longtime criticism of being overly simplistic. To alleviate this issue, the dendritic neuron model (DNM), which employs non-linear information processing capabilities of dendrites, has been widely used for prediction and classification tasks. In this study, we innovatively propose a hybrid approach to co-evolve DNM in contrast to back propagation (BP) techniques, which are sensitive to initial circumstances and readily fall into local minima. The whale optimization algorithm is improved by spherical search learning to perform co-evolution through dynamic hybridizing. Eleven classification datasets were selected from the well-known UCI Machine Learning Repository. Its efficiency in our model was verified by statistical analysis of convergence speed and Wilcoxon sign-rank tests, with receiver operating characteristic curves and the calculation of area under the curve. In terms of classification accuracy, the proposed co-evolution method beats 10 existing cutting-edge non-BP methods and BP, suggesting that well-learned DNMs are computationally significantly more potent than conventional McCulloch-Pitts types and can be employed as the building blocks for the next-generation deep learning methods.
Traffic signs give drivers instructions and information that are indispensable in protecting human life and property. Therefore, automatic traffic sign recognition is an important computer vision task [1,2,3]. In recent years, driver assistance systems (DAS) [4,5,6,7] have developed rapidly. Traffic sign recognition is an integrated function of DAS. Normally, professional equipment is mounted on the top of a vehicle to capture traffic sign images. Under real-world conditions, these images are distorted due to various natural and human factors including vehicle speed, weather, destroyed signs, angles, etc. Hence, we apply data augmentation to simulate all kinds of situations. These enhancement techniques refer to rotations, crop, scale, lighting variations and weather conditions changes, which may eventually decrease the traffic sign recognition network performance.
As for the field of traffic sign recognition, it is a famous problem of computer vision tasks. A lot of literature [8,9,10] has studied the topic of traffic sign detection, and some reviews are available in [11,12]. In [13], an algorithm based on Gaussian kernel support vector machines (SVMs) [14] is utilized for traffic sign classification. From experimental results, the proposed algorithm is robust under various conditions including translation, rotation and scale. Ayoub [15] presented a random forest classifier and gave satisfactory results on the Swedish Traffic Signs Dataset. Lu [16] proposed a graph embedding approach that preserved the sparse representation property by using L2, 1-norm. Experiments demonstrate that the proposed approach outperformed previous traffic sign recognition approaches.
In this paper, we introduce the interference module inspired by [17], which dynamically aggregates information by improving the representation way of information according to different semantic contents of an image. In quantum mechanics, a wave function containing both amplitude and phase [18] represents an entity. We describe each entity that is generated by the convolutional filters as a wave to realize the information aggregation procedure dynamically. The amplitude is the real-value feature representing the content of each entity, while the phase term is represented as a complex value. These wave-like entities intervene with each other and close phases tend to enhance each other. The whole framework is constructed by stacking the interference module and channel-mixing multi-layer perceptrons (MLPs). Figure 1 summarizes the architecture. We further conduct ablation experiments and analyze the performance of the proposed model. The final experimental results demonstrate the visible priority of the existing architectures (shown in Table 2).
The contributions of our work are as follows: we propose a novel method (WiNet) for traffic sign recognition. A new convolution structure is introduced for learning multi-scale features and it is successfully combined with a wave function. Additionally, our model achieves the most performance compared with several concurrent works based on ResMLP [19], ResNet50, PVT [20] and ViT [21]. We also test the robustness of all models on datasets with a sufficient number of synthetic samples. Furthermore, we analyze the effects of type of representation on the overall wave interference network that dynamically aggregates information by improving the representation of them according to different semantic contents of an image.
The rest of the paper is organized as follows: Section 2 reviews related methods applied to traffic sign recognition. Section 3 introduces the formulation and architecture of the proposed model. We present experiments and implementation details in Section 4, and further analyze the effectiveness of different modules and activation functions in Section 5. Finally, we draw conclusions in Section 6.
In recent decades, many methods have been proposed for traffic sign recognition. We briefly review related literature [22,23]. It can be divided into traditional methods, methods based convolutional neural networks (CNNs) and methods with attention mechanisms.
Traditional methods for traffic sign recognition: Before CNNs, traffic sign recognition depended on hand-crafted features. In [15], the comparison of different combinations of four features was conducted, including the histogram of oriented gradients (HOG), Gabor, local binary pattern (LBP) and local self-similarity (LSS). The authors tested the proposed method on the Swedish Traffic Signs Data set. Machine learning methods have also been utilized to solve related problems, such as random forests, logistic regression [24] and SVM [25].
CNN for traffic sign recognition: With the rapid development of memory and computation, CNN-based architectures have been the mainstream in the computer vision field. Many works have used CNNs for traffic sign classification. The committee of the CNN-based approach [26] obtains a high recognition rate of 99.15%, which is above the human recognition rate of 98.98%. In the GTSRB competition in 2011, multi-scale CNNs [27] made full use of local and global features and established a new record of an error rate of 1.03%. [28] proposes a novel deep network for traffic sign classification that achieves outstanding performance on GTSRB. The author utilized spatial transformer layers [29] and a modified version of the inception module [30] to build the model. The well-designed inception module allows the network to classify intraclass samples precisely. The spatial transformer layer improves the robustness of the network to deformations such as translation, rotation and scaling of input images.
Attention mechanism for traffic sign recognition: The attention mechanism can adapt to select important information from the input feature. For the outstanding performance, attention mechanism and variants have been applied to a variety of tasks. These improvements include channel attention [31], spatial attention [32] and global attention [33]. This paper [34] proposed an attention-based convolutional pooling neural network (ACPNN). Convolutional pooling replaces max pooling. The ACPNN was validated on the German Traffic Sign Recognition Benchmark (GTSRB). Experiments showed it was robust against external noises and increased recognition accuracy. Based on the ice environment traffic sign recognition benchmark (ITSRB), the author proposed an attention network based on high-resolution traffic sign classification (PFANet) [35] and reached 93.57% accuracy. At the same time, its performance on the GTSRB was as good as the newest and most effective networks.
In this section, we describe our proposed network for traffic-sign recognition in detail. First, we describe the interference module. Then, we describe the whole structure of our proposed network. At last, we describe our data augmentation techniques.
In recent years, many neural network modules based on CNN have been proposed such as [36,37]. In the interference module, the extraction structure of features based on convolution adopts residual-like connections within a single residual block [38]. After the input is normalized, it is split into two subsets which have the same spatial size and number of channels, denoted by X1 and X2, respectively. Each Xi corresponds to a 3×3 convolution, denoted by Ki(), where i∈{1,2}. X2 is added with the output of K1() and then fed into K2(). At last, a 1×1 convolution makes sure the channel number is the same as that of the following module. The output result can be calculated using the following formula:
Ei=Conv1×1(K2(K1(X1)+X2)),i=1,2,...,n. | (1) |
We name the output Ei. The final output Ei can capture feature information from a larger receptive field. Different channels often contain the complete content of different objects in deep neural networks [39]. By using a 1×1 convolution, different channels are assigned to corresponding weights. It is worth noting that the feature extraction structure not only achieves adaptability in the spatial dimension [36] but also adaptability in the channel [36] dimension.
The output Ei is the feature map extracted by the convolutional filters. To describe the feature map concretely, we refer to each feature as an entity. Next, each entity will be converted into a wave. After all, waves have aggregated each other, the new feature map will be generated and fed into the next stage.
An entity is represented as a wave Bj with both amplitude and phase information. A wave can be formulated by
Bj=|Aj|⨂eiθj,j=1,2,...,n,(2) |
where Bj returns the j-th wave, i is the imaginary unit and i2=−1. The |.| denotes the absolute value operation. ⊗ means element-wise multiplication. The amplitude |Aj| represents the information of each entity. A periodic function of the eiθj makes its values distribute over a fixed range. θj is the phase and points out the current location of an entity in a period.
In Eq (2), a wave is represented a complex-value. To embed it in the module, we expand it with Euler's formula. It can be written as
Bj=|Aj|⨂cosθj+i|Aj|⊗sinθj,j=1,2,...,n.(3) |
To get the above equation, we are required to design the form of information expression about both amplitude and phase. The amplitude |Aj| is a real-value feature in the formula. In fact, the absolute operation in the formula is not implemented. A feature map xj=RN×C is taken as the input, and we use a plain channel_FC operation [17] to estimate the amplitude. Specifically, a Tanh activation is adopted to gain the nonlinearity ability. Compared with Relu activation, we found Tanh activation achieves significantly better performance. As can be seen from Figure 3, a wave with a phase has a direction. The value range of Tanh activation is -1–1. Positive and negative numbers can be used to represent different directions. So Tanh activation can help models achieve better performance. Comparisons of Tanh activation with Relu activation are shown in Table 6.
In the above paragraphs, we presented the relevant mathematical expressions. Next, we introduce a concrete evaluation expression about the phase and the result of two waves superimposed.
ChannelFC(xj,Wc)=Wcxj,j=1,2,...,n, | (4) |
where the learnable parameter Wc is the weight. The phase plays an important role in the whole module. It points out the current location of an entity in a period. We take the simplest method to estimate the phase whose parameters are represented with the output value from Tanh activation.
To dynamically adjust the relationship between different entities with fixed parameters, we take the token_FC operation [17] to aggregate information. The token-FC is formulated as:
Token_FC(xj,Wc)=∑KWtjkxk,j=1,2,...,n, | (5) |
where the learnable parameter Wt is the weight. A feature map xk∈RN×C is taken as the input. Here, j means the j-th entity. Finally, the real-value output Oj can be written as:
Oj=∑kWtjkxkcosθk+Wijkxksinθk,j=1,2,...,n, | (6) |
where the learnable parameter Wt and Wi are the weights. As can be seen from Figure 3, when two waves have a similar phase, the output Oj tends to be enhanced. The same semantic content of the input feature map can be extracted in Figure 7 of Section 4.
In the channel dimension, the 1×1 convolution is conducive to fusing information among different channels. To enhance the information fusion ability in the spatial dimension, we use MLPs to exchange information among different entities. Before MLPs, we also apply batch normalization. We adopt residual learning to fusion information from MLPs. As Table 6 illustrates, MLPs significantly improve the performance compared with the network structure not containing MLPs.
In terms of the network architecture, our model is a simple hierarchical structure with 4 stages. Each stage will decrease the output spatial resolution, i.e., H/2×W/2, H/4×W/4, H/8×W/8 and H/16×W/16. Here, H and W represent the height and width of the input image. The number of output channels is increasing with the decrease of resolution. The detailed configuration can be seen in Table 1.
Stage | Output size | Blocks |
1 | H/2×W/2×64 | 2 |
2 | H/4×W/4×128 | 3 |
3 | H/8×W/8×256 | 4 |
4 | H/16×W/16×512 | 2 |
At the beginning of each stage, we downsample the input and control the downsampling rate by using the stride number. In the first stage, we use a 6×6 convolution with stride 2 to embed an input image with the shape H×W×3. In the following three stages, there is a 3×3 convolution with stride 2 to downsample the input data. Note that all other layers in a stage keep the same output size. We use a global average pooling operation for the feature map from the last stage and then adopt a linear classifier to predict the logits.
In general, it is very difficult to estimate which model gives better performance. Many authors evaluate their models on different datasets. To enrich the set of traffic signs, some authors sample images from multiple datasets to perform the evaluation [40,41,42]. On the other hand, lots of authors use their own private datasets [43,44] instead of public datasets [45,46]. In order to be fair, we adopt the ratio of the training set and test set in the CTSRD database that ensures different models are implemented at the same benchmark. We carried out several data augmentation techniques to extend the training set for addressing the challenges from various realistic scenarios.
To demonstrate the generalization of our model, we train our model on another well-known benchmark. The German Traffic Sign Recognition Benchmark (GTSRB) [47] consists of more than 50,000 images in which the classes of traffic signs are more than 40, but we consider only images with a size of at least 30 pixels. All images are resized to 64×64 resolution. The remaining images are ignored on account of the low human visual identification rate. Many models [48,49,50,51] have obtained good results on this dataset, so the comparison between different models is more convincing.
Data augmentation has shown its validity in deep networks. It effectively expands the size of the training set, which is an important factor to consider when training models. A sufficient amount of training data contributes to modulating millions of learnable parameters during the training phase. In real-world situations, traffic signs may be distorted in shape because of human factors. Traffic sign images may contain various appearance distortions such as brightness and contrast. In order to simulate various physical changes in real-world scenarios, we apply image processing techniques to each image randomly and expand the original training dataset as large as five times. Every image is only augmented by one of the available methods in processing, which makes sure that each augmentation operation has the same probability of being implemented. Some samples of original and sample results from these techniques are presented in Figure 4. There are many imaging processing libraries. We apply Albumentations, except for rain, to complete all options. To recapitulate briefly, we perform four classes of augmentations in the data preprocessing stage:
● Random weather. We apply data augmentation to achieve weather effects such as sunny, foggy, rainy and snowy. Synthetic samples keep the size and aspect ratio of input images.
● Random blur. We apply data augmentation to blurred original images. Methods of data augmentation include Blur, MotionBlur, GaussianBlur and MedianBlur in Albumentations.
● Random affine. Geometric transformations for data augmentation are common and effective methods. PiecewiseAffine, Affine and RandomResizedCrop in Albumentations are selected to generate synthetic samples.
● Gause noise. Gauss Noise in Albumentations generates a matrix of random values. Moreno-Barea [52] found adding noise to images can make models more robust on nine datasets from the UCI repository [53].
In this section, we conduct a few analytical experiments to explore our proposed model. To make the comparison fair, we carried out all experiments under the same settings. We report our results on CTSRD, which achieves significant improvements compared with other state-of-the-art models. In addition, we verify the effectiveness of Tanh activation and MLPs. Result visualizations further show the effectiveness of our model.
We implemented our experiments on a PC with an Intel i5-11400H, an NVIDIA GeForce RTX 3050 GPU, and 16 GB RAM. All models are carried out with TensorFlow. During training, we ensure all models use the same parameters for fairness. We train all models using Adam with the same initial learning rate of 10−4. Due to the limitation of the video card, the mini-batch size is set to 64. After considering the resolution of input images, the image is resized to 64 pixels in both width and height. To fuse spatial dimension information, we set the window size to 3 empirically in the interference module. All models are trained for 60 epochs on the original dataset. We train again all models for 30 epochs when using data augmentation and greatly increasing the size of the original dataset.
Table 2 presents the comparison of WiNet with other models, including ResMLP, ResNet50, PVT and ViT. WiNet achieves the best accuracy, which outperforms ResMLP, ResNet50, PVT and ViT by 5.8%, 2.4%, 23.5% and 11%, respectively. This clearly demonstrates that our model has a strong ability to extract features. In comparison to ResNet50 with fewer parameters, our WiNet not only has larger throughputs but also performs better. In particular, WiNet achieves 100% accuracy when images using data augmentation were added to the training set.
Model | Params | Throughput (image/s) | Original dataset | Original dataset including synthetic examples |
Top-1 Accuracy (%) | Top-1 Accuracy (%) | |||
ResMLP | 14.52 M | 13 | 94.0 | 97.2 |
ResNet50 | 23.7 M | 19 | 97.4 | 99.7 |
PVT | 12.3 M | 13 | 76.3 | 89.5 |
ViT | 85.8 M | 11 | 88.8 | 95.7 |
ours | 26.1 M | 14 | 99.8 | 100 |
We give quality histograms in Figure 5, while corresponding statistics (median, mean and standard deviation of all models) are provided in Table 3. It can be seen that the distribution of predicted probability for our model is closer to 1 than for other models trained on CTSRD simultaneously. Statistics further verify the results that WiNet has a higher median value, mean value and a smaller discrete degree.
Model | Median | Mean | Std. Dev. |
ResMLP | 0.99 | 0.96 | 0.109 |
ResNet50 | 1.00 | 0.99 | 0.047 |
PVT | 0.95 | 0.85 | 0.184 |
ViT | 0.99 | 0.94 | 0.121 |
ours | 1.00 | 0.99 | 0.034 |
In order to further analyze the model's performance, we utilize data augmentation including weather, Gauss, blur and affine on the testing set, and then make predictions. Data augmentation has made the distribution of the testing set different from the training set, so it can be used to test the model's robustness. Table 4 lists the predicted results of the model on the testing set. In the above case, our model shows good performance. We use models, which are trained on the training set with synthetic samples, to predict this testing set. As can be seen from Table 5, the performance of WiNet surpasses ResNet50 and has a sharp boost with +2.71, +3.52, +2.21 and +2.76 points for weather, Gauss, blur and affine, respectively. It indicates that our model's effectiveness can be fully exploited on a larger dataset.
Model | Weather | Gauss | Blur | Affine |
ResMLP | 44.08% | 53.01% | 51.86% | 54.71% |
ResNet50 | 70.26% | 65.05% | 64.09% | 65.75% |
PVT | 35.66% | 32.25% | 32.80% | 37.46% |
ViT | 48.14% | 47.34% | 47.09% | 48.35% |
ours | 73.37% | 61.48% | 60.48% | 62.64% |
Model | Weather | Gauss | Blur | Affie |
ResNet50 | 91.22% | 91.57% | 90.97% | 91.27% |
ours | 93.93% | 95.09% | 93.18% | 94.03% |
Except for considering the accuracy of our model, it is crucial to test the generalization of our model by training on different datasets. We divided the dataset from GTSRB into training and validating datasets according to the original proportion. We trained the model with 30 epochs, batch size 64, learning rate 0.0001 and Adam optimizer. The solid curves show the accuracy change, while the dashed curves show the loss change. Here, we can see from Figure 6 that the highest accuracies of both training (21,792 images) and validating datasets (6893 images) are 1.00. The overall trends of accuracies for both training and validating datasets are increasing with epochs. After around 16 epochs, only slight fluctuations can be observed. We observed similar but opposite trends in the loss-changing curves. The losses of both training and testing datasets are very small. In these curves, we can see our proposed model has the advantages of minor loss and high accuracy. This result indicates that our model keeps a good generalization ability rather than only fitting CTSRD.
In this section, we ablate some design components to explore their effectiveness. For a more intuitive feeling, we draw some feature maps with heatmaps in the aggregating process.
To evaluate the performance of design components, we take WiNet with Tanh activation and MLPs as the baseline model. WiNet-Relu is built by replacing Tanh activation with Relu activation on the baseline model. WiNet-noMLP is equal to the baseline model removing MLPs. Ablation results are reported in Table 6. It is obvious that two components play an important part in learning effective parameters. Tanh activation improves the accuracy of WiNet by about 1.2% compared with WiNet-Relu. Without MLPs, the model's performance is down by at least 0.4% (99.8% vs 99.4%).
Model | WiNet-Relu | WiNet-noMLP | WiNet |
Top-1 Accuracy | 98.5% | 99.4% | 99.8% |
Diagrams are an important tool to help people intuitively understand the world. In Section 3.1 (Figure 3 and Eq (6)), we analyze the superposition of two waves with different phases. In order to have a better understanding of the effects of the representation type, we take the visualized feature maps of a traffic sign as an example. From Figure 7, we can clearly see that the visualized feature maps of the first stage with the similar contents are aggregated together. Similar parts in the picture have a closer phase reformulated. Similar parts gradually stick out in the aggregating process, so we get a strong stereo effect from the picture. As the number of network layers deepens, the model extracts more abstract features.
We propose a novel deep learning architecture for traffic sign recognition, whose mechanism for aggregating information is different from the existing transformer, CNN architectures or MLP architectures. In the proposed approach, we combine a new CNN-based module with a wave function successfully. Firstly, we get multi-scale feature representations by the CNN-based module. Then, we utilize channel-FC operations to estimate the amplitude and phase information. Amplitude and phase are key parameters to dynamically modulate relationship entities with similar contents. Extensive experimental evaluations are performed according to different strategies to explore the superiority of the proposed architecture or understand how that works. We will explore further how to use the information representation in different fields or directly use the information representation to preprocess raw data. We also hope our work can encourage people to get new ideas from physical phenomenon.
The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.
This work is jointly supported by the National Natural Science Foundation of China under Grant 61976055, the special fund for education and scientific research of Fujian Provincial Department of Finance under Grant GY-Z21001 and Scientific Research Foundation of Fujian University of Technology under Grant GY-Z22071.
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
[1] |
C. Lee, H. Hasegawa, S. Gao, Complex-valued neural networks: a comprehensive survey, IEEE/CAA J. Autom. Sin., 9 (2022), 1406–1426. https://doi.org/10.1109/JAS.2022.105743 doi: 10.1109/JAS.2022.105743
![]() |
[2] | Y. LeCun, Y. Bengio, G. Hinton, Deep learning, Nature, 521 (2015), 436–444. https://doi.org/10.1038/nature14539 |
[3] |
Z. Zhang, J. Geiger, J. Pohjalainen, A. E. D. Mousa, W. Jin, B. Schuller, Deep learning for environmentally robust speech recognition: an overview of recent developments, ACM Trans. Intell. Syst. Technol., 9 (2018), 1–28. https://doi.org/10.1145/3178115 doi: 10.1145/3178115
![]() |
[4] |
Y. Guo, Y. Liu, A. Oerlemans, S. Lao, S. Wu, M. S. Lew, Deep learning for visual understanding: a review, Neurocomputing, 187 (2016), 27–48. https://doi.org/10.1016/j.neucom.2015.09.116 doi: 10.1016/j.neucom.2015.09.116
![]() |
[5] |
D. Guo, M. Zhong, H. Ji, Y. Liu, R. Yang, A hybrid feature model and deep learning based fault diagnosis for unmanned aerial vehicle sensors, Neurocomputing, 319 (2018), 155–163. https://doi.org/10.1016/j.neucom.2018.08.046 doi: 10.1016/j.neucom.2018.08.046
![]() |
[6] |
J. Cheng, M. Ju, M. Zhou, C. Liu, S. Gao, A. Abusorrah, et al., A dynamic evolution method for autonomous vehicle groups in a highway scene, IEEE Internet Things J., 9 (2021), 1445–1457. https://doi.org/10.1109/JIOT.2021.3086832 doi: 10.1109/JIOT.2021.3086832
![]() |
[7] |
J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, et al., Highly accurate protein structure prediction with AlphaFold, Nature, 596 (2021), 583–589. https://doi.org/10.1038/s41586-021-03819-2 doi: 10.1038/s41586-021-03819-2
![]() |
[8] | I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, et al., Generative adversarial networks, arXiv preprint, (2014), arXiv: 1406.2661. https://doi.org/10.48550/arXiv.1406.2661 |
[9] | B. Zoph, Q. V. Le, Neural architecture search with reinforcement learning, arXiv preprint, (2016), arXiv: 1611.01578. https://doi.org/10.48550/arXiv.1611.01578 |
[10] |
S. J. Pan, Q. Yang, A survey on transfer learning, IEEE Trans. Knowl. Data Eng., 22 (2009), 1345–1359, https://doi.org/10.1109/TKDE.2009.191 doi: 10.1109/TKDE.2009.191
![]() |
[11] |
P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, et al., Advances and open problems in federated learning, Found. Trends Mach. Learn., 14 (2021), 1–210. https://doi.org/10.1561/2200000083 doi: 10.1561/2200000083
![]() |
[12] |
Y. Zhang, Q. Yang, A survey on multi-task learning, IEEE Trans. Knowl. Data Eng., 34 (2022), 5586–5609. http://doi.org/10.1109/TKDE.2021.3070203 doi: 10.1109/TKDE.2021.3070203
![]() |
[13] | A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, et al., Attention is all you need, Adv. Neural Inf. Process. Syst. (NIPS), 30 (2017), 5998–6008. |
[14] |
F. Han, J. Jiang, Q. H. Ling, B. Y. Su, A survey on metaheuristic optimization for random single-hidden layer feedforward neural network, Neurocomputing, 335 (2019), 261–273. https://doi.org/10.1016/j.neucom.2018.07.080 doi: 10.1016/j.neucom.2018.07.080
![]() |
[15] |
D. Yarotsky, Error bounds for approximations with deep ReLU networks, Neural Networks, 94 (2017), 103–114. http://doi.org/10.1016/j.neunet.2017.07.002 doi: 10.1016/j.neunet.2017.07.002
![]() |
[16] | S. Ruder, An overview of gradient descent optimization algorithms, arXiv preprint, (2016), arXiv: 1609.04747. https://doi.org/10.48550/arXiv.1609.04747 |
[17] |
G. E. Hinton, S. Osindero, Y. W. Teh, A fast learning algorithm for deep belief nets, Neural Comput., 18 (2006), 1527–1554. http://doi.org/10.1162/neco.2006.18.7.1527 doi: 10.1162/neco.2006.18.7.1527
![]() |
[18] |
S. Sun, Z. Cao, H. Zhu, J. Zhao, A survey of optimization methods from a machine learning perspective, IEEE Trans. Cybern., 50 (2019), 3668–3681. https://doi.org/10.1109/TCYB.2019.2950779 doi: 10.1109/TCYB.2019.2950779
![]() |
[19] |
M. A. Ferrag, L. Shu, O. Friha, X. Yang, Cyber security intrusion detection for agriculture 4.0: machine learning-based solutions, datasets, and future directions, IEEE/CAA J. Autom. Sin., 9 (2021), 407–436. https://doi.org/10.1109/JAS.2021.1004344 doi: 10.1109/JAS.2021.1004344
![]() |
[20] | Z. Lu, H. Pu, F. Wang, Z. Hu, L. Wang, The expressive power of neural networks: a view from the width, Adv. Neural Inf. Process. Syst. (NIPS), 30, (2017). |
[21] |
H. P. Beise, S. D. Da Cruz, U. Schröder, On decision regions of narrow deep neural networks, Neural Networks, 140 (2021), 121–129. https://doi.org/10.1016/j.neunet.2021.02.024 doi: 10.1016/j.neunet.2021.02.024
![]() |
[22] |
J. He, H. Yang, L. He, L. Zhao, Neural networks based on vectorized neurons, Neurocomputing, 465 (2021), 63–70. https://doi.org/10.1016/j.neucom.2021.09.006 doi: 10.1016/j.neucom.2021.09.006
![]() |
[23] |
S. Q. Zhang, W. Gao, Z. H. Zhou, Towards understanding theoretical advantages of complex-reaction networks, Neural Networks, 151 (2022), 80–93. https://doi.org/10.1016/j.neunet.2022.03.024 doi: 10.1016/j.neunet.2022.03.024
![]() |
[24] |
S. Ostojic, N. Brunel, From spiking neuron models to linear-nonlinear models, PLoS Comput. Biol., 7 (2011), e1001056. https://doi.org/10.1371/journal.pcbi.1001056 doi: 10.1371/journal.pcbi.1001056
![]() |
[25] |
T. Zhou, S. Gao, J. Wang, C. Chu, Y. Todo, Z. Tang, Financial time series prediction using a dendritic neuron model, Knowl. Based Syst., 105 (2016), 214–224. https://doi.org/10.1016/j.knosys.2016.05.031 doi: 10.1016/j.knosys.2016.05.031
![]() |
[26] |
T. Zhang, C. Lv, F. Ma, K. Zhao, H. Wang, G. M. O'Hare, A photovoltaic power forecasting model based on dendritic neuron networks with the aid of wavelet transform, Neurocomputing, 397 (2020), 438–446. https://doi.org/10.1016/j.neucom.2019.08.105 doi: 10.1016/j.neucom.2019.08.105
![]() |
[27] | S. Ghosh-Dastidar, H. Adeli, Spiking neural networks, Int. J. Neural Syst., 19 (2009), 295–308. https://doi.org/10.1142/S0129065709002002 |
[28] |
S. R. Kheradpisheh, T. Masquelier, Temporal backpropagation for spiking neural networks with one spike per neuron, Int. J. Neural Syst., 30 (2020), 2050027. https://doi.org/10.1142/S0129065720500276 doi: 10.1142/S0129065720500276
![]() |
[29] |
X. Wang, X. Lin, X. Dang, Supervised learning in spiking neural networks: a review of algorithms and evaluations, Neural Networks, 125 (2020), 258–280. https://doi.org/10.1016/j.neunet.2020.02.011 doi: 10.1016/j.neunet.2020.02.011
![]() |
[30] |
L. Deng, Y. Wu, X. Hu, L. Liang, Y. Ding, G. Li, et al., Rethinking the performance comparison between SNNs and ANNs, Neural Networks, 121 (2020), 294–307. https://doi.org/10.1016/j.neunet.2019.09.005 doi: 10.1016/j.neunet.2019.09.005
![]() |
[31] |
S. WoĖniak, A. Pantazi, T. Bohnstingl, E. Eleftheriou, Deep learning incorporating biologically inspired neural dynamics and in-memory computing, Nat. Mach. Intell., 2 (2020), 325–336. https://doi.org/10.1038/s42256-020-0187-0 doi: 10.1038/s42256-020-0187-0
![]() |
[32] |
P. Poirazi, A. Papoutsi, Illuminating dendritic function with computational models, Nat. Rev. Neurosci., 21 (2020), 303–321. https://doi.org/10.1038/s42256-020-0187-0 doi: 10.1038/s42256-020-0187-0
![]() |
[33] |
A. Gidon, T. A. Zolnik, P. Fidzinski, F. Bolduan, A. Papoutsi, P. Poirazi, et al., Dendritic action potentials and computation in human layer 2/3 cortical neurons, Science, 367 (2020), 83–87. https://doi.org/10.1126/science.aax6239 doi: 10.1126/science.aax6239
![]() |
[34] |
R. L. Wang, Z. Lei, Z. Zhang, S. Gao, Dendritic convolutional neural network, IEEJ Trans. Electron. Inf. Syst., 17 (2022), 302–304. https://doi.org/10.1002/tee.23513 doi: 10.1002/tee.23513
![]() |
[35] | J. Li, Z. Liu, R. L. Wang, S. Gao, Dendritic deep residual learning for COVID-19 prediction, IEEJ Trans. Electron. Inf. Syst., 18 (2022) 297–299. https://doi.org/10.1002/tee.23723 |
[36] |
Z. H. Zhan, L. Shi, K. C. Tan, J. Zhang, A survey on evolutionary computation for complex continuous optimization, Artif. Intell. Rev., 55 (2022), 59–110. https://doi.org/10.1007/s10462-021-10042-y doi: 10.1007/s10462-021-10042-y
![]() |
[37] | J. Greensmith, U. Aickelin, S. Cayzer, Introducing dendritic cells as a novel immune-inspired algorithm for anomaly detection, in International Conference on Artificial Immune Systems (ICARIS 2005), Springer, (2005), 153–167. https://doi.org/10.1007/11536444_12 |
[38] | M. London, M. Häusser, Dendritic computation, Annu. Rev. Neurosci., 28 (2005), 503–532. https://doi.org/10.1146/annurev.neuro.28.061604.135703 |
[39] |
H. Agmon-Snir, C. E. Carr, J. Rinzel, The role of dendrites in auditory coincidence detection, Nature, 393 (1998), 268–272. https://doi.org/10.1038/30505 doi: 10.1038/30505
![]() |
[40] |
T. Euler, P. B. Detwiler, W. Denk, Directionally selective calcium signals in dendrites of starburst amacrine cells, Nature, 418 (2002), 845–852. https://doi.org/10.1038/nature00931 doi: 10.1038/nature00931
![]() |
[41] | C. Koch, T. Poggio, V. Torre, Nonlinear interactions in a dendritic tree: localization, timing, and role in information processing, Proc. Natl. Acad. Sci., 80 (1983), 2799–2802. https://doi.org/10.1073/pnas.80.9.2799 |
[42] |
S. Chavlis, P. Poirazi, Drawing inspiration from biological dendrites to empower artificial neural networks, Curr. Opin. Neurobiol., 70 (2021), 1–10. https://doi.org/10.1016/j.conb.2021.04.007 doi: 10.1016/j.conb.2021.04.007
![]() |
[43] |
J. Guerguiev, T. P. Lillicrap, B. A. Richards, Towards deep learning with segregated dendrites, Elife, 6 (2017), e22901. https://doi.org/10.7554/eLife.22901 doi: 10.7554/eLife.22901
![]() |
[44] |
T. Moldwin, M. Kalmenson, I. Segev, The gradient clusteron: a model neuron that learns to solve classification tasks via dendritic nonlinearities, structural plasticity, and gradient descent, PLoS Comput. Biol., 17 (2021), e1009015. https://doi.org/10.1371/journal.pcbi.1009015 doi: 10.1371/journal.pcbi.1009015
![]() |
[45] |
C. Koch, I. Segev, The role of single neurons in information processing, Nat. Neurosci., 3 (2000), 1171–1177. https://doi.org/10.1038/81444 doi: 10.1038/81444
![]() |
[46] |
P. Poirazi, T. Brannon, B. W. Mel, Pyramidal neuron as two-layer neural network, Neuron, 37 (2003), 989–999. https://doi.org/10.1016/S0896-6273(03)00149-1 doi: 10.1016/S0896-6273(03)00149-1
![]() |
[47] |
R. Legenstein, W. Maass, Branch-specific plasticity enables self-organization of nonlinear computation in single neurons, J. Neurosci., 31 (2011), 10787–10802. https://doi.org/10.1523/JNEUROSCI.5684-10.2011 doi: 10.1523/JNEUROSCI.5684-10.2011
![]() |
[48] | I. S. Jones, K. P. Kording, Might a single neuron solve interesting machine learning problems through successive computations on its dendritic tree, Neural Comput., 33 (2021), 1554–1571. https://doi.org/10.1162/neco_a_01390 |
[49] | Y. Todo, H. Tamura, K. Yamashita, Z. Tang, Unsupervised learnable neuron model with nonlinear interaction on dendrites, Neural Networks, 60 (2014), 96–103. https://doi.org/10.1016/j.neunet.2014.07.011 |
[50] |
S. Gao, M. Zhou, Y. Wang, J. Cheng, H. Yachi, J. Wang, Dendritic neuron model with effective learning algorithms for classification, approximation, and prediction, IEEE Trans. Neural Networks Learn. Syst., 30 (2019), 601–614. https://doi.org/10.1109/TNNLS.2018.2846646 doi: 10.1109/TNNLS.2018.2846646
![]() |
[51] |
Y. Todo, Z. Tang, H. Todo, J. Ji, K. Yamashita, Neurons with multiplicative interactions of nonlinear synapses, Int. J. Neural Syst., 29 (2019), 1950012. https://doi.org/10.1142/S0129065719500126 doi: 10.1142/S0129065719500126
![]() |
[52] |
X. Luo, X. Wen, M. Zhou, A. Abusorrah, L. Huang, Decision-tree-initialized dendritic neuron model for fast and accurate data classification, IEEE Trans. Neural Networks Learn. Syst., 33 (2022), 4173 – 4183. https://doi.org/10.1109/TNNLS.2021.3055991 doi: 10.1109/TNNLS.2021.3055991
![]() |
[53] |
J. Ji, Y. Tang, L. Ma, J. Li, Q. Lin, Z. Tang, et al., Accuracy versus simplification in an approximate logic neural model, IEEE Trans. Neural Networks Learn. Syst., 32 (2020), 5194–5207. https://doi.org/10.1109/TNNLS.2020.3027298 doi: 10.1109/TNNLS.2020.3027298
![]() |
[54] | S. Gao, M. Zhou, Z. Wang, D. Sugiyama, J. Cheng, J. Wang, et al., Fully complex-valued dendritic neuron model, IEEE Trans. Neural Networks Learn. Syst., 1–14. https://doi.org/10.1109/TNNLS.2021.3105901 |
[55] | Y. Tang, J. Ji, S. Gao, H. Dai, Y. Yu, Y. Todo, A pruning neural network model in credit classification analysis, Comput. Intell. Neurosci., 2018 (2018), 9390410. https://doi.org/10.1155/2018/9390410 |
[56] |
Z. Lei, S. Gao, Z. Zhang, M. Zhou, J. Cheng, MO4: a many-objective evolutionary algorithm for protein structure prediction, IEEE Trans. Evol. Comput., 26 (2022), 417–430. https://doi.org/10.1109/TEVC.2021.3095481 doi: 10.1109/TEVC.2021.3095481
![]() |
[57] |
J. X. Mi, J. Feng, K. Y. Huang, Designing efficient convolutional neural network structure: a survey, Neurocomputing, 489 (2022), 139–156. https://doi.org/10.1016/j.neucom.2021.08.158 doi: 10.1016/j.neucom.2021.08.158
![]() |
[58] |
Z. H. Zhan, J. Y. Li, J. Zhang, Evolutionary deep learning: a survey, Neurocomputing, 483 (2022), 42–58. https://doi.org/10.1016/j.neucom.2022.01.099 doi: 10.1016/j.neucom.2022.01.099
![]() |
[59] | Z. Wang, S. Gao, M. Zhou, S. Sato, J. Cheng, J. Wang, Information-theory-based nondominated sorting ant colony optimization for multiobjective feature selection in classification, IEEE Trans. Cybern., 1–14. https://doi.org/10.1109/TCYB.2022.3185554 |
[60] |
Y. Yu, Z. Lei, Y. Wang, T. Zhang, C. Peng, S. Gao, Improving dendritic neuron model with dynamic scale-free network-based differential evolution, IEEE/CAA J. Autom. Sin., 9 (2022), 99–110. https://doi.org/10.1109/JAS.2021.1004284 doi: 10.1109/JAS.2021.1004284
![]() |
[61] |
Y. Yu, S. Gao, Y. Wang, Y. Todo, Global optimum-based search differential evolution, IEEE/CAA J. Autom. Sin., 6 (2018), 379–394. https://doi.org/10.1109/JAS.2019.1911378 doi: 10.1109/JAS.2019.1911378
![]() |
[62] | S. Mirjalili, A. Lewis, The whale optimization algorithm, Adv. Eng. Software, 95 (2016), 51–67. https://doi.org/10.1016/j.advengsoft.2016.01.008 |
[63] | A. Kumar, R. K. Misra, D. Singh, S. Mishra, S. Das, The spherical search algorithm for bound-constrained global optimization problems, Appl. Soft Comput., 85 (2019), 105734. https://doi.org/10.1016/j.asoc.2019.105734 |
[64] | K. Wang, Y. Wang, S. Tao, Z. Cai, Z. Lei, S. Gao, Spherical search algorithm with adaptive population control for global continuous optimization problems, Appl. Soft Comput., 132 (2023), 109845. https://doi.org/10.1016/j.asoc.2022.109845 |
[65] |
N. Takahashi, K. Kitamura, N. Matsuo, M. Mayford, M. Kano, N. Matsuki, et al., Locally synchronized synaptic inputs, Science, 335 (2012), 353–356. https://doi.org/10.1126/science.1210362 doi: 10.1126/science.1210362
![]() |
[66] |
W. Chen, J. Sun, S. Gao, J. J. Cheng, J. Wang, Y. Todo, Using a single dendritic neuron to forecast tourist arrivals to Japan, IEICE Trans. Inf. Syst., 100 (2017), 190–202. https://doi.org/10.1587/transinf.2016EDP7152 doi: 10.1587/transinf.2016EDP7152
![]() |
[67] | J. Ji, S. Song, Y. Tang, S. Gao, Z. Tang, Y. Todo, Approximate logic neuron model trained by states of matter search algorithm, Knowl. Based Syst., 163 (2019), 120–130. https://doi.org/10.1016/j.knosys.2018.08.020 |
[68] | Z. Wang, S. Gao, J. Wang, H. Yang, Y. Todo, A dendritic neuron model with adaptive synapses trained by differential evolution algorithm, Comput. Intell. Neurosci., 2020 (2020), 2710561. https://doi.org/10.1155/2020/2710561 |
[69] | Z. Song, Y. Tang, J. Ji, Y. Todo, Evaluating a dendritic neuron model for wind speed forecasting, Knowl. Based Syst., 201 (2020), 106052. https://doi.org/10.1016/j.knosys.2020.106052 |
[70] | R. Tanabe, A. S. Fukunaga, Improving the search performance of SHADE using linear population size reduction, in 2014 IEEE Congress on Evolutionary Computation (CEC), IEEE, (2014), 1658–1665. https://doi.org/10.1109/CEC.2014.6900380 |
[71] | Z. Xu, Z. Wang, J. Li, T. Jin, X. Meng, S. Gao, Dendritic neuron model trained by information feedback-enhanced differential evolution algorithm for classification, Knowl. Based Syst., 233 (2021), 107536. https://doi.org/10.1016/j.knosys.2021.107536 |
[72] | R. Jiang, M. Yang, S. Wang, T. Chao, An improved whale optimization algorithm with armed force program and strategic adjustment, Appl. Math. Modell., 81 (2020), 603–623. https://doi.org/10.1016/j.apm.2020.01.002 |
[73] |
S. Gao, Y. Yu, Y. Wang, J. Wang, J. Cheng, M. Zhou, Chaotic local search-based differential evolution algorithms for optimization, IEEE Trans. Syst. Man Cybern. Syst., 51 (2021), 3954–3967. https://doi.org/10.1109/TSMC.2019.2956121 doi: 10.1109/TSMC.2019.2956121
![]() |
[74] | W. Dong, L. Kang, W. Zhang, Opposition-based particle swarm optimization with adaptive mutation strategy, Soft Comput., 21 (2017), 5081–5090. https://doi.org/0.1007/s00500-016-2102-5 |
[75] | J. Too, M. Mafarja, S. Mirjalili, Spatial bound whale optimization algorithm: an efficient high-dimensional feature selection approach, Neural Comput. Appl., 33 (2021), 16229–16250. https://doi.org/10.1007/s00521-021-06224-y |
[76] | Y. Gao, C. Qian, Z. Tao, H. Zhou, J. Wu, Y. Yang, Improved whale optimization algorithm via cellular automata, in 2020 IEEE International Conference on Progress in Informatics and Computing (PIC), IEEE, (2020), 34–39. https://doi.org/10.1109/PIC50277.2020.9350796 |
[77] |
J. Zhang, J. S. Wang, Improved whale optimization algorithm based on nonlinear adaptive weight and golden sine operator, IEEE Access, 8 (2020), 77013–77048. https://doi.org/10.1109/ACCESS.2020.2989445 doi: 10.1109/ACCESS.2020.2989445
![]() |
[78] | C. Tang, W. Sun, W. Wu, M. Xue, A hybrid improved whale optimization algorithm, in 2019 IEEE 15th International Conference on Control and Automation (ICCA), IEEE, (2019), 362–367. https://doi.org/10.1109/ICCA.2019.8900003 |
[79] | U. Škvorc, T. Eftimov, P. Korošec, CEC real-parameter optimization competitions: progress from 2013 to 2018, in 2019 IEEE Congress on Evolutionary Computation (CEC), IEEE, (2019), 3126–3133. https://doi.org/10.1109/CEC.2019.8790158 |
[80] |
T. Jiang, S. Gao, D. Wang, J. Ji, Y. Todo, Z. Tang, A neuron model with synaptic nonlinearities in a dendritic tree for liver disorders, IEEJ Trans. Electr. Electron. Eng., 12 (2017), 105–115. https://doi.org/10.1002/tee.22350 doi: 10.1002/tee.22350
![]() |
[81] |
J. F. Khaw, B. Lim, L. E. Lim, Optimal design of neural networks using the taguchi method, Neurocomputing, 7 (1995), 225–245. https://doi.org/10.1016/0925-2312(94)00013-I doi: 10.1016/0925-2312(94)00013-I
![]() |
[82] | J. Carrasco, S. García, M. Rueda, S. Das, F. Herrera, Recent trends in the use of statistical tests for comparing swarm and evolutionary computing algorithms: practical guidelines and a critical review, Swarm Evol. Comput., 54 (2020), 100665. https://doi.org/10.1016/j.swevo.2020.100665 |
[83] |
J. Misra, I. Saha, Artificial neural networks in hardware: a survey of two decades of progress, Neurocomputing, 74 (2010), 239–255. https://doi.org/10.1016/j.neucom.2010.03.021 doi: 10.1016/j.neucom.2010.03.021
![]() |
[84] | S. Mittal, A survey of FPGA-based accelerators for convolutional neural networks, Neural Comput. Appl., 32 (2020), 1109–1139. https://doi.org/10.1007/s00521-018-3761-1 |
[85] | H. Zhang, M. Gu, X. Jiang, J. Thompson, H. Cai, S. Paesani, et al., An optical neural chip for implementing complex-valued neural network, Nat. Commun., 12 (2021), 1–11. https://doi.org/10.1038/s41467-020-20719-7 |
[86] | H. Shayanfar, F. S. Gharehchopogh, Farmland fertility: a new metaheuristic algorithm for solving continuous optimization problems, Appl. Soft Comput., 71 (2018), 728–746. https://doi.org/10.1016/j.asoc.2018.07.033 |
[87] |
F. S. Gharehchopogh, B. Farnad, A. Alizadeh, A modified farmland fertility algorithm for solving constrained engineering problems, Concurrency Comput. Pract. Exper., 33 (2021), e6310. https://doi.org/10.1002/cpe.6310 doi: 10.1002/cpe.6310
![]() |
[88] | A. Hosseinalipour, F. S. Gharehchopogh, M. Masdari, A. Khademi, A novel binary farmland fertility algorithm for feature selection in analysis of the text psychology, Appl. Intell., 51 (2021), 4824–4859. https://doi.org/10.1007/s10489-020-02038-y |
[89] | B. Abdollahzadeh, F. S. Gharehchopogh, S. Mirjalili, African vultures optimization algorithm: a new nature-inspired metaheuristic algorithm for global optimization problems, Comput. Ind. Eng., 158 (2021), 107408. https://doi.org/10.1016/j.cie.2021.107408 |
[90] | B. Abdollahzadeh, F. S. Gharehchopogh, N. Khodadadi, S. Mirjalili, Mountain gazelle optimizer: a new nature-inspired metaheuristic algorithm for global optimization problems, Adv. Eng. Software, 174 (2022), 103282. https://doi.org/10.1016/j.advengsoft.2022.103282 |
[91] |
B. Abdollahzadeh, F. Soleimanian Gharehchopogh, S. Mirjalili, Artificial gorilla troops optimizer: a new nature-inspired metaheuristic algorithm for global optimization problems, Int. J. Intell. Syst., 36 (2021), 5887–5958. https://doi.org/10.1002/int.22535 doi: 10.1002/int.22535
![]() |
[92] |
W. R. Taylor, S. He, W. R. Levick, D. I. Vaney, Dendritic computation of direction selectivity by retinal ganglion cells, Science, 289 (2000), 2347–2350. https://doi.org/10.1126/science.289.5488.2347 doi: 10.1126/science.289.5488.2347
![]() |
Stage | Output size | Blocks |
1 | H/2×W/2×64 | 2 |
2 | H/4×W/4×128 | 3 |
3 | H/8×W/8×256 | 4 |
4 | H/16×W/16×512 | 2 |
Model | Params | Throughput (image/s) | Original dataset | Original dataset including synthetic examples |
Top-1 Accuracy (%) | Top-1 Accuracy (%) | |||
ResMLP | 14.52 M | 13 | 94.0 | 97.2 |
ResNet50 | 23.7 M | 19 | 97.4 | 99.7 |
PVT | 12.3 M | 13 | 76.3 | 89.5 |
ViT | 85.8 M | 11 | 88.8 | 95.7 |
ours | 26.1 M | 14 | 99.8 | 100 |
Model | Median | Mean | Std. Dev. |
ResMLP | 0.99 | 0.96 | 0.109 |
ResNet50 | 1.00 | 0.99 | 0.047 |
PVT | 0.95 | 0.85 | 0.184 |
ViT | 0.99 | 0.94 | 0.121 |
ours | 1.00 | 0.99 | 0.034 |
Model | Weather | Gauss | Blur | Affine |
ResMLP | 44.08% | 53.01% | 51.86% | 54.71% |
ResNet50 | 70.26% | 65.05% | 64.09% | 65.75% |
PVT | 35.66% | 32.25% | 32.80% | 37.46% |
ViT | 48.14% | 47.34% | 47.09% | 48.35% |
ours | 73.37% | 61.48% | 60.48% | 62.64% |
Model | Weather | Gauss | Blur | Affie |
ResNet50 | 91.22% | 91.57% | 90.97% | 91.27% |
ours | 93.93% | 95.09% | 93.18% | 94.03% |
Model | WiNet-Relu | WiNet-noMLP | WiNet |
Top-1 Accuracy | 98.5% | 99.4% | 99.8% |
Stage | Output size | Blocks |
1 | H/2×W/2×64 | 2 |
2 | H/4×W/4×128 | 3 |
3 | H/8×W/8×256 | 4 |
4 | H/16×W/16×512 | 2 |
Model | Params | Throughput (image/s) | Original dataset | Original dataset including synthetic examples |
Top-1 Accuracy (%) | Top-1 Accuracy (%) | |||
ResMLP | 14.52 M | 13 | 94.0 | 97.2 |
ResNet50 | 23.7 M | 19 | 97.4 | 99.7 |
PVT | 12.3 M | 13 | 76.3 | 89.5 |
ViT | 85.8 M | 11 | 88.8 | 95.7 |
ours | 26.1 M | 14 | 99.8 | 100 |
Model | Median | Mean | Std. Dev. |
ResMLP | 0.99 | 0.96 | 0.109 |
ResNet50 | 1.00 | 0.99 | 0.047 |
PVT | 0.95 | 0.85 | 0.184 |
ViT | 0.99 | 0.94 | 0.121 |
ours | 1.00 | 0.99 | 0.034 |
Model | Weather | Gauss | Blur | Affine |
ResMLP | 44.08% | 53.01% | 51.86% | 54.71% |
ResNet50 | 70.26% | 65.05% | 64.09% | 65.75% |
PVT | 35.66% | 32.25% | 32.80% | 37.46% |
ViT | 48.14% | 47.34% | 47.09% | 48.35% |
ours | 73.37% | 61.48% | 60.48% | 62.64% |
Model | Weather | Gauss | Blur | Affie |
ResNet50 | 91.22% | 91.57% | 90.97% | 91.27% |
ours | 93.93% | 95.09% | 93.18% | 94.03% |
Model | WiNet-Relu | WiNet-noMLP | WiNet |
Top-1 Accuracy | 98.5% | 99.4% | 99.8% |