GastroFuse-Net: an ensemble deep learning framework designed for gastrointestinal abnormality detection in endoscopic images

1.   Introduction

Endoscopic examinations are widely recognized as the traditional method for identifying gastrointestinal troubles, owing to their well-established efficacy. During an endoscopy, a patient's inner organs are meticulously examined to identify any underlying issues, enabling the medical team to determine the most effective course of treatment based on the patient's symptoms. An endoscope, a bendy tube with a fixed camera at one end and a light source attached, is used in this approach. The camera takes pictures of the organs, that are analyzed in greater detail. Different types of endoscopy procedures exist, depending on the purpose of the examination of the structures being observed and the equipment used. Endoscopes can be introduced into the body through a surgical incision, the mouth, or the esophagus ^[1].

The human digestive system is renowned for its diverse array of mucosal traits, presenting a broad spectrum of illnesses ranging from minor maladies to potentially life-threatening diseases. Given that there are 3.5 million reported cases and over 2.2 million deaths related to malignancies each year globally ^[2], it is essential to prioritize accurate and timely diagnosis for effective treatment and reducing illness and mortality rates ^[3,4]. Therefore, it is crucial to enhance the performance of clinical examinations and implement systematic screening strategies.

Computer-assisted automatic diagnosis is a recent subject of studies that shows the potential to revolutionize healthcare systems and scientific practice. Within the healthcare domain, deep convolutional neural networks (DCNNs) constitute a promising area to explore, as they possess the capability to aid medical experts in turning in remarkable care on a massive scale. This has been substantiated via numerous studies ^[5,6,7,8,9]. A vast volume of advanced image data, coupled with superior algorithms, enables the effective utilization of a DCNN-based system in the identification of gastrointestinal lesions inside endoscopic images ^[10,11,12]. The accurate and timely identification of ailments holds paramount importance because it directly influences remedy planning and patient monitoring ^[13,14,15].

Over 15 years, there has been an active search amongst researchers to analyze the usage of computer algorithms for the purpose of detecting problems within the human gastrointestinal machine via the examination of endoscopic images ^[16,17,18]. Recent clinical studies have proven that deep learning models have promising outcomes in identifying irregularities, especially within the realm of gastrointestinal polyp detection ^[19]. Nevertheless, the diagnostic efficacy of these models is significantly reliant on the quantity and quality of the data at hand.

This research employs the Kvasir dataset ^[20], which incorporates 8000 endoscopic images meticulously annotated by proficient endoscopists. The dataset consists of 8 unique classes, containing anatomical landmarks, medical outcomes, and gastrointestinal endoscopic approaches. The key goal of this study is to design a revolutionary algorithm based on Convolutional Neural Networks (CNN) tailored for the automated classification of anomalies inside the gastrointestinal system across multiple classes. The proposed deep learning framework aspires to correctly classify endoscopic images of the gastrointestinal tract into various categories with minimal preprocessing and optimized augmentation strategies. This specialized architecture demonstrates the potential to enhance the performance of the multi-class classification procedure, outperforming gastroenterologists in terms of precision and results. The results show that the proposed deep learning-based methodology can help gastroenterologists in categorizing gastrointestinal problems. The following are the contributions to this research:

• Designing a simple yet effective shallow CNN model with a careful choice of fewer layers to save computational resources and maintain competitive endoscopic images.

• Designing a deep CNN model with more layers to learn intricate patterns associated with various gastrointestinal conditions. This hierarchical feature learning significantly boosted the model's ability to discriminate and identify complex patterns.

• Introducing a comprehensive CNN-based architecture (called GastroFuse-Net) that combines knowledge from the designed shallow and deep CNN architectures in a novel way using feature concatenation. This method best utilizes the deep model's proficiency in complex feature extraction and the shallow model's capacity to collect wide context, producing a synergistic and all-encompassing classification framework.

• Assessing the performance of the proposed models using precision, recall, F1-score, Mathew's correlation coefficient (MCC), and accuracy metrics on the test dataset. These evaluations demonstrate the intricate balance between model complexity and performance, offering a comprehensive comprehension of the consequences of layer depth in CNN architectures for classifying gastrointestinal abnormalities.

The remaining sections of the paper are organized in the following manner: In Section 2, we conduct a thorough literature analysis focusing on deep learning models employed for classifying gastrointestinal abnormalities using endoscopy images. In Section 3, we detail the dataset used in this study, outlining its characteristics and the specific preprocessing steps undertaken. Additionally, the proposed novel architectures tailored to the task of gastrointestinal abnormality classification have been explained in this section. Section 4 is dedicated to evaluating the performance of our proposed models based on precision, recall, F1-score, Matthew's correlation coefficient, and accuracy. The research is concluded in Section 5 by summarizing the essential findings and insights derived from the experiments.

2.   Related work

A number of countries are making an investment resource into AI (Artificial Intelligence) research right now, with an exclusive emphasis on computer vision and deep learning ^[21,22,23]. However, getting medical data might be complicated because of restricted availability imposed by means of regulatory constraints and a scarcity of human expertise needed for manually labeled training data. These limits make building computerized examination structures difficult. In response, distinct strategies for automating the identification of anomalies inside the human gastrointestinal tract using machine learning and endoscopic images have been developed by the research groups. The time-consuming nature of manually examining large sets of gastric images, necessitating professional knowledge, has been highlighted in recent literature reviews on gastrointestinal anomalies ^[24,25]. To tackle this trouble, it is viable to develop AI-powered diagnostic tools that may automate the interpretation of large endoscopic data. Existing machine learning and deep learning techniques are presented in the subsequent sub-sections.

2.1.   Existing machine learning techniques

In ^[26], the authors proposed a methodology by hybridizing Haralick texture features and Local Binary Pattern visual descriptors to classify gastrointestinal abnormalities from endoscopic images. The researchers carried out individual training of logistic regression models for each variable and used an ensemble approach to make the very last prediction. This technique obtained an accuracy of 94% with an F1-score of 0.76 and an MCC of 0.73 on testing data. In ^[27], a computer-aided detection system was designed to reduce missed polyps during video endoscopy. In this method, color wavelet features from endoscopic images were used to train a Support Vector Machine (SVM) classifier. The obtained results demonstrated the effectiveness of this technique with an accuracy of 98.34%, sensitivity of 98.67%, and specificity of 98.23%. In ^[28], gastrointestinal abnormalities were detected and classified using a variety of machine learning classifiers including logistic regression, decision tree, naïve bayes, SVM, and random forest. Among these methods, logistic regression emerged as the top performer. In ^[24], the authors detected gastric abnormalities using an SVM classifier achieving an accuracy of 0.86. Another study ^[29] identified multiple gastrointestinal disease detection using Linear Discriminant Analysis (LDA) and achieved an accuracy of 0.91, precision of 0.87, recall of 0.85, and F1-score of 0.86.

2.2.   Existing deep learning techniques

In a study ^[30], the authors generated function vectors from geometric patterns extracted from images with the use of Inception-v3 and VGG16 networks, and the SVM classifier was trained on those extracted features. Their proposed technique correctly identified anomalies of the gastrointestinal system in endoscopic images with an MCC of 0.826. Another computational model stimulated from the Inception-v1 network to identify gastrointestinal issues and anatomical landmarks correctly was proposed ^[31]. The model consisted of convolutional layers of various sizes and pooling layers. The researchers employed an extensive range of data augmentation procedures to optimize performance. However, their method faced challenges in distinguishing between Dyed Lifted Polys and Dyed Resection Margins, in addition to discerning between classes of polyps and ulcerative colitis.

Researchers ^[32] proposed a ResNet-50 architecture to facilitate the automated analysis of endoscopic data. This approach took into account the interrelationships among the extracted variables. In a recent work ^[33], NASNet, Inception-v4, and Inception-ResNet-v2 architectures have been used to identify anatomical landmarks and detect sick tissue inside the human gastrointestinal system. Preprocessing techniques were additionally used to improve the quality of the images, resulting in an MCC of 0.93. An attention-based model was introduced in a separate study ^[34] to divide endoscopic images into four categories. During the subsequent phase of this study, an anomaly identification methodology was employed to discern atypical types inside the initial stage. Researchers ^[35] presented a series of picture preparation procedures, which are then followed by the implementation of the EWT (Empirical Wavelet Transform). The decomposed images were then used as input for the proposed CNN model for disease classification at two levels. The results indicated an accuracy of 96.65% and MCC of 0.9298 in the first level of classification. In the second level of classification, 94.25% accuracy and 0.810 MCC was achieved. Researchers ^[36] presented the classification of gastrointestinal tract disorders using different transfer learning models. The best results were achieved using EfficientNetB0 with 98.01% accuracy, a precision of 98%, and a recall of 98%.

We present the GastroFuse-Net model, a new CNN-based architecture designed specifically for automatically classifying anomalies in the gastrointestinal system. This model is innovative and differs from previous studies. GastroFuse-Net combines the knowledge from both shallow and deep CNN architectures by merging their features through concatenation. This innovative strategy maximizes the deep model's ability to extract intricate features and the shallow model's capability to capture extensive context. The objective is to address the constraints identified in previous research, such as difficulties in differentiating between distinct gastrointestinal categories and enhancing the overall accuracy of classification. GastroFuse-Net effectively combines the advantages of shallow and deep models to provide a more complete and synergistic approach to classifying gastrointestinal abnormalities.

3.   Material and methods

A deep learning-based solution has been proposed in this study for the automatic diagnosis of gastrointestinal abnormalities using deep learning. The proposed methodology carried out for this research is shown in Figure 1. Details of each block have been discussed in the sub-sections below.

Figure 1.  Proposed Methodology.

DownLoad: Full-Size Img PowerPoint

3.1.   Dataset description

The dataset utilized in this investigation was acquired from the Kvasir dataset ^[20]. There are eight distinct types of images stored in the Kvasir Database with 1000 images belonging to each category. Additional classifications of these photos include three for anatomical landmarks, three for pathological stages, and two for lesion eradication. To classify anatomical features, we have the pylorus, the z-line, and the cecum. The pylorus is the area of anatomy that spans the gap between the small intestine's first segment and the stomach. The z-line represents the anatomical boundary, where the esophagus transitions into the stomach. Examining this particular landmark holds significance in disease identification, as the manifestation of Esophagitis is commonly observed at this site. The cecum serves as the initial segment of the large intestine, and the achievement of reaching this anatomical marker is regarded as the culmination of a colonoscopy procedure. Esophagitis, polyps, and ulcerative colitis are the three disease-related conditions. Esophagitis is a medical condition characterized by inflammation of the esophagus, leading to the development of a mucosal rupture at the z-line. Ulcerative colitis is a pathological condition characterized by inflammation of the colon, specifically the large intestine. Polyps are aberrant growths that develop within the large intestine and can potentially progress into a precancerous state. Each of these categories represents a crucial aspect that a gastroenterologist examines during an endoscopic procedure. Two distinct categories are associated with the excision of lesions, namely dyed-lifted polyps and dyed resection margins.

A dye is used for polyp removal to enhance the visibility of the polyp, and a procedure called "lifting" is used to separate polyps from the surrounding tissue. Before removing a polyp, images are taken of it that have been dyed and lifted, and afterward, the resection margins are stained. The dataset exhibits a range of image resolutions from 720x576 pixels to 1920x1072 pixels. Figure 2 displays image samples for each class in the dataset. Before training the model, the dataset was partitioned into three sets: Training, Test set, and Validation set. A total of 80% of the data were devoted to the training set, while the remaining 20% were reserved for the test set. A subset containing 10% was allocated for validation among the total training samples. Table 1 provides the distribution of sample sizes across different classes within each data set.

Figure 2.  Sample Images from the dataset: (a) Cecum, (b) Pylorus, (c) Z-line, (d) Esophagitis, (e) Ulcerative Colitis, (f) Polyps, (g) Dyed Lifted Polyps, and (h) Dyed Resection Margins.

DownLoad: Full-Size Img PowerPoint

Table 1.  Dataset description with Train, Test and Validation Splits.

Class Name	Total Images in the dataset	Train Set	Augmented Train Set	Validation Set	Test Set
Ceum	1000	720	5760	80	200
Pylorus	1000	720	5760	80	200
Z-line	1000	720	5760	80	200
Esophagitis	1000	720	5760	80	200
Ulcerative Colitis	1000	720	5760	80	200
Polyps	1000	720	5760	80	200
Dyed Lifted Polyps	1000	720	5760	80	200
Dyed Resection Margins	1000	720	5760	80	200
Total Images	8000	5760	46,080	640	1600

---

 | Show Table

DownLoad: CSV

3.2.   Data pre-processing

Effective data preparation is crucial for enhancing the efficiency of machine learning and computer vision applications that depend on visual input. Image normalization and scaling are essential stages in this procedure. Image normalization transforms pixel values into a standardized scale from 0 to 1. Normalization is performed to ensure consistent intensity levels across all pictures. This stage is critical because it reduces the potential dominance of some images, that could arise from their more extensive pixel range, during the training process.

Moreover, the normalization procedure is essential in promoting convergence throughout the training of a model. After normalizing the images, they were uniformly reduced to dimensions of 256 by 256 pixels. Image scaling guarantees the standardization of the input size for the model. Resizing helps decrease computational complexity, hence improving the model's capacity to learn more efficiently from the given data. Using the methods of image normalization and scaling to a size of 256 by 256, the preprocessed dataset is efficiently optimized for subsequent machine-learning tasks.

3.3.   Data augmentation

Data augmentation techniques are applied to the images within the training set for each class to address the problems associated with data insufficiency and overfitting, raising the training set's image count effectively as a result. Regarding the proficient and accurate implementation of data augmentation, many data transformations have been employed, including vertical and horizontal flipping, rotation at angles of 30, 45, and 60 degrees in a clockwise direction, a shear range of 0.2 factor, cropping by a factor of 0.2, and adjusting brightness by a value of 0.2. During the training process, on-the-fly image augmentation operations were performed. All the data transformations were employed simultaneously for the images to diversify the augmented dataset comprehensively while preserving the original image integrity and variability. The method of enhancing images has been achieved via the OpenCV library. Figure 3 depicts the visual representations of the original and augmented images pertaining to the Ulcerative Colitis and Polyps classes.

Figure 3.  Sample Data Transformations on Training Set.

DownLoad: Full-Size Img PowerPoint

3.4.   Proposed CNN-based architectures

We proposed a CNN-based architecture for identifying gastrointestinal abnormalities, a challenging task even for experienced gastroenterologists. Initially, two CNN-based architectures are presented, namely deep CNN and shallow CNN. These networks are designed to capture different aspects of the input gastrointestinal images and are independently trained on the dataset.

Deep CNN is responsible for capturing intricate features and learning hierarchical representations of the input images. It is composed of four convolutional blocks as shown in Figure 4(a). Three convolutional layers and a max-pooling layer make up each block. Convolutional layers are used for feature extraction and its operation is given by Eq (1).

Figure 4.  Proposed architecture of GastroFuse-Net consisting of (a) Deep Convolutional Neural Network (Deep CNN) and (b) Shallow Convolutional Neural Network (Shallow CNN).

DownLoad: Full-Size Img PowerPoint

where Z[i, j] is the output feature map as position (i, j), X is the input layer, W is the convolutional filter, F is the filter size and b is the bias term. Convolutional layers pose a challenge due to the generation of a large number of neurons. To address this issue, CNN models employ pooling layers. These layers aggregate groups of pixels into a single representative pixel. Specifically, max pooling layers identify sets of pixels and replace them with the maximum value within that set, as described in Eq (2).

where Pmax(i.j) represents the output of maxpooling operation at position (i, j) and s is the stride. Also, a batch normalization layer is applied after each convolutional layer to improve training stability and convergence. Mathematically, batch normalization is given by Eqs (3) and (4):

where ${\widehat{X}}_{i, j}$ is the normalized output, ${X}_{i, j}$ is the input, $\mu$ is the mean, ${\sigma }^{2}$ is the variance, $ϵ$ is a small constant for numerical stability, $\mathrm{\gamma }$ is a learnable scale parameter, $\beta$ is a learnable shift parameter, and ${Y}_{i, j}$ is the final output after batch normalization. Batch normalization ensures that the input to subsequent layers is normalized, facilitating more stable and efficient training of deep CNNs. As shown in Figure 4(a), the filter size is increased by a multiple of 2 in each block, starting with 32 filters and a 3x3 kernel size in the first block doubling the number of filters in subsequent blocks. Following the four convolutional blocks, a global average pooling layer is applied to obtain a fixed-length feature vector. The global average pooling layer computes the average value of each feature map across the entire spatial dimensions. Mathematically, for a given feature map $\stackrel{-}{X}$ with dimensions H X W X C, the global average pooling operation is defined by Eq (5):

The vector obtained from global average pooling layer is fed into a fully connected network of two dense layers. The first dense layer has 256 neurons, and the second dense layer has eight neurons, representing each class of gastrointestinal abnormality. This fully connected network is responsible for performing the final classification.

On the other hand, as shown in Figure 4(b), the shallow CNN focuses on extracting more local and low-level features from the input images. It consists of three convolutional blocks, each containing two convolutional layers followed by a max-pooling layer. To prevent overfitting, a dropout layer is applied after each max-pooling layer. The dropout layer introduces regularization during training by randomly deactivating a fraction of neurons, and it is given by Eq (6).

where p is the dropout probability, and the layer randomly sets a fraction of input values to zero during each forward pass. The kernel size for each convolutional layer is set to 3x3, and the number of filters increases from 32 in the first block to 64 in the second block and then to 86 in the third block, as shown in Figure 4(b).

After obtaining results from two networks, we performed feature fusion by concatenating the extracted features from both networks to leverage the complementary characteristics of the deep and shallow CNNs. Specifically, features from the penultimate layer of the deep CNN were extracted and concatenated with the corresponding features from the shallow CNN. This process combines the high-level semantic information learned by the deep CNN with the local and low-level details captured by the shallow CNN. After feature fusion, the combined feature set was classified using an artificial neural network, resulting in a novel architecture called GastroFuse-Net. Its integrated feature representation enables the model to exploit richer features, enhancing its ability to differentiate between gastrointestinal abnormalities. The formal definition of GastroFuse-Net is given by Algorithm 1.

3.5.   Experimental setup

The proposed network for classifying gastrointestinal abnormalities was trained using the backpropagation method with the gradient descent algorithm to optimize biases and weights. The convolutional layers were applied with ReLU activation function for introducing non-linearity and capturing complex patterns, while softmax was used for multi-class classification in the output layer. The Adam optimizer with a fixed learning rate of 1e-3 was employed for 50 epochs with 32 batch size. The categorical cross-entropy loss function was used for training to minimize dissimilarity between predicted probabilities and actual labels. Categorical cross entropy is given by Eq (7):

where y represents the true label and $\widehat{y}$ represents the predicted label and the sum is taken over all classes. A validation set was utilized to optimize the hyperparameters, adjusting learning rate, batch size, and other relevant parameters for best performance. The implementation of experiments was done using Keras and TensorFlow on an NVIDIA GeForce GTX 1080 graphics card for computational power. The finalized model's performance was evaluated on a separate test set which provided a fair assessment of the model's ability to generalize and classify gastrointestinal abnormalities on unseen data.

3.6.   Performance metrics

Different performance criteria have been used to evaluate how well a model can identify gastrointestinal problems from endoscopic pictures. The model's predictions are broken down in depth in the confusion matrix that has been obtained, which is the first achievement. The matrix enables the generation of other indicators for performance, and it offers insights into the types of errors generated by models and by displaying true positives, true negatives, false positives, and false negatives.

The confusion matrix was used to construct some indicators for performance which incorporates Matthews Correlation Coefficient (MCC), accuracy, precision, recall, and specificity. By dividing the total number of correctly classified cases by the size of the test set, accuracy can be understood as the overall correctness of the predictions. The formula of accuracy is given by Eq (8).

where TP, TN, FP and FN are true positives, true negatives, false positives and false negatives obtained from the confusion matrix. Moreover, accuracy may not be adequate when dealing with imbalanced datasets or classes of varying importance. By calculating the accurate percentage for predicted positive instances relative to all instances predicted as positive, precision measures the model's capacity to prevent false positives. Precision is given by Eq (9).

Similarly, recall determines how effectively the model can detect positive instances by determining what positive examples percentage are true positives. Recall is given by Eq (10).

When dealing with imbalanced datasets or trying to lessen false positives and negatives, the F1 score is a good balanced measure because it combines recall and accuracy. The F1-score is given by Eq (11).

To evaluate the model's accuracy for avoiding false positives in the negative class, specificity was used in addition to recall. Specificity is given by Eq (12)

Finally, when it comes to classification problems, Matthews Correlation Coefficient (MCC) shows a balanced performance metric that takes all four metrics into account. MCC is given by Eq (13).

4.   Results and discussions

In this section, the performance of the proposed architectures is discussed. Accuracy plots, loss plots and confusion matrix along with other performance parameters for each model are presented in subsequent sections.

4.1.   Performance of shallow CNN

The variations in accuracy and loss over 50 epochs during the training of the proposed shallow CNN is shown in Figures 5(a) and 5(b), respectively. It can be seen in these figures that there are continuously increasing and decreasing fluctuations in both training and validation accuracy and loss. Despite these fluctuations, an overall positive trend of increasing accuracy and decreasing loss indicates that the model is learning and improving over time.

Figure 5.  Plots obtained while training of Shallow CNN (a) Accuracy Plot and, (b) Loss Plot.

DownLoad: Full-Size Img PowerPoint

The confusion matrix obtained on the test data set from Shallow CNN is shown in Figure 6 and the performance metrics obtained from it for each class are presented in Table 2. The results demonstrate that the model achieved an impressive overall accuracy of 90.3%, indicating its proficiency in correctly classifying most samples. Notably, the model performed exceptionally well in accurately identifying samples from the Cecum and Pylorus classes. However, some misclassifications were observed where samples from these classes were incorrectly categorized as either Polyps or Ulcerative Colitis.

Figure 6.  Confusion matrix of the Shallow CNN model.

DownLoad: Full-Size Img PowerPoint

Table 2.  Performance Parameters obtained using Shallow CNN.

Class Name	Precision	Recall	F1-score	Specificity	MCC
Dyed Lifted Polyps	0.891	0.900	0.895	0.984	0.880
Dyed Resection Margins	0.927	0.900	0.913	0.990	0.901
Esophagitis	0.865	0.900	0.882	0.980	0.865
Cecum	0.942	0.905	0.923	0.992	0.913
Pylorus	0.989	0.895	0.939	0.998	0.933
Z-line	0.873	0.900	0.886	0.981	0.870
Polyps	0.833	0.900	0.865	0.974	0.846
Ulcerative Colitis	0.895	0.900	0.897	0.985	0.883
Average	0.902	0.900	0.900	0.986	0.886

---

 | Show Table

DownLoad: CSV

Additionally, the classes Dyed Lifted Polyps and Dyed Resection Margins were frequently misclassified interchangeably. Furthermore, the class Z-line showed a considerable number of misclassifications, predominantly being classified as Esophagitis. These findings emphasize the model's strengths in distinguishing certain classes effectively. However, it also highlights areas, where the model may require further improvement, especially in differentiating between classes that share similar features.

4.2.   Performance of Deep CNN

Figures 7(a) and 7(b) illustrate the changes in accuracy and loss during the training of a deep CNN model over 50 epochs. A notable observation is that the deep CNN model exhibits much lower training and validation accuracy and loss fluctuations than the previously evaluated shallow CNN model. However, it is worth noting that after approximately ten epochs, the validation loss and training loss seem to plateau, showing minor fluctuations around a similar value. This observation suggests that the model might have reached a saturation point where further training does not significantly improve its performance. Hence, this could indicate that additional model enhancement techniques could be explored to achieve even better results.

Figure 7.  Plots obtained while training of Deep CNN (a) Accuracy Plot and, (b) Loss Plot.

DownLoad: Full-Size Img PowerPoint

The confusion matrix of the deep CNN model and its corresponding performance metrics are presented in Table 3 and Figure 8, demonstrating its impressive classification performance. The deep CNN model improved significantly, correctly classifying 96.2% of the samples compared to the shallow CNN. Analyzing the confusion matrix, we observe that, similar to the shallow CNN, the class "Z-line" has been misclassified as "Esophagitis". However, the number of misclassified samples has reduced, indicating the deep CNN's enhanced ability to discriminate between these classes. Additionally, while some instances of "Dyed Resection Margins" were misclassified as "Dyed Lifted Polyps", the frequency of misclassifications is lower than in the shallow CNN model. The considerable accuracy gain achieved by the deep CNN model and its reduced misclassification errors indicate its superior performance in handling complex patterns and improving overall class separability.

Table 3.  Performance Parameters were obtained using Deep CNN.

Class Name	Precision	Recall	F1-score	Specificity	MCC
Dyed Lifted Polyps	0.955	0.960	0.957	0.993	0.951
Dyed Resection Margins	0.979	0.960	0.969	0.997	0.965
Esophagitis	0.927	0.960	0.943	0.989	0.935
Cecum	0.989	0.960	0.974	0.998	0.971
Pylorus	0.990	0.990	0.990	0.998	0.988
Z-line	0.949	0.930	0.939	0.992	0.930
Polyps	0.936	0.960	0.948	0.990	0.940
Ulcerative Colitis	0.955	0.960	0.957	0.993	0.951
Average	0.960	0.960	0.960	0.994	0.954

---

 | Show Table

DownLoad: CSV

Figure 8.  Confusion matrix of Deep CNN model.

DownLoad: Full-Size Img PowerPoint

4.3.   Performance of GastroFuse-Net

Figures 9(a) and 9(b) show the change in accuracy and loss during training of GastroFuse-Net. The results demonstrate a smooth and steady progression of accuracy, with a maximum training accuracy of 99.9% and a validation accuracy of 82.4%, which indicates that the model effectively learned from the combined features and achieved remarkable accuracy, surpassing the individual networks. Moreover, the training loss for GastroFuse-Net showed a significant decrease to 0.0342, while the validation loss decreased to 0.235. This decline in loss signifies that the model has gained a better understanding of the underlying patterns in the data, reinforcing its ability to make precise predictions.

Figure 9.  Plots obtained while training of GastroFuse-Net (a) Accuracy Plot and, (b) Loss Plot.

DownLoad: Full-Size Img PowerPoint

The confusion matrix and performance metrics obtained for GastroFuse-Net, as shown in Figure 10 and Table 4, reveal its improved performance, correctly classifying 98.5% of the samples, surpassing the accuracy achieved by both Deep and Shallow CNN models. However, some challenges persist, particularly in distinguishing between the Polyps and Ulcerative Colitis classes, where a few samples are misclassified interchangeably. Additionally, there were three instances, where samples of Esophagitis were incorrectly predicted as Z-line class. Furthermore, minimal misclassifications were observed between the Dyed Lifted Polyps and Dyes Resection Margins classes. Despite these minor misclassifications, GastroFuse-Net demonstrates significant progress in accurately classifying most samples, showcasing its potential for enhanced performance over individual CNN models. These findings underscore the effectiveness of feature fusion in improving classification accuracy and highlight the importance of further addressing specific misclassification patterns to enhance the model's capabilities.

Figure 10.  Confusion matrix of GastroFuse-Net.

DownLoad: Full-Size Img PowerPoint

Table 4.  Performance Parameters of GastroFuse-Net.

Class Name	Precision	Recall	F1-score	Specificity	MCC
Dyed Lifted Polyps	0.985	0.985	0.985	0.997	0.982
Dyed Resection Margins	0.985	0.985	0.985	0.997	0.982
Esophagitis	0.985	0.985	0.985	0.997	0.982
Cecum	1.000	0.985	0.992	1.000	0.991
Pylorus	1.000	0.990	0.995	1.000	0.994
Z-line	0.975	0.985	0.980	0.996	0.977
Polyps	0.975	0.980	0.977	0.996	0.974
Ulcerative Colitis	0.975	0.985	0.980	0.996	0.977
Average	0.985	0.985	0.984	0.997	0.982

---

 | Show Table

DownLoad: CSV

4.4.   Comparison with the state-of-the-art

The proposed model has been compared with the state-of-the-art shown in Table 5. The proposed model achieved the highest precision of 0.98, recall of 0.98, F1-score of 0.99, specificity of 1.00, MCC of 0.98, and accuracy of 0.985 as compared to the other models in the literature.

Table 5.  State-of-the-art comparison of the proposed model.

Reference	Precision	Recall	F1-score	Specificity	MCC	Accuracy
[30]	0.8475	0.84	0.8475	0.97	0.82	0.96
[20]	0.82	0.82	0.82	0.97	0.80	0.95
[37]	0.97	0.97	0.97	-	-	0.973
[32]	0.89	0.89	0.89	0.99	0.87	0.984
[38]	0.96	0.96	0.96	-	-	0.96
[39]	-	-	-	-	-	0.97
[15]	0.94	-	0.93	-	-	0.94
[35]	-	-	-	-	0.9298	0.96
[36]	0.98	0.98	-	-	-	0.9801
Proposed Shallow CNN	0.902	0.900	0.900	0.986	0.886	0.903
Proposed Deep CNN	0.960	0.960	0.960	0.994	0.954	0.962
Proposed GastroFuse-Net	0.985	0.985	0.984	0.997	0.982	0.985

---

 | Show Table

DownLoad: CSV

5.   Discussions

The results obtained demonstrate that the proposed models are effective in the classification of gastrointestinal abnormalities. It is worth noting that GastroFuse-Net achieved a substantially higher accuracy rate of 98.5% than both Deep CNN and Shallow CNN. The comparison of the performance parameters obtained for each proposed model is shown in Figure 11. It can be seen in this figure that with feature fusion in GastroFuse-Net, the capability of the model to classify the images increased. For the shallow CNN model, accuracy of 0.9, MCC of 0.89, F1-score of 0.9, and specificity of 0.98. recall of 0.9 and precision of 0.9 are achieved. By increasing layers in the Deep CNN model, accuracy increased to 0.96, MCC to 0.95, F1-score to 0.96, specificity to 0.99, recall to 0.96 and precision to 0.96. However, when the combined features of both models were fused in GastroFuse-Net, the best results were obtained with accuracy increasing to 0.985, MCC increased to 0.98, F1-score increased to 0.9, specificity to 1.00, recall to 0.98 and precision increased to 0.99.

Figure 11.  Performance comparison of the proposed models.

DownLoad: Full-Size Img PowerPoint

It is worth mentioning that GastroFuse-Net demonstrated an exceptional capacity to differentiate between classes, as indicated by its elevated precision, recall, and specificity. The enhanced performance of GastroFuse-Net can be explained by the strategic concatenation of features, which effectively merges the advantageous attributes of shallow and deep models. The Shallow CNN exhibited competence in comprehending broad contexts, whereas the Deep CNN demonstrated an exceptional performance in extracting intricate features.

Furthermore, a runtime analysis showing the amount of time needed to train each suggested model is given in Table 6. The Deep CNN model had a training duration of 22,926.2 seconds (about 6.4 hours), the Shallow CNN model took around 10,383.4 seconds (approximately 2.88 hours), and GastroFuse-Net had a training time of roughly 16,028.1 seconds (approximately 4.45 hours). Although GastroFuse-Net demonstrated improved performance, it reached this level of accuracy within a training duration that falls between the more time-consuming Deep CNN model and the quicker Shallow CNN model. This highlights efficacy of GastroFuse-Net as it provides a convincing trade-off between excellent classification accuracy and comparatively short training times in contrast to Deep CNN and Shallow CNN.

Table 6.  Run Time analysis of different proposed models.

Model Name	Time taken to train the model
Deep CNN	22,926.2 s
Shallow CNN	10,383.4 s
GastroFuse-Net	16,028.1 s

---

 | Show Table

DownLoad: CSV

6.   Conclusions

Deep learning has demonstrated its efficacy as a beneficial tool for analyzing medical images in gastroenterology. Medical practitioners frequently depend on their specialized knowledge and years of practice to analyze images, resulting in subjective assessments and possible delays in diagnosing. Deep learning systems can address these constraints by offering unbiased and efficient categorizations of gastrointestinal disorders from endoscopic images.

Prompt and specific diagnosis is critical in medical practice, as it could substantially affect patient results. Mistakes in diagnosis can result in misguided treatments and extended hospital stays. Employing computer-aided diagnosis algorithms can lower the load of gastroenterologists and mitigate the ability for misdiagnosis. Accurate identification of gastrointestinal ailments, inclusive of colon cancer and polyps, is mainly essential because of their great effect on world health.

The GastroFuse-Net is a data-driven deep convolutional neural network (DCNN) method developed to deal with the difficulty of detecting gastrointestinal issues. The model's feature fusion method, which integrates the advantage of shallow and deep CNN models, achieved exceptional performance. It correctly categorized 98.5% of the samples with high precision, recall, specificity, F1-score, and Matthews Correlation Coefficient (MCC) values of 0.985, 0.985, 0.984, 0.997, and 0.982, respectively.

Although the outcomes are promising, there are possibilities for future enhancement and research. Possible improvements include expanding the training dataset, examining higher-decision pictures, and modifying the model for use in different medical imaging domains. Gastric endoscopists might consider the suggested algorithm a second opinion, particularly when it is difficult to get specialized medical support. GastroFuse-Net demonstrates promising abilities in helping healthcare practitioners in the appropriate and quick diagnosis of gastrointestinal issues. Additional clinical data and the exploration of progressive algorithms will improve the successful incorporation of this technology into clinical exercise.

Use of AI tools declaration

The authors have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgment

The authors present their appreciation to King Saud University for funding the publication of this research through Researchers Supporting Program (RSPD2024R809), King Saud University, Riyadh, Saudi Arabia.

Conflict of interest

Frank Werner, the Guest Editor for the journal of MBE, was not involved in the editorial review or the decision to publish this article. All authors declare that there is no conflict of interest.