Loading [MathJax]/jax/output/SVG/jax.js
Research article Special Issues

Predicting odor profile of food from its chemical composition: Towards an approach based on artificial intelligence and flavorists expertise

  • The two authors share first authorship
  • Odor is central to food quality. Still, a major challenge is to understand how the odorants present in a given food contribute to its specific odor profile, and how to predict this olfactory outcome from the chemical composition. In this proof-of-concept study, we seek to develop an integrative model that combines expert knowledge, fuzzy logic, and machine learning to predict the quantitative odor description of complex mixtures of odorants. The model output is the intensity of relevant odor sensory attributes calculated on the basis of the content in odor-active comounds. The core of the model is the mathematically formalized knowledge of four senior flavorists, which provided a set of optimized rules describing the sensory-relevant combinations of odor qualities the experts have in mind to elaborate the target odor sensory attributes. The model first queries analytical and sensory databases in order to standardize, homogenize, and quantitatively code the odor descriptors of the odorants. Then the standardized odor descriptors are translated into a limited number of odor qualities used by the experts thanks to an ontology. A third step consists of aggregating all the information in terms of odor qualities across all the odorants found in a given product. The final step is a set of knowledge-based fuzzy membership functions representing the flavorist expertise and ensuring the prediction of the intensity of the target odor sensory descriptors on the basis of the products' aggregated odor qualities; several methods of optimization of the fuzzy membership functions have been tested. Finally, the model was applied to predict the odor profile of 16 red wines from two grape varieties for which the content in odorants was available. The results showed that the model can predict the perceptual outcome of food odor with a certain level of accuracy, and may also provide insights into combinations of odorants not mentioned by the experts.

    Citation: N. Mejean Perrot, Alice Roche, Alberto Tonda, Evelyne Lutton, Thierry Thomas-Danguin. Predicting odor profile of food from its chemical composition: Towards an approach based on artificial intelligence and flavorists expertise[J]. Mathematical Biosciences and Engineering, 2023, 20(12): 20528-20552. doi: 10.3934/mbe.2023908

    Related Papers:

    [1] Zhi-Jie Wang, Hong-Rui Sun . Normalized solutions for Kirchhoff equations with Choquard nonlinearity: mass Super-Critical Case. Communications in Analysis and Mechanics, 2025, 17(2): 317-340. doi: 10.3934/cam.2025013
    [2] Yangyu Ni, Jijiang Sun, Jianhua Chen . Multiplicity and concentration of normalized solutions for a Kirchhoff type problem with L2-subcritical nonlinearities. Communications in Analysis and Mechanics, 2024, 16(3): 633-654. doi: 10.3934/cam.2024029
    [3] Xueqi Sun, Yongqiang Fu, Sihua Liang . Normalized solutions for pseudo-relativistic Schrödinger equations. Communications in Analysis and Mechanics, 2024, 16(1): 217-236. doi: 10.3934/cam.2024010
    [4] Qi Li, Yuzhu Han, Bin Guo . A critical Kirchhoff problem with a logarithmic type perturbation in high dimension. Communications in Analysis and Mechanics, 2024, 16(3): 578-598. doi: 10.3934/cam.2024027
    [5] Shengbing Deng, Qiaoran Wu . Existence of normalized solutions for the Schrödinger equation. Communications in Analysis and Mechanics, 2023, 15(3): 575-585. doi: 10.3934/cam.2023028
    [6] Wenqian Lv . Ground states of a Kirchhoff equation with the potential on the lattice graphs. Communications in Analysis and Mechanics, 2023, 15(4): 792-810. doi: 10.3934/cam.2023038
    [7] Yan-Fei Yang, Chun-Lei Tang . Positive and sign-changing solutions for Kirchhoff equations with indefinite potential. Communications in Analysis and Mechanics, 2025, 17(1): 159-187. doi: 10.3934/cam.2025008
    [8] Henryk Żołądek . Normal forms, invariant manifolds and Lyapunov theorems. Communications in Analysis and Mechanics, 2023, 15(2): 300-341. doi: 10.3934/cam.2023016
    [9] Panyu Deng, Jun Zheng, Guchuan Zhu . Well-posedness and stability for a nonlinear Euler-Bernoulli beam equation. Communications in Analysis and Mechanics, 2024, 16(1): 193-216. doi: 10.3934/cam.2024009
    [10] Chang-Jian Wang, Jia-Yue Zhu . Global existence and uniform boundedness to a bi-attraction chemotaxis system with nonlinear indirect signal mechanisms. Communications in Analysis and Mechanics, 2023, 15(4): 743-762. doi: 10.3934/cam.2023036
  • Odor is central to food quality. Still, a major challenge is to understand how the odorants present in a given food contribute to its specific odor profile, and how to predict this olfactory outcome from the chemical composition. In this proof-of-concept study, we seek to develop an integrative model that combines expert knowledge, fuzzy logic, and machine learning to predict the quantitative odor description of complex mixtures of odorants. The model output is the intensity of relevant odor sensory attributes calculated on the basis of the content in odor-active comounds. The core of the model is the mathematically formalized knowledge of four senior flavorists, which provided a set of optimized rules describing the sensory-relevant combinations of odor qualities the experts have in mind to elaborate the target odor sensory attributes. The model first queries analytical and sensory databases in order to standardize, homogenize, and quantitatively code the odor descriptors of the odorants. Then the standardized odor descriptors are translated into a limited number of odor qualities used by the experts thanks to an ontology. A third step consists of aggregating all the information in terms of odor qualities across all the odorants found in a given product. The final step is a set of knowledge-based fuzzy membership functions representing the flavorist expertise and ensuring the prediction of the intensity of the target odor sensory descriptors on the basis of the products' aggregated odor qualities; several methods of optimization of the fuzzy membership functions have been tested. Finally, the model was applied to predict the odor profile of 16 red wines from two grape varieties for which the content in odorants was available. The results showed that the model can predict the perceptual outcome of food odor with a certain level of accuracy, and may also provide insights into combinations of odorants not mentioned by the experts.



    After the novel coronavirus pneumonia (Corona Virus Disease 2019, COVID-19) was first discovered in November 2019, it caused extremely high morbidity and mortality worldwide in a short period of time [1]. Radiological examination can be used for COVID-19 screening, that is, radiologists perform chest imaging (such as chest X-ray or computed tomography imaging) of the patient, and can look for visual indicators related to the COVID-19 virus infection [2]. Early studies have found that once COVID-19 patients are infected, there will be abnormalities in lung imaging [3,4,5]. For example, Huang et al. found in their research that most COVID-19 positive cases showed bilateral radiological abnormalities in CXR images. These abnormalities are most common with ground glass opacity, patchy shadows, warped spreading and lung consolidation. Therefore, researchers believe that lung imaging can be used as the main tool for COVID-19 screening in the hardest-hit areas [6].

    In recent years, artificial intelligence (AI) technology represented by convolutional neural networks (CNN) has made great achievements in computer vision. As the CT radiography mode has been proved to have higher sensitivity and specificity than the RT-PCR detection method, many researchers have begun to use CT and CRX image data sets to train automatic classification AI algorithms. Ozturk et al. [7] proposed a deep model that uses DarkNet as a classifier and has an end-to-end architecture. However, the number of data sets needs to be increased to make the model more robust and accurate. Y. Pathak et al. [8] used transfer learning methods to classify patients infected with COVID-19. The author introduces the smooth loss function top-2 with cost-sensitive attributes to process noise and unbalanced data sets, and uses a 10-fold cross-validation method to prevent overfitting. Narin et al. [9] used three advanced deep learning models (ResNet50, Inception v3 and Inception-ResNetV2) to classify CRX images (50 COVID-19 images vs. 50 non-COVID-19 images). In June 2021, Adel Oulefki et al. proposed a method to automatically segment and measure COVID-19 lung infection area [10]. In November 2021, Chen Zhao et al. used chest CT imaging features for lung segmentation and COVID-19 automatic detection [11]. In December 2021, Juan Juan He et al. used the evolutionary countermeasure network with gradient punishment to realize the segmentation task of COVID-19 infection [12]. In December 2021, Nan Mu et al. applied progressive global perception and local polishing network to the segmentation task of pulmonary infection in COVID-19 CT image [13]. In 2021, Qimao proposed the intelligent immune clonal optimization algorithm for pulmonary nodule classification [14]. In February 2022, X. Liu et al. used scribble annotation to segment the infected area of COVID-19 CT image [15].

    Main contributions:

    (1) This research proposes a two-level deep network model, which mainly realizes the classification of lung CT images. The first level of the model uses convolutional neural networks to locate lung regions in lung CT images. The second level of the model uses the capsule network to classify and predict the segmented images.

    (2) SegNet-Lung segmentation model is used to segment and preprocess the lung region of lung CT image (the segmentation model in this study is named SegNet-Lung model) in order to extract shape features such as color and texture from lung CT image. At the same time, the unlabeled lung CT images are predicted and labeled as the training/test data set in the classification processing stage.

    (3) The CapsNet-COVID19 capsule network classification model is established to classify and reconstruct the segmented lung CT images, and the column diagram is used to visualize the prediction results. By stopping early and increasing the amount of training data, the over fitting of the model caused by excessive iterative training in the training process is avoided. The confusion matrix, accuracy, boundary loss, reconstruction loss and total loss are used to quantitatively evaluate the capsule network model.

    (4) Our classification method is suitable for medical image classification with complex background, low recognition rate, blurred boundaries and large image noise. The accuracy of our method is 84.291% on the test set and 100% on the training set. We believe that this classification method is of great value for monitoring and controlling the growth of patients in COVID-19 infected areas.

    All human lung CT image data used in this research are derived from the 2019 Novel Coronavirus Information Database (2019nCoVR) provided on the website of China National Bioinformatics Center [16]. The lung CT image data in this database is provided by the China Chest CT Image Investigation Association (CC-CCII), so 2019nCoVR has high professional and practical value. There are three main types of lung CT images in 2019nCoVR: The Novel coronavirus pneumonia data set caused by the SARS-CoV-2 virus, the common pneumonia data set, and the normal lung data set. The amount of data in 2019nCoVR is relatively large. In view of the limitations of the experimental hardware equipment in this research, we selected 29,278 images with clear and obvious lung texture characteristics from the information database as the data set of this research. An example of lung CT images in the dataset is shown in Figure 1.

    Figure 1.  Example of lung CT images. From left to right: COVID-19, common pneumonia, and normal lung CT images.

    The main features of lung CT images of COVID-19 patients are early unilateral or bilateral lungs with localized inflammatory infiltration, which is mainly manifested as patches, masses, segments or subpleural subsegmental ground glass shadow. In the late stage, consolidation is dominant in the lungs [17]. Common pneumonia is generally divided into lobar pneumonia and lobular pneumonia. The early stage of lobar pneumonia is mainly manifested as a ground-glass-like density shadow in a certain lung lobe with fuzzy edges. Lobular pneumonia is common in the middle and lower parts of the lungs. The local bronchial vascular bundles are thickened, and there are blurry patches of varying sizes. The texture of the lungs of normal people is very clear, and there is no thickening, disorder or abnormal lines. At the same time, there are no obvious high-density or low-density dense shadows and nodules in both lungs, and no pathological manifestations such as hyperplasia, cavitation, calcification and exudation. The CT images of common pneumonia and COVID-19 overlap to a certain extent, and it is often difficult to make an accurate judgment of these two types of pneumonia by the human eye alone. Therefore, it is necessary to increase the diagnostic accuracy of COVID-19 with the help of image analysis technology.

    This research uses a deep network model to analyze lung CT images to achieve automatic segmentation and classification of a large number of lung CT images (three main types). First, use an image segmentation model similar to U-NET [18] to segment the lung area of the lung CT image (the segmentation model in this research is named SegNet-Lung segmentation model). This model can extract shape features such as color and texture from lung CT images. Then, a capsule network model [19] was established to classify and predict the segmented lung CT images (the classification model in this research is named CapsNet-COVID19 classification model). The specific process is as follows:

    (1) Train the segmentation model SegNet-Lung segmentation model using the labeled lung CT image set. Then, the trained SegNet-Lung segmentation model is used to segment the lung CT images to be classified.

    (2) Use a self-editing program to unify the format of the segmented lung CT images: (a) The edge of the lung region in the image predicted by SegNet-Lung segmentation model is outlined with red lines; (b) Extract the lung region of the outlined lung CT image and hide other irrelevant regions; (c) Convert the segmented lung region image format into the input format available for the CapsNet-COVID19 classification model.

    (3) The above-mentioned image set is divided into train set, validation set and test set according to an appropriate ratio. Through training the CapsNet-COVID19 classification model, the prediction of image classification is finally realized.

    Figure 2.  Classification process of lung CT images.

    The data sets of lung CT images used in this research include: COVID-19, common pneumonia (CP) and normal lungs (Normal). The numbers of them are 9420, 9858 and 10,000 respectively. Set the resolution of the image set to 128 × 128 and the format to.tif. If the lung CT images are directly classified, the image classification accuracy will be reduced because the images are interfered by the external environment when they are taken. Therefore, it is necessary to segment the lung area of the lung CT image. We have about 2532 images of COVID-19 lung CT masks that have been annotated by experts (as shown in Figures 3 and 4). Then, they are used as the train set and validation set of the SegNet-Lung segmentation model, and the unlabeled lung CT images to be classified are used as the test set. Among them, the training set and validation set (90%) are used to train the SegNet-Lung segmentation model to determine the network parameters of the segmentation model, and the test set (10%) is used to test and predict the recognition rate and generalization of the trained segmentation model (Table 1). The test set is used to test and predict the segmentation performance of the trained SegNet-Lung segmentation model. Finally, the trained SegNet-Lung segmentation model is used for segmentation prediction and labeling of unlabeled lung CT images. Therefore, the labeled lung CT images can be used as the training/test data set of CapsNet-COVID19 classification model in the next step.

    Figure 3.  Lung CT image masks (lung area) manually marked by experts.
    Figure 4.  Lung CT image masks (lung area) manually marked by experts.
    Table 1.  The number of images of various data sets.
    Training/Validation Data Test Set Total (Labeled Images)
    2279 (90%) 253 (10%) 2532

     | Show Table
    DownLoad: CSV

    The SegNet-Lung segmentation model used in this study is improved on the basis of the U-NET model. This model uses Nconvolution instead of MaxPool and adds a random dropout layer with a parameter of 0.5. At the same time, the Sigmoid function is used in the output convolutional layer. During model training, the batch_size parameter is set to 64, and the convolutional layer uses a 3 × 3 filter and activation function ELU, which can reduce the feature dimension of the subsequent pooling layer and avoid overfitting. The filter sizes used by the largest pooling layer and upsampling layer are 3 × 3 and 2 × 2. After the convolutional layers are merged, the network structure of the model will be flattened. The SegNet-Lung segmentation model structure is shown in Figure 5.

    Figure 5.  The structure of the SegNet-Lung segmentation model.

    The structure of nconvolution of covseg net model is convolution + batchnormalization + ELU. Batch normalization (BN) is used in the convolution process, which can forcibly pull the more and more biased data distribution in model training back to the standard distribution through certain standardized means. In this way, the active input value can fall in the area where the nonlinear function is sensitive to the input, and the output of the network will not be very large. Therefore, using the structure of nconvolution instead of maxpool can speed up the training and convergence of the network, control the gradient explosion, prevent the gradient from disappearing and prevent over fitting. At the same time, the concept structure is introduced into covseg net model, which stacks the convolution layer of 1 × 1, 3 × 3 and 5 × 5 and the pooling layer of 3 × 3 together. On the one hand, it increases the width of the network, on the other hand, it increases the adaptability of the network to scale.

    We use Focal-Tversky Unet model and nn-Unet model reported in isbi as comparison models (Table 2). Through comparison, it is found that in the focus segmentation task of COVID-19 CT image, the evaluation indexes such as Dice and Precision of COVSeg-NET model proposed in this study are better than the other two models. Especially for the situation that the number of samples in medical images is small and the problem space is small, COVSeg-NET model can achieve the goal of medical image segmentation by using a variety of ways of data set amplification and preprocessing.

    Table 2.  Performance index values of the segmentation results of the COVSeg-NET model: Dice value, precision rate, and recall rate. And compare with other methods.
    Method Dice Precision Recall
    COVSeg-NET 55.5% 83.3% 44.5%
    Focal-Tversky Unet [20] 48.4% 48.3% 61.9%
    nn-Unet [21] 4.5% 75.4% 48.6%

     | Show Table
    DownLoad: CSV

    The trained SegNet-Lung segmentation model is used to segment all lung CT images to be classified (the remaining 29,278 lung CT images without segmentation annotations), and the segmentation marks are automatically labeled. Then use a self-editing program to extract the lung area in the image. Some examples of image annotation are shown in Figure 6.

    Figure 6.  An example of the segmented image set predicted by the trained SegNet-Lung segmentation model. The images in the first line are the original CT images of the lungs. The images of the second line are the masks of the lung area predicted by the SegNet-Lung segmentation model. The images in the third line are the lung regions marked with red lines using a self-programmed program. The images in the fourth line are the segmented lung region images.

    Next, the CapsNet-COVID19 classification model will be used to classify the pre-segmented lung CT images. CapsNet-COVID19 has made many adjustments and improvements to the CapsNet model.

    In the CapsNet model [19], the capsule can contain the attributes and characteristics of the entity. The training goal of the capsule is to capture the possibility of features and their variants, not the characteristics of specific variants. Therefore, the capsule can be used not only to detect image features, but also to learn image variants. In this way, the same capsule can detect the same object category with different directions.

    Figure 7.  The propagation process between capsule units in the CapsNet model.

    The input ui of the capsule is a vector, apply a transformation matrix Wij to get the output of the upper layer of the capsule. For an m × k matrix, the k-dimensional ui is converted to the m-dimensional ˆuji((m×k)×(k×1)m×1), and then the weight cij is used to calculate the weighted sum sij:

    ˆuji=Wijuisj=icijˆuji (1)

    cij is the coupling coefficient trained by iterative dynamic routing, and jcij is designed as a sum. This model does not use the ReLU function, but uses a squashing function to scale the vector between 0 and 1:

    vj=sj21+sj2sjsj (2)

    The squashing function sets the small vector as a zero vector and the long vector as a unit vector:

    vjsjsj for sj is short vjsjsj for sj is long  (3)

    Dynamic routing is located between the capsule layers, that is, between PrimaryCaps and DigitCaps in the structure diagram. This algorithm describes the calculation method of the connection weight between the th layer and the (+1)th layer capsule, and calculates the correlation score bij according to the similarity:

    bijˆujivj (4)

    The coupling coefficient cij is calculated by the softmax function of bij:

    cij=expbijkexpbik (5)

    After multiple iterations and updates, the obtained bij is more accurate:

    bijbij+ˆujivj (6)

    The 29,278 lung CT images processed by lung region segmentation can be used as the train/test data set of the CapsNet-COVID19 classification model (as shown in Figure 8 and Table 3). In order to train and test the CapsNet-COVID19 classification model, 20,000 images of the data set are used as the train set, and the remaining 9278 images are used as the test set and validationset, and the image size is set to 32 × 32.

    Figure 8.  The lung CT image after segmentation prediction (part of the example), which is then used for training in the CapsNet-COVID classification model.
    Table 3.  Number of images in different data sets in CapsNet-COVID19 classification model.
    Train Set Validation Set Test Set
    20,000 (68.3%) 3428 (11.7%) 5850 (20%)

     | Show Table
    DownLoad: CSV

    Since lung CT images are more difficult to identify and classify than ordinary images, the CapsNet-COVID19 classification model needs to be continuously modified during training to achieve better classification results. Tables 4 and 5 show the setting of the parameters of the CapsNet-COVID19 classification model.

    Table 4.  Parameter settings of CapsNet-COVID19 classification model.
    Name Filter Size Number of Filters CapsNet-COVID19
    Conv1 layer 9 × 9 256
    Conv2 layer 6 × 6 64
    Conv2 layer dropout value 0.7
    Caps1 layer 5 × 5 16
    Caps2 layer 32
    Learning rate 0.0001
    Number of routes 2

     | Show Table
    DownLoad: CSV
    Table 5.  Other parameter values of CapsNet-COVID19 classification model.
    Name CapsNet-COVID19
    Input image pixel size 32 × 32
    Transformation matrix Wij 16 × 32
    PrimaryCaps layer capsule dimension 16-D
    Number of PrimaryCaps layer capsules 32
    DigitCaps layer capsule shape 32 × 3
    Number of categories 3
    Does the image to be classified have segmentation preprocessing Segmentation preprocessing

     | Show Table
    DownLoad: CSV

    The classification process of the CapsNet-COVID19 classification model is as follows:

    (1) Input the segmented lung CT image training data set into the CapsNet-COVID19 classification model. The image set enters Conv1 first, passes through 256 9 × 9 kernel convolutions, and outputs feature maps of 256 channels (Figure 9);

    Figure 9.  CapsNet-COVID19 classification model structure.

    (2) Then the data enters the PrimaryCaps layer. After the kernel is 5 × 5 convolution, the image size is reduced from 24 × 24 to 10 × 10. The output shape of the PrimaryCaps layer is 10 × 10 × 16 × 16;

    (3) Then the data enters the DigiCaps layer, and the 16-dimensional capsule is converted into a 32-dimensional capsule using the 16 × 32 transformation matrix Wij. The output of the DigiCaps layer is 3 × 32, which means 3 types of 32-dimensional capsules;

    (4) Finally, the data is input to the squashing nonlinear function, and the vector vj. is output. Take each vector vj. as a capsule of class j, where the length of vj. represents the probability of the image being recognized as each class. Finally, after the fully connected FC layer, a 32 × 32 image can be reconstructed. The reconstructed image is shown in Figure 10.

    Figure 10.  The reconstructed CT images of the lung.

    The hardware and software configuration used in the lung CT image classification experiment is as follows:

    (1) Hard disk: 16 GB RAM, 1 TB + 256 GB;

    (2) Graphics processor: NVIDIA Geforce GTX 1070 8 G;

    (3) Central processing unit: Intel Core i7-6700HQ processor;

    (4) Operating system: Ubuntu 16.04 operating system;

    (5) Software: Python, Tensorflow, Cuda8.0, NumPy, docopt, Sklearn, Matplotlib.

    The training of the CapsNet-COVID19 classification model on the GPU takes about 2 hours and 24 minutes. The confusion matrix can be used to evaluate the classification effect of the CapsNet-COVID19 classification model, as shown in Figure 11 and Table 6. Among them, True Positive (TP): predict the number of positive cases predicted as positive; False Negative (FN): predict the number of positive cases predicted as negative; False Positive (FP): predict the number of negative cases predicted as positive; True Negative (TN): predict the number of negative cases as negative cases. In a predictive classification model, it is usually hoped that the more accurate the predictive result, the better. That is, the larger the observed and predicted values of TP and TN, the better, on the contrary, the smaller the observed values of FP and FN, the better.

    Figure 11.  Confusion matrix obtained by using CapsNet-COVID19 classification model to classify lung CT images.
    Table 6.  Confusion matrix obtained by using CapsNet-COVID19 classification model to classify lung CT images.
    COVID-19 CP Normal
    COVID-19 0.956 0.0313 0.013
    CP 0.187 0.729 0.084
    Normal 0.114 0.043 0.844

     | Show Table
    DownLoad: CSV

    In addition, the evaluation indicators of the CapsNet-COVID19 classification model also include accuracy, boundary loss, reconstruction loss, and total loss. Among them, the accuracy rate refers to the ratio of the correct recognition of lung CT image categories to the total number of lung CT images:

     Accuracy = correctly identified images Totalnumberofimages (7)

    Margin Loss: For each category c that appears in the lung CT image, the capsule uses a separate loss function Lc:

    Lc=Tcmax(0,m+vc)2+λ(1Tc)max(0,vcm)2 (8)

    Among them, if there are Tc=1,m+=0.9, and m=0.1 for the object of the class, the downward weighting of λ (the default is 0.5) prevents the initial learning and shrinks the activity vectors of all classes. The length of the instantiation output vector represents the probability of the existence of each capsule entity. Only when there is a lung CT image type in the input image, this type c has the longest vector output.

    Reconstruction Loss: It is mainly used to promote the encoding of the input images by the DigitCaps layer:

    R=mean(( decoded images  origin images )2) (9)

    Among them, R represents reconstruction loss, decoded images represents the encoded images, and origin images represents the original images.

    Total Loss is the sum of boundary loss and reconstruction loss:

    F=( Margin Loss )+λ( Reconstruction Loss ) (10)

    Among them, F is the total loss, λ = 0.392, therefore, the boundary loss should always dominate the reconstruction loss. If the reconstruction loss is greater than the total loss, the model will try to accurately match the output image with the input image of the train data set, resulting in an overfitting of the model and the train data.

    Due to the large noise interference of the lung CT image data set used in this research, the feature difference between lung CT image categories is not obvious enough, and the image recognition is low. Therefore, when the lung CT image classification model is initially trained, the model is prone to overfitting during training.

    The reasons for the over fitting of the model are as follows: (1) the amount of data in the training set is not large enough, and the training set cannot evaluate the whole data distribution, resulting in the over fitting of the model; (2) The classification model is over iterative trained in the training process, which leads to the model fitting the noise in the data and the non representative characteristics in the samples, which also leads to the over fitting of the training model; (3) The noise interference of image data set is large, the feature difference between lung CT image categories is not obvious, and the image recognition degree is low, which makes the medical image more difficult to recognize than ordinary images, resulting in over fitting of the model.

    In order to avoid or reduce the over fitting phenomenon of CapsNet-COVID19 classification model, the main methods adopted in this study are as follows: first, we widely collect open data sets to ensure that the number of images in the training set is as large as possible, so as to prevent the over fitting problem. Second, in the process of training CapsNet-COVID19 classification model, when the number of iterations is set to be large and the performance of the model in the validation set/test set begins to decline, early stop measures shall be implemented to prevent over fitting caused by continuous training. At the same time, save a copy of the network model corresponding to the current number of iterations and use it as the final/optimal network model.

    As shown in Figure 12, during the training process of the CapsNet-COVID19 classification model, the accuracy of the train set and the test set continue to increase as the number of iterations increases. Among them, the red polyline represents the change of the accuracy of the train set, and the blue polyline represents the change of the accuracy of the test set. As the number of iterations increased, the accuracy of the train set approached 100%. In the later stage, the accuracy of the test set fluctuated between 80% and 98%, and the highest value reached 98% accuracy at 85k step. At the same time, as the number of iterations increases, the loss value gradually decreases and tends to stabilize. At the 130 k step position, an early stop was adopted to prevent over-fitting. It should be noted that the CapsNet-COVID19 classification model gives a separate boundary loss for each capsule of the characterization category, and the total loss is the sum of the losses of all categories. The overall evaluation index of the CapsNet-COVID19 classification model is: the accuracy of the train set is 100%, the accuracy of the test set is 84.291%, and the total loss value is 11.066%. It can be seen that the CapsNet-COVID19 classification model is a robust model (Table 7).

    Figure 12.  Edge loss, reconstruction loss and total loss of the CapsNet-COVID19 classification model (the blue polyline represents the corresponding value of the training set, and the red polyline represents the corresponding value of the test set). The abscissa of the graph is the number of iterations, and the ordinate is the index value.
    Table 7.  Loss value, training set accuracy and test set accuracy of CapsNet-COVID19 classification model.
    Method Loss Train accuracy Test accuracy
    CapsNet-COVID19 11.066% 100% 84.291%

     | Show Table
    DownLoad: CSV

    We compare the performance of the CapsNet-COVID19 classification model with several better lung CT image classification methods, as shown in Table 8. The CapsNet-COVID19 classification model has a clear advantage in accuracy over the other models mentioned in the table. The number of images that the CapsNet-COVID19 classification model can process at the same time also has greater advantages than other methods. Our model can meet the requirements of massive lung CT image analysis and improve the efficiency of image analysis.

    Table 8.  Comparison of accuracy of different classification methods.
    Method Type Number Accuracy
    CapsNet-COVID19 Model Chest CT COVID-19: 9420
    CP: 9858
    Normal: 10,000
    84.291% (Test set) 100% (Train set)
    169-layer DenseNet [22] Chest CT COVID-19: 449
    CP: 495
    Normal: 425
    83.3%
    InceptionV3 [22] Chest CT COVID-19: 449
    CP:495
    Normal: 425
    82.7%
    ResNet50 [23] Chest CT COVID-19: 349
    Non-COVID-19: 463
    77.4%
    M-Inception [24] Chest CT COVID-19: 44
    Normal: 55
    73.1%

     | Show Table
    DownLoad: CSV

    Using CapsNet-COVID19 classification model can achieve classification prediction. Input the lung CT image into the CapsNet-COVID19 classification model, and then the type and Softmax value of the image can be predicted. As shown in Figure 13. The classification method of this research is suitable for complicated medical image classification problems such as complex background, low recognition degree, blurred boundary and large image noise.

    Figure 13.  Using CapsNet-COVID19 classification model to predict the category of lung CT images.

    In this study, a lung CT image classification method based on improved deep learning model is proposed. The main work and results are as follows: firstly, the SegNet-Lung segmentation model is used to segment and predict 29,278 lung CT images. Among them, SegNet-Lung segmentation model uses the structure of nconvolution (convolution2d + batchnormalization + ELU) instead of maxpool, which can speed up the training and convergence of the network, control the gradient explosion, prevent the gradient from disappearing and prevent over fitting. Secondly, the CapsNet-COVID19 classification model is used to classify and predict 29,278 segmented and preprocessed lung CT images with high accuracy. Among them, the CapsNet-COVID19 classification model improves the capsule shape of the primarycaps layer and digicaps layer in the capsule classification model. Thirdly, the evaluation method of classification model is proposed, including accuracy, boundary loss, reconstruction loss, total loss and other indicators, and the over fitting problem is solved by using early stop technology and increasing training data set. The accuracy of the test set of the classification model proposed in this study is higher than that of 169 layer densenet, inception V3, resnet50, m-inception and other methods, and once the model is trained, it can realize the rapid prediction of lung CT test images. The classification model improves the analysis efficiency of lung CT images and can meet the requirements of massive image analysis.

    We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.



    [1] D. E. Hornung, M. P. Enns, The contributions of smell and taste to overall intensity: A model, Percept. Psychophys., 39 (1986), 385–391. https://doi.org/10.3758/BF03207066 doi: 10.3758/BF03207066
    [2] B. Lorrain, J. Ballester, T. Thomas-Danguin, J. Blanquet, J. M. Meunier, Y. Le Fur, Selection of potential impact odorants and sensory validation of their importance in typical chardonnay wines, J. Agri. Food Chem., 54 (2006), 3973–3981.
    [3] C. Flavián, S. Ibáñez-Sánchez, C. Orús, The influence of scent on virtual reality experiences: The role of aroma-content congruence, J. Business Res., 123 (2021), 289–301. https://doi.org/10.1016/j.jbusres.2020.09.036 doi: 10.1016/j.jbusres.2020.09.036
    [4] R. Hudson, Editorial note, Chem. Senses, 25 (2000), 693–693. https://doi.org/10.1046/j.1442-9993.2000.00002.x doi: 10.1046/j.1442-9993.2000.00002.x
    [5] H. Stone, R. N. Bleibaum, H. A. Thomas, Sensory evaluation practices, Academic press, 2012.
    [6] G. Ferrari, O. Lablanquie, R. Cantagrel, J. Ledauphin, T. Payot, N. Fournier, et al., Determination of key odorant compounds in freshly distilled cognac using GC-o, GC-MS, and sensory evaluation, J. Agri. Food Chem., 52 (2004), 5670–5676.
    [7] S.-J. Lee, A. C. Noble, Characterization of odor-active compounds in californian chardonnay wines using GC-olfactometry and GC-mass spectrometry, J. Agri. Food Chem., 51 (2003), 8036–8044.
    [8] A. Pérez-Silva, E. Odoux, P. Brat, F. Ribeyre, G. Rodriguez-Jimenes, V. Robles-Olvera, et al., GC–MS and GC–olfactometry analysis of aroma compounds in a representative organic aroma extract from cured vanilla (vanilla planifolia g. jackson) beans, Food Chem, 99 (2006), 728–735. https://doi.org/10.1016/j.foodchem.2005.08.050 doi: 10.1016/j.foodchem.2005.08.050
    [9] M. Brattoli, E. Cisternino, P. Dambruoso, G. de Gennaro, P. Giungato, A. Mazzone, et al., Gas chromatography analysis with olfactometric detection (GC-o) as a useful methodology for chemical characterization of odorous compounds, Sensors, 13 (2013), 16759–16800. https://doi.org/10.3390/s131216759 doi: 10.3390/s131216759
    [10] T. Thomas-Danguin, C. Sinding, S. Romagny, F. E. Mountassir, B. Atanasova, E. Le Berre, et al., The perception of odor objects in everyday life: A review on the processing of odor mixtures, Front. Psychol., 5 (2014).
    [11] W. Grosch, Evaluation of the key odorants of foods by dilution experiments, aroma models and omission, Chem. Senses, 26 (2001), 533–545. https://doi.org/10.1093/chemse/26.5.533 doi: 10.1093/chemse/26.5.533
    [12] H. Guth, Quantitation and sensory studies of character impact odorants of different white wine varieties, J. Agri. Food Chem., 45 (1997), 3027–3032.
    [13] V. Ferreira, N. Ortín, A. Escudero, R. López, J. Cacho, Chemical characterization of the aroma of grenache rosé wines: aroma extract dilution analysis, quantitative determination, and sensory reconstitution studies, J. Agri. Food Chem., 50 (2002), 4048–4054.
    [14] A. Escudero, B. Gogorza, M. A. Melús, N. Ortín, J. Cacho, V. Ferreira, Characterization of the aroma of a wine from maccabeo. key role played by compounds with low odor activity values, J. Agri. Food Chem., 52 (2004), 3516–3524.
    [15] V. Ferreira, M. Sáenz-Navajas, E. Campo, P. Herrero, A. de la Fuente, P. Fernández-Zurbano, Sensory interactions between six common aroma vectors explain four main red wine aroma nuances, Food Chem., 199 (2016), 447–456. https://doi.org/10.1016/j.foodchem.2015.12.048 doi: 10.1016/j.foodchem.2015.12.048
    [16] Y. Zheng, B. G. Sun, M. M. Zhao, F. P. Zheng, M. Q. Huang, J. Y. Sun, et al., Characterization of the key odorants in chinese zhima aroma-type baijiu by gas chromatography–olfactometry, quantitative measurements, aroma recombination, and omission studies, J. Agri. Food Chem., 64 (2016), 5367–5374.
    [17] W. S. Cain, Odor intensity: Differences in the exponent of the psychophysical function, Percept. Psychophys., 6 (1969), 349–354. https://doi.org/10.3758/BF03212789 doi: 10.3758/BF03212789
    [18] P. A. Edwards, P. C. Jurs, Correlation of odor intensities with structural properties of odorants, Chem. Senses, 14 (1989), 281–291. https://doi.org/10.1093/chemse/14.2.281 doi: 10.1093/chemse/14.2.281
    [19] M. H. Abraham, R. Sanchez-Moreno, J. E. Cometto-Muniz, W. S. Cain, An algorithm for 353 odor detection thresholds in humans, Chemical Senses, 37 (2012), 207–218. https://doi.org/10.1093/chemse/bjr094 doi: 10.1093/chemse/bjr094
    [20] A. Keller, R. C. Gerkin, Y. F. Guan, A. Dhurandhar, G. Turu, B. Szalai, et al., Predicting human olfactory perception from chemical features of odor molecules, Science, 355 (2017), 820–826. https://doi.org/10.1126/science.aal2014 doi: 10.1126/science.aal2014
    [21] R. M. Khan, C.-H. Luk, A. Flinker, A. Aggarwal, H. Lapid, R. Haddad, et al., Predicting odor pleasantness from odorant structure: Pleasantness as a reflection of the physical world, J. Neurosci., 27 (2007), 10015–10023. https://doi.org/10.1523/JNEUROSCI.1158-07.2007 doi: 10.1523/JNEUROSCI.1158-07.2007
    [22] A. Arshamian, R. C. Gerkin, N. Kruspe, E. Wnuk, S. Floyd, C. O'Meara, et al., The perception of odor pleasantness is shared across cultures, Current Biol., 32 (2022), 2061–2066. https://doi.org/10.1016/j.cub.2022.02.062 doi: 10.1016/j.cub.2022.02.062
    [23] L. Shang, C. J. Liu, Y. Tomiura, K. S. Hayashi, Machine-learning-based olfactometer: Prediction of odor perception from physicochemical features of odorant molecules, Anal. Chem., 89 (2017), 11999–12005. https://doi.org/10.1021/acs.analchem.7b02389 doi: 10.1021/acs.analchem.7b02389
    [24] K. Snitz, A. Yablonka, T. Weiss, I. Frumin, R. M. Khan, N. Sobel, Predicting odor perceptual similarity from odor structure, PLoS Comput. Biol., 9 (2013), e1003184. https://doi.org/10.1371/journal.pcbi.1003184 doi: 10.1371/journal.pcbi.1003184
    [25] A. C. Noble, S. E. Ebeler, Use of multivariate statistics in understanding wine flavor, Food Rev. Int., 18 (2002), 1–21. https://doi.org/10.1081/FRI-120003414 doi: 10.1081/FRI-120003414
    [26] M. Aznar, R. López, J. Cacho, V. Ferreira, Prediction of aged red wine aroma properties from aroma chemical composition. partial least squares regression models, J. Agri. Food Chem., 51 (2003), 2700–2707.
    [27] S. G. Penella, R. Boulanger, I. Maraval, G. Kopp, M. Corno, B.Fontez, et al., Link between flavor perception and volatile compound composition of dark chocolates derived from trinitario cocoa beans from dominican republic, Molecules, 28 (2023), 3805. https://doi.org/10.3390/molecules28093805 doi: 10.3390/molecules28093805
    [28] B Veinand, Sensory testing with flavourists: challenges and solutions, In Rapid Sensory Profiling Techniques, pages 383–399. Elsevier, 2015.
    [29] J.-N. Jaubert, C. Tapiero, J.-C. Dore, The field of odors: toward a universal language for odor relationships, Perfumer & Flavor., 20 (1995), 1–16.
    [30] S. Mirri, C. Prandi, M. Roccetti, P. Salomoni, Handmade Narrations: Handling digital narrations on food and gastronomic culture, J. Comput. Cultural Herit., 10 (2017), 1–17.
    [31] M. Sicard, C. Baudrit, M. N. Leclerc-Perlat, P. H. Wuillemin, N. Perrot, Expert knowledge integration to model complex food processes. application on the camembert cheese ripening process, Expert Syst. Appl., 38 (2011), 11804–11812.
    [32] N. Perrot, I. C. Trelea, C. Baudrit, G. Trystram, P. Bourgine, Modelling and analysis of complex food systems: State of the art and new trends, Trends Food Sci. Technol., 22 (2011), 304–314.
    [33] L. A. Zadeh, Fuzzy sets, Inform. Control, 8 (1965), 338–353. https://doi.org/10.1016/S0019-9958(65)90241-X
    [34] N. Perrot, I. Ioannou, I. Allais, C. Curt, J. Hossenlopp, G. Trystram, Fuzzy concepts applied to food product quality control: A review, Fuzzy Sets Syst., 157 (2006), 1145–1154. https://doi.org/10.1016/j.fss.2005.12.013 doi: 10.1016/j.fss.2005.12.013
    [35] J. Tan, X. Gao, D. E. Gerrard, application of fuzzy sets and neural networks in sensory analysis, J. Sensory Studies, 14 (1999), 119–138. https://doi.org/10.1111/j.1745-459X.1999.tb00108.x doi: 10.1111/j.1745-459X.1999.tb00108.x
    [36] V. J. Davidson, J. Ryks, T. Chu, Fuzzy models to predict consumer ratings for biscuits based on digital image features, IEEE Transact. Fuzzy Syst., 9 (2001), 62–67.
    [37] I. Ioannou, N. Perrot, J. Hossenlopp, G. Mauris, G. Trystram, The fuzzy set theory: a helpful tool for the estimation of sensory properties of crusting sausage appearance by a single expert, Food Quality Prefer., 13 (2002), 589–595. https://doi.org/10.1016/S0950-3293(02)00045-9 doi: 10.1016/S0950-3293(02)00045-9
    [38] S. Mukhopadhyay, G. C. Majumdar, T. K. Goswami, H. N. Mishra, Fuzzy logic (similarity analysis) approach for sensory evaluation of chhana podo, LWT - Food Sci. Technol., 53 (2013), 204–210.
    [39] C. Debjani, S. Das, H. Das, Aggregation of sensory data using fuzzy logic for sensory quality evaluation of food, J. Food Sci. Technol., 50 (2011), 1088–1096.
    [40] K. J. Shinde, I. L. Pardeshi, Fuzzy logic model for sensory evaluation of commercially available jam samples, J. Ready Eat Food, 1 (2014), 78–84.
    [41] O. Folorunso, Y. Ajayi, T. Shittu, Fuzzy-rule-based approach for modeling sensory acceptabitity of food products, Data Sci. J., 8 (2009), 70–77. https://doi.org/10.2481/dsj.007-006 doi: 10.2481/dsj.007-006
    [42] A. Villière, R. Symoneaux, A. Roche, A. Eslami, N. Perrot, Y. Le Fur, et al., Comprehensive sensory and chemical data on the flavor of 16 red wines from two varieties: sensory descriptive analysis, hs-spme-gc-ms volatile compounds quantitative analysis, and odor-active compounds identification by hs-spme-gc-ms-o, Data Brief, 24 (2019), 103725. https://doi.org/10.1016/j.dib.2019.103725 doi: 10.1016/j.dib.2019.103725
    [43] A. Loison, R. Symoneaux, P. Deneulin, T. Thomas-Danguin, C. Fant, L. Guérin, et al., Exemplarity measurement and estimation of the level of interjudge agreement for two categories of french red wines, Food Quality Prefer., 40 (2015), 240–251. https://doi.org/10.1016/j.foodqual.2014.10.001 doi: 10.1016/j.foodqual.2014.10.001
    [44] P. Pollien, A. Ott, F. Montigon, M. Baumgartner, R. Muñoz-Box, A. Chaintreau, Hyphenated headspace-gas chromatography-sniffing technique: Screening of impact odorants and quantitative aromagram comparisons, J. Agri. Food Chem., 45 (1997), 2630–2637.
    [45] S. Arctander, Perfume and flavor chemicals, Montlcair, N.J. USA, 1969.
    [46] W. Luebke, the good scents company, https://WWW.thegoodscentscompany.com/index.html, 1980.
    [47] L. A. Zadeh, Fuzzy logic and approximate reasoning, Synthese, 30 (1975), 407–428. https://doi.org/10.1007/BF00485052 doi: 10.1007/BF00485052
    [48] D. Dubois, H. Prade, Fuzzy sets and systems: Theory and applications, Academic press, NY, USA, 1969.
    [49] S. Arlot, A. Celisse, A survey of cross-validation procedures for model selection, Statistics Surveys, 4 (2010), 40–79.
    [50] W. Guang, M. Baraldo, M. Furlanut, Calculating percentage prediction error: A user's note, Pharmacolog. Res., 32 (1995), 241–248. https://doi.org/10.1016/S1043-6618(05)80029-5 doi: 10.1016/S1043-6618(05)80029-5
    [51] A. Roche, N. M. Perrot, T. Thomas-Danguin, Oops, the ontology for odor perceptual space: From molecular composition to sensory attributes of odor objects, Molecules, 2022. https://doi.org/10.3390/molecules272227888 doi: 10.3390/molecules272227888
    [52] N. Hansen, Towards a New Evolutionary Computation Studies in Fuzziness and Soft Computing, In chapter The CMA evolution strategy: A comparing review, (eds) J.A. Lozano, P. Larrañaga, I. Inza, I. Bengoetxea, 192 (2006), Springer, 75–102. https://doi.org/10.1080/02693445.1948.12035642
  • This article has been cited by:

    1. Jingyao Liu, Qinghe Feng, Yu Miao, Wei He, Weili Shi, Zhengang Jiang, COVID-19 disease identification network based on weakly supervised feature selection, 2023, 20, 1551-0018, 9327, 10.3934/mbe.2023409
    2. Zhan He, Chunju Zhang, Shu Wang, Jianwei Huang, Xiaoyun Zheng, Weijie Jiang, Jiachen Bo, Yucheng Yang, A Grad-CAM and capsule network hybrid method for remote sensing image scene classification, 2024, 18, 2095-0195, 538, 10.1007/s11707-022-1079-x
    3. Wafa Gtifa, Marwa Fradi, Anis Sakly, Mohsen Machhout, Refining COVID‐19 Lesion Segmentation in Lung CT Scans Using Swarm Intelligence and Evolutionary Algorithms, 2025, 21, 1478-5951, 10.1002/rcs.70044
    4. A. V. P. Sarvari, K. Sridevi, Optimized Residual Attention Based Generalized Adversarial Network for COVID‐19 Classification Using Chest CT Images, 2025, 41, 0824-7935, 10.1111/coin.70031
  • Reader Comments
  • © 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Metrics

Article views(2739) PDF downloads(172) Cited by(4)

Figures and Tables

Figures(7)  /  Tables(6)

/

DownLoad:  Full-Size Img  PowerPoint
Return
Return

Catalog