Involvement of smart technologies in an advanced supply chain management to solve unreliability under distribution robust approach

1.   Introduction

Machinery has become an integral part of human life, especially considering the technological advancements related to Industry 4.0. Failures occurring on such crucial machinery lead to unplanned downtime, ultimately resulting in loss in economic aspects ^[1]. This is catastrophic in industry and public transport since these failures stop production, causing hassles to the public. Hence, machine diagnostics is of high importance in such instances. Fault prediction in an early stage dramatically improves the machine's lifetime, reducing costs and preventing downtimes.

Air and oil leaks are two of the predominant operational failures in public transport modalities, especially in metro trains, which are our prime objective. Air leaks are prone to occur in the dryer component, whereas oil leaks are prone to occur in the compressor component ^[2]. Various sensors, like pressure transducers, pneumatic sensors, motor current, etc., are used to analyze and diagnose the faults in metro trains ^[3,4,5]. An abnormal change can be observed in these sensors in the event of a fault in that component. Continuous monitoring of the vulnerable components with the sensors mentioned above can help identify the occurrence of a fault in that component ^[6].

Predictive maintenance has been an emerging technology in machine diagnostics, aiming to predict faults early and perform maintenance to prevent catastrophic events ^[7]. Also, anomaly detection based on sensor data on an early scale will reduce maintenance expenses and avoid downtime. Data curation, data pre-processing, diagnosis, and decision-making are the critical aspects of predictive maintenance ^[8]. This data-driven approach has been proven effective due to the vast availability of data and intelligent algorithms for automated analysis ^[9].

Artificial intelligence plays a promising role in fault prediction and predictive maintenance. Several machines and deep learning algorithms are trained on continuous data from sensors attached to the target machinery ^[10]. The proposed work uses machine and deep learning algorithms for anomaly detection (air and oil failure) on the air production unit of metro trains and real-time dashboard development, which is the first of this work as per our knowledge. The following section explains the existing works on machine usage and deep learning algorithms for predictive maintenance.

The following are the contributions of the proposed work:

1. To develop a deep learning algorithm for the simultaneous identification of the type and location of the fault, along with GPS quality monitoring from sensor data in metro trains.

2. To integrate an explainable AI technique into the model's prediction and highlight the key sensors contributing to the fault.

3. To develop a dashboard integrating the sensor data analytics as visual graphs, the deep learning model, and the explainable AI results for analysis by engineers.

This work will be of great aid to maintenance engineers for fault analysis in sensor values and assessment of GPS signals in real time. This predictive maintenance application can aid in reducing the downtime and service costs of machine parts, if found damaged.

The following is the outline of the research paper. Section 2 briefs about the existing work relevant to the field of interest. Section 3 explains the material description, the pre-processing techniques used, the different machine learning algorithms, the training parameters, and the methodologies for dashboard development. Section 4 represents the proposed methods' results, graphs, and supporting diagrams. Section 5 compares the proposed work with existing works and draws significant conclusions and future scope.

2.   Related works

The detection of air leakage from the pneumatic door of a train is an attempt to reduce train downtime. Deep learning algorithms were applied for automatic feature extraction from extensive data obtained from continuous monitoring by sensors for the task of fault detection ^[11]. The OSR (open set recognition) concept was used for multi-task classification to predict the known class and detect unknown samples. A lightweight convolutional neural network (CNN) model streamlined with the OSR technique was trained to predict the air leakage. An 8-layer neural network consisting of 6 convolutions and two dense layers was used for air leakage. This model was trained using an SGD (stochastic gradient descent) optimizer with a learning rate of 0.001 for a batch size of 64.

The server air leakage in the breaking pipe results in breaking issues and decreases the train reliability ^[12]. Due to the visual constraints for air leak detection, the paper proposes a framework for the simultaneous prediction of the type and severity of air leakage using anomaly detection methods based on the on-and-off logs of the compressor. Around 632,683 data points were collected from May 2016 to October 2016 from 178 VIRM trains, of which 6957 are labeled as "Air Leakage" and 625,726 are labeled as "Normal". They have used a logistic classifier model for two different classes of compressor behavior for each separate train. One defines the boundary by separating two classes under everyday situations, and the other models the distribution of the compressor idle time and run time separately using logistic functions. It also further detects the context of compressor idle time erroneously classified as a compressor run time, aiding in anomaly detection. A density-based unsupervised clustering approach is adopted for anomaly detection before four weeks and can pre-filter anomalies to prevent false alarms.

The challenges encountered by traditional manufacturing companies during their transition to intelligent factories, notably the scarcity of historical data for training machine learning models, were addressed by Mohan Rajashekarappa et al. ^[13]. A novel approach of artificially inducing anomalies for data labeling was introduced, and it underscored the importance of proactive readiness for potential future disruptions in newly installed systems. Through two experiments focused on air leakage detection, the proposed methodology demonstrates exceptional performance with RUS-Boosted bagged trees, yielding 98.73% accuracy, 99.40% precision, recall of 99.21%, and an F1 score of 99.30% on the test data.

The critical issue of energy efficiency and fault detection in air conditioning systems emphasizes their intricate nature and substantial energy consumption impact ^[14]. The study comprises two essential components: First, it investigates the ramifications of various faults within the air conditioning system on its coefficient of performance (COP), shedding light on the potential energy wastage associated with these faults. The research convincingly demonstrates that different faults lead to varying degradation levels in the COP. Second, the paper evaluates the effectiveness of three supervised learning classifier models in classifying these faults: deep learning, support vector machine (SVM), and multi-layer perceptron (MLP). The research assesses the performance of these classifiers across six distinct fault classes, revealing that different faults indeed exert varying negative impacts on the COP.

Predicting air failure of the air production unit (APU) in metro trains. The dataset used for this task was MetroPT, a 6-month analysis of metro trains in Portugal comprising analog, digital, and GPS sensors ^[15]. The GPS information was excluded from the dataset, and the timestamp was encoded using the label encoding technique. A random forest classifier algorithm was used for the multi-class classification of air failure prediction. The data was undersampled and then split into training and testing sets. A feature importance visualization technique was employed to identify the root cause of the air failure. The random forest classifier produced 85% and 97% accuracies for the binary and multi-class classification tasks, respectively.

A deep learning neural network for anomaly detection in metro trains was developed by Davari et al. ^[16]. The algorithms used for this task were the sparse autoencoder and variational autoencoder. This work is an unsupervised learning approach for anomaly detection of air failures in trains. The MetroPT dataset was used for this work with sensors placed in the air production unit. The two versions of the autoencoder were used for sensor data reconstruction, and a low-pass filter was used to perform anomaly detection and detect faults. The autoencoder algorithms using the digital data produced precision, recall, and F1 scores of 44%, 13%, and 32% better than that of the algorithms trained on the analog data.

An expert system for the multi-objective optimization of equipment was developed for highway optimization by Ali et al. ^[17]. The particle swarm optimization was used to simultaneously optimize the time, cost, and quality of the equipment for construction. This method reduced the time and cost by 35.4% and 39.1%, respectively. The application of predictive maintenance in concrete manufacturing was done by Alshboul et al. ^[18]. Seven different classification algorithms were used, out of which the cat boost classifier produced an F1-score of 0.985, an accuracy of 0.984, a recall of 0.983, and a ROC curve area of 0.984. A comparative analysis of machine learning algorithms for concrete strength estimation was performed by Alshboul et al. ^[19]. Three machine learning algorithms, namely, XGBoost, LighGBM, and genetic programming, were used, out of which the LightGBM and XGBoost algorithms surpassed other studied algorithms with a coefficient of determination of 95.74% and 93.27%, respectively.

The proposed work aims to develop a decision process for failure prediction and identification of the failure type and location using machine learning and multi-task models using deep learning algorithms.

3.   Materials and methods

3.1.   Proposed workflow

The proposed work adopts the following workflow consisting of different blocks: data acquisition, data pre-processing, feature pre-processing, visualization, model development, validation, and deployment. Figure 1 represents the proposed workflow visually.

3.2.   Dataset description

The dataset used for this work is named MetroPT ^[17,18], comprising of sensor information related to urban metro trains in Portugal collected during the year 2022. The dataset comprises different analogue, digital, and GPS sensors continuously capturing data from the metro trains for six months. The MetroPT dataset has been curated to develop AI algorithms for automated fault prediction and predictive maintenance of metro trains based on sensor data. Table 1 represents the different sensors, their description, and the unit of measurement used for acquiring the MetroPT dataset.

3.3.   Data pre-processing

The dataset comprises around 15 million sensor data records from Jan 1, 2022 to Jun 30, 2022. The dataset obtained from the source is not directly labeled. The dataset owners have provided information about the failure, like the start time, end time, type, and location of the failure. Based on the start and end times, the appropriate timestamps were found and the values between those timestamps were coded according to the fault type. Table 2 shows the statistical description of the dataset for the parameters, namely, TP2, TP3, H1, DV_pressure (DVP), Reservoirs, Oil_temperature (OT), Flowmeter, Motor_current (MoC), COMP, DV Electric (DVE), Towers, MPG, LPS, Pressure_switch (PrS), Oil_level (OL), Caudal_impulses (CaI), GPS Longitude (GPSLong), GPS Latitude (GPSLat), GPS Speed, GPS Quality, month, day, hour, minute, second. The statistical values of mean, standard deviation (std), minimum value, 25%, 50%, 75%, and maximum values for a dataset are evaluated here. Table 3 represents the label code for the different types of faults occurring in the metro train.

Timestamp data cannot be processed by machine learning and deep learning algorithms. Hence, it needs to be processed. This issue was tackled by extracting the month, week, day, hour, minute, and second information from the timestamp using the Pandas functionalities. Finally, the timestamp feature is removed from the dataset. Two new columns, one for the type of failure and another for the location of the failure, were created from the labels. These columns are created as these are the labels for the multi-task model. Table 4 represents the different failure types and location codes.

The final step in the data pre-processing was the operation of undersampling, especially in the dataset of stage 1. The stage-1 dataset comprises 2, 12,104 samples under the fault condition, about 1.63$\%$ of the entire data. Around 98.4$\%$ of the data belongs to the regular class, proving the dataset is highly imbalanced. Hence, the samples from the regular class were undersampled. Around 28, 00,000 randomly selected samples were taken from the entire dataset, which was used as the data for the first stage. The final dataset comprises about 25 input features and three target vectors. Table 5 represents the description of all the targets in the final dataset.

3.4.   Feature pre-processing and visualization

The dataset has no null or duplicate values since the dataset is obtained from sensors that continuously monitor the trains. The feature scaling technique, normalization, was adopted to bring all the features to the same scale (0-1). Appropriate data visualization techniques were used for the dataset's univariate, bivariate, and multivariate analysis. The dataset comprises continuous (analogue sensors) and categorical features (digital sensors); we have split them for data visualization. A histogram with a kernel density estimator function is used to visualize the continuous features in the dataset. A pie chart is used to visualize the categorical features. Finally, the information was visualized using a map that represents the train's route along with the train's speed. Figures 2, 4, 3, and 5 represent the visualization plots obtained from the continuous, categorical, GPS, and entire dataset, respectively. After visualization, a stratified split of ratio 80:20 was made for the training and testing sets, respectively.

3.5.   Multi-task neural network

Multi-task learning is the ability of a neural network to simultaneously obtain multiple outputs from a single input. In this work, based on the sensor information, the multi-task neural network is designed to simultaneously predict the type and location of the failure in the metro train. The multi-task neural network has common pre-processing layers, followed by branches, each corresponding to a particular task. Hence, the multi-task neural network uses parallel processing to simultaneously identify the type and location of the fault and assess the GPS quality.

The multi-task neural network comprises shared layers and task-specific layers. The input layer, a single hidden layer with four neurons, and regularization layers like dropout and batch normalization are common for both tasks. In contrast, there are individual output layers for each task. The output layer comprises three neurons for the task of failure identification, one neuron for the task of location identification, and one neuron for the task of GPS quality identification. Figure 6 represents the architecture diagram of the proposed multi-task model.

3.6.   Training methodologies

The abovementioned multi-task neural networks were trained using the Adam optimizer and a combination of categorical and binary cross-entropy loss functions. The batch size was set to 5000, and the models were trained for 20 epochs. A hybrid loss function was used to train the multi-task model since two tasks (fault type and location identification) were multi-class. In contrast, the task of GPS quality identification is binary. Equation 3.1 represents the hybrid loss function used to train the multi-task model.

3.7.   Evaluation metrics

Evaluation is an essential component of the proposed workflow. Model evaluation is done to identify the model's performance on the testing set. Evaluation of the testing set is essential to identify if the trained model has overfit or underfit. The following performance metrics are derived from the confusion matrix.

The confusion matrix comprises four values: true positive, true negative, false positive, and false negative. The diagonal elements of the matrix indicate the correctly classified samples (true positive and true negative), and the non-diagonal elements indicate the misclassified samples (false positive and false negative). The following are the different metrics used to evaluate the trained models.

3.7.1.   Accuracy

Accuracy is defined as the ratio of the correct classifications to that of the total classifications. Accuracy is considered the gold standard metric for the evaluation of classification algorithms. The formula for accuracy is mentioned in Equation 3.2.

3.7.2.   Precision

Precision is defined as the ratio of true positives to that of the total positives. Precision is one of the metrics used to analyze a model's performance in class imbalance conditions. The formula for precision is mentioned in Equation 3.3.

3.7.3.   Recall

Recall is defined as the ratio of true positives to that of total samples. Recall is another metric that is used to analyze multi-class classification under the condition of class imbalance. Equation 3.4 represents the mathematical formula for recall.

3.7.4.   AUC score

AUC-ROC is expanded as the area under the regional operating characteristic curve. The ROC curve is the plot between the false positive and true positive values. The area under that curve is termed an AUC score. An AUC value of less than 0.5 is considered a terrible score, a score of 0.5 is considered a random guess, and a score of more than 0.8 is considered a good score.

3.8.   Interpretability techniques

Model interpretability has been a focus and requirement for which the demand has risen in recent years. Many machine learning and deep learning algorithms are considered black boxes, providing output for input data without any logical interpretations. This is needed in areas like healthcare, where life-concerning critical decisions are made. In such instances, providing interpretability to the model by providing explanations of the predictions can greatly aid clinicians.

In this work, the local interpretable model agnostic explanations (LIME) ^[19] is used to derive the interpretations of the complex ensemble classifier. As the name suggests, LIME works locally, meaning it works on individual data samples. Also, this method is model agnostic, meaning it works for all models. LIME works by mapping a simple interpretable model (like linear regression) on a complex model ^[20]. The local region of the data space is considered, where synthetic samples are generated based on original samples. These synthetic samples are labeled based on the prediction of the complex model. Then, a simple interpretable linear regression is trained on the synthetic labeled data. The coefficients of the trained linear regression model represent the interpretations of the complex model on the local space. The LIME tabular function from the LIME library is used to get the interpretations for individual data samples.

The LIME function explains the type of fault, location of the fault, and quality of the GPS sensor, respectively. The tabular explainer from LIME was used to explain the predicted instances. A LIME explainer was used for each task: failure type, failure location, and GPS quality. The LIME explainer generates a figure that shows the input that contributed to that particular class (positive) and the features that contributed to the counter-class (negative). Hence, we can understand the features that positively and negatively contribute to a particular class from the plot.

3.9.   Dashboard development

The ultimate aim of the work is to develop a dashboard for real-time data analytics and predictions. A website was developed using a Python-based web development tool. The website will request the data recorded from the sensors as an Excel sheet of CSV (comma-separated values). The data visualization and prediction tasks are done simultaneously upon receiving the data. On the data visualization task, a stacked area chart of the continuous features, a stacked bar chart for the categorical features, and a map depicting the speed and route of the train are made. For the data prediction task, the latest values are processed and sent to the trained multi-task model for predictions on the failure type, failure location, and quality of the GPS, which are displayed on the website. Finally, the LIME explanations for the model on the provided sample are given for all three tasks: failure type identification, failure location identification, and GPS quality assessment.

4.   Results

The developed multi-task model was trained on the training set with the above-mentioned epochs and batch size. Table 6 represents the performance metrics of the model on the training and testing datasets for the fault type and location identification, respectively.

The trained multi-task ANN model produced 98.89%, 99.12% and 99.24% accuracy for failure type identification, failure location identification and GPS quality assessment. Also, the trained model's precision, recall and AUC values are high, indicating that the model overcomes class imbalance issues.

The performance plots representing the values of the performance metrics for each epoch in the training phase are shown in Figure 7. The loss gradually decreases in each task's epoch for fault, location, and GPS quality, as shown in Figure 7(a). In contrast, the recall, AUC, and precision are increasing for each epoch, as shown in Figure 7(b), Figure 7(c), and Figure 7(d), respectively. This shows no fluctuations in the training phase and no signs of varying gradients.

The performance plots represent the values of the performance metrics for each epoch in the testing phase. The loss gradually decreases in each task's epoch, as shown in Figure 8(a). In contrast, the recall, AUC, and precision are increasing for each epoch for fault, location, and GPS quality as shown in Figure 8(b), Figure 8(c), and Figure 8(d). The precision, recall, and AUC values for GPS quality reached the maximum in the initial epochs and remained the same for the rest. Also, fluctuations in the precision values are observed for the tasks of fault type and fault location identification.

The confusion matrices on the testing set represent that the trained model has produced high true negatives and true positives. In contrast, it has produced very few false positives and false negatives. The confusion matrix for the fault type is shown in Figure 9(a), The fault location confusion matrix is shown in Figure 9(b) and the confusion matrix for GPS quality is shown in Figure 9(c).

The classification report for fault type classification indicates high precision, recall, and F1 scores for all three classes, showing no sign of class imbalance. The classification report for the trained multi-task model for fault type identification on the testing set is shown in Table 7. The classification report for the trained multi-task model for fault location identification on the testing set is shown in Table 8. The classification report for the trained multi-task model for GPS quality assessment on the testing set is shown in Table 9.

A website was developed using Streamlit and hosted online using the Streamlit share, see ^[21] for the website URL. Figures 10 and 11 represent the snips of the developed website about data visualization and predictive modeling. Figures 12, 13, and 14 represent the LIME explanations for the tasks of failure type identification, failure location identification, and GPS quality assessment, respectively. Figure 10 represents the area chart for the continuous features and the bar chart for the categorical features. From this bar, anomalies in the sensor values can be visually identified. Figure 11 represents the map plot for the GPS data; the dots represent the map's route based on the latitude and longitude values, whereas the dot's intensity depicts the speed. This graph identifies the train's route and the crucial locations at which the train went fast/slow. Also, the multi-task neural network predictions are mentioned on the website for each dataset instance.

Table 10 compares the performance of the proposed multi-task neural network to that of the existing works related to the application of predictive maintenance in train fault analysis using sensor data.

5.   Discussion

Based on our knowledge, only two works ^[15,16] use artificial intelligence for the predictive maintenance of urban metro trains. Najjar et al. ^[15] worked on predicting air failure of the air production unit (APU) in metro trains. The dataset used for this task was MetroPT, a 6-month analysis of metro trains in Portugal comprising analog, digital, and GPS sensors. The GPS information was excluded from the dataset, and the timestamp was encoded using the label encoding technique. A random forest classifier algorithm was used for the multi-class classification of air failure prediction. The data was undersampled and then split into training and testing sets. A feature importance visualization technique was employed to identify the root cause of the air failure. The proposed work produced testing accuracies of 84$\%$ and 87$\%$ on the binary and multi-class classification tasks and F1 scores in the ranges of 0.83–0.5 and 0.73–0.97 for the binary and multi-class classification tasks. The proposed model produced better results than the work and has also included oil failures in addition to air failures. Davari et al. ^[16] developed a deep learning neural network for anomaly detection in metro trains. The algorithms used for this task were the sparse autoencoder and variational autoencoder. This work is an unsupervised learning approach for anomaly detection of air failures in trains. The sparse autoencoder trained on the digital data produced 42$\%$ more than those trained on analog data. Also, the variational autoencoder performed better than the sparse autoencoder by 37$\%$. The proposed work considered both analog, digital, and GPS sensors worked on both air and oil failures, and produced state-of-the-art results.

The multi-task model has produced excellent results on the training and testing datasets. The performance plots prove that the model has been trained perfectly and shows no signs of overfitting or underfitting. Also, the confusion matrices and classification report suggest that the trained model is generalized and does not exhibit any signs of class imbalance.

The proposed work has some advantages in comparison to the existing works. The proposed work addresses all issues faced in the metro trains (air and oil failures). Another advantage is the excellent results obtained by the trained models. The third advantage is agility; the proposed multi-task model has taken less time to train and predict batch data. Finally, the explainable AI technique, namely LIME, has been implemented to provide interpretations to the outputs given by the multi-task model. This provides belief in the model prediction and can also be useful for engineers to deeply analyze the issue.

Figure 12 represents the LIME explanation for a local instance related to the task of fault type prediction. As observed from the plot, the predicted instance is air failure with 100$\%$ confidence, and the features positively and negatively contribute to the prediction.

Figure 13 represents the LIME explanation for a local instance related to the task of fault location prediction. As observed from the plot, the predicted instance is a client with 100$\%$ confidence, and the features positively and negatively contribute to the prediction. Motor current, DV pressure, and the day positively contributed to the prediction, followed by GPS speed and reservoir. The GPS longitude, minute, and flowmeter have negatively contributed to the prediction.

Figure 14 represents the LIME explanation for a local instance related to the task of GPS quality prediction. As observed from the plot, the predicted instance is air failure with 100$\%$ confidence, and the features positively and negatively contribute to the prediction. Oil temperature has majorly contributed positively, whereas H1, day, and minute have majorly contributed negatively. For all plots, the range or condition of the input features is given, which might be of great use for the fault analysis.

One reason for achieving good results is the split of the output labels into fault type and location. This reduces the number of interdependent classes in each stage, which might have improved the performance of the algorithms. Also, we observed a performance rise of 6$\%$ when the features were standardized. Considering the deep learning aspects, we developed a multi-task model which splits the classes into individual tasks, allowing for more attention, resulting in better results and quicker periods.

However, the study has some limitations. First, the dataset was undersampled to 30, 00,000 data points, roughly 20$\%$ of the entire dataset. The second one was the expansion of the target vectors into new columns, which introduced more computations and the need to train more models. While this approach was considered to improve the holistic performance of the models, this resulted in the creation of multiple datasets and, ultimately, multiple ML models for training, leading to computational costs. The third one was the limited selection of machine learning algorithms. Many good machine learning algorithms like support vector machine and ensemble learning techniques were not implemented due to the computational constraints and long training durations (the SVM algorithm did not train even after 30 minutes!).

6.   Conclusions

Hence in this work, a multi-task model was developed for the identification of failures simultaneously. The proposed method has produced 98.89%, 99.12%, and 99.24% accuracies in the testing set for failure type, failure location, and GPS quality predictions, respectively, exceeding the state-of-the-art methods. The model produced 99.56%, 99.67%, and 99.84% precision in the testing set for failure type, failure location, and GPS quality predictions, respectively. The high accuracy and precision values indicate the good performance of the model and no signs of class imbalance. The deep learning model took 43 seconds for training and 1 second for inferencing for test data, showing fast predictions, needed for predictive maintenance applications. Moreover, a real-time interactive dashboard was developed, performing dynamic data visualization and predictions. Finally, using the LIME explainable AI technique provides explanations for the predictions, adding belief and better analysis for engineers. The developed system would be advantageous for engineers to perform fault analysis and predictive maintenance effectively. Future work will use a database to store the streaming data and deploy the system in real time. Also, we will develop deep learning algorithms on the entire dataset and employ online learning strategies to update the learned model in real time.

Author Contributions

Pratik Vinayak Jadhav: Data Curation and Analysis, Research Design and Methodology, writing draft. Sairam V. A: Data Curation and Analysis, Research Design and Methodology, writing draft. Siddharth Sonkavade: Data Curation and Analysis, Research Design and Methodology, writing draft. Shivali Amit Wagle : Conceptualization, Supervision, Project Development, Writing, Review, and Editing. Preksha Pareek: Conceptualization, Supervision, Project Development, Writing, and Review. Ketan Kotecha: Writing, Review, and Editing, Funding and Resources. Tanupriya Choudhury: Data Analysis, Writing, Review, and Editing.

Conflict of interest

The authors declare no conflict of interest.