A machine learning approach to differentiate between COVID-19 and influenza infection using synthetic infection and immune response data

1.   Introduction

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and influenza viruses cause COVID-19 and influenza diseases, respectively, and mainly infect the upper and lower respiratory tract^[1,2]. Both infections present similar primary symptoms such as cough, fever, sore throat, runny or stuffy nose, tiredness, and body aches ^[3,4]. Early on in infection this can lead to a clinical dilemma in diagnosis ^[5,6,7]. Recently, COVID-19 has, through its worse overall decompensation due to its intensive transmission and vascular effects, caused an unrivaled global crisis ^[8,9,10,11]. As the globe moves to endemicity, as the striking COVID-19 outbreak continues, the concurrence of COVID-19 and influenza epidemics is impending. This motivates the current study, to design a data analysis tool that can accurately differentiate between these two infections.

One way to rapidly classify patients with influenza or COVID-19 could be through machine learning approaches. Preliminary investigation illustrated the potentials of machine-learning models for accurately distinguishing between these two viral infections, using demographics, body mass index, and vital signs in infected patients ^[8]. Herein, we used a simple ML-based classification to identify patients with influenza or SARS-CoV-2 based on the main features of the within-host viral dynamics and the immune response. During the past decade, in-host mathematical modelling has become an increasingly powerful tool to study inter and intracellular viral infection and the ensuing immune response ^[12]. Such mathematical models can deepen our understanding of virus spread within organs leading to antiviral drug inventions and optimized treatment regimens. Furthermore, using the mechanistic model to generate synthetic patient data for various infections can help us to mitigate difficulties related to clinical data analyses, such as time-inconsistent data sets that can cause biased results ^[13].

Recent studies have suggested that that artificial intelligence (AI) and machine learning (ML) methods can perform as well as or even better than humans at significant healthcare tasks, such as diagnosing disease ^{[14,15,16,17]}. We apply a basic mathematical model on the cellular scale (the so-called target cell-limited model ^[18,19]) fit to two different sets of in vivo data for COVID-19 and influenza infections, to create virtual patient cohorts. Using our multi-class classifier, the patients are differentiated between the two infections. This is conducted over the entire infection and for early time-points only. Herein, we show that, with just some important in-host measurements, our method is able to discern which virus has infected a patient with a high degree of certainty. Such results can lead to the development of rapid diagnostic tests in future to aid in early patient diagnosis They can also be used in clinical trials of new therapeutics and vaccines to determine the need for new participant enrolments, the number of measurements needed from each participant, and what would be best to measure to show if a new vaccine or therapeutic is effective ^[20,21,22].

This paper is organized as follows: In section 2, through subsection 2.1, we discuss the in-host mathematical modelling of influenza and COVID-19 and parameter estimation. In subsection 2.2, we use the mechanistic model to generate synthetic patient data. In subsections 2.3 we study developing and evaluating a supervised machine learning method to discriminate the patients with different infections. The Interpretability of the developed model is discussed in subsection 2.4. The results of the prediction are presented in section 3 through subsection 3.1. Subsection 3.2 discusses the importance of the data features and determines the dominant features. The paper concludes with a discussion in Section 4.

2.   Method

2.1.   Mechanistic models

We employed a target-cell limited model of viral dynamics using five differential equations that track susceptible target cells $(T)$, infected cells in the eclipse phase $(I_1)$, productively infected cells $(I_2)$, virus $(V)$, and interferon $(F)$ in-host. Figure 1 presents a flow diagram of the model. The system of ordinary differential equations is as follows:

Briefly, virus particles $V$ can infect susceptible target cells $T$ to produce infected cells. This is represented by the term $\beta TV$. Newly infected cells first enter the eclipse phase $I_1$ and become productively infected cells $I_2$ when within-cell processes that program the cell to make new virus particles are completed. The eclipse phase takes, on average, $1/k$ time units. Productively infected cells produce new virus particles with a rate of $p$, and the virus particles are cleared from the system with a rate of $c$. We assumed that productively infected target cells have a death rate $\delta$. Susceptible target cells can be protected from infection by Type I interferon (IFNI), $F$. Type I interferons protect neighboring cells from infection and elicit an immune response ^[23,24]. They are central to combating different virus infections and are regularly measured in clinical trials or infection studies in humans and animals ^[25]. We assumed that interferon production is proportional to the number of productively infected cells ^{[18,19,26,27]}, that interferon has a natural decay rate $d$, and that interferon protects susceptible cells by removing them from the susceptible target cells population, with a rate $\phi F$. This term was ignored in ^[18] for influenza infection. The model described by Eq. 2.1 was used in ^[18] and ^[19] to examine the kinetics of influenza A and SARS-CoV-2 viral dynamics, respectively. For the sake of simplicity, we have ignored a half-day lag in IFNI response that was considered in ^[18].

2.1.1.   Parameter estimation

Model parameters for influenza A infection were fit to data from an experimental H1N1 influenza A/Hong Kong/123/77 infection for six patients ^[18] and for SARS-CoV-2 from thirteen untreated patients infected with severe acute respiratory syndrome-coronavirus ^[19]. The geometric average parameter values along with their $95\%$ confidence intervals and units are summarized in Table 1. We assumed that the initial number of target cells, $T_0$, is equal to the total number of target cells in the upper respiratory tract and set $T_0 = 4\times 10^8$ cells. In ^[19] the authors considered that the target cells distributed in a volume of 30 mL. Assuming that $1\%$ of these cells expresses the angiotensin-converting enzyme 2 (ACE2) as a receptor for SARS-CoV-2, the target cell concentration, $T_0$, was expressed as $1.33 \times 10^5$ cell/ml. Model variables with initial values were estimated as in Table 2.

2.2.   Generation of virtual patients

To generate a cohort of virtual patients, we followed a technique similar to the one used in ^[24]. Each patient is distinguished by five different in-host measurements, $\{T, I_1, I_2, V, F\}$, that are the solutions of Equation 2.1 for different sets of model parameters. Initial parameter sets representing individual virtual patients were drawn from normal distributions with means fixed to the corresponding parameter value in Table 1 and standard deviations derived from confidence interval measurements. Standard deviations were obtained from standard errors, confidence intervals, and $t$ statistics which measure the size of the difference relative to the variation in the sample data. For each parameter value, the standard deviation was obtained by dividing the length of the confidence interval by standard errors width ($2\times t-value$) and then multiplying by the square root of the sample size as follows

Standard errors must be of means calculated from within each parameter confidence interval. The $t-value$ for a $95\%$ confidence interval from a sample size of $N$ was then obtained in Microsoft Excel using the $tinv$ function (i.e. $tinv(1-0.95, N-1)$). From ^[18], the sample size for the influenza cohort is 6 patients infected by H1N1 influenza A/Hong Kong/123/77 infection. The COVID-19 cohort consisted of 13 untreated patients infected with severe acute Respiratory syndrome-coronavirus2 ^[19]. Therefore, the $t-value$ for influenza patients is 2.571 and for COVID-10 patients is about 2.179. From normal distributions with standard deviations, $\sigma$, and means, $\mu$, as the original parameter values, we then generated normal distributions covering values lying around each parameter value such that $|\mu\pm\sigma-\mu| < h$. Herein, the parameter $h$ is the user-defined value as a measure of data diversity. In the other words, the bigger the parameter $h$, the more diverse the synthetic data. Accordingly, the external noise can affect the data through the parameter $h$. The dynamics of 100 virtual patients from each cohort are shown in Figure 2. The diversity of patient data is mainly reflected in various viral load levels to agree with prior studies that different viral load is associated with the severity of diseases or different factors such as age or sex of the patients ^[32].

2.2.1.   Consistency of the data

Generating data with time consistency for different cohorts of infections is of great importance. Data inconsistency can lead to loss of information or biased results. Since the influenza mechanistic model predicts faster clearance of influenza-infected cells than SARS-CoV-2 ^[19], the infection period for influenza and COVID-19 patient dynamics are not the same, see Figure 2. Therefore we limited the consistency of flu/COVID-19 cohorts to have the same number of data points during the infection time. Hereupon, as an example, we divided the main infection period (i.e., ^{[1,2,3,4,5,6]} days for influenza patients and ^{[10,11,12,13,14,15,16,17,18,19,20]} days for COVID-19 patients) into ten different sub-intervals with half-day length time steps for influenza patients and one and half-day length time steps for COVID-19 patients (see Figure 3). Hence, despite having different infection periods and time steps with different lengths to report the new virtual data point, the total number of data for the two different cohorts was the same.

In addition to the total infection period, we were also interested in studying the viral load dynamics in the early period of infection. The median incubation period for influenza A(B) virus is estimated to be $1.4 (0.6)$ days, and for SARS-CoV-2 is around $5-6$ days ^[33]. Therefore, we assumed the time interval $[0.9, 1.3]$ days for influenza, and $[5-6.5]$ days for COVID-19 cohorts, corresponding to $[10^2-10^4] Copies/ml$ viral load. Dividing each interval into three different sub-intervals to get the time steps with length one-sixth of a day for Influenza and half a day for COVID-19 patients, we had four consistent data points for each patient.

2.3.   Predictive model development

To distinguish between patients who encounter COVID-19 from those who are exposed to influenza, we developed a predictive model based on some biological feature selections. Accordingly, we adopted Logistic regression with $\ell_1$-regularization, referred to Lasso (stands for least absolute shrinkage and selection operator) Regression, as an appropriate technical classification. Lasso regression is widely used for many supervised classification problems based on the concept of probability ^[34]. It can simplify the model complexity by removing irrelevant features of the data set. Recently, this algorithm was used by Han and et al. to find some additional novel immune features that accurately identified patients before the clinical diagnosis of preeclampsia ^[35].

Logistic regression, which is a special case of linear regression and used for binary classification, is defined by the following sigmoid function

in which $X$ is the $(n \times p)$ model feature matrix of $n = 100$ patients and $p = 5$ biological hallmarks. Defining the cost/objective ($C$) function of logistic regression in mean squared error format leads to a non-convexity that makes it difficult to optimally converge. Therefore, it is represented by the following equations

where $Y$ is a binary response vector of outcome (CVOID-19 vs flu). Compressing the above two equations inside a single function, we have

Replacing the sigmoid function from equation (2.3) and applying a penalty term equal to the absolute value of the magnitude of coefficients, we can reach the following objective function (after doing some mathematical simplifications) ^[35]

The penalty term which is called the $\ell_1$-regularization term is added to prevent data over-fitting. The model objective is to find a specific solution with a best-optimized cost function.

For model training and testing, we developed a $\mathcal{K}$-fold cross-validation strategy, which is a re-sampling method to evaluate machine learning models on a limited data sample. The procedure has a single parameter called $\mathcal{K}$ which displays the number of groups that a given data sample is to be split into. As such, the procedure is often called $\mathcal{K}$-fold cross-validation. Therefore, our regression model is not tailored to a particular data set and is exposed to all available samples of a given subject in the training set. This approach implies that the training procedure was entirely blinded to the synthetic patient data sets, and ensures the presumed independence from any intra-subject correlations that are required for Lasso classification. We fixed the number of folds of the data as $\mathcal{K} = 5$. Running the analysis on each fold, the predicted outcome will be the one with the least estimated prediction error. The regularization parameter $\alpha$ is estimated by a cross-validation procedure.

2.3.1.   Evaluating model performance

The discriminating ability of the developed model in predicting patients with influenza from COVID-19 was evaluated using AUC (Area Under The Curve) ROC (Receiver Operating Characteristics) curve analysis. AUC - ROC curve is one of the most important evaluation metrics to visualize the performance of multi-class classification problems. ROC represents a probability curve of sensitivity (true positive rate = $\frac{TP}{TP+FN}$) against 1-specificity (false positive rate = $\frac{FP}{FP+TN}$) and AUC is a performance measure of discrimination. In the other words, the AUC score is a criterion that explains how well the model is capable of discerning different cohorts. Generally, an AUC closer to 1 indicates a better overall diagnostic performance of influenza classes as influenza or COVID-19 to COVID-19.

2.4.   Model interpretability

From ^[36,37], "Interpretability" is the degree to which a human can understand the cause of a decision and consistently predict the model's result. The higher the interpretability of a machine learning model, the better understanding of why certain predictions have been made. Interpretable machine learning models are beneficial to extract the relevant knowledge from relationships either contained in data or learned by the model ^[38,39].

Here, we looked at the regularization path which is a plot of all coefficients values against the values of $\alpha$ in-$\ell_1$ penalization term, to see the behavior of the Lasso regression and interpret the prediction outcomes. The main purpose of Lasso regression is to classify groups of data by providing feature coefficients that can select the important features and maintain model regularization to avoid over-fitting the data. Therefore, the Lasso path can give us an idea of the feature's importance.

3.   Results

3.1.   Prediction of Influenza versus COVID-19 infection

In this study we developed a classifier in the Lasso framework to identify patients with either influenza or COVID-19, based on four major entities of viral dynamics, $\{T(t), I_1(t), I_2(t), V(t)\}$, and one main factor of host immune response, type I interferon $(F(t))$, as the entry data features. The model was trained on data from 100 virtual patient-level data sets in each infection cohort without noise, and it was externally validated on testing sets with demographic noise (reflected in diverse viral load levels). Results in Figures 4, 5 and 6 reflect the Lasso predictions using the entire infection period (see Section 2.2.1). In Figures 4 and 5, two-dimensional scatter plots are used to compare ground truth to regression predicted values based on all model features. The hue spectrum from light to dark illustrates the probability of being in the influenza (blue) or COVID-19 (red) group. In the other words, the darker the colors, the better the prediction. Considering three attributes in the data, the predicted outcomes are improved. This is shown in three-dimensional scatter plots in Figure 6 of the ground truth and regression predicted values. ROC AUC = $95\%$ indicates a satisfactory performance of the model to distinguish between COVID-19 and influenza patients. We note that our analysis was also completed on data from 1000 virtual patients, and a similar result was obtained, ROC AUC = $93\%$. See Figure 7 for more details.

3.1.1.   Early days of infection

We examined the model prediction for the data generated at the early days of infection after the incubation period. The results are shown in Figure 8 based on the model features. From the figure, we can see that there are some mispredictions, for small values of $I_1(t), I_2(t), V(t)$, and $F(t)$, especially when $I_2(t)$ is plotted as a function of $I_1(t)$ or $V(t)$ is plotted in terms of $I_2(t)$. In the other words, for this range of values, the influenza patients were misdiagnosed with COVID-19. In an attempt to find the reason, we compared correlations between the different variables in our model. See Figure 9. Here, we see small regions of overlap between influenza and COVID-19 models. Accordingly, the compatibility of the results between the two infections may lead to some overlaps in the model predictions. However, the ability of the model in the prediction of infections when the patients were monitored by $V(t)$/$F(t)$ as a function of $I_1(t)$, panels (b) and (c), or $F(t)$ in terms of $I_2(t)$/$V(t)$, panels (e) and (f), can be satisfactory, and thus can serve as benchmarks for clinical diagnosis. The model had a ROC AUC of $91\%$ on the external validation data set for early infection – see Figure 7.

3.2.   Significance of the features

To investigate the importance of various data features we created our $\ell_1$-regularization path, which was the best way to see the behavior of the Lasso regression. The regularization path is a plot of all coefficient values in terms of the regularization parameter. Figure 10 illustrates the selection path of each feature with its corresponding coefficient in terms of the logarithm of the regularization parameter $\alpha$. For each value of $\alpha$, the path method on the Lasso object returns the coefficients that solve the logistic regression problem with that parameter value. The optimal value of $-\log (\alpha)$ was estimated at around 3.25 for the test set distributed over the entire infection course, and 3.04 when the early days of infection were studied. The results suggested a higher coefficient value for viral load $V(t)$ and productively infected cells $I_2(t)$ compared to the other features. In the same analysis on 1000 virtual patient data sets, the viral load had the predominant identifying role.

4.   Discussion

This study presents a machine learning model to effectively classify influenza and COVID-19 virtual patients using in-host patient data. Our model employed a Lasso regression classifier trained to identify between two hundred patients, highlighted by a ROC AUC of $95\%$. Using within-host model structures from the literature, we generated synthetic data with five in-host measurements including target cells, eclipse phase, and productively infected cells, viral load, and type I IFN. Analyzing the feature importance revealed that the viral load and the productively infected cells are the most important components to determine if a patient is infected by influenza or SARS-CoV-2.

While our machine learning model was built on synthetic data distributed during the main infection period, it ascertained a good performance (ROC AUC = $91\%$) even for the early days of infection after the incubation period. However, in early infection, there were some exceptions for the small values of in-host features where the influenza patients were misdiagnosed as COVID-19. The reason was explained by the fact that during the early days of infection, influenza and COVID-19 patients have comparable in-host measurements that lead to some errors in discriminating the patients. This is interpreted as a limitation of our model even though the ROC AUC was still very high. A future extension of our work here will be in developing dynamic models which take more immune entities into account and end in a better classifier.

Our model was trained and successfully evaluated on synthetic data. The model, however, could be applied to animal or human clinical data. This could be useful, for example, if a clinical trial is complicated by the existence of an infectious disease with similar infection characteristics. The model could be applied as a low-cost classification system that would not require expensive virus typing procedures and could rely solely on viral load and interferon measurements. Additionally, the AI/ML method could be applied to determine when a clinical trial using a continuous enrollment design has accumulated sufficient data to determine whether a new pharmaceutical is effective. We note that studies like ^[8] that focus analysis on demographic and observational data can be cheaper to conduct than a study requiring viral load or immune system measurements, but these data can also be subject to inconsistencies and bias, affecting classification outcomes. In a future study, we will expand our analysis to a model of in-host measurements and observational data to determine if specific combinations of in-host and observational data that best classify influenza and COVID-19 infections differ.

Fourteen different AI/ML techniques in disease predictions were reviewed in ^[40]. Quiroz-Juárez and et al. developed an effective machine-learning algorithm for the identification of high-risk COVID-19 patients ^[41]. Some AI approaches that have significant contributions in the fields of health care were presented in ^[42] and their applications in confronting COVID-19, such as diagnosis and drug development were studied. Salehi and et al. studied machine and deep learning-based architectures performance for classification of coronavirus images such as X-ray and computed tomography ^[20]. Our machine learning model was developed in the Lasso framework. Ridge regression or Partial least squares discriminant analysis (PLS-DA) also can be employed, and require only small changes to our method to include this. The model demonstrated a satisfactory performance by using either Ridge or PLS regression – (ROC AUC = $95\%$) for the main infection period and –(ROC AUC = $89\%$) for the early days of infection.

Acknowledgments

We acknowledge the support of the Natural Sciences and Engineering Research Council of Canada (NSERC) and the NRC Pandemic Response Challenge Program Grant No. PR016-1.

Conflict of interest

The authors declare there is no conflict of interest.