Identifiability investigation of within-host models of acute virus infection

1.   Introduction

The study of host-virus interactions using dynamical models (within-host models) has improved our understanding of the mechanistic interactions that govern chronic infections caused by pathogens such as human immunodeficiency virus ^[1,2] and hepatitis B virus^[4], and mechanistic interactions that govern acute infections caused by pathogens such as influenza virus ^[5,6], dengue virus ^[7,8], Zika virus ^[10], and severe acute respiratory syndrome coronavirus 2 ^[11,12]. Regardless of the virus considered, the most basic within-host model has a general structure that includes the interaction between the cells susceptible to the virus, the cells infected by the virus, and the virus at short (acute) and long (chronic) timescales. The emergence of unexpected dynamics in the virus data, new information about the virus' life-cycle, data describing host immunity to the infection, or a combination of some or all of the above, may require addition of complexity into the within-host modeling process (see ^[13,14] for a review).

Data fitting techniques for simple or complex within-host models use (normalized) least-squares approaches, in which the Euclidean distance between the data and the mathematical model is minimized with respect to the unknown parameters. The first step in the parameter estimation algorithm is to provide an initial guess for each parameter based on prior biological knowledge, such as the duration of eclipse stages, life-span of an infected cell and/or virus in vitro, and knowledge from modeling of virus dynamics of related viruses. When prior knowledge is unknown, the user makes the assumption that any positive parameter guess is acceptable. Then, an optimization search algorithm is employed until a termination criterion is reached. For many within-host mathematical models and corresponding datasets, the optimization is ill-posed due to the structure of the model and/or the frequency of the data ^[15]. As a result, some parameters may be difficult or impossible to quantify. To determine whether the uncertainty in parameter estimations is due to the model or the data, both structural and practical identifiability questions need to be addressed.

Structural identifiability investigates the ability of a model to reveal its unknown parameters from noise-free infinite amount of data ^[16,17,18]. When nonstructural identifiability of parameters occurs, it is important to find the source of non-identifiability, such as correlation between model parameters. This allows the user to propose additional assumptions needed to make the model structurally identifiable. Only after the structural identifiability of the unknown parameters is guaranteed, can one conduct data fitting schemes to estimate parameter values.

Practical identifiability investigates the ability of a model to reveal unknown structurally identifiable parameters under scarce and noisy (among subjects) data, often examined using Monte Carlo simulations ^[18,19,20], the Fisher information matrix (FIM) or correlation matrix ^{[16,21,22,23]}, Bayesian method ^[24], and the profile likelihood method ^[25,26,27]. As with the structural identifiability, it is important to identify whether the practical identifiability issues are due to model structure. Additionally, it is important to determine whether increased data frequency, availability of data measurements for more than one model variable, and/or relaxing restrictions imposed on the unknown parameters can improve practical identifiability issues.

To address these important considerations in model validation, one needs to compare a set of models for the same virus infection system and the same empirical data. Here, we accomplish that by investigating four previously developed models of influenza A virus (IAV) infection in mice ^[28]. The first three models, all validated with the same virus titer dataset, are ranging from the basic within-host model to models with increased complexity through the addition of nonlinear terms and/or the inclusion of additional variables for the host cell populations infected by the influenza virus. The fourth model is the most complex, due to the addition of both nonlinear terms and variables for the host immune system. This results in a large number of unknown parameters. To compensate for the added complexity, this model is validated with two datasets: the same virus titer data and an additional immune cell population data.

The goal of this study is to determine how model complexity and data availability induce uncertainty in parameter estimates. Using the proposed models as proof of concept, we aim to provide a framework for model validation, from structural to practical identifiability, that can be generalized to other models of virus infections.

2.   Within-host influenza models

We consider four within-host models of acute infections used to describe influenza A virus infection in mice ^[5]. They all describe the same influenza A virus titer data, but they account for increased biological complexity, as follows. Model 1 assumes that influenza A virus infects all available susceptible target cells before being cleared according to first order infected cells death and viral clearance rates (target cell limitation); Model 2 explains an observed viral biphasic decay in the data by assuming a second order (density dependent) infected cell killing rate; Model 3 explains an observed viral expansion delay in the data by assuming the presence of an eclipse phase; and Model 4 utilizes a secondary immune cells dataset by adding a model population that describes immune-mediated antiviral responses. With each model, we include biological realism that describes the dynamics of virus expansion and decay in more detail, while at the same time increasing model complexity through the addition of nonlinearities and increased numbers of model parameters. The flow charts of the four models are presented in Figure 1. Below we describe all models in detail, and address the ability of accounting for complexity given the available data by investigating structural and practical identifiability of each model considered.

Model 1 is the classical target-cell limitation model of viral infection, which considers the interaction between target cells, infected cells, and virus as follows ^[5,28]. Target cells, $T$, interact with the virus, $V$, at rate $\beta$ to become infected cells $I$. Infected cells die at per capita rate $\delta$ and produce virus at rate $\pi$. Virus is cleared at rate $c$. Model 1 is described by the system of ordinary differential equations (ODE) Eq (2.1) below,

with initial conditions $T(0) = T_0,$ $I(0) = I_0,$ and $V(0) = 0$.

Experimental data has shown that, following peak expansion, virus decays in a biphasic manner. To capture the dynamics of viral decay, a modified death rate has been considered. It assumes that the rate of infected cell clearance increases as the density of infected cells decreases, as described by $\delta/(K_{\delta}+I)$, where $\delta$ is the maximum per capita death rate and $K_{\delta}$ is infected cell population where death rate is half-maximal ^[28]. This leads to the modified target-cell limitation Model 2 given by the ODE system Eq (2.2) below,

with initial conditions $T(0) = T_0,$ $I(0) = I_0,$ and $V(0) = 0$.

It was observed experimentally that, following influenza A virus exposure, there is a delay between infection of target cells and viral production by infected cells ^[29]. The delay was accounted for by assuming that, upon infection, cells enter an eclipse phase $I_1$, where cells are infected but do not produce virus. They become productively infected $I_2$ after $1/k$ days ^[6], where $1/k$ is the average time spent in eclipse phase. This leads to the target-cell limitation model with eclipse phase Model 3 given by the ODE system Eq (2.3) below,

with initial conditions $T(0) = T_0,$ $I_1(0) = I_0$, $I_2(0) = 0,$ and $V(0) = 0$.

The first three models do not explicitly account for any immune responses, but indirectly assume infected cell loss at nonspecific rate $\delta$ (or $\delta/(K_\delta +I_2)$) and viral clearance at nonspecific rate $c$. The observed biphasic viral decay captured by Models 2 and 3 given by Eqs (2.2) and (2.3), however, has the additional feature that the timing of the second phase viral decay coincides with the development of adaptive immune cells in the form of $\mathrm{CD8}^+$ T cells, which are responsible for killing infected cells and resolving the infection ^[5]. To account for adaptive immunity (especially in the presence of immune cell data), an additional variable $E$ is considered. It only accounts for the effector $\mathrm{CD8}^+$ T cell population (and ignores the memory $\mathrm{CD8}^+$ T cell population), as follows. In the absence of infection, a baseline of influenza A virus-specific effector $\mathrm{CD8}^+$ T cells are present, $E(0) = E_0$. Infection results in recruitment of additional effector $\mathrm{CD8}^+$ T cells at a rate proportional to the productively infected cells $I_2$. This is modeled in a density dependent manner at rate $\lambda/(K_E+E)$, where $\lambda$ is the maximum influx and $K_E$ is the effector $\mathrm{CD8}^+$ T cell population where the influx is half-maximal. Effector $\mathrm{CD8}^+$ T cells proliferate in the presence of infection. This is modeled by a delayed term $\eta I_2(t-\tau_I) E$, which assumes that expansion occurs following interaction between effector $\mathrm{CD8}^+$ T cells and cells that became productively infected $\tau_I$ days ago. To account for effector $\mathrm{CD8}^+$ T cells function, the model assumes that effector $\mathrm{CD8}^+$ T cells kill infected cells in a density dependent manner modeled by the term $\delta_E/(K_{\delta}+I_2)$, where $\delta_E$ is the maximum per capita killing rate and $K_{\delta}$ is the $I_2$ concentration where the killing is half-maximal. A nonspecific infected cell killing rate $\delta$ is still considered. The resulting delay differential equations (DDE) immune model is described by the DDE system Eq (2.4) below,

with initial conditions $T(0) = T_0,$ $I_1(0) = I_0$, $V(0) = 0,$ $E(0) = E_0$, and $I_{2}(t) = 0$ for $-\tau_I\leq t\leq 0$.

To unify the goal of investigating uncertainty in parameter estimates when fitting ODE systems of virus dynamics to data, we first approximate the DDE system given by Eq (2.4) with an ODE system as follows ^[30]. For a delay of $\tau_I$ days, we incorporate $n$ dummy variables which all span $\tau_I/n$ days in the variable $I_2$'s dynamics. Briefly, we let $y_i$ be the productively infected cell populations at times $t-\frac{i}{n}\tau_I$ days post infection, for $i = 1, ..., n$, and consider the following ODE system for dummy variables $y_i(t)$,

with $y_i(0) = 0$ for $i = 1, ..., n$. Then, the delayed productively infected cell population is given by

Without loss of generality, we assume $n = 3$. The corresponding immune Model 4 is given by the ODE system Eq (2.7) below,

with initial conditions $T(0) = T_0,$ $I_1(0) = I_0$, $I_2(0) = 0$, $V(0) = 0,$ $E(0) = E_0$, and $y_i(0) = 0$ for $i = 1, 2, 3$.

3.   Structural identifiability analysis

To study the structural identifiability of the Models 1–4, we rewrite them in the following general form

and the observations as

Here, $x$ denotes the state variables, $\boldsymbol{p}$ is the parameter vector, and $y$ is the output (given by the empirical data), also called the observations. The generic model given by Eq (3.1) is called structurally identifiable if the parameter vector $\boldsymbol{p}$ can be determined uniquely from the observations given by the smooth curve $y(t)$. Otherwise, it is said to be unidentifiable. The formal definition of structural identifiability is provided below.

Definition 1. Let $\boldsymbol{p}$ and $\hat{{\boldsymbol{ p }}}$ be two distinct parameter vectors. Model Eq (3.1) is said to be globally (uniquely) structurally identifiable if and only if,

Definition 2. Model Eq (3.1) is said to be locally structurally locally identifiable if for any $\boldsymbol{p}$ within an open neighborhood of $\hat{{\boldsymbol{ p }}}$ in the parameter space,

Various methods have been proposed for analyzing the structural identifiability of ODE models ^[16,17,31]. In this study, we employ the differential algebra approach. It performs the elimination of unobserved state variables, resulting in equations expressed as functions of model parameters and observed state variables. These are referred to as the input-output equations, and are differential-algebraic polynomials consisting of the outputs, $y(t)$, with model parameters, $\boldsymbol{p}$, as coefficients. The formal definition of structural identifiability within the differential algebra approach for model Eq (3.1) is provided below.

Definition 3. Let $c({\boldsymbol{p}})$ denote the coefficients of the input-output equation corresponding to model Eq (3.1). We say that model Eq (3.1) is structurally identifiable from unlimited observations $y(t)$ if and only if,

Studying structural identifiability of ODE models using the differential algebra methods can be accomplished using several platforms and available open-source software. Here, we present three such platforms: the differential algebra for identifiability of system (DAISY) ^[32], the identifiable combinations web application (COMBOS) ^[33], and the StructuralIdentifiability.jl in JULIA ^[34].

There are many similarities among the three methods. All of them offer insights into the structural identifiability status of each parameter by categorizing them into locally identifiable, globally identifiable, or non-identifiable. They employ a differential elimination method to calculate input-output equations of the considered system, and test the one-to-one map between the coefficients of the input-output equations and model parameters. COMBOS and the StructuralIdentifiability.jl package in JULIA are superior to DAISY, as they provide globally identifiable parameter correlations in an otherwise non-identifiable system. Even though DAISY does not print parameter correlations, the correlations can be derived using the coefficients of the input-output equations and algebraic manipulations in software such as MATHEMATICA. Of the three software, COMBOS does not print the input-output equations, making for a faster (yet more opaque) platform. Previous studies have shown that COMBOS works best for small to medium-size models and is not assured for models with large parameter vectors ^[33,35,36]. While highly similar, it is up to the user to determine which software is best suited for studying the identifiability of the models considered.

3.1.   Structural identifiability results

To determine whether the considered models can reveal their parameters, we examine the structural identifiability of Models 1–3, given by Eqs (2.1)–(2.3), under unlimited observations of viral load and the structural identifiability of Model 4, given by Eq (2.7), under unlimited combined observations of viral load and effector $\mathrm{CD8}^+$ T cell concentrations. We used the differential algebra software DAISY.

3.1.1.   DAISY-based structural identifiability results for Model 1

We assume that all Model 1's parameters ${\boldsymbol{p}} = \{\beta, \delta, \pi, c\}$ are unknown and that we have unlimited empirical observations of the viral load, $y(t) = V(t)$. Using DAISY ^[32], we obtain the following input-output equation in variable $V$ and model parameters $\boldsymbol{p}$,

By Definition 3, we need to examine whether another set of parameters, $\hat{ {\boldsymbol{p}}} = \{\hat{\beta}, \hat{\delta}, \hat{\pi}, \hat{c}\}$ can produce the same empirical observation $V(t)$, making the map from the parameter space $\boldsymbol{p}$ to the coefficients of input-output equation Eq (3.3) one-to-one. The coefficients of input-output equation Eq (3.3) are $c({\boldsymbol{p}}) = \{ \beta, c+\delta, c \delta\}$. To determine whether the map from the parameter space ${\boldsymbol{p}}$ to the coefficients $c({\boldsymbol{p}})$ is one-to-one, we set $c({\boldsymbol{p}}) = c(\hat{{\boldsymbol{p}}})$, which is the following system:

Solving Eq (3.4) results in the following two sets of solutions:

Hence, only the infection rate $\beta$ is globally structurally identifiable, while the infected cells killing rate $\delta$ and the virus clearance rate $c$ are locally identifiable. Lastly, the virus production rate $\pi$ does not appear in the input-output equation Eq (3.3). Therefore, it is not structurally identifiable. We summarize the results for Model 1 below (see Table 1).

Proposition 1. Model 1 given by Eq (2.1) is not structured to identify all of its parameters from unlimited viral load observations, $V(t)$. More precisely, parameter $\beta$ is globally structurally identifiable, parameters $c$ and $\delta$ are locally structurally identifiable, and parameter $\pi$ is not structurally identifiable. Moreover, Model 1 is globally structural identifiable under known initial conditions.

3.1.2.   DAISY-based structural identifiability results for Model 2

We assume that all parameters ${\boldsymbol{p}} = \{\beta, \delta, K_\delta, \pi, c\}$ of Model 2, given by Eq (2.2), are unknown and that we have unlimited empirical observations of the viral load, $y(t) = V(t)$. Using DAISY, we obtain the following input-output equation,

As before, we examine whether another set of parameters, $\hat{ {\boldsymbol{p}}}$, can produce the same empirical observation $V(t)$, making the map from the parameter space $\boldsymbol{p}$ to the coefficients of input-output equation Eq (3.5) one-to-one. If we set $c({\boldsymbol{p}}) = c(\hat{{\boldsymbol{p}}})$, we obtain

with solutions

Hence, Model 2 is not structurally identifiable. In particular, infection rate $\beta$, viral clearance rate $c$, and the products $\delta \pi$, $K_\delta \pi$ (but not the individual parameters $\delta$, $\pi$, and $K_\delta$) are globally identifiable. Since the correlations $\delta \pi$ and $K_\delta \pi$ are known, fixing one of these parameters can make model Eq (2.2) identifiable. We summarize the structureal identifiability results for Model 2 below (see Table 1).

Proposition 2. Model 2 given by Eq (2.2) is not structured to identify all of its parameters from unlimited viral load observations, $V(t)$. More precisely, parameters $\beta$ and $c$ are globally structurally identifiable. Moreover, the parameter products $\delta \pi$ and $K_\delta \pi$ are globally identifiable. Since the correlations are known, fixing $\delta$, $\pi$, or $K_\delta$ makes the Model 2 globally structurally identifiable from unlimited observations $V(t)$. Moreover, Model 2 is globally structural identifiable under known initial conditions.

3.1.3.   DAISY-based structural identifiability results for Model 3

We assume that all parameters ${\boldsymbol{p}} = \{\beta, \delta, k, \delta_E K_\delta, \pi, c\}$ of Model 3, given by Eq (2.3), are unknown and that we have unlimited empirical observations of the viral load, $y(t) = V(t)$. Using DAISY, we can derive the input-output equations (they are too messy and will not be presented here). As before, we examine whether another set of parameters, $\hat{ {\boldsymbol{p}}}$, can produce the same empirical observation $V(t)$, making the map from parameter space $\boldsymbol{p}$ to coefficients of input-output equation (not shown) one-to-one. If we set $c({\boldsymbol{p}}) = c(\hat{{\boldsymbol{p}}})$, we obtain

Hence, Model 3 is not structurally identifiable. In particular, the infection rate $\beta$, the eclipse parameter $k$, the viral clearance rate $c$, the ratio $\delta/ K_\delta$, and the product $K_\delta \pi$ (but not the individual parameters $\delta$, $\pi$, and $K_\delta$) are globally identifiable. Since the correlations are known, fixing one of these parameters makes the model Eq (2.3) identifiable. We summarize the results for Model 3 below (see Table 1).

Proposition 3. Model 3 given by Eq (2.3) is not structured to identify all of its parameters from unlimited viral load observations, $V(t)$. More precisely, parameters $\beta$, $k$, and $c$ are globally structurally identifiable. Moreover, the parameter ratio $\delta/K_\delta$ and the parameter product $K_\delta \pi$ are globally identifiable. Since the correlations are known, fixing the parameter $\delta$, $\pi$, or $K_\delta$ makes Model 3 globally structurally identifiable from unlimited observations $V(t)$. Moreover, Model 3 is globally structural identifiable under known initial conditions.

3.1.4.   DAISY-based structural identifiability results for Model 4

To study the structural identifiability of Model 4 (given by Eq (2.7)), we assume that all parameters, ${\boldsymbol{p}} = \{\beta, \delta, k, \delta_E K_\delta, \pi, c\, \lambda, \eta, d_E, \tau_I, E_0\}$, are unknown and that we have unlimited empirical observations for the viral load $y_1(t) = V(t)$ and the effector cell CD8$^+$ T cell data $y_2(t) = E(t)$. Using DAISY, we can obtain input-output equations (they are messy and will not be presented here). As before, we examine whether another set of parameters, $\hat{ {\boldsymbol{p}}}$, can produce the same empirical observations $V(t)$ and $E(t)$, making the map from the parameter space $\boldsymbol{p}$ to the coefficients of input-output equations (not shown) one-to-one. If we set $c({\boldsymbol{p}}) = c(\hat{{\boldsymbol{p}}})$, we obtain

Hence, Model 4 is not structurally identifiable. In particular, the infection rate $\beta$, the eclipse parameter $k$, the viral clearance rate $c$, the effector cells death rate $d_E$, the generic killing rate $\delta$, the half-maximal level $K_E$, the delay $\tau_I$, the ratios $\lambda/ \eta$ and $\pi/\eta$, and the products $d_E \eta$ and $K_\delta \eta$ (but not the individual parameters $\delta_E, \eta, K_\delta, \pi, \lambda$) are globally identifiable. If the parameter $\eta$ is fixed, then the model Eq (2.7) becomes identifiable. We summarize the results for Model 4 below (see Table 1).

Proposition 4. Model 4 given by Eq (2.7) is not structured to identify all of its parameters from unlimited viral load and effector cell observations, $V(t)$ and $E(t)$. More precisely, parameters $\beta$, $k$, $c$, $d_E$, $\delta$, $K_E$, and $\tau_I$ are globally structurally identifiable. Moreover, the parameter ratios $\lambda/ \eta$ and $\pi/\eta$ and the parameter products $d_E \eta$ and $K_\delta \eta$ are globally identifiable. If the parameter $\eta$ is fixed, then Model 4 becomes globally structurally identifiable from unlimited observations $V(t)$ and $E(t)$.

We do not know (from DAISY) whether knowing initial conditions guarantees global stability of Model 4 (see Table 1).

3.2.   Comparison among structural identifiability software

Studying structural identifiability of ODE models can be achieved using software other than DAISY To determine how these methods compare, results from three platforms, DAISY, COMBOS ^[33], and StructuralIdentifiability.jl in JULIA ^[34], for Models 1–4 are presented side by side in Table 1.

We find that all three software uncover the same structural identifiability results for Models 1–3. On the other hand, DAISY and StructuralIdentifiability.jl in JULIA uncover the same identifiability results (while COMBOS cannot find results) for Model 4 under unknown initial conditions. Even though Models 3 and 4 employ different interpretations of the parameter correlations among platforms, simple algebraic manipulations show that the obtained correlations are equivalent. Given the similarity in the results among Models 1–3, it is up to the user to decide which of the three software is best suited for their analysis. Similarly, given the similarity in the results among DAISY and StructuralIdentifiability.jl in JULIA for Model 4 with unknown initial conditions, it is up to the user to decide which of the two software is best suited for their analysis. However, only StructuralIdentifiability.jl in JULIA can be used to determine the structural identifiability of Model 4 with unknown $E_0$ and known other initial conditions. Hence, for larger systems with nonlinear terms of interactions, this method should be employed.

4.   Data fitting methods

4.1.   Empirical data

We use previously published longitudinal influenza A infectious virus titer and $\text{CD8}^+$ T cell data in mice from Smith et al. ^[5]. Adult mice were inoculated intranasally with 75 $\text{TCID}_{50}$ of mouse adapted influenza A/Puerto Rico/8/34 (H1N1) (PR8) virus.

Total infectious virus ($\log10 \text{ TCID}_{50}$ per lung) was measured for ten mice each day. Nine days after inoculation, the infectious virus was no longer detectable in any of the mice. Therefore, we only consider infectious virus titer data from the first nine days post inoculation in our analyses. We let $\mathbb{E}(V_{data}(i))$ be the mean infectious virus titer data at day $i = \{1, ..., 9\}$ and $\mathbb{V}\textbf{ar}(V_{data}(j))$ be the infectious virus titer variance at days $i = \{1, ..., 9\}$ among the ten mice.

Moreover, total effector $\text{CD8}^+$ T cells (cells per lung) were measured daily for five mice. Since influenza A-specific effector $\text{CD8}^+$ T cells were detectable for all twelve days of the study, we consider effector $\text{CD8}^+$ T cells data from the first twelve days post inoculation in our analyses. We let $\mathbb{E}(E_{data}(j))$ be the mean $\text{CD8}^+$ T cell data (per lung) at day $j = \{1, ..., 12\}$ and $\mathbb{V}\textbf{ar}(E_{data}(j))$ be the $\text{CD8}^+$ T cell data variance at days $j = \{1, ..., 12\}$ among the five mice.

4.2.   Model parameters and initial conditions

For all models, we assume known initial conditions $T(0) = 10^7$ cells/ml, $I(0) = 75$ cells/ml, and $V(0) = 0$ virus/ml as in ^[5]. For Models 3 and 4, we additionally assume that $I_2(0) = 0$, and for Model 4, we assume $y_i(0) = 0$, for $i = 1, 2, 3$. For Model 4, we assume $E(0) = E_0$ is unknown, therefore adding $E_0$ to the parameter vector to be estimated from the data. Lastly, we assume all parameters are unknown. When parameters are either very large or very small, we estimate their value on natural logarithmic scale. In particular, we estimate $\bf{p_1} = \{ \ln(\beta), \delta, \pi, c \}$ for Model 1, $\bf{p_2}$$= \{ \ln(\beta), \ln(\delta), \ln(K_{\delta}), \pi, c \}$ for Model 2, $\bf{p_3}$$= \{\ln(\beta), \ln(\delta), \ln(K_{\delta}), \pi, c, k\}$ for Model 3, and $\bf{p_4}$$= \{\ln(\beta), \delta, \ln(K_{\delta}), \pi, c, k, \delta_E, \ln(\eta), \ln(\lambda), d_E, \ln(K_E), \tau_I, \ln(E_0) \}$ for Model 4.

4.3.   Data fitting algorithm

To estimate parameters $\bf{p_1}$–$\bf{p_3}$, we fit the predicted viral load $\log_{10} V^w_{model}(i)$ given by Models 1–3 to the longitudinal mean (among the ten mice) infectious virus ($\log_{10} \text{ TCID}_{50}$ per lung) $\mathbb{E}(V_{data}(i))$, knowing that the variance in the data at day $i$ is $\mathbb{V}\textbf{ar}(V_{data}(i))$, for $i = \{1...9\}$ days. We assume that the data satisfies the following statistical model ^[19,37]

where $V^w_{model}(i)$ is the predicted virus trajectory given by Model w at days $i = \{1, ..., 9\}$ post infection; $\bf{p_1}$$= \{ \ln(\beta), \delta, \pi, c \}$, $\bf{p_2}$$= \{ \ln(\beta), \ln(\delta), \ln(K_{\delta}), \pi, c \}$, and $\bf{p_3}$$= \{\ln(\beta), \ln(\delta), \ln(K_{\delta}), \pi, c, k\}$; and $\epsilon_i$ are independent and identically distributed with mean zero and standard deviation $\sigma$. Given the statistical model Eq (4.1), we assume that the measured data, $\mathbb{E}(V_{data}(i))$, follows a normal distribution with a mean equal to the model prediction $\log_{10} V^w_{model}(i)$ and with variance equal to $\sigma^{2}\mathbb{V}\textbf{ar}(V_{data}(i))$. Moreover, the availability of measurements from several animals that vary with time allows us to account for the change in data variance over time, $\mathbb{V}\textbf{ar}(V_{data}(i))$. Therefore, we consider the following functional (weighted residual sum of squares), to estimate the model parameters,

Consequently, parameters of Models 1–3 are estimated by minimizing the weighted least-squares given by

Moreover, to estimate parameters $\bf{p_4}$, we fit both the predicted viral load $\log_{10} V_{model}^4(i)$, given by Model 4, to the longitudinal mean (among ten mice) infectious virus $\mathbb{E}(V_{data}(i))$ (knowing that the variance in the data at days $i = \{1...9\}$ is $\mathbb{V}\textbf{ar}(V_{data}(i))$) and the predicted effector cell population $\log_{10} E_{model}^4(j)$, given by Model 4, to the longitudinal mean (among five mice) $\mathrm{CD8}^+$ T cell data $\mathbb{E}(E_{data}(j))$ (knowing that the variance in the data at days $j = \{1...12\}$ is $\mathbb{V}\textbf{ar}(E_{data}(j))$). We assume that the data is satisfying the following statistical model ^[19,37]

where $V^4_{model}(i)$ is the predicted virus trajectory given by Model 4 at days $i = \{1, ..., 9\}$ post infection, $E^4_{model}(j)$ is the predicted $\mathrm{CD8}^+$ T cell population given by Model 4 at days $j = \{1, ..., 12\}$ post infection, and $\bf{p_4}$$= \{\ln(\beta), \delta, \ln(K_{\delta}), \pi, c, k, \delta_E, \ln(\eta), \ln(\lambda), d_E,$ $\ln(K_E), \tau_I, \ln(E_0) \}$. Here, $\epsilon_i$ and $\eta_j$ are independent and identically distributed with mean zero and standard deviations $\sigma_V$ and $\sigma_E$, respectively. As before, the measured data $\mathbb{E}(E_{data}(j))$ follows a normal distribution whose mean is the model prediction $\log_{10} E^w_{model}(i)$ and whose variance is $\sigma_E^{2}\mathbb{V}\textbf{ar}(E_{data}(j))$. We consider the following functional (weighted residual sum of squares),

Consequently, parameters of Model 4 are estimated by minimizing the weighted least-squares given by,

Note that we weighted the virus and effector cells contributions, with weights $u_1 = 1$ and $u_2 = \max_j \mathbb{V}\textbf{ar}(E_{data}(j))/\max_i \mathbb{V}\textbf{ar}(V_{data}(i))$. We minimize all least-square functionals $\text{RSS}_w$ using the fmincon function in MATLAB with ranges for parameters $\mathbf{p_w}$ given in Table 2.

4.4.   Model selection

To compare Models 1–4, we calculate the corrected Akaike information criteria (AICc), given below

where $n$ is the number of data points used to estimate parameters $\mathbf{p_w}$ and $M$ is the number of estimated parameters. In Models 1–3, $n = 9$ and $M = 4, 5$, and $6$, respectively. In Model 4, $n = 21$ and $M = 13$.

4.5.   Model prediction confidence interval

To quantify the uncertainty associated with predicted solutions of each model, we perform parametric bootstrapping. It is a simulation-based method which assumes that data comes from a known distribution with unknown parameters. For Models 1–3, we assume that the predicted viral population for best parameter estimates, $\log_{10} V^w_{model}(i, \mathbf{p_w})$, is the mean of the data's normal distribution and $\sigma^2 \mathbb{V}\textbf{ar}(V_{data}(i))$ is its variance (see Eq (4.1)). Then, $\sigma$ can be approximated as follows,

(see Banks et al. for a full derivation ^[37]). Here, $n = 9$ is the number of viral samples and $M$ is the number of parameters ($M = 4$, $M = 5$, and $M = 6$ for Models 1–3, respectively). To find a confidence region in our model predictions, we generate $1000$ simulated datasets using the distribution space given by Eq (4.1), and fit Models 1–3 to each datasets.

Similarly, for Model 4, assuming that viral data and effector cell data come from distributions with means $\log_{10} V^4_{model}(i, \mathbf{p_4})$ and $\log_{10} E^4_{model}(i, \mathbf{p_4})$ (the predicted variables for best parameter fits) and that $\sigma_V^2 \mathbb{V}\textbf{ar}(V_{data}(i))$ and $\sigma_E^2 \mathbb{V}\textbf{ar}(E_{data}(j))$ are the variances, then

and

as before. Here, $n_V = 9$ is the number of viral samples, $n_E = 12$ is the number of $\mathrm{CD8}^+$ T cell samples, $n_{tot} = n_V+n_E = 21$ is the number of total data samples, and $M = 13$ is the number of parameters fitted.

5.   Data fitting results

We fitted Models 1–3 to previously published longitudinal influenza A infectious virus titer and we fitted Model 4 to both longitudinal influenza A infectious virus titer and longitudinal $\text{CD8}^+$ T cell data in infected mice ^[5], using a normalized least-square optimization algorithm (see Section 4). The results from fitting $V(t)$ given by Models 1–3 to viral load data are shown in Figure 2A–C and the best parameter fits are given in Table 3. The results from fitting both $V(t)$ and $E(t)$ given by Model 4 to viral titer and effector cell data are shown in Figure 2D and the best parameter fits are given in Table 3. Model selection, using the corrected $AIC_c$, predicts that Model 4 best describes the data (see Table 3). To quantify the uncertainty associated with predicted solutions of each model, we find a $90\%$ confidence region in our model predictions (see Section 4.5), illustrated by shaded gray areas in Figure 2A–C for Models 1–3 and by gray and blue shaded regions in Figure 2D for Model 4. We see large error regions in virus population predictions for all models during the decay phase (gray regions in Figure 2A–D). Moreover, Models 2–4 better capture the virus population expansion phase compared to Model 1 (gray regions in Figure 2B–D versus gray region in Figure 2A). Lastly, the largest error in $\mathrm{CD8}^+$ T cell prediction in Model 4 occurs in the second week of infection (blue region in Figure 2D).

6.   Practical identifiability analysis

While structural identifiability investigates whether parameters can be uniquely determined from a model given unlimited data in the absence of measurement error or noise, practical identifiability determines whether parameters can be accurately identified in real-world scenarios, where observed discrete and variable among subject data is contaminated with measurement errors. We and others have employed several methods to study practical identifiability of within-host and infectious disease models, such as Monte Carlo simulations ^[18,19,20], Fisher information matrix or correlation matrix ^{[16,21,22,23]}, and Bayesian methods ^[24]. In this study, we use the profile likelihood method ^[25,38] described in detail below.

Consider that the vector of parameters p is partitioned into $\textbf{p}$ = ($r$, $\textbf{s}$), where $r$ represents the parameter whose practical identifiability we are investigating and s represents the vector of remaining parameters. The profile likelihood of $r$ is given by,

where $\text{RSS}$ is the objective functional used for data fitting (in our case, Eq (4.2) for Models 1–3 and Eq (4.6) for Model 4). In other words, PL$(r)$ finds the minimum of the objective functional $\text{RSS}(r, s)$ for an array of fixed $r$ values over the space of the remaining parameters $\textbf{s}$. The shape of PL$(r)$ informs the identifiability of $r$, with a u-shaped PL$(r)$ that exceeds a threshold (corresponding to a chosen confidence level) indicating practical identifiability of $r$ and a flat PL$(r)$ indicating nonpractical identifiability of $r$.

We estimate PL($r$) over a mesh of equally spaced values of $r$, centered at the best-fit estimate $\hat{r}$, with the number of mesh points chosen to have enough data to generate a confidence interval for $\hat{r}$, as follows. If we consider a model with parameters $\textbf{p}$ unknown and a model with parameters $\textbf{s}$ unknown, we obtain two nested models that differ by parameter $r$. It has been shown that the likelihood ratio of the nested models converges to a $\chi^2$ distribution with one degree of freedom, $df = 1$ (see Corrollary 2 in ^[38] for more detail). This helps us define the $\Delta$-level likelihood-based confidence interval for parameter $r$ to be

where $\Delta$ is the percentile of the $\chi^2$ distribution with $df = 1$, and $J$ is the weighted least-squares functional at the best parameter estimate ^[38]. This can be summarized, as follows.

Definition 4. Let $\text{CI} = \{r|\text{PL}(r) < J+\Delta\}$ be the likelihood-based confidence interval for parameter $r$.

i. If $\text{CI}\subset[r_1, r_2]$, where $r_1$ and $r_2$ are finite, then parameter $r$ is practically identifiable.

ii. If either $r_1$ or $r_2$ is infinite, then parameter $r$ is not practically identifiable.

A model is practically identifiable if all parameters are practically identifiable.

For Models 1–3, we generated the profile likelihoods $\text{PL}(r)$ for each parameter $r \in \{\mathbf{p_w} \}$ for $\mathbf{w} = \{1, 2, 3\}$ by fitting the functional $\text{RSS}_w(\mathbf{p_w})$ given by Eq (4.2) and Model $\mathbf{w}$ to mean population viral load data. We obtained best estimates for the remaining parameters $\textbf{s}$ over a mesh of equally spaced, known $r$ values. Similarly, for Model 4, we generated the profile likelihood $\text{PL}(r)$ for unknown parameters $r \in \{\mathbf{p_4} \}$ by fitting functional $\text{RSS}_4(\mathbf{p_4})$ given by Eq (4.6) simultaneously to the mean population viral titer and the mean population effector cell data. We obtained best estimates for the remaining parameters $\textbf{s}$ over a mesh of equally spaced, known $r$ values.

Additionally, to further explore the relationship between data availability and practical identifiability, we generated profile likelihoods for parameters $\mathbf{p_w}$ and Model $\mathbf{w}$ using simulated, noise-free, high frequency datasets. In particular, we assumed that the virus titer data was collected every fourteen minutes (for a total of 899 evenly spaced points) for Models 1–3 and that both the virus titer data and the effector cell data were collected every fourteen minutes (for a total of 899 evenly spaced points of each data type) for Model 4. We fitted each model to this high frequency data and generated profile likelihoods of the resulting parameter values. In all of our models, we chose $\Delta$ to be the 90-th percentile of the $\chi^2$-distribution, $\chi^2_{90}$ ^[38]. This guaranteed us a $90\%$ confidence interval in the estimated parameter $r$.

6.1.   Practical identifiability results

Since we determined that Models 1–4 are structurally identifiable under known initial conditions and unlimited data, we were allowed to search for best estimates for all models' parameters. The resulting fitting routine may still be ill-posed, given that the data consisted of discrete (daily) datasets that varied among the infected mice, rather than the unlimited data required by the structural identifiability theory. Therefore, we performed practical identifiability for Models 1–4 under the discrete subject data in ^[28]. We used the Profile Likelihood practical identifiability method ^{[23,25,26,27]}, which has the advantage of not only determining whether a parameter is practically identifiable, but also of determining a $90\%$ confidence interval for the parameter space where practical identifiability occurs.

When using the empirical population mean virus titer data in ^[5] for Model 1, we found that $\beta$ is practically identifiable with $90\%$ confidence interval $CI\subset[1.64\times10^{-6}, 2.94\times10^{-5}]$ and $\pi$ is practically identifiable with $90\%$ confidence interval $CI\subset[0.31, 3.18]$, respectively. On the other hand, both $\delta$ and $c$ are not practically identifiable, with identical $90\%$ confidence intervals $CI\subset[0.87, \infty]$ (see Figure 3A). Adding high frequency data and rerunning the profile likelihood analysis resulted in practical identifiability of all four parameters, consistent with the structural identifiability results for Model 1 (see Figure 3B).

Similar to Model 1, when using the empirical population mean virus titer data in ^[5] for Model 2, we found that $\delta$, $K_\delta$, and $\pi$ are practically identifiable with $90\%$ confidence intervals $CI\subset[1.41\times10^6, 2.06\times10^6]$, $CI\subset[1.74\times10^4, 4.19\times10^5]$, and $CI\subset[0.44, 5.24]$, respectively. On the other hand, both $\beta$ and $c$ are not practically identifiable, with $90\%$ confidence intervals $CI\subset[6.62\times10^{-6}, \infty]$ and $CI\subset[3.98, \infty]$, respectively (see Figure 3C). Adding high frequency data and rerunning the profile likelihood analysis resulted in practical identifiability of all five parameters, consistent with the structural identifiability results for Model 2 (see Figure 3D).

For Model 3, when using the empirical population mean virus titer data in ^[5], we found that $\beta$, $\delta$, $K_\delta$, and $\pi$ are practically identifiable with $90\%$ confidence intervals $CI\subset[2.17\times10^{-5}, 1.60\times10^{-4}]$, $CI\subset[1.47\times10^6, 2.03\times10^6]$, $CI\subset[1.59\times10^5, 4.32\times10^5]$, and $CI\subset[1.07, 4.49]$, respectively. On the other hand, both $c$ and $k$ are not practically identifiable, with $90\%$ confidence intervals $CI\subset[3.98, \infty]$ and $CI\subset[\infty, \infty]$, respectively (see Figure 3E). Adding high frequency data and rerunning the profile likelihood analysis did not result in practical identifiability of all six parameters (see Figure 3F). However, if we additionally relaxed constraints on parameters $c$ and $k$ to range in the $[0, 1000]$ and $[0, 50]$ intervals, compared to the constraints chosen in Table 2, we observed practical identifiability of all five parameters, consistent with the structural identifiability results for Model 3 (see Figure 4).

For Model 4, when using the discrete empirical population mean virus titer data and the empirical population mean effector cell data in ^[5] simultaneously, we found that $\pi$ and $E_0$ are practically identifiable with $90\%$ confidence intervals $CI\subset[0.29, 2.49]$ and $CI\subset[66, 5.41\times10^{-4}]$. Parameters $k$, $\lambda$, and $K_E$ are not practically identifiable with the same $90\%$ confidence interval $CI\subset[-\infty, \infty]$. Parameters $\beta$, $\delta_E$, $K_\delta$, $\eta$, and $c$ are also not practically identifiable with $90\%$ confidence intervals $CI\subset[8.00\times10^{-6}, \infty]$, $CI\subset[1.10, \infty]$, $CI\subset[42.98, \infty]$, $CI\subset[5.37\times10^{-8}, \infty]$, and $CI\subset[3.89, \infty]$, respectively. Lastly, parameters $\delta$, $d_E$, and $\tau$ are not practically identifiable on the positive domain with an undefined lower bound (ULB) for the $90\%$ confidence interval and a finite upper bound. In particular, $CI\subset[ULB, 0.63]$, $CI\subset[ULB, 3.20]$, $CI\subset[ULB, 6.99]$, for $\delta$, $d_E$, and $\tau_I$, respectively (see Figure 5A). Adding high frequency data and rerunning the profile likelihood analysis resulted in practical identifiability of all thirteen parameters, consistent with the structural identifiability results for Model 4 (see Figure 5B).

7.   Discussion

In this study, we investigated the conditions needed to ensure model identifiability in four models of influenza A virus dynamics in infected mice. To apply the same methodology and software, all considered models were either modeled by systems of ODEs (Models 1–3 given by equations Eqs (2.1)–(2.3)) or approximated by a system of ODEs (Model 4 given by Eq (2.7)). The considered models differ in the number of equations (corresponding to the number of variables) from three in Models 1 and 2, to four in Model 3 to eight in Model 4. Consequently, the number of unknown parameters is a maximum of four in Model 1, a maximum of five in Model 2, a maximum of six in Model 3, and a maximum of thirteen in Model 4. Lastly, the terms of interaction include only mass-action and linear terms for Model 1 and mass-action, linear terms, and density dependence terms for Models 2–4.

We found that the increased complexity needed to capture biological realism comes with a cost. It resulted in increased uncertainty in parameter estimates not only when discrete and noisy virus and immune cell empirical data is used for validation but also when we assumed (hypothetically) that unlimited data is available. This means that data fitting should not be conducted until it is established that parameters can be revealed from unlimited data under the considered model structure. In other words, the first step in the model validation is determining whether all unknown parameters are structurally identifiable (see Figure 6).

When it comes to investigating the structural identifiability of systems of ODEs several software platforms are available. Here, we compared results from three of them: DAISY ^[32], COMBOS ^[33], and StructuralIdentifiability.jl in JULIA ^[34]. For Models 1–3 and unlimited virus titer data, we found the same classification for the structurally identifiable parameters and the same (or equivalent) correlations between the nonstructurally identifiable parameters, regardless of the software used (Table 1). For the more complex Model 4 and unlimited virus titer and effector CD8$^+$ T cell data, however, only StructuralIdentifiability.jl in JULIA found that the model is structurally identifiable under known (with the exception of initial effector population, $E_0$) initial conditions (Table 1). When initial conditions are unknown, we found identical classification for structurally identifiable parameters and equivalent correlations between the nonstructurally identifiable parameters among StructuralIdentifiability.jl in JULIA and DAISY (Table 1). COMBOS cannot handle the structural stability analyses for Model 4, regardless of whether initial conditions are known or not (Table 1). While increased difficulty in analyzing the structural identifiability of Model 4 is not surprising given its increased dimensionality (eight equations), multiple parameters (thirteen), and complex terms of interaction, this model is validated with two datasets (virus titer load and effector CD8$^+$ T cells). Our analysis showed that the addition of data for another model variable did not compensate for the size of the model and number of unknown parameters.

Interestingly, we found that all parameters (for all models) are structurally identifiable under known (with the exception of initial effector population, $E_0$) initial conditions. Given that this is an inoculation study, the assumption of known viral inoculum (set by the experiment) and of known initial target cell population (approximated based on the animal body weight) is not unreasonable. When Models 1–4 are used to model natural infection, however, such initial conditions would be unknown due to differences in individual routes of infection, heterogeneity in individual immune responses, and variability in patient susceptibility. Under such unknowns, Models 1–4 would become structurally unidentifiable. Hence, it would be impossible to estimate all parameters even in the presence of unlimited data. A reduction of the unknown parameter space (based on the reported parameter correlations) would be needed before model validation can be attempted.

We next validated Models 1–3 with discrete (daily) virus titer data (for the first nine days) and validated Model 4 with discrete (daily) virus titer data (for the first nine days) and discrete (daily) CD8$^+$ T cell data (for the first twelve days). Model selection (based on $AIC_c$) favored Model 4 as the best model (Table 3). Interestingly, Model 1 was the second best model, even though it had the largest $90\%$ error region around the predicted mean virus fit (Figure 2, gray shaded regions). All models perform the worst during the contraction interval (Figure 2, gray and blue shaded regions), suggesting uncertainty in death rates estimates (for the virus and infected cells).

We used the best parameter estimates obtained through data fitting for Models 1–4 to further investigate their practical identifiablity. Knowing that data used for validation was collected daily and that there was variability among the subjects at each time point, we wanted to determine whether there is uncertainty in estimated parameters. When it comes to practical identifiability, several methods are available, such as the Monte Carlo simulations ^[18,19,20], the Fisher information matrix or correlation matrix ^{[16,21,22,23]}, and Bayesian methods ^[24]. In this study, we used the profile likelihood method ^[25,38] for two main reasons. First, it allowed us to not just classify the models as practically or non-practically identifiable, but to determine a $90\%$ confidence interval for each practically identifiable parameter. Second, it allowed us to determine the required assumptions needed to improve practical identifiability, while maintaining biological realism (for example, by imposing positivity for all parameters).

We found that none of the models are practically identifiable for the daily empirical data collected in ^[28] and the parameter range restrictions imposed in Table 2 (see Figures 3A, C, E, and 5A). While Model 1, Model 2, and Model 4 become practically identifiable if we assume data is collected every fourteen minutes (see Figures 3A, D, and 5B), Model 3 does not (see Figure 3F). For this model, we can achieve practical identifiability only when we assume that the viral clearance rate can reach values as high as $c = 500$ per day (corresponding to a life-span for the influenza virus of $2.9$ minutes), and that the epithelial cells spend $1/k = 1.2$ hours in the eclipse phase before they become productively infected (see Figure 4). While large influenza clearance rates have been reported before ^[28], the eclipse phase $1/k$ is assumed to be conserved in a tight interval of 4–6 hours in most respiratory infections ^[6,11,28]. Therefore, this parameter is not practically identifiable from Model 3 even when data is collected at high frequency. This is a situation where a parameter should be removed from the data fitting routine in order to improve the uncertainty in the estimates of the remaining parameters (see Figure 6).

Our study has several limitations. First, Model 4 was originally expressed as a five order system of DDEs. Given the lack of methods that can be used to determine the structural identifiability of DDEs, we approximated it with an eight order system of ODEs. More work is needed to determine whether we maintain (or improve) the practical identifiability results when the DDE system is used in the place of the ODE system. Second, we assumed that all model parameters are unknown. It is feasible that the practical identifiability will be improved if certain parameters (such as the eclipse phase) were assumed known. Lastly, all our practical identifiability results come in the context of daily data collection. It would be interesting to see how data collected with random frequency (especially unavailability of data measurements before peak viral load) changes the results. In conclusion, we investigated the structural and practical identifiability of four nested ODE models of influenza A virus infection in mice. We determined the tradeoff between model complexity (defined as combined system dimension, number of unknown parameters, nonlinearity in model interactions), data availability, and our ability to reliably estimate model parameters. We presented solutions for improving model identifiability. While our results dependent on the structure of the models considered the available data, the methods are generalizable and their use is needed to improve reliability and reproducibility of parameter estimates in other systems of ODEs applied to discrete biological data.

Identifiability analysis has critical implications for experimental design, particularly when it comes to ensuring that the data obtained in these experiments will provide accurate estimation of parameters. For instance, if a model is not structurally identifiable even with noise-free unlimited data, then no experimental design will allow for estimation of certain parameters. Furthermore, structural identifiability analysis informs us which variables need to be measured in order to obtain reliable parameter estimates. Therefore, these experiments can be designed with some structurally identifiable models in mind. On the other hand, practical identifiability reveals the optimal data sampling frequency, where the data is more informative for certain parameters. These results will refine the experimental design to obtain data at those times and reduce the uncertainty in parameter estimates.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Author contribution statement

Conceptualization: NT and SMC; analysis: YRL and NHB; funding acquisition SMC and NT; writing - original draft preparation: YRL, NHB, NT and SMC; writing - review and editing: YRL, NHB, NT and SMC.

Acknowledgments

SMC and NHB acknowledge partial support from National Science Foundation grant No. 2051820 and NIH NIGMS 1R01GM152743-01. This research was enabled in part through the Virginia Tech Center for the Mathematics of Biosystems (VTCMB-033500). NT acknowledges partial support from National Science Foundation grant DMS 1951626. NT and YRL acknowledge support from NIH NIGMS 1R01GM152743-01.

Conflict of interest

Necibe Tuncer is one of the special issue editors for Mathematical Biosciences and Engineering (MBE) and was not involved in the editorial review or the decision to publish this article. All authors declare no conflict of interest.