A matrix analysis of BLMBPs under a general linear model and its transformation

1.   Introduction

In this paper, we consider the standard linear model

where it is assumed that ${\bf y} \in {\mathbb R}^{n \times 1}$ is vector of observable random variables, ${\bf X} \in {\mathbb R}^{n \times p}$ are known matrices of arbitrary ranks ($0 \leq r({\bf X}) \leq \min\{n, p\})$, $\boldsymbol{\theta} \in {\mathbb R}^{p \times 1}$ is a vector of fixed but unknown parameters, and $\boldsymbol{\nu} \in {\mathbb R}^{n \times 1}$ is a random error vector. In order to carry out reasonable estimation and statistical inference in the context of (1.1), we assume that the expectation vector and the covariance matrix of $\boldsymbol{\nu}$ are given without loss of generality by

In the sequel, we assume that ${{\bf{\Sigma}}}\in {\mathbb R}^{n \times n}$ is a known positive semi-definite matrix of arbitrary rank in order to derive general and precise conclusions under the given model assumptions. Once the work on the general assumptions is established, we can further, as usual in parametric regression analysis, let ${{\bf{\Sigma}}}$ be certain specified forms with known or unknown entries, and then derive various concrete inference results.

The assumptions in the contexts of (1.1) and (1.2) are typical in form for a complete specification of the general linear model (for short, GLM). Observe that there are two unknown vectors $\boldsymbol{\theta}$ and $\boldsymbol{\nu}$ in (1.1). Then, we can figure out that an obligatory task in statistical inference under (1.1) and (1.2) is to predict the two unknown vectors simultaneously:

where it is assumed that ${\bf A} \in {\mathbb R}^{s \times p}$ and ${\bf B} \in {\mathbb R}^{s \times n}$ are known matrices of arbitrary ranks. This vector obviously includes all the unknown vectors in (1.1), such as $\boldsymbol{\theta}$ and $\boldsymbol{\nu}$, as its special cases. It is easy to see that under (1.1) and (1.2), we have

In the investigation of linear statistical models for regression, it is a common inference problem to propose and characterize various reasonable connections between two different given models under the given model assumptions. One concrete problem of such kind is to investigate the relationships between a given linear model (called the original model) and certain types of its transformed models. Sometimes, the transformed models are required to meet with certain necessary requirements in the statistical inferences of the original linear model. Now let us consider ${\mathscr M}$ in (1.1) and its transformed models. In such a case, we may be faced with different transformed forms of the model in accordance with linear transformations of observable random vector ${\bf y}$. Generally speaking, various possible transformed models of ${\mathscr M}$ in (1.1) are often obtained by pre-multiplying ${\bf T}$ by a given matrix. For example,

is a common transformed form of ${\mathscr M}$ in (1.1), where ${\bf T} \in {\mathbb R}^{m \times n}$ is a known transformation matrix of arbitrary rank. Below, we present a group of well-known cases of the transformed model for different choices of the transformation matrix ${\bf T}$ in (1.5).

${\rm (a)}$ We first divide the original model ${\mathscr M}$ in (1.1) as

by the partitions of the vectors and matrices in the model. Then we take the transformation matrices ${\bf T}_{1} = [{\bf I}_{n_{1}}, \, {\bf 0}]$ and ${\bf T}_{2} = [{\bf 0}, \, {\bf I}_{n_{2}}]$ in (1.5) to obtain the following two sub-sample models:

where ${\bf y}_{i} \in {\mathbb R}^{n_{i} \times 1}, {\bf X}_{i} \in {\mathbb R}^{n_{i} \times p}, \boldsymbol{\theta} \in {\mathbb R}^{p \times 1},$ $\boldsymbol{\nu}_{i} \in {\mathbb R}^{n_{i} \times 1}$ and $n = n_{1} + n_{2}.$ They can also be viewed as adding or deleting certain regression equations in a given GLM. Also, we can say that these two individual models occur in two periods of observations of data.

${\rm (b)}$ Assume that a concrete form of ${\mathscr M}$ in (1.1) is given by

In this case, taking the two transformation matrices ${\bf T}_{1} = [{\bf I}_{n_{1}}, \, {\bf 0}]$ and ${\bf T}_{2} = [{\bf 0}, \, {\bf I}_{n_{2}}]$, we obtain the following two sub-sample models:

where ${\bf y}_{i} \in {\mathbb R}^{n_{i} \times 1}, {\bf X}_{i} \in {\mathbb R}^{n_{i} \times p_{i}}, \boldsymbol{\theta}_{i} \in {\mathbb R}^{p_{i} \times 1},$ $\boldsymbol{\nu}_{i} \in {\mathbb R}^{n_{i} \times 1}$, $n = n_{1} + n_{2}$, $p = p_{1} + p_{2}$. The two models are known as seemingly unrelated linear models, which are linked to each other by the correlated error terms across the models, where all the given matrices and the unknown vectors in the two models are different.

Due to the linear nature of ${\mathscr M}$ in (1.1), we see the expectations and covariance matrices of ${\bf y}, {\bf T} {\bf y}$ and $\boldsymbol{\tau}$ under the assumptions in (1.1) and (1.2):

Now we mention some backgrounds of this current study. For unknown parameters in a given regression model, statisticians are able to adopt different optimal criteria in order to obtain proper predictions and estimations of the unknown parameters. In comparison, the best linear unbiased prediction, the best linear unbiased estimation and the least squares estimation are best known among others because they have many excellent mathematical and statistical properties and performances. There were many deep and fruitful works in the statistical literatures related to these predictions and estimations. However, it is a common fact in statistical practice that the unknown parameters in a given model may not be predictable or estimable. Instead, it is necessary to choose certain biased predictions and biased estimations for the unknown parameters. For example, Rao described the bias between estimators and unknown parameter functions, constructed the minimum biased estimation class, selected the one with the minimum variance in the minimum biased estimation class and then defined the best linear minimum biased estimation. Especially when the unknown parameter function is an estimable function, the best linear minimum biased estimation is the classic best linear unbiased estimation. It can be seen from (1.1)–(1.8) that a given model and its transformed models are not necessarily equivalent in form. Hence, the predictors/estimators of unknown vectors that are going to derive under these models have different algebraic expressions and properties. Yet, some transformations of observable random vectors may preserve enough information for predicting/estimating unknown vectors in the original model. Therefore, it is natural to consider certain links between the predictors/estimators obtained from an original model and its transformed models in statistical inferences of these models. Traditionally, the problems of characterizing relationships between predictions/estimations of unknown vectors in an original model and its transformed models were known as linear sufficiency problems, which were first considered in ^[1,3]. Many scholars also studied the relationship between estimations under a given original model and its transformed model from different aspects. For instance, Baksalary and Kala considered the problem on linear transformations of GLMs preserving the best linear unbiased estimations under the general Gauss-Markoff model in ^[1]; Xie studied in ^[30] the best linear minimum biased estimations under a given GLM and discussed the problem of the linear transformation preserving the best linear minimum biased estimations. Also, the subject of this kind was sufficiently approached in ^{[7,9,14,18,20,33]} among others.

Given the model assumptions in (1.1)–(1.8), the purpose of this paper is to provide a unified theoretical and conceptual exploration for solving the best linear minimum biased prediction (for short, BLMBP) problems under a GLM and its transformed general linear models (for short, TGLMs) through the skillful and effective use of a series of exact and analytical matrix analysis tools. The remaining part of this current paper is organized as follows. In the second section, we introduce notation and serval matrix analysis tools and techniques that we shall utilize to characterize matrix equalities and matrix set inclusions that involve generalized inverses of matrices. In the third section, we introduce the definitions of the linear minimum biased predictor (for short, LMBP) and the BLMBP of $\boldsymbol{\tau}$ in (1.3), as well as basic estimation and inference theory regarding the LMBP and BLMBP, including their analytical expressions and their mathematical and statistical properties and features in the contexts of (1.1)–(1.8). In the fourth section, we address the problems regarding the relationships between the BLMBPs under a GLM and its TGLMs using the powerful matrix rank and inertia methodology. The fifth section presents a special example related to the main findings in the preceding sections. Some conclusions and remarks are given in the last section.

2.   Some preliminaries

We begin with the introduction of notation used in the sequel. ${\mathbb R}^{m \times n}$ denotes the collection of all $m \times n$ matrices over the field of real numbers, and the symbols ${\bf M}^{\prime}$, $r({\bf M})$ and ${\mathscr R}({\bf M})$ denote the transpose, the rank and the range (column space) of ${\bf M} \in \mathbb R^{m \times n}$, and ${\bf I}_m$ denotes the identity matrix of order $m$. The Moore–Penrose generalized inverse of ${\bf M}$, denoted by ${\bf M}^{\dagger}$, is defined to be the unique solution ${\bf G}$ satisfying the four matrix equations ${\bf M} {\bf G} {\bf M} = {\bf M}$, ${\bf G} {\bf M} {\bf G} = {\bf G}$, $({\bf M} {\bf G})^{\prime} = {\bf M} {\bf G}$ and $({\bf G} {\bf M})^{\prime} = {\bf G} {\bf M}$. Let ${\bf P}_ {\bf M} = {\bf M} {\bf M}^{\dagger}$, ${\bf M}^{\!\perp} = {\bf E}_ {\bf M} = {\bf I}_m - {\bf M} {\bf M}^{\dagger}$ and ${\bf F}_ {\bf M} = {\bf I}_n - {\bf M}^{\dagger} {\bf M}$ denote the three orthogonal projectors (symmetric idempotent matrices) induced from ${\bf M}$, which will help in briefly denoting calculation processes related to generalized inverses of matrices, where both ${\bf E}_{ {\bf M}}$ and ${\bf F}_{ {\bf M}}$ satisfy ${\bf E}_{ {\bf M}} = {\bf F}_{ {\bf M}^{\prime}}$ and ${\bf F}_{ {\bf M}} = {\bf E}_{ {\bf M}^{\prime}}$ and the ranks of ${\bf E}_{ {\bf M}}$ and ${\bf F}_{ {\bf M}}$ are $r({\bf E}_{ {\bf M}}) = m - r({\bf M})$ and $r({\bf F}_{ {\bf M}}) = n - r({\bf M})$. Two symmetric matrices ${\bf M}$ and ${\bf N}$ of the same size are said to satisfy the inequalities ${\bf M} \succcurlyeq {\bf N}$, ${\bf M} \preccurlyeq {\bf N},$ ${\bf M} \succ {\bf N}$ and ${\bf M} \prec {\bf N}$ in the Löwner partial ordering if ${\bf M} - {\bf N}$ is positive semi-definite, negative semi-definite, positive definite and negative definite, respectively. Further information about the orthogonal projectors ${\bf P}_ {\bf M}$, ${\bf E}_ {\bf M}$ and ${\bf F}_ {\bf M}$ and their various applications in the theory of linear statistical models can be found, e.g., in ^{[10,13,16,19]}. It is also well known that the Löwner partial ordering between two symmetric matrices is a surprisingly strong and useful property in matrix analysis. The reader is referred to ^[16] and the references therein for more results and facts regarding the issues of the Löwner partial ordering in statistical theory and applications. Recently, the authors of ^{[2,4,5,6,23,25,26,29]} proposed and approached a series of research problems concerning the relationships of different kinds of predictions of unknown parameters in regression models using the rank and inertia methodology in matrix analysis, and provided a variety of simple and reasonable equivalent facts related to the relationship problems. In this paper, we also adopt the rank and inertia methodology to approach the relationship problems regarding different estimations and predictions.

As preliminaries that can help readers in getting familiar with the features and usefulness of the matrix rank methodology, we present in the following a list of commonly used results and facts about ranks of matrices and matrix equations, which are well known or easy to prove. We shall use them in the descriptions and simplifications of various complicated matrix expressions and matrix equalities that occur in the statistical inference of a GLM and its TGLMs in the following sections.

Lemma 2.1 (^[28]). Let ${\mathscr A}$ and ${\mathscr B}$ be two sets composed by matrices of the same size.

Lemma 2.2 (^[12]). Let ${\bf A} \in {\mathbb R}^{m \times n},$ ${\bf B} \in {\mathbb R}^{m \times k},$ and ${\bf C} \in {\mathbb R}^{l \times n}.$ Then$,$

In particular$,$ the following results hold$:$

${\rm (a)}$ $r[{\bf A}, \, {\bf B}] = r({\bf A}) \Leftrightarrow {\mathscr R}({\bf B}) \subseteq {\mathscr R}({\bf A}) \Leftrightarrow {\bf A} {\bf A}^{\dagger} {\bf B} = {\bf B} \Leftrightarrow {\bf E}_{ {\bf A}} {\bf B} = {\bf 0}.$

${\rm (b)}$ $r\! \begin{bmatrix} {\bf A} \\ {\bf C} \end{bmatrix} = r({\bf A}) \Leftrightarrow {\mathscr R}({\bf C}^{\prime}) \subseteq {\mathscr R}({\bf A}^{\prime}) \Leftrightarrow {\bf C} {\bf A}^{\dagger} {\bf A} = {\bf C} \Leftrightarrow {\bf C} {\bf F}_{ {\bf A}} = {\bf 0}.$

Lemma 2.3 (^[22]). Assume that five matrices ${\bf A}_1,$ ${\bf B}_1,$ ${\bf A}_2,$ ${\bf B}_2$ and ${\bf A}_3$ of appropriate sizes satisfy the conditions ${\mathscr R}({\bf A}_{1}^{\prime}) \subseteq {\mathscr R}({\bf B}_{1}^{\prime}),$ ${\mathscr R}({\bf A}_{2}) \subseteq {\mathscr R}({\bf B}_{1}),$ ${\mathscr R}({\bf A}_{2}^{\prime}) \subseteq {\mathscr R}({\bf B}_{2}^{\prime})$ and ${\mathscr R}({\bf A}_3) \subseteq {\mathscr R}({\bf B}_{2}).$ Then$,$

Lemma 2.4 (^[21,27]). Let ${\bf A} \in {\mathbb R}^{m \times n}, {\bf B} \in {\mathbb R}^{m \times k}$ and ${\bf C} \in {\mathbb R}^{l \times n}$ be given. Then, the maximum and minimum ranks of ${\bf A} - {\bf B} {\bf Z}$ and ${\bf A} - {\bf B} {\bf Z} {\bf C}$ with respect to a variable matrix ${\bf Z}$ of appropriate sizes are given by the following closed-form formulas$:$

Below we offer some existing formulas and results regarding general solutions of a basic linear matrix equation and a constrained quadratic matrix optimization problem.

Lemma 2.5 (^[15]). Let ${\bf A} \in {\mathbb R}^{m \times n}$ and ${\bf B} \in {\mathbb R}^{p \times n}.$ Then$,$ the linear matrix equation ${\bf Z} {\bf A} = {\bf B}$ is solvable for ${\bf Z} \in {\mathbb R}^{p \times m}$ if and only if ${\mathscr R}({\bf A}^{\prime}) \supseteq {\mathscr R}({\bf B}^{\prime}),$ or equivalently$,$ ${\bf B} {\bf A}^{\dagger} {\bf A} = {\bf B}.$ In this case$,$ the general solution of the equation can be written in the parametric form

where ${\bf U} \in {\mathbb R}^{p \times m}$ is an arbitrary matrix$.$

Lemma 2.6 (^[28]). Let ${\bf A} \in {\mathbb R}^{m \times n},$ ${\bf B} \in {\mathbb R}^{m \times k}$ and assume that ${\mathscr R}({\bf A}) = {\mathscr R}({\bf B}).$ Then

Lemma 2.7 (^[24]). Let

where it is assumed that ${\bf A} \in {\mathbb R}^{p \times q},$ ${\bf B} \in {\mathbb R}^{n \times q},$ ${\bf C} \in {\mathbb R}^{p\times m}$ and ${\bf D} \in {\mathbb R}^{n\times m}$ are given$,$ ${\bf M} \in {\mathbb R}^{m \times m}$ is positive semi-definite and the matrix equation ${\bf Z} {\bf A} = {\bf B}$ is solvable for ${\bf Z}\in {\mathbb R}^{n \times p}.$ Then$,$ there always exists a solution ${\bf Z}_0$ of ${\bf Z} {\bf A} = {\bf B}$ such that

holds for all solutions of ${\bf Z} {\bf A} = {\bf B},$ and the matrix ${\bf Z}_0$ that satisfies the above inequality is determined by the following consistent matrix equation:

In this case$,$ the general expression of ${\bf Z}_0$ and the corresponding $f({\bf Z}_0)$ and $f({\bf Z})$ are given by

where ${\bf G} = {\bf B} {\bf A}^{\dagger} {\bf C} + {\bf D},$ ${\bf T} = ({\bf A}^{\!\perp} {\bf C} {\bf M} {\bf C}^{\prime} {\bf A}^{\!\perp})^{\dagger}$ and ${\bf U} \in {\mathbb R}^{n\times p}$ is arbitrary$.$

In order to describe the relationships between BLMBPs under different regression models, we need to adopt the following definition to characterize possible equality between two random vectors ^[28].

Definition 2.8. Let ${\bf y}$ be as given in (1.1)$,$ let $\{ {\bf L}_{1}\}$ and $\{ {\bf L}_{2}\}$ be two matrix sets and let ${\bf L}_{1} {\bf y}$ and ${\bf L}_{2} {\bf y}$ be any two linear predictors of $\boldsymbol{\tau}$ in (1.3).

${\rm (a)}$ $\{ {\bf L}_{1} {\bf y}\} \cap \{ {\bf L}_{2} {\bf y}\} \neq \emptyset$ holds definitely, i.e, $\{ {\bf L}_{1}\} \cap \{ {\bf L}_{2}\} \neq \emptyset$ if and only if

${\rm (b)}$ The vector set inclusion $\{ {\bf L}_{1} {\bf y}\} \subseteq \{ {\bf L}_{2} {\bf y}\}$ holds definitely, i.e, $\{ {\bf L}_{1}\} \subseteq \{ {\bf L}_{2}\}$ if and only if

${\rm (c)}$ $\{ {\bf L}_{1} {\bf y}\} \cap \{ {\bf L}_{2} {\bf y}\} \neq \emptyset$ holds with probability $1$ if and only if

${\rm (d)}$ The vecctor set inclusion $\{ {\bf L}_{1} {\bf y}\} \subseteq \{ {\bf L}_{2} {\bf y}\}$ holds with probability $1$ if and only if

3.   Fundamentals of the LMBP/LMBE and BLMBP/BLMBE

Recall in parametric regression analysis that if there exists a matrix ${\bf L}$ such that ${\rm E}({\bf L} {\bf y} - \boldsymbol{\tau}) = ({\bf L} {\bf X} - {\bf A}) \boldsymbol{\theta} = {\bf 0}$ holds for all $\boldsymbol{\theta}$, the parametric parameter vector $\boldsymbol{\tau}$ in (1.3) is said to predictable under the assumptions in (1.1) and (1.2). Otherwise, there does not exist an unbiased prediction of $\boldsymbol{\tau}$ under (1.1) and (1.2), and therefore, we have to seek certain biased predictions of $\boldsymbol{\tau}$ according to various specified optimization criteria. In this section, we shall adopt the following known definitions of the LMBP and BLMBP of $\boldsymbol{\tau}$ (cf. [17, p.337]).

Definition 3.1. Let the parametric vector $\boldsymbol{\tau}$ be as given in (1.3)$.$

${\rm (a)}$ The LMBP of $\boldsymbol{\tau}$ in (1.3) under (1.1) is defined to be

where the matrix $\widehat{ {\bf L}}$ satisfies

${\rm (b)}$ The LMBP of $\boldsymbol{\tau}$ in (1.3) under (1.5) is defined to be

where the matrix $\widehat{ {\bf K}}$ satisfies

Theorem 3.2. Under the notations in Definition 3.1$,$ the following results hold$:$

Proof. Note first that

where ${\bf T} {\bf X} {\bf F}_{ {\bf T} {\bf X}} = {\bf 0}$. Note that ${{\rm{tr}}}(({\bf K} {\bf T} {\bf X} - {\bf A}({\bf T} {\bf X})^{\dagger} {\bf T} {\bf X})({\bf K} {\bf T} {\bf X} - {\bf A}({\bf T} {\bf X})^{\dagger} {\bf T} {\bf X})^{\prime}) \geq 0$ for all ${\bf K} \in {\mathbb R}^{s \times m}$ and the matrix equation ${\bf K} {\bf T} {\bf X} = {\bf A}({\bf T} {\bf X})^{\dagger} {\bf T} {\bf X}$ is solvable for ${\bf K} \in {\mathbb R}^{s \times m}$. In this case, we obtain

and

thus establishing (3.6). Letting ${\bf T} = {\bf I}_{n}$ leads to (3.5). □

Definition 3.3. Let the parametric vector $\boldsymbol{\tau}$ be as given in (1.3).

${\rm (a)}$ If $\widehat{ {\bf L}}$ satisfies

holds in the Löwner partial ordering, then the linear statistic $\widehat{ {\bf L}} {\bf y}$ is defined to be the BLMBP of $\boldsymbol{\tau}$ in (1.3) under (1.1), and is denoted by

${\rm (b)}$ If $\widehat{ {\bf K}}$ satisfies

holds in the Löwner partial ordering, then the linear statistic $\widehat{ {\bf K}} {\bf T} {\bf y}$ is defined to be the BLMBP of $\boldsymbol{\tau}$ in (1.3) under (1.5), and is denoted by

If ${\bf B} = {\bf 0}$ or ${\bf A} = {\bf 0}$ in (1.3), then the $\widehat{ {\bf K}} {\bf T} {\bf y}$ in (3.11) are defined to be the best linear minimum biased estimator (BLMBE) and the BLMBP of ${\bf A} \boldsymbol{\theta}$ and ${\bf B} \boldsymbol{\nu}$ in (1.3) under (1.5), respectively, and are denoted by

It is easy to verify that the difference of ${\bf K} {\bf T} {\bf y} - \boldsymbol{\tau}$ under (1.5) can be written in the following form:

Hence, the covariance matrix of ${\bf K} {\bf T} {\bf y} - \boldsymbol{\tau}$ can be written as

Our main results on the BLMBPs of $\boldsymbol{\tau}$ in (1.3) are given below.

Theorem 3.4. Let the parametric vector $\boldsymbol{\tau}$ be as given in (1.3) and define ${\bf W} = [{\bf T} {\bf X}({\bf T} {\bf X})^{\prime}, \, {\rm Cov}({\bf T} {\bf y})({\bf T} {\bf X})^{\perp}]$ and ${\bf D} = {\rm Cov}(\boldsymbol{\tau}, \, {\bf T} {\bf y}).$ Then

The matrix equation in (3.13) is solvable for $\widehat{ {\bf K}},$ i.e.$,$

holds under (3.6)$,$ while the general expression of $\widehat{ {\bf K}}$ and the corresponding ${\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})$ can be written in the following form

where ${\bf U}_1 \in {\mathbb R}^{s \times m}$ is arbitrary$.$ Furthermore$,$ the following results hold$.$

${\rm (a)}$ $r[{\bf T} {\bf X}, \, {\bf T} {{\bf{\Sigma}}} {\bf T}^{\prime}({\bf T} {\bf X})^{\perp}] = r[{\bf T} {\bf X}, \, ({\bf T} {\bf X})^{\perp} {\bf T} {{\bf{\Sigma}}} {\bf T}^{\prime}] = r[{\bf T} {\bf X}, \, {\bf T} {{\bf{\Sigma}}}]$ and

${\mathscr R}[{\bf T} {\bf X}, \, {\bf T} {{\bf{\Sigma}}} {\bf T}^{\prime}({\bf T} {\bf X})^{\perp}] = {\mathscr R}[{\bf T} {\bf X}, \, ({\bf T} {\bf X})^{\perp} {\bf T} {{\bf{\Sigma}}} {\bf T}^{\prime}] = {\mathscr R}[{\bf T} {\bf X}, \, {\bf T} {{\bf{\Sigma}}}].$

${\rm (b)}$ $\widehat{ {\bf K}} {\bf T}$ in (3.15) is unique if and only if ${\mathscr R}({\bf T}) \subseteq {\mathscr R}[{\bf T} {\bf X}, \, {\bf T} {{\bf{\Sigma}}}].$

${\rm (c)}$ ${\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})$ is unique if and only if ${\bf T} {\bf y} \in {\mathscr R}[{\bf T} {\bf X}, \, {\bf T} {{\bf{\Sigma}}}]$ holds with probability $1.$

${\rm (d)}$ The covariance matrix of ${\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})$ is given by

the covariance matrix between ${\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})$ and $\boldsymbol{\tau}$ is given by

the difference of ${\rm Cov}(\boldsymbol{\tau})$ and ${\rm Cov}({\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau}))$ is given by

the covariance matrix of $\boldsymbol{\tau} - {\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})$ is given by

${\rm (e)}$ If ${\bf B} = {\bf 0}$ or ${\bf A} = {\bf 0}$ in (1.3), then

Proof. Eq (3.13) is obviously equivalent to

Since ${{\bf{\Sigma}}} \succcurlyeq {\bf 0}$, the optimization problem in (3.22) is a special case of (2.10). By Lemma 2.7, the solution of (3.22) is determined by the matrix equation in (3.13). This equation is consistent as well under (3.6), and the general solution of the equation and the corresponding BLMBP are given in (3.15). Result (a) is well known; see ^[11,16]. Results (b) and (c) follow from the conditions $[{\bf T} {\bf X}, \, {\bf T} {{\bf{\Sigma}}} {\bf T}^{\prime}({\bf T} {\bf X})^{\perp}]^{\perp} {\bf T} = {\bf 0}$ and $[{\bf T} {\bf X}, \, {\bf T} {{\bf{\Sigma}}} {\bf T}^{\prime}]^{\perp} {\bf T} {\bf y} = {\bf 0}$ holds with probability $1$.

Taking the covariance operation of (3.15) yields (3.16). Also from (1.8) and (3.15), the covariance matrix between ${\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})$ and $\boldsymbol{\tau}$ is

thus establishing (3.17). Combination of (1.4) and (3.16) yields (3.18). Substitution of (3.15) into (3.12) and then simplification yields (3.19). □

Some conclusions for a special case of Theorem 3.4 are presented below without proof.

Corollary 3.5. Let the parametric vector $\boldsymbol{\tau}$ be as given in (1.3)$,$ and define ${\bf V} = [{\bf X} {\bf X}^{\prime}, \, {\rm Cov}({\bf y}) {\bf X}^{\perp}]$ and ${\bf C} = {\rm Cov}(\boldsymbol{\tau}, \, {\bf y}).$ Then

The matrix equation in (3.23) is solvable for $\widehat{ {\bf L}},$ i.e.$,$

holds under (3.5)$,$ while the general expression of $\widehat{ {\bf L}}$ and the corresponding ${\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})$ can be written in the following form

where ${\bf U}_2 \in {\mathbb R}^{s \times n}$ is arbitrary$.$ Furthermore$,$ the following results hold$.$

${\rm (a)}$ $r[{\bf X}, \, {{\bf{\Sigma}}} {\bf X}^{\perp}] = r[{\bf X}, \, {\bf X}^{\perp} {{\bf{\Sigma}}}] = r[{\bf X}, \, {{\bf{\Sigma}}}]$ and ${\mathscr R}[{\bf X}, \, {{\bf{\Sigma}}} {\bf X}^{\perp}] = {\mathscr R}[{\bf X}, \, {\bf X}^{\perp} {{\bf{\Sigma}}}] = {\mathscr R}[{\bf X}, \, {{\bf{\Sigma}}}].$

${\rm (b)}$ $\widehat{ {\bf L}}$ in (3.25) is unique if and only if $r[{\bf X}, \, {{\bf{\Sigma}}}] = n.$

${\rm (c)}$ ${\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})$ is unique if and only if ${\bf y} \in {\mathscr R}[{\bf X}, \, {{\bf{\Sigma}}}]$ holds with probability $1.$

${\rm (d)}$ The covariance matrix of ${\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})$ is given by

the covariance matrix between ${\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})$ and $\boldsymbol{\tau}$ is given by

the difference of ${\rm Cov}(\boldsymbol{\tau})$ and ${\rm Cov}({\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau}))$ is given by

the covariance matrix of $\boldsymbol{\tau} - {\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})$ is given by

Corollary 3.6. Let the parametric vector $\boldsymbol{\tau}$ be as given in (1.3). Then$,$ the following results hold$:$

${\rm (a)}$ The BLMBP of $\boldsymbol{\tau}$ can be decomposed as the sum

and they satisfy

${\rm (b)}$ For any matrix ${\bf P} \in {\mathbb R}^{t \times s},$ and the following equality holds$:$

${\rm (c)}$ The BLMBP of $\boldsymbol{\tau}$ can be decomposed as the sum

and they satisfy

${\rm (d)}$ For any matrix ${\bf P} \in {\mathbb R}^{t \times s},$ and the following equality holds$:$

Proof. Notice that the arbitrary matrix ${\bf U}_1$ in (3.15) can be rewritten as ${\bf U}_1 = {\bf V}_{1} + {\bf V}_{2}$, while the matrix $[{\bf A}({\bf T} {\bf X})^{\prime}, \, {\bf D}({\bf T} {\bf X})^{\!\perp}]$ in (3.15) can be rewritten as

Correspondingly, ${\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})$ in (3.15) can be rewritten as the sum:

thus establishing (3.30). From (3.20) and (3.21), the covariance matrix between ${\rm BLMBE}_{ {\mathscr N}}({\bf A} \boldsymbol{\theta})$ and ${\rm BLMBP}_{ {\mathscr N}}({\bf B} \boldsymbol{\nu})$ is given by

Applying (2.6) to the right-hand side of the above equality and then simplifying by Theorem 3.4(a)$,$ (2.3), and (2.5), we obtain

which implies that ${\rm Cov}({\rm BLMBE}_{ {\mathscr N}}({\bf A} \boldsymbol{\theta}), \, {\rm BLMBP}_{ {\mathscr N}}({\bf B} \boldsymbol{\nu}))$ is a zero matrix, thus establishing (3.31). Equation (3.32) follows from (3.30) and (3.31). Result (b) follows directly from (3.15). Results (c) and (d) are special cases of (a) and (b). □

4.   Relationships between the BLMBPs under a GLM and its TGLMs

One of the main tasks in the statistical inference of parametric regression models is to characterize connections between different predicttions/estimattions of unknown parameters. In this section, we study the relationships between the BLMBPs under GLM and its TGLMs. Because the coefficient matrices $\widehat{ {\bf K}} {\bf T}$ and $\widehat{ {\bf L}}$ in (3.15) and (3.25) are not necessarily unique, we use

to denote the collections of all the coefficient matrices and the corresponding BLMBPs. In order to characterize the relations between the collections of the coefficient matrices in (4.1), it is necessary to discuss the following four cases$:$

${\rm (a)}$ $\{\widehat{ {\bf L}}\} \cap \{\widehat{ {\bf K}} {\bf T}\} \neq \emptyset,$ so that $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \cap \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\} \neq \emptyset$ holds definitely$;$

${\rm (b)}$ $\{\widehat{ {\bf L}}\} \supseteq \{\widehat{ {\bf K}} {\bf T}\},$ so that $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \supseteq \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds definitely$;$

${\rm (c)}$ $\{\widehat{ {\bf L}}\} \subseteq \{\widehat{ {\bf K}} {\bf T}\},$ so that $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \subseteq \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds definitely$;$

${\rm (d)}$ $\{\widehat{ {\bf L}}\} = \{\widehat{ {\bf K}} {\bf T}\},$ so that $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} = \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds definitely$.$

In order to characterize the relations between the collections of the random vectors in (4.1), it is necessary to discuss the following four cases$:$

${\rm (a)}$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \cap \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\} \neq \emptyset$ holds with probability $1;$

${\rm (b)}$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \supseteq \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds with probability $1;$

${\rm (c)}$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \subseteq \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds with probability $1;$

${\rm (d)}$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} = \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds with probability $1.$

Our main results are given below.

Theorem 4.1. Let ${\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})$ and ${\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})$ be as given in (3.15) and (3.25)$,$ respectively and define

Then$,$ the following results hold$.$

${\rm (a)}$ There exist $\widehat{ {\bf L}}$ and $\widehat{ {\bf K}}$ such that $\widehat{ {\bf L}} = \widehat{ {\bf K}} {\bf T}$ if and only if ${\mathscr R}({\bf \Gamma}^{\prime}) \subseteq {\mathscr R}({\bf \Lambda}^{\prime}).$ In this case$,$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \cap \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\} \neq \emptyset$ holds definitely$.$

${\rm (b)}$ $\{\widehat{ {\bf L}}\} \supseteq \{\widehat{ {\bf K}} {\bf T}\}$ if and only if ${\mathscr R}({\bf \Gamma}^{\prime}) \subseteq {\mathscr R}({\bf \Lambda}^{\prime}).$ In this case$,$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \supseteq \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds definitely$.$

${\rm (c)}$ $\{\widehat{ {\bf L}}\} \subseteq \{\widehat{ {\bf K}} {\bf T}\}$ if and only if $r\!\begin{bmatrix} {\bf \Lambda} \\ {\bf \Gamma}\end{bmatrix} = r({\bf T}) + r({\bf X}) + r[{\bf X}, \, {{\bf{\Sigma}}}] - n.$ In this case, $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \subseteq \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds definitely.

${\rm (d)}$ $\{\widehat{ {\bf L}}\} = \{\widehat{ {\bf K}} {\bf T}\}$ if and only if ${\mathscr R}({\bf \Gamma}^{\prime}) \subseteq {\mathscr R}({\bf \Lambda}^{\prime})$ and $r[{\bf T} {\bf X}, \, {\bf T} {{\bf{\Sigma}}}] = r[{\bf X}, \, {{\bf{\Sigma}}}] + r({\bf T}) - n.$ In this case, $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} = \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds definitely.

Proof. From (3.15) and (3.25), the difference $\widehat{ {\bf L}} - \widehat{ {\bf K}} {\bf T}$ can be written as

where ${\bf Q} = [{\bf A} {\bf X}^{\prime}, \, {\bf B} {{\bf{\Sigma}}} {\bf X}^{\perp}][{\bf X} {\bf X}^{\prime}, {{\bf{\Sigma}}} {\bf X}^{\perp}]^{\dagger} - [{\bf A}({\bf T} {\bf X})^{\prime}, \, {\bf B} {{\bf{\Sigma}}} {\bf T}^{\prime}({\bf T} {\bf X})^{\perp}][{\bf T} {\bf X}({\bf T} {\bf X})^{\prime}, {\bf T} {{\bf{\Sigma}}} {\bf T}^{\prime}({\bf T} {\bf X})^{\perp}]^{\dagger} {\bf T}$ and ${\bf U}_1 \in {\mathbb R}^{s \times m}$ and ${\bf U}_2 \in {\mathbb R}^{s \times n}$ are arbitrary. Applying (2.8) to (4.3) gives

It is easy to obtain by (2.3), (2.4) and elementary block matrix operations (EBMOs) that

and

Substituting (4.5) and (4.6) into (4.4) yields

Setting the right-hand side of (4.7) equal to zero and applying Lemma 2.2(b) yields the equivalent condition in (a). Applying (2.8) to (4.3) yields

and by (2.9) and (4.5),

Combining (4.8) and (4.9) yields

Setting the right-hand side of (4.10) equal to zero yields $r\!\begin{bmatrix} {\bf \Lambda} \\ {\bf \Gamma}\end{bmatrix} = r({\bf \Gamma}).$ Thus, the statement in (b) holds.

By a similar approach, we can obtain

as required for the statement in (c). Combining (b) and (c) yields (d). □

Theorem 4.2. Let ${\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})$ and ${\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})$ be as given in (3.15) and (3.25)$,$ respectively, and let ${\bf \Lambda}$ and ${\bf \Gamma}$ be as given in (4.2)$.$ Then$,$ the following six statements are equivalent$:$

${\rm (a)}$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \cap \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\} \neq \emptyset$ holds definitely$.$

${\rm (b)}$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \cap \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\} \neq \emptyset$ holds with probability $1.$

${\rm (c)}$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \supseteq \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds with probability $1.$

${\rm (d)}$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \subseteq \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds with probability $1.$

${\rm (e)}$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} = \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds with probability $1.$

${\rm (f)}$ ${\mathscr R}({\bf \Gamma}^{\prime}) \subseteq {\mathscr R}({\bf \Lambda}^{\prime}).$

Proof. It can be seen from Lemma 2.6 and Definition 2.8(c) that (a) is equivalent to

Substituting the coefficient matrices in (3.15) and (3.25) into (4.12) and simplifying, we obtain

where ${\bf J} = [{\bf A} {\bf X}^{\prime}, \, {\bf B} {{\bf{\Sigma}}} {\bf X}^{\perp}] - [{\bf A}({\bf T} {\bf X})^{\prime}, \, {\bf B} {{\bf{\Sigma}}} {\bf T}^{\prime}({\bf T} {\bf X})^{\perp}][{\bf T} {\bf X}({\bf T} {\bf X})^{\prime}, {\bf T} {{\bf{\Sigma}}} {\bf T}^{\prime}({\bf T} {\bf X})^{\perp}]^{\dagger} {\bf T}[{\bf X} {\bf X}^{\prime}, \, {{\bf{\Sigma}}} {\bf X}^{\perp}]$ and ${\bf U}_1 \in {\mathbb R}^{s \times m}$ is arbitrary. From Lemma 2.5, the matrix equation is solvable for ${\bf U}_1$ if and only if

Applying (2.3) and (2.4), and simplifying, leads to

and

Substituting (4.14) and (4.15) into (4.13) leads to $r\!\begin{bmatrix} {\bf \Lambda} \\ {\bf \Gamma}\end{bmatrix} = r({\bf \Gamma}),$ thus establishing the equivalence of (a) and (e).

From Lemma 2.6 and Definition 2.8(d) that (b) is equivalent to

From (2.7), (3.15), (3.23), (3.25) and (4.14),

Setting the right-hand side of (4.17) equal to zero yields $r\!\begin{bmatrix} {\bf \Lambda} \\ {\bf \Gamma}\end{bmatrix} = r({\bf \Gamma}),$ thus establishing the equivalence of (b) and (e).

Similarly, we are able to obtain

Thus, (4.18) is equivalent to $r\!\begin{bmatrix} {\bf \Lambda} \\ {\bf \Gamma}\end{bmatrix} = r({\bf \Gamma}).$ Combining results (b) and (c) leads to the equivalence of (d) and (e). □

Combining Theorems 4.1 and 4.2, we obtain the following result:

Corollary 4.3. Let ${\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})$ and ${\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})$ be as given in (3.15) and (3.25)$,$ respectively, and let ${\bf \Lambda}$ and ${\bf \Gamma}$ be as given in (4.2)$.$ Then$,$ the following six statistical statements are equivalent$:$

${\rm (a)}$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \cap \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\} \neq \emptyset$ holds definitely$.$

${\rm (b)}$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \supseteq \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds definitely.

${\rm (c)}$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \cap \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\} \neq \emptyset$ holds with probability $1.$

${\rm (d)}$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \supseteq \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds with probability $1.$

${\rm (e)}$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} \subseteq \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds with probability $1.$

${\rm (f)}$ $\{{\rm BLMBP}_{ {\mathscr M}}(\boldsymbol{\tau})\} = \{{\rm BLMBP}_{ {\mathscr N}}(\boldsymbol{\tau})\}$ holds with probability $1.$

The results in the above theorems and corollaries can be simplified further for different choices of the matrices in (1.3), such as, ${\bf A} = {\bf K}$ and ${\bf B} = {\bf 0}$. Hence, many specified conclusions can further be obtained on the relationships between a GLM and its TGLMs. Assume that $\boldsymbol{\tau}$ in (1.3) is predictable under (1.1) and (1.5). We obtain that the BLMBP of $\boldsymbol{\tau}$ is just its BLUP, and the main results in the paper are the classic theory on the BLUPs of $\boldsymbol{\tau}$ under (1.1) and (1.5). Therefore, these works are certain extensions of the classic BLUP theory.

5.   An example

Assume that a concrete form of ${\mathscr M}$ in (1.1) is given by

In this case, taking two transformation matrices ${\bf T}_{1} = [{\bf I}_{n_{1}}, \, {\bf 0}]$ and ${\bf T}_{2} = [{\bf 0}, \, {\bf I}_{n_{2}}]$, we obtain the following two sub-sample models:

where it is assumed that ${\bf y}_{i} \in {\mathbb R}^{n_{i} \times 1}, {\bf X}_{i} \in {\mathbb R}^{n_{i} \times p_{i}}, \boldsymbol{\theta}_{i} \in {\mathbb R}^{p_{i} \times 1},$ $\boldsymbol{\nu}_{i} \in {\mathbb R}^{n_{i} \times 1}$, $n = n_{1} + n_{2}$, $p = p_{1} + p_{2}$. For illustrating the results in Section 4, let ${\bf A} = [{\bf X}_1, \, {\bf 0}], {\bf B} = {\bf 0}$ and ${\bf A} = [{\bf 0}, \, {\bf X}_2], {\bf B} = {\bf 0}$ in (1.3), respectively. From Theorems 4.1 and 4.2,

Obviously, the equivalent conditions all hold in Theorems 4.1 and 4.2. Thus, we can easily describe the relations between the corresponding estimators.

6.   Conclusions

We have provided algebraic and statistical analysis of a biased prediction problem when a joint parametric vector is unpredictable under a given GLM, and obtained an abundance of exact formulas and facts about the BLMBPs of the joint parametric vector in the contexts of a GLM and its TGLMs. All the findings in this article are technically formulated or denoted in certain analytical expressions or explicit assertions through the surprise use of specified matrix analysis tools and techniques. Hence, it is not difficult to understand these results and facts from both mathematical and statistical aspects. In view of this fact, we can take these obtained in the preceding sections as a group of theoretical contributions in the statistical inference under general linear model assumptions. Consequently, we are able to utilize the statistical methods developed in this article to provide additional insight into various concrete inference problems and subjects related to GLMs. Correspondingly, we point out that the main conclusions presented in this work have certain significant applications in the field of inverse scattering problems. The reader is referred to ^[8,31,32] on the topic of inverse scattering problems.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

We are grateful to anonymous reviewers for their helpful comments and suggestions. The second author was supported in part by the Shandong Provincial Natural Science Foundation $\#$ZR2019MA065.

Conflict of Interest

The authors declare that they have no conflicts of interest.