Pilot estimators for a kind of sparse covariance matrices with incomplete heavy-tailed data

1.   Introduction

Let $\mathbf{X}$ be a $p$-dimensional random vector. Estimating its covariance matrix $\mathbf{\Sigma} = (\sigma_{uv})_{p\times p}$ is of interest in high-dimensional statistics (Mendelson and Zhivotovskiy ^[1], Dendramis et al. ^[2] and Zhang et al. ^[3]). Until now, a commonly adopted strategy for evaluating the covariance matrix has been to impose sparse structure on itself (Belomestny^[4], Kang and Deng^[5], Bettache et al.^[6] and Liang et al. ^[7]).

If $\mathbf{X}$ is sub-Gaussian, Bickel and Levina^[8], Cai and Liu^[9] and Cai and Zhou^[10] considered all the rows or columns of the covariance matrix belonging to $l_q$-ball, weighted $l_q$-ball or weak $l_q$-ball as a kind of sparse assumption. Moreover, they proposed the corresponding thresholding estimators and established the convergence rates in sense of probability or expectation respectively.

When each component of $\mathbf{X} = (X_1, \cdots, X_p)^T$ is subject to heavy-tailed distribution, i.e., the distribution of $X_u$ satisfies $\int_\mathbb{R} e^{tx}dF_u(x) = \infty$ for $t > 0$, Avella-Medina et al.^[11] introduced a pilot estimator $\tilde{\mathbf{\Sigma}} = (\tilde{\sigma}_{uv})_{p\times p}$ satisfying

for positive constant $C_0$ and $\log p = o(n)$. Where $\varepsilon_{n, p}$ is a deterministic positive sequence and satisfies $\lim\limits_{n, p\rightarrow \infty}\varepsilon_{n, p} = 0$. Avella-Medina et al. pointed out the sample covariance matrix

must be a pilot estimator if $\mathbf{X}_1, \cdots, \mathbf{X}_n$ are i.i.d. sub-Gaussian random samples. In addition, some other pilot estimators were provided under bounded fourth moment assumption. The authors also considered convergence rate of the thresholding pilot estimator in terms of probability when the rows or columns of the covariance matrix are in weighted $l_q$-ball.

However, missing data (also called incomplete data) always occurs in high-dimensional sampling setting, see Hawkins et al.^[12], Lounici^[13] and Loh and Wain-wright^[14]. Instead of obtaining whole i.i.d. samples $\mathbf{X}_1, \cdots, \mathbf{X}_n$, one can only collect some parts of them. Let the vector $\mathbf{S}_i\in\{0, 1\}^p\; (i = 1, \cdots, n)$ denote by

where $X_{iu}$ and $S_{iu}$ are the $u$-th coordinate of the $\mathbf{X}_i$ and $\mathbf{S}_i$ respectively. This paper denotes the samples with missing values by $\mathbf{X}_i^* = (X_{i1}^*, \cdots, X_{ip}^*)^T$ where $X_{iu}^* = X_{iu}S_{iu}$. The following missing mechanism introduced by Cai and Zhang^[15] is adopted.

Assumption 1.1 (Missing completely at random). $\mathbf{S} = \{\mathbf{S}_1, \cdots, \mathbf{S}_n\}$ can be either deterministic or random and is independent of $\mathbf{X} = \{\mathbf{X}_1, \cdots, \mathbf{X}_n\}$.

Define

i.e., $n_{uv}^*$ is the number of the $u$-th and $v$-th entries of $\mathbf{X}_i^*$ being both observed. For convenience, let

Then, it is easy to see

Meanwhile, the generalized sample mean $\bar{\mathbf{X}}^* = (\bar{X}_u^*)_{1\leq u\leq p}$ is defined by

and the generalized sample covariance matrix $\hat{\mathbf{\Sigma}}^* = (\hat{\sigma}_{uv}^*)$ is given by

Our goal is to construct the thresholding estimator of the sparse covariance matrix $\mathbf{\Sigma}$ based on incomplete heavy-tailed data. Furthermore, the convergence rates of the thresholding estimator are investigated in terms of probability and expectation respectively.

The rest of paper is organized as follows. Section 2 introduces the definition of generalized pilot estimator based on missing data. Then under bounded fourth moment assumption, two kinds of generalized pilot estimators are given. In Section 3, we construct the thresholding generalized pilot estimator and explore its convergence rates in sense of probability under spectral and Frobenius norms respectively. In Section 4, the convergence rates are given in terms of expectation under an extra mild condition. Then, Section 5 investigates the numerical performances of the thresholding generalized Huber pilot estimator and thresholding generalized truncated mean pilot estimator respectively and compares these two estimators with the adaptive thresholding estimator proposed by Cai and Zhang ^[15].

2.   Generalized pilot estimator

Definition 2.1. Any symmetric matrix $\tilde{\mathbf{\Sigma}}^* = (\tilde{\sigma}_{uv}^*)_{p\times p}$ based on incomplete data $\mathbf{X}_{1}^*, \cdots, \mathbf{X}_{n}^*$ is said to be a generalized pilot estimator of $\mathbf{\Sigma}$, if for $L > 0$ there exists constant $C_0(L)$ such that

holds with $\log p = o(n_{\min}^*)$.

Remark 2.1. If one can obtain complete data, the generalized pilot estimator defined by (2.1) coincides with the pilot estimator proposed by Avella-Medina et al.^[11] except $\varepsilon_{n, p}$ is replaced by $O(p^{-L})$.

Remark 2.2. If $\mathbf{X}$ is a sub-Gaussian random vector and the items $\sigma_{uu}\; (u = 1, \cdots p)$ of $\mathbf{\Sigma}$ are uniformly bounded, the generalized sample covariance matrix $\hat{\mathbf{\Sigma}}^*$ given by (1.1) must be a generalized pilot estimator of $\mathbf{\Sigma}$.

In fact, Theorem 3.1 in ^[15] tells that for any $0 < x\leq1$ there exists constants $C, \; c > 0$ such that

By $n_{\min}^*\leq n_{uv}^*$ and $\log p = o(n_{\min}^*)$, one knows $\log p = o(n_{uv}^*).$ If $x = \sqrt{(2+L)\log p/(cn_{uv}^*)}$ with $L > 0$, (2.2) reduces to

with $C_0(L) = \sqrt{(2+L)\sigma_{uu}\sigma_{vv}/c}.$

Furthermore,

Therefore, $\hat{\mathbf{\Sigma}}^*$ given by (1.1) is a generalized pilot estimator of $\mathbf{\Sigma}$.

We introduce the following theorem in order to provide two other kinds of generalized pilot estimators under bounded fourth moment assumption.

Theorem 2.1. Suppose $\max\limits_{1\leq u\leq p}\mathbb{E}|X_u|^4\leq k^4$, $\log p = o(n_{\min}^*)$, $\mathbb{E}X_u = \mu_u, \; \mathbb{E}(X_uX_v) = \mu_{uv}$, and Assumption 1.1 holds. If $\tilde{\mu}_{u}^*$ and $\tilde{\mu}_{uv}^*$ satisfy

with absolute constants $L > 0$ and $c\geq1$ then $\tilde{\mathbf{\Sigma}}^* = (\tilde{\sigma}^*_{uv})_{p\times p}: = (\tilde{\mu}_{uv}^*-\tilde{\mu}_{u}^*\tilde{\mu}_{v}^*)_{p\times p}$ must be a generalized pilot estimator of $\mathbf{\Sigma}$.

Proof. Let $K: = ck\sqrt{2+L}$. Thus,

thanks to condition (2.3). Moreover,

Similarly, one derives

due to (2.4) and $c\geq1.$

By $\max\limits_{1\leq u\leq p}\mathbb{E}|X_u|^4\leq k^4$, one obtains $|\mu_u|\leq (\mathbb{E}|X_u|^4)^{1/4}\leq k\; (u = 1, \cdots, p)$ and

the above last inequality follows from $K\geq ck.$ Thus, one concludes

thanks to (2.5) and (2.6).

Since $n_{uv}^*\leq\min\{n_{u}^*, n_{v}^*\}$, the above result reduces to

Furthermore, according to $\log p = o(n_{\min}^*)$ and $n_{\min}^*\leq n_{uv}^*$, one knows $\log p = o(n_{uv}^*)$. Therefore, $(\log p)/n_{uv}^*\leq\left((\log p)/n_{uv}^*\right)^{1/2}$ and

hold.

Note that

Then,

follows from (2.7) and (2.8).

Hence, $\tilde{\mathbf{\Sigma}}^*$ is a generalized pilot estimator of $\mathbf{\Sigma}$ with $C_0(L) = 2ck^2(1+c)(2+L)$. □

We shall give two generalized pilot estimators based on incomplete heavy-tailed samples.

Denote Huber function by

where $\alpha > 0$ and $\psi(x) = \begin{cases} x, & |x|\leq 1, \\ \text{sign}\; x, & |x| > 1\end{cases}.$ For any constant $L > 0$, let $(\tilde{\mu}^*_{H})_u\; (u = 1, \cdots, p)$ satisfy

with $\alpha_u: = \sqrt{n_u^*\zeta^2/(2+L)\log p}$ and $\zeta\geq\sqrt{DX_u}$. Similarly, $(\tilde{\mu}^*_{H})_{uv}\; (u, v = 1, \cdots, p)$ satisfies

with $\alpha_{uv}: = \sqrt{n_{uv}^*\zeta_1^2/(2+L)\log p}$ and $\zeta_1\geq\sqrt{D(X_uX_v)}$. Then, we have the following estimator.

Example 2.1. (Generalized Huber estimator). Suppose conditions of Theorem 2.1 hold, then $\tilde{\mathbf{\Sigma}}^*_{H}: = \Big((\tilde{\mu}^*_{H})_{uv}- (\tilde{\mu}^*_{H})_u(\tilde{\mu}^*_{H})_v\Big)_{p\times p}$ is a generalized pilot estimator of $\mathbf{\Sigma}$, where $(\tilde{\mu}^*_{H})_j\; (j = u, v)$ and $(\tilde{\mu}^*_{H})_{uv}$ are defined by (2.9) and (2.10).

Proof. With the definition of $X_{iu}^*$, (2.9) is equivalent to

where $A_u = \{i:S_{iu}\neq 0\}.$ Obviously, $|A_u| = \sum_{i = 1}^nS_{iu}$. By the definition of $n_u^*$, we have $|A_u| = n_u^*.$

Similarly, we find (2.10) is equivalent to

with $A_{uv} = \{i:S_{iu}S_{iv}\neq 0\}$ and $|A_{uv}| = n_{uv}^*.$

By $\max\limits_{1\leq u\leq p}\mathbb{E}|X_u|^4\leq k^4$, we get

On the other hand,

due to Cauchy-Schwarz inequality. Thus,

Obviously, it holds

thanks to $n_{\min}^*\leq n_{uv}^*\leq n_{u}^*$ and $\log p = o(n_{\min}^*).$

According to (2.11)–(2.13) and Theorem 5 in ^[16], we know that if $(2+L)\log p/n_u^*\leq 1/8$ and $(2+L)\log p/n_{uv}^*\leq 1/8$ then

i.e., $(\tilde{\mu}^*_{H})_u$ and $(\tilde{\mu}^*_{H})_{uv}$ reach the expected results (2.3) and (2.4).□

In order to give another generalized pilot estimator, let $(\tilde{\mu}^*_{T})_u\; (u = 1, \cdots, p), \; (\tilde{\mu}^*_{T})_{uv}\; (u, v = 1, \cdots, p)$ be defined by

respectively where $L > 0$, $\beta\geq\sqrt{\mathbb{E}|X_u|^2}$ and $\beta_1\geq\sqrt{\mathbb{E}|X_uX_v|^2}.$ Then, we have the second estimator.

Example 2.2 (Generalized truncated mean estimator). Suppose conditions of Theorem 2.1 hold. Then, $\tilde{\mathbf{\Sigma}}^*_{T}: = \Big((\tilde{\mu}^*_{T})_{uv}-(\tilde{\mu}^*_{T})_u (\tilde{\mu}^*_{T})_v\Big)_{p\times p}$ is a generalized pilot estimator of $\mathbf{\Sigma}$, where $(\tilde{\mu}^*_{T})_j\; (j = u, v)$ and $(\tilde{\mu}^*_{T})_{uv}$ are defined by (2.14) and (2.15).

Proof. We first show $(\tilde{\mu}^*_{T})_u$ satisfies (2.3). According to $\max\limits_{1\leq u\leq p}\mathbb{E}|X_u|^4\leq k^4$, we have

So, (2.14) is equivalent to

where $A_u = \{i:S_{iu}\neq 0\}.$ Let $a: = k\sqrt{n_u^*/(2+L)\log p}$. We derive

Therefore, upon combining $\mathbb{E}|X_u|^2\leq k^2$ and $|A_u| = n_u^*$

According to $\mathbb{E}(X_{iu}^2\mathbf{1}{\{|X_{iu}|\leq a\}})\leq k^2$ and Bernstein's inequality in ^[17],

for any $t > 0$.

By (2.16) and (2.17) and taking $t = (2+L)\log p$, we have

Substituting $a = k\sqrt{n_u^*/(2+L)\log p}$ into the above inequality, we know

which is the expected condition (2.3) of Theorem 2.1.

Similarly, we can derive

i.e., the condition (2.4) of Theorem 2.1 holds.□

3.   Convergence rates in terms of probability

We introduce the thresholding function and the space of sparse covariance matrices.

Definition 3.1. For any constant $\lambda > 0$, a real valued function $\tau_{\lambda}(\cdot)$ is said to be thresholding function if

(i) $\tau_{\lambda}(z) = 0, \; |z|\leq\lambda$;

(ii) $|\tau_{\lambda}(z)-z|\leq\lambda$;

(iii) $|\tau_{\lambda}(z)|\leq c_0|y|$ for $|z-y|\leq\lambda$ and the constant $c_0 > 0$.

In fact, many functions satisfy conditions (i)–(iii). For example, the soft thresholding function $\tau_{\lambda}(z) = \text{sign}(z)(|z|-\lambda)_{+}$, the adaptive lasso thresholding function $\tau_{\lambda}(z) = z(1-|\lambda/z|^{\eta})$ with $\eta\geq1$ and the smoothly clipped absolute deviation thresholding rule proposed by Rothman et al.^[18].

This paper considers the following class of covariance matrices introduced by ^[15]

Next, we define the thresholding generalized pilot estimator $(\tilde{\mathbf{\Sigma}}^*)^{\tau} = ((\tilde{\sigma}_{uv}^*)^{\tau})_{p\times p}$ and consider its convergence rates in terms of probability under spectral and Frobenius norms respectively over the parametric space $\mathcal{H}(s_{n, p})$.

Let $\tilde{\mathbf{\Sigma}}^* = (\tilde{\sigma}_{uv}^*)$ be a generalized pilot estimator and define

where $\tau_{\lambda_{uv}}(\cdot)$ is the thresholding function with

The constant $\delta$ will be specified in the proving process of Lemma 3.1.

The following lemma is useful for inferring Theorem 3.1 and Theorem 4.1.

Lemma 3.1. Suppose $\min\limits_{u}\sigma_{uu}\geq\gamma > 0$, $\log p = o(n_{\min}^*)$ and Assumption 1.1 hold. Denote the events $Q_1, \; Q_2$ as

Then, for any $L > 0$

(i) there exists $C_1(L) > 0$ such that

holds under the event $Q_1\cap Q_2.$

(ii) $\mathbb{P}(Q_1\cap Q_2)\geq1-O(p^{-L})$.

Proof. (i) Under the event $Q_1$, one knows

thanks to conditions (ii) and (iii) of Definition 3.1.

Define

where $C_0(L)$ is given in Definition 2.1. By (3.2) when the event $Q_2$ happens as well (3.6) reduces to

According to (3.5) and (3.8), one obtains

under the event $Q_1\cap Q_2$. Therefore,

holds due to $n_{\min}^*\leq n_{uv}^*\leq n$. This reaches the conclusion (i) of Lemma 3.1.

(ii) In order to show $\mathbb{P}(Q_1\cap Q_2)\geq1-O(p^{-L})$, one first estimates $\mathbb{P}(Q_2^c).$

Clearly,

and

Define the event

Since $\tilde{\mathbf{\Sigma}}^* = (\tilde{\sigma}_{uv}^*)_{p\times p}$ is a generalized pilot estimator of $\mathbf{\Sigma} = (\sigma_{uv})_{p\times p}$ then one gets

By $\log p = o(n_{\min}^*)$ and $n_{\min}^*\leq n_{uv}^*$, one knows $\log p = o(n_{uv}^*).$ Furthermore, it holds

under the event $E$ because of $\min_{u}\sigma_{uu}\geq\gamma$. Hence,

Next to estimate $\mathbb{P}(Q_1^c).$ One observes $\tilde{\sigma}_{uu}^*\tilde{\sigma}_{vv}^*\geq\sigma_{uu}\sigma_{vv} -|\tilde{\sigma}_{uu}^*-\sigma_{uu}||\tilde{\sigma}_{vv}^*|-|\tilde{\sigma}_{vv}^*-\sigma_{vv}||\tilde{\sigma}_{uu}^*| -|\tilde{\sigma}_{uu}^*-\sigma_{uu}||\tilde{\sigma}_{vv}^*-\sigma_{vv}|$ and it follows that

holds true on the event $E$ due to $\min_{u}\sigma_{uu}\geq\gamma$ and $\log p = o(n_{uv}^*)$. Hence,

Let $\lambda_{uv} = \delta\sqrt{\tilde{\sigma}_{uu}^*\tilde{\sigma}_{vv}^*\log p/n_{uv}^*}$ given by (3.2). It can be shown that

follows from (3.10).

Note that $\tilde{\mathbf{\Sigma}}^* = (\tilde{\sigma}_{uv}^*)$ is a generalized pilot estimator of $\mathbf{\Sigma}$. Then, one derives

thanks to $\delta = \sqrt{2}C_0(L)/\gamma$ defined in (3.7). Substituting the above result into (3.11) gives

Combining this with (3.9), one obtains the stated result

□

Finally, we give the upper bounds of $\|(\tilde{\mathbf{\Sigma}}^*)^{\tau}-\mathbf{\Sigma}\|_{2, F}$ in terms of probability and $\|\mathbf{A}\|_{2, F}$ denotes the spectral and Frobenius norms of matrix $\mathbf{A}$ respectively.

Theorem 3.1. Suppose $\min\limits_{u}\sigma_{uu}\geq\gamma > 0$, $\log p = o(n_{\min}^*)$ and Assumption 1.1 hold. Then,

Proof. (i) Define the event $Q: = Q_1\cap Q_2$, where $Q_1, \; Q_2$ are given by (3.3) and (3.4) respectively. Then, it is easy to see

thanks to Lemma 3.1 and $\mathbf{\Sigma}\in\mathcal{H}(s_{n, p})$.

Gersgorin theorem tells $\|(\tilde{\mathbf{\Sigma}}^*)^{\tau}-\mathbf{\Sigma}\|_2 \leq\|(\tilde{\mathbf{\Sigma}}^*)^{\tau}-\mathbf{\Sigma}\|_1$ and this combining (3.12) implies

on the event $Q.$

On the other hand, Lemma 3.1 tells $\mathbb{P}(Q)\geq1-O(p^{-L})$. Hence, Theorem 3.1(i) holds.

(ii) One observes

and it follows

on the event $Q$ according to Lemma 3.1.

Note that $\max\limits_v\sum\limits_{u = 1}^p|a_{uv}|^2\leq (\max\limits_v\sum\limits_{u = 1}^p|a_{uv}|)^2.$ Then, (3.13) reduces to

as long as $\mathbf{\Sigma}\in\mathcal{H}(s_{n, p})$.

Therefore, conclusion (ii) reaches since Lemma 3.1 says $\mathbb{P}(Q)\geq1-O(p^{-L})$.

(iii) By $\max_{u}\sigma_{uu}\leq M$, one knows

Furthermore, it holds

under the event $Q$ due to (3.13) and $\mathbf{\Sigma}\in\mathcal{H}(s_{n, p})$.

Thus, the claim (iii) follows from Lemma 3.1 immediately.□

Remark 3.1. Theorem 3.1(i) generalizes the result of ^[15] which requires $\mathbf{X}$ is the sub-Gaussian random vector. In addition, if $n_{\min}^* = n$, Theorem 3.1(i) yields the result of ^[11] thanks to the parametric class $\mathcal{H}(s_{n, p})$ containing the class of sparse covariance matrices defined in ^[11].

Remark 3.2. From the proving process of Theorem 3.1(i), we find that

under the event $Q$.

Furthermore, let $\|\mathbf{A}\|_\omega$ denote the matrix $l_{\omega}$-operator norm of $\mathbf{A}$, Lemma 7.2 in ^[19] tells

for any symmetric matrix $\mathbf{A}$. Hence,

holds under the event $Q$.

Then, using Lemma 3.1 indicates

4.   Convergence rates in terms of expectation

This section studies the convergence rates of the thresholding generalized pilot estimator $(\tilde{\mathbf{\Sigma}}^*)^{\tau}$ in terms of expectation over $\mathcal{H}(s_{n, p})$.

We introduce the following technical lemma.

Lemma 4.1. Let $\min_{u}\sigma_{uu}\geq\gamma > 0$, $\log p = o(n_{\min}^*)$, $p\geq (n_{\min}^*)^{\xi}(\xi > 0)$, $\mathbb{E}|\tilde{\sigma}_{uv}^*-\sigma_{uv}|^{2}\leq M$ and Assumption 1.1 holds. Then,

Where $Q_1, Q_2$ are defined by (3.3) and (3.4). $x\lesssim y$ denotes $x\leq cy$ with a absolute constant $c > 0.$

Proof. Denote $Q: = Q_1\cap Q_2$ and

Then, $I_{n, p}\leq s_{n, p}\mathbb{P}(Q^c)$.

According to Lemma 3.1, one knows $\mathbb{P}(Q^c)\leq O(p^{-L})$ and $I_{n, p}\lesssim s_{n, p}p^{-L}.$ Taking $L = \xi^{-1}+3 > 0$, one obtains

due to $p\geq (n_{\min})^{\xi}$. Hence, it follows that

which is the desired conclusion (i).

Similarly, the definition of $\mathcal{H}(s_{n, p})$ and Lemma 3.1 imply

Moreover, the above result combining (4.1) concludes (ii).

To show (iii), Hölder inequality tells

On the other hand, it holds

Furthermore, one obtains

due to the given condition $\mathbb{E}|\tilde{\sigma}_{uv}^*-\sigma_{uv}|^{2}\leq M$. Hence,

follows from $\mathbb{P}(Q^c)\leq O(p^{-L})$.

By $L = \xi^{-1}+3$ and the assumptions $p\geq (n_{\min}^*)^{\xi}$, one finds

Substituting this into (4.2) implies the expected result (iii) holding.□

Theorem 4.1. Let $(\tilde{\mathbf{\Sigma}}^*)^{\tau} = ((\tilde{\sigma}_{uv}^*)^{\tau})_{p\times p}$ given by (3.1), $\mathbb{E}|\tilde{\sigma}_{uv}^*-\sigma_{uv}|^{2}\leq M$, $\min\limits_{u}\sigma_{uu}\geq\gamma > 0$ and Assumption 1.1 holds. If $\log p = o(n_{\min}^*)$, $p\geq (n_{\min}^*)^{\xi}(\xi > 0)$ and $s_{n, p}\gtrsim1$ then

Proof. (i) Let the event $Q: = Q_1\cap Q_2$ where $Q_1, Q_2$ are given by (3.3) and (3.4). Then, by Gersgorin theorem $\|(\tilde{\mathbf{\Sigma}}^*)^{\tau}-\mathbf{\Sigma}\|_2\leq\|(\tilde{\mathbf{\Sigma}}^*)^{\tau} -\mathbf{\Sigma}\|_1$ we have

Clearly, $\|(\tilde{\mathbf{\Sigma}}^*)^{\tau}-\mathbf{\Sigma}\|_1 : = \max\limits_v\sum_{u = 1}^p|\tau_{\lambda_{uv}}(\tilde{\sigma}_{uv}^*)-\sigma_{uv}|$ and it follows

under the event $Q$ thanks to Lemma 3.1 and the definition of $\mathcal{H}(s_{n, p})$. Hence,

Then, we just need to show

for finishing the proof of (i).

According to condition (iii) of Definition 3.1, we obtain $|\tau_{\lambda_{uv}}(\tilde{\sigma}_{uv}^*)|\le c_0|\tilde{\sigma}_{uv}^*|$ and

By $|\sigma_{uv}|\leq\sqrt{\sigma_{uu}\sigma_{vv}}$ and $\log p = o(n_{\min}^*)$, we know

due to $n_{\min}^*\leq n$. Hence, it holds

and

Therefore, (4.3) follows from Lemma 4.1(i), (iii) and $s_{n, p}\gtrsim1.$ This reaches (i).

(ii) To show (ii), we observe

Clearly,

According to Lemma 3.1, we have

on the event $Q$. Furthermore,

holds due to the definition of $\mathcal{H}(s_{n, p})$.

Hence, it suffices to prove

By (4.4), we find

Substituting the above inequality into (4.5) leads to

Since $\sqrt{|a|+|b|}\leq\sqrt{|a|}+\sqrt{|b|}$ and $(\max\limits_{v}\sum\limits_{u = 1}^p|a_{uv}|^2)^{\frac{1}{2}} \leq\max\limits_{v}\sum\limits_{u = 1}^p|a_{uv}|$, we obtain

Thus, (4.7) follows from Lemma 4.1(i), (iii) and $s_{n, p}\gtrsim1.$

(iii) By $\max\limits_u\sigma_{uu}\leq M$, we obtain

On the other hand, (4.6), (4.9) and $\mathbf{\Sigma}\in\mathcal{H}(s_{n, p})$ tells

under the event $Q$. Therefore,

Using (4.8) and (4.9), we have

Hence, it holds

due to Lemma 4.1(ii), (iii) and $s_{n, p}\gtrsim1.$ Finally, conclusion (iii) follows from (4.10) to (4.11). □

Remark 4.1. The upper bound of Theorem 4.1(i) is optimal due to Proposition 3.1 in ^[15]. In addition, Theorem 4.1(i) performs better than Theorem 3.1 of ^[15] which requires $\mathbf{X}$ to be sub-Gaussian.

Remark 4.2. From the proving process of Theorem 4.1(i), we observe

Note that $\|\mathbf{A}\|_\omega\leq\|\mathbf{A}\|_1(1\leq\omega\leq\infty)$ for any symmetric matrix $\mathbf{A}$. Then,

holds.

Remark 4.3. The condition

in Lemma 4.1 and Theorem 4.1 is mild. In fact, the generalized Huber estimator (Example 2.1) and generalized truncated mean estimator (Example 2.2) both satisfy (4.12). The details can be found in Appendix.

5.   Simulation studies

Let $(\tilde{\mathbf{\Sigma}}^*_H)^{\tau}$ and $(\tilde{\mathbf{\Sigma}}^*_T)^{\tau}$ be defined by (3.1) and (3.2), this section investigates the numerical properties and performances of the estimators $(\tilde{\mathbf{\Sigma}}^*_H)^{\tau}$, $(\tilde{\mathbf{\Sigma}}^*_T)^{\tau}$ and compares these two estimators with the adaptive thresholding estimator $\hat{\mathbf{\Sigma}}^{at}$ proposed by ^[15]. The following two types of sparse covariance matrices are considered:

Model 1. (Rothman et al., ^[18]) $\mathbf{\Sigma} = (\sigma_{uv})_{p\times p}$ with $\sigma_{uv} = \max\{1-|u-v|/5, 0\}.$

Model 2. (Cai and Zhang, ^[15]) $\mathbf{\Sigma} = \mathbf{I}_p+ (\mathbf{D}+\mathbf{D}^T)/(\|\mathbf{D}+\mathbf{D}^T\|_2+0.01)$, where $\mathbf{D} = (d_{uv})_{p\times p}$ is given by $d_{uu} = 0\; (u = 1, \cdots, p)$ and

Under each model we generate random samples $\mathbf{X}_i\in\mathbb{R}^p\; (i = 1, \cdots, n)$ by two different scenarios:

(i) $\mathbf{X}_i$ are independently drawn from multivariate $t$-distribution $t_{\nu}(0, \mathbf{\Sigma})$ with freedom $\nu = 4.5$;

(ii) $\mathbf{X}_i$ are independently drawn from multivariate skewed $t$-distribution $st_{\nu}(0, \mathbf{\Sigma}, \epsilon)$ with freedom $\nu = 5$ and skew parameter $\epsilon = 10.$

In each simulation setting we adopt the following two cases of the missingness for the data matrix $\mathbf{Y} = (\mathbf{X}_1, \cdots, \mathbf{X}_n)$ with $\mathbf{X}_i = (X_{i1}, \cdots, X_{ip})^T$ which proposed by Cai and Zhang^[15]. The first case is missing uniformly and completely at random(MUCR) in which every entries $X_{ik}$ are observed with probability $0 < \rho\leq1$. The second case is missing not uniformly but completely at random(MCR) in which $\mathbf{Y}$ is divided into four equal-size parts,

where every entries of $\mathbf{Y}_{11}, \; \mathbf{Y}_{22}$ are observed with probability $0 < \rho^{(1)}\leq1$ every entries of $\mathbf{Y}_{12}, \; \mathbf{Y}_{21}$ are observed with probability $0 < \rho^{(2)}\leq1.$

Moreover, for each procedure we set $p = 50, 200, 300$ and $n = 50, 100, 200$ respectively and 50 replications are used. Meanwhile, we choose the soft thresholding rule and measure the errors by the spectral and Frobenius norm respectively in each setting. The tuning parameter in thresholding estimator is chosen by 10-fold cross-validation which is explained in Section 4 of Cai and Zhang ^[15], and unspecified tuning parameters in the generalized pilot estimator are chosen by the method suggested in Section 6 of Avella-Medina et al.^[11].

Tables 1 and 2 demonstrate that thredsholding estimators $(\tilde{\mathbf{\Sigma}}^*_H)^{\tau}$ and $(\tilde{\mathbf{\Sigma}}^*_T)^{\tau}$ perform better than the adaptive thresholding estimator $\hat{\mathbf{\Sigma}}^{at}$ under both MUCR and MCR settings. Moreover, thresholding generalized Huber estimator $(\tilde{\mathbf{\Sigma}}^*_H)^{\tau}$ outperforms thresholding generalized truncated mean estimator $(\tilde{\mathbf{\Sigma}}^*_T)^{\tau}$. We also find that the errors decrease if sample size $n$ gets larger. Meanwhile, we observe that the errors under Model 1 is larger than under Model 2 since the covariance matrix in Model 1 is more dense than in Model 2. All these numerical results are consistent with our theoretical results.

6.   Conclusions

In this paper, we propose the generalized pilot estimator in the presence of incomplete heavy-tailed data. Moreover, two kinds of generalized pilot estimators are provided under the bounded fourth moment assumption while lots of previous studies hinged upon the sub-Gaussian condition. In addition, we establish the thresholding pilot estimator for a family of sparse covariance matrices and give the convergence rates in terms of probability and expectation respectively.

In the future, we may consider the compositional data with missing data under lower bounded moment assumption by referring Li et al. ^[20]. Moreover, we can adopt the different methods to estimate the sparse covariance matrix with incomplete data such as the proximal distance algorithm ^[21] or continuous matrix shrinkage ^[22].

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

This paper is supported by the National Natural Science Foundation of China (No. 12171016).

Conflict of interest

All authors declare no conflicts of interest in this paper.

Appendix

In order to show Example 2.1 and Example 2.2 satisfying condition (4.12), we introduce a Proposition A.1.

Proposition A.1. Let $\max\limits_{1\leq u\leq p}\mathbb{E}|X_u|^{4} \leq k^4$, $\mathbb{E}X_u = \mu_u$, $\mathbb{E}(X_uX_v) = \mu_{uv}$, and Assumption 1.1 holds. If $\tilde{\mu}_u^*$ and $\tilde{\mu}_{uv}^*$ satisfy

where $A, B\subseteq\{1, \cdots, n\}$. Then, $\tilde{\mathbf{\Sigma}}^* = (\tilde{\sigma}_{uv}^*)_{p\times p} = (\tilde{\mu}_{uv}^*-\tilde{\mu}_u^*\tilde{\mu}_v^*)_{p\times p}$ obeys (4.12).

Proof. It suffices to prove

By $\max\limits_{1\leq u\leq p}\mathbb{E}|X_u|^{4}\leq k^4$, one knows $\mathbb{E}|X_u|\leq(\mathbb{E}|X_u|^{4})^{1/4}\leq k$ and

Thus, it holds

which reaches (A.3).

For (A.4), one observes

According to (A.1) and Jensen's inequality, it follows

Furthermore, upon combining Cauchy–Schwarz inequality leads to

Similarly, (A.2) implies

By Cauchy-Schwarz inequality and $\max\limits_{1\leq u\leq p}\mathbb{E}|X_u|^{4}\leq k^4$, one finds

Hence,

Finally, the expected conclusion (A.4) follows from (A.5)–(A.7). This completes the proof of Proposition A.1.□

Now, based on Proposition A.1 we verify two kinds of generalized pilot estimators (Example 2.1 and Example 2.2) satisfying (4.12).

For the generalized truncated mean estimator $\tilde{\mathbf{\Sigma}}^*_{T}: = \Big((\tilde{\mu}^*_{T})_{uv}-(\tilde{\mu}^*_{T})_u (\tilde{\mu}^*_{T})_v\Big)_{p\times p}$, it is easy to see that $(\tilde{\mu}^*_{T})_u$, $(\tilde{\mu}^*_{T})_{uv}$ obey (A.1), (A.2) respectively.

By (2.14), we know

where $A_u = \{i: S_{iu}\neq 0\}$ and $|A_u| = n_u^*$. Similarly, it holds

with $A_{uv} = \{i: S_{iu}S_{iv}\neq 0\}$ and $|A_{uv}| = n_{uv}^*.$ The above two inequalities imply $(\tilde{\mu}^*_{T})_{u}, \; (\tilde{\mu}^*_{T})_{uv}$ satisfying (A.1), (A.2) respectively.

In fact, it is hard to check the generalized Huber estimator

satisfying (4.12) due to the structures of $(\tilde{\mu}^*_H)_u$ and $(\tilde{\mu}^*_{H})_{uv}$ being unclear. But we can consider a special case.

Proposition A.2. Let $A_u = \{i:S_{iu}\neq 0\}, \; A_{uv} = \{i:S_{iu}S_{iv}\neq 0\}$. If $\alpha_u, \; \alpha_{uv}$ defined in (2.9) and (2.10) obey

respectively. Then, $(\tilde{\mu}^*_H)_u, \; (\tilde{\mu}^*_{H})_{uv}$ satisfy (A.1), (A.2).

Proof. For $i\in A_u$, it holds

Obviously, (2.9) is equivalent to

By the definition of $\psi_{\alpha_u}(x)$, we have

Note that $\sum\limits_{i\in A_u}\psi_{\alpha_u}(X_{iu}-(\tilde{\mu}_H^*)_u)$ is the continuous and decreasing function about $(\tilde{\mu}_H^*)_u$. Then, the solution of equation $\sum\limits_{i\in A_u}\psi_{\alpha_u}(X_{iu}-(\tilde{\mu}_H^*)) = 0$ belongs to the interval $(\max\limits_{i\in A_u}X_{iu}-\alpha_u, \min\limits_{i\in A_u}X_{iu}+\alpha_u)$.

Hence, we obtain $\max\limits_{\{i\in A_u\}}X_{iu}-\alpha_u < (\tilde{\mu}_H^*)_u < \min\limits_{\{i\in A_u\}}X_{iu}+\alpha_u$ and

Furthermore, the above inequality and definition of $\psi_{\alpha_u}(x)$ implies

Therefore, $(\tilde{\mu}_H^*)_u = (n_u^*)^{-1}\sum_{i\in A_u}X_{iu}$ satisfies (A.1).

Following the similar discussion, we can derive $(\tilde{\mu}_H^*)_{uv}$ satisfying (A.2) with

□

In fact, the condition in Proposition A.2 is easy to satisfy, since $\log p = o(n_{\min}^*)$ and $n_{\min}^*\leq n_{uv}^*\leq n_{u}^*$ lead to large enough $\alpha_{u}$ and $\alpha_{uv}.$