Semi-supervised estimation for the varying coefficient regression model

1.   Introduction

Semi-supervised learning first appeared in machine learning literature, used to describe a situation where some data are labeled and the rest are unlabeled ^[10]. Conceptually, it is between supervised and unsupervised learning. It allows you to take advantage of a large amount of unlabeled data available and smaller labeled datasets in many instances. Semi-supervised learning occurs when the label variable is difficult to observe and may require complex or expensive processes. Specifically, a sample of $n$ observations in the joint distribution $({{\bf{X}}}, Y)$ is given, where $Y$ is the label variable and ${\bf{{X}}}$ contains the covariates. In addition, an additional $m$ samples are observed with only ${\bf{X}}$ given. The aim is to study the relationships between ${\bf{X}}$ and $Y$ using additional unlabeled data.

For semi-supervised learning, much literature focuses on the case that $Y$ takes a small number of values, which can be reduced to the case of classification tasks, such as ^[19]. In recent years, research in this field has focused on neural network-based models and generative learning algorithm problems. ^[13] introduced a simple and computationally efficient algorithm for training deep neural networks in a semi-supervised learning paradigm: interpolating Consistency Training (ICT). ^[8] introduced an innovative framework for federated transfer learning (FTL) known as Semi-Supervised Federated Heterogeneous Transfer Learning (SFHTL), to utilize unlabeled non-overlapping samples, addressing the challenge of model overfitting caused by the limited overlap of training samples in federated learning (FL) scenarios. Based on compatibility conditions in the semi-supervised probably approximately correct (PAC) theory, ^[7] demonstrated why labeled heterogeneous source data and unlabeled target data help reduce target risk. Based on this theory, two algorithms were designed as a proof of concept. One is the kernel heterogeneous domain alignment (KHDA) algorithm, which is a kernel-based algorithm, and the other is the joint mean embedded alignment (JMEA) algorithm based on neural networks. At the same time, some scholars have studied the regression problem. ^[14] classified different semi-supervised methods into two categories: distribution-based and marginal-based. The distribution method relied on the assumption that the conditional expectation $E(Y|{\bf{X}})$ was linked to the marginal distribution of ${\bf{X}}$. The marginal-based approach used additional information on ${\bf{X}}$. Some other studies, such as ^[18], took into account $Y$ sequential values and used unlabeled data to learn the structure of ${\bf{X}}$ so that non-parametric regression could be better estimated. These efforts are very helpful where non-parametric regression is useful and unlabeled data is available. ^[15] proposed an estimator of the population mean using unlabeled data combined with least squares. ^[16] provided a semi-supervised reasoning framework focused on the mean and variance of the responses, allowing covariates to be much larger in size than the sample sizes, and provided new estimates of the mean and variance of response variables. ^[4] considered the linear regression problem in a semi-supervised setting and proposed a class of highly efficient adaptive semi-supervised estimators (EASE) to improve the estimation efficiency. Additionally, they applied this method to the study of electronic medical records on autoimmunity. ^[2] proposed a correction estimator that effectively integrates labeled and unlabeled data, called Corrected High-dimensional Inference of Variance Interpretation Estimators (CHIVE), which achieved minimax optimal convergence speed under a general semi-supervised framework. ^[1] studied linear regression in a semi-supervised setting and, even in a framework where $E(Y|{\bf{X}})$ was not linear, additional information about the distribution of ${\bf{X}}$ was helpful to construct better estimates than standard least squares estimates. However, there are few studies on semi-supervised learning in the varying coefficient models. Therefore, we want to extend the intercept model method of ^[1] to the varying coefficient model and investigate the estimation of the coefficient function under semi-supervised learning.

Varying coefficient models constitute a versatile and expansive category of statistical models encompassing well-known structures like additive models, partial linear models, single-index coefficient regression models and adaptive varying coefficient partial linear models. Researchers have investigated varying coefficient models, seeking to understand and refine their applications across diverse statistical contexts. The exploration of these models extends beyond conventional frameworks, uncovering novel insights that contribute to the evolving landscape of statistical methodology. ^[6] introduced a robust two-step estimation approach designed for the coefficient function. The methodology involves the construction of a reliable estimator for the coefficient function, accompanied by a thorough analysis of the asymptotic mean squared error and the convergence velocity of the estimator. This method not only advances the precision of coefficient function estimation but also contributes insights into the statistical properties of the estimator under consideration. ^[3] investigated the generalized varying coefficient model, focusing on both estimation and hypothesis testing. Their methodological approach centered on crafting a robust estimation for the coefficient function, leveraging the local polynomial regression technique to enhance accuracy. ^[9] proposed a semiparametric estimator for the heteroscedastic single-index varying coefficient model. The estimator is proved to attain the semiparametric efficiency bound. ^[17] considered an estimating equations approach to parameter estimation in an adaptive varying coefficient linear quantile model. They proposed estimating equations for the index vector of the model in which the unknown nonparametric functions were estimated by minimizing the check loss function, resulting in a profiled approach.

Our goal here is to study the more efficient estimates of the one-dimensional varying coefficient model with unlabeled data as a pioneer in the semi-supervised varying coefficient modeling problems. The intercept model is introduced into the varying coefficient models, and the extra information from the unlabeled data is combined to improve the performance of the estimators.

The rest of the article is organized as follows. Section 2 provides the basic setting and the proposed methods. Theoretical properties are also given in Section 2. In Section 3, we conduct Monte Carlo simulation studies to examine the finite sample performance of the proposed procedure. We also use the proposed procedure in an example with real data.

2.   Materials and methods

2.1.   Varying coefficient model

In the study of the demand for shared bicycles ($Y$), many studies focus on the impact of the temperature ($X$), and how the relationships between them may change with time ($T$). Thus the varying coefficient model is a good choice. Nevertheless, in real-world scenarios, it is common that the labeled variable $Y$ may not be entirely observed, and only observations of the covariate $(X, T)$ are available. In such cases, the introduction of semi-supervised learning becomes imperative. In semi-supervised learning, the data structure involves a set of $n$ observations sampled from the joint distribution $(Y, X, T)$, and an additional $m$ samples are observed with only $(X, T)$ given. The model leverages both labeled and unlabeled data, allowing us to make more informed and robust estimates by incorporating the partially observed labeled samples along with the unlabeled data. This approach is effective when dealing with practical problems where complete observations of the labeled variable $Y$ may be limited.

To study the estimates of the varying coefficient model with unlabeled data, we introduce our basic method and consider the following model,

where $Y$ is the label variable, ${X}$ is the one-dimensional covariant, $T$ is the dependent variable, $g(\cdot)$ is the unknown measurable function on $R$ and $\varepsilon$ is the random error with $E(\varepsilon|T, {X}) = 0$.

2.2.   Locally weighted linear regression estimates

In the supervised situation, where the data from model (2.1) are completely observed, the coefficient function $g(t)$ of model (2.1) can be estimated by the locally weighted linear regression estimation method ^[5]. For any given point $t_0$, use a linear function in a neighborhood of $t_0$,

We can minimize the following objective functions by $a$ and $b$ to obtain the estimator of $g(t_0)$,

where $g(t_0) = a$, $hg'(t_0) = b$ and $K(\cdot)$ is some kernel function $K_h(\cdot) = K(\cdot/h)/h$ with bandwidth $h$. Let $Z_i = \left(X_i, X_i(\frac{T_i-t_0}{h}) \right)$ and $\beta(t_0) = \left(a, b \right)^\top = \left(g(t_0), hg'(t_0) \right)^\top$. Thus, $\hat{\beta}(t_0) = \left(\hat{a}, \hat{b} \right)^\top = \left(\hat{g}(t_0), h\hat{g}'(t_0) \right)^\top$ is

where $D$ is the $n\times2$ matrix and its $i$th line elements are $\left(X_i, X_i\frac{T_i-t_0}{h} \right)$, $Y = \left(Y_1, Y_2, ..., Y_n \right)^\top$, $K_w = diag \left(K_h(T_1-t_0), K_h(T_2-t_0), ..., K_h(T_n-t_0) \right)$. Thus, the locally weighted linear regression estimate for the coefficient function $g(t_0)$ of model (2.1) is $\hat{g}_L(t_0) = e^\top \hat{\beta}(t_0)$, where $e = (1, 0)^{\top}$.

2.3.   Intercept model with total information

To take full advantage of the information from the unlabeled data, we want to discuss the form of estimation in the semi-supervised case. First, we assume that the total information is completely known.

Motivated by ^[1], for model (2.1), we have

So, we get the following model through multiplying model (2.1) by $\frac{X}{E(X^2|T)}$,

where $\tilde{Y} = \frac{Y{X}}{E\left({X}^2 \mid T\right)}$, $\tilde{g}(T) = \left(g(T), a(T) \right)$, $\tilde{{\bf{X}}}_1 = \left(\tilde{X}_{01}, \tilde{X}_{02} \right)^\top$, $\tilde{X}_{01} = 1$, $\tilde{X}_{02} = \frac{{X}^2}{E\left({X}^2 \mid T\right)}-1$ and $\tilde{\varepsilon}$ is the remainder term. Under the total information situation, $E({X}^2|T)$ is known. The multiplication term makes the expectation of $\tilde{Y}$ to be $E(\tilde{Y}|T) = g(T)$. Therefore, by introducing this multiplication term into our estimation process, we can leverage the information from the unlabeled data set more effectively.

After introducing the intercept model, we have the locally weighted regression estimator of $\beta(t_0)$,

where the $i$th line elements of $\tilde{D}$ are $(\tilde{X}_i^\top, \tilde{X}_i^\top(\frac{T_i-t_0}{h}))$ and $\tilde{X_i} = (\tilde{X}_{01i}, \tilde{X}_{02i})^\top$, $\tilde{X}_{01i} = 1$, $\tilde{X}_{02i} = \frac{X_i^2}{E(X^2|T_i)}-1$, $i = 1, 2, \ldots, n$, $\hat{\tilde{\beta}}(t_0) = \left(\begin{array}{c} \hat{\tilde{g}}(t_0) \\ h\hat{\tilde{g}}'(t_0) \end{array} \right)$. Thus, the total information estimate of $\tilde{g}(t_0)$ is $\hat{g}_{TI}(t_0) = \tilde{e}^\top \hat{\tilde{\beta}}(t_0)$, where $\tilde{e} = (1, 0, 0, 0)^{\top}$.

2.4.   Intercept model with partial information

In practical problems, the overall information is not completely known, so we can only extract partial information through the observed unlabeled data. We consider the estimation under partial information (PI) i.e., the semi-supervised setting.

For semi-supervised data, consider $n$ independent and identically distributed observations $(X_1, T_1, Y_1), (X_2, T_2, Y_2), \ldots, (X_n, T_n, Y_n)$ in a joint distribution $G$, and an additional set $(X_{n+1}, T_{n+1}), (X_{n+2}, T_{n+2}), \ldots, (X_{n+m}, T_{n+m})$ of $m$ independent observations from the marginal distribution. Model (2.5) becomes

where $\check{Y} = \frac{Y{X}}{\check{E}\left(X^2 \mid T\right)}$, $\check{g}(T) = \left(g(T), a(T) \right)$, $\check{{\bf{X}}}_2 = \left(\check{X}_{01}, \check{X}_{02} \right)^\top$, $\check{X}_{02} = \frac{{X}^2}{\check{E}\left({X}^2 \mid T\right)}-1$, $\check{X}_{01} = {1}$ and $\check{\varepsilon}$ is the remainder term. $\check{E}({X}^2|T)$ is the estimated conditional expectation under $T$ for $X^2$ based on the semi-supervised data, $\check{E}(X^2|T) = \frac{\sum_{i = 1}^{n+m}X^2_iK_h(T_i-T)}{\sum_{i = 1}^{n+m}K_h(T_i-T)}$. Therefore, the semi-supervised estimator of $\beta(t)$ is

where the $i$th line elements of $\check{D}$ are $(\check{X}_i^\top, \check{X}_i^\top(\frac{T_i-t_0}{h}))$ and $\check{X}_i = (\check{X}_{01i}, \check{X}_{02i})^\top$, $\check{X}_{01i} = 1$, $\check{X}_{02i} = \frac{X_i^2}{\check{E}(X^2|T_i)}-1$, $i = 1, 2, \ldots, n$, $\hat{\check{\beta}}(t_0) = \left(\begin{array}{c} \hat{\check{g}}(t_0) \\ h\hat{\check{g}}'(t_0) \end{array} \right)$. The semi-supervised estimator of the coefficient function $\check{g}(t_0)$ of the model (2.7) is $\hat{g}_{PI}(t_0) = \check{e}^\top \hat{\check{\beta}}(t_0)$, where $\check{e} = (1, 0, 0, 0)^{\top}$.

2.5.   Theoretical properties

In this subsection, the following conditions are listed to study the theoretical properties of the proposed estimating procedures.

(1) The density function $f(t)$ of $T$ is continuous over the interval [0, 1] and takes values greater than 0.

(2) The varying coefficient function $g(t)$ and $E(Y|T = t)$ have continuous derivatives up to order 2, and are bounded away from zero.

(3) The kernel function $K_h(\cdot)$ is a bounded symmetric density function satisfying Lipschitz continuity in the interval (-1, 1) with a window width of $0 < h$ and $h = O(n^{-\frac{1}{5}})$.

(4) $\Gamma(t) = E({X}^2|T = t)$ is non-singular on the interval [0, 1] and has a continuous second derivative.

(5) $E(\varepsilon^2|T = t, X = x)$ has bounded partial derivatives up to order 2 and is bounded away from zero.

(6) There exists $s > 2$ that makes $E(|{X}|^{2s}) < \infty$ and $E(|Y|^{2s}) < \infty$.

Remark 1. Conditions (1)–(4) are general assumptions for varying coefficient models, which can be found in Tang and Zhou ^[12].

Theorem 1. If conditions (1)–(6) hold, and $\varepsilon({X}-E({X}|T))$ has a finite second order moment, then,

where $\sigma_{L}^2 = \nu_0f^{-1}(t_0)[\Gamma^{-1}(t_0)]^2E(X^2\varepsilon^2|T = t_0)$, $\mu_2 = \int t^2K(t)dt$, $\nu_0 = \int K^2(t)dt$.

Proof. First, we consider the varying coefficient model (2.1), known by the properties of the varying coefficient model,

where $D$ is the $n\times2$ matrix and its line $i$ elements are $\left(X_i, X_i\frac{T_i-t_0}{h} \right)$, $\Gamma(t_0) = E({X}^2|T_i = t_0)$, $f(t)$ is the density function of $T$ and $\mu_2 = \int t^2K(t)dt$. Since

then we have,

Define the above as $Y = D\beta(t_0)+\Delta_gX+\varepsilon$, where $Y = (Y_1, Y_2, \ldots, Y_n)^\top$, $\Delta_g = diag\left((g(T_1)-g(t_0)-g'(t_0)(T_1-t_0)), \ldots, (g(T_n)-g(t_0)-g'(t_0)(T_n-t_0))\right)$ and $X = (X_1, X_2, \ldots, X_n)^\top$. We have

Therefore,

Let $\frac{1}{n}D^{\top} K_w\Delta_gX = A_1^{(1)}$, $\frac{1}{n}D^\top K_w\varepsilon = A_2^{(2)}$, $A_1 = e_1^\top A_1^{(1)}$, $A_2 = e_1^\top A_2^{(2)}$, where $e_1$ is a unit vector with elements $(1, 0)^{\top}$. For $A_1$,

Since $\hat{\beta}(t_0)-\beta(t_0) = \left(\begin{array}{c} \hat{g}(t_0) \\ h\hat{g}'(t_0) \end{array} \right)-\left(\begin{array}{c} g(t_0) \\ hg'(t_0) \end{array} \right)$, we have

It should be noted that $\frac{1}{n}\sum_{i = 1}^{n}X_iK_h(T_i-t_0)\varepsilon_i\sim N (0, \frac{1}{nh}E(X_i^2\varepsilon_i^2|t_0)f(t_0)\nu_0)$, where $\nu_0 = \int K^2(t)dt$. Thus,

and it follows that

□

Theorem 2. Assuming that $\left(\tilde{Y}, \tilde{X}_{01}, \tilde{X}_{02} \right)$ has a finite second-order moment, the total information estimator of the intercept model has

where $\sigma_{TI}^2 = \nu_0f^{-1}(t_0)E(\tilde{\varepsilon}^2|t_0)$, $\sigma_{L}^2 = \sigma_{TI}^2+\sigma_{diff1}^2$, $\sigma_{diff1}^2 = nhE\{\frac{1}{f(t_0)}\frac{1}{n}\sum_{i = 1}^{n}K_h(T_i-t_0)(\frac{\varepsilon_i X_i}{E({X}^2|T_i)}-\tilde{\varepsilon}_i)\}^2$.

Proof. Before discussing the form of estimation in the semi-supervised case, we assume that all information is completely known and consider the estimation with complete data information. Consider the intercept model (2.5) under the complete data. We have the objective function

where, $\tilde{X_i} = (\tilde{X}_{01i}, \tilde{X}_{02i})^\top$.

Let the line $i$ elements of $\tilde{D}$ be $(\tilde{X}_i^\top, \tilde{X}_i^\top(\frac{T_i-t_0}{h}))$, $\tilde{Y} = (\tilde{Y_1}, \tilde{Y_2}, \cdots, \tilde{Y_n})^\top$, $\hat{\tilde{\beta}} = \left(\begin{array}{c} \hat{\tilde{g}}(t_0) \\ h\hat{\tilde{g}}'(t_0) \end{array} \right)$, so locally weighted linear regression by varying coefficients shows that $\hat{\tilde{\beta}}(t_0) = (\tilde{D}^\top K_w\tilde{D})^{-1}(\tilde{D}^\top K_w\tilde{Y})$, $\hat{\tilde{g}}(t_0) = e_1^\top\hat{\tilde{\beta}}(t_0)$, where the first element of $e_1$ is 1 and the others are 0.

In addition, we have

where $\tilde{\Gamma}(t_0) = E(\tilde{{\bf{X}}}\tilde{{\bf{X}}}^{\top}|t_0)$ and $\tilde{{\bf{X}}} = \left(\begin{array}{c} \tilde{X}_1^\top \\ \vdots \\ \tilde{X}_n^\top \end{array}\right)$.

Consider $\frac{1}{n}\tilde{D}^\top K_w\tilde{Y} = \left(\begin{array}{c} \frac{1}{n}\tilde{{\bf{X}}}^\top K_w\tilde{Y} \\ \frac{1}{n}\tilde{{\bf{X}}}^\top T_h^\top K_w\tilde{Y} \end{array} \right)$ and

We can define

where $\Delta_{ga} = diag\left(\left(\begin{array}{c} g(T_1)-g(t_0)-g'(t_0)(T_1-t_0) \\ a(T_1)-a(t_0)-a'(t_0)(T_1-t_0) \end{array} \right), \ldots, \left(\begin{array}{c} g(T_n)-g(t_0)-g'(t_0)(T_n-t_0) \\ a(T_n)-a(t_0)-a'(t_0)(T_n-t_0) \end{array} \right)\right)$. Therefore,

Note that

We have the following form,

and

Condisering $\frac{1}{n}\tilde{D}^\top K_w\tilde{{\bf{X}}}\Delta_{ga}$ and $\frac{1}{n}\tilde{D}^\top K_w\tilde{\varepsilon}$, let $\frac{1}{n}\tilde{D}^\top K_w\tilde{{\bf{X}}}\Delta_{ga} = \tilde{A}_1^{(1)}$, $\frac{1}{n}\tilde{D}^\top K_w\tilde{\varepsilon} = \tilde{A}_2^{(2)}$, $\tilde{A}_1 = e_1^\top\tilde{A}_1^{(1)}$, $\tilde{A}_2 = e_1^\top\tilde{A}_2^{(2)}$. We have

and

Therefore,

It is easy to obtain $\tilde{A}_1 = \tilde{A}_{11}+\tilde{A}_{12} = \frac{1}{2}g''(t_0)h^2\mu_2f(t_0)+o(h^4)$. On the other hand,

Since $\hat{\tilde{\beta}}-\tilde{\beta} = (\frac{1}{n}\tilde{D}^\top K_w\tilde{D})^{-1}(\frac{1}{n}\tilde{D}^\top K_w\tilde{{\bf{X}}}\Delta_{ga}+\frac{1}{n}\tilde{D}^\top K_w\tilde{\varepsilon})$, so we have

Since

letting $\hat{\tilde{g}}(t_0) = \hat{g}_{TI}(t_0)$, we can get

where $\sigma^2_{TI} = v_0f^{-1}(t_0)E(\tilde{\varepsilon}^2|t_0)$.

We have

Note that

we have

We can determine that

That is, $\sigma_L^2\geq\sigma_{TI}^2$, $\sigma_{L}^2 = \sigma_{TI}^2+\sigma_{diff1}^2$, where

□

Theorem 3. Considering the partial information estimator of the intercept model in a semi-supervised setting, when $lim\frac{n}{n+m} = v$, we have

where $\sigma_{PI}^2 = \nu_0f^{-1}(t_0)E(\check{\varepsilon}^2|t_0)$, $\sigma_{PI}^2 = \sigma_{TI}^2+v\sigma_{diff1}^2$.

Proof. Considering that in a practical problem the overall information is not completely known, so we can only extract partial information from the observed unlabeled data, we next consider the estimation under partial information (PI). We study the intercept model (2.7) under the semi-supervised setting.

In the same way as (ii), the local linear regression by varying coefficients shows that $\hat{\check{\beta}}(t_0) = (\check{D}^\top K_w\check{D})^{-1}(\check{D}^\top K_w\check{Y})$, $\hat{\check{g}}(t_0) = e_1^\top\hat{\check{\beta}}(t_0)$. Where the line $i$ elements of $\check{D}$ are $(\check{X}_i^\top, \check{X}_i^\top(\frac{T_i-t_0}{h}))$ and $\check{X}_i = (\check{X}_{01i}, \check{X}_{02i})^\top$.

First, let us say

and let $n+m = N$. When $T = t$, we say $m(t) = E(X^2|t)$.

From the properties of nonparametric kernel estimation, we can get

and from that, we can get

So we have

and we let $O_{p}((\frac{\ln N}{Nh})^{\frac{1}{2}}+h^2) = \bigtriangleup _{h_N}$. It is easy to get that

From the above description, we have

We have the following form:

Similar to the previous steps,

Let $\check{{\bf{X}}} = \left(\begin{array}{c} \check{X}_1^\top \\ \vdots \\ \check{X}_n^\top \end{array}\right)$, and considering $\frac{1}{n}\check{D}^\top K_w\check{{\bf{X}}}\Delta_{ga}$ and $\frac{1}{n}\check{D}^\top K_w\check{\varepsilon}$, let $\frac{1}{n}\check{D}^\top K_w\check{{\bf{X}}}\Delta_{ga} = \check{A}_1^{(1)}$, $\frac{1}{n}\check{D}^\top K_w\check{\varepsilon} = \check{A}_2^{(2)}$, $\check{A}_1 = e_1^\top\check{A}_1^{(1)}$, $\check{A}_2 = e_1^\top\check{A}_2^{(2)}$, where the first element of $e_1$ is 1 and the others are 0. We have

Therefore,

where

and

Similarly for $\tilde{A}_1$, $\check{A}_1 = \check{A}_{11}+\check{A}_{12} = \frac{1}{2}g''(t_0)h^2\mu_2f(t_0)+o(h^4)+\bigtriangleup_{h_N}$. On the other hand,

Note that $\hat{\check{\beta}}-\check{\beta} = (\frac{1}{n}\check{D}^\top K_w\check{D})^{-1}(\frac{1}{n}\check{D}^\top K_w\check{{\bf{X}}}\Delta_{ga}+\frac{1}{n}\check{D}^\top K_w\check{\varepsilon})$, so we have

Since

letting $\hat{\check{g}}(t_0) = \hat{g}_{PI}(t_0)$, we can get

where $\sigma^2_{PI} = v_0f^{-1}(t_0)E(\check{\varepsilon}^2|t_0)$. We have

where

Let $nhE(\frac{1}{f(t_0)}\frac{1}{n}\sum_{i = 1}^{n}K_h(T_i-t_0)(\check{\varepsilon}_i-\tilde{\varepsilon}_i))^2 = \sigma^2_{diff2}$, so we have

Since

and

we can get

where $\bigtriangleup_{h_{N_n}} = \bigtriangleup_{h_N}+o(\frac{1}{nh})$, and

where

and

So, we can get, for any given $t_0$

Suppose $lim\frac{n}{n+m} = v$, we can get $\sigma^2_{diff2} = v\sigma^2_{diff1}$, then we have $\sigma_{PI}^2 = \sigma_{TI}^2+v\sigma_{diff1}^2$. □

Remark 2. Theorems 1–3 show that if $\sigma_{diff1}^2$ is not equal to zero, then $\sigma_{L}^2$ is larger than $\sigma_{TI}^2$, and, further, if $v < 1$, then $\sigma_{L}^2$ is also larger than $\sigma_{PI}^2$. This means that the proposed semi-supervised estimators could be more efficient than the supervised estimators.

3.   Simulation studies and results

To study the finite sample performance of the proposed semi-supervised estimates, this section uses some numerical simulations and a real data analysis to compare the estimated performances of the locally weighted linear estimation of the varying coefficient model (LWLR), the total information estimator based on the intercept model (TI) and the semi-supervised estimator under the partial information setting (PI).

3.1.   Monte Carlo simulations

Example. Consider

where ${X}\sim N(0, 1)$, $T$ comes from the uniform distribution $U(0, 1)$ and random error $\delta \sim N(0, 1)$. At the same time, $\varepsilon1$ and $\varepsilon2$ are affected by $X$ and $T$. The labeled samples $(Y_i, X_i, T_i)$, $i = 1, 2, ..., n$ and the unlabeled samples $(X_i, T_i)$, $i = n+1, n+2, ..., n+m$ are from Model (Ⅰ) or (Ⅱ) with sample size $n$ and $m$, respectively. Let the kernel function be the Epanechnikov kernel function, $K(u) = 0.75(1-u^2)I(|u|\leq1)$. We determine the bandwidth $h$ of the kernel function through the process of cross-validation (CV). By employing CV, we aim to identify an optimal bandwidth value that enhances the performance of the kernel function in the given context. For numerical simulation, the labeled sample size is $n$ = $10, 20, 40, 60$ and the unlabeled sample size is $m = n, 3n, 10n$ for each model. To compare the pros and cons of the estimators obtained by different methods, the estimation curves are obtained by 1000 repeated simulations, and the estimation effect is evaluated by the root mean squared error (RMSE) and standard deviation (STD) of the estimations, where $RMSE = \sqrt{\frac{1}{q}\sum_{j = 1}^{q}\frac{1}{n}\sum_{k = 1}^{n}[\hat{g}_j(t_k)-g(t_k)]^2}$, and $STD = \sqrt{\frac{1}{n}\sum_{k = 1}^{n}\frac{1}{q}\sum_{j = 1}^{q}[\hat{g}_j(t_k)-\frac{1}{q}\sum_{j = 1}^{q}\hat{g}_j(t_k)]^2}$. The expression "$\hat{g}_j(t_k)$" represents the $j$th estimation of the coefficient function at the discrete time point $t_k$, where $\{t_k, k = 1, 2, \ldots, n\}$ refers to the appropriate grid point. The simulation results are shown in Table 1.

As can be seen from Table 1, the proposed total information estimator and the semi-supervised partial information estimator based on the intercept model are generally superior to the ordinary locally weighted linear regression estimator, and the RMSE and STD obtained by the former two methods are generally smaller than the latter one. The proposed semi-supervised intercept model can improve the efficiency of the estimator.

3.2.   Real data application

In recent years, the introduction of shared bicycles has enriched the travel and transportation options of urban residents and solved the "last kilometer" travel problem of residents. Shared cycling has a significant impact on building a larger riding community, minimizing greenhouse gas emissions and improving public health and transport issues. But, shared bicycle travel is easily affected by the weather environment, where the temperature is an important disturbance factor. We study the data set of the number of shared bicycles rented hourly in Seoul from 2017.12–2018.11 ^[11]. There are a total of N = 8760 samples in the data set. To study the superiority of the estimation method proposed in this paper, the variables selected include count of bikes rented at each hour (vehicles, $Y$), hour of the day ($0:00-23:00$, $T$) and temperature ($^\circ$C, $X$). At the same time, n = 1000 labeled samples and m = 7760 unlabeled samples are randomly selected in the original data set to construct a semi-supervised setting, and the semi-supervised intercept model method proposed in this paper is used for analysis. We establish the following model:

Compare the estimated $R^2$ obtained by semi-supervised intercept model estimation $R^2_{PI}$, the local linear regression estimation $R^2_{LWLR_N}$ under all supervised data and $R^2_{LWLR_n}$ under $n$ supervised samples. At the same time, in order to eliminate the influence of randomness, we repeated $K = 50$ times and calculated the mean of the corresponding $R^2(k)$ respectively, that is, $R^2 = \frac{1}{K}\sum_{k = 1}^K R^2(k)$. The goodness of fit based on $N$ samples is $R^2_{LWLR_N} = 0.8531$, the goodness of fit based on n samples is $R^2_{LWLR_n}$ = $0.7651$ and $R^2_{PI} = 0.8411$, the effect is significantly better than that of the local linear regression estimation without using unlabeled data.

4.   Conclusions

Semi-supervised data is becoming more common, and most semi-supervised learning methods focus on classification tasks, or solving linear regression models, with less emphasis on varying coefficient models. Therefore, a good estimate of the coefficient function of the varying coefficient model is given in this work. The key idea is to introduce an intercept model to replace the original varying coefficient regression model and perform the estimation under a semi-supervised setting; that is, the information of unlabeled data is utilized in the estimation process. It is further proved that the new estimates have good asymptotic properties. At the same time, the asymptotic properties of the new estimate is better than that of the conventional locally weighted linear regression estimators.

Finally, the method is applied to study the effect of temperature on the demand for shared bicycle rental. The coefficient function of the shared bicycle rental demand model is well estimated, and the demand is predicted.

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in the creation of this article.

Acknowledgments

Peng Lai's research is supported by National Natural Science Foundation of China (11771215). Yanqiu Zhou's research is supported by Guangxi Science and Technology Base and Talent Project (2020ACI9151), and Guangxi University Young and Middle-aged Teachers' Basic Research Ability Improvement Project (2021KY0343).

Conflict of interest

The authors declare that they have no conflicts of interest.