Weak convergence of the conditional single index $U$-statistics for locally stationary functional time series

1.   Introduction and motivations

In the evolutionary trajectory of asymptotic outcomes related to $U$-statistics, with a particular emphasis on independent and identically distributed random variables, pivotal contributions can be ascribed to esteemed figures like ^[93,99,181], among others. When extending these advancements to accommodate scenarios involving weak dependency assumptions, notable references include ^{[30,39,40,66,119,120]}. For a comprehensive grasp of $U$-statistics and $U$-processes, scholars are directed to seminal works such as ^{[11,12,14,31,113,117]}. A substantial leap forward in the theoretical landscape of $U$-processes is accredited to ^[64], who made significant contributions by assimilating insights from empirical process theory. Their introduction of innovative techniques, including decoupling inequality and randomization, played a pivotal role in propelling the theoretical framework forward. The applications of $U$-processes traverse diverse statistical domains, encompassing testing for qualitative features of functions in nonparametric statistics ^[1,88,118], cross-validation for density estimation ^[144], and establishing limiting distributions of M-estimators ^{[11,64,157,158]}. In the realm of machine learning, $U$-statistics find multifaceted applications in clustering, image recognition, ranking, and learning on graphs. The natural estimates of risk prevalent in various machine learning contexts often manifest in the form of $U$-statistics, as elucidated in ^[54]. Instances of $U$-statistics are also discerned in various contexts, such as empirical performance measures in metric learning, exemplified by ^[50]. When confronted with $U$-statistics characterized by random kernels exhibiting diverging orders, pertinent literature includes contributions from ^{[85,98,152,162]}. Infinite-order $U$-statistics manifest as invaluable tools for constructing simultaneous prediction intervals, providing insights into the uncertainty inherent in ensemble methods like subbagging and random forests, as explicated in ^[146]. The MeanNN approach, introduced by ^[75] for estimating differential entropy, intricately involves the utilization of the $U$-statistic. Additionally, ^[129] proposes a novel test statistic for goodness-of-fit tests, employing $U$-statistics. A model-free approach to clustering and classifying genetic data based on $U$-statistics is explored by ^[55], presenting alternative perspectives driven by the adaptability of $U$-statistics to a diverse array of genetic issues and their capability to accommodate various data types. Furthermore, ^[125] advocates for the natural application of $U$-statistics in examining random compressed sensing matrices in the non-asymptotic regime. For the latest references in this context, please consult ^[43,163,164]. In the realm of nonparametric density and regression function estimation, ^[166] introduces a class of estimators for \(r^{(m)}(\varphi, \mathbf{ t}) \), referred to as conditional \(U\)-statistics. These estimators can be perceived as an extension of the Nadaraya-Watson estimates for regression functions, initially proposed by ^[139,184]. The nonparametric domain of density and regression function estimation has been a focal point for statisticians and probabilists over numerous years, resulting in the evolution of various methodologies. Kernel nonparametric function estimation methods, in particular, have garnered substantial attention. For a thorough exploration of the research literature and statistical applications in this field, one is encouraged to consult ^{[67,74,95,140,160,182]}, and the pertinent references therein.

This investigation delves into the intricacies of nonparametric conditional $U$-statistics. To facilitate our exploration, we commence by introducing the estimators proposed by ^[166]. Consider a regular sequence of random elements $\{(\mathbf{X}_i, Y_i): i\in \mathbb{N}^*\}$, where $\mathbf{X}_i \in \mathbb{R}^d$ and $Y_i \in \mathcal{Y}$, a Polish space, with $\mathbb{N}^* = \mathbb{N} \backslash\{0\}$. Let $\varphi: \mathcal{Y}^m\rightarrow \mathbb{R}$ be a measurable function. In this paper, our central focus revolves around the estimation of the conditional expectation or regression function:

for $\mathbf{ t} \in \mathbb{R}^{dm}$, provided it exists, namely, when $\mathbb{E}\left(\left\lvert\varphi(Y_{1}, \ldots, Y_{m})\right\rvert\right) < \infty$. We introduce a kernel function $K:\mathbb{R}^d\rightarrow \mathbb{R}$ with support contained in $[-B, B]^d$, where $B > 0,$ adhering to the following conditions:

Reference ^[166] introduced a class of estimators for $r^{(m)}(\varphi, \mathbf{ t})$, known as conditional $U$-statistics, defined for each $\mathbf{ t}\in\mathbb{R}^{dm}$ as:

where $I_n^m$ is the set of all $m$-tuples of different integers between $1$ and $n$:

and $\{h_{n}\}_{n\geq1}$ is a sequence of positive constants converging to zero at the rate $nh_{n}^{dm} \rightarrow \infty$. In the specific scenario of $m = 1$, where $r^{(m)}(\varphi, t)$ simplifies to $r^{(1)}(\varphi, t) = \mathbb E(\varphi(\mathbf{ Y})\mid \mathbf{ X} = t)$, the estimator by Stute transforms into the Nadaraya-Watson estimator of $r^{(1)}(\varphi, t)$. The study conducted by ^[154] focused on estimating the rate of uniform convergence in $\mathbf{ t}$ of $\widehat{r}_n^{(m)}(\varphi, \mathbf{t}; h_n)$ to $r^{(m)}(\varphi, \mathbf{ t})$. In ^[148], the paper discusses and compares the limit distributions of $\widehat{r}_n^{(m)}(\varphi, \mathbf{ t}; h_n)$ with those obtained by Stute. ^[97] extended the results of ^[166] to weakly dependent data under appropriate mixing conditions (also see ^[17]). They applied these findings to verify the Bayes risk consistency of corresponding discrimination rules similar to ^[167] and Section 5.1. In ^[169], symmetrized nearest neighbor conditional $U$-statistics are proposed as alternatives to the usual kernel-type estimators, and reference can also be made to ^[48]. ^[86] explored the functional conditional $U$-statistic and established its finite-dimensional asymptotic normality. Despite the subject's importance, nonparametric estimation of conditional $U$-statistics in a functional data framework has received relatively limited attention. Recent advancements are presented in ^[48], addressing problems related to uniform bandwidth consistency in a general setting. In ^[104], the test of independence in the functional framework based on the Kendall statistics was investigated, which can be considered as particular cases of $U$-statistics. Extending this exploration to conditional empirical $U$-processes in the functional setting is practically useful and technically more challenging. Two perspectives on conditional $U$-processes are presented 1) they are infinite-dimensional versions of conditional $U$-statistics (with one kernel) and 2) they are stochastic processes that are nonlinear generalizations of conditional empirical processes. Both views are valuable because: 1) from a statistical standpoint, considering a rich class of statistics is more interesting than a single statistic; 2) mathematically, insights from empirical process theory can be applied to derive limit or approximation theorems for $U$-processes. Importantly, extending $U$-statistics to $U$-processes demands substantial effort and different techniques, and generalization from conditional empirical processes to conditional $U$-processes is highly nontrivial.

The prevalent practice of assuming stationarity in time series modeling has prompted the development of various models, techniques, research, and methodologies. However, this assumption may not always be suitable for spatio-temporal data, even with detrending and deseasonalization. Many pivotal time series models exhibit nonstationarity, observed in diverse physical phenomena and economic data, rendering classical methods ineffective. To address this challenge, the concept of the locally stationary random process was introduced by ^[161]. This type of process approximates a non-stationary process by a stationary one locally over short periods. The intuitive concept of local stationarity is also explored in the works of ^{[57,58,142,149,153]}, among others. The groundbreaking work of ^[57] notably serves as a robust foundation for the inference of locally stationary processes. In addition to generalizing stationary processes, this innovative approach eliminates time-varying parameters. Over the past decade, the theory of empirical processes for locally stationary time series has garnered significant attention. Empirical processes theory plays a crucial role in addressing statistical problems and has expanded into time series analysis and regression estimation. Relevant references in this context include ^[59,179] and more recent contributions such as ^[134,147]. The extension of the previously discussed exploration to conditional empirical $U$-processes bears significant interest from both practical and theoretical standpoints. We specifically delve into the domain of conditional $U$-processes indexed by a class of functions within the framework of functional data. Building upon insights from ^[8], functional data analysis (FDA) emerges as a statistical field dedicated to analyzing infinite-dimensional variables such as curves, sets, and images. Experiencing remarkable growth over the past two decades, FDA has become a crucial area of investigation in data science, fueled by advancements in data collection technology during the "Big Data" revolution. For an introduction to FDA, readers can refer to the books by ^[78,151], providing fundamental analysis methods and case studies across various domains like criminology, economics, archaeology, and neurophysiology. Notably, the extension of probability theory to random variables taking values in normed spaces predates recent literature on functional data, with foundational knowledge available in ^[9,87]. In the context of regression estimation and nonparametric models for data in normed vector spaces, valuable references include ^[78,136], along with additional contributions from ^[32,102,126]. Modern empirical process theory has been applied to functional data, as demonstrated by ^[82], who established uniform consistency rates for functionals of the conditional distribution, including the regression function, conditional cumulative distribution, and conditional density. ^[109] extended this by providing consistency rates for various functional nonparametric models, uniformly in bandwidth (UIB consistency). Recent advancements in this field can be explored through references such as ^{[3,40,42,49,68,69,137]}. This strongly motivates the consideration of regression models that offer dimension reduction. Single index models are widely used to achieve this by assuming that the predictors' influence on the response can be simplified to a single index. This index represents a projection in a specified direction and is combined with a nonparametric link function, simplifying the predictors to a one-dimensional index while still incorporating important characteristics. Additionally, because the nonparametric link function only operates on a one-dimensional index, these models are not affected by the problem of having a high number of dimensions, known as the curse of dimensionality. The single index model extends the concept of linear regression by incorporating a link function equivalent to the identity function; for further details, interested readers can refer to ^{[23,91,123,138,171,183]}.

Recent progress in functional data analysis underscores the need for developing models to address the challenges of dimensionality reduction (refer to ^[90,126] for recent surveys, and also ^{[4,5,40,41,165]}). In response to this, semiparametric approaches emerge as promising solutions. The functional single-index model (FSIM) has gained attention in this context, with exploration by ^[2,16,79]. Furthermore, ^[106] proposed functional single-index composite quantile regression, estimating the unknown slope function and link function through $B$-spline basis functions. A functional single index model with coefficient functions restricted to a subregion was introduced by ^[143]. The estimation of a general functional single index model, where the conditional distribution depends on the functional predictor via a single index structure, was investigated by ^[188]. Innovatively, ^[173] developed a new estimation method that combines functional principal component analysis, $B$-spline modeling, and profile estimation for parameters and functions. Addressing the estimation of the functional single index regression model with missing responses for strong mixing time series data, ^[127,128] made valuable contributions. ^[76] introduced a functional single-index varying coefficient model with the functional predictor as the single-index part. Utilizing functional principal component analysis and basis function approximation, they obtained estimators for slope and coefficient functions, proposing an iterative estimating procedure. An automatic and location-adaptive procedure for estimating regression in an FSIM based on k-Nearest Neighbors ($k$NN) principles was presented by ^[145]. Motivated by imaging data analysis, ^[121] proposed a novel functional varying-coefficient single-index model for regression analysis of functional response data on a set of covariates. Investigating a functional Hilbertian regressor for nonparametric estimation of the conditional cumulative distribution with a scalar response variable in a single index structure, ^[15] made notable contributions. An alternative approach was introduced by ^[52], extending to the multi-index case without anchoring the true parameter on a prespecified sieve. Their detailed theoretical analysis of a direct kernel-based estimation scheme establishes a polynomial convergence rate.

The primary objective of this paper is to scrutinize a comprehensive framework for the single-index conditional $U$-process of any fixed order, indexed by a class of functions within a nonparametric context. Specifically, we explore the conditional $U$-process in the realm of functional covariates, considering the potential non-stationary nature of functional time series. The main aim of this study is to offer an initial and comprehensive theoretical examination within this specific context. To achieve this, we skillfully apply large sample theory techniques developed for empirical processes and $U$-empirical processes. This paper meticulously tackles various technical hurdles. Initially, it delves into the nonlinear expansion of the single index concept and conditional $U$-statistics. Subsequently, it addresses the extension of the Hoeffding decomposition to non-stationary time series. Finally, it confronts the complexity stemming from the unbounded function class, leading to extensive and intricate proofs.

The manuscript's organization is structured as follows. Section 2 provides a detailed exposition of our theoretical framework, elucidating essential definitions and contextual explanations while introducing technical assumptions. Our principal findings are presented in Sections 3 and 4. Specifically, Section 3 unveils convergence rate results, reintroducing the pivotal Hoeffding decomposition technique. Accommodating our outcomes on weak convergence, Section 4 delves into the details of these results. Section 5 accentuates selected applications. In Section 6, we explore bandwidth selection methodologies utilizing cross-validation procedures. Concluding reflections are encapsulated in Section 7. The comprehensive proofs are furnished in Section 8. Lastly, Appendix 13 provides technical properties and lemmas for easy reference.

2.   Background and preliminaries

2.1.   Notation

In this document, the notation $a_n \lesssim b_n$ is employed to signify the existence of a constant $C$, which is independent of $n$ and may vary between lines, unless explicitly specified otherwise. This constant is such that $a_n \leq C b_n$ holds for all $n$. Additionally, the notation $a_n \ll b_n$ indicates that $a_n / b_n \rightarrow 0$ as $n \rightarrow \infty$. When both $a_n \lesssim b_n$ and $b_n \lesssim a_n$ hold, their equivalence is denoted as $a_n \sim b_n$. Moreover, $(i_1, \ldots, i_m)$ is denoted as $\mathbf{i}$, and $(i_1/n, \ldots, i_m/n)$ as $\mathbf{i}/n$. For any $c, d \in \mathbb{R}$, the expressions $c \vee d = \max\{c, d\}$ and $c \land d = \min\{c, d\}$ are employed. The notation $\left \lfloor a \right \rfloor$ signifies the integer part of a number. Additionally, for $m < n$, where $m$ and $n$ are positive integers, $C_m^n = \frac{n!}{(n-m)! m!}$ is defined. The set $I_n^m$ is introduced as

comprising all $m$-tuples of distinct integers between $1$ and $n$.

2.2.   Model

Consider the stochastic processes $\{Y_{i, n}, X_{i, n}\}_{i = 1}^n$, where $Y_{i, n}$ takes values in a space $\mathcal{Y}$, and $X_{i, n}$ is in an abstract space $\mathcal{H}$. We assume $\mathcal{H}$ is a semi-metric vector space with a semi-metric $d(\cdot, \cdot)$^* which, in most applications, would be a Hilbert or Banach space. We consider the semi-metric $d_{\theta}(\cdot, \cdot)$ associated with the single-index $\theta \in\mathcal H$, defined as

^*A semi-metric (or pseudo-metric) $d(\cdot, \cdot)$ is a metric allowing $d_{\theta_1}(x_1, x_{2}) = 0$ for some $x_{1}\neq x_{2}$.

Consider any function $\varphi(\cdot)$ of $k$ variables (the $U$-kernel) such that $\varphi(Y_{1}, \ldots, Y_{m})$ is integrable. For $\mathbf{x} = (x_1, \ldots, x_m) \in \mathcal{H}^m$ and $\boldsymbol{\theta} = (\theta_1, \ldots, \theta_m)\in \Theta^m \subset \mathcal{H}^m$, define the regression functional parameter as

In this study, we consider the following model:

where $\left\{\varepsilon_{\mathbf i}\right\}_{\mathbf i \in I_n^m}$ is a sequence of univariate independent and identically distributed random variables, independent of $\left\{X_{i, n}\right\}_{i = 1}^{n}$. We denote $\sigma\left(\frac{\mathbf{i}}{n}, X_{\mathbf{i}, n}\right)\varepsilon_{\mathbf{i}}$ as $\varepsilon_{\mathbf{i}, n}$. Furthermore, we assume that the process is a locally stationary functional time series. In a heuristic sense, a process $\left\{X_{i, n}\right\}$ is considered locally stationary if it displays approximately stationary behavior locally in time. The regression function $r^{(m)}(\varphi, \cdot, \mathbf x, \boldsymbol{\theta})$ is allowed to change smoothly over time, depending on a rescaled quantity ${\bf i}/n$ rather than on the specific point ${\bf i}$ (where ${\bf i}$ typically represents time in a time series framework).

2.3.   Local stationarity

We delve into the exploration of non-stationary processes characterized by dynamics evolving gradually over time, manifesting behaviors akin to stationarity at a local level. This conceptual realm has undergone comprehensive scrutiny, exemplified by references such as ^{[57,105,115,131,141,185]}. For illustration, consider a continuous function $a:[0, 1] \rightarrow \mathbb{R}$ and a sequence of i.i.d. random variables $\left(\varepsilon_i\right)_{i \in \mathbb{N}}$. The stochastic process $X_{i, n} = a(i / n)+\varepsilon_i$, where $i \in\{1, \ldots, n\}$ and $n \in \mathbb{N}$, can exhibit "almost" stationary behavior for $i$ close to a specific point $i^*$ (e.g., $i^*$ in $\{1, \ldots, n\}$), given that $a\left(i^* / n\right) \approx a(i / n)$. However, this process is not strictly weakly stationary. To capture this type of gradual change, the concept of local stationarity was introduced by ^[57], wherein the spectral representation of the underlying stochastic process is locally approximated. In our framework, the process $\{X_{i, n}\}$ can be stochastically approximated by a stationary process $\{ X_{i, n}^{(u)}\}$ around each rescaled time point $u$, specifically for those values of $i$ where $i/n-u$ is small. Since our focus is on functional data, we define a functional time series as locally stationary if it can be locally approximated by a stationary functional time series. We will provide a standard definition of local stationarity.

Definition 2.1 (local stationarity). For a sequence of stochastic processes, indexed by $n \in \mathbb{N}$ and taking values in $\mathcal{H}$, denoted as $\{ X_{i, n}\}$, it is deemed locally stationary if, for all rescaled times $u\in [0, 1]$, there exists an associated $\mathcal{H}$-valued process $\{ X_{i}^{(u)}\}$ that is strictly stationary. This association is characterized by the inequality:

This holds for all $1 \leq i \leq n$, where $\{U_{i, n}^{(u)}\}$ is a positive valued process satisfying $\mathbb E[(U_{i, n}^{(u)})^{\rho}] < C$ for some $\rho > 0$, $C < \infty$. These conditions are independent of $u$, $i$, and $n$.

Definition 2.1 represents a natural extension of the concept of local stationarity for real-valued time series introduced in ^[57]. In a more specific context, ^[176] and ^[175] elaborate on this definition, considering $\mathcal{H}$ as a Hilbert space $L_\mathbb{R}^2[0, 1]$. Here, all real-valued functions are square-integrable with respect to the Lebesgue measure on the interval $[0, 1]$, equipped with the inner product $L_2$-norm:

where $f, g\in L_{\mathbb{R}}^{2}([0, 1])$. These authors also provide sufficient conditions for an $L_{\mathbb{R}}^{2}([0, 1])$-valued stochastic process $\{X_{i, n}\}$ to satisfy (2.3) with $d(f, g) = \|f-g\|_{2}$ and $\rho = 2$. Additionally, they define $L_E^p(T, \mu)$ as the Banach space of all strongly measurable functions $f: T \rightarrow E$ with finite norm:

for $1 \leqslant p < \infty$ and with finite norm

for $p = \infty$. In this context, $\mathcal H_{\mathbb{C}} = L_{\mathbb{C}}^2([0, 1])$.

Remark 2.2. ^[176] generalizes the definition of local stationary processes, initially proposed by ^[56], to the functional setting in the frequency domain. This extension is made under the following assumptions:

(A1) (i) $\left\{\varepsilon_i\right\}_{i\in \mathbb{Z}}$ is a weakly stationary white noise process taking values in $\mathcal H$ with a spectral representation given by

where $Z_\omega$ is a $2 \pi$-periodic orthogonal increment process taking values in $\mathcal H_{\mathbb{C}}$;

(ii) the functional process $X_{i, n}$ with $i = 1, \ldots, n$ and $n \in \mathbb{N}$ is given by

with the transfer operator $\mathcal{A}_{j, \omega}^{(n)} \in \mathcal{B}_p$ and an orthogonal increment process $Z_\omega$.

(A2) There exists $\mathcal{A}:[0, 1] \times[-\pi, \pi] \rightarrow S_p\left(H_{\mathbb{C}}\right)$ with $\mathcal{A}_{u, \cdot} \in \mathcal{B}_p$ and $\mathcal{A}_{u, \omega}$ being continuous in $u$ such that for all $n \in \mathbb{N}$

They have proved in [176, Proposition 2.2] that:

Proposition 2.3. Suppose that assumptions (A1) and (A2) hold. Then, $\left\{X_{i, n}\right\}$ is a locally stationary process in $\mathcal H$.

In our investigation of non-stationary processes characterized by gradually evolving dynamics over time, exhibiting behaviors reminiscent of stationarity at a localized scale, we draw upon an extensive body of research documented in notable references such as ^{[57,105,115,131,141,185]}. As an illustrative example, let $a:[0, 1] \rightarrow \mathbb{R}$ be a continuous function, and consider a sequence of independent and identically distributed random variables $\left(\varepsilon_i\right)_{i \in \mathbb{N}}$. The stochastic process $X_{i, n} = a(i / n)+\varepsilon_i$, where $i \in\{1, \ldots, n\}$ and $n \in \mathbb{N}$, may demonstrate "almost" stationary behavior for $i$ in proximity to a specific point $i^*$ (e.g., $i^*$ in $\{1, \ldots, n\}$), under the condition that $a\left(i^* / n\right) \approx a(i / n)$. However, it is important to note that this process does not strictly adhere to weak stationarity. To capture this gradual transition, the concept of local stationarity was introduced by ^[57], wherein the spectral representation of the underlying stochastic process is locally approximated. Within our framework, the process $\{X_{i, n}\}$ can be stochastically approximated by a stationary process $\{ X_{i, n}^{(u)}\}$ around each rescaled time point $u$, particularly for those values of $i$ where $i/n-u$ is small. Given our emphasis on functional data, we define a functional time series as locally stationary if it can be approximated locally by a stationary functional time series. The subsequent section will present a standard definition of local stationarity, building upon the foundation laid out in ^[176].

Theorem 2.4. Consider a white noise process $\left\{\varepsilon_i\right\}_{i \in \mathbb{Z}}$ in $L_H^2(\Omega, \mathbb{P})$ where $H = L^2([0, 1])$, and let $\left\{X_{i, n}\right\}$ be a sequence of functional autoregressive processes defined as

with $B_{u, j} = B_{0, j}, C_u = C_0$ for $u < 0$, and $B_{u, j} = B_{1, j}, C_u = C_1$ for $u > 1$. If the process satisfies, for all $u \in[0, 1]$ and $p = 2$ or $p = \infty$, the conditions

(i) $C_u$ is an invertible element of $S_{\infty}(H)$;

(ii) $B_{u, j} \in S_p(\mathcal H)$ for $j = 1, \ldots, m$ with $\sum_{j = 1}^m\left\|B_{u, j}\right\|_l < 1$ and $B_{u, 0} = I_H$;

(iii) the mappings $u \mapsto B_{u, j}$ for $j = 1, \ldots, m$ and $u \mapsto C_u$ are continuous in $u \in[0, 1]$ and differentiable on $u \in(0, 1)$ with bounded derivatives,

then the process $\left\{X_{i, n}\right\}$ satisfies (A2) with

and, consequently, is locally stationary.

2.4.   Small-ball probability

Handling infinite-dimensional spaces presents a notable technical hurdle due to the absence of a universal reference measure, such as the Lebesgue measure. Consequently, defining a density function for the functional variable becomes elusive. To surmount this challenge, we leverage the concept of "small-ball probability". In particular, we address the concentration of the probability measure for the functional variable within a small-ball using the function $\phi_x(\cdot)$. For a fixed $x\in \mathcal{H}$ and every $r > 0$, the function $\phi_{x, \theta}(r)$ is defined as:

Here, $\mathcal{H}$ is equipped with the semi-metric $d(\cdot, \cdot)$, and $B_\theta(x, r)$ represents a ball in $\mathcal{H}$ with center $x \in \mathcal{H}$ and radius $r$. For $\mathbf x = (x_1, \ldots, x_m) \in \mathcal H^m$ and $\boldsymbol{\theta} = (\theta_1, \ldots, \theta_m) \in \Theta^m$, we define

Further elucidation and examples concerning small-ball probability can be explored in Remark 3.6.

2.5.   Mixing conditions

Statistical observations commonly exhibit a certain degree of dependence rather than complete independence. The concept of mixing serves as a quantitative measure of the proximity of a sequence of random variables to independence, facilitating the extension of traditional results applicable to independent sequences to sequences that are weakly dependent or mixing. The development of the theory of mixing conditions has emerged from the recognition that time series manifest "asymptotic independence" properties, thereby facilitating their analysis and statistical inference. Consider a probability space $(\Omega, \mathcal{F}, \mathbb{P})$. Let ${Z}_{i, n}, {Z}_{2, n}, \ldots$ be a sequence of random variables on a probability space $\left(\Omega, \mathcal{D}, \mathbb{P}\right)$. For an array $\left\{Z_{i, n}: 1 \leq i \leq n\right\}$, the coefficients are defined as

where $\sigma(Z)$ represents the $\sigma$-field generated by $Z$. The array $\left\{Z_{i, n}\right\}$ is considered $\beta$-mixing if $\beta(k) \rightarrow 0$. It is crucial to note that $\beta$-mixing implies $\alpha$-mixing. Throughout the ensuing discussion, we assume that the sequence of random elements $\{(X_{i, n}, Y_{i, n}), i = 1, \ldots, n; n \geq 1\}$ is absolutely regular. Remarkably, Markov chains exhibit $\beta$-mixing under the milder Harris recurrence condition, provided the underlying space is finite ^[62]. Additional rationale for favoring regular processes over strongly mixing processes is provided in the concluding remarks (Section 7).

2.6.   Kernel estimation

We endeavor to estimate the regression function as denoted in (2.1). The kernel estimator is formally defined as

where $K_{1}(\cdot)$ and $K_{2}(\cdot)$ denote one-dimensional kernel functions. Here, let $h = h_{n}$ be a bandwidth with the property that $h \to 0$ as $n \to \infty$. The function $\varphi: \mathcal{Y}^m \longrightarrow \mathbb{R}$ is symmetric and measurable, belonging to a class of functions denoted as $\mathscr{F}_m$. Importantly, this estimator is a conditional $U$-statistic utilizing the sequence of random variables $\{Y_{i, n}, X_{i, n}\}_{i = 1}^n$ and the kernel $\varphi \times K_1\times K_2$. The introduction of such statistics was pioneered by ^[166]. To investigate the weak convergence of the conditional empirical process and the conditional $U$-process within the functional data framework, we introduce some necessary notations. Consider the class

which consists of real-valued symmetric measurable functions on $\mathcal{Y}^{m}$ with a measurable envelope function :

For kernel functions $K_1(\cdot)$ and $K_2(\cdot)$, as well as a subset $S_{\mathcal{H}}\subset \mathcal{H}$, we define the pointwise measurable class of functions for $1\leq m \leq n$ and $\boldsymbol{\theta} = (\theta_1, \ldots, \theta_m)$:

and

The conditional $U$-process indexed by $\mathscr{F}_{m}\mathfrak{K}^m_\Theta$ is defined by

Observing the importance of point-wise measurability in our context, it enables us to state our results conventionally, adhering to the classical definition of probability, without invoking the abstract notions of outer probability or outer expectation ^[178].

Remark 2.5. The bandwidth $h$ remains consistent across all directions, simplifying the analysis in cases involving product kernels. Nevertheless, the results can be readily adapted to scenarios with non-product kernels and varying bandwidths.

Remark 2.6. Our estimator differs from the conventional conditional $U$-statistics not only in the nature of the sequence $\{X_i\}_i$ but also in the inclusion of a kernel in the time direction. As a result, we attain smoothness from both the covariate direction $(X_{i, n})$ and the temporal dimension, allowing us to capture the characteristics of a regression model evolving over time.

2.7.   VC-type classes of functions

Examining functional data through asymptotic methods involves delving into concentration properties elucidated by the concept of small-ball probability. When scrutinizing a process characterized by a set of functions, it becomes imperative to consider additional topological concepts, including metric entropy and VC-subgraph classes (referred to as "VC", inspired by Vapnik and Cervonenkis).

Definition 2.7. Let $\mathcal{S}_{\mathcal{E}}$ be a subset of a semi-metric space $\mathcal{E}$. A finite set of points $\{e_1, \ldots, e_N\}\subset \mathcal{E}$ is considered a $\varepsilon$-net of $\mathcal{S}_{\mathcal{E}}$ for a given $\varepsilon > 0$ if:

If ${N_{\varepsilon}}(\mathcal{S}_{\mathcal{E}})$ is the cardinality of the smallest $\varepsilon$-net (i.e., the minimal number of open balls of radius $\varepsilon$) in $\mathcal{E}$ needed to cover $\mathcal{S}_{\mathscr{H}}$, then the Kolmogorov's entropy (metric entropy) of the set $\mathcal{S}_{\mathcal{E}}$ is defined as the quantity:

Kolmogorov introduced the concept of metric entropy, extensively explored in various metric spaces, as indicated by its name (cf. ^[112]). Dudley (^[70]) utilized this concept to establish sufficient conditions for the continuity of Gaussian processes, forming the foundation for significant generalizations of Donsker's theorem regarding the weak convergence of the empirical process. Consider two subsets, $\mathcal{B}_\mathcal{H}$ and $\mathcal{S}_{\mathcal{H}}$, in the semi-metric space $\mathcal{H}$ with Kolmogorov's entropy (for radius $\varepsilon$) denoted as $\psi_{\mathcal{B}_{\mathcal{H}}}(\varepsilon)$ and $\psi_{\mathcal{S}_{\mathcal{H}}}(\varepsilon)$, respectively. The Kolmogorov entropy for the subset $\mathcal{B}_\mathcal{H}\times \mathcal{S}_\mathcal{H}$ of the semi-metric space $\mathcal{H}^2$ is given by:

Thus, $m\psi_{\mathcal{S}_{\mathcal{H}}}(\varepsilon)$ represents the Kolmogorov entropy of the subset $\mathcal{S}_{\mathcal{H}}^m$ in the semi-metric space $\mathcal{H}^m$. If $d$ denotes the semi-metric on $\mathcal{H}$, a semi-metric on $\mathcal{H}^m$ can be defined as:

for $\mathbf{x} = (x_1, \ldots, x_m)$, $\mathbf{z} = (z_1, \ldots, z_m) \in\mathcal{H}^m$. The choice of the semi-metric is crucial in this type of analysis, and readers can find insightful discussions on this topic in [78, Chapters 3 and 11]. Furthermore, in this context, we must also address another topological concept: VC-subgraph classes.

Definition 2.8. A class of subsets $\mathcal{C}$ on a set $C$ is called a VC-class if there exists a polynomial $P(\cdot)$ such that, for every set of $N$ points in $C$, the class $\mathcal{C}$ picks out at most $P(N)$ distinct subsets.

Definition 2.9. A class of functions $\mathscr{F}$ is called a VC-subgraph class if the graphs of the functions in $\mathscr{F}$ form a VC-class of sets. In other words, if we define the subgraph of a real-valued function $f$ on $S$ as the following subset $\mathcal{G}_f$ on $S \times \mathbb{R}$:

the class $\{\mathcal{G}_f : f \in \mathscr{F}\}$ is a VC-class of sets on $S \times \mathbb{R}$. Informally, a VC-class of functions is characterized by having a polynomial covering number (the minimal number of required functions to make a covering on the entire class of functions).

A VC-class of functions $\mathscr{F}$ with an envelope function $F$ has the following entropy property. For a given $1\leqslant q < \infty$, there exist constants $a$ and $b$ such that:

for any $\epsilon > 0$ and each probability measure such that $QF^q < \infty$. Several references provide sufficient conditions under which (2.9) holds, such as [144, Lemma 22], [72, §4.7.], [178, Theorem 2.6.7], [114, §9.1], [65, §3.2], ^[33,34,45], offering further discussions.

2.8.   Assumptions

For the reader's convenience, we have compiled the essential assumptions as follows:

Assumption 1. [Model and distribution assumptions]

i) The process $\{X_{i, n}\}$ is locally stationary and satisfies that for each time point $u \in [0, 1]$, there exists a stationary process $\{X_{i}^{(u)}\}$ such that

with $\mathbb{E}[(U_{i, n}^{(u)})^{\rho}] < C$ for some $\rho > 0$, $C < \infty$.

ii) Let $B(x, h)$ be a ball centered at $x \in \mathscr{H}$ with radius $h$, defined in Section 2.4, and let $c_{d} < C_{d}$ be positive constants. For all $\mathbf{u} \in [0, 1]^m$,

where $\phi(0) = 0$ and $\phi(u)$ is absolutely continuous in a neighborhood of the origin, $f_{1}(x)$ is a non-negative functional in $x \in \mathscr{H}$, and

iii) There exist constants $C_{\phi} > 0$ and $\varepsilon_{0} > 0$ such that for any $0 < \varepsilon < \varepsilon_{0}$,

iv) Let $\psi(h_n) \to 0$ as $h \to 0$, and $f_{2}(\mathbf{x})$ is a non-negative functional in $\mathbf{x}: = (x_1, \ldots, x_m) \in \mathcal{H}^m$

We will also assume that the ratio $\psi(h_n)/\phi^{2}(h_n)$ is bounded.

Assumption 2. [Kernel assumptions]

i) $K_{1}(\cdot)$ is a symmetric kernel around zero, bounded, and possesses compact support, i.e., $K_{1}(v) = 0$ for all $\mid v \mid > C_{1}$ for some $C_{1} < \infty$. Additionally,

and $K_{1}(\cdot)$ is Lipschitz continuous, i.e.,

for some $C_{2} < \infty$ and all $v_{1}, v_{2} \in \mathbb{R}$.

ii) The kernel $K_{2}(\cdot)$ is non-negative, bounded, and has a compact support in $[0, 1]$, such that $0 < K_{2}(0)$ and $K_{2}(1) = 0$. Alternatively, $K_2(\cdot)$ can be viewed as an asymmetrical triangular kernel, i.e., $K_2(x) = (1 - x) \mathbb{1}_{(x \in [0, 1])}$, and $K_{2}(\cdot)$ is Lipschitz continuous, i.e.,

Moreover, $K^\prime_{2}(v) = dK_{2}(v)/dv$ exists on $[0, 1]$, and for two real constants $-\infty < C^\prime_{1} < C^\prime_{2} < 0$, we have:

Assumption 3. [Smoothness]

i) $r^{(m)}(\mathbf{u}, x)$ is twice continuously partially differentiable with respect to $\bf u$. We also assume that

for some $c_{m} > 0$, $\alpha > 0$ and $\mathbf{x} = (x_1, \ldots, x_m), \mathbf{z} = (z_1, \ldots, z_m) \in\mathcal{H}^m.$

ii) $\sigma:[0, 1]\times \mathcal{H}^m \to \mathbb{R}$ is bounded by some constant $C_{\sigma} < \infty$ from above and by some constant $c_{\sigma} > 0$ from below, that is, for all $\bf u$ and $\bf x$,

iii) $\sigma(\cdot, \cdot, \cdot)$ is Lipschitz continuous with respect to $\bf u$.

iv) $\sup_{\mathbf{u} \in [0, 1]^m}\sup_{z:{d_{\mathcal{H}^m}(\bf x, \bf z)}\leq \varepsilon}\lvert\sigma(\boldsymbol{\theta}, \mathbf u, \mathbf x)-\sigma(\boldsymbol{\theta}, \mathbf{u}, \mathbf{z})\rvert = o(1)$ as $\varepsilon \to 0$.

Let ${\mathfrak W}_{\mathbf{i}, \varphi, n}$ be an array of one-dimensional random variables. In this study, this array will be equal to ${\mathfrak W}_{\mathbf{i}, \varphi, n} = 1$ or ${\mathfrak W}_{\mathbf{i}, \varphi, n} = \varepsilon_{\mathbf{i}, n}$.

Assumption 4. [Mixing]

i) For $\zeta > 2$ and $C < \infty$, we have

and

ii) The $\beta$-mixing coefficients of the array $\{X_{i, n}, {\mathfrak W}_{i, n}\}$ satisfy $\beta(k) \leq Ak^{-\gamma}$ for some $A > 0$ and $\gamma > 2$. Additionally, we assume that $\delta+1 < \gamma(1-{\frac{2}{\nu}})$ for some $\nu > 2$ and $\delta > 1-{\frac{2}{\nu}}$, along with the condition

as $n \to \infty$, where $a_{n} = \bigg[(\phi(h_n))^{-(1-{\frac{2}{\nu}})/\delta}\bigg]$ and for all $\alpha > 0$.

iii) For some $\zeta_{0} > 0$, as $n \to \infty$, we have

iv) Both $n h^{2m+1}_n$ and $n h^m\phi(h_n)^m$ tend to infinity as $n$ goes to infinity.

Assumption 5. [Blocking assumptions] There exists a sequence of positive integers $\{v_{n}\}$ satisfying $v_{n}\to \infty$, $v_{n} = o(\sqrt{nh\phi(h_n)})$ and $\sqrt{\frac{n} { h\phi(h_n)}}\beta(v_{n}) \to \infty$ as $n \to \infty$.

Assumption 6. [Class of functions assumptions]

The classes of functions $\mathfrak{K}^m_\Theta$ and $\mathscr{F}_m$ are such that :

i) The class of functions $\mathscr{F}_m$ is bounded and its envelope function satisfies for some $0 < M < \infty:$

ii) The class of functions $\mathscr{F}_m\mathfrak{K}^m_\Theta$ is supposed to be of VC-type with envelope function previously defined. Hence, there are two finite constants $b$ and $\nu$ such that:

for any $\epsilon, \nu > 0$ and each probability measure such that $Q(F)^2 < \infty$.

iii) The class of functions $\mathscr{F}_m$ is unbounded and its envelope function satisfies for some $\zeta > 2:$

iv) The metric entropy of the class $\mathscr{FK}$ satisfies, for some $\; 1\leq \zeta < \infty$ :

2.9.   Comments on the assumptions

To establish the groundwork for our analysis, we draw inspiration from seminal works such as ^{[78,87,116,133,179]}. Our assumptions play a pivotal role in shaping the properties of the random processes under consideration. Starting with Assumption 1, we formalize the local stationarity property of $X_i$ and introduce conditions related to the distribution behavior of the variables. Equation (2.10) governs the small-ball probability around zero, representing the standard condition for small-ball probability. This equation implies that the small-ball probability can be approximated as the product of two independent functions $\phi^m(\cdot)$ and ${f}_1(\cdot)$. For instance, when $m = 1$, references such as ^[135] (diffusion process), ^[25] (Gaussian measure), ^[122] (general Gaussian process), and ^[133] (strongly mixing processes) provide context. The function $\phi(\cdot)$ can take various forms, such as $\phi(\epsilon) = \epsilon^{\delta}\exp(-C/\epsilon^{a})$ for Ornstein-Uhlenbeck and general diffusion processes. Further examples and discussions can be found in ^[81] and Remark 3.6. Assumption 1 iv) details the behavior of the joint distribution near the origin, aligning with assumptions made by ^[87] in the context of density estimation for functional data. Assumption 2 encompasses the kernel assumptions commonly used in nonparametric functional estimation. Notably, the Parzen symmetric kernel is inadequate due to the positivity of the random process $D_i = d\left(x, X_i\right)$; thus, $K_2(\cdot)$ with support $[0, 1]$ is considered. The kernel $K_2(\cdot)$ is a symmetric type Ⅱ kernel belonging to the family of continuous kernels (triangle, quadratic, etc.). Compact support on $[0, 1]$ is assumed for the kernels to derive an expression for the asymptotic variance. The Lipschitz-type assumptions on $K_2(\cdot)$ and $\sigma(\cdot, \cdot)$ (Assumption 2ⅱ) and Assumption 3ⅲ)) are crucial for obtaining the convergence rate. Assumption 3 restricts the growth of $r^{(m)}(\cdot)$ and $\sigma(\cdot)$ and places bounds on these functions to prevent rapid growth outside a large bound. It is tailored to ensure the convergence rate and forms an integral part of the overall analysis. Assumption 4 ⅱ) is a standard mixing condition necessary for establishing asymptotic normality and the asymptotic negligibility of the bias, consistent with ^[133]. The variables ${\mathfrak W}_{\mathbf{i}, n}$ are not necessarily bounded, and there is a tradeoff between the decay of the mixing condition and the order ${\zeta}$ of the moment $\sup_{\mathbf{x} \in \mathcal{H}^m}\mathbb{E}\lvert {\mathfrak W}_{\mathbf{i}, n}\rvert^{\zeta} \leq C$. Assumption 4 ⅲ) and ⅳ) are technical conditions crucial for obtaining the desired results, addressing the uniform convergence rate and the bias and convergence rate of the general estimator. Assumption 6 asserts that the class of functions satisfies certain entropy conditions. Part ⅱ) and ⅲ) are interconnected, with the former stating that the class is bounded. However, in the context of proving the functional central limit theorem for conditional $U$-processes indexed by an unbounded class of functions, part ⅲ) supersedes the first one. Assumption 6 ⅱ) ensures that $\mathcal{F}$ is VC type with characteristics $b$ and $n$ for the envelope $F\kappa^m$. Since $F \in L^2(\mathbb P)$ by Assumption 6, Dudley's criterion on the sample continuity of Gaussian processes implies that the function class $\mathcal{F}$ is $\mathbb P$-pre-Gaussian. These general assumptions, inclusive of the mentioned conditions, provide adequate flexibility given the diverse components in our main results. They encapsulate and leverage the topological structure of functional variables, the probability measure within the functional space, the concept of measurability applied to the function class, and the uniformity regulated by entropy properties.

Remark 2.10. It is worth noting that Assumption 6 iii) can be replaced by more general hypotheses regarding the moments of $Y$, as discussed in ^[65]. The alternative assumption takes the following form:

iii)$^{\prime\prime}$ We introduce $\{\mathcal{M}(x) : x \geq 0\}$ as a non-negative continuous function, increasing on $[0, \infty)$, and such that, for some $s > 2$, eventually as $x \rightarrow \infty$:

For each $t \geq \mathcal{M}(0)$, we define $\mathcal{M}^{inv}(t) \geq 0$ such that $\mathcal{M}(\mathcal{M}^{inv}(t)) = t$. Additionally, we assume that:

The following choices for $\mathcal{M}(\cdot)$ are particularly interesting:

(i) $\mathcal{M}(x) = x^{\xi}$ for some $\xi > 2$;

(ii) $\mathcal{M}(x) = \exp (s x)$ for some $s > 0$.

These alternative formulations provide broader flexibility in defining the moments of $Y$, accommodating various scenarios and enhancing the applicability of the analysis.

3.   Uniform convergence rates for kernel estimators

Before expressing the asymptotic behavior of our estimator represented in (2.5), we will generalize the study to a $U$-statistic estimator defined by:

where ${\mathfrak W}_{\mathbf{i}, \varphi, n}$ is an array of one-dimensional random variables. In this study, we use the results with ${\mathfrak W}_{\mathbf{i}, \varphi, n} = 1$ and ${\mathfrak W}_{\mathbf{i}, \varphi, n} = \varepsilon_{\mathbf{i}, n}$.

3.1.   Hoeffding's decomposition

Note that $\widehat{\psi}(\mathbf{u}, \mathbf{x}, \boldsymbol{\theta}, \varphi)$ is a classical $U$-statistic with a kernel depending on $n$. We define

thus, the $U$-statistic in (3.1) can be viewed as a weighted $U$-statistic of degree $m$:

We can write Hoeffding's decomposition in this case as in ^[94]. If we do not assume symmetry for ${\mathfrak W}_{\mathbf{i}, \varphi, n}$ or $H$, we must define:

● The expectation of $H(Z_{i_1}, \ldots, Z_{i_m})$:

● For all $\ell \in \{1, \ldots, m\}$ the position of the argument, construct the function $\pi_\ell$ such that:

● Define:

Hence, the first-order expansion of $H(\cdot)$ will be seen as:

with $\mathbb{P}$ as the underlying probability measure, and define

Then, the first-order projection can be defined as:

where

For the remainder terms, we denote by $\mathbf{i} \backslash i_\ell : = (i_1, \ldots, i_{l-1}, i_{l+1}, \ldots, i_m)$ and for $\ell \in \{1, \ldots, m\}$, let

where

defined in (3.7), this projection derives us to the following remainder term:

Finally, using Eqs (3.9) and (3.11), and under conditions that :

we get the ^[99] decomposition:

For more details, the interested reader can refer to [94, Lemma 2.2].

3.2.   Uniform convergence rate

We commence by presenting the following general proposition.

Proposition 3.1. Let $\mathscr{F}_m\mathfrak{K}^m_\Theta$ denote a measurable VC-subgraph class of functions, adhering to Assumption 6. Suppose that Assumptions 1–4 are satisfied. In such case, the ensuing result holds:

The proof of Proposition 3.1 is deferred to Section 4.1.

Remark 3.2. Elaborating on Proposition 3.1, we can delve into the uniform convergence rate of the kernel estimator $\widetilde{r}_{n}^{(m)}(\varphi, \mathbf{u}, \mathbf{x}, \boldsymbol{\theta}; h_{n})$. It is crucial to emphasize that when $m = 1$ and the function $\varphi$ remains constant, the outcomes align with the pointwise convergence rate of the regression function for a strictly stationary functional time series, as discussed in ^[78].

The subsequent theorem presents the uniform convergence rate of the kernel estimator (2.5).

Theorem 3.3. Let $\mathscr{F}_m\mathfrak{K}^m_\Theta$ be a measurable VC-subgraph class of functions complying with Assumption 6. Suppose Assumptions 1–4 are fulfilled. Then, we have:

The proof of Theorem 3.15 is postponed until Section 4.1.

Remark 3.4. It is possible to consider the setting of $\Theta = \Theta_n$ and assume that

and

where $b_n$ tends to zero, as in ^[145].

Remark 3.5. In contrast to Theorem 4.2 in ^[179] and akin to Theorem 3.1 in ^[116], our formulation excludes the bias term arising from the approximation error of $X_{i, n}$ by $X_{i}^{(u)}$. Under our assumptions, the approximation error is negligibly small compared to $h^{2m \wedge \beta}_n$.

Remark 3.6. In nonparametric problems, the infinite dimensionality of the target function is typically determined by the smoothness condition, specifically in Assumptions 2 i) and 3 i). This primarily affects the bias component of the convergence rates, represented by terms like $O\left(h^{2m\wedge\alpha}_n\right)$ in Theorem 3.3. Other terms in the convergence rates stem directly from dispersion effects and are inherently linked to the concentration properties of the probability measure of the variable $X$. These terms can be expressed as

where small-ball probabilities are quantified and controlled by the mean of the function $\phi(\cdot)$ defined in (2.4). The rate of convergence is influenced by the concentration of the measure of the process $X$; less concentration leads to a slower rate of convergence. Unfortunately, solutions to $\mathbb{P}\left(X \in B(x, r)\right)$ are known for very few random variables (or processes) $X$, even when $x = 0$. In certain functional spaces, considering $x \neq 0$ introduces considerable difficulties that might not be surmountable. Authors often focus on Gaussian random elements, and for a comprehensive overview of main results on small-ball probability, refer to ^[122]. In many scenarios, it is convenient to assume that

where, to ensure the identifiability of the decomposition, a normalizing restriction is necessary, such as

The factorization (3.16) is not a stringent assumption; it holds under appropriate hypotheses (see, for instance, ^[27,122]). The advantage of assuming (3.16) is two-fold. Firstly, the function $\psi(x)$ can be viewed as a surrogate density of the functional random element $X$ and can be leveraged in various contexts. The interested reader can explore its potential in works like ^[26,83,87], where the surrogate density is estimated and used to define a notion of mode or for classification purposes. Second, the function $\phi(h_n)$ acts as the volumetric term and can be employed to assess the complexity of the probability law of the process $X$ (see ^[28]). In the special multi-(but finite)-dimensional scenario where $\mathbf{X} \in \mathbb{R}^d$, the relation (2.4) is satisfied under standard assumptions with $\phi_x(h_n) \sim C_x h^d$, commonly known as the curse of dimensionality (see ^[77,80]). Here, we would instead refer to it as the curse of infinite dimension, specifically highlighting the effects of small-ball probabilities. The inherent nature of these probability effects involving small-balls becomes apparent within our infinite-dimensional framework. The remainder of this remark will focus on applying our approach to various continuous-time processes, where the probabilities associated with small-balls have already been identified. For more details on the following examples, refer to ^[80].

(i) Consider the space $\mathcal{C}([0, 1], \mathbb{R})$ equipped with the supremum norm, and its associated Cameron-Martin space

Let's examine the fractional Brownian motion $\zeta^{\mathrm{FBM}}$ with parameter $\delta, 0 < \delta < 2$. small-ball probabilities in this context have been extensively studied. According to [122,Theorems 3.1 and 4.6], we have

Notably, our crucial relation (2.4) is trivially satisfied for the fractional Brownian motion by choosing the function $\phi(\cdot)$ in the form

(ii) Consider a centered Gaussian process $\zeta^{\mathrm{GP}} = \left\{\zeta_t^{\mathrm{GP}}, 0 \leqslant t \leqslant 1\right\}$. This process can be expressed using the Karhunen-Loève decomposition as follows:

where $\lambda_i$ are the eigenvalues of the covariance operator of $\zeta^{\mathrm{GP}}$, $f_i$ are the associated orthonormal eigenfunctions, and ${\mathfrak W}_i$ are independent standard normal real random variables. For any fixed $k \in \mathbb{N}^*$, let $\Pi_k$ be the orthogonal projection onto the subspace spanned by the eigenfunctions $\left\{f_1, \ldots, f_k\right\}$. Define a semi-metric by

Using the Karhunen-Loève expansion, we obtain

where $x_i = \int_0^1 x(t) f_i(t) \mathrm{d} t$, and $Z_i$ are the components of the vector $Z = \left(Z_1, \ldots, Z_k\right)$, exhibiting the Euclidean norm structure on $\mathbb{R}^k$. Due to the independence of $Z_i$ with densities with respect to the Lebesgue measure, we have

(iii) Consider the space $\mathcal{C}([0, 1], \mathbb{R})$ of continuous real-valued functions on $[0, 1]$, equipped with the supremum norm denoted by $\|\cdot\|_{\infty}$. Let $\mathbb P^{\mathrm{W}}$ be the Wiener measure on $\mathcal{C}([0, 1], \mathbb{R})$, and define the Cameron-Martin space of $\mathcal{C}([0, 1], \mathbb{R})$ as

Consider the Ornstein-Uhlenbeck process $\zeta^{\mathrm{OU}}$ with $\zeta_0^{\mathrm{OU}} = 0$ and

Wiener measures for small centered balls are known to be of the form [^[25], p. 187]:

By extending this result to any small-ball probability measure through the Cameron-Martin space characterization, we have

Since the Ornstein-Uhlenbeck process has a probability measure absolutely continuous with respect to $P^{\mathrm{W}}$, we can directly state

Our crucial relation (2.4) is trivially satisfied for this Ornstein-Uhlenbeck process by choosing the function $\phi_x(\cdot)$ in the form

4.   Weak convergence for kernel estimators

In this section, we are interested in studying the weak convergence of the conditional $U$-processes under absolute regular observations. Observe that

where

Under the same assumption in Theorem 3.3, we will show in the next theorem that

and

Then, we have

where

is the "bias" term and $\frac{\widehat{g}_{1}(\boldsymbol{\theta}, \mathbf u, \mathbf x)}{ \widetilde{r}_1(\varphi, \boldsymbol{\theta}, \mathbf{u}, \mathbf{x})}$ is the "variance" term. Let us define, for $\varphi_1, \varphi_2\in \mathcal F_m$

In the following, we would set $K_{2}(\cdot)$ as the asymmetrical triangle kernel, that is, $K_{2}(x) = (1-x) \mathbb 1_{(x \in [0, 1])}$ to simplify the proof. The main results of this section are given in the following theorems.

Theorem 4.1. Let $\mathscr{F}_m\mathfrak{K}^m_\Theta$ be a measurable VC-subgraph class of functions, and assume that all the assumptions of Section 2.8 are satisfied for both cases ${\mathfrak W}_{\mathbf{i}, \varphi, n} = 1$ and ${\mathfrak W}_{\mathbf{i}, \varphi, n} = \varepsilon_{\mathbf{i}, n}$. Then, as $n\to \infty$, the $U$-process

converges to a Gaussian process $\mathfrak G_{n}$ over $\mathscr{F}_m\mathfrak{K}^m_\Theta$, whose sample paths are bounded and informally continuous with respect to the $\Vert\cdot\Vert_2$-norm, with the covariance function given in (4.2).

The proof of Theorem 4.2 is postponed until Section 4.1.

To examine the weak convergence of our estimator using the standard procedure, involving Hoeffding decomposition, finite-dimensional convergence, and equicontinuity, we can turn to the following theorem. In the proof of this theorem, we express the conditional $U$-process in terms of a $U$-process based on a stationary sequence, illustrating its convergence to a Gaussian process. This convergence is established in the distribution sense within $l^{\infty}(\mathscr{F}_m\mathfrak{K}^m_\Theta)$, the space of bounded real functions on $\mathscr{F}_m\mathfrak{K}^m_\Theta$, as defined in ^[100]. For further details, refer to ^[6,71], or ^[178].

Theorem 4.2. Assume $\mathscr{F}_m\mathfrak{K}^m_\Theta$ is a measurable VC-subgraph class of functions, and all assumptions in Section 2.8 are satisfied. If in addition $n\phi_{\mathbf x, \boldsymbol\theta}^{1/m}(h_n)h^{m+2(2m\wedge\alpha)}\rightarrow 0$ as $n\to \infty$, then we have

converges in law to a Gaussian process $\left\{ \mathbb{G}(\psi):\psi\in\mathscr{F}_m\mathfrak{K}^m_\Theta\right\}$ in $l^{\infty}(\mathscr{F}_m\mathfrak{K}^m_\Theta)$ that admits a version with uniformly bounded and uniformly continuous paths with respect to the $\Vert\cdot\Vert_2-$norm, and its covariance function is given in (4.2).

The proof of Theorem 4.2 is deferred to Section 8.

Remark 4.3. To eliminate the bias term, it is necessary to have $n\phi^{1/m}_{\mathbf x, \boldsymbol\theta}(h_n)h^{m+2(2m\wedge\alpha)}\rightarrow 0$ as $n \rightarrow \infty$. As a consequence, the last condition, along with $nh^m\phi^{1/m}_{\mathbf x, \boldsymbol\theta}(h_n)\rightarrow \infty$, holds as long as $h_n = n^{-\xi}$ and $\phi_{\mathbf x, \boldsymbol\theta}(h_n) = h_n^{mc}$, where $0 < c < 1-{\frac{1}{\xi m}}$ and $\frac{1}{m(1+c)+ 2(2m\wedge \alpha)} < \xi < \frac{1}{m(1-c)}$.

Remark 4.4. The validity of the results remains intact even when replacing the entropy condition with the bracketing condition. In particular, the existence of constants $C_0 > 0$ and $v_0 > 0$ ensures the specified inequality, as described in the remark. In our framework, the choice of the kernel function is flexible, with minimal restrictions, as long as some mild conditions are satisfied. However, the selection of the bandwidth introduces challenges, and it is crucial for achieving a favorable rate of consistency. The bandwidth choice significantly impacts the bias and variance trade-off in the estimator. Therefore, adopting a bandwidth that adjusts based on specific criteria, available data, and location is more suitable. The discussion on this topic can be found in ^[44,46,132]. Establishing uniform-in-bandwidth central limit theorems in our context would be particularly interesting.

Remark 4.5. We can consider the scenario where $\Theta = \Theta_n$, where $\Theta_n$ satisfies the conditions $\operatorname{card}\left(\Theta_n\right) = n^\alpha$ with $\alpha > 0$, and for every $\theta \in \Theta_n$, we have

where $b_n$ converges to zero, as discussed in ^[145].

The functional directions set, $\Theta_n$, is constructed following a similar approach to ^[2,145], as outlined below:

(i) Each direction $\theta \in \Theta_n$ is derived from a $d_n$-dimensional space formed by $\mathrm{B}$-spline basis functions, denoted by $\left\{e_1(\cdot), \ldots, e_{d_n}(\cdot)\right\}$. Thus, we express directions as:

(ii) The set of coefficient vectors in (4.3), denoted by $\mathcal{V}$, is generated through the following steps:

Step 1. For each $\left(\beta_1, \ldots, \beta_{d_n}\right) \in \mathcal{C}^{d_n}$, where $\mathcal{C} = \left\{c_1, \ldots, c_J\right\} \subset \mathbb{R}^J$ represents a set of $J$ 'seed-coefficients', construct the initial functional direction as

Step 2. For each $\theta_{\text {init }}$ from Step 1 satisfying $\theta_{\text {init }}\left(t_0\right) > 0$, where $t_0$ denotes a fixed value in the domain of $\theta_{\text {init }}(\cdot)$, compute $\langle\theta_{\text {init }}, \theta_{\text {init }}\rangle$ and form $\left(\alpha_1, \ldots, \alpha_{d_n}\right) = \left(\beta_1, \ldots, \beta_{d_n}\right) /\langle\theta_{\text {init }}, \theta_{\text {init }}\rangle^{1 / 2}$.

Step 3. Define $\mathcal{V}$ as the collection of vectors $(\alpha_1, \ldots, \alpha_{d_n})$ obtained in Step 2. Consequently, the final set of permissible functional directions is represented as

5.   Applications

While only the subsequent examples will be provided in this section, they serve as prototypes for a range of problems that can be explored in a comparable manner.

5.1.   Discrimination

Now, we apply the results to the discrimination problem described in Section 3 of ^[168], also referring to ^[167]. We will employ similar notation and settings. Let $\varphi(\cdot)$ be any function taking at most finitely many values, say $1, \ldots, M$. The sets

then yield a partition of the feature space. Predicting the value of $\varphi(Y_{1}, \ldots, Y_{m})$ is tantamount to predicting the set in the partition to which $(Y_{1}, \ldots, Y_{m})$ belongs. For any discrimination rule $g$, we have

where

The above inequality becomes equality if

$\mathfrak G_{0}(\cdot)$ is called the Bayes rule, and the pertaining probability of error

is called the Bayes risk. Each of the above unknown functions $\mathfrak{M}^{j}$'s can be consistently estimated by one of the methods discussed in the preceding sections. Let, for $1\leq j\leq M$,

Set

Let us introduce

Then, one can show that the discrimination rule $\mathfrak G_{0, n}(\cdot)$ is asymptotically Bayes' risk consistent

This follows from the apparent relation:

5.2.   Metric learning

Metric learning, a field that has garnered significant attention in recent years, revolves around adapting the metric to the underlying data. Extensive discussions on metric learning and its applications are available in ^[19,54]. This concept has proven valuable in diverse domains, including computer vision, information retrieval, and bioinformatics. To demonstrate the practicality of metric learning, we delve into the metric learning problem for supervised classification outlined in ^[54]. Consider independent copies $\left(X_{1}, Y_{1}\right), \ldots, \left(X_{n}, Y_{n}\right)$ of a $\mathscr{H}\times \mathcal{Y}$-valued random couple $(X, Y)$, where $\mathscr{H}$ is a feature space, and $\mathcal{Y} = \{1, \ldots, C\}$ (with $C \geq 2$), representing a finite set of labels. Let $\mathcal{D}$ be a set of distance measures $D: \mathscr{H} \times \mathscr{H} \rightarrow \mathbb{R}_{+}$. In this context, the objective of metric learning is to find a metric under which pairs of points with the same label are close to each other, while those with different labels are far apart. The natural way to define the risk of a metric $D$ is given by:

where $\phi(u)$ is a convex loss function upper-bounding the indicator function $\mathbb{1}\{u \geq 0\}$, for instance, the hinge loss $\phi(u) = \max (0, 1-u)$. To estimate $R(D)$, we consider the natural empirical estimator:

which is a one-sample $U$-statistic of degree two with a kernel given by:

The convergence to (5.1) of a minimizer of (5.2) has been investigated within the frameworks of algorithmic stability ^[107], algorithmic robustness ^[18], and the theory of $U$-processes under suitable regularization ^[50].

5.3.   Kendall rank correlation coefficient

To test the independence of one-dimensional random variables $Y_1$ and $Y_2$, ^[110] proposed a method based on the $U$-statistic $K_{n}$ with the kernel function :

Its rejection on the region is of the form $\left\{\sqrt{n} K_{n} > \gamma\right\}$. In this example, we consider a multivariate case. To test the conditional independence of $\boldsymbol{\xi}, \boldsymbol{\eta}: Y = (\boldsymbol{\xi}, \boldsymbol{\eta})$ given $X$, we propose a method based on the conditional U-statistic :

where $\mathbf{x} = \left(x_{1}, x_{2}\right) \in \mathbb{I} \subset \mathbb{R}^{2}$ and $\varphi(\cdot)$ is Kendall's kernel (5.3). Suppose that $\boldsymbol{\xi}$ and $\boldsymbol{\eta}$ are $d_{1}$ and $d_{2}$-dimensional random vectors respectively and $d_{1}+d_{2} = d$. Furthermore, suppose that $\mathrm{Y}_{1}, \ldots, \mathrm{Y}_{n}$ are observations of $(\boldsymbol{\xi}, \boldsymbol{\eta})$, we are interested in testing :

Let $\mathbf{a} = \left(\mathbf{a}_{1}, \mathbf{a}_{2}\right) \in \mathbb{R}^{d}$ such as $\|\mathbf{a}\| = 1$ and $\mathbf{a}_{1} \in \mathbb{R}^{d_{1}}, \mathbf{a}_{2} \in \mathbb{R}^{d_{2}}$, and $\mathrm{F}(\cdot), \mathrm{G}(\cdot)$ be the distribution functions of $\boldsymbol{\xi}$ and $\boldsymbol{\eta}$, respectively. Suppose $F^{a_{1}}(\cdot)$ and $G^{a_{2}}(\cdot)$ to be continuous for any unit vector $\mathbf{a} = \left(\mathbf{a}_{1}, \mathbf{a}_{2}\right)$ where $\mathrm{F}^{\mathbf{a}_{1}}(t) = \mathbb{P}\left(\mathbf{a}_{1}^{\top} \boldsymbol{\xi} < t\right)$ and $\mathrm{G}^{\mathbf{a}_{2}}(t) = \mathbb{P}\left(\mathbf{a}_{2}^{\top} \boldsymbol{\eta} < t\right)$ and $\mathbf{a}_{1}^{\mathrm{T}}$ means the transpose of the vector $\mathbf{a}_{i}, 1 \leqslant i \leqslant 2$. For $n = 2$, let $Y^{(1)} = \left(\boldsymbol{\xi}^{(1)}, \boldsymbol{\eta}^{(1)}\right)$ and $Y^{(2)} = \left(\boldsymbol{\xi}^{(2)}, \boldsymbol{\eta}^{(2)}\right)$ such as $\boldsymbol{\xi}^{(i)} \in \mathbb{R}^{d_{1}}$ and $\boldsymbol{\eta}^{(i)} \in \mathbb{R}^{d_{2}}$ for $i = 1, 2$, and :

An application of Theorem 3.3 gives

5.4.   Conditional U-statistics for censored data

Consider a triple $(Y, C, {X})$ of random variables defined in $\mathbb{R} \times \mathbb{R} \times \mathcal{H}$. Here, $Y$ is the variable of interest, $C$ is a censoring variable, and ${X}$ is a concomitant variable. Throughout, we work with a sample $\{(Y_{i}, C_{i}, { X}_{i})_{1\leq i\leq n}\}$ of independent and identically distributed replication of $(Y, C, { X})$, $n \geq 1$. Actually, in the right censorship model, the pairs $(Y_{i}, C_{i})$, $1 \leq i \leq n$, are not directly observed, and the corresponding information is given by $Z_{i} : = \min\{Y_{i}, C_{i}\}$ and $\Delta_{i} : = \mathbb{1}\{Y_{i}\leq C_{i}\}$, $1 \leq i \leq n$. Accordingly, the observed sample is

For example, survival data in clinical trials or failure time data in reliability studies are often subject to such censoring. To be more specific, many statistical experiments result in incomplete samples, even under well-controlled conditions. For example, clinical data for surviving most types of diseases are usually censored by other competing risks to life, which result in death. In the sequel, we impose the following assumptions upon the distribution of $({ X}, Y)$. For $-\infty < t < \infty$, set

the right-continuous distribution functions of $Y$, $C$, and $Z$, respectively. For any right-continuous distribution function $\mathfrak L$ defined on $\mathbb{R}$, denote by

the upper point of the corresponding distribution. Now consider a pointwise measurable class $\mathscr{F}$ of real measurable functions defined on $\mathbb{R}$, and assume that $\mathscr{F}$ is of VC-type. We recall the regression function of $\psi(Y)$ evaluated at $\langle{ X}, \theta\rangle = \langle{ t}, \theta\rangle$, for $\psi \in \mathscr{F}$ and $t \in\mathcal H$, given by

when $Y$ is right-censored. To estimate $r^{(1)}(\psi, \cdot)$, we make use of the inverse probability of censoring weighted (I.P.C.W.) estimators that have recently gained popularity in the censored data literature (see ^[51,111]). The key ideas of I.P.C.W. estimators are as follows. Introduce the real-valued function $\Phi_{\psi}(\cdot, \cdot)$ defined on $\mathbb{R}^{2}$ by

Assuming the function $G(\cdot)$ to be known, first note that $\Phi_{\psi}(Y_{i}, C_{i}) = \Delta_{i}\psi(Z_{i})/(1 -G(Z_{i}))$ is observed for every $1 \leq i \leq n$. Moreover, under the Assumption (I) below:

(I) $C$ and $(Y, \mathbf{ X})$ are independent.

We have

Therefore, any estimate of $r^{(1)}(\Phi_{\psi}, \cdot)$, which can be built on fully observed data, turns out to be an estimate for $r^{(1)}(\psi, \cdot)$ too. Thanks to this property, most statistical procedures that provide estimates of the regression function in the uncensored case can be naturally extended to the censored case. For instance, kernel-type estimates are straightforward to construct. Set, for $\mathbf{ x}\in \mathcal{I}$, $h\geq 0$, $1\leq i\leq n$,

In view of (5.6)–(5.8), whenever $G(\cdot)$ is known, a kernel estimator of $r^{(1)}(\psi, \cdot)$ is given by

The function $G(\cdot)$ is generally unknown and has to be estimated. We will denote by $G^{*}_{n}(\cdot)$ the Kaplan-Meier estimator of the function $G(\cdot)$ ^[108]. Namely, adopting the conventions

and $0^{0} = 1$ and setting

we have

Given this notation, we will investigate the following estimator of $r^{(1)}(\psi, \cdot)$

refer to ^[111,130]. Adopting the convention $0/0 = 0$, this quantity is well defined, since $G_{n}^{*}(Z_{i}) = 1$ if and only if $Z_{i} = Z_{(n)}$ and $\Delta_{(n)} = 0$, where $Z_{(k)}$ is the $k$th ordered statistic associated with the sample $(Z_{1}, \ldots, Z_{n})$ for $k = 1, \ldots, n$ and $\Delta_{(k)}$ is the $\Delta_{j}$ corresponding to $Z_{k} = Z_{j}$. A right-censored version of an unconditional $U$-statistic with a kernel of degree $m\geq 1$ is introduced by the principle of a mean preserving reweighting scheme in ^[60]. Reference ^[170] has proved almost sure convergence of multi-sample $U$-statistics under random censorship and provided application by considering the consistency of a new class of tests designed for testing equality in distribution. To overcome potential biases arising from right-censoring of the outcomes and the presence of confounding covariates, ^[53] proposed adjustments to the classical $U$-statistics. ^[186] proposed a different way in the estimation procedure of the $U$-statistic by using a substitution estimator of the conditional kernel given the observed data. We also refer to ^[45]. To our best knowledge, the problem of the estimation of the conditional $U$-statistics in the censored setting with variable bandwidth was opened up to the present, and it gives the primary motivation for the study of this section. A natural extension of the function defined in (5.6) is given by

From this, we have an analogous relation to (5.7) given by

An analogue estimator to (1.1) in the censored case is given by

where, for $\mathbf{i} = (i_{1}, \ldots, i_{k})\in I(k, n)$,

The estimator that we will investigate is given by

The main result of this section is given in the following corollary.

Corollary 5.1. Let $\mathscr{F}_m\mathfrak{K}^m_\Theta$ be a measurable VC-subgraph class of functions complying with Assumption 6. Suppose Assumptions 1–4 are fulfilled. Then we have:

This last result is a direct consequence of Theorem 3.3 and the law of iterated logarithm for $G_{n}^{*}(\cdot)$ established in ^[84] ensures that

For more details refer to ^[34].

5.5.   Conditional U-statistics for left truncated and right censored data

Keeping in mind the notation of the preceding section, we now introduce a truncation variable denoted as $L$ and assume that $(L, C)$ is independent of $Y$. Let us consider a situation where we have random vectors $(Z_i \epsilon_i, \Delta_i)$, with $\epsilon_i = \mathbb 1 (L_i \leq Z_i)$. In this section, we aim to define $U$-statistics conditional for data that are left truncated and right censored (LTRC), by following ideas from ^[172] in the unconditional setting. To achieve this, we propose an extension of the function $(5.6)$ for LTRC data as follows:

According to (5.12), we get that

An analog estimator to (1.1) for LTRC data can be expressed as follows:

where $\overline{\omega}_{n, \mathbf{ i}}^{(k)}(\boldsymbol{\theta}, \mathbf u, \mathbf{x})$ is defined as in (5.14). As $\mathbb P\left(L_i < Z_i < C_i\right)$ is not known, we need to estimate it. We introduce $N_i(t) =$ $\mathbb{1}(L_i < Z_i \leq t, \Delta_i = 1)$ and $N_i^c(t) = \mathbb{1}\left(L_i < Z_i \leq t, \Delta_i = 0\right)$ as the counting process corresponding to the variable of interest and the censoring variable. Furthermore, let

and

We introduce the risk indicators as $R_i(t) = \mathbb{1}\left(Z_i \geq t \geq L_i\right)$ and

It is important to note that the risk set $R(t)$ at $t$ contains the subjects who entered the study before $t$ and are still under study at $t$. Indeed, $N_i^c(t)$ is a local sub-martingale with the appropriate filtration $\mathbb{F}_t$. The martingale associated with the censoring counting process with filtration $\mathbb{F}_t$ is given by

Here, $\lambda_c(\cdot)$ represents the hazard function associated with the censoring variable $C$ under left truncation. The cumulative hazard function for the censoring variable $C$ is defined as

Denote

Now, we define the sub-distribution function of $T_1$ corresponding to $\Delta_1 = 1$ and $\epsilon_1 = 1$ as

Let

where $h_1(x) = \mathbb E\left(\psi\left(\left(T_1, \Delta_1\right), \ldots, \left(T_k, \Delta_k\right)\right) \mid\left(T_1, \Delta_1\right) = \left(x, \Delta_1\right)\right)$. Also, denote $\tilde z(t) = \mathbb P\left(T_1 \geq t \geq L_1\right)$. Then, an estimate for the survival function of the censoring variable $C$ under left truncation, denoted as $\widehat{K}_c(\cdot)$, see ^[174], can be formulated as follows:

Similar to the Nelson-Aalen estimator (for instance, see ^[7]), the estimator for the cumulative hazard function of the censoring variable $C$ under left truncation is represented as:

In both the definitions presented in (5.18) and (5.19), we make the assumption that $\tilde Z(t)$ is non-zero with probability one. The interrelation between $\widehat{K}_c(\tau)$ and $\widehat{\Lambda}_c(\tau)$ can be expressed as:

Let $a_K = \inf \{t: K(t) > 0\}$ and $b_K = \sup \{t: K(t) < 1\}$ denote the left and right endpoints of the support. For LTRC data, as in ^[187], $F(\cdot)$ is identifiable if $a_G \leqslant a_W$ and $b_G \leqslant b_W$. By Corollary 2.2. ^[187], for $b < b_W$ we readily infer that

From the above, the estimator (5.17) can be rewritten directly as follows:

The last estimator is the conditional version of that studied in ^[172]. Following the same reasoning of the Corollary 5.1, one can infer that as $n\rightarrow \infty,$

6.   The bandwidth selection criterion

Various methodologies have been devised to formulate asymptotically optimal bandwidth selection rules for nonparametric kernel estimators, particularly for the Nadaraya-Watson regression estimator. Key contributions have been made by ^{[42,92,96,150]}. The proper selection of bandwidth is pivotal, whether in the standard finite-dimensional case or in the infinite-dimensional framework, to ensure robust practical performance. Currently, to the best of our knowledge, such investigations do not exist for addressing a general functional conditional $U$-statistic. However, an extension of the leave-one-out cross-validation procedure allows us to define, for any fixed $\mathbf{j} = (j_1, \ldots, j_m)\in I_n^m$:

where:

Equation (6.1) denotes the leave-out-$\left(\mathbf{X}_{\mathbf{j}}, \mathbf{Y}_{\mathbf{j}}\right)$ estimator of the functional regression and can also serve as a predictor of $\varphi(\mathbf{ Y}_{{j_{1}}}, \ldots, \mathbf{ Y}_{{j_{m}}}): = \varphi(\mathbf{ Y}_{\mathbf j})$. To minimize the quadratic loss function, we introduce the following criterion. Let $\mathcal{W}(\cdot)$ be a known non-negative weight function:

Building upon the concepts advanced by ^[150], a judicious approach for selecting the bandwidth is to minimize the aforementioned criterion. Therefore, we choose $\widehat{h}_{n} \in [a_n, b_n]$, minimizing among $h \in [a_n, b_n]$:

Following the approach proposed by ^[20], where bandwidths are locally determined through a data-driven method involving the minimization of a functional version of a cross-validated criterion, we can substitute (6.2) with:

where

In practice, one takes, for $\mathbf{ i} \in I_n^m$, the uniform global weights $\widetilde{\mathcal{W}}\left(\mathbf{X}_{\mathbf{i}}\right) = 1$, and the local weights

For conciseness, we have exclusively discussed the widely used cross-validated selected bandwidth method. However, this approach can be generalized to other bandwidth selectors, including those based on Bayesian principles ^[156].

7.   Concluding remarks

This manuscript introduces the theory of single-index $U$-processes tailored specifically for locally stationary variables within the functional data paradigm. The primary objective is to leverage functional local stationary approximations to facilitate asymptotic analyses in the statistical inference of non-stationary time series. We underscore the significance of adopting absolutely regular conditions or $\beta$-mixing conditions, which are independent of the entropy dimension of the class, a departure from other mixing conditions. In contrast to strong mixing, $\beta$-mixing offers greater flexibility, allowing decoupling and accommodation of diverse examples. Reference ^[103] provided a comprehensive characterization of stationary Gaussian processes satisfying the $\beta$-mixing condition. Additionally, $\beta$-mixing aligns with the $L_2(\mathbb{P})$-norm, playing a crucial role. Unlike strong mixing, which demands a polynomial rate of decay for strong mixing coefficients contingent on the entropy dimension of the function class, $\beta$-mixing involves the $L_1$-norm and the metric entropy function $H(\cdot, T, d)$. This function is defined concerning the pseudo-metric $d(s, t) = \sqrt{\operatorname{Var}(\mathbb{G}(s)-\mathbb{G}(t))}$ for a Gaussian process $\mathbb{G}(\cdot)$. The definition satisfies the integrability condition:

Consequently, we establish the rate of convergence, demonstrating that, under suitable conditions, the kernel estimator $\widetilde{r}_{n}^{(m)}(\varphi, \mathbf{u}, \mathbf{x}, \boldsymbol{\theta}; h_{n})$ constructed with bandwidth $h$ converges to the regression operator $r^{(m)}(\varphi, \boldsymbol{\theta}, \mathbf{u}, \mathbf{x})$ with a rate:

The presented rate underscores the importance of the small-ball probability function, impacting the concentration of functional variables $X_i$. The second term is linked to the bias of the estimate, dependent on the smoothness of the operator $r^{(m)}(\varphi, \boldsymbol{\theta}, \mathbf{u}, \mathbf{x})$ and its parameter $\alpha$, as indicated by the Lipschitz condition. The interconnected nature of the concentration of functional variables $X$, the small-ball probability, and the convergence rate are crucial for achieving a more efficient estimator with less dispersed variables and a higher small-ball probability. In the context of empirical process settings, the rate of convergence is established over a subset $[Ch, Ch-1]^m \times \{\mathbf{x}\}^m$, and for forecasting purposes, it can be extended to a subset $[Ch-1, 1]^m \times \{\mathbf{x}\}^m$ using one-sided kernels or boundary-corrected kernels. This extension necessitates ensuring that the kernels have compact support and are Lipschitz. The weak convergence is established through classical procedures, involving finite-dimensional convergence and the equicontinuity of conditional $U$-processes. Finite-dimensional convergence is achieved through Hoeffding decomposition, followed by approaching independence through a block decomposition strategy, ultimately leading to proving a central limit theorem for independent variables. The equicontinuity aspect requires meticulous attention due to the comprehensive framework considered. These results provide a solid theoretical foundation for our methodologies, extending non-parametric functional principles to a generic dependent structure—a relatively new and significant research area. It is crucial to note that, when dealing with highly dependent data, mixing, often adopted for simplicity, may not be suitable. The ergodic framework eliminates the need for frequently used strong mixing conditions, along with their variations for measuring dependence, and the more intricate probabilistic computations they entail (see ^[36,37,47]). An intriguing avenue for exploration is the $k$NN estimator:

where

with \(B(t, h) = \left\{z \in \mathcal{H}: d(z, t) \leqslant h\right\}\) representing a ball in \(\mathcal{H}\) with center \(t \in \mathcal{H} \) and radius \(h\), and \(\mathbb{1}_{A}\) being the indicator function of the set \(A\), as detailed in ^[3]. These findings open avenues for various applications, such as data-driven automatic bandwidth selection and confidence band construction. We propose the intriguing notion that bootstrap methods, as outlined in ^[35,38], provide valuable insights when applied in the functional context, especially the functional variant of the wild bootstrap, employed for smoothing parameter selection. It is crucial to acknowledge that the theoretical underpinnings of this bandwidth selection method using the functional wild bootstrap are still an area with unresolved challenges. Finally, change-point detection has emerged as a widely utilized technique for recognizing specific points within a data series when a stochastic system experiences abrupt external perturbations. The occurrence of these alterations can be attributed to several circumstances, hence their identification can greatly enhance their comprehension. The application of change-point analysis has been observed in numerous stochastic processes across a wide range of scientific domains. However, the investigation of change-point analysis for conditional $U$-statistics remains an unexplored and demanding research issue.

8.   Mathematical developments

In this section, we focus on proving our results, using the notation introduced earlier. We start by presenting the following lemma before delving into the proof of the main results. The proof techniques follow and extend those of ^[165] to the single index setting. Additionally, we incorporate certain intricate steps from ^[13], as observed in ^[39,40].

Proof of Proposition 3.1. As mentioned earlier, our statistic is a weighted $U$-statistic, expressed as a sum of $U$-statistics through the Hoeffding decomposition. We will delve into the details of this decomposition in Sub-section 3.1 to achieve our desired outcomes. In that specific section, we observed that:

where the linear term $\widehat{\psi}_{1, \mathbf{i}}(\mathbf{u}, \mathbf{x}, \boldsymbol{\theta}, \varphi)$ and the residual term $\widehat{\psi}_{2, \mathbf{i}}(\mathbf{u}, \mathbf{x}, \boldsymbol{\theta}, \varphi)$ are precisely defined in (3.9) and (3.11), respectively. Our goal is to establish that the linear term governs the convergence rate of this statistic, while the remaining term converges to zero almost surely as $n\rightarrow \infty$. We will initiate the analysis by addressing the first term in the decomposition. Consider $B = [0, 1]$, with \(\alpha_{n} = \sqrt{\log n / n h^m \phi^m(h_n)}\) and \(\tau_{n} = \rho_{n} n^{1 / \zeta}\), where \(\zeta\) is a positive constant specified in Assumption 4, part ⅰ), and \(\rho_{n} = (\log n)^{\zeta_{0}}\) for some \(\zeta_{0}>0\). Define

and

It is evident that we have

First, we can see that

We deduce that

where

where

Thus, we acquire

Consequently, we deduce that

Next, let's consider

To achieve the aimed result, we will cover the region $B^m = [0, 1]^{m}$ by

for some radius $r$. Hence, for each $\mathbf{u} = (\mathbf{u}_1, \ldots, \mathbf{u}_m)\in [0, 1]^{m}$, there exists $\mathbf{l}(\mathbf{u}) = (l(\mathbf{u}_1), \ldots, l(\mathbf{u}_m))$, where $\forall 1 \leq i \leq m, 1\leq l(\mathbf{u}_i)\leq N_{(\mathbf{u})}$ and such that

then, for each $\mathbf{u}\in [0, 1]^{m}$, the closest center will be $\mathbf{u}_\mathbf{l}(\mathbf{u})$, and the ball with the closest centre will be defined by

In the same way, $\Theta^m\times \mathcal{H}^m$ should be covered by

for some radius $r$. Hence, for each $\mathbf{x} = (x_1, \ldots, x_m)\in \mathcal{H}^m$, there exists $\mathbf{l}(\mathbf{x}) = (l(x_1), \ldots, l(x_m))$, where $\forall 1 \leq i \leq m, 1\leq l(x_i)\leq {N_{(x)}}$ and such that

then, for each $\mathbf{x}\in \mathcal{H}^m$, the closest center will be $\mathbf{x}_\mathbf{l}(\mathbf{x})$, and the ball with the closest centre will be defined by

We define

We can show that, for $(u, x) \in B _{j, n}$ and $n$ large enough,

Let

Note that $\mathbb E\left[\left\lvert \bar{\psi}_{1}^{(1)}(\varphi, \boldsymbol{\theta}, \mathbf{u}, \mathbf{x})\right\rvert\right] \leq M < \infty$ for some sufficiently large $M$. Then, we obtain

Therefore

where, for $N_{\mathscr{F}_m\mathfrak{K}^m_\Theta}N_{(\theta)}^m N_{(x)}^m N_{(u)}$ denotes the covering number related respectively to the class of functions $\mathscr{F}_m\mathfrak{K}^m_\Theta$, the balls that cover $[0, 1]^m$ and the balls that cover $\mathcal{H}^m\times\Theta^m$.

Notice that $Q_{1, n}$ and $Q_{1, n}$ might be treated in the same way, so, we restrict our attention to $Q_{1, n}$. Write:

Note that the array $\left\{\Phi_{i, n}(u, x)\right\}$ is $\alpha$-mixing for each fixed $(u, x)$ with mixing coefficients $\beta_{\Phi, n}$ such that $\beta_{\Phi, n}(k) \leq \beta(k)$. We apply Lemma A.4 with

and $b_{n} = C \tau_{n}$for sufficiently large $C > 0$ and $S_n = \alpha_n^{-1} \tau_{n}^{-1}$. As same as [133, Theorem 2], we can see that $\sigma_{S_{n}, n}^{2} \leq C^\prime S_n h^m \phi^m(h_n)$, and we obtain:

To get the last inequality, we must choose $M > C^\prime$. Since $N \leq C h^{-m}\phi(h_n) \alpha_{n}^{-m}$, it follows that

with

For $M$ sufficiently large, we can see that $R_{1 n} \leq n^{-\varsigma}$ for some small $\varsigma > 0$. As $\frac{\phi_{n} \log T}{T^{\theta} h^{d+1}} = o(1)$ by assumption, we further get that

Under Assumption 4 ⅱ), we establish that $R_{2n}$ tends to zero, confirming the obtained result. Now, let's move on to the nonlinear segment of the Hoeffding decomposition. Here, the goal is to illustrate that

The conclusive phase in establishing Proposition 3.1 involves leveraging Lemma 8.2 to demonstrate the convergence of the nonlinear term to zero. □

Proof of Theorem 3.3. Equation (4.1) in Section 4 shows that

where

The proof of this theorem is intricate and divided into the following four steps; wherein each step, our objective is to demonstrate that:

Step 1.

Step 2.

Step 3.

Step 4.

Step 1 is an immediate consequence of Proposition 3.1. The validity of the second step is ensured by substituting $\varphi(Y_{i_1}, \ldots, Y_{i_m})$ with

and applying Proposition 3.1. We will now proceed with the demonstration of Step 4. Consider

where

For $W\equiv 1$, the preceding proposition established that

Therefore, it is evident that

Furthermore, we have

The final outcome arises from the Lipschitz continuity of $K_2(\cdot)$ (as stipulated in Assumption 2, ⅰ)), along with the utilization of Assumption 1 ⅰ) and Lemma A.2). This holds uniformly in $\mathbf{u}$. Furthermore,

uniformly in $\mathbf{u}$. Then, we obtain

Take $K_0:[0, 1]\rightarrow \mathbb{R}$ to be a Lipschitz continuous function with its support in $[0, q]$ for some $q > 1$, and ensure that $K_0(x) = 1$ for all $x \in [0, 1]$. Importantly,

where $Q_i$ can be defined as follows

such that

Observe that

Utilizing Assumption 3 (ⅰ), the properties of $r(\boldsymbol{\theta}, \mathbf u, \mathbf x)$ allow us to establish that

With Assumption 2, part ⅱ) in mind, we will employ Lemma 8.1 and Eq (8.55) to verify that:

Likewise, it can be observed that

and

Concerning the final term, we derive

By utilizing Lemma A.1 along with the inequality (2.12) and considering Assumption 1, it becomes evident that

Keep in mind

we deduce that

Certainly, within the framework of our assumptions, the approximation error can be considered as

With this inequality, the proof is concluded. □

Proof of Theorem 4.1. The goal is to establish weak convergence, encompassing finite-dimensional convergence and asymptotic equicontinuity, for the stochastic $U$-process:

across all the specified function classes within the framework. According to [64, Section 4.2], finite-dimensional convergence asserts that every finite set of functions $f_1, \ldots, f_q$ in $L_2$

convergences to the corresponding finite-dimensional distributions of the process $G_p$. By leveraging Cramér-Wold and considering the countability of various classes, we can simplify the scenario from the weak convergence of the $U$-process to the weak convergence of $U$-statistics with the kernel $f_r$ for all $r \in \{1, \ldots, q\}$. As the $U$-process is a linear operator, our focus narrows down to demonstrating the convergence of $\sqrt{nh^m\phi^{1/m}_{\mathbf x, \boldsymbol\theta}(h_n)} D_{n}(f_r)$ towards a Gaussian distribution. Therefore, for a fixed kernel, we have:

where

We begin treating each term. For this sake, we will calculate the variance of $\widehat{G}(\boldsymbol{\theta}, \mathbf u, \mathbf x)$. Take

Observe that

Looking on $V_1$, we have

The last part of (8.23) follows from the smoothness assumption on $r^{(m)}$ in 3 ⅰ)

Combining the latter inequalities with Eqs (8.60) and (8.58) from Lemma 8.1 where:

to get

A deep sight into the work of ^[10], specially Lemma 2, makes us see $V_2$ as follows:

where

and $j_{1} = i_{2}-i_{1}$, $j_{l} = \min \left(i_{2 l-1}-i_{2 l-2}, i_{2 l}-i_{2 l-1}\right)$ for $2 \leq l \leq m-1$, and $j_{m} = i_{2 m}-i_{2 m-1}$. If we designate $j_{1} = \max \left(j_{1}, \ldots, j_{m}\right)$, we can align the original sequence $\left\{X_{1}, \ldots, X_{n}\right\}$ with another sequence characterized by independent blocks $\{i_{1}, i_{2}, \ldots, i_{2 m}\}$ while preserving the identical block distribution. It is straightforward to demonstrate now that

This indicates the quadratic-mean convergence of $\widehat{G}(\boldsymbol{\theta}, \mathbf u, \mathbf x)$ with the specified rate as follows:

Let's remember that

then

In the next step, we will consider the first part of the last equation.

The intricate relationship between weak convergence and $\widehat{g}_1$ is further substantiated. To elucidate this connection, we undertake the following steps:

(1) Truncation of the function $\widehat{g}_1$ is performed, given the unbounded nature of the function class.

(2) The convergence of the remainder term resulting from truncation to zero is established.

(3) Hoeffding's decomposition is applied to the truncated part.

(4) The convergence to zero of the non-linear term in this decomposition is validated.

(5) The weak convergence to the linear term is established by demonstrating finite-dimensional convergence and asymptotic equicontinuity.

These steps closely parallel the proof strategy employed in Theorem 4.2. Consequently, the proof is concluded. □

Proof of Theorem 4.2. Keep in mind that

Let's define

and the $U$-empirical process is defined to be

Subsequently, we have

To ensure the weak convergence of our estimator, it is essential to first establish it for $\mu_n(\varphi, \mathbf{i})$. As mentioned earlier, dealing with unbounded function classes necessitates truncating the function $\mathfrak G_{\varphi, \mathbf{i}}(\textbf{x}, \textbf{y})$. Specifically, for $\lambda_n = n^{1/\zeta}$, where $\zeta > 2$, we have:

We can write the $U$-statistic as follows :

The first term is the truncated part and the second is the remaining one. We have to prove that:

(1) $\mu_n^{(T)}(\varphi, \mathbf{i})$ converges to a Gaussian process.

(2) The remainder part is negligible, in the sense that

For the initial step, we will utilize the Hoeffding decomposition, akin to the one presented in the previous Subsection 3.1, with the sole modification of replacing ${\mathfrak W}_{\mathbf{i}, n}$ with $\varphi(Y_{\mathbf{i}, n})$

where

The convergence of $\mathfrak{U}_{2, n}(\varphi, \mathbf{i})$ to zero in probability has been established by Lemma 8.2. Therefore, it suffices to demonstrate the weak convergence of $\mathfrak{U}_{1, n}(\varphi, \mathbf{i})$ to a Gaussian process denoted as $\mathbb{G}(\varphi)$. To accomplish this, we will proceed with finite-dimensional convergence and equicontinuity. Finite-dimensional convergence asserts that for every finite set of functions $f_1, \ldots, f_q$ in $L_2$, where $\tilde{\mathfrak{U}}$ represents the centered form of ${\mathfrak{U}}$:

convergences to the corresponding finite-dimensional distributions of the process $\mathbb{G}(\varphi)$. We only need to demonstrate that for every fixed collection $(a_1, \ldots, a_k) \in \mathbb{R}$, we have

where

Take

By linearity of $\Psi(\cdot)$, we have to see that $\tilde{\mathfrak{U}}_{1, n}(\Psi, \mathbf{i}) \rightarrow \mathbb{G}(\Psi)$. Let us denote

We have

Now, we shall employ the blocking procedure for this empirical process. We intend to partition the set $\{1, \ldots, n\}$ into $2\nu_n+1$ subsets, each containing small and large blocks. In alignment with the notation used in Lemma 8.2, we denote the size of large blocks as $a_n$ and the size of small blocks as $b_n$, satisfying:

In this case, we can see that :

where

First, we aim to prove that

to show that the case of summation over the small blocks and the summation over the last one is asymptotically negligible. Hence, we infer

We have :

Thus, we have

and

where $\lambda_i = j(a_n+b_n)+a_n$, but for $j \neq k, \vert \lambda_i- \lambda_{i^\prime} +l_1-l_2\vert \geq b_n$, then

here, the use of Davydov's lemma (Lemma A.7) is necessary, we have

it follows that

where the last inequality follows also from (8.36) and the size of $b_n$. Then, (8.41) and (8.42) shows us that

Using the same footsteps, we find that

By (8.36), we find that

Now, we need to establish that the summands in $\tilde{ \mathfrak{U}}^{(1)}_{1, n}$ are asymptotically independent, allowing us to apply the conditions of Lindeberg-Feller for asymptotic finite normality. We can utilize Lemma A.8, where $\widehat{ \mathfrak{U}}^{(1)}_{a, n}$ is $\mathcal{F}_{i_a}^{j_a}$-measurable with $i_a = a(a_n +b_n)+1$ and $j_a = a(a_n+b_n)+a_n$, giving us

which tends to zero using (8.36), then the asymptotic independence is achieved. We can see also that

Up to this point, we have addressed the final condition for finite-dimensional convergence. It is important to observe that, for sufficiently large $n$, the set

is empty, thus

Therefore, we conclude the proof of finite-dimensional convergence. Now, we shift our focus to asymptotic equicontinuity, aiming to demonstrate that:

where

for

Now, we will employ the chaining technique from ^[13] and follow the approach outlined in ^[39] for the conditional setting. The fundamental idea is to decompose a sequence $({ X}_1, \ldots, { X}_n)$ into $2\upsilon_n$ equal-sized blocks, each of length $a_n$, and a residual block of length $n-2\upsilon_na_n$, where, for $1\leqslant j\leqslant \upsilon_n$:

The values of $\upsilon_n, a_n$ are given in the following. Another ingredient is essential, in this proof, that is a sequence of independent blocks $({\zeta}_1, \ldots, {\zeta}_n)$ such as:

In the same line as ^[39], the results of the work of ^[73] on $\beta$-mixing are applied, and get, for any measurable set $A$:

We will focus solely on the independent blocks represented as $\zeta_i = (\eta_i, {\varsigma}_i)$ sequences, rather than dealing with the dependent variables. We will utilize a strategy akin to the one employed in Lemma 8.2 to transition from the sequence of locally stationary random variables to the stationary one:

We choose

and

Exploiting condition (ⅴ) from Assumption 4, we deduce that $(\upsilon_n-1)\beta_{a_n}\longrightarrow 0$ as $n\rightarrow \infty$. This primarily pertains to the first term on the right-hand side of (8.51). Given the independence of the blocks, we symmetrize the sequence using a set $\{\epsilon_j\}_{j\in \mathbb{N}^*}$ of i.i.d. Rademacher variables, where

It is noteworthy that the sequence $\{\epsilon_j\}_{j\in \mathbb{N}^*}$ is independent of the sequence $\left\{\boldsymbol{\xi}_i = ({\varsigma}_i, {\zeta}_i)\right\}_{i \in \mathbb{N}^*}$. Now, our objective is to establish, for all $\epsilon > 0$,

Define the semi-norm:

and the covering number defined for any class of functions $\mathscr{E}$ by :

Building upon the preceding discussion, we can bound (8.52), with further elaboration available in ^[39]. Similarly, by following the methodology in ^[39] and referring back to ^[13], the independence between the blocks, coupled with Assumption 6 ⅱ) and the utilization of [89, Lemma 5.2], ensures equicontinuity, thus laying the foundation for weak convergence. Now, our task is to establish that:

In this proof, for clarity, we present the case where $m = 2$, with different sizes for $a_n$ and $b_n$, where $b_n$ denotes the size of the alternate blocks. Both $a_n$ and $b_n$ satisfy:

and set, for $1 \leqslant j \leqslant v_{n}-1:$

We have:

We will employ blocking arguments and address the resulting terms. Let's begin by examining the first term $\rm I^{\prime}$. We have:

Notice that $\upsilon_n \beta_{b_n}\to 0$ and recall that for all $\varphi \in \mathscr{F}_m,$ and :

Hence, by the symmetry of $F(\cdot)$:

We are going to use Chebyshev's inequality, Hoeffding's trick and inequality, respectively to obtain:

Under Assumption 6 ⅲ), we have for each $\lambda > 0:$

which tends to $0$ as $n\to\infty$. Terms $\rm II^{\prime}$, $\rm V^{\prime}$, and $\rm VI^{\prime}$ will be handled similarly to the previous term. However, $\rm II^{\prime}$ and $\rm VI^{\prime}$ deviate from this pattern due to the variables $\{{\zeta}_i, {\zeta}_j\}_{i, j \in H_p^{(U)} }$ (or $\{{\zeta}_i, {\zeta}_j\}_{i, j \in T_p^{(U)} }$ for $\rm VI^{\prime}$) belonging to the same blocks. Regarding term $\rm IV^{\prime}$, its analysis can be inferred from the study of terms $\rm I^{\prime}$ and $\rm III^{\prime}$. Considering term $\rm III^{\prime}$, we have:

We also have

Since the Eq (8.55) is still satisfied, the problem is reduced to

we follow the same procedure as in (8.56). The rest has just been shown to be asymptotically negligible.

Finally, with

and for

the weak convergence of our estimator is accomplished. □

Technical lemmas

The forthcoming proof relies on the arguments delineated in ^[39,40,165], extended to the single index model framework.

Lemma 8.1. Let $K_2(\cdot)$ denote one dimensional kernel function satisfying Assumption 2 part i), if Assumption 1, then:

Proof of Lemma 8.1. For the first inequality $i)$, by assuming that the kernel function $K_2(\cdot)$ is an asymmetrical triangle kernel, that is, $K_2(x) = (1 -x) \mathbb{1}_{(x \in [0, 1])}$, we have

For the second inequality $ii)$, we have:

By linearity of the expectation, inequality $i)$ and for

using Assumption 1 part ⅳ), the proof of this inequality holds. Now, we consider the last one. Set

We have

The final result is established by utilizing the small-ball lower bound provided in (2.11). Consequently, inequality (8.60) follows. □

In the ensuing discussion, we will present a lemma that can be regarded as a technical result in the proof of our proposition.

Lemma 8.2. Consider $\mathscr{F}_{m}\mathfrak{K}^m_\Theta$ as a uniformly bounded class of measurable canonical functions, where $m\geq 2$. Suppose there exist finite constants $\boldsymbol{a}$ and $\boldsymbol{b}$ such that the $\mathscr{F}_{m}\mathfrak{K}^m_\Theta$ covering number satisfies:

for every $\epsilon > 0$ and every probability measure $Q$. If the mixing coefficients $\beta$ of the local stationary sequence $\{Z_i = (X_{\mathbf{i}, n}, {\mathfrak W}_{\mathbf{i}, n})\}_{i \in \mathbb{N}^\star}$ satisfy:

for some $r > 1$, then:

Remark 8.3. As mentioned before, ${\mathfrak W}_{\mathbf{i}, n}$ will be equal to $1$ or $\varepsilon_{\mathbf{i}, n} = \sigma\left(\frac{\mathbf{i}}{n}, X_{\mathbf{i}, n}\right)\varepsilon_{\mathbf{i}}$. In the proof of the previous Lemma, ${\mathfrak W}_{\mathbf{i}, n}$ will be equal $\varepsilon_{\mathbf{i}, n} = \sigma\left(\frac{\mathbf{i}}{n}, X_{\mathbf{i}, n}\right)\varepsilon_{\mathbf{i}}$, and we will use the notation $\mathfrak{W}^{(u)}_{\mathbf{i}, n}$ to indicate $\sigma\left(\mathbf{u}, \mathbf{x}\right)\varepsilon_{\mathbf{i}}$

Proof of Lemma 8.2. The proof of this lemma relies on the blocking method, specifically drawing upon techniques introduced by ^[13]. The central idea involves partitioning the strictly stationary sequence $(Z_1, \ldots, Z_n)$ into $2n$ blocks, each of length $a_n$, along with a residual block of length $n-2v_n a_n$. This approach, known as Bernstein's method and discussed in ^[22], facilitates the application of symmetrization and various techniques designed for i.i.d. random variables. To establish the independence between the blocks, the smaller blocks are placed between two consecutive larger blocks, and their contribution should be asymptotically negligible. Next, introduce the sequence of independent blocks $\left(\eta_{1}, \ldots, \eta_{n}\right)$ such that:

An application of the result of ^[73] implies that for any measurable set $A$:

Since we are working with a locally stationary sequence $(X_1, \ldots, X_n)$, the sequence of independent blocks used subsequently is denoted by $\{\eta_i\}_{i\in \mathbb{N}^*}$. We decompose the process based on the distribution of these blocks:

(I): The same type of block but not the same block: Assume that the sequence of independent blocks $\{\eta_i\}_{i\in \mathbb{N}^*}$ is of size $a_n$. An application of (8.61) shows that:

By the fact that

and

We keep the choice of $b_n$ and $\upsilon_n$ such that

which implies that $2\upsilon_n \alpha_{b_n} \to 0$ as $n \to \infty$, so the term to consider is the second summand. For the second part of the inequality, we turn to the work of ^[11] in the non-fixed kernel setting. Specifically, we define

and $\mathcal{F}_{i_{1}, \ldots, i_m}$ represents a collection of kernels and the corresponding class of functions associated with this kernel. Subsequently, we will apply [64, Theorem 3.1.1 and Remarks 3.5.4 part 2] for decoupling and randomization. Given our assumption that $m = 2$, we can observe that:

where $D_{nh}^{(U_1)}$ is the diameter of $\mathcal{F}_{i_1, i_2}$according to the distance $\widetilde{d}_{nh, 2}^{(1)},$ respectively defined as

and :

Let's consider another semi-norm $\widetilde{d}_{nh, 2}^{(2)} :$

One can see that

We readily infer that

where $\lambda_n \to 0.$ We have

where $a_n$ and $\lambda_n$ must be chosen in such a way that the following relation will be achieved

Utilizing the triangle inequality along with Hoeffding's trick, we readily obtain that

where $\left\{{\eta}^\prime_i\right\}_{i \in \mathbb{N}^*}$ are independent copies of $({\eta}_i)_{i \in \mathbb{N}^*}$. By imposing:

we readily infer that

Symmetrizing the last inequality in (8.72) and subsequently applying Proposition A.6 from the Appendix yields

where

and for $\xi_{1\boldsymbol{.}} {K}_{2, 1}\mathfrak{W}^\prime\; , \; \xi_{2\boldsymbol{.}} {K}_{2, 2}\mathfrak{W}^{\prime\prime} \in \mathcal{F}_{ij}$ :

By the fact that:

so:

we have the convergence of (8.74) to zero. For the choice of $a_n,$ $b_n$ and $\upsilon_n$, it should be noted that all the values satisfying (8.54), (8.68), (8.71), (8.73), and (8.74) are suitable.

(II): The same block:

Similar to $\rm I$, we can show that both the first and second terms in the previous inequality are of order $o_ \mathbb{P}(1)$. Therefore, as in the preceding proof, it is enough to establish

Notice that when we consider a uniformly bounded class of functions, we obtain uniformity in $B^m \times \mathscr{F}_2\mathscr{K}^2$

This implies that we have to prove that, for $\mathbf{u} \in B^m$

As for empirical processes, to prove (8.77), it suffices to symmetrize and show that

In a similar way as in (8.69), we infer that :

where

and the semi-metric $\widetilde{d}_{nh, 2}^{(3)}$ is defined by

Since we are trading uniformly bounded classes of functions, we infer that

Since $a_n^{3/2}(n)^{-1}\phi^{-1}(h_n)\to 0$, $D_{nh}^{(U_3)}\to 0$, we obtain $\rm{II}\to 0$ as $n \to \infty$.

(III): Different types of blocks:

As mentioned earlier, we have addressed the first and second summands in the previous inequality. What remains is the last summation, where the application of (8.61) reveals that

we have

using Condition (8.63) and the choice of $a_n$, $b_n$ and $\upsilon_n$. For $p = 1$ and $p = \nu_n$:

For $2\leqslant p\leqslant \upsilon_n-1$, we obtain:

therefore it suffices to treat the convergence:

Using similar arguments as in ^[13], we apply the standard symmetrization and:

where

In a similar way as in (8.69), we infer that

where

Since we have

and considering the semi-metric

We show that the expression in (8.81) is bounded as follows

by choosing $\gamma_n = n^{-\alpha}$ for some $\alpha > (17r-26)/60r$, we get the convergence to zero of the previous quantity. To bound the second term on the right-hand side of (8.79), we observe that

Now, we apply the square root trick to the last expression conditional on $H_{1}^{U}$. Denoting $\mathbb{E}_T$ as the expectation with respect to $\sigma\{{\eta}_{i_2}: i_2 \in T_q, q\geqslant 3\}$, we assume that any class of functions $\mathscr{F}_m$ is unbounded, and its envelope function satisfies, for some $p > 2$:

for $2r/(r-1) < s < \infty$, (in the notation in of [89, Lemma 5.2]).

where

and

Nevertheless, since we require $t > 8M_n$ and $m \to \infty$, using similar arguments as in [13, page 69], we achieve the convergence of (8.81) and (8.82) to zero.

(IV): Different types of blocks: The target here is to prove that:

We have

Hence the proof of the lemma is complete. □

Use of AI tools declaration

The authors declare they have not used Artificial Intelligence (AI) tools in creating this article.

Acknowledgements

The authors would like to thank the Editor-in-Chief, an Associate-Editor, and three referees for their extremely helpful remarks, which resulted in a substantial improvement of the original form of the work and a presentation that was more sharply focused.

Conflict of interest

The author declares that he has no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix

A.   Additional information

This appendix includes additional information that is integral to achieving a more comprehensive understanding of the paper.

Lemma A.1 (^[165]). Let $I_{h} = \left[C_{1} h, 1-C_{1} h\right]$. Suppose that kernel $K_{1}$ satisfies Assumption 2 part(i). Then, for $q = 0, 1, 2$ and $m > 1$:

Lemma A.2 (^[165]). Suppose that kernel $K_{1}$ satisfies Assumption 2 part (i) and let $g:[0, 1] \times \mathscr{H} \rightarrow \mathbb{R}$, $(\mathbf{u}, \mathbf{x}) \mapsto g(\mathbf{u}, \mathbf{x})$ be continuously differentiable with respect to $\mathbf{u}$. Then,

Lemma A.3 (^[124]). Let $\left\{Z_{i, n}\right\}$ be a zero-mean triangular array such that $\left\lvert Z_{i, n}\right\rvert \leq b_{n}$ with $\alpha$-mixing coefficients $\alpha(k)$. Then, for any $\varepsilon > 0$ and $S_{n} \leq n$ with $\epsilon > 4 S_{n} b_{n}$,

Lemma A.4. Let $\left\{Z_{i, n}\right\}$ be a zero-mean triangular array such that $\left\lvert Z_{i, n}\right\rvert \leq b_{n}$ with $\beta$-mixing coefficients $\beta(k)$. Then, for any $\varepsilon > 0$ and $S_{n} \leq n$ with $\epsilon > 4 S_{n} b_{n}$,

Proof of Lemma A.4. Using Lemma A.3 and the fact that for any $\sigma$-algebra $\mathcal{A}$ and $\mathcal{B}$, $\alpha(\mathcal{A}, \mathcal{B}) \subseteq \beta(\mathcal{A}, \mathcal{B})$, Lemma A.7 holds. □

Lemma A.5 (^[63]). Let $X_{1}, \ldots, X_{n}$ be a sequence of independent random elements taking values in a Banach space $(B, \|\cdot\|)$ with $\mathbb E X_{i} = 0$ for all $i$. Let $\left\{\varepsilon_{i}\right\}$ be a sequence of independent Bernoulli random variables independent of $\left\{X_{i}\right\}$. Then, for any convex increasing function $\Phi$,

Proposition A.6 (^[11]). Let $\{{X}_i : i \in {n}\}$ be a process satisfying, for $m \geq 1$:

and the semi-metric :

There exists a constant $K = K(m)$ such that :

where $D$ is the $\rho$-diameter of ${n}$.

Lemma A.7 (^[61]). Suppose that $X$ and $Y$ are random variables which are $\mathscr{G}$ and $\mathscr{H}$-measurable, respectively, and that $\mathbb E|X|^{p} < \infty, \mathbb E|Y|^{q} < \infty$, where $p > 0$,

Then,

Proof of Lemma A.7. This Lemma follows directly using Lemma A.7 and the fact that for any $\sigma$-algebra $\mathcal{A}$ and $\mathcal{B}$, $\alpha(\mathcal{A}, \mathcal{B}) \subseteq \beta(\mathcal{A}, \mathcal{B})$.□

Lemma A.8 (^[180]). Let $V_{1}, \ldots, V_{L}$ be strongly mixing random variables measurable with respect to the $\sigma$-algebras $\mathscr{F}_{i_{1}}^{j_{1}}, \ldots, \mathscr{F}_{i_{L}}^{j_{L}}$ respectively with $1 \leq i_{1} < j_{1} < i_{2} < \cdots < j_{L} \leq n, i_{l+1}-j_{l} \geq w \geq 1$ and $\left\lvert V_{j}\right\rvert \leq 1$ for $j = 1, \ldots, L$. Then,

where $\alpha(w)$ is the strongly mixing coefficient.

A1.   Examples of classes of functions

Example A.9. The set $\mathscr{F}$ of all indicator functions ${\rm 1\!I}_{\{(-\infty, t]\}}$ of cells in $\mathbb{R}$ satisfies :

for any probability measure $\mathbb{P}$ and $\epsilon\leq 1$. Notice that :

For more details and discussion on this example refer to Example 2.5.4 of ^[178] and [114, p. 157]. The covering numbers of the class of cells $(-\infty, t]$ in higher dimension satisfy a similar bound, but with higher power of $(1/\epsilon)$, see Theorem 9.19 of ^[114].

Example A.10. (Classes of functions that are Lipschitz in a parameter, Section 2.7.4 in ^[178]). Let $\mathscr{F}$ be the class of functions $x\mapsto \varphi(t, x)$ that are Lipschitz in the index parameter $t\in T$. Suppose that:

for some metric $d$ on the index set $T$, the function $\kappa(\cdot)$ defined on the sample space $\mathcal{X}$, and all $x$. According to Theorem 2.7.11 of ^[178] and Lemma 9.18 of ^[114], it follows, for any norm $\|\cdot\|_{{\mathscr F}}$ on $\mathscr{F}$, that :

Hence if $(T, d)$ satisfy

then the conclusions holds for $\mathscr{F}$.

Example A.11. Let us consider as an example the classes of functions that are smooth up to order $\alpha$ defined as follows, see Section 2.7.1 of ^[178] and Section 2 of ^[177]. For $0 < \alpha < \infty$ let $\lfloor \alpha \rfloor$ be the greatest integer strictly smaller than $\alpha$. For any vector $k = (k_{1}, \ldots, k_{d})$ of $d$ integers define the differential operator

Then, for a function $f:\mathcal{X}\rightarrow \mathbb{R}$, let

where the suprema are taken over all $x, y$ in the interior of $\mathcal{X}$ with $x \neq y$. Let $C_{M}^{\alpha}(\mathcal{X})$ be the set of all continuous functions $f: \mathcal{X}\rightarrow \mathbb{R}$ with

Note that for $\alpha \leq 1$ this class consists of bounded functions $f$ that satisfy a Lipschitz condition. ^[112] computed the entropy of the classes of $C_{M}^{\alpha}(\mathcal{X})$ for the uniform norm. As a consequence of their results, ^[177] shows that there exists a constant $K$ depending only on $\alpha, d$ and the diameter of $\mathcal{X}$ such that for every measure $\gamma$ and every $\epsilon > 0$,

${\mathcal N}_{[\; ]}$ is the bracketing number, refer to Definition 2.1.6 of ^[178] and we refer to Theorem 2.7.1 of ^[178] for a variant of the last inequality. By Lemma 9.18 of ^[114], we have

A2.   Examples of U-kernels

In this section, we present some classical $U$-kernels.

Example A.12. ^[99] introduced the parameter

where $D(y_1, y_2) = F(y_1, y_2)-F(y_1, \infty) F(\infty, y_2)$ and $F(\cdot, \cdot)$ is the distribution function of $Y_1$ and $Y_2$. The parameter $\triangle$ has the property that $\triangle = 0$ if and only if $Y_1$ and $Y_2$ are independent. From ^[117], an alternative expression for $\triangle$ can be developed by introducing the functions

and

We have

Example A.13. (Hoeffding's $D$). From the symmetric kernel,

we recover Hoeffding's $D$ statistic, a rank-based $U$-statistic of order 5, and gives rise to Hoeffding's $D$ correlation measure $\mathbb{E} h_{D}$.

Example A.14. (Blum-Kiefer-Rosenblatt's $R$). The symmetric kernel

yields Blum-Kiefer-Rosenblatt's $R$ statistic (^[24]),

Example A.15. (Bergsma-Dassios-Yanagimoto's $\tau^{*}$). ^[21] introduced a rank correlation statistic as a $U$-statistic of order 4 with the symmetric kernel

Here, $\mathbb{1}\left(y_{1}, y_{2} < y_{3}, y_{4}\right): = \mathbb{1}\left(y_{1} < y_{3}\right) \mathbb{1}\left(y_{1} < y_{4}\right) \mathbb{1}\left(y_{2} < y_{3}\right) \mathbb{1}\left(y_{2} < y_{4}\right).$

Example A.16. The Wilcoxon Statistic. Suppose that $E \subset \mathbb{R}$ is symmetric around zero. As an estimate of the quantity

it is pertinent to consider the statistic

which is relevant for testing whether or not $\mu$ is located at zero.

Example A.17. The Takens estimator. Denote by $\|\cdot\|$ the usual Euclidean norm on $\mathbb{R}^d$. In ^[29], the following estimate of the correlation integral,

is considered:

In the case where a scaling law holds for the correlation integral, i.e., when there exists $\left(\alpha, r_0, c\right) \in$ $\mathbb{R}_{+}^{* 3}$ such that $C_F(r) = c \cdot r^{-\alpha}$ for $0 < r \leq r_0$, the $U$-statistic

is used in order to build the Takens estimator $\hat{\alpha}_n = -T_n^{-1}$ of the correlation dimension $\alpha$.

Example A.18. Let $\widehat{{Y}_1{Y}_2}$ denote the oriented angle between ${Y}_1, {Y}_2 \in T$, $T$ is the circle of radius 1 and center $0$ in $\mathbb{R}^{2}$. Let:

Reference ^[159] has used this kernel in order to propose a $U$-process to test uniformity on the circle.

Example A.19. For $m = 3$, let :

We have

and the corresponding conditional $U$-Statistic can be considered a conditional analog of the Hollander-Proschan test-statistic (^[101]). It may be used to test the hypothesis that the conditional distribution of $Y_1$ given ${X}_1 = { t}$, is exponential, against the alternative that it is of the New-Better than-Used-type.

Example A.20. The Gini mean difference. The Gini index provides another popular measure of dispersion. It corresponds to the case where $E \subset \mathbb{R}$ and $h(x, y) = |x-y|$ :

Example A.21 (^[113]). Let the sample central moments of any order $m = 2, 3, \ldots$ be given by

In this case, the $U$-statistic has a symmetric kernel

where summation is carried out over all permutations $\left(i_1, \ldots, i_m\right)$ of the numbers $(1, \ldots, m)$. In particular, if $m = 3$, then

In the case of $m = 2$,

For the kernel

the corresponding $U$-statistic is the sample variance

we refer also to ^[155].