Limit theorems for nonparametric conditional U-statistics smoothed by asymmetric kernels

1.   Introduction

Motivated by numerous applications, the theory of $U$-statistics (introduced in the seminal work by ^[80] and ^[71]) and $U$-processes has garnered considerable attention over the past decades. $U$-statistics are instrumental in solving complex statistical problems such as density estimation, nonparametric regression tests, and goodness-of-fit tests. Specifically, $U$-statistics are essential in analyzing estimators (including function estimators) with varying degrees of smoothness. For instance, Stute in ^[135] applied almost sure uniform bounds for $\mathbb{P}$-canonical $U$-processes to analyze the product limit estimator for truncated data. In ^[8], Arcones and Wang presented two novel tests for normality based on $U$-processes. Leveraging results from ^[68], Schick et al. in ^[126] introduced new normality tests using weighted $L_{1}$-distances between the standard normal density and local $U$-statistics based on standardized observations. In ^[87], Joly and Lugosi discussed estimating the mean of multivariate functions under possibly heavy-tailed distributions and introduced the median-of-means approach based on $U$-statistics. $U$-processes are crucial tools in various statistical applications, including testing for qualitative features of functions in nonparametric statistics ^[67], cross-validation for density estimation, and establishing the limiting distributions of $M$-estimators (see, e.g., ^[7]). Foundational asymptotic results for $U$-statistics under the assumption of independent and identically distributed (i.i.d.) random variables were provided by ^[71,80,146] and ^[59]. Under weak dependency assumptions, asymptotic results were demonstrated in ^[18,48]. For a comprehensive resource on $U$-statistics and $U$-processes, readers may refer to ^{[7,19,92,98,100]}, and ^[46]. Recent advancements and references can be found in ^[30,31]. $U$-statistics also appear naturally in other contexts, such as counting occurrences of certain subgraphs (e.g., triangles) in random graph theory ^[85]. In machine learning, $U$-statistics are utilized in a wide range of problems, including clustering, image recognition, ranking, and learning on graphs, where risk estimates often take the form of $U$-statistics. For instance, the ranking problem can be framed as pairwise classification, with the empirical ranking error being a $U$-statistic of order 2 ^[37]. For $U$-statistics with random kernels of diverging orders, see ^[60,75,131]. Infinite-order $U$-statistics are valuable for constructing simultaneous prediction intervals that quantify the uncertainty of ensemble methods such as sub-bagging and random forests ^[118]. The Mean nearest-neighbors approach for differential entropy estimation introduced by ^[58] is a specific application of $U$-statistics. Using $U$-statistics, ^[107] proposed a new test statistic for goodness-of-fit tests. In ^[45], Cybis et al. explored a model-free approach for clustering and classifying genetic data using $U$-statistics, offering alternative perspectives on these problems. In ^[103], Lim and Stojanovic proposed employing $U$-statistics for analyzing random compressed sensing matrices in the non-asymptotic regime. In ^[13], Bello et al. introduced a comprehensive framework for clustering within multiple groups using a $U$-statistics-based approach designed for high-dimensional datasets. In a related context in ^[91], Kim and Ramdas focused on dimension-agnostic inference, developing methods whose validity remains independent of assumptions regarding dimension versus sample size. Their approach utilized variational representations of the existing test statistics, incorporating sample splitting and self-normalization to yield a refined test statistic with a Gaussian limiting distribution. This involved modifying degenerate $U$-statistics by dropping diagonal blocks and retaining off-diagonal blocks. Further exploration by ^[39] involved $U$-statistics-based empirical risk minimization, while in ^[86], Janson examined asymmetric $U$-statistics based on a stationary sequence of $m$-dependent variables, with applications motivated by pattern matching in random strings and permutations. Additionally in ^[139], Sudheesh et al. developed innovative $U$-statistics considering left truncation and right censoring, proposing a straightforward nonparametric test for assessing independence between time to failure and cause of failure in competing risks under such censoring conditions. In ^[95], Le Minh investigated the quadruplet $U$-statistic with applications in network analysis statistical inference. In ^[102], Li et al. introduced a learning framework utilizing pairwise loss and minimizing empirical risk through $U$-processes and Rademacher complexity. In ^[66], Ghannadpour et al. demonstrated the high efficacy of using $U$-statistics to identify modification zones, accounting for the structural characteristics of the data and neighboring samples. In ^[44], Cintra et al. presented a model-free technique using $U$-statistics to construct control charts for effectively monitoring batch processes, considering multiple sources of variability. In ^[81], Huang et al. examined distributed inference for two-sample $U$-statistics in the context of large datasets, proposing blockwise linear two-sample $U$-statistics to reduce computational complexity. Finally in ^[122], Randles proposed a general method for obtaining asymptotic distribution theory for $U$-statistics with estimated parameters, extended recently by ^[50].

In this paper, we consider the conditional $U$-statistics introduced by ^[134]. These statistics generalize the Nadaraya-Watson estimators of a regression function by ^[112] and ^[150]. Specifically, let $\{(\mathbf{X}_i, \mathbf{Y}_i), i \in \mathbb{N}^*\}$ be a sequence of i.i.d. random elements with $\mathbf{X}_i \in \mathbb{R}^d$ and $\mathbf{Y}_i \in \mathbb{R}^q$, where $d, q \geq 1$. Let $\varphi: \mathbb{R}^{qm} \rightarrow \mathbb{R}$ be a measurable function. We focus on estimating the conditional expectation or regression function

whenever it exists, i.e., $\mathbb{E}(|\varphi(\mathbf{Y}_1, \ldots, \mathbf{Y}_m)|) < \infty$. We introduce a kernel function $K: \mathbb{R}^d \rightarrow \mathbb{R}$ with support contained in $[-B, B]^d$, $B > 0$, satisfying

In ^[134], Stute introduced a class of estimators for $r^{(m)}(\varphi, \mathbf{t})$, called conditional $U$-statistics, defined for each $\mathbf{t} \in \mathbb{R}^{dm}$ as

where

is the set of all $m$-tuples of distinct integers between $1$ and $n$, and $\{h_n\}_{n \geq 1}$ is a sequence of positive constants converging to zero at the rate $nh_n^{dm} \rightarrow \infty$. For $m = 1$, $r^{(m)}(\varphi, \mathbf{t})$ reduces to $r^{(1)}(\varphi, \mathbf{t}) = \mathbb{E}(\varphi(\mathbf{Y}) \mid \mathbf{X} = \mathbf{t})$, and Stute's estimator becomes the Nadaraya-Watson estimator of $r^{(1)}(\varphi, \mathbf{t})$ given by

In ^[128], Sen gave the rate of uniform convergence in $\mathbf{t}$ of $\widehat{\widehat{r}}_n^{(m)}(\varphi, \mathbf{t}; h_n)$ to $r^{(m)}(\varphi, \mathbf{t})$. In ^[121], Rao and Sen discussed the limit distributions of $\widehat{\widehat{r}}_n^{(m)}(\varphi, \mathbf{t}; h_n)$ and compared them with those obtained by Stute. In ^[74], Harel and Puri extended the results of ^[134] to weakly dependent data under appropriate mixing conditions and applied their findings to verify the Bayes risk consistency of the corresponding discrimination rules. In ^[138], Stute proposed symmetrized nearest-neighbors conditional $U$-statistics as alternatives to the usual kernel-type estimators. In ^[53], Dony and Mason established a much stronger form of consistency, namely, uniform in $\mathbf{t}$ and bandwidth consistency (i.e., $h_n \in [a_n, b_n]$ where $a_n < b_n \rightarrow 0$ at a specific rate) of $\widehat{\widehat{r}}_n^{(m)}(\varphi, \mathbf{t}; h_n)$. Additionally, they demonstrated uniform consistency over $\varphi \in \mathscr{F}$ for a suitably restricted class of functions $\mathscr{F}$, extended in ^{[24,25,30,31,33,35]}. The main tool in their results was the use of the local conditional $U$-process investigated in ^[68]. ^[6], Arcones examined the distributional and almost sure point-wise Bahadur-Kiefer representation for $U$-quantiles, demonstrating that the order of this representation is contingent on the local variance of the empirical process of the $U$-statistic structure at the $U$-quantile. Additionally in ^[4], Arcones explored the order of the Kolmogorov-Smirnov distance for the bootstrap of $U$-quantiles. For further details on $U$-quantile estimation, readers are referred to the works of ^{[47,76,145,151,156,157,158]}.

It is well-known that using standard symmetric kernels to estimate unknown curves on a bounded support, such as on a half-real line or a compact set, leads to boundary bias near the boundary (refer to ^{[51,55,73,110,113,119,124,140,147,152]}). Despite various boundary correction methods proposed since the early works of ^[111] and ^[88] on boundary-adapted kernel estimation, smoothing with a nonstandard asymmetric kernel function has emerged as a viable alternative (see ^[62]). An asymmetric kernel, based on a probability density function (pdf) with the same support as the curves, inherently avoids boundary bias. Additionally, the kernel shape varies according to the position where smoothing is performed, allowing for adaptive smoothing. Over two decades have passed since the introduction of asymmetric kernels, and numerous studies have reported favorable evidence from their application to empirical models in economics and finance. For instance in ^[40], Chen considered kernel estimators using nonnegative kernels to estimate pdfs with compact supports. In ^[41], Chen examined regression estimation, comparing a beta smoother with a local linear smoother. In ^[20], Bouezmarni and Rolin studied the asymptotic properties of the beta kernel density estimator proposed by ^[40], deriving the exact asymptotic behavior of the expected $L_1$-loss and demonstrating uniform weak consistency for continuous underlying density functions. In ^[155], Zhang and Karunamuni showed that the performance of beta kernel estimator is very similar to that of the reflection estimator, which avoids the boundary problem only for densities with a shoulder at the support endpoints. In ^[16], Bertin and Klutchnikoff analyzed the beta kernel density estimator from an asymptotic minimax perspective, and this was later corrected and extended to the general setting of the Dirichlet kernel by ^[17]. When the underlying density has a fourth-order derivative, in ^[82], Igarashi improved the beta kernel estimator using bias correction techniques based on two beta kernel estimators with different smoothing parameters (see ^[77] for more details on asymmetric kernels and their applications). Various other statistical topics related to beta kernels are addressed in ^{[38,83,94,154]}. Additionally, Bernstein polynomial density and distribution estimators have gained popularity and have been discussed in several works. In ^[144], Vitale first proposed the Bernstein estimator for estimating a density function for an i.i.d. sample $\{X_n\}_{n \geq 1}$. The asymptotic normality of the resulting density estimator, such as the uniform weak law of large numbers, the central limit theorem, and uniform strong consistency are studied in ^[133] and ^[64]. In ^[141], Tenbusch investigated the Bernstein estimation in the multivariate context, obtaining the bias, variance, and Mean Squared Error (MSE) for the Bernstein estimator, focusing on the two-dimensional unit simplex and the unit square $[0, 1]^2$. Several extensions have been provided by ^{[2,9,12,15,89,120,142,149]}. In ^[96], Leblanc demonstrated that Bernstein estimators of distribution functions have excellent boundary properties, including the absence of boundary bias (see also ^[97]). The latest results were generalized in ^[116].

However, little is known about the uniform consistency and convergence rates of asymmetric nonlinear kernel estimators. This paper aims to fill this gap by considering a general framework for conditional $U$-statistics. This study examines the uniform convergence and the asymptotic normality of nonparametric estimators on a compact set smoothed using an asymmetric kernel. The compactness of the support frequently emerges due to the data's nature or as a theoretical concept in econometrics. Economic and financial variables described as shares or proportions are typically constrained within a specific range, with an upper bound for the former and a lower bound for the latter. Examples include the allocation of funds, the distribution of budget resources, the percentage of unemployed individuals, desired exchange rate ranges, and failure-to-repay and recovery rates. Compactness is required for certain aspects, such as the support of nonparametric copulas (see, e.g., ^[125]), the nonparametric part of partial linear regressions (see, e.g., ^[153]), and the covariates used for nearest-neighbor matching (see, e.g., ^[1]). Additionally, the fully nonparametric estimator for first-price auctions relies on the compactness of the supports of the distributions of private values and observed bids (see ^[70] for more information). In ^[3], Aitchison first defined the Dirichlet kernel estimator for density estimation and compared their performance empirically with an alternative approach, the logistic-normal kernel method, where the data on the simplex is first transformed to $\mathbb{R}^d$ via an additive log-ratio transformation, followed by multivariate Gaussian kernel smoothing. In ^[36], Brown and Chen first studied beta kernel $(d = 1)$ theoretically in the context of smoothing for regression curves with equally spaced and fixed design points. The asymptotics of the point-wise bias, the integrated variance, and the mean integrated squared error (MISE) for the estimator of the regression function were found, with the optimal MISE shown to be $O(n^{-4/5})$. These results extended to the beta kernel estimator some findings from ^[133], who worked with the closely related Bernstein estimator. In ^[40], Chen first studied the beta kernel estimator theoretically in the context of density estimation. In ^[41], Chen generalized the results of ^[36] to arbitrary collections of fixed design points using a ^[63] type estimator. In ^[20], Bouezmarni and Rolin computed the asymptotics of the expected average absolute error for the beta kernel estimator, extending analogous results for traditional kernel estimators found in Theorem 2 of ^[52]. In ^[123], Renault and Scaillet first used a beta kernel to estimate recovery rate densities of defaulted bonds. In ^[21], Bouezmarni and Rombouts generalized the results of ^[40,42] to the multidimensional setting, considering the products of one-dimensional asymmetric kernels. In ^[155], Zhang and Karunamuni showed that the performance of the beta kernel estimator is similar to that of the reflection estimator of ^[127], which avoids the boundary problem only for densities exhibiting a shoulder condition at the support endpoints. It should be noted that Bernstein density estimators, studied theoretically by authors such as ^{[11,61,65,69,78,84,105,108,109,148]}, share many of the same asymptotic properties with proper reparameterization. For an overview of the extensive literature on Bernstein estimators, see ^[115].

This paper presents a first exploration into establishing a rigorous theoretical foundation for asymmetric kernel-type estimators applied to conditional $U$-statistics, including Dirichlet kernels, multivariate beta kernels, and Bernstein polynomials. However, as we elucidate later, this challenge goes beyond merely combining ideas from existing literature. It necessitates intricate mathematical derivations to address the inherent nonlinearity characterizing conditional $U$-statistics. To the best of our knowledge, this broader contextualization remains unexplored within the scholarly domain, presenting a more formidable task compared to the Nadaraya-Watson estimator or the density estimator. Our analysis leverages Hoeffding's decomposition, a fundamental tool in the analysis of $U$-statistics. Specifically, we offer novel insights into the Dirichlet kernel regression estimator within the framework of Hoeffding's decomposition. We encounter a challenge concerning the applicability of the martingale structure to the nonlinear terms in Hoeffding's decomposition within our context. To address this, we employ exponential inequality principles drawn from the intricate theory of $U$-processes. By exploring the conditional $U$-statistics smoothed by Dirichlet kernels and Bernstein polynomial, we present several interesting new results regarding the Nadaraya-Watson estimators that are of independent interest. Furthermore, we broaden the scope of our investigation to beta kernel estimation by exploring the realm of mixed-type vectors.

The remainder of the paper is structured as follows. In Section 3, we introduce the conditional $U$-statistics estimator using Dirichlet kernel for the first time in the literature. We commence by establishing uniform convergence for the regression estimator in Section 3.1, specifically in Theorem 3.1, which holds independent significance. Extensions of this result are then presented in Section 3.2, as shown in Corollary 3.5. The asymptotic normality of the proposed estimators is explored in Section 3.3, wherein Theorem 3.6 and Corollary 3.7 are discussed. In Section 4, we consider the conditional $U$-statistics using Bernstein polynomials. Section 4.1 is devoted to the Nadaraya-Watason estimator (see Theorem 4.2 and Corollary 4.3) while Section 4.2 investigates the conditional U-statistics presenting the main results in Theorem 4.5 and Corollary 4.7. Moving forward, Section 5 unveils the estimator of conditional $U$-statistics through beta kernel smoothing. Section 5.1 outlines our findings regarding weak uniform convergence, detailed in Corollary 5.4. In Section 5.3, we establish the strong uniform convergence of the beta conditional $U$-statistics estimator, as presented in Corollary 5.8. Furthermore, in Section 5.4, we introduce the conditional $U$-statistics estimators for mixed categorical and continuous data for the first time. The main result regarding weak uniform convergence is stated in Corollary 5.9. Section 9 provides numerical experiments that illustrate the favorable small-sample properties of the proposed method. Finally, we provide concluding remarks and discuss potential future developments in Section 10. All proofs are consolidated in Section A to maintain the coherence of the presentation, while a selection of pertinent technical results is provided in the Appendix.

2.   Preliminaries and estimation procedure

Let us consider a sequence of i.i.d. random vectors $\{(\mathbf{ X}_i, \mathbf{ Y}_i), i\in \mathbb{N}^*\}$, where $\mathbf{ X}_i \in \mathcal{X}\subseteq[0, 1]^d$, and the $\mathbf{Y}_i\in \mathcal{Y}: = \mathbb{R}^q$. Let $\varphi: \mathcal{Y}^{ m}\rightarrow \mathbb{R}$ be a measurable function. In this paper, we are primarily concerned with the estimation of the conditional expectation, or regression function of $\varphi(\mathbf{ Y}_{1}, \ldots, \mathbf{ Y}_{m})$ evaluated at $(\mathbf{ X}_{1}, \ldots, \mathbf{ X}_{m}) = \tilde{\mathbf{x}}$, given by

whenever it exists, i.e., $\mathbb{E}\left(\left|\varphi(\mathbf{ Y}_{1}, \ldots, \mathbf{ Y}_{m})\right|\right) < \infty$. ^[134] presented a class of estimators for $r^{(m)}(\varphi, \tilde{\mathbf{x}})$, called the conditional $U$-statistics, which is defined for each $\tilde{\mathbf{x}}\in \mathcal{X}^m$ and $\ell \in \left\{1, 2, 3\right\}$ to be:

where $\bar{\boldsymbol{\Lambda}}_{n, \ell}(\tilde{\mathbf{x}}) = (\boldsymbol{\Lambda}_{n, \ell}(\mathbf{x}_1), \ldots, \boldsymbol{\Lambda}_{n, \ell}(\mathbf{x}_m))$ will be specified later in the sections below. In the particular case $m = 1$, the $r^{(m)}(\varphi, \tilde{\mathbf{x}})$ is reduced to $r^{(1)}(\varphi, \tilde{\mathbf{x}}) = \mathbb E(\varphi(\mathbf{ Y})|\mathbf{ X} = \mathbf{x})$ and Stute's estimator becomes the Nadaraya-Watson estimator of $r^{(1)}(\varphi, \tilde{\mathbf{x}})$ is given by

Throughout this paper, any multivariate point will be written in bold. To avoid confusion, we note $\mathbf{x} = (x_1, \ldots, x_d)$ for $\mathbf{x} \in \mathcal{X}$ and we denote $\tilde{\mathbf{x}}: = (\mathbf{x}_1, \ldots, \mathbf{x}_m)$ an $m$-tuple of multivariate points $\mathbf{x}_i \in \mathcal{X}$, $1\leq i\leq m$. Accordingly, we denote $\mathbf{1} = (1, \ldots, 1)$ is a $d$-dimensional vector whose components are all equal to 1, and $\tilde{\mathbf{1}} = (\mathbf{1}, \ldots, \mathbf{1})$ an $m$-tuple of points $\mathbf{1}$. From now on, we shall use the following notation:

We also define for all $\tilde{\mathbf{x}} = (\mathbf{x}_1, \ldots, \mathbf{x}_m) \in \mathcal{X}^m$ and $\ell \in \left\{1, 2, 3\right\}$

where $\tilde{\mathcal{K}}_{\bar{\boldsymbol{\Lambda}}_{n, \ell}(\tilde{\mathbf{x}})}(\tilde{\mathbf{t}}): = \prod\limits_{i = 1}^m K_{\boldsymbol{\Lambda}_{n, \ell}(\mathbf{x}_i)}(\mathbf{t}_i)$. For $\ell \in \left\{1, 2, 3\right\}$, we now define

We can see that

In establishing the uniform consistency of $\widehat{r}_{n, \ell}^{(m)}(\varphi, \tilde{\mathbf{x}}, \bar{\boldsymbol{\Lambda}}_{n, \ell}(\tilde{\mathbf{x}}))$ with respect to $r^{(m)}(\varphi, \tilde{\mathbf{x}})$, an alternative and more suitable centering factor will be considered instead of the expectation $\mathbb{E}\left[ \widehat{r}_{n, \ell}^{(m)}(\varphi, \tilde{\mathbf{x}}, \bar{\boldsymbol{\Lambda}}_{n, \ell}(\tilde{\mathbf{x}}))\right]$, which may either be non-existing or computationally challenging to determine. This alternative centering is defined as

The notation and facts presented below should be included in the continuation of this discussion. For a kernel $L$ of $m \geq 1$ variables we define

Suppose that $L$ is a function of $k \geq 1$ variables, symmetric in its entries. Then, the Hoeffding projections (see ^[80] and ^[46]) with respect to $\mathbb P$, for $1 \leq m \leq k$, are defined as

and

For some measures $\mathbb Q_{i}$ on $S$, we denote

and $\Delta_{\mathbf x}$ denotes the Dirac measure at point $\mathbf{x} \in \mathcal{X}$. Then, the decomposition of ^[80] gives

which is easy to check. For $L \in L_2\left(\mathbb P^k\right)$ this denotes an orthogonal decomposition and $\mathbb E(\pi_m L \mid\mathbf X_2, \ldots, \mathbf X_m) = 0$ for $m\geq 1$; then, the kernels $\pi_{m, k} L$ are canonical for $\mathbb P$. Also, $\pi_{m, k}, m \geq 1$, are nested projections, that is, $\pi_{m, k} \circ \pi_{m^\prime, k} = \pi_{m, k}$ if $m \leq m^\prime$, and

For example,

Remark 2.1. The functions $\mathcal{G}_{\varphi, \tilde{\mathbf{x}}, \ell}$ are not necessarily symmetric; when we need to symmetrize them, we have

where $\tilde{\mathbf{t}}_{\sigma} = (\mathbf{t}_{\sigma_1}, \ldots, \mathbf{t}_{\sigma_m})$ and $\tilde{\mathbf{y}}_{\sigma} = (\mathbf{y}_{\sigma_1}, \ldots, \mathbf{y}_{\sigma_m})$. After symmetrization, the expectation

and the $U$-statistic

do not change.

Before presenting the conditions and primary results, let's delve into the following notation. For $a > 0$,

is the gamma function; for $\mathbf{z} \in \mathbb{R}^d, \nabla\{h(\mathbf{z})\}$ signifies a $d$-column vector of the first-order partial derivatives of a function $h(\mathbf{z})$;

where $f_{\mathbf{X}}(\cdot)$ is the marginal pdf of $\mathbf{X}\in \mathcal{X}$ and we denote

The expression "$X \stackrel{\mathcal D}{ = } Y$" denotes that the random variable $X$ has the same distribution as $Y$, while "a.s." stands for "almost surely". Moreover, $\|\mathbf{A}\|$ represents the Frobenius norm of the matrix $\mathbf{A}$, defined as $\|\mathbf{A}\| = \left\{\operatorname{tr}\left(\mathbf{A}^{\top} \mathbf{A}\right)\right\}^{1 / 2}$.

2.1.   Conditions and comments

(C.1) The function $\mathcal{R}(\varphi, \cdot)$ is Lipschitz continuous with respect to $\tilde{\mathbf{x}} \in \mathcal{X}^m$;

(C.2) The second-order partial derivatives of $\tilde{f}(\tilde{\mathbf{x}})$ and $\mathcal{R}(\varphi, \tilde{\mathbf{x}})$ are continuous on $\tilde{\mathbf{x}} \in(0, 1)^{dm}$;

(C.3) There are some constants $\gamma > 0$ and $C_1 \in[1, \infty)$ such that $\mathbb{E}|\varphi(\tilde{\mathbf{Y}})|^{2+\gamma} < \infty$ and

Comments:

Notice that (C.2) implies that there is some constant $C_0\in [1, \infty)$ such that

The uniform boundedness condition (2.6) in (C.3) implies that $\mathbb{E}\left(|\varphi(\tilde{\mathbf{Y}})|^{2+\gamma} \mid \tilde{\mathbf{X}} = \tilde{\mathbf{x}}\right)$ is allowed to diverge at boundaries but no faster than $\{f(\mathbf{x}_1)\cdots f(\mathbf{x}_m)\}^{-1}$. A similar condition can be found, for instance, in [72, Assumption 2] and [94, Assumption A3]. Notice that (C.3) is important to the truncation method. Note that (C.3) may be replaced by more general hypotheses upon moments of $\tilde{\mathbf{ Y}}$ as in ^[14], that is

$\textbf{(C.3)}^{\prime\prime}$ We denote by $\{\mathcal{M}(x) : x \geq 0\}$ a nonnegative continuous function, increasing on $[0, \infty)$ such that, for some $s > 2$, ultimately as $x \uparrow \infty$,

For each $t \geq \mathcal{M}(0)$, we define $\mathcal{M}^{inv}(t) \geq 0$ by $\mathcal{M}(\mathcal{M}^{inv}(t)) = t$. We assume further that:

The following choices of $\mathcal{M}(\cdot)$ are of particular interest:

(ⅰ) $\mathcal{M}(x) = x^{p}$ for some $p > 2$ or

(ⅱ) $\mathcal{M}(x) = \exp (s x)$ for some $s > 0$.

3.   Conditional U-statistics using Dirichlet kernels

In this section, we consider $\mathcal{X} = \mathbb S_{d, 1}$, a $d$-dimensional simplex, defined by

and its interior

where $\|\mathbf{x}\|_1: = \sum\limits_{i = 1}^d\left|x_i\right|$ and $d \in \mathbb{N}$. For $\alpha_1, \ldots, \alpha_d, \beta > 0$, the density of the $\operatorname{Dirichlet}(\boldsymbol{\alpha}, \beta)$ distribution is

We refer to [93, Chapter 49] and ^[114]. In ^[3], Aitchison and Lauder introduced a significant aspect wherein the kernel $K_{\boldsymbol{\alpha}, \beta}(\cdot)$ form alters with the position $\mathbf{x}$ within the simplex. This adaptation mitigates the boundary bias issue prevalent in conventional estimators, where the kernel remains constant across all points. Throughout this section, for each $j = 1, \ldots, m$, we set

The smoothing parameter $\breve b$, denoted as $\breve b(n)$, inherently depends on the sample size $n$. Now, we can introduce a new conditional $U$-statistic regression estimator using the Dirichlet kernel by replacing (3.1) in (2.2), that is,

Below, we will present key findings regarding the regression function in the scenario where $m = 1$. These findings are essential for examining the estimator (3.2).

3.1.   Nonparametric regression estimation

We consider the following quantities:

and

In this section, we establish uniform strong consistency of the estimator defined by

Finally, we represent the expectation of $\widehat{g}_n(\varphi, \mathbf{x}, \boldsymbol{\Lambda}_{n, 1})$ as

where $\boldsymbol{\xi}_{\mathbf x} \sim \operatorname{Dirichlet}\left(\boldsymbol{\alpha}, \beta\right)$ and $\mathcal{R}(\varphi, \mathbf{x}) = f(\mathbf{x})r^{(1)}(\varphi, \mathbf{x})$. To derive uniform consistency results, as usual, we rewrite

For $\delta > 0$, define

The Dirichlet Nadaraya-Watson kernel estimators have not yet been proven to yield the following result.

Theorem 3.1. Assume that (C.1) and (C.3) hold. If, in addition, $\breve{b}^{-d} \leq n$ as $n \rightarrow \infty$, then, as $n \rightarrow \infty$,

In particular, if $|\log \breve b|^2 \breve b^{-2 d-1} = o\left(n /(\log n)^3\right)$ as $n \rightarrow \infty$, then

3.2.   Uniform convergence of conditional U-statistics

In this section, we unveil our principal findings regarding the uniform almost sure consistency for the conditional $U$-statistics. Below, we state the uniform consistency of conditional $U$-statistics when the function $\varphi$ is not necessarily bounded.

Theorem 3.2. If (C.2.) and (C.3.) hold, then as $n \rightarrow \infty,$ we have

Theorem 3.3. If (C.2) and (C.3.) hold, then as $n \rightarrow \infty,$ we have

Theorem 3.4. If (C.2) holds, then, we have

Corollary 3.5. Under the assumptions of Theorems 3.3 and 3.4, we have, as $n \rightarrow \infty,$

3.3.   Asymptotic normality

Within this section, we establish the asymptotic normality for the estimator defined in (3.2). To achieve this, we rely on the following set of assumptions:

(A.1) Let $\tilde{\mathbf{x}} = (\mathbf{x}_1, \ldots, \mathbf{x}_m)$ be a point of continuity for each

where

(A.2) The density function $f(\cdot)$ is continuous at each $\mathbf{x}_j$, $1\leq j \leq m,$ with $f(\mathbf{x}_j) > 0$;

(A.3) $r_{j, l, s}(\cdot, \cdot, \cdot)$ is bounded in a neighborhood of $(\tilde{\mathbf{x}}, \tilde{\mathbf{x}}, \tilde{\mathbf{x}}) \in \mathbb S_{d, 1}^{3m}$, where, for all $1\leq j, l, s\leq m$

and for $1\leq s\leq 3$, $\tilde{\mathbf{z}}_{sm} = (\mathbf{z}_{(s-1)m+1}, \ldots, \mathbf{z}_{(s-1)m+j-1}, \mathbf{z}, \mathbf{z}_{(s-1)m+j+1}, \ldots, \mathbf{z}_{sm})$;

(A.4) $r_{1, 2}^{(m)}(\cdot, \cdot)$ is bounded in a neighborhood of $(\tilde{\mathbf{x}}, \tilde{\mathbf{x}})$, where

$\textbf{(A.5)}$$r^{(m)}(\varphi, \cdot)$ admits an expansion

as $\Delta \rightarrow 0$, for all $\mathbf{t}$ in a neighborhood of $\tilde{\mathbf{x}}$.

Below, we write $Z \stackrel{\mathcal{D}}{ = } \mathcal{N}(\mu, \sigma^{2})$ whenever the random variable $Z$ follows a normal law with expectation $\mu$ and variance $\sigma^{2}$, ${\stackrel{\mathcal{D}}{\rightarrow }}$ denotes convergence in distribution. We also denote

where

Our main result in this section is summarized as follows.

Theorem 3.6. If (A.1)–(A.4) and (C.2.) hold and if $r^{(m)}(\cdot, \cdot)$ is continuous at $\tilde{\mathbf{x}}$, then

where

The proof of Theorem 3.6 is postponed to Appendix-1 A.

Corollary 3.7. If, in addition to the assumptions of Theorem 3.6, (A.5) holds, then

In addition, we have

provided that $n\breve{b}^{(d+2)/4} \rightarrow 0$.

Remark 3.8. ^[117] According to Theorem 3.1.15 in ^[119], the convergence rate for the conventional $d$-dimensional kernel density estimator for i.i.d. data, using bandwidth $h$, is $O\left(n^{-1 / 2} h^{-d / 2}\right)$. In contrast, the estimator $\widehat{f}_n(\mathbf{x}, \boldsymbol{\Lambda}_{n, 1})$ achieves a convergence rate of $O\left(n^{-1 / 2} \breve{b}^{-d / 4}\right)$. Consequently, the relationship between the bandwidths of $\widehat{f}_n(\mathbf{x}, \boldsymbol{\Lambda}_{n, 1})$ and the traditional multivariate kernel density estimator is expressed as $\breve{b} \approx h^2$.

Remark 3.9. In their work, ^[117] demonstrated that for all $\mathbf{x}\in \text{Int}(\mathbb{S}_{d, 1})$ and as $n$ tends to infinity, the MSE of the estimator $\widehat{f}_n(\mathbf{x}, \boldsymbol{\Lambda}_{n, 1})$ with respect to the density function $f(\cdot)$ can be expressed as:

where $\psi(\cdot)$ is defined in Eq (B.1) in Lemma B.4, and

In particular, if $f(\mathbf{x}) \cdot g(\mathbf{x}) \neq 0$, the asymptotically optimal choice of $\breve{b}$, concerning MSE, is given by:

with

Furthermore, if $n^{2 /(d+4)} \breve{b}$ tends to $\lambda$ for some $\lambda > 0$ as $n$ approaches infinity, then

4.   Conditional U-statistics using Bernstein polynomials

This section delves into the asymptotic properties of the conditional $U$-statistics using Bernstein polynomials. Let $F(\cdot)$ represent any joint cumulative distribution function on $\mathbb {S}_{d, 1}$, where values outside $\mathbb S_{d, 1}$ are either 0 or 1. Following ^[115,116], we define the Bernstein polynomial of order $\vartheta$ for $F(\cdot)$ as follows

where the weights are probabilities from the $\operatorname{multinomial}(\vartheta, \mathbf{x})$ distribution

The Bernstein estimator of $F(\cdot)$, denoted by $F_{n, \vartheta}^{\star}(\cdot)$, is the Bernstein polynomial of order $\vartheta$ for the empirical cumulative distribution function

where $\mathbf{X}_1, \ldots, \mathbf{X}_n$ are i.i.d according to $F(\cdot)$ and $\mathbf{1}\{A\}$ denotes as usual the indicator function of the set $A$. Precisely, we define

For a density $f(\cdot)$ supported on $\mathbb {S}_{d, 1}$, we define the Bernstein density estimator of $f(\cdot)$ as

In this expression, we let

and $(\vartheta-1+d)! /(\vartheta-1)!$ serves as a scaling factor proportional to the inverse of the volume of the hypercube $\left(\mathbf{k} / \vartheta, (\mathbf{k}+\mathbf{1}) / \vartheta\right]$. It's worth noting that replacing this scaling factor by $\vartheta^d$ in $(4.1)$ maintains $\hat{f}_{n, \vartheta}(\cdot)$ as an asymptotic density function. The asymptotic results presented in this paper remain largely unchanged under both definitions of the density estimator.

Remark 4.1. ^[115] A different expression for the Bernstein density estimator (4.1) can be formulated as a specific finite mixture of Dirichlet densities, that is,

where the density value of the $\operatorname{Dirichlet}(\alpha, \beta)$ distribution at $\mathbf{x} \in\mathbb {S}_{d, 1}$ is given by

For further insights, refer to ^[115].

The conditional $U$-statistics smoothed by the Bernstein polynomials are defined as, for each $\tilde{\mathbf{x}}\in \mathbb {S}_{d, 1}^m$

In the particular case $m = 1$, the $r^{(m)}(\varphi, \tilde{\mathbf{x}})$ is reduced to $r^{(1)}(\varphi, \mathbf{ x}) = \mathbb E(\varphi(\mathbf{ Y})|\mathbf{ X} = \mathbf{ x})$ and Stute's estimator becomes the Nadaraya-Watson estimator of $r^{(1)}(\varphi, \mathbf{ x})$

4.1.   Nonparametric regression estimation

In this section, we prove the uniform strong consistency of the regression estimator for $m = 1$.

Theorem 4.2. Assume that (C.1)–(C.3) hold. If $2\leq\vartheta\leq \frac{n}{\log n}$, as $n\longrightarrow\infty$, then

and

Corollary 4.3. Under the assumptions of Theorem 4.2, we have as $n\longrightarrow\infty$

In addition, if $\vartheta^{2d-1} = o(n/\log n)$, then

4.2.   Conditional U-statistics

In this section, we study the uniform strong consistency of the conditional $U$-statistics using Bernstein polynomials.

Theorem 4.4. Assume that (C.2) and (C.3) hold. If $2\leq\vartheta\leq \frac{n}{\log n}$, as $n\longrightarrow\infty$, then

Theorem 4.5. Assume that (C.2) and (C.3) hold. If $2\leq\vartheta\leq \frac{n}{\log n}$, as $n\longrightarrow\infty$, then

Theorem 4.6. Assume that (C.1) and (C.2) hold. If $2\leq\vartheta\leq \frac{n}{\log n}$, as $n\longrightarrow\infty$, then

Corollary 4.7. Under the assumptions of Theorems 4.5 and 4.6, as $n \longrightarrow \infty$, we have

In addition, if $\vartheta^{2d-1} = o(n/\log n)$, then

Remark 4.8. ^[115] It is worth mentioning that, similar to Remark 3.8, the convergence rate for the conventional $d$-dimensional kernel density estimator for i.i.d. data, using bandwidth $h$, is $O\left(n^{-1 / 2} h^{-d / 2}\right)$. However, the estimator $\widehat{f}_n(\mathbf{x}, \vartheta)$ achieves a convergence rate of $O\left(n^{-1 / 2} \vartheta^{d / 4}\right)$. Consequently, the relationship between the bandwidths of $\widehat{f}_n(\mathbf{x}, \vartheta)$ and the traditional multivariate kernel density estimator is expressed as $\vartheta \approx h^{-2}$.

Remark 4.9. ^[115] demonstrated that the Mean Squared Error (MSE) of the density estimator $\widehat{f}_n(\mathbf{x}, \vartheta)$ satisfies for all $\mathbf{x}\in \text{Int}(\mathbb{S}_{d, 1})$ and as $n$ tends to infinity:

where

and

In particular, when $f(\mathbf{x}) \cdot b(\mathbf{x}) \neq 0$, the asymptotically optimal choice for $\vartheta$, minimizing MSE, is:

with corresponding MSE:

Moreover, in the more general case where $n^{2 /(d+4)} \vartheta^{-1} \rightarrow \lambda > 0$, as $n \rightarrow \infty$, the MSE becomes:

5.   Conditional U-statistics estimators using beta kernel

Throughout this section, it is assumed, as in ^[79], without loss of generality, that the compact set is a $d$-dimensional unit hypercube $[0, 1]^d$. Among all asymmetric kernels, our particular focus is on the beta kernel by ^[40]. The kernel takes the form

where

and $B(\breve{\alpha}, \breve{\beta}) = \int_0^1 y^{\breve{\alpha}-1}(1-y)^{\breve{\beta}-1} d y$ for $\breve{\alpha}, \breve{\beta} > 0$ is the beta function. To cope with multivariate problems, we construct a product kernel for $\breve{\boldsymbol{\alpha}} = (\breve{\alpha}_1, \ldots, \breve{\alpha}_d)$ and $\breve{\boldsymbol{\beta}} = (\breve{\beta}_1, \ldots, \breve{\beta}_d)$

where $\mathbf{u}: = \left(u_1, \ldots, u_d\right) \in[0, 1]^d, \mathbf{x}: = \left(x_1, \ldots, x_d\right)\in[0, 1]^d$, and $\mathbf{b}: = \left(b_1, \ldots, b_d\right) \in \mathbb{R}_{+}^d$ are $d$-dimensional vectors of data points, design points, and the smoothing parameter. We consider

where

In the particular case $m = 1$, the Nadaraya-Watson estimator of $r^{(1)}(\varphi, \tilde{\mathbf{x}})$ of ^[79] is given by

5.1.   Conditions and comments

Our analysis starts from demonstrating weak uniform consistency with rates of the sample average estimator (5.1) for (2.1) on a $dm$-hyper-rectangle

where the boundary parameters $\eta: = \left(\eta_1, \ldots, \eta_d\right)$ are either fixed or shrunk to zero at a suitable rate. To deliver the results, we impose the following conditions.

(C.4) $\mathbf{b}_j: = \mathbf{b}_j(n) = \left(b_{j_1}, \ldots, b_{j_d}\right) > 0$ and $\boldsymbol{\eta}_j: = \boldsymbol{\eta}_j(n) = \left(\eta_{j_1}, \ldots, \eta_{j_d}\right) > 0$, $j = 1, \ldots, m$, satisfy for $i = 1, \ldots, d$, $b_{j_i}, \eta_{j_i}\rightarrow 0$, $\frac{b_{j_i}}{\eta_{j_i}} \rightarrow 0$, and

The conditions on $\eta_{j_i}$ in (C.4) are intended for the case of an expanding set. In particular, the condition $b_{j_i} / \eta_{j_i} \rightarrow 0$ means that the boundary parameter $\eta_{j_i}$ must shrink to zero at a slower rate than $b_{j_i}$, this is crucial for Stirling's approximation to the gamma function. This condition was used ^[79] for the novel proof of the convergence results that we have extended to our setting.

5.2.   Weak uniform convergence of conditional U-statistics

In the following theorem, we state that the weak uniform convergence of conditional U-statistics, in the particular case of $m = 1$, reduces to the results obtained in ^[79].

Theorem 5.1. If (C.2)–(C.4) hold, then, as $n \rightarrow \infty$,

Theorem 5.2. If (C.2)–(C.4) hold, then, as $n \rightarrow \infty$,

Theorem 5.3. If (C.2) holds, then, as $n \rightarrow \infty$,

Corollary 5.4. Under the assumptions of Theorem 5.2 and Theorem 5.3, as $n \rightarrow \infty$,

5.3.   Strong uniform convergence of conditional U-statistics

In this section, we establish strong uniform consistency with rates of $\widehat{r}_{n, 3}^{(m)}(\varphi, \tilde{\mathbf{x}}; \bar{\boldsymbol{\Lambda}}_{n, 3}(\tilde{\mathbf{x}}))$. However, to do so, the assumption on smoothing parameters must be suitably strengthened.

(C.4') $\mathbf{b}_j: = \mathbf{b}_j(n) = \left(b_{j_1}, \ldots, b_{j_d}\right) > 0$ and $\boldsymbol{\eta}_j: = \boldsymbol{\eta}_j(n) = \left(\eta_{j_1}, \ldots, \eta_{j_d}\right) > 0$, $j = 1, \ldots, m$, satisfy for $i = 1, \ldots, d$, $b_{j_i}, \eta_{j_i}\rightarrow 0$, $\frac{b_{j_i}}{\eta_{j_i}} \rightarrow 0$ and

for some constant $\kappa \in[0, 1)$, as $n \rightarrow \infty$.

The condition (5.6) is stronger than $\log n /\left(n \sqrt{ \prod\limits_{j = 1}^m\left(\prod\limits_{i = 1}^{d} b_{j_i} \eta_{j_i}\right)}\right) \longrightarrow 0$ in (C.4) in that the former implies the latter. Under this condition, the statement in Corollary 5 can be strengthened to almost sure convergence. The following theorems generalize the results of ^[79] that are given for $m = 1$.

Theorem 5.5. If (C.2)–(C.3) and (C.4') hold, then, as $n \rightarrow \infty$,

Theorem 5.6. If (C.2)–(C.3) and (C.4') hold, then, as $n \rightarrow \infty$,

Theorem 5.7. If (C.2) holds, then

Corollary 5.8. Under the assumptions of Theorems 5.6 and 5.9, as $n \rightarrow \infty$,

5.4.   Conditional U-statistics estimators for mixed categorical and continuous data

Let's delve into the methodology for handling a discrete random variable $Z$, which can assume $c$ distinct values, $\{0, 1, \ldots, c-1\}$, where $c\geq 2$; we refer to ^{[43,101,104,106,132]}. We categorize this variable as either unordered or ordered, as the kernels utilized for these two types differ slightly. For an unordered variable, the univariate discrete kernel takes the form

Here, $v$ represents the data point, $z$ denotes the design point, and $\lambda \in(0, 1)$ denotes the bandwidth. Conversely, the univariate discrete kernel for an ordered variable is given by

Moving on to the product discrete kernel, when $q_1 (\leq q)$ out of $q$ discrete variables are unordered, it becomes

Here, $\mathbf{v}: = \left(v_1, \ldots, v_q\right)$, $\mathbf{z}: = \left(z_1, \ldots, z_q\right)$, and $\lambda: = \left(\lambda_1, \ldots, \lambda_q\right)$. Combining this with the product beta kernel $K_{\breve{\boldsymbol{\alpha}}, \breve{\beta}}(\mathbf{u})$ yields the product kernel for mixed categorical and continuous data

Given this kernel and $n$ i.i.d. observations $\left\{\left(Y_i, \mathbf{X}_i, \mathbf{Z}_i\right)\right\}_{i = 1}^n \in \mathbb{R} \times[0, 1]^d \times \mathbb{S}_{\mathbf{Z}}$, where $\mathbb{S}_{\mathbf{Z}}: = \prod_{k = 1}^q\left\{0, 1, \ldots, c_k-1\right\}$, we turn to a regression estimator of the conditional mean

This estimator, denoted as $\widehat{r}_{n, 3}^{(m)}(\varphi, \tilde{\mathbf{x}}; \bar{\boldsymbol{\Lambda}}_{n, 3}(\tilde{\mathbf{x}}))$, is expressed as

5.4.1.   Weak uniform convergence

Before we state the uniform convergence results of the estimator, we assume

(C.1') $\left\{\left(Y_i, \mathbf{X}_i, \mathbf{Z}_i\right)\right\}_{i = 1}^n \in \mathbb{R} \times[0, 1]^d \times \mathbb{S}_{\mathbf{Z}}$ are i.i.d. random variables;

(C.2') Let $\tilde{f}(\tilde{\mathbf{x}}, \tilde{\mathbf{y}})$ be the joint pdf of $(\tilde{X}, \tilde{Y})$. Then, the second-order derivatives of $\tilde{f}(\tilde{\mathbf{x}}, \tilde{\mathbf{y}})$ and $\tilde{g}(\tilde{\mathbf{x}}, \tilde{\mathbf{y}}): = r^{(m)}(\varphi, \tilde{\mathbf{x}}, \tilde{\mathbf{z}})\tilde{f}(\tilde{\mathbf{x}}, \tilde{\mathbf{y}})$, with respect to $\tilde{\mathbf{x}}$, are continuous on $\tilde{\mathbf{x}} \in(0, 1)^{dm}$;

(C.3') There are some constants $\gamma > 0$ and $C_1 \in[1, \infty)$ such that $\mathbb{E}|\varphi(\mathbf{Y})|^{2+\gamma} < \infty$ and

(C.4") $\mathbf{b}_j: = \mathbf{b}_j(n) = \left(b_{j_1}, \ldots, b_{j_d}\right) > 0$, $\boldsymbol{\eta}_j: = \boldsymbol{\eta}_j(n) = \left(\eta_{j_1}, \ldots, \eta_{j_d}\right) > 0$, $j = 1, \ldots, m$, and ${\lambda}_k: = {\lambda}_k \in (0, 1)$, $k = 1, \ldots, q$, satisfy for $i = 1, \ldots, d$, $b_{j_i}, \eta_{j_i}\rightarrow 0$, $\frac{b_{j_i}}{\eta_{j_i}} \rightarrow 0$, $\lambda_k\rightarrow0$, and

(C.4"') $\mathbf{b}_j: = \mathbf{b}_j(n) = \left(b_{j_1}, \ldots, b_{j_d}\right) > 0$, $\boldsymbol{\eta}_j: = \boldsymbol{\eta}_j(n) = \left(\eta_{j_1}, \ldots, \eta_{j_d}\right) > 0$, $j = 1, \ldots, m$, and ${\lambda}_k: = {\lambda}_k \in (0, 1)$, $k = 1, \ldots, q$ satisfy, for $i = 1, \ldots, d$, $b_{j_i}, \eta_{j_i}\rightarrow 0$, $\frac{b_{j_i}}{\eta_{j_i}} \rightarrow 0$, $\lambda_k\rightarrow0$, and

for some constant $\kappa \in[0, 1)$, as $n \rightarrow \infty$;

(C.5) Let $f_n^m: = \inf_{\tilde{\mathbf{x}}\in \mathbb{S}_{X}^m }\tilde{f}(\tilde{\mathbf{x}}) > 0$, $f_n^m$ tends to zero as $n\longrightarrow \infty$, and

Corollary 5.9. If (C.1')–(C.3'), (C.4") and (C.5) hold, then, as $n \rightarrow \infty$,

6.   Applications

6.1.   Discrimination problems

Now, we apply the results of the problem of discrimination described in Section 3 of ^[137], referring also to ^[136]. We will use a similar notation and setting. Let $\varphi(\cdot)$ be any function taking at most finitely many values, say, $1, \ldots, M$. The sets

then yield a partition of the feature space. Predicting the value of $\varphi(\mathbf Y_{1}, \ldots, \mathbf Y_{k})$ is tantamount to predicting the set in the partition to which $(\mathbf Y_{1}, \ldots, \mathbf Y_{k})$ belongs. For any discrimination rule $g$, we have

where

The above inequality becomes an equality if

$g_{0}(\cdot)$ is called the Bayes rule, and the pertaining probability of error

is called the Bayes risk. Each of the above unknown function $\mathfrak{M}^{j}$'s can be consistently estimated by one of the methods discussed in the preceding sections. Let, for $1\leq j\leq M$ and $\ell\in\{1, 2, 3\}$,

Set

Let us introduce

The discrimination rule $g_{0, n, \ell}(\cdot)$ is asymptotically Bayes' risk consistent

This follows from Corollaries 3.5, 4.7, or 5.8 and the obvious relation

6.2.   Generalized U-statistics

The extension to the case of several samples is straightforward. Consider $\tilde\ell$ independent collections of independent observations

Let, for $\mathbf{ t} \in \mathbb R^{d(k_1+\cdots+k_\ell)}$,

where $\varphi$ is assumed, without loss of generality, to be symmetric within each of its $\tilde\ell$ blocks of arguments. Corresponding to the "kernel" $\varphi$ and assuming $n_{1} \geq k_{1}, \ldots, n_{k} \geq k_{\tilde\ell}$, the conditional $U$-statistic for estimation of $r^{(k, \tilde\ell)}(\varphi, \mathbf{ t})$ is defined, for $\ell\in\{1, 2, 3\}$, by

where

Here $\left\{i_{j 1}, \ldots, i_{j m_j}\right\}$ denotes a set of $k_{j}$ distinct elements of the set $\left.\{1, 2, \ldots, n_j\right\}$ $1 \leq j \leq \tilde\ell$, and $\sum_{c}$ denotes summation over all such combinations. The extension of ^[80] treatment of one-sample $U$ -statistics to the $\tilde\ell$ sample case is due to ^[99] and ^[54]. One can use Corollaries 3.5, 4.7, or 5.8 to infer that

6.3.   Kendall rank correlation coefficient

To test the independence of one-dimensional random variables $Y_1$ and $Y_2$, ^[90] proposed a method based on the $U$-statistic $K_{n}$ with the kernel function:

Its rejection region is of the form $\left\{\sqrt{n} K_{n} > \gamma\right\}$. In this example, we consider a multivariate case. To test the conditional independence of $\boldsymbol{\xi}, \boldsymbol{\eta}: Y = (\boldsymbol{\xi}, \boldsymbol{\eta})$ given $X$, we propose a method based on the conditional $U$-statistic, for $\ell\in\{1, 2, 3\}$,

where $\mathbf{t} = \left(t_{1}, t_{2}\right) \in \mathbb{I} \subset \mathbb{R}^{2}$ and $\varphi(\cdot)$ is Kendall's kernel (6.3). Suppose that $\boldsymbol{\xi}$ and $\boldsymbol{\eta}$ are $d_{1}$ and $d_{2}$-dimensional random vectors, respectively, and $d_{1}+d_{2} = d$. Furthermore, suppose that $\mathrm{Y}_{1}, \ldots, \mathrm{Y}_{n}$ are observations of $(\boldsymbol{\xi}, \boldsymbol{\eta})$, and we are interested in testing :

Let $\mathbf{a} = \left(\mathbf{a}_{1}, \mathbf{a}_{2}\right) \in \mathbb{R}^{d}$ such as $\|\mathbf{a}\| = 1$ and $\mathbf{a}_{1} \in \mathbb{R}^{d_{1}}, \mathbf{a}_{2} \in \mathbb{R}^{d_{2}}$, and $\mathrm{F}(\cdot), \mathrm{G}(\cdot)$ be the distribution functions of $\boldsymbol{\xi}$ and $\boldsymbol{\eta}$ respectively. Suppose $F^{a_{1}}(\cdot)$ and $G^{a_{2}}(\cdot)$ to be continuous for any unit vector $\mathbf{a} = \left(\mathbf{a}_{1}, \mathbf{a}_{2}\right)$ where $\mathrm{F}^{\mathbf{a}_{1}}(t) = \mathbb{P}\left(\mathbf{a}_{1} \boldsymbol{\xi} < t\right)$ and $\mathrm{G}^{\mathbf{a}_{2}}(t) = \mathbb{P}\left(\mathbf{a}_{2} \boldsymbol{\eta} < t\right)$ and $\mathbf{a}_{1}^{\top}$ means the transpose of the vector $\mathbf{a}_{i}, 1 \leqslant i \leqslant 2$. For $n = 2$, let $Y^{(1)} = \left(\boldsymbol{\xi}^{(1)}, \boldsymbol{\eta}^{(1)}\right)$ and $Y^{(2)} = \left(\boldsymbol{\xi}^{(2)}, \boldsymbol{\eta}^{(2)}\right)$ such as $\boldsymbol{\xi}^{(i)} \in \mathbb{R}^{d_{1}}$ and $\boldsymbol{\eta}^{(i)} \in \mathbb{R}^{d_{2}}$ for $i = 1, 2$, and:

An application of Corollaries 3.5, 4.7, or 5.8 gives

7.   Examples

Generally speaking, we may take for $h$ any function that has been found interesting in the unconditional setup; cf. ^[129]. As mentioned before, the case $m = 1$ leads to the Nadaraya-Watson estimator if we set $\varphi(\cdot) = {\rm id}; \varphi(\cdot) = \mathbf 1_{(-\infty, x]}(\cdot)$ yields the empirical conditional d.f. evaluated at $x$. We now discuss several examples for $m = 2$.

Example 7.1. ^[134] For

we obtain

In this case,

Compare $\rho^2$ with $\zeta_1$ in ^[129], page 182.

Example 7.2. ^[134] Assume $Y_i = \left(Y_{i 1}, Y_{i 2}\right)^{\top}$, and define $h$ by

that is, $m = 2$, and

In particular,

the conditional covariance of $Y_1$ given $X_1 = x_1$.

8.   The bandwidth selection criterion

Following ^[53] and ^[30], the leave-one-out cross validation procedure allows us to define, for any fixed $\mathbf{i} = (i_1, \ldots, i_m)\in I(m, n)$ and $\ell\in\{1, 2, 3\},$

To minimize the quadratic loss function, we introduce the following criterion, we have for some (known) nonnegative weight function $\mathcal{W}(\cdot)$, for $\ell\in\{1, 2, 3\},$

where

A natural way for choosing the bandwidth is to minimize the preceding criterion, so let's choose $\widehat{h}_{n, \ell}$ minimizing

One can replace (8.2) by

where

In practice, one takes for $\mathbf{ i} \in I(m, n)$, the uniform global weights $\widetilde{\mathcal{W}}(\tilde{\mathbf{X}}_{\mathbf{i}}) = 1$, and the local weights

For the sake of brevity, we have just considered the most popular method, that is, the cross-validated selected bandwidth. This may be extended to any other bandwidth selector, such as the bandwidth based on Bayesian ideas ^[32,33].

9.   Monte Carlo experiments

The program codes are implemented in $\texttt{R}$. Our simulation setup closely follows that of ^[49] and ^[23,32]. Specifically, we consider drawing i.i.d. random samples $(X_i, Y_{i, 1}, Y_{i, 2})$ for $i = 1, \dots, n$, with univariate explanatory variables. We assume that the underlying distribution is continuous. Note that the conditional Kendall-$\tau$, as described in ^[90], can be defined as follows:

Motivated by these expressions, ^[49] introduced several kernel-based estimators of $\tau_{1, 2|X = u}$:

where $w_{i, n}(\cdot)$ is a sequence of Nadaraya-Watson weights defined by

with the notation $K_h(\cdot): = h^{-1} K(\cdot/h)$ for some kernel $K(\cdot)$ on $\mathbb R$, and $h = h(n)$ denotes a usual bandwidth sequence that tends to zero as $n\to\infty$. In the simulation study, we consider the following simple case of bounded explanatory variables, as follows:

$-$ The variable $X$ is uniformly on $(0, 1)$. Conditionally on $X = u$, $Y_1 | X = u$ and $Y_2 | X = u$ both follow a Gaussian distribution $N(u, 1)$. Their associated conditional copula is Gaussian, and their conditional Kendall-$\tau$ is given by

We make use of the following kernels:

● The tricube kernel:

● the Gaussian kernel:

● the Epanechnikov kernel ^[57]:

● the beta kernel:

with the weight

where $\alpha_1 = u/h+1$ and $\alpha_2 = (1-u)/h+1$,

● the Bernstein polynomials: For $x \in[0, 1], \vartheta \in \mathbb{N}, k = 0, \ldots, \vartheta$, set $b_{k, \vartheta}(x): = \left(\begin{array}{c}\vartheta \\ k\end{array}\right) x^k$ $(1-x)^{\vartheta-k}$, and

For the estimators based on the tricube kernel, the Gaussian kernel, and the Epanechnikov kernel, we use the "normal scale rule" or the rule-of-thumb method; for instance, see ^[130], to select the bandwidth. Specifically, we set $h$ to be $\alpha_h \hat{\sigma}(X_1, \ldots, X_n) n^{-1/5}$, where $\alpha_h$ is some positive constant and $\hat{\sigma}(X_1, \ldots, X_n)$ is the empirical standard deviation of $X$. The parameter $\alpha_h$ is calculated by minimizing the $L_2$ distance between the kernel density estimator and the theoretical density. The choice of $\alpha_h$ is not the optimal one, since we are choosing this in order to minimize the distance between the densities rather than between the regression functions. This choice is sufficient for our needs. The flexibility of this choice is due to the rule-of-thumb method. For the sake of effective calculations of these measures, the theoretical density can be replaced by the empirical counterparts based, for example, on 10000 simulations. Corollary 4 clearly shows that the choice of $\vartheta$ is critical for accurately estimating the Kendall-$\tau$. According to ^[10], it is noted that for the distribution function, $\vartheta$ can be as large as $n / \log n$, whereas for density estimation, it is preferable to have $\vartheta = \mathrm{o}(n / \log n)$. In our simulations, we choose $\vartheta = \lceil n / \log n \rceil$, where $\lceil x \rceil$ denotes the ceiling of $x$. This framework allows us to examine the small sample performances of the estimators $\widehat{\tau}_{1, 2|X = u}^{(\ell)}$ for $\ell = 1, 2, 3$. Consequently, we compute our estimators for each of the kernels mentioned above and for each $n \in \{ 500, 1000, 2000 \}$. We consider three local measures of goodness-of-fit: for a given $u$ and for any Kendall-$\tau$ estimate (say $\widehat\tau_{1, 2|X = u }$), let

● the (local) bias: ${\rm Bias}(u) : = \mathbb{E}[\widehat \tau_{1, 2| X = u}] - \widehat\tau_{1, 2| X = u}$,

● the (local) standard deviation: ${\rm Sd}(u) : = \mathbb{E} \Big[ \big(\widehat \tau_{1, 2| X = u} - \mathbb{E}[\widehat \tau_{1, 2| X = u}] \big)^2 \Big]^{1/2}$,

● the (local) MSE: ${\rm MSE}(u) : = \mathbb{E} \Big[ \big(\widehat\tau_{1, 2| X = u} - \tau_{1, 2| X = u} \big)^2 \Big]$.

We also consider the integrated versions of these measures with respect to the usual Lebesgue measure over the entire support of $t$, denoted as $IBias$, $ISd$, and $IMSE$. For effective computation, we replace the theoretical expectations with their empirical counterparts based on 1000 simulations.

As is common in inferential contexts, larger sample sizes generally yield better performance, see Figures 1–6. Simple inspection of the results in Table 1 shows that larger sample sizes $n$ lead to smaller $IBias$, $ISd$, and $IMSE$. The parameters of the beta kernel and Bernstein polynomial can be adjusted such that the mode, median, or mean aligns with the point $x$. This variable smoothing enables these asymmetric kernel estimators to outperform traditional kernel estimators near the boundary of the support by reducing bias. Additionally, because variable smoothing is directly integrated into the parameterization of the kernel function, asymmetric kernel estimators are generally simpler to implement compared to boundary kernel methods. It is important to note that for $m = 2$, the tendency for larger variance when $x_1 = x_2$ occurs because only data from the single neighborhood of $x_1$ is used. In contrast, when $x_1 \neq x_2$, data from two potentially disjoint sets are incorporated. Specifically, for Kendall's tau, the case $x_1 = x_2 = u$ is not ideal for minimizing variance. To provide methodological recommendations for using the proposed estimators, it would be beneficial to conduct extensive Monte Carlo experiments comparing our procedures with other alternatives in the literature. However, this is beyond the scope of the present paper.

10.   Concluding remarks

Our study initiated exploration into the theoretical aspects of the Dirichlet kernel estimator, as initially proposed by ^[3], when applied to conditional $U$-statistics within the $dm$-dimensional simplex. By extending the unidimensional beta kernel estimator introduced by ^[40], the Dirichlet kernel estimator effectively mitigates boundary bias. Our comprehensive analysis established both its asymptotic normality and uniform strong consistency. Furthermore, we have considered the estimation of conditional $U$-statistics using Bernstein polynomials. In deriving our results for these estimators, we also presented novel insights into the Nadaraya-Watson estimators employing Bernstein polynomials, which are of independent interest. Additionally, we introduce a beta kernel estimator specifically designed for conditional $U$-statistics, providing an extensive suite of uniform consistency results along with associated rates. Our rigorous analysis demonstrated both weak and strong uniform convergence, leveraging the expansion of compact sets and general sequences of smoothing parameters as the analytical foundation. Also, we conducted some simulations to illustrate the small sample performances of the estimators. One aspect that remains unexplored in this paper is the optimal selection of smoothing parameters, a topic of significant importance warranting dedicated research effort, which we defer to a forthcoming investigation.

Several avenues exist for further development of our approach. Extending our results to encompass $k$-nearest-neighbors estimators is of notable interest, although achieving this goal necessitates the development of new technical arguments, as it currently lies beyond reasonable expectations. Exploring the realm of $k$-nearest-neighbors estimators would broaden the scope of our research and yield valuable insights into their performance and properties (see ^[31,56]). The literature concerning asymmetric kernels for dependent data remains underdeveloped. Extending our results to account for dependence poses a more formidable challenge compared to previous extensions, as it necessitates the formulation of new probabilistic results, given that those employed in our present analysis are tailored specifically to i.i.d. samples (refer to ^[26,34]). Change-point detection has become an essential tool for identifying points in a data sequence where a stochastic system undergoes sudden external influences. This can significantly enhance our understanding of the underlying processes. While change-point analysis has been widely applied to various stochastic processes across numerous scientific fields (refer to ^[27,28,29]), its application to conditional $U$-statistics constitutes an unexplored and challenging research topic that warrants further investigation. Missing data are prevalent in modern statistics, presenting significant challenges across various applications. For example, missing data can occur when information is gathered from sources that measure different variables, such as in healthcare, where patient data may vary between clinics or hospitals. Additional causes of missing data include sensor failure, data censoring, and privacy concerns, among many others. It would be interesting to extend the results of the present paper to the missing data or censored data framework as in ^[22]. Spatial data, gathered from measurement sites across various disciplines, frequently appears in research fields such as econometrics, epidemiology, environmental science, image analysis, oceanography, meteorology, geostatistics, and many others. Extending this paper to effectively handle spatial data is a challenging task. Our results will be expanded in future investigations by examining the weak convergence of the conditional $U$-processes. This will require additional effort and advanced techniques to establish tightness.

Author contributions

Salim Bouzebda, Amel Nezzal and Issam Elhattab: Conceptualization, formal analysis, investigation, methodology, software, validation, visualization, writing – original draft, writing – review & editing. All authors have read and approved the final version of the manuscript for publication

Acknowledgements

The authors extend their sincere gratitude to the Editor-in-Chief, the Associate Editor, and the five referees for their invaluable feedback. Their insightful comments have greatly refined and focused the original work, resulting in a markedly improved presentation.

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Salim Bouzebda is the(a) Guest Editor of special issue "Advances in Statistical Inference and Stochastic Processes: Theory and Applications" for AIMS Mathematics. Prof. Salim Bouzebda was not involved in the editorial review and the decision to publish this article.

A.   Appendix-1

This section is dedicated to proving our results. We will continue to use the previously established notation. A crucial element in our proofs involves the truncation of the $U$-statistics. Specifically, we represent the $U$-statistics $u_{n, \ell}(\varphi, \tilde{\mathbf{x}})$ for $\ell \in \{1, 2, 3\}$ as follows:

where for $\ell \in \{1, 2, 3\}$ and some $\omega_{n, \ell}$ (to be specified later in the proof of each section), we have:

Here, $u_{n, \ell}^{(T)}(\varphi, \tilde{\mathbf{x}})$ is the truncated part, and $u_{n, \ell}^{(R)}(\varphi, \tilde{\mathbf{x}})$ is the remainder part. We establish the uniform convergence rates of $u_{n, \ell}(\varphi, \tilde{\mathbf{x}})$ to $\mathbb{E}[u_{n, \ell}(\varphi, \tilde{\mathbf{x}})]$ based on the convergence rates of $u_{n, \ell}\left(\mathcal{G}_{\varphi, \tilde{\mathbf{x}}, \ell}^{(T)}\right)$ to $\mathbb{E}[u_{n, \ell}\left(\mathcal{G}_{\varphi, \tilde{\mathbf{x}}, \ell}^{(T)}\right)]$, while demonstrating that the remainder part is asymptotically negligible. Next, we can use these results to deduce the convergence rates of the stochastic part of the estimators $\widehat{r}_{n, \ell}^{{(m)}}(\varphi, \tilde{\mathbf{x}}; \bar{\boldsymbol{\Lambda}}_{n, \ell}(\tilde{\mathbf{x}}))$. Indeed, we can clearly see that based on the classical decomposition, for $\ell\in \{1, 2, 3\}$:

Later, based on the imposed regularity conditions in each section, we can easily control the terms $\left| u_{n, \ell}(\varphi, \tilde{\mathbf{x}})\right|$ and $\left| \mathbb{E}\left[u_{n, \ell}(\varphi, \tilde{\mathbf{x}})\right]\right|$ (including the particular case when $\varphi\equiv 1$), uniformly in $\tilde{\mathbf{x}}$ to obtain the desired rates of convergence. Lastly, we need to study the bias of each estimator. It is worth noting that the proof of the bias term for the three estimators proposed in this paper is based on the following decomposition for $\ell \in \left\{1, 2, 3\right\}$. We have

As a matter of fact, (A.3) implies that it suffices to control the term $\left| \mathbb{E}\left(u_{n, \ell}(\varphi, \tilde{\mathbf{x}})\right)-\mathcal{R}(\varphi, \tilde{\mathbf{x}})\right|$ uniformly in $\tilde{\mathbf{x}}$, to establish the desired results, as we can see in the sequel.

A.1.   Proofs of Section 3: Dirichlet kernels

A.1.1.   Proofs of Section 3.1

The regression proof for the case where $m = 1$ closely resembles the one given in ^[117]. We include it here in full detail for the reader's convenience and to ensure it is self-contained. However, the result concerning the regression function smoothed by the Dirichlet kernel has not been addressed in the literature, providing the primary motivation for presenting it in this paper.

Proof of Theorem 3.1. Observe that

Keep in mind the definition of the set $\mathbb{S}_{d, 1}(\delta)$ given in (3.4). To begin, we need to prove the following result:

The proof of (A.4) follows the same analogy as in ^[117] while applying the necessary changes to fit our context. We have

where, for $i = 1, \ldots, n$,

For some sequence $\omega_{n, 1}$ tending to infinity, we also consider the following notation:

This allows us to write

where

We also denote

The following proposition, that is close to Proposition 1 of ^[117], will play an instrumental role in the sequel.

Proposition A.1. Let $\mathbf{x} \in \mathbb{S}_{d, 1}(\breve{b}(d+1))$, $n \geq 1$, $0 < \breve{b} < \left(e^{-16 \sqrt{2}} \wedge d^{-1}\right)$, $0 < a \leq e^{-1}\|f\|_{\infty}|\log \breve b| / \breve {b}^{d+1 / 2}$, and take the unique

Then, for all $h \in \mathbb{R}$,

where $C_{\varphi, d} > 0$ is a constant that depends only on the function $\varphi(\cdot)$ and the dimension $d$.

Proof of Proposition A.1. Following a similar approach to the proof in ^[117], we apply a union bound to show that the probability in (A.9) can be bounded as follows:

To clarify our notation, for any subset $\mathfrak{A} \subset \mathbb{R}^d$ and any point $\mathbf{x} \in \mathbb{R}^d$, we define $\mathbf{x} + \mathfrak{A} = \{\mathbf{x} + \mathbf{y} : \mathbf{y} \in \mathfrak{A}\}$. To bound term (A.10), consider the assumption $\mathbf{x} \in \mathbb{S}_{d, 1}(\breve{b}(d+1))$ and $\mathbf{x}^{\prime} = \mathbf{x} + [-\breve{b}, \breve{b}]^d$. This implies that both $\mathbf{x}$ and $\mathbf{x}^{\prime}$ are in $\mathbb{S}_{d, 1}(\breve{b})$, leading to the following relations:

and for $\mathbf{x}^{\prime}$,

Consequently, we have:

Combining these results with our assumption in (A.8) and the upper bound on the Dirichlet density from Lemma A.5 in ^[117], we obtain:

Given the assumptions $0 < \delta \leq e^{-1}$ and $0 < \breve{b} < \left(e^{-16 \sqrt{2}} \wedge d^{-1}\right)$, it follows that:

Term (A.11) represents the probability of encountering "too many bad observations", meaning too many $\mathbf{x}_i$s near the boundary of the simplex where the partial derivatives of the Dirichlet density with respect to $\alpha_1, \ldots, \alpha_d$, and $\beta$ diverge. We can control this term using a concentration bound. To begin, note that the volume of $\mathbb{S}_{d, 1} \setminus \mathbb{S}_{d, 1}(\delta)$ is at most $2 d \delta / d!$. Specifically, $\mathbb{S}_{d, 1}(\delta)$ forms a simplex of side-length $1-2\delta$ within $\mathbb{S}_{d, 1}$, so:

where we used the inequality $(1+x)^n \geq 1 + nx$, which holds for all $n \in \mathbb{N}$ and $x \geq -1$. From (A.18) and knowing that $\|f\|_{\infty}$ is finite (since $f(\cdot)$ is continuous by assumption and $\mathbb{S}_{d, 1}$ is compact), we get:

Applying Hoeffding's inequality and condition (A.8), we obtain:

To bound the third probability in (A.13), the main idea is to decompose the supremum using a chaining argument and apply concentration bounds on the increments at each level of the $d$-dimensional tree. With the notation $\mathcal{H}_k : = 2^{-k} \cdot \breve{b} \mathbb{Z}^d$, we have the following embedded sequence of lattice points:

Hence, for $\mathbf{x} \in \mathbb{S}_{d, 1}(\breve{b}(d+1))$ fixed, and for any $\mathbf{x}^{\prime} \in \mathbf{x} + [-\breve{b}, \breve{b}]^d$, let $\left(\mathbf{x}_k\right)_{k \in \mathbb{N}_0}$ be a sequence that satisfies:

and

Since the map $\mathbf{x} \mapsto \frac{1}{n} \sum_{i = 1}^n Z_{i, b}^{(T)}(\mathbf{x})$ is almost surely continuous,

and since $\sum_{k = 0}^{\infty} \frac{1}{2(k+1)^2} \leq 1$, we have the inclusion of events,

By a union bound and the fact that $\left|\mathcal{H}_k \cap [-\breve{b}, \breve{b}]^d\right| \leq 2^{(k+2)d}$,

Using Azuma's inequality and Lemma A.9 ^[117] (note that $\mathbf{x} \in \mathbb{S}_{d, 1}(\breve{b}(d+1))$ and $\mathbf{x}^{\prime} \in \mathbf{x} + [-\breve{b}, \breve{b}]^d$ implies $\mathbf{x}_k \in \mathbb{S}_{d, 1}(\breve{b})$ for all $k \in \mathbb{N}_0$, so that $\alpha_1 = \left(\mathbf{x}_k\right)_1 / \breve{b} + 1, \ldots, \alpha_d = \left(\mathbf{x}_k\right)_d / \breve{b} + 1, \beta = \left(1 - \left\|\mathbf{x}_k\right\|_1\right) / \breve{b} + 1 \geq 2$ for all $k \in \mathbb{N}_0$) and (A.7), the above is

The minimum of $k \mapsto 0.99 \cdot 2^{2k-1} (k+1)^{-4}$ on $\mathbb{N}_0$ is larger than, say, $1/16$, so we deduce

for some large constant $C_{f, d} > 0$. Putting (A.17), (A.19), and (A.21) together in (A.13) concludes the proof of Proposition A.1. □

The following corollary is very similar to Corollary 2 of ^[117].

Corollary A.2 (Large deviation estimates). Recall $Z_{i, b}^{(T)}(\mathbf{x})$ defined in (A.5). Let $\mathbf{x} \in\mathbb{S}_{d, 1}(\breve{b}(d+1)), n \geq 100^6 d^6, n^{-1 / d} \leq \breve{b}\leq\left(e^{-16 \sqrt{2}} \wedge d^{-1}\right)$ $0 < a \leq e^{-1}\|f\|_{\infty}|\log \breve b| / \breve {b}^{d+1 / 2}$, and take the unique

Then, we have

where $C_{f, d} > 0$ is a constant that depends only on the density $f(\cdot)$ and the dimension $d$.

Proof of Corollary A.2. By applying a union bound, we find that the probability in Eq (A.22) can be bounded as follows:

The first probability can be bounded using Proposition 1, and the second probability can be similarly bounded by applying Azuma's inequality and Lemma 4 from ^[117], as was done in Eq (A.20). Now, we turn to the proof of (A.4). We start by noting:

To prove (A.4), we need to show that the remainder term is asymptotically negligible, i.e.,

This follows directly from the proof of the remainder term for the $U$-statistics developed subsequently. Additionally, we need to prove:

This equation is obtained by a union bound over the suprema on hypercubes of width $2\breve{b}$ centered at each $\mathbf{x} \in 2\breve{b}\mathbb{Z}^d \cap \mathbb{S}_{d, 1}(\breve{b}(d+1))$, using the large deviation estimates in Corollary A.2, and choosing

The upper bound condition on $a$ is satisfied as long as $100 d^2 (\log n)^{3/2} / (\sqrt{n}) \leq e^{-1}$, which is valid if $n \geq 100^6 d^6$. For the unique $\delta \in (0, e^{-1}]$ that satisfies

we obtain:

The condition on $\delta$ in (A.25) implies:

since the function $x \mapsto x |\log x|$ is increasing on $(0, e^{-1}]$. Using (A.26) in (A.25), we get:

Since we assumed that $\breve{b} \geq n^{-1/d}$, the above is $\leq C_{f, d} n^{-3}$, which is summable. By our choice of $a$ in (A.24) and the Borel-Cantelli lemma, we obtain

Now, we only need to study the bias term,

Using the same reasoning as ^[79], we have

where $\mathbf{\zeta}_{\mathbf{x}} = (\zeta_{x_1}, \ldots, \zeta_{x_d})\sim \operatorname{Dirichlet}\left(\boldsymbol{\alpha}, \beta\right)$. By a second-order Taylor expansion around $\mathbf{\zeta}_{\mathbf{x}} = \mathbf{x}$, we have

for some $\overline{\mathbf{x}}$ joining $\mathbf{\zeta}_{\mathbf{x}}$ and $\mathbf{x}$. In addition, for all $j, k \in\{1, \ldots, d\}$, straightforward calculations yield (for instance, see ^[117]):

Then, the Cauchy-Schwartz inequality, (A.28), and (A.29) yields:

Finally, we obtain

and

By integrating the acquired findings with the given information,

gives us the desired result. Therefore, the proof is conclusive. □

□

A.1.2.   Proofs of Section 3.2

Proof of Theorem 3.2. Let $\mathbf{x} \in \mathbb{S}_{d, 1}(\breve{b}(d+1))$, $n \geq 1$, $0 < \breve{b} < \left(e^{-16 \sqrt{2}} \wedge d^{-1}\right)$, $0 < a \leq e^{-1}\|f\|_{\infty}|\log \breve b| / \breve {b}^{d+1 / 2}$, and take the unique

Define $\tilde{\mathbf{x}}' = (\mathbf{x}'_1, \ldots, \mathbf{x}'_m)$ such that $\tilde{\mathbf{x}}' \in \tilde{\mathbf{x}}+[-\breve{\mathbf{b}}, \breve{\mathbf{b}}]^m$, where $\tilde{\mathbf{x}} = (\mathbf{x}_1, \ldots, \mathbf{x}_m)\in \mathbb{S}^m_{d, 1}(\breve{b}(d+1))$, and $\breve{\mathbf{b}}: = (b, \ldots, b)$ is a $d$-dimensional vector, then we have:

As we explained before, to establish uniform convergence rates, we will be studying the convergence of the truncated part and the remainder part of $u_{n, 1}(\varphi, \tilde{\mathbf{x}})$ respectively.

Truncated Part:

Notice that

where

Mirroring the approach used in the proof of Theorem 3.1, we begin by establishing continuity estimates for the random fields $\tilde{\mathbf{z}}\mapsto \mathcal{W}^{(T)}(\tilde{\mathbf{z}})$ so that we get to control the probability that $\mathcal{W}^{(T)}(\tilde{\mathbf{z}})$ and $\mathcal{W}^{(T)}(\tilde{\mathbf{z}}')$ are too far apart when $\tilde{\mathbf{z}} = (\tilde{\mathbf{x}}, \tilde{\mathbf{y}})$ and $\tilde{\mathbf{z}}' = (\tilde{\mathbf{x}}', \tilde{\mathbf{y}})$ are close. Building on the framework established in Proposition 1 of ^[117], we present the following proposition to determine the behavior of $\left(u_{n, 1}^{(m)}(\mathcal{W}^{(T)}(\tilde{\mathbf{x}}', \tilde{\mathbf{y}}))-u_{n, 1}^{(m)}(\mathcal{W}^{(T)}(\tilde{\mathbf{x}}, \tilde{\mathbf{y}}))\right)$.

Proposition A.3. Let $\mathbf{x} \in \mathbb{S}_{d, 1}(\breve{b}(d+1))$, $n \geq 1$, $0 < \breve{b} < \left(e^{-16 \sqrt{2}} \wedge d^{-1}\right)$, $0 < a \leq e^{-1}\|f\|_{\infty}|\log \breve b| / \breve {b}^{d+1 / 2}$, and take the unique

Then, for all $h \in \mathbb{R}$, we have

where $C_{\varphi, dm} > 0$ is a constant that depends only on the function $\varphi(\cdot)$, the dimension $d$, and the degree $m$.

Proof of Proposition A.3. Similar to the proof of Proposition 1, by a union bound the probability in (A.33) is

where $C_m^n$ as usual is equal to $n!/\{m!(n-m)!\}$. Let us begin with (A.34). Following the same reasoning as Proposition A.1, on the event

we have

The latter equation is obtained by (A.14), (A.15), and Lemma 2 (^[117]). Therefore, we have

Note that $0 < \delta \leq e^{-1}$ and $0 < \breve{b} < \left({e^{-8 \sqrt{2}}} \wedge d^{-1}\right)$ by assumption. This implies that the probability in (A.34) equals zero. Next, we apply Hoeffding's inequality to control the probability in (A.35). Since $0\leq \mathbf{1}_{\left\{\tilde{\mathbf{x}} \in \mathbb S_{d, 1}^m \backslash \mathbb S_{d, 1}^m(\delta)\right\}}\leq 1$ and

we have}, for $t = 2\cdot2^m\|f\|^m_{\infty} \delta^m-\mu$, by considering (A.32)

Moving on to (A.36), then

where the linear term

can be treated similarly to the proof of Proposition 1. Now, for the nonlinear term, let us first introduce the following class of functions:

then we have for all $\varepsilon > 0$

We have

Using the same reasoning as in ^[5], one can find a positive constant $c_0 > 0$ such that

Now, making use of (A.32) and applying Proposition 4 of ^[5] gives us, for $\varepsilon = a^m n^{-1 / 2}$,

We can find a constant $\mathbb C_1 > 0$, such that

which implies

Therefore, we readily infer that

Hence, the proof of the proposition is complete by an application of the Borel-Cantelli lemma. □

Remainder Part:

We now consider the remaining part. Recall that the $U$-statistic $u_{n, 1}^{(R)}(\varphi, \tilde{\mathbf{x}})$ is related to the unbounded kernel given by

We have to establish that it is negligible, meaning that

For $\tilde{\mathbf{x}}, \tilde{\mathbf{y}} \in \mathbb S_{d, 1}^m$, observe that

Taking into account that $\widetilde{F}$ is symmetric, we have

where $u_{n, 1}^{(m)}\left(\widetilde{F}(\mathbf{y})\mathbf{1}_{\{ \varphi(\mathbf{y}) > \lambda\xi_n^{1/(1+\gamma)}\}}\right)$ is a $U$-statistic based on the $U$- kernel $\widetilde{F}\mathbf{1}_{\{ \varphi > \lambda\xi_n^{1/(1+\gamma)}\}}$,

and

Therefore, as $n \longrightarrow \infty$, we have, almost surely,

Hence, to achieve the proof, it remains to establish that

An application of the Chebyshev's inequality, for any $\eta > 0$, gives

so by using the fact that

we deduce that

Finally, note that (A.41) implies

The preceding results of the arbitrary choice of $\lambda > 0$ show that (A.43) holds, which, by combining with (A.42) and (A.41), completes the proof of (A.40). We finally obtain

Hence, the proof is complete. □

Proof of Theorem 3.3. Recall (A.2), and we have:

where

Notice that, given the imposed hypothesis and previously obtained results, and for some positive constants $c_1, c_2 > 0$, we readily infer

Hence by Theorem 3.2, for some $c" > 0$, we get with probability 1:

Hence, the proof is complete. □

Proof of Theorem 3.4. By (A.3), it suffices to establish

and

Let us start with (A.45). We have

where $\tilde{\boldsymbol{\xi}}_{\tilde{\mathbf{x}}} = (\xi_{\mathbf{x}_1}, \ldots, \xi_{\mathbf{x}_m})$ such that $\mathbf{\xi}_{\mathbf{x}_j} = \left(\xi_{x_{j_1}}, \ldots, \xi_{x_{j_d}}\right) \sim \operatorname{Dirichlet}(\boldsymbol{\alpha}_j, \beta_j)$; $j = 1, \ldots, m$. By a second-order Taylor expansion around $\tilde{\boldsymbol{\xi}}_{\tilde{\mathbf{x}}} = \tilde{\mathbf{x}}$, we get

for some $\tilde{\underline{\mathbf{x}}}$ joining $\tilde{\boldsymbol{\xi}}_{\tilde{\mathbf{x}}}$ and $\tilde{\mathbf{x}}$. Keep in mind that $\mathbf{\xi}_{\mathbf{x}_j}$ are vectors whose components are beta-distributed, which implies that

Then, the Cauchy-Schwartz inequality, (A.47), and (A.48) yields uniformly on $\tilde{\mathbf{x}} \in \mathbb{S}^m_{d, 1}$

Consequently, we deduce that

which implies that for $\varphi\equiv 1$,

As an immediate consequence of the last two equations, we infer that

This completes the proof of the theorem. □

A.1.3.   Proofs of the results of Section 3.3

Before we start the proofs, we will state some lemmas that are necessary to obtain the desired results. It is worth mentioning that we will follow the steps of ^[134] while making the appropriate changes to fit our general setting.

Lemma A.4. Under assumptions (A.1)–(A.4), and if $\mathbb{E}\varphi^2 < \infty$, the Hájek projection $\widehat{U}_{n, 1}$ of $\mathbb{U}$ satisfies, as $n\rightarrow \infty$:

(i)

where

(ii) and if, in addition, assumption (A.5) is verified, we have

In the following lemma, we show that $\mathbb{U}$ has the same asymptotic distribution as $\widehat{U}_{n, 1}$.

Lemma A.5. Under assumption (A.1)–(A.6), we have, as $n\rightarrow \infty$,

Specification of $\sigma^2(\varphi)$ leads to the following lemma.

Lemma A.6. ^[134] Under assumptions (A.1)–(A.6), we have, as $n\rightarrow \infty$,

with

and where for two functions $g_1(\cdot)$ and $g_2(\cdot)$,

with $\mathbf Y$ entering in the $j^{\rm th}$ and $l^{\rm th}$ positions.

Proof of Lemma A.4. We write

The Hájek projection $\widehat{U}_{n, 1}(\varphi, \tilde{\mathbf{x}})$ of $\mathbb{U}_{n, 1}(\varphi, \tilde{\mathbf{x}})$ satisfies

with

and $\varphi_{n, j}({\mathbf{x}}, {\mathbf{y}})$ is defined by

where $\mathbb{P}$ represents the underlying probability measure. By independence, we get

Using the fact that $(\mathbf{X}, \mathbf{Y})$, $(\mathbf{X}_i, \mathbf{Y}_i)_{1\leq i\leq 2m}$ are i.i.d, we get

By condition (A.2), we have, for $\mathbf{x}_j\neq\mathbf{x}_l$

Consequently,

In the case of $\mathbf{x}_j = \mathbf{x}_l$ and $\tilde{\mathbf{x}} = (\mathbf{x}_1, \ldots, \mathbf{x}_m)$ being a point of continuity for $r_{j, l}^{(m)}$, we have

by a differentiation argument. Now, provided that the density function $f(\cdot)$ is continuous and $f(\mathbf{x}_j) = f(\mathbf{x}_l) > 0$, by the fact that

where

and if $\boldsymbol{\gamma}_{\mathbf{x}} \sim \operatorname{Dirichlet}\left(\frac{2\mathbf{x}_j}{\breve{b}}+\textbf{1}, \frac{2\left(1-\|\mathbf{x}_j\|_1\right)}{\breve{b}}+1\right)$, then

where

We readily infer that

Hence, by Lemma B.4, we have

Using the fact that $r^{(m)}(\varphi, \tilde{\mathbf{x}})$ is bounded in the neighborhood of $\tilde{\mathbf{x}}$, then $\theta_n, n\ge 1$ is bounded, therefore we have $\breve{b}^{1/2}\theta_n^2 \rightarrow 0$, from which we have

where $\sigma^2(\varphi)$ is defined in (A.52), which yields the assertion (ⅰ). Now, to prove (A.53), we only need to verify Lyapunov's condition for third moments, i.e.,

Using the fact that

an upper bound for the absolute third moment of $\bar{\varphi_n}(\mathbf{X}, \mathbf{Y})$ is dominated by sums of the form

Following the same steps as in ^[134], we may restrict ourselves to triples $(i, j, l)$ such that $\mathbf{x}_i = \mathbf{x}_j = \mathbf{x}_l$. Under (A.5), we have

taking into account (A.1), which gives us the desired result. This concludes the proof. □

Proof of Lemma A.5. To study the asymptotic distribution of $U_n$, we need to bound the variance of $U_n -\widehat{U}_{n, 1}$. To do that, it is sufficient to show that

As in ^[134], using the centered variance formula for a centered, or zero mean $U$-statistic of degree $m$, for $\mathbf{Z}_i, i\ge 1$ i.i.d., we have

with a non-necessary symmetric $U$-Kernel $\tilde{G}(\cdot)$, that is square-integrable, which gives us

where $\Delta_1$ and $\Delta_2$ represent positions of some length $1 \leq r \leq m$, and

with the $y$ 's in position $\tilde\Delta_{2}$ coincide with the $z$ 's in position $\tilde\Delta_{1}$ and are taken from $z_{m+1}, \ldots, z_{2m-r}$ otherwise. Moreover, $\Sigma^{(r)}$ represents the summation over all positions $\tilde\Delta_{1}, \tilde\Delta_{2}$ with a cardinality of $r$, and $F(\cdot)$ denotes the common distribution function of the $Z$'s. When considering $V_{n} = U_{n}-\hat{U}_{n}$ and recalling $\tilde{G}$ from ^[129] (in the symmetric case), we obtain

Furthermore, by (A.6), we infer that

In conclusion, we have

Hence, the proof is complete. □

Proof of Theorem 3.6. To obtain the desired result, we shall first apply the Cramér-Wold device to investigate the asymptotic behavior of the two-dimensional vector

where $\mathbb{U}_{n, 1}(\varphi_1, \tilde{\mathbf{x}})$ and $\mathbb{U}_{n, 1}(\varphi_2, \tilde{\mathbf{x}})$ are $U$-statistics with $U$ kernels $G_{\varphi_1, \tilde{\mathbf{x}}, 1}(\cdot)$ and $G_{\varphi_2, \tilde{\mathbf{x}}, 1}(\cdot)$ respectively, satisfying the smoothness assumptions of Lemma A.5. Let $c_1$ and $c_2$ be any two real numbers. We can see that

which means we can apply Lemma A.5. The limit distribution of $\widehat{r}^{(m)}_{n}(\varphi, \tilde{\mathbf{x}}, \mathbf{b}_n)$ may now be easily deduced from Lemma A.6. We have

Let us define, as in ^[134],

which means

Since, by continuity of $r^{(m)}(\varphi, \tilde{\mathbf{x}})$, we have

and

From Lemma A.6, we deduce that

where

and $\rho^{2}$ is defined in (3.10). This concludes the proof of Theorem 3.6. □

A.2.   Proof of Section 4: Bernstein polynomials

Proof of Theorem 4.2. In order to prove Theorem 4.2, we need

and

The result in (A.59) is established in the following proposition.

Proposition A.7. Assume that condition (C.2) holds. We have, uniformly for $\mathbf{x} \in\mathbb{S}_{d, 1}$,

where

It is important to mention that this proposition closely follows Proposition 2 in ^[115]. However, it offers greater generality as it holds for any function $\varphi(\cdot)$ that satisfies condition (C.3), thereby extending its range of applications.

Proof of Proposition A.7. We first remark that

For all $\varepsilon > 0$, there exists a decreasing sequence $(\delta_{n, \varepsilon})_n$, $0 < \delta_{n, \varepsilon}\leq 1$ such that for all $\mathbf{x}, \mathbf{y}\in\mathbb{S}_{d, 1}$, $\|\mathbf{x}-\mathbf{y}\|\leq \delta_{n, \varepsilon}$. By a Taylor's expansion for any $\mathbf{k}$ such that $\|\mathbf{k}/\vartheta-\mathbf{x}\|_1 = o(1)$, we obtain

If we multiply the last expression by $\vartheta^{-d} \cdot \frac{(\vartheta-1+d)!}{(\vartheta-1)!} P_{\mathbf{k}, \vartheta-1}(\mathbf{x})$ and sum over all $\mathbf{k} \in \mathbb{N}_0^d \cap(\vartheta-1)\mathbb{S}_{d, 1}$, then considering the well-known identities

and the fact that

it yields

assuming that $\|\mathbf{k} / \vartheta-\mathbf{x}\|_1 = o(1)$ decays slowly enough to 0 that the contributions coming from outside the bulk are negligible. Hence, the proof of the proposition is complete. □

Next, based on the results of Proposition A.7, we can see that

which implies considering the Lipschitz continuity of $\mathcal{R} (\varphi, \cdot)$ uniformly for $\mathbf{x} \in \mathbb{S}_{d, 1}$

Finally, all it takes to get the bias term is to apply the Cauchy-Schwarz inequality in the last expression, followed by an application of (A.62), and we obtain

Use the fact that

and

Combining (A.63) with the fact that $r^{(1)}(\varphi, \mathbf{x}) = \frac{\widehat{g}_{n, \vartheta}(\varphi, \mathbf{x})}{f(\mathbf{x})}\cdot \frac{f(\mathbf{x})}{\widehat{f}_{n, \vartheta}(\mathbf{x})}$, completes the proof of (A.59). Now, we only need to prove (A.60), noticing that

where

For every $\vartheta$, the random variables $Y_{1, \vartheta}, \ldots, Y_{n, \vartheta}$ are i.i.d. and centered. Set

then, we have

where

So we can write

To obtain the desired result, we need to prove that

and

Note that the proof of $\mathcal{T}_2$ can be derived directly from the proof of the remainder part of the conditional U-statistics in the following section. Moving on to $\mathcal{T}_1$, we have

where

Set $\varsigma_n: = \sqrt{\frac{\log n}{n}}$, and

By a union bound on $\mathbf{k} \in \mathbb{N}_0^d \cap(\vartheta-1)\mathbb{S}_{d, 1}$ (there are at most $\vartheta^d$ such points) and Bernstein's inequality, we have, for all $\rho > 0$,

where the second inequality assumes that $\vartheta \leq n / \log n$ (equivalently, $\varsigma_n\leq \vartheta^{-1 / 2}$), and $c \geq \rho$ is a Lipschitz constant for $f(\cdot)$. If we choose $\rho = \rho(c, d) > 0$ large enough, then the righthand side of (A.67) is summable in $n$, and the Borel-Cantelli lemma implies

Hence, the proof of Theorem 4.2 is complete. □

Proof of Theorem 4.4. Recall the notation (A.1). We will start by investigating the term $u_{n, 2}^{(T)}(\varphi, \tilde{\mathbf{x}}).$

Truncated Part:

Notice that

where

It suffices to consider

We have $\mathbb{E}(Z_i) = 0$, and by condition (C.2),

Applying Bernstein's inequality on

we have

where $C_{\delta, dm}$ is a Lipschitz constant of the cumulative distribution function of $\tilde{\mathbf{X}}$. We obtain, for $\kappa > 0$,

Remainder Part:

Recall that

Now, using the fact that for $\left|\varphi(\mathbf{\tilde{\mathbf{Y_i}}})\right| > \omega_{n, 2}$, we have $(\left|\varphi(\mathbf{\tilde{\mathbf{Y_i}}})\right|/\omega_{n, 2})^{1+\gamma} > 1$, which implies that

where, by Assumption (C.3), we have

Therefore, if we set of $\omega_{n, 2} = (\vartheta^{-m/2}\varsigma_n)^{-1/(1+\gamma)}$, it follows that $\left|\mathbb{E}\left[u_{n, 2}^{(R)}(\varphi, \tilde{\mathbf{x}})\right]\right|\leq O(\vartheta^{-m/2}\varsigma_n)$ uniformly on $\tilde{\mathbf{x}} \in \mathbb{S}_{d, 1}^m$. Consequently, Markov's inequality gives us

Hence, the proof is complete. □

Proof of Theorem 4.5. The proof of Theorem 4.5 is similar to the proof of Theorem 3.3. Using the decomposition (A.2) and the fact that for some positive constants $c_1, c_2 > 0$, we readily infer

Set $\varsigma_{n, 2}: = \vartheta^{m(d-1/2)}(n^{-1}\log n)^{1/2}$. By Theorem 4.4, for some $c" > 0$, we get with probability 1:

Hence, the proof is complete. □

Proof of Theorem 4.6. We have

It suffices to control

and that

Let us start with (A.71), and we have the following

Proposition A.8. Assume that condition (C.2) holds. We have, uniformly for $\tilde{\mathbf{x}} \in\mathbb{S}^m_{d, 1}$,

where

Proof of Proposition A.8. Let us start with

By a Taylor's expansion for any $\tilde{\mathbf{k}}$ such that $\|\frac{\tilde{\mathbf{k}}}{\vartheta}-\tilde{\mathbf{x}}\|_1 = o(1)$, we obtain

Multiplying the last expression by $\vartheta^{-dm} \cdot\left(\frac{(\vartheta-1+d)!}{(\vartheta-1)!}\right)^m\prod\limits_{j = 1}^m P_{\mathbf{k}_j, \vartheta-1}(\mathbf{x}_j)$, summing over all $\tilde{\mathbf{k}}\in (\mathbb{N}_0^d \cap(\vartheta-1)\mathbb{S}_{d, 1})^m$, and using the same principle as the identities (A.61) and (A.62) leads to

assuming that $\|\frac{\tilde{\mathbf{k}}}{\vartheta}-\tilde{\mathbf{x}}\|_1 = o(1)$ decays slowly enough to 0 so that the contributions coming from outside the bulk are negligible. Hence, the proof of the proposition is complete. □

Now, recall that based on (A.73), we get that

and multiplying the last expression by

and summing over all

we obtain

which implies that together with the fact that the function $\mathcal{R}(\varphi, \cdot)$ is Lipscitz continuous

Finally, an application of the Cauchy-Schwarz inequality combined with the identity (A.62) gives us uniformly for $\tilde{\mathbf{x}} \in \mathbb{S}^m_{d, 1}$,

Therefore, we obtain

It remains to prove (A.72), which can easily be verified by simply taking $\varphi\equiv 1$ in the above equation. Combining the previously obtained results,

and if we suppose that $\inf\limits_{\tilde{\mathbf{x}}\in\mathbb{S}^m_{d, 1}}\tilde{f}(\tilde{\mathbf{x}}) > 0$, we can infer that

Hence, the proof is complete. □

A.3.   Proof of Section 5: Beta kernels

Let $\mathbf{A}_h, h = 1, \ldots, N_n^d$ be the $h$-th sub-hyper-rectangle. Also let $\mathbf{x}_h$ be the most distant point in $\mathbf{A}_h$ from the origin, that is, $\mathbf{x}_h: = \arg \max _{\mathbf{x} \in \mathbf{A}_h}\|\mathbf{x}\|$. Suppose that the design point $\mathbf{x}$ falls into $\mathbf{A}_h$. Then, for all $\tilde{\mathbf{x}} = (\mathbf{x}_1, \ldots, \mathbf{x}_m)$, we denote $\tilde{\mathbf{x}}_{h} = (\mathbf{x}_{1, h}, \ldots, \mathbf{x}_{m, h})$ such that $\tilde{\mathbf{x}}_{h}: = \arg \max _{\tilde{\mathbf{x}} \in \mathbf{A}_h^m}\|\tilde{\mathbf{x}}\|$. For $\tilde{\mathbf{x}} = (\mathbf{x}_1, \ldots, \mathbf{x}_m) \in \mathbb S_{\tilde{\mathbf{x}}}: = \prod\limits_{i = 1}^m \mathbb{S}_{\mathbf{X}_i}$, where

the boundary parameters $\boldsymbol{\eta}_i: = \left(\eta_{i_1}, \ldots, \eta_{i_d}\right)$ either are fixed or shrink to zero at a suitable rate. For each $1\leq i\leq m$, we divide every edge of the $d$-hyper-rectangles $\mathbb{S}_{\mathbf{X}_i}$ into $N_n$ evenly spaced grids, resulting in $N_n^d$ identical sub-hyper-rectangles. For any $\tilde{\mathbf{x}} = (\mathbf{x}_1, \ldots, \mathbf{x}_m) \in \mathbb{S}_{\mathbf{X}}^m$, there exists $\boldsymbol{\ell}(\tilde{\mathbf{x}}) = (\ell(\mathbf{x}_1), \ldots, \ell(\mathbf{x}_m))$ such that for all $1\leq i \leq m$, $1\leq \ell(\mathbf{x}_i) \leq N_n^d$, and

For each $\tilde{\mathbf{x}}\in \mathbb{S}_{\mathbf{X}}^m$, we consider the $U$-statistic as

Before proceeding, we borrow a few lemmas from ^[79], all of which are key building blocks for the technical proofs below. Throughout, $\theta_{x_j}$ denotes a beta random variable so that

Lemma A.9. Let $\theta_{x_j}$ and $\theta_{x_k}$ be independent for $j \neq k$. Then, as $n \rightarrow \infty$, we have

Lemma A.10. Suppose that $b( = b(n) > 0)$ and $\eta( = \eta(n) > 0$) satisfy $b, \eta \rightarrow 0$ and $b / \eta \rightarrow 0$ as $n \rightarrow \infty$. Then, as $n \rightarrow \infty$, we have

Lemma A.11. Under the same condition as in Lemma A.10, as $n \rightarrow \infty$, we have

where $\gamma = 0.5772 \ldots$ is Euler's constant.

A.3.1.   Proof of Theorem 5.1

Proof of Theorem 5.1. To establish this theorem, we'll have to truncate the conditional $U$-statistic. First, let's introduce the following notation:

From (A.1), we know that we can write

Using the same truncation technique we used in the previous sections' proof, we need to establish the results for the truncated and remainder parts.

Truncated Part:

Let us remark that

where

We apply Lemma B.2 on the function $H^{(T)}(\cdot, \cdot)$. Throughout the rest of the proof, we suppose that the function $\mathcal{G}_{\varphi, \tilde{\mathbf{x}}, 3}^{(T)}$ is symmetric. Moreover, by Lemma A.10, for a sufficiently large $n$, we readily infer

We also remark that

One can easily derive

Using Lyapunov's inequality (2.7), and condition (C.3), for $C_0, C_1\ge 1$, we have

In addition, recall that

By Lemma of ^[40], the first term is bounded by $b^{-1 / 2}(1+b)^{3 / 2} /\{2 \sqrt{\pi} \sqrt{x(1-x)}\}$ for a sufficiently large $n$. The second term is the pdf of Beta $\{2 x / b+1, 2(1-x) / b+1\}$. Therefore, we derive

For a sufficiently large $n, b_{j_1}, \ldots, b_{j_d}$ and $\eta_{{j_1}}, \ldots, \eta_{j_d}$, $1\leq j\leq m$ are no greater than $1 / 2$, and thus

where $\rho^2: = C_0 C_1\left(\frac{3}{4} \sqrt{\frac{3}{\pi}}\right)^{dm}$. For any $\varepsilon > 0$ and $n$ large enough, we get that

Taking into account $\phi_n = o(1)$ and $\frac{2}{3}\left(\frac{9}{4\sqrt{\pi}}\right)^{dm}\frac{\varepsilon\phi_n^{1-1/(1+\gamma)}}{\rho}\leq 1$ for sufficiently large $n$, it follows that

On the other hand, we have

We highlight that

Hence, the rate of

is determined by

By the mean-value theorem, we have

for some $\tilde{\underline{\mathbf{x}}}$ joining $\tilde{\mathbf{x}}$ and $\tilde{\mathbf{x}}_{\boldsymbol{\ell}(\tilde{\mathbf{x}})}$. For $k = 1, \ldots, m$, observe that

where, by Lemmas A.10 and A.11, for $\ell = 1, \ldots, d$, we have

uniformly on $(\tilde{\mathbf{x}}, \tilde{\mathbf{u}}) \in \mathbf{A}_h^m \times [0, 1]^{dm}$ and

which implies

Using the fact that $\sup\limits _{\tilde{\mathbf{x}} \in \mathbf{A}_h^m}\left\|\tilde{\mathbf{x}}-\tilde{\mathbf{x}}_{\boldsymbol{\ell}(\tilde{\mathbf{x}})}\right\| = O(N_n^{-m})$, it follows that

uniformly on $(\tilde{\mathbf{x}}, \tilde{\mathbf{u}}) \in \mathbf{A}_h^m \times [0, 1]^{dm}$. Next, making use of (A.78), we have

Just like in the bounded scenario, the progression from (A.79) to (A.80) arises from Jensen's inequality and certain properties of the absolute value function. We can deduce that

For sufficiently large $n$ and each $m\geq2$, for some $\varepsilon > 0$, we infer that

Continue, now, with (A.76), by imposing that the kernel function $\mathcal{G}_{\varphi, \tilde{\mathbf{x}}_\mathbf{\ell}, 3}^{(T)}(\cdot)$ is symmetric and the $U$-statistic is decomposed according to ^[80] decomposition, that is,

Let us first start with the linear term. We have

From Hoeffding's projection (A.81), we have

Set

It's evident that $Z_i^{(T)}$ are independent and identically distributed random variables with a mean of zero, and

Making use of (A.75) and an application of Bernstein's inequality, for some $\varepsilon > 0$, yields

Moving to the nonlinear term, we will prove that for $2\leq q \leq m$:

which implies that, for $1\leq i\leq m$ and $\boldsymbol{\ell} = (\ell_1, \ldots, \ell_m)$:

To prove the abovementioned equation, we need to apply Proposition 1 of ^[5] (see Lemma B.3). We can see that $\mathcal{G}_{\varphi, \tilde{\mathbf{x}}_\mathbf{\ell}, 3}^{(T)}$ is bounded by $\left(\frac{9}{4\sqrt{\pi}}\right)^{dm}\frac{\phi_n^{2-1/(1+\gamma)}}{\log(n)}$, hence for $\varepsilon > 0$, we have

where $\varepsilon_0 = \frac{\varepsilon}{\sqrt{n}}$. Now for $t = \varepsilon\rho\phi_n$, Lemma B.3 gives

By the last result, it follows that there exists $\varepsilon > 0$ in such a way that

where

Therefore, for each $\varepsilon_0 > 0$, $1\leq i\leq m$ and $\ell = (\ell_1, \ldots, \ell_m)$, we infer that

By combining (A.75) and (A.83), for some $\varepsilon > 0$, it follows that

which implies for $\varepsilon = 2\sqrt{5d}$, as $n \rightarrow \infty$,

Hence, the proof is complete. □

Reminder Part:

Notice that

Now, using the fact that for $\left|\varphi(\mathbf{\tilde{\mathbf{Y_i}}})\right| > \omega_{n, 3}$, we have $(\left|\varphi(\mathbf{\tilde{\mathbf{Y_i}}})\right|/\omega_{n, 3})^{1+\gamma} > 1$, which implies that

where, by Assumption (C.3) and the fact that $\tilde{\mathcal{K}}_{\bar{\boldsymbol{\Lambda}}_{n, 3}(\tilde{\mathbf{x}})}(\cdot)$ is the density function of the product of $dm$ independent beta random variables $\boldsymbol{\theta}_{\mathbf{x}_i}: = (\theta_{\mathbf{x}_1}, \ldots, \theta_{\mathbf{x}_m}) \in [0, 1]^{dm}$, $i = 1, \ldots, dm$, we have

Hence, by the definition of $\omega_{n, 3}$, $\left|\mathbb{E}\left[u_{n, 3}^{(R)}(\varphi, \tilde{\mathbf{x}})\right]\right|\leq O(\phi_n)$ uniformly on $\tilde{\mathbf{x}} \in \mathbb{S}_{\mathbf{X}}^m$. Consequently, Markov's inequality gives us

Hence, the proof is complete.

Proof of Theorem 5.2. Similar to the proof of the bias terms in the previous sections, we know that based on (A.2) :

In addition, we have for some positive constants $c_1, c_2 > 0$,

Hence by Theorem 5.1, for some $c" > 0$, we get with probability 1:

Hence, the proof is complete. □

Proof of Theorem 5.3. To obtain the desired results, we need to prove that:

We first remark that

where $\boldsymbol{\theta}_{\tilde{\mathbf{x}}} = (\theta_{\mathbf{x}_1}, \ldots, \theta_{\mathbf{x}_m}) \in [0, 1]^{dm}$. Following the same reasoning as the proof of Theorem 4.4, a second-order Taylor expansion around $\boldsymbol{\theta}_{\tilde{\mathbf{x}}} = \tilde{\mathbf{x}}$ gives us:

for some $\tilde{\underline{\mathbf{x}}}$ joining $\boldsymbol{\theta}_{\tilde{\mathbf{x}}}$ and $\tilde{\mathbf{x}}$. Taking into account condition (C.2) and Lemma A.9 implies that

Taking $\varphi\equiv 1$ in the above equation gives us

Combining these two results leads to

and if we suppose that $\inf\limits_{\tilde{\mathbf{x}}\in \mathbb{S}^m_{\mathbf{X}}}\tilde{f}(\tilde{\mathbf{x}}) > 0$, we can infer that

Hence, the proof is complete. □

Proof of Theorem 5.5. Using the notation established in the proof of Theorem 5.1, and employing a reasoning akin to that of ^[79], we proceed to redefine $\omega_{n, 3}$ and $N_n$,

for an arbitrarily small $\varepsilon$. In order to prove Theorem 5.5, we need to demonstrate that

Similar to the proof of Theorem 5.1, for the reminder part, (A.85) and (A.86) give us

Moreover, by (C.3) and Markov's inequality, we infer readily that

Using the Borel-Cantelli lemma for a sufficiently large $n$, this gives us $|\varphi(\tilde{\mathbf{Y}}_n)|\leq\omega_{n, 3}$ with probability 1. This implies that $\left|\varphi(\tilde{\mathbf{Y}}_i)\right| \leq \omega_{n, 3}$ for any $i \leq n$ with probability 1 for a sufficiently large $n$. It follows that $u_{n, 3}^{(R)}(\varphi, \tilde{\mathbf{x}}) = 0$ with probability $1$, that is,

uniformly on $\tilde{\mathbf{x}} \in \mathbb{S}_{\mathbf{X}}^m$. Next, observe that

Hence, we infer that

Then, (A.75) and (A.83) hold for a sufficiently large $n$. In addition, (C.4') implies that

where the last inequality holds because $\left(\prod\limits_{j = 1 }^m \left\{\left(\prod\limits_{i = 1}^d b_{j_{i}} \eta_{j_{i}}\right)^{-\frac{1}{2}}\right\}\right)^{\frac{\kappa}{2}} / \log(n)$ is bounded. Then, picking

yields $N_n^d n^{-K^2 / 4} = O\left\{n^{-(1+\varepsilon)}\right\}$, so that

Therefore, by the Borel-Cantelli lemma, we obtain

Hence, the proof is complete. □

Proof of Theorem 5.6. The proof of Theorem 5.6 is done in the same fashion as the proof of Theorem 5.2, combining (A.2) with the results of Theorem 5.5. □

Proof of Theorem 5.7. The proof of Theorem 5.7 is the same as the proof of Theorem 5.3. □

B.   Appendix-2

This appendix contains supplementary information that is an essential part of providing a more comprehensive understanding of the paper.

Lemma B.1 (Lemma 2.2.9, ^[143]). Let $X_1, \ldots, X_n$ be independent random variables with bounded ranges $[-M, M]$ and zero means. Then,

for all $t$ and $v \geq \operatorname{Var}\left(\sum\limits_{i = 1}^n X_i\right)$.

Lemma B.2 (Theorem A. page 201, ^[129]). Let $f$ be a symmetric function taking its variables from $\mathbb S_{d, 1}$ satisfying $\left\Vert f \right\Vert_{\infty}\leq c$,

and

then for $t > 0$ and $n\geq m,$ we have:

Lemma B.3 (Proposition.1, ^[5]). If $G: S^{m} \rightarrow \mathbb{R}$ is a measurable symmetric function with $\|G\|_{\infty} = b$, then

Lemma B.4 (Lemma 1, ^[117]). We have, as $b \rightarrow 0$ and uniformly for $\mathbf{x} \in\mathbb{S}_{d, 1}$,

Furthermore, for any subset $\emptyset \neq \mathcal{J} \subseteq[d]$, and any $\kappa \in(0, \infty)^d$,

where $\psi(\cdot)$ and $\psi_{\mathcal{J}}(\cdot)$ are defined for every subset of indices $\mathcal{J} \subseteq[d]$, by

Lemma B.5 (Lemma 2, ^[117]). If $\alpha_1, \ldots, \alpha_d, \beta \geq 2$, then

Lemma B.6 (Lemma 3, ^[117]). If $\alpha_1, \ldots, \alpha_d, \beta \geq 2$, then for all $\mathbf{x} \in \operatorname{Int}\left(\mathbb{S}_{d, 1}\right)$,

Lemma B.7 (Lemma 4, ^[117]). If $\alpha_1, \ldots, \alpha_d, \beta, \alpha_1^{\prime}, \ldots, \alpha_d^{\prime}, \beta^{\prime} \geq 2$, and $\mathbf{X}$ is $F$ distributed with a bounded density $f$ supported on $\mathbb{S}_{d, 1}$, then

where $\boldsymbol{\alpha} \vee \boldsymbol{\alpha}^{\prime}: = \left(\max \left\{\alpha_i, \alpha_i^{\prime}\right\}\right)_{i \in[d]}, \beta \vee \beta^{\prime}: = \max \left\{\beta, \beta^{\prime}\right\}$, and $\beta \wedge \beta^{\prime}: = \min \left\{\beta, \beta^{\prime}\right\}$. Furthermore, let

Then, for $0 < \delta \leq e^{-1}$, we have