We suggest a dependence coefficient between a categorical variable and some general variable taking values in a metric space. We derive important theoretical properties and study the large sample behaviour of our suggested estimator. Moreover, we develop an independence test which has an asymptotic \chi^2-distribution if the variables are independent and prove that this test is consistent against any violation of independence. The test is also applicable to the classical K-sample problem with possibly high- or infinite-dimensional distributions. We discuss some extensions, including a variant of the coefficient for measuring conditional dependence.
For some categorical variable Y let \mathcal{L}_Y be the class of bijections between the K levels of Y and the integers \{1,\ldots, K\}. For any \varepsilon>0 there exists a categorical variable Y and a random variable X such that
Let U be uniformly distributed on the interval (0,1) and independent of X. Set {Q_k(X)=P(Y=k|X)} and define the variable
Then (\widetilde Y,X) and (Y,X) are identically distributed.
(i) The variables X and Y are independent if and only if Y and Y' are independent. (ii) There exists f\colon \mathcal{H}\to \{1,\ldots, K\} such that Y=f(X), if and only if there is a function \ell\in\mathcal{L}_Y such that Y=\ell(Y').
Note that p_{j}=q_{j}. Hence, in the population version we will replace the q_j’s by p_j’s.
The statement of our Lemma 2 is analogue to Theorem 1 of [fuchs_quantifying_2024], where it is formulated for continuous variables in terms of copulas.
It can be readily verified that
Thus, setting Q(X)=(Q_1(X),\ldots, Q_K(X))' and defining D=\mathrm{diag}(p_{1},\ldots, p_{K}), we may alternatively write
We will refer later to the alternative representations (3) and (3).
The dependence coefficient {\psi(X,Y)} satisfies properties ?–?. Moreover, \psi(X,Y)=\psi(X,\ell(Y)) for any \ell\in\mathcal{L}_Y.
Nearest neighbours have also been used by [azadkia_simple_2021] in the construction of their estimator for \xi.
We have been tacitly assuming that the nearest neighbours are unique. Essentially this means that the distance between two variables d(X_1,X_2) has a continuous distribution. From a technical point of view uniqueness of N(i) is not important for our results, but it allows to avoid some extra discussions (e.g. how to break ties) which are not essential for conveying the main ideas. Hence, for the rest of this paper this assumption is supposed to hold.
The computationally most intensive part in the calculation of \widehat\psi is finding the nearest neighbour indices {N(i)}. Since (approximate) nearest neighbour search can be done in {O(n\log n)} time, the computation time for \widehat\psi is {O(n\log n)} as well.
Assume that one of the following conditions holds:
(a) For almost all u we have f(X_{N(1)},u)\stackrel{P}{\to} f(X_1,u) for n\to\infty.
(b) For all k=1,\ldots,K, the mapping x\mapsto Q_k(x) is P_X-almost everywhere continuous.
Suppose that Assumption 1 holds. Then \widehat\psi(X,Y) converges in probability to {\psi(X,Y)}.
Assume that \mathcal{H}=\mathbb{R}^d, then Assumption 1 holds.
Let \mathcal{V}_k=\{(x,u)\in \mathcal{H}\times (0,1)\colon f(x,u)=k\}. If we have
then Assumption 1 holds.
Let Assumption 1 hold and assume that there is a sequence (k_n) such that
then the estimator \widehat\psi(X,Y) is strongly consistent.
We have L_n=o_P(n^{1/4}).
While L_n remains bounded if X takes values in \mathbb{R}^p, this is typically not the case in infinite dimensional vector spaces. Indeed, as demonstrated in Theorem 2 of [hormann_azadkiachatterjees_2024], one can give explicit examples of distributions for which the sequence L_n diverges at a polynomial rate, but Assumption 2 is satisfied.
Assume that X and Y are independent, set
and let \Sigma_n=\mathrm{Var}(P_n|X_1,\ldots,X_n).
(i) Then under Assumption 1
with
(ii) Let Assumption 2 hold and suppose that we have some b>0 such that
Then for large enough n the matrix \Sigma_n is invertible and
If \widehat \Sigma_n denotes the empirical analogue to \Sigma_n with p_k replaced by their estimators \widehat p_k, then we also have
To clarify the relationship between \mathcal{I}_n and \widehat\psi(X,Y), we refer to the representation of \psi(X,Y) given in (3). Observe that P_n/\sqrt{n} serves as the sample analogue of \mathrm{vec}(\mathrm{Var}(Q^{[-K]}(X))), where Q^{[-K]}(X) excludes the redundant component
from Q(X). To account for dependencies between the coordinates of P_n and to obtain a pivotal limit distribution, the weighting matrix of the quadratic form is suitably adjusted.
Suppose \mathcal{H}=\mathbb{R}^d and assume that X_i have a continuous density. Then we have W_{n}\stackrel{\mathcal{P}}{\longrightarrow} \gamma_d\vcentcolon=\frac{\mathrm{vol}(\mathcal{S}_d)}{\mathrm{vol}(\mathcal{S}_d\cup \mathcal{S}_{1,d})}, where \mathcal{S}_d is the unit-sphere in \mathbb{R}^d and \mathcal{S}_{1,d} is an affine shift of \mathcal{S}_d by distance 1. It holds that \gamma_d\in (1/2, 2/3], with \gamma_1=2/3 and \gamma_d\searrow 1/2 for d\to\infty.
Suppose that Y is a binary variable and that X takes values in \mathbb{R}^d. Then if X and Y are independent we have
Let Assumptions 1 and 2 hold. Assume that X and Y are not independent. Then \mathcal{I}_n\stackrel{\mathcal{P}}{\longrightarrow}\infty as n\to\infty. The divergence rate is of order \sqrt{n}.
As outlined in Section ?, testing independence between X and Y is equivalent to testing whether the K subsamples of X associated with different levels of Y originate from the same distribution. The test proposed above therefore also contributes to the K-sample problem, with particular relevance in the challenging setting of high- or even infinite-dimensional distributions.
The dependence coefficient {\psi(X,Y)} satisfies ?–?.
The conditional dependence measure \psi(Z,Y|X) satisfies properties ?–?. Moreover, {\psi(Z,Y|X)=\psi(Z,\ell(Y)|X)} for any {\ell\in\mathcal{L}_Y}.
One can also use \widehat\psi to get a test for conditional dependence by using the conditional randomization testing [candes_panning_2018] or conditional permutation testing [berrett_conditional_2020] frameworks.
If {\|.\|} is a strictly monotone norm and \gamma is a function as described above, then the measure \psi_{\|.\|,\gamma} satisfies the properties ?–? as well as ?–?.
Theorem 7 provides some intuition about the nature of our approach. Determining whether alternative norms yield more powerful independence tests necessitates an investigation into the large-sample behaviour of these coefficients. This analysis lies beyond the scope of the current study and is left for future research.
Note that in the extreme case \varepsilon=0 we have Y=B almost surely, and hence a degenerate case. As noted by [chatterjee_new_2021], in this case Y is both, independent of X and equal to a function of X and the measure \xi is undefined, as the denominator in (?) is equal to zero. In this specific example, each of the two encodings captures the “closeness” to this case of combined full dependence and independence from a different perspective. In turn, one can easily derive that {\psi(X,Y)=1/2} for all {\varepsilon\in (0,1)}.
We denote \Sigma_{g,n}=\mathrm{Var}(P_n|\mathcal{G}_n=g) and let \mathcal{K}_n, n\geq 1, be a sequence of collections of nearest neighbour graphs g with n vertices, such that
and such that the coefficient W_{n} defined in (?) and corresponding to g is at most {1-b/2}. Then, for any vector c on the (K-1)^2-dimensional unit sphere it holds that
Under the assumptions of Theorem 7 it holds that
with equality if and only if Y is independent of Z conditionally on X.
Let X_i be i.i.d. random elements in a separable metric space \mathcal{H}. Then as n\to\infty we have (i) X_{N(1)}\stackrel{a.s.}{\longrightarrow} X_1 and (ii) P(N(1)=N(2))=o(1).
Suppose Assumption 1 holds. Then for n\to\infty, we have
Let \mathcal{X}_1,\ldots, \mathcal{X}_n be a collection of arbitrary metric spaces and set {\mathcal{X}=\mathcal{X}_1\times\ldots\times\mathcal{X}_n}. Let \mathcal{Y}\subset \mathcal{X} and let X be an \mathcal{X}-valued random variable. Denote {p=P(X\notin \mathcal Y)}. If f:\mathcal{X}\to\mathbb{R} is such that for all pairs x,y\in\mathcal Y which are only distinct in the i-th coordinate we have
then it holds that
[combes_extension_2023] states Lemma S3 for \mathcal{X}_i = \mathbb{R}^p, but this is not relevant for the proof and it remains valid for any metric spaces.
It holds that
[[rinott_normal_1994], Thm. 2.2] Let V_1,\ldots,V_n be random variables having a dependence graph whose maximal degree is strictly less than k and satisfy
|V_i|\leq~B almost surely for i=1,\ldots,n,
\mathbb{E}\, V_i =0 and
\operatorname{Var}(\sum_{i=1}^n V_i)=1.
Then
Note that the term W_{n}' doesn’t appear in \Sigma_n. The corresponding summands cancel.
Fix K and let {E\in \mathbb{R}^{(K-1)\times (K-1)}} be a matrix with all entries equal to 1 and {I} be the identity matrix in {\mathbb{R}^{(K-1)\times (K-1)}}. Let D=\mathrm{diag}(p_1,\ldots, p_{K-1}) and denote A^{2\otimes}=A\otimes A. Finally, define the matrix {\mathcal{W}_n\in\mathbb{R}^{(K-1)^2\times (K-1)^2}} with
for 1\leq k_1,l_1,k_2,l_2\leq K-1. Then the following factorization holds:
The text on this page is from arXiv:2509.10268v2 by Siegfried Hörmann and Daniel Strenger-Galvis, under the licence CC BY 4.0. The corpus made these changes: it read the items out of the LaTeX source, it expanded the macros of the paper, it put each proof after its statement, and it converted the text to HTML. The formulas are the TeX of the paper, and KaTeX renders them. A formula or a reference can look different from the PDF. The page is published under CC BY 4.0.
The record of each item is in analysis/objects/2509/2509.10268.jsonl.