Random feature maps are used to decrease the computational cost of kernel machines in large-scale problems. The Mondrian kernel is one such example of a fast random feature approximation of the Laplace kernel, generated by a computationally efficient hierarchical random partition of the input space known as the Mondrian process. In this work, we study a variation of this random feature map by applying a uniform random rotation to the input space before running the Mondrian process to approximate a kernel that is invariant under rotations. We obtain a closed-form expression for the isotropic kernel that is approximated, as well as a uniform convergence rate of the uniformly rotated Mondrian kernel to this limit. To this end, we utilize techniques from the theory of stationary random tessellations in stochastic geometry and prove a new result on the geometry of the typical cell of the superposition of uniformly rotated Mondrian tessellations. Finally, we test the empirical performance of this random feature map on both synthetic and real-world datasets, demonstrating its improved performance over the Mondrian kernel on a dataset that is debiased from the standard coordinate axes.
The limiting kernel of the uniformly rotated Mondrian process with lifetime \lambda > 0 in \mathbb{R}^d is of the form
where \omega_d is the surface area of the unit ball in \mathbb{R}^d.
For any bounded domain \mathcal{X} \subset \mathbb{R}^d and small enough \delta > 0, we have that
where k_M : \mathbb{R}^d \times \mathbb{R}^d \to \mathbb{R} is the uniformly rotated Mondrian kernel of order M.
[Schneider2008, Equation 4.3] For any nonnegative measurable function f on the space of nonempty compact sets in \mathbb{R}^d, and for any stationary random tessellation X, we have that
where Z is the typical cell of the X.
Let X be the superposition of M independent rotated Poisson Manhattan tessellations X_1, \dots, X_M of intensity \lambda > 0 in \mathbb{R}^d, and let Z be the typical cell of X. First, it holds that
where V(C) is the volume of a compact, convex body C \subset \mathbb{R}^d and \kappa_d is the volume of the unit ball in \mathbb{R}^d. Second, it holds that
where r(C) is the inradius of a compact, convex body C \subset \mathbb{R}^d. Finally, it holds that
where R(C) is the circumradius of a compact, convex body C \subset \mathbb{R}^d.
We first compute the limiting kernel of a Mondrian process under a fixed rotation R \in SO_d, and we then compute the limiting kernel of the uniformly rotated Mondrian process by conditioning on the rotation R \in SO_d.
For a fixed rotation R \in SO_d, letting k_\infty be the rotated Mondrian limiting kernel and \hat{k}_\infty be the Mondrian limiting kernel, we have from Proposition 1 in [Balog2016] that
Letting \mu be the normalized Haar measure on SO_d, we compute that
from the law of total probability. Finally, from the translation invariance of the Haar measure \mu, we have from right multiplication by the rotation S \in SO_d that points x - x' in the direction of the axis-aligned unit vector e_1 \in \mathbb{R}^d that
as is desired, completing our computation of the limiting kernel of the uniformly rotated Mondrian. \square
We will first let B_r be a ball of radius r in \mathbb{R}^d such that \mathcal{X} \subseteq B_r. We will then let \mathcal{U} be an \varepsilon-grid covering of a concentric B_{R + \varepsilon} for some R > r and \varepsilon > 0 to be specified later. Letting X be the superposition of M independently rotated Mondrian tessellations, we will consider the following four events:
A_1 = \{\text{every cell of }X\text{ with a circumcenter}
\text{ outside of }B_R\text{ is disjoint from }B_r\},
A_2 = \{\text{every cell of }X\text{ with a circumcenter in }
\text{ }B_R\text{ contains a point of }\mathcal{U}\},
A_3 = \{\text{every cell of }X\text{ with a circumcenter}\text{ in}
B_R\text{ has a diameter less than or equal}
\text{ to }\delta/4\lambda \sqrt{d}\},
A_4 = \{\text{the }\delta/2\text{-approximation holds on }\mathcal{U}\},
where the \delta-approximation on an \varepsilon-grid covering \mathcal{U} is that
for all u,u' \in \mathcal{U}. It holds (see the Appendix) that A_1 \cap A_2 \cap A_3 \cap A_4 implies that the \delta-approximation holds on \mathcal{X}, and so we have that
from a union bound. It remains to bound the probability of the four events A_1^c, A_2^c, A_3^c, A_4^c. For A_1^c, letting c(C) be the circumcenter of a nonempty compact, convex set, we note that
By Campbell’s theorem and Lemma 1, we have that
where Z is the typical cell of X and \kappa_d is the volume of the unit ball in \mathbb{R}^d. Deriving similar bounds for the other events, we find that
This expression, when minimized with respect to \varepsilon, gives us the desired result for small enough \delta > 0. \square
Let X be the superposition of M independent rotated Poisson Manhattan tessellations X_1, \dots, X_M all with intensity \lambda > 0. For n=1, \ldots, M, note that the Poisson Manhattan tessellation X_n has directional distribution
for some orthonormal set of vectors \{u_{n,1}, \dots, u_{n,d}\}, and thus the associated zonoid of X_n has support function
The above quantity achieves the same maximum and minimum values over the unit sphere ||u||_{2} = 1 as
Thus, by the inequality ||u||_2 \leq ||u||_1 \leq \sqrt{d}||u||_2 that is tight over the unit sphere ||u||_{2} = 1, we have that
From Page 158 of [Schneider2008], we have that the associated zonoid of the superposition X of the independent stationary Poisson hyperplane processes X_1, \ldots, X_M is additive in the sense that
where the associated sum is the Minkowski sum of the associated zonoids, i.e., we have that \Pi_X = \left\{v_1 + \dots + v_M : v_n \in \Pi_{X_n}, 1 \leq n \leq M \right\}. Furthermore, as the support function of the Minkowski sum of convex bodies is additive, it follows that
From Theorem 10.3.3 and (10.4) in [Schneider2008], it holds that \mathbb{E}[V(Z)] = V(\Pi_X)^{-1}. Thus, we have that
This completes our computation of bounds for the expected volume of the typical cell of the superposition of independent rotated Mondrian tessellations. \square
Let X be the superposition of M independent rotated Poisson Manhattan tessellations X_1, \dots, X_M all with intensity \lambda > 0 in \mathbb{R}^d, where each X_n has directional distribution
for an orthonormal set of vectors \{u_{n,1}, \dots, u_{n,d}\}. It holds that X is the same in distribution as \widetilde{X} \cap U, where \widetilde{X} is an axis-aligned Poisson Manhattan tessellation in \mathbb{R}^{Md} with intensity \widetilde{\lambda} = M^{3/2}\lambda > 0 and U = \text{Im}(T) is a d-dimensional linear subspace of \mathbb{R}^{Md}, where the orthogonal transformation T : \mathbb{R}^{d} \to \mathbb{R}^{Md} is represented by the matrix
First, we recall from Page 158 of [Schneider2008] that the directional distribution of X is given by
We then have from (4.60) in [Schneider2008] that the intensity of the point process X \cap \text{Span}(u) for any u \in S^{d-1} is given by twice the support function h_{\Pi_X}(u) of the associated zonoid \Pi_X of X, i.e., we have that
as in our computation in proving Equation (1). Similarly, for any \widetilde{u} \in S^{Md - 1}, recalling that
is the directional distribution of \widetilde{X}, we have that
We will then consider the linear transformation T : \mathbb{R}^d \to \mathbb{R}^{Md} as in (1). We note for any u \in \mathbb{R}^d that
and so we have that
By uniqueness on Page 131 of [Schneider2008], it follows that X is distributed the same as \widetilde{X} \cap U, where U = \text{Im}(T). Finally, we note since each \{u_{n, 1}, \dots, u_{n, d}\} is an orthonormal set of vectors, the matrix
is orthogonal and
where u_{n,i} = (u_{n,i}^{(1)}, \dots, u_{n,i}^{(d)}) are the coordinates of the vector u_{n,i}. We thus have for all 1 \leq j_1, j_2 \leq d that
We thus have that T : \mathbb{R}^d \to \mathbb{R}^{Md} is an orthogonal transformation, and so U is a d-dimensional linear subspace of \mathbb{R}^{Md} such that T preserves the structure of X as a random tessellation as is desired. \square
We will let \widetilde{X} and U = \text{Im}(T) be as defined in Proposition 1 for X a superposition of M independent rotated Poisson Manhattan tessellations X_1, \dots, X_M so that X and \widetilde{X} \cap U are the same in distribution. We have in particular for the typical cell Z of X that
where the typical cell \widetilde{Z} of \widetilde{X} is distributed as
We recall that the particular orthogonal lift T : \mathbb{R}^d \to \mathbb{R}^{Md} of X is represented by the matrix
for orthonormal sets of vectors \{u_{1,i}, \dots, u_{d,i}\} with each u_{n,i} \in \mathbb{R}^d for all 1 \leq i \leq M. Furthermore, we have for all fixed t_1, \dots, t_{Md} \in (0, \infty) that
where we recall since T is orthogonal that u_1, \dots, u_d \in \mathbb{R}^{Md} is orthogonal. We will then define
so that
where each v_{n,1}, \dots, v_{n,d} \in \mathbb{R}^d is orthonormal. It follows for any
that
where (\dagger) holds since each set \{v_{n,1}, \dots, v_{n,d}\} is orthogonal. We have thus computed an upper bound of the form
for any particular set of rotated Poisson Manhattan tessellations X_1, \dots, X_M. Finally, since \widetilde{X} is a Poisson Manhattan tessellation in \mathbb{R}^{Md} of intensity \lambda M^{3/2} > 0, we know that t_k \sim \text{Exp}(2 \lambda \sqrt{M}/d). It follows that each t_k\sqrt{M} \sim \text{Exp}(2\lambda /d) so that
Finally, since for all independent and identically distributed random variables Y_1, \dots, Y_n it holds that
we have that
as is desired, completing our computation of an upper bound for the circumradius of the typical cell of the superposition of independent rotated Mondrian tessellations. \square
[Schneider2008, Equation 4.3] For any nonnegative measurable function f on the space of nonempty compact sets in \mathbb{R}^d and for any stationary random tessellation X, we have that
where Z is the typical cell of the X.
We will consider the definitions and events as are detailed in the outline from Section ?. From A_1 \cap A_2 \cap A_3, we have for every x \in \mathcal{X} that there is a point u \in \mathcal{U} such that x,u are in the same cell of X and
Then, for any x,x' \in \mathcal{X} we have that
where u,u' \in \mathcal{U} are as described above. Since x,u and x',u' are in the same cells of X,
for all x,x' \in \mathcal{X}. We then note from A_4 that \sup_{x,x' \in \mathcal{X}} \left| k_M(u, u') - k_\infty(u, u') \right| \leq \delta/2. We finally note from Theorem 1 that
where (\mathrm{a}) follows from the inequality \left|e^{-|x|} - e^{-|y|}\right| \leq \left|x - y\right|, (\mathrm{b}) follows from the inequality \left|\left|v\right|\right|_{1} \leq \sqrt{d} \left|\left|v\right|\right|_{2}, and (\mathrm{c}) holds from
We thus have that the \delta-approximation on \mathcal{X} is satisfied on the event A_1 \cap A_2 \cap A_3 \cap A_4, and so
from a union bound. We will now bound these four probabilities. First, we saw from the outline above that
Second, we see that for A_2^c,
where we have again used Campbell’s Theorem in the last equality. Since r(C) \geq \varepsilon/2 \sqrt{d} implies that C contains a point of \mathcal{U}, we note that
where in the last inequality we have used the fact that 1 - e^{-x} \leq x for all x\in \mathbb{R}.
Third, we see that for A_3^c,
with R(C + y) = R(C) for all cells C since the circumradius is translation invariant. We thus have from (1) that
Finally, we see that for A_4^c by Hoeffding’s inequality that
For any \varepsilon-grid covering \mathcal{U} of B_{R+\varepsilon}, we have that
since \varepsilon < 2r. We thus have that
for all small enough 0 < \varepsilon < 1. This gives us that
The above expression is minimized at
and so we have that
for constants c_1, c_2, c_3, c_4 not depending on M, \delta and function
which satisfies \alpha(\delta) - \delta/4d^{3/2} < 0 and (\alpha(\delta) - \delta/4d^{3/2}) \to 0 slower than -M\delta^2/(4d+2) \to 0 as \delta \to 0. Thus, letting R - r > 0 be sufficiently small, it holds for small enough \delta > 0 that
completing our proof of the rate of uniform convergence of the uniformly rotated Mondrian kernel to its limiting kernel. \square
The text on this page is from arXiv:2502.04323v2 by Calvin Osborne and Eliza O'Reilly, under the licence CC BY 4.0. The corpus made these changes: it read the items out of the LaTeX source, it expanded the macros of the paper, it put each proof after its statement, and it converted the text to HTML. The formulas are the TeX of the paper, and KaTeX renders them. A formula or a reference can look different from the PDF. The page is published under CC BY 4.0.
The record of each item is in analysis/objects/2502/2502.04323.jsonl.