Equality cases in the Anantharam-Jog-Nair inequality

We’ll use the properties of the Föllmer drift summarized in Appendix ?. Begin by fixing centered \mu_i\in \mathcal{P}(E_i), 1\leq i \leq k, and let (W_t)_{t\geq 0} be a Brownian motion on E_0 with \operatorname{Cov}(W_1) = \operatorname{id}_{E_0}. By Theorem 28 and (?), there is a drift U_t = \int_0^t u_s ds such that \mathbb{E}[u_t]=0 and (\pi_{E_i}(u_t))_{i=1}^k are independent for all 0\leq t\leq 1,

and D(\mu_i\|\gamma_{E_i}) = \frac{1}{2} \int_{0}^1\mathbb{E} | \pi_{E_i}(u_s) |^2 ds for each 1\leq i\leq k. Therefore,

where (?) follows from (?) and the properties of u_t, and (?) follows from (?) and Proposition 26 (with construction (?)) because B_j W_1 \sim \gamma_{E^j}, due to B_j B_j^T = \operatorname{id}_{E^j} by assumption. Now, expanding the relative entropies in terms of Shannon entropies and second moments, the second-moment terms cancel due to independence and (?), giving

for any X_i\sim \mu_i \in \mathcal{P}(E_i) and X\sim \otimes_{i=1}^k \mu_i, where the centering assumption can be removed due to translation invariance of Shannon entropy. The fact that X\sim \gamma_{E_0} is an extremizer follows immediately from the scaling condition (?) (see Remark 6) and the observation that B_j X\sim \gamma_{E^j} (since B_j B_j^T = \operatorname{id}_{E^j}).

(i)\Rightarrow(ii): Let X be an extremizer in (4), and put Z_n := n^{-1/2} \sum_{\ell=1}^n X^{(i)}, where X^{(1)},X^{(2)}, \dots are i.i.d. copies of X, which we assume to be zero-mean without loss of generality. By an application of Proposition 13 and the scaling condition (?) (which holds by finiteness of C_g(\mathbf{c},\mathbf{d},\mathbf{B})), we have that Z_n is an extremizer in (4) for all n\geq 1. By an application of the entropic central limit theorem [barronCLT, CarlenSoffer], it follows that Z\sim N(0,\operatorname{Cov}(X)) is also an extremizer in (4).

(ii)\Rightarrow(i): This follows immediately from Theorem 4.

(ii)\Rightarrow(iii): If (\mathbf{c},\mathbf{d},\mathbf{B}) is Gaussian-extremizable, then there exist K^*_i \in \mathbf{S}^+(E_i), 1\leq i \leq k which maximize

where K:= \operatorname{diag}(K_1, \dots, K_k) (note this implies B_j K^* B_j^T is invertible for each 1\leq j \leq m). This means, for any index i and any A_i \in \mathbf{S}(E_i), we can consider the perturbation K_i = K_i^* + \epsilon A_i for \epsilon sufficiently small, and the function value cannot increase. By first-order Taylor expansion, this implies

where \langle \cdot, \cdot \rangle is the Hilbert–Schmidt (trace) inner product. By arbitrariness of A_i, we conclude (?).

(iii)\Rightarrow(iv): Let K be as in (?). The equivalent datum (\mathbf{c},\mathbf{d},\mathbf{B'}) defined by

is AJN-geometric. Indeed, B_j' B_j'^T = \operatorname{id}_{E^j} and (?) gives

(iv)\Rightarrow(ii): Let (\mathbf{c},\mathbf{d},\mathbf{B'}) be the geometric datum equivalent to (\mathbf{c},\mathbf{d},\mathbf{B}). In the notation of (3), for any X_i \in \mathcal{P}(E_i), 1\leq i \leq k and X = (X_1,\dots, X_k), we have by a change of variables

where we have defined Y_i := C_i X_i, and Y = (Y_1, \dots, Y_k). Since each C_i is invertible, it is clear that X is a (Gaussan-)extremizer for (\mathbf{c},\mathbf{d},\mathbf{B}) if and only if Y is a (Gaussan-)extremizer for (\mathbf{c},\mathbf{d},\mathbf{B'}). The latter is Gaussian-extremizable by the assumption of geometricity and Proposition 7, so the claim follows.

Define Z^+ = (Z_1^+, \dots, Z_k^+) and Z^- = (Z_1^-, \dots, Z_k^-), where

Observe that

In the above, (Proof) is due to independence; (Proof) follows due to orthogonality of the transformation (X_i,Y_i) \to (Z_i^+,Z_i^-) and the chain rule; (Proof) is two applications of (4); (Proof) follows because conditioning reduces entropy; (?) is due to the chain rule and orthogonality of the transformation (B_j Z^+, B_jZ^-) \to (B_j X, B_j Y); (?) is again due to independence.

Since X and Y are extremal by assumption, we have equality throughout. This implies Z^+ is also extremal, and hence we conclude X+Y is extremal by the scaling condition (?).

We first note that \Pi_{E^j} K_{dep} is orthogonal to \Pi_{E^j} K, for any independent subspace K. Indeed, by definition of an independent subspace, we either have \Pi_{E^j} K=\{0\} or \Pi_{E^j} K=K. The former is trivially orthogonal to \Pi_{E^j} K_{dep}, and the latter is orthogonal to \Pi_{E^j} K_{dep} since K_{dep} is orthogonal to K by definition and \Pi_{E^j} is self-adjoint. Indeed,

This establishes (?).

Now, using the decomposition (?) and the scaling condition (?) (which holds by AJN-geometricity), we have

To summarize,

Since each independent subspace is of product form, the dimension condition (?) implies, for each 1\leq i \leq k and 1\leq \ell\leq n_i,

Likewise, since K_{dep} = \oplus_{i=1}^k K_0^i is of product form, (?) also implies

Comparing against (?), we necessarily have equality in (?) and (?), which proves that K_{dep} is critical. Thus, there exists at least one critical decomposition of K_{dep} (the trivial one), and Proposition 15 follows.

It remains to show (?). By induction, it suffices to show if K\subset E_0 is a critical subspace, and K = K_1 \oplus K_2 is a critical decomposition, then \Pi_{E^j} K_1 and \Pi_{E^j} K_2 are orthogonal complements in \Pi_{E^j} K. The proof is similar to that of [bennett2008brascamp, Lemma 7.12]. Letting \Pi_{K_1}: E_0 \to E_0 denote the orthogonal projection onto K_1, we have that \Pi_{E^j}\Pi_{K_1} is a contraction in E_0, so \operatorname{Tr}(\Pi_{E^j}\Pi_{K_1})\leq \dim(\Pi_{E^j}K_1). Since K_1 is critical, it is product-form by definition and therefore \Pi_{K_1} = \sum_{i=1}^k \Pi_{E_i}\Pi_{K_1}\Pi_{E_i}. From (?), this implies

Since K_1 is critical, we have equality throughout, implying \operatorname{Tr}(\Pi_{E^j}\Pi_{K_1})= \dim(\Pi_{E^j}K_1) for each j. From this, we can conclude that \Pi_{K_1}\Pi_{E^j} is an isometry from \Pi_{E^j}K_1 into K_1, and similarly \Pi_{K_2}\Pi_{E^j} is an isometry from \Pi_{E^j}K_2 into K_2. Since K_1 and K_2 are orthogonal complements in K, it follows that \Pi_{E^j} K_1 and \Pi_{E^j} K_2 are orthogonal complements in \Pi_{E^j} K.

Let X_i \sim \mathcal{P}(E_i), 1\leq i \leq k be independent and satisfy (i)-(ii), and let X=(X_1,\dots, X_k). By the orthogonal decomposition (?) and the independence assumptions imposed by (i), we can decompose

where all entropies are computed with respect to the subspace being projected upon. In the proof of Lemma 21, we found (?) was met with equality. So, whenever E_i contains an independent subspace (i.e., n_i\geq 1), we have

Now, using the decomposition (?) and the independence assumptions imposed by (i), an application of the above identity followed by (Proof) reveals

In summary,

where any entropies over the trivial subspace \{0\} are to be neglected.

It remains to show the RHS is zero. By (ii) and translation invariance of entropy, we can assume that \Pi_{K^{\ell}_{dep}}(X)\sim N(0,\sigma_{\ell}^2 \operatorname{id}_{K^{\ell}_{dep}}) for each 1\leq \ell\leq n. Using the independence assumption in (ii) and the decomposition (?), we can express

Since each K_{dep}^{\ell} is critical by definition, we have

where we used the independence assumption in (ii) for the last line. Putting everything together shows

so that (i) and (ii) are sufficient conditions for the X_i’s to be extremal, since C_g(\mathbf{c},\mathbf{d},\mathbf{B})=0 by Proposition 7.

Let \mu_i \in \mathcal{P}(E_i), 1\leq i \leq k satisfy

under the prevailing assumption of AJN-geometricity; this is the same as equality in (?). Without loss of generality, we can assume each \mu_i is centered. Moreover, since the extremizers of the AJN inequality are closed under convolutions (Proposition 13) and standard Gaussians are extremal in the geometric AJN inequality (Proposition 7), we can assume without loss of generality that each \mu_i is of the form

for some extremal \widetilde{\mu}_i \in \mathcal{P}(E_i), 1\leq i \leq k. Indeed, X \sim \otimes_{i=1}^k \mu_i satisfies (i)-(ii) if and only if X+Z satisfies (i)-(ii) for Z\sim \gamma_{E_0}, independent of X.

1ex

Necessity of condition (i): In the proof of Proposition 7, the sole inequality is (?). Hence, properties of the drift u_t warrant a closer inspection; we follow the approach developed in [EldanMikulincer]. Toward this end, let f denote the density of \mu_1\otimes \cdots \otimes \mu_k with respect to \gamma_{E_0}, and define the function

where (P_t)_{t\geq 0} denotes the heat semigroup. Define the matrix-valued function

which, for each 0\leq t \leq 1, takes the block-diagonal form \Gamma_t = \operatorname{diag}(\Gamma^1_t , \dots, \Gamma^k_t) with \Gamma_t^i\in \mathbf{S}^+(E_i) due to the product form of the density f and Lemma 24 applied to (Proof).

Now, consider the Wiener space of continuous functions \mathbb{W} = \{\omega: [0,1] \to E_0;~ \omega(0) = 0\}, equipped with the uniform norm, the Borel sets \mathcal{B}, and the Wiener measure \gamma. Let X_t(\omega) = \omega(t) be the coordinate process, and \mathcal{F} = (\mathcal{F}_t)_{ 0\leq t \leq 1} be the natural filtration of (X_t)_{0\leq t \leq 1}. We’ll work on the filtered probability space (\mathbb{W}, \mathcal{B},\nu, \mathcal{F}), where \nu is the Brownian bridge

By the representation theorem for Brownian bridges, we have

where X \sim \mu_1\otimes \cdots \otimes \mu_k and Z\sim \gamma_{E_0} are independent. Writing u_t \equiv u_t(X_t), the classical de Bruijn identity, parametrized with respect to the bridge (?), gives

where the expectation is with respect to \nu. Moreover, if we define the \mathcal{F}_t-adapted process (W_t)_{0\leq t \leq 1} by the equation

then (W_t)_{0\leq t \leq 1} is a Brownian motion by an application of Girsanov’s theorem [lehec2013representation, EldanLee]. Using the SDE (?) and the heat equation

we can apply Itô’s formula to u_t to reveal the relationship

with \Gamma_t\equiv \Gamma_t(X_t). Rearranging and integrating gives

In particular, equality in (?) together with the computation in (Proof) gives the following representation for the entropies in terms of the \Gamma_t^i processes:

Next, positive-definiteness of \Gamma_t and the assumption that B_j B_j^T = \operatorname{id}_{E^j} together justify the definition of a new process (\widetilde{W}^j_t)_{0\leq t\leq 1} via

By Lévy’s characterization, this process is a Brownian motion, since it has quadratic covariation

Putting things together, observe that definitions and (?) give

Thus, by (?) and an application of Lemmas 22 and 23, we have

We have equality throughout due to (Proof). Since X_t has full support for each 0 < t \leq 1 and (t,x)\mapsto \Gamma_t(x) is smooth by the regularizing properties of the heat semigroup, Lemma 22 and the above equality implies that

By definition, this implies that, for each t\in (0,1), we have

Since f is assumed regular by virtue of (Proof), the above also holds for t=1 by continuity of the derivatives of the heat semigroup. Since f = \prod_{i=1}^k f_i by definition, where each f_i is a density on E_i with respect to \gamma_{E_i}, the above imposes a block-diagonal structure on the Hessian of \log f_i, which can be summarized as

whenever x,y are vectors from distinct spaces in the decomposition (?). This implies, for each 1\leq i \leq k, that the density f_i has product form

establishing necessity of (i).

1ex

Necessity of condition (ii): Having established necessity of (i), the initial calculations in the proof of sufficiency hold, leading to the conclusion (Proof). The reduced datum (\mathbf{c},\mathbf{d},\mathbf{B}_{K_{dep}}) obtained by restricting the maps in \mathbf{B} to domain K_{dep} remains AJN-geometric, so without loss of generality, we can assume for simplicity that there are no independent subspaces henceforth; i.e., K_{dep} \equiv E_0. As in the previous step, we let f denote the density of X\sim \mu_1\otimes \cdots \otimes \mu_k with respect to \gamma_{E_0}.

Letting definitions from the previous step prevail, Lemma 24 implies that u_t has linear growth in x for each 0 < t < 1. Hence, we are justified in taking the Fourier transform, which we denote by \hat{u}_t. By (?), u_t is additively separable in the variables \Pi_{E_j} x and (\operatorname{id}_{E_0}-\Pi_{E_j})x, and therefore \hat{u}_t is supported on H^j \cup (H^j)^{\perp} for each 1\leq j \leq m (where H^j denotes the complex Hilbert space E^j + \mathbf{i} E^j). Similarly, since u_t is additively separable in the variables \pi_{E_1}(x), \dots, \pi_{E_k}(x), it follows that \hat{u}_t is supported on \cup_{i=1}^k H_i (where, H_i := E_i + \mathbf{i} E_i). Taking intersections, we find \hat{u}_t is supported on the set

where the equality follows by the assumption that there are no independent subspaces. A tempered distribution with Fourier transform supported at the origin is a polynomial [Rudin91, p. 194], so the linear growth estimate in Lemma 24 implies that x\mapsto u_t(x) is affine for each 0 < t < 1. As a consequence of its defnition, \Gamma_t is therefore deterministic for each 0 < t < 1, in the sense that \Gamma_t(x) does not depend on x. Using the Itô isometry, we conclude from the representation \int_{0}^1 \Gamma_t d W_t \overset{\text{law}}{=} X that X is Gaussian with covariance

Note that \Sigma has diagonal form

due to independence of the coordinates of X.

From (?), we have \Pi_{E^j} \Sigma = \Pi_{E^j} \Sigma \Pi_{E^j} for each 1\leq j\leq m. This implies that if v\in E_0 is an eigenvector of \Sigma with eigenvalue \lambda, then \Pi_{E^j} v is an eigenvector of \Pi_{E^j} \Sigma \Pi_{E^j} with eigenvalue \lambda. In particular, if we consider the spectral decomposition \Sigma = \sigma_1^2 \Pi_{K^{1}_{dep}} + \cdots \sigma_n^2 \Pi_{K^{n}_{dep}} with \sigma^2_{1}, \dots, \sigma^2_{n} distinct, then we have the orthogonal decomposition

where we note each K^{\ell}_{dep} is product-form due to (?). To see that E_0 = \oplus_{\ell=1}^n K^{\ell}_{dep} is a critical decomposition, observe that

where (?) is the extremality assumption; (?) is due to (?) and the spectral decomposition of \Sigma; (?) is the dimension condition (?); and (?) follows due to the orthogonal decomposition E_i = \oplus_{\ell=1}^n \pi_{E_i} K_{dep}^{\ell} for each 1\leq i\leq k, because each K_{dep}^{\ell} is of product-form. Since we have equality throughout, this implies K_{dep}\equiv E_0 = \oplus_{\ell=1}^n K_{dep}^{\ell} is a critical decomposition, as desired. Since K_{dep}^{1}, \dots, K_{dep}^{n} are eigenspaces of \Sigma, (ii) holds.

Using the block-diagonal structure of A and the definition of AJN-geometricity, we have

where the inequality follows because square root is operator monotone. More precisely, AJN-geometricity implies

so that operator monotonicity of square root gives B_j A B_j^T \leq (B_j A^2 B_j^T)^{1/2}. Equality in (22) is therefore equivalent to equality above, which can be rewritten as

Define the drift

We claim that W_1 + \int_{0}^1 u_t dt \sim \mu. To see this, write

where we used the stochastic Fubini theorem. Now, by Proposition 26 and the data processing inequality, Itô’s isometry, and Fubini’s theorem, we have

Let \rho denote the density of X with respect to Lebesgue measure on E. By direct calculation, we can reparametrize P_{1-t}f in terms of \rho as

Hence,

Regularity estimates for evolution of densities under (P_t)_{t\geq 0} imply

for some finite constant c_s depending only on s and the second moments of \rho (see, e.g., [PolyanksiyWu, Proposition 2]). Hence, the first claim follows.

For the second claim, we have \rho = P_{1}\rho_0 for some density \rho_0. Hence, by the semigroup property combined with (Proof), we have

By a simple convexity calculation [EldanLee, Lemma 1.3], it holds that \nabla^2 (\log P_s g)\geq -\frac{1}{s}\operatorname{id}_E for any density g and s>0, so we find

For each 1\leq i\leq k, let \mathbb{W}_i = C^0([0,1]; E_i), \mathcal{G}_i be its natural filtration, \mathcal{B}_i be the corresponding Borel \sigma-algebra, and \gamma_i the Wiener measure. Define measure \nu_i\ll \gamma_i on (\mathbb{W}_i,\mathcal{B}_i) by

By Proposition 27 and the subsequent discussion, there exists a drift U_t^i= \int_{0}^{t} u^i_s ds and a standard Brownian motion B^i, both carried on (\mathbb{W}_i, \mathcal{B}_i, \nu_i, \mathcal{G}_i), such that \mu_i \sim B^i_1 + U^i_1 and

Now, put everything together on the product space \Omega = \prod_{i=1}^k (\mathbb{W}_i\times \mathbb{W}_i) equipped with its natural filtration, the Borel sets, and the product measure \mathbb{P} = \otimes_{i=1}^k P_{B^i U^i}.