Sparse Signal Processing with Frame Theory

We first note that \Phi being (K,\delta)-RIP trivially implies that \Phi is (K,\delta+\varepsilon)-RIP for every \varepsilon>0. It therefore suffices to show that the expression for \delta_K in (4) satisfies two criteria: (i) \Phi is (K,\delta_K)-RIP, and (ii) \Phi is not (K,\delta)-RIP for any \delta<\delta_K. To this end, pick some K-sparse vector x. To prove (i), we need to show that

Let \mathcal{K}\subseteq\{1,\dots,N\} be the size-K support of x, and let x_\mathcal{K} be the corresponding subvector. Then rearranging (Proof) gives

Since the expression for \delta_K in (4) maximizes (Proof) over all supports \mathcal{K} and entry values x_\mathcal{K}, the inequality necessarily holds; that is, \Phi is necessarily (K,\delta_K)-RIP. Furthermore, equality is achieved by the support \mathcal{K} which maximizes (4) and the eigenvector x_\mathcal{K} corresponding to the largest eigenvalue of \Phi_\mathcal{K}^*\Phi_\mathcal{K}^{}-I_K; this proves (ii).

To verify \Phi is a tight frame, note that the inner product of any two distinct rows of \Phi is zero, as they are the sum of the inner products of the corresponding rows of the \Phi_j’s over all j=1,\dotsc,v; for any j, these shorter inner products are necessarily zero, as they either correspond to inner products of distinct rows of H_j or to inner products with zero vectors. Moreover, the rows of \Phi have constant norm: as noted in (ii) above, each row of A^\mathrm{T} contains k ones; since each H_j has unimodular entries, the squared-norm of any row of \Phi is the squared-scaling factor \frac{k-1}{v-1} times a sum of \smash{k(1+\frac{v-1}{k-1})} ones, which, as is necessary for any unit norm tight frame, equals the redundancy \smash{\frac NM=k(1+\frac{k-1}{v-1})}.

Having that \Phi is tight, we show \Phi is also equiangular. We first note that the columns of \Phi have unit norm: the squared-norm of any column of \Phi is \smash{\frac{k-1}{v-1}} times the squared-norm of a column of one of the \Phi_j’s; since the entries of H_j are unimodular and (iii) above gives that each column of A^\mathrm{T} contains \smash{\frac{v-1}{k-1}} ones, the squared-norm of any column of \Phi is \smash{(\frac{k-1}{v-1})(\frac{v-1}{k-1})1=1}, as claimed. Moreover, the inner products of any two distinct columns of \Phi has constant modulus. Indeed, the fact (iv) that any two distinct columns of A^\mathrm{T} have but a single entry of mutual support implies the same is true for columns of \Phi that arise from distinct \Phi_j blocks, implying the inner product of such columns is \smash{\frac{k-1}{v-1}} times the product of two unimodular numbers. That is, the squared-magnitude of the inner products of two columns that arise from distinct blocks is \smash{\frac{N-M}{M(N-1)}=(\frac{k-1}{v-1})^2}, as needed. Meanwhile, the same holds true for columns that arise from the same block \Phi_j. To see this, note that since H_j is a scalar multiple of a unitary matrix, its columns are orthogonal. Moreover, \Phi_j contains all but one of the H_j’s rows, namely one for each of the 1-valued entries of A^\mathrm{T}, à la (iii). Thus, the inner products of the portions of H_j that lie in \Phi_j are their entire inner product of zero, less the contribution from the left-over entries. Overall, the inner product of two columns of \Phi that arise from the same \Phi_j block is \smash{\frac{k-1}{v-1}} times the negated product of one entry of H_j and the conjugate of another; since the entries of H_j are unimodular, we have that the squared-magnitude of such inner products is \smash{\frac{N-M}{M(N-1)}=(\frac{k-1}{v-1})^2}, as needed.

Thus \Phi is an ETF. Moreover, as noted above, its redundancy is \smash{\frac NM=k(1+\frac{k-1}{v-1})}. All that remains to verify is its density: as the entries of each H_j are all nonzero, the proportion of \Phi’s nonzero entries is the same as that of the incidence matrix A, which is clearly \frac kv, having k ones in each v-dimensional row. Moreover, substituting \smash{N=v(1+\frac{v-1}{k-1})} and \smash{M=\frac{v(v-1)}{k(k-1)}} into the quantity \smash{\frac{N-1}{M(N-M)}} reveals it to be \frac{k^2}{v^2}, and so the density can be alternatively expressed as \smash{(\frac{N-1}{M(N-M)})^{\frac12}}.

To prove the necessary conditions on M and N, recall that Steiner ETFs, namely those ETFs produced by Theorem 7, have \smash{N=v(1+\frac{v-1}{k-1})} and \smash{M=\frac{v(v-1)}{k(k-1)}}. Together, these two equations imply N=v+kM. Solving for k and substituting the resulting expression into \smash{N=v(1+\frac{v-1}{k-1})} yields the quadratic equation 0=(M-1)v^2+2(N-M)v-N(N-M). With some algebra, the only positive root of this equation can be found to be \smash{v=\frac{N\alpha}{1+\alpha}}, as claimed. Substituting this expression for v into N=v+kM yields \smash{k=\tfrac{N}{M(1+\alpha)}}. Having v and k, the previously mentioned relations bk=vr and v-1=r(k-1) imply \smash{r=\frac{v-1}{k-1}=\frac1\alpha} and \smash{b=\frac vkr=M}, as claimed.

The second set of conclusions is the result of applying Theorem 7 to the aforementioned (2,k,v)-Steiner ETFs that are guaranteed to exist for all sufficiently large v, provided \smash{\frac{v-1}{k-1}} and \smash{\frac{v(v-1)}{k(k-1)}} are integers. The final set of conclusions are then obtained by applying this fact in the special cases where v is either jk(k-1)+1 or jk(k-1)+k. In particular, if v=jk(k-1)+1 then \smash{\frac{v-1}{k-1}=jk} and M=\smash{\frac{v(v-1)}{k(k-1)}=j\big(jk(k-1)+1\big)} are integers, and the resulting ETF of (jk+1)\big(jk(k-1)+1\big) vectors has a redundancy of \smash{k+\frac1j} that tends to k for large j; such an ETF can be real if there exists a real Hadamard matrix of size jk+1. Meanwhile, if v=jk(k-1)+k then \smash{\frac{v-1}{k-1}=jk+1} and M=\smash{\frac{v(v-1)}{k(k-1)}=(jk+1)\big(j(k-1)+1\big)} are integers, and the resulting ETF of k(jk+2)\big(j(k-1)+1\big) vectors has a redundancy of \smash{k\frac{jk+2}{jk+1}} that tends to k for large j; such an ETF can be real if there exists a real Hadamard matrix of size jk+2.

Take \mathcal{K},\mathcal{K}'\subseteq\{1,\ldots,N\} such that |\mathcal{K}|,|\mathcal{K}'|\leq K and n\in\mathcal{K}\setminus\mathcal{K}'. Then the left-hand inequality of the restricted isometry property gives

For a fixed |\mathcal{K}|, we will find a lower bound for

Since we can have |\mathcal{K}\cap\mathcal{K}'|>\frac{|\mathcal{K}|}{2}, we know \frac{|\mathcal{K}|-2|\mathcal{K}\cap\mathcal{K}'|}{|\mathcal{K}'|}<0 when (Proof) is minimized. That said, |\mathcal{K}'| must be as small as possible, i.e., |\mathcal{K}'|=|\mathcal{K}\cap\mathcal{K}'|. Thus, when (Proof) is minimized, we have

i.e., |\mathcal{K}\cap\mathcal{K}'| must be as large as possible. Since n\in\mathcal{K}\setminus\mathcal{K}', we have |\mathcal{K}\cap\mathcal{K}'|\leq|\mathcal{K}|-1. Therefore,

Substituting (Proof) into (Proof) gives

Since this bound holds for every n, \mathcal{K} and \mathcal{K}' with n\in\mathcal{K}\setminus\mathcal{K}', we have (9).

To bound \mathrm{P}_\mathrm{I}(\Phi,\tau,\{x_k\}_{k=1}^K), assume a given user n is innocent, i.e., H_0(n). Then the test statistic for our detector (?) is given by

By the symmetry of z’s Gaussian distribution, we know the projection \langle z,\frac{\varphi_n}{\|\varphi_n\|}\rangle also has Gaussian distribution with mean zero and variance \sigma^2, meaning our test statistic T_n(y) has Gaussian distribution with mean \sum_{k\in\mathcal{K}}x_k\langle \frac{\varphi_k}{\|\varphi_k\|},\frac{\varphi_n}{\|\varphi_n\|}\rangle and variance \frac{\sigma^2}{\gamma^2}. Furthermore, since the normalized fingerprints form an ETF with worst-case coherence \mu, we can use the triangle inequality to bound the mean of T_n(y):

We use this to bound the false-positive probability for user n:

Since this bound holds for all coalitions, weight assignments and innocent users, this bound must also hold for \mathrm{P}_\mathrm{I}(\Phi,\tau,\{x_k\}_{k=1}^K).

Next, to bound \mathrm{P}_\mathrm{II}(\Phi,\tau,\{x_k\}_{k=1}^K), assume a given user n is guilty, i.e., H_1(n). In this case, the test statistic for our detector (?) is given by

As before, T_n(y) has Gaussian distribution with variance \frac{\sigma^2}{\gamma^2}, but this time, the mean is

As such, the false-negative probability for user n is

Applying the definition of \mathrm{P}_\mathrm{II}(\Phi,\tau,\{x_k\}_{k=1}^K) therefore gives

Suppose a Vandermonde matrix is equal norm and tight. Note that a zero base will produce the zeroth identity basis element \delta_0. Letting \mathcal{P} denote the indices of the nonzero bases, the fact that the matrix is full rank implies |\mathcal{P}|\geq M-1. Also, equal norm gives that the frame element length

is constant over n\in\mathcal{P}. Since \sum_{m=0}^{M-1}x^{2m} is strictly increasing over 0<x<\infty, there exists c>0 such that |\alpha_n|^2=c for all n\in\mathcal{P}. Next, tightness gives that the rows have equal norm, implying that the first two rows have equal norm, i.e., |\mathcal{P}|c=|\mathcal{P}|c^2. Thus c=1, and so the nonzero bases are in the complex unit circle. Furthermore, since the zeroth and first rows have equal norm by tightness, we have |\mathcal{P}|=N, and so every base is in the complex unit circle.

Now consider the inner product between Vandermonde frame elements whose bases \{e^{2\pi i x_n}\}_{n=1}^N come from the complex unit circle:

We will show that the worst-case coherence comes from the two closest bases. Consider the following function:

Figure ? gives a plot of this function in the case where M=5. We will prove two things about this function:

• (i) \tfrac{d}{dx}g(x)<0 for every x\in(0,\tfrac{1}{2M}),

• (ii) g(x)\leq g(\tfrac{1}{2M}) for every x\in(\tfrac{1}{2M},1-\tfrac{1}{2M}).

First, we claim that (i) and (ii) are sufficient to prove our result. To establish this, we first show that the two closest bases e^{2\pi ix_{n'}} and e^{2\pi ix_{n''}} satisfy |x_{n'}-x_{n''}|\leq\frac{1}{2M}. Without loss of generality, the n’s are ordered in such a way that \{x_n\}_{n=0}^{N-1}\subseteq[0,1) are nondecreasing. Define

and let n' be the n which minimizes d(x_n,x_{n+1}). Since the minimum is less than the average, we have

provided N\geq2M. Note that if we view \{x_n\}_{n\in\mathbb{Z}_N} as members of \mathbb{R}/\mathbb{Z}, then d(x_n,x_{n+1})=x_{n+1}-x_n. Since g(x) is even, then (i) implies that |\langle \varphi_{n'+1},\varphi_{n'}\rangle|^2=g(x_{n'+1}-x_{n'}) is larger than any other g(x_p-x_{p'})=|\langle \varphi_p,\varphi_{p'}\rangle|^2 in which x_p-x_{p'}\in[0,\tfrac{1}{2M}]\cup[1-\tfrac{1}{2M},1). Next, (?) and (ii) together imply that |\langle \varphi_{n'+1},\varphi_{n'}\rangle|^2=g(x_{n'+1}-x_{n'})\geq g(\tfrac{1}{2M}) is larger than any other g(x_p-x_{p'})=|\langle \varphi_p,\varphi_{p'}\rangle|^2 in which x_p-x_{p'}\in(\tfrac{1}{2M},1-\tfrac{1}{2M}), provided N\geq2M. Combined, (i) and (ii) give that |\langle \varphi_{n'+1},\varphi_{n'}\rangle| achieves the worst-case coherence of \{\varphi_n\}_{n\in\mathbb{Z}_N}. Additionally, (i) gives that the worst-case coherence |\langle \varphi_{n'+1},\varphi_{n'}\rangle| is minimized when x_{n'+1}-x_{n'} is maximized, i.e., when the x_n’s are equally spaced in the unit interval.

To prove (i), note that the geometric sum formula gives

where the final expression uses the identity 1-\cos(2z)=2\sin^2 z. To show that g is decreasing over (0,\frac{1}{2M}), note that the base of (?) is positive on this interval, and performing the quotient rule to calculate its derivative will produce a fraction whose denominator is nonnegative and whose numerator is given by

This factor is zero at x=0 and has derivative:

which is strictly negative for all x\in(0,\tfrac{1}{2M}). Hence, (?) is strictly negative whenever x\in(0,\tfrac{1}{2M}), and so g'(x)<0 for every x\in(0,\tfrac{1}{2M}).

For (ii), note that for every x\in(\frac{1}{2M},1-\frac{1}{2M}), we can individually bound the numerator and denominator of what the geometric sum formula gives:

For (i), we first define D to be the N\times N diagonal matrix whose diagonal entries are \{\omega^n\}_{n=0}^{N-1}. Note that, since \omega^{(m+1)n}=\omega^n\omega^{mn}, translating the row indices \mathcal{M} by 1 corresponds to multiplying \Phi on the right by D. For some set \mathcal{K}\subseteq\mathbb{Z}_N of size M:=|\mathcal{M}|, let \Phi_\mathcal{K} denote the M\times M submatrix of \Phi whose columns are indexed by \mathcal{K}, and let D_\mathcal{K} denote the M\times M diagonal submatrix of D whose diagonal entries are indexed by \mathcal{K}. Then since D_\mathcal{K} is unitary, we have

Thus, if \Phi is full spark, |\mathrm{det}((\Phi D)_\mathcal{K})|=|\mathrm{det}(\Phi_\mathcal{K})|>0, and so \Phi D is also full spark. Using this fact inductively proves (i) for all translations of \mathcal{M}.

For (ii), let \Psi denote the submatrix of rows indexed by A\mathcal{M}. Then for any \mathcal{K}\subseteq\mathbb{Z}_N of size M,

Since A is relatively prime to N, multiplication by A permutes the elements of \mathbb{Z}_N, and so A\mathcal{K} has exactly M distinct elements. Thus, if \Phi is full spark, then \mathrm{det}(\Psi_\mathcal{K})=\mathrm{det}(\Phi_{A\mathcal{K}})\neq0, and so \Psi is also full spark.

For (iii), we let \Psi be the (N-M)\times N submatrix of rows indexed by \mathcal{M}^\mathrm{c}, so that

We will use contraposition to show that \Phi being full spark implies that \Psi is also full spark. To this end, suppose \Psi is not full spark. Then \Psi has a collection of N-M linearly dependent columns \{\psi_i\}_{i\in \mathcal{K}}, and so there exists a nontrivial sequence \{\alpha_i\}_{i\in \mathcal{K}} such that

Considering \psi_i=\Psi\delta_i, where \delta_i is the ith identity basis element, we can use (?) to express this linear dependence in terms of \Phi:

Rearranging then gives

Here, we note that x is nonzero since \{\alpha_i\}_{i\in \mathcal{K}} is nontrivial, and that x\in\mathrm{Range}(\Phi^*\Phi). Furthermore, whenever j\not\in \mathcal{K}, we have from (?) that

and so x\perp\mathrm{Span}\{\Phi^*\Phi\delta_j\}_{j\in \mathcal{K}^\mathrm{c}}. Thus, the containment \mathrm{Span}\{\Phi^*\Phi\delta_j\}_{j\in \mathcal{K}^\mathrm{c}}\subseteq\mathrm{Range}(\Phi^*\Phi) is proper, and so

Since the M\times M submatrix \Phi_{\mathcal{K}^\mathrm{c}} is rank-deficient, it is not invertible, and therefore \Phi is not full spark.

Let \Phi denote the resulting M\times(N+K) frame. We start by verifying that \Phi is unit norm. Certainly, the identity basis elements have unit norm. For the remaining frame elements, the modulus of each entry is determined by D, and so the norm squared of each frame element is

To demonstrate that \Phi is tight, it suffices to show that \Phi\Phi^*=\frac{N+K}{M}I_M. The rows of DH are orthogonal since they are scaled rows of the DFT, while the rows of the identity portion are orthogonal because they have disjoint support. Thus, \Phi\Phi^* is diagonal. Moreover, the norm squared of each of the first K rows is N(\frac{N+K-M}{MN})+1=\frac{N+K}{M}, while the norm squared of each of the remaining rows is N(\frac{N+K}{MN})=\frac{N+K}{M}, and so \Phi\Phi^*=\frac{N+K}{M}I_M.

To show that \Phi is full spark, note that every M\times M submatrix of DH is invertible since

by Chebotarëv’s theorem. Also, in the case where K=M, we note that the M\times M submatrix of \Phi composed solely of identity basis elements is trivially invertible. The only remaining case to check is when identity basis elements and columns of DH appear in the same M\times M submatrix \Phi_\mathcal{K}. In this case, we may shuffle the rows of \Phi_\mathcal{K} to have the form

Since shuffling rows has no impact on the size of the determinant, we may further use a determinant identity on block matrices to get

Since A is a multiple of a square submatrix of the N\times N DFT, we are done by Chebotarëv’s theorem.

(\Leftarrow) We will use Lemma 21 to demonstrate that \Phi is full spark. To apply this lemma, we need to establish that (21) is not a multiple of p, and to do this, we will show that there are as many p-divisors in the numerator of (21) as there are in the denominator. We start by counting the p-divisors of the denominator:

For each pair of integers k,a\geq 1, there are \max\{M-ap^k,~0\} factors in (Proof) of the form \ell=ap^k. By adding these, we count each factor \ell as many times as it can be expressed as a multiple of a power of p, which equals the number of p-divisors in \ell. Thus, the number of p-divisors of (Proof) is

Next, we count the p-divisors of the numerator of (21). To do this, we use the fact that \mathcal{M} is uniformly distributed over the divisors of N. Since N is a power of p, the only divisors of N are smaller powers of p. Also, the cosets of \langle p^k\rangle partition \mathcal{M} into subsets S_{k,b}:=\{m_i\equiv b\mod p^k\}. We note that m_j-m_i is a multiple of p^k precisely when m_i and m_j belong to the same subset S_{k,b} for some 0\leq b<p^k. To count p-divisors, we again count each factor m_j-m_i as many times as it can be expressed as a multiple of a prime power:

Write M=qp^k+r with 0\leq r<p^k. Then q=\lfloor\frac{M}{p^k}\rfloor. Since \mathcal{M} is uniformly distributed over p^k, there are r subsets S_{k,b} with q+1 elements and p^k-r subsets with q elements. We use this to get

Rearranging and substituting M=qp^k+r then gives

Thus, there are as many p-divisors in the numerator (?) as there are in the denominator (Proof), and so (21) is not divisible by p. Lemma 21 therefore gives that \Phi is full spark.

(\Rightarrow) We will prove that this direction holds regardless of whether N is a prime power. Suppose \mathcal{M}\subseteq\mathbb{Z}_N is not uniformly distributed over the divisors of N. Then there exists a divisor d of N such that one of the cosets of \langle d\rangle intersects \mathcal{M} with \leq\lfloor \frac{M}{d}\rfloor-1 or \geq\lceil\frac{M}{d}\rceil+1 indices. Notice that if a coset of \langle d\rangle intersects \mathcal{M} with \leq\lfloor \frac{M}{d}\rfloor-1 indices, then the complement \mathcal{M}^\mathrm{c} intersects the same coset with \geq\lceil\frac{N-M}{d}\rceil+1=\lceil\frac{|\mathcal{M}^\mathrm{c}|}{d}\rceil+1 indices. By Theorem 14(iii), \mathcal{M} produces a full spark harmonic frame precisely when \mathcal{M}^\mathrm{c} produces a full spark harmonic frame, and so we may assume without loss of generality that there exists a coset of \langle d\rangle which intersects \mathcal{M} with \geq\lceil\frac{M}{d}\rceil+1 indices.

To prove that the rows with indices in \mathcal{M} are not full spark, we find column entries which produce a singular submatrix. Writing M=qd+r with 0\leq r<d, let \mathcal{K} contain q=\lfloor\frac{M}{d}\rfloor cosets of \langle\frac{N}{d}\rangle along with r elements from an additional coset. We claim that the DFT submatrix with row entries \mathcal{M} and column entries \mathcal{K} is singular. To see this, shuffle the rows and columns to form a matrix A in which the row entries are grouped into common cosets of \langle d\rangle and the column entries are grouped into common cosets of \langle\frac{N}{d}\rangle. This breaks A into rank-1 submatrices: each pair of cosets a+\langle d\rangle and b+\langle\frac{N}{d}\rangle produces a submatrix

for some index sets \mathcal{I} and \mathcal{J}; this is a rank-1 outer product. Let \mathcal{L} be the largest intersection between \mathcal{M} and a coset of \langle d\rangle. Then |\mathcal{L}|\geq\lceil\frac{M}{d}\rceil+1 is the number of rows in the tallest of these rank-1 submatrices. Define A_\mathcal{L} to be the M\times M matrix with entries A_\mathcal{L}[i,j]=A[i,j] whenever i\in\mathcal{L} and zero otherwise. Then

Since A-A_\mathcal{L} has |\mathcal{L}| rows of zero entries, we also have

Moreover, since we can decompose A_\mathcal{L} into a sum of \lceil\frac{M}{d}\rceil zero-padded rank-1 submatrices, we have \mathrm{Rank}(A_\mathcal{L}) \leq \lceil\frac{M}{d}\rceil. Combining this with (?) and (?) then gives that \mathrm{Rank}(A)\leq M-1, and so the DFT submatrix is not invertible.

Denoting \omega:=e^{-2\pi i/N}, then for every m=1,\ldots,M, we have \omega_m=\omega^{k_m} for some 0\leq k_m<N. Defining the polynomial Q(z):=P(z^{k_1},\ldots,z^{k_M}), we have Q(\omega)=0 by assumption. Also, Q(z) is a polynomial with integer coefficients, and so it must be divisible by the minimal polynomial of \omega, namely, the cyclotomic polynomial \Phi_N(z). Evaluating both polynomials at z=1 then gives that p=\Phi_N(1) divides Q(1)=P(1,\ldots,1).

We wish to show that \mathrm{det}(\omega_n^m)_{m\in\mathcal{M},1\leq n\leq M}\neq0 for all M-tuples of distinct Nth roots of unity \{\omega_n\}_{n=1}^M. Define the polynomial D(z_1,\ldots,z_M):=\mathrm{det}(z_n^m)_{m\in\mathcal{M},1\leq n\leq M}. Since columns i and j of (z_n^m)_{m\in\mathcal{M},1\leq n\leq M} are identical whenever z_i=z_j, we know that D vanishes in each of these instances, and so we can factor:

for some polynomial P(z_1,\ldots,z_M) with integer coefficients. By Lemma 20, it suffices to show that P(1,\ldots,1) is not a multiple of p, since this implies D(\omega_1,\ldots,\omega_M) is nonzero for all M-tuples of distinct Nth roots of unity \{\omega_n\}_{n=1}^M.

To this end, we proceed by considering

To compute A, we note that each application of z_j\frac{\partial}{\partial z_j} produces terms according to the product rule. For some terms, a linear factor of the form z_j-z_i or z_i-z_j is replaced by z_j or -z_j, respectively. For each the other terms, these linear factors are untouched, while another factor, such as P(z_1,\ldots,z_M), is differentiated and multiplied by z_j. Note that there are a total of M(M-1)/2 linear factors, and only M(M-1)/2 differentiation operators to apply. Thus, after expanding every product rule, there will be two types of terms: terms in which every differentiation operator was applied to a linear factor, and terms which have at least one linear factor remaining untouched. When we evaluate at z_1=\cdots=z_M=1, the terms with linear factors vanish, and so the only terms which remain came from applying every differentiation operator to a linear factor. Furthermore, each of these terms before the evaluation is of the form P(z_1,\ldots,z_M)\prod_{1\leq i<j\leq M}z_j, and so evaluation at z_1=\cdots=z_M=1 produces a sum of terms of the form P(1,\ldots,1); to determine the value of A, it remains to count these terms. The M-1 copies of z_M\frac{\partial}{\partial z_M} can only be applied to linear factors of the form z_M-z_i, of which there are M-1, and so there are a total of (M-1)! ways to distribute these operators. Similarly, there are (M-2)! ways to distribute the M-2 copies of z_{M-1}\frac{\partial}{\partial z_{M-1}} amongst the M-2 linear factors of the form z_{M-1}-z_i. Continuing in this manner produces an expression for A:

For an alternate expression of A, we substitute the definition of D(z_1,\ldots,z_M) into \eqref{eq.A defn}. Here, we exploit the multilinearity of the determinant and the fact that (z_n\frac{\partial}{\partial z_n})z_n^m=mz_n^m to get

where the final equality uses the fact that (m^{n-1})_{m\in\mathcal{M},1\leq n\leq M} is the transpose of a Vandermonde matrix. Equating (?) to (?) reveals that (21) is an expression for P(1,\ldots,1). Thus, by assumption, P(1,\ldots,1) is not a multiple of p, and so we are done.

We will give a randomized polynomial-time reduction from Problem 23 to Full Spark. As such, suppose we are given a bipartite graph G, in which every edge is between the disjoint sets A and B. Take M:=|B| and N:=|A|. Using this graph, we randomly draw an M\times N matrix \Phi using the following process: for each i\in B and j\in A, pick the entry \Phi_{ij} randomly from \{1,\ldots,N2^{N+1}\} if i\leftrightarrow j in G; otherwise set \Phi_{ij}=0. In Proposition 3.11 of [Marx:tcs09], it is shown that the columns of \Phi form a representation of the transversal matroid of G with probability \geq\frac{1}{2}. For the moment, we assume that \Phi succeeds in representing the matroid.

Since the girth of the original matroid equals the spark of its representation, for each K=1,\ldots,M, we test whether \mathrm{Spark}(\Phi)>K. To do this, take H to be some M\times P full spark frame. We will determine an appropriate value for P later, but for simplicity, we can take H to be the Vandermonde matrix formed from bases \{1,\ldots,P\}; see Lemma 12. We claim we can randomly select K indices \mathcal{K}\subseteq\{1,\ldots,P\} and test whether H_\mathcal{K}^*\Phi is full spark to determine whether \mathrm{Spark}(\Phi)>K. Moreover, after performing this test for each K=1,\ldots,M, the probability of incorrectly determining \mathrm{Spark}(\Phi) is \leq\frac{1}{2}, provided P is sufficiently large.

We want to test whether H_\mathcal{K}^*\Phi is full spark and use the result as a proxy for whether \mathrm{Spark}(\Phi)>K. For this to work, we need to have \mathrm{Rank}(H_\mathcal{K}^*\Phi_{\mathcal{K}'})=K precisely when \mathrm{Rank}(\Phi_{\mathcal{K}'})=K for every \mathcal{K}'\subseteq\{1,\ldots,N\} of size K. To this end, it suffices to have the nullspace \mathcal{N}(H_\mathcal{K}^*) of H_\mathcal{K}^* intersect trivially with the column space of \Phi_{\mathcal{K}'} for every \mathcal{K}'. To be clear, it is always the case that \mathrm{Rank}(H_\mathcal{K}^*\Phi_{\mathcal{K}'})\leq\mathrm{Rank}(\Phi_{\mathcal{K}'}), and so \mathrm{Rank}(\Phi_{\mathcal{K}'})<K implies \mathrm{Rank}(H_\mathcal{K}^*\Phi_{\mathcal{K}'})<K. If we further assume that \mathcal{N}(H_\mathcal{K}^*)\cap\mathrm{Span}(\Phi_{\mathcal{K}'})=\{0\}, then the converse also holds. To see this, suppose \mathrm{Rank}(H_\mathcal{K}^*\Phi_{\mathcal{K}'})<K. Then by the rank-nullity theorem, there is a nontrivial x\in\mathcal{N}(H_\mathcal{K}^*\Phi_{\mathcal{K}'}). Since H_\mathcal{K}^*\Phi_{\mathcal{K}'}x=0, we must have \Phi_{\mathcal{K}'}x\in\mathcal{N}(H_\mathcal{K}^*), which in turn implies x\in\mathcal{N}(\Phi_{\mathcal{K}'}) since \mathcal{N}(H_\mathcal{K}^*)\cap\mathrm{Span}(\Phi_{\mathcal{K}'})=\{0\} by assumption. Thus, \mathrm{Rank}(\Phi_{\mathcal{K}'})<K by the rank-nullity theorem.

Now fix \mathcal{K}'\subseteq\{1,\ldots,N\} of size K such that \mathrm{Rank}(\Phi_{\mathcal{K}'})=K. We will show that the vast majority of choices \mathcal{K}\subseteq\{1,\ldots,P\} of size K satisfy \mathcal{N}(H_\mathcal{K}^*)\cap\mathrm{Span}(\Phi_{\mathcal{K}'})=\{0\}. To do this, we consider the columns \{h_k\}_{k\in\mathcal{K}} of H_\mathcal{K} one at a time, and we make use of the fact that \mathcal{N}(H_\mathcal{K}^*)=\bigcap_{k\in\mathcal{K}}\mathcal{N}(h_k^*). In particular, since H is full spark, there are at most M-K columns of H in the orthogonal complement of \mathrm{Span}(\Phi_{\mathcal{K}'}), and so there are at least P-(M-K) choices of h_{k_1} for which \mathcal{N}(h_{k_1}^*) does not contain \mathrm{Span}(\Phi_{\mathcal{K}'}), i.e.,

Similarly, after selecting the first J h_k’s, we have \mathrm{dim}(S)=K-J, where

Again, since H is full spark, there are at most M-(K-J) columns of H in the orthogonal complement of S, and so the remaining P-(M-(K-J)) columns are candidates for h_{k_{J+1}} that give

Overall, if we randomly pick \mathcal{K}\subseteq\{1,\ldots,P\} of size K, then

where the final step is by Bernoulli’s inequality. Taking a union bound over all choices of \mathcal{K}'\subseteq\{1,\ldots,N\} and all values of K=1,\ldots,M then gives

Thus, to make the probability of failure \leq\frac{1}{2}, it suffices to have P=M^32^{N+1}.

In summary, we succeed in representing the original matroid with probability \geq\frac{1}{2}, and then we succeed in determining the spark of its representation with probability \geq\frac{1}{2}. The probability of overall success is therefore \geq\frac{1}{4}. Since our success probability is, at worst, polynomially small, we can apply amplification to achieve an exponentially small error probability.

Given a matrix \Phi=[\varphi_1\cdots\varphi_N], we want to upper-bound the smallest \delta for which (1-\delta)\|x\|^2\leq\|\Phi x\|^2\leq(1+\delta)\|x\|^2, or equivalently:

for every nonzero 2K-sparse vector x. We observe from (Proof) that we may take x to have unit norm without loss of generality. Letting \mathcal{K} denote a size-2K set that contains the support of x, and letting \{x_k\}_{k\in\mathcal{K}} denote the corresponding entries of x, the triangle inequality gives

Since \sum_{i\in\mathcal{K}}|x_i|^2=1, the second term of (Proof) satisfies

and so it remains to bound the first term of (Proof). To this end, we note that for each i,j\in\mathcal{K} with j\neq i, the term \langle x_i\varphi_i,x_j\varphi_j\rangle appears in

as many times as there are size-K subsets of \mathcal{K} which contain i but not j, i.e., \binom{2K-2}{K-1} times. Thus, we use the triangle inequality and the definition of restricted orthogonality to get

At this point, x having unit norm implies (\sum_{i\in\mathcal{I}}|x_i|^2)^{1/2}(\sum_{j\in\mathcal{K}\setminus\mathcal{I}}|x_j|^2)^{1/2}\leq\frac{1}{2}, and so

Applying both this and (Proof) to (Proof) gives the result.

Given arbitrary disjoint subsets \mathcal{I},\mathcal{J}\subseteq\{1,\ldots,N\} with |\mathcal{I}|,|\mathcal{J}|\leq K, we will bound the following quantity three times, each time with different constraints on \{x_i\}_{i\in\mathcal{I}} and \{y_j\}_{j\in\mathcal{J}}:

To be clear, our third bound will have no constraints on \{x_i\}_{i\in\mathcal{I}} and \{y_j\}_{j\in\mathcal{J}}, thereby demonstrating restricted orthogonality. Note that by assumption, (Proof) is \leq\hat\theta(|\mathcal{I}||\mathcal{J}|)^{1/2} whenever the x_i’s and y_j’s are in \{0,1\}. We first show that this bound is preserved when we relax the x_i’s and y_j’s to lie in the interval [0,1].

Pick a disjoint pair of subsets \mathcal{I}',\mathcal{J}'\subseteq\{1,\ldots,N\} with |\mathcal{I}'|,|\mathcal{J}'|\leq K. Starting with some k\in\mathcal{I}', note that flat restricted orthogonality gives that

for every disjoint \mathcal{I},\mathcal{J}\subseteq\{1,\ldots,N\} with |\mathcal{I}|,|\mathcal{J}|\leq K and k\in\mathcal{I}. Thus, we may take any x_k\in[0,1] to form a convex combination of these two expressions, and then the triangle inequality gives

Since (?) holds for every disjoint \mathcal{I},\mathcal{J}\subseteq\{1,\ldots,N\} with |\mathcal{I}|,|\mathcal{J}|\leq K and k\in\mathcal{I}, we can do the same thing with an additional index i\in\mathcal{I}' or j\in\mathcal{J}', and replace the corresponding unit coefficient with some x_i or y_j in [0,1]. Continuing in this way proves the claim that (Proof) is \leq\hat\theta(|\mathcal{I}||\mathcal{J}|)^{1/2} whenever the x_i’s and y_j’s lie in the interval [0,1].

For the second bound, we assume the x_i’s and y_j’s are nonnegative with unit norm: \sum_{i\in\mathcal{I}}x_i^2=\sum_{j\in\mathcal{J}}y_j^2=1. To bound (Proof) in this case, we partition \mathcal{I} and \mathcal{J} according to the size of the corresponding coefficients:

Note the unit-norm constraints ensure that \mathcal{I}=\bigcup_{k=0}^\infty\mathcal{I}_k and \mathcal{J}=\bigcup_{k=0}^\infty\mathcal{J}_k. The triangle inequality thus gives

By the definitions of \mathcal{I}_{k_1} and \mathcal{J}_{k_2}, the coefficients of \varphi_i and \varphi_j in (?) all lie in [0,1]. As such, we continue by applying our first bound:

We now observe from the definition of \mathcal{I}_k that

Thus for any positive integer t, the Cauchy-Schwarz inequality gives

and similarly for the \mathcal{J}_k’s. For a fixed K, we note that (?) is minimized when K^{1/2}2^{-t}=\frac{t^{-1/2}}{2\log 2}, and so we pick t to be the smallest positive integer such that K^{1/2}2^{-t}\leq\frac{t^{-1/2}}{2\log 2}. With this, we continue (?):

From here, we claim that t\leq\lceil\frac{\log K}{\log 2}\rceil. Considering the definition of t, this is easily verified for K=2,3,\ldots,7 by showing K^{1/2}2^{-s}\leq\frac{s^{-1/2}}{2\log 2} for s=\lceil\frac{\log K}{\log 2}\rceil. For K\geq8, one can use calculus to verify the second inequality of the following:

meaning t\leq\lceil\frac{\log K}{\log 2}\rceil. Substituting t\leq\frac{\log K}{\log 2}+1 and t\geq1 into (?) then gives

with C_0\approx 5.77, C_1\approx 11.85. As such, (Proof) is \leq C'\hat\theta\log K with C'=C_0+\frac{C_1}{\log 2} in this case.

We are now ready for the final bound on (Proof) in which we apply no constraints on the x_i’s and y_j’s. To do this, we consider the positive and negative real and imaginary parts of these coefficients:

and similarly for the y_j’s. With this decomposition, we apply the triangle inequality to get

Finally, we normalize the coefficients by (\sum_{i\in\mathcal{I}}x_{i,k_1}^2)^{1/2} and (\sum_{j\in\mathcal{J}}y_{j,k_2}^2)^{1/2} so we can apply our second bound:

where C=4C'\approx74.17 by the Cauchy-Schwarz inequality, and so we are done.

Considering Lemma 28, it suffices to show that \Phi has restricted orthogonality and that \delta_1 is sufficiently small. First, to demonstrate restricted orthogonality, it suffices to demonstrate FRO by Theorem 30, and so we will ensure that the following quantity is small:

Notice that X_m:=\sum_{i\in\mathcal{I}}\varphi_i[m] and Y_m:=\sum_{j\in\mathcal{J}}\varphi_j[m] are mutually independent over all m=1,\ldots,M since \mathcal{I} and \mathcal{J} are disjoint. Also, X_m is Gaussian with mean zero and variance \frac{|\mathcal{I}|}{M}, while Y_m similarly has mean zero and variance \frac{|\mathcal{J}|}{M}. Viewed this way, (Proof) being small corresponds to the sum of independent random variables Z_m:=X_mY_m having its probability measure concentrated at zero. To this end, Theorem 32 is naturally applicable, as the absolute central moments of a Gaussian random variable X with mean zero and variance \sigma^2 are well known:

Since Z_m=X_mY_m is a product of independent Gaussian random variables, this gives

Further since \mathbb{E}|Z_m|^2=\frac{|\mathcal{I}||\mathcal{J}|}{M^2}, we may define L:=2\frac{(|\mathcal{I}||\mathcal{J}|)^{1/2}}{M} to get (32). Later, we will take \hat\theta<\delta<\sqrt{2}-1<\frac{1}{2}. Considering

we therefore have (32), which in this case has the form

where the probability is doubled due to the symmetric distribution of \sum_{m=1}^M Z_m. Since we need to account for all possible choices of \mathcal{I} and \mathcal{J}, we will perform a union bound. The total number of choices is given by

and so the union bound gives

Thus, Gaussian matrices tend to have FRO, and hence restricted orthogonality by Theorem 30; this is made more precise below.

Again by Lemma 28, it remains to show that \delta_1 is sufficiently small. To this end, we note that M\|\varphi_n\|^2 has chi-squared distribution with M degrees of freedom, and so we can use another (simpler) concentration-of-measure result; see Lemma 1 of [LaurentM:00]:

for any t>0. Specifically, we pick

and we perform a union bound over the N choices for \varphi_n:

To summarize, Lemma 28, the union bound, Theorem 30, and (Proof) and (?) give

and so M\geq\frac{33C^2}{\delta^2}K\log^2 K\log N gives that \Phi has (2K,\delta)-RIP with high probability.

Take t:=\frac{\delta}{3K^{1/2q}}-(\frac{K}{M})^{1/2} and pick \mathcal{K}\subseteq\{1,\ldots,N\}. Then Theorem II.13 of [DavidsonS:01] states

Continuing, we use the fact that \lambda(\Phi_\mathcal{K}^*\Phi_\mathcal{K}^{})=\sigma(\Phi_\mathcal{K})^2 to get

where the last inequality follows from the fact that (\frac{K}{M})^{1/2}+t<1. Since \Phi_\mathcal{K}^*\Phi_\mathcal{K}^{} and I_K are simultaneously diagonalizable, the spectrum of \Phi_\mathcal{K}^*\Phi_\mathcal{K}^{}-I_K is given by \lambda(\Phi_\mathcal{K}^*\Phi_\mathcal{K}^{}-I_K)=\lambda(\Phi_\mathcal{K}^*\Phi_\mathcal{K}^{})-1. Combining this with (Proof) then gives

Considering \mathrm{Tr}[A^{2q}]^\frac{1}{2q}=\|\lambda(A)\|_{2q}\leq K^\frac{1}{2q}\|\lambda(A)\|_\infty, we continue:

From here, we perform a union bound over all possible choices of \mathcal{K}:

Rearranging M\geq\frac{81}{\delta^2}K^{1+1/q}\log\frac{eN}{K} gives K^{1/2}\leq\frac{\delta M^{1/2}}{9K^{1/2q}\log^{1/2}(eN/K)}\leq\frac{\delta M^{1/2}}{9K^{1/2q}}, and so

Combining (Proof) and (Proof) gives the result.

The Gershgorin analysis (?) gives the bound \delta_K\leq(K-1)\mu, and so it suffices to prove \delta_K\geq(K-1)\mu. Since K\leq\omega(G)+1, there exists a clique of size K in the join of G with a vertex. Let \mathcal{K} denote the vertices of this clique, and take S_\mathcal{K} to be the corresponding Seidel adjacency submatrix. In this case, S_\mathcal{K}=I_K-J_K, where J_K is the K\times K matrix of all 1’s. Observing the decomposition \Phi_\mathcal{K}^*\Phi_\mathcal{K}^{}=I_K+\mu S_\mathcal{K}, it follows from (4) that

which concludes the proof.

First, we note that Theorem 16 used Chebotarëv’s theorem [StevenhagenL:mi96] to prove that the spark of the M\times 2M Paley ETF \Phi is M+1, that is, every size-M subcollection of columns of \Phi forms a spanning set. Thus, for every \mathcal{K}\subseteq\{1,\ldots,2M\} of size \leq M, the smallest singular value of \Phi_\mathcal{K} is positive. It remains to show that the square of the largest singular value is strictly less than 2. Let x be a unit vector for which \|\Phi_\mathcal{K}^*x\|=\|\Phi_\mathcal{K}^*\|_2. Then since the spark of \Phi is M+1, the columns of \Phi_{\mathcal{K}^\mathrm{c}} span, and so

where the final step follows from (i) and (ii) of Section ?, which imply \Phi\Phi^*=2I_M.

We start by showing \omega(G)+1\leq M. Suppose otherwise: that there exists a clique \mathcal{K} of size M+1 in the join of a vertex with G. Then the corresponding sub-Gram matrix of the Paley ETF has the form \Phi_\mathcal{K}^*\Phi_\mathcal{K}^{}=(1+\mu)I_{M+1}-\mu J_{M+1}, where \mu=p^{-1/2} is the worst-case coherence and J_{M+1} is the (M+1)\times (M+1) matrix of 1’s. Since the largest eigenvalue of J_{M+1} is M+1, the smallest eigenvalue of \Phi_\mathcal{K}^*\Phi_\mathcal{K}^{} is 1+p^{-1/2}-(M+1)p^{-1/2}=1-\frac{1}{2}(p+1)p^{-1/2}, which is negative when p\geq 5, contradicting the fact that \Phi_\mathcal{K}^*\Phi_\mathcal{K}^{} is positive semidefinite.

Since \omega(G)+1\leq M, we can apply Lemma 39 and Theorem 37 to get

and rearranging gives the result.

Let x be as in Definition 44. Note that (44) is equivalent to \big|\|\Phi x\|^2-\|x\|^2\big|\leq\delta\|x\|^2. Defining \mathcal{K}:=\{n:|x_n|>0\}, then the Cauchy-Schwarz inequality gives

where the last inequality uses the fact that \|\cdot\|\leq\sqrt{K}\|\cdot\|_\infty in \mathbb{C}^K. We now consider Lemma 3 of [bajwa:jcn10], which states that for any \varepsilon \in [0,1) and a \geq 1, \|(\Phi_\mathcal{K}^*\Phi_\mathcal{K}^{}-I_K)x_\mathcal{K}\|_\infty \leq \varepsilon \|x_\mathcal{K}\| with probability exceeding 1-4Ke^{- (\varepsilon-\sqrt{K}\nu)^2/16(2+a^{-1})^2\mu^2} provided K \leq \min\{\varepsilon^2\nu^{-2}, (1+a)^{-1}N\}. We claim that (Proof) together with Lemma 3 of [bajwa:jcn10] guarantee \big|\|\Phi x\|^2-\|x\|^2\big|\leq\delta\|x\|^2 with probability exceeding 1-\frac{4K}{N^2}. In order to establish this claim, we fix \varepsilon=10\mu\sqrt{2\log{N}} and a=2\log{128}-1. It is then easy to see that (SCP-1) gives \varepsilon < 1, and also that (SCP-2) and 2K\log{N} \leq M give K \leq \varepsilon^2\nu^{-2}/9. Therefore, since the assumption that N \geq 128 together with 2K\log{N} \leq M implies K \leq (1+a)^{-1}N, we obtain e^{- (\varepsilon - \sqrt{K}\nu)^2/16(2+a^{-1})^2\mu^2} \leq \frac{1}{N^2}. The result now follows from the observation that 2K\log{N} \leq \frac{\delta^2}{100\mu^2} implies \sqrt{K}\varepsilon \leq \delta.

To begin, note that since \|\Phi\|_2^2\geq\frac{N}{M}, we have from (46) that K\leq M/(2\log{N}). It is then easy to conclude from Theorem 5 of [bajwa:jcn10] that \hat{\mathcal{K}} satisfies \mathcal{T}_\sigma(t) \cap \mathcal{T}_\mu(t) \subseteq \hat{\mathcal{K}} \subseteq \mathcal{K} with probability exceeding 1 - 6N^{-1}. Therefore, conditioned on the event \mathcal{E}_1 := \{\mathcal{T}_\sigma(t) \cap \mathcal{T}_\mu(t) \subseteq \hat{\mathcal{K}} \subseteq \mathcal{K}\}, we can make use of the triangle inequality to write

Next, we may use (46) and the fact that \Phi satisfies the strong coherence property to conclude from [tropp:cras08] (see, e.g., Proposition 3 of [bajwa:jcn10]) that \|\Phi_\mathcal{K}^*\Phi_\mathcal{K}^{}-I_K\|_2 < e^{-1/2} with probability exceeding 1 - 2N^{-1}. Hence, conditioning on \mathcal{E}_1 and \mathcal{E}_2 := \{\|\Phi_\mathcal{K}^*\Phi_\mathcal{K}^{}-I_K\|_2 < e^{-1/2}\}, we have that (\Phi_{\hat{\mathcal{K}}})^\dagger = (\Phi_{\hat{\mathcal{K}}}^* \Phi_{\hat{\mathcal{K}}}^{})^{-1} \Phi_{\hat{\mathcal{K}}}^* since \Phi_{\hat{\mathcal{K}}} is a submatrix of a full column rank matrix \Phi_\mathcal{K}. Therefore, given \mathcal{E}_1 and \mathcal{E}_2, we may write

and so substituting (Proof) into (Proof) and applying the triangle inequality gives

Since, given \mathcal{E}_1, we have that \Phi_{\hat{\mathcal{K}}}^* \Phi_{\hat{\mathcal{K}}}^{} - I_K and \Phi_{\hat{\mathcal{K}}}^* \Phi_{\mathcal{K} \setminus \hat{\mathcal{K}}}^{} are submatrices of \Phi_\mathcal{K}^* \Phi_\mathcal{K}^{} - I_K, and since the spectral norm of a matrix provides an upper bound for the spectral norms of its submatrices, we have the following given \mathcal{E}_1 and \mathcal{E}_2: \|\Phi_{\hat{\mathcal{K}}}^* \Phi_{\mathcal{K} \setminus \hat{\mathcal{K}}}^{}\|_2 \leq e^{-1/2} and \|(\Phi_{\hat{\mathcal{K}}}^* \Phi_{\hat{\mathcal{K}}}^{})^{-1}\|_2 \leq \frac{1}{1-e^{-1/2}}. We can now substitute these bounds into (Proof) and make use of the fact that \|\Phi_{\hat{\mathcal{K}}}^* z\| \leq |\hat{\mathcal{K}}|^{1/2}\|\Phi_{\hat{\mathcal{K}}}^* z\|_\infty to conclude that

given \mathcal{E}_1 and \mathcal{E}_2. At this point, define the event \mathcal{E}_3 = \{\|\Phi_{\hat{\mathcal{K}}}^* z\|_\infty \leq 2\sqrt{\sigma^2 \log{N}}\} and note from Lemma 6 of [bajwa:jcn10] that \Pr(\mathcal{E}_3^\mathrm{c}) \leq 2(\sqrt{2\pi\log{N}}~N)^{-1}. A union bound therefore gives (46) with probability exceeding 1 - 10N^{-1}. For (46), note that \hat{\mathcal{K}} \subseteq \mathcal{K} implies |\hat{\mathcal{K}}| \leq K, and so \mathcal{T}_\sigma(t) \cap \mathcal{T}_\mu(t) \subseteq \hat{\mathcal{K}} implies that \|x_{\mathcal{K} \setminus \hat{\mathcal{K}}}\| \leq \|x_{\mathcal{K} \setminus (\mathcal{T}_\sigma(t) \cap \mathcal{T}_\mu(t))}\| = \|x - x_T\|.

Let \mathcal{K} be the support of x, and define \mathcal{I} := \{n : |x_n| = \alpha\}. We wish to show that \mathcal{I}\subseteq\mathcal{T}_\mu(t), since this implies \|x_{\mathcal{K} \setminus \mathcal{T}_\mu(t)}\| \leq \|x_{\mathcal{K} \setminus \mathcal{I}}\| = \mathrm{O}(\sqrt{\sigma^2 K\log{N}}). In order to prove \mathcal{I}\subseteq\mathcal{T}_\mu(t), notice that

and so combining this with the fact that \|\Phi\|_2^2\geq\frac{N}{M} gives

Therefore, provided c_0\leq\frac{tc_1}{20\sqrt{2}}, we have that \mathcal{I}\subseteq\mathcal{T}_\mu(t).

For condition (i), we have

The Welch bound (Theorem 3) therefore gives \nu=\frac{1}{N-1}\big(\frac{N}{M}-1\big)=\frac{N-M}{M(N-1)}\leq\mu\sqrt{\frac{N-M}{M(N-1)}} \leq\frac{\mu}{\sqrt{M}}. For condition (ii), we have

Considering the Welch bound, it suffices to show \frac{1}{N-1}\leq\frac{1}{\sqrt{M}}\sqrt{\frac{N-M}{M(N-1)}}. Rearranging gives

When N=2M, the left-hand side of (Proof) becomes (M-1)^2, which is trivially nonnegative. Otherwise, we have

In this case, by the quadratic formula and the fact that the left-hand side of (Proof) is concave up in N, we have that (Proof) is indeed satisfied. For condition (iii), we use the triangle and Cauchy-Schwarz inequalities to get

Considering the Welch bound, it suffices to show \frac{\sqrt{N}+1}{N-1}\leq\frac{1}{\sqrt{M}}\sqrt{\frac{N-M}{M(N-1)}}. Taking x:=\sqrt{N} and rearranging gives a polynomial: x^4-(M^2+M+1)x^2-2M^2x-M(M-1)\geq0. By convexity and monotonicity of the polynomial in [M+\frac{3}{2},\infty), it can be shown that the largest real root of this polynomial is always smaller than M+\frac{3}{2}. Also, considering it is concave up in x, it suffices that \sqrt{N}=x\geq M+\frac{3}{2}, which we have since N\geq M^2+3M+3\geq(M+\frac{3}{2})^2.

Theorem 49(i) can be shown to hold with probability exceeding 1 - 2N^{-1} by using a bound on the norm of a Gaussian random vector in Lemma 1 of [LaurentM:00] and a bound on the magnitude of the inner product of two independent Gaussian random vectors in Lemma 6 of [haupt:tit10]. Specifically, pick any two distinct indices i, j \in \{1,\dots,N\}, and define probability events \mathcal{E}_1 := \{|\langle \psi_i,\psi_j\rangle| \leq \varepsilon_1\}, \mathcal{E}_2 := \{\|\psi_i\|^2 \geq M(1 - \varepsilon_2)\}, and \mathcal{E}_3 := \{\|\psi_j\|^2 \geq M(1 - \varepsilon_2)\} for \varepsilon_1 = \sqrt{15M\log{N}} and \varepsilon_2 = \sqrt{(12\log{N})/M}. Then it follows from the union bound that

One can verify that \Pr(\mathcal{E}_2^\mathrm{c}) = \Pr(\mathcal{E}_3^\mathrm{c}) \leq N^{-3} because of Lemma 1 of [LaurentM:00], and we further have \Pr(\mathcal{E}_1^\mathrm{c}) \leq 2N^{-3} because of Lemma 6 of [haupt:tit10] and the fact that M \geq 60\log{N}. Thus, for any fixed i and j, |\langle \varphi_i,\varphi_j\rangle| \leq \sqrt{15\log{N}}/(\sqrt{M} - \sqrt{12\log{N}}) with probability exceeding 1 - 4N^{-3}. It therefore follows by taking a union bound over all \binom{N}{2} choices for i and j that Theorem 49(i) holds with probability exceeding 1 - 2N^{-1}.

Theorem 49(ii) can be shown to hold with probability exceeding 1 - 6N^{-1} by appealing to the preceding analysis and Hoeffding’s inequality for a sum of independent, bounded random variables [hoeffding:jasa63]. Specifically, fix any index i \in \{1,\dots,N\}, and define random variables Z_{ij} := \frac{1}{N-1}\langle \varphi_i, \varphi_j\rangle. Next, define the probability event

Using the analysis for the worst-case coherence of \Phi and taking a union bound over the N-1 possible j’s gives \Pr(\mathcal{E}_4^\mathrm{c}) \leq 4N^{-2}. Furthermore, taking \varepsilon_3 := \sqrt{15\log{N}}/(M - \sqrt{12M\log{N}}), then elementary probability analysis gives

where \mathbb{S}^{M-1} denotes the unit hypersphere in \mathbb{R}^M, \mathrm{H}^{M-1} denotes the (M-1)-dimensional Hausdorff measure on \mathbb{S}^{M-1}, and p_{\varphi_i}(x) denotes the probability density function for the random vector \varphi_i. The first thing to note here is that the random variables \{Z_{ij}: j\not=i\} are bounded and jointly independent when conditioned on \mathcal{E}_4 and \varphi_i. This assertion mainly follows from Bayes’ rule and the fact that \{\varphi_j:j\not=i\} are jointly independent when conditioned on \varphi_i. The second thing to note is that \mathbb{E}[Z_{ij}~|~\mathcal{E}_4, \varphi_i] = 0 for every j\neq i. This comes from the fact that the random vectors \{\varphi_n\}_{n=1}^N are independent and have a uniform distribution over \mathbb{S}^{M-1}, which in turn guarantees that the random variables \{Z_{ij}: j\not=i\} have a symmetric distribution around zero when conditioned on \mathcal{E}_4 and \varphi_i. We can therefore make use of Hoeffding’s inequality [hoeffding:jasa63] to bound the probability expression inside the integral in (?) as

which is bounded above by 2N^{-2} provided M \leq \frac{N-1}{4\log{N}}. We can now substitute (?) into (?) and take the union bound over the N possible choices for i to conclude that Theorem 49(ii) holds with probability exceeding 1 - 6N^{-1}.

Lastly, Theorem 49(iii) can be shown to hold with probability exceeding 1 - 3N^{-1} by using a bound on the spectral norm of standard Gaussian random matrices reported in [rudelson:icm10] along with Lemma 1 of [LaurentM:00]. Specifically, define an N \times N diagonal matrix D := \mathrm{diag}(\|\psi_1\|^{-1}, \dots, \|\psi_N\|^{-1}), and note that the entries of \Psi := \Phi D^{-1} are independently and normally distributed with zero mean and unit variance. We therefore have from (2.3) in [rudelson:icm10] that

In addition, we can appeal to the preceding analysis for the probability bound on Theorem 49(i) and conclude using Lemma 1 of [LaurentM:00] and a union bound over the N possible choices for i that

Finally, since \|\Phi\|_2 \leq \|\Psi\|_2 \|D\|_2, we can take a union bound over (?) and (?) to argue that Theorem 49(iii) holds with probability exceeding 1 - 3N^{-1}.

The complete result now follows by taking a union bound over the failure probabilities for the conditions (i)-(iii) in Theorem 49.

The claim that \Phi is tight follows trivially from the fact that the rows of F are orthogonal and that the rows of \Phi correspond to a subset of the rows of F. Next, we define the probability events \mathcal{E}_1 := \{|\mathcal{M}| \leq \frac{3}{2}M\} and \mathcal{E}_2 := \{|\mathcal{M}| \geq \frac{1}{2}M\}, and claim that \Pr(\mathcal{E}_1^\mathrm{c} \cup \mathcal{E}_2^\mathrm{c}) \leq N^{-1} + N^{-2}. The proof of this claim follows from a Bernstein-like large deviation inequality. Specifically, note that |\mathcal{M}| = \sum_{i=0}^{N-1} B_i with \mathbb{E}[|\mathcal{M}|] = M, and so we have from Theorems A.1.12 and A.1.13 of [alon:00] and page 4 of [RudelsonV:08] that for any \varepsilon_1 \in [0,1),

Taking \varepsilon_1 := \frac{1}{2}, then a union bound gives \Pr(\mathcal{E}_1^\mathrm{c} \cup \mathcal{E}_2^\mathrm{c}) \leq N^{-1} + N^{-2} provided M \geq 16 \log{N}. Conditioning on \mathcal{E}_1 \cap \mathcal{E}_2, we have that Theorem 51(i) holds trivially, while Theorem 51(ii) follows from Lemma 48. Specifically, we have that \frac{N}{3} \geq M guarantees N \geq 2|\mathcal{M}| because of the conditioning on \mathcal{E}_1 \cap \mathcal{E}_2, which in turn implies that \Phi satisfies either condition (i) or (ii) of Lemma 48, depending on whether 0\in\mathcal{M}. This therefore establishes that Theorem 51(i)-(ii) simultaneously hold with probability exceeding 1 - N^{-1} - N^{-2}.

The only remaining claim is that \mu \leq \varepsilon_2 := \sqrt{(118(N-M)\log{N})/MN} with high probability. To this end, define p:=\frac{M}{N}, and pick any two distinct indices i, j \in \{0,\dots,N-1\}. Note that

where the last equality follows from the fact that F has orthogonal columns. Next, we write F_{ki}\overline{F_{kj}}=\cos(\theta_k)+\mathrm{i}\sin(\theta_k) for some \theta_k\in[0,2\pi). Then applying the union bound to (?) and to the real and imaginary parts of F_{ki}\overline{F_{kj}} gives

where the last term follows from (Proof) and the fact that M \geq 16 \log{N}. Define random variables Z_k:=(B_k-p)\cos(\theta_k). Note that the Z_k’s have zero mean and are jointly independent. Also, the Z_k’s are bounded by 1-p almost surely since |(B_k-p)\cos(\theta_k)|\leq\max\{p,1-p\} and N\geq 2M. Moreover, the variance of each Z_k is bounded: \mathrm{Var}(Z_\ell)\leq p(1-p). Therefore, we may use the Bernstein inequality for a sum of independent, bounded random variables [bennett:jasa62] to bound the probability that |\sum_{k=0}^{N-1} Z_k| deviates from \varepsilon_3 := \frac{M\varepsilon_2}{4}:

Similarly, the probability that |\sum_{k=0}^{N-1} (B_k - p)\sin(\theta_k)| > \varepsilon_3 is also bounded above by 2N^{-3}. Substituting these probability bounds into (?) gives |\langle \varphi_i,\varphi_j\rangle| > \varepsilon_2 with probability at most 5N^{-3} provided M \geq 16\log{N}. Finally, we take a union bound over the \binom{N}{2} possible choices for i and j to get that Theorem 51(iii) holds with probability exceeding 1 - 3N^{-1}.

The result now follows by taking a final union bound over \mathcal{E}_1^\mathrm{c} \cup \mathcal{E}_2^\mathrm{c} and \{\mu > \varepsilon_2\}.

The tightness claim follows from [lawrence:jfaa05], in which it was shown that Gabor frames generated by nonzero seed vectors are tight. The bound on average coherence is a consequence of Theorem 7 of [bajwa:jcn10] concerning arbitrary Gabor frames. The claim concerning \mu_\Phi follows directly from [StrohmerH:03], while the claim concerning \mu_\Psi is a simple consequence of Theorem 5.1 of [pfander:tsp08].

It is easy to verify that \Phi is a unit norm tight frame using the geometric sum formula. Also, since the frame elements sum to zero and N\geq 2M, the claim regarding average coherence follows from Lemma 48(ii). It remains to prove \mu_\Phi\leq\mu_\Psi. For each pair of indices i,j\in\{1,\ldots,N\}, we have

and so |\langle \varphi_i,\varphi_j\rangle|=|\mathrm{Re}\langle \psi_i,\psi_j\rangle|\leq|\langle \psi_i,\psi_j\rangle|. This gives the result.

For the tightness claim, we use the linearity of the trace map to write the inner product of rows x and y:

This expression is 2^{tm} when x=y. Otherwise, note that \alpha_0\mapsto(-1)^{\mathrm{Tr}[\alpha_0(x+y)]}\in\{\pm1\} defines a homomorphism on \mathbb{F}_{2^m}. Since (x+y)^{-1}\mapsto-1, the inverse images of \pm1 under this homomorphism must form two cosets of equal size, and so \sum_{\alpha_0\in\mathbb{F}_{2^m}}(-1)^{\mathrm{Tr}[\alpha_0(x+y)]}=0, meaning distinct rows in \Phi are orthogonal. Thus, \Phi is a unit norm tight frame.

For the worst-case coherence claim, we first note that the linearity of the trace map gives

i.e., every inner product between columns of \Phi is a sum over another column. Thus, there exists \alpha\in\mathbb{F}_{2^m}^{t+1} such that

where the last equality is by the identity (x+y)^{2^i+1}=x^{2^i+1}+y^{2^i+1}+\sum_{j=0}^{i-1}(xy)^{2^j}(x+y)^{2^i-2^{j+1}+1}, whose proof is a simple exercise of induction. From here, we perform a change of variables: u:= x+y and v:= xy. Notice that (u,v) corresponds to (x,y) for some x\neq y whenever (z+x)(z+y)=z^2+uz+v has two solutions, that is, whenever \smash{\mathrm{Tr}(\frac{v}{u^2})=0}. Since (u,v) corresponds to both (x,y) and (y,x), we must correct for under-counting:

where the second equality is by repeated application of \mathrm{Tr}(z)=\mathrm{Tr}(z^2), and

To bound \mu, we will count the u’s that produce nonzero summands in (?).

For each u\neq0, we have a homomorphism \chi_u\colon\{v\in\mathbb{F}_{2^m}:\mathrm{Tr}(\frac{v}{u^2})=0\}\rightarrow\{\pm1\} defined by \chi_u(v):=(-1)^{\mathrm{Tr}[p(u)v]}. Pick u\neq0 for which there exists a v such that both \smash{\mathrm{Tr}(\frac{v}{u^2})=0} and \mathrm{Tr}[p(u)v]=1. Then \chi_u(v)=-1, and so the kernel of \chi_u is the same size as the coset \smash{\{v\in\mathbb{F}_{2^m}:\mathrm{Tr}(\frac{v}{u^2})=0,\chi_u(v)=-1\}}, meaning the summand associated with u in (?) is zero. Hence, the nonzero summands in (?) require \smash{\mathrm{Tr}(\frac{v}{u^2})=0} and \mathrm{Tr}[p(u)v]=0. This is certainly possible whenever p(u)=0. Exponentiation gives

which has degree 2^{2t-1}-2^{t-1}. Thus, p(u)=0 has at most 2^{2t-1}-2^{t-1} solutions, and each such u produces a summand in (?) of size 2^{m-1}. Next, we consider the u’s for which \smash{\mathrm{Tr}(\frac{v}{u^2})=0}, \mathrm{Tr}[p(u)v]=0, and p(u)\neq0. In this case, the hyperplanes defined by \smash{\mathrm{Tr}(\frac{v}{u^2})=0} and \mathrm{Tr}[p(u)v]=0 are parallel, and so \smash{p(u)=\frac{1}{u^2}}. Here,

which has degree 2^{2t-1}+2^{t-1}. Thus, \smash{p(u)=\frac{1}{u^2}} has at most 2^{2t-1}+2^{t-1} solutions, and each such u produces a summand in (?) of size 2^{m-1}. We can now continue the bound from (?): 2^{2m}\mu^2\leq 2^m+2(2^{2t-1}-2^{t-1}+2^{2t-1}+2^{t-1})2^{m-1}\leq 2^{m+2t+1}. From here, isolating \mu gives the claim.

Lastly, for average coherence, pick some x\in\mathbb{F}_{2^m}. Then summing the entries in the xth row gives

That is, the frame elements sum to a multiple of an identity basis element: \sum_{\alpha\in\mathbb{F}_{2^m}^{t+1}}\varphi_\alpha=2^{(t+1/2)m}\delta_0. Since every entry in row x=0 is \smash{\frac{1}{\sqrt{2^{m}}}}, we have \langle \varphi_{\alpha'},\sum_{\alpha\in\mathbb{F}_{2^m}^{t+1}}\varphi_\alpha \rangle=\frac{2^{(t+1)m}}{2^m} for every \alpha'\in\mathbb{F}_{2^m}^{t+1}, and so by Lemma 48(i), we are done.

As in the proof of Theorem 66, we relate packing N unit vectors to packing 2N points in the hypersphere \mathbb{S}^{M-1}\subseteq\mathbb{R}^M. The argument in the proof of Lemma 65 generalizes so that two of the 2N points must reside in some closed hyperspherical cap of hypersurface area \frac{1}{2N}\mathrm{H}^{M-1}(\mathbb{S}^{M-1}). Therefore, the smallest angle \alpha between these points is no more than twice the radius of this cap. Let C(\gamma) denote a hyperspherical cap of angular radius \gamma. Then we use hyperspherical coordinates to get

We wish to solve for \gamma, but analytically inverting \int_0^\gamma\sin^{M-2}\phi ~\mathrm{d}\phi is difficult. Instead, we use \sin\phi\geq\frac{2\phi}{\pi} for \phi\in[0,\frac{\pi}{2}]. Note that we do not lose generality by forcing \gamma\leq\frac{\pi}{2}, since this is guaranteed with N\geq2. Continuing (Proof) gives

Using the formula for a hypersphere’s hypersurface area, we can express the left-hand side of (Proof):

Isolating 2\gamma above and using \alpha\leq2\gamma and \mu=\cos\alpha gives (64). The second part of the result comes from a simple application of Stirling’s approximation.

We first claim there exists a closed spherical cap in \mathbb{S}^{2} with area \smash{\frac{4\pi}{N}} that contains two of the N points. Suppose otherwise, and take \gamma to be the angular radius of a spherical cap with area \smash{\frac{4\pi}{N}}. That is, \gamma is the angle between the center of the cap and every point on the boundary. Since the cap is closed, we must have that the smallest angle \alpha between any two of our N points satisfies \alpha>2\gamma. Let C(p,\theta) denote the closed spherical cap centered at p\in \mathbb{S}^2 of angular radius \theta, and let P denote our set of N points. Then we know for p\in P, the C(p,\gamma)’s are disjoint, \frac{\alpha}{2}>\gamma, and \bigcup_{p\in P}C(p,\tfrac{\alpha}{2})\subseteq \mathbb{S}^2, and so taking 2-dimensional Hausdorff measures on the sphere gives

a contradiction.

Since two of the points reside in a spherical cap of area \smash{\frac{4\pi}{N}}, we know \alpha is no more than twice the radius of this cap. We use spherical coordinates to relate the cap’s area to the radius: \smash{\mathrm{H}^2(C(\cdot,\gamma))=2\pi\int_0^\gamma\sin\phi~\mathrm{d}\phi=2\pi(1-\cos\gamma)}. Therefore, when \smash{\mathrm{H}^2(C(\cdot,\gamma))=\frac{4\pi}{N}}, we have \gamma=\cos^{-1}(1-\frac{2}{N}), and so \alpha\leq2\gamma gives the result.

Packing N unit vectors in \mathbb{R}^3 corresponds to packing 2N antipodal points in \mathbb{S}^2, and so Lemma 65 gives \alpha\leq2\cos^{-1}(1-\frac{1}{N}). Applying the double angle formula to

gives the result.

Take two frames \Phi and \Psi such that \Psi=\Phi D. The first claim is immediate. Next, the Gram matrices are related by \Psi^*\Psi=D^*\Phi^*\Phi D. Since corresponding off-diagonal entries are equal in modulus, we know the worst-case coherences are equal. Finally, \|\Psi\|_2^2=\|\Psi\Psi^*\|_2^2=\|\Phi DD^*\Phi^*\|_2^{}=\|\Phi\Phi^*\|_2^{}=\|\Phi\|_2^2, and so we are done.

Take \{R_n\}_{n=1}^N to be a Rademacher sequence that independently takes values \pm1, each with probability \frac{1}{2}. We use this sequence to randomly flip \Phi; define Z:=\Phi~\mathrm{diag}\{R_n\}_{n=1}^N. Note that if \smash{\Pr(\nu_Z\leq\frac{\mu_\Phi}{\sqrt{M}})>0}, we are done. Fix some i\in\{1,\ldots,N\}. Then

We can view \sum_{j\neq i} R_j\langle \varphi_i,\varphi_j\rangle as a sum of N-1 independent zero-mean complex random variables that are bounded by \mu_\Phi. We can therefore use a complex version of Hoeffding’s inequality [hoeffding:jasa63] (see, e.g., Lemma 3.8 of [bajwa:thesis]) to bound the probability expression in (Proof) as \leq4e^{-(N-1)/4M}. From here, a union bound over all N choices for i gives \Pr(\nu_Z\leq\frac{\mu_\Phi}{\sqrt{M}})\geq 1-4Ne^{-(N-1)/4M}, and so M < \frac{N-1}{4\log 4N} implies \Pr(\nu_Z\leq\frac{\mu_\Phi}{\sqrt{M}})>0, as desired.

Considering Lemma 48(iii), it suffices to have \|\sum_{n=1}^N \psi_n\|^2\leq N. We will use induction to show \|\sum_{n=1}^k \psi_n\|^2\leq k for k=1,\ldots,N. Clearly, \|\sum_{n=1}^1 \psi_n\|^2=\|\varphi_n\|^2=1\leq1. Now assume \|\sum_{n=1}^k \psi_n\|^2\leq k. Then by our choice for \psi_{k+1} in Algorithm ?, we know that \|\sum_{n=1}^k\psi_n+\psi_{k+1}\|^2\leq\|\sum_{n=1}^k\psi_n-\psi_{k+1}\|^2. Expanding both sides of this inequality gives

and so \mathrm{Re}\langle\sum_{n=1}^k\psi_n,\psi_{k+1}\rangle\leq0. Therefore,

where the last inequality uses the inductive hypothesis.