arXiv reader

The stochastic block model has the overlap graph property for modularity

Shankar Bhamidi, David Gamarnik, Remco van der Hofstad, Nelly Litvak, Pawel Pralat, Fiona Skerman and Yasmin Tousinejad

arXiv
2605.10911v1 · PDF · 11 May 2026
Categories
math.PR, cs.CC, cs.DS, math.CO, math.ST, stat.TH
Licence
CC BY 4.0 The site shows the text.

Abstract

The overlap gap property (OGP) is a statement about the geometry of near-optimal solutions. Exhibiting OGP implies failure of a class of local algorithms; and has been observed to coincide with conjectured algorithmic limits in problems with statistical computational gap. We consider the Stochastic Block Model (SBM), where the graph has a planted partition with k equal-size blocks which form the ‘communities’, and where, for parameters p>q, vertices within the same community connect with probability p, while vertices in different communities connect with probability q, independently across pairs of vertices. Modularity–based clustering algorithms have become ubiquitous in applications. This article studies theoretical limits of local algorithms based on the modularity score on the SBM. We establish that modularity exhibits OGP on the SBM. This rules out a class of local algorithms based on modularity for recovery in the SBM, and shows slow mixing time for a related Markov Chain. Theoretically this is one of the few instances where OGP has been established for a ‘planted’ model, as most such analyses to date consider the ‘null’ model. As part of our analysis, we extend a result by Bickel and Chen 2009, who established that with high probability, the modularity optimal partition of SBM is o(n) local moves away from the planted partition, where n is the graph size. We show that, with high probability, any partition with modularity score sufficiently near the optimal value is close to the planted partition.

Items 22

Definition

Definition 1 (Overlap gap property – planted model)

YLH0

For a graph G_n with planted partition \mathcal{P}, the optimisation problem \max_\mathcal{A} q_\mathcal{A}(G_n) exhibits OGP with values \mu > 0 and 0 \leq \nu_1 < \nu_2 if the following holds: For any partition \mathcal{B} for which q_\mathcal{B}(G_n)\geq q^{\star}(G_n)-\mu, it holds that either d(\mathcal{B}, \mathcal{P})\leq \nu_1, or that d(\mathcal{B}, \mathcal{P})\geq\nu_2. Furthermore, d(\mathcal{B}', \mathcal{P})\geq \nu_2 does indeed occur for some partition \mathcal{B}' with q_{\mathcal{B}'}(G_n)\geq q^{\star}(G_n) - \mu.

Theorem

Theorem 2 (Modularity has OGP)

7RMO

Fix real numbers a>b>0, and integer k\geq 3. For any \nu\in \left(\frac{1}{2(k-1)}, \frac{1}{k}\right) and \varepsilon>0, there exist \mu = \mu(\nu) and c=c(0.007) such that the following holds for n large enough.

Let p=p(n)=\omega\; a / n and q=q(n)=\omega \; b/n be such that \omega>c, and let G \in \mathcal{G}(n,k,p,q) with planted partition \mathcal{P}. Then with probability at least 1-0.007, for every k-part partition \mathcal{A} with

q_\mathcal{A}(G) \ge \frac {a-b}{a+(k-1)b} \left( 1 - \frac {1}{k} - \frac {2}{k^2} \right) - \mu,

either d(\mathcal{A}, \mathcal{P}) \le \frac {{1}}{2(k-1)} or d(\mathcal{A}, \mathcal{P}) \ge \nu. Moreover, there are partitions \mathcal{A} satisfying the latter.

Theorem

Theorem 3 (Informal)

AQO6

For every \zeta>0, there exists a large enough inverse temperature parameter \beta>0, such that when the chain is initiated at distance at least 1/k from the ground truth, the time to reach a partition close to the ground truth is at least \exp(\Theta(n)) with high probability.

Theorem

Theorem 4

J07H

Fix real numbers a>b>0, integer k\geq 2 and let \omega=\omega(n) \to \infty as n \to \infty.

Let p=\omega\, a/n and q=\omega\, b /n, and let G \in \mathcal{G}(n,k,p,q) with the planted partition \mathcal{P}. Then,

d\big( \arg \max q_\mathcal{A}(G) , \, \mathcal{P} \big) = o(1).
Theorem

Theorem 5 (Partitions with near-optimal modularity are close to the planted)

91RD

Fix a>b>0 and integer k\ge 2. Let p=p(n)=\omega\; a / n and q=q(n)=\omega \; b/n. For any 0.007>0 and \delta<\frac{a-b}{a+(k-1)b} \cdot \frac{2}{k^2}, there exists c=c(0.007) such that, if \omega>c for n large enough, then the following holds with probability at least 1-0.007 provided n is large enough.

Let G \in \mathcal{G}(n,k,p,q) with the planted partition \mathcal{P}. Then, q_\mathcal{A}(G)>q^{\star}(G) - \delta for some k-part partition \mathcal{A} implies that d(\mathcal{A}, \mathcal{P}) < \delta' + \varepsilon, where \delta' =\delta'(\delta) \in \big(0,\frac{1}{k(k-1)}\big).

Theorem

Theorem 6 (Maximal modularity at distance at most 1/k)

C4IV

Fix a>b>0. Let p=p(n)=\omega\; a / n and q=q(n)=\omega \; b/n. For any 0.007>0, there exists c=c(0.007) such that if \omega>c then the following holds with probability at least 1-0.007 provided n is large enough.

Let G \in \mathcal{G}(n,k,p,q) with the planted partition \mathcal{P}. For any d \in \left[0,\frac{1}{k}\right], let

H(d) = \max_{\mathcal{A}: |\mathcal{A}|\le k} \left\{ q_\mathcal{A}(G) : d - \frac{1}{\sqrt{n}} \le d(\mathcal{A}, \mathcal{P}) \le d + \frac{1}{\sqrt{n}} \right\}.

Then,

\left| H(d) - \frac {a-b}{a+(k-1)b}\cdot h(d) \right| \le 0.007,

where

h(d)= 1 - \frac {1}{k} - 2d\big (1-d(k-1)\big).
Theorem

Theorem 7 (Maximal modularity)

MMME

Fix a>b>0 and integer k\geq 2. Let p=p(n)=\omega\; a / n and q=q(n)=\omega \; b/n. For any 0.007>0, there exists c=c(0.007) such that, if \omega>c for n large enough, then the following holds with probability at least 1-0.007 provided n is large enough.

Let G \in \mathcal{G}(n,k,p,q) and denote the planted partition by \mathcal{P}. Then,

\left| q_\mathcal{P}(G) - \frac {a-b}{a+(k-1)b} \left( 1 - \frac {1}{k} \right) \right| \le \frac {0.007}{2}

and

\Big| q^{\star}(G) - q_\mathcal{P}(G) \Big| \le \frac {0.007}{2}.

As a result,

\left| q^{\star}(G) - \frac {a-b}{a+(k-1)b} \left( 1 - \frac {1}{k} \right) \right| \le 0.007.
Lemma

Lemma 8

YR8U

Suppose that n is divisible by k. Let \mathcal{A} be any partition of [n] with k parts, and let \mathcal{P} be the planted partition. Let X=X(\mathcal{A})=(x_{ij})_{1\leq i \leq j \leq k} be the signature of \mathcal{A}. Then,

d(\mathcal{A}, \mathcal{P}) = 1-\tfrac{1}{k}\max_{\sigma} \sum_{i=1}^k x_{\sigma(i) i},

where the maximum is taken over all permutations of [k].

Moreover, d(\mathcal{A}, \mathcal{P}) \leq 1 -1/k and this upper bound is sharp, that is, there exists a partition \mathcal{A} such that d(\mathcal{A}, \mathcal{P}) = 1 -1/k, provided that n is divisible by k^2.

Lemma

Lemma 9 (Concentration of modularity)

HD9H

Fix a>b>0 and integer k\geq 2. Let p=p(n)=\omega\; a / n and q=q(n)=\omega \; b/n. For any 0.007>0, there exists c=c(0.007) such that if \omega>c for n large enough, then the following holds with probability at least 1-0.007 provided n is large enough.

Let G \in \mathcal{G}(n,k,p,q). For all k-part partitions \mathcal{A},

\left| q_\mathcal{A}(G) - \frac{a-b}{(a+(k-1)b)} \cdot \frac {g(X)}{k^2} \right| < 0.007,

where X = X(\mathcal{A}) =(x_{ij})_{1\leq i,j \leq k} is the signature of \mathcal{A} and g(X) is defined in (?).

Lemma

Lemma 10

DNGV

Let 0\leq t <1. Define \mathcal{X}_k(t) to be the family of matrices X=(x_{ij})_{1\leq i \leq j \leq k} such that

\begin{aligned} &x_{ij} \geq 0, &\mbox{for all $i,j \in [k]$},\\[0.5em] &\sum_i x_{ij} = 1 &\mbox{for all $j \in [k]$}, \\ &\sum_{i\neq j}x_{ij} = t, \mbox{ and }\\ &\sum_{i}x_{ii} \geq \max_{\sigma} \sum_i x_{\sigma(i)i}. \end{aligned}

Let g(X) be defined as in (?). Then,

\max_{X\in \mathcal{X}_k(t)} g(X) = k(k\!-\!1) - 2t \big( k\!-\!t(k\!-\!1)\big)\;\; \mbox{and} \;\; \arg\max_{X\in \mathcal{X}_k(t)} g(X) = \{ X_k^{(ij)}(t) \; : \; i\neq j \},

where, for i\neq j, X^{(ij)}_k(t) is the k\times k matrix with x_{ij}=t, x_{jj}=1-t, all other diagonal elements 1 and non-diagonal elements 0 (i.e., x_{aa}=1 for all a\neq j and x_{ab}=0 for all a\neq b such that (a,b)\neq (i,j)).

Lemma

Lemma 11

90CS

Consider \mathcal{X}_k(t) as defined in (?), and g(X) as defined in (?). Then, for any t>1,

\max_{X\in \mathcal{X}_k(t)} g(X) < k(k-1)-2.
Proposition

Proposition 12

9A53

Let \mathcal{X}_t' be defined as in (?) with the additional restriction that \sum_j x_{ij} =1 for all i\in[k]. Then g^{\rm bal}(t)=\max_{X \in \mathcal{X}_t'}g(X) is strictly decreasing on 0<t<k-1.

Proposition

Proposition 13

6EUG

For every 0.007>0,\zeta>0, there exists large enough \beta>0 such that

\begin{aligned} \sum_{\mathcal{A}\colon d(\mathcal{A},\mathcal{P}) \le \zeta} {\mathbb P}_{\rm Gibbs}(\mathcal{A}) \ge 1-0.007. \end{aligned}
Theorem

Theorem 14

07FI

Consider the MCMC initiated at the Gibbs distribution conditioned on being in \mathcal{E}_{\rm far}. Namely, suppose

\begin{aligned} {\mathbb P}\left(\mathcal{A}_0=\mathcal{A}\right) = {{\mathbb P}_{\rm Gibbs}(\mathcal{A}) \over \sum_{\mathcal{B}\in \mathcal{E}_{\rm far}} {\mathbb P}_{\rm Gibbs}(\mathcal{B}) }, \end{aligned}

for all \mathcal{A}\in \mathcal{E}_{\rm far}, and {\mathbb P}\left(\mathcal{A}_0=\mathcal{A}\right)=0 otherwise. Let 0.22=\min\lbrace t\colon \mathcal{A}_t \in \mathcal{E}_{\rm close}\rbrace. There exist c_3,c_4>0 such that with probability at least 1-0.007 (with respect to the randomness of the graph)

\begin{aligned} {\mathbb P}\left(0.22 \ge \exp(c_3 n)\right) \ge 1-\exp(-c_4 n), \end{aligned}

where the probability is with respect to the random choices of the MCMC.

Lemma

Lemma 15

T9CR

Fix integer k \ge 2. For integer n \ge k that is divisible by k, let P_1 \cup \cdots \cup P_k be a partition of V=[n], where |P_j| = n/k for all j \in [k]. Let w=w(n, k, p, q) be the weight function on node set V with w_{uv}=p if u and v are in the same part P_j for some j \in [k] and with w_{uv}=q otherwise. Then, for all k-part partitions \mathcal{A} with signature X=X(\mathcal{A})=(x_{ij})_{1 \le i,j \le k},

q_\mathcal{A}(w) = (1+O(1/n)) \, \frac{p-q}{p+(k-1)q} \cdot \frac {g(X)}{k^2},

where g(X) is defined as in (?).

Proposition

Proposition 16

3J0Q

Given 0.007>0, there exists c=c(0.007) such that the following holds. Let w satisfy e_w(V)\geq c. Then, with probability at least 1-0.007, for any partition \mathcal{A} which is (0.007/4)–fat for w,

|q_\mathcal{A}(G)-q_\mathcal{A}(w)| \leq 0.007.
Lemma

Lemma 17

9H34

For each w on E, and each 0<\eta \leq 1, there is an \eta-fat partition \mathcal{B} of V such that q_{\mathcal{B}}(w) > q^{\star}(w) - 2\eta. Indeed, given any partition \mathcal{A} of V, the greedy amalgamating algorithm uses a linear number of operations and constructs an \eta-fat partition \mathcal{B} =\mathcal{A}(G,\eta,\mathcal{A}) such that

2\zeta < q_{\mathcal{A}}(w) - q_{\mathcal{B}}(w) < 2\eta,

where \zeta = \sum_{A\in \mathcal{A}_{\eta \; \rm{small}}} {\rm vol}_w(A)/{\rm vol}_w(V).

Lemma

Lemma 18

8TKR

Let H=(V,E) be a graph and let \mathcal{A} be a partition of V.

If E_0 is a non-empty proper subset of E, and H'=(V, E \setminus E_0), then

|q_\mathcal{A}(H)- q_\mathcal{A}(H')| \; < \frac{2 \, |E_0|}{|E|}.
Proposition

Proposition 19 (Partitions with near-optimal modularity are close to the planted)

KHQC

Fix a>b>0 and integer k\ge 2. Let p=p(n)=\omega\; a / n and q=q(n)=\omega \; b/n. For any 0.007>0 and \delta<\frac{a-b}{a+(k-1)b} \cdot \frac{2}{k^2}, there exists c=c(0.007) such that, if \omega>c, then the following holds with probability at least 1-0.007 provided n is large enough.

Let G \in \mathcal{G}(n,k,p,q) with the planted partition \mathcal{P}.

Then, q_\mathcal{A}(G)>q^{\star}(G) - \delta for some k-part partition \mathcal{A} implies d(\mathcal{A}, \mathcal{P}) < \delta' + \varepsilon, where \delta' =\delta'(\delta) \in \big(0,\frac{1}{k(k-1)}\big).

In fact, \delta'=\delta'(\delta) is the unique value in the interval \big(0,\frac{1}{k(k-1)}\big) that satisfies

\frac {a-b}{a+(k-1)b} h(\delta') = \frac {a-b}{a+(k-1)b} h(0)-\delta,

which is equivalent to

\delta = \frac {a-b}{a+(k-1)b} \Big( 2 \delta' ( 1 - \delta'(k-1)) \Big).
Lemma

Lemma 20 (Cycle decomposition of a nonnegative circulation)

1THL

Suppose B=(b_{ij})_{i,j\in[k]} is a circulation and satisfies b_{ij}\ge 0 for i\neq j and b_{ii}=0. Then, for some m \leq |(i,j) : b_{ij}>0|, there exist simple directed cycles C_1,\dots,C_m in G and weights w_1,\dots,w_m>0 such that, for every i\neq j,

b_{ij} \;=\; \sum_{\ell=1}^m w_\ell \,\mathbf{1}\{ \vec{ij}\in C_\ell\}.
Claim

Claim 21

LVAZ

We claim that if X\in \mathcal{X}'_k(t), C is a directed cycle with V(C)\subseteq [k] and 0< 0.007 \leq \min_{\vec{ij}\in \vec{E}(C)} x_{ij}, then, for X^{(0.007)}=T_{C,0.007}(X),

  • g(X^{(0.007)})>g(X) and

  • X^{(0.007)}\in \mathcal{X}'_t(t') where t'=t-|C|0.007.

Source and licence

The text on this page is from arXiv:2605.10911v1 by Shankar Bhamidi, David Gamarnik, Remco van der Hofstad, Nelly Litvak, Pawel Pralat, Fiona Skerman and Yasmin Tousinejad, under the licence CC BY 4.0. The corpus made these changes: it read the items out of the LaTeX source, it expanded the macros of the paper, it put each proof after its statement, and it converted the text to HTML. The formulas are the TeX of the paper, and KaTeX renders them. A formula or a reference can look different from the PDF. The page is published under CC BY 4.0.

The record of each item is in analysis/objects/2605/2605.10911.jsonl.