Example 1
6DJBIn the scalar case with \eta\mu=1, the mapping reduces to Q(z)=\max\{z-1,0\}. Taking z_1=0 and z_2=2 gives
Q (\frac{z_1+z_2}{2} )=0,~
\frac{Q(z_1)+Q(z_2)}{2}=\frac{1}{2}.
An Alternating Minimization Algorithm for Federated Sparse Orthogonal Nonnegative Matrix Factorization
- arXiv
- 2610.21455v1 · PDF · 15 October 2026
- Categories
- math.OC
- Licence
- CC BY 4.0 The site shows the text.
Abstract
In this paper, we develop an alternating minimization algorithm for federated sparse orthogonal nonnegative matrix factorization (ONMF) with emphases on controlling client drift. To reduce communication frequency, clients perform multiple local updates between communication rounds, biasing updates toward their respective local objectives and giving rise to client drift, particularly under data heterogeneity. In federated sparse ONMF, global nonsmooth regularization further complicates drift control, as proximal mappings generally do not commute with averaging. To address these challenges, we develop an effective optimization algorithm that introduces variable splitting to decouple proximal operator evaluation from communication, allowing clients to handle joint sparsity regularization locally. In addition, the algorithm incorporates a correction term into the local gradients to adjust the update directions and mitigate client drift. We theoretically establish that the proposed algorithm achieves sublinear convergence to a bounded residual error under some mild conditions. Numerical experiments demonstrate favorable convergence behavior on a synthetic dataset and competitive clustering performance on real-world multi-view datasets.
Items 23
Lemma 1
WC6WSuppose that \varphi(\cdot) is continuously differentiable with an L-Lipschitz gradient on a convex region containing Z,Y,W, and let h(\cdot) be proper, closed, and convex. If Y=P_{\eta\mu}(Z-\eta D) for \eta>0 and W\in\text{dom}~h(\cdot), then
\begin{aligned}
\varphi(Y)+h(Y) &\leq \varphi(W)+h(W) +\langle\nabla\varphi(Z)-D,Y-W\rangle\\
&-\frac{1}{\eta}\langle Y-Z,Y-W\rangle +\frac{L}{2}\|Y-Z\|_F^2 +\frac{L}{2}\|W-Z\|_F^2.
\end{aligned}
Used by Proof
Proof of Lemma 1
Following the proximal optimality condition and the subgradient inequality, it derives
\begin{aligned}
h(Y)-h(W)\leq-\langle D,Y-W\rangle-\frac{1}{\eta}\langle Y-Z,Y-W\rangle.
\end{aligned}
The Lipschitz continuity of \nabla\varphi implies
\begin{aligned}
\varphi(Y)-\varphi(W) \leq\langle\nabla\varphi(Z),Y-W\rangle+\frac{L}{2}\|Y-Z\|_F^2 +\frac{L}{2}\|W-Z\|_F^2.
\end{aligned}
Adding the two inequalities proves the result.
Remark 1
ZNLOUnder the assumptions and penalty parameter conditions of Theorem 3.1 of [jiang2023exact], a global minimizer of the penalized problem can be used to recover a global minimizer of (?). In the convergence analysis, U_p^{s,k+1} denotes the recovered global minimizer.
Remark 2
I54IAt local iteration k, the proximal parameter is \eta=(k+1)\alpha. The pre-proximal variable accumulates gradient updates without incorporating the results of the proximal mappings. Thus, each post-proximal iterate is obtained by applying the proximal mapping to the initial point of the round adjusted by the accumulated gradients. As explained by Zhang et al. [zhang2026nonconvex], the proximal step size should match the cumulative step size of these gradient updates, which is (k+1)\alpha after k+1 local updates.
Assumption 1
BEFLThere exist constants L_{1,p}>0, p=1,\ldots,m, such that, for any feasible U_p and any V_1,V_2 generated by the algorithm,
\begin{aligned}
\|\nabla_{U_p}f_p(U_p,V_1)-\nabla_{U_p}f_p(U_p,V_2)\|_F\leq L_{1,p}\|V_1-V_2\|_F.
\end{aligned}
Assumption 2
HIDQThe proximal basis subproblem (?) and its penalty formulation satisfy the conditions of Theorem 3.1 in [jiang2023exact] at every local update.
Assumption 3
XM2FThere exists a constant c_h>0 such that \|(X-Y)/\tilde\eta\|_F\leq c_h for every proximal evaluation Y=\mathcal P_{\tilde\eta\mu}(X) generated by Algorithm ?.
Lemma 2
UOEGIf the parameter \tilde\eta satisfies 0<\tilde\eta\leq\frac{\eta_g}{\sqrt{20}L_2}, then, for every s\geq1, it holds that
\begin{aligned}
\sum_{k=0}^{\tau-1}\sum_{p=1}^{m}\|V_p^{s,k}-V^s\|_F^2 &\le 6\tau\|\Lambda^s-\overline{\Lambda}^s\|_F^2+24m\tau^3\alpha^2c_h^2\\
&+6m\tau^3\alpha^2\|\widetilde{\nabla}h(\widetilde V^{s+1})
+\nabla_V f(\{U_p^{s,0}\},V^s)\|_F^2\\
&+12\alpha^2\tau^4\sum_{p=1}^{m}\sum_{k=0}^{\tau-1}L_{3,p}^2\|\Delta U_p^{s,k}\|_F^2,
\end{aligned}
where \Lambda_{a}^s=\frac1m\sum_{p=1}^{m}\Lambda_p^s,~\Lambda^s=\text{col}~\{\Lambda_1^s,\cdots,\Lambda_m^s\},~\overline{\Lambda}^s=\text{col}~\{\Lambda_{a}^s,\cdots, \Lambda_{a}^s\},~\widetilde{\nabla}h(\widetilde V^{s+1})\in\partial h(\widetilde V^{s+1}),~ \Delta U_p^{s,k} =U_p^{s,k+1}-U_p^{s,k}, and \Lambda_p^s =\alpha\tau ( \nabla_V f_p(U_p^{s,0},V^s) +\frac{1}{m}\sum_{p=1}^{m}G_p^{s-1} -G_p^{s-1} ).
Lemma 3
IKPYFor every s=1,\cdots,T, it holds that
\begin{aligned}
&\frac{1}{m}\|\Lambda^{s+1}-\overline{\Lambda}^{s+1}\|_F^2- 2\alpha^2\tau^2L_2^2 \|V^{s+1}-V^s\|_F^2\\
&\leq\frac{4\alpha^2\tau L_2^2}{m}\sum_{p=1}^{m}\sum_{k=0}^{\tau-1}\|V_p^{s,k}-V^s\|_F^2+\frac{2\alpha^2\tau^2(\tau-1)}{m}\sum_{p=1}^{m}\sum_{k=0}^{\tau-1}L_{3,p}^2\|\Delta U_p^{s,k}\|_F^2.
\end{aligned}
Lemma 4
56E9Under the update rule U_p^{s+1,0}=U_p^{s,\tau}, for communication round s, it holds that
\begin{aligned}
&F(\{U_p^{s+1,0}\},V^{s+1})-F(\{U_p^{s,0}\},V^{s+1})\\
&\leq -\frac{\rho_1}{4mL_{3,a}^2} \sum_{p=1}^{m}\sum_{k=0}^{\tau-1}L_{3,p}^2\|\Delta U_p^{s,k}\|_F^2 + \frac{1}{m\rho_1} \sum_{p=1}^{m}\sum_{k=0}^{\tau-1} L_{1,p}^2 \|V_p^{s,k}-V^{s+1}\|_F^2,
\end{aligned}
where L_{3,a}=\max_{p}L_{3,p}.
Lemma 5
DYSRSuppose that Assumption 1-3 hold. Then, for communication round s, local iteration k, and client p, it has
\begin{aligned}
r_p^{s,k+1} \leq
2L_{1,p}^{2} \|V_p^{s,k}-V^s\|_F^2 + 2\rho_1^2 \|\Delta U_p^{s,k}\|_F^2,
\end{aligned}
where
\begin{aligned}
r_p^{s,k+1} &= \text{dist}^{2}~( 0, \nabla_{U_p}f_p(U_p^{s,k+1},V^s) + N_{\mathcal O_p}(U_p^{s,k+1}) ),\\
\mathcal O_p &= \{ U\in\mathbb R^{d_p\times r}: U\geq 0,\ U^\top U=I \}
\end{aligned}
is the feasible set for the local matrix, and N_{\mathcal O_p}(U) denotes the limiting normal cone to \mathcal O_p at the matrix U.
Theorem 1
QDV1Under Assumptions 1-3, let L_{1,a}=\max_{p}L_{1,p}. If the step sizes and the proximal parameter satisfy
\begin{aligned}
& 0<\tilde{\eta}\leq\frac{1}{10L_2},~\eta_g\geq\frac{3}{2},\\
&\rho_1> \max\{ 72\tau\tilde{\eta}L_{1,a}^2, 8L_{3,a}^2( \tilde{\eta} ( \frac{\tau+1}{2} + 12L_2^2\alpha^2\tau^3 ) \\
&+ \frac{ 2\alpha^2\tau^2(\tau-1) + 48\alpha^4\tau^5L_2^2 }{\tilde{\eta}}), 8\sqrt{3} L_{3,a}L_{1,a}\alpha\tau^2 \}.
\end{aligned}
Then, there exist constants c_1,c_2,c_3>0 such that, for every T\geq1,
\frac{1}{T}\sum_{s=1}^{T} (c_1R_V^s +c_2C_s +c_3 A_s ) \leq \frac{\Phi^1}{T}
+c_4 c_h^2.
Consequently,
\frac{1}{T}\sum_{s=1}^{T}R_V^s \leq \frac{\Phi^1}{c_1T}+ \frac{c_4}{c_1}c_h^2,
where c_1=(\tilde{\eta} (\frac12-L_2\tilde{\eta} -6L_2^2\tau^2\alpha^2 ) - \frac{24\alpha^4\tau^4L_2^2}{\tilde{\eta}} - \frac{12\tau^3\alpha^2L_{1,a}^2}{\rho_1}),~ c_4 = 24\tilde{\eta}L_2^2\tau^2\alpha^2 + \frac{96\alpha^4\tau^4L_2^2}{\tilde{\eta}} + \frac{48\tau^3\alpha^2L_{1,a}^2}{\rho_1}, and under (1), it holds that c_1>0.
Proof of Theorem 1
The proof proceeds in four steps.
Step 1: It reformulates the global update for V^{s+1}.
Algorithm ? gives V^s=\mathcal P_{\tilde\eta\mu}(\overline V^s). The definition of f(\{U_p\},V) yields
\nabla_V f(\{ U^{s,0}\},V^s)
= \frac{1}{m}\sum_{p=1}^{m} \nabla_V f_p(U_p^{s,0},V^s).
and introduce the averaged local gradient direction
G^s=\frac{1}{m}\sum_{p=1}^{m}G_p^s.
Summing the local updates in Algorithm ? and using \widehat V_p^{s,0}=V^s yields
\begin{aligned}
\widehat V_p^{s,\tau}=V^s-\alpha\sum_{k=0}^{\tau-1} (\nabla_V f_p (U_p^{s,k+1},V_p^{s,k}) +C_p^s).
\end{aligned}
Substituting (?) into the global update gives
\begin{aligned}
\overline {V}^{s+1}&= V^s-\frac{\eta_g\alpha}{m}\sum_{p=1}^{m}\sum_{k=0}^{\tau-1}
(\nabla_V f_p(U_p^{s,k+1},V_p^{s,k})+C_p^s) \\
&= V^s -\eta_g\alpha\tau
(\frac{1}{m\tau}\sum_{p=1}^{m}\sum_{k=0}^{\tau-1}\nabla_V f_p( U_p^{s,k+1},V_p^{s,k})+\frac{1}{m}\sum_{p=1}^{m}C_p^s).
\end{aligned}
Since the client drift correction terms satisfy \frac{1}{m}\sum_{p=1}^{m}C_p^s =0, and \tilde{\eta}=\eta_g\alpha\tau, the global pre-proximal variable can be equivalently written as \overline {V}^{s+1}= V^s-\tilde{\eta}G^s. Consequently, by applying the global proximal operator, the global post-proximal iterate satisfies
V^{s+1} =\mathcal P_{\tilde{\eta}\mu}( V^s-\tilde{\eta}G^s).
Step 2: It bounds the change in F(\{U_p^{s,0}\},V^s) over one communication round.
Applying Lemma 1 to the virtual iterate the virtual iterate \widetilde{V}^{s+1}, using Y=\widetilde{V}^{s+1},Z=W=V^s, and D=\nabla_V f(\{ U^{s,0}\},V^s) yields
\begin{aligned}
F(\{ U^{s,0}\},\widetilde V^{s+1}) &\leq F(\{ U^{s,0}\},{V}^{s}) +(\frac{L_2}{2}- \frac{1}{\tilde{\eta}})\| \widetilde{V}^{s+1} - {V}^{s} \|_{F}^2.
\end{aligned}
For the actual iterate V^{s+1}, using Y={V}^{s+1},X={V}^{s} ,W=\widetilde{V}^{s+1} and D=G^{s} in Lemma 1 yields
\begin{aligned}
&\quad F(\{ U^{s,0}\},V^{s+1})\\
&\leq F(\{ U^{s,0}\},\widetilde{V}^{s+1}) - \frac{1}{\tilde{\eta}} \langle V^{s+1}-V^s, V^{s+1}-\widetilde V^{s+1} \rangle + \frac{L_2}{2} \| {V}^{s+1} - {V}^{s} \|_{F}^2\\
&+ \langle \nabla_V f(\{ U^{s,0}\},{V}^{s}) - G^{s}, {V}^{s+1} - \widetilde{V}^{s+1} \rangle + \frac{L_2}{2} \| \widetilde{V}^{s+1} - {V}^{s}\|_{F}^2\\
&\leq F(\{U^{s,0}\},\widetilde V^{s+1}) + \langle \nabla_V f(\{U^{s,0}\},V^s)-G^s,
V^{s+1}-\widetilde V^{s+1} \rangle \\
& + (\frac{L_2}{2} - \frac{1}{2\tilde{\eta}}) \|V^{s+1}-V^s\|_F^2 + ( \frac{L_2}{2} + \frac{1}{2\tilde{\eta}} ) \|\widetilde V^{s+1}-V^s\|_F^2 - \frac{1}{2\tilde{\eta}} \|V^{s+1}-\widetilde V^{s+1}\|_F^2.
\end{aligned}
To eliminate the virtual function value F(\{ U^{s,0}\},\widetilde{V}^{s+1}), the above inequalities can be added to obtain
\begin{aligned}
&\quad F(\{ U^{s,0}\},V^{s+1})\\
&\leq F(\{ U^{s,0}\},{V}^{s})+ (\frac{L_2}{2}- \frac{1}{2\tilde{\eta}})\| V^{s+1}-V^s \|_{F}^2 \\
&+ \underbrace{\langle \nabla_V f(\{ U^{s,0}\},V^s)-G^s,
V^{s+1}-\widetilde V^{s+1} \rangle}_{(\text{I})}\\
&+ (L_2-\frac{1}{2\tilde{\eta}}) \| \widetilde{V}^{s+1} - V^s \|_{F}^2- \underbrace{\frac{1}{2\tilde{\eta}}\| {V}^{s+1} - \widetilde{V}^{s+1} \|_{F}^2}_{(\text{II})}.
\end{aligned}
By Young’s inequality, (\text{I})-(\text{II}) \leq\|\nabla_V f( \{U^{s,0}\},V^s)-G^s\|_{F}^{2}. Using the definitions of \nabla_V f(\{U_p^{s,0}\} ,V^s) and G^s, it has
\begin{aligned}
\nabla_V f( \{U^{s,0}\},V^s)-G^s = \frac{1}{m\tau} \sum_{p=1}^{m}
\sum_{k=0}^{\tau-1}( \nabla_V f_p(U_p^{s,0},V^s) - \nabla_V f_p(U_p^{s,k+1},V_p^{s,k})).
\end{aligned}
Hence, by Jensen’s inequality,
\begin{aligned}
\|\nabla_V f( \{U^{s,0}\},V^s)-G^s\|_F^2 \leq \frac{1}{m\tau} \sum_{p=1}^{m} \sum_{k=0}^{\tau-1} \| \nabla_V f_p(U_p^{s,0},V^s) - \nabla_V f_p(U_p^{s,k+1},V_p^{s,k}) \|_F^2 .
\end{aligned}
Moreover, by the Lipschitz continuity properties established above,
\begin{aligned}
& \| \nabla_V f_p(U_p^{s,0},V^s) - \nabla_V f_p(U_p^{s,k+1},V_p^{s,k}) \|_F^2 \\
&\leq 2L_{3,p}^2 \|U_p^{s,k+1}-U_p^{s,0}\|_F^2 + 2L_2^2 \|V_p^{s,k}-V^s\|_F^2 .
\end{aligned}
Combining the above inequalities gives
\begin{aligned}
(\text{I})-(\text{II})
&\leq \frac{\tilde\eta L_2^2}{m\tau} \sum_{p=1}^{m} \sum_{k=0}^{\tau-1} \|V_p^{s,k}-V^s\|_F^2 + \frac{\tilde\eta}{m\tau} \sum_{p=1}^{m} \sum_{k=0}^{\tau-1} L_{3,p}^2 \|U_p^{s,k+1}-U_p^{s,0}\|_F^2 .
\end{aligned}
Substituting this equation into (?), it has
\begin{aligned}
F(\{ U^{s,0}\},V^{s+1}) &\leq F(\{ U^{s,0}\},{V}^{s})+
(\frac{L_2}{2}- \frac{1}{2\tilde{\eta}})\|V^{s+1}-V^s\|_F^2\\
&+\frac{\tilde{\eta}L_2^2}{m\tau}\sum_{p=1}^{m}\sum_{k=0}^{\tau-1}\|V_p^{s,k}-V^s\|_F^2+ (L_2-\frac{1}{2\tilde{\eta}})\|\widetilde V^{s+1}-V^s\|_F^2 \\
&+\frac{\tilde{\eta}}{m\tau}\sum_{p=1}^{m}\sum_{k=0}^{\tau-1}
L_{3,p}^2\|U_p^{s,k+1}-U_p^{s,0}\|_F^2 .
\end{aligned}
By Lemma 4, the change caused by the local U_p updates between two communication rounds is bounded by
\begin{aligned}
&\quad F(\{U_p^{s+1,0}\},V^{s+1}) - F(\{U_p^{s,0}\},V^{s+1}) \\
&\leq -\frac{\rho_1}{4mL_{3,a}^2} \sum_{p=1}^{m}\sum_{k=0}^{\tau-1} L_{3,p}^2 \|\Delta U_p^{s,k}\|_F^2 + \frac{1}{m\rho_1} \sum_{p=1}^{m}\sum_{k=0}^{\tau-1} L_{1,p}^2 \|V_p^{s,k}-V^{s+1}\|_F^2.
\end{aligned}
The last term in (?) can be controlled as follows. Since V_p^{s,k}-V^{s+1} = (V_p^{s,k}-V^s)+(V^s-V^{s+1}), it has
\begin{aligned}
&\quad\frac{1}{m\rho_1} \sum_{p=1}^{m}\sum_{k=0}^{\tau-1} L_{1,p}^2 \|V_p^{s,k}-V^{s+1}\|_F^2\\
&\leq \frac{2L_{1,a}^2}{m\rho_1} \sum_{p=1}^{m}\sum_{k=0}^{\tau-1} \|V_p^{s,k}-V^s\|_F^2 + \frac{2\tau L_{1,a}^2}{\rho_1} \|V^{s+1}-V^s\|_F^2 .
\end{aligned}
Applying Lemma 2 to the first term on the right hand side of (?) gives
\begin{aligned}
&\quad\frac{2L_{1,a}^2}{m\rho_1} \sum_{p=1}^{m}\sum_{k=0}^{\tau-1} \|V_p^{s,k}-V^s\|_F^2
\\
&\leq \frac{12\tau L_{1,a}^2}{m\rho_1} \|\Lambda^s-\overline{\Lambda}^s\|_F^2
+ \frac{12\tau^3\alpha^2L_{1,a}^2}{\rho_1} \| \widetilde{\nabla}h(\widetilde V^{s+1}) + \nabla_V f(\{U_p^{s,0}\},V^s) \|_F^2 \\
&+ \frac{48\tau^3\alpha^2L_{1,a}^2}{\rho_1} c_h^2 + \frac{24\alpha^2\tau^4L_{1,a}^2}{m\rho_1} \sum_{p=1}^{m}\sum_{k=0}^{\tau-1} L_{3,p}^2 \|\Delta U_p^{s,k}\|_F^2 .
\end{aligned}
Step 3: It establishes a recursion for the auxiliary function \Phi^s.
Next, multiplying Lemma 3 by 1/\tilde{\eta} and using Lemma 2, it obtains
\begin{aligned}
\frac{1}{m\tilde{\eta}} \|\Lambda^{s+1}-\overline{\Lambda}^{s+1}\|_F^2 &\leq \frac{2\alpha^2\tau^2L_2^2}{\tilde{\eta}} \|V^{s+1}-V^s\|_F^2 + \frac{24\alpha^2\tau^2L_2^2}{m\tilde{\eta}}
\|\Lambda^s-\overline{\Lambda}^s\|_F^2 \\
&+ \frac{96\alpha^4\tau^4L_2^2}{\tilde{\eta}}c_h^2 + \frac{24\alpha^4\tau^4L_2^2}{\tilde{\eta}}\|
\widetilde{\nabla}h(\widetilde V^{s+1}) + \nabla_V f(\{U_p^{s,0}\},V^s) \|_F^2 \\
&\quad+ \frac{ 2\alpha^2\tau^2(\tau-1) + 48\alpha^4\tau^5L_2^2
}{ m\tilde{\eta} } \sum_{p=1}^{m}\sum_{k=0}^{\tau-1} L_{3,p}^2 \|\Delta U_p^{s,k}\|_F^2 .
\end{aligned}
Adding (?), (?), (?), (?), and (?), and subtracting F^\star from both sides, it gets
\begin{aligned}
\Phi^{s+1} &\leq \Phi^s -c_1 \| \widetilde{\nabla}h(\widetilde V^{s+1}) + \nabla_V f(\{U_p^{s,0}\},V^s) \|_F^2 - \frac{c_2}{m}\|\Lambda^s-\overline{\Lambda}^s\|_F^2\\
& -\frac{c_3}{m} \sum_{p=1}^{m}\sum_{k=0}^{\tau-1} L_{3,p}^2 \|\Delta U_p^{s,k}\|_F^2 + c_4c_h^2-c_5 \|V^{s+1}-V^s\|_F^2,
\end{aligned}
where c_1 and c_4 are defined in Theorem 1, and c_2=\frac{1}{\tilde\eta} -6\tilde\eta L_2^2 -\frac{24\alpha^2\tau^2L_2^2}{\tilde\eta} -\frac{12\tau L_{1,a}^2}{\rho_1}, c_3=\frac{\rho_1}{4L_{3,a}^2} -\tilde\eta ( \frac{\tau+1}{2}+12L_2^2\alpha^2\tau^3 ) -\frac{2\alpha^2\tau^2(\tau-1) +48\alpha^4\tau^5L_2^2}{\tilde\eta} -\frac{24\alpha^2\tau^4L_{1,a}^2}{\rho_1}, c_5= \frac{1-4\alpha^2\tau^2L_2^2}{2\tilde{\eta}} - \frac{L_2}{2} - \frac{2\tau L_{1,a}^2}{\rho_1}. The parameter conditions imply \alpha\tau=\frac{\tilde\eta}{\eta_g} \leq\frac{2\tilde\eta}{3},~ L_2\tilde\eta\leq\frac{1}{10},~ \frac{2\tau\tilde\eta L_{1,a}^2}{\rho_1}<\frac{1}{36}. Consequently,
\begin{aligned}
\tilde\eta c_5
&=\frac12-2\alpha^2\tau^2L_2^2
-\frac{L_2\tilde\eta}{2}
-\frac{2\tau\tilde\eta L_{1,a}^2}{\rho_1}\\
&>\frac12-\frac{2}{225}-\frac{1}{20}-\frac{1}{36}
=\frac{31}{75}>0.
\end{aligned}
Thus, c_5>0, and omitting the nonpositive term -c_5\|V^{s+1}-V^s\|_F^2 yields
\begin{aligned}
\Phi^{s+1} &\leq \Phi^s -c_1 \| \widetilde{\nabla}h(\widetilde V^{s+1}) + \nabla_V f(\{U_p^{s,0}\},V^s) \|_F^2 - \frac{c_2}{m}\|\Lambda^s-\overline{\Lambda}^s\|_F^2\\
& -\frac{c_3}{m} \sum_{p=1}^{m}\sum_{k=0}^{\tau-1} L_{3,p}^2 \|\Delta U_p^{s,k}\|_F^2 + c_4c_h^2.
\end{aligned}
Following the definition of C_s,A_s, and R_V^s in (?) and (?), it has
\begin{aligned}
\Phi^{s+1}-\Phi^s \leq & -c_1R_V^s -c_2C_s -c_3 A_s +c_4 c_h^2.
\end{aligned}
Step 4: It bounds the average stationarity residual \frac{1}{T}\sum_{s=1}^{T}R_V^s.
Summing the inequality (?) over s=1,\cdots,T yields
\begin{aligned}
&\Phi^1-\Phi^{T+1} + T c_4c_h^2\geq c_3 A_s +c_1R_V^s + c_2C_s.
\end{aligned}
Since \Phi^{T+1}\geq0, it has
\begin{aligned}
&\Phi^1 + T c_4c_h^2\geq \sum_{s=1}^{T}(c_1R_V^s + c_2C_s +c_3 A_s) .
\end{aligned}
Since C_s,A_s\geq0 and c_1,c_2,c_3>0, it follows that
\begin{aligned}
&\frac{1}{T} \sum_{s=1}^{T} R_V^s \leq \frac{\Phi^1}{c_1 T} + \frac{c_4}{c_1}c_h^2 .
\end{aligned}
The proof is complete.
Theorem 2
F67GSuppose that Assumption 1-3 hold and the parameter conditions of Theorem 1 are satisfied. Define
R_U^s = \frac{1}{m}\sum_{p=1}^{m} r_p^{s,\tau}.
Then,
\frac{1}{T}\sum_{s=1}^{T}R_U^s
\leq \frac{\kappa (\Phi^1-\underline{\Phi})}{T}
+ (\kappa c_4+b_1 )c_h^2,
where \underline{\Phi}=\inf_s\Phi^s, \kappa=\max\{b_2/c_1,b_3/c_2,b_4/c_3\}, and b_1=48\tau^3\alpha^2L_{1,a}^2. The constants b_2,b_3,b_4 are defined in the proof.
Proof of Theorem 2
Let L_{3,i}=\min_{p }L_{3,p}>0. Applying Lemma 5 to k=\tau-1, and enlarging the resulting terms by summing over all local iterations, it obtains
R_U^s \leq 2L_{1,a}^2\frac{1}{m}\sum_{p=1}^{m} \sum_{k=0}^{\tau-1} \|V_p^{s,k}-V^s \|_F^2 + \frac{2\rho_1^2}{L_{3,i}^2}A_s.
Moreover, Lemma 2 gives
\begin{aligned}
&\frac{1}{m}\sum_{p=1}^{m} \sum_{k=0}^{\tau-1} \|V_p^{s,k}-V^s \|_F^2 \leq 6\tau C_s +24\tau^3\alpha^2c_h^2
+6\tau^3\alpha^2R_V^s
+12\alpha^2\tau^4A_s.
\end{aligned}
Substituting (?) into (Proof) yields
R_U^s \leq b_2R_V^s +b_3C_s +b_4A_s +b_1c_h^2,
where
\begin{aligned}
b_2 =12\tau^3\alpha^2L_{1,a}^2,~b_3=12\tau L_{1,a}^2,b_4= 24\alpha^2\tau^4L_{1,a}^2 +\frac{2\rho_1^2}{L_{3,i}^2}.
\end{aligned}
Let
\kappa= \max \{ \frac{b_2}{c_1}, \frac{b_3}{c_2}, \frac{b_4}{c_3} \}.
It follows from (?) that
R_U^s \leq \kappa ( c_1R_V^s+c_2C_s+c_3A_s ) +b_1c_h^2.
Summing (?) from s=1 to s=T and using \Phi^{T+1}\geq\underline{\Phi} give
\sum_{s=1}^{T} (
c_1R_V^s+c_2C_s+c_3A_s )
\leq \Phi^1-\underline{\Phi}+T c_4c_h^2,
where \underline{\Phi}=\inf_s\Phi^s. Therefore,
\sum_{s=1}^{T}R_U^s \leq
\kappa (\Phi^1-\underline{\Phi}) +T (\kappa c_4+b_1 )c_h^2.
Dividing both sides by T proves (2).
Theorem 3
6T3FSuppose that the conditions of Theorems 1 and 2 hold. Define the overall stationarity residual as
R^s = R_V^s + \frac{1}{m}\sum_{p=1}^{m} R_{U,p}^{s,\tau}.
Then,
\frac{1}{T}\sum_{s=1}^{T}R^s
\leq \frac{( \frac{1}{c_1}+\kappa )(\Phi^1-\underline{\Phi})}{T} + (\frac{c_4}{c_1}+\kappa c_4+b_1)c_h^2.
Proof of Theorem 3
Remark 3
2L2DTheorem 3 bounds the average combined residual by a term that decays as O(1/T) for fixed parameters and a remaining error term. The latter depends on the step sizes, the number of local updates, and the proximal parameter through c_4, c_1, \kappa, and b_1. To examine the controllability of the error term, fix \tau, \eta_g, and \rho_1, and set \alpha=\tilde\eta/(\eta_g\tau). For sufficiently small \tilde\eta satisfying the parameter conditions, the coefficients satisfy c_1=O(\tilde\eta), c_4=O(\tilde\eta^2), \kappa=O(1), and b_1=O(\tilde\eta^2). Provided that the relevant Lipschitz constants and the initial auxiliary function gap remain uniformly bounded, the estimate becomes
\frac1T\sum_{s=1}^{T}R^s
\leq O(\frac1{T\tilde\eta})+O(\tilde\eta)c_h^2.
Thus, decreasing the effective global step size reduces the remaining error bound, and increasing the number of rounds reduces the initial-error contribution. The average residual bound tends to zero when \tilde\eta\to0 and T\tilde\eta\to\infty.
Proof
7X9SIf \tau=1, then V_{p}^{s,k}=V^{s}.
If \tau\ge 2,
\begin{aligned}
\widehat{V}_p^{s,k+1} &= \widehat{V}_p^{s,k} - \alpha ( \nabla_{V_p} f_p(U_p^{s,k+1},V_p^{s,k}) + C_p^{s} )\\
&=\widehat{V}_p^{s,0}- \alpha \sum_{l=0}^{k} (\nabla_{V_p} f_p(U_p^{s,l+1},V_p^{s,l}) + C_p^{s}).
\end{aligned}
Since \widehat{V}_p^{s,0}=\mathcal{P}_{\tilde{\eta}\mu} (\overline {V}^{s})=V^{s} and {V}_p^{s,k+1}=\mathcal{P}_{(k+1)\alpha\mu}(\widehat{V}_p^{s,k+1}), applying (Proof) it has
\begin{aligned}
\|V_{p}^{s,k+1}-V^{s}\|_{F}^{2} &=\|\mathcal{P}_{(k+1)\alpha\mu}(\widehat{V}_p^{s,k+1})-\mathcal{P}_{\tilde{\eta}\mu}(\overline {V}^{s})\|_{F}^{2}\\
&=\|\mathcal{P}_{(k+1)\alpha\mu}(V^s- \alpha \sum_{l=0}^{k} (\nabla_V f_p(U_p^{s,l+1},V_p^{s,l})+ C_p^{s}))-V^s\|_{F}^{2}.
\end{aligned}
Define H_p^s=\nabla_V f_p(U_p^{s,0},V^s),H^s=\frac{1}{m}\sum_{p=1}^{m}H_p^s. The right side of (?) can be interpreted as an inexact projected gradient descent, executed from the starting point V^{s} using the inexact gradient - \alpha \sum_{l=0}^{k} (\nabla_V f_p(U_p^{s,l+1},V_p^{s,l})+ C_p^{s}). This inspires us to bound (?) using the deviation from the exact projected gradient descent step with the same initial point \widetilde{V}_{pgd}=\mathcal{P}_{(k+1){\eta}}(V^{s}-(k+1)\alpha H^s). In this way,
\begin{aligned}
&\quad\|V_{p}^{s,k+1}-V^s\|_{F}^{2}\\
&=\|\mathcal{P}_{(k+1)\alpha\mu}(V^s- \alpha \sum_{l=0}^{k} (\nabla_{V_p} f_p(U_p^{s,l+1},V_p^{s,l}) +C_p^{s}))-\widetilde{V}_{pgd}+\widetilde{V}_{pgd}-{V}^{s}\|_{F}^{2}\\
&\le \underbrace{2\|\mathcal{P}_{(k+1)\alpha\mu}(V^{s}- \alpha \sum_{l=0}^{k} (\nabla_{V_p} f_p(U_p^{s,l+1},V_p^{s,l})+ C_p^{s}))-\widetilde{V}_{pgd}\|_{F}}_{(\text{III})} +\underbrace{2\|\widetilde{V}_{pgd}-{V}^{s}\|_{F}^{2}}_{(\text{IV})}.
\end{aligned}
Since
\begin{aligned}\widetilde{V}_{pgd}= \arg\min_{Z}\frac{1}{2}\|V^s-(k+1)\alpha H^s-Z\|_{F}^{2}+(k+1)\alpha h(Z),\end{aligned}
so \widetilde{V}_{pgd} satisfies the optimality condition
\begin{aligned}(k+1)\alpha(\tilde{\nabla}h(\widetilde{V}_{pgd})+H^s)+\widetilde{V}_{pgd}-\mathcal{P}_{\tilde{\eta}\mu} (\overline {V}^{s})=0,\end{aligned}
for some \tilde{\nabla}h(\widetilde{V}_{pgd})\in\partial h(\widetilde{V}_{pgd}). This implies that
\begin{aligned}
\widetilde{V}_{pgd}-V^s= -(k+1)\alpha(\tilde{\nabla}h(\widetilde{V}_{pgd})+H^s).
\end{aligned}
Note that the server step size is \tilde{\eta} and not (k+1)\alpha. To correct for this, it uses the iterate \widetilde{V}^{s+1} defined in (?), which is the same as \widetilde{V}_{pgd} except that the step size (k+1)\alpha is replaced by \tilde{\eta}. Following similar steps as above, the optimality condition for \widetilde{V}^{s+1} implies that
\begin{aligned}
\widetilde{V}^{s+1}-V^s= -\tilde{\eta}(\tilde{\nabla}h(\widetilde{V}^{s+1})+H^s),
\end{aligned}
for some \tilde{\nabla}h(\widetilde{V}^{s+1})\in\partial h(\widetilde{V}^{s+1}). For (IV), according to Assumption 3, it has
\begin{aligned}
(\text{IV})&=2\|(k+1)\alpha(\tilde{\nabla}h(\widetilde{V}_{pgd})+H^s)\|_{F}^{2}\\
&=2\|(k+1)\alpha(\tilde{\nabla}h(\widetilde{V}_{pgd})+H^s-\tilde{\nabla}h(\widetilde{V}^{s+1})+H^s) +(k+1)\alpha(\tilde{\nabla}h(\widetilde{V}^{s+1})+H^s)\|_{F}^{2}\\
&=2\|(k+1)\alpha(\tilde{\nabla}h(\widetilde{V}_{pgd})-\tilde{\nabla}h(\widetilde{V}^{s+1}))+(k+1)\alpha(\tilde{\nabla}h(\widetilde{V}^{s+1})+H^s)\|_{F}^{2}\\
&\le 4\tau^{2}\alpha^{2} 4B_{g}^{2}+4\tau^{2}\alpha^{2}\|\tilde{\nabla}h(\widetilde{V}^{s+1})+H^s\|_{F}^{2}.
\end{aligned}
By the definition of \widetilde{V}_{pgd} and the nonexpansiveness of the proximal operator, it has
\begin{aligned}
(\text{III})&=2\|\mathcal{P}_{(k+1)\alpha\mu}(V^s- \alpha \sum_{l=0}^{k} (\nabla_{V_p} f_p(U_p^{s,l+1},V_p^{s,l})+ C_p^{s}))-\mathcal{P}_{(k+1){\alpha}}(V^s-(k+1)\alpha H^s)\|_{F}^{2}\\
&\le 2\|\alpha \sum_{l=0}^{k} (\nabla_{V_p} f_p(U_p^{s,l+1},V_p^{s,l})+ C_p^{s})-(k+1)\alpha H^s\|_{F}^{2}\\
&\le 2\alpha^2\| \sum_{l=0}^{k} (\nabla_{V_p} f_p(U_p^{s,l+1},V_p^{s,l})+ C_p^{s}-H^s)\|_{F}^{2}.\\
\end{aligned}
According to the definition of \Lambda_p^s and \Lambda_{a}^s, the correction term satisfies
H_p^s+C_p^s-H^s= \frac{1}{\alpha\tau}
(\Lambda_p^s-\Lambda_{a}^s).
It has
\begin{aligned}
(\text{III})&\le 2\alpha^2\| \sum_{l=0}^{k} (\nabla_{V_p} f_p(U_p^{s,l+1},V_p^{s,l})-h_{p}^s+h_{p}^s+ C_p^{s}-H^s)\|_{F}^{2}\\
&=2\alpha^2\| \sum_{l=0}^{k} (\nabla_{V_p} f_p(U_p^{s,l+1},V_p^{s,l})-h_{p}^s)+(k+1)(h_{p}^s+ C_p^{s}-H^s)\|_{F}^{2}\\
&\le 4\alpha^2\| \sum_{l=0}^{k} (\nabla_{V_p} f_p(U_p^{s,l+1},V_p^{s,l})-h_{p}^s)\|_{F}^{2}+4 \alpha^2\frac{(k+1)^{2}}{\alpha^2\tau^2}
\|\Lambda_p^s-\Lambda_{a}^s\|_{F}^2.
\end{aligned}
The first term contains both the V drift and the U drift. Specifically,
\begin{aligned}
&\quad\nabla_{V_p} f_p(U_p^{s,l+1},V_p^{s,l})-H_p^s\\
&=\underbrace{\nabla_{V_p} f_p(U_p^{s,l+1},V_p^{s,l})-\nabla_V f_p(U_p^{s,l+1},V^s)}_{\text{$V$ drift}}\\
&+\underbrace{\nabla_V f_p(U_p^{s,l+1},V^s)-\nabla_V f_p(U_p^{s,0},V^s)}_{\text{$U$ drift}}.
\end{aligned}
Using the Lipschitz continuity gives
\begin{aligned}
\|\nabla_{V_p} f_p(U_p^{s,l+1},V_p^{s,l})-H_p^s\|_F^2\leq 2L_2^2\|V_p^{s,l}-V^s\|_F^2+2L_{3,p}^2\|U_p^{s,l+1}-U_p^{s,0}\|_F^2.
\end{aligned}
Using \|\sum_{l=0}^{k}A_l\|_F^2\leq(k+1)\sum_{l=0}^{k}\|A_l\|_F^2, it obtains
\begin{aligned}
(\text{III})\leq &8(k+1)\alpha^2L_2^2\sum_{l=0}^{k}\|V_p^{s,l}-V^s\|_F^2\\
&+8(k+1)\alpha^2L_{3,p}^2\sum_{l=0}^{k}\|U_p^{s,l+1}-U_p^{s,0}\|_F^2 +4(\frac{k+1}{\tau})^2\|\Lambda_p^s-\Lambda_{a}^s\|_F^2.
\end{aligned}
Now, plugging (III) and (IV) into (?) yields
\begin{aligned}
\|V_{p}^{s,k+1}-V^s\|_{F}^{2}
&\le 8(k+1)\alpha^2L_2^2\sum_{l=0}^{k}\|V_p^{s,l}-V^s\|_F^2+ 4\|\Lambda_p^s-\Lambda_{a}^s\|_F^2+16\tau^2\alpha^2c_h^2 \\
&+ 4\tau^2\alpha^2\|\widetilde\nabla h(\widetilde V^{s+1})+H^s\|_F^2 +8(k+1)\alpha^2L_{3,p}^2\sum_{l=0}^{k}\|U_p^{s,l+1}-U_p^{s,0}\|_F^2.
\end{aligned}
To get a recursion for the drift error from (?), let
\begin{aligned}B^{s}&=4\|\Lambda_p^s-\Lambda_{a}^s\|_F^2+16\tau^2\alpha^2c_h^2+
4\tau^2\alpha^2\|\widetilde\nabla g(\widetilde V^{s+1})+H^s\|_F^2,\\
~E_{U}^{k}&=8(k+1)\alpha^2L_{3,p}^2
\sum_{l=0}^{k}\|U_p^{s,l+1}-U_p^{s,0}\|_F^2,\end{aligned}
and define \mathcal{S}_{p,k}^{s}=\sum_{l=0}^{k}\|V_p^{s,l}-V^s\|_{F}^{2}.
\begin{aligned}
\mathcal{S}_{p,k+1}^{s}-\mathcal{S}_{p,k}^{s}\le 8\tau\alpha^{2}L_{V}^{2}\mathcal{S}_{p,k}^{s}+B^{s}+E_{U}^{k}.
\end{aligned}
Under the step-size condition \alpha\tau L_2\leq\frac{1}{\sqrt{20}}, it has 8\tau\alpha^2L_2^2\leq\frac{2}{5\tau}. Consequently, (1+8\tau\alpha^2L_2^2)^\tau\le\exp (\frac{2}{5} )\approx 1.492, it concludes that
\begin{aligned}
\mathcal S_{p,\tau-1}^s\leq1.492\tau A_p^s+1.492\sum_{k=0}^{\tau-2}E_{U}^{k}.
\end{aligned}
Summing (?) over all the client, it has
\begin{aligned}
\sum_{k=0}^{\tau-1}\sum_{p=1}^{m}\|V_p^{s,k}-V^s\|_F^2
&\leq 6\tau \|\Lambda^s-\overline{\Lambda}^s\|_F^2 + 24m\tau^3\alpha^2c_h^2 \\
&+ 6m\tau^3\alpha^2 \| \widetilde{\nabla}h(\widetilde V^{s+1}) + \nabla_Vf(\{U_p^{s,0}\},V^s) \|_F^2 \\
&+ 12\alpha^2\tau^4 \sum_{p=1}^{m}\sum_{k=0}^{\tau-1} L_{3,p}^2 \|\Delta U_p^{s,k}\|_F^2.
\end{aligned}
Uses Assumption 3
Proof
YJQTBy the definition of \Lambda_{p}^{s+1} and \Lambda_{a}^{s+1}, it has
\begin{aligned}
\frac{1}{m}\|\Lambda^{s+1}-\overline{\Lambda}^{s+1}\|_{F}^{2}\le\frac{\alpha^2\tau}{m}\sum_{p=1}^{m}\sum_{k=0}^{\tau-1}\|\nabla_V f_p(U_p^{s+1,0},V^{s+1})-\nabla_{V_p} f_p(U_p^{s,k+1},V_p^{s,k})\|_F^2,
\end{aligned}
where the inequality follows from the fact that \sum_{p=1}^{m}\|Y_p-\frac{1}{m}\sum_{q=1}^{m}Y_q\|_F^2\leq\sum_{p=1}^{m}\|Y_p\|_F^2 and \|\sum_{k=0}^{\tau-1}Y_k\|_F^2\leq\tau\sum_{k=0}^{\tau-1}\|Y_k\|_F^2 and
\begin{aligned}
\Lambda_p^{s+1}-\Lambda_{a}^{s+1}
&=\alpha\sum_{k=0}^{\tau-1}(\nabla_V f_p(U_p^{s+1,0},V^{s+1})-
\nabla_{V_p} f_p(U_p^{s,k+1},V_p^{s,k})\\
&-\frac{1}{m}\sum_{p=1}^{m}(\nabla_V f_p(U_p^{s+1,0},V^{s+1})-\nabla_{V_p} f_p(U_p^{s,k+1},V_p^{s,k}))).
\end{aligned}
For every client p and local iteration k, adding and subtracting \nabla_V f_p(U_p^{s+1,0},V^s) and \nabla_V f_p(U_p^{s,k+1},V^s) gives
\begin{aligned}
&\quad \nabla_V f_p(U_p^{s+1,0},V^{s+1})-\nabla_{V_p} f_p(U_p^{s,k+1},V_p^{s,k})\\
&=\nabla_V f_p(U_p^{s+1,0},V^{s+1})-\nabla_V f_p(U_p^{s+1,0},V^s)+\nabla_V f_p(U_p^{s+1,0},V^s)\\
&-\nabla_V f_p(U_p^{s,k+1},V^s)+\nabla_V f_p(U_p^{s,k+1},V^s)-\nabla_{V_p} f_p(U_p^{s,k+1},V_p^{s,k}).
\end{aligned}
Using the Lipschitz continuity and the inequality \|x+y+z\|_F^2\leq2\|x\|_F^2+4\|y\|_F^2+4\|z\|_F^2, it obtains
\begin{aligned}
&\quad\|\nabla_V f_p(U_p^{s+1,0},V^{s+1})-\nabla_{V_p} f_p(U_p^{s,k+1},V_p^{s,k})\|_F^2\\
&\leq2L_2^2\|V^{s+1}-V^s\|_F^2+4L_2^2\|V_p^{s,k}-V^s\|_F^2 +4L_{3,p}^2\|U_p^{s+1,0}-U_p^{s,k+1}\|_F^2.
\end{aligned}
Under the condition U_p^{s+1,0}=U_p^{s,\tau}, it follows that
\begin{aligned}
U_p^{s+1,0}-U_p^{s,k+1}=\sum_{l=k+1}^{\tau-1}(U_p^{s,l+1}-U_p^{s,l}).
\end{aligned}
By the Cauchy-Schwarz inequality,
\begin{aligned}
\sum_{k=0}^{\tau-1}\|U_p^{s+1,0}-U_p^{s,k+1}\|_F^2 &\leq\sum_{k=0}^{\tau-1}(\tau-k-1)\sum_{l=k+1}^{\tau-1}\|U_p^{s,l+1}-U_p^{s,l}\|_F^2\\
&\leq\frac{\tau(\tau-1)}{2}\sum_{l=0}^{\tau-1}\|U_p^{s,l+1}-U_p^{s,l}\|_F^2.
\end{aligned}
Substituting (?) and (?) into (Proof) yields
\begin{aligned}
&\quad\frac{1}{m}\|\Lambda^{s+1}-\overline{\Lambda}^{s+1}\|_F^2-2\alpha^2\tau^2L_2^2\|V^{s+1}-V^s\|_F^2\\
&\leq\frac{4\alpha^2\tau L_2^2}{m}\sum_{p=1}^{m}\sum_{k=0}^{\tau-1}\|V_p^{s,k}-V^s\|_F^2 +\frac{2\alpha^2\tau^2(\tau-1)}{m}\sum_{p=1}^{m}\sum_{k=0}^{\tau-1}L_{3,p}^2\|\Delta U_p^{s,k}\|_F^2.
\end{aligned}
Applying Lemma 2 to the local V drift term gives
\begin{aligned}
&\quad\frac{1}{m}\|\Lambda^{s+1}-\overline{\Lambda}^{s+1}\|_F^2-2\alpha^2\tau^2L_2^2\|V^{s+1}-V^s\|_F^2\\
&\leq\frac{24\alpha^2\tau^2L_2^2}{m}\|\Lambda^s-\overline{\Lambda}^s\|_F^2 +24\alpha^4\tau^4L_2^2\|\widetilde{\nabla}h(\widetilde V^{s+1})+\nabla_V f(\{U_p^{s,0}\},V^s)\|_F^2\\
&+96\alpha^4\tau^4L_2^2c_h^2+\frac{2\alpha^2\tau^2(\tau-1)+48\alpha^4\tau^5L_2^2}{m}\sum_{p=1}^{m}\sum_{k=0}^{\tau-1}L_{3,p}^2\|\Delta U_p^{s,k}\|_F^2.
\end{aligned}
The proof is complete.
Uses Lemma 2
Proof
GW3YUnder Assumption 2, it ensure that U_p^{s,k+1} is a global minimizer of subproblem (?). Since U_p^{s,k}\in\mathcal O_p is feasible for this subproblem and the proximal term vanishes at U_p^{s,k}, it has
\begin{aligned}
&f_p(U_p^{s,k+1},V_p^{s,k}) +\frac{\rho_1}{2} \|\Delta U_p^{s,k}\|_F^2\leq f_p(U_p^{s,k},V_p^{s,k}).
\end{aligned}
Therefore,
\begin{aligned}
&f_p(U_p^{s,k+1},V_p^{s,k}) - f_p(U_p^{s,k},V_p^{s,k})\leq -\frac{\rho_1}{2} \|\Delta U_p^{s,k}\|_F^2.
\end{aligned}
Next,
\begin{aligned}
&\quad f_p(U_p^{s,k+1},V^{s+1}) - f_p(U_p^{s,k},V^{s+1}) \\
&= f_p(U_p^{s,k+1},V_p^{s,k}) - f_p(U_p^{s,k},V_p^{s,k})+( f_p(U_p^{s,k+1},V^{s+1}) \\
& - f_p(U_p^{s,k+1},V_p^{s,k}))-( f_p(U_p^{s,k},V^{s+1}) - f_p(U_p^{s,k},V_p^{s,k})).
\end{aligned}
Let J_p(U) = f_p(U,V^{s+1})-f_p(U,V_p^{s,k}). Then the last two bracketed terms of (Proof) are J_p(U_p^{s,k+1})-J_p(U_p^{s,k}). Since the feasible set induced by U_p\geq 0 and U_p^\top U_p=I is not convex, the Euclidean line segment between U_p^{s,k} and U_p^{s,k+1} may not remain feasible. Hence, it uses the integral form along the Euclidean segment. Since f_p is continuously differentiable in an open neighborhood containing this segment, it has
\begin{aligned}
&J_p(U_p^{s,k+1})-J_p(U_p^{s,k}) = \int_0^1 \langle \nabla_U J_p ( U_p^{s,k} +t(U_p^{s,k+1}-U_p^{s,k}) ), U_p^{s,k+1}-U_p^{s,k} \rangle dt .
\end{aligned}
By the Cauchy–Schwarz inequality, it follows that
\begin{aligned}
& |J_p(U_p^{s,k+1})-J_p(U_p^{s,k}) | \leq \sup_{t\in[0,1]} \| \nabla_U J_p ( U_p^{s,k} +t(U_p^{s,k+1}-U_p^{s,k}) ) \|_F \|U_p^{s,k+1}-U_p^{s,k} \|_F .
\end{aligned}
Moreover, since \nabla_U J_p(U) = \nabla_U f_p(U,V^{s+1}) - \nabla_U f_p(U,V_p^{s,k}), the cross-Lipschitz continuity of \nabla_U f_p with respect to V gives
\begin{aligned}
\|\nabla_U J_p(U) \|_F
&= \| \nabla_U f_p(U,V^{s+1}) - \nabla_U f_p(U,V_p^{s,k}) \|_F \leq L_{1,p} \|V^{s+1}-V_p^{s,k} \|_F .
\end{aligned}
Therefore,
\begin{aligned}
&( f_p(U_p^{s,k+1},V^{s+1}) - f_p(U_p^{s,k+1},V_p^{s,k})) -( f_p(U_p^{s,k},V^{s+1}) - f_p(U_p^{s,k},V_p^{s,k}))\\
&\leq L_{1,p} \|V^{s+1}-V_p^{s,k}\|_F \|U_p^{s,k+1}-U_p^{s,k}\|_F.
\end{aligned}
Applying Young’s inequality gives
\begin{aligned}
& L_{1,p} \|V^{s+1}-V_p^{s,k}\|_F \|U_p^{s,k+1}-U_p^{s,k}\|_F \\ &\leq \frac{\rho_1}{4} \|\Delta U_p^{s,k}\|_F^2 + \frac{L_{1,p}^2}{\rho_1} \|V^{s+1}-V_p^{s,k}\|_F^2.
\end{aligned}
Combining (Proof), (Proof), (Proof), and (Proof), we obtain
\begin{aligned}
&f_p(U_p^{s,k+1},V^{s+1}) - f_p(U_p^{s,k},V^{s+1}) \\
&\leq -\frac{\rho_1}{4} \|\Delta U_p^{s,k}\|_F^2 + \frac{L_{1,p}^2}{\rho_1} \|V^{s+1}-V_p^{s,k}\|_F^2 .
\end{aligned}
Summing (Proof) over k=0,\cdots,\tau-1 gives
\begin{aligned}
&f_p(U_p^{s,\tau},V^{s+1}) - f_p(U_p^{s,0},V^{s+1}) \\
&\leq -\frac{\rho_1}{4} \sum_{k=0}^{\tau-1} \|\Delta U_p^{s,k}\|_F^2 + \frac{1}{\rho_1} \sum_{k=0}^{\tau-1} L_{1,p}^2 \|V^{s+1}-V_p^{s,k}\|_F^2 .
\end{aligned}
Using U_p^{s+1,0}=U_p^{s,\tau}, summing over all clients, and multiplying by 1/m, it has
\begin{aligned}
&\frac1m\sum_{p=1}^{m}f_p(U_p^{s+1,0},V^{s+1}) - \frac1m\sum_{p=1}^{m}
f_p(U_p^{s,0},V^{s+1}) \\
&\leq -\frac{\rho_1}{4m} \sum_{p=1}^{m}\sum_{k=0}^{\tau-1} \|\Delta U_p^{s,k}\|_F^2 + \frac{1}{m\rho_1} \sum_{p=1}^{m}\sum_{k=0}^{\tau-1} L_{1,p}^2 \|V^{s+1}-V_p^{s,k}\|_F^2 .
\end{aligned}
Since h(V^{s+1}) is independent of \{U_p\}, the above inequality proves (4).
Finally, because L_{3,p}\leq L_{3,a}, it has \|\Delta U_p^{s,k}\|_F^2 \geq \frac{L_{3,p}^2}{L_{3,a}^2} \|\Delta U_p^{s,k}\|_F^2. Substituting this inequality into (?) yields (4). This completes the proof.
Proof
QXURUnder Assumption 2, it ensures that U_p^{s,k+1} is a global minimizer, and hence a local minimizer, of subproblem (?). Since the objective is continuously differentiable and \mathcal O_p is closed, the generalized Fermat rule gives
0\in\nabla_{U_p}f_p(U_p^{s,k+1},V_p^{s,k})+\rho_1(U_p^{s,k+1}-U_p^{s,k})+N_{\mathcal O_p}(U_p^{s,k+1}).
Therefore, there exists Z_p^{s,k+1}\in N_{\mathcal O_p}(U_p^{s,k+1}) such that
\nabla_{U_p}f_p(U_p^{s,k+1},V_p^{s,k})+\rho_1(U_p^{s,k+1}-U_p^{s,k})+Z_p^{s,k+1}=0.
Using this normal vector in the definition of r_p^{s,k+1}, it obtains
\begin{aligned}
r_p^{s,k+1}
&\leq\|\nabla_{U_p}f_p(U_p^{s,k+1},V^s)+Z_p^{s,k+1}\|_F^2\\
&=\|\nabla_{U_p}f_p(U_p^{s,k+1},V^s)-\nabla_{U_p}f_p(U_p^{s,k+1},V_p^{s,k})-\rho_1(U_p^{s,k+1}-U_p^{s,k})\|_F^2\\
&\leq2L_{1,p}^2\|V_p^{s,k}-V^s\|_F^2+2\rho_1^2\|\Delta U_p^{s,k}\|_F^2,
\end{aligned}
where the last inequality follows from the cross Lipschitz continuity of \nabla_{U_p}f_p and \|A+B\|_F^2\leq2\|A\|_F^2+2\|B\|_F^2. This completes the proof.
Uses Assumption 2
Source and licence
The text on this page is from arXiv:2610.21455v1 by Yanjiao Zhu, Xianchao Xiu, Yongyang Xiong and Wanquan Liu, under the licence CC BY 4.0. The corpus made these changes: it read the items out of the LaTeX source, it expanded the macros of the paper, it put each proof after its statement, and it converted the text to HTML. The formulas are the TeX of the paper, and KaTeX renders them. A formula or a reference can look different from the PDF. The page is published under CC BY 4.0.
The record of each item is in analysis/objects/2610/2610.21455.jsonl.