Deep collocation method: A framework for solving PDEs using neural networks with error control

Mingxing Weng, Zhiping Mao and Jie Shen

arXiv: 2502.17203v2 · PDF · 7 March 2025
Categories: math.NA, cs.NA
Licence: CC BY 4.0 The site shows the text.

Abstract

Neural networks have shown significant potential in solving partial differential equations (PDEs). While deep networks are capable of approximating complex functions, direct one-shot training often faces limitations in both accuracy and computational efficiency. To address these challenges, we propose an adaptive method that uses single-hidden-layer neural networks to construct basis functions guided by the equation residual. The approximate solution is computed within the space spanned by these basis functions, employing a collocation least squares scheme. As the approximation space gradually expands, the solution is iteratively refined; meanwhile, the progressive improvements serve as reliable a posteriori error indicators that guide the termination of the sequential updates. Additionally, we introduce adaptive strategies for collocation point selection and parameter initialization to enhance robustness and improve the expressiveness of the neural networks. We also derive the approximation error estimate and validate the proposed method with several numerical experiments on various challenging PDEs, demonstrating both high accuracy and robustness of the proposed method.

Items 5

Proof

Proof 1

XSUC

Direct calculation gives

\@bsphack \protected@write\@auxout{} {\string \@ifundefined{r@equ:es_expand} \relax {\gdef \relax { \@latex@warning@no@line{Label `equ:es_expand' multiply defined}} \GenericError{ \space\space\space\@spaces\@spaces\@spaces }{ SIAM Label Error: There were multiply-defined labels }{ Correct multiply-defined labels. }{} } \global\@namedef{r@equ:es_expand}{{\p@equationSM\arabic{equation}}{SM\arabic{page}}}} \@esphack {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_s^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}^2 = {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_{s+1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}^2 + {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert v_{s+1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}^2 + 2\left(\langle \mathcal{L}e_{s+1}^*, \mathcal{L}v_{s+1}^*\rangle_\Omega + \lambda^2\langle\mathcal{B}e_{s+1}^*, \mathcal{B}v_{s+1}^*\rangle_{\partial\Omega}\right).

We claim that

\@bsphack \protected@write\@auxout{} {\string \@ifundefined{r@equ:inequality} \relax {\gdef \relax { \@latex@warning@no@line{Label `equ:inequality' multiply defined}} \GenericError{ \space\space\space\@spaces\@spaces\@spaces }{ SIAM Label Error: There were multiply-defined labels }{ Correct multiply-defined labels. }{} } \global\@namedef{r@equ:inequality}{{\p@equationSM\arabic{equation}}{SM\arabic{page}}}} \@esphack \langle \mathcal{L}e_{s+1}^*, \mathcal{L}v_{s+1}^*\rangle_{\Omega} + \lambda^2\langle\mathcal{B}e_{s+1}^*, \mathcal{B}v_{s+1}^*\rangle_{\partial\Omega} \geq 0.

Assuming (1) holds, it follows that

{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_s^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}^2 \geq {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_{s+1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}^2 + {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert v_{s+1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}^2 \geq {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_{s+1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}^2.

The equality holds if and only if {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert v_{s+1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert} = 0, which implies v_{s+1}^* = u_{s+1}^* - u_{s}^* = 0.

It remains to prove (1). Suppose, for the sake of contradiction, that (1) does not hold. Define v_{s+1}^\eta = (1-\eta)v_{s+1}^* \in \text{span}\Psi_{s+1}. Then, we have

\begin{aligned} {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_s^*-v_{s+1}^\eta \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}^2 =& {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_{s}^*-v_{s+1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}^2 + \eta\big(\eta{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert v_{s+1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}^2 \\ &+ 2\langle \mathcal{L}v_{s+1}^*,\mathcal{L}e_{s+1}^*\rangle_{\Omega}+ 2\lambda^2\langle \mathcal{B}v_{s+1}^*,\mathcal{B}e_{s+1}^*\rangle_{\partial\Omega}\big). \end{aligned}

By choosing \eta such that

0 < \eta < -\frac{2\left(\langle \mathcal{L}v_{s+1}^*, \mathcal{L}e_{s+1}^*\rangle_{\Omega} + \lambda^2\langle \mathcal{B}v_{s+1}^*, \mathcal{B}e_{s+1}^*\rangle_{\partial\Omega}\right)}{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert v_{s+1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}^2},

we would obtain {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_s^* - v_{s+1}^\eta \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}^2 < {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_s^* - v_{s+1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}^2, contradicting the definition of v_{s+1}^* as the minimizer

\@bsphack \protected@write\@auxout{} {\string \@ifundefined{r@equ:v_def} \relax {\gdef \relax { \@latex@warning@no@line{Label `equ:v_def' multiply defined}} \GenericError{ \space\space\space\@spaces\@spaces\@spaces }{ SIAM Label Error: There were multiply-defined labels }{ Correct multiply-defined labels. }{} } \global\@namedef{r@equ:v_def}{{\p@equationSM\arabic{equation}}{SM\arabic{page}}}} \@esphack \begin{aligned} v_{s+1}^* &= u_{s+1}^*-u_s^* =\underset{v\in\text{span}\Psi_{s+1}}{\arg\min} {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u-v \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}-u_s^*\\ &=\underset{v^\prime\in\text{span}\Psi_{s+1}}{\arg\min} {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u-(u_s^*+v^\prime) \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}+u_s^*-u_s^*\\ &=\underset{v^\prime\in\text{span}\Psi_{s+1}}{\arg\min} {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_s^*-v^\prime \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}. \end{aligned}

Therefore, (1) must hold.

Theorem

Theorem 2 (Universal Approximation)

6Y5C

Suppose 1 \leq p < \infty, 0 \leq s < \infty, and \Omega \subset \mathbb{R}^d is compact. If \sigma \in C^s(\Omega) is non-constant and bounded, then the space

V^\sigma := \bigcup_{N=1}^\infty V_N^\sigma

is dense in the Sobolev space

W^{s, p}(\Omega) := \left\{v \in L^p(\Omega): D^\alpha v \in L^p(\Omega), \ \forall |\alpha| \leq s \right\}.

Proof

Proof 2

IQC4

Selecting v^\prime=\psi_{s}\in\Psi_{s} in (?) and utilizing (?), we obtain

\begin{aligned} \left| {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u - u_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert} - {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert v_{s}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert} \right| \leq {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u - u_{s-1}^* - v_{s}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert} \leq {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_{s-1}^* - \psi_{s} \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert} \leq \tau_s{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}. \end{aligned}

Thus, we obtain the following bounds:

{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u - u_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert} \leq {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert v_{s}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert} + \tau_s {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u - u_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert},

and

{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u - u_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert} \geq {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert v_{s}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert} - \tau_s {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u - u_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}.

Combining these inequalities yields (?).

Proof

Proof 3

J6HK

For each stage s>0, it is obvious that {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u-u_{s}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}\leq {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u-u_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}. In addition, we observe that u_{s-1}^*+{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u-u_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}/{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert \psi_{s} \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}\cdot\psi_{s}\in\Psi_{s}. According to the definition of u_{s}^*, we have

\@bsphack \protected@write\@auxout{} {\string \@ifundefined{r@equ:error_reduce} \relax {\gdef \relax { \@latex@warning@no@line{Label `equ:error_reduce' multiply defined}} \GenericError{ \space\space\space\@spaces\@spaces\@spaces }{ SIAM Label Error: There were multiply-defined labels }{ Correct multiply-defined labels. }{} } \global\@namedef{r@equ:error_reduce}{{\p@equationSM\arabic{equation}}{SM\arabic{page}}}} \@esphack \begin{aligned} {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u-u_{s}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}&\leq{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u-\left(u_{s-1}^*+\frac{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u-u_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}}{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert \psi_{s} \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}}\cdot\psi_{s}\right) \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}\\ &={\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u-u_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert \frac{u-u_{s-1}^*}{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u-u_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}}-\frac{\psi_{s}}{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert \psi_{s} \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}} \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}\\ &< 2\tau_{s}{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u-u_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}. \end{aligned}

The final step follows thanks to

\@bsphack \protected@write\@auxout{} {\string \@ifundefined{r@equ:error_assumption} \relax {\gdef \relax { \@latex@warning@no@line{Label `equ:error_assumption' multiply defined}} \GenericError{ \space\space\space\@spaces\@spaces\@spaces }{ SIAM Label Error: There were multiply-defined labels }{ Correct multiply-defined labels. }{} } \global\@namedef{r@equ:error_assumption}{{\p@equationSM\arabic{equation}}{SM\arabic{page}}}} \@esphack \begin{aligned} {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert \frac{e_{s-1}^*}{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}} - \frac{\psi_{s}}{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert \psi_{s} \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}} \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert} &\leq {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert \frac{e_{s-1}^*}{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}} - \frac{\psi_{s}}{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}} \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert} + {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert \frac{\psi_{s}}{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}} - \frac{\psi_{s}}{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert \psi_{s} \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}} \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}\\ &=\frac{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_{s-1}^*-\psi_{s} \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}}{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}} + \frac{\left|{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert \psi_{s} \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}-{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}\right|}{{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert e_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert \psi_{s} \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}}\cdot{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert \psi_{s} \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert} \leq 2\tau_s. \end{aligned}

Thus, we have

{\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u-u_s^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert}\leq {\left\vert\kern-0.25ex\left\vert\kern-0.25ex\left\vert u-u_{s-1}^* \right\vert\kern-0.25ex\right\vert\kern-0.25ex\right\vert} \cdot \min \left\{1,2 \tau_s\right\}.

Then, the estimate (?) follows.

Remark

Remark 5

2KSL

r@rem:width_choice Label ‘rem:width_choice’ multiply defined

SIAM Label Error: There were multiply-defined labels Correct multiply-defined labels. r@rem:width_choiceequationSMpage The selection of the network width plays an important role in balancing convergence speed and computational efficiency. Due to its iterative nature, Algorithm ? allows flexibility to vary the network width N_s at each stage. Although in theory a fixed width N_s \equiv N can still achieve convergence by performing sufficient stages [ainsworth_galerkin_2021, Prop 2.12], it actually results in slow convergence in practice. The primary reason is that a fixed-width network has limited capacity to capture the higher-resolution features of the residual. ? further suggests how this stagnation can be avoided. Observe that if \tau_s = \tau < 1/2 for all s, then the sequence \left\{n(\tau, u_{s-1}^*)\right\}_{s=1}^S is expected to be non-decreasing. In particular, the error for u_s^* is reduced by a factor of 2 \tau compared to the error for u_{s-1}^*. To ensure that \psi_s can capture the higher-resolution features of u - u_{s-1}^* (as compared to the relatively lower-resolution features of u - u_{s-2}^*), it is necessary that n(\tau, u_{s-1}^*) > n(\tau, u_{s-2}^*). Therefore, to achieve geometric convergence, the network width N_s must be increased progressively as s grows. In the experiments, following the setting in [ainsworth_galerkin_2021], we set the network width as N_s = N_0 \times 2^s, where N_0\in \mathbb{Z}^+ is a predefined width.

Source and licence

The text on this page is from arXiv:2502.17203v2 by Mingxing Weng, Zhiping Mao and Jie Shen, under the licence CC BY 4.0. The corpus made these changes: it read the items out of the LaTeX source, it expanded the macros of the paper, it put each proof after its statement, and it converted the text to HTML. The formulas are the TeX of the paper, and KaTeX renders them. A formula or a reference can look different from the PDF. The page is published under CC BY 4.0.

The record of each item is in analysis/objects/2502/2502.17203.jsonl.