Point processes are widely used statistical models for continuous-time discrete event data, such as medical records, crime reports, and social network interactions, to capture the influence of historical events on future occurrences. In many applications, however, event times are not observed exactly, motivating the need to incorporate time uncertainty into point process modeling. In this work, we introduce a framework for modeling time-uncertain self-exciting point processes, known as Hawkes processes, possibly defined over a network. We begin by formulating the model in continuous time under assumptions motivated by real-world scenarios. By imposing a time grid, we obtain a discrete-time model that facilitates inference and enables computation via first-order optimization methods such as gradient descent and variational inequality (VI). We establish a parameter recovery guarantee for VI inference with an O(1/k) convergence rate using k steps. Our framework accommodates non-stationary processes by representing the influence kernel as a matrix (or tensor on a network), while also encompassing stationary processes, such as the classical Hawkes process, as a special case. Empirically, we demonstrate that the proposed approach outperforms existing baselines on both simulated and real-world datasets, including the sepsis-associated derangement prediction challenge and the Atlanta Police Crime Dataset.
The notations used in this work are standard. For a vector x \in \mathbb{R}^n, we use both bracket x(u) or subscript x_u to denote the u-th entry of the vector. When to emphasize that x is a vector, we use the boldface symbol \boldsymbol{x}; \otimes means vector outer-product, i.e., for u, v \in \mathbb{R}^n, u \otimes n is an n-by-n matrix. a \vee b := \max\{ a, b\}, and a \wedge b := \min \{ a, b\}. \mathbb{R}_+ = (0, \infty) denotes the set of strictly positive real numbers.
(A1) The consecutive intervals \Delta_i do not overlap, i.e., \Delta_{i-1}^r \le \Delta_{i}^l for all i.
The log-likelihood of \mathcal{L} as in (?) has the expression
By Assumption (A1), we know that \Delta_k^r > \Delta_k^l \ge \Delta_{k-1}^r, and then we have
This leads to the expression of the likelihood \mathcal{L} as follows, denoting \Delta_{0}^r = 0,
Recall the definition of \lambda(t) as in (?), then (?) can be written by integrals of \lambda(t), which leads to the expression of \mathcal{L} as
Taking the log proves (3.1).
We choose the form of the conditional intensity function in (?) because it is consistent with the earlier Hawkes conditional intensity model specified in (?). Specifically, if the uncertainty window \Delta_i shrinks to a single point t_k, i.e., |\Delta_i|\rightarrow 0 for all i, then, by that t_i \in (\Delta_i^l, \Delta_i^r] according to (?) (and assuming the continuity of the function k), the expression (?) is reduced to (?). In this limit, the time-uncertain model for the conditional intensity function \lambda(t) recovers the classic “time-certain” Hawkes model.
In (?), we have assumed the most general form of kernel k(t',t), which is allowed to be non-stationary. The time-invariant kernel is a special case when k(t',t) = \psi( t- t' ) for some function \psi. Later in Section ?, we will first present the general case of a time-varying kernel (corresponding to the recovery of a matrix), and then discuss the special case of the time-invariant kernel (corresponding to the recovery of a vector) in Section ?.
(A2) The uncertain event time window \Delta_i can only have the end points \Delta_i^l and \Delta_i^r taking value on the evenly spaced discrete time grid points \{ j h: j \in [N]\}.
Under Assumption (A2), for j \in [N],
By (?) and under Assumption (A2), we have
By the definition of \Lambda_j in (?),
The expression (3.2) then follows by the definition of K_{i,j} in (?).
(A3) The uncertainty window \Delta_i falls on exact one of the N intervals, i.e., \Delta_i = I_{j(i)} for some j(i) \in [N]. In other words, t_i^l = t_i^r = j(i).
Under the unit-uncertainty setting Assumption 3(A3) and using the formulation of the Bernoulli process y_t, for any t\in [N], \Lambda_t \in \sigma\{ y_i, i\le t-1\} and
Under (A3), the definition of \Lambda_t in (?) shows that \Lambda_t \in \sigma\{ y_i, i\le t-1 \}. The expression of log-likelihood as in (?) gives that
This gives that, for any t \in [ N],
Thus, by definition, \Pr [y_t = 1 | y_i, i\le t-1 ] = 1 - e^{ - h \Lambda_t}.
Under Assumption 4, - \mathbb{E} L(z) is \kappa'-strongly convex on \Theta_K, where \kappa' := e^{-2hb} (b/B)^2 h^2 e^{-B \tau_{\rm max}}.
Direct computation gives that
For any z \in \Theta_K, by Assumption 4(ii), b \le \Lambda_t( z) \le B a.s., and then \frac{e^{- h \Lambda_t (z)}}{(1- e^{- h \Lambda_t (z)})^2} \ge \frac{e^{-hB }}{(1- e^{-hB })^2} a.s. This gives that
Meanwhile, because \eta_t \in \sigma\{ y_i, i<t\},
This means that \mathbb{E} (- L(z) ) is \lambda-strongly convex on \Theta_K where \lambda = h^2 \frac{e^{-hB }}{(1- e^{-hB })^2} (1- e^{-hb}) \rho. To prove the proposition, it suffices to show that \lambda \ge \kappa'. This can be verified by the definition of \rho in Lemma 4.3 and using that x e^{-x} \le 1-e^{-x} \le x, \forall x \ge 0.
There exist constants B, b > 0 s.t.
(i) The true conditional intensity \Lambda_t^* satisfies
(ii) There exists a non-empty, compact, and convex set \Theta_K \subset \mathbb{R}^{NN'} such that the true kernel parameter z^* \in \Theta_K, and for any z \in \Theta_K,
Under Assumption 4(i), we have that \mu \in [b,B], and any realization of the binary sequence \{y_t, -N' < t \le N\} happens w.p. > 0. Consequently, it always holds that b \le \Lambda_t^* \le B, \forall t= -N'+1, \cdots, N.
Consider when t= -N'+1, since there is no history, \Lambda_{-N'+1}^* = \mu, and then (?) implies that b \le \mu \le B.
The claim that any realization of the binary sequence \{y_t, -N' < t \le N\} happens w.p. > 0 is equivalent to that any realization of the binary sequence \{y_j, -N'+1 \le j \le t\}, for any -N' < t \le N, happens w.p. > 0. We prove this claim by induction.
First, for t= -N'+1, \Pr [ y_{-N'+1} =0] = e^{-h \mu} \in (e^{-hB}, e^{-hb}). Thus both \Pr [ y_{-N'+1} =0] and \Pr [ y_{-N'+1} =1] >0.
Suppose the claim holds for t-1,
where the second inequality is by Lemma 3.3. By induction hypothesis, \Pr [ (y_{ -N'+1 }, \cdots, y_{t-1}) ] > 0, then (?) holds a.s. means that \Lambda_t^* as a function evaluated at the outcome (y_{ -N'+1 }, \cdots, y_{t-1}) needs to satisfy that b \le \Lambda_t^* \le B. This means that both e^{-h \Lambda_t^*} and 1-e^{-h \Lambda_t^*} > 0. As a result, \Pr [ (y_{ -N'+1 }, \cdots, y_{t}) ] > 0 in both cases of y_t = 0 or 1. This proves the claim for t based on the induction hypothesis on t-1. We thus have proved that the claim holds for any t.
Finally, because the conditional intensity \Lambda_t^* is a random variable determined by the outcome (y_{ -N'+1 }, \cdots, y_{t-1}), any any realization of this binary sequence up to time t-1 happens w.p. >0, then b \le \Lambda_t^* \le B a.s. means that for any realization of (y_{ -N'+1 }, \cdots, y_{t-1}) this inequality holds.
Under Assumption 4(i),
For any v \in \mathbb{R}^{N N'}, we also write v in the pattern of matrix K indexed by v_{i,t}, -N' < i \le N, 1 \le t \le N, cf. (?), then
where the second equality is by (?). For each t, we lower bound \mathbb{E} \left( \sum_{ i = t- N' }^{t-1} y_i v_{i, t} \right)^2 by enumerating over the realizations of (y_{t-N'}, \cdots, y_{t-1} ), a binary sequence of length N', that equals (0, \cdots, 1, \cdots, 0), namely only y_i = 1 for some i and the other y_j’s are all zero. By the derivation in the proof of Lemma 3.3, we know that
and then, applying Lemma 4.2, we have
As a result, for each t,
and then
This proves the lemma by putting back to (Proof).
Under Assumption 4, G(z) is \kappa-monotone on \Theta_K, where
By definition of G(z) in (?), for any z, \tilde z \in \Theta_K,
where in the second equality we used that \Lambda_t ( z ) = \mu + \langle \eta_t , z \rangle. Observe an elementary fact about the function \phi(x) = 1-e^{-x}: for any \beta >0,
The claim can be verified by the Mean Value Theorem. Using this fact and that \Lambda_t(z), \Lambda_t(\tilde z) \in [b,B] a.s. under Assumption 4(ii), (Proof) continues as
By Lemma 4.3 which holds under Assumption 4(i), we have
It remains to show that e^{-hB} h \rho \ge \kappa to finish the proof of the lemma. By definition,
by that hN' = \tau_{\rm max}, where in the inequality we used the fact that x e^{-x} \le 1-e^{-x}, \forall x \ge 0.
Under Assumption 4,
suppose y^{(k)} = \{ y_t^{(k)} \}_t are i.i.d. trajectories across k, and z_k is computed from (?) starting from z_0 \in \Theta_K, with step size
where \kappa is as in (4.4). Then, with C:= B(T+ \tau_{\rm max} ) \wedge (N + N'), we have
Due to the projection operator in (?), we have that z_k \in \Theta_K for all k. Let \| \cdot \| always denote the 2-norm. By definition of the projection operator (?), the convexity of \Theta_K and that z^* \in \Theta_K, we have
and by expanding the r.h.s. we have
Taking expectation on both sides, and defining d_k := \frac{1}{2} \mathbb{E} \| z_k - z^*\|^2, we have
The third term on the r.h.s. is upper bounded by 2 \gamma_k^2 C^2 by Lemma B.1(ii).
To handle the second term on the r.h.s. of (Proof), first note that z_{k-1} is in the \sigma-algebra generated by up to (k-1)-th trajectories, and then
where we used that \mathbb{E} [ \hat G^{(k)}(z_{k-1}) | y^{(m)}, m=1, \cdots k-1 ] = G(z_{k-1}) because y^{(k)} is independent from the previous k-1 trajectories. Next, recall (?), we have
where the last inequality is by Lemma 4.4 and that both z_{k-1} and z^* are in \Theta_K. Taking expectations on both sides gives
Putting back to (Proof), we have
where, by our choice of \gamma_k, \kappa \gamma_k = 1/(k+1) \le 1/2 for all k\ge 1, and thus, the factor 1- 2 \kappa \gamma_k is non-negative. We now claim that for all k,
which, if true, proves the theorem. The proof of (?) follows the argument to prove the same claim in the proof of [juditsky2019signal, Proposition 2], and we include the details for completeness.
We prove (?) by induction. When k=0, we have d_0 \le D^2/2 where D > 0 is the diameter of the domain \Theta_K. Suppose D = \|z_+ - z_-\|, where z_+, z_- \in \Theta_K. We then have
This gives that D \le 2C/\kappa, and then we have
which proves (?) when k=0.
Suppose the claim holds up to k-1; we prove it also holds for k. Because k \ge 1, we have 1- 2 \kappa \gamma_k \ge 0, then, by (?), and that d_{k-1} \le S/k (the induction hypothesis),
This proves the claim (?) by induction.
We consider the limit when the length of the time grid interval h \to 0, where the kernel matrix K approaches the kernel function k(t',t) and the time-uncertainty model (the unit-uncertainty setting) approaches the time-certain case. We assume that the time horizon T = h N and the influence time lag \tau_{\rm max} = h N' stay as O(1) constants in the limit, and we call this the continuity limit. In other words, N \sim N' \sim 1/h. We also assume that the constants b and B in Assumption 4 are fixed positive constants.
In this asymptotic, for large N (and N') and equivalently small h, C \sim B(T + \tau_{\rm max}) is O(1), and \kappa \sim h^2 e^{-B \tau_{\rm max}}, and thus the factor C/\kappa in the convergence bound in Theorem 4.5 scales as e^{B\tau_{\rm max}}/h^2, which \sim h^{-2} and we kept the exponential factor in the constant.
Suppose the influence kernel is time-invariant, under Assumption 4(i) and the assumption (ii’) on \Theta_\psi as above, G_z(\psi) is \kappa_s-monotone on \Theta_\psi, where
We claim that under Assumption 4(i),
If true, then using the same argument as in the proof of Lemma 4.4 one can show that for any \psi, \tilde \psi \in \Theta_\psi,
which proves that G_s is (e^{-hB} h \rho_s)-monotone on \Theta_\psi, and that is the same as declared in the lemma.
It remains to prove the claim (Proof) to prove the lemma. For any v \in \mathbb{R}^{N'},
To lower bound \mathbb{E} \left( \sum_{ i = t- N' }^{t-1} y_i v_{t-i} \right)^2 for each t, again we have (?) due to Lemma 4.2, and then
Thus, \mathbb{E} \sum_{t=1}^N \left( \sum_{ i = t- N' }^{t-1} y_i v_{t-i} \right)^2 \ge N \rho \| v\|_2^2. This proves that \mathbb{E} \sum_{t= 1}^N \xi_t \otimes \xi_t \succeq \lambda I_{N'} where
where in the inequality we used x e^{-x} \le 1-e^{-x}, \forall x \ge 0.
Recall that \bar{\lambda}(t) = \sum_{u \in\mathcal{V}} \lambda(t,u),
Using the expression (?), we have that, denoting \Delta_{0}^r = 0,
This proves the expression (5.1).
We consider the limit where the length of the window \Delta_i shrinks to zero, that is, \Delta_i^{l}, \, \Delta_i^{r} \to t_i the certain event time for all i. We claim that in this case,
This is can be derived from (?), which gives that the l.h.s. can be written as
and then using the continuity of \lambda(t,u) (and \bar{\lambda}(t)) with respect to t. The equation (5.1) leads to the expression
where we recall that in the classical (time-certain) model, \mathcal{F}_{k}:=\sigma\{ (t_i, u_i), \, i=1,\cdots, k \}. Then, the log-likelihood of the n events can be shown to be
which recovers the classical model without time uncertainty, see e.g. Eqn (8) in [reinhart2018review].
Consider the special case where the network \mathcal{V} only has one node u. In this case, \bar{\lambda}(t)= \lambda(t,u) and define it to be \lambda(t). Then, one can verify that
This equality reduces the conditional probability (?) to (Proof), and the log-likelihood (5.1) to (3.1). That is, the model is reduced to the time-only case in Section ?.
For any u', u \in \mathcal{V} and 1 \le i < j \le N, k(t', t; u', u) is constant on (t',t) \in I_i \times I_j, and we denote the value as K_{i,j}(u', u).
Under (A3) and Assumption 5, for any u \in \mathcal{V}, \lambda(t,u) as defined in (?) is constant over t \in I_j. We denote the value as \Lambda_j(u), and
Under Assumption 3(A3) and Assumption 5, the log-likelihood defined in (5.1) can be written as
Under (A3), we have (?) hold, namely
Applying Lemma ?, ?, we have
and meanwhile,
This proves the expression of \ell as (5.3).
Under Assumption 3(A3) and Assumption 5,
and using the formulation of the Bernoulli process \mathbf{y}_t on \mathcal{V}, for any t\in [N], \mathbf{\Lambda}_t \in \sigma\{ \mathbf{y}_i, i\le t-1 \} and
where \Phi: \mathbb{R}_+^V \to \mathbb{R}_+^V is defined by
Because \boldsymbol x \in \mathbb{R}_+^V satisfies \boldsymbol x^T \mathbf{ 1} > 0, (?) is well-defined.
The expression (?) implies that \mathbf{\Lambda}_t is in \sigma\{ \mathbf{y}_i, i\le t-1 \}. From the expression of the log-likelihood (?), one can verify that
and for each u\in\mathcal{V},
As a result,
where in the second equality we used that \bar{\Lambda}_t = \mathbf{\Lambda}_t^T \mathbf{1} by definition (?). Writing in vector form, this gives \mathbb{E} [ \mathbf{y}_t | \mathbf{y}_i, i\le t-1 ] = \Phi(\boldsymbol x) with \boldsymbol x = h \mathbf{\Lambda}_t \in \mathbb{R}_+^V.
For data with N'=8 and N=32, both TULIK-VI and TULIK-GD can learn the true model parameters (\mu and kernel matrix) well, resulting in good prediction performance, as shown quantitatively in Table ?. The proposed TULIK models outperform the other three baselines, where TULIK-VI gains slightly better performance than TULIK-GD.
Both TULIK-VI and TULIK-GD can learn the vector \mu and the kernel tensor, leading to accurate prediction as quantitatively revealed in Table ?. Similar to the time-only examples, the proposed TULIK models yield smaller prediction errors than the other baselines, where TULIK-VI is slightly better than TULIK-GD.
The performance of all methods is reported in Table ?. We evaluate True Positive Rate (TPR), True Negative Rate (TNR), and Balanced Accuracy (BA), on the testing data. These evaluation metrics are standard for sepsis prediction [physionetChallenge]. As shown in the table, TULIK-VI consistently outperforms the baseline methods.
Table ? presents the prediction performance of all methods. We report TPR, TNR, and BA on the testing data. For the burglary crime data, TPR and TNR are computed by averaging across all nodes for t \in (0, 16], treating event occurrences as positives and non-events as negatives. TULIK-VI achieves noticeably higher prediction accuracy compared to the baseline methods.
Under Assumption 4, let C be as defined in Theorem 4.5,
(i) Define G_1(z) := \mathbb{E} \sum_{t=1}^N \phi( h \Lambda_t(z)) \eta_t, then \forall z \in \Theta_K, \| G_1(z) \|_2 \le C.
(ii) \forall z \in \Theta_K, \mathbb{E} \| \hat G^{(m)}( z ) \|_2^2 = \mathbb{E} \| \sum_{t=1}^N \left(\phi( h \Lambda_t (z)) - y_t \right) \eta_t \|_2^2 \le 4 C^2.
We first prove (i), which provides useful intermediate estimates to prove (ii). The norm \| \cdot \| always denotes 2-norm.
Proof of (i): \| G_1(z) \|^2 = \| \mathbb{E} \sum_{t=1}^N \phi( h \Lambda_t(z)) \eta_t \|^2 \le \mathbb{E} \| \sum_{t=1}^N \phi( h \Lambda_t(z)) \eta_t \|^2 by Jensen’s inequality. By definition of \eta_t as a vector in \mathbb{R}^{NN'}, we know that \langle \eta_t, \eta_{t'} \rangle =0 when t \neq t'. Meanwhile, \|\eta_t\|^2 = \sum_{i = t-N'}^{t-1} y_i. Thus,
where the 2nd inequality is by that, under Assumption 4(ii),
using the elementary relation that 1-e^{-x} \le x, \forall x \ge 0. We claim that
If true, then putting together, we have
namely \| G_1(z) \| \le ( (hB) \wedge 1) (N + N') which equals C since hB(N + N') = B (T+ \tau_{\rm max}).
Proof of Claim (?): Observe that for any t, we always have
and then
By linearity of expectation, we have
which proves the claim.
Proof of (ii): Again using that \langle \eta_t, \eta_{t'} \rangle =0 when t \neq t', and \|\eta_t\|^2 = \sum_{i = t-N'}^{t-1} y_i, we have
In the proof of (i), we have already shown that \mathbb{E} \sum_{t=1}^N \phi( h \Lambda_t (z))^2 \sum_{i = t-N'}^{t-1} y_i \le C^2. To handle the second term in the bracket, by (?) we have
which again is upper bounded by C^2 as has been shown in the proof of (i). Putting together, we have that (?) is upper bounded by 4 C^2.
Under (A3), \Delta_i = I_{j(i)} for some j(i) \in [N], and then for t \in I_j, by (?),
because \Delta_i^r < t if and only if j(i) < j. By Assumption 5, for t' \in I_{j(i)} and t \in I_j, k(t', t; u_i, u) \equiv K_{ j(i), j} (u_i, u). Thus,
The r.h.s. is a constant independent of t, and this proves the lemma.
In this time-invariant kernel example, the kernel \psi and the scalar \mu are well recovered, as illustrated by Figure ? and the small REs in Table ?.
The text on this page is from arXiv:2411.02694v2 by Xiuyuan Cheng, Tingnan Gong and Yao Xie, under the licence CC BY 4.0. The corpus made these changes: it read the items out of the LaTeX source, it expanded the macros of the paper, it put each proof after its statement, and it converted the text to HTML. The formulas are the TeX of the paper, and KaTeX renders them. A formula or a reference can look different from the PDF. The page is published under CC BY 4.0.
The record of each item is in analysis/objects/2411/2411.02694.jsonl.