Following the proof of [VZ08, Prop.3.1] the restriction morphism factors as the composite
\operatorname{End}_k(\operatorname{M}(X))\xrightarrow{res_{X/k}} \operatorname{End}_{X}(\operatorname{M}(X_X))\xrightarrow{res_{k(X)/X}}\operatorname{End}_{k(X)}(\operatorname{M}(X_{k(X)}))
where the middle term denotes endomorphisms in the category of Chow motives over the relative base X, the first map is injective, and the second map is induced by passing to the generic point of X. Observe that the kernel of res_{k(X)/X} is the nilpotent ideal by the proof of [VZ08, Lem.3.2] extended in [Gi25, §3.5].
Now the kernel of the restriction res_{l/k}\colon \operatorname{End}_k(\operatorname{M}(X)) \to \operatorname{End}_l(\operatorname{M}(X_l)) is contained in the kernel of res_{F/k}, where F contains both l and k(X), so is nilpotent.