Block stochastic gradient descent for linear ill-posed problems in Hilbert spaces under the power-type source condition

Rong Wang; De-Han Chen; Ting Cheng; Daijun Jiang; Rong Wang; De-Han Chen; Ting Cheng; Daijun Jiang

doi:10.3934/era.2026334

Electronic Research Archive

2026, Volume 34, Issue 10: 7779-7810. doi: 10.3934/era.2026334

Previous Article Next Article

Research article Special Issues

Block stochastic gradient descent for linear ill-posed problems in Hilbert spaces under the power-type source condition

1.
School of Mathematics and Statistics, Central China Normal University, Wuhan 430079, China
2.
Hubei Key Laboratory of Mathematical Sciences, Central China Normal University, Wuhan 430079, China
3.
Key Lab NAA–MOE, Central China Normal University, Wuhan 430079, China

Received: 31 May 2026 Revised: 23 August 2026 Accepted: 25 August 2026 Published: 09 September 2026

In this paper, we consider linear ill-posed inverse problems in Hilbert spaces under the classical power-type source condition. We propose a block stochastic gradient descent (Block SGD) algorithm, which generalizes the classical SGD method by allowing updates on data blocks rather than single indices. Under the power-type source condition, we derive a novel mean squared error bound between the true solution and the regularized solution, in terms of the step size, the batch size, and the block number. In particular, our estimate shows that the stochastic error decreases as the batch size increases, which is confirmed by numerical experiments.
- stochastic gradient descent,
- regularization property,
- ill-posed problems,
- convergence rates,
- power-type source condition
Citation: Rong Wang, De-Han Chen, Ting Cheng, Daijun Jiang. Block stochastic gradient descent for linear ill-posed problems in Hilbert spaces under the power-type source condition[J]. Electronic Research Archive, 2026, 34(10): 7779-7810. doi: 10.3934/era.2026334

Related Papers:

Abstract

In this paper, we consider linear ill-posed inverse problems in Hilbert spaces under the classical power-type source condition. We propose a block stochastic gradient descent (Block SGD) algorithm, which generalizes the classical SGD method by allowing updates on data blocks rather than single indices. Under the power-type source condition, we derive a novel mean squared error bound between the true solution and the regularized solution, in terms of the step size, the batch size, and the block number. In particular, our estimate shows that the stochastic error decreases as the batch size increases, which is confirmed by numerical experiments.

References

[1]	H. W. Engl, M. Hanke, A. Neubauer, Regularization of inverse problems, Springer Dordrecht, 2020. Available from: https://link.springer.com/book/9780792341574.
[2]	B. Hofmann, P. Mathé, M. Schieck, Modulus of continuity for conditionally stable ill-posed problems in Hilbert space, J. Inverse Ill-Posed Probl., 16 (2008), 567–585. https://doi.org/10.1515/JIIP.2008.030 doi: 10.1515/JIIP.2008.030
[3]	B. Jin, X. Lu, On the regularization property of stochastic gradient descent, Inverse Probl., 35 (2019), 015004. https://doi.org/10.1088/1361-6420/aaea2a doi: 10.1088/1361-6420/aaea2a
[4]	B. Jin, Z. Zhou, On the convergence of stochastic variance reduced gradient for linear inverse problems, Inverse Probl., 42 (2026), 045006. https://doi.org/10.1088/1361-6420/ae5845 doi: 10.1088/1361-6420/ae5845
[5]	S. Lu, P. Mathé, Stochastic gradient descent for linear inverse problems in Hilbert spaces, Math. Comput., 91 (2022), 1763–1788. https://doi.org/10.1090/mcom/3714 doi: 10.1090/mcom/3714
[6]	T. Jahn, B. Jin, On the discrepancy principle for stochastic gradient descent, Inverse Probl., 36 (2020), 095009. https://doi.org/10.1088/1361-6420/abaa58 doi: 10.1088/1361-6420/abaa58
[7]	M. J. Ehrhardt, Ž. Kereta, J. Liang, J. Tang, A guide to stochastic optimisation for large-scale inverse problems, Inverse Probl., 41 (2025), 053001. http://doi.org/10.1088/1361-6420/adc0b7 doi: 10.1088/1361-6420/adc0b7
[8]	J. Lin, L. Rosasco, Optimal rates for multi-pass stochastic gradient methods, J. Mach. Learn. Res., 18 (2017), 1–47. Available from: http://jmlr.org/papers/v18/17-176.html.
[9]	S. Kassing, S. Weissmann, L. Döring, Controlling the flow: Stability and convergence for stochastic gradient descent with decaying regularization, in Advances in Neural Information Processing Systems, 38 (2026), 83893–83932. https://doi.org/10.52202/085713-2530
[10]	Z. Zhou, On the convergence of a data-driven regularized stochastic gradient descent for nonlinear ill-posed problems, SIAM J. Imag. Sci., 18 (2025), 388–448. https://doi.org/10.1137/24M1647394 doi: 10.1137/24M1647394
[11]	A. Beck, L. Tetruashvili, On the convergence of block coordinate descent type methods, SIAM J. Optim., 23 (2013), 2037–2060. https://doi.org/10.1137/120887679 doi: 10.1137/120887679
[12]	Q. Jin, X. Lu, L. Zhang, Stochastic mirror descent method for linear ill-posed problems in Banach spaces, Inverse Probl., 39 (2023), 065010. https://doi.org/10.1088/1361-6420/accd8e doi: 10.1088/1361-6420/accd8e
[13]	B. Jin, Z. Zhou, J. Zou, On the saturation phenomenon of stochastic gradient descent for linear inverse problems, SIAM/ASA J. Uncertainty Quantif., 9 (2021), 1553–1588. https://doi.org/10.1137/20M1374456 doi: 10.1137/20M1374456
[14]	R. Gu, Z. Fu, B. Han, H. Fu, Stochastic gradient descent method with convex penalty for ill-posed problems in Banach spaces, Inverse Probl., 41 (2025), 055003. https://doi.org/10.1088/1361-6420/adc94d doi: 10.1088/1361-6420/adc94d
[15]	B. Jin, Ž. Kereta, On the convergence of stochastic gradient descent for linear inverse problems in Banach spaces, SIAM J. Imag. Sci., 16 (2023), 671–705. https://doi.org/10.1137/22M1518542 doi: 10.1137/22M1518542
[16]	J. Huang, Q. Jin, X. Lu, L. Zhang, On early stopping of stochastic mirror descent method for ill-posed inverse problems, Numer. Math., 157 (2025), 539–571. https://doi.org/10.1007/s00211-025-01458-7 doi: 10.1007/s00211-025-01458-7
[17]	B. Jin, Z. Zhou, J. Zou, On the convergence of stochastic gradient descent for nonlinear ill-posed problems, SIAM J. Optim., 30 (2020), 1421–1450. https://doi.org/10.1137/19M1271798 doi: 10.1137/19M1271798
[18]	Q. Jin, L. Chen, Stochastic variance reduced gradient method for linear ill-posed inverse problems, Inverse Probl., 41 (2025), 055014. https://doi.org/10.1088/1361-6420/add04b doi: 10.1088/1361-6420/add04b
[19]	Q. Jin, Y. Liu, Convergence analysis of a stochastic heavy-ball method for linear ill-posed problems, J. Comput. Appl. Math., 470 (2025), 116702. https://doi.org/10.1016/j.cam.2025.116702 doi: 10.1016/j.cam.2025.116702
[20]	Y. Zhang, C. Chen, Stochastic asymptotical regularization for linear inverse problems, Inverse Probl., 39 (2023), 015007. https://doi.org/10.1088/1361-6420/aca70f doi: 10.1088/1361-6420/aca70f
[21]	H. Long, Y. Zhang, Stochastic asymptotical regularization for nonlinear ill-posed problems, Inverse Probl., 41 (2025), 065017. https://doi.org/10.1088/1361-6420/addffc doi: 10.1088/1361-6420/addffc
[22]	A. Klenke, Probability Theory: A Comprehensive Course, Springer, London, 2008. https://doi.org/10.1007/978-1-84800-048-3
[23]	K. Ito, B. Jin, Inverse Problems: Tikhonov Theory and Algorithms, World Scientific, 2014.
[24]	B. Jin, Z. Kereta, Y. Xia, Stochastic Gradient Descent for Nonlinear Inverse Problems in Banach Spaces, SIAM J. Imag. Sci., 19 (2026), 1244–1271. https://doi.org/10.1137/25M1822289 doi: 10.1137/25M1822289
[25]	T. Hytönen, J. van Neerven, M. Veraar, L. Weis, Analysis in Banach spaces, Volume I: Martingales and Littlewood-Paley Theory, Springer Cham, 2016. https://doi.org/10.1007/978-3-319-48520-1
[26]	K. Schmüdgen, Unbounded Self-adjoint Operators on Hilbert Space, Springer Dordrecht, 2012. https://doi.org/10.1007/978-94-007-4753-1
[27]	B. Simon, Loewner's Theorem on Monotone Matrix Functions, Springer Cham, 2019. https://doi.org/10.1007/978-3-030-22422-6
[28]	Y. Zhang, C. Chen, Stochastic linear regularization methods: random discrepancy principle and applications, Inverse Probl., 40 (2024), 025007. https://doi.org/10.1088/1361-6420/ad149e doi: 10.1088/1361-6420/ad149e
[29]	M. Geissert, Applications of discrete maximal $L_p$ regularity for finite element operators, Numer. Math., 108 (2007), 121–149. https://doi.org/10.1007/s00211-007-0110-1 doi: 10.1007/s00211-007-0110-1
[30]	R. Wang, D. H. Chen, T. Cheng, D. Jiang, A new strategy for convergence rates of non-stationary asymptotical regularization method for linear inverse problems, Inverse Probl., 41 (2025), 105007. https://doi.org/10.1088/1361-6420/ae0e4b doi: 10.1088/1361-6420/ae0e4b

Reader Comments

Your name:*

Email:*
© 2026 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)