A new operator splitting method with application to feature selection

Yunda Dong; Yiyi Li; Yunda Dong; Yiyi Li

doi:10.3934/math.2025488

AIMS Mathematics

2025, Volume 10, Issue 5: 10740-10763. doi: 10.3934/math.2025488

Previous Article Next Article

Research article

A new operator splitting method with application to feature selection

Yunda Dong ^,,
Yiyi Li

School of Mathematics and Statistics, Zhengzhou University, Zhengzhou, 450001, China

Received: 19 February 2025 Revised: 18 April 2025 Accepted: 24 April 2025 Published: 12 May 2025
MSC : 65K05, 49M30, 46N10, 90C31

In this article, we consider the problem of finding a zero of a system of monotone inclusions in Hilbert spaces. Notably, each of these monotone inclusions comprises three operators, with two of them being linearly composed. To address this challenge, we propose a new splitting method that, at each iteration, essentially necessitates the computation of three individual resolvents, corresponding to each operator within the monotone inclusion. Under the weakest possible conditions, with the help of characteristic operator techniques, we analyze the weak convergence properties of our proposed method, which is facilitated by the introduction of a novel inequality. Numerical results demonstrate the practical usefulness of this method in solving large-scale rare feature selection in deep learning.
- monotone inclusion,
- splitting method,
- characteristic operator,
- weak convergence,
- feature selection
Citation: Yunda Dong, Yiyi Li. A new operator splitting method with application to feature selection[J]. AIMS Mathematics, 2025, 10(5): 10740-10763. doi: 10.3934/math.2025488

Related Papers:

Abstract

In this article, we consider the problem of finding a zero of a system of monotone inclusions in Hilbert spaces. Notably, each of these monotone inclusions comprises three operators, with two of them being linearly composed. To address this challenge, we propose a new splitting method that, at each iteration, essentially necessitates the computation of three individual resolvents, corresponding to each operator within the monotone inclusion. Under the weakest possible conditions, with the help of characteristic operator techniques, we analyze the weak convergence properties of our proposed method, which is facilitated by the introduction of a novel inequality. Numerical results demonstrate the practical usefulness of this method in solving large-scale rare feature selection in deep learning.

References

[1]	B. Vũ, A splitting algorithm for dual monotone inclusions involving cocoercive operators, Adv. Comput. Math., 38 (2013), 667–681. https://doi.org/10.1007/s10444-011-9254-8 doi: 10.1007/s10444-011-9254-8
[2]	D. Hieu, L. Vy, P. Quy, Three-operator splitting algorithm for a class of variational inclusion problems, Bull. Iran. Math. Soc., 46 (2020), 1055–1071. https://doi.org/10.1007/s41980-019-00312-5 doi: 10.1007/s41980-019-00312-5
[3]	O. Iyiola, C. Enyi, Y. Shehu, Reflected three-operator splitting method for monotone inclusion problem, Optim. Method. Softw., 37 (2022), 1527–1565. https://doi.org/10.1080/10556788.2021.1924715 doi: 10.1080/10556788.2021.1924715
[4]	E. Chouzenoux, M. Corbineau, J. Pesquet, A proximal interior point algorithm with applications to image processing, J. Math. Imaging Vis., 62 (2020), 919–940. https://doi.org/10.1007/s10851-019-00916-w doi: 10.1007/s10851-019-00916-w
[5]	Y. Tang, M. Wen, T. Zeng, Preconditioned three-operator splitting algorithm with applications to image restoration, J. Sci. Comput., 92 (2022). https://doi.org/10.1007/s10915-022-01958-w doi: 10.1007/s10915-022-01958-w
[6]	A. Padcharoen, D. Kitkuan, W. Kumam, P. Kumam, Tseng methods with inertial for solving inclusion problems and application to image deblurring and image recovery problems, Comput. Math. Method. M., 3 (2021), e1088. https://doi.org/10.1002/cmm4.1088 doi: 10.1002/cmm4.1088
[7]	V. Nguyen, N. Vinh, Two new splitting methods for three-operator monotone inclusions in Hilbert spaces, Set-Valued Var. Anal., 32 (2024), 26. https://doi.org/10.1007/s11228-024-00730-6 doi: 10.1007/s11228-024-00730-6
[8]	B. Tan, X. Qin, J. Yao, Strong convergence of self-adaptive inertial algorithms for solving split variational inclusion problems with applications, J. Sci. Comput., 87 (2021), 20. https://doi.org/10.1007/s10915-021-01428-9 doi: 10.1007/s10915-021-01428-9
[9]	P. Johnstone, J. Eckstein, Single-forward-step projective splitting: Exploiting cocoercivity, Comput. Optim. Appl., 78 (2021), 125–166. https://doi.org/10.1007/s10589-020-00238-3 doi: 10.1007/s10589-020-00238-3
[10]	P. Johnstone, J. Eckstein, Projective splitting with forward steps, Math. Program., 191 (2022), 631–670. https://doi.org/10.1007/s10107-020-01565-3 doi: 10.1007/s10107-020-01565-3
[11]	P. Lions, B. Mercier, Splitting algorithms for the sum of two nonlinear operators, SIAM J. Numer. Anal., 16 (1979), 964–979. https://doi.org/10.1137/0716071 doi: 10.1137/0716071
[12]	Y. Dong, A. Fischer, A family of operator splitting methods revisited, Nonlinear Anal., 72 (2010), 4307–4315. https://doi.org/10.1016/j.na.2010.02.010 doi: 10.1016/j.na.2010.02.010
[13]	Y. Dong, Douglas-Rachford splitting method for semi-definite programming, J. Appl. Math. Comput., 51 (2016), 569–591. https://doi.org/10.1007/s12190-015-0920-8 doi: 10.1007/s12190-015-0920-8
[14]	H. He, D. Han, A distributed Douglas-Rachford splitting method for multi-block convex minimization problems, Adv. Comput. Math., 42 (2016), 27–53. https://doi.org/10.1007/s10444-015-9408-1 doi: 10.1007/s10444-015-9408-1
[15]	Y. Dong, A new splitting method for systems of monotone inclusions in Hilbert spaces, Math. Comput. Simulat., 203 (2023), 518–537. https://doi.org/10.1016/j.matcom.2022.06.023 doi: 10.1016/j.matcom.2022.06.023
[16]	Y. Dong, X. Zhu, An inertial splitting method for monotone inclusions of three operators, Int. J. Math. Stat. Oper. Res., 2 (2022), 43–60. https://doi.org/10.47509/IJMSOR.2022.v02i01.04 doi: 10.47509/IJMSOR.2022.v02i01.04
[17]	J. Eckstein, A simplified form of block-iterative operator splitting and an asynchronous algorithm resembling the multi-block alternating direction method of multipliers, J. Optim. Theory Appl., 173 (2017), 155–182. https://doi.org/10.1007/s10957-017-1074-7 doi: 10.1007/s10957-017-1074-7
[18]	X. Zhu, Inertial splitting methods for monotone inclusions of three operators (Thesis), Zheng Zhou University, 2020.
[19]	Y. Dong, Extended splitting methods for systems of three-operator monotone inclusions with continuous operators, Math. Comput. Simulat., 223 (2024), 86–107. https://doi.org/10.1016/j.mat-com.2024.03.024 doi: 10.1016/j.mat-com.2024.03.024
[20]	X. Yan, J. Bien, Rare feature selection in high dimensions, J. Am. Stat. Assoc., 116 (2020), 887–900. https://doi.org/10.1080/01621459.2020.1796677 doi: 10.1080/01621459.2020.1796677
[21]	G. Minty, Monotone (nonlinear) operators in Hilbert space, Duke Math. J., 29 (1962), 341–346. https://doi.org/10.1215/S0012-7094-62-02933-2 doi: 10.1215/S0012-7094-62-02933-2
[22]	J. Eckstein, B. Svaiter, A family of projective splitting methods for the sum of two maximal monotone operators, Math. Program., 111 (2008), 173–199. https://doi.org/10.1007/s10107-006-0070-8 doi: 10.1007/s10107-006-0070-8
[23]	P. Latafat, P. Patrinos, Asymmetric forward-backward-adjoint splitting for solving monotone inclusions involving three operators, Comput. Optim. Appl., 68 (2017), 57–93. https://doi.org/10.1007/s10589-017-9909-6 doi: 10.1007/s10589-017-9909-6
[24]	P. Tseng, A modified forward-backward splitting method for maximal monotone mappings, SIAM J. Control Optim., 38 (2000), 431–446. https://doi.org/10.1137/S0363012998338806 doi: 10.1137/S0363012998338806
[25]	R. Rockafellar, Monotone operators and the proximal point algorithm, SIAM J. Control Optim., 14 (1976), 877–898. https://doi.org/10.1137/0314056 doi: 10.1137/0314056
[26]	Y. Dong, Weak convergence of an extended splitting method for monotone inclusions, J. Global Optim., 79 (2021), 257–277. https://doi.org/10.1007/s10898-020-00940-w doi: 10.1007/s10898-020-00940-w
[27]	Q. Dong, M. Su, Y. Shehu, Three-operator reflected forward-backward splitting algorithm with double inertial effects, Optim. Method. Softw., 39 (2024), 431–456. https://doi.org/10.1080/10556788.2024.2307470 doi: 10.1080/10556788.2024.2307470
[28]	K. Bredies, E. Chenchene, D. Lorenz, E. Naldi, Degenerate preconditioned proximal point algorithms, SIAM J. Optim., 32 (2022), 2376–2401. https://doi.org/10.1137/21M1448112 doi: 10.1137/21M1448112

Reader Comments

Your name:*

Email:*
© 2025 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)