A nonsmooth, nonconvex optimization approach over sphere constraints for Variants of regularized CCA and SVD

Amir Beck; Raz Sharon; Amir Beck; Raz Sharon

doi:10.3934/jimo.2026084

Journal of Industrial and Management Optimization

2026, Volume 22, Issue 5: 2301-2318. doi: 10.3934/jimo.2026084

Previous Article Next Article

Research article Special Issues

A nonsmooth, nonconvex optimization approach over sphere constraints for Variants of regularized CCA and SVD

Amir Beck ^,,
Raz Sharon

School of Mathematical Sciences, Tel Aviv University, Tel Aviv 69978, Israel

Received: 08 December 2025 Revised: 08 February 2026 Accepted: 20 March 2026 Published: 13 April 2026
90C26, 90C30

In this paper, we study a general nonsmooth, nonconvex optimization problem over a cross product of spheres, suitable for various well-known variants of the canonical correlation analysis (CCA) and singular value decomposition (SVD) problems that promote sparsity and smoothness. We propose an alternating minimization algorithm for a smooth approximation of this problem and study its rate of convergence. Numerical experiments demonstrate the potential of the suggested method.
- sparse CCA,
- sparse SVD,
- nonsmooth nonconvex optimization,
- alternating minimization,
- sphere constraints,
- smooth approximation,
- manifold optimization
Citation: Amir Beck, Raz Sharon. A nonsmooth, nonconvex optimization approach over sphere constraints for Variants of regularized CCA and SVD[J]. Journal of Industrial and Management Optimization, 2026, 22(5): 2301-2318. doi: 10.3934/jimo.2026084

Related Papers:

Abstract

In this paper, we study a general nonsmooth, nonconvex optimization problem over a cross product of spheres, suitable for various well-known variants of the canonical correlation analysis (CCA) and singular value decomposition (SVD) problems that promote sparsity and smoothness. We propose an alternating minimization algorithm for a smooth approximation of this problem and study its rate of convergence. Numerical experiments demonstrate the potential of the suggested method.

References

[1]	D. M. Witten, R. Tibshirani, T. Hastie, A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis, Biostatistics, 10 (2009), 515–534. https://doi.org/10.1093/biostatistics/kxp008 doi: 10.1093/biostatistics/kxp008
[2]	X. Suo, V. Minden, B. Nelson, R. Tibshirani, M. Saunders, Sparse canonical correlation analysis, arXiv preprint arXiv: 1705.10865, (2017). https://doi.org/10.48550/arXiv.1705.10865
[3]	Q. Mai, X. Zhang, An iterative penalized least squares approach to sparse canonical correlation analysis, Biometrics, 75 (2019), 734–744. https://doi.org/10.1111/biom.13043 doi: 10.1111/biom.13043
[4]	M. Lee, H. Shen, J. Z. Huang, J. S. Marron, Biclustering via sparse singular value decomposition, Biometrics, 66 (2010), 1087–1095. https://doi.org/10.1111/j.1541-0420.2010.01392.x doi: 10.1111/j.1541-0420.2010.01392.x
[5]	Z. Hong, H. Lian, Sparse-smooth regularized singular value decomposition, J. Multivariate Anal., 117 (2013), 163–174. https://doi.org/10.1016/j.jmva.2013.02.011 doi: 10.1016/j.jmva.2013.02.011
[6]	D. R. Hardoon, S. Szedmak, J. Shawe-Taylor, Canonical correlation analysis: An overview with application to learning methods, Neural Comput., 16 (2004), 2639–2664. https://doi.org/10.1162/0899766042321814 doi: 10.1162/0899766042321814
[7]	D. P. Bertsekas, J. N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods, Prentice Hall, 1989.
[8]	J. M. Ortega, W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970.
[9]	S. J. Wright, Coordinate descent algorithms, Math. Program., 151 (2015), 3–34. https://doi.org/10.1007/s10107-015-0892-3 doi: 10.1007/s10107-015-0892-3
[10]	A. Beck, M. Teboulle, Smoothing and first order methods: a unified framework, SIAM J. Optim., 22 (2012), 557–580. https://doi.org/10.1137/100818327 doi: 10.1137/100818327
[11]	I. Daubechies, R. DeVore, M. Fornasier, C. S. Güntürk, Iteratively reweighted least squares minimization for sparse recovery, Comm. Pure Appl. Math., 63 (2010), 1–38. https://doi.org/10.1002/cpa.20303 doi: 10.1002/cpa.20303
[12]	A. Beck, On the convergence of alternating minimization for convex programming with applications to iteratively reweighted least squares and decomposition schemes, SIAM J. Optim., 25 (2015), 185–209. https://doi.org/10.1137/13094829X doi: 10.1137/13094829X
[13]	A. Beck, Quadratic matrix programming, SIAM J. Optim., 17 (2007), 1224–1238. https://doi.org/10.1137/05064816X doi: 10.1137/05064816X
[14]	G. Pataki, The geometry of semidefinite programming, Handbook of Semidefinite Programming: Theory, Algorithms, and Applications, Kluwer Academic Publishers, 2000, 29–65. https://doi.org/10.1007/978-1-4615-4381-7_3
[15]	Y. Ye, S. Zhang, New results on quadratic minimization, SIAM J. Optim., 14 (2003), 245–267. https://doi.org/10.1137/S105262340139001X doi: 10.1137/S105262340139001X
[16]	P. A. Absil, R. Mahony, R. Sepulchre, Optimization Algorithms on Matrix Manifolds, Princeton University Press, 2008. https://doi.org/10.1515/9781400830244
[17]	N. Boumal, An Introduction to Optimization on Smooth Manifolds, Cambridge University Press, 2023. https://doi.org/10.1017/9781009166164
[18]	A. Beck, First-Order Methods in Optimization, SIAM, 2017. https://doi.org/10.1137/1.9781611974997
[19]	C. Helmberg, F. Rendl, R. J. Vanderbei, H. Wolkowicz, An interior-point method for semidefinite programming, SIAM J. Optim., 6 (1996), 342–361. https://doi.org/10.1137/0806020 doi: 10.1137/0806020

Reader Comments

Your name:*

Email:*
© 2026 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)