An alternating minimization algorithm for sparse convolutive non-negative matrix factorization with $ \ell_1 $-norm

Yijia Zhou; Yijia Zhou

doi:10.3934/era.2025346

Electronic Research Archive

2025, Volume 33, Issue 12: 7841-7865. doi: 10.3934/era.2025346

Previous Article Next Article

Research article Special Issues

An alternating minimization algorithm for sparse convolutive non-negative matrix factorization with $ \ell_1 $-norm

Yijia Zhou ^,

Dalian Neusoft University of Information, Dalian 116086, China

Received: 23 July 2025 Revised: 15 November 2025 Accepted: 05 December 2025 Published: 25 December 2025

Convolutive non-negative matrix factorization has been a dominant analytical technique for deriving interpretable insights from data in speech processing, image analysis, data mining, biomedicine, and other fields. In this paper, a sparse convolutive non-negative matrix factorization model was introduced by incorporating an $ \ell_1 $ regularization on representation matrices. This enhancement not only preserved the inherent characteristics of convolutive non-negative matrix factorization, but also promoted sparse data representation, thereby facilitating more efficient data storage and analysis. An alternating minimization algorithm for the presented model was proposed by integrating the alternating direction method of multipliers with the accelerated iterative shrinkage-thresholding algorithm. In addition, a convergence result was presented that the convergence point of the algorithm necessarily constitutes a stable point of the problem. Experimental results showed that the proposed algorithm yielded sparser solutions for synthetic data designed to simulate sparse representation scenarios, and achieved practical applicability in speech dataset, validating its potential for real-world signal processing tasks.
Citation: Yijia Zhou. An alternating minimization algorithm for sparse convolutive non-negative matrix factorization with $ \ell_1 $-norm[J]. Electronic Research Archive, 2025, 33(12): 7841-7865. doi: 10.3934/era.2025346

Related Papers:

Abstract

Convolutive non-negative matrix factorization has been a dominant analytical technique for deriving interpretable insights from data in speech processing, image analysis, data mining, biomedicine, and other fields. In this paper, a sparse convolutive non-negative matrix factorization model was introduced by incorporating an $ \ell_1 $ regularization on representation matrices. This enhancement not only preserved the inherent characteristics of convolutive non-negative matrix factorization, but also promoted sparse data representation, thereby facilitating more efficient data storage and analysis. An alternating minimization algorithm for the presented model was proposed by integrating the alternating direction method of multipliers with the accelerated iterative shrinkage-thresholding algorithm. In addition, a convergence result was presented that the convergence point of the algorithm necessarily constitutes a stable point of the problem. Experimental results showed that the proposed algorithm yielded sparser solutions for synthetic data designed to simulate sparse representation scenarios, and achieved practical applicability in speech dataset, validating its potential for real-world signal processing tasks.

References

[1]	M. S. Bartlett, J. R. Movellan, T. J. Sejnowski, Face recognition by independent component analysis, IEEE Trans. Neural Netw., 13 (2002), 1450–1464. https://doi.org/10.1109/TNN.2002.804287 doi: 10.1109/TNN.2002.804287
[2]	K. Guo, X. Xu, F. Qiu, J. Chen, A novel incremental weighted PCA algorithm for visual tracking, in 2013 IEEE International Conference on Image Processing, IEEE, (2013), 3914–3918. https://doi.org/10.1109/ICIP.2013.6738806
[3]	J. Gan, T. Liu, L. Li, J. Zhang, Nonnegative matrix factorization: A survey, Comput. J., 64 (2021), 1080–1092. https://doi.org/10.1093/comjnl/bxab103 doi: 10.1093/comjnl/bxab103
[4]	F. Dargahi, S. Babaie-Kafaki, Z. Aminifard, M. Hajian-Berenjestanaki, Solving an augmented nonnegative matrix factorization model by modified scaled nonmonotone memoryless BFGS methods devised based on the ellipsoid vector norm, Math. Methods Appl. Sci., 48 (2025), 9088–9097. https://doi.org/10.1002/mma.10781 doi: 10.1002/mma.10781
[5]	V. Y. F. Tan, C. Fevotte, Automatic relevance determination in non-negative matrix factorization with the beta-divergence, IEEE Trans. Pattern Anal. Mach. Intell., 35 (2012), 1592–1605. https://doi.org/10.1109/TPAMI.2012.240 doi: 10.1109/TPAMI.2012.240
[6]	D. Lee, H. Seung, Learning the parts of objects by non-negative matrix factorization, Nature, 401 (1999), 788–791. https://doi.org/10.1038/44565 doi: 10.1038/44565
[7]	E. Gonzalez, Y. Zhang, Accelerating the lee-seung algorithm for nonnegative matrix factorization, Dept. Comput. Appl. Math., Rice Univ., Houston, TX, Tech. Rep. TR-05-02, 2005 (2005), 1–13.
[8]	P. O. Hoyer, Non-negative sparse coding, in Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing, IEEE, (2002), 557–565. https://doi.org/10.1109/NNSP.2002.1030067
[9]	P. O. Hoyer, Non-negative matrix factorization with sparseness constraints, preprint, arXiv: cs/0408058.
[10]	R. Peharz, F. Pernkopf, Sparse nonnegative matrix factorization with $\ell_0$-constraints, Neurocomputing, 80 (2012), 38–46. https://doi.org/10.1016/j.neucom.2011.09.024 doi: 10.1016/j.neucom.2011.09.024
[11]	J. Yang, Y. Zhang, Alternating direction algorithms for $\ell_1$-problems in compressive sensing, SIAM J. Sci. Comput., 33 (2011), 250–278. https://doi.org/10.1137/090777761 doi: 10.1137/090777761
[12]	M. Gong, X. Jiang, H. Li, K. C. Tan, Multiobjective sparse non-negative matrix factorization, IEEE Trans. Cybern., 49 (2019), 2941–2954. https://doi.org/10.1109/TCYB.2018.2834898 doi: 10.1109/TCYB.2018.2834898
[13]	Q. M. Pham, D. Lachmund, D. N. Hao, Convergence of proximal algorithms with stepsize controls for non-linear inverse problems and application to sparse non-negative matrix factorization, Numer. Algor., 85 (2020), 1255–1279. https://doi.org/10.1007/s11075-019-00864-x doi: 10.1007/s11075-019-00864-x
[14]	M. S. Islam, Y. Zhu, M. I. Hossain, R. Ullah, Z. Ye, Supervised single channel dual domains speech enhancement using sparse non-negative matrix factorization, Digital Signal Process., 100 (2020), 102697. https://doi.org/10.1016/j.dsp.2020.102697 doi: 10.1016/j.dsp.2020.102697
[15]	P. Deng, T. Li, H. Wang, D. Wang, S. Horng, R. Liu, Graph regularized sparse non-negative matrix factorization for clustering, IEEE Trans. Comput. Soc. Syst., 10 (2023), 910–921. https://doi.org/10.1109/TCSS.2022.3154030 doi: 10.1109/TCSS.2022.3154030
[16]	W. Min, T. Xu, X. Wan, T. H. Chang, Structured sparse non-negative matrix factorization with $\ell_{2, 0}-$norm, IEEE Trans. Knowl. Data Eng., 35 (2023), 8584–8595. https://doi.org/10.1109/TKDE.2022.3206881 doi: 10.1109/TKDE.2022.3206881
[17]	W. Wang, J. Meng, H. Li, J. Fan, Non-negative matrix factorization for overlapping community detection in directed weighted networks with sparse constraints, Chaos, 33 (2023), 053111. https://doi.org/10.1063/5.0152280 doi: 10.1063/5.0152280
[18]	A. Degleris, N. Gillis, A provably correct and robust algorithm for convolutive nonnegative matrix factorization, IEEE Trans. Signal Process., 68 (2020), 2499–2512. https://doi.org/10.1109/TSP.2020.2984163 doi: 10.1109/TSP.2020.2984163
[19]	Y. Li, X. Zhang, M. Sun, Y. Hu, L. Li, Online convolutive non-negative bases learning for speech enhancement, IEICE Trans. Fundam. Electron. Commun. Comput. Sci., E99.A (2016), 1609–1613. https://doi.org/10.1587/transfun.E99.A.1609 doi: 10.1587/transfun.E99.A.1609
[20]	D. Fagot, H. Wendt, C. Fevotte, P. Smaragdis, Majorization-minimization algorithms for convolutive NMF with the beta-divergence, in 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, (2019), 8202–8206. https://doi.org/10.1109/ICASSP.2019.8683837
[21]	M. Sun, Y. Li, J. F. Gemmeke, X. Zhang, Speech enhancement under low SNR conditions via noise estimation using sparse and low-rank NMF with Kullback-Leibler divergence, IEEE/ACM Trans. Audio Speech Lang. Process., 23 (2015), 1233–1242. https://doi.org/10.1109/TASLP.2015.2427520 doi: 10.1109/TASLP.2015.2427520
[22]	E. Mackevicius, A. Bahle, A. Williams, S. Gu, N. Denisenko, M. Goldman, et al., Unsupervised discovery of temporal sequences in high-dimensional datasets with applications to neuroscience, eLife, 8 (2019), e38471. https://doi.org/10.7554/eLife.38471 doi: 10.7554/eLife.38471
[23]	P. D. O'Grady, B. A. Pearlmutter, Discovering speech phones using convolutive non-negative matrix factorization with a sparseness constraint, Neurocomputing, 72 (2008), 88–101. https://doi.org/10.1016/j.neucom.2008.01.033 doi: 10.1016/j.neucom.2008.01.033
[24]	P. D. O'Grady, B. A. Pearlmutter, Convolutive non-negative matrix factorization with a sparseness constraint, in 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing, IEEE, (2006), 427–432. https://doi.org/10.1109/MLSP.2006.275588
[25]	L. Zhang, C. Jia, X. Zhang, G. Min, L. Zeng, Speech enhancement based on convolutive nonnegative matrix factorization with sparseness constraints (in Chinese), J. Data Acquis. Process., 29 (2014), 259–264. https://doi.org/10.16337/j.1004-9037.2014.02.016 doi: 10.16337/j.1004-9037.2014.02.016
[26]	C. Lu, M. Tian, J. Zhou, H. Wang, L. Tao, A single-channel speech enhancement approach using convolutive non-negative matrix factorization with $L_{1/2}$ sparse constraint (in Chinese), Acta Acust., 42 (2017), 377–384. https://doi.org/10.15949/j.cnki.0371-0025.2017.03.016 doi: 10.15949/j.cnki.0371-0025.2017.03.016
[27]	F. Wang, H. Ke, H. Ma, Y. Tang, Deep wavelet temporal-frequency attention for nonlinear fMRI factorization in ASD, Pattern Recogn., 165 (2025), 111543. https://doi.org/10.1016/j.patcog.2025.111543 doi: 10.1016/j.patcog.2025.111543
[28]	F. Wang, H. Ke, C. Cai, Deep wavelet self-attention non-negative tensor factorization for non-linear analysis and classification of fMRI data, Appl. Soft Comput., 182 (2025), 113522. https://doi.org/10.1016/j.asoc.2025.113522 doi: 10.1016/j.asoc.2025.113522
[29]	F. Wang, H. Ke, Y. Tang, Fusion of generative adversarial networks and non-negative tensor decomposition for depression fMRI data analysis, Inf. Process. Manage., 62 (2025), 103961. https://doi.org/10.1016/j.ipm.2024.103961 doi: 10.1016/j.ipm.2024.103961
[30]	H. Ke, F. Wang, H. Bi, H. Ma, G. Wang, B. Yin, Unsupervised deep frequency-channel attention factorization to non-linear feature extraction: A case study of identification and functional connectivity interpretation of Parkinson's disease, Expert Syst. Appl., 243 (2024), 122853. https://doi.org/10.1016/j.eswa.2023.122853 doi: 10.1016/j.eswa.2023.122853
[31]	H. Ke, D. Chen, Q. Yao, Y. Tang, J. Wu, J. Monaghan, et al., Deep factor learning for accurate brain neuroimaging data analysis on discrimination for structural MRI and functional MRI, IEEE/ACM Trans. Comput. Biol. Bioinform., 21 (2024), 582–595. https://doi.org/10.1109/TCBB.2023.3252577 doi: 10.1109/TCBB.2023.3252577
[32]	X. Fu, K. Huang, N. D. Sidiropoulos, W. K. Ma, Nonnegative matrix factorization for signal and data analytics identifiability algorithms and application, IEEE Signal Process. Mag., 36 (2019), 59–80. https://doi.org/10.1109/MSP.2018.2877582 doi: 10.1109/MSP.2018.2877582
[33]	P. Smaragdis, Convolutive speech bases and their application to supervised speech separation, IEEE Trans. Audio Speech Lang. Process., 15 (2007), 1–12. https://doi.org/10.1109/TASL.2006.876726 doi: 10.1109/TASL.2006.876726
[34]	P. Smaragdis, Non-negative matrix factor deconvolution; extraction of multiple sound sources from monophonic inputs, in Independent Component Analysis and Blind Signal Separation, Springer, (2004), 494–499. https://doi.org/10.1007/978-3-540-30110-3_63
[35]	Y. Mai, L. Lan, N. Guan, X. Zhang, Z. Luo, Transductive convolutive non-negative matrix factorization for speech separation, in 2015 4th International Conference on Computer Science and Network Technology (ICCSNT), IEEE, (2015), 1400–1404. https://doi.org/10.1109/ICCSNT.2015.7490990
[36]	D. D. Lee, H. S. Seung, Algorithms for non-negative matrix factorization, in Proceedings of the 14th International Conference on Neural Information Processing Systems, MIT Press, (2001), 535–541.
[37]	C. J. Lin, On the convergence of multiplicative update algorithms for nonnegative matrix factorization, IEEE Trans. Neural Networks, 18 (2007), 1589–1596. https://doi.org/10.1109/TNN.2007.895831 doi: 10.1109/TNN.2007.895831
[38]	P. Paatero, U. Tapper, Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values, Environmetrics, 5 (1994), 111–126. https://doi.org/10.1002/env.3170050203 doi: 10.1002/env.3170050203
[39]	Y. Zhang, An alternating direction algorithm for nonnegative matrix factorization, in Technical Report TR10-03, CAAM, Rice University, (2010), 1–14.
[40]	D. L. Sun, C. Fevotte, Alternating direction method of multipliers for non-negative matrix factorization with the beta-divergence, in 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, (2014), 6201–6205. https://doi.org/10.1109/ICASSP.2014.6854796
[41]	W. Deng, W. Yin, On the global and linear convergence of the generalized alternating direction method of multipliers, J. Sci. Comput., 66 (2016), 889–916. https://doi.org/10.1007/s10915-015-0048-x doi: 10.1007/s10915-015-0048-x
[42]	R. Glowinski, A. Marroco, Sur lapproximation, par elements finis dordre un, et la resolution, par penalisation-dualite dune classe de problemes de dirichlet non lineaires, Rev. Fr. Autom. Inf. Rech. Oper. Anal. Numer., 9 (1975), 41–76.
[43]	D. Gabay, B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite element approximation, Comput. Math. Appl., 2 (1976), 17–40. https://doi.org/10.1016/0898-1221(76)90003-1 doi: 10.1016/0898-1221(76)90003-1
[44]	Y. Li, R. Wang, Y. Fang, M. Sun, Z. Luo, Alternating direction method of multipliers for convolutive non-negative matrix factorization, IEEE Trans. Cybern., 53 (2023), 7735–7748. https://doi.org/10.1109/TCYB.2022.3204723 doi: 10.1109/TCYB.2022.3204723
[45]	T. Xie, H. Zhang, R. Liu, H. Xiao, Accelerated sparse non-negative matrix factorization for unsupervised feature learning, Pattern Recognit. Lett., 156 (2022), 46–52. https://doi.org/10.1016/j.patrec.2022.01.020 doi: 10.1016/j.patrec.2022.01.020
[46]	Y. Y. Xu, W. T. Yin, Z. W. Wen, Y. Zhang, An alternating direction algorithm for matrix completion with nonnegative factors, Front. Math. China, 7 (2012), 365–384. https://doi.org/10.1007/s11464-012-0194-5 doi: 10.1007/s11464-012-0194-5
[47]	J. S. Garofolo, L. F. Lamel, W. M. Fisher, J. G. Fiscus, D. S. Pallett, N. L. Dahlgren, et al., TIMIT acoustic-phonetic continuous speech corpus LDC93S1, Ling. Data Consortium, 1993. https://doi.org/10.35111/17gk-bn40

Reader Comments

Your name:*

Email:*
© 2025 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)