Generalization analysis of tuning-free, Markov-ensemble SVM with distributed applications

Hongwei Jiang; Yujing Yang; Bin Zou; Jie Xu; Hongwei Jiang; Yujing Yang; Bin Zou; Jie Xu

doi:10.3934/math.2026564

AIMS Mathematics

2026, Volume 11, Issue 5: 13683-13709. doi: 10.3934/math.2026564

Previous Article Next Article

Research article

Generalization analysis of tuning-free, Markov-ensemble SVM with distributed applications

1.
School of Science, Shenyang University of Technology, Shenyang 110870, China
2.
Faculty of Mathematics and Statistics, Hubei Key Laboratory of Applied Mathematics, Hubei University, Wuhan 430062, China
3.
Faculty of Computer Science, Hubei University, Wuhan 430062, China

Received: 30 January 2026 Revised: 20 April 2026 Accepted: 28 April 2026 Published: 15 May 2026
MSC : 65C40, 68Q32

Although support vector machine (SVM) is an important algorithm, known hyperparameter tuning methods are typically time-consuming and susceptible to the influence of noise samples in large datasets. In particular, the selection of SVM hyperparameters is especially challenging in distributed learning. Therefore, this paper proposed a novel linear kernel SVM based on non-hyperparameter tuning. Its core idea was to train multiple SVM models using different regularization hyperparameters, and then integrate them into a final SVM model. To further increase the diversity of the resulting SVM models, Markov sampling was employed to generate different training subsets prior to training each SVM model. This paper derived the SVM based on non-hyperparameter tuning (SNHT) algorithm and proved its consistency. As an application, SNHT was applied to distributed learning. The performance of SNHT was validated through experiments on benchmark datasets.
- SVM,
- non-hyperparameter tuning,
- Markov sampling,
- generalization bounds,
- distributed learning
Citation: Hongwei Jiang, Yujing Yang, Bin Zou, Jie Xu. Generalization analysis of tuning-free, Markov-ensemble SVM with distributed applications[J]. AIMS Mathematics, 2026, 11(5): 13683-13709. doi: 10.3934/math.2026564

Related Papers:

Abstract

Although support vector machine (SVM) is an important algorithm, known hyperparameter tuning methods are typically time-consuming and susceptible to the influence of noise samples in large datasets. In particular, the selection of SVM hyperparameters is especially challenging in distributed learning. Therefore, this paper proposed a novel linear kernel SVM based on non-hyperparameter tuning. Its core idea was to train multiple SVM models using different regularization hyperparameters, and then integrate them into a final SVM model. To further increase the diversity of the resulting SVM models, Markov sampling was employed to generate different training subsets prior to training each SVM model. This paper derived the SVM based on non-hyperparameter tuning (SNHT) algorithm and proved its consistency. As an application, SNHT was applied to distributed learning. The performance of SNHT was validated through experiments on benchmark datasets.

References

[1]	V. N. Vapnik, Statistical learning theory, New York: Wiley, 1998.
[2]	H. G$\ddot{\mathrm{u}}$ney, A fast-optimizing and adaptable intrusion detection system based on progressively optimized support vector machines, Concurr. Comput. Pract. Exp., 37 (2025), e70156. https://doi.org/10.1002/cpe.70156 doi: 10.1002/cpe.70156
[3]	K. Ramu, S. Patthi, Y. N. Prajapati, J. V. N. Ramesh, S. Banerjee, K. B. V. Rao, et al., Hybrid CNN-SVM model for enhanced early detection of Chronic kidney disease, Biomed. Signal Process. Control, 100 (2025), 107084. https://doi.org/10.1016/j.bspc.2024.107084 doi: 10.1016/j.bspc.2024.107084
[4]	J. K. Myilvahanan, N. M. Sundaram, Support vector machine-based stock market prediction using long short-term memory and convolutional neural network with aquila circle inspired optimization, Netw. Comput. Neural Syst., 36 (2025), 1185–1220. https://doi.org/10.1080/0954898X.2024.2358957 doi: 10.1080/0954898X.2024.2358957
[5]	G. H. Huang, T. G. Xue, W. H. Chen, L. L. Huang, Q. Dai, J. Y. Jiang, SVM-LncRNAPro: An SVM-based method for predicting long noncoding RNA promoters, IET Syst. Biol, 19 (2025), e70013. https://doi.org/10.1049/syb2.70013 doi: 10.1049/syb2.70013
[6]	C. Singh, N. Jain, N. Adlakha, K. R. Pardasani, Type-2 fuzzy support vector machine model for conformational epitope prediction, Netw. Model. Anal. Health Inform. Bioinform., 14 (2025), 4. https://doi.org/10.1007/s13721-024-00498-7 doi: 10.1007/s13721-024-00498-7
[7]	H. J. Lin, H. S. H. Chung, C. X. Lin, D. Xie, Q. L. Deng, M. C. Lyu, et al., Improved fault diagnosis capability in CHBMCs: Counter design for multiple OC switches via an E-SVM unit, IEEE Trans. Power Electron., 41 (2026), 2358–2376. https://doi.org/10.1109/TPEL.2025.3618228 doi: 10.1109/TPEL.2025.3618228
[8]	A. Singhal, Seema, A. M. Saeed, R. Tiwari, A. Chaudhary, Hybrid fractional thermoelastic-machine learning framework for heat and mass transfer in skin tissue: Enhanced simulations using Atangana-Baleanu, Cattaneo-Vernotte models, and KNN-SVM classifiers, Int. Commun. Heat Mass Transf., 171 (2026), 110074. https://doi.org/10.1016/j.icheatmasstransfer.2025.110074 doi: 10.1016/j.icheatmasstransfer.2025.110074
[9]	T. Zhang, Statistical behavior and consistency of classification methods based on convex risk minimization, Ann. Statist., 32 (2004), 56–85. https://doi.org/10.1214/aos/1079120130 doi: 10.1214/aos/1079120130
[10]	I. Steinwart, Consistency of support vector machines and other regularized kernel classifiers, IEEE Trans. Inf. Theory, 51 (2005), 128–142. https://doi.org/10.1109/TIT.2004.839514 doi: 10.1109/TIT.2004.839514
[11]	D. R. Chen, Q. Wu, Y. M. Ying, D. X. Zhou, Support vector machine soft margin classifiers: Error analysis, J. Mach. Learn. Res., 5 (2004), 1143–1175.
[12]	I. Steinwart, A. Christmann, Fast learning from non-i.i.d. observations, In: Proceedings of the 22nd international conference on neural information processing systems, 2009, 1768–1776.
[13]	J. Xu, Y. Y. Tang, B. Zou, Z. B. Xu, L. Q. Li, Y. Lu, B. et al., The generalization ability of SVM classification based on Markov sampling, IEEE Trans. Cybern., 45 (2015), 1169–1179. https://doi.org/10.1109/TCYB.2014.2346536
[14]	M. Kafai, K. Eshghi, CROification: Accurate kernel classification with the efficiency of sparse linear SVM, IEEE Trans. Pattern Anal. Mach. Intell., 41 (2019), 34–48. https://doi.org/10.1109/TPAMI.2017.2785313 doi: 10.1109/TPAMI.2017.2785313
[15]	Y. Kong, Y. Fu, Max-margin action prediction machine, IEEE Trans. Pattern Anal. Mach. Intell., 38 (2016), 1844–1858. https://doi.org/10.1109/TPAMI.2015.2491928 doi: 10.1109/TPAMI.2015.2491928
[16]	Y. W. Chang, C. J. Hsieh, K. W. Chang, M. Ringgaard, C. J. Lin, Training and testing low-degree polynomial data mappings via linear SVM, J. Mach. Learn. Res., 11 (2010), 1471–1490.
[17]	S. Litayem, A. Joly, N. Boujemaa, Hash-based support vector machines approximation for large scale prediction, In: Proceedings of the British machine vision conference, 2012, 86.1–86.11. https://doi.org/10.5244/C.26.86
[18]	X. Q. Jiao, H. Lian, J. M. Liu, Y. Y. Zhang, Linear convergence of proximal gradient method for linear sparse SVM, Neural Netw., 194 (2026), 108162. https://doi.org/10.1016/j.neunet.2025.108162 doi: 10.1016/j.neunet.2025.108162
[19]	C. R. Rao, Y. Wu, Liner model selection by cross-validation, J. Stat. Plan. Infer., 128 (2005), 231–240. https://doi.org/10.1016/j.jspi.2003.10.004 doi: 10.1016/j.jspi.2003.10.004
[20]	H. Li, Q. X. Huang, C. Wang, An early warning model for student status based on genetic algorithm-optimized radial basis kernel support vector machine, J. Inf. Process. Syst., 20 (2024), 263–272. https://doi.org/10.3745/JIPS.02.0213 doi: 10.3745/JIPS.02.0213
[21]	D. Anguita, A. Ghio, L. Oneto, S. Ridella, In-sample model selection for support vector machines, In: The 2011 international joint conference on neural networks, 2011, 1154–1161. https://doi.org/10.1109/IJCNN.2011.6033354
[22]	Y. X. Hu, H. T. Zhang, Chaos optimization method of SVM parameters selection for chaotic time series forecasting, Phys. Procedia, 25 (2012), 588–594. https://doi.org/10.1016/j.phpro.2012.03.130 doi: 10.1016/j.phpro.2012.03.130
[23]	A. Popov, A. Sautin, Selection of support vector machines parameters for regression using nested grids, In: 2008 Third international forum on strategic technologies, 2008,329–331. https://doi.org/10.1109/IFOST.2008.4602974
[24]	Z. Zhai, B. Gu, C. Deng, H. Huang, Global model selection via solution paths for robust support vector machine, IEEE Trans. Pattern Anal. Mach. Intell., 47 (2025), 1331–1347. https://doi.org/10.1109/TPAMI.2023.3346765 doi: 10.1109/TPAMI.2023.3346765
[25]	J. C. Shao, X. N. Zhou, Q. K. Shao, H. L. Chen, B. J. Pan, A novel lymph node metastasis prediction method for gastric cancer: Enhanced support vector machine with polar lights optimization, Biomed. Signal Process. Control, 111 (2026), 108349. https://doi.org/10.1016/j.bspc.2025.108349 doi: 10.1016/j.bspc.2025.108349
[26]	Y. L. Yuan, G. Y. Chong, J. J. Ren, W. Zhao, Y. A. Li, Z. X. Wang, et al., Musk ox optimizer (MO): A novel optimization algorithm and its application. Cluster Comput., 28 (2025), 1041. https://doi.org/10.1007/s10586-025-05735-w doi: 10.1007/s10586-025-05735-w
[27]	J. J. Zeng, Y. Duan, D. Wang, B. Zou, Y. Yin, J. Xu, Generalization performance of Lagrangian support vector machine based on Markov sampling, J. Stat. Plan. Infer., 214 (2021), 89–104. https://doi.org/10.1016/j.jspi.2020.09.001 doi: 10.1016/j.jspi.2020.09.001
[28]	Z. Wang, T. Liang, B. Zou, Y. L. Cai, J. Xu, X. G. You, Incremental Fisher linear discriminant based on data denoising, Knowl.-Based Syst., 237 (2022), 107799. https://doi.org/10.1016/j.knosys.2021.107799 doi: 10.1016/j.knosys.2021.107799
[29]	S. Geman, D. Geman, Stochastic relaxation, Gibbs distribution and the Bayesian restoration of images, IEEE Trans. Pattern Anal. Mach. Intell., 6 (1984), 721–741. https://doi.org/10.1109/TPAMI.1984.4767596 doi: 10.1109/TPAMI.1984.4767596
[30]	R. A. Fisher, The use of multiple measures in taxonomic problems, Ann. Eugen., 7 (1936), 179–188. https://doi.org/10.1111/j.1469-1809.1936.tb02137.x doi: 10.1111/j.1469-1809.1936.tb02137.x
[31]	F. Cucker, S. Smale, On the mathematical foundations of learning, Bull. Amer. Math. Soc., 39 (2002), 1–49. https://doi.org/10.1090/S0273-0979-01-00923-5 doi: 10.1090/S0273-0979-01-00923-5
[32]	Q. Wu, Y. M. Ying, D. X. Zhou, Learning rates of least-square regularized regression, Found. Comput. Math., 6 (2006), 171–192. https://doi.org/10.1007/s10208-004-0155-9 doi: 10.1007/s10208-004-0155-9
[33]	D. X. Zhou, Capacity of reproducing kernel spaces in learning theory, IEEE Trans. Inf. Theory, 49 (2003), 1743–1752. https://doi.org/10.1109/TIT.2003.813564 doi: 10.1109/TIT.2003.813564
[34]	L. Devroye, L. Gy$\ddot{\mathrm{o}}$rfi, G. Lugosi, A probabilistic theory of pattern recognition, New York: Springer, 1996. https://doi.org/10.1007/978-1-4612-0711-5
[35]	H. Z. Tong, D. R. Chen, L. Z. Peng, Analysis of support vector machine regression, Found. Comput. Math., 9 (2009), 243–257. https://doi.org/10.1007/s10208-008-9026-0 doi: 10.1007/s10208-008-9026-0
[36]	I. Steinwart, C. Scovel, Fast rates for support vector machines using Gaussian kernels, Ann. Statist., 35 (2007), 575–607. https://doi.org/10.1214/009053606000001226 doi: 10.1214/009053606000001226
[37]	G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, et al., LightGBM: A highly efficient gradient boosting decision tree, In: Proceedings of the 31st international conference on neural information processing systems, 2017, 3149–3157.
[38]	D. E. Rumelhart, G. E. Hinton, R. J. Williams, Learning representations by back-propagating errors, Nature, 323 (1986), 533–536. https://doi.org/10.1038/323533a0 doi: 10.1038/323533a0
[39]	L. Breiman, Random Forests, Mach. Learn., 45 (2001), 5–32. https://doi.org/10.1023/A:1010933404324 doi: 10.1023/A:1010933404324
[40]	F. Wilcoxon, Individual comparisons by ranking methods, Biom. Bull., 1 (1945), 80–83. https://doi.org/10.2307/3001968 doi: 10.2307/3001968

Reader Comments

Your name:*

Email:*
© 2026 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)