Transfer learning for sparse linear discriminant analysis based on mean-variance similarity

Ping Wang; Xiaoyu Ma; Huanlin Li; Lu Lin; Ping Wang; Xiaoyu Ma; Huanlin Li; Lu Lin

doi:10.3934/math.20261352

AIMS Mathematics

2026, Volume 11, Issue 10: 34699-34729. doi: 10.3934/math.20261352

Previous Article Next Article

Research article

Transfer learning for sparse linear discriminant analysis based on mean-variance similarity

1.
Shandong Management University, Jinan, China
2.
National University of Defense Technology, Changsha, China
3.
Shandong University, Jinan, China

Received: 23 July 2026 Revised: 09 October 2026 Accepted: 15 October 2026 Published: 26 October 2026
MSC : 62H30, 62R07

We study transfer learning for high-dimensional linear discriminant analysis (LDA) with auxiliary data. To describe the differences between the target and source domains, we introduce a mean–variance similarity condition based on class-specific means and covariance structures. We first derive an exact source-domain moment relation and use it to construct a transfer learning estimator, called Trans-LPD. For a fixed set of auxiliary studies, we establish estimation and misclassification-risk bounds, which show the effects of the enlarged sample size and transfer bias. A data-driven adaptive procedure is then proposed to rank the auxiliary studies and select the final classifier using a target validation sample. The adaptive Trans-LPD classifier is shown to be asymptotically no worse than the target-only method. Simulation studies and an application to TCGA gene-expression data are also presented to illustrate the numerical performance of the proposed method.
- transfer learning,
- linear programming discriminant,
- mean-variance similarity,
- high-dimensional data,
- moment equation
Citation: Ping Wang, Xiaoyu Ma, Huanlin Li, Lu Lin. Transfer learning for sparse linear discriminant analysis based on mean-variance similarity[J]. AIMS Mathematics, 2026, 11(10): 34699-34729. doi: 10.3934/math.20261352

Related Papers:

Abstract

We study transfer learning for high-dimensional linear discriminant analysis (LDA) with auxiliary data. To describe the differences between the target and source domains, we introduce a mean–variance similarity condition based on class-specific means and covariance structures. We first derive an exact source-domain moment relation and use it to construct a transfer learning estimator, called Trans-LPD. For a fixed set of auxiliary studies, we establish estimation and misclassification-risk bounds, which show the effects of the enlarged sample size and transfer bias. A data-driven adaptive procedure is then proposed to rank the auxiliary studies and select the final classifier using a target validation sample. The adaptive Trans-LPD classifier is shown to be asymptotically no worse than the target-only method. Simulation studies and an application to TCGA gene-expression data are also presented to illustrate the numerical performance of the proposed method.

References

[1]	L. Torrey, J. Shavlik, Transfer learning, In: Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques, 2010,242–264. https://doi.org/10.4018/978-1-60566-766-9.ch011
[2]	L. Lin, W. Li, A Correlation-ratio transfer learning and variational Stein's paradox, arXiv preprint, 2022, arXiv: 2206.06086. https://doi.org/10.48550/arXiv.2206.06086
[3]	L. Lin, W. Li, Source-function weighted-transfer learning for nonparametric regression with seemingly similar sources, arXiv preprint, 2023, arXiv: 2302.11222. https://doi.org/10.48550/arXiv.2302.11222
[4]	S. Pan, Q. Yang, A survey on transfer learning, IEEE T. Knowl. Data Eng., 22 (2010), 1345–1359. https://doi.org/10.1109/TKDE.2009.191 doi: 10.1109/TKDE.2009.191
[5]	S. Mei, W. Fei, S. Zhou, Gene ontology based transfer learning for protein subcellular localization, BMC Bioinform., 12 (2011), 44. https://doi.org/10.1186/1471-2105-12-44 doi: 10.1186/1471-2105-12-44
[6]	H. C. Shin, Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning, IEEE T. Med. Imaging, 35 (2016), 1285–1298. https://doi.org/10.1109/TMI.2016.2528162 doi: 10.1109/TMI.2016.2528162
[7]	T. Turki, Z. Wei, J. Wang, Transfer learning approaches to improve drug sensitivity prediction in multiple myeloma patients. IEEE Access, 5 (2017), 7381–7393. https://doi.org/10.1109/ACCESS.2017.2696523
[8]	Y. Sun, Y. Hu, Integrative analysis of multi-omics data for discovery and functional studies of complex human diseases, Adv. Genet., 93 (2016), 147–190. https://doi.org/10.1016/bs.adgen.2015.11.004 doi: 10.1016/bs.adgen.2015.11.004
[9]	Y. Hu, M. Li, Q. Lu, H. Weng, J. Wang, S. M. Zekavat, et al., A statistical framework for cross-tissue transcriptomewide association analysis, Nat. Genet., 51 (2019), 568–576. https://doi.org/10.1038/s41588-019-0345-7 doi: 10.1038/s41588-019-0345-7
[10]	Z. Huang, Y. Song, Q. Zhang, Individualized treatment rules based on adaptive transfer-dragonnet, Stat. Comput., 35 (2025), 165. https://doi.org/10.1007/s11222-025-10704-9 doi: 10.1007/s11222-025-10704-9
[11]	H. Daumé III, Frustratingly easy domain adaptation, In: Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, 2007,256–263.
[12]	W. Pan, Q. Yang, Transfer learning in heterogeneous collaborative filtering domains, Artif. Intell., 197 (2013), 39–55. https://doi.org/10.1016/j.artint.2013.01.003 doi: 10.1016/j.artint.2013.01.003
[13]	T. Cai, H. Wei, Transfer learning for nonparametric classification: Minimax rate and adaptive classifier, Ann. Stat., 49 (2021), 100–128. https://doi.org/10.1214/20-AOS1949 doi: 10.1214/20-AOS1949
[14]	S. Li, T. Cai, R. Duan, Targeting underrepresented populations in precision medicine: A federated transfer learning approach, arXiv preprint, 2021. https://doi.org/10.48550/arXiv.2108.12112
[15]	S. Li, T. Cai, H. Li, Transfer learning for high-dimensional linear regression: Prediction, estimation, and minimax optimality, J. Roy. Stat. Soc. Ser. B, 84 (2022), 149–173. https://doi.org/10.1111/rssb.12479 doi: 10.1111/rssb.12479
[16]	S. Li, T. Cai, H. Li, Estimation and inference with proxy data and its genetic applications, arXiv preprint, 2022. https://doi.org/10.48550/arXiv.2201.03727
[17]	S. Li, T. Cai, H. Li, Transfer learning in large-scale graphical models with false discovery rate control, J. Am. Stat. Assoc., 118 (2023), 2171–2183. https://doi.org/10.1080/01621459.2022.2044333 doi: 10.1080/01621459.2022.2044333
[18]	T. Gu, Y. Han, R. Duan, Robust angle-based transfer learning in high dimensions, J. Roy. Stat. Soc. Ser., 87 (2025), 723–745. https://doi.org/10.1093/jrsssb/qkae111 doi: 10.1093/jrsssb/qkae111
[19]	Y. Tian, Y. Feng, Transfer learning under high-dimensional generalized linear models, J. Am. Stat. Assoc., 118 (2022), 2684–2697. https://doi.org/10.1080/01621459.2022.2071278 doi: 10.1080/01621459.2022.2071278
[20]	F. Sun, Q. Zhang, Robust transfer learning of high-dimensionalgeneralized linear model, Physica A, 618 (2023), 128674. https://doi.org/10.1016/j.physa.2023.128674 doi: 10.1016/j.physa.2023.128674
[21]	H. Lin, M. Reimherr, On transfer learning in functional linear regression, arXiv preprint, 2023. https://doi.org/10.48550/arXiv.2206.04277
[22]	M. Wu, L. Zhang, Z. Wang, D. Christiani, X. Lin, Sparse linear discriminant analysis for simultaneous testing for the significance of a gene set/pathway and gene selection, Bioinformatics, 25 (2009), 1145–1151. https://doi.org/10.1093/bioinformatics/btp019 doi: 10.1093/bioinformatics/btp019
[23]	D. Witten, R. Tibshirani, Covariance-regularized regression and classification for high dimensional problems, J. Roy. Stat. Soc. Ser. B, 71 (2009), 615–636. https://doi.org/10.1111/j.1467-9868.2009.00699.x doi: 10.1111/j.1467-9868.2009.00699.x
[24]	J. Shao, Y. Wang, X. Deng, S. Wang, Sparse linear discriminant analysis with high dimensional data, Ann. Stat., 39 (2011), 1241–1265. https://doi.org/10.1214/10-AOS870 doi: 10.1214/10-AOS870
[25]	T. Cai, W. Liu, A direct estimation approach to sparse linear discriminant analysis, J. Am. Stat. Assoc., 106 (2011), 1566–1577. https://doi.org/10.1198/jasa.2011.tm11199 doi: 10.1198/jasa.2011.tm11199
[26]	Q. Mai, H. Zou, M. Yuan, A direct approach to sparse discriminant analysis in ultrahigh dimensions, Biometrika, 99 (2012), 29–42. https://doi.org/10.1093/biomet/asr066 doi: 10.1093/biomet/asr066
[27]	J. Fan, Y. Feng, X. Tong, A road to classification in high dimensional space: the regularized optimal affine discriminant, J. Roy. Stat. Soc. Ser. B, 74 (2012), 745–771. https://doi.org/10.1111/j.1467-9868.2012.01029.x doi: 10.1111/j.1467-9868.2012.01029.x
[28]	M. Azizyan, A. Singh, L. Wasserman, Minimax theory for high-dimensional Gaussian mixtures with sparse mean separation, In: Proc. Advances in Neural Information Processing Systems, Lake Tahse, Dec. 5th-10th, 2013, 2139–2147.
[29]	T. Li, X. Yi, C. Carmanis, P. Ravikumar, Minimax Gaussian classification & clustering, In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, 2017, 1–9.
[30]	T. Cai, L. Zhang, High dimensional linear discriminant analysis: Optimality, adaptive algorithm and missing data, J. Roy. Stat. Soc. Ser. B, 81 (2019), 675–705. https://doi.org/10.1111/rssb.12326 doi: 10.1111/rssb.12326
[31]	S. Safo, E. Min, L. Haine, Sparse linear discriminant analysis for multiview structured data, Biometrics, 78 (2022), 612–623. https://doi.org/10.1111/biom.13458 doi: 10.1111/biom.13458
[32]	T. Anderson, An introduction to multivariate statistical analysis, 3 Eds., New Jersey: Wiley-Interscience, 2003, 1566–1568.
[33]	Y. Guo, T. Hastie, R. Tibshirani, Regularized linear discriminant analysis and its application in microarrays, Biostatistics, 8 (2007), 86–100. https://doi.org/10.1093/biostatistics/kxj035 doi: 10.1093/biostatistics/kxj035
[34]	A. Rothman, P. Bickel, E. Levina, J. Zhu, Sparse permutation invariant covariance estimation, Electron. J. Stat., 2 (2008), 494–515. https://doi.org/10.1214/08-EJS176 doi: 10.1214/08-EJS176
[35]	P. Hall, D. M. Titterington, J. H. Xue, Median-based classifiers for high-dimensional data, J. Am. Stat. Assoc., 104 (2009), 1597–1608. https://doi.org/10.1198/jasa.2009.tm08107 doi: 10.1198/jasa.2009.tm08107
[36]	J. Huang, A. J. Smola, A. Gretton, K. M. Borgwardt, B. Schölkopf, Correcting sample selection bias by unlabeled data, In: Proceedings of NIPS, 2006,601–608. https://doi.org/10.7551/mitpress/7503.003.0080
[37]	B. Quanz, J. Huan, Large margin transductive transfer learning, In: Proceedings of CIKM, 2009, 1327–1336. https://doi.org/10.1145/1645953.1646121
[38]	S. Hanneke, S. Kpotufe, A no-free-lunch theorem for multitask learning, arXiv preprint, 2020. https://doi.org/10.48550/arXiv.2006.15785
[39]	R. Glowinski, A. Marrocco, Sur l'D. Gabay and B. Mercierapproximation, paréléments finis d'ordre un, et la résolution, par pénalisation-dualité, d'une classe de problémes de Dirichlet non linéaires, Rev. Francaise D'automatique, Informatique, Recherche Opérationnelle, 2 (1975), 41–76.
[40]	D. Gabay, B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite element approximation, Comput. Math. Appl., 2 (1976), 17–40. https://doi.org/10.1016/0898-1221(76)90003-1 doi: 10.1016/0898-1221(76)90003-1
[41]	Z. Lu, T. Pong, Y. Zhang, An alternating direction method for finding Dantzig selectors, Comput. Stat. Data Anal., 56 (2012), 4037–4046. https://doi.org/10.1016/j.csda.2012.04.019 doi: 10.1016/j.csda.2012.04.019
[42]	X. Wang, X. Yuan, The linearized alternating direction method of multipliers for Dantzig selector, SIAM J. Sci. Comput., 34 (2012), 2792–2811. https://doi.org/10.1137/110833543 doi: 10.1137/110833543
[43]	L. Merchante, Y. Grandvalet, G. Govaert, An efficient approach to sparse linear discriminant analysis, In: Proceedings of the 29th International Conference on Machine Learning, 2012.
[44]	T. Hastie, A. Buja, R. Tibshirani, Penalized discriminant analysis, Ann. Stat., 23 (1995), 73–102. https://doi.org/10.1214/aos/1176324456 doi: 10.1214/aos/1176324456

Reader Comments

Your name:*

Email:*
© 2026 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)