Layer-wise FCFL: Federated learning with scaling-factor-based layer-wise fuzzy clustering

Milin Zhao; Guangwei Xu; Jianbo Lu; Haopu Lv; Wenxin Zhao; Peng Zhang; Liwei Li; Yang Lu; Milin Zhao; Guangwei Xu; Jianbo Lu; Haopu Lv; Wenxin Zhao; Peng Zhang; Liwei Li; Yang Lu

doi:10.3934/era.2026335

Electronic Research Archive

2026, Volume 34, Issue 10: 7811-7832. doi: 10.3934/era.2026335

Previous Article Next Article

Research article Special Issues

Layer-wise FCFL: Federated learning with scaling-factor-based layer-wise fuzzy clustering

1.
School of Computer Science and Technology, Shandong University, Shandong Provincial Key Laboratory of Computing-Network Integration, Shandong University, Qingdao, China
2.
MCC Capital Engineering & Research Incorporation Limited, Beijing, China
3.
School of Robotics and Automation, Nanjing University, Suzhou, China
^† These authors contributed equally to this work

Received: 03 June 2026 Revised: 09 August 2026 Accepted: 20 August 2026 Published: 10 September 2026

Federated learning is a distributed learning paradigm in which clients collaboratively train a shared model through local computation and server-side aggregation of model updates, without sharing raw data. However, under non-independent and identically distributed data settings, the aggregation performance of conventional FL often degrades significantly. Although clustered FL is an effective approach to mitigating data heterogeneity, existing methods still exhibit several limitations, including high computational overhead from full-model parameter clustering, insufficient characterization of layer-wise difference under monolithic whole-model clustering and aggregation, and inadequate information utilization caused by hard clustering. To address these issues, we proposed a new layer-wise fuzzy clustered FL paradigm that uses the scaling factors of batch normalization layers as local data representations. Low-dimensional BN scaling factors were used to replace full-model parameters for lightweight clustering. Then, we introduced a layer-wise independent clustering and aggregation mechanism that performs client clustering and model aggregation separately for different network layers to better capture layer-specific heterogeneity. Finally, fuzzy C-means (FCM) was applied at each layer to enable fuzzy clustering, allowing each client to belong to multiple clusters and enabling membership-weighted aggregation for improved information utilization. Extensive experiments on the MNIST, EMNIST, CIFAR-10, and CIFAR-100 datasets with SIMPLE-CNN and VGG16 demonstrated the effectiveness of the proposed approach over several state-of-the-art clustered FL baselines, with up to 7.10% gains in test accuracy over the strongest baseline. Further ablation studies showed that both the proposed fuzzy clustering mechanism and layer-wise aggregation strategy provide significant benefits, and their combination achieved the best overall performance.
- federated learning,
- data heterogeneity,
- fuzzy clustering,
- layer-wise clustering,
- batch normalization
Citation: Milin Zhao, Guangwei Xu, Jianbo Lu, Haopu Lv, Wenxin Zhao, Peng Zhang, Liwei Li, Yang Lu. Layer-wise FCFL: Federated learning with scaling-factor-based layer-wise fuzzy clustering[J]. Electronic Research Archive, 2026, 34(10): 7811-7832. doi: 10.3934/era.2026335

Related Papers:

Abstract

Federated learning is a distributed learning paradigm in which clients collaboratively train a shared model through local computation and server-side aggregation of model updates, without sharing raw data. However, under non-independent and identically distributed data settings, the aggregation performance of conventional FL often degrades significantly. Although clustered FL is an effective approach to mitigating data heterogeneity, existing methods still exhibit several limitations, including high computational overhead from full-model parameter clustering, insufficient characterization of layer-wise difference under monolithic whole-model clustering and aggregation, and inadequate information utilization caused by hard clustering. To address these issues, we proposed a new layer-wise fuzzy clustered FL paradigm that uses the scaling factors of batch normalization layers as local data representations. Low-dimensional BN scaling factors were used to replace full-model parameters for lightweight clustering. Then, we introduced a layer-wise independent clustering and aggregation mechanism that performs client clustering and model aggregation separately for different network layers to better capture layer-specific heterogeneity. Finally, fuzzy C-means (FCM) was applied at each layer to enable fuzzy clustering, allowing each client to belong to multiple clusters and enabling membership-weighted aggregation for improved information utilization. Extensive experiments on the MNIST, EMNIST, CIFAR-10, and CIFAR-100 datasets with SIMPLE-CNN and VGG16 demonstrated the effectiveness of the proposed approach over several state-of-the-art clustered FL baselines, with up to 7.10% gains in test accuracy over the strongest baseline. Further ablation studies showed that both the proposed fuzzy clustering mechanism and layer-wise aggregation strategy provide significant benefits, and their combination achieved the best overall performance.

References

[1]	Intersoft Consulting, General Data Protection Regulation (GDPR), 2018. Available from: https://gdpr-info.eu/.
[2]	N. Rieke, J. Hancox, W. Li, F. Milletari, H. R. Roth, S. Albarqouni, et al., The future of digital health with federated learning, npj Digital Med., 3 (2020), 119. https://doi.org/10.1038/s41746-020-00323-1 doi: 10.1038/s41746-020-00323-1
[3]	P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, et al., Advances and open problems in federated learning, Found. Trends Mach. Learn., 14 (2021), 1–210. https://doi.org/10.1561/2200000083 doi: 10.1561/2200000083
[4]	B. McMahan, E. Moore, D. Ramage, S. Hampson, B. A. y Arcas, Communication-efficient learning of deep networks from decentralized data, in Artificial Intelligence and Statistics, (2017), 1273–1282.
[5]	A. Hard, K. Rao, R. Mathews, S. Ramaswamy, F. Beaufays, S. Augenstein, et al., Federated learning for mobile keyboard prediction, preprint, arXiv: 1811.03604.
[6]	Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, V. Chandra, Federated learning with non-iid data, preprint, arXiv: 1806.00582.
[7]	F. Sattler, K. Müller, W. Samek, Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints, IEEE Trans. Neural Networks Learn. Syst., 32 (2020), 3710–3722. https://doi.org/10.1109/TNNLS.2020.3015958 doi: 10.1109/TNNLS.2020.3015958
[8]	A. Ghosh, J. Chung, D. Yin, K. Ramchandran, An efficient framework for clustered federated learning, in Advances in Neural Information Processing Systems, 33 (2020), 19586–19597.
[9]	T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, V. Smith, Federated optimization in heterogeneous networks, in Proceedings of Machine Learning and Systems, 2 (2020), 429–450.
[10]	M. G. Arivazhagan, V. Aggarwal, A. K. Singh, S. Choudhary, Federated learning with personalization layers, preprint, arXiv: 1912.00818.
[11]	T. Li, S. Hu, A. Beirami, V. Smith, Ditto: Fair and robust federated learning through personalization, in International Conference on Machine Learning, (2021), 6357–6368.
[12]	Z. Zhu, J. Hong, J. Zhou, Data-free knowledge distillation for heterogeneous federated learning, in International Conference on Machine Learning, (2021), 12878–12889.
[13]	Y. Jiang, S. Wang, V. Valls, B. J. Ko, W. Lee, K. K. Leung, et al., Pfa: Privacy-preserving federated adaptation for effective model personalization, in Proceedings of the 2021 ACM SIGCOMM Workshop on Network Meets AI & ML (NetAI), (2021), 61–67.
[14]	Y. Kim, E. Al Hakim, J. Haraldson, H. Eriksson, J. M. B. da Silva, C. Fischione, Dynamic clustering in federated learning, in ICC 2021-IEEE International Conference on Communications, IEEE, (2021), 1–6. https://doi.org/10.1109/ICC42927.2021.9500877
[15]	B. Liu, Y. Cai, Z. Zhang, Y. Li, L. Wang, D. Li, et al., Distfl: Distribution-aware federated learning for mobile scenarios, Proc. ACM Interact. Mobile Wearable Ubiquitous Technol., 5 (2021), 1–26. https://doi.org/10.1145/3494966 doi: 10.1145/3494966
[16]	E. Yoo, H. Ko, S. Pack, Fuzzy clustered federated learning algorithm for solar power generation forecasting, IEEE Trans. Emerging Top. Comput., 10 (2022), 2092–2098. https://doi.org/10.1109/TETC.2022.3142886 doi: 10.1109/TETC.2022.3142886
[17]	C. Li, G. Li, P. K. Varshney, Federated learning with soft clustering, IEEE Internet Things J., 9 (2022), 7773–7782. https://doi.org/10.1109/JIOT.2021.3113927 doi: 10.1109/JIOT.2021.3113927
[18]	Y. Deng, A. Wang, L. Zhang, Y. Lei, B. Li, Y. Li, FedRFC: Federated learning with recursive fuzzy clustering for improved non-iid data training, Future Gener. Comput. Syst., 160 (2024), 835–843. https://doi.org/10.1016/j.future.2024.06.049 doi: 10.1016/j.future.2024.06.049
[19]	Z. Liu, J. Li, Z. Shen, G. Huang, S. Yan, C. Zhang, Learning efficient convolutional networks through network slimming, in 2017 IEEE International Conference on Computer Vision (ICCV), (2017), 2755–2763.
[20]	D. Bau, B. Zhou, A. Khosla, A. Oliva, A. Torralba, Network dissection: Quantifying interpretability of deep visual representations, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017), 6541–6549.
[21]	M. D. Zeiler, R. Fergus, Visualizing and understanding convolutional networks, in European Conference on Computer Vision, Springer International Publishing, Cham, (2014), 818–833. https://doi.org/10.1007/978-3-319-10590-1_53
[22]	I. S. Dhillon, D. S. Modha, Concept decompositions for large sparse text data using clustering, Mach. Learn., 42 (2001), 143–175. https://doi.org/10.1023/A:1007612920971 doi: 10.1023/A:1007612920971
[23]	J. C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithms, Springer Science & Business Media, 2013.
[24]	R. L. Thorndike, Who belongs in the family, Psychometrika, 18 (1953), 267–276.
[25]	P. J. Rousseeuw, Silhouettes: A graphical aid to the interpretation and validation of cluster analysis, J. Comput. Appl. Math., 20 (1987), 53–65. https://doi.org/10.1016/0377-0427(87)90125-7 doi: 10.1016/0377-0427(87)90125-7
[26]	J. A. Hartigan, M. A. Wong, Algorithm as 136: A k-means clustering algorithm, J. R. Stat. Soc. C, 28 (1979), 100–108. https://doi.org/10.2307/2346830 doi: 10.2307/2346830
[27]	J. F. Kolen, T. Hutcheson, Reducing the time complexity of the fuzzy c-means algorithm, IEEE Trans. Fuzzy Syst., 10 (2002), 263–267. https://doi.org/10.1109/91.995126 doi: 10.1109/91.995126
[28]	D. Fenoglio, M. Li, P. Barbiero, N. Lane, M. Langheinrich, M. Gjoreski, Flux: Efficient descriptor-driven clustered federated learning under arbitrary distribution shifts, in Advances in Neural Information Processing Systems, 38 (2026), 71229–71292.

Reader Comments

Your name:*

Email:*
© 2026 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)