A decision-level fusion hybrid deep learning framework for high-precision automated optical inspection of VCSEL semiconductor devices

Kyu-Jeong Choi; Jin-Taek Seong; Kyu-Jeong Choi; Jin-Taek Seong

doi:10.3934/math.2026594

AIMS Mathematics

2026, Volume 11, Issue 5: 14487-14521. doi: 10.3934/math.2026594

Previous Article Next Article

Research article Special Issues

A decision-level fusion hybrid deep learning framework for high-precision automated optical inspection of VCSEL semiconductor devices

Kyu-Jeong Choi ^1,2,
Jin-Taek Seong ^{1
,
,}

1.
Graduate School of Data Science, Chonnam National University, Gwangju 61186, Republic of Korea
2.
Opticis Company Ltd., Seongnam-si 13354, Republic of Korea

Received: 20 March 2026 Revised: 14 April 2026 Accepted: 24 April 2026 Published: 22 May 2026
MSC : 68T07, 68T45

This paper presents a hybrid deep learning framework for the automated optical inspection (AOI) of vertical cavity surface-emitting laser (VCSEL) semiconductor devices. Manual inspection is limited by subjective inconsistency and operator fatigue, while high-end commercial AOI systems impose substantial costs that are often impractical for small and medium sized manufacturers. The proposed system adopts a decision-level fusion architecture in which defect-specific binary classifiers are aggregated through an OR-gate. This design prioritizes recall so that any defect flagged by at least one sub-classifier triggers rejection, reducing the risk of defect escape. An industrial dataset of 22, 410 images with severe class imbalance (e.g., crack defects comprising less than 0.4% of all labels) was used for training, with targeted augmentation applied to minority classes. Five fold cross-validation yielded an overall accuracy of 98.7% and an F1-score of 93.5%. Deployment on an active production line reduced per-unit inspection time from 17.7 s to 1.5 s (an 89% reduction) and recorded a secondary defect escape rate of 0.00%.
- automated optical inspection,
- vertical cavity surface-emitting laser,
- decision-level fusion,
- hybrid deep learning,
- surface defect detection
Citation: Kyu-Jeong Choi, Jin-Taek Seong. A decision-level fusion hybrid deep learning framework for high-precision automated optical inspection of VCSEL semiconductor devices[J]. AIMS Mathematics, 2026, 11(5): 14487-14521. doi: 10.3934/math.2026594

Related Papers:

Abstract

This paper presents a hybrid deep learning framework for the automated optical inspection (AOI) of vertical cavity surface-emitting laser (VCSEL) semiconductor devices. Manual inspection is limited by subjective inconsistency and operator fatigue, while high-end commercial AOI systems impose substantial costs that are often impractical for small and medium sized manufacturers. The proposed system adopts a decision-level fusion architecture in which defect-specific binary classifiers are aggregated through an OR-gate. This design prioritizes recall so that any defect flagged by at least one sub-classifier triggers rejection, reducing the risk of defect escape. An industrial dataset of 22, 410 images with severe class imbalance (e.g., crack defects comprising less than 0.4% of all labels) was used for training, with targeted augmentation applied to minority classes. Five fold cross-validation yielded an overall accuracy of 98.7% and an F1-score of 93.5%. Deployment on an active production line reduced per-unit inspection time from 17.7 s to 1.5 s (an 89% reduction) and recorded a secondary defect escape rate of 0.00%.

References

[1]	S. Sundaram, A. Zeid, Artificial intelligence-based smart quality inspection for manufacturing, Micromachines, 14 (2023), 570. https://doi.org/10.3390/mi14030570 doi: 10.3390/mi14030570
[2]	3D automated optical inspection (AOI) equipment market, Market growth reports, 2025. Available from: https://www.marketgrowthreports.com/market-reports/3d-automated-optical-inspection-aoi-equipment-market-100380.
[3]	A. Adrian, AOI equipment: unveiling the hidden costs that impact your ROI, AllPCB Blog, 2025. Available from: https://www.allpcb.com/blog/pcb-assembly/aoi-equipment-unveiling-the-hidden-costs-that-impact-your-roi.html.
[4]	Z. H. Ren, F. Z. Fang, N. Yan, Y. Wu, State of the art in defect detection based on machine vision, Int. J. Precis. Eng. Manuf.-Green Tech., 9 (2022), 661–691. https://doi.org/10.1007/s40684-021-00343-6 doi: 10.1007/s40684-021-00343-6
[5]	J.-H. Park, Y.-S. Kim, H. Seo, Y.-J. Cho, Analysis of training deep learning models for PCB defect detection, Sensors, 23 (2023), 2766. https://doi.org/10.3390/s23052766 doi: 10.3390/s23052766
[6]	D. Tabernik, S. Šela, J. Skvarč, D. Skočaj, Segmentation-based deep-learning approach for surface-defect detection, J. Intell. Manuf., 31 (2020), 759–776. https://doi.org/10.1007/s10845-019-01476-x doi: 10.1007/s10845-019-01476-x
[7]	S. B. Jha, R. Babiceanu, P. Shekhar, S. Namilae, Deep convolutional neural network-based automated optical inspection for aerospace components, Digital Engineering, 7 (2025), 100062. https://doi.org/10.1016/j.dte.2025.100062 doi: 10.1016/j.dte.2025.100062
[8]	J. Terven, D.-M. Córdova-Esparza, J.-A. Romero-González, A comprehensive review of yolo architectures in computer vision: From yolov1 to yolov8 and yolo-nas, Mach. Learn. Knowl. Extr., 5 (2023), 1680–1716. https://doi.org/10.3390/make5040083 doi: 10.3390/make5040083
[9]	W. Y. Lv, Y. A. Zhao, Q. Y. Chang, K. Huang, G. Z. Wang, Y. Liu, Rt-detrv2: Improved baseline with bag-of-freebies for real-time detection transformer, 2024, arXiv: 2407.17140.
[10]	V. De Ridder, B. Dey, S. Halder, B. Van Waeyenberge, Semi-diffusioninst: A diffusion model based approach for semiconductor defect classification and segmentation, 2023 International Symposium ELMAR, Zadar, Croatia, 2023, 61–66. https://doi.org/10.1109/ELMAR59410.2023.10253920
[11]	F. Mohammad, D. Ryu, Semiconductor wafer map defect classification with tiny vision transformers, 2025, arXiv: 2504.02494.
[12]	I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, et al., Generative adversarial networks, Commun. ACM, 63 (2020), 139–144. https://doi.org/10.1145/3422622 doi: 10.1145/3422622
[13]	J. Redmon, A. Farhadi, YOLOv3: An incremental improvement, 2018, arXiv: 1804.02767. https://doi.org/10.48550/arXiv.1804.02767
[14]	K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun, Deep residual learning for image recognition, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016,770–778. https://doi.org/10.1109/CVPR.2016.90
[15]	A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. H. Zhai, T. Unterthiner, et al., An image is worth 16x16 words: Transformers for image recognition at scale, 2021, arXiv: 2010.11929.
[16]	X. B. Wang, S. Gao, Y. T. Zou, J. L. Guo, C. Wang, IH-ViT: Vision transformer-based integrated circuit appear-ance defect detection, 2023, arXiv: 2302.04521.
[17]	A. Krizhevsky, I. Sutskever, G. E. Hinton, ImageNet classification with deep convolutional neural networks, Commun. ACM, 60 (2017), 84–90. https://doi.org/10.1145/3065386 doi: 10.1145/3065386
[18]	W. Ullah, S. U. Khan, M. J. Kim, A. Hussain, M. Munsif, M. Y. Lee, et al., Industrial defective chips detection using deep convolutional neural network with inverse feature matching mechanism, J. Comput. Des. Eng., 11 (2024), 326–336. https://doi.org/10.1093/jcde/qwae019 doi: 10.1093/jcde/qwae019
[19]	W. J. Tang, Y. J. Deng, X. Luo, RST-YOLOv8: An improved chip surface defect detection model based on YOLOv8, Sensors, 25 (2025), 3859. https://doi.org/10.3390/s25133859 doi: 10.3390/s25133859
[20]	J. J. Hwang, H. W. Lee, Y. S. Park, Multi-sensor and decision-level fusion-based structural damage detection using 1-D CNN, Sensors, 21 (2021), 3950. https://doi.org/10.3390/s21123950 doi: 10.3390/s21123950
[21]	N. G. Yu, H. Z. Li, Q. Xu, A full-flow inspection method based on machine vision to detect wafer surface defects, Math. Biosci. Eng., 20 (2023), 11821–11846, https://doi.org/10.3934/mbe.2023526 doi: 10.3934/mbe.2023526
[22]	Y. C. Du, J. P. Chen, H. Zhou, X. L. Yang, Z. Q. Wang, J. Zhang, et al., An automated optical inspection (AOI) platform for three-dimensional (3D) defects detection on glass micro-optical components (GMOC), Opt. Commun., 545 (2023), 129736. https://doi.org/10.1016/j.optcom.2023.129736 doi: 10.1016/j.optcom.2023.129736
[23]	H. Fu, Y. M. Lai, C. R. Pan, S. W. Zhang, L. P. Bai, J. Li, A central array method to locate chips in AOI systems in semiconductor manufacturing, Electronics, 13 (2024), 1070. https://doi.org/10.3390/electronics13061070 doi: 10.3390/electronics13061070
[24]	T.-Y. Kim, S. Park, C.-K. Lim, J.-H. Choi, C.-H. An, L.-H. Song, et al., Deep learning-based detection of defects in wafer buffer zone during semiconductor packaging process, Multiscale Sci. Eng., 6 (2024), 25–32. https://doi.org/10.1007/s42493-024-00103-z doi: 10.1007/s42493-024-00103-z
[25]	A. B. Goti, Automated optical inspection (AOI) based on IPC standards, International Journal of Engineering and Computer Science, 13 (2025), 26928–26947.
[26]	C.-J. Fu, H.-L. Chen, H.-Y. Tseng, Application of artificial intelligence to improve chip defect detection using semiconductor equipment, Eng. Proc., 98 (2025), 26. https://doi.org/10.3390/engproc2025098026 doi: 10.3390/engproc2025098026
[27]	J. M. Johnson, L. M. Khoshgoftaar, Survey on deep learning with class imbalance, J. Big Data, 6 (2019), 27. https://doi.org/10.1186/s40537-019-0192-5 doi: 10.1186/s40537-019-0192-5
[28]	M. S. Shelke, P. R. Deshmukh, V. K. Shandilya, A review on imbalanced data handling using undersampling and oversampling technique, International Journal of Recent Trends in Engineering and Research, 3 (2017), 444–449.
[29]	N. Sarafianos, X. Xu, I. A. Kakadiaris, Deep imbalanced attribute classification using visual attention aggregation, In: Computer vision–ECCV 2018, Cham: Springer, 2018,708–725. https://doi.org/10.1007/978-3-030-01252-6_42
[30]	J. Byrd, Z. C. Lipton, What is the effect of importance weighting in deep learning, Proceedings of the 36th International Conference on Machine Learning, Long Beach, California, 2019,872–881.
[31]	N. V. Chawla, K. W. Bowyer, L. O. Hall, W. P. Kegelmeyer, SMOTE: Synthetic minority over-sampling technique, J. Artif. Intell. Res., 16 (2002), 321–357. https://doi.org/10.1613/jair.953 doi: 10.1613/jair.953
[32]	E. Goceri, Medical image data augmentation: techniques, comparisons and interpretations, J. Artif. Intell. Rev., 56 (2023), 12561–12605. https://doi.org/10.1007/s10462-023-10453-z doi: 10.1007/s10462-023-10453-z
[33]	P. Y. Simard, D. Steinkraus, J. C. Platt, Best practices for convolutional neural networks applied to visual document analysis, Seventh International Conference on Document Analysis and Recognition, Edinburgh, UK, 2003,958–963. https://doi.org/10.1109/ICDAR.2003.1227801
[34]	S. S. Lee, Noisy replication in skewed binary classification, Comput. Stat. Data An., 34 (2000), 165–191. https://doi.org/10.1016/S0167-9473(99)00095-X doi: 10.1016/S0167-9473(99)00095-X
[35]	S. Wang, Evaluation of impact of image augmentation techniques on two tasks: Window detection and window states detection, Results in Engineering, 24 (2024), 103571. https://doi.org/10.1016/j.rineng.2024.103571 doi: 10.1016/j.rineng.2024.103571
[36]	M. Buda, A. Maki, M. A. Mazurowski, A systematic study of the class imbalance problem in convolutional neural networks, Neural Networks, 106 (2018), 249–259. https://doi.org/10.1016/j.neunet.2018.07.011 doi: 10.1016/j.neunet.2018.07.011
[37]	A. More, Survey of resampling techniques for improving classification performance in unbalanced datasets, 2023, arXiv: 1608.06048.
[38]	C. Elkan, The foundations of cost-sensitive learning, The 17th International Joint Conference on Artificial Intelligence, Seattle, WA, USA, 2001,973–978.
[39]	K. M. Ting, An instance-weighting method to induce cost-sensitive trees, IEEE T. Knowl. Data En., 14 (2002), 659–665. https://doi.org/10.1109/TKDE.2002.1000348 doi: 10.1109/TKDE.2002.1000348
[40]	A. K. Menon, S. Jayasumana, A. S. Rawat, H. Jain, A. Veit, S. Kumar, Long-tail learning via logit adjustment, 2021, arXiv: 2007.07314.
[41]	H. Cho, W. Koo, H. Kim, Prediction of highly imbalanced semiconductor chip-level defects in module tests using multimodal fusion and logit adjustment, IEEE T. Semiconduct. M., 36 (2023), 425–433. https://doi.org/10.1109/TSM.2023.3283101 doi: 10.1109/TSM.2023.3283101
[42]	V. Vasan, N. V. Sridharan, R. J. Balasundaram, S. Vaithiyanathan, Ensemble-based deep learning model for welding defect detection and classification, Eng. Appl. Artif. Intel., 136 (2024), 108961. https://doi.org/10.1016/j.engappai.2024.108961 doi: 10.1016/j.engappai.2024.108961
[43]	J. Choi, D. Suh, M.-O. Otto, Boosted stacking ensemble machine learning method for wafer map pattern classification, CMC-Comput. Mater. Con., 74 (2022), 2945–2966. https://doi.org/10.32604/cmc.2023.033417 doi: 10.32604/cmc.2023.033417
[44]	J. Bai, D. Wu, T. Shelley, P. Schubel, D. Twine, J. Russell, et al., A comprehensive survey on machine learning driven material defect detection, ACM Comput. Surv., 57 (2025), 1–36. https://doi.org/10.1145/3730576 doi: 10.1145/3730576
[45]	K. Fan, P. Peng, H. P. Zhou, L. L. Wang, Z. Y. Guo, Real-time high-performance laser welding defect detection by combining acgan-based data enhancement and multimodel fusion, Sensors, 21 (2021), 7304. https://doi.org/10.3390/s21217304 doi: 10.3390/s21217304
[46]	O. Ronneberger, P. Fischer, T. Brox, U-Net: Convolutional networks for biomedical image segmentation, Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015, Cham: Springer, 2015,234–241. https://doi.org/10.1007/978-3-319-24574-4_28
[47]	C. Cortes, V. Vapnik, Support-vector networks, Mach. Learn., 20 (1995), 273–297. https://doi.org/10.1007/BF00994018 doi: 10.1007/BF00994018
[48]	T. M. Cover, P. E. Hart, Nearest neighbor pattern classification, IEEE T. Inform. Theory, 13 (1967), 21–27. https://doi.org/10.1109/TIT.1967.1053964 doi: 10.1109/TIT.1967.1053964
[49]	D. R. Cox, The regression analysis of binary sequences, J. R. Stat. Soc. B, 20 (1958), 215–232. https://doi.org/10.1111/j.2517-6161.1958.tb00292.x doi: 10.1111/j.2517-6161.1958.tb00292.x
[50]	L. Breiman, Random forests, Mach. Learn., 45 (2001), 5–32. https://doi.org/10.1023/A:1010933404324 doi: 10.1023/A:1010933404324
[51]	J. H. Friedman, Greedy function approximation: A gradient boosting machine, Ann. Statist., 29 (2001), 1189–1232. https://doi.org/10.1214/aos/1013203451 doi: 10.1214/aos/1013203451

Reader Comments

Your name:*

Email:*
© 2026 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)