Analysis and comparison for image colorization with machine learning based on PyTorch and ChromaGAN

Jinyi Luo; Xi Li; Jinyi Luo; Xi Li

doi:10.3934/era.2025241

Electronic Research Archive

2025, Volume 33, Issue 9: 5377-5400. doi: 10.3934/era.2025241

Previous Article Next Article

Research article Special Issues

Analysis and comparison for image colorization with machine learning based on PyTorch and ChromaGAN

Jinyi Luo ,
Xi Li ^,

School of Mathematical Sciences, Chengdu University of Technology, Sichuan 610059, China

Received: 28 July 2025 Revised: 29 August 2025 Accepted: 02 September 2025 Published: 12 September 2025

Colorization of grayscale images has found widespread applications across artistic, historical, scientific, medical, and industrial domains. Traditional manual colorization methods, however, are labor-intensive, time-consuming, and prone to subjective interpretation. In recent years, various deep learning (DL) approaches have been explored to automate the colorization process. Despite this progress, the efficacy and accuracy of these DL methods remain largely unexamined. One of the primary challenges is the accurate handling of color boundaries, which significantly affects the realism and visual clarity of the colorized output. In this study, we investigated the impact of different DL algorithms and training epochs on the quality of colorized images, comparing them to their corresponding ground truths. To explore these effects, a semantic-guided adversarial generative framework was employed, and five commonly used DL algorithms were compared.
- image colorization,
- deep learning,
- ChromaGAN
Citation: Jinyi Luo, Xi Li. Analysis and comparison for image colorization with machine learning based on PyTorch and ChromaGAN[J]. Electronic Research Archive, 2025, 33(9): 5377-5400. doi: 10.3934/era.2025241

Related Papers:

Abstract

Colorization of grayscale images has found widespread applications across artistic, historical, scientific, medical, and industrial domains. Traditional manual colorization methods, however, are labor-intensive, time-consuming, and prone to subjective interpretation. In recent years, various deep learning (DL) approaches have been explored to automate the colorization process. Despite this progress, the efficacy and accuracy of these DL methods remain largely unexamined. One of the primary challenges is the accurate handling of color boundaries, which significantly affects the realism and visual clarity of the colorized output. In this study, we investigated the impact of different DL algorithms and training epochs on the quality of colorized images, comparing them to their corresponding ground truths. To explore these effects, a semantic-guided adversarial generative framework was employed, and five commonly used DL algorithms were compared.

References

[1]	S. Huang, X. Jin, Q. Jiang, L. Liu, Deep learning for image colorization: Current and future prospects, Eng. Appl. Artif. Intell. , 114 (2022), 105006. https://doi.org/10.1016/j.engappai.2022.105006 doi: 10.1016/j.engappai.2022.105006
[2]	P. Vitoria, L. Raad, C. Ballester, Chromagan: Adversarial picture colorization with semantic class distribution, in 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), (2020), 2434–2443. https://doi.org/10.1109/WACV45572.2020.9093389
[3]	A. Levin, D. Lischinski, Y. Weiss, Colorization using optimization, ACM Trans. Graphics, 23 (2004), 689–694. https://doi.org/10.1145/1015706.1015780 doi: 10.1145/1015706.1015780
[4]	Y. C. Huang, Y. S. Tung, J. C. Chen, S. W. Wang, J. L. Wu, An adaptive edge detection based colorization algorithm and its applications, in Proceedings of the 13th annual ACM International Conference on Multimedia, (2005), 351–354. https://doi.org/10.1145/1101149.1101223
[5]	L. Yatziv, G. Sapiro, Fast image and video colorization using chrominance blending, IEEE Trans. Image Process. , 15 (2006), 1120–1129. https://doi.org/10.1109/TIP.2005.864231 doi: 10.1109/TIP.2005.864231
[6]	Q. Luan, F. Wen, D. Cohen-Or, L. Liang, Q. Xu, Y. Shum, Natural image colorization, in Proceedings of the 18th Eurographics Conference on Rendering Techniques, (2007), 309–320.
[7]	T. Welsh, M. Ashikhmin, K. Mueller, Transferring color to greyscale images, ACM Trans. Graphics, 21 (2002), 277–280. https://doi.org/10.1145/566654.566576 doi: 10.1145/566654.566576
[8]	R. Ironi, D. Cohen-Or, and D. Lischinski. Colorization by example, in Rendering Techniques, (2005), 201–210.
[9]	Y. W. Tai, J. Jia, K. Tang, Local color transfer via probabilistic segmentation by expectation-maximization, in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), 1 (2005), 747–754. https://doi.org/10.1109/CVPR.2005.215
[10]	G. Charpiat, M. Hofmann, B. Scholkopf, Automatic image colorization via multimodal predictions, in Computer Vision – ECCV 2008, (2008), 126–139. https://doi.org/10.1007/978-3-540-88690-7_10
[11]	A. Y. Chia, S. Zhuo, R. K. Gupta, Y. W. Tai, S. Y. Cho, P. Tan, et al., Semantic colorization with internet images, ACM Trans. Graphics, 30 (2011), 1–8. https://doi.org/10.1145/2070781.2024190 doi: 10.1145/2070781.2024190
[12]	R. K. Gupta, A. Y. Chia, D. Rajan, E. S. Ng, Y. Huang, Image colorization using similar images, in Proceedings of the 20th ACM International Conference on Multimedia, (2012), 369–378. https://doi.org/10.1145/2393347.2393402
[13]	Z. Cheng, Q. Yang, B. Sheng, Deep colorization, in Proceedings of the IEEE International Conference on Computer Vision, (2015), 415–423. https://doi.org/10.1109/ICCV.2015.55
[14]	S. Iizuka, E. Simo-Serra, H. Ishikawa, Let there be color!: joint end-to-end learning of global and local image priors for automatic image colorization with simultaneous classification, ACM Trans. Graphics, 35 (2016), 110. https://doi.org/10.1145/2897824.2925974 doi: 10.1145/2897824.2925974
[15]	R. Zhang, P. Isola, A. A. Efros, Colorful image colorization, in Computer Vision – ECCV 2016, (2016), 649–666. https://doi.org/10.1007/978-3-319-46487-9_40
[16]	M. H. Noaman, H. Khaled, H. M. Faheem, Image colorization: A survey of methodologies and techniques, in Proceedings of the International Conference on Advanced Intelligent Systems and Informatics 2021, (2021), 115–130. https://doi.org/10.1007/978-3-030-89701-7_11
[17]	S. Anwar, M. Tahir, C. Li, A. Mian, F. S. Khan, A. W. Muzaffar, Image colorization: A survey and dataset, Inf. Fusion, 114 (2025), 102720. https://doi.org/10.1016/j.inffus.2024.102720 doi: 10.1016/j.inffus.2024.102720
[18]	R. Zhang, J. Y. Zhu, P. Isola, X. Geng, S. Lin, T. Yu, et al., Real-time user-guided image colorization with learned deep priors, ACM Trans. Graphics, 36 (2017), 119. https://doi.org/10.1145/3072959.3073703 doi: 10.1145/3072959.3073703
[19]	M. He, D. Chen, J. Liao, P. V. Sander, L. Yuan, Deep exemplar-based colorization, ACM Trans. Graphics, 37 (2018), 47. https://doi.org/10.1145/3197517.3201365 doi: 10.1145/3197517.3201365
[20]	P. Isola, J. Y. Zhu, T. Zhou, A. A. Efros, Image-to-image translation with conditional adversarial networks, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2017), 5967–5976. https://doi.org/10.1109/CVPR.2017.632
[21]	Y. Cao, Z. Zhou, W. Zhang, Y. Yu, Unsupervised diverse colorization via generative adversarial networks, in Machine Learning and Knowledge Discovery in Databases, (2017), 151–166. https://doi.org/10.1007/978-3-319-71249-9_10
[22]	S. Yoo, H. Bahng, S. Chung, J. Lee, J. Chang, J. Choo, Coloring with limited data: Few-shot colorization via memory augmented networks, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2019), 11283–11292.
[23]	M. Arjovsky, S. Chintala, L. Bottou, Wasserstein generative adversarial networks, in Proceedings of the 34th International Conference on Machine Learning, (2017), 214–223.
[24]	I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, A. C. Courville, Improved training of Wasserstein GANs, in Advances in Neural Information Processing Systems, 30 (2017).
[25]	K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, preprint, arXiv: 1409.1556.
[26]	A. Odena, V. Dumoulin, C. Olah, Deconvolution and checkerboard artifacts, Distill, 2016. https://doi.org/10.23915/distill.00003
[27]	M. D. Zeiler, G. W. Taylor, R. Fergus, Adaptive deconvolutional networks for mid and high level feature learning, in 2011 International Conference on Computer Vision, (2011), 2018–2025. https://doi.org/10.1109/ICCV.2011.6126474
[28]	M. Zhang, M. Li, J. Yu, L. Chen, Aesthetic photo collage with deep reinforcement learning, IEEE Trans. Multimedia, 25 (2023), 4653–4664. https://doi.org/10.1109/TMM.2022.3180217 doi: 10.1109/TMM.2022.3180217
[29]	R. Zhang, P. Isola, A. A. Efros, E. Shechtman, O. Wang, The unreasonable effectiveness of deep features as a perceptual metric, in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2018), 586–595. https://doi.org/10.1109/CVPR.2018.00068

Reader Comments

Your name:*

Email:*
© 2025 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)