Feature group tabular transformer: a novel approach to traffic crash modeling and causality analysis

Oscar Lares; Hao Zhen; Jidong J. Yang; Oscar Lares; Hao Zhen; Jidong J. Yang

doi:10.3934/aci.2025003

Applied Computing and Intelligence

2025, Volume 5, Issue 1: 29-56. doi: 10.3934/aci.2025003

Previous Article Next Article

Research article

Feature group tabular transformer: a novel approach to traffic crash modeling and causality analysis

Smart Mobility and Infrastructure Lab, College of Engineering, University of Georgia, Athens, GA 30602, USA

Academic Editor: Chih-Cheng Hung

Received: 05 December 2024 Revised: 30 December 2024 Accepted: 31 December 2024 Published: 09 January 2025

Reliable and interpretable traffic crash modeling is essential for understanding causality and improving road safety. This study introduced a novel approach to predicting crash types by utilizing a comprehensive dataset fused from multiple sources, including weather data, crash reports, high-resolution traffic information, pavement geometry, and facility characteristics. An essential part of our proposed approach was a feature group tabular transformer (FGTT) model, which organizes disparate data into meaningful feature groups, represented as tokens. These group-based tokens serve as rich semantic components, enabling effective identification of collision patterns and interpretation of causal mechanisms. The FGTT model was compared with widely used tree ensemble models, including random forest, XGBoost, and CatBoost, demonstrating better predictive performance. Furthermore, the attention heatmaps from the FGTT model revealed key influential factor interactions, providing fresh insights into the underlying causality of distinct crash types.
- road safety,
- crash analysis and modeling,
- causality analysis,
- tabular transformer,
- tree ensemble,
- feature grouping,
- attention heatmap
Citation: Oscar Lares, Hao Zhen, Jidong J. Yang. Feature group tabular transformer: a novel approach to traffic crash modeling and causality analysis[J]. Applied Computing and Intelligence, 2025, 5(1): 29-56. doi: 10.3934/aci.2025003

Related Papers:

Abstract

Reliable and interpretable traffic crash modeling is essential for understanding causality and improving road safety. This study introduced a novel approach to predicting crash types by utilizing a comprehensive dataset fused from multiple sources, including weather data, crash reports, high-resolution traffic information, pavement geometry, and facility characteristics. An essential part of our proposed approach was a feature group tabular transformer (FGTT) model, which organizes disparate data into meaningful feature groups, represented as tokens. These group-based tokens serve as rich semantic components, enabling effective identification of collision patterns and interpretation of causal mechanisms. The FGTT model was compared with widely used tree ensemble models, including random forest, XGBoost, and CatBoost, demonstrating better predictive performance. Furthermore, the attention heatmaps from the FGTT model revealed key influential factor interactions, providing fresh insights into the underlying causality of distinct crash types.

References

[1]	National Highway Traffic Safety Administration, Early estimate of motor vehicle traffic fatalities in 2023, U.S. Department of Transportation, 2023. Available from: https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/813561.
[2]	A Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, et al., Attention is all you need, Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017, 6000–6010.
[3]	D. Kim, S. Washington, J. Oh, Modeling crash types: new insights into the effects of covariates on crashes at rural intersections, J. Transp. Eng., 132 (2006), 282–292. https://doi.org/10.1061/(ASCE)0733-947X(2006)132:4(282) doi: 10.1061/(ASCE)0733-947X(2006)132:4(282)
[4]	K. Assi, Traffic crash severity prediction—a synergy by hybrid principal component analysis and machine learning models, Int. J. Environ. Res. Public Health, 17 (2020), 7598. https://doi.org/10.3390/ijerph17207598 doi: 10.3390/ijerph17207598
[5]	G. Kadilar, Effect of driver, roadway, collision, and vehicle characteristics on crash severity: a conditional logistic regression approach, Int. J. Inj. Control Sa., 23 (2016), 135–144. https://doi.org/10.1080/17457300.2014.942323 doi: 10.1080/17457300.2014.942323
[6]	Q. Zeng, W. Hao, J. Lee, F. Chen, Investigating the impacts of real-time weather conditions on freeway crash severity: a bayesian spatial analysis, Int. J. Environ. Res. Public Health, 17 (2020), 2768. https://doi.org/10.3390/ijerph17082768 doi: 10.3390/ijerph17082768
[7]	R. Spicer, A. Vahabaghaie, D. Murakhovsky, G. Bahouth, B. Drayer, S. Lawrence, Effectiveness of advanced driver assistance systems in preventing system-relevant crashes, SAE Int. J. Adv. Curr. Prac. in Mobility, 3 (2021), 1697–1701. https://doi.org/10.4271/2021-01-0869 doi: 10.4271/2021-01-0869
[8]	M. Ahmed, M. Abdel-Aty, The viability of using automatic vehicle identification data for real-time crash prediction, IEEE Trans. Intell. Transp., 13 (2011), 459–468. https://doi.org/10.1109/TITS.2011.2171052 doi: 10.1109/TITS.2011.2171052
[9]	A. Høye, I. Hesjevoll, Traffic volume and crashes and how crash and road characteristics affect their relationship–-a meta-analysis, Accident Anal. Prev., 145 (2020), 105668. https://doi.org/10.1016/j.aap.2020.105668 doi: 10.1016/j.aap.2020.105668
[10]	C. Xu, K. Ozbay, H. Liu, K. Xie, D. Yang, Exploring the impact of truck traffic on road segment-based severe crash proportion using extensive weigh-in-motion data, Safety Sci., 166 (2023), 106261. https://doi.org/10.1016/j.ssci.2023.106261 doi: 10.1016/j.ssci.2023.106261
[11]	N. Dutta, M. Fontaine, Improving freeway segment crash prediction models by including disaggregate speed data from different sources, Accident Anal. Prev., 132 (2019), 105253. https://doi.org/10.1016/j.aap.2019.07.029 doi: 10.1016/j.aap.2019.07.029
[12]	R. Avelar, K. Dixon, S. Ashraf, A comparative analysis on performance of severe crash prediction methods, Transport. Res. Rec., 2672 (2018), 109–119. https://doi.org/10.1177/0361198118794052 doi: 10.1177/0361198118794052
[13]	M. Bedard, G. Guyatt, M. Stones, J. Hirdes, The independent contribution of driver, crash, and vehicle characteristics to driver fatalities, Accident Anal. Prev., 34 (2002), 717–727. https://doi.org/10.1016/S0001-4575(01)00072-0 doi: 10.1016/S0001-4575(01)00072-0
[14]	A. Iranitalab, A. Khattak, Comparison of four statistical and machine learning methods for crash severity prediction, Accident Anal. Prev., 108 (2017), 27–36. https://doi.org/10.1016/j.aap.2017.08.008 doi: 10.1016/j.aap.2017.08.008
[15]	B. Naik, L. Tung, S. Zhao, A. Khattak, Weather impacts on single-vehicle truck crash injury severity, J. Safety Res., 58 (2016), 57–65. https://doi.org/10.1016/j.jsr.2016.06.005 doi: 10.1016/j.jsr.2016.06.005
[16]	K. Santos, J. Dias, C. Amado, A literature review of machine learning algorithms for crash injury severity prediction, J. Safety Res., 80 (2022), 254–269. https://doi.org/10.1016/j.jsr.2021.12.007 doi: 10.1016/j.jsr.2021.12.007
[17]	C. Morris, J. Yang, Understanding multi-vehicle collision patterns on freeways—a machine learning approach, Infrastructures, 5 (2020), 62. https://doi.org/10.3390/infrastructures5080062 doi: 10.3390/infrastructures5080062
[18]	C. Morris, J. Yang, Effectiveness of resampling methods in coping with imbalanced crash data: crash type analysis and predictive modeling, Accident Anal. Prev., 159 (2021), 106240. https://doi.org/10.1016/j.aap.2021.106240 doi: 10.1016/j.aap.2021.106240
[19]	A. Géron, Hands-on machine learning with Scikit-Learn, Keras, and TensorFlow, Sebastopol: O'Reilly Media, Inc., 2022.
[20]	C. Dong, C. Shao, J. Li, Z. Xiong, An improved deep learning model for traffic crash prediction, J. Adv. Transport., 2018 (2018), 3869106. https://doi.org/10.1155/2018/3869106 doi: 10.1155/2018/3869106
[21]	K. Sattar, F. Chikh Oughali, K. Assi, N. Ratrout, A. Jamal, S. Masiur Rahman, Transparent deep machine learning framework for predicting traffic crash severity, Neural Comput. Applic., 35 (2023), 1535–1547. https://doi.org/10.1007/s00521-022-07769-2 doi: 10.1007/s00521-022-07769-2
[22]	H. Li, Y. Li, A novel explanatory tabular neural network to predicting traffic incident duration using traffic safety big data, Mathematics, 11 (2023), 2915. https://doi.org/10.3390/math11132915 doi: 10.3390/math11132915
[23]	X. Wang, M. Feng, Freeway single and multi-vehicle crash safety analysis: influencing factors and hotspots, Accident Anal. Prev., 132 (2019), 105268. https://doi.org/10.1016/j.aap.2019.105268 doi: 10.1016/j.aap.2019.105268
[24]	S. Geedipally, D. Lord, Investigating the effect of modeling single-vehicle and multi-vehicle crashes separately on confidence intervals of poisson–-gamma models, Accident Anal. Prev., 42 (2010), 1273–1282. https://doi.org/10.1016/j.aap.2010.02.004 doi: 10.1016/j.aap.2010.02.004
[25]	Weather underground, Wundermap weather information, The Weather Company, LLC., 2023. Available from: https://www.wunderground.com/wundermap?lat=33.751&lon=-84.39.
[26]	H. Khanum, A. Garg, M. Faheem, Accident severity prediction modeling for road safety using random forest algorithm: an analysis of indian highways, F1000Res., 12 (2023), 494. https://doi.org/10.12688/f1000research.133594.2 doi: 10.12688/f1000research.133594.2
[27]	T. Chen, C. Guestrin, Xgboost: a scalable tree boosting system, Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016,785–794. https://doi.org/10.1145/2939672.2939785 doi: 10.1145/2939672.2939785
[28]	Y. Yang, K. Wang, Z. Yuan, D. Liu, Predicting freeway traffic crash severity using xgboost-bayesian network model with consideration of features interaction, J. Adv. Transport., 2022 (2022), 4257865. https://doi.org/10.1155/2022/4257865 doi: 10.1155/2022/4257865
[29]	L. Prokhorenkova, G. Gusev, A. Vorobev, A. Dorogush, A. Gulin, Catboost: unbiased boosting with categorical features, Proceedings of the 32nd International Conference on Neural Information Processing Systems, 2018, 6639–6649.
[30]	A. Hasan, M. Jalayer, S. Das, M. Kabir, Application of machine learning models and shap to examine crashes involving young drivers in new jersey, International Journal of Transportation Science and Technology, 14 (2024), 156–170. https://doi.org/10.1016/j.ijtst.2023.04.005 doi: 10.1016/j.ijtst.2023.04.005
[31]	J. Li, X. Wang, X. Yang, Q. Zhang, H. Pan, Analyzing freeway safety influencing factors using the catboost model and interpretable machine-learning framework, shap, Transport. Res. Rec., 2678 (2024), 563–574. https://doi.org/10.1177/03611981231208903 doi: 10.1177/03611981231208903
[32]	S. Lundberg, G. Erion, S. Lee, Consistent individualized feature attribution for tree ensembles, arXiv: 1802.03888. https://doi.org/10.48550/arXiv.1802.03888
[33]	R. Turner, D. Eriksson, M. McCourt, J. Kiili, E. Laaksonen, Z. Xu, I. Guyon, Bayesian optimization is superior to random search for machine learning hyperparameter tuning: analysis of the black-box optimization challenge 2020, Proceedings of the NeurIPS 2020 Competition and Demonstration Track, 2021, 3–26.
[34]	A. Victoria, G. Maragatham, Automatic tuning of hyperparameters using bayesian optimization, Evol. Syst., 12 (2021), 217–223. https://doi.org/10.1007/s12530-020-09345-2 doi: 10.1007/s12530-020-09345-2
[35]	T. Akiba, S. Sano, T. Yanase, T. Ohta, M. Koyama, Optuna: a next-generation hyperparameter optimization framework, Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019, 2623–2631. https://doi.org/10.1145/3292500.3330701 doi: 10.1145/3292500.3330701
[36]	T. Lin, P. Goyal, R. Girshick, K. He, P. Dollár, Focal loss for dense object detection. Proceedings of IEEE International Conference on Computer Vision (ICCV), 2017, 2980–2988. https://doi.org/10.1109/ICCV.2017.324 doi: 10.1109/ICCV.2017.324
[37]	J. Tian, P. Tsai, K. Zhang, X. Cai, H. Xiao, K. Yu, et al., Synergetic focal loss for imbalanced classification in federated xgboost, IEEE Transactions on Artificial Intelligence, 5 (2023), 647–660. https://doi.org/10.1109/TAI.2023.3254519 doi: 10.1109/TAI.2023.3254519
[38]	X. Liu, J. Lu, X. Chen, Y. Fong, X. Ma, F. Zhang, Attention based spatio-temporal graph convolutional network with focal loss for crash risk evaluation on urban road traffic network based on multi-source risks, Accident Anal. Prev., 192 (2023), 107262. https://doi.org/10.1016/j.aap.2023.107262 doi: 10.1016/j.aap.2023.107262
[39]	R. Yu, Y. Wang, Z. Zou, L. Wang, Convolutional neural networks with refined loss functions for the real-time crash risk analysis, Transport. Res. C-Emer., 119 (2020), 102740. https://doi.org/10.1016/j.trc.2020.102740 doi: 10.1016/j.trc.2020.102740
[40]	M. Hernandez, G. Epelde, A. Alberdi, R. Cilla, D. Rankin, Synthetic data generation for tabular health records: a systematic review, Neurocomputing, 493 (2022), 28–45. https://doi.org/10.1016/j.neucom.2022.04.053 doi: 10.1016/j.neucom.2022.04.053
[41]	Y. Jo, C. Oh, S. Kim, Estimation of heavy vehicle-involved rear-end crash potential using wim data, Accident Anal. Prev., 128 (2019), 103–113. https://doi.org/10.1016/j.aap.2019.04.005 doi: 10.1016/j.aap.2019.04.005

Reader Comments

Your name:*

Email:*
© 2025 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)