Correlation filter learning with dual-modal motion fusion and total variation for UAV tracking

Yu-Feng Yu; Ziwei Wang; Xiaoying Tan; Yang Zhang; Ke-Kun Huang; Yu-Feng Yu; Ziwei Wang; Xiaoying Tan; Yang Zhang; Ke-Kun Huang

doi:10.3934/era.2026249

Electronic Research Archive

2026, Volume 34, Issue 8: 5580-5611. doi: 10.3934/era.2026249

Previous Article Next Article

Research article

Correlation filter learning with dual-modal motion fusion and total variation for UAV tracking

1.
Department of Statistics, Guangzhou University, Guangzhou, China
2.
Department of Computer and Information Science, University of Macau, Macau 999078, China
3.
School of Mathematics, Jiaying University, Meizhou 514015, China

Received: 22 April 2026 Revised: 15 June 2026 Accepted: 17 June 2026 Published: 30 June 2026

Unmanned aerial vehicle (UAV) visual tracking is an important task in low-altitude perception and aerial monitoring, where trackers must handle fast camera motion, scale variation, occlusion, background clutter, and limited onboard computational resources. These challenges can easily lead to tracking drift or failure. To address these issues, we propose a correlation filter tracker named correlation filter learning with dual-modal motion fusion and total variation for UAV tracking (CFLDT). First, the search region is updated using the dual-modal motion fusion (DMMF), and features are then extracted from the calibrated area. This process integrates local motion cues and correlation response information to improve target localization under complex UAV motion. In addition, a multi-response spatial penalty matrix (MRSP) is introduced to dynamically constrain the filter, while the environment-aware regularization term is optimized through the total variation constraint module (TVCM). Finally, both the MRSP and the environment-aware regularization term are incorporated into the filter training process, yielding a more discriminative model for distinguishing the target from the background. Extensive comparative experiments on multiple UAV tracking datasets demonstrate that CFLDT achieves competitive tracking accuracy, robustness, and real-time performance.
- UAV target tracking,
- total variation,
- correlation filtering
Citation: Yu-Feng Yu, Ziwei Wang, Xiaoying Tan, Yang Zhang, Ke-Kun Huang. Correlation filter learning with dual-modal motion fusion and total variation for UAV tracking[J]. Electronic Research Archive, 2026, 34(8): 5580-5611. doi: 10.3934/era.2026249

Related Papers:

Abstract

Unmanned aerial vehicle (UAV) visual tracking is an important task in low-altitude perception and aerial monitoring, where trackers must handle fast camera motion, scale variation, occlusion, background clutter, and limited onboard computational resources. These challenges can easily lead to tracking drift or failure. To address these issues, we propose a correlation filter tracker named correlation filter learning with dual-modal motion fusion and total variation for UAV tracking (CFLDT). First, the search region is updated using the dual-modal motion fusion (DMMF), and features are then extracted from the calibrated area. This process integrates local motion cues and correlation response information to improve target localization under complex UAV motion. In addition, a multi-response spatial penalty matrix (MRSP) is introduced to dynamically constrain the filter, while the environment-aware regularization term is optimized through the total variation constraint module (TVCM). Finally, both the MRSP and the environment-aware regularization term are incorporated into the filter training process, yielding a more discriminative model for distinguishing the target from the background. Extensive comparative experiments on multiple UAV tracking datasets demonstrate that CFLDT achieves competitive tracking accuracy, robustness, and real-time performance.

References

[1]	S. Li, D. Yeung, Visual object tracking for unmanned aerial vehicles: A benchmark and new motion models, in Proceedings of the AAAI Conference on Artificial Intelligence, 31 (2017). https://doi.org/10.1609/aaai.v31i1.11205
[2]	M. Mueller, N. Smith, B. Ghanem, A benchmark and simulator for uav tracking, in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, 9905 (2016), 445–461. https://doi.org/10.1007/978-3-319-46448-0_27
[3]	D. Du, Y. Qi, H. Yu, Y. Yang, K. Duan, G. Li, et al., The unmanned aerial vehicle benchmark: Object detection and tracking, in Proceedings of the European Conference on Computer Vision (ECCV), (2018), 370–386.
[4]	M. Bakirci, Internet of things-enabled unmanned aerial vehicles for real-time traffic mobility analysis in smart cities, Comput. Electr. Eng., 123 (2025), 110313. https://doi.org/10.1016/j.compeleceng.2025.110313 doi: 10.1016/j.compeleceng.2025.110313
[5]	W. Ouyang, J. Mu, X. Jing, Y. Wang, Efficient vehicle recognition and tracking for UAV-enabled intelligent transport systems: A multi-agent reinforcement learning method, IEEE Trans. Intell. Transp. Syst., 26 (2025), 20930–20940. https://doi.org/10.1109/TITS.2025.3601740 doi: 10.1109/TITS.2025.3601740
[6]	J. Liu, H. Wang, C. Ma, Y. Su, X. Yang, Siamdmu: Siamese dual mask update network for visual object tracking, IEEE Trans. Emerging Top. Comput. Intell., 8 (2024), 1656–1669. https://doi.org/10.1109/TETCI.2024.3353674 doi: 10.1109/TETCI.2024.3353674
[7]	Z. Xin, J. Yu, X. He, Y. Song, H. Li, Siamraan: Siamese residual attentional aggregation network for visual object tracking, Neural Process. Lett., 56 (2024), 98. https://doi.org/10.1007/s11063-024-11556-6 doi: 10.1007/s11063-024-11556-6
[8]	Z. Zhang, Z. Guo, L. Wang, Y. Li, Ctiftrack: Continuous temporal information fusion for object track, Expert Syst. Appl., 262 (2025), 125654. https://doi.org/10.1016/j.eswa.2024.125654 doi: 10.1016/j.eswa.2024.125654
[9]	K. Wang, Z. Wang, X. Zhang, M. Liu, Bstrack: Robust UAV tracking using feature extraction of bilateral filters and sparse attention mechanism, Expert Syst. Appl., 267 (2025), 126202. https://doi.org/10.1016/j.eswa.2024.126202 doi: 10.1016/j.eswa.2024.126202
[10]	H. Wu, Y. Chen, C. Wu, R. Zhang, K. Chen, A multi-scale cyclic-shift window transformer object tracker based on fast fourier transform, Electron. Res. Arch., 33 (2025), 3638–3672. https://doi.org/10.3934/era.2025162 doi: 10.3934/era.2025162
[11]	F. Li, C. Tian, W. Zuo, L. Zhang, M. Yang, Learning spatial-temporal regularized correlation filters for visual tracking, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2018), 4904–4913.
[12]	S. Moorthy, Y. H. Joo, Adaptive spatial-temporal surrounding-aware correlation filter tracking via ensemble learning, Pattern Recogn., 139 (2023), 109457. https://doi.org/10.1016/j.patcog.2023.109457 doi: 10.1016/j.patcog.2023.109457
[13]	H. Zhang, Y. Li, Y. Yang, Y. Feng, Y. Li, C. Deng, et al., UAV tracking based on correlation filters with dynamic aberrance-repressed temporal regularizations, IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 16 (2023), 7749–7762. https://doi.org/10.1109/JSTARS.2023.3306273 doi: 10.1109/JSTARS.2023.3306273
[14]	K. Chen, Y. Chen, R. Zhang, Y. Chen, H. Han, Y. He, et al., Correlation filter object tracking algorithm based on spatial and channel attention mechanism, Electron. Res. Arch., 33 (2025), 4857–4892. https://doi.org/10.3934/era.2025219 doi: 10.3934/era.2025219
[15]	F. Zhang, S. Ma, Y. Zhang, Z. Qiu, Perceiving temporal environment for correlation filters in real-time UAV tracking, IEEE Signal Process. Lett., 29 (2021), 6–10. https://doi.org/10.1109/LSP.2021.3120943 doi: 10.1109/LSP.2021.3120943
[16]	C. Wu, J. Shen, K. Chen, Y. Chen, Y. Liao, UAV object tracking algorithm based on spatial saliency-aware correlation filter, Electron. Res. Arch., 33 (2025), 1446. https://doi.org/10.3934/era.2025068 doi: 10.3934/era.2025068
[17]	Y. Xue, T. Shen, G. Jin, L. Tan, N. Wang, L. Wang, et al., Handling occlusion in UAV visual tracking with query-guided redetection, IEEE Trans. Instrum. Meas., 73 (2024), 1–17. https://doi.org/10.1109/TIM.2024.3440378 doi: 10.1109/TIM.2024.3440378
[18]	Z. Huang, C. Fu, Y. Li, F. Lin, P. Lu, Learning aberrance repressed correlation filters for real-time UAV tracking, in Proceedings of the IEEE/CVF International Conference on Computer Vision, (2019), 2891–2900.
[19]	Y. F. Yu, Z. Chen, Y. Zhang, C. Zhang, W. Ding, Learning dynamic-sensitivity enhanced correlation filter with adaptive second-order difference spatial regularization for UAV tracking, IEEE Trans. Intell. Transp. Syst., 26 (2025), 7211–7230. https://doi.org/10.1109/TITS.2025.3533953 doi: 10.1109/TITS.2025.3533953
[20]	Y. Zhang, Y. F. Yu, L. Chen, W. Ding, Robust correlation filter learning with continuously weighted dynamic response for UAV visual tracking, IEEE Trans. Geosci. Remote Sens., 61 (2023), 1–14. https://doi.org/10.1109/TGRS.2023.3325337 doi: 10.1109/TGRS.2023.3325337
[21]	J. Wen, H. Chu, Z. Lai, T. Xu, L. Shen, Enhanced robust spatial feature selection and correlation filter learning for UAV tracking, Neural Netw., 161 (2023), 39–54. https://doi.org/10.1016/j.neunet.2023.01.003 doi: 10.1016/j.neunet.2023.01.003
[22]	G. Zheng, C. Fu, J. Ye, F. Lin, F. Ding, Mutation sensitive correlation filter for real-time UAV tracking with adaptive hybrid label, in 2021 IEEE International Conference on Robotics and Automation (ICRA), (2021), 503–509. https://doi.org/10.1109/ICRA48506.2021.9561931
[23]	C. Fu, J. Jin, F. Ding, Y. Li, G. Lu, Spatial reliability enhanced correlation filter: An efficient approach for real-time UAV tracking, IEEE Trans. Multimedia, 26 (2024), 4123–4137. https://doi.org/10.1109/TMM.2021.3118891 doi: 10.1109/TMM.2021.3118891
[24]	F. Lin, C. Fu, Y. He, W. Xiong, F. Li, Recf: Exploiting response reasoning for correlation filters in real-time UAV tracking, IEEE Trans. Intell. Transp. Syst., 23 (2021), 10469–10480. https://doi.org/10.1109/TITS.2021.3094654 doi: 10.1109/TITS.2021.3094654
[25]	Y. Chen, K. Chen, Four mathematical modeling forms for correlation filter object tracking algorithms and the fast calculation for the filter, Electron. Res. Arch., 32 (2024), 4684–4714. https://doi.org/10.3934/era.2024213 doi: 10.3934/era.2024213
[26]	H. K. Galoogahi, A. Fagg, S. Lucey, Learning background-aware correlation filters for visual tracking, in Proceedings of the IEEE International Conference on Computer Vision, (2017), 1135–1143.
[27]	C. Fu, J. Xu, F. Lin, F. Guo, T. Liu, Z. Zhang, Object saliency-aware dual regularized correlation filter for real-time aerial tracking, IEEE Trans. Geosci. Remote Sens., 58 (2020), 8940–8951. https://doi.org/10.1109/TGRS.2020.2992301 doi: 10.1109/TGRS.2020.2992301
[28]	F. Lin, C. Fu, Y. He, F. Guo, Q. Tang, Bicf: Learning bidirectional incongruity-aware correlation filter for efficient UAV object tracking, in 2020 IEEE International Conference on Robotics and Automation (ICRA), (2020), 2365–2371. https://doi.org/10.1109/ICRA40945.2020.9196530
[29]	M. Danelljan, G. Bhat, F. S. Khan, M. Felsberg, Eco: Efficient convolution operators for tracking, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, (2017), 6638–6646.
[30]	C. Fu, J. Ye, J. Xu, Y. He, F. Lin, Disruptor-aware interval-based response inconsistency for correlation filters in real-time aerial tracking, IEEE Trans. Geosci. Remote Sens., 59 (2020), 6301–6313. https://doi.org/10.1109/TGRS.2020.3030265 doi: 10.1109/TGRS.2020.3030265
[31]	Y. Li, C. Fu, F. Ding, Z. Huang, G. Lu, Autotrack: Towards high-performance visual tracking for UAV with automatic spatio-temporal regularization, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2020), 11923–11932.
[32]	J. Ye, C. Fu, F. Lin, F. Ding, S. An, G. Lu, Multi-regularized correlation filter for UAV tracking and self-localization, IEEE Trans. Ind. Electron., 69 (2021), 6004–6014. https://doi.org/10.1109/TIE.2021.3088366 doi: 10.1109/TIE.2021.3088366
[33]	Y. Zhang, Y. F. Yu, K. K. Huang, Y. Wang, Channel attentional correlation filters learning with second-order difference for UAV tracking, IEEE Geosci. Remote Sens. Lett., 20 (2023), 1–5. https://doi.org/10.1109/LGRS.2023.3311441 doi: 10.1109/LGRS.2023.3311441
[34]	J. Lin, J. Peng, J. Chai, Real-time UAV correlation filter based on response-weighted background residual and spatio-temporal regularization, IEEE Geosci. Remote Sens. Lett., 20 (2023), 1–5. https://doi.org/10.1109/LGRS.2023.3272522 doi: 10.1109/LGRS.2023.3272522
[35]	Y. F. Yu, Y. Zhang, L. Chen, P. Ge, C. L. P. Chen, Multi-scale enhanced features correlation filters learning with dual second-order difference for UAV tracking, IEEE Trans. Intell. Veh., 9 (2024), 3232–3245. https://doi.org/10.1109/TIV.2024.3355171 doi: 10.1109/TIV.2024.3355171
[36]	Z. An, X. Wang, B. Li, J. Fu, Learning spatial regularization correlation filters with the Hilbert-Schmidt independence criterion in RKHS for UAV tracking, IEEE Trans. Instrum. Meas., 72 (2023), 1–12. https://doi.org/10.1109/TIM.2023.3265106 doi: 10.1109/TIM.2023.3265106
[37]	L. Chen, Y. Liu, Y. Wang, An efficient spatial-temporal UAV visual tracker with the temporal enhancement model update strategy, Signal Image Video Process., 19 (2025), 217. https://doi.org/10.1007/s11760-024-03772-3 doi: 10.1007/s11760-024-03772-3
[38]	Y. F. Yu, X. Tan, Y. Zhang, L. Chen, W. Ding, K-nearest neighbor correlation filters learning with p-laplacian regularization for visual tracking, Tsinghua Sci. Technol., (2025). https://doi.org/10.26599/TST.2025.9010049
[39]	N. Wang, Y. Song, C. Ma, W. Zhou, W. Liu, H. Li, Unsupervised deep tracking, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2019), 1308–1317.
[40]	J. Zhao, K. Dai, D. Wang, H. Lu, X. Yang, Online filtering training samples for robust visual tracking, in Proceedings of the 28th ACM International Conference on Multimedia, (2020), 1488–1496. https://doi.org/10.1145/3394171.3413930
[41]	B. Wang, W. Li, B. Zhang, Y. Liu, J. Du, Correlation filters for UAV online tracking based on complementary appearance model and reversibility reasoning, IEEE Trans. Circuits Syst. Video Technol., 34 (2024), 3983–3997. https://doi.org/10.1109/TCSVT.2023.3325672 doi: 10.1109/TCSVT.2023.3325672
[42]	J. Zhang, Y. He, W. Feng, J. Wang, N. N. Xiong, Learning background-aware and spatial-temporal regularized correlation filters for visual tracking, Appl. Intell., 53 (2023), 7697–7712. https://doi.org/10.1007/s10489-022-03868-8 doi: 10.1007/s10489-022-03868-8
[43]	Z. Cao, C. Fu, J. Ye, B. Li, Y. Li, Hift: Hierarchical feature transformer for aerial tracking, in Proceedings of the IEEE/CVF International Conference on Computer Vision, (2021), 15457–15466.
[44]	I. Sosnovik, A. Moskalev, A. Smeulders, Scale equivariance improves siamese tracking, in 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), (2021), 2764–2773.
[45]	Z. Cao, Z. Huang, L. Pan, S. Zhang, Z. Liu, C. Fu, Tctrack: Temporal contexts for aerial tracking, in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2022), 14778–14788.
[46]	H. Zuo, C. Fu, S. Li, K. Lu, Y. Li, C. Feng, Adversarial blur-deblur network for robust UAV tracking, IEEE Robot. Autom. Lett., 8 (2023), 1101–1108. https://doi.org/10.1109/LRA.2023.3236584 doi: 10.1109/LRA.2023.3236584
[47]	B. Yan, H. Peng, J. Fu, D. Wang, H. Lu, Learning spatio-temporal transformer for visual tracking, in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), (2021), 10428–10437.
[48]	B. Kang, X. Chen, D. Wang, H. Peng, H. Lu, Exploring lightweight hierarchical vision transformers for efficient visual tracking, in 2023 IEEE/CVF International Conference on Computer Vision (ICCV), (2023), 9578–9587.
[49]	Y. Cui, T. Song, G. Wu, L. Wang, Mixformerv2: Efficient fully transformer tracking, Adv. Neural Inf. Process. Syst., 36 (2023), 58736–58751.
[50]	Y. Wu, X. Wang, D. Zeng, H. Ye, X. Xie, Q. Zhao, et al., Learning motion blur robust vision transformers with dynamic early exit for real-time UAV tracking, Expert Syst. Appl., 297 (2024), https://doi.org/10.1016/j.eswa.2025.129445
[51]	S. Li, X. Yang, X. Wang, D. Zeng, H. Ye, Q. Zhao, Learning target-aware vision transformers for real-time UAV tracking, IEEE Trans. Geosci. Remote Sens., 62 (2024), 1–18. https://doi.org/10.1109/TGRS.2024.3417400 doi: 10.1109/TGRS.2024.3417400
[52]	C. Xue, B. Zhong, Q. Liang, Y. Zheng, N. Li, Y. Xue, et al., Similarity-guided layer-adaptive vision transformer for UAV tracking, in Proceedings of the Computer Vision and Pattern Recognition Conference, (2025), 6730–6740.
[53]	Y. Wu, X. Wang, X. Yang, M. Liu, D. Zeng, H. Ye, et al., Learning occlusion-robust vision transformers for real-time UAV tracking, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, (2025), 17103–17113.
[54]	X. Yang, D. Zeng, X. Wang, H. Ye, X. Xie, Q. Zhao, et al., Towards real-time UAV tracking with adaptive and background-aware vision transformers, IEEE Trans. Circuits Syst. Video Technol., 36 (2026), 7395–7410. https://doi.org/10.1109/TCSVT.2026.3652352 doi: 10.1109/TCSVT.2026.3652352
[55]	K. Yin, J. Feng, S. Dong, Y. Long, Mmkltrack: Illumination-robust UAV tracking with progressive localization, Expert Syst. Appl., 300 (2026), 130207. https://doi.org/10.1016/j.eswa.2025.130207 doi: 10.1016/j.eswa.2025.130207

Reader Comments

Your name:*

Email:*
© 2026 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)