Export file:


  • RIS(for EndNote,Reference Manager,ProCite)
  • BibTex
  • Text


  • Citation Only
  • Citation and Abstract

Regularized kernel matrix decomposition for thermal video multi-object detection and tracking

1 Department of Electrical Engineering, University of Texas at Arlington, Arlington, TX, 76019, USA
2 Department of Mathematics, University of Tennessee at Knoxville, TN, 37996, USA

This paper derives a novel algorithm for joint detection and tracking of multiple moving objects in thermal videos. The problem of determining multiple objects in a frame sequence is formulated as the task of factorizing a properly defined kernel covariance matrix into sparse factors. The support of these factors will point to the indices of the pixels that form each object. A coordinate descent approach is utilized to determine the sparse factors, and extract the object pixels. A centroid pixel is estimated for each object which is subsequently tracked via Kalman filtering. A novel interplay between the sparse kernel covariance factorization scheme along with Kalman filtering is proposed to enable joint object detection and tracking, while a divide and conquer strategy is put forth to reduce computational complexity and enable efficient tracking. Extensive numerical tests on both synthetic data and thermal video sequences demonstrate the effectiveness of the novel approach and superior tracking performance compared to existing alternatives.
  Article Metrics


1. Bar-Shalom Y, (2001) EstimationWith Applications to Tracking and Navigation. New York: Wiley.

2. Bertsekas DP, (2003) Nonlinear Programming, Second Edition, Athena Scientific.

3. Black MJ, Jepson AD (1998) Eigentracking: Robust matching and tracking of articulated objects using a view-based representation. Int J Comput Vision 26: 63–84.    

4. Chen IK, Hsu SL, Chi CY, et al. (2014) Automatic video segmentation and object tracking with real-time RGB-D data Proc of the IEEE Intl Conf on Consum Electr: 486–487.

5. Cielniak G, Duckett T, Lilienthal AJ (2007) Improved data association and occlusion handling for vision-based people tracking by mobile robots. Proc of IEEE Intl Conf on Intell Robot Syst.

6. Davis JW, Sharma V (2007) Background-subtraction using contour-based fusion of thermal and visible imagery. Comput Vis Image Und 106: 162–182.    

7. Ennulat RD, Pommerrenig D (1988) Uncooled high resolution infrared imaging plane.

8. Hanif M, Ali U (2006) Optimized visual and thermal image fusion for e cient face recognition. Proc of IEEE Intl Conf on Inform Fusion.

9. Harchaoui Z, Bach F (2007) Image classification with segmentation graph kernels. Proc of IEEE Conf on Comput Vis and Pattern Recogn.

10. Hartigan JA, Wong MA (1979) Algorithm AS 136: A K-means clustering algorithm. J R Stat Soc C-Appl 28: 100–108.

11. Heo J, Kong SG, Abidi BR, et al. (2004) Fusion of visual and thermal signatures with eyeglass removal for robust face recognition. Proc of IEEE Conf on Comput Vis and Pattern Recogn Workshop, 122–122.

12. Hofmann T, Schölkopf B, Smola AJ (2008) Kernel methods in machine learning. Ann Stat, 1171–1220.

13. Horn RA, (1985) Matrix Analysis. Cambridge, U.K.: Cambridge Univ. Press.

14. Isola P, Zoran D, Krishnan D, et al. (2014) Crisp boundary detection using pointwise mutual information. European Conference on Computer Vision, 799–814.

15. Jiang M, Pan Z, Tang Z (2017) Visual object tracking based on Cross-modality Gaussian-Bernoulli deep Boltzmann machines with RGB-D sensors. Sensors 121.

16. Kang K, Maroulas V, Schizas I, et al. (2018) Improved distributed particle filters for tracking in wireless sensor network. Comput Stat Data An 117: 90–108.    

17. Kay SM, (1993) Fundamental of Statistical Signal Processing: Estimation Theory, Prentice Hall.

18. Kwak N (2012) Kernel discriminant analysis for regression problems. Pattern Recogn 45: 2019–2031.    

19. Luber M, Spinello L, Arras KO (2011) People tracking in RGB-D data with on-line boosted target models. Proc of IEEE Intl Conf on Intell Robot Syst.

20. Maroulas V, Stinis P (2012) Improved particle filters for multi-target tracking. J Comput Phys 231: 602–611.    

21. IEEE OTCBVS WS Series Bench; Roland Miezianko, Terravic Research Infrared Database. Available: http://vcipl-okstate.org/pbvs/bench/

22. Padole CN, Alexandre LA (2010) Motion based particle filter for human tracking with thermal imaging. Proc of IEEE Intl Conf on Emerging Trends in Engineering and Technology (ICETET): 158–162.

23. Peng J, Zhou Y, Chen CLP (2015) Region-kernel-based support vector machines for hyperspectral image classification IEEE T Geosci Remote 53: 4810–4824.

24. Ren G, Maroulas V, Schizas ID (2015) Distributed Sensors-Targets Spatiotemporal Association and Tracking. IEEE Taes 51: 2570–2589.

25. Ren G, Maroulas V, Schizas ID (2016) Decentralized Sparsity-Based Multi-Source Association and State Tracking. Signal Process 120: 627–643.    

26. Ren G, Maroulas V, Schizas ID (2016) Exploiting sensor mobility and covariance sparsity for distributed tracking of multiple targets. J Adv Sig Pr 2016: 53.

27. Rosipal R, Girolami M, Trejo LJ, et al. (2001) Kernel PCA for feature extraction and de-noising in nonlinear regression. Neural Comput Appl 10: 231–243.    

28. Schizas ID (2013) Distributed informative-sensor identification via sparsity-aware matrix factorization. IEEE Trans on Sig Proc 61: 4610–4624.    

29. Teichman A, Lussier JT, Thrun S (2013) Learning to segment and track in RGBD. IEEE T Autom Sci Eng 10: 841–852.    

30. Tibshirani R (1996) Regression shrinkage and selection via the Lasso. J R Stat Soc B 58: 267–288.

31. Treptow A, Cielniak G, Duckett T (2005) Active people recognition using thermal and grey images on a mobile security robot. Proc of IEEE Intl Conf on Intell Robot Syst.

32. Tseng P (2001) Convergence of a block coordinate descent method for nondifferentiable minimization. J Opt Theory App 109: 475–494.    

33. Vert JP, Tsuda K, Schölkopf B (2004) A primer on kernel methods. Kernel Methods in Comput Biol: 35–70.

34. Zhang K, Zhang L, Yang and M (2012) Real-time compressive tracking. European Conference on Computer Vision: 864–877.

35. Xu F, Liu X, Fujimura K (2005) Pedestrian detection and tracking with night vision. IEEE T Intell Transp: 63–71.

36. Yasuno M, Yasuda N, Aoki M (2005) Pedestrian detection and tracking in far infrared images. IEEE Conf on Intell Transport S: 131–136.

37. Zou H, Hastie T, Tibshirani R (2006) Sparse principal component analysis. J Comput Graph Stat: 15.

© 2018 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution Licese (http://creativecommons.org/licenses/by/4.0)

Download full text in PDF

Export Citation

Article outline

Show full outline
Copyright © AIMS Press All Rights Reserved