Study of visual SLAM methods in minimally invasive surgery

Liwei Deng; Zhen Liu; Tao Zhang; Zhe Yan; Liwei Deng; Zhen Liu; Tao Zhang; Zhe Yan

doi:10.3934/mbe.2023203

Mathematical Biosciences and Engineering

2023, Volume 20, Issue 3: 4388-4402. doi: 10.3934/mbe.2023203

Previous Article Next Article

Research article Special Issues

Study of visual SLAM methods in minimally invasive surgery

Heilongjiang Provincial Key Laboratory of Complex Intelligent System and Integration, School of Automation, Harbin University of Science and Technology, Harbin 150080, China

Received: 14 June 2022 Revised: 24 October 2022 Accepted: 18 November 2022 Published: 23 December 2022

In recent years, minimally invasive surgery has developed rapidly in the clinical practice of surgery and has gradually become one of the critical surgical techniques. Compared with traditional surgery, the advantages of minimally invasive surgery include small incisions and less pain during the operation, and the patients recover faster after surgery. With the expansion of minimally invasive surgery in several medical fields, traditional minimally invasive techniques have bottlenecks in clinical practice, such as the inability of the endoscope to determine the depth information of the lesion area from the two-dimensional images obtained, the difficulty in locating the endoscopic position information and the inability to get a complete view of the overall situation in the cavity. This paper uses a visual simultaneous localization and mapping (SLAM) approach to achieve endoscope localization and reconstruction of the surgical region in a minimally invasive surgical environment. Firstly, the K-Means algorithm combined with the Super point algorithm is used to extract the feature information of the image in the lumen environment. Compared with Super points, the logarithm of successful matching points increased by 32.69%, the proportion of effective points increased by 25.28%, the error matching rate decreased by 0.64%, and the extraction time decreased by 1.98%. Then the iterative closest point method is used to estimate the position and attitude information of the endoscope. Finally, the disparity map is obtained by the stereo matching method, and the point cloud image of the surgical area is finally recovered.

Keywords:

Citation: Liwei Deng, Zhen Liu, Tao Zhang, Zhe Yan. Study of visual SLAM methods in minimally invasive surgery[J]. Mathematical Biosciences and Engineering, 2023, 20(3): 4388-4402. doi: 10.3934/mbe.2023203

Related Papers:

[1]	Mapundi K. Banda, Michael Herty, Axel Klar . Gas flow in pipeline networks. Networks and Heterogeneous Media, 2006, 1(1): 41-56. doi: 10.3934/nhm.2006.1.41
[2]	Michael Herty . Modeling, simulation and optimization of gas networks with compressors. Networks and Heterogeneous Media, 2007, 2(1): 81-97. doi: 10.3934/nhm.2007.2.81
[3]	Michael Herty, Veronika Sachers . Adjoint calculus for optimization of gas networks. Networks and Heterogeneous Media, 2007, 2(4): 733-750. doi: 10.3934/nhm.2007.2.733
[4]	Martin Gugat, Falk M. Hante, Markus Hirsch-Dick, Günter Leugering . Stationary states in gas networks. Networks and Heterogeneous Media, 2015, 10(2): 295-320. doi: 10.3934/nhm.2015.10.295
[5]	Mapundi K. Banda, Michael Herty, Axel Klar . Coupling conditions for gas networks governed by the isothermal Euler equations. Networks and Heterogeneous Media, 2006, 1(2): 295-314. doi: 10.3934/nhm.2006.1.295
[6]	Claus Kirchner, Michael Herty, Simone Göttlich, Axel Klar . Optimal control for continuous supply network models. Networks and Heterogeneous Media, 2006, 1(4): 675-688. doi: 10.3934/nhm.2006.1.675
[7]	Fabian Rüffler, Volker Mehrmann, Falk M. Hante . Optimal model switching for gas flow in pipe networks. Networks and Heterogeneous Media, 2018, 13(4): 641-661. doi: 10.3934/nhm.2018029
[8]	Didier Georges . Infinite-dimensional nonlinear predictive control design for open-channel hydraulic systems. Networks and Heterogeneous Media, 2009, 4(2): 267-285. doi: 10.3934/nhm.2009.4.267
[9]	Simone Göttlich, Oliver Kolb, Sebastian Kühn . Optimization for a special class of traffic flow models: Combinatorial and continuous approaches. Networks and Heterogeneous Media, 2014, 9(2): 315-334. doi: 10.3934/nhm.2014.9.315
[10]	Gunhild A. Reigstad . Numerical network models and entropy principles for isothermal junction flow. Networks and Heterogeneous Media, 2014, 9(1): 65-95. doi: 10.3934/nhm.2014.9.65

Abstract

References

[1]	T. N. Robinson, G. V. Stiegmann, Minimally invasive surgery, Endoscopy, 36 (2004), 48–51. https://doi.org/10.1055/s-2004-814113 doi: 10.1055/s-2004-814113
[2]	J. Perissat, Laparoscopic cholecystectomy, a treatment for gallstones: From idea to reality, World J. Surg., 23 (1999), 328–331.
[3]	J. Chen, Y. Guo, Observation and study on the therapeutic effect of endoscopy combined with minimally invasive abdominal surgery robot in the treatment of gallstones, in Proceedings of 2019 International Conference on Biology, Chemistry and Medical Engineering Francis, (2019), 79–84.
[4]	J. Xiao, Q. Wu, D. Sun, C. He, Y. Chen, Classifications and functions of vitreoretinal surgery assisted robots-a review of the state of the Art, in 2019 International Conference on Intelligent Transportation, Big Data & Smart City (ICITBS) IEEE, (2019), 474–484. https://doi.org/10.1109/ICITBS.2019.00122
[5]	C. Siristatidis, C. Chrelias, Feasibility of office hysteroscopy through the "see and treat technique" in private practice: A prospective observational study, Arch. Gynecol. Obstet., 283 (2011), 819–823. https://doi.org/10.1007/s00404-010-1431-3 doi: 10.1007/s00404-010-1431-3
[6]	P. Cheeseman, R. Smith, M. Self, A stochastic map for uncertain spatial relationships, in 4th International Symposium on Robotic Research, (1987), 467–474.
[7]	P. Mountney, D. Stoyanov, A. Davison, G. Yang, Simultaneous stereoscope localization and soft-tissue mapping for minimal invasive surgery, in International Conference on Medical Image Computing and Computer-Assisted Intervention Springer, Berlin, Heidelberg, (2006), 347–354. https://doi.org/10.1007/11866565_43
[8]	G. Mattioli, V. Rossi, F. Palo, M. C. Y. Wong, P. Gandullia, S. Arrigo, et al., Minimal invasive approach to paediatric colorectal surgery, J. Ped. Endosc. Surg., 3 (2021), 129–139. https://doi.org/10.1007/s42804-020-00090-6 doi: 10.1007/s42804-020-00090-6
[9]	P. Mountney, G. Z. Yang, Dynamic view expansion for minimally invasive surgery using simultaneous localization and mapping, in 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE, (2009), 1184–1187. https://doi.org/10.1109/IEMBS.2009.5333939
[10]	B. Lin, Visual SLAM and Surface Reconstruction for Abdominal Minimally Invasive Surgery, University of South Florida, 2015.
[11]	L. Chen, W.Tang, N. W. John, T. R. Wan, J. J. Zhang, Augmented reality for depth cues in monocular minimally invasive surgery, preprint, arXiv: 1703.01243.
[12]	A. Marmol, P. Corke, T. Pinot, ArthroSLAM: Multi-sensor robust visual localization for minimally invasive orthopedic surgery, in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) IEEE, (2018), 3882–3889. https://doi.org/10.1109/IROS.2018.8593501
[13]	C. Girerd, A. V. Kudryavtsev, P. Rougeot, P. Renaud, K. Rabenorosoa, B. Tamadazte, Automatic tip-steering of concentric tube robots in the trachea based on visual slam, in IEEE Transactions on Medical Robotics and Bionics, 2 (2020), 582–585. https://doi.org/10.1109/TMRB.2020.3034720
[14]	I. Font, S. Weiland, M. Franken, M. Steinbuch, L. Rovers, Haptic feedback designs in teleoperation systems for minimal invasive surgery, in 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583), (2004), 2513–2518. https://doi.org/10.1109/ICSMC.2004.1400707
[15]	S. Seung, B. Kang, H. Je, J. Park, K. Kim, S. Park, Tele-operation master-slave system for minimal invasive brain surgery, in 2009 IEEE International Conference on Robotics and Biomimetics (ROBIO), (2009), 177–182. https://doi.org/10.1109/ROBIO.2009.5420619
[16]	D. DeTone, T. Malisiewicz, A. Rabinovich, Superpoint: Self-supervised interest point detection and description, in Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition Workshops, (2018), 224–236. https://doi.org/10.1109/CVPRW.2018.00060
[17]	E. Mühe, Laparoscopic cholecystectomy—late results, in Die Chirurgie und ihre Spezialgebiete Eine Symbiose, (1991), 416–423. https://doi.org/10.1007/978-3-642-95662-1_189
[18]	J. Gimenez, A. Amicarelli, J. M. Toibero, F. di Sciascio, R. Carelli, Iterated conditional modes to solve simultaneous localization and mapping in markov random fields context, Int. J. Autom. Comput., 15 (2018), 310–324. https://doi.org/10.1007/s11633-017-1109-4 doi: 10.1007/s11633-017-1109-4
[19]	N. Ketkar, J. Moolayil, Convolutional neural networks, in Deep Learning with Python, Springer International Publishing, (2017), 197–242. https://doi.org/10.1007/978-1-4842-5364-9_6
[20]	Z. Zhang, Flexible camera calibration by viewing a plane from unknown orientations, in Proceedings of the Seventh Ieee International Conference on Computer Vision, (1999), 666–673.
[21]	D. G. Lowe, Distinctive image features from scale-invariant keypoints, Int. J. Comput. Vision, 60 (2004), 91–110. https://doi.org/10.1023/B:VISI.0000029664.99615.94 doi: 10.1023/B:VISI.0000029664.99615.94
[22]	T. Qin, P. Li, S. Shen, Vins-mono: a robust and versatile monocular visual-inertial state estimator, in IEEE Transactions on Robotics, 34 (2018), 1004–1020. https://doi.org/10.1109/TRO.2018.2853729
[23]	O. Garcıa, J. Civera, A. Gueme, V. Munoz, J. M. M. Montiel, Real-time 3d modeling from endoscope image sequences, Adv. Sens. Sensor Integr. Med. Robot., (2009), 1–3.
[24]	M. A. Fischler, R. C. Bolles, Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography, Commun. ACM, 24 (1981), 381–395. https://doi.org/10.1145/358669.358692 doi: 10.1145/358669.358692
[25]	F. Wolfgang, B. P. Wrobel, Bundle adjustment, in Photogrammetric Computer Vision, (2016), 643–725. https://doi.org/10.1007/978-3-319-11550-4_15

This article has been cited by:

1.	Dustin Carrión-Ojeda, Rigoberto Fonseca-Delgado, Israel Pineda, Analysis of factors that influence the performance of biometric systems based on EEG signals, 2021, 165, 09574174, 113967, 10.1016/j.eswa.2020.113967
2.	Leila Farsi, Siuly Siuly, Enamul Kabir, Hua Wang, Classification of Alcoholic EEG Signals Using a Deep Learning Method, 2021, 21, 1530-437X, 3552, 10.1109/JSEN.2020.3026830
3.	Sumair Aziz, Muhammad Umar Khan, Zainoor Ahmad Choudhry, Afeefa Aymin, Adil Usman, 2019, ECG-based Biometric Authentication using Empirical Mode Decomposition and Support Vector Machines, 978-1-7281-2530-5, 0906, 10.1109/IEMCON.2019.8936174
4.	Muhammad Umar Khan, Sumair Aziz, Sara Ibraheem, Aqsa Butt, Hira Shahid, 2019, Characterization of Term and Preterm Deliveries using Electrohysterograms Signatures, 978-1-7281-2530-5, 0899, 10.1109/IEMCON.2019.8936292
5.	Dustin Carrion-Ojeda, Hector Mejia-Vallejo, Rigoberto Fonseca-Delgado, Pilar Gomez-Gil, Manuel Ramirez-Cortes, 2019, A method for studying how much time of EEG recording is needed to have a good user identification, 978-1-7281-5666-8, 1, 10.1109/LA-CCI47412.2019.9037054
6.	Andrei V. Kelarev, Xun Yi, Hui Cui, Leanne Rylands, Herbert F. Jelinek, A survey of state-of-the-art methods for securing medical databases, 2018, 5, 2375-1576, 1, 10.3934/medsci.2018.1.1
7.	Lu-di Wang, Wei Zhou, Ying Xing, Na Liu, Mahmood Movahedipour, Xiao-guang Zhou, A novel method based on convolutional neural networks for deriving standard 12-lead ECG from serial 3-lead ECG, 2019, 20, 2095-9184, 405, 10.1631/FITEE.1700413
8.	Qingxue Zhang, Dian Zhou, Xuan Zeng, 2017, Hear the heart: Daily cardiac health monitoring using Ear-ECG and machine learning, 978-1-5386-1104-3, 448, 10.1109/UEMCON.2017.8249110
9.	Yaroslav Voznyi, Mariia Nazarkevych, Volodymyr Hrytsyk, Nataliia Lotoshynska, Bohdana Havrysh, DESIGN OF BIOMETRIC PROTECTION AUTHENTIFICATION SYSTEM BASED ON K-AVERAGE METHOD, 2021, 4, 2663-4023, 85, 10.28925/2663-4023.2021.12.8595
10.	Mariya Nazarkevych, Yaroslav Voznyi, Volodymyr Hrytsyk, Ivanna Klyujnyk, Bohdana Havrysh, Nataliia Lotoshynska, 2021, Identification of Biometric Images by Machine Learning, 978-1-6654-4296-1, 95, 10.1109/ELIT53502.2021.9501064
11.	Ömer Kasim, Mustafa Tosun, Biometric Authentication from Photic Stimulated EEG Records, 2021, 35, 0883-9514, 1407, 10.1080/08839514.2021.1981660
12.	Fatma Mallouli, Nesrine Khelifi, Aya Hellal, Imen Ferjani, Nada Chaabane, Mejda Dakhlaoui, Houda Chamakhi, 2023, Biometric Authentification Comparison: Toward Secure Human Recognition, 979-8-3503-6151-3, 1264, 10.1109/CSCI62032.2023.00206
13.	Ilija Tanasković, Ljiljana B. Lazarević, Goran Knežević, Nikola Milosavljević, Olga Dubljević, Bojana Bjegojević, Nadica Miljković, CardioPRINT: Biometric identification based on the individual characteristics derived from the cardiogram, 2025, 265, 09574174, 126018, 10.1016/j.eswa.2024.126018
14.	Vinodhini Chinnayan Meiyalagan, Sabeenian Royappan Savarimuthu, 2025, 3279, 0094-243X, 020170, 10.1063/5.0263047

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)