Teleoperation control of a wheeled mobile robot based on Brain-machine Interface

Su-na Zhao; Yingxue Cui; Yan He; Zhendong He; Zhihua Diao; Fang Peng; Chao Cheng; Su-na Zhao; Yingxue Cui; Yan He; Zhendong He; Zhihua Diao; Fang Peng; Chao Cheng

doi:10.3934/mbe.2023170

Mathematical Biosciences and Engineering

2023, Volume 20, Issue 2: 3638-3660. doi: 10.3934/mbe.2023170

Previous Article Next Article

Research article

Teleoperation control of a wheeled mobile robot based on Brain-machine Interface

1.
College of Electrical and Information Engineering, Zhengzhou University of Light Industry, Zhengzhou 450000, China
2.
Zhongshan Institute, University of Electronic Science and Technology of China, Zhongshan 528402, China
3.
Key Laboratory of Bionic Engineering, Ministry of Education, Jilin University, Changchun 130022, China
4.
Weihai Institute for Bionics, Jilin University, Weihai 264402, China

Academic Editor: Hamidreza Namazi

Received: 12 October 2022 Revised: 01 November 2022 Accepted: 24 November 2022 Published: 09 December 2022

This paper presents a novel teleoperation system using Electroencephalogram (EEG) to control the motion of a wheeled mobile robot (WMR). Different from the other traditional motion controlling method, the WMR is braked with the EEG classification results. Furthermore, the EEG will be induced by using the online BMI (Brain Machine Interface) system, and adopting the non-intrusion induced mode SSVEP (steady state visually evoked potentials). Then, user's motion intention can be recognized by canonical correlation analysis (CCA) classifier, which will be converted into motion commands of the WMR. Finally, the teleoperation technique is utilized to manage the information of the movement scene and adjust the control instructions based on the real-time information. Bezier curve is used to parameterize the path planning of the robot, and the trajectory can be adjusted in real time by EEG recognition results. A motion controller based on error model is proposed to track the planned trajectory by using velocity feedback control, providing excellent track tracking performance. Finally, the feasibility and performance of the proposed teleoperation brain-controlled WMR system are verified using demonstration experiments.
- BMI,
- SSVEP-based system,
- Bezier curve,
- CCA
Citation: Su-na Zhao, Yingxue Cui, Yan He, Zhendong He, Zhihua Diao, Fang Peng, Chao Cheng. Teleoperation control of a wheeled mobile robot based on Brain-machine Interface[J]. Mathematical Biosciences and Engineering, 2023, 20(2): 3638-3660. doi: 10.3934/mbe.2023170

Related Papers:

Abstract

This paper presents a novel teleoperation system using Electroencephalogram (EEG) to control the motion of a wheeled mobile robot (WMR). Different from the other traditional motion controlling method, the WMR is braked with the EEG classification results. Furthermore, the EEG will be induced by using the online BMI (Brain Machine Interface) system, and adopting the non-intrusion induced mode SSVEP (steady state visually evoked potentials). Then, user's motion intention can be recognized by canonical correlation analysis (CCA) classifier, which will be converted into motion commands of the WMR. Finally, the teleoperation technique is utilized to manage the information of the movement scene and adjust the control instructions based on the real-time information. Bezier curve is used to parameterize the path planning of the robot, and the trajectory can be adjusted in real time by EEG recognition results. A motion controller based on error model is proposed to track the planned trajectory by using velocity feedback control, providing excellent track tracking performance. Finally, the feasibility and performance of the proposed teleoperation brain-controlled WMR system are verified using demonstration experiments.

References

[1]	Z. Li, W. Yuan, S. Zhao, Z. Yu, Y. Kang, C. P. Chen, Brain-actuated control of dual-arm robot manipulation with relative motion, IEEE Trans. Cognit. Dev. Syst., 11 (2017), 51–62. https://doi.org/10.1109/TCDS.2017.2770168 doi: 10.1109/TCDS.2017.2770168
[2]	Y. Yuan, W. Su, Z. Li, G. Shi, Brain-computer interface-based stochastic navigation and control of a semiautonomous mobile robot in indoor environments, IEEE Trans. Cognit. Dev. Syst., 11 (2018), 129–141. https://doi.org/10.1109/TCDS.2018.2885774 doi: 10.1109/TCDS.2018.2885774
[3]	Y. Chae, J. Jeong, S. Jo, Toward brain-actuated humanoid robots: asynchronous direct control using an EEG-based BCI, IEEE Trans. Rob., 28 (2012), 1131–1144. https://doi.org/10.1109/TRO.2012.2201310 doi: 10.1109/TRO.2012.2201310
[4]	L. Tonin, F. C. Bauer, J. D. R. Millan, The role of the control framework for continuous teleoperation of a brain-machine interface-driven mobile robot, IEEE Trans. Rob., 36 (2019), 78–91. https://doi.org/10.1109/TRO.2019.2943072 doi: 10.1109/TRO.2019.2943072
[5]	S. Zhao, Z. Li, R. Cui, Y. Kang, F. Sun, R. Song, Brain-machine interfacing-based teleoperation of multiple coordinated mobile robots, IEEE Trans. Ind. Electron., 64 (2016), 5161–5170. https://doi.org/10.1109/TIE.2016.2606089 doi: 10.1109/TIE.2016.2606089
[6]	X. Deng, Z. Yu, C. Lin, Z. Gu, Y. Li, A bayesian shared control approach for wheelchair robot with brain machine interface, IEEE Trans. Neural Syst. Rehabil. Eng., 28 (2019), 328–338. https://doi.org/10.1109/TNSRE.2019.2958076 doi: 10.1109/TNSRE.2019.2958076
[7]	Z. Li, S. Zhao, J. Duan, C. Y. Su, C. Yang, X. Zhao, Human cooperative wheelchair with brain-machine interaction based on shared control strategy, IEEE/ASME Trans. Mechatron., 22 (2016), 185–195. https://doi.org/10.1109/TMECH.2016.2606642 doi: 10.1109/TMECH.2016.2606642
[8]	B. Rebsamen, E. Burdet, C. Guan, H. Zhang, C. Teo, Q. Zeng, et al., Controlling a wheelchair indoors using thought, IEEE Intell. Syst., 22 (2007), 18–24. https://doi.org/10.1109/MIS.2007.26 doi: 10.1109/MIS.2007.26
[9]	E. Yin, Z. Zhou, J. Jiang, Y. Yu, D. Hu, A dynamically optimized SSVEP brain-computer interface (BCI) speller, IEEE Trans. Biomed. Eng., 62 (2014), 1147–1456. https://doi.org/10.1109/TBME.2014.2320948 doi: 10.1109/TBME.2014.2320948
[10]	T. Vouga, K. Z. Zhuang, J. Olivier, M. A. Lebedev, M. A. Nicolelis, M. Bouri, et al., EXiO-A brain-controlled lower limb exoskeleton for rhesus macaques, IEEE Trans. Neural Syst. Rehabil. Eng., 25 (2017), 131–141. https://doi.org/10.1109/TNSRE.2017.2659654 doi: 10.1109/TNSRE.2017.2659654
[11]	F. Janabi-Sharifi, I. Hassanzadeh, Experimental analysis of mobile-robot teleoperation via shared impedance control, IEEE Trans. Syst. Man Cybern. Part B Cybern., 41 (2010), 591–606. https://doi.org/10.1109/TSMCB.2010.2073702 doi: 10.1109/TSMCB.2010.2073702
[12]	C. Escolano, J. M. Antelis, J. Minguez, A telepresence mobile robot controlled with a nonoimvasive brain-computer interface, IEEE Trans. Syst. Man Cybern. Part B Cybern., 42 (2011), 793–804. https://doi.org/10.1109/TSMCB.2011.2177968 doi: 10.1109/TSMCB.2011.2177968
[13]	I. Iturrate, J. M. Antelis, A. Kubler, J. Minguez, A noninvasive brain-actuated wheelchair based on a P300 neurophysiological protocol and automated navigation, IEEE Trans. Rob., 25 (2009), 614–627. https://doi.org/10.1109/TRO.2009.2020347 doi: 10.1109/TRO.2009.2020347
[14]	A. Kelly, N. Chan, H. Herman, D. Huber, R. Meyers, P. Rander, et al., Real-time photorealistic virtualized reality interface for remote mobile robot control, Int. J. Rob. Res., 30 (2011), 384–404. https://doi.org/10.1177/0278364910383724 doi: 10.1177/0278364910383724
[15]	M. Lepetic, G. Klancar, I. Skrjanc, D. Matko, B. Potocnik, Time optimal path planning considering acceleration limits, Rob. Auton. Syst., 45 (2003), 199–210. https://doi.org/10.1016/j.robot.2003.09.007 doi: 10.1016/j.robot.2003.09.007
[16]	T. C. Liang, J. S. Liu, G. T. Hung, Y. Z. Chang, Practical and flexible path planning for car-like mobile robot using maximal-curvature cubic spiral, Rob. Auton. Syst., 52 (2005), 312–335. https://doi.org/10.1016/j.robot.2005.05.001 doi: 10.1016/j.robot.2005.05.001
[17]	E. Papadopoulos, I. Papadimitriou, I. Poulakakis, Polynomial-based obstacle avoidance techniques for nonholonomic mobile manipulator systems, Rob. Auton. Syst., 51 (2005), 229–247. https://doi.org/10.1016/j.robot.2005.03.006 doi: 10.1016/j.robot.2005.03.006
[18]	L. Fowler, J. Rogers, Bzier curve path planning for parafoil terminal guidance, J. Aerosp. Inf. Syst., 11 (2014), 300–315. https://doi.org/10.2514/1.I010124 doi: 10.2514/1.I010124
[19]	J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, T. M. Vaughan, Brain-computer interfaces for communication and control, Clin. Neurophysiol., 113 (2002), 767–791. https://doi.org/10.1016/S1388-2457(02)00057-3 doi: 10.1016/S1388-2457(02)00057-3
[20]	E. Candes, J. Romberg, T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inf. Theory, 52 (2006), 489–509. https://doi.org/10.1109/TIT.2005.862083 doi: 10.1109/TIT.2005.862083
[21]	J. W. Choi, R. E. Curry, G. H. Elkaim, Continuous curvature path generation based on Bzier curves for autonomous vehicles, IAENG Int. J. Appl. Math., 40 (2010), 91–101. https://doi.org/10.1016/j.cej.2005.12.011 doi: 10.1016/j.cej.2005.12.011
[22]	K. G. Jolly, R. S. Kumar, R. vijayakumar, A Bezier curve based path planning in a multi-agent robot soccer system without violating the acceleration limits, Rob. Auton. Syst., 57 (2009), 23–33. https://doi.org/10.1016/j.robot.2008.03.009 doi: 10.1016/j.robot.2008.03.009
[23]	Z. Sun, F. Li, X. Duan, L. Jin, Y. Lian, S. Liu, et al., A novel adaptive iterative learning control approach and human-in-the-loop control pattern for lower limb rehabilitation robot in disturbances environment, Auton. Rob., 45 (2021), 595–610. https://doi.org/10.1007/s10514-021-09988-3 doi: 10.1007/s10514-021-09988-3
[24]	L. Jin, J. Li, Z. Sun, J. Lu, F. Wang, Neural dynamics for computing perturbed nonlinear equations applied to ACP-based lower limb motion intention recognition, IEEE Trans. Syst. Man Cybern.: Syst., 52 (2021), 5105–5113. https://doi.org/10.1109/TSMC.2021.3114213 doi: 10.1109/TSMC.2021.3114213
[25]	Z. Sun, G. Wang, L. Jin, C. Cheng, B. Zhang, J. Yu, Noise-suppressing zeroing neural network for online solving time-varying matrix square roots problems: A control-theoretic approach, Expert Syst. Appl., 192 (2022), 116272. https://doi.org/10.1016/j.eswa.2021.116272 doi: 10.1016/j.eswa.2021.116272
[26]	Z. Sun, T. Shi, L. Jin, B. Zhang, Z. Pang, J. Yu, Discrete-time zeroing neural network of $O (\tau_{4})$ pattern for online solving time-varying nonlinear optimization problem: Application to manipulator motion generation, J. Franklin Inst., 358 (2021), 7203–7220. https://doi.org/10.1016/j.jfranklin.2021.07.006 doi: 10.1016/j.jfranklin.2021.07.006
[27]	W. Qi, H. Su, A cybertwin based multimodal network for ecg patterns monitoring using deep learning, IEEE Trans. Ind. Inf., 18 (2022), 6663–6670. https://doi.org/10.1109/TII.2022.3159583 doi: 10.1109/TII.2022.3159583
[28]	K. Liu, Y. Liu, Y. Zhang, L. Wei, Z. Sun, L. Jin, Five-step discrete-time noise-tolerant zeroing neural network model for time-varying matrix inversion with application to manipulator motion generation, Eng. Appl. Artif. Intell., 103 (2021), 104306. https://doi.org/10.1016/j.engappai.2021.104306 doi: 10.1016/j.engappai.2021.104306
[29]	H. Su, Y. Hu, H. R. Karimi, A. Knoll, G. Ferrigno, E. D. Momi, Improved recurrent neural network-based manipulator control with remote center of motion constraints: Experimental results, Neural Networks, 131 (2020), 291–299. https://doi.org/10.1016/j.neunet.2020.07.033 doi: 10.1016/j.neunet.2020.07.033
[30]	H. Su, W. Qi, Y. Schmirander, S. E. Ovur, S. Cai, X. Xiong, A human activity-aware shared control solution for medical human–robot interaction, Assem. Autom., 42 (2022), 388–394. https://doi.org/10.1108/AA-12-2021-0174 doi: 10.1108/AA-12-2021-0174
[31]	H. Su, A. Marian, S. E. Ovur, A. Menciassi, G. Ferrigno, E. D. Momi, Toward teaching by demonstration for robot-assisted minimally invasive surgery, IEEE Trans. Autom. Sci. Eng., 18 (2021), 484–494. https://doi.org/10.1109/TASE.2020.3045655 doi: 10.1109/TASE.2020.3045655
[32]	J. Chen, H. Qiao, Motor-cortex-like recurrent neural network and multitask learning for the control of musculoskeletal systems, IEEE Trans. Cognit. Dev. Syst., 14 (2020), 424–436. https://doi.org/10.1109/TCDS.2020.3045574 doi: 10.1109/TCDS.2020.3045574
[33]	H. Su, S. E. Ovur, X. Zhou, W. Qi, G. Ferrigno, E. D. Momi, Depth vision guided hand gesture recognition using electromyographic signals, Adv. Rob., 34 (2020), 985–997. https://doi.org/10.1080/01691864.2020.1713886 doi: 10.1080/01691864.2020.1713886
[34]	H. Su, W. Qi, C. Yang, J. Sandoval, G. Ferrigno, E. De Momi, Deep neural network approach in robot tool dynamics identification for bilateral teleoperation, IEEE Rob. Autom. Lett., 5 (2020), 2943–2949. https://doi.org/10.1109/LRA.2020.2974445 doi: 10.1109/LRA.2020.2974445
[35]	J. Chen, H. Qiao, Muscle-synergies-based neuromuscular control for motion learning and generalization of a musculoskeletal system, IEEE Trans. Syst. Man Cybern.: Syst., 51 (2020), 3993–4006. https://doi.org/10.1109/TSMC.2020.2966818 doi: 10.1109/TSMC.2020.2966818

Reader Comments

Your name:*

Email:*
© 2023 the Author(s), licensee AIMS Press. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0)