Detecting performance difficulty of learners in colonoscopy: Evidence from eye-tracking

  • Xin Liu University of Science and Technology Beijing; University of Alberta
  • Bin Zheng University of Alberta
  • Xiaoqin Duan Jilin University Second Hospital; University of Alberta
  • Wenjing He University of Manitoba
  • Yuandong Li Shanxi Bethune Hospital
  • Jinyu Zhao University of Alberta
  • Chen Zhao University of Science and Technology Beijing; Beijing Key Laboratory of Knowledge Engineering for Materials Science
  • Lin Wang University of Alberta
DOI: https://doi.org/10.16910/jemr.14.2.5
Keywords: colonoscopy, simulation, eye-tracking, navigation, Deep Convolutional Generative Adversarial Networks (DCGANs), Long Short-Term Memory (LSTM) networks

Abstract

Eye-tracking can help decode the intricate control mechanism in human performance. In healthcare, physicians-in-training requires extensive practice to improve their healthcare skills. When a trainee encounters any difficulty in the practice, they will need feedback from experts to improve their performance. The personal feedback is time-consuming and subjected to bias. In this study, we tracked the eye movements of trainees during their colonoscopic performance in simulation. We applied deep learning algorithms to detect the eye-tracking metrics on the moments of navigation lost (MNL), a signature sign for performance difficulty during colonoscopy. Basic human eye gaze and pupil characteristics were learned and verified by the deep convolutional generative adversarial networks (DCGANs); the generated data were fed to the Long Short-Term Memory (LSTM) networks with three different data feeding strategies to classify MNLs from the entire colonoscopic procedure. Outputs from deep learning were compared to the expert’s judgment on the MNLs based on colonoscopic videos. The best classification outcome was achieved when we fed human eye data with 1000 synthesized eye data, where accuracy (90%), sensitivity (90%), and specificity (88%) were optimized. This study built an important foundation for our work of developing a self-adaptive education system for training healthcare skills using simulation.

Author Biographies

Xin Liu, University of Science and Technology Beijing; University of Alberta

Xin Liu received her B.E. degree in electronic information engineering and Ph.D. degree in control science and engineering from the University of Science and Technology Beijing, China, in 2009 and 2015, respectively. From 2015 to 2017, she held a postdoctoral position in information and communication systems. Since 2017, she has been an Assistant Professor and M.E. Supervisor with the Computer Science and Technology Department, School of Computer and Communication Engineering, University of Science and Technology Beijing. From 2019 to 2020, she was a Visiting Professor with the Department of Surgery, University of Alberta. Her research interests include high-level data analysis, intelligent information processing, data mining, and decision making in medicine.

Bin Zheng, University of Alberta

Bin Zheng received his M.D. and M.S. degree in Medicine from China, and Ph.D. degree in Kinesiology from Simon Fraser University, Canada. Differing from most medical researches focusing on patients, he put healthcare providers under the spotlight. Explicitly, he studies the performance and cognition of physicians and surgeons during surgery, especially in the image-guided and robotic surgery, such as endoscopic and laparoscopic surgery. Currently, he is the Associate Professor in Surgery and the Endowed Research Chair in Surgical Simulation in the Department of Surgery of the University of Alberta. He collaborates with surgeons, engineers, clinical educators, and psychologists to develop artificial intelligence and simulation programs for training surgeons. His long-term goal is to prompt collaboration between engineers and healthcare providers for improving care quality and patient safety.

Xiaoqin Duan, Jilin University Second Hospital; University of Alberta

Xiaoqin Duan received her M.D. and Ph.D. degree from Jilin University, China. She is currently an Associate Professor in Rehabilitation Medicine. From 2019 to 2020, she was a Visiting Professor with the Department of Surgery, University of Alberta. She was awarded as “Excellent Young Rehabilitation Doctor” by China Rehabilitation Medicine Association in 2017 and 2018. Her research focuses on medical education and rehabilitation-engineering interdisciplinary. She collaborates with engineers to develop the rehabilitation robot for patients with paralysis. Prompting collaboration between engineers and physicians to improve patients' ability in daily life is her long-term research goal.

Wenjing He, University of Manitoba

Wenjing He completed her Ph.D. in surgery at the University of Alberta in 2019. She is currently a research associate working at Department of Surgery, the University of Manitoba. Her Ph.D. thesis research titled "Surgical Team and Team Assessment: Psychomotor Evidence". She was a general surgeon from China and then moved to Canada for her Ph.D. study. Combining her medical background and training in human factors research, her research interests include eye hand coordination of surgeons and objective assessment of teamwork using eye tracking devices and the application in medical team training.

Yuandong Li, Shanxi Bethune Hospital

Yuandong Li received his M.S. and Ph.D. in oncology at Tongji Medical College, Huazhong University of Science and Technology in China. He was a post-doctor in Surgery Simulation Research Lab at the University of Alberta to focus on the simulation training for physicians. He is currently an Associate Professor in Surgery at the Shanxi Bethune Hospital. He received the Shanxi Province Prize for Progress in Science and Technology. Combining his clinic medicine, his research interests include the treatment of cancer, surgical training skills, and digital technology application.

Jinyu Zhao, University of Alberta

Jinyu Zhao received a double Masters degree in electrical and computer engineering and physical chemistry from the University of Alberta and Northeast Petroleum University, in 2019 and 2017 respectively. She is currently a research assistant at the Surgical Simulation Research Lab at the University of Alberta with a research focus on deep learning for human behavior analysis.

Chen Zhao, University of Science and Technology Beijing; Beijing Key Laboratory of Knowledge Engineering for Materials Science

Chen Zhao is a second-year M.E. student in computer technology at the University of Science and Technology Beijing. Her research interests include machine learning in medicine data processing and computer-aided pattern recognition aiming at improving the interpretability of artificial intelligent systems for healthcare.

Lin Wang, University of Alberta

Lin Wang received her Ph.D. in psychology from the University of Alberta in 2018, where she focused on human spatial navigation in virtual reality. She was a post-doctoral researcher in the Department of Biomedical Engineering, School of Medicine, Johns Hopkins University from 2019 to 2020. She is currently a post-doctoral researcher in the Department of Surgery of the University of Alberta. She has been investigating rehabilitation of patients after acute vestibular loss. Her research aims at assisting clinicians to assess patients and prescribe medication more precisely using advanced technology and eventually improving rehabilitation outcome.

Published
2021-07-13
How to Cite
Liu, X., Zheng, B., Duan, X., He, W., Li, Y., Zhao, J., Zhao, C., & Wang, L. (2021). Detecting performance difficulty of learners in colonoscopy: Evidence from eye-tracking. Journal of Eye Movement Research, 14(2). https://doi.org/10.16910/jemr.14.2.5
Issue
Vol. 14 No. 2 (2021)
Section
Articles

License

Copyright (c) 2021 Xin Liu, Bin Zheng, Xiaoqin Duan, Wenjing He, Yuandong Li, Jinyu Zhao, Chen Zhao, Lin Wang

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.