March 13, 2017 weblog
Using a machine-learning algorithm to rate surgeons on suturing skill level
(Tech Xplore)—A team of researchers at the University of Georgia has used a machine-learning algorithm to create a system to accurately rate the suturing skill level of a surgeon. In their paper uploaded to the preprint server arXiv, the team describes what went into development of the algorithm, how it was taught to perform, and how accurate it was compared to human assessors.
To make it all the way through medical school, internships and other training to become a surgeon, a student must demonstrate a variety of skills, both mental and physical. One of the most basic of those is suturing and tying knots once a wound has been closed. But as it turns out, despite showing sufficient proficiency to become certified, not all surgeons are created equal when it comes to closing up a patient's wounds.
As part of an effort to apply testing methodologies to physical procedures similar to those used during the development of drugs, the researchers sought to test the feasibility of creating systems to watch a surgeon perform certain procedures, and then to rate them on their skill level. To that end, the researchers created a system meant to judge just one skill: applying sutures and tying them off.
To create such a system, the researchers filmed 41 surgeons and nurses suturing test boards made of foam—each wore accelerometers on their hands to capture all of the action. The team then showed the videos to a clinician who rated the skill level of the subjects shown in the videos. Next, the video was fed to a computer running a machine-learning algorithm along with the scores from the clinician, which gave the system a basis for rating the work under review. Finally, the clinician's scores were removed from the system and it was then asked to rate suturing capabilities by itself. In looking at the results, the team found the new system to be 93.2 percent accurate in matching the rating of the original clinician rating.
The team also found that including data from the accelerometer did not improve accuracy, and in fact actually reduced effectiveness. The researchers suggest their system, or one like it, might one day be used by surgeons in training to get feedback on their skill level prior to undergoing critique by other surgeons.
More information: Video and Accelerometer-Based Motion Analysis for Automated Surgical Skills Assessment, arXiv:1702.07772 [cs.CV] arxiv.org/abs/1702.07772
Abstract
Purpose: Basic surgical skills of suturing and knot tying are an essential part of medical training. Having an automated system for surgical skills assessment could help save experts time and improve training efficiency. There have been some recent attempts at automated surgical skills assessment using either video analysis or acceleration data. In this paper, we present a novel approach for automated assessment of OSATS based surgical skills and provide an analysis of different features on multi-modal data (video and accelerometer data). Methods: We conduct the largest study, to the best of our knowledge, for basic surgical skills assessment on a dataset that contained video and accelerometer data for suturing and knot-tying tasks. We introduce "entropy based" features - Approximate Entropy (ApEn) and Cross-Approximate Entropy (XApEn), which quantify the amount of predictability and regularity of fluctuations in time-series data. The proposed features are compared to existing methods of Sequential Motion Texture (SMT), Discrete Cosine Transform (DCT) and Discrete Fourier Transform (DFT), for surgical skills assessment. Results: We report average performance of different features across all applicable OSATS criteria for suturing and knot tying tasks. Our analysis shows that the proposed entropy based features out-perform previous state-of-the-art methods using video data. For accelerometer data, our method performs better for suturing only. We also show that fusion of video and acceleration features can improve overall performance with the proposed entropy features achieving highest accuracy. Conclusions: Automated surgical skills assessment can be achieved with high accuracy using the proposed entropy features. Such a system can significantly improve the efficiency of surgical training in medical schools and teaching hospitals.
via Newscientist
© 2017 Tech Xplore
