Analytics of deep model-based spatiotemporal and spatial feature learning methods for surgical action classification

Classification of surgical actions from a real anatomy video sequence is a challenging task due to limited visibility, poorer lighting conditions, lower contrast, and obscured frames of a video sequence. Several deep model-based spatiotemporal and spatial feature learning methods have been presented to classify surgical actions. However, some of the methods have been evaluated on synthetic data due to unavailability of sufficient labeled data. Conversely, some methods have been evaluated on a real anatomy dataset, but produce lower accuracy. Therefore, in this paper, first we analyze the effects of both feature learning methods on surgical action classificaiton from a real anatomy dataset. Thereafter, we propose new methods to enhance surgical action classification. Specific contributions in this paper are as follows. First, two novel deep model-based spatiotemporal feature learning methods are proposed to classify surgical actions. Second, a hypothesis is proposed, stating that the elimination of spatiotemporal features does not affect the performance of the method. Third, to test the proposed hypothesis, a spatial feature learning method comprised of a unique custom Convolutional Neural Network (CNN) is also proposed. Fourth, performance analysis of the proposed spatiotemporal and spatial feature learning methods is presented using a real anatomy Surgical Actions 160 dataset. The experimental results demonstrate that the MobileNetV2-based spatial feature learning method achieves the highest accuracy of 97% in classifying surgical actions. It outperforms the other pre-trained CNN models significantly. Additionally, MobileNetV2 exhibits lower training time.