Document Type
Article
Publication Date
8-1-2021
Publication Title
Sensors
Abstract
Accurate semantic image segmentation from medical imaging can enable intelligent vision-based assistance in robot-assisted minimally invasive surgery. The human body and surgical procedures are highly dynamic. While machine-vision presents a promising approach, sufficiently large training image sets for robust performance are either costly or unavailable. This work examines three novel generative adversarial network (GAN) methods of providing usable synthetic tool images using only surgical background images and a few real tool images. The best of these three novel approaches generates realistic tool textures while preserving local background content by incorporating both a style preservation and a content loss component into the proposed multi-level loss function. The approach is quantitatively evaluated, and results suggest that the synthetically generated training tool images enhance UNet tool segmentation performance. More specifically, with a random set of 100 cadaver and live endoscopic images from the University of Washington Sinus Dataset, the UNet trained with synthetically generated images using the presented method resulted in 35.7% and 30.6% improvement over using purely real images in mean Dice coefficient and Intersection over Union scores, respectively. This study is promising towards the use of more widely available and routine screening endoscopy to preoperatively generate synthetic training tool images for intraoperative UNet tool segmentation.
Keywords
Generative adversarial networks, Medical imaging, Robot-assisted minimally invasive surgery, Surgical tool segmentation, UNet
Volume
21
Issue
15
DOI
10.3390/s21155163
ISSN
14248220
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.
Rights
© 2021 by the authors
Recommended Citation
Su, Yun Hsuan; Jiang, Wenfan; Chitrakar, Digesh; Huang, Kevin; Peng, Haonan; and Hannaford, Blake, "Local Style Preservation in Improved GAN-Driven Synthetic Image Generation for Endoscopic Tool Segmentation" (2021). Engineering: Faculty Publications, Smith College, Northampton, MA.
https://scholarworks.smith.edu/egr_facpubs/131
Comments
Archived as published. Open access article.