Surgical Planning Laboratory - Brigham & Women's Hospital - Boston, Massachusetts USA - a teaching affiliate of Harvard Medical School

Surgical Planning Laboratory

The Publication Database hosted by SPL

All Publications | Upload | Advanced Search | Gallery View | Download Statistics | Help | Import | Log in

Endoscopic Image Analysis in Semantic Space

1Kitware Inc., Chapel Hill, NC, USA.
2Statistical Visual Computing Laboratory, UC San Diego, USA.
3Department of Computer Sciences, University of Salzburg, Austria.
4Department for Internal Medicine, St. Elisabeth Hospital, Vienna, Austria.
5Department of Clinical Pathology, Medical University of Vienna, Austria.
Elsevier Science
Publication Date:
Med Image Anal
Volume Number:
Issue Number:
Med Image Anal. 2012 Oct;16(7):1415-22.
PubMed ID:
Image understanding, Pit pattern analysis, Semantic modeling
Appears in Collections:
R01 CA138419/CA/NCI NIH HHS/United States
U54 EB005149/EB/NIBIB NIH HHS/United States
Generated Citation:
Kwitt R., Vasconcelos N., Rasiwasia N., Uhl A., Davis B., Häfner M., Wrba F. Endoscopic Image Analysis in Semantic Space. Med Image Anal. 2012 Oct;16(7):1415-22. PMID: 22717411. PMCID: PMC3772630.
Downloaded: 853 times. [view map]
Paper: Download, View online
Export citation:
Google Scholar: link

A novel approach to the design of a semantic, low-dimensional, encoding for endoscopic imagery is proposed. This encoding is based on recent advances in scene recognition, where semantic modeling of image content has gained considerable attention over the last decade. While the semantics of scenes are mainly comprised of environmental concepts such as vegetation, mountains or sky, the semantics of endoscopic imagery are medically relevant visual elements, such as polyps, special surface patterns, or vascular structures. The proposed semantic encoding differs from the representations commonly used in endoscopic image analysis (for medical decision support) in that it establishes a semantic space, where each coordinate axis has a clear human interpretation. It is also shown to establish a connection to Riemannian geometry, which enables principled solutions to a number of problems that arise in both physician training and clinical practice. This connection is exploited by leveraging results from information geometry to solve problems such as (1) recognition of important semantic concepts, (2) semantically-focused image browsing, and (3) estimation of the average-case semantic encoding for a collection of images that share a medically relevant visual detail. The approach can provide physicians with an easily interpretable, semantic encoding of visual content, upon which further decisions, or operations, can be naturally carried out. This is contrary to the prevalent practice in endoscopic image analysis for medical decision support, where image content is primarily captured by discriminative, high-dimensional, appearance features, which possess discriminative power but lack human interpretability.

Additional Material
1 File (39.147kB)
Kwitt-MedImageAnal2012-fig3.jpg (39.147kB)