In addition to its research activities, the Surgical Planning Laboratory at Brigham and Women's Hospital in Boston provides surgeons with 3D reconstructions from cross-sectional MR or CT data, to be used both in pre-operative planning and as an aid during surgery. In the cross-sectional images, important structures (e.g. blood vessels, tumor, brain structures) are segmented, and yield separate structures in the reconstruction. Even after the application of algorithms that smooth and simplify the underlying triangle mesh representation of these reconstructions, the models often consist of nearly 100,000 triangles. As a result, the surgeon must use a graphics workstation in the lab to view the models and interact with them. In addition, while these medical datasets are a valuable tool for teaching complex anatomy, most medical students do not have access to the 3D graphics hardware necessary for viewing and interaction. We developed a Java applet, the ``SPL Anatomy Browser,'' that utilizes MLIs to permit remote users (e.g. surgeons and students) to access and interact with the 3D reconstructions from surgical cases.
The applet's display is divided into three areas: the first displays the 3D model, the second displays the cross-sectional slices from which it was derived, and the third displays a hierarchical description of the structures present in the models. In the 3D area, the user may select from among six views of the model (top, bottom, left, right, front, and back), and by using key-modified mouse clicks on structures in the model, he or she can remove structures, display annotation associated with a structure, or request cross-hairs on the cross-sectional slice displays corresponding to the mouse position in the 3D model. In both the 3D and cross-sectional slice display areas, the user can execute a fixed-resolution zoom by dragging the mouse to draw a bounding box over the region of interest. In the hierarchy, the user may select/de-select structures for permanent annotation, adjust the structure's transparency, and expand or collapse the hierarchy to adjust the level of detail in the 3D display. For example, the effect of collapsing the sub-structures listed as part of the brain is to adjust the coloring of the model so that all of the different brain lobes are uniformly colored, to appear as a single structure.
To support easy generation of data for the applet we wrote a back-end script that runs on a graphics workstation in the lab. To use the script, the user needs only to view the 3D model, select appropriate lighting, and enter information about the original cross-sectional slices. The back-end automatically generates compressed MLIs for each of the six views. Finally, the user must provide a text file that describes the hierarchy of structures present in the model, and the desired colors for each part. Using these tools, any 3D model that is generated in the day-to-day operations of the lab can be imported into the applet with minimum effort. Click here for a screen shot of the Anatomy Browser displaying an abdominal dataset.
To demonstrate the Anatomy Browser's potential use in medical education, we imported models from our brain atlas [21] [12], an extensively segmented brain 3D reconstruction. By viewing and interacting with these models within the Anatomy Browser, medical students and teachers anywhere on the Internet can utilize an valuable educational resource that was previously inaccessible to anyone outside the lab. In fact, even researchers with access to graphics workstations found interaction with the 3D models to be sluggish due to the dataset's extraordinary size, a problem that is alleviated by viewing the models using MLIs.
