Arm exercise. Slice of 3D dataset a) at exercise, b) at rest, c) deformation field overlayed on exercise slice
Nonrigid registration techniques are designed to allow the warping of an anatomical structure onto another. This process is intended to provide a spatial normalization of the data so as to be able to compare structures scanned in different times, from different patients or with different imaging modalities. Applications of nonlinear registration include population-based atlas construction, where maps from different individuals are warped onto a canonical reference system, intersubject comparison of anatomy and function (for example to control for anatomic variability in functional studies), intrasubject warping for development studies and follow-up assessment of therapy and disease and also for intraoperative image-guided procedures, where high resolution preoperative scans are warped onto intraoperative ones.
A key step for most of these applications is template-based segmentation, where anatomic knowledge is incorporated into tissue classification algorithms by warping an atlas of normal anatomy into patient specific scans.
Warping can be considered an optimization problem where the parameters of a transformation relating two datasets are computed in order to maximize the similarity between the target and warped datasets subject to some constraints. Different schemes correspond to how the similarity is computed (ie. sum of squared differences, normalized cross-correlation, mutual information, etc), if the transformation is directly estimated from volumetric intensity information or from geometrical features extracted from the data (ie. points, curves, surfaces), how constraints are imposed to regularize the solution (ie. kinematic or optical flow models, biomechanical models, statistical dependence, etc.) and how the optimization is solved (ie. gradient descent, multigrid methods, direct search, etc.)
We have devoted considerable research efforts over the past several years to the nonrigid registration problem and we have developed some new schemes that have proved their success in different applications. Some of our results are briefly sumarized next with links to different papers published by our group on this topic.
The automatic segmentation of MRI images of the human brain can be approached as a registration problem. The elastic matching algorithm developed by Joachim Dengler, is a practical solution to the problem of fast elastic matching [1]. The elastic deformation between a volumetric atlas of the brain developed at the SPL and the brain of a patient with a large tumour was computed. The calculation required about 90 minutes real time and was done on an IBM RS/6000 workstation. The movie was generated by taking 30 steps of the deformation and shows the warping of the normal atlas into the configuration of the patient and back again. Each match step consists of a 3D volume of 256x256x123 voxels, projecting the atlas labels onto the patient's anatomy. Each data set was volume rendered on a Connection Machine to generate the movie frames.
The anatomical knowledge represented in the atlas and the fast elastic matching algorithm are allowing us to develop fully automatic algorithms for the segmentation of MRI of the human brain. As an example, this registration movie illustrates the process we have used to successfully identify a tumour from MRI. At first the atlas brain, shown surface rendered in blue, is aligned with the patient brain (rendered slightly transparent) using a rigid transformation. Then a non-rigid elastic match is computed to capture local shape differences between the atlas and patient brains. The movie shows the deformation field being applied to the atlas in 10 steps of 10% of the total deformation, and then being reversed and then being applied again. With the atlas in place, a localized analysis of voxel intensities allowed the tumour to be identified. It is shown rendered in green.
Michael Kaus investigated in his PhD Thesis [2] the performance of several optical flow methods for the purpose of aligning segmented images. Adaptive regularization of the optical flow is considered to be the most efficient way of constraining optival flow estimation but it leads to significant missregistrations if the underlying data has not been correctly segmented or if there are discontinuities in the displacement field. He consequently proposes a new method based on a probabilistic similarity function and offers a multigrid solution to the optical flow PDE.
These ideas are used in the framework of adaptive template-moderated classification to succesfully segment meningiomas and low grade gliomas in MRI [3].
We have developed a nonrigid registration algorithm [4] that finds the transformation that minimizes the sum of squared differences between the target and warped datasets constrained by an elastic model of the different structures that are present. This method is able to cope with the actual physical properties of the different structures. A global solution is found using a finite-element scheme that makes use of a tetrahedral mesh generator specifically designed for this project. In the figures warps estimated from muscle exercise imaging and ventricular deformation in multiple sclerosis are shown.
Arm exercise. Slice of 3D dataset a) at exercise, b) at rest, c)
deformation field overlayed on exercise slice
Enlarging ventricles. a) slice of difference between segmented
images at both time points (gray means no difference), b) deformation
field superimposed on same image at the first time point. c) close-up
One limitation of this method is the expense of computing the volumetric forces that would deform the elastic model of the tissues. Our previous approach implicitly made use of optical flow to obtain these forces which were not explicitly computed but a linear approximation of their contribution included in the goal function. Our next generation approach [5] makes use of a surface matching approach to precompute the forces on some boundary surfaces and interpolate the displocements in the inner volume using our previous finite-element based warping algorithm. Some results when the method is applied to intraoperative MRI registration in the following figures.
3D surface renderings of active surfaces ( a) brain surface, b)
lateral ventricles) with color-coded intensity of deformation field
a) Volumetric deformation field and initial landmarks (green)
overlayed on initial intraoperative image slice, b) Same slice of
deformed initial image with deformed initial landmarks (red), c) Same
slice of target image with deformed landmarks
3D Volumetric Deformation field (downsampled 12x, scaled 2x) with
orthogonal cuts through target intraoperative MR image and
transparently overlayed color coded intensity of the deformation
field a)Axial view, gravity is downwards. b)Coronal view, gravity
goes from left to right
We have also extended image matching schemes to deal with general tensor data [6]. In this case we automatically detect points with highly structured neighbors in the target dataset and we perform a direct search for a best match for every of these points in the other datset. We are currently using appropriate extensions of the sum of squared differences and of the normalized cross-correlation to compute the local similarity functions. The displacement in the remaining areas is carried out using optimal estimation theory by means of a Kriging estimator and assuming correlation properties for the deformation in different tissues. The whole approach is embedded in a Gaussian pyramid. This method can be considered a statistical counterpart of physic-based schemes, where the interpolation of the displacement is obtained using statistical principles instead of biomechanical models. The method has been used to estimate interpatient variability from tensor diffusion maps of the brain.
a) b) c)
DT-MRI interpatient warping. a,b) DT-MRI of different
individuals. c) zoomed T2W of the posterior corpus of a) and
estimated deformation.
MRI-guided brachytherapy is used at our institution for the treatment of prostate cancer. Preoperative 1.5T MRI data is acquired and segmented. Intraoperative guidance today is carried out using intraoperative 0.5T MRI in which, compared to preoperative MRI, it is difficult to perceive the prostate peripheral zone. 1.5T MRI can be used to indicate the location of the peripheral zone of the prostate for surgical planning, but the different positioning of the patient for preoperative MRI and for intraoperative MRI and brachytherapy leads to nonrigid deformation of the prostate, making direct comparison difficult. Early initial experiments have indicated that the technology we have used for capturing intraoperative brain deformation is also successful in capturing the nonrigid deformation that occurs here.
The figure below shows a 1.5T MRI data
rendered before deformation. We can see the bladder (light blue) and
the rectum (green), as well as the the tetrahedral mesh of the
Central Zone (red) and the Peripheral Zone (blue). The Peripheral
Zone (PZ) is where 75% of cancer occurs.
Below is a snapshot of the 1.5T MRI original data, a deformation of the prostate from this data, and the 0.5T MRI data to which the 1.5T data was matched. Comparison
of the overlap of the matched prostate
and the prostate seen in the 0.5T MRI scan indicate the central zone
and peripheral zone are both brought into alignment by the matching
process.
[3] Kaus et al. Segmentation of Meningiomas and Low Grade Gliomas in MRI.
In Proceedings of Second
International Conference on Medical Image Computing and
Computer-assisted Interventions, Cambridge, U.K. 1999. pp.1-10. SPL Technical Report #137, posted November 1999.
[4] Ferrant et al. 3D Image Matching Using a Finite Element Based Elastic
Deformation Model.(450kB
zipped PostScript version)
In Proceedings of Second International
Conference on Medical Image Computing and Computer-assisted
Interventions, Cambridge, U.K. 1999. pp.202-209. SPL Technical
Report #133, posted October 1999.
[5] Ferrant et al. Registration of 3D Intraoperative MR Images of
the Brain Using a Finite Element Biomechanical Model.
In Proceedings of Third International
Conference on Medical Image Computing and Computer-assisted
Interventions, Pittsburg, Pennsylvania, USA. 2000. pp.19-28. SPL Technical Report #177, posted October 2000.
[6] Ruiz-Alzola et al. Nonrigid Registration of 3D Scalar, Vector
and Tensor Medical Data.
In Proceedings of Third International
Conference on Medical Image Computing and Computer-assisted
Interventions, Pittsburg, Pennsylvania, USA. 2000. pp.541-550. SPL Technical Report #174, posted October 2000.