1216 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 24, NO. … · 1216 IEEE TRANSACTIONS ON MEDICAL...

1216 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 24, NO. 9, SEPTEMBER 2005

Large Deformation Diffeomorphic Metric Mappingof Vector Fields

Yan Cao*, Michael I. Miller, Senior Member, IEEE, Raimond L. Winslow, and Laurent Younes

Abstract—This paper proposes a method to match diffusiontensor magnetic resonance images (DT-MRIs) through the largedeformation diffeomorphic metric mapping of vector fields, fo-cusing on the fiber orientations, considered as unit vector fields onthe image volume. We study a suitable action of diffeomorphismson such vector fields, and provide an extension of the Large Defor-mation Diffeomorphic Metric Mapping framework to this type ofdataset, resulting in optimizing for geodesics on the space of dif-feomorphisms connecting two images. Existence of the minimizersunder smoothness assumptions on the compared vector fields isproved, and coarse to fine hierarchical strategies are detailed, toreduce both ambiguities and computation load. This is illustratedby numerical experiments on DT-MRI heart images.

Index Terms—Diffeomorphism, diffusion tensor MRI, imageregistration, variational methods, vector field.

I. INTRODUCTION

D IFFUSION tensor magnetic resonance imaging(DT-MRI) probes and quantifies the anisotropic diffusion

of water molecules in biological tissues, making it possible tononinvasively infer the architecture of the underlying structures[1]–[4]. The measurement at each voxel in a DT-MRI imagevolume is a symmetric second-order tensor. Orientation ofthe principal eigenvector of the diffusion tensor is known toalign with fiber tracts in brain [1] and heart [3]. Consequently,DT-MRI is becoming a routine magnetic resonance imagingtechnique for studying fiber orientation in biological tissue.

DT-MRI provides useful physiological information nonin-vasively, not only about the fiber structure of normal tissue,but also about its changes in development, aging, disease anddegeneration. It has already been shown to be of value instudies of neuroanatomy, fiber connectivity, and brain develop-ment. DT-MRI has been used in the investigation of cerebralischemia, brain maturation and traumatic brain injury. It also

Manuscript received April 14, 2005; revised June 13, 2005. This work wassupported in part by the National Institutes of Health (NIH) under Grant 1-R01-HL70894, in part by the Falk Medical Trust, in part by the NIH under GrantP41-RR15241, Grant P50 MH71616, and Grant AG20012. The Associate Ed-itor responsible for coordinating the review of this paper and recommending itspublication was B. Lelieveldt. Asterisk indicates corresponding author.

*Y. Cao is with the Center for Imaging Science, Johns Hopkins University,Baltimore, MD 21218 USA (e-mail: [email protected]).

M. I. Miller is with the Center for Imaging Science and the Department ofBiomedical Engineering, Johns Hopkins University, Baltimore, MD 21218 USA(e-mail: [email protected]).

R. L. Winslow is with the Center for Cardiovascular Bioinformatics & Mod-eling and the Whitaker Biomedical Engineering Institute, Johns Hopkins Uni-versity, Baltimore, MD 21218 USA (e-mail: [email protected]).

L. Younes is with the Center for Imaging Science and the Department ofApplied Mathematics and Statistics, Johns Hopkins University, Baltimore, MD21218 USA (e-mail: [email protected]).

Digital Object Identifier 10.1109/TMI.2005.853923

promises to further our understanding of brain disorders andabnormalities such as stroke, tumors and metabolic disorders,epilepsy, multiple sclerosis, schizophrenia, Alzheimer’s dis-ease and cognitive impairment [5]. DT-MRI is also useful inthe study of normal and abnormal cardiac electro-mechanicalproperties since both geometry and fiber orientation play acritical role in the electro-mechanical behavior of the heart,such as electrical propagation [6]–[8], mechanical propertiessuch as stress and strain [9], [10]. Fiber orientation is knownto be altered in various cardiac diseases such as ischemic heartdisease [11] and ventricular hypertrophy [12]–[14]. Detailedinformation of the cardiac macrostructure in normal and dis-eased hearts is, therefore, of fundamental importance to theunderstanding of electro-mechanical properties in health anddisease. At the moment, the real value of DTMRI applied toanalysis of cardiac structure is not clinical, but rather, as animportant research tool. The imaging sessions require approx-imately 40–60 hours per heart and also require that the heartis in immobilized fixation. The importance of DT-MRI hasto do with the fact that it provides for detailed anatomicalreconstruction times that are many orders of magnitude fasterthan can be achieved using traditional histological approaches(manual dissection and mapping of the ventricles pioneered bythe Auckland University group of P. Hunter [15]).

Our work is motivated by the investigation of how remod-eling of ventricular geometry and fiber organization influencesventricular conduction. To do this, it is necessary to develop andapply mathematical methods for identifying statistically signif-icant changes in ventricular geometry and fiber organizationthat occur during remodeling. We seek to develop a transforma-tion-based probabilistic atlas describing variation of ventriculargeometry and fiber organization in normal and failing humanand canine ventricles. Our approach is to apply the theory oflarge deformation diffeomorphic metric mapping to transform atemplate heart into target hearts, computing the metric distancesof the transformations required for registration as a measure ofthe differences between template and target, and building prob-ability distributions describing variability of geometry of targethearts in terms of the variability of these transformations. Thetemplate we consider is the anatomy which includes both theshape geometry and the fiber organization.

We hypothesize that use of DT-MRI data during registrationwill provide more structural information to guide the registra-tion process. For example, DT-MRI images of brain providesstructure to white matter regions while conventional MRI im-ages describe it as a homogeneous region [16]. The cardiacventricles are also known to have a complex fiber architecture,

0278-0062/$20.00 © 2005 IEEE

CAO et al.: LARGE DEFORMATION DIFFEOMORPHIC METRIC MAPPING OF VECTOR FIELDS 1217

with fiber orientation changing as a function of transmural loca-tion [17], [18]. Fiber tracts in the heart are arranged as counter-wound helices, with the spatial orientation of the helices varyingby approximately 120 through the wall of the heart. In addition,extreme fiber rotation occurs near the apex of the heart, wherethe spiraling epicardial fibers plunge into the infundibulum toemerge on the endocardial surface. The helical structure of theheart is shown in the left panel of Fig. 11, in which fiber tracingmethods are applied to diffusion tensor MRI data to reconstructfiber trajectories in the cardiac ventricles. Right panel of Fig. 11shows the extreme fiber rotation that occurs near the apex of theheart. Strong orientation of fiber structure can be observed. Reg-istration making use of both cardiac geometry as well as fiberorientation may be more accurate than that achieved using geo-metric information alone.

Image registration for quantitative analysis of tissue ge-ometry and fiber structure based on DT-MRI data is muchmore complicated than that based on scalar image data sinceDT-MRIs contain structural information which is affectedby the transformation. Though significant efforts have beendirected at scalar image registration, little work has been doneon matching tensor images. Alexander and Gee [16] adaptedthe multi-resolution elastic matching algorithm [19], [20] formatching DT-MRIs. Different tensor similarity measures todrive the elastic matching were compared empirically. Tensorreorientation is not included in the energy function, but tensorsare reoriented in each iteration according to the estimateddisplacement field. Ruiz-Alzola [21] proposed a unified frame-work for registration of medical volumetric multi-valued databy matching local areas with high structure. Local structureis detected by measuring “cornerness.” Then, the Krigingestimator was used to interpolate the displacement field inthe remaining areas. Tensor reorientation is performed afterthe registration. Guimond et al. [22] proposed a deformableregistration method using tensor characteristics that are in-variant to rigid transformations to avoid the complexity oftensor reorientation. The invariants used in their experimentsare the tensor eigenvalues. Rohde et al. [23] proposed anintensity based registration method capable of performingaffine and nonlinear registration of multichannel images. Themultichannel similarity measure considered both the similaritybetween same-index image channels and the similarity betweenimage channels with different index. But tensor reorientation isperformed after affine registration and after the nonlinear reg-istration is complete. All the methods above do not have tensorreorientation in the optimization, which means the measureof difference does not take all the available information intoaccount.

This paper proposes a method to match DT-MRIs throughthe large deformation diffeomorphic metric mapping of vectorfields, focusing on fiber orientation, considered as unit vectorfields on the image volume. Following the Large DeformationDiffeomorphic Metric Mapping paradigm, we model the imagespace as the orbit of a template under the action of a group ofdiffeomorphisms. Optimal matching is computed by optimizingfor the best geodesic path in connecting the template to itstarget. We define an action of diffeomorphisms on fiber orienta-

tion, which is notably different in nature from the usual actionon scalar images. Our group action is identical to that proposedby Alexander et al. [24], [25] restricted to the first eigenvector ofthe tensor. Our method includes tensor reorientation in the op-timization, hence, the measure of difference is calculated cor-rectly during the optimization. Our approach also insures thesmoothness and consistency of the Jacobian which plays a fun-damental role in transforming vector information under the dif-feomorphisms.

Our approach is developed on the basis of the previous studiesof variational problems on flows of diffeomorphism for imagematching, along the lines of Grenander’s deformable templatetheory, as developed in Dupuis et al. [26], Trouvé [27], [28],Miller et al. [29] and Beg et al. [30]. The transformations gen-erated using this method are invertible and smooth. Local neigh-borhood anatomic structure is also preserved. More recent andrelated work includes Avants and Gee [31], Joshi et al. [32],Marsland and Twining [33].

In our approach, anatomy is modeled as a deformable tem-plate. The images are defined on an open bounded setand are viewed as the orbit under the diffeomorphic transforma-tions acting on a template. Here, two things are very impor-tant, the diffeomorphic transformations and the definition ofthe action.

Elements in are defined through the solutions of the non-linear Eulerian transport equation

(1)

For each , is a vector field on which belongs to someHilbert space . The meaning of this equation is that we havea path on the space of transformations , where is the time.Under the action of this flow, a particle which starts at a point

, traces out a path , where ,is the speed at time .One way to build the Hilbert space is through a differen-

tial operator, like in [26], [30]. Another way is to construct itthrough kernels using the theory of reproducing kernel Hilbertspace (RKHS) [34], as in [35]. In our approach, we will buildthe appropriate Hilbert space through kernels.

The usual action of diffeomorphisms on scalar images is viathe composition of the image with the inverse diffeomorphism.Under this definition of action, only the background coordinatespace changes, with the range of image values remaining fixed.This action definition cannot be applied to DT-MRI becausefiber orientation also has to be affected by the deformation. Weneed to extend the action to also operate on vector fields. Thisis detailed in the next section.

II. FORMULATION OF THE PROBLEM

First, we introduce some notation. An -valued functionis treated as a 1 column vector, is the th compo-

nent of . The Jacobian operator gives the matrixfor -valued functions and the 1row vector for scalar valued functions de-fined on -dimensional domain. The Hessian operator givesthe matrix like structure for -valued functions

and the matrix


for scalar valued functions defined on a -dimen-sional domain. In fact, is the Hessian of . We use theEinstein summation convention, summation is implied when anindex is repeated on upper and lower levels. The notationdenotes the value of at .

We model the images as the orbit of a template under dif-feomorphic transformations. Let the background space be abounded domain in and a group of diffeomorphisms on

. Let the images be functions that associate toeach point a vector, the principal direction of the diffu-sion tensor. (For outside the object, the vector will be zero.The object can be part of the image which we are interested inor the whole image.) acts on the set of all images. We mayview the object as a manifold in and the fiber orientation asa vector field on the manifold. The usual action of diffeomor-phisms on vector fields operates according to the Jacobian, with

, here is a point on the manifold,is a tangent vector at and is the Jacobian of the transfor-mation evaluated at . But if we apply that rule, the length ofthe vector also changes. That means the tissue micro-structurechanges during the deformation, which is not desirable. So werenormalize the vector to the original length. The action is de-fined as follows,

Definition: For any and

when

when(2)

Here, is the Euclidean norm in .If the transformation is , then the action of on a vector

transforms the vector to another vector which is de-fined in (2). (See Fig. 1.) To understand the definition, considera point on the deformed grid, the corresponding point in thetemplate image is . This point will move to the po-sition after the mapping has been applied. The vector at thispoint will be transformed by the Jacobian at point

, i.e., to be , andthen renormalized to the original length to become the vector atpoint in the transformed space under . It can be shown thatthe action defined above is a group action using the chain ruleof differentiation and the fact that a Jacobian acts linearly.

Alexander et al. [24], [25] considered different strategies forthe way a deformation field should act on a tensor-valued image.An eigenvector deformation strategy gave the best results intheir experiments. Suppose the eigenvectors of the diffusiontensor are , and and the Jacobian of the deformationis . The eigenvector deformation strategy assumes that theeigenvalues of the diffusion tensor do not change, the principaleigenvector changes according to the Jacobian, but preserve thelength, i.e., changes to . The plane generated bythe first and the second principal vectors and changes tothe plane generated by and . Our action here is the re-striction of this strategy to the first eigenvector.

Proposition 2.1: Define property on .For any and

Fig. 1. Action of the transformation'. For a point y on the grid in the range ofmapping, the corresponding point in the template image is x = ' (y). Thispoint will move to the position y after the mapping has been applied. The vectorat this point ' (y) will be transformed by the Jacobian D' at point ' (y)and then renormalized to the original length to become the vector at point y inthe transformed space under '.

and if satisfies the property , then

In the remainder of this paper, we assume the image satisfiesthe property , and will not single out the case when .This property can be satisfied by smoothing the image at theboundary of the set .

Let the notation denote the composition. Then, is the solution at time of the equation

with initial condition . The functionis called the flow associated to starting at . It

is defined on . In physical terms, is the motionof a particle which starts at at time . For , wehave . This equation states that the movementfrom to is the same as the movements from to , and thenfrom to .

Given two elements and , we would like to find an op-timal matching between and which minimizes a distancefunction on -dimensional vector fields, , in thespace of diffeomorphisms. The mapping is generated as theend-point of the flow of smooth time-dependentvector fields via (1).

In the image, if the object which we are interested in has beensegmented, then all the points inside the object are representedby vectors with length greater than 0, all the background pointsare represented by zero vectors. If the object has not been seg-mented, our method can still be applied using the original vectorfield as long as the template and target images have similar back-ground. Segmentation is commonly used in medical image pro-cessing.

We require that nonzero vectors prefer matching othernonzero vectors rather than zero vectors which essentiallycorrespond to the background. This is not guaranteed by theEuclidean distance, since for any two vectors and in ,

if, where is the angle between and . Therefore, we

define . Hence

(3)

The penalty term is added to force nonzero vectors to prefermatching nonzero vectors. If the object has been segmented,then the norm of the vector field forms a image which shows


the geometry of the object. So in this case, intuitively, the firstterm is for matching vectors, the second the term is for matchinggeometric shape. We summarize the problem as follows.

Problem Statement: Estimation of the optimal diffeomor-phic transformation connecting two images of vector fieldsand is done via the variational problem shown in (4) at thebottom of the page, where is an appropriate functionalnorm on the velocity field .

Remark 1: We would like to point out that the extension ofthe Alexander and Gee’s work to large deformation theory isnontrivial. Our method is quite different from Alexander andGee’s. We use the same strategy to reorient the DT-MRI dataand formulate the problem in a variational framework, that is allwhere it is in common. They adopted a scalar elastic matchingalgorithm to do the DT-MRI matching. Our formulation is basedon diffeomorphic flows, resulting in optimizing for geodesics onthe space of diffeomorphisms connecting two images. Estab-lishing this methodology continues our efforts on establishingthe construction and computation of metric spaces for studyinganatomical structures.

The optimization strategies are also very different. Supposethe two DT-MRI images to be matched are and , thedeformation is . We denote the deformed version ofunder , i.e., . Alexander and Gee’s energy functionto be minimized is

Alexander and Gee’s algorithm works as follows:Start from and . Alternate between the two steps:

1)

2)Suppose this algorithm converges with solutions and ,

then

In fact, depends on , the solution above is not the solutionof

Our algorithm does not have this problem. We viewed the en-ergy function as a function of , calculated the first-order varia-tion with respect to and use a gradient descent based methodto get the solution. Tensor reorientation is included in the mini-mization process. Convergence of the algorithm is guaranteed.

III. FIRST VARIATION OF THE ENERGY FUNCTION

To solve the variational problem (4), we need to compute thefirst-order variation of the energy function with respect to thevector field .

Definition: Let and be time-dependent vector fields,, such that for each , and ,

. For a function , we define

(5)

Here, is called the Gâteaux differential of at with in-crement and is called the Fréchet derivative of [36].

We construct the Hilbert space using the theory of RKHS[34]. RKHS has the reproducing property, that is, there exists aKernel function such that, for any , and ,we have . Some commonly usedkernels include Gaussian kernels andpolynomial kernels .

In our case, the background space is a bounded domainin , is a vector field with zero boundary conditions. Sowe need the kernel function to have compact support in . TheHilbert Space is defined as

(6)

where is continuous, positive definite and has com-pact support in . The inner product on is definedas . The reproducing kernel is

.The smoothness of in depends on the smoothness of

the kernel function . We should choose a proper kernel,such that is embedded in , to assure that there al-ways exists a minimizer of the energy function defined by (4).See Appendix I for the existence theorem and proof. Let bea Hilbert space which is embedded in , then the flow

associated to is for all times a diffeomorphism of [26].The velocity vector field solving the variational problem (4) de-fines a “geodesic” path on the manifold of diffeomorphisms andthe length of this path is a metric distance betweenthe images connected by the diffeomorphism [27].

Theorem 3.1: Suppose is a continuously differentiableidealized template image of vector fields and satisfies the prop-erty , is the target image, then for inexact matching ofand is given by (4) and the Euler-Lagrange equation corre-sponding to the first-order variation of is given by (7) (shown

(4)


at the bottom of the page), where denotes the th columnof , , ,and .

See Appendix II for the proof.Remark 2: We use the RKHS [34] formalism to build the

Hilbert space instead of the differential operator formalismsince it provide a convenient framework for efficient com-putations and it allows more general boundary conditions tobe imposed on the velocity fields. We use the zero boundaryconditions in our implementation. Other boundary conditionslike natural (Neumann) boundary conditions can be imposedtoo. It is not natural to impose boundary conditions on theedges of the image, but as long as the object we are interestedis not too close to the boundary, the effects will be very small.

The theory of RKHS was established in 1950 and has sincedeveloped into a beautiful mathematical edifice. These ab-stract objects have found applications in a spread of areas ofmathematics as well as in some related disciplines, such asapproximation theory, probability, numerical analysis, regres-sion methods [37], orthogonal expansions, inequalities, signalprocessing, functional analysis and complexity.

Remark 3: To take advantage of the Fourier transform inthe calculations related to the kernel, we define

, where is continuous, positive definiteand translation invariant, i.e., is a function of ,

is a smooth function. For example, in a two-dimensional(2-D) case, we may take be a Gaussian function andbe the function shown in Fig. 2.

Defining the kernel this way has the advantage that the cal-culations related to the kernel can be computed efficiently byFourier transformation. Suppose , then

Here, we use and to denote the Fourier transform andthe inverse Fourier transform.

Fig. 2. This figure shows an example of � function.

Remark 4: In our definition of the Hilbert space , the el-ement is defined by means of functions . In our exper-iments, we find it is more efficient to work on directly. Wedefine

(8)

So, for , we have

Using the same method as above we can prove (9), shown atthe bottom of the page.

IV. NUMERICAL IMPLEMENTATION

This section presents details of the numerical implementa-tion of the estimation of the optimal vector field . Let

be the 2D/3D background space. is discretizedinto a uniform square/cube mesh. The time space is discretized

(7)

(9)


as , and . Letdenote the value of at , in time step for iteration.

A. Gradient Descent Scheme Based Optimization

The variational optimization of the energy functional is per-formed in a steepest descent scheme

(10)

where is computed using (9). The derivatives are calcu-lated via central differences with enforcement thatif the function involves and . The integrations arecalculated by Fourier transformation since they can be viewedas convolutions. We use the Gaussian kernel functions as inour experiments. Other kernel functions can be used.

The step size is decided by a line search algorithm in thedirection of steepest descent. We use the golden section searchalgorithm [38]. Suppose the minimum is bracketed in a tripletof points, , such that is less than bothand . The next point to be tried is that which is a fraction0.382 into the larger of the two intervals (measuring from thecentral point of the triplet, ). Then, we form a new bracketingtriplet of points. If , then the new bracketing tripletof points is ; otherwise, if , then the newbracketing triplet is . This process is repeated until thedistance between the two outer points of the triplet is tolerablysmall. We initially bracket a minimum by an exponential search.

B. Integration of Velocity Field to Generate Maps

Another issue is how to integrate the velocity field to generatethe map and . Both and satisfy a pure advectionequation. For , , ; for ,

, , then

This equation is solved using a semi-Lagrangian scheme.Such schemes regulate the number and distribution of the fluidparcels by choosing a completely new set of parcels at everytime step. The parcels making up this set are those arriving ateach node on a regularly spaced grid at the end of each step.This scheme keeps the parcels evenly distributed throughout thefluid and facilitates the computation of spatial derivatives viafinite difference. Semi-Lagrangian schemes avoid the primarysource of nonlinear instability because the nonlinear advec-tion terms appearing in the Eulerian form of the momentumequations are eliminated when those equations are expressedin a Lagrangian frame of reference. They are, in particular,not subject to Courant-Friedrichs-Lewy stability conditions[39] which restrict the efficiency of purely Eulerian schemes.Note that, because of the effect of space interpolation, standardnumerical schemes for the solution of differential equations arenot adapted to our problem.

We use a second-order semi-Lagrangian scheme [40] to solvethe equation. Let and be the discretization of the timespace and background space, then a semi-Lagrangian approx-imation to the advection equation can be written in the form

Fig. 3. Top left shows the template image, top right shows the target image.Bottom row shows the matching results with two images superimposed, blackthe template and gray the target. Left panel shows the shape matching case, rightpanel shows the vector field matching case.

Fig. 4. The left panel shows the energy for vector field difference at eachiteration; the right panel shows the energy for shape difference at each iteration.

, where denotes the depar-ture point of a trajectory originating at time and arriving at

. A fully second-order semi-Lagrangian scheme canbe obtained by (i) using toesti-mate the midpoint of the back trajectory, (ii) computing

by linearly interpolating and extrapolating the velocity field, (iii) de-

termining the departure point from , (iv)evaluating using quadratic interpolation, (v) setting

to this value.

C. Hierarchical Multiresolution Matching Strategy

A hierarchical multiresolution matching strategy is used toreduce ambiguity issues and computation load. It is employedfrom coarse to fine, and results achieved on one resolution areconsidered as approximations for the next finer level. We gen-erate the image pyramid by reducing the resolution from onelevel to the next by a factor of 2. We use norm-preserving linearinterpolation to convert the vector field from one level to thenext finer level, i.e., both the vector and the norm of the vectorare interpolated linearly and the norm of the interpolated vectoris normalized to the interpolated norm.

D. Rigid Matching

We performed a rigid transformation matching before ap-plying the gradient descent scheme. We use the second-ordercentral moments of the objects. All these moments form amatrix, the eigenvectors of this matrix are called the principalaxes. The rigid matching is performed by aligning the center ofgravity and the principal axes.


Fig. 5. The first 5 panels show how the template image deforms to the target image on the geodesic path, with two images superimposed, black the template andgray the target. The last panel shows the mapping ' on the mesh grid.

E. The Matching Algorithm

At a given resolution, perform the following steps.

1) Initialize the algorithm with , ,or any initial guess.

For each iteration :2) calculate and using the semi-Lagrangian

scheme.3) Calculate the new gradient using (9).4) Calculate the new velocity . Find

the optimal step size using golden section line search al-gorithm in the direction of steepest descent. If ,stop the iteration, otherwise, return to step 2).

The multiresolution matching algorithm is as follows: 1) rigidmatching; 2) generate multi-resolution image pyramid by re-ducing the resolution from one level to the next by a factor of 2.From the coarsest level to the finest level, do the following: a).Use the fixed resolution matching algorithm to find the optimalvector field for that level. b). Use linear interpolation to con-vert the optimal vector field from one level to the next finerlevel.

V. EXPERIMENTAL RESULTS

The algorithm is implemented in C++ for both 2-D and three-dimensional (3-D) vector field images. In our experiments, thetime interval of the flow is discretized into 20 steps, with stepsize . We use and . The parametershould be big enough such that the matching terms in the energyfunction are dominant. The parameter controls the balance ofthe two matching terms. We use 0.075 as the standard deviation

Fig. 6. This figure shows the shape of various normal canine hearts. The topleft panel is chosen as the template, all others are targets.

of the Gaussian kernel with the grid normalized to . If thewidth of the kernel is too large, the constraints on the deforma-tion will be too strong, hence, the template may not have enoughfreedom to deform to the target; if the width of the kernel is toosmall, the deformation may not be smooth enough due to dis-cretization errors. We choose the width experimentally.

We first provide a simple example to show the difference be-tween matching only the shapes and matching the vector fields.See Fig. 3. In this example, the template and the target havethe same rectangular shape, but different vector fields inside.There are two major structures, one with angle 45 (with re-spect to -axis) vectors, another with horizontal vectors. Boththe template and the target are composed with these structures,


Fig. 7. Normal canine heart data example. Several slices are shown. Each panel shows the distribution of the primary eigenvectors within an equatorial short axiscross section. Orientation of the line segments indicates eigenvector orientation. The normalized out-of-plane component of each vector is shown according to theindicated color code.

Fig. 8. This figure shows the matching results for various normal canine hearts. Odd columns show the two geometries superimposed after rigid motion; the blueone is the template and the red one is the target. Even columns show the final matching results with color representing the absolute value of the dot product of thecorresponding vectors. Red color means total alignment.

but the structures are distributed in different regions of the rec-tangle. We applied two matching methods to this example, shapematching and vector field matching. In the shape matching case,the energy function only contains the smoothness term and theshape difference term. We can see that the boundaries of thetwo rectangles match well, but the two vector fields inside therectangles do not match at all. In the vector matching case, theenergy function contains the smoothness term, the shape differ-ence term and the vector difference term. In this case, both thegeometries and the vector fields associated to them match well.Fig. 4 shows the comparison of the energy at each iteration. Ifmatching only the shapes, the energy for vector field differencestops decreasing at some point since it is not included in theminimization.

We applied our algorithm on 3-D heart diffusion tensorMRI data from the Center for Cardiovascular Bioinformaticsand Modeling, Johns Hopkins University, in which image

segmentation and volumetric reconstruction of the hearthave been performed [41], [42]. DT-MRI imaging was per-formed with m in-plane, and m out-of-planeresolution. Epicardial and endocardial boundaries were esti-mated using diffusion-weighted short axis slices using asemiautomated image segmentation tool, see [41] for details.Fig. 7 shows an example of the data. For the 3-D experiments,the image is resampled to size 128 128 96 with each voxela 0.625 0.625 0.8 mm rectangular prism and the matchingis performed on 3 levels of resolutions.

Fig. 5 shows the experiment results on matching two 2-Dslices of normal canine hearts. They are projections of the 3-Ddata in plane. We show a very small example here since it isdifficult to show the details of large dense vector fields on paper.

We now provide 3-D experiments on cardiac data. Fig. 6shows the shape of the template and the targets. Fig. 8 showsthe matching results for various 3-D DT-MRI data sets obtained


Fig. 9. Top row shows the distribution of the primary eigenvectors within an equatorial short axis cross section. Orientation of the line segments indicateseigenvector orientation. The normalized out-of-plane component of each vector is shown according to the indicated color code. Left column shows a slice of thetarget, right column show the corresponding slice of the deformed template. Top row is the complete view. Second row and third row show the enlargement ofregion A and region B respectively.

Fig. 10. This figure shows the distribution of the primary eigenvectors within an equatorial short axis cross section. Orientation of the line segments indicateseigenvector orientation. First panel shows a slice from the target with color representing the normalized out-of-plane component of each vector. Second panel andthird panel show the slice of the deformed template with color representing the absolute value of the dot product of the corresponding vectors with respect to thematching. Red color means total alignment. Second panel shows the result of matching the vector field, third panel shows the result of matching only the supportof the vector field.

from a set of normal canine hearts. Fig. 9 shows a detailed viewof one slice.

The experimental results are very good. Most areas inside theobjects align well. The zones which do not align properly arethose within which the vector fields have large variations. Thealignment on the boundary is not as good as inside. This is due tothe noise on the boundary and the side effect of the interpolationcaused by discontinuity. This may be reduced by smoothing thevector fields before matching.

This approach transfers the problem of quantifying thevariation of the objects to the mathematically tractable problem

of analyzing the computed transformations. An object withsimilar structures to the template will require transformationsclose to the identity (the un-deformed grid). Also, once atransformation that registers the template and target hearts iscomputed, it is then possible to identify the correspondingregions and compare the anatomical features. The validityof analyzing the deformations in terms of the metric on thediffeomorphism group has been shown in the experimentsperformed in [33]. Figs. 10, 12, and 13 shows the comparisonof matching the vector field and matching only the supportof the vector field (shape) with respect to one pair of normal


Fig. 11. Fiber structure of the canine ventricle. Left panel shows fiber tracingof a canine heart. Right panel shows fiber rotation at the cardiac apex. Colorrepresents fiber inclination angle. Image provided by P. Helm [44], Center forCardiovascular Bioinformatics and Modeling, Johns Hopkins University.

Fig. 12. This figure shows the comparison of matching the vector field (leftcolumn) and matching only the support of the vector field (right column).The first row shows the displacement in millimeter. The second row showsthe determinant of the Jacobian matrix. The third row shows the rotation partof the Jacobian matrix, the rotation angle in radius. The last row shows thenormalized difference of the eigenvalues of the Jacobian matrix. (See the textfor the details.).

canine hearts. Fig. 10 shows corresponding equatorial shortaxis cross sections. Fig. 14 shows the histogram of the absolutevalue of the dot product map of the corresponding vectors.Matching the whole vector field captures fiber structure in-formation that the support shape does not have, hence, gives

Fig. 13. This figure shows several slices of the rotation angle in radius. Leftcolumn is for matching the vector field and right column is for matching onlythe support of the vector field. Matching the whole vector field captures fiberstructure information that the support shape does not have.

better result. The first row of Fig. 12 shows the displacement inmillimeters, i.e., how much the grid moves at each voxel duringthe transformation. They are similar. The second row showsthe determinant of the Jacobian matrix of the transformation.Both are homogeneous, i.e., the range of the values are small,which means there are no large compressions and expansions.The polar decomposition theorem of Cauchy [43] states that anonsingular matrix equals an orthogonal matrix either premulti-plied or postmultiplied by a positive definite symmetric matrix.If we apply this theorem to the Jacobian matrix , we get

, in which is a proper orthogonal matrixand and are positive definite symmetric matrices. Notethat the decomposition of the Jacobian matrix is uniquein that , and are uniquely determined by . Theorthogonal factor captures the rotation information in theJacobian. The third row in Fig. 12 shows the rotation part of theJacobian matrix, the rotation angle. Fig. 13 shows the rotationangle of several slices. We can see that the rotation angles aredifferent in some regions which means matching the wholevector field captures fiber structure information that the supportshape does not have. When the match is only based on shape,the deformation computed is whatever optimizes the energy


Fig. 14. This figure shows the histogram of the absolute value of the dotproduct map of the corresponding vectors for one pair of hearts. Left panelshows the full figure, right panel shows the enlargement of the region with highvalues. Red line shows the histogram of matching the vector field, blue lineshows the histogram of matching only the support of the vector field. Matchingthe whole vector field captures fiber structure information that the supportshape does not have, hence, gives better result.

Fig. 15. Conditions before matching. This figure shows several slices of thefractional anisotropy weighted color-coded orientation map of two humanbrains. Left column shows the template, right column shows the target. Eachrow shows the slice with the same slice number. In the color-coded map, redindicates fibers running along the right-left direction, green anterior-posterior(up-down direction), and blue inferior-superior (perpendicular to the plane).

function; when the match is based on the vector field, there areconstraints on how the vector field should change, the rotationhas to be coming from the data. The rotation part of the Jacobianmatrix does not itself provide all the re-alignment of the vectorinformation, since it neglects the shear component. In Fig. 12we plot the normalized difference of the eigenvalues of the

Fig. 16. Top row shows the distribution of the primary eigenvectors.Orientation of the line segments indicates eigenvector orientation. Redindicates fibers running along the right-left direction, green anterior-posterior(up-down direction), and blue inferior-superior (perpendicular to the plane).The length of the vector is weighted by the fractional anisotropy. First panelshows a complete slice of the target, second panel shows the enlargement of theregion marked in the first panel. Second row shows the corresponding region ofthe deformed template. Left panel shows the result of matching the vector field,right panel shows the result of matching only the support of the vector field.

Fig. 17. Matching results. This figure shows several slices of the color-codedabsolute value of the dot product map of the corresponding vectors. Left columnshows the result of matching the vector field and right column shows the result ofmatching only the support of the vector field. The two matchings are consistent.Only pixels with the fractional anisotropy higher than a threshold of 0.10 areshown. Matching the whole vector field captures fiber structure information thatthe support shape does not have, hence, gives slightly better result.


Fig. 18. This figure shows the histogram of the absolute value of the dotproduct map of the corresponding vectors for one pair of human brains. Redline shows the histogram of matching the vector field, blue line shows thehistogram of matching only the support of the vector field. Matching the wholevector field captures fiber structure information that the support shape does nothave, hence, gives slightly better result.

Jacobian matrix to see the shear effects. Suppose theeigenvalues of the Jacobian matrix are , and ,we define the normalized difference of the eigenvalues as

.The normalized difference of the eigenvalues of the Jacobianmatrix are similar in matching the vector field and matchingonly the support of the vector field (shape). This example showswe can use the Jacobian matrix of the deformation to analysisvariations of fiber orientations.

We also applied our algorithm to one pair of normal humanbrains. The data is from Department of Radiology, Johns Hop-kins University School of Medicine. The image is resampled tosize 128 128 64 with each voxel a 0.9375 0.9375 2.5mm rectangular prism and the matching is performed on 2levels of resolutions. Fig. 15 shows several slices of the frac-tional anisotropy weighted color-coded orientation map of thetwo brains. Figs. 16–18 shows the matching results. From thefigures, we can see that matching the whole vector field givesslightly better result compared to only matching the geometricshape of the two brains.

VI. CONCLUSION

In conclusion, we have presented in this paper the large de-formation diffeomorphic metric mapping of vector fields. Theoptimal mapping is the endpoint of a geodesic path on the man-ifold of diffeomorphisms connecting two vector fields. Findingthe optimal mapping and the geodesic path is formulated as avariational problem over a vector field. We give existence re-sults and a gradient descent based multi-resolution algorithm.Our method includes the vector reorientation in the optimiza-tion, hence, the measure of difference is calculated correctlyduring the optimization. We did the experiments on various 2-Dand 3-D heart diffusion tensor MRIs and got good results. Wealso provide preliminary results on brain data. The frameworkwe provide works on matching general vector fields, there is norestrictions on the norm of the vectors as long as the vectors atcorresponding regions of the images have similar lengths, which

is a reasonable assumption. We expect this method be a usefultool for analysis diffusion tensor MRIs and other images whichhave similar properties.

APPENDIX IPROOF OF THE MINIMIZATION EXISTENCE THEOREM

Theorem 1.1: Let be a Hilbert space which is embeddedin . There always exists a minimizer of the energyfunction defined by (4).

Proof: We extend the method used in [26] and [27]. Letbe the set of time-dependent vector field, such that for each , , . Then,is a Hilbert space with norm . We

can consider this space with the weak topology, i.e., iffor all .

has an infimum in since . Let bea minimizing sequence, so is bounded, hence, it has asubsequence (still denote by ) which weakly converges to .

Use the same method in [26], we can show that convergesto uniformly in as .

For fixed , is the solution of the linear differen-tial equation

(11)

with initial condition .From (11), we get .

(12)

where , and.

Consider the function . It is a linear

functional on . It is continuous since

. The first term of (12) goes to zero asgoes to since weakly converges to in . The secondterm goes to zero too since uniformlyin . So we have for all , ,

. So


and

Hence, by the dominated convergence theorem

and

Note that

since weakly converges to and is Hilbert. So

Hence, is a minimizer.Remark 5: The minimizer is not necessarily unique.

APPENDIX IIPROOF OF THE FIRST VARIATION THEOREM

We first state some lemmas without proof since the calcula-tions are standard, then we prove the theorem about .

Lemma 2.1: Let and in , then for

This lemma is proved in [30], but stated in a different way.Lemma 2.2: Let and in , then for ,

To simplify the notation, we drop the superscript in the re-maining calculations. Recall that we define the following vari-ables: , , ,and .

Using Lemma 2.1 and Lemma 2.2, we can prove the fol-lowing lemma.

Lemma 2.3: Let and in , then for

Theorem 4.2: Suppose is a continuously differentiableidealized template image of vector fields and satisfies the prop-erty , is the target image, then for inexact matching ofand is given by (4) and the equation at the bottom of the page.

Proof: Let

We have


Suppose is the self-reproducing kernel of , then isa matrix. We denote . Let bethe th column of . We have the equation at the bottomof the page.

ACKNOWLEDGMENT

The authors would like to thank P. Helm of the Center for Car-diovascular Bioinformatics and Modeling, Johns Hopkins Uni-

so we have


versity, for assistance on the heart data and S. Mori, Departmentof Radiology, Johns Hopkins University School of Medicine andKennedy Krieger Institute, for providing the human brain data.

REFERENCES

[1] C. Pierpaoli, P. Jezzard, P. J. Basser, A. Barnett, and G. D. Chiro, “Dif-fusion tensor MR imaging of the human brain,” Radiology, vol. 201, no.3, pp. 637–648, 1996.

[2] S. Mori and P. B. Barker, “Diffusion magnetic resonance imaging: Itsprinciple and applications,” Anatom. Rec. (New Anatom.), vol. 257, pp.102–109, 1999.

[3] D. F. Scollan, A. Holmes, R. L. Winslow, and J. Forder, “Histologicalvalidation of myocardial microstructure obtained from diffusion tensormagnetic resonance imaging,” Am. J. Physiol. (Heart CirculatoryPhysiol.), vol. 275, pp. 2308–2318, 1998.

[4] J. Dou, W.-Y. I. Tseng, T. G. Reese, and V. J. Wedeen, “Combined diffu-sion and strain MRI reveals structure and function of human myocardiallaminar sheets in vivo,” Magn. Reson. Med., vol. 50, pp. 107–113, 2003.

[5] P. C. Sundgren, Q. Dong, D. Gómez-Hassan, S. K. Mukherji, P. Maly,and R. Welsh, “Diffusion tensor imaging of the brain: Review of clinicalapplications,” Neuroradiology, vol. 46, pp. 339–350, 2004.

[6] A. Kanai and G. Salama, “Optical mapping reveals that repolarizationspreads anisotropically and is guided by fiber orientation in guinea pighearts,” Circ. Res., vol. 77, no. 4, pp. 784–802, 1995.

[7] D. E. Roberts, L. T. Hersh, and A. M. Scher, “Influence of cardiac fiberorientation on waterfront voltage, conduction velocity and tissue resis-tivity in the dog,” Circ. Res., vol. 44, no. 5, pp. 701–712, 1979.

[8] B. Taccardi, “Anatomical architecture and electrical activity of theheart,” Acta Cardiologica, vol. 52, no. 2, pp. 91–105, 1997.

[9] L. K. Waldman, “Relation between transmural deformation and localmyofiber direction in canine left ventricle,” Circ. Res., vol. 63, no. 3, pp.550–562, 1988.

[10] J. H. Omens, K. D. May, and A. D. McCulloch, “Transmural distributionof three-dimensional strain in the isolated arrested canine left ventricle,”Am. J. Physiol., vol. 261, pp. 918–928, 1991.

[11] S. A. Wickline et al., “Structural remodeling of human myocardial tissueafter infarction. Quantification with ultrasonic backscatter,” Circ. Res.,vol. 85, no. 1, pp. 259–268, 1992.

[12] T. Koide, T. Narita, and S. Sumino, “Hypertrophic cardiomyopathywithout asymmetric hypertrophy,” Br. Heart J., vol. 47, no. 5, pp.507–510, 1982.

[13] W. C. Roberts and V. J. Ferrans, “Pathologic anatomy of the cardiomy-opathies. Idiopathic dilated and hypertrophic types, infiltrative types,and endomyocardial disease with and without eosinophilia,” Hum.Pathol., vol. 6, no. 3, pp. 287–342, 1975.

[14] F. Tezuka, “Muscle fiber orientation in normal and hypertrophiedhearts,” Tohoku J. Exp. Med., vol. 117, no. 3, pp. 289–297, 1975.

[15] P. M. Nielsen, I. J. L. Grice, B. H. Smaill, and P. J. Hunter, “Mathematicalmodel of geometry and fibrous structure of the heart,” Am. J. Physiol.,vol. 260, pp. 1365–1378, 1991.

[16] D. C. Alexander and J. C. Gee, “Elastic matching of diffusion tensor im-ages,” Comput. Vis. Image Understanding, vol. 77, pp. 233–250, 2000.

[17] D. Strrter, H. Spotnitz, D. Patel, J. Ross, and E. Sonnenblick, “Fiberorientation in the canine left ventricle during diastole and systole,” Circ.Res., vol. 24, pp. 339–347, 1969.

[18] P. M. Nielsen, I. J. L. Grice, B. H. Smaill, and P. J. Hunter, “Mathematicalmodel of geometry and fibrous structure of the heart,” Am. J. Physiol.,vol. 260, pp. 1365–1378, 1991.

[19] R. Bajcsy and S. Kovacic, “Multi-resolution elastic matching,” Comput.Vis., Graphics Image Process., vol. 46, no. 1, pp. 1–21, 1989.

[20] J. C. Gee and R. K. Bajcsy, “Elastic matching: Continuum mechanicaland probabilistic analysis,” in Brain Warping. New York: Academic,1998, pp. 193–198.

[21] J. Ruiz-Alzola, C.-F. Westin, S. K. Warfield, C. Alberola, S. Maier, andR. Kikinis, “Nonrigid registration of 3-D tensor medical data,” Med.Image Anal., vol. 6, pp. 143–161, 2002.

[22] A. Guimond, C. R. G. Guttmann, S. K. Warfield, and C.-F. Westin, “De-formable registration of DT-MRI data based on transformation invarianttensor characteristics,” presented at the 2002 IEEE Int.l Symp. Biomed.Imag., Washington, DC, Jul. 2002.

[23] G. K. Rohde, S. Pajevic, and C. Pierpaoli, “Multi-channel registrationof diffusion tensor images using directional information,” presented atthe 2004 IEEE Int. Symp. Biomedical Imaging: From Nano to Macro,Arlington, VA, Apr. 2004.

[24] D. C. Alexander, C. Pierpaoli, P. J. Basser, and J. C. Gee, “Spatial trans-formations of diffusion tensor magnetic resonance images,” IEEE Trans.Med. Imag., vol. 20, no. 11, pp. 1131–1139, Nov. 2001.

[25] D. C. Alexander, J. C. Gee, and R. Bajcsy, “Strategies for data reori-entation during nonrigid warps of diffusion tensor images,” in LectureNotes in Computer Science. Berlin, Germany: Springer-Verlag, 1999,Proceedings of MICCAI 1999, pp. 463–472.

[26] P. Dupuis, U. Grenander, and M. I. Miller, “Variational problems onflows of diffeomorphisms for image matching,” Quart. Appl. Math., vol.3, pp. 587–600, 1998.

[27] A. Trouvé. (1995) An Infinite Dimensional Group Approach for Physicsbased Models in Pattern Recognition. Center for Imaging Science. [On-line]. Available: http://www.cis.jhu.edu/

[28] , “Diffeomorphic groups and pattern matching in image analysis,”Int. J. Comput. Vis., vol. 28, pp. 213–221, 1998.

[29] M. I. Miller, A. Trouvé, and L. Younes, “On the metrics and Euler-La-grange equations of computational anatomy,” Annu. Rev. Biomed. Eng.,vol. 4, pp. 375–405, 2002.

[30] M. F. Beg, M. I. Miller, A. Trouvé, and L. Younes, “Computing metricsvia geodesics on flows of diffeomorphisms,” Int. J. Comput. Vis., vol.61, no. 2, pp. 139–157, 2005.

[31] B. Avants and J. C. Gee, “Geodesic estimation for large deformationanatomical shape averaging and interpolation,” NeuroImage (Suppl. 1),vol. 23, pp. S139–S150, 2004.

[32] S. Joshi, B. Davis, M. Jomier, and G. Gerig, “Unbiased diffeomorphicatlas construction for computational anatomy,” NeuroImage (Suppl. 1),vol. 23, pp. S151–S160, 2004.

[33] S. Marsland and C. J. Twining, “Constructing diffeomorphic represen-tations for the groupwise analysis of nonrigid registrations of medicalimages,” IEEE Trans. Med. Imag., vol. 23, no. 8, pp. 1006–1020, Aug.2004.

[34] N. Aronszajn, “Theory of reproducing kernels,” Trans. Am. Math. Soc.,vol. 68, no. 3, pp. 337–404, May 1950.

[35] V. Camion and L. Younes, “Geodesic interpolating splines,” presentedat the 3rd Int Workshop on Energy Minimization Methods in ComputerVision and Pattern Recognition, Sophia Antipolis, France, Sep. 2001.

[36] D. G. Luenberger, Optimization by Vector Space Methods. New York:Wiley, 1969.

[37] G. Wahba, Spline Models for Observational Data. Philadelphia, PA:SIAM (Soc. Ind. Appl. Math.), 1990, p. 169.

[38] W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Nu-merical Recipes in C, The Art of Scientific Computing, 2nd ed. Cam-bridge, U.K.: Cambridge Univ. Press, 1992.

[39] K. W. Morton and D. F. Mayers, Numerical Solution of Partial Differ-ential Equations. Cambrisge, U.K.: Cambridge Univ. Press, 1996.

[40] D. R. Durran, Numerical Methods for Wave Equations in GeophysicalFluid Dynamics. Berlin, Germany: Springer-Verlag, 1999.

[41] D. F. Scollan, A. Holmes, J. Zhang, and R. L. Winslow, “Reconstructionof cardiac ventricular geometry and fiber orientation using magnetic res-onance imaging,” Ann. Biomed. Eng., vol. 28, pp. 934–944, 2000.

[42] M. F. Beg, P. A. Helm, E. McVeigh, M. I. Miller, and R. L. Winslow,“Computational cardiac anatomy using magnetic resonance imaging,”Magn. Reson. Med., to be published.

[43] P. R. Halmos, Finite Dimensional Vector Spaces, 2nd ed. New York:Van Nostrand, 1958.

[44] P. Helm, M. F. Beg, M. I. Miller, and R. L. Winslow, “Measuring andmapping cardiac fiber and laminar architecture using diffusion tensorMR imaging,” Ann. New York Acad. Sci., vol. 1047, pp. 296–307, 2005.

1216 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 24, NO. … · 1216 IEEE TRANSACTIONS ON MEDICAL...

Documents

Transcript of 1216 IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 24, NO. … · 1216 IEEE TRANSACTIONS ON MEDICAL...