Real-Time Myocardial Segmentation Using Coupled Active Geometric Functions

1,2Qi Duan, Member, IEEE, 1Andrew F. Laine, Member, IEEE, and 3Vinay M. Pai

�

Abstract—Myocardial segmentation is essential for quantitative evaluation of cardiac functional images. As imaging techniques advance, 3D and 4D image data have become available. These data can provide clinically important cardiac dynamic information at high spatial or temporal resolution. However, the enormous amount of information contained in these data has also raised a challenge for traditional image analysis algorithms in terms of efficiency and clinical workflow. In this context, an automated real-time myocardial segmentation framework based on coupled Active Geometric Functions was proposed and tested on 414 frames of Phase Train Imaging data, a real-time cardiac MR imaging technique, with an average temporal resolution of 2 ms. The performance of myocardial segmentation was visually and quantitatively validated. Implemented in Matlab©, the current method takes less than 1.2 ms per cardiac phase, allowing realization of true real-time online segmentation.

I. INTRODUCTION

ARDIAC cine-MRI images are becoming more important in clinical diagnosis and treatment of heart disease.

Recent advances of high speed imaging, such as Phase Train Imaging (PTI) [1], can provide large image data sets with very high temporal resolution (1.5-3 milliseconds). Such data sets may be very valuable for diagnosing and detecting mechanical dyssynchrony in systole and early diastole, which previously has been limited by temporal resolution. In order to quantitatively evaluate cardiac functional images, segmentation of endocardial and epicardial boundaries of the myocardium is essential. Several automated or semi-automated segmentation algorithm have been developed to minimize human effort involved in this step.

Based on the mathematical foundation of each method, segmentation approaches can be roughly divided into several classes: classification (e.g. thresholding, k-means), region growing (such as fuzzy connectedness [2]), active shape [3] and active appearance models [4], deformable models (e.g.

ia.edu

Dr. Qi Duan is with the Department of Biomedical Engineering, Columbia University, 1210 Amsterdam Avenue, MC-8904, Engineering Terrace, Room 351, New York, NY 10027 USA, and the Department of Radiology, NYU Medical Center, 660 1st Ave. CBI, New York, NY 10016 USA (phone: 646-334-5173; fax: 212-854-5995; e-mail: qd2002@columb , Qi.Duan@nyumc.org).

Prof. Andrew F. Laine is with the department of Biomedical Engineering, Columbia University, 1210 Amsterdam Avenue, MC-8904, Engineering Terrace, Room 351, New York, NY 10027 USA (e-mail: laine@columbia.edu).

.edu

Prof. Vinay M. Pai is with the Department of Radiology, SUNY Upstate Medical University, 750 East Adams St, Syracuse, NY 13210 USA (e-mail: PaiV@upstate ).

snake [5], level set [6-9]), and stochastic methods (Markov random field [10], graph cut [11]). Hybrid methods [12] combining different existing methods have also been proposed. Among segmentation methods, deformable models are still widely used in medical image analysis, especially for cardiac imaging.

The first deformable model parametric formulation was proposed by Kass, et al in 1987 [5]. In 1998, Xu et al [13] proposed the Gradient Vector Flow (GVF) method to overcome several drawbacks in the original snake framework. In order to handle topological changes, especially in 3D,in the late 1990s, Sethian, et al. [14] proposed level set framework by utilizing level set functions with higher dimensionality than the data. In 2001, Chan and Vese [15] proposed their famous “active contour without edges”. The driving forces were derived via energy minimization of the Mumford-Shah segmentation functional [16]. Their method has been widely used in ultrasound segmentation [17], brain segmentation [18], and many other applications. However, the introduction of level set functions implicitly increased the number of parameters of the surface model, which increases the demand for computational power. Although many optimization modifications such as narrow-banding or fast marching schemes have been proposed, the level-set framework is still, however, a relatively “slow” approach especially for 3D or 4D data.

However, the increased data content in high speed imaging necessitates development of highly efficient image processing algorithms in order to minimize processing times. In this context, a fully automated real-time segmentation framework was developed in this paper and illustrated on 414 frames of clinical PTI data. The proposed method is based on a newly developed deformable model framework, Active Geometric Functions, also previously referred as Surface Function Actives [19]. In this work, this novel framework was further extended to two coupled geometric functions in order to simultaneously segment endocardium and epicardium from the PTI image series.

II. METHODOLOGY

A. General Active Geometric Functions Framework 1) Interface Representation

In all deformable model methods, interface representation is fundamental. Mathematically, there are two ways to represent the interface: � Explicit representation: Representing the surface by

C

30th Annual International IEEE EMBS ConferenceVancouver, British Columbia, Canada, August 20-24, 2008

978-1-4244-1815-2/08/$25.00 ©2008 IEEE. 3385

explicitly listing the coordinates of the boundary points (i.e. a parametric representation). This is the representation that original snakes [20] used;

� Implicit representation: Representing the surface by embedding the boundary as the iso-value curves of some function f called the representation function.Level set functions [6-9] are a good example by embedding the interface as the zero level-set of a distance function.

2) Geometric Function Most of the recent efforts in segmentation based on

implicit interface representation have focused on the level-set framework, given its advantages for topological changes and feasibility to represent convoluted surfaces. As mentioned above, level set functions add one extra dimension beyond the dimensionality of the image data. For example, to represent a surface in 3D space, the level set function corresponding to the surface will be a tri-variate function (or quad-variate during the actual iteration). For comparison, original parametric deformable models only required a list of point coordinates in 3D. For level set, this extra dimension brings various benefits as well as additional computation load, which may degrade computational efficiency.

By thinking in the opposite way of level set frameworks, it is very natural to consider dimensionality reduction in surface representation to reduce the computational complexity. Using terminology of interface representation, we are looking for a representation function which has fewer dimensions than the image data, i.e. using a 2D function to represent a 3D surface in space. We call such function a geometric function.

Mathematically, in N dimensional space, we can define a geometric function as a special set of functions representing one of the coordinates constrained by the others. Without losing generality, we can assume that this special coordinate is x0 and the other coordinates are x1 to xN-1. That is:

N-1:g �� �

1

1

. (1) 0 1( , )Nx g x x �� � The corresponding representation function f is defined as:

. (2) 0 1( , )Nf x g x x �� � �So that the corresponding boundary is the zero-value curve

of the function f, i.e. f=0.3) Driving Forces

Similar to other deformable models, we adopted a variational framework in deriving the driving forces. For example, we can use the Mumford-Shah segmentation energy functional:

� � 22

\ \( , )

C CE F C F G dV F dV ds�

� � � � � ��� ��� ��

��C

� �� (3)

in which C denotes the smoothed and closed segmented interface, G represents the observed image data, F is a piecewise smoothed approximation of G with discontinuities only along , and denotes the image domain. Given the flexibility of variational frameworks, other segmentation energy functionals may also be easily adopted.

��

C��

B. Coupled Active Geometric Functions Each AGF model could divide an image into two

partitions, i.e. the object (or the foreground) and the background. In order to simultaneously segment multiple objects, similar to level set framework [18, 21, 22], more than one deformable model could be introduced at the same time. In order to realize simultaneous multi-object segmentation, these deformable models will be coupled together via some mechanism, such as distance [22] or multi-phase fashion [18, 21].

One benefit of implicit surface representation is that the function value of the representation function can be assigned with some physical meanings that may assist segmentation during iteration. In the level set framework, signed distance functions [6, 7] are usually chosen as the level set function. Such choice of representation functions not only offers an immediate measure or approximation of the distance to the current interface for any given point, but also has nice features, such as unitary slope, that may simplify the energy minimization equations. As another implicit surface representation approach, the function values of Geometric Functions also have physical meaning, i.e. the coordinate values along the direction of one spatial basis function. This generally does not correspond to the Euclidean distance for a given point to the current surface. However, recalling the definition of coordinate values, it is obvious that this value is a distance measure along the corresponding basis direction. If this basis is linear, the function value is thus a signed distance function, with some scaling factor, along the basis direction. Moreover, such distance measures usually is an upper bound to the true point-to-surface distance. And sometimes it also has clinical importance and diagnostic value. For example, in cardiac diagnosis, radial displacement and radial thickening are important metrics for cardiac dynamics. By choosing a coordinate system with a radial direction, such as polar coordinates, the Geometric Function values can be immediate measures of radial distance.

This nice feature can also be used in coupling Active Geometric Functions models. Specifically in myocardial segmentation in 2D spatial + time cine series, two geometric functions were introduced to simultaneously segment endocardium and epicardium. Both models were driven by forces computed via energy minimization. The total energy is defined by combination of Mumford-Shah energy functional and a penalty term computed from signed radial distance between two surfaces. Specifically, assuming 1C

���2 and C���

are the two coupled surfaces with corresponding geometric functions g1 and g2, then in polar coordinates system, we have

� �1r g � �2r g�� and �� 1. We shall assume C���

2

targets the

endocardium and C���

targets epicardium. Generally, these two models divide an image into four different partitions [18, 21], with the average intensity values within each partition defined as c00,c10,c01, and c11, as shown in Figure 1.

3386

Figure 1: Partitioning of the image into four phases using two level set functions. Average intensity values are designed as c00,c10,c01,c11.

Let a function d be a membership penalty function defining a penalty associated with its function variable. In our implementation, a linear penalty function similar to wavelet soft-thresholding [23] function was used. However, more sophisticated membership function such as fuzzy membership function [24] can be used. The transform function is:

max max

min max

min min

,( ) 0,

,

x d x dd x d x d

x d x d

� ���� � ��� � ��

, (4)

with dmin and dmax defining acceptable range of membership. The shape of the membership penalty function is given in Figure 2.

�

Figure 2: Shape of membership penalty function. Lastly, a smoothness constraint was superimposed to each

AGF model to ensure the papillary muscles were excluded in the endocardial segmentation.

Thus, the final energy functional can be defined as:

� � � �

� � �

1 12 2

1 12 2

1 2

1 22 2

11 11 10 10

2 201 01 00 00

21 2 2 1

( , , )

( )

insideC insideCinsideC outsideC

outsideC outsideCinsideC outsideC

C C

E F C C

u c u c

u c u c

ds ds d g g

� �

� �

� � �

� � � �

� � � �

� � � �

��� ����

��� ���

� � �

(5)

with , 0,1; 0,1ij i j� � � as the parameters balancing the homogeneity measures, 1 2,� � as the weighting for the smoothness constraint for each AGF model, and � as the parameter controlling the weighting for membership penalty.

III. RESULTS

The proposed algorithm was applied to 414 frames of clinical PTI data with average temporal resolution of 2 milliseconds. Each image frame had a dimension of 160x192 pixels. Manual tracing of the endocardium and epicardium was also performed by an experienced expert serving as a gold standard to evaluate the performance of the proposed multi-phase AGF method. The algorithm was preliminarily

implemented in Matlab© (The Mathworks, Natick, MA). 1C���

2C���

The coupled Active Geometric Functions models were automatically initialized as two tiny circles with different radii at the center of the image at the first frame of series, as shown in Figure 3.

Figure 3: Initialization at the first frame. After initialization, two AGF models started to evolve

under the force derived via energy minimization of the total energy functional defined by equation (5) until convergence. Subsequently the segmentation results on the current frame were propagated into the next one, utilizing the high temporal resolution that the PTI data possesses, followed by curve evolution until convergence and propagation to next frame. Figure 4 shows 4 sample frames of the segmentation results taken at different phases of the cardiac cycle. Both endocardium and epicardium surfaces were accurately segmented on all frames with papillary muscle successfully excluded.d

t=0. 166 s t=0 s

d_min 0 d_max

t=0.494 s t=0.740 s

Figure 4: Illustration of segmentation of endocardium and epicardium on 4 frames out of 414 frames. The red curves indicate the automated endocardium segmentation; the green ones are automated epicardium segmentation.

The proposed method took 500 ms to segment all 414 frames. On average, it took SFA 6 iterations to reach a stable endocardial and epicardial segmentation with each iteration using 0.194 ms under a Matlab© implementation. All computations were executed on a 2.4GHz 64-bit AMD server, running Red Hat Linux Enterprise AS. Quantitative evaluations were performed both on endocardial and epicardial segmentation volume, in terms of area difference, true positive fraction, and false positive fraction. For endocardial segmentation, the area difference (mean ± standard deviation) was 8.7% ± 5.9%; the true positive fraction was 93.3% ± 7.0%; and the false positive fraction was 7.0% ± 4.6%. For epicardial segmentation, the area difference was 6.8% ± 5.3%; the true positive fraction was

3387

95.5% ± 3.6%; and the false positive fraction was 7.8% ± 5.2%. Average distance between automated segmented surfaces and manually traced surfaces was 3.0 pixels ± 2.4 pixels. Comparison metrics from a recent systematic study on cardiac MRI segmentation [25] were used as a reference, which suggested that our results were comparable to level set based methods as well as inter-observer variance. Note that the PTI images have slightly coarser resolution as well as a slightly blurry appearance than regular cardiac cine MRI due to undersampling in the phase-encoding direction, which may increase the inter-observer variance as presented in [25]

IV. DISCUSSION

Under a non-optimized Matlab© implementation, the proposed method still achieved a computational cost of 1.2 ms per frame, which is already less than the temporal sampling rate of the data. A “C” implementation would be considerably faster than Matlab. With this extraordinaryefficiency, the proposed method can not only be used for offline analysis, but also enable online real-time segmentation, which can also use applied to other applications requiring real-time feedback, such as an interventional procedures or quantifying cardiac mechanical dyssynchrony.

Also since the Geometric Function values are already radial distance measures, clinically important metrics, such as radial displacement and thickening can be directly derived from the Geometric Function values with simple calculations such as subtraction and division.

V. CONCLUSION

An automated real-time segmentation based on multi-phase SFA was developed. The proposed method was tested on 414 frames of PTI data with 2ms temporal resolution. The performance of endocardial and epicardial segmentation was visually and quantitatively validated. Implemented in Matlab©, current method took less than 1.2 ms per cardiac phase. Given the fact that the proposed algorithm is faster than image acquisition, this new framework can not only realize real-time image analysis for diagnosis, but also provide the possibility for online segmentation during acquisition or support interactive acquisition protocols.

REFERENCES

[1] V. Pai, L. Axel, and P. Kellman, "Phase Train Approach for very high temporal resolution cardiac imaging," J Cardiovasc Magn Reson, vol. 7, 1, pp. 98-99, 2005.

[2] J. K. Udupa, W. L., S. Samarasekera, et al., "Multiple sclerosis lesion quantification using fuzzy Connectedness principles," IEEETransactions in Medical Imaging, vol. 16, pp. 598-609, 1997.

[3] T. F. Cootes, C. J. Taylor, D. H. Cooper, et al., "Active Shape Models-Their Training and Application," Computer Vision and Image Understanding, vol. 61, 1, pp. 38-59, 1995.

[4] T. F. Cootes, G. J. Edwards, and C. J. Taylor, "Active Appearance Models," in Lecture Notes in Computer Science. vol. 1407: Springer Berlin / Heidelberg, 1998, pp. 484-498.

[5] M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: Active contour models," International Journal of Computer Vision, vol. 1, pp. 321-331, 1987.

[6] J. Sethian, Level set methods and fast marching methods, 2 ed. vol. 3. Cambridge: Cambridge University Press, 1999.

[7] S. Osher and R. Fedkiw, Level set methods and dynamic implicit surfacesvol. 153. New York: Springer, 2003.

[8] T. F. Chan and L. A. Vese, "Active contours without edges," IEEETransactions on Image Processing, vol. 10, 2, pp. 266 - 277, 2001.

[9] E. Angelini, S. Homma, G. Pearson, et al., "Segmentation of real-time three-dimensional ultrasound for quantification of ventricular function: a clinical study on right and left ventricles," Ultrasound in Medicine and Biology, vol. 31, 9, pp. 1143-1158, 2005.

[10] K. Held, E. R. Kops, B. J. Krause, et al., "Markov random field segmentation of brain MR images," IEEE Trans Med Imaging., vol. 16, 6, pp. 878-886, 1997.

[11] Y. Y. Boykov and M.-P. Jolly, "Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D Images," in Eighth International Conference on Computer Vision (ICCV'01), 2001, pp. 105-112.

[12] Y. Jin, C. Imielinska, A. Laine, et al., "Segmentation and Evaluation of Adipose Tissue From Whole Body MRI Scans.," in Proceedings of the Sixth International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI 2003), Montreal Canada, 2003, pp. 635-642.

[13] C. Xu and J. L. Prince, "Snakes, Shapes and Gradient Vector Flow," IEEE Transaction on Image Processing, vol. 7, 3, pp. 359-369, 1998.

[14] J. A. Sethian, Level Set Methods and Fast Marching Methods: Evolving interfaces in computational geometry, fluid mechanics, computer vision, and materials science. Cambridge, UK: Cambridge University Press, 1999.

[15] T. F. Chan and L. A. Vese, "Active Controus Without Edges," IEEETransactions on Image Processing, vol. 10, 2, pp. 266-277, 2001.

[16] D. Mumford and J. Shah, "Boundary detection by minimizing functional," in International Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 1985, pp. 22-26.

[17] E. Angelini, A. Laine, S. Takuma, et al., "LV volume quantification via spatio-temporal analysis of real-time 3D echocardiography," IEEETransactions on Medical Imaging, vol. 20, 6, pp. 457-469, June 2001.

[18] T. Song, E. D. Angelini, B. D. Mensh, et al., "Comparison study of clinical 3D MRI brain segmentation evaluation," in Annual International Conference IEEE Engineering in Medicine and Biology Society (EMBS),San Francisco, CA, USA, 2004, pp. 1671 - 1674

[19] Q. Duan, "Real-Time 4D Cardiac Image Analysis for Left Ventricular Function," Ph.D. Thesis, Biomedical Engineering, Columbia University, 2007

[20] M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: Active contour models," in Proceedings of 1st International Conferences on Computer Vision, 1987, pp. pp. 259-268.

[21] E. D. Angelini, T. Song, B. D. Mensh, et al., "Brain MRI Segmentation with Multiphase Minimal Partitioning: A Comparative Study," International Journal of Biomedical Imaging, vol. 2007, Article ID 10526, pp. 1-15, 2006.

[22] X. Zeng, L. H. Staib, R. T. Schultz, et al., "Segmentation and measurement of the cortex from 3-D MR images using coupled-surfaces propagation," IEEE Transactions on Medical Imaging, vol. 18, 10, pp. 927-937, 1999.

[23] Y. Jin, E. Angelini, and A. Laine, "Wavelets in Medical Image Processing: Denoising, Segmentation, and Registration," in Handbook of Medical Image Analysis: Advanced Segmentation and Registration Models, J. Suri, D. L. Wilson, and S. Laximinarayan, Eds. New York NY: Kluwer Academic Publishers, 2004.

[24] J. K. Udupa and S. Samarasekera, "Fuzzy connectedness and object definition: Theory, algorithms, and applications in image segmentation," Graphical Models and Image Processing, vol. 58, 3, pp. 246-261, 1996.

[25] Q. Duan, D. Moses, M. B. Srichai, et al., "Semi-Automatic Ventricular Border Segmentation Package Based on Multi-Phase Levelset Segmentation," in International Society for Magnetic Resonance in Medicine(ISMRM) 14th Scientific Meeting & Exhibition, Seattle, WA, USA, 2006.

3388