Co-Registration of Intracardiac Echocardiography (ICE) and ... · three-dimensional CT volume. ICE...
Transcript of Co-Registration of Intracardiac Echocardiography (ICE) and ... · three-dimensional CT volume. ICE...
Co-Registration of Intracardiac Echocardiography (ICE) and Computed Tomography (CT) for the Purpose of
Guiding Structural Heart Disease Intervention
by
Jonathan E. Toma, MD, FRCPC
A thesis submitted in conformity with the requirements for the degree of Masters of Science Department of Medical Biophysics
University of Toronto
© Copyright by Jonathan E. Toma 2019
ii
ii
Co-Registration of Intracardiac Echocardiography (ICE) and
Computed Tomography (CT) for the Purpose of Guiding
Structural Heart Disease Interventions
Jonathan Toma
Masters of Science
Department of Medical Biophysics University of Toronto
2019
Abstract
We demonstrate in this work a system for tracking the position and orientation of an intracardiac
echocardiography (ICE) image within a CT volume for the purpose of improving imaging
guidance for cardiac interventions. The method begins with a user-defined registration step,
followed by subsequent automated local searches within the CT volume. The local searches are
performed by a series of individual 2D-2D co-registration steps over various positions and
orientations within the 3D space using an edge-sensitive algorithm. A stepwise sampling method
is used to reduce the total number of processing steps. Validation testing using a silicone cardiac
phantom over 10 pullback positions demonstrated average position and orientation localization
errors of 4.1 mm and 8.1 degrees respectively. A high degree of rotation symmetry in the model
likely contributed significantly to the orientation accuracy results. Further work is required to
reduce computation time for the algorithm, including the use of parallel processing.
iii
iii
Acknowledgments
I would like to extend my heartfelt appreciation to my supervisors Drs. Graham Wright and
Brian Courtney for their support in this thesis work. This work would not be possible without
their ongoing support and guidance. Thank you to the additional members of my Masters
committee, Drs. Anne Martel and David Goertz for their advice throughout the Masters process.
Thank you to the members of Dr. Graham Wright’s lab, including Labonny Biswas and
Sabastian Ferguson for their advice and support regarding software development. Thank you to
the members of Brian Courtney’s lab, including Dr. Natasha Alvez-Kotsev and Dr. Jill Weyers
for supporting experiments, data collection, and providing insights with respect to data
interpretation.
Thank you to the employees of Conavi Medical Inc, including Bogdan Neagu for providing
experimental catheters and phantoms, and providing advice regarding experimental setup.
Thank you to the administrative support personnel from Dr. Wright’s lab and the Department of
Medical Biophysics, including Kimberly Allan, Donna Marie Pow, and Merle Casci for their
advice and guidance throughout the Masters process.
Thank you to Professor Jan Modersitzki and his group at the Institute for Mathematics and Image
Computing from the University of Lübeck their work in developing a suite of registration tools
for MATLAB used in our work.
Finally, I would like to thank most my wife Stephanie, and my children Audrey and Benjamin
(both of whom were born during this degree) for their continuing love and support.
iv
iv
Table of Contents
Chapter 1: Introduction and Background .................................................................................. 1
1.1 Clinical Background .......................................................................................................... 1
1.1.1 Structural Heart Disease Intervention ...................................................................... 1
1.1.2 Aortic Stenosis ......................................................................................................... 4
1.1.3 Surgical Aortic Valve Replacement (SAVR) and Transaortic Valve Implantation (TAVI) ................................................................................................ 6
1.1.4 Evolving Role of Imaging Guidance in TAVI ........................................................ 8
1.1.5 Case for the Use of Intracardiac Echocardiography (ICE) during TAVI .............. 14
1.2 Technical Background .................................................................................................... 17
1.2.1 Medical Image Co-Registration ............................................................................. 18
1.2.2 Search Method within the CT Volume .................................................................. 29
1.2.3 Prior Work in Cardiac Ultrasound-CT/MR Co-Registration for Use in Procedural Navigation and as an Adjunct to Magnetic Tracking Systems ........... 31
Chapter 2: ICE-CT Co-Registration for the Purpose of Guiding Cardiovascular Interventions ................................................................................................................................ 33
2.1 Introduction ..................................................................................................................... 33
2.2 Methods ............................................................................................................................ 34
2.2.1 Software environment ............................................................................................... 34
2.2.2 Process Overview ..................................................................................................... 35
2.2.3 2D-2D Image Co-Registration .................................................................................. 37
2.2.4 Search Method within the CT volume ...................................................................... 39
2.3 Experimental Setup ......................................................................................................... 40
2.4 Results ............................................................................................................................... 41
2.4.1 2D-2D Co-Registration ............................................................................................. 42
v
v
2.4.2 Search Method within CT Volume ........................................................................... 43
2.5 Discussion .......................................................................................................................... 47
2.6 Conclusion ......................................................................................................................... 49
Chapter 3: Summary and Future Directions ............................................................................ 51
References ..................................................................................................................................... 61
1
1
Chapter 1
Introduction and Background
The purpose of our work is to use software co-registration methods to automatically determine
the position and orientation of an intracardiac echocardiography (ICE) imaging surface within a
three-dimensional CT volume. ICE is a catheter-based ultrasound technique where high-
resolution B-mode ultrasound images and Doppler information are obtained from within the
heart and displayed to the operator in real-time to help guide minimally invasive procedures. One
major drawback of the technique is the difficulty in recognizing the orientation of the imaging
surface relative to three-dimensional (3D) cardiac anatomy from the ICE images alone. A
software algorithm capable of demonstrating to the user the 3D anatomic context of the ICE
imaging surface would therefore provide a significant advantage to the operator. The algorithm
developed in our work is intended for novel applications of ICE imaging, including transaortic
valve implantation (TAVI), which is a minimally invasive method to implant an aortic valve
bioprosthesis in patients with severe aortic valve disease. It is vitally important in this procedure
to understand the location of the current imaging surface relative to neighboring critical
structures, such as the aortic wall, coronary ostia (origins), and the plane of the aortic annulus. In
the upcoming sections, we will review the clinical indications, rationale for the use of ICE in
TAVI, medical imaging co-registration, and how co-registration techniques may be used to
determine the location of a two-dimensional surface within a three-dimensional volume.
1.1 Clinical Background
1.1.1 Structural Heart Disease Intervention
Structural heart disease (SHD) intervention represents a rapidly growing sub-specialty within the
field of interventional cardiology.1–3 Physicians practicing SHD intervention use minimally
invasive techniques to repair pathologies affecting large structures of the heart, such as
narrowing or leakage of heart valves, or anomalous connections between cardiac chambers.3
These procedures are commonly performed using device implants through a percutaneous route,
2
2
which generally describes the method for accessing deeper structures in the body through the
skin without the use of a large surgical incision. Percutaneous access in the field of structural
heart disease intervention is most commonly performed by entering a blood vessel using a
needle-and-wire technique (Seldinger technique), and subsequently using this access to pass a
catheter through the blood vessel to the area of interest.4 A medical device is then passed through
the catheter to its destination where it may perform its intended action. This technique has been
adopted from earlier work in coronary and peripheral angioplasty; however, there are several
unique considerations in the structural heart disease field, including the larger vessel diameters
required for the passage of equipment, greater importance of properly sealing the port of entry,
and the risk of damaging (or causing embolization of) adjacent tissues or critical structures.1–3,5
Another unique consideration in this field is the role of adequate imaging guidance, given the
soft tissue composition and mobility of the target pathology, the size of the equipment typically
used in these procedures, and the proximity to adjacent critical structures.6–8 These procedures
are typically performed in an x-ray fluoroscopy suite, or cardiac catheterization laboratory,
which does not routinely make use of imaging modalities with soft-tissue contrast, such as
ultrasound. When soft tissue contrast is required, such as when performing transcatheter
(catheter-based) mitral valve repair, imaging guidance is generally performed using
transesophageal echocardiography (TEE).9–11 During TEE guidance, an imaging probe is placed
in the lower esophagus via the patient’s mouth, and usually requires administration of general
anaesthesia for the patient to tolerate this technique for extended periods of time (Figure
1.1A).6,7,9,12 The main benefit of this ultrasound imaging method is the vastly superior spatial and
temporal resolution of TEE (Figure 1.1B) compared to its surface counterpart, transthoracic
echocardiography (TTE). In the lower esophagus there is less tissue separation between the
probe and the left side of the heart as compared to TTE, as well as fewer tissue and air-tissue
interfaces producing ultrasound scatter between the probe and target area of interest.12 Later in
this introduction, we will discuss the role of intracardiac echocardiography (ICE), an emerging
catheter-based technique for providing ultrasound-based soft-tissue contrast during SHD
intervention, without the need for general anaesthesia.
3
3
Figure 1.1: Transesophageal Echocardiography is performed by inserting the imaging probe into
the lower esophagus via the mouth (A). There is far less tissue separation between the imaging
probe and the heart using this method. TEE is able to produce imaging with vastly superior
spatial and temporal resolution compared to transthoracic echocardiography (B). Source:
National Heart, Lung, and Blood Institute; National Institutes of Health; U.S. Department of
Health and Human Services.
Computed tomography (CT) represents a widely used imaging tool for procedural planning in
SHD intervention.6–8,13,14 Most contemporary procedures in this field benefit from the prior
knowledge of three-dimensional anatomy derived from CT. An operator can perform detailed
geometric measurements regarding the procedural approach, as well as the size and shape of the
area of interest within the heart, which often is used in determining the size of the prosthesis
required.6,14 Specialized software may help inform the operator of the fluoroscopic angle on the
live procedural x-ray system required to accomplish a specific task without the use of live soft-
tissue contrast.15,16 For example, as a result of CT procedural planning, the use of TEE during
transcatheter aortic valve intervention dropped by almost half (47%) in a large French registry of
patients undergoing aortic valve intervention in a relatively short period of time between 2010
and 2014.17
4
4
One of the particular strengths of using CT-based imaging for procedural guidance is the three-
dimensional anatomic context provided, particularly with respect to adjacent critical
structures.6,7,13,14,16 For example, when performing aortic valve intervention, operators are able to
understand the distance from the aortic valve to the origins of the coronary arteries, as well as the
length of the valve leaflets that will be displaced in their direction in order to accurately assess
the risk of coronary artery occlusion during the procedure (a complication associated with both a
high rate of morbidity and mortality). The information from CT may either inform the operator
of patients who are at prohibitive risk of undergoing such interventions due to risk of coronary
occlusion, or may dictate an alternative strategy to addressing the risk, such as stenting the
coronary origin(s).18–20
1.1.2 Aortic Stenosis
Aortic stenosis is a life-threatening clinical condition defined by narrowing of the aortic valve,
which is most commonly caused by chronic buildup of calcium on the surface of the valve
leaflets, and leaflet fibrosis (scarring, Figure 1.2).21–23 It is a condition predominantly affecting
the elderly, and it is estimated that one in eight persons over the age of 75 has aortic stenosis in
its moderate or severe form.24 There are, however, genetic conditions predisposing subjects to
development of aortic stenosis at earlier ages, such as bicuspid aortic valve disease, which affects
1-2% of the general population,25 but represents as high as 50% of aortic stenosis cases at a mean
age of 70 years old referred for surgical management.26 Patients suffering from severe aortic
stenosis can experience symptoms such as dyspnea (shortness of breath), angina (chest pain),
syncope (loss of consciousness), or even sudden death due to the condition. The average length
of time between the onset of symptoms due severe aortic stenosis and eventual death is
unfortunately only 1-3 years,27,28 which represents a worse prognosis than most forms of
cancer.29
5
5
Figure 1.2: Progression of calcific aortic stenosis from a normal aortic valve (far left), to aortic
valve sclerosis (calcium present, but no evidence obstruction), to mild-moderate and severe
aortic stenosis (right).30 Reproduced with permission from Otto, E. M. Calcific aortic stenosis –
time to look more closely at the valve. N. Engl. J. Med. 359, 1395-1398 (2008), Copyright
Massachusetts Medical Society.
It is possible for aortic stenosis to be detected as an asymptomatic heart murmur during a milder
phase, which then may be followed conservatively over time until the development of
symptoms.31 Some cases of severe symptomatic aortic stenosis arise incidentally when a patient
is evaluated in the clinic or emergency setting for new-onset dyspnea, angina, or syncope. The
standard of evaluation of the severity of aortic stenosis is transthoracic echocardiography (TTE).
Using TTE, physicians measure the relatively higher velocity of blood travelling through the
narrow aortic valve orifice to estimate the gradient (pressure difference) across the aortic valve,
and estimate an aortic valve area using the principles of the Bernoulli equation.32 Aortic stenosis
is considered severe if the peak velocity across the aortic valve exceeds 400 cm/s, the mean
gradient (based on the average of the Doppler velocity profile) exceeds 40 mmHg, or the
estimated aortic valve area is under 1.0 cm2.33 A physician following a patient with aortic
stenosis will typically perform routine TTE studies to follow the progression of the disease, and
will seek definitive intervention (surgery or TAVI) for the patient if the measurements suggest
severe aortic stenosis in the presence of symptoms. Other reasons to proceed to intervention in
patients who have minimal or no symptoms include weakness of the heart muscle (depressed
ejection fraction), another reason for heart surgery (such as the need for coronary bypass), or
very severe pathology indicated by measurements (velocity greater than 500 cm/second, mean
gradient greater than 50 mmHg, or valve area less than 0.8 cm2).33 Depressed ejection fraction
and very severe aortic stenosis are acted upon earlier despite the presence or lack of symptoms
due to the significantly higher risk of sudden cardiac death.34
6
6
1.1.3 Surgical Aortic Valve Replacement (SAVR) and Transaortic Valve Implantation (TAVI)
Traditional open-heart surgery in the presence of severe aortic valve disease has been performed
in clinical practice since the 1950’s.35 Surgical aortic valve replacement (SAVR) is performed by
a cardiac surgeon in an operating room, and typically requires a sternotomy (opening of the chest
by splitting the sternum), cardioplegia (stopping of the heart), and a bypass pump circuit in order
to circulate blood during the operation. During AVR, the new valve sewn in the aortic position
may be either metallic (mechanical), or may consist of a bovine or porcine pericardial patch
attached to a metallic support frame (bioprosthetic). Mechanical valves are known to be more
durable than bioprosthetic valves over the long-term, however require anticoagulation (blood
thinning) at a high threshold over a patient’s lifetime to prevent valve thrombosis (abnormal
clotting of the valve). Bioprosthetic valves are more similar in form and function to a native
aortic valve, and will last on average for a period of 10-15 years before suffering from severe
structural degeneration, causing either severe aortic insufficiency (valve leak), stenosis, or a mix
of stenosis and regurgitation requiring re-intervention.36,37 Patients spend an average of 9 days in
hospital following SAVR,38 and it is associated with a mortality rate as low as 1.4% in low-risk
patients;39 however, this number can be substantially higher in elderly patients or those with
significant co-morbidities.40 The current accepted “gold standard” for assessment of operative
risk of death either in-hospital, or at 30 days, is the Society of Thoracic Surgeons (STS) risk
score. Patients with an operative mortality of <4% are considered “low-risk”, 4-8% are
considered moderate risk, and >8% are considered “high-risk” for complications from cardiac
surgery.41 Before the widespread use of TAVI, patients who were either deemed high risk or
inoperable for traditional cardiac surgery were often palliated when their aortic valve disease
progressed to the point of causing repeat hospitalizations for refractory heart failure and eventual
death.27
TAVI is a catheter-based technique for implanting a bioprosthetic aortic valve for patients with
severe aortic valve disease, without requiring an open-chest operation (Figure 1.3). The
procedure is typically performed either through the femoral artery – transfemoral approach, or
directly through the tip of the heart using a “keyhole” incision – transapical approach. The safety
of the transfemoral over transapical approach has been well proven in recent years, and is thus
the preferred method of access for most TAVI operators, unless severe femoral or iliac vessel
7
7
disease is present.42–44 The new valve is advanced to the position of the diseased aortic valve in a
collapsed, or “crimped”, form mounted on a catheter until deployed. TAVI valves are expanded
to their natural size and shape either by balloon inflation (balloon-expandable valve), or by the
outward pressure of the valve frame once unsheathed (self-expanding valve). The metallic valve
support frame of the TAVI prosthesis holds the diseased native valve permanently open, and the
bioprosthetic leaflets mounted in the centre of the frame begin working immediately as a
normally functioning aortic valve.
Figure 1.3: Transaortic valve implantation (TAVI) is a non-surgical method to place an aortic
bioprosthesis within a diseased aortic valve (left), most often due to aortic stenosis (top right).
The frame of the bioprosthesis is held in place by the old diseased valve, which it displaces
towards the aortic root wall (bottom right).43 Reproduced with permission from Smith, C. R. et.
8
8
al. Transcatheter versus surgical aortic-valve replacement in high-risk patients. N. Engl. J. Med.
364, 2187-2198 (2011), Copyright Massachusetts Medical Society.
Several clinical trials have demonstrated the benefit of performing TAVI for symptomatic severe
aortic stenosis in selected patient groups; however, the indication for the procedure and group of
eligible patients continues to expand. The PARTNER 1B trial published in 2010 demonstrated a
significant mortality benefit of performing TAVI compared to medical therapy alone for patients
with symptomatic severe aortic stenosis who are deemed not suitable candidates for open
surgery.45 The PARTNER 1A trial published in 2011 demonstrated the non-inferiority of TAVI
compared to SAVR for both safety and efficacy in patients deemed high risk for surgery
(determined by cardiac surgeon using guideline of STS risk score > 10%).43 The transcatheter
patients were at slightly higher risk of stroke and vascular complications; however, the surgical
group was at higher risk of major bleeding and new-onset atrial fibrillation. The PARTNER II
trial published in 2016 demonstrated non-inferiority of TAVI compared to SAVR for various
measures of safety and efficacy in patients deemed moderate risk for traditional surgery (STS
risk score 4-8%).42 The transfemoral-access cohort of the study actually had lower rates of death
or disabling stroke at 2 years compared to open surgery. The PARTNER III trial was recently
published in March 2019, which demonstrated superiority of TAVI via transfemoral access
compared to traditional open surgery over one year in low-risk patients with respect to a
composite outcome of death from any cause, stroke, or rehospitalization.46 Secondary endpoints
also demonstrated significantly lower rates of stroke, new-onset atrial fibrillation, and shorter
index hospitalizations in the TAVI group compared to surgery. Although the PARTNER series
of trials were performed using exclusively the Edwards Sapien™ family of TAVI bioprostheses,
the U.S. COREVALVE High Risk Study,47 SURTAVI,48 REPRISE III,49 Evolut Low Risk
Trial,50 and many other trials have demonstrated similar results for various competing TAVI
valve systems.
1.1.4 Evolving Role of Imaging Guidance in TAVI
Given the minimally invasive nature of TAVI, performing the procedure safely and effectively
relies heavily on the appropriate use of imaging guidance.6–8,14,15,19 As previously mentioned,
TAVI is performed in an x-ray fluoroscopy suite, either a routine cardiac catheterization
9
9
laboratory (also known as a ‘cath lab’), or a hybrid cath lab-operating room. In the earlier days of
TAVI implementation, the majority of patients underwent a planning transesophageal
echocardiogram (TEE), cardiac catheterization, and an aortic-phase contrast CT of the vessels
extending from the aortic arch to the level below the femoral artery bifurcations. The purpose of
the planning TEE was to assess the aortic annulus diameter, distance from the annulus to the
coronary ostia (coronary origins), leaflet lengths, and assess the degree of calcification of the
valve, aortic annulus, and the left ventricular outflow tract (LVOT).6 The annulus diameter from
TEE was used in conjunction with the measurements from CT to determine the size of the
prosthesis required. The amount of calcification at the level of the aortic annulus and the LVOT
helped predict the difficulty in expanding the valve, the risk of causing aortic rupture (an almost
certainly fatal complication), and paravalvular leak (leak around the valve frame) due to irregular
depositions of calcium. The distance from the level of the annulus to the coronary ostia, along
with the leaflet length allowed operators to assess the risk of occluding one or both coronaries
following valve deployment.6,7,13 As mentioned previously, a large French registry of TAVI
patients had demonstrated the rates of planning TEE on average from 2010 to 2014 had dropped
by 48%, and likely has continued to drop in contemporary practice due to the advances in
multidetector CT.17
Contemporary multislice, gated CT technology has allowed CT imaging to effectively replace
TEE for the above measures, and its widespread use for anatomic pre-procedural assessment has
changed the algorithm for assessment for TAVI eligibility substantially.6,14,16,19 A typical TAVI
planning CT study will include measures of aortic annulus diameter and perimeter at a defined
phase of the cardiac cycle, sinus of valsalva diameters (base of the cusps), sinotubular junction
(where cusps meet aorta), coronary ostia distances from the aortic annulus, and diameters of all
vessels extending from the access site to the valve (Figure 1.4).14,16 CT has been demonstrated to
more accurately predict post-implant paravalvular leak compared to TEE, given the shape of the
annulus is slightly more often oval than circular.51 CT images may be easily re-formatted in the
plane of the aortic annulus in post-processing, whereas TEE relies on the skill of the operator
during the study to correctly align with the plane of the annulus. The aortic annulus mean
diameter (at the level of the virtual basal ring, Figure 1.4) is used more commonly for sizing of
balloon-expandable aortic valves given the outward pressure has a tendency to ‘circularize’ the
annulus, whereas self-expanding valves are sized to perimeter at the same level, due to their
10
10
tendency to conform more to the pre-existing annulus shape. The pattern of calcium at the level
of the annulus and the LVOT play a particularly important role in predicting where paravlavular
leak may occur.52,53 Asymmetric calcium patterns in the sub-annular region or the LVOT
(especially towards the left coronary cusp) have been particularly predictive of aortic rupture
when using balloon-expandable valves, and therefore patients with this pattern often have their
procedures performed using self-expanding prostheses.54
Figure 1.4: Three-dimensional (3D) representation of aortic valve cusp layout. 3D arrangement
of the three aortic valve cusps is shown in magenta. Virtual basal ring (purple) represents the
circular plane in contact with the base of all three cusps (primary measure for TAVI valve
sizing). Anatomic ventriculo-arterial junction (light blue) represents the plane dividing
ventricular and aortic ‘compartments’ when aortic valve is closed. Sinotubular junction (darker
blue) represents the plane where the cusps transition into the tubular portion of the aorta.6
Reproduced with permission Bleakley, C. & Monaghan, M. J. The pivotal role of imaging in
TAVR procedures. Curr. Cardiol. Rep. 20, 9 (2018).
Specialized image processing software has allowed TAVI operators to estimate the optimal
fluoroscopic angle for valve deployment, orthogonal to the plane of the aortic annulus, using the
pre-procedure CT study.14,15 For fluoroscopy, the goal for the operator is to produce a view such
that the bases of the three aortic cusps are linear, visible, and equally spaced (Figure 1.5). This
represents a view perpendicular to the aortic annulus with greatest separation between the left
and non-coronary cusp.55 A virtual fluoroscopic view may be generated in the image processing
software based on the CT anatomy, which informs the operator of the optimal angle prior to the
start of the procedure.
11
11
Figure 1.5: Representation of ideal alignment of left (L), right (R), and non-coronary (N) cusps
(left image) by aortic root angiography (right image).55 Reproduced with permission from Kasel,
A. M. et. al. Fluoroscopy-guided aortic root imaging for TAVR. JACC Cardiovas. Imaging. 6,
274-275 (2013).
The particular value of a fluoroscopic angle orthogonal to the plane of the annulus is in
estimating the depth of the TAVI prosthesis implant upon deployment.56 Given the fluoroscopic
image is a 2D projection of the TAVI prosthesis passing through the ‘imaginary’ plane of the
aortic annulus, any projection off-axis will significantly either over-, or underestimate the valve
implant depth. For example, the optimal implant depth of a CoreValve Evolut TAVI
bioprosthesis is between 2 and 6 mm from the base of the valve frame to the plane of the aortic
annulus. If the frame is lower (i.e. distance is greater than 6mm), the patient is at significant risk
of requiring a permanent pacemaker due to iatrogenic heart block. This potential complication is
due to the outward pressure of the valve frame on the adjacent AV node, or bundle branch fibers
in the interventricular septum. If the frame is higher (ie. distance is less than 2 mm), there is a
significant risk of paravalvular leak due to an ‘incomplete seal’ between the valve frame and the
native valve, and a non-trivial risk of valve embolization (valve frame moving completely out of
the plane of the annulus).52,56 If there is significant paravalvular leak due to an incomplete seal, it
is often required to place a second valve within the first valve at a lower position immediately
after the first one is placed to correct the problem. If valve embolization occurs at the time of the
procedure, the operator can use a snare to move the valve around the aortic arch to the level of
the descending aorta, and place a new TAVI valve in the correct position.57 The above
circumstances involve significant periods of hemodynamic instability prior to the placement of
the second valve (assuming the patient survives the event); furthermore, the placement of a
12
12
second valve predisposes the patient to a higher rate of long-term complications, including valve
thrombosis. For these reasons, it is of paramount importance to achieve the correct fluoroscopic
angle perpendicular to the plane of the aortic annulus, which is greatly informed by the pre-
operative CT.
The technique of using fluoroscopy as the sole imaging modality for guiding valve deployment
during a TAVI procedure represents the majority of cases in contemporary practice due to the
wealth of information provided by the planning CT study; however, a subset of operators
continues to use live TEE for selected cases for important reasons outlined below. As mentioned
earlier, TEE is performed by introducing an ultrasound imaging probe into the lower esophagus
via the patients’ mouth. TEE provides substantially better spatial and temporal resolution
compared to TTE due to the close proximity of the imaging probe to the left side of the heart,
and fewer tissue or air-tissue interfaces between the probe and the area of interest.12 In order to
perform TEE imaging during a TAVI procedure, a patient must be placed under general
anesthesia to tolerate the placement of the TEE probe for an extended period of time, compared
to the option of heavy conscious sedation, which may be employed in cases using fluoroscopy
alone.6,7,11,13 The use of general anesthesia requires heavier sedation, and involves placement of
an endotracheal tube to support the patient’s breathing throughout the procedure. Heavy
conscious sedation involves administration of significant amounts of sedation under the
supervision of an anesthesiologist; however, the patient breathes spontaneously throughout the
procedure. Patients under heavy conscious sedation compared to those under general anesthesia
are less prone to hemodynamic instability requiring the use of vasopressors to support their blood
pressure throughout the case.
Despite the drawbacks of general anesthesia, TEE continues to be used for imaging guidance in
selected cases due to the high-resolution soft-tissue contrast and real-time hemodynamic
information provided to the operator. A few definite advantages of intra-procedural TEE imaging
are: 1) the ability to determine the depth of implant in real-time without the use of contrast prior
to valve deployment; 2) the ability to assess the degree of paravalvular leak post-valve
deployment more accurately and without the use of contrast; and 3) the ability to monitor the
opening of the native valve leaflets in real-time during valve deployment, especially their
position with respect to the coronary ostia.11,13 For these reasons, intraprocedural TEE imaging is
particularly useful in cases where the operator would seek to limit exposure to contrast (such as
13
13
for patients with significant renal insufficiency), for patients with irregular patterns of calcium at
the level of the annulus or LVOT who may be more prone to paravalvular leak, or for patients
with low coronary ostia height and/or long native leaflet length who may be at higher risk for
coronary occlusion post-valve deployment. The late consequences of paravalvular leak post-
TAVI have been well-studied, and a published review of literature recently highlighted that it is
associated with a 2-fold increase in mortality at 2 years compared to trace or no paravalvular
aortic insufficiency at time of discharge from hospital.53
The issue of paravalvular leak is particularly important when attempting to perform TAVI in the
population with bicuspid aortic valve disease. As mentioned earlier, bicuspid aortic valve disease
represents as high as 50% of the aortic valve disease cases in patients at a mean age of 70 years
old referred for surgical management.26 The predominantly genetic condition causes the patient
to have two, rather than three aortic valve leaflets/cusps. These patients are prone to developing
aggressive forms of aortic stenosis at an earlier age.25 A subset of these patients is either
unsuitable or unwilling to under traditional open cardiac surgery and thus must to be considered
for definitive management with TAVI. The main issue arising in this group is the predominantly
ovoid shape of the aortic annulus (far less circular than the traditional tricuspid aortic valve
population) predisposing them to paravalvular leak. The availability of live Doppler for
paravalvular leak assessment afforded by TEE is therefore highly advantageous in performing
TAVI for these patients. The operator is better informed during the procedure regarding the exact
depth of implant and the amount of post-dilation of the valve required to more effectively reduce
paravalvular leak compared with angiography alone.58
Although there are several important advantages listed above of performing TEE during TAVI,
balancing these risks against the risk and logistics required for placing a patient under general
anesthesia continues to favor not performing TEE. Unfortunately, fewer patients are benefiting
from the real-time soft tissue and hemodynamic information obtained by TEE as the popularity
of x-ray-alone procedures continues to grow. The pressure from hospital administrations to
perform these procedures under conscious sedation is heavily influenced by reducing costs of
these procedures and associated length of stay in hospital.
14
14
1.1.5 Case for the Use of Intracardiac Echocardiography (ICE) during TAVI
Intracardiac echocardiography (ICE) is a catheter-based ultrasound imaging technique that
provides high spatial and temporal resolution images of the anatomy within the heart
surrounding the catheter. An ICE catheter is introduced via a venous approach (either femoral or
jugular vein) to the right side of the heart, where it is able to visualize nearby cardiac structures
with resolution similar to TEE. The ICE catheter contains a handle component that allows the
operator to steer the tip of the catheter and change the orientation of the imaging plane without
the help of a dedicated sonographer, which represents a significant advantage of this technique.
The main disadvantage of performing imaging from this perspective is the learning curve
required to understand the unique views inherent in ICE imaging. Users tend to overcome this
limitation with further experience; however it would be beneficial for the operator to have the
position and orientation of the ICE catheter displayed with 3D anatomic context, which is the
main purpose of our work.
Contemporary ICE imaging catheters can provide an imaging depth of penetration of up to 15
cm, which is adequate to visualize any structure within the heart. There are two main types of
ICE catheters available: 1) phased-array catheters, which provide a 90-degree sector view, and 2)
mechanical rotating element catheters, which provide a 360-degree view. A phased-array
imaging catheter, such as the Acuson Acunav ICE imaging system (Siemens, Mountain View,
CA, USA), uses a 64-element transducer with frequencies between 5 and 10 MHz to produce a
90-degree sector image (Figure 1.6A). A mechanical rotating transducer, such as the Conavi
Foresight ICE catheter (Conavi Medical, Toronto, ON, Canada) uses a single rotating imaging
element operating at 9MHz to produce a 360-degree view (Figure 1.6B).
15
15
Figure 1.6: Images acquired from phased-array (left) and mechanical rotating element (right)
ICE catheters. Image on the left was acquired within right ventricle (RV) providing a 90-degree
sector view of RV (superior chamber) and LV (inferior chamber). Image on the right features a
mechanical rotating element within a catheter tip (bullseye) in the RA (right atrium) beside
membranous portion of interatrial septum (immediately adjacent to catheter tip between 2 and 7
o’clock). LA, left atrium; IAS, interatrial septum; ICE, intracardiac echo catheter tip.
Thomas Bartel et al. published a review article in the European Heart Journal regarding the
benefits of the use of ICE imaging during TAVI.59 The most pronounced benefit of ICE imaging
compared TEE according to the review was the advantage of ICE being compatible with
monitored anesthesia care without endotracheal intubation (conscious sedation), as mentioned in
the previous section. Further benefits include uninterrupted procedural monitoring, lack of
fluoroscopic interference (which can often be problematic with TEE), and precise Doppler-based
assessment of pulmonary artery pressures.59,60 The article describes how uninterrupted ICE
imaging could be safely performed throughout the procedure with the tip of the catheter
positioned in the right atrium. Live assessment of paravalvular leak post-valve deployment using
ICE imaging is demonstrated in Figure 1.7. ICE is also particularly well suited for monitoring for
catastrophic complications, such as aortic rupture, wire perforation resulting in pericardial
effusion, and coronary occlusion.59 Such complications are almost certainly fatal if not
recognized and acted upon by the operator in a short period of time. Many hybrid catheterization
laboratories can be converted to a traditional cardiac operating room in these circumstances in
order to immediately repair the underlying structure and reverse the effects of the complication.
16
16
Figure 1.7: Use of ICE in monitoring for presence of paravalvular leak (PVL or PL) post-valve
deployment. Position of the ICE catheter within the right atrium in order to image transcatheter
heart valve (THV) in short axis, shown in A. Short-axis images of mild PVL (B), encompassing
< 10% of circumference. Severe PVL demonstrated in C, comprising of > 20% of circumference.
No persistent paravalvular leak demonstrated in D, after post-dilation (balloon expansion) of
valve demonstrated in C. SVC, superior vena cava; RV, right ventricle.59 Reproduced with
permission from Bartel, T., Edris, A., Velik-Salchner, C., & Muller, Silvana. Intracardiac
echocardiography for guidance of transcatheter aortic valve implantation under monitored
sedation: a solution to a dilemma? Eur. Heart. J. Cardiovasc. Imaging. 17, 1-8 (2016).
As the population of patients eligible for TAVI over SAVR continues to grow, and particularly
in the relatively younger demographic (aged 40-70), there will be an increasing imperative for
aggressive valve expansion, and avoidance of catastrophic complications. For example, if a
patient aged 50 undergoes a TAVI, it is expected that with an anticipated valve longevity of 10-
15 years that this patient will likely undergo an additional 2 valve-in-valve replacement
procedures in their lifetime. Each successive replacement adds a new layer of prosthesis at the
level of the annulus (given the old layer is not removed), thus limiting the ability to maximize the
17
17
area of the orifice, or effective orifice area (EOA). Retrospective studies have demonstrated a
clear link between lower starting EOA post-TAVI and clinical events,61 as well as earlier valve
deterioration. In addition, the degree of paravalvular leak is also associated with earlier re-
intervention, which is of particular importance to younger patients given they may have to face
an increase in the number of repeat procedures. ICE is particularly useful in evaluating adequacy
of expansion of the prosthesis post-deployment compared to fluoroscopy, as well as the degree
of paravalvular leak, which will more accurately guide appropriate valve post-dilation. For these
reasons, ICE will potentially impact the durability of the TAVI prosthesis and rates of re-
intervention in these patients in a significant way.
The purpose of our work is to create software for the purpose of tracking the position and
orientation of an ICE imaging surface (either plane or curved surface) within the prior CT
volume using co-registration techniques. By using these software techniques, the program may
act independently of any position sensors or physical tracking devices. We believe that
displaying the ICE imaging surface in context of the three-dimensional surrounding anatomy will
help TAVI operators better appreciate the location of their imaging surface with respect to
neighboring critical structures or physical boundaries and make more informed clinical decisions
while using an ICE imaging catheter.
1.2 Technical Background In our work, we intend to use image-processing techniques to identify the position and
orientation of an ICE imaging surface (either plane or curved surface) in the context of a three-
dimensional CT volume. In broad terms, locating a two-dimensional surface within a three-
dimensional volume may be accomplished either by: 1) hardware methods, such as magnetic
position sensors, or 2) software methods that seek to align common features in the images, such
as image co-registration used in conjunction with a volume search method. The advantages of a
hardware-based solution include: 1) more efficient temporal updating due to the lack of
necessary image processing, and 2) less prone to registration errors caused by the introduction of
new equipment compared to software methods alone.62,63 Hardware methods such as magnetic
tracking systems, however, require dedicated equipment both externally and within the imaging
catheter to function, suffer from concerns regarding accuracy, and are prone to interference from
18
18
external magnetic signals,63,64 which are commonly present in cardiac catheterization
laboratories.
Software methods for determining the position and orientation of a surface within a 3D volume
are able to operate independent of specific hardware, and therefore may be inherently equipment
and equipment vendor-independent. These methods do not interfere with workflow in the
catheterization laboratory or operating room since they do not require any additional external
hardware, such as magnetic position sensors, or shielding from magnetic fields that may interfere
with tracking. Although lag-time from image processing requirements is a key concern regarding
software methods, the use of parallel processing techniques, such as graphic processor units
(GPUs), increases the processing efficiency of these methods by hundreds or even thousands-
fold. The techniques may also be adopted for various combinations of 2D and tomographic
imaging combinations, such as ultrasound-MRI, with only minor adjustments to the underlying
algorithm.
For the above reasons, we have chosen a software-based method for tracking the position and
orientation of a 2D ICE imaging surface within a CT volume. In this section, we will provide a
background on medical image co-registration, which serves as the basis for our method, as well
as prior work in cardiac ultrasound-CT co-registration.
1.2.1 Medical Image Co-Registration
Medical image co-registration is the process of aligning two or more medical images taken with
different acquisition properties, or different forms of contrast in order to benefit from the
information gathered from each imaging modality, and their corresponding overlap. It is
common to perform medical image co-registration for imaging modalities with vastly different
anatomic contrast mechanisms, or to overlap high-fidelity anatomic images with those
demonstrating functional or metabolic information. For example, a clinician may seek to align
magnetic resonance (MR) images of a prostate with transrectal ultrasound (TRUS) in order to
help guide prostate needle biopsy for detection of prostate cancer,65 or combine anatomic images
of a lung tumor derived from CT with images of metabolic activity derived from positron
emission tomography, or PET, in order to better guide radiation therapy.63,66
19
19
Template and Reference Images
Medical image co-registration may be generally viewed as an optimization problem that acts to
minimize an associated cost function as a measure of how closely the two images are aligned.
The problem of medical image co-registration may be broken down into several steps in order to
gain an easier understanding of the overall process. The problem begins with two medical
images; one image serves as the template, T, which will later be transformed, and the other as
the reference (or static) image, R. The purpose of optimization-based co-registration is to
perform some transform of the template image, T, such that it more closely resembles the
reference image, R.
Multilevel Image Representation
Multilevel image representation is a pre-processing step for image co-registration, which
involves developing progressively coarser representations of the medical image data. The
algorithm first attempts to align features in the coarse representation of the image prior to
aligning images containing finer features. Coarse representations of images are generated by
averaging pixel values over a defined space, depending on the ‘level’ of the representation. An
example of multilevel representation involving ICE and CT representations of a silicon cardiac
phantom is shown in Figure 1.8.
20
20
Figure 1.8: An example of multilevel representations: ICE (A) and CT (B) images of a silicon
cardiac phantom (black) in a holder (white) are used for 2D-2D co-registration testing.
Multilevel representations of the overlaid transformed ICE (C-F, red) and CT images (C-F,
background) are shown. The algorithm starts by aligning coarse features (C), and progressively
aligns finer features as more details are added (D-F).
Advantages of performing medical image co-registration in an iterative manner using multilevel
image representations include: 1) optimization is far less computationally expensive in earlier
stages using coarse representations of the images, and 2) by aligning anatomically coarse
features earlier in the process, the algorithm is less likely to get caught within false local minima
when attempting to align finer features. A set of user-defined constraints, or regularizers, act on
the data at each level of image co-registration to prevent the process from tracking within false
local minima. The regularizer effectively prevents large-scale transformations from being
performed between iterations, reducing the scope of the search at each level, and thus successive
21
21
multilevel representations may undergo fewer processing steps at each level, saving a significant
amount of overall computation time.
Distance Measures
The purpose of image co-registration is to minimize the difference between two images, which
can be quantitatively analyzed as a distance measure. In broad terms, a distance measure,
D[T[y], R], is a measure of the similarity between the transformed template image, T[y], and
the reference image, R. It may be measured using a number of different approaches, chosen
depending on the characteristics of the imaging modalities under comparison. Commonly used
distance measures in medical image co-registration include sum of square differences (SSD),
normalized cross-correlation (NCC), mutual information (MI), and normalized gradient fields
(NGF).
In SSD, the ‘distance’ measured between the template and the reference image is based solely on
the sum of the differences between the template and reference squared.67 Mathematically, it is
represented as follows:
𝒟[𝒯[𝑦], ℛ] = 12, -𝒯[𝑦](𝑥) − ℛ(𝑥)23𝑑𝑥
5(Eq1)
W represents the generic interval over which the integration occurs. The SSD distance measure is
most appropriately suited for images with similar contrast, typically acquired with the same
modality, such that a transform of the template produces the reference image.
Normalized cross-correlation assumes a linear relationship between the pixel intensities of the
template and reference images, which can be expressed as: λT[y] = μR, where λ and μ are
scalars. ℝ9 represents the generic interval of real numbers in d dimensions over which
integration occurs.68 An expression to assess the degree to which two images are correlated, <T, R>, is written as:
⟨𝒯, ℛ⟩ = , 𝒯(𝑥)ℛ(𝑥)𝑑𝑥ℝ<
(Eq2)
22
22
In order to normalize this value to a scalar between 0 and 1, a normalized cross-correlation
divides the above value by the product of the square root of the template and reference images
correlated with themselves, such that:
NCC[𝒯,ℛ] = ⟨𝒯, ℛ⟩‖𝒯‖‖ℛ‖ ,where
‖𝒯‖ = D⟨𝒯, 𝒯⟩(Eq3)
The above measure of NCC produces a value that reaches a maximum of 1 when the images are
the same within a scalar term. In order to use NCC in a traditional minimization problem, the
distance measure is expressed as:
𝒟HII[𝒯,ℛ] = 1 − NCC[𝒯,ℛ]3 = 1 −⟨𝒯, ℛ⟩3
‖𝒯‖3‖ℛ‖3 (Eq4)
The values of the NCC in the equation are squared in order to place equal weighting for directly
proportional, and inversely proportional correlations, thus imaging modalities with inverted color
scale are also viewed as highly correlated. An example of this principle is brain MR images
acquired using T1 and T2-weighted protocols.
Mutual information assumes no linear relationship between the information contained in T and
R. A histogram of the values contained in T and R is created, which summarizes the
coincidences of all possible pairs of values that can be compared between them.69,70 A measure
of the entropy, H, reflects the ‘sharpness’ of the histogram, and is a key determinant of the
distance measure. Due to the complexity of the mathematical underpinnings of this distance
measure, we will refer the reader to Chapter 7 in ‘FAIR: Flexible Algorithms for Image
Registration’ by Jan Modersitzki for further details.71 Mutual information is particularly well
suited for imaging modalities with drastically different contrast mechanisms, such as comparing
tomographic imaging techniques (CT, MR) with metabolic imaging (PET, SPECT, Thallium).70
The normalized gradient fields (NGF) distance measure is based on comparing the normalized
pattern of image intensity changes between the template and the reference. It is far less restrictive
than the SSD and NCC methods, which rely heavily on comparing the pixel values throughout
the image, however far less general than the MI method, which analyzes the entropy of a mutual
23
23
information histogram. The concept of using changes in image intensity to compare two imaging
modalities was borne out of the observations that: 1) intensity changes naturally play a dominant
role in co-registration processes, and 2) anatomic boundaries with sharp changes in image
intensity are often well-preserved between imaging modalities. In other words, areas of image
intensity changes in one imaging modality should correspond to the image intensity changes
based on anatomic boundaries in another.
To perform analysis based on NGF, gradient representations of the images are created in order to
map the changes in image intensities.72,73 Given the magnitude of change varies across different
imaging modalities, gradient representations, such as ∇T, are divided by the magnitude of the
entire gradient field, |∇T|, thus focusing solely on locations of changes within the image rather
than their magnitude. The normalized gradient field of the template image, n[T] or n[T, η], is
defined as:
𝑛[𝒯] = 𝑛[𝒯, 𝜂] = ∇𝒯
D|∇𝒯|3 +𝜂3(Eq5)
where η is a scalar which serves as an ‘edge parameter’, such that |∇T| > η at a point that is
considered an edge, and |∇T| < η at a point which is considered within the noise level. For
example, by reducing η, the distance measure is more sensitive to detect edges, however will be
more susceptible to falsely assuming that noise-level variations represent anatomic boundaries.71
A comparison of the gradient representations of the template, n[T], and the reference, n[R], images is performed by determining the area spanned by the two vectors as a measure of linear
dependency. This is performed by multiplying the transpose of the template by the reference, and
subsequently squaring the result in order to remove bias for the absolute direction of change. The
resulting value, (n[T]T . n[R])2, is a maximum when edges align, and thus to use this in a
traditional minimization problem (over interval W), it is re-termed as: 1- (n[T]T . n[R])2. Thus,
mathematically, the minimization problem for NGF may be expressed as:
𝒟HQR[𝒯, ℛ] = NGF[𝒯,ℛ] = , 1 − (𝑛[𝒯(𝑥)]U5
𝑛[ℛ(𝑥)])3(Eq6)
24
24
Regularization and the Ill-Posed Problem
Co-registration may be viewed as an ill-posed problem resulting in a highly non-convex cost-
function (Figure 1.9).74,75 In a non-convex circumstance, several unique and independent
solutions can be inferred at locations of various independent local minima. A way to resolve the
discrepancy of multiple solutions to an ill-defined problem is to introduce regularization, a term
whose purpose is to modify the problem such that there is a unique solution, or resolve the non-
convexity of the cost function. If multiple potential solutions exist, a term that differentiates the
quality of the solutions can lead to one that is unique based on user-defined constraints.
Figure 1.9: Graphical examples of convex (left) and non-convex (right) cost functions.74
Reproduced with permission from Reali, F., Priami, C., & Marchetti, L. Optimization algorithms
for computational systems biology. Front. Appl. Math. Stat. 3, 6 (2017).
Elastic-based regularization is the most common form of regularization used in contemporary
image co-registration algorithms to solve the issue of non-convexity of the cost function. An
elastic regularizer is based on the concept of potential energy created by deforming an elastic
material.76 It is applied to a non-convex cost function as a method to penalize solutions requiring
large transformations (in an elastic manner) in order to optimize the distance measured.
Mathematically, it is expressed as:
𝒮[𝑦] = 12,
|ℬ[𝑦]|3𝑑𝑥,whereℬ = YD𝜇∇ 00 D𝜇∇
Dλ + 𝜇𝜕^ Dλ + 𝜇𝜕3
_5
(Eq7)
25
25
λ and μ represent the Lamé constants, which are material-dependent quantities that arise in
elastic stress-strain relationships. In the elastic stress-strain model, μ can be thought of as the
shear modulus. Ñ represents the divergence operator, and 𝝏1 and 𝝏2 represent first- and second-
order differential operators. W represents the generic interval over which the integration occurs.67
As mentioned previously, the main advantages of using regularizers in the image co-registration
process are to: 1) improve image co-registration accuracy by preventing the process from
tracking within false local minima, and 2) improve processing efficiency by limiting the scope of
transformations, and thus processing steps or computation time, at each iteration of a multilevel
image representation.
Transformations
In the co-registration process, iterative transformations of the template image, T, are performed,
and the resulting image is compared to the reference image, R, at each iteration by means of the
distance measure and the regularizer. If a transformation results in a smaller ‘distance’ measured,
the resulting template image may be viewed as more ‘similar’ to the reference. As mentioned
previously, transformations of the template image, T[y], may be classified as rigid
transformations, involving only rotation or translation, or spline-based transformations, which
also includes more free-form transformations such as shearing and scaling of individual
components of the image.77
Mathematically, a 2D rigid transformation is expressed as:
𝑦^ = cos(𝑤^) 𝑥^ − sin(𝑤^)𝑥3 +𝑤3
𝑦3 = sin(𝑤^) 𝑥^ + cos(𝑤^)𝑥3 +𝑤h
ory=wx,wherew=[𝑤^;𝑤3;𝑤h] ∈ ℝh(Eq8)
For each point (x1, x2) in an image, a rigid translation can be defined into another space (y1, y2).
w = [𝑤^; 𝑤3;𝑤h] denotes a translation vector. ℝh denotes real numbers in three dimensions.
Rigid transformations are particularly useful in maintaining computational efficiency in the co-
registration process given they are limited to three degrees of freedom (DoF). The computational
26
26
efficiency is achieved at a cost of robustness, especially in comparing imaging modalities with
vastly different contrast mechanisms and artifacts, which may somewhat distort geometry in
different manners.
Spline-based transformations require a greater deal of computational processing given the ability
to act differently on individual components of the image. Mathematically, they are far more
complex than rigid body transformations:
𝑦^ = 𝑥^ + n n 𝑤op,oq^ 𝑏op(𝑥^)𝑏oq(𝑥3
sq
oqt^
)andsp
opt^
𝑦3 = 𝑥3 + n n 𝑤op,oq3 𝑏op(𝑥^)𝑏oq(𝑥3
sq
oqt^
)(Eq9)sp
opt^
In the equations above, bj denotes the splines, w1 and w2 denote the coefficients of y1 and y2
respectively, and p1 and p2 are the number of coefficients in the spline expansion.
In order to limit the computational processing requirement of spline-based transformation,
regularizers places constraints on which types of transformations can be reasonably performed
for the data set. For example, elastic regularizers will limit transformations that result in
significant asymmetric compression, or scaling, of individual image components due to the large
‘elastic potential energy’ that would be required to perform such a transformation.
Spline-based transformations are particularly useful in circumstances where anatomic boundaries
may be ‘warped’ compared to one another due to differences in image acquisition properties. It is
important to reconcile, however, whether anatomic boundaries align due to the deformation
process itself, rather than common anatomic boundaries.
Putting Together the Optimization Problem
Co-registration may be viewed as an optimization problem whose aim is to find a transformed
version of the template image, T, which most closely resembles the reference image, R. The
optimization framework is expressed mathematically as follows:
27
27
𝒥[𝑦] = 𝒟[𝒯[𝑦], ℛ] + 𝑆[𝑦]z{⎯} min (Eq 10)
In the above formula, J[y] is the parameter being optimized, known as the joint functional, y
represents the transformation, D[T[y], R] is the distance measured between the transformed
version of the template, T[y] and R, and S[y] is the regularizer (or set of constraints) applied to
the transformation, y.
In the optimization process, the template and the reference images first undergo pre-processing
steps, where multilevel representations of both images are created. The multilevel images are
generated using a defined quantity of pixel averaging, which is used to define each ‘level’ of
image representation. At the lowest defined level of image representation, the template image, T,
enters the processing loop, and is initially compared to the reference image, R, using the pre-
defined distance measure and the regularizer term. The regularizer term differentiates the quality
of the solution based on user-defined constraints. In the case of elastic regularization, it is the
elastic potential energy difference between two versions of the image based on the
transformation required to produce them. The joint functional, J, is used to determine the
‘difference’ between the template image, T, and the reference image, R, using the combination
of the distance measure, and the additional values introduced by the regularizer term. An optimal
transformation is determined based on the transformation producing the lowest value in the cost
function (i.e. lowest J-value). As the loop repeats, the difference in the joint functional, J,
between the images in the previous and the current iterations is used to determine how much
‘optimization’ has occurred in the last round, denoted by J-Jold. During iterations of the
optimization process, the value of J-Jold continues to decline when the template appears more
similar to the reference in successive steps. A stopping condition is met when J-Jold crosses a
pre-determined lower threshold value. When the stopping condition is met, the transformed
template is thought to be as close to the reference as reasonably achievable. A graphical
representation of the algorithm at each multilevel representation is illustrated in Figure 1.10.
The algorithm repeats this optimization process for each level in the multilevel representation,
typically three to four times, based on user specification. Given the initial iteration is performed
using a coarse representation of the image, coarse image features are aligned first in the co-
registration process. Successive iterations start with a version of the template image transformed
28
28
from the previous iteration. The use of regularizers in successive steps ensure that alignment of
finer features does not deviate significantly from alignment of coarse features in prior iterations.
The 2D-2D ICE-CT co-registration process in our work performs the co-registration process
outlined above with three multilevel representations, normalized gradient fields as the distance
measure, elastic regularizers, and spline-based transformations.
Figure 1.10: 2D-2D Co-Registration Loop: 2D ICE surface (template) and 2D CT slice
(reference) undergo initial pre-processing (multilevel representation). Template and reference
images are compared using a chosen distance metric (NGF) to produce a distance measure. A
regularization term (elastic regularization in our case, S) sets the search constraints (ie. penalties
applied for large transformations). The template is subsequently transformed using a pre-
determined series of transformation (spline-based transformations). A joint functional, J, which
is a linear combination of the distance measure and the regularizer term is determined as a as a
measure of the quality of the overlap between the template and the reference. The process is
repeated until a pre-determined stopping condition (ie. threshold J-value) is met.
29
29
1.2.2 Search Method within the CT Volume
Geometric Representation of the ICE and CT Image Surfaces
In our work, the volume search method consists of performing an exhaustive search of the
position and orientation of a 2D ICE imaging surface within a 3D CT volume. Due to acquisition
properties of the ICE imaging catheter, the 2D ICE imaging surface is represented as a shallow
cone in three-dimensional space specified by the ICE catheter (usually with 10 degrees
curvature). At the centre of the ICE image is the ICE catheter itself, which is subtracted from the
image during analysis, and thus the ICE imaging surface is represented in 3D space by a shallow
cone with a missing section at the apex that is the radius of the ICE catheter. Subsequent
comparisons to CT data in our work require comparing the conical ICE imaging surface to CT
data with similar conical geometry extracted from the 3D CT volume. Shallow conical
geometries are extracted from the 3D CT volume by extracting CT data points according to the
coordinates of the surface of a conical mesh. The conical representations of both the ICE and CT
data are compared for the purpose of the co-registration algorithm.
Constructing the Cost Function
The joint functional, or J-value, of each individual 2D-2D image co-registration represents a
measure of the quality of the overlap. If multiple co-registrations are performed using CT
surfaces from various positions and orientations within the CT volume and a single ICE image,
the co-registration with the lowest J would represent the likeliest position and orientation of the
ICE image within the CT volume. For the purpose of our work, a 5-dimensional matrix of J-
values resulting from co-registration of CT surfaces at defined x, y, z positions within the CT
volume and at defined pitch and yaw angles (in-plane rotation addressed by 2D-2D co-
registration process) represents the cost function of the search. In our work, the cost function
represents a search within a given space (for example a 32 mm width and 16 degree arc) with a
defined resolution (for example 1 mm spacing and 1 degree resolution). The resulting cost
function in our example containing 32 points in each x, y, z direction, and 16 points in each pitch
and yaw rotation contains over 8 million individual elements, and thus processing shortcuts to
improve efficiency of the search are necessary to significantly reduce processing time.
30
30
Optimizing Efficiency of Volume Search
The shape of the associated cost function will heavily influence the strategy employed for
searching for a 2D surface within a 3D volume. In a circumstance of a highly convex cost
function (Fig 1.9a), a gradient search algorithm such as the gradient descent method is a
computationally efficient way to solve the optimization problem. In a gradient descent method, a
limited initial sampling of the cost function is performed. The derivative of the cost function of
the initial sampled version will indicate the areas and direction of the steepest negative slope,
which will be used as a starting point for a more detailed (or well-sampled) search. The process
continues in the direction of the slope until a unique solution is found at the minimum of the
curve. This method is used successfully in examples where solutions converge on a unique
global minimum.
In a problem involving a highly non-convex cost function (Figure 1.9B), such as the co-
registration methods used in our work, a more exhaustive form of searching within a 3D volume
is performed. To maintain computational efficiency, our method uses a series of under-sampled
passes in order to perform more detailed sampling only within the troughs of the cost function.
Each pass (n=4, 3, 2, 1) under-samples the data at a ratio of 2n-1 (8:1, 4:1, 2:1, 1:1), however only
the J-values falling below (100% - (n x 10%)) of the range of J-values (|Jmax – Jmin|) will be
sampled in the next layer. For example, the first layer is under-sampled at a ratio of 8:1, and only
the data falling below 60% of (|Jmax – Jmin|) will be sampled in the next layer. The second layer is
under-sampled at a ratio of 4:1, and only the data falling below 70% of (|Jmax – Jmin|) will be
sampled in the next layer, and so on. The above formulae were derived empirically from
experimentation with a model dataset.
A final step to increase processing efficiency is the high-resolution position and angle update
performed at the end of the volume search. For example, if the lowest position resolution using
the above sampling technique is 2x, the data surrounding the solution may be sampled with a
resolution of 1x by fixing the angle and searching locally for the best position. The algorithm
subsequently fixes position and searches locally for an angle with 1x resolution. This step may
be included if the resolution of the CT volume is less than the resolution of the volume search.
The result is an efficient search algorithm that performs high-resolution sampling only at the
lowest points of each trough where the optimization solutions are found. In an example of a
31
31
search using our algorithm with a 32 mm and 16-degree radius and 5 degrees of freedom, a 1:1
exhaustive search would require processing 8,388,608 co-registrations, however using the
progressive sampling method listed above, it is reduced to 174,618 co-registrations (48 times
faster) without suffering a theoretical loss of accuracy.
1.2.3 Prior Work in Cardiac Ultrasound-CT/MR Co-Registration for Use in Procedural Navigation and as an Adjunct to Magnetic Tracking Systems
Prior research in the field of intra-operative or intra-procedural co-registration of cardiac
ultrasound with a pre-procedural tomographic imaging modality, such as CT or MR, has been
conducted for various applications. Significant contributions to the field have been made by Dr.
Terry Peters and his team at the Robarts Research Institute in London, Canada.64,78–86 Co-
registration and hardware tracking efforts from this group using cardiac ultrasound and/or
tomographic imaging include: 1) magnetic position sensing of instruments during cardiovascular
interventions,62,63 2) co-registration of CT with TEE,78–83,85,86 3) use of pre-operative 4D
tomographic images,86,87 and 4) embedding a phased-array ICE catheter within the delivery
catheter combined with magnetic tracking.85 A number of the above projects were performed
with the main purpose of reducing contrast exposure during minimally invasive cardiovascular
procedures, which may reduce the risk of long-term kidney injury caused by iodine-based
contrast agents.
An example of magnetic position sensing alone includes a system for magnetic tracking during
deployment of thoracic aortic stent grafts.62 In this example, magnetic tracking sensors were
embedded in the device catheter and cannula for a thoracic aortic stent, as well as the ultrasound
probe to gauge the position of the device with respect to the intra-operative live ultrasound image
plane. Using the live ultrasound and magnetic position tracking information, it is possible to
perform this procedure in an x-ray fluoroscopy suite with minimal or no exposure to iodine
contrast. Tracking in this application was performed with localization errors of 1.4 mm and 4.2
mm in phantom and animal experiments respectively.
Examples of co-registration of ultrasound with CT also include combined systems involving the
use of both magnetic tracking and software image co-registration. In one example, the position of
32
32
the TEE probe contained a magnetic tracking system whose tracked position and orientation
would be used to determine the corresponding 2D plane in the pre-operative 3D CT volume.79
Retrospectively gated CT acquisitions were obtained at ten different points in the cardiac cycle in
the pre-operative setting, thus creating a 4D CT data set. Image co-registration between the live
ultrasound image and the CT plane at the corresponding point in the cardiac cycle (based on
ECG gating) was performed with a target registration error of 1.7 mm. Another example
incorporated an ICE imaging catheter within the delivery system with a magnetic tracking
system used to determine its position and orientation.85 Live ICE-CT co-registration could have
been performed using this technique, similar to the prior example; however, this was not
included in the paper.
The above examples include the use hardware tracking systems due to the computational
complexity involved in software-only methods. With further advances in the field of parallel
processing techniques, predominantly the use of GPU-based hardware acceleration, software-
only methods are potentially realizable with almost real-time temporal resolution.
33
33
Chapter 2
ICE-CT Co-Registration for the Purpose of Guiding Cardiovascular Interventions
2.1 Introduction Imaging plays a crucial role in performing transaortic valve implantation (TAVI), including the
use of tomographic imaging (such as CT or MR) for procedural planning and x-ray and
ultrasound-based techniques for live imaging guidance.6–8,14,15 In the earlier experience with
TAVI, procedures were almost exclusively performed under general anaesthesia with the use of
transesophageal echocardiography (TEE) and fluoroscopy, which provided the operator with
soft-tissue contrast and Doppler flow assessment in real-time.11–13 Advances in the use of CT for
the purpose of pre-procedure measurements and determining optimal intra-procedural
fluoroscopic angles have enabled operators to perform TAVI successfully using only
fluoroscopy.14,15,55 By relying solely on fluoroscopy, the majority of TAVI operators have
transitioned from performing procedures under general anaesthesia to performing them under
heavy conscious sedation without the need for intubation.17 Although reducing the rates of
general anaesthesia has effectively improved procedural safety and post-operative inpatient
length of stay, the loss of soft tissue contrast and hemodynamic information afforded by TEE has
some considerable drawbacks, including limitations in quantifying the degree of paravalvular
leak (PVL) post-valve deployment, which is well-correlated with adverse clinical events.53
Intracardiac echocardiography (ICE) represents a novel alternative to TEE for intra-procedural
imaging guidance for TAVI.59 ICE is an ultrasound-based imaging technology which uses either
a single rotating mechanical element or phased-array ultrasound transducer at the distal end of
the catheter to provide images with soft tissue contrast and colour Doppler information in real-
time without the need for increased levels of sedation. A common concern among early-
experience operators with ICE is the difficulty in appreciating the three-dimensional context of
the acquired ultrasound images. Given patients undergoing TAVI require a planning CT study, a
solution demonstrating the position and orientation of the ICE imaging surface in context of the
previously acquired 3D CT anatomy would be highly desirable.
34
34
Methods for determining the position and orientation of an ICE imaging surface within a 3D
image volume interprocedurally, including hardware and software techniques have been
previously attempted. Please refer to section 1.2.3 for further details regarding prior navigation
attempts using hardware techniques.
The use of software tracking methods based on alignment of anatomic features within the images
represents a robust approach in determining position and orientation of the catheter tip, however
at a cost of significant computational complexity. The main advantages of using such techniques
are the lack of dedicated hardware, interference with operative workflow, and interference from
external electromagnetic signals. Medical image co-registration may be used in these methods to
test the alignment of anatomic features within the images and provide metrics of the quality of
the alignment produced. The significant challenge of aligning images from two drastically
different modalities may be overcome by extracting well-preserved features such as major
anatomic boundaries in the comparison. Combined with three-dimensional search techniques,
these methods may be used to determine which image pairs produce the optimal alignment, and
thus reflect the most likely position and orientation of the catheter tip within the 3D volume.
Three-dimensional searches are limited in scope by only searching a small volume around the
last known location. Based on these principles, we present a system for determining the position
and orientation of an ICE imaging surface within a 3D CT volume using automated co-
registration methods for the purpose of improving imaging guidance using ICE during TAVI.
2.2 Methods
2.2.1 Software environment Software for the purposes of our study was developed in the MATLAB programming
environment (MATLAB & Simulink, Mathworks Inc, Natick, MA), using the Flexible
Algorithms for Image Registration (FAIR) package (Jan Modersitzki, et al, Institute of
Mathematics and Image Computing, University of Lübeck, Germany). A custom user interface
was developed for this application that demonstrates the current ICE imaging surface, the 3D CT
volume, and a three-dimensional representation of the position and orientation of the ICE surface
(shown in Figure 2.1). Using the software interface, the user is able to select the initial position
and orientation of the search, define the search constraints (including position and angle
35
35
constraints), the degree of curvature of the ICE imaging surface (as specified by the ICE
catheter), and sampling interval of the initial search. The conical representation of the ICE
imaging surface in 3D space may either be represented as the ICE image, or as a CT
representation of the ultrasound surface (as seen in Figure 2.1). Extracting CT data along this
geometry was created using the Cone MATLAB plugin (Waldemar Swiercz, et al, Harvard
University, MA) to define the inner and outer cone radii. Polar images were constructed by
extracting data along the lines connecting points at defined angles along the inner and outer radii.
Polar images were subsequently converted to Cartesian images, thus resulting in planar
representations of the conical geometry. Representing the CT data in this format allowed for
comparisons of the conical ultrasound geometry to similar geometry extracted from the 3D CT
volume.
Figure 2.1: Single 2D ICE imaging surface (top, left) and 3D CT volume (bottom, left) are
loaded into custom user interface. CT representation of the ICE imaging surface (right, blue) in
best estimated position and orientation is shown in context of CT volume.
2.2.2 Process Overview The search for the estimated position and orientation of the ICE imaging surface within a 3D CT
volume begins with a user-defined best estimate. The user starts by selecting their estimate of the
initial position and orientation of the ICE catheter tip within the CT volume. A location with
36
36
clearly defined unique anatomic features is advised as a starting point to reduce error introduced
in this first step. Following the initial ‘user defined’ step, the algorithm performs an automatic
local search in order to optimize position and orientation, and subsequent steps of the algorithm
are performed automatically within a limited search space. The user may specify the maximum
distance and orientation changes involved in the search, as well as the resolution (ie. spacing
between points) in the search. For example, for the purpose of our experiment, the maximum
distance searched for each step was 16 mm from the starting point (total 32 mm width) with a 0.5
mm final resolution, and the orientation search was limited to a 16-degree arc (-8 degrees and + 8
degrees from the starting orientation) with a 1-degree final resolution. The approach of using
limited local searches between each step was chosen due to processing efficiency, without
suffering a significant loss in accuracy as long as the catheter tip did not move outside of the
search space between steps. A local search strategy represents a reasonable method for tracking
the location of an ICE catheter since it typically does not undergo large rotations and translations
within a limited time period during invasive cardiac procedures.
The search for position and orientation of the ICE catheter within a limited volume at each step
comprises of numerous individual 2D-2D image co-registrations at various positions and
orientations. The algorithm extracts a planar image representation of a conical CT surface at a
specified position and orientation, and with a pre-defined curvature angle (based on the curvature
of the ICE surface). A 2D-2D image co-registration is performed between the extracted CT data
and the ICE image (outlined in detail in section 2.2.3). A resulting value, J, known as a joint
functional, is derived from the 2D-2D co-registration algorithm and quantifies the quality of the
overlap between the two images. The algorithm repeats this process for each position and
orientation within the local search volume at a pre-determined spacing interval. The matrix of J-
values at various positions and orientations thus represents a multidimensional cost function. The
position and orientation with the lowest J-value, or lowest point in the cost function (ie. the best
overlap of the images), represents the estimated position and orientation of the ICE catheter. An
exhaustive search method is used for the position and orientation search due to the presence of
multiple minima and non-convexity of the cost function. A strategy of under-sampling the cost
function, and subsequently performing more detailed sampling of data falling under a certain
pre-defined threshold was performed (outlined in detail in section 2.2.4). The algorithm thus
employs an exhaustive search of a cost function derived from joint functional values from
37
37
individual 2D-2D co-registration between a static ICE image and CT images at various positions
and orientations to determine its estimate of the position and orientation of the ICE catheter tip.
2.2.3 2D-2D Image Co-Registration Image Pre-Processing
For the purpose of the co-registration process, the CT data will be identified as the reference
image, and the ultrasound image as the template image. ICE and CT images underwent initial
pre-processing where fine features were smoothed in a stepwise manner, creating what is known
as a multilevel representation. The use of multilevel representation in our work is explained
further in section 1.2.1 (under Multilevel Image Representation heading). Coarse representations
of the images were aligned using the 2D-2D co-registration algorithm, the output of which
served as the starting point for the next iteration of the algorithm. Subsequently, less smoothed
versions of the images underwent the 2D-2D co-registration process. The images underwent this
process three times prior to final image alignment.
Distance Measure
Normalized gradient fields (NGF) were used as the distance measure for the 2D-2D co-
registration process. The distance measure, D[T[y], R], is a measure of the similarity between
the transformed template image, T[y], and the reference image, R. Further details regarding the
use of NGF as a distance measure in the 2D-2D co-registration process may be found in section
1.2.1 (under Distance Measures heading). Gradient representations of the ultrasound and CT
image were used in our work since they represent the location of image intensity changes, rather
than reflecting the absolute magnitude of the intensity changes. The use of NGF as a distance
measure therefore is ideally suited for comparing vastly different imaging modalities, which both
contain well-preserved anatomic boundary information, however with differing magnitudes of
change at boundaries, such as those seen in ultrasound and CT.
Regularization
Elastic-based regularization was employed in our study to help resolve the non-convexity of the
cost function. An elastic regularizer is based on the concept of potential energy created by
deforming an elastic material. When used in the context of comparing two images, quantitative
38
38
penalties are applied for large transformations in an elastic manner when compared to the
previous iteration. Further details may be found in section 1.2.1 (under Regularization and the
Ill-Posed Problem heading).
Transformation
Spline-based transformations were used for the purpose of our study, which includes rotation,
translation, and some free-form transformations such as rotation, shearing and scaling of
individual components of the image. Non-rigid transformations are useful in comparing images
from imaging modalities with vastly different acquisition mechanisms given the distortions and
artifacts that may occur at anatomic boundaries. The use of regularization limits substantial
image deformation that would potentially cause a reference image to co-register to a significantly
distorted version of the template image, rather than the image itself. Further details may be
found in section 1.2.1 (under Transformations heading).
Solving the 2D-2D Optimization Problem
The optimization framework used in our work to find a transformed version of the template
image, T, which most closely resembles the reference image, R, is expressed mathematically as
follows:
𝒥[𝑦] = 𝒟[𝒯[𝑦], ℛ] + 𝑆[𝑦]z{⎯} min (Eq 10)
In the above formula, J[y] is the parameter being optimized, known as the joint functional, y
represents the transformation, D[T[y], R] is the distance measured between the transformed
version of the template, T[y] and R, and S[y] is the regularizer applied to the transformation, y.
In our work, the template and reference images are pre-processed as multilevel representations
prior to entering the co-registration loop. The template and reference images are compared using
the NGF distance measure and regularization terms. A pre-prescribed transformation is
performed and the loop repeats with the transformed template image compared to the reference
image. If the J-value between the images is less than the prior iteration, the transformed image
serves as the starting point for the next iteration. The loop repeats until the difference between
the J-value of the prior iteration and the current iteration reaches a lower threshold value. The
process repeats a total of three times, one for each set of multilevel representations. The resulting
39
39
transformation of the prior multilevel representation serves as a starting point for the next
iteration. Further details regarding this process may be found in Section 1.2.1 (under the heading
Putting Together the Optimization Problem).
2.2.4 Search Method within the CT volume Cost Function
For the purpose of our work, the cost function for the localization search was constructed using
joint functional values, or J[y], at each of the x/y/z translation and pitch/yaw rotation steps in the
volume search. J[y] may therefore be viewed as a cost function in five dimensions. Further
details regarding constructing the cost function may be found in Section 1.2.2 (under the heading
ConstructingtheCostFunction).
Optimizing Search Efficiency
Efficient and accurate processing of a cost-function representing a 5-DoF search within three-
dimensional space requires several designed shortcuts in order to reduce computation time. For
the purpose of our work, the 5-DoF search began with an initial user-defined starting point as a
‘best estimate’ of the position and orientation of the ICE imaging surface within the CT volume.
In each iteration of the 5-DoF search, the search was limited within a 16 mm translation distance
from the starting point (total 32 mm search width) and 16 degrees of rotation (-8 to +8 degrees
pitch and yaw). Given an ICE catheter does not typically perform large rotations and translations
suddenly during an invasive cardiac procedure, the limited search scope represents a reasonable
shortcut.
The matrix of J-values derived from the 5-DoF search within three-dimensional space is an
inherently non-convex cost-function with several local minima. An exhaustive search strategy
must therefore be employed to better resolve the difference between these local minima. A
layered progressive under-sampling strategy was used in our work in order to maintain
processing efficiency. AfinalsteptoincreaseprocessingefficiencyistheHigh-Resolution
PositionandAngleUpdateperformedattheendofthevolumesearch.Further details
regarding the Progressive Sampling Method and High-Resolution Position and Angle Update
40
40
used in our work may be found in section 1.2.2 (under the heading Optimizing Efficiency of
Volume Search).
When an updated ICE image was received, the algorithm would repeat the above process using
the position and orientation determined by the previous iteration as the starting point. The GUI in
MATLAB displayed the updated plane position and orientation in 3D space in the context of
axial and sagittal CT planes, as shown in Figure 2.1.
2.3 Experimental Setup A custom silicon cardiac phantom was used for the purpose of our study. CT images were
acquired using a Toshiba Acquilion One™ CT scanner (Toshiba Inc, Tokyo, Japan). ICE images
of the phantom were acquired using a Conavi Foresight™ ICE imaging catheter (Conavi Medical
Inc, Toronto, Canada). The silicon phantom was placed in a custom water bath container, which
contained a hemostatic valve built into its side to allow passage of the ICE catheter without
leaking the water content. The ICE catheter was placed in an initial position, and both ICE
images and CT volumes were acquired at this location, and 9 additional manual pullback
positions. The pullback positions were each not a standardized length, however did not exceed
the maximum distance (16 mm) searched by the algorithm during each iteration. CT scans were
performed with the ICE catheters at each pullback location. The CT volumes were used to
confirm the position and orientation of the ICE catheter tip at each pullback location. A final CT
volume was acquired with the ICE catheter removed from the phantom as the CT data set for the
automated localization portion of our study.
CT images were exported in DICOM format, including standard DICOM headers. ICE images
were exported to PGM format via third-party image-processing software from Conavi Medical
Inc. Header information including appropriate scaling and the conical angle of the ICE surface
were exported from the third-party software. Image data were extracted from the respective
DICOM and PGM formats, and corrective scaling of the images were applied in MATLAB.
Initially, the first of ten ICE images and the CT volume without the ICE catheter were loaded in
our custom software. The experiment began with a manual user-defined best estimate of the ICE
41
41
catheter position and orientation. Following this first step, the algorithm updated the initial
position based on the CT volume search described in section 2.2. The true position and
orientation of the ICE imaging catheter tip was known based on the CT scan acquired with the
ICE catheter in its initial position. Localization error for position was defined as the three-
dimensional distance between the true tip position and the algorithms’ estimated position.
Localization error for orientation was defined as the three-dimensional difference in pitch and
yaw between the known and estimated orientations. In-plane rotation, or roll, was not factored
into the error calculation.
The final estimated catheter position for each pullback step served as the starting point for the
next search. A total of ten catheter tip position and orientation estimates were performed in
succession by the algorithm. It is possible that the position of the next pullback location falls
outside the search radius if the error introduced by the preceding step in addition to the distance
in the current step is larger than the search radius. Using our software, it is possible for the user
to perform a manual correction if the estimated position strays too far from the true catheter tip
location. We did not employ this strategy for the purpose of our experiment.
2.4 Results A detailed description of the steps performed in the algorithm are outlined in section 2.2. In
brief, the algorithm performs a series of 2D-2D co-registration steps with a static ICE image and
conical CT surfaces at various positions and orientations within the CT volume. Each 2D-2D co-
registration produces a joint functional, or J-value, which is used to construct a 5-dimensional
cost function. Given the J-value represents the quality of overlap of the images in the 2D-2D co-
registration, the position and orientation of the lowest J-value represents the most likely position
and orientation of the ICE image within the CT volume. We present here the results of the 2D-
2D co-registration process and the search method within the CT volume, which comprises of
numerous 2D-2D co-registrations.
42
42
2.4.1 2D-2D Co-Registration 2D-2D ICE-CT co-registrations were performed in our work using: 1) multilevel representation,
2) normalized gradient fields (NGF) distance metric, 3) elastic regularizers, and 4) spline-based
transformation. The results of a sample 2D-2D co-registration are shown in Figure 2.2. In this
illustration, the red outline in Figure 2.2 C-E represents the contour of the simulated RV contour
(7 o’clock to 2 o’clock in Figure 2.2 B) during the co-registration process in context of the CT
data. Figure 2.2 C-E demonstrate multilevel image representations of the ICE (red) and CT data
(background). Initial registration is performed by aligning coarse features of the transformed ICE
image and CT (C). Finer features are aligned progressively as further details are added to the
images (D,E).
Figure 2.2: An example of 2D-2D Co-Registration: ICE (A) and CT (B) images of a silicon
cardiac phantom (black) in a holder (white) are used for 2D-2D co-registration testing.
Multilevel representations of a portion of the transformed ICE image (C-F, red) and the entire
43
43
CT image (C-E, background) are shown. The algorithm starts by aligning coarse features (C),
and progressively aligns finer features as more details are added (D,E).
2.4.2 Search Method within CT Volume The experimental setup for our work is outlined in Section 2.3. Localization error for position
was defined as the three-dimensional distance between the true tip position determined by CT
and the position estimated by the algorithm (Figure 2.4). Localization error for orientation was
defined as the three-dimensional difference in pitch and yaw between the known and estimated
orientations. In-plane rotation, or roll, was not factored into the error calculation. The average
localization error for position and orientation over the ten pullback positions used in our
experiment were 4.1 mm and 8.1 degrees respectively. Figure 2.3 illustrates the estimated
position and orientation of the ICE imaging catheter traveling through a cardiac phantom from
the apex to the base of the right side of the heart (top row) compared to the actual trajectory
verified by CT scans (bottom row). The relatively high average error for the orientation estimates
was heavily influenced by the rotational symmetry of the cardiac phantom about the x-axis (or
pitch; refer to Discussion section for details).
Figure 2.3: Example of the algorithm following a live ICE imaging plane through 3D CT
volume. Top row represents the estimated trajectory of the ICE catheter based on the volume
search method. Bottom row represents the actual trajectory of the catheter based on identified
44
44
position and orientation on CT scans. In both rows, the CT representation of initial position of
the 2D ICE image in the silicone phantom shown on right. Ten subsequent CT representations of
ICE imaging surface position and trajectory (red line) shown in middle. Final position and
trajectory are shown on right side.
Localization error as described above was calculated for each of the ten steps in the experiment.
Each step may be viewed independently as a process that begins at the ‘estimated prior location’
and attempts to locate the ‘actual current location’ (Figure 2.4). Thus, the ‘distance between the
prior estimated location and current actual location’ must fall within the ‘maximum search
distance’ (in our case 16 mm and 8 degrees) in order for the algorithm to be able to identify the
correct current position and orientation.
Figure 2.4: Graphical representation of locations and distances during each pullback step.
Estimated prior location serves as a starting point for the search. The distance between the prior
estimated location and the actual current location represents the distance from the prior step to
the current true position. The distance between prior and current actual locations is the distance
travelled by the catheter during each pullback. Localization error is defined as the distance
between the estimated and actual locations. The maximum search distance represents the
maximum distance the algorithm searches from its starting point. The total distance searched in
each dimension is twice this value.
Figure 2.5 and 2.6 demonstrate the localization error for position and orientation as a function of
the pullback step number in millimeters and degrees respectively (blue lines). The distances
between the starting position and orientation for each pullback step and the actual position and
45
45
orientation for each pullback in millimeters and degrees respectively are shown as green lines.
The distance between actual location of prior iteration and actual location of current pullback
step in millimeters is shown as a purple line in Figure 2.5. Given the angle was fixed throughout
the experiment, this measure was not shown in Figure 2.6. Maximum search distances in
millimeters and degrees are outlined as red dotted lines.
Figure 2.5: Distance measured (in millimeters) between points and associated error
measurements. Distance between the actual location of catheter tip during prior pullback step and
the actual location during current step (purple) represents the pullback distance. Distance
between the estimated location of catheter tip during prior pullback step and the actual location
during current step (green). Localization error: distance between actual location of catheter tip
and estimated location in the current step (blue). Maximum search distance is outlined as a red
dotted line. The extent of the search represents twice this value (-16 mm to +16 mm).
46
46
Figure 2.6: Distance measured (in degrees) and associated error measurements. Distance
between the estimated catheter tip orientation of previous pullback step and the current step true
orientation. Orientation error: distance (in degrees) between actual orientation of catheter tip and
estimated orientation of current iteration (blue). Maximum search distance (in degrees) is
outlined as a red dotted line. The extent of the search represents twice this value (-8 degrees to
+8 degrees).
A majority of the estimated prior position estimates remained within the maximum search
distance of the actual current location. During iteration 9 in the distance measurements and
iterations 8-10 of the angle measurements, the distance between the estimated prior and current
actual locations fell outside the maximum search distance. In these circumstances, it is not
possible for the algorithm to identify the current actual location given it is out of bounds of the
maximum search distance. The distance measure recovered by locating a different minimum in
the cost function that lay within the search radius and thus ‘recovered’; however the angle
measurements would likely continue to track in a false local minimum in further iterations in this
scenario. The angle estimates likely continued to follow false minima given the rotational
symmetry of the cardiac phantom about the x-axis (please refer to next section for details).
47
47
2.5 Discussion In our work, we present a software system for tracking the position and orientation of a 2D ICE
imaging surface within a previously acquired 3D CT volume. The algorithm begins with an
initial user-defined step, followed by a limited local search (32 mm translation and 16 degrees
rotation total) during each iteration of the process. Testing was performed using a custom-built
silicone cardiac phantom, and images acquired using a Toshiba CT scanner and 10 Fr Conavi
Foresight™ ICE imaging catheter. ICE imaging surface was tracked through the simulated right
ventricle of the 3D CT volume in ten successive pullback steps with an average localization error
for position of 4.1 mm. The localization error was measured by comparing the locations of the
ICE surface estimated by the algorithm and scanned location of the ICE catheter tip on CT. A
version of the CT of the cardiac phantom acquired with the ICE catheter removed was used to
test the performance of the algorithm.
The average localization error with respect to orientation for the ten pullback positions was 8.1
degrees, which was influenced by the rotational symmetry of the cardiac phantom. Rotations
about the x-axis particularly yielded similar RV cavity outlines, thus multiple false minima, as
demonstrated in Figure 2.7. Given the NGF distance measure assesses distance based on
geometric boundaries, the similar RV outlines caused the algorithm to erroneously “rotate” the
estimated orientation about the x-axis. This concept is well-illustrated by following the rotational
trajectory of the estimated surface during pullback in the top row of Figure 2.3. In more realistic
phantom or in-vivo anatomy, the rotation errors seen in this experiment will be much less likely
to occur due to the lack of rotational symmetry (due to irregular surfaces, and both near-field and
far-field additional structures, such as chordae or papillary muscles). Further work is required to
assess orientation estimate accuracy in models with less rotational symmetry.
48
48
Figure 2.7: Demonstration of rotational symmetry about the x-axis. In images to the far left and
right, the x, y, z position remained constant, however the rotation about the x-axis (or ‘pitch’)
varied between +10 degrees and -10 degrees. The 2D CT representations of the conical surfaces
extracted at each angle (20 degrees apart) are shown in the centre column. The centre column
demonstrates how geometric outlines at these varied angles would be difficult to discern without
any additional anatomic features, such as those found in detailed phantoms or in-vivo anatomy.
A key to maintaining processing efficiency is limiting the scope of the searches performed
during each iteration. Our experiment demonstrates that the algorithm is able to track the
location of the ICE imaging surface within a reasonable error range, however if the estimated
prior location (or orientation) of the search falls outside the maximum search distance of the
actual current location due to tracking a false local minimum, the algorithm may or may not
‘recover’. One possible solution to this dilemma is to increase the maximum search radius;
however, at a significant cost of processing efficiency. Strategies to potentially address this issue
will be discussed in Chapter 3. Further research is required to determine strategies to increase the
search area while maintaining processing efficiency.
For the purpose of our study, the search algorithm was developed in the MATLAB programming
environment, which provides a robust set of software development tools, however at a significant
cost of processing efficiency. The algorithm requires approximately 175,000 independent steps
(each requiring approximately 1 second), which are processed linearly in its current form,
totalling approximately 48 hours per iteration. Although this timeframe is unacceptable for its
49
49
intended use in structural heart disease, the algorithm will operate more efficiently using CPU
multithreading and GPU parallel processing techniques. For example, Xu, et al from our research
group developed a similar system for co-registering real-time with static MR images that was
accelerated by 139-fold using a graphics card with a modest only 192 CUDA cores. It is
estimated that using a contemporary high-end graphics workstation, such as one using four
NVIDIA Tesla V100 with a combined 20,480 CUDA cores could accelerate the process by over
14,800-fold. In this scenario, the rate of position updates would accelerate from the current rate
of 2 days per update to 12 seconds per update. Such a system would provide an acceptable
refresh speed for the purpose of tracking a live imaging surface during TAVI.
Using our system, an operator may perform ICE imaging during structural heart cases, such as
TAVI, with a better understanding of the three-dimensional anatomic context surrounding the
imaging catheter. For many early-experience ICE users, understanding the position and
orientation of the catheter tip represents a significant barrier to adoption. With further
experience, ICE allows the operator to visualize the valve prosthesis and surrounding soft tissue
simultaneously, as well as native valve deformation and Doppler information in real-time.
Operators will be better equipped to assess prosthesis positioning with respect to the native
valve, risk of coronary occlusion, and development of paravalvular leak, all in real-time and
without the use of additional nephrotoxic contrast injections. The localization error of 4.1 mm for
position represents an acceptable range for the purpose of guiding structural heart procedures.
For example, the most position-sensitive measure performed during TAVI relates to the risk of
coronary occlusion following valve deployment. In this case, the coronary origins usually
measure between 4-6 mm, and the borderline valve-to-coronary height is 10 mm. An estimate
within 4.1 mm will allow the operator to position their imaging surface within the vicinity of this
sensitive measure with subsequent fine adjustments as necessary. Using ICE imaging, the
operator is thus able to monitor the coronary origin in real-time as the prosthesis is deployed, and
the native leaflets are deformed in the direction of the ostium.
2.6 Conclusion The ICE-CT co-registration process described in this paper represents an accurate method for
determining the location of a 2D ICE imaging surface within a 3D CT volume. The software is
50
50
able to display the 2D ICE surface’s location in context of the prior CT volume. The algorithm
will potentially help structural heart operators appreciate the three-dimensional anatomic context
of the ICE imaging surface. Further work is required to improve processing efficiency and
increase the search area, including the use of multithreading and parallel processing techniques.
51
51
Chapter 3
Summary and Future Directions
In our work, we present a system for localization of an ICE imaging surface within a three-
dimensional CT volume. The localization algorithm begins with a user-defined step, which
provides the starting point for the position and orientation search. Subsequent steps involve
performing a series of image 2D-2D image co-registrations within a limited range of positions
and orientations (typically 32 mm and 16 degrees), and using the resulting ‘J-value’ (co-
registration metric determining the quality of the overlap) to determine the best estimate of the
position and orientation of the ICE image surface within the CT volume. The individual two-
dimensional co-registration steps are performed in MATLAB using the Flexible Algorithm for
Image Registration (FAIR) package. The algorithm uses three levels of multilevel image
representation as pre-processing steps. In this form of pre-processing, pixel values are averaged
in progressive steps such that coarse features of the images are aligned first, prior to alignment of
fine image features. The distance measure used in the algorithm to compare ICE and CT is
normalized gradient fields (NGF). NGF compares the locations of magnitude changes (ignoring
the absolute magnitude of the change), such as those found at major anatomic boundaries, which
are well preserved in both ICE and CT. Elastic regularization helps prevent tracking within false
minima by applying penalties to large transformations in a manner that simulates elastic potential
energy. Spline-based transformations are applied to the template during each iteration of the
algorithm, and compared to the reference image using NGF as a distance metric and an elastic
regularizer. Figure 2.2 in Chapter 2 shows a graphical representation of the 2D-2D portion of the
algorithm. The search for the ICE image surface within the CT volume is performed with 5
degrees of freedom (x/y/z translation, and pitch/yaw). In-plane rotation, the 6th degree of
freedom, is accounted for in the 2D-2D registration process. A sampling shortcut, known in our
work as progressive sampling, is performed to maintain processing efficiency of the 3D search.
Progressive sampling involves under-sampling the data set, and subsequently performing more
detailed sampling of data falling below as certain threshold value. Ultimately, the algorithm is
capable of tracking the location of an ICE imaging catheter within a silicone cardiac phantom
over ten iterations with an average localization error of 4.1 mm.
52
52
The average localization error for orientation was 8.1 degrees, which was similar to the extent of
the orientation search. Figure 2.6 demonstrates the error in orientation estimates related to
pullback position. The relatively high error value was heavily influenced by the rotational
symmetry of the cardiac phantom. Figure 2.3 demonstrates the estimated path of the ICE
imaging surface based on the algorithm (top row), compared to the actual trajectory confirmed
by CT images of the ICE catheter during pullback (bottom row). The predominant rotational
change throughout the estimated trajectory is rotation around the x-axis (or ‘pitch’). Figure 2.7
demonstrates how such rotational error can occur in the cardiac phantom used in this experiment,
which would greatly improve in the in-vivo circumstance given the extra anatomic details
present in both the near-field and far-field (such as irregular chamber boundaries, chordae,
papillary muscles, etc).
A future phantom experimental design to more accurately test the orientation error would
involve a phantom model with more realistic asymmetric anatomic features, such as right
ventricular papillary muscles, trabeculae, and a moderator band. A similar experiment with ten
pullback positions may be performed, and compared to the results of the experiment performed
in our work to assess the role of anatomic symmetry in orientation accuracy. An in-vivo
experimental design, such as a porcine animal study, with CT scans performed during each
pullback iteration for the purposes of position and orientation error assessment would also help
determine the role of irregular anatomic features in ICE surface localization accuracy.
In-Vivo Preclinical Studies
The next step in experimental testing will involve a preclinical in-vivo animal test, such as a
porcine model, to test the difference in performance of the algorithm between a simple cardiac
phantom and in-vivo anatomy similar to a human. A catheter pullback study may be performed
within a CT scanner, similar to the phantom study performed in our work, and would help clarify
whether the algorithm performs localization estimates more accurately in-vivo due to the
additional anatomic details present, or whether it is more prone to tracking into false minima.
The Conavi Foresight™ ICE catheter image software displays the ECG signal acquired during
imaging, and therefore allows the operator to time comparisons of ICE and CT images at
identical portions of the cardiac cycle. By timing the image co-registration to occur during early
or mid-diastole, it will reduce the potential error in co-registration due to cardiac motion. It will
53
53
also be necessary to obtain CT scans at each pullback location to confirm the true position and
orientation of the ICE catheter in order to determine the error values between the estimate and
true positions and orientations. Figure 3.1 demonstrates roughly co-aligned ICE and CT frames
from a porcine study performed in 2017 as an illustration of how these in-vivo images compare.
The data acquired during the study shown in the figure is not optimized for the purposes of using
our algorithm due to the irregular contrast opacification of the right heart chambers, and
inadequate opacification of the left heart chambers. This example illustrates how an edge-
sensitive distance measure, such as NGF may be affected by improper CT contrast
administration. It remains to be seen how the algorithm performs with image artifacts present (in
both ICE and CT), such as metal artifacts caused by cardiac pacemaker leads.
Figure 3.1: Single frame representation of ICE (left) and CT (right) surfaces demonstrating ICE
catheter in right atrium during porcine study. CT image created by extracting shallow cone in
three dimensions from CT volume in roughly similar orientation to ICE surface. ICE imaging
probe tip seen in centre of ICE image. ICE and CT representations of right atrium, aortic root
and pulmonary artery shown above. Data acquired during the above study is not optimized for
the purposes of using our algorithm due to the irregular contrast opacification of the right heart
chambers, and inadequate opacification of the left heart chambers. Future animal study will
require attention to appropriate contrast deliver and CT gating techniques in order to perform
accurate localization.
54
54
Addressing the Limited Search Area
A key consideration in improving the algorithm towards a clinically useful product is addressing
the limited search area. In the current version, it is possible for the algorithm to track an ICE
surface within a false local minimum, and the algorithm is unable to recover since the position
and/or orientation fall outside the maximum search distance. By expanding the search area, this
scenario is less likely; however, by increasing the search area, additional false local minima may
be uncovered beyond the perimeter of the previous maximum search distance. There are also
significant computer processing costs in increasing search area. In order to limit computational
costs, one may consider reducing final localization resolution, increase the angle searched in
exchange for a reduction in distance searched (or vice-versa), or modify the sampling ratio at
each step of the under-sampling strategy. Further work is required to better understand the
interplay between search area, search resolution, sampling ratios, and accuracy of the algorithm.
A strategy to increase the search area while maintaining processing efficiency is to perform
segmentation of the CT volume prior to the search in order to identify “vascular spaces”. This
method will ensure that the ICE catheter is tracked within vessels or cardiac chambers where it is
possible to be located, and ignoring non-vascular spaces in order to improve efficiency.
Improving Processing Efficiency: Parallel Processing Techniques
The algorithm designed in our work was developed in MATLAB using the FAIR image co-
registration package developed by Modersitzki,etal. Although both MATLAB and the FAIR
package provide robust development environments, there is a significant cost of processing
speed and efficiency during program runtime. In its current iteration, the algorithm requires
approximately 2 days to complete approximately 175,000 independent calculations required for
each localization step, each averaging approximately 1 second to complete. The timeframe
outlined is thus not compatible with imaging support during structural heart interventions. The
use of parallel processing methods afforded by graphics processing units (GPUs) has the
potential to significantly accelerate processes such as ours that require numerous small,
independent calculations. For example, a study performed by Xu, et al, from our research lab
were able to accelerate respiratory motion correction during co-registration of real-time with
static MR by an average of 139-fold using a GPU with a modest only 192 CUDA cores. Using a
55
55
contemporary high-end graphics workstation, such as one containing four NVIDIA Tesla V100
GPUs with a combined 20,480 CUDA cores could potentially accelerate the process by over
14,800-fold. It is therefore conceivable that combining implementation in a more efficient
programming environment, such as C#, and running the independent processes in parallel over a
high-end graphics workstation could reduce the time between iterations from 2 days to 12
seconds. Limiting the co-registration iterations to vascular-only territories may reduce data
processing by up to 50% further, thus providing an update speed of only 6 seconds. A refresh
rate in this timeframe would ultimately be compatible with assisting imaging guidance for
structural heart intervention.
Improving Processing Efficiency: Predictive Algorithms
A further step to consider in accelerating the algorithm to real-time capabilities for guiding
structural heart interventions is employing the use of predictive algorithms (such as Kalman
filtering and artificial neural networks). Kalman filtering, or linear quadratic estimation (LQE), is
an approach to predicting data patterns based on a prior data sample collected over time. The
process is used to estimate output variables of a control system prone to stochastic disturbances
cause by noise in the input dataset.88 The algorithm works by combining the known information
regarding the system dynamics learned by the training data, with the probabilistic information
regarding the noise. Its strength lies in the ability to predict past, present and future data, even
when the causes of the noise of the input signal is poorly understood. Kalman filtering has been
used extensively in aerospace navigation, automated driving, and real-time image processing.
For our purposes, Kalman filtering would be particularly useful for predicting the motion of the
ICE catheter. For example, the estimated position predicted by our algorithm will track the
movement of ICE catheter is within an average error of 4.1 mm. The movement of the ICE
catheter, combined with the noise introduced by our 4.1 mm error range will likely produce
‘jumps’ in position estimates. By using the Kalman filtering technique, the algorithm will predict
the location of next movement based on physical principles (which may be further enhanced by
the addition of hardware sensors), and uses our algorithm to ‘steer’ the updated predicted
position. The resulting updated position is based on the covariance between the path predicted by
prior data (along with physical principles) and the estimates produced by our algorithm.
56
56
Artificial neural networks (ANNs) are a subset of machine learning which make use of a series
of interconnected nodes, each known as an “artificial neuron”, to make predictions regarding the
outcome of a system. The connections between artificial neurons and the neurons themselves
carry as specific weighting factor, which is modified as a specific training data set is introduced.
ANNs are adept at predicting outcomes based on training data sets, without necessarily
understanding the underlying mechanism causing the outcome. In this fashion, they are
particularly well-suited for applications where prior behaviour of the system somewhat predicts
future events. For our purposes, ANNs may be used to predict the motion of the ICE catheter
with fewer 3D searches performed, as well as dictate the regions in 3D space to perform more
aggressive 2D-2D co-registration sampling in the 3D search portion of the algorithm. In
combination with our progressive sampling method, ANNs may substantially reduce the number
of 2D-2D co-registrations required in a search of 3D space required to accurately locate the
position and orientation of the ICE catheter.
Addressing Introduction of New Equipment and Tissue Deformation
In future versions of the algorithm developed in our work, it will be necessary to consider the
effect of tissue deformation and introduction of new equipment in the live imaging space. Given
the algorithm attempts to track the location of the live imaging surface within the pre-procedure
CT volume, it stands to reason that the introduction of foreign equipment during the procedure
may result in incorrect localization. This issue may be addressed using one or both of the
following strategies: 1) combination with a magnetic tracking system, such as the system
described by Luo, et al,62 and 2) computed simulation of introduction of equipment in the
imaging field. In the first example, our algorithm may be employed to make small position and
orientation adjustments for a magnetic tracking system. The advantage of this approach is that
using the position information from the MTS system, only require a small local search would be
required to update both position and orientation. A combined system would likely draw on the
strengths of each tracking technique without a significant delay in tracking if using the parallel
processing techniques described above. In the second example, retaining a software-only
solution may still be possible if the algorithm can accurately predict the appearance of certain
medical instruments in the imaging field. Given the imaging views used during TAVI are
predominantly in short-axis, and the equipment used are narrow catheters, the space occupied by
57
57
the equipment in the imaging field will be predominantly small, however simulating the
instrument and acoustic shadowing cast will remain a significant challenge.
Working Towards 3D-3D Image Co-Registration
The Conavi Foresight™ ICE catheter is capable of performing 3D ICE imaging by modifying
the rotation speed of the ultrasound transducer located at the distal tip. The mechanical rotating
element is a hexagonal disc that revolves about the longitudinal axis and acquires images
perpendicular to the catheter at speeds of approximately 20 Hz. The disc tilts to a ‘forward-
looking’ position at rotation speeds of 70 Hz (Figure 3.2).
Figure 3.2: 10-French Conavi Foresight™ ICE catheter (shown left) with proximal connector,
and handle for deflection. Imaging tip (shown right) rotates about the longitudinal axis as speeds
of roughly 20 Hz. By increasing the rotational speed to up to 70 Hz, the ultrasound transducer
tilts to a ‘forward-looking’ position, and acquires images of anatomy ahead of the catheter. The
‘forward-looking’ ICE image surface is a conical surface with a much steeper angle and smaller
radius compares to the ‘side-looking’ surface. Reproduced with permission from Conavi Medical
Inc®.
The catheter is capable of performing ‘sweeps’ in three-dimensional space by modifying the
rotational speed of the distal tip of the catheter over time. In a single sweep, the catheter is
capable of combining images at progressive angles to form a 3D ICE volume. This may be
performed in approximately one second. Using maximum intensity projection techniques, a
58
58
surface rendering of the three-dimensional volume is shown to the user based on pixel intensities
(shown in Figure 3.3).
Figure 3.3: Example of Conavi Foresight™ 3D intra-procedural imaging. By modifying the
rotational speed of the distal tip of the ICE catheter, the ultrasound moves to a progressively
more ‘forward-looking’ position. A resulting ‘sweep’ by modifying the rotational speed over
time produces a 3D image data set. The above is a maximum intensity projection of the catheter
performing a 3D sweep from within the right atrium. Reproduced with permission from Conavi
Medical Inc®.
A potential future direction for localization of the ICE catheter within a CT volume will depend
on co-registering maximum intensity projections of both ICE and CT in three dimensions. In this
case, the 2D localization may serve as a starting point, however with the capabilities of aligning
anatomic details in three dimensions, the catheter orientation results would be theoretically much
more accurate. 3D/3D co-registration between multiple imaging modalities has been developed
in prior work, and proven to be highly accurate.89
Final Thoughts Regarding Clinical Utility
59
59
The use of ICE imaging represents an emerging practice in the field of structural heart disease
intervention, including transaortic valve implantation (TAVI), and has the potential to improve
procedural guidance, as well as monitoring for procedural complications. ICE is particularly
useful in imaging the valve prosthesis location during deployment, monitoring the location of the
displaced native aortic leaflets in context of the coronary origins, and monitoring for the degree
of paravalvular leak post-valve deployment in real-time. Structural heart disease operators
(cardiologist or cardiac surgeon) would potentially make use of the localization algorithm
described in our work to track the anatomic location of an ICE imaging surface used during
TAVI within a previously acquired 3D CT volume. By demonstrating the location of the ICE
surface in this context, the operator will gain an enhanced understanding of the three-
dimensional anatomic context of the ICE imaging surface. A specific example of clinical
application of ICE imaging during TAVI that would benefit from an appreciation of three-
dimensional anatomic context provided by our algorithm is during deployment of a TAVI valve
with borderline low valve-to-coronary heights, typically in the 8-10 mm range. During the pre-
procedure planning CT study, the TAVI operator assesses the height of the coronary origins from
the native valve and the native valve leaflet lengths. During valve implantation, the native valve
leaflets are propped permanently in the ‘open’ position, and thus with low coronary heights there
is risk of causing iatrogenic coronary occlusion. This situation is easily predicted if the leaflet
lengths are longer than the estimated coronary heights. In this circumstance, the patient will
benefit from intentional leaflet laceration using a ‘Basilica’ procedure, however it is quite
technically difficult and introduces several new risks beyond TAVI itself. There are many
circumstances, however, where the leaflet lengths are ‘borderline’ to produce coronary occlusion
(ie. leaflet length is within 1-2 mm of coronary height), and require diligent intra-procedural
monitoring. In these circumstances, the operator may use ICE to monitor for coronary occlusion
during valve deployment. If the operator observes occlusion, the coronary origin may be stented
(which effectively displaces the valve away from the coronary origin) using a coronary wire left
in place prior to valve deployment as a precaution. In monitoring for this complication, the
operator will require a clear understanding of where the coronary origins lie with respect to the
current ICE imaging surface, and thus, a system that can provide this information within 4 mm
(which is roughly the diameter of the coronary origin) would be highly desirable. Additionally,
this information is available to the operator with minimal disruption to current TAVI workflow.
60
60
Final Thoughts
The potential combination of the use of parallel processing techniques, predictive algorithms,
machine learning, and computed simulation of procedural equipment will likely provide the
performance capabilities necessary to introduce the use of our algorithm into the hands of
structural heart operators for clinical application. Further pre-clinical and clinical experiments
will be required to confirm whether the addition of the localization algorithm described will
enhance the operators’ understanding the three-dimensional context of the current imaging plane.
In conclusion, if the algorithm performs as this author predicts in the real-world clinical setting,
it has the potential to improve procedural guidance during TAVI by introducing early
recognition of common serious complications, such as iatrogenic coronary occlusion, or
persisting paravalvular leak.
61
61
References 1. Asgar, A. W. et al. Structural Heart Disease Intervention: The Canadian Landscape. Can.
J. Cardiol. 33, 1197–1200 (2017).
2. Franke, J., Steinberg, D. H. & Sievert, H. Interventional treatment of structural heart disease. Minim. Invasive Ther. Allied Technol. 18, 110–121 (2009).
3. Steinberg, D. H., Staubach, S., Franke, J. & Sievert, H. Defining structural heart disease in the adult patient: current scope, inherent challenges and future directions. Eur. Hear. J. Suppl. 12, E2–E9 (2010).
4. Higgs, Z. C. J., Macafee, D. A. L., Braithwaite, B. D. & Maxwell-Armstrong, C. A. The Seldinger technique: 50 Years on. Lancet 366, 1407–1409 (2005).
5. Noble, S. & Roffi, M. Overcoming the Challenges of the Transfemoral Approach in Transcatheter Aortic Valve Implantation. Interv. Cardiol. Rev. 8, 131–134 (2013).
6. Bleakley, C. & Monaghan, M. J. The Pivotal Role of Imaging in TAVR Procedures. Curr. Cardiol. Rep. 20, (2018).
7. Carminati, M. et al. Role of imaging in interventions on structural heart disease. Expert Rev. Cardiovasc. Ther. 11, 1659–1676 (2013).
8. Cavalcante, J. L. & Schoenhagen, P. Role of cross-sectional imaging for structural heart disease interventions. Cardiol. Clin. 31, 467–478 (2013).
9. Cavalcante, J. L., Rodriguez, L. L., Kapadia, S., Tuzcu, E. M. & Stewart, W. J. Role of echocardiography in percutaneous mitral valve interventions. JACC. Cardiovasc. Imaging 5, 733–746 (2012).
10. Katz, W. E., Conrad Smith, A. J., Croc, F. W. & Cavalcante, J. L. Echocardiographic evaluation and guidance for MitraClip procedure. Cardiovasc. Diagn. Ther. 7, 616–632 (2017).
11. Lee, A. P. W. et al. Role of real time three-dimensional transesophageal echocardiography in guidance of interventional procedures in cardiology. Heart 96, 1485–1493 (2010).
12. Sengupta, P. P. & Khandheria, B. K. Transoesophageal echocardiography. Heart 91, 541–547 (2005).
13. Wunderlich, N. C., Kux, H., Kreidel, F., Birkemeyer, R. & Siegel, R. J. The Changing Paradigm in the Treatment of Structural Heart Disease and the Need for the Interventional Imaging Specialist. Interv. Cardiol. Rev. 11, 135–139 (2016).
14. Litmanovich, D. E. et al. Imaging in Transcatheter Aortic Valve Replacement (TAVR): role of the radiologist. Insights Imaging 5, 123–145 (2014).
15. Binder, R. K. et al. Prediction of Optimal Deployment Projection for Transcatheter Aortic
62
62
Valve Replacement. Circ. Cardiovasc. Interv. 5, 247–252 (2012).
16. Blanke, P., Schoepf, U. J. & Leipsic, J. A. CT in Transcatheter Aortic Valve Replacement. Radiology 269, 650–669 (2013).
17. Auffret, V. et al. Temporal Trends in Transcatheter Aortic Valve Replacement in France: FRANCE 2 to FRANCE TAVI. J. Am. Coll. Cardiol. 70, 42–55 (2017).
18. Akinseye, O. A., Jha, S. K. & Ibebuogu, U. N. Clinical outcomes of coronary occlusion following transcatheter aortic valve replacement: A systematic review. Cardiovasc. Revascularization Med. 19, 229–236 (2018).
19. Barbanti, M. Avoiding Coronary Occlusion and Root Rupture in TAVI – The Role of Pre-procedural Imaging and Prosthesis Selection. Interv. Cardiol. Rev. 10, 94–97 (2015).
20. Sultan, I., Siki, M., Wallen, T., Szeto, W. & Vallabhajosyula, P. Management of coronary obstruction following transcatheter aortic valve replacement. J. Card. Surg. 32, 777–781 (2017).
21. Carità, P. et al. Aortic stenosis : insights on pathogenesis and clinical implications. J. Geriatr. Cardiol. 13, 489–498 (2016).
22. Spaccarotella, C., Mongiardo, A. & Indolfi, C. Pathophysiology of Aortic Stenosis and Approach to Treatment With Percutaneous Valve Implantation. Circ. J. 75, 11–19 (2010).
23. Sverdlov, A. L. et al. Pathogenesis of aortic stenosis: not just a matter of wear and tear. Am J Cardiovasc Dis 1, 185–199 (2011).
24. Nkomo, V. T. et al. Burden of valvular heart diseases: a population-based study. Lancet 368, 1005–1011 (2006).
25. Mordi, I. & Tzemos, N. Bicuspid Aortic Valve Disease : A Comprehensive Review. Cardiol. Res. Pract. 2012, 1–7 (2012).
26. Roberts, W. C. & Ko, J. M. Frequency by decades of unicuspid, bicuspid, and tricuspid aortic valves in adults having isolated aortic valve replacement for aortic stenosis, with or without associated aortic regurgitation. Circulation 111, 920–925 (2005).
27. Horstkotte, D. & Loogen, F. The natural history of aortic valve stenosis. Eur. Heart J. 9 Suppl E, 57–64 (1988).
28. Iung, B. & Vahanian, A. Degenerative calcific aortic stenosis: a natural history. Heart 98, iv7-iv13 (2012).
29. Teppo, L. et al. Cancer patient survival--patterns, comparisons, trends--a population-based Cancer Registry study in Finland. Acta Oncol. (Madr). 38, 283–294 (1999).
30. Otto, C. M. Calcific aortic stenosis--time to look more closely at the valve. The New England journal of medicine 359, 1395–1398 (2008).
63
63
31. Grimard, B. H., Safford, R. E. & Burns, E. L. Aortic Stenosis: Diagnosis and Treatment. Am. Fam. Physician 93, 371–378 (2016).
32. Rijsterborgh, H. & Roelandt, J. Doppler assessment of aortic stenosis: Bernoulli revisited. Ultrasound Med. Biol. 13, 241–248 (1987).
33. Nishimura, R. A. et al. 2014 AHA / ACC Guideline for the Management of Patients With Valvular Heart Disease. J. Am. Coll. Cardiol. 63, e57-185 (2014).
34. Taniguchi, T. et al. Sudden Death in Patients With Severe Aortic Stenosis : Observations from the CURRENT AS Registry. J. Am. Heart Assoc. 7, (2018).
35. Edwards, W. S. & Smith, L. Aortic valve replacement with a subcoronary ball valve. Surg. Forum 9, 309–313 (1958).
36. Chikwe, J., Filsoufi, F. & Carpentier, A. F. Prosthetic valve selection for middle-aged patients with aortic stenosis. Nat. Rev. Cardiol. 7, 711–719 (2010).
37. Rodriguez-Gabella, T., Puri, R., Voisine, P., Rodés-Cabau, J. & Pibarot, P. Aortic Bioprosthetic Valve Durability: Incidence, Mechanisms, Predictors, and Management of Surgical and Transcatheter Valve Degeneration. J. Am. Coll. Cardiol. 70, 1013–1028 (2017).
38. Auensen, A. et al. Morbidity outcomes after surgical aortic valve replacement. Open Hear. 4, 1–11 (2017).
39. Johnston, L. E. et al. Outcomes for Low-Risk Surgical Aortic Valve Replacement: A Benchmark For Aortic Valve Technology. Ann. Thorac. Surg. 104, 1282–1288 (2018).
40. Thourani, V. H. et al. Outcomes of surgical aortic valve replacement in high-risk patients: A multiinstitutional study. Ann. Thorac. Surg. 91, 49–56 (2011).
41. Vassileva, C. M. et al. Evaluation of the society of thoracic surgeons online risk calculator for assessment of risk in patients presenting for aortic valve replacement after prior coronary artery bypass graft: An analysis using the STS Adult Cardiac Surgery Database. Ann. Thorac. Surg. 100, 2109–2116 (2015).
42. Leon, M. B. et al. Transcatheter or Surgical Aortic-Valve Replacement in Intermediate-Risk Patients. N. Engl. J. Med. 374, 1609–1620 (2016).
43. Smith, C. R. et al. Transcatheter versus Surgical Aortic-Valve Replacement in High-Risk Patients. N. Engl. J. Med. 364, 2187–2198 (2011).
44. Rougé, A. et al. Mid-term results of 150 TAVI comparing apical versus femoral approaches. J. Cardiothorac. Surg. 10, 1–10 (2015).
45. Leon, M. B. et al. Transcatheter Aortic-Valve Implantation for Aortic Stenosis in Patients Who Cannot Undergo Surgery. N. Engl. J. Med. 363, 1597–1607 (2010).
64
64
46. Mack, M. J. et al. Transcatheter Aortic-Valve Replacement with a Balloon-Expandable Valve in Low-Risk Patients. N. Engl. J. Med. (2019). doi:10.1056/NEJMoa1814052
47. Adams, D. H. et al. Transcatheter Aortic-Valve Replacement with a Self-Expanding Prosthesis. N. Engl. J. Med. 370, 1790–1798 (2014).
48. Reardon, M. J. et al. Surgical or Transcatheter Aortic-Valve Replacement in Intermediate-Risk Patients. N. Engl. J. Med. 376, 1321–1331 (2017).
49. Feldman, T. E. et al. Effect of Mechanically Expanded vs Self-Expanding Transcatheter Aortic Valve Replacement on Mortality and Major Adverse Clinical Events in High-Risk Patients With Aortic Stenosis: The REPRISE III Randomized Clinical Trial. J. Am. Med. Assoc. 319, 27–37 (2018).
50. Popma, J. J. et al. Transcatheter Aortic-Valve Replacement with a Self-Expanding Valve in Low-Risk Patients. N. Engl. J. Med. (2019). doi:10.1056/NEJMoa1816885
51. Kempfert, J., Linden, A. Van, Lehmkuhl, L., Rastan, A. J. & Holzhey, D. Aortic annulus sizing : echocardiographic versus computed tomography derived measurements in comparison with direct surgical sizing. Eur. J. Cardio-Thoracic Surg. 42, 627–633 (2012).
52. Feuchtner, G. et al. Prediction of Paravalvular Regurgitation After Transcatheter Aortic Valve Implantation by Computed Tomography : Value of Aortic Valve and Annular Calcification. Ann. Thorac. Surg. 96, 1574–1580 (2013).
53. Généreux, P. et al. Paravalvular leak after transcatheter aortic valve replacement: The new achilles’ heel? A comprehensive review of the literature. J. Am. Coll. Cardiol. 61, 1125–1136 (2013).
54. Barbanti, M. et al. Anatomical and procedural features associated with aortic root rupture during balloon-expandable transcatheter aortic valve replacement. Circulation 128, 244–253 (2013).
55. Kasel, A. M., Cassese, S., Leber, A. W., von Scheidt, W. & Kastrati, A. Fluoroscopy-Guided Aortic Root Imaging for TAVR. JACC Cardiovasc. Imaging 6, 274–275 (2013).
56. Petronio, A. S. et al. Optimal Implantation Depth and Adherence to Guidelines on Permanent Pacing to Improve the Results of Transcatheter Aortic Valve Replacement With the Medtronic CoreValve System. JACC Cardiovasc. Interv. 8, 837–846 (2015).
57. Guerios, E. E. et al. Second valve implantation for the treatment of a malpositioned transcatheter aortic valve. J. Invasive Cardiol. 24, 457–462 (2012).
58. Hayashida, K. et al. Transcatheter aortic valve implantation for patients with severe bicuspid aortic valve stenosis. Circ. Cardiovasc. Interv. 6, 284–291 (2013).
59. Bartel, T., Edris, A., Velik-salchner, C. & Mu, S. Intracardiac echocardiography for guidance of transcatheter aortic valve implantation under monitored sedation : a solution to a dilemma ? Eur. Heart J. 17, 1–8 (2016).
65
65
60. Vitulano, N. & Narducci, M. L. Technology update: intracardiac echocardiography – a review of the literature. Med. Devices Evid. Res. 8, 231–239 (2015).
61. Pibarot, P. et al. Incidence and Sequelae of Prosthesis-Patient Mismatch in Transcatheter Versus Surgical Valve Replacement in High-Risk Patients With Severe Aortic Stenosis. J. Am. Coll. Cardiol. 64, 1323–1334 (2014).
62. Luo, Z. et al. Magnetic navigation for thoracic aortic stent-graft deployment using ultrasound image guidance. IEEE Trans. Biomed. Eng. 60, 862–871 (2013).
63. Franz, A. M. et al. Electromagnetic tracking in medicine--a review of technology, validation, and applications. IEEE Trans. Med. Imaging 33, 1702–1725 (2014).
64. Peters, T. M. & Linte, C. A. Image-guided interventions and computer-integrated therapy: Quo vadis? Med. Image Anal. 33, 56–63 (2016).
65. Hu, Y. et al. MR to ultrasound registration for image-guided prostate interventions. Med. Image Anal. 16, 687–703 (2012).
66. Fitton, I. S. F. et al. Impact of Anatomical Location on Value of CT-PET Co-Registration for Delineation of Lung Tumors. Int. J. Radiat. Oncol. 70, 1403–1407 (2008).
67. Modersitzki, J. Numerical Methods for image registration. Oxford Univ. Press. New York (2004).
68. Kanekoa, S., Yutaka, S. & Satoru, I. Using selective correlation coefficient for robust image registration. Pattern Recognit. 19, 1165–1173 (2003).
69. Maes, F., Collignon, A., Vandermeulen, D., Marchal, G. & Suetens, P. Multimodality image registration by maximization of mutual information. IEEE Trans. Med. Imaging 16, 187–198 (1997).
70. Pluim, J. P. W., Maintz, J. B. A. & Viergever, M. A. Mutual-information-based registration of medical images: a survey. IEEE Trans. Med. Imaging 22, 986–1004 (2003).
71. Modersitzki, J. FAIR: Flexible Algorithms for Image Registration. Soc. Ind. Appl. Math. 95–115 (2009).
72. Hodneland, E., Lundervold, A., Rorvik, J. & Munthe-Kaas, A. Z. Normalized gradient fields for nonlinear motion correction of DCE-MRI time series. Comput. Med. Imaging Graph. 38, 202–210 (2014).
73. Haber, E. & Modersitzki, J. Intensity gradient based registration and fusion of multi-modal images. Dep. Math. Comput. Sci. Emory Univ. (2004).
74. Reali, F., Priami, C. & Marchetti, L. Optimization Algorithms for Computational Systems Biology. Front. Appl. Math. Stat. 3, 1–12 (2017).
75. Fischer, B. & Modersitzki, J. Ill-posed medicine — an introduction to image registration
66
66
Ill-posed medicine — an introduction to image. Inverse Probl. 24, 1–16 (2008).
76. Broit, C. Optimal registration of deformed images. Ph.D. thesis, Comput. Inf. Sci. Univ. Pennsylvania, USA 121, 167 (1981).
77. Holden, M. A review of geometric transformations for nonrigid body registration. IEEE Trans. Med. Imaging 27, 111–128 (2008).
78. Huang, X. et al. Dynamic 3D ultrasound and MR image registration of the beating heart. Med. Image Comput. Comput. Assist. Interv. 8, 171–178 (2005).
79. Huang, X. et al. Dynamic 2D ultrasound and 3D CT image registration of the beating heart. IEEE Trans. Med. Imaging 28, 1179–1189 (2009).
80. Lang, P. et al. Surface-Based CT-TEE Registration of the Aortic Root. IEEE Trans. Biomed. Eng. 60, 3382–3390 (2013).
81. Li, F. P., Rajchl, M., White, J. A., Goela, A. & Peters, T. M. Ultrasound guidance for beating heart mitral valve repair augmented by synthetic dynamic CT. IEEE Trans. Med. Imaging 34, 2025–2035 (2015).
82. Li, F. P., Rajchl, M., Moore, J. & Peters, T. M. A mitral annulus tracking approach for navigation of off-pump beating heart mitral valve repair. Med. Phys. 42, 456–468 (2015).
83. Linte, C. A., Moore, J., Wedlake, C. & Peters, T. M. Evaluation of model-enhanced ultrasound-assisted interventional guidance in a cardiac phantom. IEEE Trans. Biomed. Eng. 57, 2209–2218 (2010).
84. Linte, C. A. et al. Accuracy considerations in image-guided cardiac interventions : experience and lessons learned. Int. J. Comput. Assist. Radiol. Surg. 7, 13–25 (2012).
85. Mcleod, A. J. et al. Phantom study of an ultrasound guidance system for transcatheter aortic valve implantation. Comput. Med. Imaging Graph. 50, 24–30 (2016).
86. Zhang, Q., Eagleson, R. & Peters, T. M. GPU-based visualization and synchronization of 4-D cardiac MR and ultrasound images. IEEE Trans. Inf. Technol. Biomed. 16, 878–890 (2012).
87. Zhang, Q., Eagleson, R. & Peters, T. M. Dynamic real-time 4D cardiac MDCT image display using GPU-accelerated volume rendering. Comput. Med. Imaging Graph. 33, 461–476 (2009).
88. Piovoso, M. & Laplante, P. A. Kalman filter recipes for real-time image processing. Real-Time Imaging 9, 433–439 (2003).
89. Li, G. et al. Accuracy of 3D volumetric image registration based on CT , MR and PET / CT phantom experiments. J. Appl. Clin. Med. Phys. 9, 17–36 (2008).