1

Contribution to Segmentation in Alzheimer’sDisease Research

Marc Garriga

Professor Josep R. CasasPhD Edu Vilaplana

Abstract

Alzheimer’s disease is the most common form of dementia and a socio-economic problem for the society nowadays. Moreover,the problem will grow in the next decades due to aging in developed countries. At pre-clinic stage, when treatments are expectedto be more effective, automatic techniques and tools allowing identification of subtle alterations have recently focused the attentionof the research community.

FreeSurfer offer modules that, although effective, still present significant segmentation problems. The Masther Thesis focuson the improvement of cortex segmentation functionalities for brain morphology analysis, with the aim to help in the validationof ongoing clinical hypotheses for the brain research project at the Alzheimer’s Hospital de Sant Pau lab.

Index Terms

Alzheimer, Segmentation, Skull-stripping, Watershed, Pial Surface.

I. STATE OF THE ART

A. Scientific Research on Alzheimer’s Disease

Alzheimer’s disease (AD) is one of the most common pathology of later life, currently incurable and terminal [2]. Evenmore, this illness is increasingly affecting the population due to the increase in life expectancy [8]. AD cannot be diagnoseduntil dementia has been clinically observed, when the patient shows cognitive deficits which causes an immediate loss onmemory and other mental capabilities.

Figure 1: Male and female expetancy at birth and gender gap, 1950-2050. Figure from [8].

Evolution of AD is divided into 3 phases regarding the patient: cognitively normal, mild cognitive impairment (MCI) anddementia. First one is related to the pre-symptomatic phase, in which individuals have pre-symptomatic AD (hypothesis ofAD). MCI phase is related to some alterations or deficits in episodic memory, but does not meet the criteria to be classifiedas dementia.

The Memory Unit of the Hospital de Sant Pau (HSP) and other scientific research institutions have shown that the pre-clinicalstage of the disease (when symptoms of dementia are not exposed yet) is vital to better understand the development of theillness. At this stage, patients are cognitively normal but some have pathological changes in the brain. Pathological changes

2

involve an atrophy of the brain surface. At pre-clinical stage, AD is only a hypothesis so these patients could die withoutexposing any symptom.

Biomarkers are indicators of changes in the brain. Neuronal degeneration and later AD symptoms have a strong correlationwith these biomarkers [10] [11]. One of the accepted models in the literature is the one proposed by Jack et al and it is shownin Figure 1. This model proposes that the initiating event in AD is related to an abnormal processing of a peptide1 known asAmyloid Beta (Aβ). During this process, cognitive responses of the patient are normal, but after a lag period, which variesfrom patient to patient, neurodegeneration becomes dominant.

Figure 2: Biomarkers of the AD evolution. Figure from [11]

As biomarkers’ level are increasing, when Aβ biomarker is at the top, the physiopathogical process is advanced enough toproduce loss of cognitive capacities.

The term ”biomarker” is not only used to denote biochemical indicators. There exist other types of biomarkers, such asimage biomarkers. One of these biomarkers is based on the structural Magnetic Resonance Imaging (MRI), which is a medicalimaging technique used in radiology to investigate the anatomy and physiology of the body. An MRI applied to the brainproduces an image which allows the distinction between different brain tissues, such as Gray Matter (GM), White Matter(WM) and Cerebrospinal Fluid (CSF).

There are different types of MRI, depending on which parameter is being measured. Moreover, depending on the fieldstrength of the magnet applied (measured in Tesla), MRI can have better or worse resolution. Different types of MRI (f.i. T1,T2, FLAIR) have distinctive contrast, saturation and brightness.

In the past, researchers analyzed brain MRI by ”hand”. Obviously, this process entailed a large amount of time and was notas precise as nowadays. At this point, research community is using automatic techniques and tools in order to analyze MRIand being capable to identify alterations on the cerebral structure and associate them to other biomarkers (f.i. physiological orbiochemical).

B. Brain Structure Extraction

MRI biomarkers can be extracted from processing and analyzing brain MRI images. There are several image processingtools and techniques which are capable of extracting valuable information about the brain anatomy (i.e. cortical thickness,surface area, curvature of the brain and labeling areas). However, all MRI processing tools use to follow a similar pipeline[5], which can be summarized as:

1) Talairach Coordinates Registration.2) Intensity Normalization.3) Skull-Stripping.

1Pepride: a molecule consisting of 2 or more amino acids.

3

4) White Matter Labeling.5) Cutting Hemisphere Planes.6) Connected Components.7) Surface Tessellation, Refinement and Deformation.8) Validation of Surface Topology and Geometry.

As this set of processes are completely automatic, an MRI analysis could fail if one or more steps have an error. This meansthat after all processes are done, the researcher has to correct manually those parts of the image (which has several cuts, as itis a 3D representation of the brain) which are not well-classified. Then, the corrected data set for the subject under analysismust be launched again from the step where the errors were produced.

For instance, and this is the topic of this master project, it is usual to have some errors on the Skull-Stripping phase.Skull-Stripping [13] refers to the process of removing the skull and outer non-brain tissue from the rest. As it is an early(and essential) process in the workflow of MRI processing, it is vital to remove completely the skull and non-brain tissue. Ifsome non-brain tissue is not well-removed from the rest, labeling of white-matter could fail. Thus, the process of pial surface2

segmentation (in which an algorithm of curve inflation, e.g. watershed, is involved) could have some failures.

Figure 3: Process of Skull-Stripping. Software: Tkmedit (Freesurfer Editor)

As explained in the previous section, different types of MRI (f.i. T1, T2, FLAIR) have distinctive properties. That impliesthat in the Skull-Stripping process, one can take profit of different types of MRI of the same individual in order to mix thoseparts of images which have better contrast, and to strip without having those errors.

C. Thesis Objective

The Image Processing Group (GPI) at Universitat Politecnica de Catalunya (UPC) is collaborating with HSP in order tomake contributions to segmentation in FreeSurfer [5] for Alzheimer’s Disease studies. At the present time, there are some toolsavailable in order to avoid hand-correction, which can take a large amount of time.

The aim of this project is to analyze two different versions of the Skull-Stripping module on FreeSurfer: v5.1 and v5.3, whichis the newest version. v5.3 allows to make a pial surface refinement by using an additional MRI type FLAIR or T2. Thesescan types allow dura to be distinguished from grey matter. FLAIR or T2 are useful for correcting automaticaly segmentationmissplacements which would not be eliminated if only a MRI type T1 was used.

At GPI there is a FreeSurfer v5.3 [7] and FSL [12] environment, both are state of the art regarding MRI analysis software.This environment was installed by Esteve Cervantes and Albert Gil [4]. In recent years, HSP has segmented his MRI databaseof AD using v5.1, and devoting significant resources to manual corrections. Upgrading version is not a possible solution, atleast in a short term. The reason is that between versions there is a meaningful change in the statistical data output (thickness,area, volume, etc). An upgrade to Stable v5.3.0 would mean to restart all the data processed until now, which would entail ahigh impact in time and data collection for researching.

2Pial surface: a surface representing the boundary between grey matter and cerebrospinal fluid

4

II. METHOD

There are several types of MRI images, depending on which parameters are measured. In fact, an important feature of MRIis that it can generate a set of images of the same subject with different contrast characteristics. In this chapter, we will explainthe basics of MRI and why they are useful in the processing stream of FreeSurfer.

A. MRI images

The human body is composed of trillions of atoms, which are the building blocks of everything. MRI produce images byreceiving different signal intensities, depending on the hydrogen content from different tissues. Hydrogen protons are neededfor MRI because they have a unique magnetic signature that is captured at the time of scan.

The MRI machine applies a radiofrequency pulse in the image region of interest. These pulses incite the protons absorbthese energy and spin. When the frequency is turned off, protons spin slower until they return to its normal position. Thecomputer captures this energy amount during the relaxation stage (relaxation time) of tissues, and then this captured energy isconverted into an image presented on a screen.

The greater the hydrogen content, the brighter the image. Moreover, the MRI scanner can be programmed in order to onlyanalize an specific direction of proton movement. At this point, we can refer to three type of basic MRI scans, depending onthe parameters (direction, movement) that are being measured:

• T1-weighted: image contrast is based on the longitudinal relaxation time (T1) of protons. Tissue with short T1 time(white-matter and dura) appears brighter in the MRI image.

• T2-weighted: image contrast is based on the transverse relaxation time (T2) of protons. Tissue with large T2 time appearsbrighter in the MRI image.

• FLAIR (Fluid-Attenuated Inversion-Recover): CSF signal, which is brigth, is substracted from the rest of the MRI image.

Figure 4: Coronal plane of T1-weighted scan (left) and FLAIR scan (right)

Of course, as each type of MRI images described above have different characteristics, they are used for different medicalpurposes. Briefly, because this is not the topic of this project, we will explain the keypoints of each MRI type in order tointroduce the pial surface refinement algorithm used in Freesurfer v5.3, and why each type is useful for.

T1-weighted is mostly used for anatomical purposes. This means that T1 is better for analyzing brain structure in order todisplay pathologies. On FreeSurfer and other anatomical analisis tools, it is common to start all the processing procedures withthis type of MRI data.

As it is show in Figure 5, the problem is rather located outside, in the non-brain tissue, in the limit between the pial surfaceand dura. Ocassionaly, given that contrast in this zone is not enought, the skullstripping process does not work very well andthere are parts of dura that still remain . As dura and cerebrospinal-fluid is needed to be completely substracted for a properbrain surface segmentation, sometimes some extra data information is needed in order to remove those parts that are still there.

5

Figure 5: Error on pial surface and white-matter surfaceestimation (red and yellow curve, respectively) due to a bad skull strip.

B. FreeSurfer Processing Stream

FreeSurfer processing stream is computed by a shell script called recon-all. This script calls all the processes neededto generate new volumes with the appropiate format. After these volumes are generated, voxel3-based and surface-basedmorphometry of brain techniques are used [Figure 6]. This set of techniques [Figure NUM] allows to segment the volume andto parcellate and label regions of interest (ROIs) of the brain.

Figure 6: Process Stream Overview Graphic

Recon-all script is divided into three set of processes [Figure 7], which can be called separately if is required (for example,to perform manual editing).

Initialy, recon-all normalizes the input MRI (rawavg.mgz) by scalating the intensity of all voxels at each slice. Result issaved as T1.mgz (in case that input recieved is a T1-weighted scan). Then, the watershed algorithm is used to find the boundarybetween skull and brain. Once the boundary is located, skull voxels are removed. The process of removing skull from brain isknown as skullstripping. The result is saved as brainmask.mgz This first processing step takes 1.5 hours and can be executedindividually by adding -autorecon1 flag on the main command.

After this first processing is done, the next step is to create the white-matter surface and pial surfaces using brainmask.mgz. Inorder to identify those voxels which have more probability to belong to white-matter, a second intensity normalization is done.Then, white-matter is isolated from everything else (output wm.mgz) and this is separated into left and right hemispheres. After

3Voxel: from volumetric pixel. Represents a value on a regular 3D grid.

6

a preprocess of white-matter filling using morphology operators, each hemisphere is tessellated (covering the filled hemispherewith triangles). When tessellation is done, white-matter surface is defined. By expanding white-matter surface in brainmask.mgzvolume pial surface is computed.

Figure 7: FreeSurfer workflow and user interaction

Of course, if white-matter labeling is not well-perfomed, white-matter surface will not follow accurately the subcorticalboundary, hence pial surface could also fail. In this case, when the processing of a subject dataset is finished, researchers useto select manually those points not labeled as white-matter (known as control points) and re-run in order to correct surface.This can be done by executing recon-all with -autorecon2 flag. The processing step takes around 10-12 hours.

After pial segmentation is done, volume labels are applied to subcortical brain structures. Label assigments are based onthe prior probabilities of voxel identity, which are assigned by the default FreeSurfer atlas (estimated by a manually labeledtraining set), and the probability of voxel identity based on the tissue location assignment of surrounding voxels.

The last process stage performs statistical evaluation. In order to compute these statistics, a spherical morphing of surfacesis computed. This process consists in inflating surface to create volume in order to register the surface to a spherical atlas [6].Once volume is registered, a second label assignment is done by parcellating volumeric cortex surface and using informationderived from atlas.

After parcellation is done, statistics as curvature index, total surface area, total gray matter volume or average corticalthickness are computed on a summary table. This whole process takes around 8-10 hours, using autorecon-3.

III. EXPERIMENTS

At this point, FLAIR and T2-weighted MRI brain images are used in order to improve cortical surface computation due toa wrong skullstripping process. This process is called Pial Surface Refinement, and was released in Stable version 5.2.0 andfixed in Stable version 5.3.0 [1].

At Alzheimer’s HSP lab FreeSurfer version 5.1 is installed. Version 5.1 does not have the Pial Surface Refinement module.This module considerably reduces (often eliminates) the amount of time required by researchers for hand-correction of thoseparts of the image not well-removed, as it is automatic and requires less time. Upgrading to v5.3 would imply redoing all thiswork without guaranteeing the same results for their baseline. However improving these results with an additional module,is acceptable for them. Therefore, the solution is to port this module to Stable v5.1.0 at HSP and verify the improvement interms of data statistics and computation time.

7

A. Pial Surface Refinement with T2/FLAIR

In this section, we explain how main programs involved in Pial Surface Refinemenet module work. These are the progamsported to old version 5.1.0 at HSP.

• bbregister: creates a T2/FLAIR registered to the T1 scan. Is the first process on pial surface refinement.

Bbregister uses an algorithm called boundary-based registration (BBR) [9] (Doug Greve et al.). This algorithm tries toalign an input scan (T2/FLAIR) into a model (in this case: a T1-weighted scan) by minimizing a cost funtion (based ingradient descent). The method is based in tissue boundaries and the fact that it is expected to see reliable changes inintensity across these boundaries.

• mri normalize: once the T2/FLAIR registration is done, this module converts an auxiliar volume from cortical reconstruc-tion as input into a new volume where white-matter image values all range around 110 voxel (3D pixel) value. This isnecessary in order to start the last step.

• mris make surfaces: firstly, this program calculates the surface that best defines the white-matter boundary (e.d white-matter surface). This procedure is made by using those voxels labeled as WM in the corresponding T1-weighted scan.

After subcortical boundary is defined, the next step is to define the pial surface. The pial surface is created by expandingthe white-matter surface until converges with gray-CSF intensity gradient. Another restriction in this expanding force isthat the generated white-matter surface and pial surface should not cross.

If mris make surfaces algorithm is refining pial surface (Chapter 3, Section C) there is no need to segment the white-matterboundary, because this is previously computed in the recon-all process (explained in Chapter 2, Section A). In this case,registered and normalized FLAIR/T2 instead of T1 is used in order to expand white-matter surface and refining pial surface.

B. FreeSurfer Merged Version

This section is concieved as a reference guide regarding the procedure that Albert Gil (Software Engineer at UPC ImageProcessing Group) and I have followed in order to port the Pial Surface Refinement module (Stable v5.3) from GPI server toAlzheimer’s Hospital de Sant Pau lab.

To begin with, first step was to identify those binaries called when pial refinement is working. Altough the main programsdescribed in the previous section are the most relevant ones in order to explain the internal process of pial refinement, FreeSurferhas hundreds of binaries which, depending on the input user setting, can change procedures and outputs. Some of these binarieshave not been described in the previous section because they have an auxiliar role in the process of cortical surface refinement(f.i. converting files, computing transforms, etc) .

Here is the list of the imported binaries and scripts to Stable v5.1:

• mri convert• bbregister (script)• mri matrix multiply• mri segreg• tkregister.tlc (script)• mri normalize• mris make surfaces

We followed the same modus operandi for all binaries, as is shown in Figure 8.

At first, we linked each 5.3 binary to Stable 5.1 version command usage. Then, for each binary replaced, we added one byone each missing lib. We wanted to mantain libraries from the current version at HSP as much as possible. Most of missinglib files were from ITK library [3]. FreeSurfer does not use ITK, but FSL does. ITK provides a large set of tools for theneuroimaging community. FSL is used as an internal call of FreeSurfer in order to compute some operations, for instance,registration matrices between scans.

8

Figure 8: Porting binaries workflow

IV. RESULTS

In this chapter we show how pial surface is corrected before Pial Surface Refinement module is used. In order to do thecomparison, we use T1-weighted MRI cortical and subcortical surfaces before and after surface refinement is done. We showhow pial surface refinement performs the accuracy in those cases where pial surfaces segment incorrectly non-brain tissue(mainly dura) as cortical matter.

However, not all the cases where there is a pial surface missplacement have a performance before this module is used. Forinstance, a typical case where wrong pial surface is not corrected by this module (and sometimes surface refinement even get aworse result) is when pial surface extends into cerebellum. As cerebellum voxels are not skull-stripped in recon-all, FLAIR/T2MRI has nothing to do in this case.

A. T1 Cortical Surface vs Pial Refinement with FLAIR

The figures shown in the left-hand have disting segmentation errors on the pial surface (segmentation in red). These subjectswere under analysis on FreeSurfer with Stable version 5.1.0 (after recon-all is done). On right-hand, the same subject is shownafter the segmentacion was correced with the Pial Surface Refinement module ported from version Stable 5.3.0 to HSP.

Figure 9:On left figure: the error is located on left hemisphere (right side of the fig.). Pial segmentation is incorporating non-braintissue, in this case, a small area of cerebrospinal fluid (CSF). On right figure: although missplacement is almost corrected,there is a small isolated area which should be removed. Furthermore, a significant gray-matter area is outside from the pial

surface curve.

9

Figure 10:On left figure: the error is located on left hemisphere. This is the result of a bad segmentation incorporating a piece of the

dura within the pial surface. Error due to a bad skullstripping preprocess. On right figure: by using a registered FLAIR scan tothe T1 scan, dura can be classified as non-brain tissue, outside the curve. However, there is a small area which is not refined.

Figure 11:On top-left figure: same situation as Fig. 10. On top-right figure: pial missplacement is completly removed. On bottom-left

figure: same figure but with voxels skull-stripped not shown. The brightest voxels of the skull are not removed on theskull-stripping process. Thus, the process of inflating white-matter surface in order to estimate pial surface is not working

well.

10

Figure 12: On left figures: pial missplacement on left hemisphere. On right figures: non-brain tissue segmented aspial surface is fully corrected by its corresponding FLAIR image.

Figure 13: On left figure: pial surface incorporates a huge area of dura on left-hemisphere. The same situation is happeningbut on right hemisphere. On right figure: Both missplacements are corrected using FLAIR, which has a high-gradient

intensity on the pial-dura boundary.

Figure 14: On left figure: pial surface penetrates into part of dura on left hemisphere.On right figure: pial surface is well refined without losing cortical matter.

11

V. CONCLUSIONS

Stripping the skull and other non-brain tissues from structural brain images is a challenge and a significal step for differentpost-processing tasks. There is such a long variability among brains, and current techniques are often susceptible to problemswhich usually require manual intervetion. Hand editing can take a long time, as one processed subject with FreeSurfer havemore than a thousand brain slices. Furthermore, each slice can have a strong dependance at some point with respect nextslice, thus researchers have to carefully visualize the points they are correcting. Although skull-stripping is an early processing,post-processes as pial surface definition and in consequence ROI labeling could be wrong estimated. The pial reconstruction isa complex process that requires the solution of subtasks such as intensity normalization, skull-stripping, filtering, segmentationand white-matter surface deformation.

At this point, authomated techniques as pial surface refinement are becoming more usual. These methods are supporting thestandard brain processing stream, and help researchers to reach a good result without being a highly consuming task in termsof time and accuracy. By working with different scans from the same subject one can take profit from difference of intensitiesto preprocess the image. Using FLAIR or T2, the pial surface can be generated using intensity gradient from these scans toguide the surface deformation. These scan types allow dura to be distinguished from grey matter, which is very useful to avoidmissplacements due to a defective skull-stripping processing.

FreeSurfer Stable v5.3 allows this type of processing. Alzheimer’s Hospital de Sant lab has a huge collection of processeddata for research purposes. These data were processed under FreeSurfer v5.1, which do not have the surface refinement option.Moreover, data statistics extracted from 5.3 version are not equal compared to version 5.1 at Hospital de Sant Pau. Upgradingto newest version would mean a change in statistics and redoing all data processed until now.

According to the objectives set out at the beginning of this Ms Thesis, we have been able to individualize the step of PialSurface Refinement. We have identified the relevant module and ported it to v5.1, without changing the rest of the procedure,as required by Hospital de Sant Pau researchers, who can now take advantage of FLAIR/T2 scans to improve their MRIsegmentation results. For this task, we have identified the input and the output data for each one of the sub-processes involvedin order to insert the Pial Surface Refinement module with the minimum disruption It has been proven that this module removesmissaligments of the pial surface without damaging other zones. However, the ported module seems to fail (or at least, doesnot work completely) in some cases where some skull intensities are too high even using FLAIR/T2 scans. This is due tothe watershed algorithm used to remove the skull from the brain. This process is critical, as is one of the first processesin FreeSurfer and other brain analyzing tools. Moreover, the brain shape is a very complex structure, and for this reasonskull-stripping stage is not robust enough in difficult cases where intensities of non-brain tissues are too equal to gray-matter.

At this moment, an improvement of skull-stripping, which is the cause of several errors, is going to be quite necessary inorder to reduce those errors on post-processes that are highly dependent of this method.

Porting this module to v5.1 entails an automated pial surface refining without the need of redoing all statistics collecteduntil now. This contribution will be upload to FreeSurfer forum in order to help those research institutions with the same issueregarding missplacements of cortical surface.

VI. ACKNOWLEDGMENTS

Thanks to professor Josep R. Casas for his support and dedicated time and for giving me the opportunity to work on this UPCcollaboration with Hospital de Sant Pau. Many thanks to Dr. Juan Fortea and Edu Vilaplana for introducing me to FreeSurfer.Finally, I would like to thank Albert Gil for helping me on porting the Pial Surface Refinement module to Alzheimer’s Hospitalde Sant Pau lab.

12

REFERENCES

[1] https://.surfer.nmr.mgh.harvard.edu/fswiki/releasenotes.[2] www.alzforum.org.[3] www.itk.org/.[4] Esteve Cervantes Martın. Segmentacio de neuroimatge amb freesurfer. 2014.[5] Anders M Dale, Bruce Fischl, and Martin I Sereno. Cortical surface-based analysis: I. segmentation and surface reconstruction. Neuroimage, 9(2):179–194,

1999.[6] Rahul S Desikan, Florent Segonne, Bruce Fischl, Brian T Quinn, Bradford C Dickerson, Deborah Blacker, Randy L Buckner, Anders M Dale, R Paul

Maguire, Bradley T Hyman, et al. An automated labeling system for subdividing the human cerebral cortex on mri scans into gyral based regions ofinterest. Neuroimage, 31(3):968–980, 2006.

[7] Bruce Fischl. Freesurfer. Neuroimage, 62(2):774–781, 2012.[8] LA Gavrilov and P Heuveline. Demographic determinants of population aging.[9] Douglas N Greve and Bruce Fischl. Accurate and robust brain image alignment using boundary-based registration. Neuroimage, 48(1):63–72, 2009.

[10] Harald Hampel, Katharina Burger, Stefan J Teipel, Arun LW Bokde, Henrik Zetterberg, and Kaj Blennow. Core candidate neurochemical and imagingbiomarkers of alzheimer’s disease. Alzheimer’s & Dementia, 4(1):38–48, 2008.

[11] Clifford R Jack, David S Knopman, William J Jagust, Leslie M Shaw, Paul S Aisen, Michael W Weiner, Ronald C Petersen, and John Q Trojanowski.Hypothetical model of dynamic biomarkers of the alzheimer’s pathological cascade. The Lancet Neurology, 9(1):119–128, 2010.

[12] T.E. Behrens M.W. Woolrich S.M. Smith M. Jenkinson, C.F. Beckmann. Fsl. Neuroimage, 9(2):782–790, 2012.[13] Stephen M Smith. Fast robust automated brain extraction. Human brain mapping, 17(3):143–155, 2002.