NeuroImage 43 (2008) 312–320

Contents lists available at ScienceDirect

NeuroImage

j ourna l homepage: www.e lsev ie r.com/ locate /yn img

Measurement of brain atrophy in subcortical vascular disease: A comparison ofdifferent approaches and the impact of ischaemic lesions

Mike O'Sullivan a,⁎, Eric Jouvent b, Philipp G. Saemann c, Jean-Francois Mangin d, Anand Viswanathan b,e,Andreas Gschwendtner a, Luc Bracoud f, Chahin Pachai f, Hugues Chabriat b, Martin Dichgans a

a Neurologische Klinik, Klinikum Groβhadern, Ludwig Maximilians University, Munich, Germanyb Department of Neurology, CHU Lariboisiere, Paris, Francec Max Planck Institute of Psychiatry, Munich, Germanyd Unité de NeuroAnatomie Fonctionelle, Service Hospitalier Frédéric Jolliot, CEA, Orsay, Francee Neurology Clinical Trials Unit, Massachusetts General Hospital, Boston, USAf Bio-Imaging Technologies SAS, Lyon, France

⁎ Corresponding author. Centre for Clinical NeuroscieLondon, Cranmer Terrace, London, SW17 0RE, UK. Fax: +

E-mail address: research@mikeosullivan.org (M. O'S

1053-8119/$ – see front matter. Crown Copyright © 20doi:10.1016/j.neuroimage.2008.07.049

a b s t r a c t

a r t i c l e i n f o

Article history:

Measurement of brain atrop Received 2 March 2008Revised 9 July 2008Accepted 9 July 2008Available online 5 August 2008

hy has been proposed as a surrogate marker in MS and degenerative dementias.Although cerebral small vessel disease predominantly affects white and subcortical grey matter, recent datasuggest that whole brain atrophy is also a good indicator of clinical and cognitive status in this disease.Automated methods to measure atrophy are available that are accurate and reproducible in disease-freebrains. However, optimal methods in small vessel disease have not been established and the impact ofischaemic lesions on different techniques has not been explored systematically. In this study, threecontrasting techniques – Statistical Parametric Mapping 5 (SPM5), SIENAX and BrainVisa – were applied tomeasure cross-sectional atrophy (brain parenchymal fraction or BPF) in a large (n=143) two-centre cohort ofpatients with CADASIL, a genetic model of small vessel disease. All three techniques showed similarsensitivity to trends in BPF associated with age and lesion load. No single technique was particularlyvulnerable to error as a result of lesions. Provided major errors in registration were excluded by visualinspection, manual correction of segmentations had a negligible impact with mean errors of 0.41% forSIENAX and 0.46% for BrainVisa. BPF correlated strongly with global cognitive function and physicaldisability, independent of the technique used. Correlation coefficients with the Minimental StateExamination score were: BrainVisa 0.58, SIENAX 0.58, SPM5 0.60 (for all, pb0.001). These results suggestthat all three methods can be applied reliably in patients with ischaemic lesions. Choice of analysis approachfor this kind of clinical question will be determined by factors other than their robustness and precision, suchas a desire to explore subtle localised changes using extensions of these processing tools.

Crown Copyright © 2008 Published by Elsevier Inc. All rights reserved.

Introduction

Measurement of brain atrophy has been proposed as a surrogatemarker for therapeutic trials in a number of diseases. Even in diseasesthat predominantly affect white matter, measurement of whole brainvolume is often more closely linked to clinical and cognitive statusthan the burden of white matter lesions (Filippi et al., 2004; Miller etal., 2002; Miller, 2004; Peters et al., 2006). In MS, this relationship hasbeen ascribed to a closer link with axonal loss (Miller et al., 2002; Tripet al., 2006). Cerebral small vessel disease (SVD) is another commonand important cause of white matter abnormality, cognitive impair-ment and dementia. Studies in this specific group (O'Sullivan et al.,2004; Peters et al., 2006), are also beginning to suggest that brain

nce, St George's, University of44 208 725 2950.ullivan).

08 Published by Elsevier Inc. All rig

atrophy is more closely aligned with cognitive status and disabilitythan the volume of typical ischaemic lesions.

A variety of automated methods are now available that allowmeasurement of brain volume with a high degree of accuracy andreproducibility, at least in disease-free brains (Fox and Schott, 2004;Smith et al., 2002b). However, optimal protocols for evaluating brainvolume in cerebrovascular disease have not yet been established.Available methods differ in a number of respects. One importantdistinction is between those that involve registration to a prioritemplates and those that do not (Anderson et al., 2007, for review seePelletier et al., 2004). Approaches to registration also differ with somemethods constrained predominantly by the skull and others con-strained by prior images of normal brain (Ashburner and Friston,2005). Subcortical vascular lesions can have a profound impact on thedistribution of image intensities across the brain and, in theory, thesechanges could affect either segmentation or registration steps.Furthermore, their impact would potentially differ between different

hts reserved.

313M. O'Sullivan et al. / NeuroImage 43 (2008) 312–320

techniques. So far, little work has focussed on the effect of vascularlesions, and different lesion loads, on the applicability of differenttechniques.

CADASIL (cerebral autosomal dominant arteriopathy with sub-cortical infarcts and leukoencephalopathy) is a rare, genetic smallvessel disease, which like the common sporadic form of SVD ischaracterised by diffuse, confluent regions of hyperintensity on FLAIRor T2-weighted images (WML) and discrete lacunar infarcts. The maindifference with sporadic disease is that patients often develop highlesion loads at a relatively young age (O'Sullivan et al., 2001). Thus,studies in CADASIL offer a stringent test of the impact of lesions onvolumetric methods and can extend our understanding of differentmethodological approaches to more extreme conditions with exten-sive, widespread lesions.

In this study, we set out to compare three widely-used, butconceptually distinct, approaches to measure brain parenchymal fraction(BPF, that is the volume of the brain parenchyma as a proportion of totalintracranial volume) in a large cross-sectional cohort with CADASIL. BPFis a measure of the “state” of brain atrophy at a point in time and themost suitable measure for cross-sectional analysis (Horsfield et al.,2003). We chose techniques that reflect the variety of differentapproaches that are available, including both registration-based andnative space techniques, and contrasting approaches to registration. Asan additional criterion we chose non-commercial algorithms that arefreely available. Three techniques were evaluated: BrainVisa, whichinvolves tissue segmentation in native space; SIENAX, a registration-based technique in which registration is constrained predominantly bythe skull (Smith et al., 2007); and Statistical Parametric Mapping 5(SPM5) in which image segmentation and registration are unified andconstrained by tissue priors (Ashburner and Friston, 2005).We set out todetermine: i.) whether these techniques were robust in CADASIL, interms of producing consistent, visually plausible brain segmentations;ii.) howmeasures of BPF compare between techniques; iii.) the impact ofsubcortical lesions – diffuseWML and lacunae – on BPF measurements;iv.) whether techniques differed in their clinical relevance in relation toglobal cognitive and disability scores. In addition, data were used fromtwo scanners with different T1-weighted sequences to provide someindication of their generalisability and relevance tomulti-centre studies.

Materials and methods

Subjects were drawn from an ongoing longitudinal study (Viswa-nathan et al., 2006). Inclusion criteriawere a definite genetic diagnosisof CADASIL and acquisition of a three-dimensional T1-weighted imageof the whole brain for volumetric assessment. 143 subjects wereincluded: 106 patients fromHôpital Lariboisière, Paris and 37 from theNeurologische Klinik, Klinikum Grosshadern, Munich. The studypopulation was made up of 63 men and 80 women, with an averageage of 52.0±11.2 years (range 24–75). Three subjects were in the 3rddecade, with 20, 42, 40, 31 and 7 subjects in the 4th to 8th decadesrespectively. The age distribution was similar in the two centres:Munich, mean±SD 47±10 years (range 32–73); Paris, mean±SD 54±11 years (range 25–76). Womenwere over-represented in the Munichcohort (27 women, 10 men) with an equal sex distribution in the Parissample (53 men, 53 women). All subjects underwent a baselineclinical assessment including collection of demographic and riskfactor status, neurological and neuropsychological assessment. Globalcognitive functionwas assessed with two scales, the Minimental StateExamination (MMSE) and the Mattis Dementia Rating Scale (MDRS)and physical disability with the modified Rankin scale.

MRI and measurement of lesion volumes

MRI scans were obtained by the use of a 1.5-T system (Vision,Siemens (Munich) or Signa General Electric Medical Systems (Paris)with a standard quadrature birdcage headcoil in both centres).

Volumetric T1-weighted acquisitions: Munich, magnetization pre-pared rapid gradient echo (MPRAGE), TR/TE/TI 11.4/4.4/500 ms, slicethickness 1.19 mm, no interslice gap, 256×256, in-plane resolution0.9×0.9 mm, flip angle 15°, acquisition time 9 min58 s; Paris, fastspoiled gradient echo (SPGR) TR/TE 9/2 ms slice thickness 0.8 mm, nointerslice gap, 256×256, in-plane resolution 1.02×1.02 mm, flip angle20°, bandwidth 122 Hz/px, acquisition time 4 min3 s). In addition,fluid-attenuated inversion recovery (FLAIR, Munich: TR/TE/TI 4284/110/1428 ms, slice thickness 5 mm, no interslice gap, 176×256, in-plane resolution 1.07×0.98 mm, flip angle 180°, acquisition time4 min42 s; Paris: TR/TE/TI 8402/161/2002 ms, slice thickness 5.5 mm,no interslice gap, 256×160, in-plane resolution 1.5×0.94 mm flipangle 90°, bandwidth 61 Hz/px, acquisition time 3 min41 s); andproton density-weighted (turbo spin echo) (Munich: TR/TE 3300/16 ms, slice thickness 5 mm, no interslice gap, 190×256, in-planeresolution 0.99×0.98 mm, flip angle 180°, acquisition time 2 min13 s;Paris: TR/TE 3300/15 ms, slice thickness 5.5 mm, no interslice gap,256×192, in-plane resolution 1.25×0.94 mm, flip angle 90°, band-width 125 Hz/px, acquisition time 9 min15 s) sequences wereacquired.

WML were delineated on FLAIR images. All FLAIR axial slices fromthe base of the cerebellum to the vertex were analyzed. A mask ofhyperintensities was generated from FLAIR images after application ofan intracranial cavity mask by applying a threshold on signal intensityderived from the signal intensities histogram. Two trained neurolo-gists using drawing tools then validated lesion boundaries andcorrected them visually if necessary. Lacunar lesions were defined asdiscrete lesions of CSF intensity on T1-weighted images consistent inappearance and location with lacunar infarction. Enlarged perivas-cular spaces – defined according to previous publications (Maclullichet al., 2004) as small (less than 2 mm) sharply delineated structuresfollowing the orientation of perforating vessels perpendicular to thebrain surface –were not included. To assess lacunar lesion volume, T1-weighted images were segmented into 4 tissue classes consisting ofwhite matter, grey matter, CSF/lacunar lesions and non-brain parti-tions. This was performed independently of the tissue class segmen-tations of the volumetric techniques using native space images. Aftersegmentation, the lowest intensity tissue class was automaticallylabelled as CSF/lacunar lesions. Subsequently, two raters verified thetissue classification and separated lacunar lesions from peripheral orventricular CSF based on anatomical configuration using custom 2Dand 3D image editing tools provided by BioImaging (BioImagingTechnologies SAS, Lyon, France). Good interrater reliability has beendemonstrated for assessment of bothWML and lacunar lesion volume(intraclass correlation coefficient 0.99 for WML volume, 0.83 forlacunar lesion volume (Viswanathan et al., 2006)).

Automated determination of the volume of the intracranial cavitywas performed using proton density-weighted images from base totop of the skull using a dedicated algorithm from BioImaging. Thisalgorithm performs intensity-based segmentation into multiple tissueclasses, detection of the edge of the outer CSF class with visualverification of tissue class assignments. The corresponding mask wasmarginally corrected if necessary.

Volumetric techniques

Table 1 summarises the main differences and implementationsteps of the three techniques.

Statistical Parametric Mapping 5 (SPM5)SPM5 (Functional Imaging Laboratory, University College London,

www.fil.ion.ucl.ac.uk/spm) was implemented employing defaultsegmentation options. In SPM5 tissue segmentation, registration andintensity non-uniformity correction are combined in a single processbased on a single generative model (Ashburner and Friston, 2005).Images are fitted to tissue priors (i.e. standard segmented images of

Table 1Comparison of techniques

SPM5 SIENAX BrainVisa

Registration Based on tissue priors (i.e. brain and CSF) Constrained by skull NoneIntensity non-uniformitycorrection and segmentation

Integrated as part of a unified generativemodel for non-uniformity correction,segmentation and registration. Usescombinations of Gaussian basis functions

Non-uniformity correction based onminimising entropy and automaticdetermination of number of Gaussianbasis functions

Non-uniformity correction based onentropy and a comparison of correctedand uncorrected image intensityhistograms (Mangin, 2000)

Visualisation/checking i.) display segmented normalised greyand white matter images

‘Rendered’ image overlays segmentationresults on native image providing easyvisual verification (displayed withMRIcro)

Partially transparent segmentationmasks overlain on native slices withAnatomist visualisation toolii.) simultaneous display of native T1 and

native space segmented imagesManual correction of minor errorsin brain outline

Not possible Possible Possible

Typical minor errors corrected – Pre-pontine soft tissue; orbits Pre-pontine soft tissue; orbits; temporalbone rarely

Quantification Requires custom post-processing script Automated AutomatedBasis of quantification Summing modulated tissue probabilities

for grey/white matter at every voxel(producing probable total brain volume)

Summing voxels ascribed to grey/whitematter

Summing voxels ascribed to grey/whitematter

314 M. O'Sullivan et al. / NeuroImage 43 (2008) 312–320

grey matter, white matter and CSF). The default priors in SPM5 arederived from the tissue probability maps of the International Consor-tium for Brain Mapping (www.loni.ucla.edu/ICBM/ICBM_TissueProb.html) derived from 452 T1-weighted scans which were aligned withan atlas space, corrected for scan inhomogeneities and classified intogrey matter, white matter and CSF (Mazziotta et al., 2001). Thesetissue priors have already been registered into MNI (MontrealNeurological Institute) space. The initial output images consist oftissue class probability maps for grey matter, white matter and CSF inthe same MNI space. These images are subsequently modulated toaccount for spatial warping during registration. For example someregions may expand by a certain volume factor during warping; inmodulation the intensities of these regions are scaled by the samefactor so that the total amount (or “volume”) of tissue remains thesame (Good et al., 2001). Summing the probabilities of the modulatedimages for grey and white matter provides a measure of “probabletotal brain volume”; adding CSF provides a measure of “probable totalintracranial volume”.

SIENAXSIENAX is the cross-sectional atrophy tool adapted from Structural

Imaging Evaluation Using Normalisation of Atrophy from the FMRIBSoftware Library (Smith et al., 2002a) (FSL, Oxford Centre forfunctional MRI of the brain, www.fmrib.ox.ac.uk/fsl). SIENAX startsby extracting brain and skull images (Smith, 2002). The brain image isthen registered into MNI152 space (Jenkinson et al., 2002; Jenkinsonand Smith, 2001) using the skull image to derive a scaling factor tocorrect for variations in head size. We performed pilot studies with 10randomly chosen datasets to test segmentation and pre-processingoptions. Cropping the images to exclude axial slices below theforamen magnum led to a clear improvement in registrations byreducing misclassification of pre-pontine skull and soft tissue.Otherwise default options (a fractional intensity threshold of 0.5with no ventrodorsal gradient) provided the most visually accurateregistrations. These parameters were then chosen and appliedidentically to all individual scans. The skull-based registration ofSIENAX also provides a value of normalised brain volume in volumetricunits (ml) but with matched skull outlines therefore removing theeffects of head size variation. As this is a commonly used measure, therelationship of this measure to clinical variables was also assessed.

BrainVisaBrainVisa is a software platform developed by CEA (Commissariat

à l'énergie Atomique), Orsay, France, (http://brainvisa.info) (Riviere etal., 2003). Unlike SPM5 and SIENAX, image processing takes place innative space. First, a correction is applied for spatial intensity non-

uniformity (bias correction) (Mangin, 2000). An intensity histogram isthen generated, with an algorithm which automatically fits severalGaussian functions and identifies those corresponding to white andgrey matter. Brain images are then eroded to break eventualconnections with non-brain tissue. Tissue histogram analysis is thenrefined after most non-brain tissue has been removed, and finally abrain mask is reconstructed using growth seeding. BrainVisa volumeswere normalised according to intracranial volume calculated fromproton-density images (Hartley et al., 2006; Jouvent et al., 2007;O'Sullivan et al., 2007). This approach was used to maintain theprinciple of measurements made in native space for the BrainVisatechnique.

Segmentation failures

All brain segmentations were checked visually for obvious errorsfollowing procedures outlined in Table 1. Segmentation errors weredivided into minor errors and major errors (or segmentation failures).Failures were defined by an obviousmismatch between themargins ofthe segmented images and brain:CSF boundary which were notamenable to minor manual correction of masks. The following minorerrors were not considered sufficient to constitute ‘failures’: inclusionof parts of the orbit, pre-pontine skull and soft tissue, carotid arteriesand parts of the sphenoid, optic chiasm or optic nerves. These errorsare common but appear minor; therefore they were accepted at thisstage but an analysis of their impact on a subset of patients wasperformed (see below).

In all cases of segmentation failure, the native T1-weighted imageswere reviewed to assess whether they displayed any of the followingpre-defined reasons for failure: movement artefact; gross ventricularenlargement; or unexpected parenchymal abnormalities (other thanlacunes and leukoaraiosis). Any additional artefacts were also notedand a single most likely reason was ascribed to the failure of brainsegmentation where possible.

Manual correction of minor segmentation errors

In a randomly selected subset of 20 patients (10 from each centre)the effect of manual correction of minor segmentation errors wasassessed. BrainVisa and SIENAX both provide scope for manualcorrection based on visual inspection of images. In BrainVisa, thevariance of grey and white matter Gaussian functions determinedduring the histogram analysis were manually modified to improve thevisual fit. If necessary, manual corrections with 2 or 3D simpledrawing tools were used. In SIENAX, common sites of misclassificationsuch as the pre-pontine tissues were manually masked from the

315M. O'Sullivan et al. / NeuroImage 43 (2008) 312–320

native images. Masks were drawn on the rendered images while thenative T1-weighted image was displayed alongside (providinganatomical corroboration, in axial, coronal and sagittal planes ifneeded). These masks were then subtracted from the native T1-weighted image, the SIENAX analysis re-run and visual evaluationrepeated until the rater was satisfied with the quality of thesegmentation. Manual mask corrections were performed by twoexperienced neurologists (MO'S with SIENAX and EJ with BrainVisa).Both were blinded to volumetric data (by relabelling of the scans withrandom letters and performing the manual correction phase beforeanalysis of the uncorrected data).

Statistical analysis

BPF was the dependent variable of choice for all analyses.Statistical analysis was performed only in those subjects where BPFmeasurements were possible with all three techniques. The statisticalapproach was based on recommended methods for comparison ofmeasurement techniques (Altman and Bland, 1983) including visua-lisation with Bland–Altman plots. These plots provide easy visualisa-tion of both:

bias — the tendency of one method to systematically producehigher or lower values than the other, and the extent to which thisis influenced by BPF. This is represented by a regression line.error — the discrepancy between measurements once systematicbias between techniques has been removed. This can be displayedby 95% confidence intervals (95% CI) around the regression line.

To assess the effect of age and lesions the whole sample wasdivided into groups (decade for age and quartiles for lacunar andWMLvolumes). Two subjects in their 20s were grouped with those in the4th decade, and 6 subjects in their 70s grouped in the 7th decade forthe purposes of analysis. Grouping subjects allows not just BPF butalso variance of BPF measurement at different ages and lesion loads tobe measured. Variance is a major determinant of statistical power in a

Fig. 1. Ventricular enlargement affected only SPM5. (A) Native space T1-weighted image of afrom SPM5. The color scheme reflects the probability of brain parenchyma (i.e. sum of greybeen ascribed probabilities mostly in the range 0.5 to 0.8. This occurred in 3 of 143 subjects wshowing correct segmentation of the ventricular margins.

study based on a surrogate marker (Sormani et al., 2004) with samplesize requirements directly proportional to the variance of an endpointmeasure. Variance arises from three main sources: i.) true variance inbrain volume between subjects; ii.) measurement error induced byimage acquisition effects such as noise, image distortion and artefact;iii.) variance introduced by error in the calculation of BPF (“analysiserror”). We were unable to calculate these separately, but asmeasurements were made on the same set of images, inter-individualvariance in true BPF and image acquisition effects were constant, soany differences in variance of measured BPF between techniques mustbe due to analysis error. We were therefore mainly interested inwhether age or higher lesion loads led to an inflation of variance forone technique in comparisonwith the others. Because variance is alsosensitive tomean BPF, a coefficient of variationwas calculated for eachsubgroup by dividing the standard deviation by the mean BPF.

In addition, we calculated a further coefficient of variation (CoV)that would reflect the impact of measurement error on sensitivity todisease-related change. The standard deviation (SD) of BPF in themostseverely affected quartile was expressed as a proportion of thedifference in mean BPF betweenmost severe and least severe quartile.Thus, CoVchange=SDtop/ (mean BPFbottom−mean BPFtop) where ‘top’and ‘bottom’ refer to most and least severely affected quartilesrespectively. A similar calculation was made for age by decade.

The effects of manual correction on BPF were assessed with pairedt-tests. Correlations with cognitive function were assessed withPearson's correlation coefficient.

To assess the effect of scanner a multiple analysis of varianceanalysis (MANOVA) was performed with BPF as the dependentvariable. In the first instance a 2×2 factorial design was used withmeasurement technique and scanner as factors and age as a covariate.The important question was whether there was a significant scannerby technique interaction to suggest that volumetric techniquesperformed differently on data from different scanners. At the nextlevel of analysis, either lacune or WML grade was entered asadditional factors. In this analysis a significant scanner by techniqueby lesion load interaction would suggest that the effect of lesions on

subject with CADASIL with ventricular enlargement. (B) Tissue probability map derivedand white matter tissue probabilities). The outer portions of the lateral ventricles haveith marked ventricular enlargement. (C) SIENAX hard segmentation of the same patient

316 M. O'Sullivan et al. / NeuroImage 43 (2008) 312–320

volumetric measurements is dependent on the scanner used. Allstatistical analyses were performed using SPSS (version 12 SPSS Inc,Chicago).

Results

Median volume of WML was 94 ml (range 4–334 ml). 130 subjectshad lacunar lesions. The mean number of lacunar lesions in thesepatients was 14 (range 1–82) and the median total volume of lacunarlesions 0.5 ml (range 0–5.7 ml).

Segmentation failures

Measurements were rejected in 9 (6.3%) of subject scans analyzedwith SPM5, 12 with SIENAX (8.3%) and 17 (11.9%) subjects withBrainVisa. In most individual subjects (24 of 30) segmentation failureoccurred with one method only and BPF measurements were derivedfrom the alternative techniques. Movement artefact was thecommonest identified cause and the main cause of failure in 8patients. In two patients, movement artefact precluded brainsegmentation with all three techniques and also made segmentationof lesions impossible. BrainVisa was most sensitive to movementartefact (failure in 7/8 cases) compared with 3/8 for SPM and 2/8 forSIENAX. Some reasons affected a single technique particularly: grossventricular enlargementwas considered responsible for 3 failureswith

Fig. 2. (A) Correlations between BPF measurements with different techniques. (B and C) Blaindividual patient. The solid regression lines reflect bias — a systematic tendency for one teintervals around this regression line reflectmeasurement error— or the tendency for two techbias over the other two techniques that was greater at smaller values of BPF but measuremenand SPM5 is not shown). BrainVisa showed a small positive bias over SPM5; neither bias no

SPM5 (Fig. 1) while marked temporal pole hypointensity (a featurecharacteristic of CADASIL) led to truncation of the temporal lobe inSIENAX in 2 cases. In 3 cases, segmentations were deemed unsatis-factory because of the presence of a large-vessel infarct extending tothe cortex. A clear reason for segmentation failure was not apparent in13 of 30 cases. Subjects with at least one segmentation failure wereolder (56±9 v. 51±11, p=0.008), more cognitively impaired (MMSE23.4±6.5 v. 26.7±4.3, p=0.02) and more disabled (modified Rankinscale 2.7±1.9 v. 0.9±1.4, pb0.001).

Measurement of parenchymal fraction

Only subjects in whom measurements were made with all threetechniques were included in subsequent analyses. Measurements ofBPF were possible with all 3 techniques in 113 subjects (83 from Paris,30 fromMunich, mean age 51±11 years, range 25–76). The numbers ofsubjects by decade included in subsequent analyses were 3, 18, 37, 28,22 and 5 for 3rd to 8th decades respectively. Fig. 2 presents correlationand Bland–Altmanplots for the different techniques. The bias betweenSIENAX and the other two techniques was greater at lower BPF values,as indicated by the negative slope of the Bland–Altman regression line.However, analysis error (indicated by the 95% confidence intervals)was not influenced by BPF: 95% CI for BPF remained within ±0.04. BPFhad no influence on bias or error for differences between BrainVisa andSPM5.

nd–Altman plots of the difference between BPF plotted against the mean BPF for eachchnique to produce larger or smaller measurements than another. The 95% confidenceniques to produce discrepant results after accounting for bias. SIENAX showed a positivet error remained small andwas uninfluenced by BPF (Figure 2B) a similar plot for SIENAXr measurement error was influenced by BPF.

Fig. 3. Effect of age and subcortical lesions on BPF measurements. The graphs plot the mean BPF for each decade or quartile of lesion severity for each of the three techniques. The error bars correspond to the standard errors of the means foreach decade/quartile group. Increasing age and increasing volume of lacunar lesions were associated with reductions in BPF, detectable with all techniques. The variance in BPF increases with age and between the first and second quartiles forWML severity. This increase in variance is similar across different techniques suggesting that no single technique is particularly prone to analysis error in a particular subset of patients.

317M.O

'Sullivanet

al./NeuroIm

age43

(2008)312

–320

Table 2Coefficient of variation (%) of brain parenchymal fraction according to decade, lesionseverity and measurement technique

Decade Lacunar lesionvolume — quartile

WML lesionvolume — quartile

4th 5th 6th 7th 1 2 3 4 1 2 3 4

SIENAX 2.3 2.9 3.5 4.1 3.3 3.7 3.0 3.6 2.2 4.3 4.5 3.5SPM5 4.2 5.8 7.2 9.3 6.8 8.0 6.7 7.4 4.4 9.4 9.1 8.2BrainVisa 3.8 5.3 7.5 8.5 7.4 7.0 5.5 7.0 4.8 8.0 8.8 7.3

Table 3Correlations with cognition and disability

MMSE(n=108)

MDRS(n=107)

Modified Rankinscale (n=105)

Parenchymal fractionSPM5 0.60⁎⁎ 0.56⁎⁎ −0.62⁎⁎BrainVisa 0.58⁎⁎ 0.53⁎⁎ −0.59⁎⁎SIENAX 0.58⁎⁎ 0.56⁎⁎ −0.60⁎⁎

OtherSIENAX normalised brain volume 0.56⁎⁎ 0.51⁎⁎ −0.58⁎⁎

Lesion measuresLacunar lesion volume −0.25⁎ −0.24⁎ 0.45⁎⁎WML volume −0.05(NS) −0.16(NS) 0.13(NS)

MMSE — Minimental State Examination; MDRS — Mattis Dementia Rating Scale.(⁎pb0.05, ⁎⁎pb0.001, NS pN0.1).

318 M. O'Sullivan et al. / NeuroImage 43 (2008) 312–320

Effect of age and lesions

Fig. 3 illustrates the effect of age and lesion grade onmeasured BPFfor each technique. Increasing age and lacune grade were associatedwith significant declines in BPF with all techniques. Table 2 shows thecorresponding coefficients of variation (CoV). Age was associated withan increase in CoV with all techniques. CoV approximately doubledbetween the first and the second quartile for white matter lesionvolume for all techniques but was little affected by lacunar lesions.Generally, CoV values were smaller for SIENAX, but this was offset bysmaller absolute differences between mildly and severely affectedindividuals. This was reflected in the measure of sensitivity to change,CoVchange, which for the effects of age was 0.71 for SIENAX, 0.69 forSPM5 and 0.77 for BrainVisa.

Brain volume measurements, global cognition and disability

Table 3 provides correlation coefficients with global cognition anddisability. Both cognitive scores and the modified Rankin scalecorrelated with BPF measurements from all three techniques.Technique had very little impact on the correlation coefficients. BPFmeasurements with all techniques were superior to volumetric lesionmeasurements as correlates of cognitive function and disability.

Effect of manual correction of brain segmentations

For both BrainVisa and SIENAX, manual correction of segmentationmasks had a very small impact on measured BPF. For SIENAX, themean (SD) corrected BPF was 0.866(0.028) compared with 0.864(0.026) uncorrected. Corresponding values for BrainVisa were 0.813(0.047) compared with 0.817(0.046). Adopting the manually correctedvolumes as the gold standard, these differences amount to errors of0.41% and 0.46% when SIENAX and BrainVisa respectively were usedwithout manual correction. Manual correction had little impact on thestrength of correlations with age or cognitive function. For example,Pearson's correlation coefficient for SIENAX and age was −0.71 foruncorrected BPF and −0.66 for corrected BPF (both pb0.001).

Effect of different scanners

There was no evidence of an effect of scanner on measured BPF(MANOVA F=0.3, NS) nor evidence of a scanner by techniqueinteraction (F=0.4, NS). In two subsequent models entering lacuneor WML grade as an additional factor, there was no evidence ofsignificant lesion by scanner or lesion by scanner by techniqueinteractions to suggest that the impact of lesions on BPF measure-ments was dependent on the scanner or T1-weighted sequence used.

Discussion

SIENAX, SPM and BrainVisa all appear to be practical options forstudies of brain volume in patients with SVD. The results suggest thatthese approaches can be used in patient groups who have extensivewhite matter lesions and lacunae. Manual correction of segmenta-tions, at least when visual inspection excludes gross errors, appears to

offer little benefit. The clinical relevance of BPF measurements, interms of their strong correlations with global cognitive function, wasmaintained with all three techniques.

The total volume of WML in this study exceeded 100 ml in 58subjects and 200 ml in 15 subjects and 67 subjects had more than 10lacunae, which indicates that this study provided a stringent test ofthe robustness of these techniques in the face of ischaemic lesions. Itseems likely that these techniques will be reliable in sporadic smallvessel disease, which is also typified by widespread WML andmultiple lacunar infarcts but with total lesion loads that are oftenlower than in CADASIL (O'Sullivan et al., 2001). White matter lesionsare also unlikely to be a major threat to reliability in diseases likesevere MS, AD and cerebral amyloid angiopathy where signal changescan be extensive, but rarely as severe as in CADASIL.

Evaluating brain volume measurements in a large cohort frommore than one centre reproduces the conditions of a clinical trial.Availability of data from two centres was a strength, as systematiceffects of scanner or imaging sequence on BPF measurements wouldbe a major problem for multi-centre clinical trials (Horsfield et al.,2003). Although the geographical separation of the centres precludeda careful analysis for inter-scanner effects (which would requirescanning the same patient on both scanners in a very shorttimeframe), we found no trends to suggest scanner effects inmultivariate analyses: there was no evidence for any interactionbetween scanner and any of the three volumetric techniques used inthis study, nor between scanner, lesion load and measurementtechnique. We deliberately chose standard 3D T1-weighted sequencesto make the analysis as relevant as possible to data acquired usingscanners and sequences in widespread clinical use. It is likely thattailoring image acquisition to the analysis tool would further improvethe quality of BPF measurements. However, the results of this studysuggest that even without specific tailoring of the acquisitionparameters good clinical validity can be obtained with differentanalysis techniques and different volumetric T1-weighted sequences.

Automatedmethods tomeasure atrophymay be particularly usefulin clinical studies and trials. Methods such as SIENAX, used in thisanalysis, provide a cross-sectional assessment of prior atrophy, whilethe longitudinal version, SIENA, can be used to track atrophy inindividuals. Clinical trial data in MS has demonstrated that suchautomated methods improve statistical power dramatically whencompared with a semi-automated approach (Sormani et al., 2004).Specifically, SIENAwas found to improve power from 32% to 73%whencompared with a technique based on seed-growing and intensitythresholding. Nevertheless, optimal protocols are yet to be establishedfor many clinical settings. The statistical power advantage of SIENA inMS is driven by a smaller variability arising from measurement error(Sormani et al., 2004). Conceivably, factors like age and different typesof parenchymal lesions could influence these errors. For this reason,we chose an approach to analysis that allowed us to compare varianceacross different subgroups of patients. Although analysis error couldnot be measured directly, differences in variance between techniques

319M. O'Sullivan et al. / NeuroImage 43 (2008) 312–320

must be due to analysis error as the measurements are made on anidentical set of scans with the same intrinsic variability. There was noclear trend to suggest higher analysis error with any one technique inany subset of patients, and all techniques detected similar patterns ofchange in BPF related to age and lesion load. The choice of technique inpractice may well rest on other factors. For example, SIENAX providesstandard options and output, requires less computing time than SPM5and less operator intervention than BrainVisa. Conversely, SPM5 andBrainVisa segmentations may be pre-requisites to other analysesoffered by these packages, including voxel-based morphometry withSPM5 (Ashburner and Friston, 2000; Good et al., 2001).

The pattern of bias observed between SIENAX and the othertechniques is difficult to explain. The approach in SIENAX involvesscaling by a factor determined from the skull outlines; intracranialcavity volume is not measured explicitly so it is hard to determine thesource of this bias. However, it is unlikely that this would affect the useof this method in a trial setting as long as only one techniquewas usedand direct comparisons of BPF derived from different methods werenot made. For SIENAX, a narrower range of BPF values was offset bylower variance. An approximate measure of sensitivity to disease-related change, CoVchange, was found to be similar for all techniques,though this needs to be interpreted with caution given the cross-sectional context.

Although this is a cross-sectional study, assessment of brainatrophy at a single timepoint is also valuable. As pointed out by Smithet al. (2007), cross-sectional atrophy measurements provide anassessment of the “state” of the brain, and the results of thisassessment are immediate, whereas measurements of “rate” emergeonly after an interval long enough to produce an accurate measure-ment of change. Furthermore, cross-sectional “state” measurementscontain information about disease progression over a long period oftime and may be a highly appropriate measure to relate to, forexample, vascular risk factors which also act over very long periods. InCADASIL, and small vessel diseases in general, the clinical onset of thedisease is variable between individuals and much of the brain damagethat occurs may occur without overt clinical symptoms. Thereforebrain imaging – for example a “state” measurement of accruedatrophy – can provide valuable information about pathologicalprogression not available from clinical assessment. This is importantas cross-sectional measurements of BPF are strong predictors oflongitudinal atrophy, at least in AD (Smith et al., 2007). The main roleof BPF measurements in clinical trials therefore will be in providing anaccurate assessment of disease state at baseline, as this is a potentialconfounding factor. Many study designs will also require accuratemeasurements of “rate” of atrophy over short periods and for thispurpose longitudinal data will be more appropriate for selecting themost sensitive methods. Studies in MS have shown that registration-based methods – where scans from two timepoints are registered toeach other and differences assessed – are generally more sensitive(Anderson et al., 2007). Both SPM5 and SIENA (the longitudinalalgorithm closely related to SIENAX) fit this criterion and can beadapted to assess both baseline disease state and longitudinal rate ofprogression.

One reason for the strong relationship between BPF and globalcognitive function – and a reason why it is a potentially valuablemarker for clinical trials – is that it may reflect a final commonpathway of brain injury, through an accumulation of insults includinglacunar strokes and diffuse white matter damage, which bothcorrelate with BPF. A global measure has advantages for trials inencompassing and summarising the effect of a broad range ofpotential mechanisms in a measure likely to be important in termsof quality of life. Investigations of specific aspects of the disease willoften require assessment of damage at a much more localised level.Such additional research questions are one factor that might guide thechoice of technique from those presented here: for example, questionsrequiring evaluation of specific cortical gyri could evolve from the

BrainVisa approach, whereas subtle local changes could be soughtthroughout the brain in a hypothesis-free approach with voxel-basedmorphometry (from SPM5) or voxel-based analysis of atrophypatterns with SIENA. Furthermore, although lesion loads correlateless well with global cognitive measures than BPF, they may be moreimportant in specific aspects of disease, especially if lesion location istaken into account (Levy-Cooperman et al., 2008).

The number of studies unsuitable for volumetric analysis variedfrom 6 to 12%. This is slightly greater than the rate of 4–9% forunusable scans inMS trials (Miller et al., 2007). Subjects with unusablescans were, as expected, older and had poorer cognitive function.These results indicate that a rate of 10% for unusable scans should beincluded in sample size calculations for trials based on atrophy invascular cognitive impairment, as has been assumed for trials in AD(Fox et al., 2000). Both general factors and factors specific for diseaseor technique were identified. For example, the truncation of thetemporal pole seen with SIENAX in three patients is likely to be aspecific problem for CADASIL as temporal pole signal abnormality ischaracteristic of this disease (Auer et al., 2001; O'Sullivan et al., 2001).Ventricular enlargement was a problem only for SPM5, probablybecause it leads to marked deviation from the tissue priors used toconstrain segmentation and registration.

One surprising aspect of these resultswas that different approachesto registration did not lead to major discrepancies, even for the mostseverely diseased brains. We had predicted that the skull-constrainedregistration of SIENAX would be more robust than unified segmenta-tion in SPM5, because diseasewould not be expected to lead to changesin skull images, but would lead to profound changes in brain imagesand deviation from the normal priors. Volumetric measurements fromSPM5 also require a modulation step, where tissue probabilities areadjusted to account for spatial warping (Good et al., 2001), anadjustment which remains, to some extent, controversial. However,apart from the problem with ventricular enlargement in only threecases, BPF measurements from SPM5 were comparable with thosefrom other techniques across the range of ages and lesion severitiesencountered in this cohort. The techniques included in this study arenot the only tools available for free download – others that supportmanual or semi-automated measurement of the total brain volumeand intracranial volume include Slicer (www.slicer.org) or the3DVIEWNIX (http://www.mipg.upenn.edu/∼Vnews/3dviewnix_ftp.html) package – but they are in widespread use and representativeof current research practice.

In summary, this study has compared BPF measurements fromthree techniques in a large cohort of patients with CADASIL. Theresults confirm that BPF is an excellentmarker of cognitive and clinicalstatus, and that the advantage of this marker over lesion measure-ments is maintained across a variety of volumetric techniques, at leastwith the MRI sequences used in this study. It remains to be seenwhether this advantage will be maintained with data of less goodquality, or whether better techniques for evaluating lesion burden,such as regional measures (Gootjes et al., 2004; Wen and Sachdev,2004), will emerge. The results suggest that automated methods canbe applied reliably to groups of subjects with marked parenchymalabnormalities and can be applied to datasets obtained from differentscanners — a pre-requisite for multi-centre clinical trials.

Acknowledgments

MO'S was an Alexander von Humboldt Fellow in Germany and wasalso supported by the European Neurological Society and the PeelMedical Research Trust. This work was supported by PHRC grant AOR02-001 (DRC/APHP) and performed with the help of ARNEVA(Association de Recherche en Neurologie VAsculaire), Hôpital Lar-iboisiere, France, a grant from the Journées de Neurologie de LangueFrançaise, France, the Deutsche Forschungsgemeinschaft (SFB596/TPA4), and a grant from EISAI Medical Res. Inc (Germany).

320 M. O'Sullivan et al. / NeuroImage 43 (2008) 312–320

References

Altman, D.G., Bland, J.M., 1983. Measurement in medicine: the analysis of methodcomparison studies. The Statistician 32, 307–317.

Anderson, V.M., Fernando, K.T., Davies, G.R., Rashid, W., Frost, C., Fox, N.C., Miller,D.H., 2007. Cerebral atrophy measurement in clinically isolated syndromes andrelapsing remitting multiple sclerosis: a comparison of registration-basedmethods. J. Neuroimaging 17, 61–68.

Ashburner, J., Friston, K.J., 2000. Voxel-based morphometry—the methods. Neuroimage11, 805–821.

Ashburner, J., Friston, K.J., 2005. Unified segmentation. Neuroimage 26, 839–851.Auer, D.P., Putz, B., Gossl, C., Elbel, G., Gasser, T., Dichgans, M., 2001. Differential lesion

patterns in CADASIL and sporadic subcortical arteriosclerotic encephalopathy: MRimaging study with statistical parametric group comparison. Radiology 218, 443–451.

Filippi, M., Rovaris, M., Inglese, M., Barkhof, F., De, S.N., Smith, S., Comi, G., 2004.Interferon beta-1a for brain tissue loss in patients at presentation with syndromessuggestive of multiple sclerosis: a randomised, double-blind, placebo-controlledtrial. Lancet 364, 1489–1496.

Fox, N.C., Schott, J.M., 2004. Imaging cerebral atrophy: normal ageing to Alzheimer'sdisease. Lancet 363, 392–394.

Fox, N.C., Cousens, S., Scahill, R., Harvey, R.J., Rossor, M.N., 2000. Using serial registeredbrain magnetic resonance imaging to measure disease progression in Alzheimerdisease: power calculations and estimates of sample size to detect treatmenteffects. Arch. Neurol. 57, 339–344.

Good, C.D., Johnsrude, I.S., Ashburner, J., Henson, R.N., Friston, K.J., Frackowiak, R.S.,2001. A voxel-based morphometric study of ageing in 465 normal adult humanbrains. Neuroimage 14, 21–36.

Gootjes, L., Teipel, S.J., Zebuhr, Y., Schwarz, R., Leinsinger, G., Scheltens, P., Moller, H.J.,Hampel, H., 2004. Regional distribution of whitematter hyperintensities in vasculardementia, Alzheimer's disease and healthy aging. Dement. Geriatr. Cogn. Disord. 18,180–188.

Hartley, S.W., Scher, A.I., Korf, E.S., White, L.R., Launer, L.J., 2006. Analysis and validationof automated skull stripping tools: a validation study based on 296MR images fromthe Honolulu Asia aging study. Neuroimage 30, 1179–1186.

Horsfield, M.A., Rovaris, M., Rocca, M.A., Rossi, P., Benedict, R.H., Filippi, M., Bakshi, R.,2003. Whole-brain atrophy in multiple sclerosis measured by two segmentationprocesses from various MRI sequences. J. Neurol. Sci. 216, 169–177.

Jenkinson, M., Smith, S., 2001. A global optimisation method for robust affineregistration of brain images. Med. Image Anal. 5, 143–156.

Jenkinson, M., Bannister, P., Brady, M., Smith, S., 2002. Improved optimization for therobust and accurate linear registration and motion correction of brain images.Neuroimage 17, 825–841.

Jouvent, E., Viswanathan, A., Mangin, J.F., O'Sullivan, M., Guichard, J.P., Gschwendtner, A.,Cumurciuc, R., Buffon, F., Peters, N., Pachai, C., Bousser, M.G., Dichgans, M., Chabriat,H., 2007. Brain atrophy is related to lacunar lesions and tissue microstructuralchanges in CADASIL. Stroke 38, 1786–1790.

Levy-Cooperman, N., Ramirez, J., Lobaugh, N.J., Black, S.E., 2008. Misclassified tissuevolumes in Alzheimer disease patients with white matter hyperintensities:importance of lesion segmentation procedures for volumetric analysis. Stroke. 39,1134–1141.

Maclullich, A.M., Wardlaw, J.M., Ferguson, K.J., Starr, J.M., Seckl, J.R., Deary, I.J., 2004.Enlarged perivascular spaces are associated with cognitive function in healthyelderly men. J. Neurol. Neurosurg. Psychiatry 75, 1519–1523.

Mangin, J.F., 2000. Entropy minimization for automatic correction of intensitynonuniformity. IEEE Work. MMBIA. IEEE Press, Hilton Head Island, South Carolina,pp. 162–169.

Mazziotta, J., Toga, A., Evans, A., Fox, P., Lancaster, J., Zilles, K., Woods, R., Paus, T.,Simpson, G., Pike, B., Holmes, C., Collins, L., Thompson, P., MacDonald, D., Iacoboni,M., Schormann, T., Amunts, K., Palomero-Gallagher, N., Geyer, S., Parsons, L., Narr, K.,Kabani, N., Le, G.G., Boomsma, D., Cannon, T., Kawashima, R., Mazoyer, B., 2001. Aprobabilistic atlas and reference system for the human brain: InternationalConsortium for Brain Mapping (ICBM). Philos. Trans. R. Soc. Lond. B. Biol. Sci. 356,1293–1322.

Miller, D.H., 2004. Brain atrophy, interferon beta, and treatment trials in multiplesclerosis. Lancet 364, 1463–1464.

Miller, D.H., Barkhof, F., Frank, J.A., Parker, G.J., Thompson, A.J., 2002. Measurement ofatrophy in multiple sclerosis: pathological basis, methodological aspects andclinical relevance. Brain 125, 1676–1695.

Miller, D.H., Soon, D., Fernando, K.T., MacManus, D.G., Barker, G.J., Yousry, T.A., Fisher,E., O'Connor, P.W., Phillips, J.T., Polman, C.H., Kappos, L., Hutchinson, M.,Havrdova, E., Lublin, F.D., Giovannoni, G., Wajgt, A., Rudick, R., Lynn, F., Panzara,M.A., Sandrock, A.W., 2007. MRI outcomes in a placebo-controlled trial ofnatalizumab in relapsing MS. Neurology 68, 1390–1401.

O'Sullivan, M., Jarosz, J.M., Martin, R.J., Deasy, N., Powell, J.F., Markus, H.S., 2001. MRIhyperintensities of the temporal lobe and external capsule in patients withCADASIL. Neurology 56, 628–634.

O'Sullivan, M., Morris, R.G., Huckstep, B., Jones, D.K., Williams, S.C., Markus, H.S., 2004.Diffusion tensor MRI correlates with executive dysfunction in patients withischaemic leukoaraiosis. J. Neurol. Neurosurg. Psychiatry 75, 441–447.

O'Sullivan, M., Ngo, E., Viswanathan, A., Jouvent, E., Gschwendtner, A., Saemann, P.G.,Duering, M., Pachai, C., Bousser, M.G., Chabriat, H., Dichgans, M., 2007. Hippocampalvolume is an independent predictor of cognitive performance in CADASIL.Neurobiol. Aging Oct 24. [Epub ahead of print].

Pelletier, D., Garrison, K., Henry, R., 2004. Measurement of whole-brain atrophy inmultiple sclerosis. J. Neuroimaging 14, 11S–19S.

Peters, N., Holtmannspotter, M., Opherk, C., Gschwendtner, A., Herzog, J., Samann, P.,Dichgans, M., 2006. Brain volume changes in CADASIL: a serial MRI study in puresubcortical ischemic vascular disease. Neurology 66, 1517–1522.

Riviere, D., Regis, J., Cointepas, Y., Papadopoulos-Orfanos, D., Cachia, A., Mangin, J.F.,2003. A freely available Anatomist/BrainVISA package for structural morphometryof the cortical sulci. Neuroimage. 19, 934.

Smith, S.M., 2002. Fast robust automated brain extraction. Hum. Brain Mapp. 17,143–155.

Smith, S.M., Zhang, Y., Jenkinson, M., Chen, J., Matthews, P.M., Federico, A., De, S.N.,2002b. Accurate, robust, and automated longitudinal and cross-sectional brainchange analysis. Neuroimage. 17, 479–489.

Smith, S.M., Rao, A., De, S.N., Jenkinson, M., Schott, J.M., Matthews, P.M., Fox, N.C., 2007.Longitudinal and cross-sectional analysis of atrophy in Alzheimer's disease: cross-validation of BSI, SIENA and SIENAX. Neuroimage. 36, 1200–1206.

Sormani, M.P., Rovaris, M., Valsasina, P., Wolinsky, J.S., Comi, G., Filippi, M., 2004.Measurement error of two different techniques for brain atrophy assessment inmultiple sclerosis. Neurology 62, 1432–1434.

Trip, S.A., Schlottmann, P.G., Jones, S.J., Li, W.Y., Garway-Heath, D.F., Thompson, A.J.,Plant, G.T., Miller, D.H., 2006. Optic nerve atrophy and retinal nerve fibre layerthinning following optic neuritis: evidence that axonal loss is a substrate of MRI-detected atrophy. Neuroimage 31, 286–293.

Viswanathan, A., Guichard, J.P., Gschwendtner, A., Buffon, F., Cumurcuic, R., Boutron, C.,Vicaut, E., Holtmannspotter, M., Pachai, C., Bousser, M.G., Dichgans, M., Chabriat, H.,2006. Blood pressure and haemoglobin A1c are associated with microhaemorrhagein CADASIL: a two-centre cohort study. Brain 129, 2375–2383.

Wen,W., Sachdev, P.S., 2004. Extent and distribution of whitematter hyperintensities instroke patients: the Sydney Stroke study. Stroke 35, 2813–2819.