MeasurementErrorinNutritionalEpidemiology:Impact,CurrentPracticeforAnalysis,and
OpportunitiesforImprovement
PamelaShaw
10May,2017
9th EMR-IBS,Thessaloniki
Outline
• Background
• MotivatingexamplesshowingimpactofMeasurementerror
• Regressioncalibration
• Literaturesurvey:methodologyandresults
• Conclusions
2
STRATOSTG4:MeasurementErrorandMisclassification
MEMBERSHIP• LaurenceFreedman,Gertner/IMS,Co-Chair• VictorKipnis,NCI,Co-Chair• RaymondCarroll,TexasA&MU• VeronikaDeffner,Munich,LMU• KevinDodd,NCI• PaulGustafson,U.BritishColumbia• RuthKeogh,LondonSchoolofHygiene• HelmutKuechenhoff,Munich,LMU• PamelaShaw,U.Pennsylvania• JanetTooze,WakeForestSchoolofMedicine
3
TG4Projects
1. LiteratureSurveyforhowmeasurementerrorisaddressedin4typesofepidemiologicalstudies
2. Guidancepaperfornutritionalepidemiologists
3.Guidancepaperforbiostatisticians
4
TG4LiteratureSurvey• Therehavebeenmanystatisticaladvancestoaddressinmeasurementerrorinthepastfewdecades
• TG4wasinterestedinassessingthecurrentpracticeforacknowledgingandaddressingmeasurementerrorinepidemiologic/observationalstudies– Wanttoidentifyknowledgegapsandopportunitiesforimprovement
• Weconductedaliteraturesurveyfocusedontypesofepidemiologicstudieswithexposuresthatarewellknowntobesubjecttomeasurementerror
5
Example1:ClassicalMeasurementerror
• Classicalmeasurementerror(CME)israndom,meanzeroerror
• CovariateX*withCMEcanbewrittenas:X*=X+u,whereuismean0errortermindependentofXandY
• SupposeY=𝛽0+𝛽1X+𝜀,thenregressingYonX*willestimateslope𝛽#∗≠𝛽1
• 𝛽#∗ willbeattenuatedtoward0
−4 −2 0 2 4
−1
0
1
2
3
4
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−4 −2 0 2 4
−1
0
1
2
3
4
●
●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
● ●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
𝜷𝟏∗ = 𝝀𝜷 , where
𝜆 =var(𝑋)
var 𝑋 + var(𝑢)
So 0< 𝜆 <16
Y
X
Example2:MeasuringDietaryIntake
• Measuringdietaryintakeisofinterestinepidemiologyasthereareanumberofdiseasesforwhichdietaryfactorsarethoughttobeimportantriskfactors,includingcancer,heartdiseaseanddiabetes
• Dietaryintakeisacomplexexposuretomeasure– Madeupofmanynutrientsobtainedfromavarietyoffoods– Containsday-to-dayvariability,possiblyalsotemporalvariability
• Thereareseveralprevailingdietaryassessmentmethods– Self-report:Foodfrequencyquestionnaire,24hourrecall,daily
foodrecord– Objectivebiomarkers:recoveryorconcentrationmarkers
7
MeasuringEnergyIntake
0 500 1000 1500 2000 2500 3000
0500
1000
1500
2000
2500
3000
Biomarker Energy
Visit 1
Vis
it 2
0 500 1000 1500 2000 2500 3000
0500
1000
1500
2000
2500
3000
FFQ Energy
Visit 1
Vis
it 2
8
𝜌 =.72 𝜌 =.70
EnergyIntakevsBodyMassIndexNeuhouser etalAJE2008
9
RegressionCalibration:AsimpleapproachtoadjustforME
PrenticeBiometrika 1982
• Supposetrueintake:X
• Error-pronemeasure:X*(FFQintake)
• Objectivebiomarker:X**=X+u
• PredictedX=E(X**|X*,Z)=E(X+u|X*,Z)=E(X|X*,Z)
=a1+a2X*+a3Z+a4 ZX*
Regressioncalibration:RegressoutcomeYonpredictedintake,othercovariatesZ
10
HRforUncalibratedvsCalibratedEnergyIntakePrentice,ShawetalAJE2009
11
SurveyAreasEachoffourtopicareashaditsownliteraturesearch
• Nutritionalintakecohortstudies(PamelaShaw/RuthKeogh)
• Dietaryintakepopulationsurveys(KevinDodd)
• Physicalactivitycohortstudies(JanetTooze)
• Airpollutioncohortstudies(VeronikaDeffner/HelmutKuechenhoff)
12
OverallApproach• Focusedonerror-pronevariableasexposureinanalysis
• Forcohortstudies,literaturesearchdoneintwostages– SearchA:Surveyrecentarticlestoassesshowoftenarticlesacknowledgedand/oraddressedmeasurementerror
– SearchB:Surveyrecentarticlesthatadjustedformeasurementerrortodescribemethodsincurrentpractice
• Questionnairesfilledoutforeachreviewedarticle
• Excludedreviews,cross-sectionalstudies,case-controlstudiesandmeta-analyses
• Eachtopicareaconductedaqualitycontrolreview– 20%re-reviewedbyindependentreviewer
13
NutritionalEpidemiologyCohortStudies:SurveyMethodology
• DateRangeA:Feb2014-Jun2015;B:Jan 2001-Jul2015
• Limitedsearchtothreecommondiseaseswithdietaryriskfactors:cancer,heartdiseaseanddiabetes– Restricteddaterangetofindabout50articlesfromSearchAand30
articlesfromSearchB
• SearchB:added(measurementerrorORmisclassificationtoSearchA– Notmanyarticles,sodidadditionalkeywordsearchesincluding:
(measurementerrorORmisclassification)ANDnutritionalepidemiology
• Physicalactivityandpollutioncohortmethodologysimilar,exceptreliedondaterangeandrandomsamplingtoreducenumberofarticlesreviewed
14
NumberofArticlesReviewed*SearchA SearchB
Nutritional Epidemiologycohort studies
51 27
DietaryIntakePopulationSurvey
67 N/A
Physical Activitycohort studies
30 40
AirPollutioncohort studies
50 25
* Number in table excludes articles that were identified by search terms but upon closer examination did not meet inclusion criteria
15
SearchASurveyResultsNutritional Epi Cohort
N= 51
Phys activity CohortN=30
Diet IntakeSurvey N=67
PollutionCohortN=50
Mention ME as potential problem n(%)
48 (94%) 17 (57%) 53/67 (79%) 20 (40%)
Used a method to adjust forME N (%)
5 (10%) 0 (0%) 19/67 (28%) 3 (6%)
% categorizing exposure
Any 50/51(98%) Exclusively 27/51 (53%)
Primary exposure
21/30 (70%)
Statistic of main interestN (%)
HR 45 (88%)OR 3 (6%)RR 2 (4%)
Slope 5(10%)
HR 11 (37%)OR/RR 9(30%)GLM 5 (17%)Other 5 (17%)
Mean 51 (76%)Median 28(42%)%-tiles 21(31%)Quality 31(46%)
16
MethodstoAddressMeasurementErrorNutritional Epi CohortN= 27*
Phys Activity Cohort N=40
Dietary Intake Pop. SurveyN=67
PollutionCohortN = 25
RegressionCalib. 26 (96%)SIMEX 1 (4%)Other 1 (4%)-----------------Search A: None 90%
RegressionCalib. 1(50%)
Other 1 (50%)-------------------Search A: None 95%
NCI 10(53%)Means 7(37%)ISU 1 (5%)MSM 1 (5%)------------------Search A: None 72%
Sens Analysis4 (80%)
Instr Variables1 (20%)
------------------Search A:None 94%
• Number excludes articles that were identified by search terms but upon review did not use a method to correct for error.
• Row percents do not add to 100% due to use of multiple methods.
17
OtherObservationsfromDietandPhysicalActivityCohortSurveys
• Commoninthecohortstudiestohavemultiplecovariateswitherror:eg diet+physicalactivity,smoking,and/oralcoholintake– Manyadjustforbothdiet+PA,only1articleadjustedforerrorinboth
physicalactivity(Zhangetal,AJE2014)– Errorsinsmoking/alcoholnotaddressed
• Mostcategorizedthecontinuousexposures– Impactsofcategorizinganexposuresubjecttoerrorareignored– Commonbelief:categorizationwilllowerimpactofmeasurementerrorin
theanalysis
• Mostpeoplewhomentionederrorasaproblemmadeanincomplete/incorrectclaim– Manyonlymentionedattenuationinfoundassociations– Someclaimednobiasinassociationssinceprospectivesubjectrecall– Someclaimednobiassinceinstrumentwasvalidated
18
OtherobservationsfromDietaryIntakePopulationSurveys
• Moststudies(80%)used24HRasprimaryinstrument– 31/53usedonly124HR,resthadrepeatsonatleastasubsample
– 8/31(26%)reportedpercentilessubjecttobias
• 16/31paperswith124HRmentionedthatusualintakeoradjustmentforwithin-personvariationwasneeded
• 8/11(73%)ofpapersusingmultiple24HRstoreportmedians/percentiles,usedacomplexmethod(NCI/MSM)
19
OtherObservationsfromtheAirPollutionCohortSurvey
• Statementsaboutthemeasurementerrorareoftenvague– Theoriginofthemeasurementerrorisoftennotclearlyspecified– Thesizeandtheimpactofthemeasurementerrorisoftennotstated
• Measurementerrorisoftenmentionedbutrarelyaddressedindetailorcorrected– Themajorityofthestudiesusedailyandspatiallyaggregateddata– TheoftenprevailingBerkson error(throughtemporalandspatial
aggregation)isnotoronlyinsufficientlydescribedanditsimplicationsarenotdiscussed
– Errorsoriginatingfromstayingindifferentmicroenvironmentsareoftenneglectedoronlypoorlyconsidered
• Manydifferentexposuremeasuresareanalyzedseparatelyorjointly;ahomogeneousprocedureislacking
20
Conclusions• Incohortstudies:measurementerroracknowledged,but
implicationsnotfullyunderstoodandcommonlynotaddressedinstatisticalanalysis– Veryfewusedmethodstoadjustformeasurementerror– ForPAstudies,littlemotivationtoadjustforerrorsincethenaïve
associationsaregenerallyalignedwithapriorihypotheses– Manystudieshadmultiplevariablesmeasuredw/error
• Indietaryintakepopulationsurveys:minoritycorrectedformeasurementerror– Majorityofthosethatdidapplyacorrectionmethodweretaking
advantageofsoftware(e.g.NCImethod)
• RegressioncalibrationmostcommonmethodtoaddressmeasurementerrorindietandPAstudies
21
Moreworkisneeded....
• Identifythevarioussourcesofmeasurementerror• Disseminateideasofmeasurementerrorcorrection
– Discussionofsoftwareinguidancedocuments,tutorialsinclinicaljournals,talksatepiandclinicalconferences
• Correctmisconceptions,suchas:– Randomerrorwon’tcausebiasinassociations– Attenuationistheonlypossibledirectionofbias– Categorizationreducestheeffectofmeasurementerror– Validatedquestionnairesdon’thavebias– Softwareisnotavailable
22
ReferencesRegressionCalibration• PrenticeRL.Covariatemeasurementerrorsandparameterestimationina
failuretimeregressionmodel.Biometrika.1982Aug1;69(2):331-42.• ZhengC,BeresfordSA,VanHornL,TinkerLF,ThomsonCA,Neuhouser ML,Di
C,MansonJE,Mossavar-Rahmani Y,SeguinR,ManiniT,LaCroix AZ,PrenticeRL.Simultaneousassociationoftotalenergyconsumptionandactivity-relatedenergyexpenditurewithrisksofcardiovasculardisease,cancer,anddiabetesamongpostmenopausalwomen.AmJEpidemiol.2014Sep1;180(5):526-35.
Simex• CookJandStefanski LA.Asimulationextrapolationmethodforparametric
measurementerrormodels.JournaloftheAmericanStatisticalAssociation1995;89:1314-1328.
• Küchenhoff,H,Mwalili SM,Lesaffre E.Ageneral method for dealing withmisclassification inregression:the misclassification SIMEX.Biometrics.2006;62(1):85-96.
23
References(2)IowaStateUniversityMethod(ISU)• Nusser,S.M.,Carriquiry,A.L.,Dodd,K.W.,andFuller,W.A.1996.ASemiparametricApproach
toEstimatingUsualIntakeDistributions.JournaloftheAmericanStatisticalAssociation,91:1440-1449.
MultipleSourceMethod(MSM)• Harttig U,Haubrock J,Knüppel S,BoeingH.2011TheMSMprogram:web-basedstatistics
packageforestimatingusualdietaryintakeusingtheMultipleSourceMethod.Eur JClin Nutr.65S1:S87-9
• Haubrock J,Nöthlings U,Volatier JL,Dekkers A,Ocké M,Harttig U,Illner AK,KnüppelS,Andersen LF,BoeingH;EuropeanFoodConsumptionValidationConsortium.EstimatingusualfoodintakedistributionsbyusingthemultiplesourcemethodintheEPIC-PotsdamCalibrationStudy.JNutr,141,914-20
NCIMethod• Tooze JA,Midthune D,DoddKW,FreedmanLS,Krebs-SmithSM,Subar AF,GuentherPM,
CarrollRJ,Kipnis V.Anewstatisticalmethodforestimatingtheusualintakeofepisodicallyconsumedfoodswithapplicationtotheirdistribution<image001.gif>.JAmDietAssoc 2006Oct;106(10):1575-87.
• Tooze JA,Kipnis V,Buckman DW,CarrollRJ,FreedmanLS,GuentherPM,Krebs-SmithSM,Subar AF,DoddKW.Amixed-effectsmodelapproachforestimatingthedistributionofusualintakeofnutrients:theNCImethod<image001.gif>.StatMed2010Nov30;29(27):2857-68.
24
25
DietaryIntakePopulationStudies:SurveyMethodology
• Daterange:Jan2012–May2015
• Term“Measurementerror”nottypicallyreferredtoindietaryintakesurveys– Understoodasvariancearoundusualintake– ConductedSearchAonly
26
PhysicalActivityCohortStudies:SurveyMethodology
• Daterange:Jan2012– Sep2015
• SearchA:Verybroadsearchterms:N=8760fromsearch;randomlyselectedN=610;N=51fromabstractreview
• SEARCHB:Added"measurementerror"ORmisreport*ORmisclassif*ORbiasORattenuat*ORcalibrat*– N=610fromsearch;N=86fromabstractreview
27
AirPollutionCohortStudies:SurveyMethodology
• Daterange:Jan2012– Dec2014
• SearchAbroadsearchwithin„WebofScience“:
– SearchBAdditionalkeywords: "measurementerror”,"measurementuncertainty”,misclassif*,attenuat*
– A:4595hits,B:386hits
• Afterabstractreview:A:431hits,B:32hits
• Randomselection:SearchA:50/SearchB:25
28
Top Related