Draft Systematic Review Supplemental File: Data Quality ...€¦ · DRAFT T able 1: Lash et al....
Transcript of Draft Systematic Review Supplemental File: Data Quality ...€¦ · DRAFT T able 1: Lash et al....
-
DRAFT
PEER REVIEW DRAFT, DO NOT CITE OR QUOTE
United States Office of Chemical Safety and Environmental Protection Agency Pollution Prevention
Risk Evaluation for Methylene Chloride
Systematic Review Supplemental File:
Data Quality Evaluation of Human Health Hazard Studies – Epidemiological Studies
CASRN: 75-09-2
H
H Cl
October, 2019, DRAFT
Cl
-
DRAFT
Table Listing1 Lash et al. 1991: Evaluation of Neurological/Behavior Outcomes . . . . . . . . . . . 32 Ott et al. 1983: Evaluation of Mortality Outcomes . . . . . . . . . . . . . . . . . . . . 63 Cherry et al. 1983: Evaluation of Neurological/Behavior Outcomes . . . . . . . . . 94 Windham et al. 2006: Evaluation of Neurological/Behavior Outcomes . . . . . . . 145 Siemiatycki 1991: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . . 186 Cantor et al. 1995: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . 217 Heineman et al. 1994: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . 248 Seidler et al. 2007: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . 289 Dosemeci et al. 1999: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . 3110 Wang et al. 2009: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . . 3411 Infante-Rivard 2005: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . 3812 Miligi et al. 2006: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . . 4213 Costantini et al. 2008: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . 4514 Radican et al. 2008: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . 4915 Radican et al. 2008: Evaluation of Respiratory Outcomes . . . . . . . . . . . . . . . 5216 Gold et al. 2010: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . . . 5517 Cocco et al. 1999: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . . 5818 Barry et al. 2011: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . . 6219 Bell et al. 1991: Evaluation of Growth (early life) and Development Outcomes . . 6520 Hearne and Pifer 1999: Evaluation of Cancer for Employees in Roll Coating
Division Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6821 Hearne and Pifer 1999: Evaluation of Cancer for All Employees Outcomes . . . . 7222 Hearne and Pifer 1999: Evaluation of Respiratory Outcomes . . . . . . . . . . . . . 7623 Hearne and Pifer 1999: Evaluation of Hematological and Immune Outcomes . . . 8024 Gibbs et al. 1996: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . . 8425 Lanes et al. 1990: Evaluation of Mortality Outcomes . . . . . . . . . . . . . . . . . . 8726 Lanes et al. 1990: Evaluation of Respiratory Outcomes . . . . . . . . . . . . . . . . . 9027 Lanes et al. 1990: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . . 9328 Lanes et al. 1990: Evaluation of Cardiovascular Outcomes . . . . . . . . . . . . . . . 9629 Lanes et al. 1993: Evaluation of Respiratory Outcomes . . . . . . . . . . . . . . . . . 9930 Lanes et al. 1993: Evaluation of Cardiovascular Outcomes . . . . . . . . . . . . . . . 10231 Lanes et al. 1993: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . . 10532 Lanes et al. 1993: Evaluation of Mortality Outcomes . . . . . . . . . . . . . . . . . . 10833 Taskinen et al. 1986: Evaluation of Reproductive Outcomes . . . . . . . . . . . . . . 11134 Soden 1993: Evaluation of Cardiovascular Outcomes . . . . . . . . . . . . . . . . . . . 11635 Soden 1993: Evaluation of Neurological/Behavior Outcomes . . . . . . . . . . . . . . 12036 Soden 1993: Evaluation of Hepatic Outcomes . . . . . . . . . . . . . . . . . . . . . . . 12437 Soden 1993: Evaluation of Hematological and Immune Outcomes . . . . . . . . . . 12838 Kalkbrenner et al. 2010: Evaluation of Neurological/Behavior Outcomes . . . . . 13239 Tomeson 2011: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . . . . 13640 Tomeson 2011: Evaluation of Cardiovascular Outcomes . . . . . . . . . . . . . . . . . 14141 Roberts et al. 2013: Evaluation of Neurological/Behavior Outcomes . . . . . . . . 14642 Christensen et al. 2013: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . 14943 Neta et al. 2012: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . . . 15244 Ruder et al. 2013: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . . 15545 Vizcaya et al. 2013: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . 15846 Morales-Suárez-Varela et al. 2013: Evaluation of Cancer Outcomes . . . . . . . . . 16147 von Ehrenstein et al. 2014: Evaluation of Neurological/Behavior Outcomes . . . . 16448 Talibov et al. 2014: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . 16749 Mattei et al. 2014: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . 17250 Brender et al. 2014: Evaluation of Cardiovascular Outcomes . . . . . . . . . . . . . 17551 Brender et al. 2014: Evaluation of Growth (early life) and Development Outcomes178
1
-
DRAFT
52 Brender et al. 2014: Evaluation of Neurological/Behavior Outcomes . . . . . . . . 18153 Silver et al. 2014: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . . 18454 Silver et al. 2014: Evaluation of Neurological/Behavior Outcomes . . . . . . . . . . 18755 Silver et al. 2014: Evaluation of Hepatic Outcomes . . . . . . . . . . . . . . . . . . . . 19056 Chaigne et al 2015: Evaluation of Hematological and Immune Outcomes . . . . . . 19357 Talbott et al 2015: Evaluation of Neurological/Behavior Outcomes . . . . . . . . . 19658 Garcia et al. 2015: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . 20059 Kumagi et al. 2016: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . 20360 Carton et al. 2017: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . 20761 Purdue et al. 2016: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . 21062 Celanese Fibers, Inc 1987: Evaluation of Hepatic Outcomes . . . . . . . . . . . . . . 21263 General Electric, Co 1990: Evaluation of Hepatic Outcomes . . . . . . . . . . . . . . 21564 General Electric, Co 1990: Evaluation of Neurological/Behavior Outcomes . . . . 21865 Gibbs 1992: Evaluation of Cancer Outcomes . . . . . . . . . . . . . . . . . . . . . . . . 22166 Gibbs 1992: Evaluation of Respiratory Outcomes . . . . . . . . . . . . . . . . . . . . . 22667 Dow Chem, Co 1976: Evaluation of Skin and Connective Tissue Outcomes . . . . 23168 Dow Chem, Co 1972: Evaluation of Skin and Connective Tissue Outcomes . . . . 23469 Ott et al. 1983: Evaluation of Hematological and Immune Outcomes . . . . . . . . 23770 Ott et al. 1983: Evaluation of Hepatic Outcomes . . . . . . . . . . . . . . . . . . . . . 241
2
-
DRAFT
T abl
e1:
Lash
etal
.19
91:
Eva
luat
ion
ofN
euro
logi
cal/
Beh
avio
rO
utco
mes
Stud
yCita
tion:
Lash,A
A;B
ecker,CE;S
o,Y;S
hore,M
(199
1).Neu
rotoxiceff
ects
ofmethy
lene
chlorid
e:Are
they
long
lastingin
human
s?Occup
ationa
lan
dEnv
ironm
entalM
edicine,
48(6),
418-42
6DataTyp
e:methy
lene
chlorid
e_retir
edworkers_de
layedverbal
mem
ory_
expo
sed-Neu
rological/Beh
avior
HERO
ID:
1350
9
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Dom
ain1:
Stud
yPa
rticipation
Metric
1:Pa
rticipan
tselection
Med
ium
×0.4
0.8
Participa
ntswereretiredairlinemecha
nics
who
had
worked
forthesame,
sing
leairlinean
dwho
were
mem
bers
ofthesamelabo
run
ion.
Both
theair-
linean
dtheun
ionprovided
inform
ationab
outthe
stud
ypo
pulation
andhistorical
occu
pation
almethy
-lene
chloride
expo
sures.
Retireesha
dto
have
worked
aminim
umof
6yearsin
oneor
moreof
14target
jobs
inorde
rto
beeligible.
Med
ical
and
demog
raph
iccriteria
forpa
rticipan
tswere
well-do
cumented
inthe
stud
yrepo
rt.
Follo
w-ups
with
survey
non-
respon
dents/no
n-pa
rticipan
tsrevealed
that
ahigh
erpe
rcentage
ofthem
hadbe
endiag
nosedwithhe
art
disease
and/
orgo
utcompa
red
tosurvey
respon
-de
nts/pa
rticipan
ts,sugg
esting
abias
toward
lower
freque
ncyof
heartdiseasein
thestud
ypo
pulation
.Add
itiona
lly,theau
thorssaythat
retirees
that
had
suffe
red
strokeswereexclud
ed,bu
tTab
le3show
sthat
4pa
rticipan
tsha
dha
dstrokes.
Metric
2:Attrit
ion
Low
×0.4
1.2
Ofthe91
potentialstud
ypa
rticipan
tswho
met
all
themed
ical
andde
mog
raph
iccriteria
andwerein-
vitedto
participatein
thefie
ldstud
y,on
ly46
(25
solvent-expo
sed,
21un
expo
sed)
participated
.The
low
participationrate
isno
texplicitly
explaine
d,al-
thou
ghalogicalassumptionmay
bethat
theseeli-
giblesubjects
electedno
tto
participate.
Metric
3:Com
paris
onGroup
Med
ium
×0.2
0.4
The
unexpo
sed
compa
rison
grou
pconsisted
ofre-
tired
airline
mecha
nics
who
had
worked
inlow-
orno
-solvent-exp
osure
jobs
(jet
engine
assembly
orroutineaircraft
mainten
ance).
The
unexpo
sed
compa
risongrou
pdiffe
redfrom
thesolvent-expo
sed
grou
pin
somede
mog
raph
iccriteria
(e.g.,ethn
icmi-
nority,Eng
lish-speaking
),bu
tmod
elswereno
tad
-justed
accordingly.
Dom
ain2:
Exp
osureCha
racterization
Con
tinu
edon
next
page
...
3
-
DRAFT
. ..c
onti
nued
from
prev
ious
page
Stud
yCita
tion:
Lash,A
A;B
ecker,CE;S
o,Y;S
hore,M
(199
1).Neu
rotoxiceff
ects
ofmethy
lene
chlorid
e:Are
they
long
lastingin
human
s?Occup
ationa
lan
dEnv
ironm
entalM
edicine,
48(6),
418-42
6DataTyp
e:methy
lene
chlorid
e_retir
edworkers_de
layedverbal
mem
ory_
expo
sed-Neu
rological/Beh
avior
HERO
ID:
1350
9
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Metric
4:Measurementof
Exp
osure
High
×0.4
0.4
Job-expo
sure
matriceswas
determ
ined
usingoccu
-pa
tion
al/h
istoricalexpo
sure
inform
ationfrom
both
the
airline
and
the
labo
run
ion.
Exp
osure
was
confi
rmed
byindu
strial
hygien
eassessments
(per-
sona
lan
darea
airmon
itoring
from
1975
throug
h19
86),
observation
ofcu
rrentworkp
lace
practices,
andinterviewswithlong
-term
employees.
Add
ition-
ally,thestud
ypo
pulation
consistedof
retirees
who
hadworkedforthesame,
sing
leairlinethroug
hout
theircareers,
andthus
theirfullworkhistorieswere
know
n.Metric
5:Exp
osurelevels
Low
×0.2
0.6
The
stud
yexam
ines
twolevelsof
expo
sure
(solvent-
expo
sedan
dun
expo
sed),b
ased
onoccu
pation
alan
dhistorical
expo
sure
inform
ationprovided
bytheair-
linean
dthelabo
run
ion.
Metric
6:Te
mpo
rality
High
×0.4
0.4
Stud
ypa
rticipan
tsthe
solvent-expo
sed
grou
pworked
inthesejobs
foran
averag
eof
11.6
years
during
thetarget
yearsof
1970
to19
84,an
dfor
anaverag
eof
23.8
yearsin
all.
Formost,
employ
-mentin
thesejobs
was
continuo
us.
Participa
nts
wereassessed
forne
urolog
ical
outcom
esinclud
ing
grip
streng
th,motor
speed,
andmem
ory.
Dom
ain3:
OutcomeAssessm
ent
Metric
7:Outcomemeasurementor
characteriz
ation
Med
ium
×0.66
71.33
Participa
ntsweretested
foranu
mbe
rof
psycho
phys-
ical
andpsycho
logicale
ndpo
ints
(gripstreng
th,sen
-sory
respon
ses,motor
speed,
short-term
visual
mem
-ory,etc.)throug
hseventest
stations
atthefie
ldsite.
Tests
weread
ministeredby
specially
traine
dexam
in-
ers(e.g.,ph
ysicians,p
sycholog
ists,n
urses)
who
were
blindto
thepa
rticipan
ts’expo
sure
grou
p.Metric
8:Rep
ortin
gBias
High
×0.33
30.33
Means
and
stan
dard
deviations
wererepo
rted
for
each
physiologicala
ndpsycho
logicaltest(along
with
pvalues).
Dom
ain4:
PotentialC
ounfou
nding/
Varia
bleCon
trol
Metric
9:Covariate
Adjustm
ent
Med
ium
×0.5
1The
statisticalan
alyses
weread
justed
only
forag
e.Metric
10:
Covariate
Cha
racterization
High
×0.25
0.25
Que
stionn
aires,
stan
dardized
tests,
and
interviews
bytheresearch
team
and/
orph
ysicians
wereused
tode
term
inepa
rticipationeligibility
andassess
po-
tentialconfou
nders.
Con
tinu
edon
next
page
...
4
-
DRAFT
. ..c
onti
nued
from
prev
ious
page
Stud
yCita
tion:
Lash,A
A;B
ecker,CE;S
o,Y;S
hore,M
(199
1).Neu
rotoxiceff
ects
ofmethy
lene
chlorid
e:Are
they
long
lastingin
human
s?Occup
ationa
lan
dEnv
ironm
entalM
edicine,
48(6),
418-42
6DataTyp
e:methy
lene
chlorid
e_retir
edworkers_de
layedverbal
mem
ory_
expo
sed-Neu
rological/Beh
avior
HERO
ID:
1350
9
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Metric
11:
Co-expo
sure
Con
foun
ding
Med
ium
×0.25
0.5
The
issue
ofpo
tentialco-exp
osures
was
notad
-dressed
inthestud
y,bu
tthere’salso
noeviden
cethat
therewereco-exp
osures
that
wereim
prop
erly
adjusted
for.
Dom
ain5:
Ana
lysis Metric
12:
Stud
yDesignan
dMetho
dsMed
ium
×0.4
0.8
Asm
alloccu
pation
alcoho
rtof
airlinemecha
nicre-
tirees
with
long
-term
methy
lene
chloride
expo
sure
was
assessed
forne
urolog
ical
outcom
es.
Datapre-
sented
asmeans/stand
ardde
viations
evalua
tedwith
t-tests.
Metric
13:
Statistic
alpo
wer
Med
ium
×0.2
0.4
The
stud
yha
dlim
ited
samplesize
(25expo
sed,
21un
expo
sed),bu
tshow
edstatistically
sign
ificant
re-
sults.
Statisticalpo
wer
appe
arssufficientto
detect
largeeff
ects.
Metric
14:
Rep
rodu
cibilityof
analyses
Med
ium
×0.2
0.4
Results
ofne
urolog
ical
assessments
wererepo
rted
asmeans/stand
ardde
viations.Ana
lysisof
effectesti-
mates
isclearlyde
scribe
d,an
dreprod
ucible.
Metric
15:
Statistic
almod
els
Med
ium
×0.2
0.4
Con
tinu
ousde
pend
entvariab
lesan
alyzed
using
t-tests.
Com
posite
scores
formem
oryan
dattention
testswerestan
dardized
forthepo
oled
grou
pof
sub-
jects.
Dom
ain6:
Other
Con
side
ratio
nsforBiomarkerSe
lectionan
dMeasurement
Metric
16:
Use
ofBiomarkerof
Exp
osure
NA
NA
Metric
17:
Effe
ctbiom
arker
NA
NA
Metric
18:
Metho
dSe
nsitivity
NA
NA
Metric
19:
Biomarkerstab
ility
NA
NA
Metric
20:
Samplecontam
ination
NA
NA
Metric
21:
Metho
drequ
irements
NA
NA
Metric
22:
Matrix
adjustment
NA
NA
Ov erallQua
lityDetermination‡
Med
ium
1.8
Extracted
Yes
?MW
F=
MetricWeigh
ting
Factor
†High=
1;Medium
=2;
Low
=3;
Una
cceptable=
4;N/A
hasno
value.
‡The
overallr
atingis
calculated
asnecessary.
EPA
may
notalwaysprovideacommentforametricthat
hasbe
encategorizedas
High.
Overallrating
=
4ifan
ymetricis
Una
cceptable
⌊ ∑ i(M
etricScore i
×MW
Fi)/
∑ jMW
Fj
⌉ 0.1(rou
ndto
thenearesttenth)
otherw
ise
,
where
High
=≥
1to
<1.
7;Med
ium
=≥
1.7to
<2.
3;Lo
w=
≥2.
3to
≤3.
0.If
thereview
erdeterm
ines
that
theoverallr
atingneedsad
justment,
theoriginal
rating
iscrossedou
tan
dan
arrow
points
tothenew
rating
.††
Thismetricmet
thecriteria
forhigh
confi
denc
eas
expe
cted
forthis
type
ofstud
y
5
-
DRAFT
T abl
e2:
Ott
etal
.19
83:
Eva
luat
ion
ofM
orta
lity
Out
com
es
Stud
yCita
tion:
Ott,MG;Sk
ory,
LK;Holde
r,BB;Bronson
,JM
;W
illiams,
PR
(198
3).
Health
evalua
tion
ofem
ployeesoccu
patio
nally
expo
sed
tomethy
lene
chlorid
eSc
andina
vian
Journa
lofW
ork,
Env
ironm
entan
dHealth
,9(Sup
pl1,Su
ppl1
),1-38
DataTyp
e:DCM_occu
patio
nal_
retrospe
ctivecoho
rt_mortality-Mortality
HERO
ID:
2914
9
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Dom
ain1:
Stud
yPa
rticipation
Metric
1:Pa
rticipan
tselection
Med
ium
×0.4
0.8
Participa
ntswereem
ployeesof
acellu
lose
triacetate
andcellu
lose
diacetatefib
erman
ufacturing
plan
tin
South
Carolinawho
had
worked
inprep
arationor
extrusionareasforat
least3mon
thsbe
tween19
54an
d19
77.A
totalo
f127
1em
ployeesfrom
thisplan
twereinclud
edin
themortalitystud
y.Con
trol
grou
ppa
rticipan
ts(948
)weredraw
nfrom
ano
n-DCM-
expo
sure
referenc
eacetatefib
erman
ufacturing
plan
tin
Virginia.
Becau
seworkassign
ments
atthis
plan
tvaried
andda
y-to-day
assign
mentrecordswereno
tkept,em
ployeeswho
workedin
compa
rableareasof
theplan
t(prepa
ration
orextrusionareas)
couldno
tbe
identifie
d.Metric
2:Attrit
ion
Low
×0.4
1.2
Attrition
was
notrepo
rted
/add
ressed
inthisrepo
rt.
Metric
3:Com
paris
onGroup
Med
ium
×0.2
0.4
Becau
seof
anab
senc
eof
workrecordsforem
ployees
ofthereferenc
eplan
t,it
could
notbe
ascertaine
dwhe
ther
participan
tsfrom
this
plan
tworkedin
sim-
ilarareas/op
erations
asthoseof
thepa
rticipan
tsfrom
theDCM-exp
osureplan
t.Add
itiona
lly,d
etails
onpa
rticipan
ts(e.g.,race,sex,
age,
etc.)
werere-
ported
lycolle
cted
,but
notrepo
rted
inthestud
yre-
port.
Dom
ain2:
Exp
osureCha
racterization
Metric
4:Measurementof
Exp
osure
High
×0.4
0.4
Eight-hrTWA
conc
entrations
andpe
akconc
entra-
tion
swerede
term
ined
forbo
thplan
ts.Persona
lair
mon
itoring(>
350samples),area
sampling(170
sam-
ples),
andshort-term
excu
rsionsampling(20sam-
ples)werepe
rformed
over
thecourse
ofa3.5-mon
thsurvey
period
inlate
1977
-early
1978
.Detailsof
the
person
alairsamplingmetho
dsarede
scribe
din
anap
pend
ixto
thestud
yrepo
rt.
Metric
5:Exp
osurelevels
Med
ium
×0.2
0.4
Occup
ationa
lDCM
expo
sure
was
catego
rized
into
threelevels
across
asufficientrang
e.
Con
tinu
edon
next
page
...
6
-
DRAFT
. ..c
onti
nued
from
prev
ious
page
Stud
yCita
tion:
Ott,MG;Sk
ory,
LK;Holde
r,BB;Bronson
,JM
;W
illiams,
PR
(198
3).
Health
evalua
tion
ofem
ployeesoccu
patio
nally
expo
sed
tomethy
lene
chlorid
eSc
andina
vian
Journa
lofW
ork,
Env
ironm
entan
dHealth
,9(Sup
pl1,Su
ppl1
),1-38
DataTyp
e:DCM_occu
patio
nal_
retrospe
ctivecoho
rt_mortality-Mortality
HERO
ID:
2914
9
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Metric
6:Te
mpo
rality
Low
×0.4
1.2
The
timefram
ebe
tween
assessed
employee
expo
-suresan
dmortalityis
unclear,
butlik
elyto
bead
e-qu
atesinc
ethisisamortalitystud
y.Cau
sesof
death
werede
term
ined
from
deathcertificates.
Mortality
withintheexpo
sedcoho
rtwas
compa
redwiththat
ofthereferenc
epo
pulation
andthegene
ralU
.S.p
op-
ulation.
Dom
ain3:
OutcomeAssessm
ent
Metric
7:Outcomemeasurementor
characteriz
ation
Med
ium
×0.66
71.33
Cau
seof
death
was
determ
ined
from
copies
death
certificatesof
death
certificatesob
tained
throug
hcompa
nyinsuranc
erecordsor
statevitalstatistics
agen
cies.The
ywere
code
dby
ano
sologist
accordingto
theRevisionof
theInternationa
lClassification
ofDiseasesin
forceat
thetimeof
death.
Mortality
withintheexpo
sedcoho
rtwas
compa
red
with
that
ofbo
ththe
correspo
nding
UnitedStates
popu
lation
andthe
referenc
epo
pulation
.Outcomes
ofapriori
interest
werede
aths
dueto
ischem
iche
art
diseasean
dmaligna
ntne
oplasm
s.Metric
8:Rep
ortin
gBias
Low
×0.33
31.0
Mortality
inform
ation
for
participan
tsis
not
re-
ported
inthis
stud
yrepo
rt.Onlymed
ianexpo
sures
arerepo
rted
.Dom
ain4:
PotentialC
ounfou
nding/
Varia
bleCon
trol
Metric
9:Covariate
Adjustm
ent
Una
ccep
table×
0.5
0.25
The
reis
nodiscussion
ofcovariatead
justments.
Metric
10:
Covariate
Cha
racterization
Una
ccep
table×
0.25
0.06
The
reis
nodiscussion
ofcovariatecharacterization
.Metric
11:
Co-expo
sure
Con
foun
ding
Low
×0.25
0.75
The
stud
yrepo
rtindicatesthat
expo
sure
toothe
rchem
icals(e.g.,metha
nol,aceton
e)was
possible
attheSo
uthCarolinaplan
t.Dom
ain5:
Ana
lysis Metric
12:
Stud
yDesignan
dMetho
dsUna
ccep
table×
0.66
70.44
Statisticala
nalyseswereno
tpresentedin
thisstud
yrepo
rt,an
dthereforeit
isdifficu
ltto
determ
ineac-
ceptab
ility
ontheba
sisof
stud
yde
sign
.Metric
13:
Statistic
alpo
wer
Med
ium
×0.33
30.67
The
stud
yinclud
ed1,27
1expo
sedem
ployeesan
d94
8un
expo
sed
employees,
thus
with
alik
ely
adequa
tesamplesize.
Metric
14:
Rep
rodu
cibilityof
analyses
Not
Rated
NA
NA
Details
ofan
alyses
aremissing
from
this
stud
yre-
port.
Con
tinu
edon
next
page
...
7
-
DRAFT
. ..c
onti
nued
from
prev
ious
page
Stud
yCita
tion:
Ott,MG;Sk
ory,
LK;Holde
r,BB;Bronson
,JM
;W
illiams,
PR
(198
3).
Health
evalua
tion
ofem
ployeesoccu
patio
nally
expo
sed
tomethy
lene
chlorid
eSc
andina
vian
Journa
lofW
ork,
Env
ironm
entan
dHealth
,9(Sup
pl1,Su
ppl1
),1-38
DataTyp
e:DCM_occu
patio
nal_
retrospe
ctivecoho
rt_mortality-Mortality
HERO
ID:
2914
9
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Metric
15:
Statistic
almod
els
Not
Rated
NA
NA
Details
onstatisticala
nalyseswereno
tpresentedin
this
stud
yrepo
rt.
Dom
ain6:
Other
Con
side
ratio
nsforBiomarkerSe
lectionan
dMeasurement
Metric
16:
Use
ofBiomarkerof
Exp
osure
NA
NA
Metric
17:
Effe
ctbiom
arker
NA
NA
Metric
18:
Metho
dSe
nsitivity
NA
NA
Metric
19:
Biomarkerstab
ility
NA
NA
Metric
20:
Samplecontam
ination
NA
NA
Metric
21:
Metho
drequ
irements
NA
NA
Metric
22:
Matrix
adjustment
NA
NA
OverallQua
lityDetermination‡
Una
ccep
table?
?2.8
Extracted
No
??Con
sistentwithou
rA
pplic
atio
nof
Syst
emat
icR
evie
win
TSC
AR
isk
Eva
luat
ions
docu
ment,ifametricforada
tasource
receives
ascoreof
Una
cceptable(score
=4),E
PAwill
determ
inethestud
yto
beun
acceptab
le.In
thiscase,o
neor
moreof
themetrics
wereratedas
unacceptab
le.Assuch,the
stud
yisconsidered
unacceptab
lean
dthescore
ispresentedsolely
toincrease
tran
sparency.
?MW
F=
MetricWeigh
ting
Factor
†High=
1;Med
ium
=2;
Low
=3;
Una
cceptable=
4;N/A
hasno
value.
‡The
overallr
atingis
calculated
asnecessary.
EPA
may
notalwaysprov
ideacommentforametricthat
hasbe
encategorizedas
High.
Overallrating
=
4ifan
ymetricis
Una
cceptable
⌊ ∑ i(M
etricScore i
×MW
Fi)/
∑ jMW
Fj
⌉ 0.1(rou
ndto
thenearesttenth)
otherw
ise
,
where
High
=≥
1to
<1.
7;Medium
=≥
1.7to
<2.
3;Lo
w=
≥2.
3to
≤3.
0.Ifthereview
erdeterm
ines
that
theoverallratingne
edsad
justment,theoriginal
rating
iscrossed
outan
dan
arrow
points
tothenew
rating
.††
Thismetricmet
thecriteria
forhigh
confi
denceas
expe
cted
forthis
type
ofstud
y
8
-
DRAFT
T abl
e3:
Che
rry
etal
.19
83:
Eva
luat
ion
ofN
euro
logi
cal/
Beh
avio
rO
utco
mes
Stud
yCita
tion:
Che
rry,
N;V
enab
les,
H;W
aldron
,HA
(198
3).The
acutebe
haviou
rale
ffectsof
solventexpo
sure
Occup
ationa
lMed
icine,
33(1),13
-18
DataTyp
e:Coh
ort_
Occup
ationa
l_DCM_Beh
avior-Neu
rological/Beh
avior
HERO
ID:
7458
2
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Dom
ain1:
Stud
yPa
rticipation
Metric
1:Pa
rticipan
tselection
Med
ium
×0.4
0.8
Somekeyelem
ents
ofthestud
yde
sign
wereno
tpre-
sented
,but
availableinform
ationindicatesalowrisk
ofselection
bias.
FactoryC
was
thefactorywith
methy
lene
chloride
expo
sure.So
mede
tails
provided
(typ
eof
shift
work,
age).The
rewerethreediffe
r-entshiftsan
dcontrols
wereselected
from
allthree
shifts.
How
ever,p
articipa
tion
ratesan
drecruitm
ent
metho
dswereno
trepo
rted
.Metric
2:Attrit
ion
Una
ccep
table×
0.4
0.16
Tab
leII
indicatesloss
ofover
half
ofthesubjects,
withno
explan
ations.Metho
dsindicatedthat
there
were56
expo
sed
subjects
and
36controlsubjects
from
factoryC,bu
tresultsin
Tab
leII
indicate
asamplesize
of44
.It
was
also
notindicatedif
the
44wereexpo
sed
subjects
only
orif
they
includ
edthecontrolsubjects.In
addition
,althou
ghthey
se-
lected
subjects
from
allthree
shifts,thereisno
infor-
mationto
indicate
that
thoseinclud
edin
theresults
werestill
from
allthreeshifts.
Metric
3:Com
paris
onGroup
Low
×0.2
0.6
Noinform
ationab
outthesimila
rity
betw
eengrou
psno
rwas
thereinform
ationto
indicate
that
controls
werematched
.Altho
ughit
was
notedthat
controls
wereselected
from
each
shift
sothat
they
workedthe
sameshift
patternas
theexpo
sedsubjects.Noothe
rinform
ation
was
provided
includ
ingif
thecontrols
wereallm
enlik
etheexpo
sedworkers.The
meanag
eof
theexpo
sedworkers
was
stated
tobe
43.8
years
old,
butno
agewas
provided
forthe36
controls.
Inad
dition
,on
ly12
ofthecontrols
werefrom
the
areasof
FactoryC
whe
retherewas
nocontactwith
solvents.
The
othe
r24
werefrom
anothe
rfactory
belong
ingto
thesamepa
rent
grou
pon
afilm
mak
ing
processidenticalt
otheexpo
sedmewitho
utsolvent
expo
sure.
Dom
ain2:
Exp
osureCha
racterization
Con
tinu
edon
next
page
...
9
-
DRAFT
. ..c
onti
nued
from
prev
ious
page
Stud
yCita
tion:
Che
rry,
N;V
enab
les,
H;W
aldron
,HA
(198
3).The
acutebe
haviou
rale
ffectsof
solventexpo
sure
Occup
ationa
lMed
icine,
33(1),13
-18
DataTyp
e:Coh
ort_
Occup
ationa
l_DCM_Beh
avior-Neu
rological/Beh
avior
HERO
ID:
7458
2
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Metric
4:Measurementof
Exp
osure
Low
×0.4
1.2
Atm
osph
eric
solventconc
entrationwas
measuredon
asub-grou
pof
men
using
individu
alpu
mps
sam-
plingon
tocharcoal
tube
s.The
solventwas
desorbed
incarbon
disulphide
andsolventconc
entrationwas
analyzed
usingga
schromatog
raph
ywitha2m
8%carbow
axcolumn.
Blood
samples
weretaken
and
measuredas
well.
The
reis
noinform
ationprov
ided
onQC
metho
dsor
recovery
ratesforthesemetho
ds.
Metric
5:Exp
osurelevels
Med
ium
×0.2
0.4
The
rang
eof
expo
sure
repo
rted
was
28-173
ppm.
Blood
solventlevelswereno
trepo
rted
.So
meresults
werepresentedas
only
expo
sedvs.un
expo
sed,
but
combine
dtheresultsforthediffe
rent
factoriesan
dcontrolsan
dwereno
tspecificformethy
lene
chloride
expo
sure
(i.e.,Fa
ctoryC)
Metric
6:Te
mpo
rality
Med
ium
×0.4
0.8
Tem
poralityisestablishe
d,bu
titisun
clearwhe
ther
expo
suresfallwithinrelevant
expo
sure
windo
wsfor
theou
tcom
eof
interest.Blood
samples
(usedin
the
analysis)wereob
tained
atthebe
ginn
ingan
den
dof
theshift.
The
seap
pear
tobe
thesametimes
that
theou
tcom
ewas
tested
.So
althou
ghthesub-
jectslik
elyworkedarou
ndmethy
lene
chloride
prior
totheou
tcom
es,thereis
noten
ough
inform
ation
provided
onho
wlong
orwhe
nan
dmeasurements
weremad
eat
thesametimeas
theou
tcom
e.How
-ever,thestud
yau
thorsap
pear
tobe
look
ingat
the
acuteeff
ects
indicating
that
thetimingmay
beap
-prop
riate.
Dom
ain3:
OutcomeAssessm
ent
Metric
7:Outcomemeasurementor
characteriz
ation
Med
ium
×0.66
71.33
Three
testswerecompleted
atthebe
ginn
ing
and
theen
dof
shift
(i.e.,
visual
analog
uescales
tore-
flect
moo
d,thedigitsymbo
lsubstitution
test
from
theWescslerAdu
ltIntelligenc
eSc
ale,
andatest
ofsimplereaction
time.
Visua
lana
logu
escales
areself-
repo
rted
rating
scales
that
wereno
tedto
have
been
show
nto
providerelia
blean
dvalid
measure
ofmoo
d.So
mede
tails
wereprovided
ontheothe
rmeasures,
butit
isno
tclearwha
tthecriteria
beingmeasured
were.
Con
tinu
edon
next
page
...
10
-
DRAFT
. ..c
onti
nued
from
prev
ious
page
Stud
yCita
tion:
Che
rry,
N;V
enab
les,
H;W
aldron
,HA
(198
3).The
acutebe
haviou
rale
ffectsof
solventexpo
sure
Occup
ationa
lMed
icine,
33(1),13
-18
DataTyp
e:Coh
ort_
Occup
ationa
l_DCM_Beh
avior-Neu
rological/Beh
avior
HERO
ID:
7458
2
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Metric
8:Rep
ortin
gBias
Low
×0.33
31.0
Correlation
swereprov
ided
formethy
lene
chloride
and4moo
dchan
gesno
tedas
part
ofthevisual
ana-
logu
escales.
Noresultswereprovided
forsimple
reaction
timein
methy
lene
chloride
workers.
Al-
thou
ghresultswerestated
tobe
inTab
leIIIa
ndmay
have
evalua
tedmethy
lene
chloride
sepa
rate
from
the
styren
eworkers,therewas
noTab
leIIIin
there-
port
noristhereadiscussion
offin
ding
sforthistest
inmethy
lene
chloride
workers.Digit
symbo
lscores
werejust
notedto
show
nodiffe
renc
e.Dom
ain4:
PotentialC
ounfou
nding/
Varia
bleCon
trol
Metric
9:Covariate
Adjustm
ent
Low
×0.66
72
All
thesubjects
werepresum
ably
male(not
clear
that
allthe
controls
were
male)
and
subjects
inbo
thexpo
sedan
dcontrolgrou
pwereselected
from
allthreeshifts,
butno
othe
rconfou
ndingvariab
les
werediscussed.
Altho
ughsubjects
wereno
tedin
the
metho
dsto
beselected
from
allthreeshifts,
notall
subjects
appe
arto
have
been
includ
edin
thean
aly-
sisan
ditisno
tclearthat
thiswas
still
accoun
tedfor
intheresults.
Age
was
mention
edfortheexpo
sed
workers,b
utwas
notmention
edforthecontrolsub
-jects.
Metric
10:
Covariate
Cha
racterization
Not
Rated
NA
NA
N/A
becauseno
covariates
werediscussed.
Metric
11:
Co-expo
sure
Con
foun
ding
Low
×0.33
31
Co-expo
sedto
metha
nol(D
CM:m
etha
nol9:1),bu
ttheco-exp
osures
wereno
tad
justed
for.
Thisco-
expo
sure
wou
ldalso
likely
bias
resultsaw
ayfrom
thenu
ll,as
itmight
contribu
teto
effects
seen
.In
addition
,con
trolswereexpo
sedto
othe
run
specified
compo
unds
aspa
rtof
thefilm
mak
ingprocessthat
couldalso
have
contribu
tedto
resultsin
thecontrol
andmay
bias
theresultstowards
thenu
ll.Dom
ain5:
Ana
lysis Metric
12:
Stud
yDesignan
dMetho
dsMed
ium
×0.4
0.8
Stud
yde
sign
was
approp
riate.
The
stud
ywas
evalu-
atingacutene
urob
ehavioraleffe
ctsan
dwas
design
edto
test
subjects
before
andafterexpo
sure.
Italso
containe
dcontrols
that
werefrom
thesameplan
tan
dun
expo
sed,
which
wou
ldalso
help
addressif
theexpo
sure
had
achroniceff
ecton
thesubjects
(thu
sloweringtheirinitialscore)
andif
thediffe
r-en
ceswerejust
basedon
working
8ho
ursan
dno
tan
effectof
expo
sure.
Con
tinu
edon
next
page
...
11
-
DRAFT
. ..c
onti
nued
from
prev
ious
page
Stud
yCita
tion:
Che
rry,
N;V
enab
les,
H;W
aldron
,HA
(198
3).The
acutebe
haviou
rale
ffectsof
solventexpo
sure
Occup
ationa
lMed
icine,
33(1),13
-18
DataTyp
e:Coh
ort_
Occup
ationa
l_DCM_Beh
avior-Neu
rological/Beh
avior
HERO
ID:
7458
2
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Metric
13:
Statistic
alpo
wer
Med
ium
×0.2
0.4
The
numbe
rof
participan
tsaread
equa
teto
detect
aneff
ectin
the
expo
sed
popu
lation
and/
orsub-
grou
psof
thetotalpo
pulation
.The
initialstud
ygrou
pwas
56expo
sedan
d36
controls.The
results
show
ed44
subjects.
Metric
14:
Rep
rodu
cibilityof
analyses
Low
×0.2
0.6
Itisap
parent
that
theau
thorsmad
eseveralcom
par-
ison
sinclud
ingcorrelations,bu
tthespecificmeth-
odsused
tode
term
inesign
ificanc
eor
correlations
was
notprovided
.The
refore,the
ycouldno
tbe
repli-
cated.
Metric
15:
Statistic
almod
els
Low
×0.2
0.6
Itisap
parent
that
theau
thorsmad
eseveralcom
par-
ison
sinclud
ingcorrelations,bu
tthespecificmeth-
odsused
tode
term
inesign
ificanc
eor
correlations
was
notprovided
.The
refore,the
ycouldno
tbe
repli-
cated.
Dom
ain6:
Other
Con
side
ratio
nsforBiomarkerSe
lectionan
dMeasurement
Metric
16:
Use
ofBiomarkerof
Exp
osure
Low
×0.16
70.5
Methy
lene
chloride
was
repo
rted
tobe
tested
inbloo
dusingahe
adspacean
alyser
and
a2m
8%carbow
axcolumn.
Blood
carboxyh
aemog
lobincon-
centration
was
also
measuredas
notedby
thestud
yau
thorscarbon
mon
oxideis
ametab
oliteof
methy
-lene
chloride
.The
reis
noinform
ationprovided
for
QC
orrecovery
rates.
Nor
istherean
yinform
ation
onho
wthemetab
oliteinform
ationwas
includ
edifat
allintheassessmentof
methy
lene
chloride
expo
sure.
Thisis
prob
ably
notavery
accu
rate
metho
d.Metric
17:
Effe
ctbiom
arker
Not
Rated
NA
NA
Nobiom
arkerof
effectwas
measured..
Metric
18:
Metho
dSe
nsitivity
Low
×0.16
70.5
LOD/L
OQ
values
areno
tstated
Metric
19:
Biomarkerstab
ility
Low
×0.16
70.5
The
reis
noinform
ationon
thestorag
eor
stab
ility
ofthesamples
norwas
thereinform
ationprovided
onwhe
nthesamples
weretested
incompa
risonto
whe
nthey
werecolle
cted
.Metric
20:
Samplecontam
ination
Low
×0.16
70.5
The
reis
noinform
ation
abou
tthecolle
ction
and
storag
eof
thesamplein
rega
rdsto
contam
ination.
Metric
21:
Metho
drequ
irements
Low
×0.16
70.5
Sigm
aH6he
adspacean
alyser
was
used
.Metric
22:
Matrix
adjustment
Low
×0.16
70.5
The
rewas
noinform
ation
abou
tad
justments
(or
not)
forthematrix.
The
reareno
unitsprovided
soit
cann
otbe
determ
ined
iftherewas
anyad
just-
ments
orho
wtheexpo
sure
was
presented.
OverallQua
lityDetermination‡
Una
ccep
table?
?2.7
Extracted
No
Con
tinu
edon
next
page
...
12
-
DRAFT
. ..c
onti
nued
from
prev
ious
page
Stud
yCita
tion:
Che
rry,
N;V
enab
les,
H;W
aldron
,HA
(198
3).The
acutebe
haviou
rale
ffectsof
solventexpo
sure
Occup
ationa
lMed
icine,
33(1),13
-18
DataTyp
e:Coh
ort_
Occup
ationa
l_DCM_Beh
avior-Neu
rological/Beh
avior
HERO
ID:
7458
2
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
??Con
sistentwithou
rA
pplic
atio
nof
Syst
emat
icR
evie
win
TSC
AR
isk
Eva
luat
ions
docu
ment,ifametricforada
tasource
receives
ascoreof
Una
cceptable(score
=4),E
PAwill
determ
inethestud
yto
beun
acceptab
le.In
thiscase,o
neor
moreof
themetrics
wereratedas
unacceptab
le.Assuch,the
stud
yisconsidered
unacceptab
lean
dthescore
ispresentedsolely
toincrease
tran
sparency.
?MW
F=
MetricWeigh
ting
Factor
†High=
1;Med
ium
=2;
Low
=3;
Una
cceptable=
4;N/A
hasno
value.
‡The
overallr
atingis
calculated
asnecessary.
EPA
may
notalwaysprov
ideacommentforametricthat
hasbe
encategorizedas
High.
Overallrating
=
4ifan
ymetricis
Una
cceptable
⌊ ∑ i(M
etricScore i
×MW
Fi)/
∑ jMW
Fj
⌉ 0.1(rou
ndto
thenearesttenth)
otherw
ise
,
where
High
=≥
1to
<1.
7;Medium
=≥
1.7to
<2.
3;Lo
w=
≥2.
3to
≤3.
0.Ifthereview
erdeterm
ines
that
theoverallratingne
edsad
justment,theoriginal
rating
iscrossed
outan
dan
arrow
points
tothenew
rating
.††
Thismetricmet
thecriteria
forhigh
confi
denceas
expe
cted
forthis
type
ofstud
y
13
-
DRAFT
T abl
e4:
Win
dham
etal
.20
06:
Eva
luat
ion
ofN
euro
logi
cal/
Beh
avio
rO
utco
mes
Stud
yCita
tion:
Windh
am,G
C;Z
hang
,L;G
unier,R;C
roen
,LA;G
rether,J
K(200
6).Autism
spectrum
disordersinrelatio
nto
distrib
utionof
hazardou
sairpo
llutantsin
theSa
nFran
ciscoBay
area
Env
ironm
entalH
ealth
Perspe
ctives,1
14(9,9),
1438
-144
4DataTyp
e:Califo
rnia_case_control_
autis
m_DCM_OR_Q4-Neu
rological/Beh
avior
HERO
ID:
1035
22
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Dom
ain1:
Stud
yPa
rticipation
Metric
1:Pa
rticipan
tselection
Med
ium
×0.4
0.8
Cases
wereidentifie
dfrom
theCalifo
rnia
Centers
forAutism
andDevelop
mentalD
isab
ilities
Research
andEpide
miology
(CADDRE)which
draw
sinform
a-tion
onASD
byactive
surveilla
nceof
Califo
rnia
De-
partmentof
Develop
mentalSe
rvices
(DDS)
andthe
KaiserPerman
ente
Med
ical
CareProgram
.Autho
rsestimated
that
thesemetho
dscaptured
75-80%
ofcasesliv
ingin
thearea
(Croen
etal.20
02);
authors
note
that
extrem
een
dsof
thesocioecono
mic
status
werelik
elyno
twellcovered.
Cases
wereinclud
edif
they
werebo
rnin
1994
andreside
din
oneof
sixSa
nFran
ciscoBay
area
coun
ties.Con
trolswereidenti-
fiedfrom
aCalifo
rnia
1994
linkedbirth-infant
death
certificate
databa
seusingthesameinclusioncrite-
ria.
Con
trolswererand
omly
selected
andmatched
onbirthmon
than
dsex(2
to1).
Metric
2:Attrit
ion
High
×0.4
0.4
Ofthecasesidentifie
din
theda
taba
ses,
expe
rtre-
view
bythePIconfi
rmed
83.3%
ASD
diag
noses,
us-
ingthesamecriteria
forallexclusion/
inclusionby
expe
rtreview
.Exclusion
from
thecontrolpo
pula-
tion
was
minim
al(n=18
)an
dwas
sufficiently
ex-
plaine
d.Metric
3:Com
paris
onGroup
High
×0.2
0.2
The
reis
someeviden
ceof
diffe
renc
esbe
tween
the
controlsan
dcases;ho
wever,p
arentala
ndchild
char-
acteristicssuch
asrace/ethnicity,materna
led
uca-
tion
,an
dpa
rity
wereconsidered
aspo
tentialcon-
foun
ders
inthestatisticalan
alysis.
Dem
ograph
icde
tails
provided
inTab
le2.
Dom
ain2:
Exp
osureCha
racterization
Metric
4:Measurementof
Exp
osure
Med
ium
×0.4
0.8
Ann
uala
verage
conc
entrationestimates
weredraw
nfrom
EPA
’sNationa
lAir
Toxics
Assessm
ent
(U.S.EPA
;41
5230
3).Con
centration
estimates
were
availableby
census
tractfor19
96that
matched
the
geocod
edad
dressesfrom
birth
certificates.
Esti-
mates
werecalculated
bysummingconc
entrations
across
variou
ssources
(mob
ile,
point,
and
area
sources).Thisrepresents
awell-establishe
dmetho
dof
determ
iningexpo
sure
toHAPsan
dwas
assessed
consistently
across
grou
ps.
Con
tinu
edon
next
page
...
14
-
DRAFT
. ..c
onti
nued
from
prev
ious
page
Stud
yCita
tion:
Windh
am,G
C;Z
hang
,L;G
unier,R;C
roen
,LA;G
rether,J
K(200
6).Autism
spectrum
disordersinrelatio
nto
distrib
utionof
hazardou
sairpo
llutantsin
theSa
nFran
ciscoBay
area
Env
ironm
entalH
ealth
Perspe
ctives,1
14(9,9),
1438
-144
4DataTyp
e:Califo
rnia_case_control_
autis
m_DCM_OR_Q4-Neu
rological/Beh
avior
HERO
ID:
1035
22
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Metric
5:Exp
osurelevels
Medium
×0.2
0.4
Forchem
ical
specifican
alyses,q
uartile
sof
expo
sure
wereused
.The
sewerede
term
ined
byexpo
sure
dis-
tributionqu
artilesin
controls.Thisrepresents
more
than
twolevels
ofexpo
sure.Meanexpo
sureswere
0.64
-0.68
ug/m
3(D
CM),
0.60
-0.61
ug/m
3(P
erc),
and0.17
-0.19ug
/m3(T
CE).
Metric
6:Te
mpo
rality
Low
×0.4
1.2
Cases
werediag
nosed
with
Autism
Spectrum
Dis-
orde
rby
age
9(suffi
cientwindo
wfordiag
nosis).
Cases
and
controls
weredraw
nfrom
apo
pulation
ofchild
renbo
rnin
1994
;how
ever,e
xposurewas
de-
term
ined
from
census
tract-levelexpo
sure
data
for
birthad
dressfrom
1996
expo
sure
estimates
(other
option
was
1994
).It
isun
clearho
wstab
lethesees-
timates
may
befrom
year
toyear.Using
expo
sure
data
from
1996
may
notaccu
rately
capturetheex-
posure
that
occu
rred
during
gestation,
butinstead
refle
ctan
earlychild
hood
developm
entalwindo
w.
Dom
ain3:
OutcomeAssessm
ent
Metric
7:Outcomemeasurementor
characteriz
ation
High
×0.66
70.67
Cases
wereidentifie
dby
CADDRE
active
surveil-
lanc
eof
Califo
rnia
Dep
artm
entof
Develop
mental
Services
andKaiserPerman
ente
records.
Identifie
dcaseswereconfi
rmed
bytheprincipa
linv
estiga
torby
diag
nosisfrom
aqu
alified
med
ical
profession
al,q
ual-
ification
forspecialed
ucationun
deran
autism
ex-
ceptiona
lity,
orau
tistic
beha
viorsap
pearingto
meet
DSM
-IV
criteria
forASD
.Thisrepresents
awell-
establishe
dmetho
dof
determ
iningan
autism
diag
-no
sis.
Metric
8:Rep
ortin
gBias
Med
ium
×0.33
30.67
Allou
tcom
esou
tlined
intheab
stract,introdu
ction,
andmetho
dswereprov
ided
intheresults.
The
num-
berof
casesan
dcontrolswas
detaile
dforsomean
al-
yses,bu
tno
tforchem
ical-spe
cific
analyses
which
wou
ldno
tallowed
forde
taile
dextraction
ofthenu
m-
berof
cases/controls.Thisis
notexpe
cted
toha
vean
appreciableim
pact
ontheresults.
Dom
ain4:
PotentialC
ounfou
nding/
Varia
bleCon
trol
Con
tinu
edon
next
page
...
15
-
DRAFT
. ..c
onti
nued
from
prev
ious
page
Stud
yCita
tion:
Windh
am,G
C;Z
hang
,L;G
unier,R;C
roen
,LA;G
rether,J
K(200
6).Autism
spectrum
disordersinrelatio
nto
distrib
utionof
hazardou
sairpo
llutantsin
theSa
nFran
ciscoBay
area
Env
ironm
entalH
ealth
Perspe
ctives,1
14(9,9),
1438
-144
4DataTyp
e:Califo
rnia_case_control_
autis
m_DCM_OR_Q4-Neu
rological/Beh
avior
HERO
ID:
1035
22
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Metric
9:Covariate
Adjustm
ent
High
×0.5
0.5
Potential
confou
ndersinclud
edmaterna
lag
e,race,
and
educ
ation,
parity,pa
ternal
race
and
age,
low
birthweigh
t,preterm
deliv
ery,
andchild
race.The
final
mod
elsinclud
echild
race,materna
lag
e,an
dmaterna
led
ucation.
Cases
andcontrols
werebirth
mon
th-an
dsex-matched
.The
authorsstated
they
didno
tinclud
ethesetw
ovariab
lesin
thefin
almod
elas
itmad
elittlediffe
renc
e.Metric
10:
Covariate
Cha
racterization
High
×0.25
0.25
Forcontrols,d
emog
raph
icda
tawerestated
tobe
ab-
stracted
from
thebirthcertificate.Dem
ograph
icin-
form
ationforcaseswas
draw
nfrom
med
ical
orDDS
records.
The
searebo
threlia
blemetho
dsof
obtain-
ingcovariateinform
ation.
Metric
11:
Co-expo
sure
Con
foun
ding
Med
ium
×0.25
0.5
App
roximately30
hazardou
sairpo
llutants(H
APs)
wereconsidered
inthis
stud
y.The
chlorina
tedsol-
vents(P
erc,
TCE,D
CM,a
ndviny
lchloride)
tend
edto
becorrelated
witheach
othe
r.TCE
was
noted
tobe
high
lycorrelated
tometals.
Che
mical-spe
cific
analyses
didno
tcontrolfor
expo
sure
toothe
rHAPs.
Altho
ugh,
therewas
noeviden
ceof
unba
lanced
co-
expo
suresby
case
status.
Dom
ain5:
Ana
lysis Metric
12:
Stud
yDesignan
dMetho
dsMed
ium
×0.4
0.8
Acase-con
trol
stud
yde
sign
was
used
toassess
re-
lation
shipsbe
tweenexpo
sure
toHAPsdu
ring
preg-
nanc
y/earlychild
hood
andthepresen
ceof
ASD
di-
agno
sisat
age9.
Metric
13:
Statistic
alpo
wer
Med
ium
×0.2
0.4
The
rewerea
sufficientnu
mbe
rof
casesan
dcon-
trolsto
detect
aneff
ect.:28
4cases,
657controls.
The
stud
yau
thorsexplicitly
stated
they
kept
birth
mon
th-an
dsex-matched
controls
who
sematched
casesdidno
tmeetthestud
y’sdiag
nostic
criteria
inorde
rto
maintainalarger
samplesize.
Metric
14:
Rep
rodu
cibilityof
analyses
Med
ium
×0.2
0.4
The
descriptionof
thean
alysis
was
sufficient.
Cut-
points
forqu
artilesof
expo
sure
andtheproced
ure
forinclusion/
exclusionof
potentialc
onfoun
ders
was
describe
d.Metric
15:
Statistic
almod
els
Med
ium
×0.2
0.4
Odd
sratios
werecalculated
forthetw
ohigh
estqu
ar-
tilesof
expo
sure
usinglogistic
regression
.The
mod
-elsan
dde
cision
son
catego
ries
ofexpo
sure
werede
-scribe
din
detailin
themetho
ds.
Dom
ain6:
Other
Con
side
ratio
nsforBiomarkerSe
lectionan
dMeasurement
Metric
16:
Use
ofBiomarkerof
Exp
osure
NA
NA
Con
tinu
edon
next
page
...
16
-
DRAFT
. ..c
onti
nued
from
prev
ious
page
Stud
yCita
tion:
Windh
am,G
C;Z
hang
,L;G
unier,R;C
roen
,LA;G
rether,J
K(200
6).Autism
spectrum
disordersinrelatio
nto
distrib
utionof
hazardou
sairpo
llutantsin
theSa
nFran
ciscoBay
area
Env
ironm
entalH
ealth
Perspe
ctives,1
14(9,9),
1438
-144
4DataTyp
e:Califo
rnia_case_control_
autis
m_DCM_OR_Q4-Neu
rological/Beh
avior
HERO
ID:
1035
22
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Metric
17:
Effe
ctbiom
arker
NA
NA
Metric
18:
Metho
dSe
nsitivity
NA
NA
Metric
19:
Biomarkerstab
ility
NA
NA
Metric
20:
Samplecontam
ination
NA
NA
Metric
21:
Metho
drequ
irements
NA
NA
Metric
22:
Matrix
adjustment
NA
NA
Ov erallQua
lityDetermination‡
Med
ium
1.7
Extracted
Yes
?MW
F=
MetricWeigh
ting
Factor
†High=
1;Medium
=2;
Low
=3;
Una
cceptable=
4;N/A
hasno
value.
‡The
overallr
atingis
calculated
asnecessary.
EPA
may
notalwaysprovideacommentforametricthat
hasbe
encategorizedas
High.
Overallrating
=
4ifan
ymetricis
Una
cceptable
⌊ ∑ i(M
etricScore i
×MW
Fi)/
∑ jMW
Fj
⌉ 0.1(rou
ndto
thenearesttenth)
otherw
ise
,
where
High
=≥
1to
<1.
7;Med
ium
=≥
1.7to
<2.
3;Lo
w=
≥2.
3to
≤3.
0.If
thereview
erdeterm
ines
that
theoverallr
atingneedsad
justment,
theoriginal
rating
iscrossedou
tan
dan
arrow
points
tothenew
rating
.††
Thismetricmet
thecriteria
forhigh
confi
denc
eas
expe
cted
forthis
type
ofstud
y
17
-
DRAFT
T abl
e5:
Siem
iaty
cki
1991
:E
valu
atio
nof
Can
cer
Out
com
es
Stud
yCita
tion:
Siem
iatycki,J(199
1).Riskfactorsforcanc
erin
theworkp
lace
#journa
l#,#
volume#
(#issue#
),#Pa
ges#
DataTyp
e:DCM_workeran
yexpo
sure_rectal
canc
er-C
ancer
HERO
ID:
1579
54
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Dom
ain1:
Stud
yPa
rticipation
Metric
1:Pa
rticipan
tselection
High
×0.4
0.4
Of
4576
eligible
male
cases
from
the
Mon
treal
metropo
litan
area
wereascertaine
dbe
tween
1979
-19
85,3
730completed
aninterviewdu
ring
thisstud
y(initiated
in19
79as
acase-con
trol
design
).Each
canc
erwas
code
dby
theInternationa
lClassification
ofDisease
forOnc
olog
y.Of54
1eligible
popu
la-
tion
malecontrols,3
75wereinterviewed
andselected
from
rand
omdigitcalling
,the
provincial
election
of19
81,wereno
ncan
cerpa
tients
hospitalized
inthe
sameinstitutions
asthosewithcanc
er-asubg
roup
ofcontrolc
ancercasesun
relatedto
occu
pation
alex-
posure
orwith
canc
erat
anothe
rsite
deem
edno
toccu
pation
ally
relevant
was
also
interviewed
Metric
2:Attrit
ion
High
×0.4
0.4
81.5%
ofeligible
casescompleted
interviews.
72%
ofcontrols.Non
respon
sesdu
eto
refusal,de
ath,
none
xtof
kinfoun
d,pa
tientdischa
rged
,no
valid
ad-
dress,
psychiatriccases,
notran
slator,or
physician
refusal
Metric
3:Com
paris
onGroup
High
×0.2
0.2
Baselinecharacteristicswerecolle
cted
from
partic-
ipan
tsan
dad
justed
for;
casesan
dcontrols
were
simila
rin
that
they
wereselected
from
Mon
treal,
Can
ada,
betw
een35
-70yearsold,
malean
drecruited
from
1979
-198
5.Dom
ain2:
Exp
osureCha
racterization
Metric
4:Measurementof
Exp
osure
Low
×0.4
1.2
Exp
osurede
term
ined
byqu
estion
naire,
nooccu
pa-
tion
alrecords.
Che
mist-hy
gien
ists
interview
consul-
tantsto
better
grasptheworking
sof
particular
in-
dustries,o
ccup
ations
wereselected
andcode
das
low
med
ium
orhigh
conc
entrations
ofexpo
sure
toaho
stof
chem
icalsba
sedon
jobtitle
Metric
5:Exp
osurelevels
Medium
×0.2
0.4
Any
orsubstantialexpo
sure
was
assign
edto
each
job
titlean
dpa
tients
wereassign
edto
oneof
the
twocatego
ries
foran
alysis.Assignm
ents
mad
eby
achem
ist-hy
gien
ist
Metric
6:Te
mpo
rality
Low
×0.4
1.2
Cases
aged
35-70,
timesinc
efirst
expo
sure
notes-
timated
;study
was
initiatedin
1979
withexpo
sures
occu
rringbe
fore
orbe
tween19
45-197
5.Dom
ain3:
OutcomeAssessm
ent
Con
tinu
edon
next
page
...
18
-
DRAFT
. ..c
onti
nued
from
prev
ious
page
Stud
yCita
tion:
Siem
iatycki,J(199
1).Riskfactorsforcanc
erin
theworkp
lace
#journa
l#,#
volume#
(#issue#
),#Pa
ges#
DataTyp
e:DCM_workeran
yexpo
sure_rectal
canc
er-C
ancer
HERO
ID:
1579
54
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Metric
7:Outcomemeasurementor
characteriz
ation
High
×0.66
70.67
Histologicalor
autopsyconfi
rmationof
prim
arytu-
mor
site.
Metric
8:Rep
ortin
gBias
High
×0.33
30.33
ORswith90
%CIs.
Dom
ain4:
PotentialC
ounfou
nding/
Varia
bleCon
trol
Metric
9:Covariate
Adjustm
ent
High
×0.5
0.5
Foreach
associationbe
tweenoccu
pation
alexpo
sure
and
cancer
type
adjustments
weremad
einclud
edag
e,he
ight,placeof
birth,
andrace
Metric
10:
Covariate
Cha
racterization
Med
ium
×0.25
0.5
Con
foun
ders
basedon
literaturean
dqu
estion
naire
data.
Metric
11:
Co-expo
sure
Con
foun
ding
Med
ium
×0.25
0.5
Adjustm
ents
forothe
roccu
pation
alexpo
sure
type
s,sm
oking,
andalcoho
lintake
weremad
e.Dom
ain5:
Ana
lysis Metric
12:
Stud
yDesignan
dMetho
dsMed
ium
×0.4
0.8
Thisis
acase-con
trol
stud
ythat
colle
cted
canc
ertype
andlifetim
eoccu
pation
alhistoryfrom
canc
erpa
tients
tode
term
ineifoccu
pation
alhistoryeff
ected
canc
errisk
Metric
13:
Statistic
alpo
wer
Med
ium
×0.2
0.4
DCM
was
notinclud
edin
Tab
le1results,
which
in-
clud
edalla
ssociation
swhe
repo
wer
was
adequa
teto
detect
a2-fold
risk
(based
on#
participan
tsan
dat
least2%
expo
sure).
DCM
was
includ
edin
Ta-
ble2which
show
selevated
ORson
ly(irrespe
ctiveof
power
tode
tect
excess
risk).
Metric
14:
Rep
rodu
cibilityof
analyses
Med
ium
×0.2
0.4
Ana
lysiswas
fully
describe
daMan
tel-Haenszela
nal-
ysis
was
performed
toan
alyzeod
dsratios
forthe
data.
Metric
15:
Statistic
almod
els
Med
ium
×0.2
0.4
Metho
dwas
tran
sparent.
AMan
tel-Haenszela
naly-
siswas
performed
toan
alyzeod
dsratios
fortheda
ta.
p-values
werecompu
tedby
theMan
tel-Haenszelchi-
squa
retest
Dom
ain6:
Other
Con
side
ratio
nsforBiomarkerSe
lectionan
dMeasurement
Metric
16:
Use
ofBiomarkerof
Exp
osure
NA
NA
Metric
17:
Effe
ctbiom
arker
NA
NA
Metric
18:
Metho
dSe
nsitivity
NA
NA
Metric
19:
Biomarkerstab
ility
NA
NA
Metric
20:
Samplecontam
ination
NA
NA
Metric
21:
Metho
drequ
irements
NA
NA
Metric
22:
Matrix
adjustment
NA
NA
OverallQua
lityDetermination‡
Med
ium
1.7
Extracted
Yes
Con
tinu
edon
next
page
...
19
-
DRAFT
. ..c
onti
nued
from
prev
ious
page
Stud
yCita
tion:
Siem
iatycki,J(199
1).Riskfactorsforcanc
erin
theworkp
lace
#journa
l#,#
volume#
(#issue#
),#Pa
ges#
DataTyp
e:DCM_workeran
yexpo
sure_rectal
canc
er-C
ancer
HERO
ID:
1579
54
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
?MW
F=
MetricWeigh
ting
Factor
†High=
1;Medium
=2;
Low
=3;
Una
cceptable=
4;N/A
hasno
value.
‡The
overallr
atingis
calculated
asnecessary.
EPA
may
notalwaysprovideacommentforametricthat
hasbe
encategorizedas
High.
Overallrating
=
4ifan
ymetricis
Una
cceptable
⌊ ∑ i(M
etricScore i
×MW
Fi)/
∑ jMW
Fj
⌉ 0.1(rou
ndto
thenearesttenth)
otherw
ise
,
where
High
=≥
1to
<1.
7;Med
ium
=≥
1.7to
<2.
3;Lo
w=
≥2.
3to
≤3.
0.If
thereview
erdeterm
ines
that
theoverallr
atingneedsad
justment,
theoriginal
rating
iscrossedou
tan
dan
arrow
points
tothenew
rating
.††
Thismetricmet
thecriteria
forhigh
confi
denc
eas
expe
cted
forthis
type
ofstud
y
20
-
DRAFT
T abl
e6:
Can
tor
etal
.19
95:
Eva
luat
ion
ofC
ance
rO
utco
mes
Stud
yCita
tion:
Can
tor,KP;S
tewart,PA
;Brin
ton,
LA;D
osem
eci,M
(199
5).Occup
ationa
lexp
osures
andfemalebreast
canc
ermortalityin
theUnited
States
Journa
lofO
ccup
ationa
land
Env
ironm
entalM
edicine,
37(3,3),33
6-34
8DataTyp
e:DCM_breast
canc
er_occu
patio
nal_
case-con
trol_OR_black2
-Can
cer
HERO
ID:
1941
30
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Dom
ain1:
Stud
yPa
rticipation
Metric
1:Pa
rticipan
tselection
High
×0.4
0.4
Cases
werewom
enwho
sede
ath
certificateslisted
breast
canc
eras
thecauseof
death(across24
U.S.
states).
Con
trolswererand
omly
selected
from
non-
canc
erde
aths,an
dfreque
ncy-matched
forag
e,gen-
der,an
drace
(fou
rcontrols
percase).
Records
were
from
years19
84to
1989
,from
ada
taba
sesupp
orted
bytheNationa
lCan
cerInstitute,
NIO
SH,an
dthe
Nationa
lCenterforHealth
Statistics.
Cases
for
which
’hom
emaker’was
thede
sign
ated
occu
pation
wereexclud
ed,leaving29
,397
white
wom
encases,
102,95
5white
wom
encontrols,4,11
2blackwom
encases,
and14
,839
blackwom
encontrols.
Metric
2:Attrit
ion
High
×0.4
0.4
Only
casesforwhich
’hom
emaker’was
thede
sig-
nated
occu
pation
were
exclud
ed(45.1%
ofwhite
wom
encases,
31.1%
ofblackwom
encases;
51.7%
ofwhite
wom
encontrols,37
.9%
ofblackwom
encon-
trols).
Metric
3:Com
paris
onGroup
High
×0.2
0.2
Con
trolswererecruitedfrom
recordsfrom
thesame
databa
sean
dforthesametimepe
riod
ascases,an
dwerefreque
ncy-matched
forag
e,gend
er,an
drace.
Dom
ain2:
Exp
osureCha
racterization
Metric
4:Measurementof
Exp
osure
Med
ium
×0.4
0.8
Exp
osurewas
estimated
usingajobexpo
sure
ma-
trix,ba
sedon
theassign
edoccu
pation
alcode
s,an
dde
velope
daccordingto
profession
aljudg
ementof
anindu
strial
hygien
ist,
inform
ationin
thegene
rallit-
eratureon
occu
pation
alexpo
sure,an
dNIO
SHan
dOSH
Aoccu
pation
alexpo
sure
databa
ses.
Exp
osure
prob
ability
andlevelwas
estimated
for31
occu
pa-
tion
alexpo
sure
catego
ries,of
which
DCM
expo
sure
was
one.
Scores
wereassign
edforprob
ability
and
levelof
expo
sure.The
rewereno
detaile
dem
ploy-
mentrecordsused
.Metric
5:Exp
osurelevels
Low
×0.2
0.6
Four
levels
ofexpo
sure
arepresented,
includ
ingno
expo
sure.Detailedrang
esforexpo
sure
areno
tin-
clud
edin
thepresentreferenc
e,bu
tmorede
tails
may
beavailablein
HERO
ID’s
7079
12an
d11
88.
Con
tinu
edon
next
page
...
21
-
DRAFT
. ..c
onti
nued
from
prev
ious
page
Stud
yCita
tion:
Can
tor,KP;S
tewart,PA
;Brin
ton,
LA;D
osem
eci,M
(199
5).Occup
ationa
lexp
osures
andfemalebreast
canc
ermortalityin
theUnited
States
Journa
lofO
ccup
ationa
land
Env
ironm
entalM
edicine,
37(3,3),33
6-34
8DataTyp
e:DCM_breast
canc
er_occu
patio
nal_
case-con
trol_OR_black2
-Can
cer
HERO
ID:
1941
30
Dom
ain
Metric
Rating†
MW
F?
Score
Com
ments
††
Metric
6:Te
mpo
rality
Med
ium
×0.4
0.8
Exp
osureislik
elyto
have
occu
rred
priorto
theou
t-come,
buttheexacttimelineof
occu
pation
alexpo
-suresin
relation
toou
tcom
eisn’tclear.
Dom
ain3:
OutcomeAssessm
ent
Metric
7:Outcomemeasurementor
characteriz
ation
High
×0.66
70.67
Outcomewas
assessed
from
causes
ofde
ath
listed
onoffi
cial
deathcertificates.
Mortalityfrom
breast
canc
erwas
determ
ined
usingtheun
derlying
causeof
death(ICD-9,code
174)
listedon
thede
athcertifi-
cate.
Metric
8:Rep
ortin
gBias
High
×0.33
30.33
One
outcom
e(breastcanc
er)was
assessed
,an
dis
approp
riately
identifie
din
thestud
yrepo
rt.
The
numbe
rsof
casesan
dcontrols
includ
edin
theas-
sessmentarealso
repo
rted
.Dom
ain4:
PotentialC
ounfou
nding/
Varia
bleCon
trol
Metric
9:Covariate
Adjustm
ent
High
×0.5
0.5
Ana
lysesweread
justed
forag
eat
timeof
death,
and/
orsocioecono
mic
class.
Results
werestratifie
dby
race.
Metric
10:
Covariate
Cha
racterization
Med
ium
×0.25
0.5
The
assign
mentof
SESwas
describe
din
thecu
rrent
referenc
eas
theSE
Sstatus
implied
bytheusua
loccu
pation
listed
foran
individu
al.
Thisis
nota
well-establishe
dmetho
d,bu
tthereisno
eviden
ceto
sugg
estthat
itis
notavalid
metho
d.Metric
11:
Co-expo
sure
Con
foun
ding
Low
×0.25
0.75
The
stud
yau
thorsdiscusspo
tentialfor
"overlap
ping
expo
sures"
and
statethis
asa
limitation.
ofthe
stud
yDom
ain5:
Ana
lysis Metric
12:
Stud
yDesignan
dMetho
dsMed
ium
×0.4
0.8
Thiscase-con
trol
stud
ycalculates
odds
ratios
and
95%
confi
denc
eintervalsforprob
ability
andlevelo
fexpo
sure
toDCM
amon
gbreast
canc
erdeaths
across
24states,from
1984
to19
89.The
design
isap
pro-
priate
forinvestigatingtheeff
ects
ofDCM
onbreast
canc
ermortality.
Metric
13:
Statistic
alpo
w