NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo)...
Transcript of NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo)...
![Page 1: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/1.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
NIH Toolbox Cognition Battery Normative Standards across Administration Platforms
Sensitivity and Validity in HIV infection
Kaitlin B. Casaletto, PhDAssistant Professor
University of California, San Francisco
![Page 2: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/2.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
Thank you!!!
Coauthors:
Anya Umlauf, M.S.Ian Abramson, PhDDonald Franklin, BA
Bob Heaton, PhD
![Page 3: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/3.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
History of the NIH Toolbox-Cognitive Battery: Platform Development
2006-2012: NIH Toolbox Development and Normative Study (desktop-based)
2012: Finalized Toolbox (web-based) released to research community
2015: Revised Cognitive Battery normative standards released (based on desktop version)
2016: iPad-based Toolbox released
OUR QUESTION:Can the normative standards be applied across administration
platforms?
![Page 4: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/4.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
To Norm, or not to Norm?
Quantify change (i.e., “impairment”) cross-sectionally
ClinicIndividual level
ResearchNo or non-demographically matched control groupRemove effects of demographics (reduce variance, covariates)
![Page 5: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/5.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
Development of Normative Standards
6,859 NIHTB Normative Study
English administration
Spanish administration
Adults (18-85yo) Children (3-17yo)
Race/Ethnicity (White/Asian, Black,
Hispanic)
Race/Ethnicity(White/Asian, Black, Hispanic, Multiracial)
Adults (18-85yo) Children (3-17yo)
Examine demographicsNormalize scoresPolynomic regression modelsSmoothing to achieve variance homogeneityExtract residuals for adjusted T-
scoresTest, confirm, validate…
![Page 6: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/6.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
Importance of Test Invariance for Normative Standards
Change to what goes into a test, change what comes out of that test!
“Never change standardized test stimuli.” –V. Malcarne, PhD
If a new score ≠ old score, norms won’t workExample: DCCS T-score = 50 + 10 * {[(DCCS scaled score) − (12.92 −
9.84*(age/100) + 1.45*(edu/10) + 0.30*male)] / [1.78 –0.63*(age/100) + 0.16*(edu/10) + 0.32*male]} / 1.28
![Page 7: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/7.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
Aims
1. Determine comparability of NIH Toolbox-CB in detecting HIV-associated neurocognitive disorders (HAND) across platforms
2. Validate the NIH Toolbox-CB in a known neurocognitive disease, HIV-associated neurocognitive disorders (HAND)
![Page 8: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/8.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
Why HIV-associated neurocognitive disorders (HAND)?
~30-50% of HIV+ individuals
Mild-to-moderate fronto-striatal patternWeaknesses: processing speed, executive functions, memory retrievalStrengths: language processingHeterogeneousLimited floor/ceiling effects
Pragmatics… HNRP
![Page 9: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/9.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
Participants: n=83 iPad and n=530 web-based NIHTB-CB assessments
Note: *p<0.05; “gold-standard” = HNRP comprehensive neuropsychological battery
iPad WebHIV+ HIV- Cohen’s d HIV+ HIV- Cohen’s d
N 57 26 404 126
Age, y 52.3 (13.7) 52.3 (13.3) 0.001 50.6 (12.3) 48.5 (15.5) 0.16
Edu, y 14.0 (2.6) 14.7 (2.9) 0.26 13.9 (2.5) 14.2 (2.4) 0.12
Sex, % M (n) 86% (49) 69.2% (18) 0.43 88.6% (358) 61.1% (77) 0.75
RaceWhiteBlackHispanic
61.4% (35)21.2% (12)17.5%)
69.2% (18)30.8% (8)0%
0.0261.1% (247)21.5% (87)17.3% (70)
55.6% (70)26.2% (33)18.3% (23)
0.12
WRAT-4 Reading 102.1 (10.9) 104 (13.5) 0.16 101.0 (14.0) 105.0 (13.7) 0.29*
“Gold-standard” Impaired, %
43.9% (25) 26.9% (7) 0.35 47.5% (192) 14.3% (18) 0.71
![Page 10: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/10.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
Equivalency smoothing helps…
Web conversion iPad conversion
Equivalency study: new individuals ”matched” to those in Norming Study and administered both web and iPad
1. C
onve
rsio
n fo
rmul
a ex
trac
ted
2. C
onve
rsio
n fo
rmul
a ap
plie
d to
HN
RP
![Page 11: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/11.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
DCCS Flanker List Sort Pic SeqMemory
Pattern Comp Vocab Reading
Coh
en’s
d
Cohen’s d: iPad > Web
Raw Equiv. Smooth (raw) T-score (demo adj.)
Equivalency smoothing usually helps…N=158 iPad vs. Web matched on age (±5yrs), edu (±2yrs), sex, race/ethnicity, HIV serostatus
* *
+
**
+
![Page 12: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/12.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
Normative standards control for demographics
00.020.040.060.080.1
0.120.14
R2
Age EffectsUnadjusted Fully-adjusted
0
0.05
0.1
0.15
0.2
0.25
R2
Education Effects
00.020.040.060.080.1
0.120.14
DCCS Flanker List sort Pic SeqMemory
PatternComp
Reading Vocab
R2
Race/Ethnicity Effects
![Page 13: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/13.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
Norm-adjusted NIHTB-CB Fluid Scores are sensitive to HIV-infection, across platforms
0
0.1
0.2
0.3
0.4
0.5
0.6
DCCS Flanker List Sort Pic SeqMemory
PatternComp
Fluid Reading Vocab Crystal
Coh
en’s
d
Web-based (n=530) iPad-based (n=83)
*
*
* *
*
*p<0.05
Cohen’s d: HIV+ < HIV-
*
*
![Page 14: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/14.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
Web and iPad, combined
NIHTB-CB is sensitive to “gold standard” HAND
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
DCCS Flanker List Sort Pic SeqMemory
PatternComp
Fluid Reading Vocab Crystal
Coh
en’s
d
All HIV+: no HAND > HAND
*all bars p’s<0.05
![Page 15: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/15.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
WRAT-4 and NIHTB-CB Crystallized scores
HIV+: r=0.83**HIV-: r=0.82**
![Page 16: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/16.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
NIHTB-CB is sensitive to HAND
Normative standards can be statistically adjusted……not ideal…work in progress
NIHTB-CB Fluid scores can detect HIV+ differenceShow effect with gold standard, comprehensive battery
NIHTB-CB Crystallized scores track with gold standard (WRAT4)Consider Crystallized based cut-offs to define Fluid “impairment”
![Page 17: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/17.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
Lessons learned
1. Tests need to be normed with the platforms/programs that will be used going forward!
2. Programs or Tests should never be changed after norming
3. If/when technology substantially advances/changes, re-norming may be necessary
![Page 18: NIH Toolbox Cognition Battery Normative Standards across ...€¦ · Spanish Adults (18 -85yo) Children (3 -17yo) Race/Ethnicity (White/Asian, Black, Hispanic) Race/Ethnicity (White/Asian,](https://reader033.fdocuments.in/reader033/viewer/2022060507/5f1fd437f45d3f1aa6510f7c/html5/thumbnails/18.jpg)
HIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGOHIV NEUROBEHAVIORAL RESEARCH PROGRAM | UNIVERSITY OF CALIFORNIA, SAN DIEGO
Questions?