Nan Rothrock, Ph.D. Northwestern University May 22, 2012.
-
Upload
kaitlyn-gunn -
Category
Documents
-
view
218 -
download
0
Transcript of Nan Rothrock, Ph.D. Northwestern University May 22, 2012.
Nan Rothrock, Ph.D.
Northwestern University
May 22, 2012
Problems in patient-reported outcome measures
PROMIS approach to PRO instrument development
Available PROMIS instruments Reliability, validity
PROMIS and the FDA
Many measures of same health concept
Widely varying quality
Difficult to compare and combine data
. . . across studies
. . . across conditions
Complex
Long
01
23
- 1
- 2
- 3
Questionnairewith a widerange -but low precision
12 Questionnaire
with a highprecision -but small range
National Institutes of Health, 2003
• Attack the Patient-Reported Outcome (PRO) “Tower of Babel”
• Harness modern psychometric methods
• Improve quality and interpretability of PROs
Bruegel, 1563
Nine-year commitment of NIH $80+ million investment 15 funded research sites
Methodology
Measures (Instruments)
Software
Item = question or statement a patient answers
Instrument = collection of items Legacy = existing instrument that is
“gold standard” or a commonly used and widely accepted instrument
Domain focused, not disease focused Domain = feeling, function or perception you
want to measure (e.g., anxiety, physical function, general health perceptions)
Item Banks A large collection of items measuring one
domain Any and all items can be used to provide a
score Can be administered as Computerized Adaptive
Tests (CATs) or fixed-length short forms
Literature review
Focus groups
Archival data
analysis Expert review/
consensus
Binning and
winnowing
Literacy level
analysis
Expert item revision
Cognitive interviews
Translation review
Large-scale testing
Validation studies
Calibration decisions
Intellectual property
Short formCAT
Statistical analysis
Domain Framework
Self-Reported
Health
Social Social HealthHealth
Mental Mental HealthHealth
Physical Physical HealthHealth
SymptomsSymptoms
FunctionFunction
AffectAffect
BehaviorBehavior
CognitionCognition
RelationshipsRelationships
FunctionFunction
Pain Behavior
Pain Interference
Sleep-related Impairment
Sleep Disturbance
Fatigue
Physical Function
Pain Interference
Fatigue
Upper Extremity
Mobility
Asthma Impact
Adult Pediatric/Parent Proxy
Sexual Function
Physical Health
Pain Intensity
Mental Health
Anxiety
Depression
Psychosocial Illness Impact
Anger
Applied Cognition Concerns
Anxiety
Depression
Anger
Adult
Applied Cognition Abilities
Alcohol Use
Alcohol Consequences
Alcohol Expectancies
Pediatric/Parent Proxy
Social Health
Ability to Participate in
Roles & Activities
Emotional Support
Companionship
Informational Support
Peer Relationships
Adult
Instrumental Support
Satisfaction with Roles &
Activities
Social Isolation
Pediatric/Parent Proxy
In the past 7 days … Never Rarely Some-times
Often Always
FATEXP 20
How often did you feel tired? 1
2
3
4
5
FATEXP 5
How often did you experience extreme exhaustion? 1
2
3
4
5
FATEXP 18
How often did you run out of energy? 1
2
3
4
5
FATIMP 33
How often did your fatigue limit you at work (include work at home)?
1
2
3
4
5
FATIMP 30
How often were you too tired to think clearly? 1
2
3
4
5
FATIMP 21
How often were you too tired to take a bath or shower? 1
2
3
4
5
FATIMP 40
How often did you have enough energy to exercise strenuously?
1
2
3
4
5
Reprinted with permission of the PROMIS Health Organization and the PROMIS Cooperative Group © 2007.
In the past 7 days … Never Rarely Some-times
Often Always
FATEXP 20
How often did you feel tired? 1
2
3
4
5
FATEXP 5
How often did you experience extreme exhaustion? 1
2
3
4
5
FATEXP 18
How often did you run out of energy? 1
2
3
4
5
FATIMP 33
How often did your fatigue limit you at work (include work at home)?
1
2
3
4
5
FATIMP 30
How often were you too tired to think clearly? 1
2
3
4
5
FATIMP 21
How often were you too tired to take a bath or shower? 1
2
3
4
5
FATIMP 40
How often did you have enough energy to exercise strenuously?
1
2
3
4
5
Available Universal Spanish
In Process German Portuguese Mandarin Chinese French Italian Norwegian Others – see
nihpromis.org/measures/translations
T Score Mean = 50 Standard Deviation = 10
Referenced to the US General Population
Adult GI Symptoms Self-efficacy for management of chronic disease
Pediatric Pain Behavior, Quality, Intensity Physical Activity Experience of Stress Subjective Well-being Impact of Child Illness on Family Family Belongingness
SE = 3.3rel = 0.90
SE = 2.3rel = 0.95
0
0.1
0.2
0.3
0.4
0.5
-4 -3 -2 -1 0 1 2 10 20 30 40 50 60 70
SF-36 10 items
HAQ 20 items
CAT 10 items
PROMISShort Form20 items
PROMISShort Form10 items
rheumatoid arthritis patients US general population
Err
or
Depression
CE
SD
-2 -1 0 1 2 3 4
010
2030
4050
60
r =0.84
-2 -1 0 1 2 3 4
080
Depression
Depression
CE
SD
-2 -1 0 1 2 3 4
010
2030
4050
60
010
2030
4050
60
0 100 250
CE
SD
PROMIS Depression
Importance of PRO development to include patient voices
Importance of sound measurement Confusion in selecting an instrument
because of huge array of choices
Ongoing discussions via Interagency Clinical Outcomes Assessment Working Group to qualify PROMIS Fatigue measures, attendance and presentations at PRO Consortium
FDA Approach evaluate content validity in each clinical population in which the measure may be used
PROMIS Approach there is commonality in patients’ experiences of symptoms/outcomes and their impact on QOL Need to re-validate a well-developed & valid
instrument in a target population is questionable
Magasi, S. et al (2011) Content validity of patient-reported outcome measures: Perspectives from a PROMIS meeting. Quality of Life Research
N
PROMIS FatigueSF v1.0 PROMIS FatigueMS
Mean SD Mean SD Expanded Disability Status Scale (EDSS) Mild (0-4) 83 52.2 8.2 52.5 9.2
Moderate (4.5-6.5)104 60.5 6.4 60.7 5.6
Severe (7.0-9.5) 43 60.7 8.3 60.5 8.7Fatigue Severity (0-10 NRS) None/Mild (0-1) 18 43.0 4.5 42.5 5.4 Moderate (2-4) 58 51.0 6.0 51.3 6.6
Severe (5-10)154 61.7 5.8 61.9 5.5
Vitality (item from the MOS) None/A little 52 63.8 5.3 64.2 5.4 Some 88 59.9 6.3 60.1 5.5 Quite a lot 44 55.7 6.6 56.0 6.8 Very Much 45 47.5 7.3 47.0 7.9
PROMIS FatigueSFv1.0 and PROMIS FatigueMS Scores
by Disability Status, Fatigue Severity, and Vitality Scores
Cook et al, QOLR, 2011
Supplement with targeted measures Item banking allows flexible item choice
without loss of a standard scoring base Alternative is a messy array of
contenders that fail to communicate across themselves regarding severity or result interpretation
Comparability Provide the ability to compare or combine results
from multiple studies.
Reliability and Validity Reduce response burden. Improve measurement precision.
Simplify administration via computer-based administration, scoring, and reporting
IRT-based MIDs on a T-Score scale
Multiple cross-sectional and longitudinal anchors (18)
Summarized with nonparametric statistics (median, interquartile range)
Instrument T-Score MID
Points
T-Score MID
Effect Sizes*
Raw Score MID
Points
Raw Score MID
Effect Sizes‡
Fatigue 3 -5 0.4 - 0.7 2-3 0.4 - 0.6
Pain Interference 4 -6 0.5 - 0.7 4-7 0.3 - 0.6
Physical Function 4 -6 0.5 - 0.7 3-6 0.3 - 0.6
Anxiety 3-5 0.3 - 0.6 3-4 0.5 - 0.6
Depression 3-5 0.3 - 0.5 3-4 0.4 - 0.6
*Calculated as the T-Score MID divided by the Assessment 1 T-Score standard deviation‡Calculated as the Raw Score MID divided by the Assessment 1 Raw Score standard deviation
Recommended IRT-based T-Score MIDs and Raw Score MIDs for PROMIS-Cancer Short Forms in Advanced Cancer Patients
CATT-Score MID
PointsT-Score MID Effect Sizes
Fatigue 4.5-5.0 0.57-0.63
Pain Interference 3.0-6.0 0.34-0.67
Physical Function 4.5-7.5 0.21-0.80
Anxiety 3.0-4.5 0.38-0.58
Depression 2.5-4.5 0.32-0.58
Recommended IRT-based T-Score MIDs for PROMIS-Cancer CATs