PCORI Methodology Standards: Academic Curriculum · “Americans are very measurement oriented …...
Transcript of PCORI Methodology Standards: Academic Curriculum · “Americans are very measurement oriented …...
PCORI Methodology Standards:
Academic Curriculum
© 2016 Patient-Centered Outcomes Research Institute. All Rights Reserved.
Prepared by Albert Wu, MD, MPH
Clifton Bingham, MD
Presented by Albert Wu, MD, MPH
Module 6c: Patient-Reported Outcomes:
Measurement and Measure Development
Category 2: Patient-Centeredness
Measurement
3
“Americans are very measurement oriented … we tend to emphasize what is
measured.”
—John Kenneth Galbraith
Measurement Matters
4
Measurement
The process of applying a standard
scale to something that varies in
the dimension of interest
5 Image: Goele. (2007). Via Wikimedia Commons: https://commons.wikimedia.org/wiki/File:Folding_Rules_2.jpg. Public domain. Accessed October 13, 2015.
How Do You Measure Health-Related Quality of Life?
Health-related quality of life is an
abstract concept (not directly
observable)
6 Image: Kr-val. (2008). Via Wikimedia Commons: https://commons.wikimedia.org/wiki/File:Altocumulus_se_2.jpg. Public domain. Accessed October 13, 2015.
There is no standard scale, so we need to specify what we want to measure
Assemble several indicators that approximate the concept
Create scale scores by combining responses to questions
Measuring Health-Related Quality of Life
7
Assumes concept being measured is unidimensional (e.g., depression)
Comprises questions/items to estimate the concept
Method to elicit responses = response scale
Summated Rating Scale
8
Goal is to measure a person’s self-reported mental health
Five questions rate extent of anxiety, depression, calm, and happiness
Responses are on a six-point “Likert-type” ordinal, adjective rating response scale
Example: Mental Health Inventory Five-Item Scale (MHI-5)
9
9b. Have you been a very nervous person?
9c. Have you felt so down in the dumps that nothing could cheer you up?
9d. Have you felt calm and peaceful?
9f. Have you felt downhearted and blue?
9h. Have you been a happy person?
Scoring Example: MHI-5 Questions
10
Check one answer:
1. All of the time
2. Most of the time
3. A good bit of the time
4. Some of the time
5. A little of the time
6. None of the time
Scoring Example: MHI-5 Responses
11
Write in Your Responses to These Items
12
Points (1 – 6) Recoded score
Nervous
Down in the dumps
Calm and peaceful
Downhearted and blue
Happy
Sum after reversing Items (5 – 30)
Sum of item scores
Recode and reverse
Linear conversion to 0-100 scale
Scaling and Scoring
13
Item Scoring
Items 9b, 9f, and 9c
Use precoded values
Items 9d and 9h
Require recoding before computation of the scale
score
14
All of the time 1 = 6
Most of the time 2 = 5
A good bit of the time 3 = 4
Some of the time 4 = 3
A little of the time 5 = 2
None of the time 6 = 1
Recode and reverse items
Compute sum of recoded items scores
Five items
Six response categories:
• Lowest possible score 5
• Highest 30
• Range 25
Transformation of raw summated score:
[(Raw scale score – lowest possible score) / possible score range] 100
For example, mental health score of 21
(21 – 5)/25] 100 = 64
Scale Scoring
15
64 on a 0-100 scale
Mean for the U.S. population is approximately 75, with standard deviation of
approximately 20
Your Score for MHI-5 Mental Health Scale
16
PROs are measured using indicators (questions), with responses summed to
approximate the construct being measured
Unidimensional constructs are commonly estimated using summated rating scales
Scores on PRO scales are expressed as raw scores and transformed linearly into 0-100
scales
Summary
17
Measure Development
18
1. Define what it is you are trying to measure
2. Identify relevant domains
3. Identify relevant items
4. Select response scale
5. Pilot-test
Steps in Developing a PRO Measure
19
A model of the main concepts of a domain and their relationships
Conceptual Model
20
Clinical observation
Theory
Research findings
Preliminary studies
Expert opinion
Identifying Relevant Domains
21
Clinical observation
Theory
Research findings
Expert opinion
Existing questions
Patient engagement
Devising Indicators/Items
22
How does your health condition affect your ability to function and how you feel?
Engaging Patients for Item Development
23
Exclude items that are …
Redundant
Awkward
Too idiomatic
Difficult to translate
Item Reduction
24
1. Specify what you are trying to measure
2. Identify relevant domains, using various sources including engaging patients
3. Identify items that tap those domains (item reduction)
4. Select a response scale
5. Pilot-test
Summary: Steps in Developing a PROM
25