SPECA Workshop on Disability Statistics, Dec 13-15, 2006 The Definition and Measurement of...
-
Upload
raymond-lambert -
Category
Documents
-
view
214 -
download
0
Transcript of SPECA Workshop on Disability Statistics, Dec 13-15, 2006 The Definition and Measurement of...
SPECA Workshop on Disability Statistics, Dec 13-15, 2006
The Definition and Measurement of Disability: The Work of the Washington Group
Jennifer Madans
National Center for Health Statistics, USA
for the Washington Group
The Washington Group on Disability Statistics
• In June of 2001, the UN International Seminar on the Measurement of Disability recommended the development of principles and standard forms for global indicators of disability to be used in censuses
• There was a broad consensus on the need for population based measures of disability for country use and for international comparisons
The Washington Group was set up to:
• Foster international cooperation in the area of health and disability statistics
• Develop disability measures suitable for censuses and surveys that will provide basic information on disability
• Untangle the web of confusing and conflicting disability estimates
Current Problem• Currently national censuses in developing
countries use one of three types of questions that provide widely differing estimates of national prevalence of disability:
• The three types of questions include• A. Generic question about the presence of a condition
• B. Generic questions about the presence in the household of a person with a disability followed by a list of impairments
• C. Checklist of impairments
Variety in National Estimates
• Developing Countries:• Cyprus – 4.0%• Uganda – 1.2%• Nigeria – 0.5%
• Developed Countries:• United Kingdom – 12.5%
Washington Group: Objectives
• Develop a small set/s of general disability measures
• Recommend extended set/s of items to measure disability as components of population surveys / supplements
• Address methodological issues associated with disability measurement
Preliminary work to meet objectives
• Clarify the purpose of data collection in order to identify appropriate measures
• Understand choices being made when time, expenses and respondent burden limit number of questions
Moving from Concept to Definition to Measurement: The Conceptual Model
ICF selected as the conceptual model:• Common point of reference • Common vocabulary• Does not provide an operational
definition or a way to measure the concepts
Moving from Concept to Definition to Measurement: The Definitional Paradox• There is no single operational definition of disability• Different operational definitions lead to different
estimates• The question you are trying to answer (the purpose)
will determine which definition to use • Need to understand the choices that are being
made when a purpose and a definition are chosen• Need to understand the choices that are being
made when time, expenses and respondent burden limit number of questions
Purpose of Data Collection
3 major classes of purposes at aggregate level
• Service Provision• Monitoring functioning in the population• Assess equalization of opportunities
2 criteria for selection of a purpose• Relevance—particularly for policy makers
and program officials• Feasibility
Purpose: Service provision
• Seeks to identify those with specific needs, usually the most serious problems
• Requires detailed information about the person and the environment
• Influenced by the organization and structure of service organizations within a particular culture
Purpose: Monitoring functioning in the population
• Seeks to identify all those with activity or participation limitation
• Response comparability problematic since participation is culturally and environmentally determined
0
5
1 0
1 5
2 0
2 5
Pro
po
rtio
n (
%)
2 0 0 5 2 0 1 0 2 0 1 5 2 0 2 0 2 0 2 5
Population reporting work limitation
Purpose: Equalization of opportunities
• Seeks to identify all those at greater risk than the general population for limitations in activity or participation
• Disability as a demographic
0
3 0
6 0
9 0
Pro
po
rtio
n (
%)
N o n d is a b le d D is a b le d
% Employed
Applying criteria to select purpose• Service provision:
• Level of detail necessary not feasible in a census format
• Nature of service provision varies across cultures
• Monitoring functioning: • Response comparability problematic since participation
is culturally and environmentally determined
• Assessing equalization of opportunities:• If we conceive of disability toward the most basic
elements of activity, without tying it to participation, we limit the number / types of questions thus enhancing feasibility
Locating Risk in the ICF ModelHealth Condition
Body Functions & Structure Participation
Environmental Factors
Personal Factors
Source: ICF, WHO, 2001
?
Moving from Concept to Definition to Measurement: Measurement of equalization of opportunities
• Locate the definition of disability at the most basic level of activity/participation in core domains
• This level is associated with the ability or inability to carry out basic bodily operations at the level of the whole person (i.e. walking, climbing stairs, lifting packages, seeing a friend across the room)
Benefits of this approach
• Development of a demographic means of understanding disability (can compare persons with and without disability)
• Connection between disability and participation can be made during data analysis
• Effectiveness of programs / policies to promote full participation can be monitored
Possible types of questions
• Questions that measure functioning in various domains such as mobility, cognition, sensory, etc.
• A qualifier would need to ascertain that the action was accomplished without human or mechanical assistance
Possible question options• Mobility
• Walking• Climbing stairs• Bending or stooping• Reaching or lifting• Using hands
• Sensory• Seeing• Hearing
• Communicating• Understanding• Speaking
• Cognitive functions• Learning
• Remembering
• Making decisions
• Concentrating• Emotional functioning
• Interpersonal interactions
• Psychological well-being
Criteria for inclusion of domains
• Cross cultural comparability• Suitability for self-report• Parsimony• Validity across various methodological
modes
WG draft questions for censuses (short disability measure)
Core Questions:1) Do you have difficulty seeing even if wearing glasses?2) Do you have difficulty hearing even if using a hearing
aid?3) Do you have difficulty walking or climbing stairs?4) Do you have difficulty remembering or concentrating?
Additional Questions:5) Do you have difficulty with (self-care such as) washing
all over or dressing?6) Because of a physical, mental, or emotional health
condition, do you have difficulty communicating (for example understanding others or others understanding you)?
Response categories: No - no difficulty; Yes – some difficulty; Yes – a lot of difficulty; Cannot do at all
Objectives
• Identify persons with similar types and degree of limitations in basic activities regardless of nationality or culture
• Represent the majority (but not all) persons with limitations in basic activities in any one nation
• Represent the most commonly occurring limitations in basic activities within any country
• Capture persons with similar problems across countries
Intended use of data
• Development of a demographic means of understanding disability: Can compare persons with and without disability on levels of participation in employment, education, or family life to see if persons with disability have achieved social inclusion
• Monitor effectiveness of programs / policies to promote full participation
• Monitor prevalence trends for persons with limitations in specific basic activity domains
Limitations• One set of measures will not satisfy
multiple needs for disability data• Excluded populations:
• Very young children• Institutionalized population• Homeless• Floating populations
• It is not our purpose to:• identify every person with a disability within every
community • replicate a population evaluated across a wider
range of domains that would be possible with other forms of data collection
Constraints of the census format
• Limited space• Indicated domains reflect types of
questions that would identify largest population for this purpose and those most culturally comparable
• Allows some flexibility for additional culturally relevant questions for specific countries
Pre-test implementation• Implementation protocols developed:
• Objectives and evaluation plan for field / cognitive testing
• Cognitive test plan• Translation• Enumerator training• Sample design issues• Plan for tabulation, analysis and report writing
• Two regional workshops successfully implemented• June 20-22, 2005 / Nairobi, Kenya• September 19-20, 2005 / Rio de Janeiro, Brazil
Standardized testing • 15 countries participating
• 13 funded via World Bank grant, 2 self-funded
• Cognitive tests in 12 countries• Congo, Egypt, Gambia, India, Kenya, Lesotho,
Mauritius, Philippines, Uganda, Mexico, Tanzania, Vietnam
• Field tests in 2 countries• Gambia, Vietnam
• Combined cognitive/field test in 3 countries• Argentina, Brazil, Paraguay
Pre-test objectives
To determine if:
1) the questions are being interpreted as intended by the developers in that they are capturing the important aspects of the functional domains selected and
2) the questions are interpreted consistently across countries.
Pre-test evaluationEvaluation
1) Validity: a) Content validity: How well WG question set
compares with expanded disability measures
b) Criterion related validity: How well individual WG questions compare to relevant similar concept in a comparison measure
c) Face validity: Does the measure ‘look to be valid’?
2) Reliability: test/re-test
Cognitive test
Cognitive test• Objective: to determine if questions are being
interpreted as intended and if interpretation is consistent across countries• WG cognitive test more structured than usual• Ensures a greater level of standardization across
test sites• Understand how the response mechanisms
operate in the different countries
Cognitive test
Components of cognitive test• Purposive samples• Asked WG questions and series of detailed
questions in the same domains• Traditional cognitive probes• Collected information on response process• Analyzed problematic response patterns• Interviewer report on problems respondent had
with questions
Detailed questions on functioning (example, cognition)
WG question: • Do you have difficulty remembering or concentrating?
Detailed questions:• Do you have difficulty remembering the names of people or places?• Do you have difficulty remembering appointments? • Do you have difficulty remembering how to get to familiar places?
• Do you have difficulty remembering important tasks, like taking medications or paying bills?
• Do you have difficulty concentrating on doing something for ten minutes?
• Do you have difficulty learning a new task, for example, learning how to get to a new place?
• Do you have difficulty finding solutions to problems in day to day life?
Information on response process
• Questions to interviewers• Need to repeat any part of the question?• Any difficulty in using response options?• Did respondents ask for clarification or
qualify their answers?• Follow-up probes to understand response
process
Correlates of cognitive difficulty
Unadjusted ORs
95% confidence interval
Concerned 11.18*** 7.34 – 17.05
Learn new task 7.93*** 4.95 – 12.70
Problem solving 7.22*** 4.59 – 11.35
Correlates of cognitive difficulty
Difficulties remembering
Unadjusted ORs
95% confidence interval
People or places
9.12*** 5.85 – 14.23
Appointments 6.55*** 4.36 – 9.82
Finding places 5.20*** 3.43 – 7.86
Important tasks 6.40*** 4.27 – 9.60
Socio- demographic correlates of cognitive difficulty
Unadjusted ORs
95% confidence interval
Gender
Male 1.04 (non-significant) 0.70 – 1.54
Age
35-54
55 +
1.05 (non-significant)
2.05***0.64 – 1.72
1.26 – 3.33
Overall, questions worked well
• Well understood• Responses consistent• Problem areas
• Glasses clause – WG is revisiting but solution has a lot to do with translation protocol
• Communication Question – WG will revisit to see if question can be simplified
• Introduction to the question set
Field test
• Conditions closely approximate how final study will be done
• See how WG core questions function in different countries
• Useful to compare WG set to a larger set of more detailed questions to determine whether the same population is identified by each set
Field test
• Undertaken in Gambia, Vietnam, and South Africa
• Checking for internal consistency• Results comparable to changes seen in other
countries with move towards functional approach to measuring disability
Field test: Analysis• Constructed three thresholds for disability based on
WG census question: • D1=1 if any difficulty• D2=1 if a lot• D3=1 if unable to do
• Constructed three thresholds for disability based on extended questions: • ED1• ED2• ED3
• What was the D measure picking up compared to the ED measures?
Example: Mobility question
• WG question: • Do you have difficulty walking or climbing steps?
• D1=1 if “some difficulty”, “a lot of difficulty”, or “cannot do at all”
• D2=1 if “a lot of difficulty” or “cannot do at all”
• D3=1 if “cannot do at all”
• Extended Questions: • Do you have difficulty going outside of your home?(0)
• Do you have difficulty walking a long distance such as a kilometer (or a mile)?
• ED1=1 if “some”, “a lot” or “cannot do” for either question
• ED2=1 if “a lot” or “cannot do at all” for either question
• ED3=1 if “cannot do at all” for either question
South Africa : N=185D vs ED (2, 3 or 4 Qs)
01020304050607080
Res
pond
ents
(n)
D1 (High) D3 (Low) ED1 (high) ED3 (Low)
Disability data collection recommendations
• Do not use the word disability• Functional approach• Separate functional domains• Scaled responses• Definition of disability can be made ex post,
and hinge on domains and thresholds used
Meeting products and more information
• Executive summaries of meetings, presentations and papers posted on the Washington Group website:• http://www.cdc.gov/nchs/citygroup.htm
• Publication of key papers in a special issue of Research in Social Science and Disability