Validity and Reproducibility of Physical Activity

Questionnaires

Lisa Chasan-Taber, Sc.D. Patty S. Freedson, Ph.D.

Research Interests

Chasan-Taber• Physical activity and

pregnancy– Effect on maternal/fetal

disorders– Development &

validation of measurement tools

• Reproductive & perinatal epidemiology

• Women's health

Freedson• Children's responses to

exercise• Exercise and women's

health• Determinants of

performance and fitness • Development and

validation of objective techniques to assess habitual physical activity

Learning Objectives

1. To understand the need for evaluation of physical activity questionnaires

2. To understand the role of subjective and objective measures of physical activity in validity/reproducibility studies.

3. To understand approaches for analyzing validity/reproducibility studies.

Performance Objectives

• Students should be able to design a study to assess the validity & reproducibility of a physical activity questionnaire.

• Be able to interpret the findings from a validity/reproducibility study.

• Be able to present findings from a reproducibility/validity study.

Overview

1. Need for Evaluation of Physical Activity Questionnaires

2. Reproducibility Studies: Design, Analysis, & Presentation

3. Validity Studies: Design, Analysis, & Presentation

4. Future Directions

Associations Between Physical Activity and

Disease

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Sedentary Occasional Conditioning Exercisers Exercisers

RR

Kujala, UM et al., JAMA 279:440-444, 1998

Why are Questionnaires used to Measure Physical Activity in Epidemiologic Studies?

• Advantages– Practical for large sample sizes– Non-reactive– Tailored to specific populations and

time periods

• Disadvantages– Precision

The Need for Validation of Physical Activity Questionnaires• Physical activity is a complex behavior

difficult to measure accurately– Based upon self-report – Includes unstructured activities

• Individuals rarely make clear changes in their activity at identifiable points in time.

• Activity intensity may vary from person to person

Definition of Terms: Reproducibility

The consistency of questionnaire measurements •on more than one administration, •to the same people, •at different times.

Definition of Terms: Validity

The degree to which the questionnaire actually measures the aspect of physical activity that it was designed to measure.• Type • Frequency• Duration• Intensity

When to Perform Reproducibility/Validity Studies?

1. For new questionnaires.2. When a questionnaire is modified.3. When questionnaires will be used

in a different population according to:

– Age– Gender– Culture– Physical activity profile

Design of Reproducibility Studies

1 s t M ea su rem en t 2 n d M ea su rem en t

Specify time interval

Analysis of Reproducibility Studies

ICC

Total Activity 0.81

By Intensity

Light -Intensity Activity 0.64

Moderate -Intensity Activity 0.78

Vigorous -Intensity Activity

0.85

By Type

Occupational Activity

Recreational Activity 0.78

Household Activity 0.73

Interpretation of Results from Reproducibility Studies

High reliability

High validity

High reliability

Low validity

Figure 1. Figure 2.

Validity Studies: Choice of a Comparison Measure

• Use superior method as a comparison.

• No gold standard exists. • Use a method with different

sources of error.

Comparison Measures• Subjective

– Physical activity logs (PALs)– 24 hour recall– 7 day recall– Previous month recall– Previous year recall

• Objective– Accelerometers– Heart rate monitoring– Doubly labeled water– Direct observation

Subjective Comparison Measures- Physical Activity Logs (PALs)

Advantages and Disadvantages of PALs as Comparison Measures

• Advantages– Open-ended– Alleviate memory difficulties

• Disadvantages– Subject cooperation and motivation– May alter normal activity– Share error of self report

Computation of Summary Estimates from Subjective Measures

Frequency

1 times/day

Duration

2 hours/day

Intensity

5 METs

Total Energy Expenditure

30 MET-hrs/day

X X

Objective Comparison Measures - Accelerometers

Example Accelerometer Data

0

1000

2000

3000

4000

5000

6000

7000

8000

0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000

Time (min)

# o

f C

ou

nts

METS

Accelerometer Counts

<3METS

3-6METS

6-9METS

>9 METS

Computation of Summary Estimates from Accelerometer

Example of Summary Estimates from Accelerometer

Day Min Min Total X Counts/ Mod Vig Counts Min

1 52 130 1247890 1122

2 16 141 1181263 1238

3 14 138 1135351 1169

Design of Validity Studies

Questionnaire1st Admin.

Questionnaire2nd Admin.

ComparisonmeasureWinter

ComparisonmeasureSpring

ComparisonmeasureSummer

Comparisonmeasure

Fall

Design of a Validation Study (Continued)

• The number of administrations of the comparison method should be based upon– Questionnaire time frame– Variation in physical activity– Subject burden

Data Analysis & Presentation

Pearson Correlation Coefficients

Total Activity 0.50

By Intensity

Light 0.45

Moderate

0.55

Vigorous 0.70

By Type

Occupational 0.47

Recreational 0.60

Household 0.58

Interpretation of Results from Validity Studies

• High validity may reflect correlated error– If both omit a source of activity– If both include questions that are

misinterpreted– If both rely upon self-report

Summary• Evaluation of physical activity

questionnaires is critical• Goal is to assess typical or usual activity • A variety of comparison methods are

available• Study design should be tailored to the

specific population• Interpretation of results should be informed

Future Directions• How much physical activity (dose) is needed

for health (response)? – Intensity– Duration– Frequency

• Improvements and standardization of methods to assess and validate physical activity.

• How sensitive are self-report and objective measures in detecting change consequent to physical activity interventions?