Evidence-Based Public Health TPHA 2013 Paul Campbell Erwin, MD, DrPH Professor and Department Head...

1

Evidence-Based Public HealthTPHA 2013

Paul Campbell Erwin, MD, DrPHProfessor and Department Head

Department of Public Health, University of Tennessee

2

Learner Objectives

1. Identify and describe the domains of EBPH;2. Understand the applicability of EBPH to

improving the efficiency, effectiveness, and accountabilities of public health practice;

3. Describe how EBPH concepts can provide the basis for stating the value of local public health to stakeholders.

4

Why do we do the things we do?

How do we decide what to do?

5


6

Why do we do the things at WORK the way that we do?


Among competing priorities, how do we decide what to do, and why?

9

Evidence-Based Public Health the process of integrating science-

based interventions with community preferences to improve population health.

Dr. Ross Brownson

Acknowledgment: To Dr. Ross Brownson and team, for all EBPH Framework slides and general approach

10

Retool

Discontinue

Disseminate widely

What is “Evidence”?

11

What is “Evidence”? Scientific literature in systematic reviews Scientific literature in one or more

journal articles Public health surveillance data Program evaluations Qualitative data

– Community members– Other stakeholders

Media/marketing data Word of mouth Personal experience

Objective

SubjectiveLike beauty, it’s in the eye of the beholder…

12

Are we talking only of scientific evidence?

13

Where am I?

You’re 30 yards above the

ground in a balloon

You must be a researcher

Yes. How did you know?

Because what you told me is

absolutely correct but completely

useless

You must be a practitioner

Yes, how did you know? Because you don’t know

where you are, you don’t know where you’re

going, and now you’re blaming meThe problem

14

15

How much Evidence is enough?

“The best is the enemy of the good”

Voltaire

The effectiveness of parachutes has not been subjected to rigorous evaluation by using randomised controlled trials…. We think that everyone might benefit if the most radical protagonists of evidence based medicine organised and participated in a double blind, randomised, placebo controlled, crossover trial of the parachute.Smith and Pell, BMJ, 2004

16

How are decisions generally made in public health settings?

17

How are decisions generally made in public health settings?

History/inertia Anecdote Pressure from policy makers or administrators Media driven Resources/funding availability (C-E) Peer reviewed literature/systematic reviews Expert opinions (e.g., academics, community

members)

OR Combined methods, based in sound science The challenge is in how to make the best use of

multiple sources of information & limited resources. 18

Some Key Characteristics of EBPH1. Making decisions based on the best

available peer-reviewed evidence (both quantitative and qualitative research);

2. Using data and information systems systematically;

3. Applying program planning frameworks (that often have a foundation in behavioral science theory);

19

Some Key Characteristics of EBPH

20

4. Engaging the community* in assessment and decision making;

5. Conducting sound evaluation; and

6. Disseminating what is learned to key stakeholders and decision makers.

* Community of fellow staff/employees, of workgroups, of neighborhoods, etc.

21

Small Group Work

What was the most recent NEW thing you did at work?

Why did you do it? What was the Evidence-Base for

decision-making? Did it make a difference?

22

EBPH is NOT Rocket Science **

What’s the problem? How big is it? What’s the cause? What do I do about it? Once I do something, how do I know

I’ve made a difference?

** But rocket science depends on the use of evidence

23

EBPH begins with The Question

24

What’s the issue?

25

EBPH begins with The QuestionFor example…..

What are the most important health problems in our community?

Given the problems of Obesity, Chronic Disease, Child Health and Health Disparities, and Substance Use, what should we focus on? And once we decide, what should we do?

What’s the best way to reduce tobacco use in our community? In our health department?

How do we reduce no-show rates in Family Planning?

26

What’s the issue?

Community Assessment

• Decide what to assess• Decide the best methods to collect the

data to answer your questions• Develop a work plan that identifies

tasks to accomplish, partner roles and responsibilities, and a time frame for completion

• Organize information as it is collected so that it can be shared with all partners, community stakeholders, and community members.

27

Community Assessment

• Present information gathered and summarized from the community assessment back to partners• Coalition/partnership• Community forum• Meetings with community groups

• Move to Action Planning – what do we do to create the desired changes?

28

Community HealthIssues

Identify and discuss

problems and causal factors

Performance Improvement

Approach to Community Health Assessment, 2012-13

Phase 2 Focused

Assessment

Phase 1 Secondary Data

Assessment

Target yourCommunity

health Intervention

programs

Target your community

health intervention

programs

30

What’s the issue?

How big is it?

Quantifying the issue

• Disease/condition vs. Death

• Incidence/Prevalence vs. Mortality

• Problem of small numbers

• Types of rates

31


• Incidence = new cases; a measure of

risk

• Prevalence = existing cases; a

measure of the overall burden of

disease

• P = I x D

32

Downloaded from: StudentConsult (on 4 September 2008 03:58 PM)© 2005 Elsevier

In a steady state: P = I x D


A program to address cardiovascular disease was put in place twenty years ago. An evaluation of the program showed that the prevalence of cardiovascular disease remained the same after twenty years. The program director was convinced this was a good program, but the evaluator retorted – “20 years and no improvement in prevalence?” How can you help this program director out – i.e., how could you explain that the program has indeed been successful despite a stable prevalence rate?

37

P = I x DTime 1: __ = 10 x 550 = 10 x 5Time 2:__ = 5 x 1050 = 10 x 5

38

Where do we get good data on Disease/Condition

Population-based

Vital Statistics• Birth and death

Reportable diseases

Registries• Birth defects• Cancer• Immunizations• Trauma

Representative Samples

National Health Interview Survey (NHIS)

National Health and Nutrition Examination Survey (NHANES)

Behavioral Risk Factor Surveillance System (BRFSS)

Youth Risk Behavior Survey (YRBS)

Hospital/facility-based

TennCareTHA

40


The problem of small numbers: most challenging when expressed as rates, when the multiplier is large and the actual numerators and denominators are relatively small

41


The problem: in 1994 a call from central office to alert me that the female African-American mortality rate for diabetes was higher in Claiborne County than for any county in the state, including Shelby County.

http://hit.state.tn.us/index.shtml

42



43

What’s the issue?

How big is it?

How important is it?

Developing a concise statement of the issue through “Data Synthesis”

Data Synthesis in MAPP

What are we synthesizing?

http://healthyknox.org/home.html

How can we create a sustainable network of partnerships that effectively contributes to improved community health?

STRATEGICISSUES

www.healthyknox.org

http://www.healthyknox.org/



How can we achieve equitable health outcomes for all community members?

STRATEGICISSUES

www.healthyknox.org




How can we position health as a consideration in community policy and planning decisions?

STRATEGICISSUES

www.healthyknox.org




51

What’s the issue?

How big is it?


What is the cause?What should we do about it?

Determining what is known using scientific literature

Do immunizations work? Does family planning work? Does WIC work? Does restaurant inspection work? Do County Health Councils work?

52

Evidence-Based Public Health

http://www.thecommunityguide.org/index.html

http://prcstl.wustl.edu/training/Pages/Evidence-Based-Public-Health-Course.aspx








Evidence-Based Public Health

For which of the following is there strong or sufficient evidence that the intervention is effective (“recommended”) in obesity prevention and control?:Provider remindersSchool-based programsWorksite programs

http://www.thecommunityguide.org/obesity/index.html

54





Ok…Programs…but what about evidence for how we operate?

Fostering more effective public health by identifying administrative evidence-based practices. Brownson RC, Allen P, Duggan K, Stamatakis KA, Erwin PC. Am J Prev Med 43(3):309 –319, 2012

55

Administrative Evidence-Based Practices

56

Domain Evidence-Based Practice

Workforce development Training

Access to technical assistance

Leadership Skills and background of leaders

Values and expectations of leaders

Participatory decision-making

Organizational climate and culture Access and free flow of information

Support of innovation and new methods

Learning orientation

Relationships and partnerships Interorganizational relationships

Vision and mission of partnerships

Financial Allocation and expenditure of resources

58

What’s the issue?

How big is it?



What are we going to do about it?

Developing & Prioritizing Options

1. Multi-voting Technique2. Strategy Grids3. Nominal Group Technique4. The Hanlon Method5. Prioritization Matrix

59


60

61


64

What’s the issue?

How big is it?




How are we going to do it, when, with what results?

Developing an Action Plan and Implementing Interventions

Assessment should guide the development of programs

Community members should participate in the process

Program should be developed with key partners or as part of coalitions (builds infrastructure for change)

Participants should develop a comprehensive intervention strategy- but may implement one step at a time

Evaluation should emphasize feedback and program improvement

The community capacity for health promoting changes should be increased – skill transfer between community & academic/practice partners (two way)

65

Program development and implementation

Source: http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html

http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html

http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html

67

What’s the issue?

How big is it?





How will we know if we have made a difference?

Why Evaluate?

68

What funders want:– Prove that their money is making a

big difference in outcomes of their choice.

What grantees/people who receive funds want: – Show that they should be given more

money to continue and expand the good work they are doing.

The Evaluation Trap

69

If evaluation is conducted for these reasons only, the result, over time, may be to undermine both political support for funding and program quality.

Why? These motivations can lead to: – looking for wrong outcomes – looking for wrong effect sizes (how big a

difference your program makes)

Logic models document your theory of change: how your program works

70

SpecifiedInterventionComponents

AndActivities

SpecifiedInterventionComponents

AndActivities

ChosenDeterminants

(Risk & ProtectiveFactors)

ChosenDeterminants

(Risk & ProtectiveFactors)

Outcomes

Interventions• Inputs• Activities

OutcomesProximalDeterminants• Behaviors• Outputs

71

Early detection of breast cancer

Outcomes

Breast cancer screening by mammography

Determinants

Provider reminder and recall systems

Activities

Evaluation Strategy 1: Start with Outcomes

We are used to thinking about activities first.

That’s OK! Build the model from left to right.

But you still need to think through why your activities “make a difference.”

72

Evaluation Strategy 2: Start Building the Model with Activities

73

SpecifiedActivitiesSpecifiedActivities

Immediate or short-term outcomes

(Exactly the sameas proximaldeterminants)

SpecifiedOutcomesSpecifiedOutcomes

Evaluation Strategy 2: Start Building the Model with Activities

Evaluation Words of Wisdom Don’t worry about the labels

– Determinants are often called outputs– Outcomes are sometimes called impacts– Sometimes logic models have separate columns

for inputs, activities, outputs, short-term outcomes, long-term outcomes, and impacts (whew!).

Focus instead on the causal mechanisms.– If you think through the logic of why your program

works, you can fill in any format the funder/evaluator requires of you.

– ….and actually benefit from the experience!

74

75

Small Group Work

What was the most recent NEW thing you did at work?

Why did you do it? What was the Evidence-Base for

decision-making? Did it make a difference? –

create a logic model with inputs, activities (outputs), and outcomes to evaluate

Contextual Factors(Individual Student Characteristics – Race, Special Needs, Language, achievement; Family Characteristics – Income, parental education, structure;School Characteristics- Average Test Scores, Mobility, %FRL, Attendance; Community – % Poverty, % Affluent, % Idle youth)

TCAP Staff

Administrative Buy-in/Support

Organizational Structure

INPUTS ACTIVITIES OUTPUTS OUTCOMES

Example of a Complex Logic Model: Turning the Corner on Achievement Program

June 1, 2006 Center for Adolescent Health, JHU

Relationship with Families

School-Level

TCAP-Level

Motivation toAchieve

New PossibleIdentities

Social Development

Intermediate Outcomes Improved

AcademicAchievement

Ultimate Outcomes

ImprovedAttendance

Access toCommunityResources

Academic•Integrated•Acceleration•Targeted

Arts & Culture

StudentAssessment

Lifestyles and Career

Reduced HealthRisk Behaviors

Improved Mental Health

Students Experience…Physical and Psychological SafetyAppropriate StructureSupportive RelationshipsOpportunities to BelongPositive Social NormsSupport for Efficacy and MatteringOpportunities for Skill BuildingIntegration of School, Family, and CommunityExposure to new opportunities

Staff Experience…Support Tools to perform

Families Experience…Engagement in EducationParenting SupportAccess to resources

Psychological and Emotional Development

Professional Development

Organizational Structure

Communication skills

Family Support Services

Community Service

Improved Social Relationships

Relationship with Teachers

Leadership skillsStudents and Families

Relationship w/ families

Relationship w/ school

Improved Civic Engagement

Evidence-Based Decision Making

78

What’s the issue?

How big is it?





How will we know if we have made a difference?

Evidence-Based Public Health TPHA 2013 Paul Campbell Erwin, MD, DrPH Professor and Department Head...

Documents

Transcript of Evidence-Based Public Health TPHA 2013 Paul Campbell Erwin, MD, DrPH Professor and Department Head...