The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis...
-
Upload
oliver-bishop -
Category
Documents
-
view
217 -
download
1
Transcript of The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis...
![Page 1: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/1.jpg)
The Path to Health and Wellness:Building Our Strength inProgram Planning and Evaluation
The Path to Health and Wellness:Building Our Strength inProgram Planning and EvaluationJanis Weber, Ph.D.Janis Weber, Ph.D.
![Page 2: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/2.jpg)
Describe a general approach to
evaluation planning using a program
logic model, performance objectives,
indicators, and data sources
Goal
![Page 3: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/3.jpg)
What comes to mind when you hear
Program Evaluation?
![Page 4: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/4.jpg)
What Is Program Evaluation?
“The systematic collection of information about
the activities, characteristics, and outcomes of
programs to make judgements about the
program, improve program effectiveness, and/or
inform decisions about future program
development.”
![Page 5: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/5.jpg)
Demonstrate accountability to stakeholders
Measure program achievement
Manage program resources
Document and improve program operations
Why Invest in Program Evaluation?
![Page 6: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/6.jpg)
Accountability
Describe relationship between activities and intended outcomes
Monitor program implementation
Measure intermediate and long-term outcomes
Educate about realistic expectations for change
![Page 7: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/7.jpg)
Evaluation
• Requires more in-depth data collection
• Measures short-term and intermediate outcomes
• Documents implementation and effectiveness
• Includes developing a program logic model
![Page 8: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/8.jpg)
Surveillance Data• Routine
• Existing resources
• Less flexibility
Evaluation Data• Occasional
• Additional resources
• Flexible
Surveillance and Evaluation
![Page 9: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/9.jpg)
Framework for Program Evaluation in Public Health
TM
![Page 10: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/10.jpg)
Standards for “Good” Evaluation
Utility
Feasibility
Propriety
Accuracy
![Page 11: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/11.jpg)
Framework for Program Evaluation in Public Health
TM
![Page 12: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/12.jpg)
A Collaborative Approach
Reduces suspicion and fear
Encourages differing perspectives and many voices
Increases awareness and commitment
Increases the possibility of reaching objectives
Increases the possibility findings will be used
![Page 13: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/13.jpg)
• Insiders
• People who manage or work in the program
Outsiders
• Those served or affected by programs, or work in partnership with the program to achieve its goals
Intended users of the evaluation
• People who are in a position to decide something about the program
Identify Stakeholders—Who?
![Page 14: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/14.jpg)
Who are your stakeholders?
![Page 15: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/15.jpg)
Example HOPE Stakeholders
• Program staff
• Local and regional coalitions
• Community leaders, members, and grantees
• Local, state, and national partners
• Program funders
• Program Users
• Federal, state and county health departments
• Professional and business entities
• Schools and educational groups
• The medical community
• Community-based organizations
![Page 16: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/16.jpg)
Example HOPE Stakeholders
• Universities and educational institutions
• Government(local, state legislators, and the governor)
• Families
• Privately owned businesses and business associations
• Faith organizations
![Page 17: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/17.jpg)
Ensure that evaluation designed to answer questions important to stakeholders
Increase likelihood of continued support
Build wider competency in evaluation
Increase possibility evaluation findings used
Involve Stakeholders—Why?
![Page 18: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/18.jpg)
Stakeholders as Team Members
• Engaging stakeholders• Diplomats• Those with diverse networks
• Describing the program• People who understand program history, purpose and operation
• Focusing the evaluation design• Decision makers who guide program direction
• Gathering credible evidence• Experienced evaluators• Social or behavioral scientists
• Justifying conclusions • Trusted people with few personal stakes
• Ensuring use and sharing lessons learned
• Advocates• Clear communicators• Creative thinkers• Members of the power structure
Step Helpful Team Members
![Page 19: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/19.jpg)
Example Stakeholder Interests
What is the program doing to address the problem?
How can I get involved in the program?
Is the program making a difference?
Is the program worth the cost?
Can the program be made more efficient?
![Page 20: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/20.jpg)
Priority Questions
Who are your stakeholders?
How can stakeholders be identified or engaged?
What role(s) do stakeholders have in planning?
Who lends credibility to an evaluation or team?
How will we integrate stakeholder values?
![Page 21: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/21.jpg)
Checklist—Engage Stakeholders
Identify stakeholders
Review list of stakeholders for inclusivity
Balance individuals and organizations
Reflect the situation of overweight in setting
Establish a method of communication
![Page 22: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/22.jpg)
Target key stakeholders for participation
Identify key areas for stakeholder input
Create a plan for strategic involvement
Bring stakeholders together as needed
Understand and reflect stakeholder values
Checklist—Engage Stakeholders
![Page 23: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/23.jpg)
Framework for Program Evaluationin Public Health
TM
![Page 24: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/24.jpg)
Our plans miscarry because they have no aim.When a man does not know what harbor he is making for,
no wind is the right wind.
—Marcus Annaeus Seneca
![Page 25: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/25.jpg)
Describe the Program
• Need
• Expected effects
• Activities
• Resources
• Stage of development
• Context
• Logic model
![Page 26: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/26.jpg)
We Must Describe the Program BeforeWe Can Begin Monitoring and Evaluation
Describing the program includes:
• Identify what activities you will do based on the needs identified from the data
• Clearly outline what the activities will accomplish immediately
• Clearly outline the impact the activities will have in the longer term
![Page 27: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/27.jpg)
What Is a Logic Model?
Disciplined way of mapping a program
Platform for discussion
Multi-purpose tool
Means and not an end
Presentation of links in a chain of reasoning
![Page 28: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/28.jpg)
INPUTS ACTIVITIES OUTCOMESOUTPUTS
Program Resources
What we do with program resources to fulfill mission
Benefits for participants during and
after program activities
Direct products of
activities
Generic Program Logic Model
![Page 29: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/29.jpg)
Demonstrate a theory of change linked to time
Educate stakeholders about realistic expectations
for change
Set ourselves up to demonstrate accountability now
and in the future
Tell a complete a story
A Continuum of Outcomes—Why?A Continuum of Outcomes—Why?
![Page 30: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/30.jpg)
A comprehensive evaluation includes A comprehensive evaluation includes process process andand outcome measures outcome measures..
![Page 31: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/31.jpg)
Process
• Were program activities accomplished?
• Were the activities implemented as planned?
Process and Outcome Evaluation
Outcome
• Is the program having the intended impact?
• Is there progress toward larger program goals?
![Page 32: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/32.jpg)
The type and quantity of services provided
The number of people receiving services
What actually happens during implementation
How much money the project is using
The staffing for services/programs
The number of coalition activities and meetings
Assessment of program fidelity
Process Evaluation
![Page 33: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/33.jpg)
Results of program services
Changes in individuals
Changes in attitudes, beliefs or behaviors
Changes in the environment
Changes in health disparities related to overweight and obesity
Outcome Evaluation
![Page 34: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/34.jpg)
How do we transition from the logic model to program evaluation?
QUESTION:
![Page 35: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/35.jpg)
Inputs
ActivitiesShort-termOutcome
IntermediateOutcome
Long-termOutcomeOutputs Goal
Evaluate Against Program Logic Model
![Page 36: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/36.jpg)
Evaluation Questions
Is your program making a difference?
Is your program effective in reducing tobacco consumption?
Can your program be improved?
What exactly is your program doing?
Is your program accomplishing what it was intended to accomplish?
Was your program implemented as planned?
Are you using resources efficiently and effectively?
Is your program’s performance on par with established standards?
![Page 37: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/37.jpg)
Performanceobjectives
Indicators
Data sources
Performanceobjectives
Indicators
Data sources
Performanceobjectives
Indicators
Data sources
Inputs
ActivitiesShort-termOutcome
IntermediateOutcome
Long-termOutcomeOutputs Goal
Evaluate Against Program Logic Model
![Page 38: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/38.jpg)
““Unless the goals are operationalized into Unless the goals are operationalized into
specific objectives, it is unlikely that a specific objectives, it is unlikely that a
plan can be implemented to meet them” plan can be implemented to meet them”
(Rossi and Freeman 1982, p 56).(Rossi and Freeman 1982, p 56).
![Page 39: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/39.jpg)
Performance Objectives
Focus program priorities
Benchmark progress over time
Set targets for accountability
![Page 40: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/40.jpg)
Strong Program Objectives Are
Specific
Measurable
Achievable
Relevant
Time-bound
![Page 41: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/41.jpg)
Increase the proportion of
youth in grades 6 to 8 who
engage in 30 minutes of
physical activity per day from
X percent in June 2007 to Y
percent in June 2008.
Specific
Measurable
Achievable
Relevant
Time-bound
Strong Program Objectives Are
![Page 42: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/42.jpg)
Specific
Measurable
Achievable
Relevant
Time-bound
Increase the proportion
of youth in grades 6 to 8
who engage in 30
minutes of physical
activity per day from X
percent in June 2007 to
Y percent in June 2008.
Strong Program Objectives Are
![Page 43: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/43.jpg)
Specific
Measurable
Achievable
Relevant
Time-bound
Increase the proportion
of youth in grades 6 to 8
who engage in 30
minutes of physical
activity per day from X
percent in June 2007 to
Y percent in June 2008.
Strong Program Objectives Are
![Page 44: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/44.jpg)
Specific
Measurable
Achievable
Relevant
Time-bound
Increase the proportion
of youth in grades 6 to 8
who engage in 30
minutes of physical
activity per day from X
percent in June 2007 to
Y percent in June 2008.
Strong Program Objectives Are
![Page 45: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/45.jpg)
Increase the proportion
of youth in grades 6 to 8
who engage in 30
minutes of physical
activity per day from X
percent in June 2007 to
Y percent in June 2008.
Specific
Measurable
Achievable
Relevant
Time-bound
Strong Program Objectives Are
![Page 46: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/46.jpg)
Specific
Measurable
Achievable
Relevant
Time-bound
Increase the proportion
of youth in grades 6 to 8
who engage in 30
minutes of physical
activity per day from X
percent in June 2007 to
Y percent in June 2008.
Strong Program Objectives Are
![Page 47: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/47.jpg)
Framework for Program Evaluation in Public Health
TM
![Page 48: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/48.jpg)
Not everything that can be counted counts andnot everything that counts can be counted.
—Albert Einstein
![Page 49: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/49.jpg)
Focus the Evaluation Design
Purpose
Users
Uses
Questions
Methods
Agreements
![Page 50: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/50.jpg)
Focus the Evaluation Design
Define the purpose(s) of your evaluation
Include process and outcome evaluation
Identify evaluation questions
Link ? to goals, outcomes, and objectives
Identify the use(s) of results
![Page 51: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/51.jpg)
Focus the Evaluation Design
Collect data to make comparisons
Review options for evaluation design
Consider a goal-based evaluation model
Make sure evaluation design “fits” questions
Seek expertise and/or review as needed
![Page 52: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/52.jpg)
CDC Framework for Program Evaluation
TM
![Page 53: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/53.jpg)
“Don’t accept your dog’s admiration asconclusive proof that you are wonderful.”
—Ann Landers
![Page 54: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/54.jpg)
Gather Credible Evidence
Indicators
Sources
Quality
Quantity
Logistics
![Page 55: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/55.jpg)
Sources of Information-People
Clients, program participants/nonparticipants
Staff, program managers, administrators
Partner agency staff
General public
Key informants
Funders
Critics/skeptics
Representatives of advocacy groups
Elected officials, legislators, policy makers
DocumentsGrant proposals, newsletters, press releasesMeeting minutes, administrative records Registration/enrollment formsPublicity materials, quarterly reportsPublications, journal articles, postersPrevious evaluation reportsNeeds assessments
Surveillance summariesDatabase recordsRecords held by funders or collaboratorsWeb pagesGraphs, maps, charts, photographs, videotapesObservationsMeetings, special events/activities, job performanceService encounters
![Page 56: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/56.jpg)
Grant proposals, newsletters, press releases Meeting minutes, administrative records Registration/enrollment forms Publicity materials, quarterly reports Publications, journal articles, posters Previous evaluation reports Needs assessments Surveillance summaries Database records Records held by funders or collaborators Web pages (electronic documents) Graphs, maps, charts, photographs, video
Sources of Information-Documents
![Page 57: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/57.jpg)
Meetings
Special events and/or activities
Job performance
Service encounters
Sources of Information-Observations
![Page 58: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/58.jpg)
Gather Credible Evidence
Confirm use of intended outcomes
Confirm outcomes are logically linked
Address a continuum of outcomes
Collect process and outcome data
Identify at least 1 indicator for each outcome
Link outcomes to indicators and data sources
![Page 59: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/59.jpg)
Gather Credible Evidence
Ask if new data collection is necessary
Add evaluation ? to existing systems
Consider mixed-method data collection
Take into account available resources
Test new instruments to identify sources of error
Consider issues of timing
![Page 60: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/60.jpg)
Framework for Program Evaluation in Public Health
TM
![Page 61: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/61.jpg)
Justify Conclusions
Standards
Analysis/synthesis
Interpretation
Recommendations
![Page 62: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/62.jpg)
Sample Benchmarks for Performance Measurement
Needs of participants
Community values, expectations, norms
Program missions, objectives
Program protocols and procedures
Change in performance over time
Performance by similar programs
Performance by a control or comparison group
![Page 63: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/63.jpg)
Sample Benchmarks for Performance Measurement
Resource efficiency
Mandates, policies, regulations, laws
Judgments by participants, experts, and funders
Institutional goals
Political ideology
Social equity
Human rights
![Page 64: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/64.jpg)
Justify Conclusions
Check data for errors
Analyze data using appropriate techniques
Consider issues of context when interpreting
Discuss alternative explanations
Compare results with those of similar programs
![Page 65: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/65.jpg)
Justify Conclusions
Assess results against available literature
Use HP2010 as a key point for comparisons
Examine the limitations of the evaluation
Document potential biases
![Page 66: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/66.jpg)
Framework for Program Evaluation in Public Health
TM
![Page 67: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/67.jpg)
Design
Preparation
Feedback
Follow-up
Dissemination
Ensure Use and Share Lessons Learned
![Page 68: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/68.jpg)
Demonstrate that resources are well spent
Aid in the formulation of budgets
Compare outcomes with those of previous years
Compare actual with intended outcomes
Identify training and technical assistance needs
Ensure Use
![Page 69: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/69.jpg)
Support annual and long-range planning
Focus attention on issues important to program
Promote your program
Identify partners for future collaborations
Enhance the public image of your program
Retain or increase funding
Provide direction for program staff
Ensure Use
![Page 70: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/70.jpg)
Ensure Use and Share Lessons Learned
What can be done to increase the use of evaluation findings?
What can be done to reduce confusion or misinterpretation?
How should results be presented for different audiences?
How are program planning and evaluation related to one another?
![Page 71: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/71.jpg)
Ensure Use and Share Lessons Learned
Design for intended use by intended users
Prepare stakeholders for eventual use
Provide continuous feedback to stakeholders
Disseminate procedures used and lessons learned to stakeholders
Draft recommendations based on audience
![Page 72: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/72.jpg)
Ensure Use and Share Lessons Learned
Revisit the purpose(s) of the evaluationwhen preparing reports and recommendations
Tailor evaluation reports to audience(s)
Present clear and succinct findings in a timely manner
Avoid jargon when presenting information to others
Document limitations of the evaluation
Disseminate via multiple venues
![Page 73: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/73.jpg)
Expensive
Too time consuming
Tangential
Technical
Not inclusive
Academic
Punitive
Political
Useless
Rethink Program Evaluation
Cost effective
Strategically timed
Integrated
Accurate
Engaging
Practical
Surprisingly helpful
Participatory
Useful
![Page 74: The Path to Health and Wellness: Building Our Strength in Program Planning and Evaluation Janis Weber, Ph.D.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e0e5503460f94af7714/html5/thumbnails/74.jpg)
Janis [email protected]