Health and Well-Being in Schools Steering Group 18 May 2009 Neil Craig Policy Evaluation & Appraisal...
-
Upload
june-kelly -
Category
Documents
-
view
213 -
download
0
Transcript of Health and Well-Being in Schools Steering Group 18 May 2009 Neil Craig Policy Evaluation & Appraisal...
Health and Well-Being in Schools Steering Group18 May 2009
Neil CraigPolicy Evaluation & Appraisal Team
NHS Health Scotland
Logic Modelling for Outcomes-Based
Approaches
Outline
• What is an “outcomes-focused approach”?
• What are logic models, how do you create them how can they help?
• Discussion
What is an outcome-based approach?
• “encourages us all to focus on the difference we make and not just the inputs or processes over which we have control.”
• “should clearly demonstrate how the activities of public bodies are aligned with the Govt’s overarching purpose through the National Performance Framework” ... [and are] “supporting the achievement of outcomes”
(Outcome based approach: Working guidance for Scottish public bodies, Sept 2008)
Why now?Part of wider performance management, outcome based
planning system:
“We are moving the whole of government to an outcomes-focused approach to performance” (Scottish Budget Spending Review, 2007)
National PerformanceFramework, HEAT & SOAs
NHS PerformanceManagement:
30 HEAT Targets
SOA: Menu of 67 local outcome
indicators
PURPOSE
PURPOSE TARGETS
STRATEGIC OBJECTIVES
NATIONAL OUTCOMES
NATIONAL INDICATORS AND TARGETS
Greener
Wealthier & Fairer
HealthierSmarter
Safer & Stronger
e.g.CHD mortality Life expectancy Child healthy weightMental health-related outcomesCompletion of child
healthy weight programmes
e.g.CHD mortality Life expectancy Child healthy weightMental health-related outcomes
We live longer, healthier lives
We have tackled the significant inequalities in
Scottish society
We have strong, resilient and supportive
communities…
We give children the
best start in life
We live in well-designed, sustainable places
We value and protect the natural
environment and cut our environmental
impact
SOAs:Guidance for 2009/10
• SOAs are the “means by which CPPs agree their strategic priorities for their local area and express those as outcomes to be delivered by partners, … while showing how they contribute to the Scottish Gov’t’s relevant National Outcomes”
• The SOA “must be a strategic document … underpinned by robust perf. management arrangements… [and] a very clear line of sight from the SOA document.”
SOAs: AccountabilityLetter from SG to Chief Execs
Signing up to an SOA means that partners• “are corporately committed to the agreed outcomes …
and … will take every opportunity to promote and support the achievement of outcomes.”
• “doing this is likely to include each partner looking at how they individually can contribute to outcomes” and
• “each partner will need to show that a ‘golden thread’ runs from the SOA through their planning, resourcing and performance management processes.”
Tools for the job? Logic models
• A convincing picture of what you are trying to achieve that shows the links between your intended inputs, activities, outputs and outcomes.
• Not an attempt to describe reality but your theory about the key things that need to happen to get to your intended outcomes
• A process and a way of thinking as well as an output
Basic model
Resources / Inputs Activities Outputs
Short-term
Outcomes
Long term
Impact
Medium term
Outcomes
ASSUMPTIONS
That affect delivery
EXTERNAL FACTORS
That affect results/impacts
Define your terms• Inputs
– What you invest– E.g. Resources, staff,
funding, equipment, facilities supplies
• Activities– Key things you will have to
do – E.g. marketing,
recruitment, training, running programmes
• Outputs– Countable products– E.g. Number of people trained,
information packs, courses run, number recruited
• Outcomes (short, interim and long-term)
– The changes achieved in your target groups
– E.g. Increased knowledge, increased confidence, improved risk factors, better health
An example: child healthy weight
LONG-TERM HIGH LEVEL
OUTCOMES
INTERMEDIATEOUTCOMES
SHORT-TERM OUTCOMES
Outputs?
Inputs?
Activities/ Processes?
Reduce the rate of increase in the proportion of
children with BMI outwith healthy range by 2018
National Indicators
Menu of Local OIs
(Proportion and no. of obese children in P1)
Increased healthy life expectancy
Reductions in morbidity and mortality from obesity-related causes
Achieve agreed completion rates for child healthy weight intervention programme by 2010/11
HEAT: H3
?
Healthy weight – Cross-sector Contributions?
Inputs
Activities
Outputs
Reach
Short-term outcomes
Intermediate outcomes
High level outcomes
NHS
What NHS does to reach target
population
Child healthy weight intervention
Overweight children & families
Completed healthy weight programmes
Planners
?
?
?
?
Schools
?
?
?
?
Reduced increase in children with BMI outwith healthy range
Improved mental wellbeingReduced inequalities in healthy life expectancy
Reduced prevalence of and inequalities in obesity-related ill-health
EnvironmentsPhysical: More opportunities to lead active lives?
Economic: Less availability/affordability of energy-rich food? Social: Changing attitudes to food and exercise?
SG, UK govts, EU
?
?
?
?
Scottish Govt
?
?
?
?
Vol orgs
?
?
?
?
?
INTERMEDIATE OUTCOMES
Strategic Healthy Weight Outcomes Logic Model (Simplified)
LONG-TERM OUTCOMES
NESTED LOGIC MODELS
Reduced Type 2
Diabetes
Reduced CHD and
stroke
Reduced cancer
NATIONALOUTCOMES
Longer, healthier
lives
Tackled significant inequalities
Other non-health
outcomes?
Reduced prevalence of overweight and
obesity
Improved energy balance
Model a
Model b
Model c
Model d
Model e
Increased food industry corporate
and social responsibility
Changes in availability and affordability of
energy-dense/low energy dense
food
Improved early years nutrition
Increased individual
empowerment and food literacy
Improved media and social
environment
Increased physical activity
Nested model for child healthy weight programmes (simplified)
ActivitiesShort-term Outcomes
(by end of programme)
Intermediate OutcomeReach
Reduced increase in
children with BMI outwith healthy range
??Other NHS?
?
??Other non-
NHS?
?
Outputs
% Target group
completed programme
Improved psychosocial outcomes e.g. self-esteem,
body image
Increased physical activity
Improved nutrition
Improved knowledge and skills
Healthy weight programme: identification, contact, assessment, interventions
5–15 year olds, >91 centile
No. sessionsType of
sessions and range ofcontent
How do you develop a model?
Look for logic in plans and draft a model
Agree intended use(s) of model(s)
Engage stakeholders and adapt and refine
Share model more widely
Use, revisit and refine it regularly
Workshops, discussions, interviews, focus groups
Process• Engage people• Set boundaries• Draw on evidence, logic and ethics• Interrogate and revisit the model
Interrogate and revisit the model
• Is it plausible? • Is it doable?• Is it testable?• Have external constraints/enablers and unintended
consequences (+/-) been considered?
Learning and reflections
• No standardised approach – to the process or the product
• The value is in the process not just the product• Models should be fit for purpose, not ‘perfect’• Logic models should be embedded in part of wider
performance management system and learning culture
• Logic modelling is limited by, but may also contribute to, the existing evidence base
Conclusions: How can LMs help?
• Help you know what and when resources are needed• Provides a focus for evaluation• Helps identify what to measure + when• In complex interventions, provides some credibility
when attribution impossible• Links with bigger picture - e.g. how programme
activities support achievement of national objectivesLogic models provide a framework for integrating
planning, delivery, evaluation and performance management
Questions and Discussion