Variations in the organisation of and outcomes from Early ...
Creating an outcomes framework for your organisation
-
Upload
mark-planigale -
Category
Business
-
view
2.130 -
download
0
description
Transcript of Creating an outcomes framework for your organisation
Mark PlanigaleResearch & Consultancy
results by design
Homelessness assistance sector, Australia, November 2010
2
Performance measurement
Compliance reporting
Program logic
3
Performance measurement Bigger picture of organisational performance
measurement
Compliance reporting currently focuses on effort
Many organisations monitor inputs and client satisfaction
But are we making a difference?
National frameworks: National Affordable Housing Agreement, National Partnership Agreement on Homelessness
‘Manufacturing-based’ performance models need to be enhanced
4
Program logic
InputsPeople and resources
• Clients
• Human resources
• Other resources
ActivitiesWhat gets done
• Functions
• Processes
• Actions
OutputsWhat clients receive
• Client participation benefits
• Client direct benefits
OutcomesChanges in clients’ lives
• Short term
• Long term
• Interim
ImpactsChanges in society
• Social outcomes
Environment • Problem or need• Barriers• Enablers
Objectives Aims
Mission Vision
5
Useful information
is produced through
well-planned systems.
6
Purpose and scope
Defined outcomes
Defined measures and tools
Data collection and storage processes
Analysis and reporting processes
Use of outcomes information
Strategies for sustaining the system
7
Clarify purpose and scopeWhy measure outcomes? Questions to consider
Compliance
Individual assessment and planning
Oversight
Quality improvement
Advocacy
Which programs?
Which populations?
What do we mean by outcomes?
8
Define key outcome areasOutcome: Questions to consider
“... a change or absence of change that results (at least in part) from actions of staff of the organisation”
Desired vs. undesired
Short term, long term, interim
Domains vs. locus of change
If the program works really well, how are clients’ lives better?
If things go wrong for our clients, where do we see this in their lives?
What is our core business?
9
Example outcomes1. Person obtains appropriate, affordable long-term
housing
2. Person takes medication more regularly
3. Person has a better understanding of tenancy rights and responsibilities
4. Person is placed on the OOH Early Housing waiting list
5. Person gains part-time employment
6. Person becomes homeless
10
Select measures and toolsMeasure: Questions to consider
“...an observable, measurable characteristic of a person or their situation, which is linked with a state or condition of interest to us”
Base measures, derived measures and KPIs
Varied vantage points
Use modules for flexibility and consistency
If our key outcomes occurred, how would we know?
Which measures have the greatest communicative power?
What data do we already have?
11
Example measuresStatus change/maintenance scale:
1. % of clients who were homeless at the end of the period
Level of functioning scale:
2. % of clients who rate their parenting skills better at exit than they did at entry
Goal attainment scaling:
3. % of clients who achieved better than expected outcomes in majority of goals reviewed this period
12
Example tool: Outcomes Star
Triangle Consulting / London Housing Foundation. http://www.homelessoutcomes.org.uk/The_Outcomes_Star.aspx
13
Collect dataData collection and entry: Questions to consider:
Has a huge impact on data quality
Should be integrated as far as possible with service delivery
Requires client consent
May need to respond flexibly to circumstances of client
Who will collect the data? What training will they require?
All clients or a sub-sample?
How and when will data be collected?
How will data be stored?
14
Analyse and reportOutcomes reporting enablers: Questions to consider:
Who will use the reports?
What levels of aggregation are useful?
How can complexity be acknowledged?
A knowledge of stakeholder requirements
Data analysis skills
Standardised:
calculation procedures
report templates
database queries
Access to data!
15
Example: control graphProportion of service episodes with housing situation improved at completion
38.4%
34.5%
31.8%
35.2%34.3%
33.3%34.4%
37.4%
28.5%
28.6%
34.9%
41.3%
20%
30%
40%
Jun
2009
Sep
2009
Dec
2009
Mar
2010
Jun
2010
Sep
2010
Dec
2010
Mar
2011
Jun
2011
Percentage of
completed service
episodes
(current quarter
n = 165,
cumulative
n = 1097)
Quarterly proportion Cumulative proportion
16
Use outcomes informationContexts for use: Questions to consider:
Individual casework: assessment, planning and review
Service review and reflection
Strategic planning: “turning the curve”
Advocacy
How can outcomes information be shared with clients?
How can staff play a part in “giving meaning” to outcomes information?
17
Caution! Outcomes monitoring data by itself cannot “prove”
service effectiveness
To build your case, triangulate with other evidence
18
Sustain the systemWhat helps?
Using the information – complete the loop
Strong commitment to and endorsement of outcomes measurement by senior management
Performance expectations – staff to collect outcomes data
Ongoing resourcing
Regular review of framework
19
Risks
Implementation process
The bottom line
20
Risks Poor value for money
Outcome measurement can be resource-intensive and time-consuming
Information produced may not be high quality
Distortion of service delivery
Staff opposition
Data collection burden
Feeling scrutinised
Negative results
21
Process tips Involve stakeholders early and ongoingly
Board, Executive, management, service delivery staff, clients
Combine top-down and bottom-up elements
Pilot locally, implement sequentially
Typically 6 – 24 months to end of pilots
Varies with size of organisation, complexity of services, and level of resourcing
22
The bottom line Client wellbeing comes first
Client rights are respected
Information produced is useful
Relevant
Reliable
Valid
System is affordable
Data collection burden on staff is minimised
23
Useful resourcesResults-Based Accountability
www.raguide.org
Friedman (2009) Trying Hard Isn’t Good Enough
UK Approaches (incl. Outcomes Star) www.homelessoutcomes.org.uk
Burns & Cupitt (2003) Managing Outcomes: A Guide for Homelessness Organisations
US: National Alliance to End Homelessness Spellman & Abbenante (2008) What Gets Measured, Gets Done: A Toolkit on
Performance Measurement for Ending Homelessness
Reference – technical and implementation issues Planigale (2010) Literature Review: Measurement of Client Outcomes in Homelessness
Services. http://www.homeground.org.au/assets/literature-review-measurement-of-client-outcomes-in-homelessness-services.pdf
Mark PlanigaleResearch & Consultancyresults by design
Phone: 0429 136 596
Mail: PO Box 754, Macleod VIC 3085
Email: [email protected]
Web: www.planigale.com