AEA Presentation: Using Data to Influence Programs and Policy
-
Upload
the-cloudburst-group -
Category
Data & Analytics
-
view
303 -
download
1
Transcript of AEA Presentation: Using Data to Influence Programs and Policy
![Page 1: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/1.jpg)
Solutions for Health, Housing and Land ● www.cloudburstgroup.com
Using Data to Influence Programs and PolicyLindsey Barranco, Ph.D. Jamie Taylor, Ph.D.
American Evaluation AssociationChicago, ILNovember 14, 2015
![Page 2: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/2.jpg)
2
Session Objectives1. To identify innovative ways existing data is used to
influence program and policy directions2. To examine multiple ways data results can be used for
mid-course corrections in programs and long-term policy impacts
3. To understand the use of data visualization tools to promote data utilization, stakeholder discussion and new policy directions
![Page 3: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/3.jpg)
3
Using Administrative Data to Drive Systems Change and Improve Programs
![Page 4: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/4.jpg)
4
Background• Homeless Management Information System (HMIS) is a
locally administered, electronic data collection system that stores longitudinal person-level information about persons who access the homeless service system.
• HMIS is HUD’s response to a Congressional Directive to capture better data on homelessness
• Can also provide communities with a comprehensive view of the nature and response to homelessness AND foster collaboration
• HUD is now requiring communities use HMIS data to report on system performance toward ending homelessness
![Page 5: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/5.jpg)
55
HUD System Performance Measures
![Page 6: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/6.jpg)
6
Benefits of Analyzing HMIS Data• Provides HUD way to benchmark measure
progress around ending homelessness • Provides communities with an indicator of their
success and challenges• Provides agencies with information about how
their program is contributing to overall system performance
• Allows communities to look at the needs of the homeless population and what is and is not working
• Allows community to monitor performance of agencies
![Page 7: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/7.jpg)
7
2014 Annual Homeless Assessment Report – Published 11/13/15
HMIS data entered at client level, aggregated quarterly for AHAR.
![Page 8: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/8.jpg)
8
2014 Annual Homeless Assessment Report – Published 11/13/15
![Page 9: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/9.jpg)
9
Using the Data to Understand the Local Homeless Service System
![Page 10: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/10.jpg)
10
• National trends in homelessness can be tracked to monitor investments made in targeted efforts, i.e. Ending Veterans homelessness and Ending Chronic Homelessness by 2016
• Great example of efforts to really USE the data collected as part of Federal reporting requirements
• Complicated process at the federal level determining how to define each measure accurately enough to ensure all communities are measuring performance in the same manner
• Communities need support in understanding how to USE their data
• Communities need support in understanding what programmatic or policy changes will “move the needle”
Lessons Learned
![Page 11: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/11.jpg)
11
RRH Analytics ProjectGOAL OF PROJECT: Capacity building to empower community leaders to use data to meet local and federal policy goals to end homelessness. Policy focus on Rapid Re-Housing (RRH) – a housing assistance approach that provides time-limited rent payments to quickly move households out of shelter and back into housing. RRH DATA ANALTICS PROJECT: Homeless Management Information System (HMIS) data analyzed in six sites across the country. Community leaders engaged in learning community: weekly meetings to review data quality, share system learning around program/provider variation & promote inquiry for RRH system level improvement plans.
RRH Data Analytics Project: 2 states, 3 cities, 1 county
![Page 12: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/12.jpg)
12
RRH Data Analytics ProjectFour-month Learning Community Process
HMIS Data Pull
Data Preparatio
n Data
AnalysisIterative
Data Reviews
Data Dashboard
s
12
• Weekly Meetings with Project Leadership– HMIS Data Pull, Data Analysis Design– Data Preparation, Data Reviews, Data Visualization
• Collaborative assessment with community partners ensured analysis congruent with expected program data
• Theory of Change based on Data Literacy Intervention
![Page 13: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/13.jpg)
13
Theory of Change to End Homelessness has Changed
Person falls into homelessness
Person sheltered
Person enters Transitional Housing
Person “ready” to re-enter community
End of Person Homelessness
![Page 14: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/14.jpg)
14
Need for Rapid Re-Housing Evidence Base
![Page 15: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/15.jpg)
15
RAPID CYCLE EVALUATION on RRH IMPACTSPropensity Score Matching (PSM) employed to assess the effect of RRH on reducing the risk of return to homelessness. With Phoenix/Maricopa County HMIS data, comparison groups that statistically looked the same were created to assess the true effects of RRH assistance.
Households were matched on: age, type of household, single parent, education, income, previous shelter stay, race, ethnicity, mental health disability, physical disability, substance use disability explanatory variables.
![Page 16: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/16.jpg)
1616
Evidence supports the claim that RRH reduces the risk of return to homelessness
![Page 17: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/17.jpg)
17
Additional PSM Analysis: Families vs. Singles
Single RRH Households Family RRH Households
*Returns to homelessness were significantly lower for households receiving RRH than for similar households that received usual care. Significance at 5% level of significance
![Page 18: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/18.jpg)
18
Lessons LearnedSystem transformation impacts of RRH data analytics project:• Accelerated state and local shift towards collective
understanding of local RRH impact on ending homelessness• Creative data visualization of analysis motivated broad
stakeholder engagement, system-wide program improvements
• Data analytic results were used by leaders to move support from temporary/transitional housing to RRH approach
• System-Rapid cycle evaluation technique (PSM) engaged social investment, motivated policy review based on evidence of RRH effects on ending homelessness
![Page 19: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/19.jpg)
19
Displaying Surveillance Data to Influence Resource Allocation
![Page 20: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/20.jpg)
20
BackgroundSuicide Deaths in Alaska – One of Highest in the Country
![Page 21: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/21.jpg)
21
Data and MethodsQuantify the Problem, Quantify the Solution
![Page 22: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/22.jpg)
22
Data and MethodsQuantify the Problem, Quantify the Solution
![Page 23: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/23.jpg)
23
Lessons LearnedData reports developed using effective visualization tools promoted positive results of training initiative in decreasing suicide risk
Communities, agencies, organizations and school systems now expanding their use of suicide prevention trainings for all staff
New suicide prevention policies and procedures are being developed locally to increase community awareness and actions on suicide risk, and prevent youth suicide death.
![Page 24: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/24.jpg)
24
Other experiences using existing data to influence policy?
![Page 25: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/25.jpg)
25
Recommendations• To empower communities to use data for program and policy
improvements, rapid evaluation methods and data literacy development is critical
• Knowledge sharing across multi-sector stakeholders requires inquiry, and the capacity to review what is and is not working with data as neutral evidence
• To motivate policy change around complex public health issues, both rigorous research evidence and the effective use of data visualization tools are critical to promote cross-sector understanding and political will
• Data-driven decision making is necessary for continuous, system-wide improvement planning that is geared towards continual mid-course corrections
![Page 26: AEA Presentation: Using Data to Influence Programs and Policy](https://reader031.fdocuments.in/reader031/viewer/2022011722/58eeab5e1a28ab2b448b45c9/html5/thumbnails/26.jpg)
26
Recommendations• Evaluation does not have to cost a fortune!
• Often there is a wealth of data available to communities for planning at little to no cost
• Look for opportunities – up front and throughout – to apply analysis results in a way that creates meaningful change
• When using administrative data – ensure you have a full understanding of the pros and cons – and what data cleaning may be required
• Include community stakeholders or practitioners in the analysis process