Implementation Evaluation of Canada’s At Home/Chez Soi Housing First Program Geoffrey Nelson, National Research Team, At Home/Chez Soi Project, Dept. of Psychology, Wilfrid Laurier University, Waterloo, Ontario, Canada Tim Aubry, National Research Team, At Home/Chez Soi Project, School of Psychology, University of Ottawa, Ottawa, Ontario, Canada 9th European Research Conference – Homelessness in Times of Crisis, FEANTSA, Warsaw, Friday, September 19, 2014
/ 2
Overview of Presentation
• Background and Description of the At Home/Chez Soi Study • Fidelity Findings • Qualitative Implementation Findings
/ 3
At Home/Chez Soi Project Description
In 2008 the Federal government allocated $110 million
At Home/Chez Soi is:
The largest study of its kind in the world (>2100 participants)
Pathways Housing First model
Implemented in 5 Canadian cities: Vancouver, Winnipeg, Toronto, Montreal, Moncton
Funding – 85% services / 15% research
A randomized controlled trial
/ 4
Housing First
• Housing is immediate and permanent • Recovery & choice emphasized • Housing is primarily scattered site, private market • Supports include guaranteed rent subsidies – clients pay 30% • Additional services (>1 visit) are voluntary • No readiness/maintenance conditions • Tenancy rights • Draws upon Pathways to Housing program model • Two service delivery approaches (ACT and ICM) for two different populations (High and Moderate Need)
/ 5
/ 6
Vancouver
•Outcomes of individuals with serious substance use issues • 3rd arm – a congregate setting
/ 7
• Focus on Aboriginal homeless population • 3rd arm included traditional healing approaches (e.g., medicine wheel)
Winnipeg
/ 8
Toronto
• 3rd arm – ethno-culturally appropriate based on anti-racist/anti-oppression model • Also emphasis on primary care/chronic physical illnesses
/ 9
• Intervention included test of a vocational approach – individualized placement and support (IPS)
Montreal
/ 10
Moncton
• Blended intervention - “flexible ACT” (FACT) model • Focus on smaller cities and rural areas
/ 11
Implementation of Housing First
• National and local training events • Technical assistance visits • Communities of practice • Quality Assurance Team conducts fidelity assessments
/ 12
What is the Purpose of the Mixed Methods Fidelity and Implementation Evaluation?
Quantitative fidelity assessment • Looks at degree of implementation of critical ingredients
• Rates programs numerically using scale
• Provides feedback for quality improvement
Qualitative process and implementation evaluations uncovers: • Strengths
• Challenges/Trouble spots
/ 13
Can Housing First Be Adapted to Local Context and Culture?
Yes, but must be careful that the adaptation does not deviate from Housing First elements/principles • Toronto’s Anti-racism, Anti-oppression ICM program
• Winnipeg’s Aboriginal-focused medicine wheel program
But “no,” when the adaptation goes against the basic principles of Housing First, because it enters a “zone of drastic mutation” (Hall & Loucks, 1978)
Housing
First
Fidelity
Adaptation
/ 15
Why is Housing First Fidelity Important?
Fidelity matters because it is related to outcomes. The greater the fidelity, the better the outcomes of Housing First
• Gilmer et al. (2014) • Davidson et al. (2014)
/ 16
Relationship of Fidelity with Program Processes and Participant Outcomes
• Higher fidelity in the in the 12 programs (5 ACT, 7 ICM) was related to greater direct and indirect service time and more contacts with participants (corr. = .55 to .60)
• Higher fidelity was associated with greater housing stability (% of time in housing) (OR = 1.11) and larger improvement in quality of life (d = .10) and community functioning (d = .11) over the course of the study
/ 17
Housing First Fidelity Domains
Housing and recovery-oriented approach – 37 item scale with items falling into 5 domains; each item completed for each program
• Housing choice and structure • Separation of housing and services • Treatment philosophy (recovery orientation) • Service array • Program structure
/ 18
Sample Housing First Fidelity Items
Housing choice and structure domain
• Housing choice • Housing availability • Permanent housing • Affordable housing • Integrated housing • Privacy
/ 19
Housing Availability Item – Anchors
All items are rated on a 4-point scale, with 4 indicating high fidelity, and 1 indicating low fidelity
• Extent to which program helps participants move quickly into units of
their choosing.
1. Less than 55% of program participants move into a unit of their choosing within 6 weeks of having a housing subsidy or receiving a voucher
2. 55-69% of program participants move into a unit of their choosing within 6 weeks of having a housing subsidy or receiving a voucher
3. 70-84% of program participants move into a unit of their choosing within 6 weeks of having a housing subsidy or receiving a voucher
4. 85% of program participants move into a unit of their choosing within 6 weeks of having a housing subsidy or receiving a voucher
/ 20
Fidelity Assessment External quality assurance team
Data Collected Time 1 – 9-13 Months of Program Operation
Time 2 – 26-31 Months of Program Operation
Staff Interviews 84 89
Consumer Focus Groups 10 11
Chart Reviews 100 102
Number of Programs
10 10
Number of Sites 5 5
/ 21
Summary of Fidelity Rating Findings
• 10 programs rated on 37 fidelity items in 5 domains • Overall, strong fidelity to the Housing first model • Also, improvement noted from early to later
implementation (71% program ingredients rated above 3 on a 4-point scale at early implementation and 78% of program ingredients rated above 3 at later implementation)
• Average fidelity rating across all items is 3.47 at early implementation and 3.62 at later implementation
• Variation at individual program level and by service delivery type (e.g., more challenges for ICM regarding service array domain)
• Both fidelity and adaptation are important
/ 22
Qualitative Implementation Evaluation • Aim: to better understand the reasons for implementation
fidelity strengths and challenges/trouble spots in the At Home/ Chez Soi project • Qualitative implementation evaluations were conducted in years one
and three of the project
• Primary questions: 1. What are implementation strengths?
2. What are implementation challenges/trouble spots?
/ 23
Qualitative Implementation Evaluation Site qualitative research teams and national qualitative research team
Data Collected Time 1 – 9-13 Months of Program Operation
Time 2 – 26-31 Months of Program Operation
Key Informant Interviews
72 36
Focus Groups 35(211) 17(99)
Number of Programs 10 10
Number of Sites
5 5
/ 24
Methods • Sampling and sample • Purposeful sampling
• 283 participants total at T1 and 135 total at T2
• Data collection • Interviews in English or French
• At participants’ workplaces or site offices
• Interviews audio recorded and transcribed verbatim
• Data analysis • Thematic analysis for site reports and cross-site report
/ 25
Results – Q1: Implementation Strengths • Partnerships and collaboration between services teams, and
with participants, community organizations, and professionals were important strengths • One respondent at the Vancouver site stated, “I’ve never worked in such a
supportive and collaborative place. The team is so committed to what it
does, and I think it’s reflected in our communication and the changes
we’ve seen in our clients.”
/ 26
Results – Q1: Implementation Strengths
• Despite low vacancy rates and lack of affordable housing in every city, all sites reported success with specific aspects of housing and rehousing participants • Across sites, this was achieved primarily by 1) creatively matching
participants with housing units; 2) learning from past experiences; 3)
achieving stability; 4) planning preventative strategies to avoid future
rehousing incidents; and 5) forming collaborative relationships with
landlords and property management companies
• “Compared to other landlords, those in our project get a great deal of
support… We underestimated how important that support is. It’s like an
extra set of hands helping them do their job.” (Vancouver team member)
/ 27
Results – Q1: Implementation Strengths
• Presence of dedicated specialists and professionals on housing and service teams • For example, the Moncton site
reported that the addition of a home
economist, psychiatric consultant, and
vocational specialist to their ACT team
enabled program delivery and support
• Toronto noted that their service
provision success could be attributed
to the high level of clinical expertise
from team leaders and psychiatrists
/ 28
Results – Q2: Implementation Challenges/Trouble Spots
• Staffing • Substantial staff turn-over across sites due to heavy workloads, pressure
to meet project timelines, and concerns about job security
• This is a common problem in these types of programs and not something
unique or unusual about the At Home/ Chez Soi project
• Scattered-site housing model • Vancouver, Winnipeg, Montreal, and Moncton faced challenges related to
scattered-site being the sole model of housing for the project, particularly
for people with more complex physical and mental health challenges
• Winnipeg suggested congregate-type housing as an alternative;
Vancouver introduced the idea of “step-up, step-down” options; and
Moncton opened a transitional apartment building
/ 29
Results – Q2: Implementation Challenges/Trouble Spots
• Participant feelings of isolation and loneliness • Isolation is a broader social issue which is more
acute for this population
• Sites attributed these difficult experiences to
participants’ changing social networks, leaving
friends behind, and vast lifestyle and living
arrangement changes
• “…the person that is addicted lives a life of
loneliness… You can’t be around your friends, so
the drug is your best friend… An agency has to
make sure that you can be there to complement
that… until they have been able and have had time
to build new friends.” (Moncton key informant)
/ 30
Results – Q2: Implementation Challenges/Trouble Spots
• Some sites had problems integrating peer support into the program • Some difficulty identifying and training potential peer specialists
• Moncton and Winnipeg sites had concerns about appropriately defining
and clarifying the roles that peer support workers should play on service
teams
• Documentation • Sites noted the need to more clearly document housing and service
teams’ work in their case notes
• Certain implementation challenges and low implementation fidelity for
certain criteria could be attributed to inconsistent record and chart-
keeping by team members
/ 31
Results – Q2: Implementation Challenges/Trouble Spots
• Most sites identified some trouble spots regarding housing between the second and third years of implementation • Challenges around finding good quality, affordable, vacant housing units in areas in
which participants desired to live
• Housing restrictions related to smoke-free and pet-free policies (Vancouver)
• In Toronto, issues of tenancy management, relationships between housing teams
and participants, dealing with unit damage, and rehousing participants emerged as
trouble-spots in housing
• “So sometimes, the only interaction we had with clients was when there was
chaos, when their was a real problem that was going on… I mean, the landlords
are calling us and complaining… but we have no idea who the person is so that
was a disadvantage to me that’s going forward.” (Toronto team member)
/ 32
Results – Q2: Implementation Challenges/Trouble Spots
• Vocational and educational goal supports • The Vancouver site reported that initially engaging
and/ or re-engaging participants in work, volunteer,
and educational activities had become difficult
• Vocational specialists often had to tend to
emergency crises rather than develop more
proactive, long-term goals with participants
• Toronto team members found limited variety and
availability of educational and employment
opportunities within the community
/ 33
Conclusions • Findings provide information about identifying and
maintaining strengths, as well as identifying and potentially preventing challenges or trouble spots when implementing complex community-based housing interventions
Top Related