Evaluation and Out of Home CareIain Matheson

ACWA Conference 2008 ACWA Conference 2008

1. What is Evaluation Research?

2. Overseas Out of Home Care Examples

3. Commissioning Evaluation

4. Some Issues for Out of Home Care

5. Developing an Evaluative Management Culture

Evaluation Research – a Definition

Professional evaluation is defined as the systematic determination of the quality or value of something (Scriven, 1991).

What is evalu(e)ation research?

Summative and formative What can be evaluated? Evaluation questions Specific logic, methodology and techniques Logic models Same data and analysis methods as applied research Evaluation v research standards for quality

Evaluation as Knowledge Production

Evaluation to Inform Decisions

Some Evaluation Approaches

CIPP model (Context, Input, Process and Product) Empowerment evaluation Experimental and quasi-experimental Fourth Generation evaluation Goal-free evaluation Indigenous-led research/evaluation

More Evaluation Approaches

Participatory evaluation Rapid reconnaissance Stakeholder evaluation Strategic evaluation Theory-based evaluation Utilisation-focused evaluation

Overseas Out of Home Care Examples

Context

Cashmore & Ainsworth (2004)

Bromfield et al (2005)

Not alone

Current Pilot of Oregon Social Learning Centre model York U and Manchester U Funded by Dept. for Education and Skills 15 evaluation sites Quasi-experimental

Treatment Foster Care in Eng. Project

Youth Horizons Trust SCD Bridging Programme - NZ

2005 SCD residential programme Centre for Social Research and Evaluation Funded by Ministry of Social Development Literature review, document analysis, observation &

interviews

Community Alternative Placement Scheme - Scotland

2002 Action for Children (NCH) project - family

placements alternatives to secure care. Moira Walker, Malcolm Hill and John Triseliotis Funded by Scottish Executive Document/data analysis and interviews –

longitudinal with control group

Tri-County Treatment Foster Care Program - Canada

Current Tri-County Treatment Foster Care Program - hard-

to-serve children Centres of Excellence for Children’s Well-being Document/data analysis and interviews –

longitudinal with control group

Foster Care to College Partnership - US

Current Completion of schooling by yp in foster care and

take up post-compulsory education Washington State Institute of Public Policy Mason Burley Administrative data and statistical analysis

Commissioning Evaluation

Scoping Budget Resources/commitment – and stakeholders Clarity on what is being evaluated Methodological robustness A matter of...timing Dissemination

Other Issues for Out of Home Care

Difficult area to research Complex intervention logic Pilots, pilots, pilots What counts as good evidence? Data v information v evaluative judgements Still not widely used in out-of-home care

Developing an Evaluative Management CultureAll forms of learning activity that aim to improve: Focus Impact Outputs and Outcomes

Thanks!

Iain Mathesoniain@mathesonassociates.co.nz

References

Alkin, M. (2004) Evaluation Roots: Tracing Theorists’ Views and Influences, Thousand Oaks, Sage

Beddoe, L., Fouche, C., Harington, P., Light, G., Lunt, N., & Yates, D. (2007). Growing research in practice; The story so far. Social Work Review, 2007(Autumn), 39 – 50.

Bromfield, L. M., Higgins, D. J., Osborn, A., Panozzo, S.,& Richardson, N. (2005). Out-of-home care in

Australia: Messages from Research. Melbourne: National Child Protection Clearinghouse, Australian

Institute of Family Studies.

Burley, M. (2007). Foster care to college partnership evaluation: Program overview and research design . Washington DC: Washington State Institute of Public Policy.

Cashmore, J. and A. F. (2004). Audit of Australian out-of-home-care research. Sydney: Association of Child Welfare Agencies.

Centre of Excellence for Children’s Well-being (2008). "Intervention with attachment disordered children in the child welfare system." Retrieved on 28 July from http://www.cecw-cepb.ca/projects/attachment_disorders_e.ht

Davidson, E. J. (2005). Evaluation Methodology Basics: The nuts and bolts of sound evaluation . Thousand Oaks, CA: Sage.

Fox, M., Martin, P., & Green, G. (2007). Doing practitioner research. London: Sage.

Patton, M. Q. (2002). Qualitative research and evaluation methods (3 ed.). Thousand Oaks , Calif. : Sage Publications.

Saville-Smith, K. (2005). Evaluation of the Youth Horizons Trust SCD bridging programme report . Ministry of Social Development, Wellington.

Scriven, M., (1991) Evaluation thesaurus (4th ed.). Newbury Park, CA: Sage.

Walker, M., Hill, M. & Triseliotis, J. (2002). Fostering and secure care: An evaluation of the community alternative

placement scheme (CAPS), Edinburgh: Scottish Executive.