SOWK 6003 Social Work Research

Week 7 Designs for Evaluating Programmes and

Practice – Experimental designs

By Dr. Paul Wong

Purposes of Research

1. Exploration

2. Description

3. Explanation

4. Evaluation

5. Multiple purposes

Program evaluation refers to the purpose of research

rather than to any specific research methods.

Purposes of program evaluation

1) to assess the ultimate success of programs;

2) to assess problems in how programs are being implemented; or

3) to obtain information needed in program planning and development

1990s in the US: program evaluation as a prerequisite for approving grant applications

Two major types of evaluations

Summative – are concerned with assessing the success of programs

Formative – are concerned on obtaining information that is helpful in planning the program and in improving its implementation and performance

Politics

The evaluation, itself, can be dealt with quite straightforwardly.

However, the POLITICS of it can be very difficult to handle.

Some common obstacles

• In-house vs. External evaluators

• Utilization of the findings: – The implications may not always be presented in a way that

nonresearchers can understand;

– Evaluation results sometimes contradict deeply held beliefs.

• Logistical and Administrative problems: – Learning as much as possible about the stakeholders;

– Maintaining ongoing mutual feedback between them and the evaluator;

– Tailoring the evaluation and its reportage to their needs and preferences as much as possible without sacrificing scientific objectivity

Types of Program Evaluation

1. Evaluating outcome and efficiency: – should strive to enhance causal inference by

using the most internally valid experimental or quasi-experimental design possible

– The assessment of efficiency asks whether program outcomes are being achieved at a reasonable cost and applies the principles of cost accounting to calculate the ratio of program benefits to costs

2) Process Evaluation – aims to investigate the “means” , not the “ends”, and focus on identifying strengths and weaknesses in program processes and recommending needed improvements e.g., (pp.323)

– What types of individuals are not being reached by the service?

– Are clients satisfied with services? Why and why not?

– Tends to rely heavily on qualitative methods

3) Needs assessment

A diagnostic evaluation

Common techniques:

− Key informants

− Community forum

− Rate under treatment approach

− Social indicator

− Community survey

− Focus groups

Randomized Controlled Trial – What is it?

•

In class activities1) Break into small groups and design program evaluation

studies regarding this class. How would they assess the need for it, its implementation, and its outcome in attaining

its goals?

Articles Critique: 1) Cheung Chau Suicide Prevention Programme

2) A local evaluation study using a RCT methodology