Monitoring and evaluation PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION.

Monitoring and evaluation

PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION


Focus of this workshop session

Monitoring Evaluation

What they are?Why they are important?

Who is involved? PlanningMethods

Psychosocial interventionsIn

dica

tors


Write a word or phrase that you associate with the term ’monitoring’ on the cards you have received.

When you have finished, display your cards on the board/wall space.

If someone has written the same word as you then put your card on top of that card.

What is monitoring?


Monitoring is the regular and continuous process of collecting and analyzing data to assess progress and development.

What is monitoring?


Two main types of monitoring in RCRC

Process oriented

Progress and development of response

Activities implemented as planned?

Use of resources

Problems? How should they be dealt with?

New opportunities for improving response?


Two main types of monitoring in RCRC

Results oriented

Direct results of the interventions

Relevance of present and planned activities

Are objectives still realistic/relevant?

Changes in targeted population / environment (impact)

Is more information needed?


Is the response still relevant to the needs of the population?

What is monitoring?

Keeps track of inputs, outputs and outcomes using indicators


What are inputs, outputs and outcomes?

Input

Any resources that is ‘put in’

to the program

E.g. funds, materials, personnel,

time

Outputs: Measurable achievements that have been ‘put out’ or produced as a result of the interventionE.g. numbers of people trained, numbers of meetings held, etc.

Outcomes: Changes that have come about as a result of the programE.g. improvement of psychosocial wellbeing, skills and knowledge on PSS improved,


INDICATORS

Brainstorm:

What are indicators?


What are indicators?

• Criteria used to measure the data collected

• Measure changes related to implementation

Examples of (outcome) indicators in PS responses:

• Decrease in stress related symptoms of population A

• Increase knowledge and skills in providing PFA

• Children regain desire to play

• Beneficiaries regain capacity to relate to others


Examples of Input, output and outcome indicators

Input

Indicators: AMOUNT $

spent; NUMBERS of

personnel working;

NUMBERS of hours

Output indicators: NUMBERS of people trained; NUMBERS of meetings held; NUMBERS of PS kits distributed

Outcome indicators: NUMBER of children playing (quantitative); self-reported INCREASE in desire for social interaction (qualitative); LEVEL of skill improvement in PSS

Keeping track of inputs, outputs and outcomes for activities - An example

Logical framework approach Indicators

Overall Goal To reduce suffering and risk for development of severe trauma of Population A

Adults and children show healthy signs of coping with impact of crisis event

Activity 1 Training in PFA

Input Resources to enable training Amount of money; personnel; training manuals; materials

Output Training has taken place Number of people trained

Outcome Volunteers can provide PFA Skill level in PFA has increased

Activity 2: Workshops with children

Input Resources for workshops Amount of money; personnel; training manuals; materials

Output Workshops are held Number of workshops held; number of children attended; number of trainers

Outcome Children are coping better Increase in children’s playfulness, self-confidence, trusting of others


Why is monitoring important?

1. Relevance

2. Responsibility and communication

3. Accountability

Simply put: Are we on track? Are we on the RIGHT track?


Who is involved in monitoring?

Responsibility

Including monitoring and evaluation

Data collection

Data collection

Other partners


In buzz groups of 2 or 3, discuss amongst yourselves what evaluations are and how they differ from monitoring?

Try to write a simple and clear definition of what an evaluation does in a psychosocial response.

What are evaluations?



“Evaluations explore whether the interventions succeeded in achieving the overall goals and aims of the psychosocial response”

Discussion points: 1. Evaluations are meant to be OBJECTIVE – what does it

mean and why is it important?



Evaluations = Monitoring + more

Final evaluationMid term evaluation

M1 M2 M3 M4 M5 M6 M7 M8

Evaluations focus on assessing OVERALL AIM/GOAL













Types of evaluations

Real time evaluations

•Early in response•Internal staff•Focus: process / operational issues

Mid-term evaluations

•Assess impact•Accountability (budgets/admin)•Guide needed adaptations

Final evaluations

•Impact and process•key successes & challenges•Invaluable info for future PS interventions


Evaluating impact

• How do you measure the impact of an intervention? • What do you need?

Data Data

DataData

Data

SAME INDICATORS


How to define Psychosocial indicators

• Assess - How is the population affected?

• What kind of change is desired?

(Goals/objectives/activities of response)

• What is the local definition of psychosocial wellbeing?


Indicators of psychosocial wellbeing


Psychosocial indicators

1. Write three things about yourself or your life that you think show to others that you are feeling good and doing well.

2. Now think and write about how others could measure

whether you felt better from one day to the next.

3. In plenary share some of your indicators of wellbeing. We

aim to come up with a good variety of indicators, and also

some indicators that may be the same for the whole group.



People have different things that indicate their psychosocialwellbeing, but if we spent some time we would probably beable to find some common indicators that everyone agreewould be both general, and specific, enough to determinethe groups’ psychosocial wellbeing.

Indicators of psychosocial wellbeing focus on: - How the individuals are doing- How the group is doing.

The nature of psychosocial is a focus on the individual’s psycheand on the social interaction and network.



We need to

help the

child

ren fe

el

better…

.

But how will we KNOW if

they feel better?

I KNOW..We will ask them how THEY know when they feel

better and then we will find ways to measure that…


Why is evaluation important?

1. Relevance (response relevant to needs of pop.)

2. Efficiency (time-frame, cost)

3. Impact (succeeded or not)

4. Effectiveness (objective of program met)

5. Sustainability (benefits continue beyond the program)


Who is involved in evaluations apart from staff, volunteers, partners?

Responsibility

Including monitoring and evaluation

Data collection

Data collection

Other partners

External consultants


Sampling

Broad & varied

Targeted populationBroad and varied


When to monitor and evaluate

PSYCHOSOCIAL INTERVENTIONS ASSESSMENT

How to conduct assessments?

Two main categories of data collection methods

Quantitative Qualitative Surveys, Key informant interviewsQuestionnaires Focus Group DiscussionsPsychometric tools Observations(measure reactions, behaviour, feelings, using a scaled measure) 1,2,3 Word descriptions

=, %, a:b












Are data needed for these indicators quantitative or qualitative? PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION


Collecting the data

1.Well planned and justifiable

2.Coordination

3.Clarifying aims and procedures

4.Participatory and collaborative

Ethical principals to data collection


Collecting the data

5. Comparison groups

6. Conduct and consent

7. Privacy and confidentially

8. Anticipate adverse consequences

Ethical principals to data collection


Planning for monitoring and evaluation

Group work: What needs to be planned to ensure good monitoring and evaluation?



1. Timing

a. Response length (short vs. long term)

b. Appropriate time for affected pop. (ex. Do not plan

monitoring activities with in-school children during exam time etc)

c. Time to plan, implement, analyze, follow up (time and

resources should be allocated)

2. Training

a. Management on PSS

b. Data collectors

c. Program team – analysis and reporting



3. Resourcesa. Staff and volunteers

b. Financial

c. Time

d. Logistics

4. Information sharinga. Internal standards and formats

b. External – different audiences

(beneficiaries/donors/public)


Role and importance of M&E in PSS

• Helps us understand local – and relevant – meaning of psychosocial wellbeing

• Keeps us on the right track

• Helps us do our best, now and in the future

Monitoring and evaluation PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION.

Documents

Transcript of Monitoring and evaluation PSYCHOSOCIAL INTERVENTIONSMONITORING AND EVALUATION.