Evaluation and Reporting extension activities

Jeanette LongAg Consulting Co

Trials – why bother?

• R, D or E?

Why evaluate??

Why Evaluate?

• Measure practice change • Ensure accountability• Measure cost to benefit • Continuously improve• Account for unintended effects• Report to funding body

Evaluation starts at the beginning of the project!

Evaluation Steps

1. Clarify the evaluation request 2. Focus the evaluation – program logic3. Identify the information you will need4. Identify information sources and methods 5. Decide on methods of analysis and reporting 6. Prepare a timeline for the evaluation – its not

all at the end

Achievement of the desired outcome

Requires a change in practice – what people do

Which requires appropriate levels of Motivation, attitudes, knowledge, and skills

Created by activities that engage people and create technologies (trials, field days, farm walks, case studies, communications, new

products etc)

Which require resources (money, time, expertise, infrastructure, human resources)

Program logic

Types of data

• Quantitative – where responses can be easily measured– “like it” scale 1-5, numbers of people etc

• Qualitative – responses from open ended questions – What changes did you make in your stubble

management as a result of attending the workshop?

Collecting the data

• Numbers of people, hits on website etc• Surveys – happy sheets • Dart boards• Facilitated discussion• Survey monkey • Case studies • Observation • Stories of significant change• Keypads• Narratives

Golden rules of questions

• Each question has ONE request in it• Use open questions • Ask exactly what you need to know and no

more• Ask about attitudes, beliefs, behaviour and

motivation• Avoid testing knowledge • Test your questions before using them

Overall how beneficial has the day been for you?

To what extent have you learnt new things relevant to your business ?

How well did the process work for you?

How likely are you to take some action from what you saw?

Multiple choice questions

• As a result of your participation in the field days for the BWY project, what actions have you taken with respect to BWY?– Followed up options with my consultant– Commenced IPM– Changed variety– Use seed dressing– Nothing – don’t have the resources at this stage– Other:

Practice Change question

Narrative

• Who – Farmer X in district Y farms X• Activity – he attended …• Reaction – was impressed by …asked lots of

questions• Action – what he/she did (practice change)• Impact – what impact had on business and

what next

Observations watching and listening

• What is the goal• Who is involved• Activities • Actions • Language • Feelings

Be aware of your own bias’s

ORID questioning approachFocused conversation

• Objective – facts– What did we do today?

• Reflective /reactive – emotions, feelings– What surprised you? How did you feel?

• Interpretive – value, meaning, purpose– What were the main messages?

• Decisional – future steps – What will you apply on your farm?

Develop Program logic and monitoring & evaluation plan

M & E data collection

Data analysis

Summarise against performance measures

Report against plan