CONFERENCE EVALUATION

DATA COLLECTION

WHEN TO COLLECT DATA?

Survey,Interviews, etc.

Polling,Observation,Focus groups,Interviews, etc.

Interviews,Surveys, etc

WHAT TO COLLECT?

You have to move from the objectives you want to evaluate to concrete questions – “operationalisation”

“Assess what were key learnings of the conference”

Change to knowledge

Which of the following issues did the conference address for you?

To what extent did the conference increase your knowledge on the following issues?

You then need to put questions into the appropriate surveys, interview guides, etc.

HOW MUCH DATA TO COLLECT?

The number or survey responses needed or persons interviewed largely depends on the size and type of the conference

You are not trying to achieve scientifically valid sampling but collecting enough feedback to make your evaluation credible and representative

Some general recommendations:

Surveys: aim to have 30 – 60% of all conference participants respond to your survey

Interviews: 8-12 interviews per demographic group is usually sufficient to gather enough feedback

GUIDANCE FOR DATA COLLECTION

Evaluators have to be impartial, cooperative and respectful

Good data collection guarantees anonymity to participants

Collect only data you need – evaluations tend to collect too much data that is never used

Develop a feeling for when you have collected enough data – this is called “saturation” – e.g. in interviewing when you keep hearing the same things with no variations

THE INTERVIEW - DATA COLLECTION

Semi-structured interviews often work well in conference evaluations

When interviewing:

Ask for amplifications, examples and have participants define terms

Use phatic language to encourage responses “I see..” “ah huh..”

Listen to what the respondents don’t tell you

Write up your notes as soon as you can after the interview

THE SURVEY - DATA COLLECTION

General advice:

Key when to give out the survey – during, at the end or after the conference (paper vs. online versions….)

Keep the survey short! Always link the survey questions back to your evaluation objectives to avoid “survey creep”

Start with more general questions (e.g. attitudes) and move to the more specific (e.g. demographic)

No need to reinvent the wheel – look at other conference surveys and use commonly used question scales and terms

Repeat questions from year to year to allow comparison

Test your survey before deployment

THE SURVEY - DATA COLLECTION

Structuring the survey:

THE SURVEY - DATA COLLECTION

Type of questions

Different question types are suited to different types of information sought:

Open questions

Closed questions:

Likert scale

Rank order scale

Multiple choice (singular response)

Multiple choice (plural response)

Binary (e.g. yes/no)

Numerical response

THE SURVEY - DATA COLLECTION

Writing question

All surveys should have at least one open-ended question - often useful for the “why” and the “how”

Only ask one piece of information per question

Be consistent in the use of terms and words

Ensure that questions are answerable

Avoid biased questions

Avoid having similar questions with same response set

For closed questions with pre-listed responses, make sure list is exhaustive (include “other” if in doubt)