Questionnaire design: Module 8; Quantitative 1 Educational research some basic concepts and...
date post
25-Mar-2019Category
Documents
view
212download
0
Embed Size (px)
Transcript of Questionnaire design: Module 8; Quantitative 1 Educational research some basic concepts and...
8Module
UNESCO International Institute for Educational Planning
Maria Teresa Siniscalcoand Nadia Auriat
Questionnaire design
Quantitative research methods in educational planning
Series editor: Kenneth N.Ross
International Institute for Educational Planning/UNESCO7-9 rue Eugne-Delacroix, 75116 Paris, FranceTel: (33 1) 45 03 77 00Fax: (33 1 ) 40 72 83 66e-mail: information@iiep.unesco.orgIIEP web site: http://www.unesco.org/iiep
The designations employed and the presentation of material throughout the publication do not imply the expression of any opinion whatsoever on the part of UNESCO concerning the legal status of any country, territory, city or area or of its authorities, or concerning its frontiers or boundaries.
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means: electronic, magnetic tape, mechanical, photocopying, recording or otherwise, without permission in writing from UNESCO (International Institute for Educational Planning).
Graphic design: Sabine LebeauTypesetting: Sabine LebeauPrinted in IIEPs printshop
September 2005 UNESCO
Quantitative research methods in educational planning
These modules were prepared by IIEP staff and consultants to be used in training workshops presented for the National Research Coordinators who are responsible for the educational policy research programme conducted by the Southern and Eastern Africa Consortium for Monitoring Educational Quality (SACMEQ).
The publication is available from the following two Internet Websites:http://www.sacmeq.org and http://www.unesco.org/iiep.
mailto:information@iiep.unesco.orghttp://www.unesco.org/iiephttp://www.sacmeq.orghttp://www.unesco.org/iiep
Educational research: some basic concepts and terminology Module 1
Content
1
1. Introduction 1
2. Initial planning 3Why a new questionnaire and when? 3
Relationships between research problems, research hypotheses, and variable construction 5
Characteristics of research hypotheses 6
Specifying variables and indicators 6
Operationalization of research questions 8
The problem of the cross-national validity of educational concepts, definitions, and data collection instruments 12
1. What is formal education? 12
2. Distinguishing public and private service providers 14
3. What is a school? 16
4. What is a student? 17
5. What is a teacher? 18
3. The design of questions 22Question structure 221. Closed questions 23
2. Open-ended questions 26
3. Contingency questions 28
Guidelines for writing questions 29
UNESCO
Educational research: some basic concepts and terminologyModule 1
II
Specifying the characteristics of respondents 34
A checklist for reviewing questionnaire items 35
4. Examples of questions 38Student background 381. Gender and age 39
2. Socio-economic background: occupation, education, and possessions 41
Teacher characteristics 47
School location 49
Learning, teaching, and school activities 511. Student reading activity 51
2. Teacher activities 52
3. School head activities 53
Attitudes, opinions, and beliefs 551. Likert scaling 55
5. Moving from initial draft to f inal version of the questionnaire 63Two widely-used patterns of question sequence 63
General guidelines for item placement 65
Covering letters and introductory paragraphs 66
Drafting instructions for answering questions 68
Training of interviewers or questionnaire administrators 70
Pre-testing the questionnaire 72
Basic steps in pre-testing 75
Reliability and validity 761. Validity 76
2. Reliability 78
UNESCO
The codebook 79
6. Further reading 84
III
Content
1 UNESCO
Introduction 1
This module provides guidance for the design of standardized questionnaires that are to be administered in school systems to students, teachers, and school heads. The module is divided into four sections that cover initial planning, the design of questions, examples of question types, and moving from a draft to a final questionnaire.
After reading this module, the reader should be able to design a quality survey questionnaire that is suitable for addressing the research issues at hand. He or she will know how to:
Decide on the target population for the questionnaire.
Identify the variables and indicators that will address the research issues and hypotheses on which data are to be collected.
Develop demographic, knowledge, attitude, and practice questions.
Close open ended quantitative and qualitative questions and design skip, filter, and contingency questions, where appropriate.
Decrease response bias and maximize response rates.
Design probe questions and interviewer or respondent instructions on the questionnaire.
Questionnaire designModule 8
2 3 UNESCO
Conduct a pilot test of the questionnaire, and adjust its final design according to the results.
Prepare a codebook for data entry.
Questionnaire designModule 8
2 3 UNESCO
This section reviews the steps required to determine the need for a new questionnaire, and looks at how a general research problem needs to be translated into a number of specific research questions and hypotheses. It examines the problem of valid cross-national instruments and provides helpful hints and recommendations for using comprehensive and precise definitions of key educational concepts.
Why a new questionnaire and when?This module addresses the planning and design of standardized questionnaires. A formal standardized questionnaire is a survey instrument used to collect data from individuals about themselves, or about a social unit such as a household or a school. A questionnaire is said to be standardized when each respondent is to be exposed to the same questions and the same system of coding responses. The aim here is to try to ensure that differences in responses to questions can be interpreted as reflecting differences among respondents, rather than differences in the processes that produced the answers.
Standardized questionnaires are often used in the field of educational planning to collect information about various aspects of school systems. The main way of collecting this information is by asking people questions either through oral interviews (face to face or telephone), or by self-administered questionnaires, or by using some combination of these two methods.
Initial planning 2
4
Questionnaire designModule 8
5
Initial planning
UNESCO
Although survey research, by definition, implies the use of some form of questionnaire to be administered to a sample of respondents, the questionnaire is simply one instrument that can be employed in the study of a research problem. As such, it may or may not be the most suitable tool for the task at hand.
Hence, before deciding on the need for a new questionnaire, one should consider whether or not some of the required information may already be available from other sources, for example, from statistics compiled by governments or research agencies, or from survey research archives. One should also consider whether a suitable questionnaire already exists that could be wholly or partially used.
The planner should also consider whether other means of data collection are more appropriate. These can be (a) field experiments, where people in treatment and control groups respond to a scenario devised by the investigators, (b) content analysis of newspapers or articles, (c) direct observation (such as counting the number of schools in a district, the number of blackboards in a school or the number of students per teacher in a given area), or (d) non-directive interviews where there are no pre-specified questions and the interviewer has a great deal of freedom in probing areas and specific issues during the course of the interview.
Among the types of information that can be collected by means of a questionnaire are facts, activities, level of knowledge, opinions, expectations and aspirations, membership of various groups, and attitudes and perceptions. In the field of educational planning, the information that is collected can be classified broadly into: (a) inputs to education (such as school resources or various background characteristics of schools, teachers or students), (b) learning and teaching processes, and (c) the outcomes of education (such as pupil achievement, attitudes towards school, and measures of school efficiency such as survival rates etc.).
4
Questionnaire designModule 8
5
Initial planning
UNESCO
Relationships between research problems, research hypotheses, and variable constructionThe development of a questionnaire commences with the transformation of general educational research and policy concerns into specific research questions for which the data are intended to supply an answer. Some examples of general educational policy and research concerns are: (a) policy-makers want to assess the supply of resources in their primary schools, (b) a curriculum expert wants to determine to what extent teaching methods explain differences in reading literacy among 9-year-old students, and (c) a national evaluation agency wants to investigate student attitudes towards science at the end of secondary school.
In the case of the above three examples, it would be necessary to establish empirical evidence for decision