RESEARCH DESIGN

Vivek Patkarvnpatkar2004@yahoo.co.in

EPISTEMOLOGYONTOLOGY METHODOLOGY METHODS SOURCES

What’s outthere to know?

Which preciseprocedure to use?

Which datacan we collect?

How can we go aboutacquiring that knowledge?

What & how canwe know about it?

Links between the Basic Elements of Research

Research Strategy: A general approachto research determined by the kind of question that the research study hopes to answer

Research Design: A general plan forimplementing a research strategy. It givesdetails like whether the study will involve groups or individuals and how many variables

Research Procedure: An exact, step-by-stepdescription of a specific research study

Inductive Research StrategyObjective is to establish generalisation

(what, why questions)

Three principles -• Accumulation: knowledge grows by the

addition of further well-attested facts.• Induction: from valid statements

describing observations, valid generalisations can be inferred.

• Instance confirmation: belief in the validity of generalisation increases with number of instances that have been observed.

Deductive Research StrategyObjective is to test generalisation, hypothesis, theory (how questions, predictions)• Put forward a hypothesis that helps

building a theory.• Deduce conclusions.• Examine critically the logic of argument

in the theory. • Test the conclusions by gathering

appropriate data.• If data are not consistent with the

conclusions, the theory is judged as false, otherwise, temporarily supported (but not proven). [K. Pooper’s Principle of Falsification]

Retroductive Research StrategyObjective is to find mechanism or lay account for explaining (how questions, evaluation)

• To explain observable phenomena try to discover suitable patterns by constructing a model of them.

• If model correctly represents the patterns, the phenomena might be causally explained

• Test further consequences of the model, If tests are successful, existence of the proposed arrangement is acceptable.

• Repeat the whole model building process to explain known arrangements.

Abductive Research StrategyObjective is to obtain technical account from lay account (exploration, description, understand)

• Meaning and intentions which direct behaviour of the people is to be captured.

• Describe this inside view and do not impose outsider views.

• Turn the lay descriptions of social life to technical descriptions of social life.

• Develop a theory or theories; it is like grasping the unknown whole from the known parts.

PURPOSES

VALIDITYMETHODOLOGY

RESEARCHQUESTION

CONCEPTUALCONTEXT

Purposes: What issues to research and why?Conceptual Context: What theories will guide the study?Research Question: What is the focus and how various sub-questions are inter-related?Methodology: What techniques to be used?Validity: How one can go wrong? Why should one believe the results obtained?

Research Interest and Idea: selecting problem & expanding

Research Design Phase:1. specifying concepts and variables in the selected problem2. operationalising concepts to help measuring variables3. choice of data collection method - primary data: Survey : questionnaire, interview, observation Experiment, Field study, Case study, Content analysis - secondary data4. sampling : Probability or Non-probability

Empirical Phase: data collection and processing

Interpretive Phase: data analysis and report writing

FOUR PHASES OF RESEARCH

Design Purpose

• To provide comprehensive answers to the research questions

• To control the variables that may influence the research outcome

{For example, in a study on voting behaviour and group conformity the variable sex may be controlled by selecting only male subjects for the research}

• To provide a built-in system to check the factors that may affect validity of the research outcome

Research Design A plan guiding the process of collecting,

analysing, validating and interpreting data is called the research design.

Helps in determining whether the obtained interpretation can be generalised to a larger population or to a different situation.

Aims at eliminating as many alternative explanations for a phenomenon as possible.

Depends on the type of research, and time and money available.

Research Design Stages¶ Planning stage: identification, selection and

formulation of research problem; formulation of

hypothesis and its linkage with theory and literature ¶ Selection stage: drawing up the design of the

experiment or inquiry, definition and measurement of variables, sampling procedures, tools and techniques of gathering data

¶ Operational stage: drawing of the finances and

budgeting, recruitment and training of the staff ¶ Completion stage: analysis, interpretation and

validation of data and report writing

Research Design Elements

1) Types of information and proposed sources that would be tapped.

2) A strategy for gathering and analysing the data.

3) An outline of the procedure that will be employed for validating the results

4) An estimate of the time and cost to complete the study.

Research Design Scope• Data gathering strategy• Sample size and selection method• Instruments that would be used:

- questionnaire & interview - passive observations

- participative observations - desk study

• Justification for particular method selection

Ethical issues that may be involved and how will they be dealt with, should also be stated.

Interestin an issue

ResearchDesign

Doingthe research

Influences

ResearchPerspective- Biographical research- Grounded theory- Social representations

Theory- Epistemology- Assumptions- Previous research

ResearchQuestions

Methods

Components

Audiencesand writing

Intendedgeneralisation

Intendedcomparisons

Sampling

Criteria andstrategies forquality

TriangulationQual/QualQual/Quant

Resources

Constructing a Research Design

Features of Good Design

§ Built around a clear research question

§ Well linked to the theoretical background

§ Manageable in resources and time

§ Clear in decision about sampling and selection of the particular methods

§ Sensitive, flexible and adaptive to the field conditions

§ Open to new insights during the research

Research Methodology, Designs & Methods

Positivist Methodology(Quantitative Approach)

Postpositivist Methodology(Qualitative Approach)

ExploratoryDesigns

CriticalDesigns

CausalDesigns

DescriptiveDesigns

ExplanatoryDesigns

InterpretiveDesigns

FocusGroup

In-depthInterview

Partici-pativeStudies

SemioticStudies

ActionRes.Studies

Ehnog-phicStudies

Herme-neuticStudies

FieldSurveys

Multi-FactorExp.

SingleFactorExp.

CaseStudies

Maths.Models

Exploratory Designs (Quantitative Studies)

• To get insight about the research problem is the objective (pilot studies)

• These are small-sample designs• They are to help evaluate the

importance of variables in the study and drop those marginally related

• They help in hypothesis formulation and testing

• Data collection is done from both primary and secondary sources

Descriptive Designs (Quantitative Studies)

• An extensive picture of the phenomenon is obtained by using a large sample

• Field studies probe a few issues in-depth• Field surveys are broader; more popular• Cross-sectional studies are one-shot

assessment of a sample of respondents, which could be divided in different classes say by income, education, gender

• Longitudinal studies use panels of the same participants over a time period to assess the change in response and measure it

Causal Designs (Quantitative Studies)

• They are relational (how one or more variables are related to each other)

or• They are experimental to know cause or

causes of change in a variable or event (what leads to what)

• Elaborate tools from statistics labelled as Design of Experiments are employed depending on research problem and environment

Hawthrone Experiment: Randomly chosen few workers of the Western Electric company (Hawthorne, New Jersey, USA) were moved to a special location and subjected to variations in working conditions like lighting, period of rest and speed of machines; surprisingly, performance increased with both positive and negative changes.

Reason:- Workers being singled out for attention turned out to be the reason and such change is captured by intervening or confounding variables [Reactivity]

Explanatory Designs (Qualitative Studies)

• Used primarily for providing description as well as meaning of the phenomenon

• Prediction is also the goal• Use of simple statistical analysis is

made for explaining difference in responses from two or more groups

Interpretive Designs (Qualitative Studies)

• To establish a meaning to an event or social situation is the basic objective

• Concentrate on norms, standards, rules and values that are held in common and how these all influence human interactions

• Study of symbols, artifacts, beliefs or meanings attributed by people in the study situation is carried out to describe and explain the phenomenon in greater depth

Critical Designs (Qualitative Studies)

• To change people’s belief is the overriding objective

• To bring people to actions that are commensurate with accepted truth and goodness is another aim

• Being increasingly used in public administration, sociological and education research

Combined Research Designs1. Archival studies: study of historical

records to establish an understanding of the circumstances that characterise an event or period (documents like pamphlets, press releases, annual reports, newspaper articles)

2. Media analysis: use of more current records of different media is done (Content Analysis is employed extensively)

3. Artifact studies: study of objects to gain understanding of culture and values of groups (study of items in the dumping sites, advertisements)

DESIGN PROPOSAL

Purposes

Conceptual Context

ResearchQuestions

Methods

Validity

Introduction

Setting

Abstract

Pilot Studies

Theories, Hypotheses

Methodology

Logical, Statistical

Implications

Steps in Sampling Survey1. Statement of objective2. Definition of the population to be

studied3. Determination of sampling frame

(listing of all the elements)4. Determination of sample size5. Selection of sampling method6. Sample selection & field work7. Determination of sampling error

SAMPLING

PLAN

Types of Sampling

Probability Sampling Non-Probability Sampling

• Simple Random• Systematic• Stratified Random• Cluster• Multi-Stage• Multi-Phase• Successive• Sequential

• Convenience• Judgment• Quota• Snowball• Theoretical

Useful for longitudinal studies

Concentration Principle:Focus the sampling on thosecases, which are particularlyimportant for the study issue

Simple Random SamplingEach element or unit in the entire population has an equal chance of selection.

Selection Method: Use the following means to pick up the sample unit till the desired number of units are selected,- Lottery system- Random Number Tables

It is simple to implement but is time consuming and laborious when the sample size is large.

Replacement Method: Selected unit may be put back and could be selected againNon-Replacement Method: Selected unit is not considered again

Systematic SamplingSelection Method: The first unit is selected randomly, but thereafter, the units are selected according to a systematic plan (sampling interval).

Systematic sampling will be more precise only if units within the sample are heterogeneous.

It is simple in operation butshould be used with caution for the population with periodicity.

Every 3rd unit

If we have 500 people and we want a sample of 50 people then the sampling interval is 500/50 = 10. Thus select a random integer from 1 to 10. Say it is 4, then sampling unit will be 4,14, 24 and so on till 500 to get 50 units.

Stratified Random SamplingSelection Method: The population is divided in non-overlapping homogeneous groups called strata and sample unit is selected randomly from each strata either by equal allocation logic or in proportion to the number of units in a stratum or according to any designated scheme.

It provides a better cross-section of the population and brings a more precision in the estimate.

HindiUrdu Marathi

English Gujarati

Newspaper Readers

Cluster Sampling

Selection Method: Divide the population in large number of segments called clusters and select the required number of clusters either by equal or unequal probabilities of selection and all the units in the selected clusters are to be surveyed.

Though it is statistically less efficient than the simple random sampling, it is less costly for covering say, a large area.

Non-Probability Sampling• Convenience sampling is used in

exploratory research for getting a gross estimate of the results, without incurring the cost or time required to select a random sample.

• Judgment sampling is an extension of convenience sampling. For example, one may decide to draw the entire sample from one "representative" college, even though the population includes all colleges.

Non-Probability Sampling (2)

• Quota sampling is the non-probability equivalent of stratified sampling. Here the strata and their proportions as they are represent in the population are fixed. Then convenience or judgment sampling is used to select the required number of units from each stratum.

• Snowball sampling is used when the desired sample characteristic is rare or it is extremely difficult to locate respondents, like in the AIDS studies. Here one element leads to another and so on.

Theoretical Sampling• If the aim of the research is to

develop a theory, this type of sampling strategy is to be employed

• Here extent of the basic population and its features are not known in advance

• Repeated drawing of sampling elements with criteria to be defined again in each step

• Sample size is not determined in advance

• Sampling is finished when theoretical saturation has been reached

Categories of Research Design

1. Experimental:- Is there a cause-and-effect relationship between two variables?

2. Quasi-Experimental:- Is there evidence of a cause-and-effect relationship between two variables? (some ambiguity would persist)

3. Non-Experimental:- Is there a relationship between two variables? (no control for confounding)

4. Correlational:- Is there a statistical relation between two variables? (no explanation)

5. Descriptive:- What is the status of individual for a specific group of individuals?

Factorial Experimental Designs• Experiment is a study in which one or

more independent variables are varied and change in the dependent variable is measured. Thus, It could be of 2X2 or of higher dimension.

• Validity is checked by employing Analysis of Variance (ANOVA) technique. (Statistical software packages are available to assist this part)

• Complete Randomisation, Randomised Blocks, Latin Squares & Graeco-Latin Squares are the common frameworks for designing experiments to eliminate bias.

Informal Experimental Designs

a) Before & after without control design:

Exp. Performance Treatment Performance

group before treatment given after treatment

(X) (Y)

Treatment effect = (Y) – (X)

Informal Experimental Designs

b) After-only with control design:

Exp. Treatment Performance

group given after treatment (Y)

Control No Performance

group treatment without treatment (Z)

Treatment effect = (Y) – (Z)

Informal Experimental Designs

c) Before & after with control design: Time 1 Time 2 Exp. Perf. before Treat. Performance group treat. (X) given after treatment (Y)

Control Perf. before Treat. Performance group treatment (A) given after treatment (Z)

Treatment effect = (Y- X) – (Z-A)

No Follow Up

Follow Up

Token GiftNo Gift

30% (30/100)

50% (50/100) 60% (60/100)

45% (45/100)

52.5% (105/200)

55% (110/200)

37.5% (75/200)

40% (80/200)

2 X 2 Design : To test the effectiveness of token gifts and follow up reminders alone and in combinationon the response rate in a mail survey.

1. Improvement by follow up : 55% – 37.5 % = 17.5%

2. Improvement by token gift : 52.5% – 40 % = 12.5%

3. Maximum response rate by follow up & token gift (60%)

Conclusions:

Some Experimental Designs

D A C C

B D B A

D C B D

A B C A

D B C A

B D A C

C A D B

A C B D

I C B A D

II A B D C

III B C D A

IV A D C B

B γ A β D δ C α

A δ B α C γ D β

D α C δ B β A γ

C β D γ A α B δ

Complete Randomisation

Latin Square Graeco-Latin Square

Randomised Blocks

Elaborate Checking

♦ Internal checking to safeguard against selecting wrong pattern as true

♦ Avoiding faulty generalisation from single event or handful observations

♦ Logical testing of arguments and examining the basic assumptions

♦ Filling up information gaps♦ Checking for coherence (a causal relation

among observations must be established)

Contd…

Elaborate Checking (2) ♦ Carrying out risk assessment♦ Breaking the creation to examine for

any flaw in any of the components, side effects, difficulty in use, changing the environment for variable testing

♦ Reviewing by experts♦ Testing through surveys, prototypes,

and experimentation

Validity Threats• Research Bias: Data selection to fit the

theory or preconceptions• Reactivity: Influence of researcher on

setting of experiments

Checks: Modus Operandi Approach [generating alternatives to eliminate]

Searching Negative Cases [rule out the possibilities]

Triangulation [supporting evidence from diverse sources]

Feedback [soliciting observations from experts]

Quasi-Statistics [some quantitative support]

Comparisons [comparing results from other studies]

Effective Research Design• Parsimony: causes advanced as

explanations for phenomena be as far as simple as possible (Occam’s razor)

• Unobtrusiveness: avoids obtrusive data collection

• Sound Methodology: selects appropriate technique for analysis

• Triangulation: provides for consistency checks

Research Study Design Steps:

1. Formulation of the research problem.

2. Decision about suitable population for the study and the sampling procedure.

3. Devising techniques for gathering data.

4. Determining the mode of administering the study.

5. Deciding the scheme for editing, coding and processing of data.

6. Indicating the procedures for data analysis.

7. Deciding about the mode of presentation of the research report.

What is the study about? What is the purpose of the study

and its scope? What are the types of data

required? Where can the needed data be

found and their sources? What will be the place or area of

the study? What periods of time will the study

include? What time is approximately

required for the study?

Research Design Decisions

Contd…

What amount of material or number of cases will be needed?

What bases will be used for the selection of the material ?

What techniques of data gathering will be adopted?

What type of sampling, if required, will be used?

How will the data be analysed? How best can all these questions

be decided upon with minimum resources?

Research Design Decisions

Check List

1. Goal clarification 2. Designing overall study & sampling plan3. Selecting the sample 4. Designing the questionnaire & cover letter 5. Conducting pilot test 6. Revising the questionnaire7. Printing final questionnaire8. Locating the sample 9. Mailing & response collection 10. Attempts to get non-respondents 11. Editing and coding the data12. Data entry and verification 13. Analysing the data14. Interpreting & validating the analysed data 15. Preparing the report

Time Cost

Research designshould incorporatesufficient control & validity checks, but also flexibilityto achieve the objective precisely