MIXED

METHODS9715 – Doctoral Seminar

Notes by M. Larsen

Nov. 28, 2014

‘The Third Methodological Movement’

‘The Third Path’‘A research paradigm whose time has

come.

WHY MIXED METHODS

DESIGN?To provide a more complete picture of the phenomenon under study

To increase data validity

Enable researcher to develop analysis and build on original data

MM research – “can help to clarify the formulation of the research problem and the most appropriate ways in which problems or aspects of problems may be theorised and studied….With multiple methods the researcher has to confront the tensions between different theoretical perspectives while at the same time considering the relationship between the data sets produced by the different methods” (Brannen, 1992, pp. 32-33)

CRITICS OF MIXED

METHODS‘Incompatibility thesis’ - qualitative and

quantitative research paradigms cannot

and should not be mixed

(purists – paradigm wars)

Critics of MM - different research

paradigms embody incompatible

assumptions about the nature of the

world and what is important to know

CHALLENGING THE DIVIDE BETWEEN

QUALITATIVE AND QUANTITATIVE

RESEARCH

Hammersley (1992) “Deconstructing the

qualitative-quantitative divide”

- the distinction between these

methodological paradigms is limited

and dangerous

Presents 7 issues concerning the

quant/qual distinction and challenges

each of them.

1) Qualitative vs quantitative

dataAssumptions:

Qual research - words

Quant – numbers

But… this distinction is problematic

- large proportion of research reports combine both

- ethnographers use words like “Regularly”, Frequently, Often, Sometimes, Generally, Typically

If this is about precision, then precision doesn’t necessarily mean the use of numbers

1) Qualitative vs quantitative data

cont’d.

“We are not faced, then, with a starkchoice between words and numbers, oreven between precise or imprecise data.Furthermore, our decisions about whatlevel of precision is appropriate in relationto any particular claim should depend onthe nature of what we are trying todescribe, on the likely accuracy of ourdescriptions, on our purposes, and on theresources available to us; not onideological commitments to onemethodological paradigm or another.”(p.43)

2) The investigation of natural vs

artificial settingsAssumptions:Quant – artificial setting – experimentalQual – naturalBut…this distinction is spurious What happens in classroom is not necessarily more natural than what goes on in a psych lab“To treat classrooms…as natural and experiments as artificial is to forget that social research is itself part of the social world.” (p. 44)

2) The investigation of natural vs

artificial settingsReactivity – individuals alter their performance or behavior due to the awareness that they are being observed -both quant and qual research can lead to reactivity(Hawthorne effect)

“The terms ‘natural’ and ‘artificial’ havemisleading connotations. And while theissue of ecological validity is important, it isnot the only important methodological issue.Nor does research in ‘natural’ settingsguarantee ecological validity, any more thanresearch in ‘artificial’ settings automaticallydebars us from it.” (p. 45)

3) A focus on meanings rather

than behaviorAssumptions:

Qual research - interpretive (meaning)

Quant research – positivist (behaviours)

But… rare that qual research simply

documents point of view of participants

Researcher is involved in interpretation of

data

Much quant research concerned with attitudes not just behavior

3) A focus on meanings rather

than behavior

“As regards differences in the approachthat attitude researchers andethnographers employ in identifyingattitudes/perspectives, the contrast isbetween the use of attitude scales andmore unstructured approaches…Hereagain, we do not have a clear-cutdistinction between two contrastingapproaches.” (p. 46)

4) Adoption or rejection of natural

science as a modelAssumptions:

Qual – reject natural science as modelQuant –natural science – exemplary

“Not even the most extreme positivistwould argue that the methods of physicsshould be applied lock, stock and barrel tothe study of the social world. And there arefew supporters of qualitative research whowould insist that there is no aspect ofnatural science method that is relevant tosocial research. What is involved here is amatter of degree.” (p. 47)

5) An inductive vs deductive

approachAssumption:

Quant = deductive or hypothetico-deductive

Qual = inductive

But… “Quant research does not always test hypotheses: its goal is often descriptive.” (Brannen, 1992, p. 8) Some quant research is concerned with

theory generationSome qual research is deductiveLots of qual research is simply descriptive

5) An inductive vs a deductive

approachAll research involves induction and deduction to some degree– impossible that researchers not be influenced by prior knowledge

“What is true is that one can distinguishbetween studies that are primarily exploratory,being concerned with generating theoreticalideas, and those which are more concernedwith testing hypotheses. But these types ofresearch are not alternatives; we need both.Nor need the former be quantitative and thelatter qualitative in other senses of thoseterms.” (p. 48)

6) The identification of cultural

patterns as against seeking scientific

lawsAssumption:

Quant –committed to discovery of scientific laws

Qual – committed to identifying cultural patterns

Yet… much quant is concerned with description

Early qual researchers justified their practice by claiming that it produced scientific laws and even today they claim their goal is theory generation

“Thus the distinction between identifying patterns and pursuing laws seems to provide little clear basis for the division between quant and qualitative methods.” (p. 50)

7) Idealism vs realism

Assumption:

Quant – realist epistemology

Qual- idealist

“More important than the empirical question ofwhether it is true that quant researchers arerealists and qual researchers idealists, though, isthe philosophical issue of whether there is anynecessary connection between qual method anda particular epistemological position….historysuggest that there is little reason to believe thatthere is such a connection. And we mustremember that there are many more than 2epistemological positions available.” (p. 51)

METHODOLOGY OF COMBINING

APPROACHES – CONSIDERATIONS

1) Timing - What will the timing of qual

and quant methods be? What will the

order be? Will it be concurrent or

consecutive?

2) Weighting Dimension -What will be

the relative importance, weight or

priority, given to qualitative &

quantitative methods?

Equal

Emphasis on Qualitative or

Quantitative

METHODOLOGY OF COMBINING

APPROACHES – CONSIDERATIONS

cont’d.3) Mixing Dimension - How will qual and quant methods be mixed? How will the 2 data sets be mixed?

Merged

Embedded within one another

Connected in another way

Complementary

Integration

Fundamental Principle of Mixed

Methods Research

Combine the methods in a way that

achieves complementary strengths and

non-overlapping weaknesses (Johnson

& Onwuegbuzie, 2004)

1) Timing

2) Weighting

3) Mixing Dimension

MIXED METHODS DESIGNS

1) Triangulation/ Multiple Methods

Within-method – the same method

being used on different occasions

Between-method – using different

methods in relation to the same topic

Purpose of triangulation – to obtain

complementary quant and qual data on

the same topic

2) Embedded Design

Different research questions require

different types of data to answer them

Complementary – qual and quant data

complement one another

1 or 2 phase

3) Explanatory Design

1st phase – quantitative (e.g. surveys)

2nd phase – qualitative (e.g. classroom

observation)

Qualitative data needed to explain results

1st phase quant – may be used to guide the

selection of 2nd phase in-depth qual study

4) Exploratory Design

1st phase – qualitative

2nd phase – quantitative

Need to develop a measurement instrument

1st phase - get a deeper understanding of the issue/phenomenon

2nd phase - survey to measure its distribution and prevalence.

PRACTICAL ISSUES

1) Politics of research – evidence-based

research - AERA Scientifically Based Research

2) Costs

3) Researchers: skills, careers,

disciplines

4) Social Organization of the Research

Team