Research Seminars inIT in Education(MIT6003)

Qualitative Educational Research

Design & Practical Concerns II

Dr Jacky Pow

Agenda

Qualitative data analysis Management of time and resources Validity and reliability in qualitative

research Practical concerns Qualitative research critique exercise

Qualitative data analysis 資料分析 Operational definition

– Behaviour – Phenomena

Unit of analysis (pre-defined in design stage) Coding (transcription and field notes) Classifying into Categories Checking inter-rater’s reliability (if any) Tallying / encoding Identifying patterns Theme development

Qualitative data analysis 資料分析 Pilot study (to foresee feasible research di

rections and testify researchable contexts) Research rigor: precise measurement, refu

tability of knowledge claims Replicability of findings: qualitative resea

rchers do not concern much about this Identify relationships among variables in

quantitative data (if necessary)

Management of time and resources

Time, personnel and financial support are critical to the success of a qualitative study:– Time (sufficient to track the phenomena as they

developed)– Personnel (more than one researchers involve

in the same study)(capable of thoroughly and efficiently gathering the needed data)

– Resources (for personnel, travel, data analysis, and report writing)

Management of time and resources

Due to the fluid and flexible nature of qualitative research, additional costs may incur

Mindful of the research scale (not too ambitious but can reveal the phenomena as far as possible)

However, determination of resources needed for the study must often wait until fundamental design decisions have been made

Objectivity of qualitative research

Objectivity is the simultaneous realization of as much reliability and validity as possible (Kirk and Miller, 1986)

Social research can never be ‘totally objective’ (Babbie, 1992) due to the ‘natural subjectivity’ of qualitative research (Marshall and Rossman, 1995)

Should not be judged in a positivists way (Winter, 1989)

Reliability 信度 Reliability is the degree to which the

finding is independent of accidental circumstances of the research (Kirk and Miller, 1986)

‘Fluidity and flexibility’ of methods (Mason, 1996 )

Qualitative study does not pretend to be replicable and qualitative researchers tend to avoid controlling the research conditions

Enhancing reliability in qualitative study

Stress the importance of repeatability of observations (Kirk and Miller, 1986)

Careful documentation of procedures for collecting and interpreting data

Ensure that the data is not only appropriate to the research questions, but also thorough, careful, honest and accurate (Mason, 1996)

Validity 效度 Validity is the degree to which the finding

is interpreted in a correct way (Kirk and Miller, 1986)

There are two kinds of validity: internal and external validity

Measure of internal validity

Internal validity (the degree to which correct conclusions about causal relations can be drawn ) is not relevant to a qualitative study (Yin, 1994), especially in descriptive and exploratory studies

Qualitative research tends not to make such causal assertions (Mertens, 1998)

Trustworthiness of the findings

Measure of external validity External validity means the degree of transfera

bility (Lincoln and Guba, 1985) or generalisability (Cohen and Manion, 1989) of the findings

Context-sensitive nature of qualitative research Cannot have high transferability and generalisa

bility – a general weakness of qualitative research

The stronger the falsification attempts a proposition has survived, the more valid, the more trustworthy the knowledge (Kvale, 1996)

Triangulation 三角測量法 Aims to enhance validity and trustworthines

s of findings / knowledge Types of triangulation (Denzin & Lincoln,

1994)– Cross-perspective– Cross-data-source– Cross-method– Cross-person

Researcher’s role management Issue of entry

– Entry negotiation requires time, patience, and sensitivity to the rhythms and norms of a group

– Building Trust– Managing the political

Reciprocity– Confidentiality and trust

Ethics– Researchers must demonstrate awareness of the

complex ethical issues (teacher as researcher)– The research will not be harmful to the subjects

Practical concerns

Limited local reference materials and literature (研究參考資料及相關文獻有限 )– Non-local materials and literature may not direc

tly applicable

Hard to find participants (研究對象難尋 ) – Don’t want to be observed or video-taped– The issue of teaching privacy– The problem of Hawthorn Effect – invalid data– Students are not really want to be participated

Practical concerns

Data Coding problem (資料登錄不易 ) – To use the data collected, we need to code them– To code the data, we need a data coding system

in place– Since the coding systems in foreign countries m

ay not suit the local context, new coding systems are needed, making the re-use of coding system difficult

– Researchers need to design their own coding categories

Practical concerns

Difficulty in transcribing (記音費時費力 ) Transcribing data is a tedious job (in genera

l, 1 hour of interview needs 6-7 hours to complete the transcription)

If you do not have the time to do the transcription, you may pay someone to do it for you. But you have to make sure s/he has the patient and ability to get the job done

Practical concerns

Inexperience in data analysis (缺乏資料分析的經驗 )– Inter-rater reliability (lack of experienced rater

s)– Operational definition are usually unclear or ke

ep changing– Easily get frustrated by the amount of time and

energy in putting data into the categories

Practical concerns

Lack of experienced qualitative researchers (熟諳質的研究方法論者不多 )– Academics and researchers usually received for

mal quantitative training– Qualitative research in education is new to man

y academics and researchers as it only be introduced in the last 20-30 years at most

– Many qualitative researchers could not complete their research because they were unable to seek help from experienced ones to solve their technical problems in data analysis

Qualitative research critique exercise