Ruth Garside

Senior Research Fellow

PenTAG, Peninsula Medical School, University of Exeter

Ruth. Garside@pms.ac.uk

Qualitative research and policy makingMay 2011. Belfast

Session plan 75 mins� Background to our experience: 10 mins

� Group work: What does good quality research and its reporting look like? 10mins

� Checklist examples from Public Health Policy Making in England 10 mins

� Break out practical session and feedback: 35 mins

� Round up (and a suggestion!) (10 mins)

Our experience

� Working with CPHE NICE in the UK

� Producing systematic reviews of qualitative research

� Tendency to assume similar concerns as for quantitative research

What makes qualitative research

“good quality”?

p.8

Alternative criteria� Wallace A, Croucher K, Quilagars D, Baldwin S.

Meeting the challenge: developing systematic reviewing in social policy. Policy and Politics 2004; 32(4):455-470.

Question Is the research question clear? E/D

1

2

Theoretical

perspective

Is the theoretical or ideological perspective of the author (or funder)

explicit?

Has this influenced the study design, methods, or research findings?

D

3 Study design Is the study design appropriate to answer the question? E

4 Context Is the context or setting adequately described? E

5

6

Sampling Is the sample adequate to explore the range of subjects and

settings?

Has it been drawn from an appropriate population?

E

7

8

Data collection Was the data collection adequately described?

Was it rigorously conducted to ensure confidence in the findings?

E

9 Data analysis Was there evidence that the data analysis was rigorously conducted

to ensure confidence in the findings?

E

10 Reflexivity Are the findings substantiated by the data and has consideration

been given to any limitations of the methods or data that may have

affected the results?

D

11 Generalisibility Do any claims to generalisibility follow logically and theoretically from

the data?

D

12 Ethics Have ethical issues been addressed and confidentiality respected? D

13 Question Is the research question clear? E

14

15

Theoretical

perspective

Is the theoretical or ideological perspective of the author (or funder)

explicit?

Has this influenced the study design, methods, or research findings?

D

16 Study design Is the study design appropriate to answer the question? E

Small group work

� Are there any challenges with using the checklists provided?

� Do you agree that the questions they are asking will assess “quality”? Why?/Why not?

� What would you use?

My preferred approach

� Technical aspects

� Trustworthiness

� Theoretical considerations

� Practical considerations

Adapted from: Popay J, Using Qualitative Research to Inform Policy and Practice. ONS, Cardiff: April 2008.

1. Technical aspects

Y/P/N Comments

1. Is the research question(s) clear?

2. Is the research question(s) suited to qual. enquiry?

Are the following clearly described?

3. Context

4. Sampling

5. Data collection

6. Analysis

Adapted from: Dixon-Woods M, Shaw RL, Agarwal S, Smith JA. The problem of appraising qualitative research. Qual Saf Health Care 2004; 13:233-225

2. TrustworthinessFor example:

� Are the design and execution appropriate to the research question?

� What evidence of reflexivity is there?

� Do the voices of the participants come through?

� Are alternative interpretations, theories etc explored?

� How well supported by the data are any conclusions?

� Are ethical considerations given appropriate thought?

� etc.

3. Theoretical considerationsFor example:

� Does the report connect to a wider body of knowledge or existing theoretical framework; and, if so

� Is this appropriate (e.g. not uncritical verification);

� Does the paper develop explanatory concepts for the findings

� etc.

4. Practical considerationsNot “is this research valid?” but rather “what is this

research valid for?”

For example

� Does this study usefully contribute to the policy question?

� Does this study provide evidence relevant to the policy setting?

� Does this study usefully contribute to the review?

Aguinaldo JP. Rethinking Validity in Qualitative Research from a Social Constructionist Perspective:

From "Is this valid research?" to "What is this research valid for?". The Qualitative Report 2004; 9(1):127-136.

Presented at the 2nd European

conference on Qualitative Research for

Policy Making, 26 -27 May 2011, Belfast

For more information

Please visit: http://www.merlien.org