Module: 14-7104-OOY-B-20123 – Portfolio of Research Skills (A-2012/3)

Quantitative Data Analysis

‘Attitudes Towards Research’ Lee James Cox

B0049872

MA in Technical Communication

1

Introduction

This presentation aims to show:

How I would go about analysing and presenting survey data

based on example of:

Excel spreadsheet of 50 summarised responses for a research question of:

Attitudes Towards Research* *Adapted by Sheffield Hallam University (SHU) based on Bjorkstrom and Hamrin [1] survey

2

Recognising limitations in the data set

1. Data set is missing key information to provide validity as to the source – Sampling or selection mechanism of how 50 subjects were chosen is not explained* – Key demographic information is missing (age and education are important variables for attitudes [2]) – Data set may not representative. [3] Desired confidence level, population size and accuracy is

required. Instead, sample is likely to be opportunistic [4]* – Data set is only for a specific time period – potentially influenced by specific/recent events

2. Raw data is missing. Only aggregated data is available which limits analysis:

– Did the Date or Time of day of the submission affect the responses? – No opportunity to identify and discard invalid responses, e.g. what if all answers were the same (all

low, all medium, or all high) – Interrelationships between an individuals answers is lost, e.g. if a person respond high to question 5,

are they also more likely to respond to question 4 as high? – Is there a correlation or cross-tabulation between a persons role and their attitudes?

3. Questions may be confusing – ‘Professional Communication’ is an abstract idea and using concrete examples would remove

misunderstandings – The Likert [5] scale would have been more conclusive and less confusing than the Low High rating.

Had subjects been able specify their level of agreement or disagreement, it would remove guess work whether low agreement = disagreement, and whether moderate means neither ‘agree nor disagree’.

*50 respondents likely to be SHU students from Nov 2011 MA in Technical Communications course

3

Collation of data

4

Even with the limitations of the data set, the data needs to be reshaped to assist with analysis:

1. Categorising: ‘Name your role’ question should be grouped to create an additional data point

2. Formatting: superfluous text and formatting in questions 2-10 should be reduced to make data manipulation and graph production easier.

3. Labelling: headings should be renamed to include ‘Agreement’ to reduce misunderstandings that headings may refere to ‘Priority’ or ‘Importance’

4. Significance: statistical inference [6] should be applied to assess whether any of the data or results may help support the theories about this research topic.

Presenting the findings

5

1. Agree a suitable format for the audience. Report with technical data or presentation with hand-outs?

2. Directly answer the research question as an executive summary highlight the most important finding, e.g.:

‘4 out of 5 students believe external researchers are needed to develop standards.’

3. Provide appropriate conclusions for format of report, e.g. recommendation and next steps

‘84% said relevant research results should be more widely spread to Professional Communicators

and therefore we recommend …’

4. Share the supporting data to establish validity of research – Raw data for other researchers to draw their own analysis and conclusions

– Summary of data in tabular and graphical formats – highlighting key findings

5. Validate interpretations and propose alternative conclusions to consider

Validity & Reliability

6

Ethics

‘SurveyMonkey’ application protected user anonymity

No personal questions to identify individuals

× No confirmation page or capturing of an email address to be rewarded with the results

Use of a Research Diary to record the research process is recommended

• Using a blog (Blogger, Wordpress) or offline in Microsoft Word

• To record what I was doing, feeling and thinking throughout the project

Excel would be the primary analysis tool because of small data set

• Easy to manipulate data – filtering, calculations, cross-tabulation

• Good range of table formats and graphs for the presentation

Testing is required

• Of conclusions - through peer review from tutors and students

• Over time - generated periodically (annually?) to trend attitude changes and issues

Weaknesses & Strengths of Quantitative Data Analysis

Weaknesses Strengths

Perception of having less ‘validity’ as questions don’t offer opportunity to clarify or challenge the questions and may not reflect subjects understanding of topic.[7]

Perception of being more ‘reliable’ because approach aims to control or eliminate extraneous variables. Results are more independent of the researcher.

Data not displaying significance may be neglected. E.g. Q7 – 2 deviations [8] from the norm said they highly agreed that doing research would complicate their professional practice but this is not explored further.

Best for testing and validating existing constructed theories and hypotheses to identify areas to deep dive with Qualitative analysis.

Large data sets may become unmanageable to restrict the scope

Matters less about selection of samples – can be more random using larger number of people.

Knowledge produced might be too abstract and general for direct application to specific situations or contexts relevant to the subject.

Can be easier to obtain precise, numerical data more quickly.

Likely to have higher credibility.

7

Conclusions

Quantitative analysis relies completely on the data set available for its validity. In this study, we show there were too many issues that invalidated it.

Unfortunately the restricted set of data also presents little opportunity to identify areas of interest for interpretation and further analysis.

8

References

[1] Björkström M.E., Hamrin, E.K.F. (2001) Swedish nurses’ attitudes towards research and development within nursing. Journal of Advanced Nursing. 34 (5): 706–14

[2] Carr, L. T. (1994). Referencing Bostrom, Hicks, Pearcey & Dyson in ‘The strengths and weaknesses of quantitative and qualitative research: what method for nursing?’. Journal of Advanced Nursing. 20, 713.

[3] Krejcie, R., Morgan, D. (1970). Determining Sample Size for Research Activities. Educational and Psychological Measurement. Online. Last accessed 1/12/13 at http://opa.uprrp.edu/InvInsDocs/KrejcieandMorgan.pdf

[4] Duffy, M.E. (1985) Designing nursing research: the qualitative-quantitative debate. Journal of Advanced Nursing. 10 (3), 225-232.

[5] Likert, R. (1932). A Technique for the Measurement of Attitudes. Archives of Psychology, 140, 1–55.

[6] Neyman, J. (1934). Statistical Inference in "On the two different aspects of the representative method: The method of stratified sampling and the method of purposive selection", Journal of the Royal Statistical Society, Vol. 97 (4), 557–625. Published by Wiley.

[7] Sandeowski, M. (1986) The problem of rigor in qualitative research. Advances in Nursing Science. 8 (2), 27-37.

[8] Cormack, D. (1991). Deviant cases. The Research Process in Nursing 2nd edition. Blackwell Scientific. Oxford.

9