Appraising academic publications: tips for effective reading and writing for quality and safety professionals Dr Anne Hogden, Dr Deborah Debono, Dr Janet Long and A/Prof David Greenfield Centre for Healthcare Resilience and Implementation Science (CHRIS) Australian Institute of Health Innovation (AIHI) Presenters 2 David, Deborah, Anne and Janet Australian Institute of Health Innovation Our mission is to enhance local, institutional and international health system decision-making through evidence; and use systems sciences and translational approaches to provide innovative, evidence-based solutions to specified health care delivery problems.3 Purpose of the webinar 4 Build on previous presentations given to ISQua Fellowship: Previous webinars - January 2015 and May 2015 Seminar at 2015 ISQua conference: Healthcare research for quality and safety professionals: study design, implementation and translation. Appraising publications is a useful skill to assist you plan and conduct your work: Reading to inform your work or conducting evaluations and research Writing up or presenting your project Reaching a high standard for publishing your manuscript in peer- reviewed journals Acting as a reviewer for peer-reviewed journals Agenda Deborah Analyse the overall coherence and cohesion of publications Identify limitations and how they could be improved Appraise the quality and rigour of each individual section Apply this knowledge to improve your own research work and writing David Anne Janet 5 Analysing coherence and cohesion of the publication Three approaches you can use, depending on your time constraints. Each approach is directed at taking a big picture look and answering several basic questions: 1.Does the aim match the method, leading to the findings and conclusion? 2.Do you find it credible? 3.Are you convinced? 6 Approach #1 Review the Abstract Questions to ask: Does the aim match the method, leading to the findings and conclusion? Do you find it credible? Are you convinced? Analysing coherence and cohesion of the publication 7 Approach #2 Examine the Introduction + Discussion and/or Conclusion Additional questions to ask: Are both sections grounded in the relevant, current literature? Is there a clear aim or objective that is answered in the discussion and conclusion? Analysing coherence and cohesion of the publication 8 Approach #3 Consider the whole paper from beginning to end, and back and forth at the same time Additional questions to ask: Is the paper well structured, written and easy to read? Are all sections grounded in the relevant, current literature? Is there a clear aim/objective that is answered in the discussion and conclusion? Do sub-sections add to addressing the research aim? Analysing coherence and cohesion of the publication 9 RATS guidelines:Relevance Appropriateness of method Transparency of procedures Soundness of interpretive approach Reference: Clark, JP. How to peer review a qualitative manuscript. In Peer Review in Health Sciences. Second edition. Edited by Godlee F, Jefferson T. London: BMJ Books; 2003: Appraising the quality and rigour of each section 10 R Relevance of study question ASK THIS OF THE MANUSCRIPT THIS SHOULD BE INCLUDED IN THE MANUSCRIPT 1.Is the research question interesting? 2.Is the research question relevant to clinical practice, public health, or policy? Research question is explicitly stated Research question is justified and linked to the existing knowledge base (empirical research, theory, policy) Appraising the quality and rigour of each section 11 A Appropriateness of method QUALITATIVE STUDIES Is qualitative methodology the best approach for the study aims? Interviews: experience, perceptions, behaviour, practice, process Focus groups: group dynamics, convenience, non-sensitive topics Ethnography: culture, organizational behaviour, interaction Textual analysis: documents Study design is described and justified, i.e. why was a particular method chosen? Appraising the quality and rigour of each section 12 QUANTITATIVE STUDIES Is quantitative methodology the best approach for the study aims? Large scale surveys Pre-and post-test design Longitudinal studies Cross-sectional studies Study design described and justified, i.e. why was a particular method chosen? Appraising the quality and rigour of each section A Appropriateness of method 13 T Transparency of procedures SAMPLING Are the participants selected the most appropriate to provide access to the type of knowledge sought by the study? Is the sample adequate? Criteria for selecting the study sample justified and explained theoretical: based on preconceived or emergent theory purposive: diversity of opinion volunteer: feasibility, hard-to-reach groups Appraising the quality and rigour of each section RECRUITMENT Was the recruitment conducted using appropriate methods? Details of how recruitment was conducted and by whom Is the sampling strategy appropriate? Could there be selection bias?Details of who chose not to participate and why 14 DATA COLLECTION Was collection of data systematic and comprehensive? Method(s) outlined and examples given (e.g., interview questions) Are characteristics of the study group and setting clear? Study group and setting clearly described Why and when was data collection stopped, and is this reasonable? End of data collection justified and described Appraising the quality and rigour of each section T Transparency of procedures 15 ROLE OF RESEARCHERS Is the researcher(s) appropriate? How might they bias (good and bad) the conduct of the study, and the results? Do the researchers occupy dual roles (clinician and researcher) Are the ethics of this discussed? Do the researchers critically examine their own influence on the formulation of the research question, data collection, and interpretation of results? Appraising the quality and rigour of each section T Transparency of procedures 16 ETHICS Was informed consent sought and granted? Informed consent process explicitly and clearly detailed Were participants anonymity and confidentiality ensured? Anonymity and confidentiality discussed Was approval from received from an appropriate ethics committee? Ethics approval cited Appraising the quality and rigour of each section T Transparency of procedures 17 S Soundness of interpretive approach ANALYSIS Is the type of analysis appropriate for the type of study? eg Thematic: exploratory, descriptive, hypothesis generating Constant comparison/grounded theory: theory generating, analytical Analytic approach is described in-depth and justified Indicators of quality: Description of how themes were derived from the data (inductive or deductive) Are the interpretations clearly presented and adequately supported by the evidence? Evidence of alternative explanations being sought Analysis and presentation of negative or deviant cases Was trustworthiness/reliability of the data and interpretations checked? Method of reliability check described and justified, e.g. was an audit trail, triangulation, or member checking employed? Did an independent analyst review data and contest themes? Appraising the quality and rigour of each section 18 DISCUSSION AND PRESENTATION Are findings sufficiently grounded in empirical, theoretical or conceptual framework? Is adequate account taken of previous knowledge and what the findings add? Findings presented with reference to existing theoretical and empirical literature, and how they contribute Are the limitations thoughtfully considered? Strengths and limitations explicitly described and discussed Is the manuscript well written and accessible? Written for a health sciences audience Appraising the quality and rigour of each section S Soundness of interpretive approach 19 What are study limitations? What are not study limitations? Why is it important to identify study limitations? Identifying limitations and how they could be improved Strengths Limitations 20 Types of limitations Theoretical limitations Methodological limitations Study design limitations Data limitations Identifying limitations and how they could be improved 21 Presenting study limitations Are they described at all? How well are they described? Critical reflection Evaluation Identifying limitations and how they could be improved 22 The implications of study limitations Identifying areas for additional research Conducting future studies Identifying limitations and how they could be improved 23 Relevance Target group Audience: journal articles, reports, conference presentations, study partners, industry Appropriateness Best method to answer the question Give reasons for your choices Transparency Keep a log What does the report look like? Soundness Results only Discussion Applying this knowledge to improve your research and writing Argue your case Convince the reader Revise, revise, revise Argue your case Convince the reader Revise, revise, revise 24 Report statistics using APA style guide E.g.The sample as a whole was relatively young (M = 19.22, SD = 3.45). The average age of students was years (SD = 3.45). The percentage of participants that were married did not differ by gender, 2 (1, N = 90) = 0.89, p =.35 There was a significant main effect for treatment, F(1, 145) = 5.43, p =.02, and a significant interaction, F(2, 145) = 3.24, p = Applying this knowledge to improve your research and writing Draft your output Check back on RATS to ensure all the criteria are addressed Authors guidelines Revise, review and rewrite Critical review partners Big picture Great work is not written it is re-written 26 Applying this knowledge to improve your research and writing Conclusions Effective reading and writing of academic publications, by: analysing the overall coherence and cohesion publications appraising the quality and rigour of each individual section identifying limitations and improvements applying this knowledge to your own research work and writing 27 Contact details 28 Centre for Healthcare Resilience and Implementation Science, Australian Institute of Health Innovation, Macquarie University Dr Anne Hogden Dr Deborah Debono Dr Janet A/Prof David Greenfield Australian Institute of Health Innovation 29 The Australian Institute of Health Innovation (AIHI) comprises three centres: Centre for Healthcare Resilience and Implementation Science (CHRIS), Centre for Health Informatics (CHI), and Centre for Health Systems and Safety Research (CHSSR). We are a major national resource for many people whose work is intrinsic to strengthening health systems, organisations and services. We strive to conduct world-class research with relevance to national and international research communities, governments, policymakers, providers of health services, managers, clinicians, patients and the community. 30 Our work underpins health reforms and systems improvement and provides new tools, perspectives and evidence to help stakeholders who are interested in making the health system more effective, efficient and productive. Our overarching aim is to produce new, high quality research evidence to support change and improvement. We are funded by the National Health and Medical Research Council, the Australian Research Council, NSW Health and many other supporters, partners and stakeholders. Australian Institute of Health Innovation 31 Centre For Healthcare Resilience and Implementation Science 32 The Centre for Healthcare Resilience and Implementation Science (CHRIS), formerly the Centre for Clinical Governance Research, has a successful track record of undertaking research and evaluation projects on health sector issues since Our core interest is to investigate issues of policy, governance and leadership in the health sector. We are involved in conducting original research providing a scholarly capability by which to evaluate health sector policies, programs and projects, and contributing to undergraduate medical, postgraduate health services management, and public health and al education. Website:http://aihi.mq.edu.au/chrishttp://aihi.mq.edu.au/chris