James Parry - Research Integrity - Institutional Respons

Integrity of research data:the challenges for institutions from an external perspective

James ParryActing Head, UK Research Integrity Office

UK Research Integrity Office

Integrity of research data

• The integrity of research data is at the heart of good research practice.

• It can be argued that it is the most important part of the research process:

‘Science as we know and practice it today cannot exist without data.’

Steneck, N., 2007. Office of Research Integrity Introduction to the Responsible Conduct of Research Revised Edition.

(Available at www.ori.hhs.gov)


Integrity of research data• Good research practice includes how data is:

Collected and recorded in an appropriate, detailed, complete and, most importantly, authorised manner

Protected and stored for an appropriate duration and in a secure and accessible form

Reported accurately and honestly Shared with researchers and others

• It also includes properly addressing issues of data ownership, including the requirements of funding bodies, research organisations and data sources.


Integrity of research data

• Consequently, how best to collect, store and share data has been discussed for sometime:

‘I have begun to think that no one ought to publish biometric results, without lodging a well-arranged and well-bound manuscript copy of his data in some place where it should be accessible, under reasonable restrictions, to those who desire to verify his work.’

Galton, F., 1901. Biometry. Biometrika.


Increasing interest and scrutiny

• There has been increased interest in - and scrutiny of - research data and its integrity in recent years.

• Examination of a variety of issues by Government, funding bodies, learned societies and other organisations, both in the UK and abroad. In some cases, this has led to new requirements or guidelines

for researchers and research organisations

• Increased interest in the media and academic press.• Increased public awareness and scrutiny, especially via

the internet.



• The House of Commons Science and Technology Committee’s inquiry into peer review also examined issues relating to research integrity and research data.

‘Access to data is fundamental if researchers are to reproduce, verify and build upon results that are reported in the literature… The work of researchers who expend time and effort adding value to their data, to make it useable by others, should be acknowledged as a valuable part of their role.’

House of Commons Science and Technology Committee, 2011. Peer Review in Scientific Publications.



• Research Councils UK Policy and Code of Conduct on the Governance of Good Research Conduct (2009): Guidance on the management of research conduct

and the investigation and reporting of misconduct. From October 2009, a requirement of all existing and

future grants and awards from the Research Councils. Requires institutional systems for good research

practice which should apply to all research by an organisation, regardless of the funding source.



• These systems should include ‘…clear requirements for preservation of relevant primary data, laboratory notebooks and other relevant materials.’

• The Policy defines unacceptable conduct as including:

‘misrepresentation of data, for example suppression of relevant findings and/or data, or knowingly, recklessly or by gross negligence [emphasis added], presenting a flawed interpretation of data…’ and

‘…mismanagement or inadequate preservation of data and/or primary materials…’



• The European Science Foundation’s European Code of Conduct for Research Integrity (2011) reiterates that integrity in research data is vital:

‘Objectivity: interpretations and conclusions must be founded on facts and data capable of proof and secondary review; there should be transparency in the collection, analysis and interpretation of data, and verifiability of the scientific reasoning.’

• The guidelines (available from www.esf.org) also include suggested requirements for data storage and accessibility.



• Increased media interest in research data and integrity: Academic press: Times Higher Education, Science, Nature and

subject-specific journals have all covered issues of research data and its integrity, often linked to misconduct cases, as well as discussing questions of data ownership and sharing.

Also in mainstream media, often covering so-called scandals.

• Increased public interest in research data and integrity: Often driven by coverage in mainstream media, particularly

regarding research perceived to be contentious. Numerous blogs and websites covering research, including

fraud and misconduct, and often scrutinising specific data.


Integrity in research data is vital• Poor practice or deliberate misconduct relating to

research data not only damages: The reliability of that data. The validity of the research as a whole.

• It can also: Be in breach of the law. Be in breach of ethical approval and other

requirements. Lead to contractual, financial and reputational liabilities.


Integrity in research data is vital• In serious cases, poor practice or deliberate

misconduct relating to research data can harm participants, patients and the public.

• It also damages public trust in research: Research is fundamentally important to our society. Based on knowledge gained from research, decisions are

made that are crucial for economic development and the general welfare of people and society.

Trust in data generated by research is based on the belief that it will be honest, accurate and objective.


UKRIO’s perspective on the issues

• UKRIO is the only dedicated research integrity body in the UK.

• We provide support to researchers, organisations and members of the public on issues of research integrity across all subjects, including: Giving advice on good practice in research. Addressing poor practice and misconduct. Creating and revising systems for research conduct. Providing education and training in research integrity.


UKRIO’s perspective on the issues

• Much of our work has involved issues relating to research data.

• Through this extensive work, we have become aware of: Two particular challenges for organisations. Several recurring themes regarding existing

practice. A number of notable issues and pitfalls.


Challenges for institutions: 1• The first challenge is obvious: to fulfil the many – and

growing – requirements for research data. Legislation and regulatory bodies. Research funders. Data sources. Institutional policies, including those of collaborating

organisations. Ethical and other approvals for research projects. Academic journals. Requirements of other countries for relevant projects.


Challenges for institutions: 1• But is meeting this challenge as straightforward as it

seems? Requirements for research data, whether mandatory or

suggested guidelines, can be many and varied. UKRIO has found that determining what requirements

have to be met for a research project and how best to meet them can often be complicated for researchers.

What seem like straightforward requirements can be hard for researchers to understand or difficult to achieve.

Researchers may also be unwilling to come forward and admit they need help or be unsure of where to get it.


Challenges for institutions: 1• Devising and supporting a consistent yet balanced

approach to research data at the institutional level can also prove to be very difficult. Organisations need to be responsive to the different

requirements relevant to different fields of study but present clear common approaches to the management of research data and its consideration throughout the lifetime of a research project.

There is no rigid, one-size-fits-all solution but there is room for common approaches to common situations.


Challenges for institutions: 2• The second challenge is to do more than just meet the

letter of the various requirements: Avoid these requirements leading to the development of

a ‘tick box’ mentality, which can easily occur. Aim to meet the practical, ethical and intellectual

challenges of carrying out high-quality research with a focus on developing ‘best practice’ rather than just ‘good practice’.

Incorporate external requirements for research data into wider institutional efforts to promote the responsible conduct of research.


Challenges for institutions: 2• There are no quick fixes to meet both the letter and

the spirit of requirements for research data: Raise awareness of what needs to be done and, most

importantly, why it matters. ‘Champions’ to promote best practice in Faculties, Schools

and Departments, and also in relevant support services. Education and training for research staff and students. Systems to define the particular ethos and requirements of

the institution’s approach to research data, and with a wider focus than only data retention and security.

Resources and practical support for researchers.


Challenges for institutions: 2• Organisations must not only set up policies but also

provide practical support to help their researchers meet institutional and external standards:

‘Organisations should have in place procedures, resources (including physical space) and administrative support to assist researchers in the accurate and efficient collection of data and its storage in a secure and accessible form.’

UK Research Integrity Office, 2009. Code of Practice for Research: Promoting good practice and preventing misconduct.

(Available at www.ukrio.org)


Recurring themes from UKRIO’s perspective: 1

• Good research practice is often thought to be self-evident, i.e. researchers naturally practise responsible conduct in research, including in relation to research data.

• In our experience, and that of others, it should be self-evident but often it is not.

• This can apply to established researchers as much as to those early in their careers.



• Are we teaching researchers what they need to know or are they picking it up as they go along?

• If the latter is the case, are they learning the right things or picking up bad habits?

‘Proper record-keeping is crucial to scientific research. But the accepted practices of record keeping and policies on custody and retention of data are usually learned passively by most scientists.’

Macrina, F. L., 2005. Scientific Integrity (Third Edition).



• A key lesson from UKRIO’s extensive case work: serious problems can occur because of a lack of foresight but could easily have been avoided.

• Anticipate what situations and problems might occur and agree jointly in advance how they might be addressed. This applies to organisations and researchers alike This can save an awful lot of time, effort and grief

later on…



• Consideration of issues relating to research data should begin early on and continue throughout the lifetime of a project:

‘Researchers should consider how data will be gathered, analysed and managed, and how and in what form relevant data will be made available to others, at an early stage of the design of the project.’

UK Research Integrity Office, 2009. Code of Practice for Research: Promoting good practice and preventing misconduct.

(Available at www.ukrio.org)



• Institutional policies and systems must be appropriate and proportionate, rather than burdensome and bureaucratic.

‘Discussions of scientific record-keeping run the risk of implying some uniform prescription for the process – a rigid method for the one correct way to do things. However… there are multiple right ways to keep scientific records.’

Macrina, F. L., 2005. Scientific Integrity (Third Edition).



• While policies and systems for research data must be robust, they should also allow appropriate scope for: The type of the research in question. The nature and quantity of data generated. The experiences and preferences of researchers.

• Organisations should monitor policies and systems and seek feedback to ensure that they are fit for purpose and not inflexible to the needs of researchers, research students and support staff.


Issues and pitfalls: external collaborations

• Organisations must not design and implement systems for research data in a vacuum.

• They need to consider the level of data management required - and the ways that data can be shared - with the move to a more collaborative research approach with private sector and other partners, including the NHS.

• Will your systems facilitate collaborative research with partner organisations in the UK and abroad or might they inadvertently hinder them?


Issues and pitfalls: does everyone know the rules?

• Ensure that all staff know what is required of them, how to achieve it and where to go for help.

• Ensure that supervisors are passing on correct practice, otherwise problems can arise: Research staff and students can sometimes engage in poor

practice - or even misconduct - by following incorrect but apparently legitimate instructions from supervisors, or at least claim that this was the case.

• Staff should remember that what was correct for one project may not be correct for similar projects, even if they appear to be identical.


Issues and pitfalls: coping with complexity

• Do you have adequate systems and resources to ensure that particularly complex data is collected, stored, verified and shared properly?

‘Our capacity to generate data sometimes outstrips our capacity to store and share it. Data storage and sharing were major problems during the early years of the Human Genome project. …Even when researchers want to, it is not always clear how they should go about collecting, storing, and sharing data responsibly.’

Steneck, N., 2007. Office of Research Integrity Introduction to the Responsible Conduct of Research Revised Edition.


Issues and pitfalls: not-so-simple mistakes

• Simple mistakes can lead to significant problems which require considerable effort and resources to resolve.

‘Unfortunately, poor documentation can shift from an inconvenience to an active danger when it obscures not just methods but errors…

‘While simple mistakes often allow for simple fixes, incomplete documentation and lack of reproducibility means that this simplicity is often hidden.’

Baggerly, K. A., and Coombes, K. R., 2009. Deriving Chemosensitivity from Cell Lines: Forensic Bioinformatics and Reproducible Research in High-Throughput Biology. The Annals of Applied Statistics.


Issues and pitfalls: other organisations and countries

• When collaborating or sharing data with research teams in other organisations or countries, are you assured of the adequacy of their data protection provisions?

• Are you fulfilling all of your legal responsibilities?• Early consideration of these issues is essential and

legal advice is often advisable.• Research ethics committees frequently want to be

reassured that contracts, duties and responsibilities relating to data have been explored and agreed.


Issues and pitfalls: scrutiny and audit

• How do you know if research staff and students are getting things right? Apart from in clinical studies, is there much scrutiny

or auditing? Is this an area where universities could perhaps learn

from other sectors and adapt practices appropriately?• If you have Faculty- or School-based systems, are

central research governance staff aware of how well these systems are working? Are they being informed of any problems?


Next steps• Ensuring integrity in research data can seem daunting,

especially considering the many different requirements and the constraints of the current economic climate.

• However, there are many sources of help available: Large body of published work on research data. Advice from regulators, research funders, professional

bodies, learned societies, academic journals and organisations such as JISC, RIN and UKRIO.

• Organisations can also help each other: it is vital that good practice is shared.


James Parry - Research Integrity - Institutional Respons

Technology

Transcript of James Parry - Research Integrity - Institutional Respons