Zoe G. Davies Centre for Evidence-Based Conservation University of Birmingham, UK

Zoe G. DaviesCentre for Evidence-Based ConservationUniversity of Birmingham, UK

Systematic Review Protocol Development

Overview

What is a protocol?

What is the value of a protocol?

Question formulation

Components of a protocol

What is a Protocol?

The starting point

Explicitly states the methodology to be followed during the systematic review process

Available for peer-review by subject experts

What is the value of a protocol?

Formalises the question under review

Helps to avoid bias

Ensures transparency

Question: formulation

Define the systematic review question

Key elements:– Subject

(i.e., unit of study to which the intervention is to be applied)

– Intervention (i.e., policy or management intervention under scrutiny)

Question: formulation

– Outcome(i.e., any measured outcome that can be used to judge the effectiveness of the intervention)

– Comparators

(e.g., intervention vs. no intervention or intervention y vs. intervention z)

Question: making it relevant

Dependent on the purpose of the systematic review– Management, policy or research driven question

Consultation with interested stakeholders, end-users and subject experts

Striking the balance– Not too broad, not too specific

Question: secondary objectives

Identify factors that may influence the outcome of studies

Also referred to as:– Reasons for heterogeneity – Effect modifiers (meta-analytical terminology)

Case Study: hedgerow corridors

Evaluate the effectiveness of habitat corridors in promoting population viability of target species and biodiversity within fragments of remnant habitat

20 years of debate within the ecological literature


Do hedgerows mitigate woodland habitat fragmentation?

Do hedgerows increase the population viability of target species occupying otherwise isolated fragments of woodland habitat?


Key question elements:– Subject: mammal, bird, invertebrate

or amphibian populations or assemblages

– Intervention: a hedgerow, or hedgerow network, connecting two or more woodland habitat

fragments


– Outcome: desired primary outcomes were change in population density for a target species or change in species richness within assemblages

– Comparator: No comparator was necessary for inclusion (although appropriate spatial or temporal controls were a prerequisite for studies to be included in any subsequent meta-analysis)


Reasons for heterogeneity: – Physical structure of the hedgerow

– Vegetation composition of the hedgerow

– Nature of the non-habitat matrix

– Life history stage of the target species (e.g., dispersing juveniles)

Protocol: background

Rational behind the systematic review

Put the review question into context – Woodland fragmentation – Habitat connectivity– Hedgerow ecology

Protocol: literature search strategy

State how and which information sources will be searched

Key words:– Reflect the population, intervention and outcome

– Consider synonyms, alternative spellings and abbreviations(e.g., colonise and colonize)

– Foreign language translations


Hedgerow* AND corridor* Hedgerow* AND movement* Hedgerow* AND dispersal Hedgerow* AND colonisation Hedgerow* AND colonization Hedgerow* AND connectivity Hedgerow* AND population* Hedgerow* AND communit*


Combinations and permutations– Trade off between effort and return– Sensitivity vs. specificity

Scope searches and refine

Search generic and specific information sources


Online databases and libraries– Scientific literature databases (e.g., WOK and JSTOR)– Statutory and non-governmental organisation websites

(e.g., Defra, EN and RSPB)– Internet search engines (e.g., Dogpile and Google Scholar)

Specialists in the field

Bibliographies

Hand searching libraries and museums

Protocol: study inclusion criteria

Based on the key elements of the question

State the filtering process to be used– Title– Abstract

(Assessed by a second review and test for agreement)

– Full text

(Assessed by a second review and test for agreement)

Protocol: study quality assessment

Hierarchy of evidence– RCT’s– Control trials without

randomisation– Site comparisons– Time series data

Used to determine study quality thresholds for included articles– Dependent on the purpose of the review

Protocol: data extraction

What data needs to be extracted from the accepted studies?

– Pilot data extraction forms

– Contact authors or organisations for retrieval of missing data

– Database or spreadsheet of all information relevant to the review

Protocol: data synthesis

How are the studies to be pooled?– How will differences in the studies be taken into

account?

Propose analyses to be undertaken

May not be possible to be specific at the protocol stage

Protocol: timescale

Set out key milestones

(e.g., completion of searching, study selection, etc.)

Possible delays

(e.g., consultation periods, inter library loans, etc.)

Some stages may overlap

Protocol: modifications

Amendments to the methodology(e.g., if there are no studies which meant the inclusion

criteria)

Modifications must be documented and justified– Maintain transparency– Allow independent parties to judge review validity

Further Information

Available from our website: – www.cebc.bham.ac.uk

Medical systematic review centres:– The Cochrane Collaboration

(www.cochrane-net.org)– NHS Centre for Reviews and Dissemination

(www.york.ac.uk/inst/crd)

Zoe G. Davies Centre for Evidence-Based Conservation University of Birmingham, UK

Documents

Transcript of Zoe G. Davies Centre for Evidence-Based Conservation University of Birmingham, UK