EPPI Reviewer webinar slides
Transcript of EPPI Reviewer webinar slides
![Page 1: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/1.jpg)
Getting to know EPPI Reviewer
James Thomas (EPPI-Centre, UCL)
Webinar
12/07/2016
![Page 2: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/2.jpg)
Outline
• Introductory overview of EPPI-Reviewer
• Integration with Archie and Revman
• More efficient screening using text mining & machine learning
• Quality assessment / data extraction in complex / qualitative reviews
• Advanced meta-analysis
![Page 3: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/3.jpg)
A (very) quick overview of EPPI-Reviewer
![Page 4: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/4.jpg)
Searching and screening Bibliographic information
Data extraction and quality/relevance assessmentSubstantive codes and textual detail (data extraction)
Characterising studies Organises and applies substantive codes (keywording)
Numerical
synthesisMeta-analysis
function
Narrative ‘empirical’
synthesis Interrogation of codes and detail
Thematic/conceptual
synthesis Inductive coding function
•Web-based browsing and interrogation of coding within and across reviews,
and review reports and summaries
• Online since 2000
EPPI-Reviewer:
web-based review management tool
![Page 5: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/5.jpg)
EPPI-Centre
EPPI-Reviewer: How it Workshttp://eppi.ioe.ac.uk/eppireviewer4/
1. Silverlight Web Application
2. Runs on the Desktop
3. Data comes from central
server
4. Concurrent use:
1. Data is saved in real time
2. Last change is retained
5. Safety:
1. Encrypted communication
2. Backups
3. Error Messages
![Page 6: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/6.jpg)
Interaction with Archie
![Page 7: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/7.jpg)
EPPI-Centre
Cochrane Author Support Tool
(CAST) and moreProduce, write and maintain Cochrane reviews, using a
range of intercommunicating tools.
Cochrane systems
Archie
RevMan
CRS-D
More
Review Production
Covidence
EPPI-Reviewer
More to come?
![Page 8: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/8.jpg)
EPPI-Centre
EPPI-Reviewer and Cochrane Systems
![Page 9: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/9.jpg)
EPPI-Centre
EPPI-Reviewer and Cochrane Systems
Manage reviews
1. Like in RevMan, you need to check it out.
2. Checked-in reviews are ReadOnly in EPPI-Reviewer
3. Archie controls who can open the review.
4. No Other Data is Exchanged!
![Page 10: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/10.jpg)
Machine learning / text mining for
reducing workload during citation screening
10
![Page 11: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/11.jpg)
EPPI-Centre
The screening process
11
1. Read title
& abstract
2. Click
include /
exclude
3. Click
‘next’ and
move on
to the next
reference
4. Repeat…
![Page 12: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/12.jpg)
EPPI-Centre
Screening prioritisation: Changing the
distribution of studies
12
Screening process (red = eligible study)
Traditional
screening
Screening
aided by
text
mining
![Page 13: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/13.jpg)
EPPI-Centre
The active learning process
13
![Page 14: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/14.jpg)
EPPI-Centre
The result
• The result is an ordered list of titles and
abstracts
• Those that are most similar to the ones already
marked as ‘include’ are at the top
• The person screening continues to screen as
usual, but behind the scenes the titles and
abstracts remaining are re-ordered regularly
(e.g. every 25 items)
14
![Page 15: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/15.jpg)
Automatic classification
Does it work?
15
![Page 16: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/16.jpg)
e.g. reviews from Cochrane Heart Group
![Page 17: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/17.jpg)
EPPI-Centre
Comparison of clinical vs public health
• Miwa M, Thomas J, O’Mara-Eves A, Ananiadou
S (2014) Reducing the screening burden in
systematic reviews through active learning on
imbalanced data sets. Journal of Biomedical
Informatics
17
![Page 18: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/18.jpg)
EPPI-Centre
Comparison: two reviews
18
“Micro nutrients”
With enhancements‘Basic’ methods
“Cooking skills”
![Page 19: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/19.jpg)
EPPI-Centre
19
On the tedium of screening many excludes towards the end of the
process: “this can be quite satisfying. By this stage of the screening
process, you are well aware of what an include ‘looks like’, and can
screen for exclusion very quickly. You can therefore screen through these
remaining studies much more quickly than the earlier studies – and
having them all coming later on does seem to make screening them
quicker too.”
“I’ll just use my brain later”. It’s
a technical process – doing all
the screening – and got much
more interesting once papers
were being read.
Process evaluation in ‘live’ reviews
![Page 20: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/20.jpg)
EPPI-CentreUsing the technology in live reviews: not all
textbook graphs!
20
![Page 21: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/21.jpg)
EPPI-Centre
… and for updating existing
reviewsWeightman A,
Thomas J, Baker P,
Lovie-Toon Y,
Francis D, O'Mara-
Eves A (2014) Text
mining for screening
efficiency? Testing
within a Cochrane
public health review.
Poster presented at
Cochrane
Colloquium 2014,
Hyderabad
![Page 22: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/22.jpg)
Active learning:
Screening in
EPPI-Reviewer
22
![Page 23: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/23.jpg)
EPPI-Centre
Citation screening in EPPI-Reviewer
23
![Page 24: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/24.jpg)
Identifying RCTs automatically –
coming soon
![Page 25: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/25.jpg)
Acknowledgements
• Many people… including: Sergio
Graziosi, Jeff Brunton, Alison O’Mara-
Eves, Ian Shemilt, Claire Stansfield
(EPPI-Centre / EPPI-Reviewer and text
mining / automation / information
science)
![Page 26: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/26.jpg)
Quality assessment /
data extraction
26
![Page 27: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/27.jpg)
Meta-analysis
27
![Page 28: EPPI Reviewer webinar slides](https://reader034.fdocuments.in/reader034/viewer/2022051405/58a2fd581a28abbf0b8c0b56/html5/thumbnails/28.jpg)
Thank you
EPPI-CentreSocial Science Research UnitInstitute of EducationUniversity of London18 Woburn SquareLondon WC1H 0NR
Tel +44 (0)20 7612 6397Fax +44 (0)20 7612 6400Email [email protected] eppi.ioe.ac.uk/
The EPPI-Centre is part of the Social Science Research Unit at
the UCL Institute of Education, University College London
SSRU website: http://www.ioe.ac.uk/ssru/
SSRU's EPPI website: http://eppi.ioe.ac.uk