10/22/2014

1

NIH Library | http://nihlibrary.nih.gov

Analytical Methods for Systematic Review

Support

Doug Joubert - MS, MLIS

Chris Belter - MLS

NIH Library

2014 MAC-MLA Meeting

Outline

Systematic Review

Review Process

Sci2 Analysis

Sci2 Screening

Conclusions

10/22/2014

2

• Title/abstract reviewing

• First time participating in screening.

• Epi Info™ 7: Free software from the CDC that allows

you to create forms & enter data.

• Science of Science (Sci2) Tool: Open source software

for creating and analyzing publication-based networks.

• Gephi: Open source network visualization software.

Systematic review tools

• Purpose of this systematic review/meta-analysis is:

• to assess the extent to which evidence-based mental

health interventions reduce racial and ethnic disparities in

access to mental health care.

• to identify potential facilitators and barriers to accessing

mental health care among racial and ethnic minorities.

• Access adhered to Penchansky and Thomas (1981)

dimensions of access, including availability, affordability,

accessibility, accommodation, and acceptability.

Systematic review overview

10/22/2014

3

Search strategy and results

• Search strategy was collaboratively developed by everyone on the systematic review team.

• Set 1: Racial or Ethnic Disparities terms (n=157).

• Set 2: Access Terms (n=75).

• Set 3: Mental Health Services (n=59).

• Set 4: Disorders: depressive disorders, anxiety disorders, schizophrenia, or bipolar disorder terms (n=68).

• APA PsycNET, Medline, and Scopus databases were searched:

• APA PsycNET (n=102)

• Medline (n=448)

• Scopus (n=823)

Review process

• First stage of screening involved screening the

title/abstract against the inclusion and exclusion criteria.

• Epi Info™ 7 was used to screen titles/abstracts.

• 4 screeners, worked in pairs.

• Applied inclusion and exclusion factors using screening

form.

• Intercoder reliability (concordance rate) was discussed

during weekly meetings.

10/22/2014

4

Epi Info™ 7 (Slide 1)

Epi Info™ 7 (Slide 2)

10/22/2014

5

Epi Info™ 7 (Slide 3)

Epi Info™ 7 (Slide 4)

10/22/2014

6

Concordance reports

Screening alternatives

10/22/2014

7

Network

Analysis

Identify new search terms to find papers missed by the initial search

Identify new search terms

Word co-occurrence

analysis

Identify new papers

Co-citation analysis

Background and approaches

10/22/2014

8

What is word co-occurrence?

Paper Title: Economic Stress and Mental Health

Network: Economic Stress

Mental Health

Word co-occurrence network

10/22/2014

9

What is co-citation?

Article

A

Article

B

Article

C

Co-citation network

10/22/2014

10

• 191 new title/abstracts screened

• 5 new articles classified as “yes”→move to second round

of review

• 18 new articles classified as “?”→move to second round of

review

• 168 new articles classified as “no’”→not included in

second round of review

• 148 (88%) of the no's were rejected because they did not meet the

“condition of interest”

• 13 (8%) of the no's were rejected because they did not meet the

“population of Interest”

• 7 (4%) of the no's were rejected because they did not meet the

“study design”

Sci2 title/abstract screening

Sci2 title/abstract screening

• Of the 23 “yes” or “?” records identified:

• 49 keywords identified

• 41 unique keywords

• 23 new keywords not included in SR search

• 2 might warrant inclusion in an updated search

• Our ratio of (keep/total screened), was compared to the

ratio (keep/total screened) from the actual systematic

review:

• Our ratio: 23/191 = 0.1204 (12.04%)

• Systematic review ratio: 71/1373 =0.0517 (5.17%)

10/22/2014

11

Conclusions

• With proper mentoring, librarians and other information

professionals can participate in title/abstract screening.

• Epi Info™ and SurveyMonkey® are easy to use tools for

developing screening forms.

• Network analysis can identify new keywords and

phrases for search query expansion.

• Co-citation analysis can identify new publications, but is

dependent on the publication set analyzed.

NIH Library | http://nihlibrary.nih.gov

Doug Joubert douglas.joubert@nih.gov

Chris Belter christopher.belter@nih.gov