A unique method for fast, high-quality systematic searching

Wichor M. Bramer

Erasmus MC

Rotterdam the Netherlands

Research Question

Is exercise therapy an effectivetreatment for hip osteoarthritis

Simple search strategy:

exercise therapy effectiveness hip osteoarthritis

155 hits

('hip osteoarthritis'/exp OR ((arthritis/de OR osteoarthritis/de) AND (hip/de OR 'hip pain'/exp)) OR (coxarth* OR ((hip OR cox) NEAR/6 (arthrit* OR arthrosis* OR arthroses* OR osteoarth* OR oa)) OR 'malum coxae senilis'):ab,ti) AND ('kinesiotherapy'/expOR exercise/exp OR physiotherapy/exp OR 'conservative treatment'/de OR (kinesiotherap* OR kinesitherap* OR exercis* OR physiotherap* OR gymnastic* OR ((movement* OR motion OR manual OR phys* OR conservative* OR nonoperat* OR nonsurg* OR non-operative OR non-surgical OR paramedic* OR para-medical) NEAR/3 (technique* OR therap* OR treat* OR isokinet* OR isomet* OR water OR management* )) OR ((muscle* OR muscul*) NEAR/3 (train*))):ab,ti)

Systematic search strategy1916 hits

Systematic searching

≠

Doing a systematic review

Systematic reviews need systematic searche(r)s but:

The systematic approach can be used for more thanjust searching systematic reviews

Performing systematic reviews is more than justsearching systematically

Creating an optimal syntax in embase.com

(Agenda for this webinar)

1. Identify most relevant emtree terms and free text words

2. Create a syntax combining those terms

3. Optimize the syntax to identify missing terms

Step 1: identifying emtree terms

One element One emtree term

Step 1: identifying emtree terms

One element One emtree term

'familial hypertrophic cardiomyopathy'

One element More emtree terms

Step 1: identifying emtree terms

One element One emtree term

'familial hypertrophic cardiomyopathy'

One element Multiple emtree terms

prevention OR prevention:lnk OR 'prevention study' OR 'preventive health service' OR …

Multiple elements One emtree term

Step 1: identifying emtree terms

One element One emtree term

'familial hypertrophic cardiomyopathy‘

One element Multiple emtree terms

prevention OR prevention:lnk OR 'prevention study' OR 'preventive health service' OR …

Multiple elements One emtree term

('cancer mortality' OR (neoplasm AND mortality))

(neoplasm OR 'cancer mortality') AND (mortalityOR 'cancer mortality')

Step 1: identifying free text words

Optimal sensitivity only by additionally searchingin title / abstract:

• Relevant, non-inverted terms from synonyms

Step 1: identifying free text words

Optimal sensitivity only by additionally searching in title / abstract:

• Relevant, non-inverted terms from synonyms

(neoplasm) tumo* OR cancer* OR neoplas*

• Relevant words / phrases from narrower terms

Step 1: identifying free text words

Optimal sensitivity only by additionally searching in title / abstract:

• Relevant, non-inverted terms from synonyms

(neoplasm) tumo* OR cancer* OR neoplas*

• Relevant words / phrases from narrower terms

(large intestine tumor)

'anus tumor' OR 'appendix tumor' OR 'cecum tumor' OR 'colon tumor' OR 'rectum tumor'

• The emtree term itself

Next: step 2 Create a syntax

Embase.com OvidSP PubMed

Boolean AND, OR, NOT

Thesaurus /exp exp …/ [mh]

---- Unexploded /de …/ [mh:noexp]

Title / abstract ():ab,ti ().ab,ti. [tiab]

Phrases ‘single quotes’ No quotesneeded

“Double quotes” or *

Proximity NEAR/n ADJn Not possible: useAND

Standard syntax (considered familiar):

If not: do attend a basic workshop, youtube video, database documentation (after this workshop)

DEMO

Example

Research Question:

Is exercise therapy an effective treatment for hip osteoarthritis

Elements:

1. Hip osteoarthritis

2. Exercise therapy

3. (Treatment effectiveness)

Hip osteoarthritis

()

Hip osteoarthritis

(''/exp)

('hip osteoarthritis'/exp)

Hip osteoarthritis

('hip osteoarthritis'/exp OR ():ab,ti)

('hip osteoarthritis'/exp OR (coxarth*):ab,ti)

('hip osteoarthritis'/exp OR (coxarth* OR (() NEAR/3 ())):ab,ti)

Hip osteoarthritis

('hip osteoarthritis'/exp OR (coxarth* OR ((hip OR cox) NEAR/3 (arth* OR osteoarth*))):ab,ti)

('hip osteoarthritis'/exp OR (coxarth* OR ((hip OR cox) NEAR/3 (arth* OR osteoarth*)) OR 'malum coxaesenilis'):ab,ti)

Exercise therapy

Exercise therapy

('kinesiotherapy'/exp OR (kinesiotherap* OR kinesitherap* OR ((exercis*) NEAR/3 (technique* OR therap* OR treat*))):ab,ti)

Exercise therapy

('kinesiotherapy'/exp OR (kinesiotherap* OR kinesitherap* OR ((exercis* OR movement*) NEAR/3 (technique* OR therap* OR treat* OR isokinet* OR isomet*)) OR ((muscle*) NEAR/3 (train*))):ab,ti)

Treatment effectiveness

Treatment effectiveness

('clinical effectiveness'/exp OR 'comparative effectiveness'/exp OR 'cost effectiveness analysis '/exp OR 'program cost effectiveness'/exp OR 'program effectiveness'/exp)

Treatment effectiveness

('clinical effectiveness'/exp OR 'comparative effectiveness'/exp OR 'cost effectiveness analysis '/exp OR 'program cost effectiveness'/exp OR 'program effectiveness'/exp OR ():ab,ti)

Treatment effectiveness

('clinical effectiveness'/exp OR 'comparative effectiveness'/exp OR 'cost effectiveness analysis '/exp OR 'program cost effectiveness'/exp OR 'program effectiveness'/exp OR (effectiv*):ab,ti)

Concept query('hip osteoarthritis'/exp OR (coxarth* OR ((hip OR cox) NEAR/3 (arth* OR osteoarth*)) OR 'malum coxaesenilis'):ab,ti) AND ('kinesiotherapy'/exp OR (kinesiotherap* OR kinesitherap* OR ((exercis* OR movement*) NEAR/3 (technique* OR therap* OR treat* OR isokinet* OR isomet*)) OR ((muscle*) NEAR/3 (train*))):ab,ti) AND ('clinical effectiveness'/exp OR 'comparative effectiveness'/exp OR 'cost effectiveness analysis '/exp OR 'program cost effectiveness'/exp OR 'program effectiveness'/exp OR (effectiv*):ab,ti)

152 hits

Step 2 : type code in Word (summary)

• Open an element with ()• For emtree terms type ''/exp and paste term• After pasting all emtree terms, type OR ():ab,ti within

that element once for free text words• Type single word synonyms first• Type (() NEAR/3 ()) somewhere in the document to

reuse• Copy (() NEAR/3 ()) into the ():ab,ti part search for

phrase variants• Untick search as broadly as possibleNext: Step 3 optimization

Step 3: optimizing your strategy

Temporarily change the syntax and check words:

Per element, keeping the rest unchanged

1. (emtree) NOT (free text)

2. (free text) NOT (emtree)

Change the full query, subtract the old results

3. replace /exp and /de with /syn

4. replace :ab,ti with :de,ab,ti

('hip osteoarthritis'/exp …) AND ('kinesiotherapy'/exp OR (kinesiotherap* OR kinesitherap* OR ((exercis* OR movement* OR motion OR manual) NEAR/3 (technique* OR therap* OR treat* OR isokinet* OR isomet* OR water)) OR ((muscle*) NEAR/3 (train*))):ab,ti) AND ('clinical effectiveness'/exp …)

('hip osteoarthritis'/exp …) AND ('kinesiotherapy'/exp NOT (kinesiotherap* OR kinesitherap* OR ((exercis* OR movement* OR motion OR manual) NEAR/3 (technique* OR therap* OR treat* OR isokinet* OR isomet* OR water)) OR ((muscle*) NEAR/3 (train*))):ab,ti) AND ('clinical effectiveness'/exp …)

('hip osteoarthritis'/exp …) AND ('kinesiotherapy'/exp OR (kinesiotherap* OR kinesitherap* OR exercis* OR physiotherap* OR ((movement* OR motion OR manual) NEAR/3 (technique* OR therap* OR treat* OR isokinet* OR isomet* OR water)) OR ((muscle*) NEAR/3 (train*))):ab,ti) AND ('clinical effectiveness'/exp …)

267 hits

('hip osteoarthritis'/exp …) AND ('kinesiotherapy'/exp OR (kinesiotherap* OR kinesitherap* OR exercis* OR physiotherap* OR ((movement* OR motion OR manual) NEAR/3 (technique* OR therap* OR treat* OR isokinet* OR isomet* OR water)) OR ((muscle*) NEAR/3 (train*))):ab,ti) AND ('clinical effectiveness'/exp …)

('hip osteoarthritis'/exp …) AND ((kinesiotherap* OR kinesitherap* OR exercis* OR physiotherap* OR ((movement* OR motion OR manual) NEAR/3 (technique* OR therap* OR treat* OR isokinet* OR isomet* OR water)) OR ((muscle*) NEAR/3 (train*))):ab,ti) AND ('clinical effectiveness'/exp …) NOT 'kinesiotherapy'/exp

('hip osteoarthritis'/exp …) AND ('kinesiotherapy'/exp OR exercise/exp OR physiotherapy/exp OR (kinesiotherap* OR kinesitherap* OR exercis* OR physiotherap* OR ((movement* OR motion OR manual) NEAR/3 (technique* OR therap* OR treat* OR isokinet* OR isomet* OR water)) OR ((muscle*) NEAR/3 (train*))):ab,ti) AND ('clinical effectiveness'/exp …)

355 hits

End Result – Systematic Review

('hip osteoarthritis'/exp OR ((arthritis/de OR osteoarthritis/de) AND (hip/de OR 'hip pain'/exp)) OR (coxarth* OR ((hip OR cox) NEAR/6 (arthrit* OR arthrosis* OR arthroses* OR osteoarth* ORoa)) OR 'malum coxae senilis'):ab,ti) AND ('kinesiotherapy'/expOR exercise/exp OR physiotherapy/exp OR 'conservative treatment'/de OR (kinesiotherap* OR kinesitherap* OR exercis* OR physiotherap* OR gymnastic* OR ((movement* OR motion OR manual OR phys* OR conservative* OR nonoperat* OR nonsurg* OR non-operative OR non-surgical OR paramedic* OR para-medical) NEAR/3 (technique* OR therap* OR treat* OR isokinet* OR isomet* OR water OR management* )) OR ((muscle* OR muscul*) NEAR/3 (train*))):ab,ti)

1916 hits

Considerations

Explode a broad term or not?

Start exploded, when many irrelevant results appear, be more specific

Truncate a term or not?

Truncate as much as possible, hardly noise

How broad should proximity be?

Start with 3, increase with 3 and

evaluate newly retrieved articles

Important to remember

• Create single line syntax in Word document

• Type code (() ; ‘’/exp; OR ():ab,ti ; (() NEAR/3 ()) ) before filling in words

• Copy emtree terms into your document

• Use the synonyms in ():ab,ti, creatingproximity searches from phrases

• Optimize by comparing emtree NOT ab,ti and the other way around

Semi-automatic and fast translation toother databases

E-mail me for more information:

• Roadmap to optimizing queries in embase.com explaining every step in detail

• Sharing Macros to translate syntax of embase.com to other databases

Contact details

E-mail: w.bramer@erasmusmc.nl

Twitter: @wichor

LinkedIn: nl.linkedin.com/in/wichor

Slideshare: www.slideshare.net/wichor

Meet me in person:

15-20 May MLA conference, Austin, TX (USA)

10-12 June EAHIL workshop, Edinburgh (UK)

Further reading:

• Bramer WM, Giustini D, Kramer BM, Anderson PF. The comparative recall of Google Scholar versus PubMed in identical searches for biomedical systematic reviews: a review of searches used in systematic reviews. SystRev. 2013 Dec 23;2:115.

• Bramer WM, Holland L, Mollema J, Hannon T, Bekhuis T. Removing duplicates in retrieval sets from electronic databases: comparing the efficiency and accuracy of the Bramer-method with other methods and software packages

• Bramer WM, Pols DHJ, Bindels PJE, Bohnen AM. A fast, reliable and objective method for creating thematic search filters in bibliographic databases [in process]

• Work in progress on coverage, retrieval and relevance ranking of the major databases for systematic reviews

Additional topics

• Speed of searches

• Quality of searches

• Success factors

• Why start in embase

• Do you need to understand the topic

• Set numbers or single line queries

• Which databases do you use

• Subheadings

• Interface choice

Speed of searches

42%

73%

91%

0%

25%

50%

75%

100%

0 30 60 90 120 150 180

Cu

mu

lati

ve p

erc

en

tage

of

SRs

(N=2

07

)

Minutes needed to create search

Proclaimer on speed

• The described method does not guarantee the speed shown here

• Nor should this be the norm

• Systematic searches (especially for reviews) should always aim at optimizing sensitivity

• Quality of the search results is what counts(see the next slide)

Quality of searches

(medianvalues)

Bramer (N=96-402)

Benchmark (N=68-93)

Difference

# terms 54 45 21%

# databases 7 3 133%# hits 2108 1399 51%# includes 36 19 89%precision 1,6% 1,2% 29%

Benchmark: All SRs published in 2014 by Dutch university hospitals (not including Erasmus MC)with a librarian co-author or librarian mentioned in the acknowledgements

Success factors

1. Experience (hardly controllable)

2. Interface choice and database order (habit, and management decision)

3. Optimization techniques (this workshop)

4. Single-line search strategies (habit)

5. Macros (available upon request)

6. The researcher immediately providing feedback on the results (habit)

Why start in embase?Source: Unpublished research Bramer (4321 included references from 96 reviews)

0% 20% 40% 60% 80% 100%

Google Scholar

Embase

Medline

first 300 first 1000 retrieval coverage citations (GS)

Do you need to fully understand the research question?

What is the Cell contribution to chondroid / endochondreal / intramembranous bone formation

((('bone development'/de OR 'bone mineralization'/de OR 'bone

maturation'/de OR ossification/de OR 'bone conduction'/de OR 'bone

transplantation'/exp OR 'bone matrix'/exp OR 'bone growth'/exp OR

'bone remodeling'/de OR ((bone NEAR/6 (minerali* OR develop* OR

format* OR synthesis* OR growth* OR graft* OR transplant* OR

engineer* OR defect* OR matri* OR remodel* OR conduct* OR maturat*))

OR osteogen* OR ossificat*):ab,ti) AND (chondroid* OR

transchondroid* OR enchondr* OR endochondr* OR intermembran* OR

cartilagin* OR (inter NEXT/1 membran*)):ab,ti) OR 'enchondral

ossification'/de) AND ('cell death'/de OR 'cell survival'/de OR

apoptosis/de OR 'cell differentiation'/de OR 'cell fate'/de OR 'host

cell'/de OR 'cell lineage'/de OR (((cell* OR msc OR osteoblast* OR

osteocyte* OR osteoclast* OR preosteoblast*) NEAR/3 (death OR

senescen* OR contribut* OR differentiat* OR fate OR plasticit* OR

reprogram* OR donor* OR host* OR recruit* OR surviv* OR viabilit* OR

lineage*)) OR apoptosis):ab,ti)

Single line approach or set numbers?

Well structured?

#1 brassica/exp

#2 brassica*:ab,ti

#3 broccoli:ab,ti

#4 #1 OR #2 OR #3

#5 neoplasm/exp

#6 neoplas*:ab,ti

#7 cancer*:ab,ti

#8 tumor*:ab,ti

#9 #5 OR #6 OR #7 OR #8

#10 #4 AND #9

#11 [animals]/lim

#12 [humans]/lim

#13 #11 AND #12

#14 #11 NOT #13

#15 #10 NOT #14

Unstructured?

(brassica/exp OR (brassica* OR

broccoli):ab,ti) AND (neoplasm/exp

OR (neoplas* OR cancer* OR

tumor*):ab,ti) NOT ([animals]/lim

NOT [humans]/lim)

Single line approach or set numbers?

Disadvantages of set numbers:

1. Optimization is complicated (cannot easily add terms)

2. Repeating a search is difficult and time consuming

3. Translating to other databases is difficult (different number of thesaurus terms)

Disadvantages of single line approach:

1. Use of many parentheses easily results in errors type parentheses before pasting words words

2. You cannot see what a single term does (spot errors) because of optimization with NOT errors are easilyrecognizable

72

When to use what databases?

(N=3661 incl ref) total retrieval unique

embase 2913 80% 470 48%

medline 2529 69% 190 19%

Web-of-Science 1907 52% 187 19%

Google Scholar 1510 41% 88 9%

Other databases 54 5%

The databases I use:Database Used in % of SR

Embase.com 100%

Medline (OvidSP) 98%

(PubMed the other 2%)

Cochrane central 97%

Web-of-Science 94%

PubMed (publisher) 90%

Google Scholar 90%

Scopus 40%

Cinahl 23%

PsycINFO 16%

Other (proquest, sportdiscus, lilacs, scielo, eric, amed etc)

<3%

What to do with subheadings

Separate element

(broccoli/de OR (broccoli* OR bassica*):ab,ti) AND (neoplasm/expOR (cancer* OR neoplas*):ab,ti) AND (prevention:lnk OR prevention/expOR (prevent*):ab,ti)

Combined element

(broccoli/de OR (broccoli* OR bassica*):ab,ti) AND

(neoplasms/exp/dm_pc OR ‘cancerprevention’/de OR ((cancer OR neoplasm) NEAR/3 (prevent*)):ab,ti)

Sensitive

More hits : 345 (101 unique)

Less chance of missing relevant

articles

Recommended for start

For SRs highly recommended

Specific

Less hits : 257 (13 unique)

Higher chance of missing relevant

articles

To reduce the number of hits

Use proximity in free text

Interface choiceEmbase.comPros- Very good relevance ranking

Cons- Translating between embase

and medline more complicated

- No wildcard (?) in truncation- Special characters not

normalized ( )

OvidSPPros- Equal syntax to medline

(OvidSP)

Cons- No relevance ranking on

complicated queries- Complex thesaurus interface

My advice: relevance ranking is very helpful duringoptimization. Problems are onlyminor. Start in embase.com andtranslate with macros