Reviewing the Literature

Prof. Jimmy Volmink

PHC Directorate

Faculty of Health Science, UCT

What is a literature review?

• Account of what is written on a particular subject by scholars and researchers

• Purpose: convey current knowledge and ideas as well as their strength and weaknesses

• May be: – part of essay, research report or thesis – stand alone exercise

Annotated bibliography vs. literature review

• AB: List of sources each accompanied by a brief note (annotation) discussing one or more aspects e.g. content, usefulness, quality, etc

• LR: Discursive writing – presents themes and concepts; integrates ideas and concepts

Objectives of literature review

1. Address a research question or thesis

2. Critically appraise research for validity

3. Synthesize and summarise current knowledge

4. Identify controversies in literature

5. Formulate questions for future research

Information explosion

• Journals

• Textbooks

• Unpublished literature

• World Wide Web

Reviews… reduce the load!!!

Reviews: what do we want ?

• A readable summary of ALL the evidence

• Unbiased

• Transparent

• Up-to-date

But is this what we get?

Corticosteroids for prevention of Corticosteroids for prevention of respiratory distress syndromerespiratory distress syndrome

• Expert review of the literature (Robertson, BMJ 1982)

• “The evidence suggests that antenatal steroids are of value only in white males, and even for them the benefit is mainly in those of 30-32 weeks’ gestation…”

Corticosteroids for prevention of Corticosteroids for prevention of respiratory distress syndromerespiratory distress syndrome

• Another view (Crowley, 1989)

• “The benefits of antenatal corticosteroids have been established. No further trials are necessary with the exception of certain specific situations (such as pre-eclampsia) or to establish other dosages or routes of administration.”

Treatments for Acute MI

• Existing evidence compared with recommendations in textbooks and journals

• “advice on some life-saving therapies has been delayed for more than a decade, while other treatments have been recommended long after controlled research has shown them to be harmful.” Antman et al. JAMA 1992; 268: 240-8

Traditional review

• Qualitative, narrative summaries of evidence on a given topic usually written by an expert in the field.

• They typically involve informal and subjective methods to collect and interpret information

Problems with traditional reviews 1. Bias in selection of included studies

• Personal (“file drawer”) bias

• Publication bias:

– Studies with statistically significant results are:

• More likely to be published

• More likely to be published in English

• More likely to be published in journals with high citation impact factors

Easterbrooke et al, Lancet

1991

Problems with traditional reviews 2. Indexing bias

AllRCTs

AllRCTs

OngoingOngoing

CompletedCompleted

RCTs notidentified byMEDLINE

RCTs notidentified byMEDLINE

RCTsidentified byMEDLINE

RCTsidentified byMEDLINE

UnpublishedUnpublished

PublishedPublished

Abstracts,theses, etcAbstracts,theses, etc

JournalsJournals

MEDLINEjournals(c. 4600)

1966+

MEDLINEjournals(c. 4600)

1966+

Non-MEDLINEjournals

(c. 12000)1966+

Non-MEDLINEjournals

(c. 12000)1966+

Problems with traditional reviews 3. Variable quality of published studies

• Fourth Congress on Peer Review in Biomedical Publication :

“Medical journals are full of serious methodological errors “

“Journal editors are giving no time, energy and thought to their craft”

“Studies are published that reach false conclusions”BMJ, Sept. 2001:323

Problems with traditional reviews 4. Effect of Study Design

PURPOSE: To assess the evidence of the effectiveness of vitamin supplementation, specifically vitamins A, C, and E; beta-carotene; folic acid; antioxidant combinations; and multivitamin supplements, in preventing cardiovascular disease.

STUDY SELECTION: The researchers selected English-language reports of randomized trials and cohort studies that assessed vitamin supplementation in western populations and reported incidence of or death from cardiovascular events.

CONCLUSIONS: Some good-quality cohort studies have reported an association between the use of vitamin supplements and lower risk for cardiovascular disease. Randomized, controlled trials of specific supplements, however, have failed to demonstrate a consistent or significant effect of any single vitamin or combination of vitamins on incidence of or death from cardiovascular disease.

Ann Intern Med. 2003;139:56-70.

Problems with traditional reviews 5. Effect of Study Size

“ … we still have no clear evidence that beta-blockers improve long-term survival after infarction despite almost 20 years of

clinical trials.”

JRA Mitchell. BMJ 1981;282:1565-70

The Science of Research Synthesis

“If science consisted of nothing but the laborious accumulation of facts, it would soon come to a standstill...…Two processes are at work side by side, the reception of new material and the digestion and assimilation of the old; and both are essential…”

Lord Rayleigh 1884

A Systematic ReviewHow is it different?

“A review in which bias has been reduced by the:

– systematic identification,

– appraisal,

– synthesis

– statistical aggregation (if relevant)

of all relevant studies on a specific topic according to a

predetermined and explicit method.”Moher et al.

Lancet; 354: 1896-900

Steps in conducting a systematic review

1. Formulate topic/question

2. Decide upon selection criteria for studies

3. Determine search strategy

4. Critically appraise research

5. Extract data

6. Synthesise and summarise data

1. The review question

• Do betablockers compared to placebo decrease morbidity and mortality in people with hypertension?

• Does the administration of Bacillus Calmette-Guerin (BCG) influence the risk of tuberculosis and all-cause mortality in populations exposed to infection?

• PICO Method

2. Selection criteria for studies

• Types of participants– People receiving BCG vaccination,

irrespective of age

• Types of interventions– BCG vaccination given by any route.

• Types of outcome measures– Death– Tuberculosis

• Types of studies – RCT

3. Search strategy

• Scope of search– Published vs. unpublished vs. grey literature– Peer-reviewed vs. non peer-reviewed– Electronic vs. paper based sources

• Search should be as comprehensive as possible

Ingredients of a comprehensive literature search

• Electronic databases (highly sensitive strategy)– MEDLINE/PUBMED– EMBASE– Cochrane Databases – CENTRAL – Other e.g. CINAHL, AIDSLINE, LILACS, etc.

• Handsearching of journals• Reference lists• Conference proceedings• Personal communication

4. Critical appraisal of research

• Different types of clinical research• Need to understand strengths and

limitations of various study designs• Know what aspects of a study to assess to

determine likelihood of bias

5. Extract data

• Pre-specify items of information to be collected

• Data extraction form

• Independent data extraction

6. Synthesise and summarise data

• Qualitative (narrative)

• Quantitative synthesis (uses statistics)

• Some statistical knowledge will be needed

Meta-analysisPooling the results

• A statistical procedure that pools the results of several independent studies considered sufficiently similar (homogeneous)

• Not appropriate if studies are too dissimilar (heterogeneous)

• Provides a quantitative summary of the overall treatment effect

• Visual display called a “forest plot”

• Estimate of effect

• Confidence intervals

• Weighting

• Overall estimate of effect

Meta-analysis and the Forest Plot

Meta-analysis Beta-carotene and cardiovascular mortality

0.1 0.5 0.75 1 1.25 1.5 1.75

Male health workers

Social insurance, men

Male chemical workers

Hyperlipidaemic men

Nursing home residents

Social insurance, women

Male physicians

Male smokers

(Ex)-smokers, asbestos workers

Trials

Cohort studies

Skin cancer patients

USA

Finland

Switzerland

USA

USA

Finland

Cohorts combined

Trials combined

Finland

USA

USA

USA

Relative risk (95% CI)

Acknowledging your sources

• Why bother:– Honesty and transparency– Show research done to reach conclusions– Allow readers to identify and retrieve

references

• Plagiarism – “taking, using and passing off as your own work the ideas or words of another”– See UCT policy

Types of Citation

Two main types:

1. Harvard: author-date style– …as one author has put it “the darkest days were

still ahead” (Weston 1988, p.45)

2. Vancouver: footnote/endnote style set by the International Committee of Medical Editors– …as one author has put it “the darkest days were

still ahead”[1]• [ ] , ( ) or supercript …ahead”1 all acceptable

In summary…

Traditional vs. systematic review

Traditional review

• Qualitative, narrative summaries of evidence on a given topic usually written by an expert in the field. Typically, involve informal and subjective methods to collect and interpret information

Systematic review

“A review in which bias has been reduced by the systematic identification, appraisal, synthesis, and, if relevant statistical aggregation of all relevant studies on a specific topic according to a predetermined and explicit method.”

Moher et al. Lancet 1999; 354: 1896-900

Cochrane review

• A systematic review produced by the Cochrane Collaboration (www.cochranelibrary.com)– Standardised format– Extensive peer review– Published electronically on the Cochrane Library (and indexed in

Medline)– Invites comments and criticism– Kept up-to-date

• Quality and reporting on average better than other systematic reviews ….but not without flaws!

Summary: Types of reviews

All reviews

Systematic reviews

Meta-analyses