Systematic Reviews

By Jonathan Tsun & Ilona Blee

What is a systematic review? Define it!

Results from similar randomized trials are brought together (“synthesized”) to help identify which forms of health care work, which don’t work, and which are harmful.

This combination of randomized trials needs to be done in a systematic way – hence it is called a systematic review

The methods used include steps to minimize bias in the process – this is done by including all of the relevant studies

What are the TWO main types of bias?

1. Selection

2. Information

What are the limitations of systematic reviews?

“Synthesis” of results from different studies can oversimplify important distinctions

Different conclusions can be reached depending on the wording of the “review question”

For example compare these two questions: to be able to know whether we can predict a problem and intervene early vs. to identify the best ways of diagnosing/identifying problems

Reviews can make it difficult for doctors to apply the results of the review to every situation they come across over-generalisation, medicine isn’t an exact science!

Findings from systematic reviews aren’t always consistent with other findings of large-scale high quality trials

What is a meta-analysis?

When the results of individual studies are combined to produce an overall statistic.

This can be done without a systematic review by combining results from more than one trial

What is the disadvantage of this?

Without the systematic review, there will be bias present if there is any from that study

This can lead to clinically misleading results!!!

Forest plot – what is it? A graphical representation of a meta analysis,

illustrating the effect sizes of individual studies and the meta-analysis as a whole.

What is “effect size”? Quantitative measure of the strength of a

phenomenon (e.g. correlation between 2 variables, etc)

It will have a list of the studies included in the meta analysis

What is the line at x=0 called? What does it indicate? Line of No Effect

If a study’s confidence interval line crosses the Line of No Effect, it demonstrates that at the given level of confidence, the study’s effect size does not differ from no effect. (AKA the study’s findings are not significant)

What does the size of the green square indicate? It is a graphical representation of the weight of that

particular study in the meta-analysis.

What do the lines sticking out from the green boxes indicate? They are graphical representations of the size of confidence

intervals.

What does the diamond represent? The effect size of the meta-analysis as a whole

What do the 2 horizontal points of the diamond indicate? What does it mean if it crosses x=0? The 2 horizontal points indicate the confidence interval for the

combined data

If it crosses the line of no effect... Answer is below in the notes.

Funnel Plot – what are they/ what do they do?

They assess whether the results of a systematic review have been influenced by “publication bias”

Publication bias occurs when the publication of research results depends on their nature and direction

eg. Positive results are more likely to be published than negative studies and can mean systematic reviews end up biased towards positive results

If the plot is symmetric, this means that there is probably no publication bias

If the plot is asymmetric, this means that publication bias is likely

Symmetrical = no publication bias!

Asymmetrical= publication bias likely

If we were to critically appraise a study, what

would we ask? (3)1. Is the study valid?

2. Are the results reliable?

3. Can I generalise from this study to my workplace/own practice? OR will the results help locally?

Why is critical appraisal important?

Ensures that relevant articles contribute towards patient care

Tries to ensure that bias is minimised

Ensures that the results from a study can be used in real life

An example of evidence-based medicine!

CURB-65

This is something you should have covered in CBL and comes up in exams/ on the wards etc.

What does it stand for?

Confusion

Urea levels of >7mmol/L

Respiratory rate >30 breaths/min

Blood pressure <90mmHg systolic over <60mmHg diastolic

Age >65 years

You get a point for each of these

What does PRISMA stand for?

Preferred Reporting Items for Systematic reviews & Meta-Analyses

In other words, helps authors report the results of a systematic review

More at: http://prisma-statement.org/

Using systematic reviews!

Where do we search for systematic reviews?

1. PICO! Population/patient problem, intervention, comparator, outcome

2. Cochrane library

3. Medline

What is PICO used for?

Asking a “good” question

What are the steps for using PICO/how do we use PICO?

How to explain this ..

Don’t forget to learn your equations and calculations!

Weighted mean difference: Mean(g1) – Mean(g2)

Standardised mean difference: Mean(g1) – Mean(g2)

pooled sd

Relative risk:

Number needed to treat (NNT):

Risk Ratio:

Be able to interpret risk ratio quite quickly

E.g. RR = 2

Means the intervention outcome is twice as likely compared to the control outcome

E.g. RR = 0.5

Means the intervention outcome is half as likely compared to the control outcome

What is bias? Any systematic error in an epidemiological study

which either underestimates or overestimates the parameter of interest because of a deficiency in the design or execution of the study.

Selection Information

Sampling bias Recall bias

Allocation bias Observer/interviewer/reporting bias

Responder bias Social desirability bias

Attrition bias

Publication bias

Healthy worker effect

Hierarchy of evidence!

1. Systematic reviews – always at the top over anything ever because of the way it minimises bias!

Meta-analyses

Non meta-analytic reviews

2. Experimental studies

Randomised controlled trials

Controlled trials

3.Observational studies

Cohort studies

Case-controlled studies

4.Descriptive studies

Cross-sectional designs

What order to the study designs go in?

Match the question with the type of evidence needed!

Question Evidence

How effective is this preventative intervention?

RCT

How effective is this therapy? RCT

How effective is this screening/diagnostic tool?

RCT

Is this a causative factor for this disease?

Cohort or case-controlled studies

What is the prognosis for this? Cohort or case-controlled studies

How do patients feel about their condition?

Qualitative experimental studies

How common is this condition? Cross-sectional studies

How cost-effective is this intervention?

RCT

The end! Any questions?