Dal grading al GRADE

Nicola MagriniArea Valutazione del Farmaco

Agenzia Sanitaria e Sociale Regionale, BolognaWHO Collaborating Centre for Evidence Based Research Synthesis

and Guideline Development

1

1. Linee-guida e raccomandazionievidence-based: definizioni e ruolo

2

Definizione di linea-guida e

raccomandazioni

• "Guidelines are recommendations intended to assist providers and recipients of health care and other stakeholders to make informed decisions. Recommendations may relate to clinical interventions, public health activities, or government policies."

WHO 2003, 2007

3

Distinguere fra qualità delle evidenzee forza delle raccomandazioni

• Si tratta di due informazioni complementari in buona parte indipendenti

• La prima valuta la qualità metodologica delle prove disponibili per stimare gli effetti del trattamento (benefici e rischi)

• La forza considera:– la fattibilità, la trasferibilità – i benefici e rischi attesi e la loro rilevanza– le implicazioni organizzative , economiche, sociali

e finanziarie (quindi rispetto al contesto) dell’interventi proposto

4

Che cosa è la qualità delle evidenze

in una linea-guida

In the context of making recommendations:

• The quality of evidence reflects the extent of our confidence that the estimates of an effect are adequate to support a particular decision or recommendation.

Forza della raccomandazione

“The strength of a recommendation reflects the extent to which we can, across the range of patients for whom the recommendations are intended, be confident that desirable effects of a management strategy outweigh undesirable effects.”

• Strong or conditional

• Forte e …

6

Vecchi sistemi di grading

• La qualità delle evidenze rifletteva sostanzialmente il disegno degli studi e la loro conduzione

• La forza delle raccomandazioni dipendeva dal disegno e in parte dalla qualità metodologica degli studi … quindi …

• Ridondanze e ripetizioni da un lato ma anche mancate considerazioni della entità dell’effetto, della eterogeneità tra gli studi e della trasferibilità dei risultati …

7

8

Raccomandazioni e evidenze

• Per poche raccomandazioni chiave:– Reflect and discuss on relevant outcomes

– Search for and retrieve all available evidence

– Identify relevant SRs

– Formally assess quality of evidence

– GRADE (systematic and transparent approach)

– But also: Panel composition and role of chair

– Manage CoI

9

The scope

• Small is beautiful (S. Hill)

• Who is the target user of the guideline

• Who it applies to

• What is covered?

– Eg diagnosis and treatment of diabetic retinopathy

• Develop key questions (<10-20…..)

10

2. Elaborazione di linee-guidaIl gruppo di lavoro (Panel)

11

12

How to improve guideline production

Attuali limiti:

• Governance and composition of the guideline committee (“what is to be decided is often already decided with the

selection of the deciders”)

• Unanimity in guideline (not a natural component in research)

• Lack of independent review (outside the accepted procedures

of scientific publications)

• CoI

Sniderman AD, Furberg CD.Why guidelines making requires reform

JAMA 2009

Group composition

• One systematic review (Murphy et al. 1998)

• Composition of panel influences recommendations– Members of a specialty are more likely to advocate techniques that

involve their specialty

• Balanced groups– Select the appropriate group leader

• Necessary technical skills– including information retrieval, systematic reviewing, health economics,

group facilitation, project management, writing and editing

• Include or have access to content experts

• No SR on how to obtain consultation, but logical reasons support this

• Up to 15 members

13

Composizione del GdL/Panel

• „Include all who are affected“

- To identify the right questions

- To identify areas of suboptimal care

- To identify feasibility of recommendations

• Consequences

- Definition of Standards of Care

- Ownership to improve implementation

14

Composizione del GdL/Panel in RER

• Cosa significa includere i competenti/responsabili …. Non esattamente „all who are affected“

- Andare verso gruppi di competenti (non necessariamente i

responsabili/primari)

- Gruppi stabili versus gruppi ad hoc

- Separare meglio fase di elaborazione da quella di disseminazione

introduzione /implementazione? Quale formazione specifica?

• Consequences

- Alta variabilità tra gruppi e tra prodotti

- Rapporti con istituzioni (e Soc. Scient.) variabili

15

Expertise needed in the group

• Medical content:

health care professionals

• Values and preferences:

patients / carers / community

• Support staff /Secretariat: metodologi, clinici part-time (epidemiologists, health economists), … administrative support

16

3. Perché il GRADE

17

Which approach?

Evidence Recommendation

• B Class I

• A 1

• IV C

Organization

� AHA

� ACCP

� SIGN

Recommendation for use of oral anticoagulation in patients with atrial fibrillation and rheumatic mitral valve disease

18

LG e sintesi evidenze

• La parte di qualità e sintesi delle evidenze

– Essenziale e importante

– Puo essere commissionata esternamente a un gruppo o dalla segreteria scientifica di supporto

– Non coinvolge direttamente il Panel (se non per commenti e integrazioni)

– È certamente la piu lunga e impegnativa assieme alla stesura del testo finale

19

Study design is important

� Early systems of grading the quality of evidence focused almost exclusively on study design

� Randomised trials provide, in general, stronger evidence than observational studies:

–RCTs start at High Quality

–Observational studies start at Low Quality� However, other factors may decrease or increase the quality of evidence

20

Quality assessment criteriaQuality of

evidence

Study design Lower if Higher if

High Randomised trial

Moderate

Low Observational

study

Very low

Study quality:

-1 Serious

limitations

-2 Very serious

limitations

-1 Important

inconsistency

Directness:

-1 Some

uncertainty

-2 Major

uncertainty

-1 Sparse or

imprecise data

-1 High probability

of reporting bias

Strong association:

+1 Strong, no

plausible

confounders

+2 Very strong,

no major

threats to

validity

+1 Evidence of a

Dose response

gradient

21

Factors that may decrease the quality of evidence

�Study limitations (risk of bias)

�Inconsistency among studies

�Indirectness of evidence

�Imprecise results

�Reporting bias

22

Vecchie gerarchi troppo semplici: l’importanza della competenza clinica

STUDY DESIGN

� Randomized Controlled

Trials

� Cohort Studies and Case

Control Studies

� Case Reports and Case

Series, Non-systematic

observations

BIAS

Expert Opinion

Exp

ert O

pin

ion

Schünemann & Bone, 200323

Strong / weak

recommendation

Quality assessment

criteria

Ratings of outcomes

Quality assessment

criteria

Quality of evidence:

estimates of benefits &

harms and risk of bias,

directness, …

Risk-benefit profile evaluation for different subgroups

Evidence

Emilia Romagna New Cancer Drugs Rec.

GRADE steps and polls• We (in Italy) adapted GRADE by:

– Rating/voting also the evaluation of the benefit-risk profile (favourable, uncertain, unfavourable)

– Defining an expected use indicator

– Incorporating the results of polls in the final document

+ expected use24

4. dalle evidenze allaraccomandazione

25

Getting from evidence to

recommendations - GRADE

Recommendations are judgments:

– Quality of evidence

– Trade off between benefits and harms

– Values and preferences

– Resource use

But judgments need to be based on the best available evidence and transparent

26

Che cosa determina la direzione e la forza

di una raccomandazioneFactors that can strengthen a recommendation

Comment

Quality of the evidence The higher the quality of evidence, the more likely is a strong recommendation.

Balance between desirable and undesirable effects

The larger the difference between the desirable and undesirable consequences, the more likely a strong recommendation warranted. The smaller the net benefit and the lower certainty for that benefit, the more likely weak recommendation warranted.

Values and preferences The greater the variability in values and preferences, or uncertainty in values and preferences, the more likely weak recommendation warranted.

Costs (resource allocation) The higher the costs of an intervention – that is, the more resources consumed – the less likely is a strong recommendation warranted

27

• Desirable outcomes– lower mortality– reduced hospital stay– reduced duration of disease– reduced resource expenditure

• Undesirable outcomes– adverse reactions– costs of treatment

• Every decision comes with desirable and undesirable consequences�Developing recommendations must include a

consideration of desirable and undesirable outcomes

Scelta degli outcomes … prima di leggere gli studi

28

GRADE:

recommendation – quality of evidence

Clear separation:

1) 4 categories of quality of evidence: ⊕⊕⊕⊕ (High), ⊕⊕⊕(Moderate), ⊕⊕(Low), ⊕(Very low)?– methodological quality of evidence– likelihood of bias– by outcome and across outcomes

2) Recommendation: 2 grades – conditional (aka weak) or strong (for or against an intervention)?– Balance of benefits and downsides, values and

preferences, resource use and quality of evidence

*www.GradeWorking-Group.org29

Che cosa è la qualità delle evidenze

in una linea-guida

In the context of making recommendations:

• The quality of evidence reflects the extent of our confidence that the estimates of an effect are adequate to support a particular decision or recommendation.

Forza della raccomandazione

“The strength of a recommendation reflects the extent to which we can, across the range of patients for whom the recommendations are intended, be confident that desirable effects of a management strategy outweigh undesirable effects.”

• Strong or conditional

• Forte e …

31

Implications of

a strong recommendation

• Patients: Most people in this situation would want the recommended course of action and only a small proportion would not

• Clinicians: Most patients should receive the recommended course of action

• Policy makers: The recommendation can be adapted as a policy in most situations

32

Implications of

a conditional/weak recommendation

• Patients: The majority of people in this situation would want the recommended course of action, but many would not

• Clinicians: Be more prepared to help patients to make a decision that is consistent with their own values/decision aids and shared decision making

• Policy makers: There is a need for substantial debate and involvement of stakeholders

33

6. How to improve transparency in goingfrom evidence to recomendations

34

Contesto, evidenze, quesito, esiti

Raccomandazione(implicazioni e indicatori)

Entità dei beneficie dei rischi

Rapporto beneficio/rischiAccettabilità e preferenze

Costi e uso risorse

35

Going from evidence to recommendationrisk benefit profile, values and preferences, costs (1/3)

36

Going from evidence to recommendationrisk benefit profile, values and preferences, costs (2/3)

37

Going from evidence to recommendationrisk benefit profile, values and preferences,

costs and fesibility (3/3)

38

7. Il processo di formulazione della raccomandazione

I fattori legati al Panel e al contesto

… e il CoI

39

Il processo: come renderlo costruttivoe trasparente?

• Group members are heterogeneous and might have different

objectives

• Chair facilitates rather than leads the group

• Common understanding of goal, tasks and ground rules

• Similar level of required know-how and skills

• Sufficient technical support in research synthesis

40

Panel: balanced participation andformal agreements

• Key task of chair

• Drafting of manuscript

• Formal consensus processes

- Delphi Method

- Nominal group process

- Voting

41

How to present controversies

• Lay out the controversies

• Describe the evidence

• Ask members to focus on the agreed upon evidence and the factors leading to a decision

• Ask whether there still is disagreement

• Vote

– Make voting explicit and transparent (ways of doing this to come tomorrow)

42

Esempi di raccomandazioni che hanno usato il GRADE

Examples of flexibility and transparency

43

LG RER 2012 e GRADE

• GREFO farmaci oncologici• diabete IGEA, Incretine, • Farmaci biologici (anti-TNF, …) molti, vari e

diversi• Diabete gestazionale, • Gravidanza fisiologicaIn corso:• Nuovi anticoagulanti?• … nuovi farmaci anti HCV, • … altri argomenti / gruppi RER (PET …)

44

LG RER 2012 e GRADE …se siete interessati potremmo farne 1 alla volta

1. GREFO farmaci oncologici2. diabete IGEA, 3. Incretine, 4. Farmaci biologici (anti-TNF, …) molti, vari e diversi5. Diabete gestazionale, 6. Gravidanza fisiologica7. Farmaci antiaggreganti IMA/SCA pre e in-hospitalIn corso:8. Nuovi anticoagulanti?9. … nuovi farmaci anti HCV, 10. … anche altri temi NON farmacol

45

GRADE ed esperienze RER/ITA

• Via via che su alza il livello di committenza della LG, più il GRADE sempre essere utile … come strada e linguaggi comuni (fornisce un buon framework)

• MA in RER …

• Sembra mancare spesso un interesse dei Panelisti (vi è anzi il timore) a trovare una posizione nuova e originale

• Molti di questi recenti esempi regionali e nazionali evidenziano come si sia giunti a posizioni di buona rilevanza …

• Fare un Network di GRADErs?

• Fare Corsi di base (e avanzati)?

46

GREFO …

• Formato attuale

• Esempi vari

• Da rivedere e snellire il formato

47

Incretine

• IGEA …

• Per incretine RER … utile

• Anche SE …

• Panel difficile e con regole troppo casalinghe …

48

Diabete gestazionale

49

NAO?

50

Domande e discussione

51

Grazie dell’attenzione

E fate buon uso del GRADE

52