Finance Forum 1 June 2011. Welcome & Overview 1 June 2011 David Sturgiss.
DSRU June 2011 1
-
Upload
fanfanlatulipe -
Category
Health & Medicine
-
view
451 -
download
3
description
Transcript of DSRU June 2011 1
An agency of the European Union
Defining objective measures of signal
prioritisation - issues to consider in the
masking effect of measures of
disproportionality François MAIGNEN (nothing to get hung about …)
Presented by: Name SurnamePosition or Unit/Sector/Section/Team
2
Plan of the presentation
Conflicts of interest
Background and rationale for the projects
Definition of objective measures of signal prioritisation
Assessment of the extent and magnitude of the masking effect of the PRR in EudraVigilance
Concluding remarks and take home messages
3
Conflict of interests
M. Hauben has contributed to this study
No external funding was received for this study
I do not have any financial interests with the Pharmaceutical industry or any IT software provider (declaration available from the Agency)
Disclaimer on the views expressed in this presentation wrt European Medicines Agency
No claim on a “better” safety profile on any medicinal product mentioned in this work should be made.
My declaration of interests can be obtained from the Agency
4
Acknowledgements
No external source of funding was used to perform this study. The following authors: FM has no conflicts of interest with the pharmaceutical industry (declaration of interest available from EMEA). M. Hauben is also working in Department of Medicine, Risk Management Strategy, Pfizer Inc., New York, New York University School of Medicine, Departments of Community and Preventive Medicine and Pharmacology, New York Medical College, Valhalla, New York, USA and for the School of Information Systems, Computing and Mathematics, Brunel University, London, England . None of the authors have any conflict of interests with any statistical software provider. M. Hauben did not (could not) access the raw data for confidentiality reasons.
5
Disclaimer
The views expressed in this presentation are the personal views of the author(s) and may not be understood or quoted as being made on behalf of or reflecting the position of the European Medicines Agency or one of its committees or working parties.
6
Signal prioritisation and serious medical events: reported rate of fatality as a prioritisation variable
About the EV-EWG IME list and lists of IMEs in general (e.g. CIOMS V)
Useful but purpose not always clear (early signal detection? Focus the detection? Signal prioritisation?)
Based on expert’s judgment
Has not been formally “validated” / tested (no standards)
Probably situation dependant
7
Concept of seriousness # linked to the outcome # surrogate for grading the severity of the reactions hence prioritisation
Grading in seriousness: death >> disability (permanent) >>> life-threatening >>> disability (temp.) >> prolongation hosp.
Variable linked to fatal outcome = reported rate of fatality
For each drug-event pair = No of reported fatal cases / total number of reported cases
Computed for the intensively monitored products
Reaction 1 Reaction 2 Reaction 3 Outcome (incl. fatal)
Surrogate to predict the outcome
8
Hazardous identification of serious events a priori
Some examples of reactions not usually considered to be serious per se which can be linked to most dramatic outcomes (e.g. dramatic increases of liver aminotransferases e.g. >100 ULN leading to liver failure, liver transplant and death)
Exhaustion/tiredness
Jaundiceincr. aminotransferases 500ULN
hyperbilirubinemia
Liver transplant Death
Prioritise these events on the associated reported outcome (here death)
9
Reported rate of fatality
Some reactions may be consistently linked to a high reported mortality rate
Some reactions are serious but do not lead to a fatal outcome
Some reactions are situation dependent (the reported rate of fatality may be highly variable)
For each of the MedDRA PT involved in a DEC in EudraVigilance, the following variables were computed across all the products involved in the reported combinations:
• Mean, min., max., range: max. – min., SD
10
Methods
DECs from last year in EudraVigilance concerning the “intensively” monitored products
Imported in ACCESS, computation of reported rate for each drug-event combination
Computation of average, min, max, SD and range for each MedDRA PT
Analysis in R using the ODBC connectivity
11
Average reported rate of fatality
Significant No of terms associated with a very low rate or no fatality (0%)
12
How does it relate to IME status?
Important No of IME terms not associated with fatal outcome (or associated with a very low reported rate)
RHS: IMEs
LHS: non-IMEs
13
How does it relate to IME status?
Reported rate of fatality for IMEs > non-IMEs
Number of events for which the reported rate is high which are non-IMEs
Very high number of IMEs for which the reported rate of fatality is zero.
IMEs useful for prioritisation?
The figure displays the boxplot of the average reported rate of fatality for non-IMEs (left)compared to IMEs (right) (red and blue line = mean rate for non-IMEs (red) vs IMEs (blue))
14
Mean / median
Difference of the mean rate / median between non-IMEs and IMEs but not very important (approx. 10%)
Confirmation that some non-IMEs are linked to a fairly high reported rate of fatality
And that some IMEs are associated to a very low rate
The analysis shows that some non-IMEs have a high minimum rate (graph not shown)
The median of the max. reported rate of fatality is quite low.
Can IMEs discriminate between serious / non-serious signals?
15
Relation between the different variables?
Lattice of the descriptive variables for the reported rate of fatality
16
17
18
19
Liver injuries
Clear relation between Reported rate # seriousness of injury and the severity of the outcome
Highest mean rate around 30% (1/3 fatal reports) with a max. at 75% (3/4)
Some inconsistencies (bilirubin disorders: hyperbilirubinaemia 18.7%, blood bilirubin increased 16.2%, blood bilirubin unconjugated increased 6.7% and bilirubin conjugated increased 6.3%)
Unclear or undefined concepts (liver disorders [?]) linked to a fairly high mean reported rate 18.9%, hepatic function abnormal 8.5% and liver function test (singular) abnormal 9.1%.
20
Data reduction (PCA)
21
Data reduction (PCA)
Principal components analysis (sorry I can’t remember whether I did it on the variance-covariance or on the correlation matrix)
Performed on Average, Min, Max, Range
99% of variability explained by first two PCs +++
First two PC closely associated to Max and Average
22
DiscussionThree set of events used for signal detection: mild reported rate of mortality, moderate and high
Reported rate of fatality can be useful (and should be used) for signal prioritisation
Needs to be considered with caution (events with rate of zero include e.g. Torsade de pointes, autism, Breast cancer in situ, Breast cancer stage I, Dermatitis exfoliative)
Does not replace DMEs
Death is not the only criterion which could be used
EudraVigilance = only serious reactions(!)
Some events are consistently associated either with low rate or conversely with very high rate
An agency of the European Union
Assessing the effect and extent of the masking effect of the PRR on SRS databasesF. Maignen / M. Hauben
24
Credits and acknowledgements
M. Hauben dragged me into this project against my will (he should be considered to be the promoter). FM devised the mathematical background. MH came up with the idea to use the EU-ADRs IMEs. FM added the additional MedDRA terms relevant to Public Health.
None of the authors have any conflict of interests with any statistical software provider. M. Hauben did not (could not and still cannot) access the raw data for confidentiality reasons.
25
Masking effect of measures of disproportionality (here = PRR)
The masking effect has first been described and identified by Gould in spontaneous reporting system databases (pharmacoepidemiology and drug safety in 2003 – 8 years ago).
The masking is a statistical artefact by which true signals are hidden by the presence of information reported with other medicines in the database. Therefore, the masking involves one given reaction and two products (the product for which the DA is conducted) and a possible masking drug.
The masking effect is a potentially important issue for Public Health which is not perfectly understood or perfectly quantified: some signals might be missed or identified with delay because of the presence (or a suspicion on the presence) of masking effect.
26
Masking effect of measures of disproportionality
In particular, there is no algorithm to identify the potential masking drugs to remove them from subsequent analyses aimed at identifying new signals using the statistical methods of signal detection based on disproportionality analysis.
We have developed an algorithm based on the computation of a simple The masking ratio has been developed to be intuitive. The highest masking drugs have the highest masking ratio.
From an underlying mathematical framework, we have developed a simple expression of the masking ratio (which can be easily computed on a database incl. No of computations and IT resources) which allow a fairly rapid identification of the main culprits.
27
28
Masking effect of measures of disproportionality
Recent studies have shown effects which were suspected from the article by Gould, that masking products are usually products for which the given reaction is known (i.e. listed in the SPC), therefore likely to have a high PRR (in the database in which the analysis is conducted) for the adverse drug event / reaction which is included in the disproportionality analysis.
Unfortunately, the authors could not conclude on any algorithm considering that this association is not systematically present (not all products with high PRR will induce a significant masking even if he masking generally involves products with a high PRR).
29
Masking effect of measures of disproportionality
The removal of the masking effect has the advantage to put the focus on a defined threshold (the signal is below or above the chosen threshold) rather than on the ranking.
Both approaches should provide a clear understanding how the removal of the masking product affects the ranking provided by the different methods of disproportionality.
30
Masking effect of measures of disproportionality (RRR)
Respective proportion of reports in the
database influences the extent of the
masking
The higher the proportion of reports
involving a product for a given
reaction the higher the masking
The lower the proportion involving a
given product over the total number of
reports in the database, the higher the
masking
31
Masking effect of measures of disproportionality
A reassuring result shows that if the masking is fairly prevalent in a database (approx. 60% of DECs have a MR > 1), however, the prevalence of a significant masking (MR > 1.1) is quite rare (0.006% of the DECs) and mainly affects events rarely reported in the database.
Some assumptions of size of the database must be checked when the model is implemented (our model is likely to be valid without any further assumptions of size) in EudraVigilance or SRS databases of a similar size.
32
33
Relation between the masking effect and the PRR (of the masking medicinal product for the given event)
MR > 1PRR > 2
34
Relation between the masking effect and the PRR (of the masking medicinal product for the given event)
MR > 1PRR > 2
35
The highest masking is induced by products known to induce the given reaction (and for which the PRR is likely to be increased)
Products of the same class induce the highest masking for similar reactions (gambling – ropinirole and pathological gambling – cabergoline, Fanconi syndromes, role of drug-drug interactions – rifampicin)
36
Exact MR
Requires an important number of cumbersome computations (easy to make a mistake in the contingency tables) since the exact MR:
-Is specific of both the DEC being masked and of the masking product, therefore the exact MR requires n*(n – 1) computations (where n is the No of DECs)
The simplified MR relies on approximations related to the size of the database (verified but it needs to be checked).
37
Simplified expression of the MRSimplified masking ratio is perfectly correlated to the exact MR
The simplified MR slightly overestimates the exact MR (not more than 10%)
Provides an excellent variable for an algorithm
But more importantly, the removal of the masking effect of the highest masking product marginally affects the ranking given by the measure of disproportionality (based on EXACT masking ratio)
The algorithm can be extended to more than one product since, by construction, the MR is multiplicative (caveat: MRC/A must be computed on new contingency table without B)
PRRA(withoutB)(withoutC)=MRB/A*MRC/A*PRRA(incl. B and C)
38
Simplified expression of the MR
39
The ranking is marginally affected by the removal of the masking effect induced by the highest masking drug (exact MR) (Ab anti Epo)
More signals
Ranking MARGINALLYaffected
40
The ranking is marginally affected by the removal of the masking effect induced by the highest masking drug (exact MR) (liver failure)
Ranking NOTaffected
41
Pitfalls•Overlapping datasets: the removal of the masking must be performed adequately (only remove the records which involve the masking product)
•Masking is different from SUBGROUPING (different algorithm which also relies on assumptions on the size of the db)
•The method of computation of the DA (report level vs event level) is likely to influence the masking (magnitude, removal of the records)
•Masking possibly more prevalent in Companies databases rather than Regulatory Authorities dbs (prevalence and highest masking effect)
•Assumptions on the size of the database must be checked in particular in small SRS databases
42
Proposed algorithm
43
Conclusions
Rationale approach to signal prioritisation and identification of the masking effect of measures of disproportionality
Simplified variable from a rather cumbersome mathematical framework
Pitfalls and caveats must be kept in mind, in particular the assumptions on the size of the database must be checked, Masking is different from subgrouping, overlapping datasets must be handled appropriately
The prevalence of significant masking (>10%) is rather low. The masking is induced by products which themselves are known to induce the reaction for which they are responsible of a masking effect
44
Conclusions
The masking marginally affects the ranking given by the measure of DA
The proportion of true signals / false positive revealed by the masking is currently unknown
The median time gained by the removal of the masking effect is also unknown
The analyses are exploratory in nature and one should not deviate from the general consensus concerning the systematic evaluation of SDRs
Masking possibly more prevalent in Companies SRS dbs