ICH M7 - Lhasa Limited
Transcript of ICH M7 - Lhasa Limited
Agenda
• The ICH M7 Guidelines
• The Process
• Negative Predictions
• Managing out of domains
• Using Strain Profiles
• Some Examples
The ICH M7 Guidelines
ICH
M7
Supports the use of in silico models
in decision-making
Adopted worldwide
Enables fast, safe decision-making
http://www.ich.org/products/guidelines/multidisciplinary/article/multidisciplinary-guidelines.html
1 Feb2018
14 March 2018
The ICH M7 Guidelines
• Control measures can be determined by considering• mutagenic potential
http://www.ich.org/products/guidelines/multidisciplinary/article/multidisciplinary-guidelines.html
The ICH M7 Guidelines
• Control measures can be determined by considering• mutagenic potential
• concentrations of impurities
http://www.ich.org/products/guidelines/multidisciplinary/article/multidisciplinary-guidelines.html
Option Control Process
1 [impurity] < specification in final drug product
2 [impurity] < specification in raw material, starting material or intermediate
3
[impurity] > specification in raw material, starting material or intermediate
+ fate and purge during subsequent steps ⇒ [impurity] in the final drug substance is
below the acceptable limit
4Sufficient knowledge of fate and purge to give confidence that final [impurity] <
acceptable limit without analytical testing
Decision-making under ICH M7
Database
searching
ICH
M7
Expert in silico
prediction
Statistical in
silico prediction
Expert
assessment
Classification
Reporting
Purge mitigation
Control
Impurity
identification
Test
Storing
Undertaking a Database Search
Public data
Pre-competitive
shared data
In-house dataClass 1
Mutagen
Carcinogen
Mutagen
Carcinogen ?
Class 2
Non-Mutagen
Class 5
Exact
match
Databases
Substructure /
similarity search Further supporting information for
expert review
Undertaking a Database Search
Public data
Pre-competitive
data
Sources include:
• FDA
• NTP
• ISSSTY
• Kirkland
• Hansen, Bursi
• MPDB
• Literature
In vitro genetox
• 185,018 | 10,602
In vivo genetox
• 12,488 | 2,995
Overall-call genetox
• 35,092 | 10,932
Carcinogenicity
• 16,398 | 3,866
Aromatic amines
• 9,402 | 666
Intermediates
• 21,507 | 1,338
Excipients
• 2,985 | 1,071
Consortia of Lhasa
members
In-house data
Freely available on-line resourcehttps://www.lhasalimited.org/products/lhasa-carcinogenicity-
database.htm
Long term carc studies
• 6,529 | 1529
Expert Review Using in silico Models
Database
searching
ICH
M7
Classification
Reporting
Purge mitigation
Control
Impurity
identification
Test
Storing
Expert in silico
prediction
Statistical in
silico prediction
Expert
assessment
Expert Review Using in silico Models
• Guidance documentation is readily available
• Establishing best practise in the application of expert review of
mutagenicity under ICH M7.
• Barber… Regul. Toxicol. Pharmacol. 2015, 73, 367
• Principles and procedures for implementation of ICH M7
recommended (Q)SAR analyses.
• Amberg… Regul. Toxicol. Pharmacol. 2016, 77, 13
• Lhasa’s website contains publications, presentations, examples...
• https://www.lhasalimited.org/ich-m7.htm
• Regulator’s website
• https://www20.anvisa.gov.br/coifaeng/calculos.html
Expert Review Using in silico Models
• “The absence of structural alerts from two complementary
(Q)SAR methodologies (expert rule-based and statistical) is
sufficient to conclude that the impurity is of no mutagenic
concern”
• “If warranted, the outcome of any computer system-based
analysis can be reviewed with the use of expert knowledge in
order to provide additional supportive evidence on relevance of
any positive, negative, conflicting or inconclusive prediction
and provide a rationale to support the final conclusion”
http://www.ich.org/products/guidelines/multidisciplinary/article/multidisciplinary-guidelines.html
• A model should meet the OECD principles• a defined endpoint;
• an unambiguous algorithm;
• a defined domain of applicability;
• appropriate measures of goodness-of-fit, robustness and predictivity;
• a mechanistic interpretation, if possible.”
• A model must also support expert review• Make predictions and have good accuracy for your chemical space
• Provide some meaningful measure of confidence
• Be regularly updated with new knowledge
• Ideally be well understood by regulatory authorities
• Be transparent and highlight areas of uncertainty
• Make predictions that you understand, support or can overturn
Expert Review – Model Selection
Distinguishing between expert and statistical systems for application under ICH M7.
Barber… Regul. Toxicol. Pharmacol. 2017, 84, 124
Derek – an Expert System
1 Defined endpoint
2 Unambiguous algorithm
3 Defined applicability domain
4 Performance measure
5 mechanistic interpretation
Derek – Positive Predictions
CFSANn=1535
100 98
80
49
0
25
50
75
100
Certain Probable Plausible Equivocal
Accuracy of predictions
Assessing confidence in predictions made by knowledge-based systems. Judson… Toxicol. Res., 2013, 2, 70
Distinguishing between expert and statistical systems for application under ICH M7. Barber…Regul. Toxicol. Pharmacol. 2017, 84, 124
• Highlights the alert and gives a level of confidence
Derek – Negative Predictions
• “I have no concerns, there is nothing new for me here..
..I’ve seen these all features before in negative compounds”
• “There is nothing new for me here..
..but there is a feature that was in a compound I falsely predicted
as negative”
• “There is a feature I haven’t seen before
..you might want to check that out!”
If you asked a human expert the absence of a positive prediction..
More confidentAmount of
expert review
Derek – Negative Predictions
?
Is it alerting?
Contains unknown
features?
Contains features in
known false negatives?
InactiveInactive with
misclassified features
Inactive with
unclassified features
N
N N YY
It’s difficult, but important, to make negative predictions. Williams… Reg. Tox. and Pharmacol. 2016, 76, 79
YPositive prediction
Derek – Negative (unclassified feature)
• Highlights fragments in contexts not seen in public data
• May be present in confidential data
• Often driven by unusual ring systems
• Expert review recommended
Derek – Negative (misclassified feature)
• Highlights fragments seen in false positive predictions
• This is still a negative prediction
• It can be triggered for many reasons
• Inconsistent data• Will flag if only seen once (and many negative tests are seen)
• An innocent spectator
• A missing alert
• Expert review recommended
• Search for additional supporting compounds containing fragment
• Review conflicting results• Higher hurdle to dismiss positive results but you can
• Non-standard assay, impure sample…
Performance against 3 proprietary data sets – frequency (negative predicitivity)
Derek – Negative Predictions
?
Is it alerting?
Contains unknown
features?
Contains features in
known false negatives?
InactiveInactive with
misclassified features
Inactive with
unclassified features
N
N N YY
It’s difficult, but important, to make negative predictions. Williams… Reg. Tox. and Pharmacol. 2016, 76, 79
YPositive prediction
86-90%
(87-94%)
7-9%
(86-93%)
3-5%
(86-95%)
Using Predictions from Expert Systems
• Positive and negative predictions are not sufficient
• OCED principles are not sufficient
• Expert review also requires:
• Levels of confidence
• Where to focus attention
• Supporting examples
• Peer-reviewed expert commentaries
• Supporting literature references
Positive
plausible
probable
certain
Equivocal
Negative
..no mis/unclassified
..with misclassified
..with unclassified
Expert Review Using in silico Models
Likely to conclude positiveVery strong evidence would be needed to overturn both
predictions
UncertainLikely to conclude positive without strong evidence to
overturn a positive prediction
Likely to conclude positiveLack of a second prediction
suggests insufficient evidence to draw any other
conclusion
In silicoprediction 1
In silicoprediction 2
Positive
Positive
Positive
O.O.D. or equivocal
Positive
Negative
Negative
O.O.D. or equivocal
Negative
Negative
UncertainConservatively could assign as positive.
May conclude negative with strong evidence showing feature driving a ‘no prediction’ is
present in the same context in known negative examples (without deactivating features)
Likely to conclude negativeExpert review should support this conclusion – e.g. by assessing any
concerning features (misclassified, unclassified, potentially reactive...)
O.O.D. = out of domain
Establishing best practise in the application of expert review of mutagenicity under ICH M7 Reg. Tox. and Pharmacol. 2015, 73, 367
Expert Review
Harder
Easiest
Expert
Review
Low confidence
Mis- or unclassified
Poor coverage of important
fragments
Own knowledge /
proprietary data disagrees
Close relevant examples
agree
Good coverage of
fragments
Both predictions agree
Fits own knowledge /
proprietary data
Conflicting predictions
Equivocal or out of domain
Expert Review – Conflicting Results
O.O.D. 2 0 3
+ 10 0 17
Equiv 8 0 6
- 38 2 12
- Equiv +
• Lhasa data sharing consortium group (n=777; 32% positive)
Frequency of Outcomes (%)
O.O.D. 29 - 62
+ 30 - 70
Equiv 23 - 51
- 8 15 34
- Equiv +
Probability of being positive (%)
Managing out of domains
https://www.lhasalimited.org/publications/dealing-with-out-of-domain-qsar-predictions-for-ich-m7-a-
regulatory-and-industrial-perspective/4476 (Kruhlak , Sept 2017)
Slides reproduced with permission of the author
Managing out of domains
https://www.lhasalimited.org/publications/dealing-with-out-of-domain-qsar-predictions-for-ich-m7-a-
regulatory-and-industrial-perspective/4476 (Kruhlak , Sept 2017)
Managing out of domains
https://www.lhasalimited.org/publications/dealing-with-out-of-domain-qsar-predictions-for-ich-m7-a-
regulatory-and-industrial-perspective/4476 (Kruhlak , Sept 2017)
Using Strain Profiles in Expert Review
• Ames test uses different strains with different sensitivities
• A negative result not tested in 5 strains may miss something
• Strain-specific models perform badly• Too many data gaps
• Too many different strains used
• You should know if a negative result for an analogue was
tested in the most appropriate strain• Particularly if this is a key piece of data for expert review
• Hypothesis-level strain information• Which strain is most sensitive for chemicals in this class?
• How comprehensive is the data for supporting negative ex’s?
• Compound-level strain information• Which strains was this compound tested in?
• Is it negative because it is missing a key strain?
Using Strain Profiles in Expert Review
Agenda
• The ICH M7 Guidelines
• The Process
• Negative Predictions
• Managing out of domains
• Using Strain Profiles
• Some Examples
Example 1
• Conflicting predictions from the expert and statistical
systems
Example 1
• Well supported alert• No reason to immediately dismiss a positive prediction
Example 1
• Positive hypothesis for the epoxide was overwhelmed by others• Close examples are also positive
Example 1
• Model’s closest compound was not tested in
5-strains or in the presence of S9
Example 1
Mutagenic
Equivocal
Pro-mutagenic
arguments
Non-mutagenic
arguments
Non-mutagenic
Key hypothesis (epoxide) is positive
and this is supported by many
relevant examples that give good
coverage of the structure Key negative compound was
only tested in 2 strains w/out
metabolic activation
Example 2
• One equivocal and one weakly positive prediction..• You could ‘conservatively’ treat as a positive prediction
but…
Example 2
• Derek specifically notes that positive results are not
driven by the acid chloride but by the solvent• Literature references are included…
Example 2
• Sarah makes a weakly positive prediction• Insufficient data for own hypothesis – not all examples relevant
• Links to the source data allows review of solvents
acetone DMSOTHF
DMSO N/A
Example 2
Mutagenic
Equivocal
Pro-mutagenic
arguments
Non-mutagenic
arguments
Non-mutagenic
Half the relevant
examples are inactive
supporting DX
comments & publ’n
Alert tells us that
activity is driven by
reaction with solvent
and not by ROCl
Example 3
• Conflicting predictions from the expert and statistical
systems
Example 3
• Clear negative prediction
..with no mis- or unclassified features
Example 3
Positive 1/6 times in TA100
Positive 1/1 in D3052
(1984)
1 positive reference
(no experimental details
or strain information..)
• Positive prediction overturned by expert• Other reasons for activity or weak positive evidence
Example 3
Mutagenic
Equivocal
Pro-mutagenic
arguments
Non-mutagenic
arguments
Non-mutagenic
Positive examples have
more plausible reasons
for activity or have weak
evidence
Example 4
• Conflicting predictions from the expert and statistical
systems
Example 4
• Specifically includes sulphinates and includes references
• Describes mechanism
Example 4
• No specific hypothesis for this functional group
• Insufficient relevant examples to accept the prediction
Example 4
Mutagenic
Equivocal
Pro-mutagenic
arguments
Non-mutagenic
arguments
Non-mutagenic
Supporting
examples not
that relevantNo hypothesis
of alkyl
sulphinate
Summary of expert review
• Good software will help you reach a conclusion• You need to be able to convince yourself!
Conflicted predictions
• use of strain information
Two weak positives were overturned
• expert system commentary + key ref’s
• links to original underlying data
Conflicted predictions
• ability to look for other causes of
activity in statistical system
• review of underlying data (of
remaining positives)
Conflicted predictions
• expert system commentary
• assessment of supporting
compounds (statistical system)
Tools to support ICH M7 assessments
Database
searching
ICH
M7
Expert in silico
prediction
Statistical in
silico prediction
Expert
assessment
Classification
Reporting
Purge mitigation
Control
Impurity
identification
Test
Storing
Head office in Leeds, UKNot-for-profit
Educational charity
A membership organisation
Data & knowledge sharing
Honest broker
Sponsor
PhDs
Teaching
lecturesWork
experienceAcademic
Agrochem
Biotech
Chemical
CROCosmetic
Generics
Personal Products
Pharma
Government
Tobacco
MIP-DILI
Regulators
(members & collaborators)
Proprietary data mining
Predictive software
(expert & machine learnt)
Purge
Toxicity Metabolism
Degradation
Undergrad
projects
Publications &
presentations
Thank you