Monitoring the Adequacy of Implementation and Adherence to ... · Implementation and Adherence to...

Monitoring the Adequacy of Implementation and Adherence to International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) GuidelinesAUTHORITY REPORT

April 2019

EC, Europe (centralised procedure with EMA)

SlideID:317SlideID:317v5

Study Background 2

The aim of this study is to monitor the adequacy of implementation and adherence to ICH Guidelines by regulatory authorities

ICH selected the Centre for Innovation in Regulatory Science (CIRS) to collaborate on the development and the conduct this study, including the design of the study questionnaire and the online data collection tool (DCT).

The questionnaire was completed by companies (assessing all the participating authorities) and by authorities (assessing themselves only) in order to undertake a gap analysis.

The authority specific results from this survey are now being shared individually with each participating authority, demonstrating the answers from the authority compared to the aggregated industry responses.

This report is specific to your authority only and does not contain data from other authorities. Please note that these slides are being treated as confidential and have not been shared outside of your organisation at this point.

Study scopeThe following guidelines were assessed:

• Tier 1: o Q1 – Stability (NOTE: this guideline was intended to be considered as a whole, but as it is made up of sub

parts, these should be taken into consideration when answering)o Q7 – Good Manufacturing Practice Guide for Active Pharmaceutical Ingredients o E6(R2) – Good Clinical Practice (GCP)

• Tier 2o E2A – Clinical Safety Data Management: Definitions and Standards for Expedited Reportingo E2B(R3) – Clinical Safety Data Management: Data Elements for Transmission of Individual Case Safety

Reportso E2D – Post-approval Safety Data Management: Definitions and Standards for Expedited Reportingo M1 – Medical Dictionary for Regulatory Activities Terminology (MedDRA)o M4 – Common Technical Document (CTD)

• Tier 3o M3(R2) - Guidance on Nonclinical Safety Studies for the Conduct of Human Clinical Trials and Marketing

Authorization for Pharmaceuticalso M8 - Electronic Common Technical Document (eCTD)o E17 - General principles for planning and design of Multi-Regional Clinical Trials

3

The following authorities were invited to participate in the study• All ICH Members • Observers (voluntary basis)

Company responses were obtained from across PhRMA, EFPIA, JPMA and BIO companies.

Four short questions were used to assess the implementation/ adherence to each of the guidelines within each authority, based on company and authority perceptions.

Study scope 4

Data provision by participant authorities

Data validation, and analysis

Presentation at ICH Meeting

Study plan for 2019

Activity Time Deliverable responsibility Data provision until 22 March All Participants Data Collection tool closes for all participants except the project coordinator from each authority

22 March CIRS

Data validation by project coordinators from each authority 22 – 29 March Project Coordinator

Data Collection tool closes to all participants 29 March CIRS

Data cleaning and analysis – reports shared with authorities April - May Participants/CIRS/ICH

Presentation at the ICH Meeting in Amsterdam (NOTE: the format of the results to be presented is under discussion)

1 - 6 June CIRS/ICH

5

Study Results - MethodPlease note that to facilitate the interpretation of the results, for each analysis, the report includes the specific question

from the survey as the first slide, followed by the results for that question based on authority and company responses as the second slide.

For example for question 1 (implementations status), the left hand side (first slide) lists the question, whereas the right hand slide (second slide) shows the analyses.

6

Study Results - MethodPlease also note that where the company and authority responses are showed on one slide, the top graph

represents responses from your authority, whereas the bottom graph shows the aggregated industry responses in relation to your own authority.

Please note that a blank in the graph indicates no response, due to the fact that that question was not applicable for the specific guideline based on how the previous question was answered e.g. blanks in Question 1.2 (modifications to guideline) would be because the answer to Question 1 (implementation status) was ‘not implemented’, ‘in the process of implementation’ or ‘not applicable’ so question 1.2 could not be answered in this case (unless question 1 ‘implemented’ response was selected).

Similarly, for the industry response a low response number (‘n’ on x-axis, which indicates number of companies responding) could be because of the above reasons, as well as the fact that a response was missing (i.e. answer not provided even if the question is applicable). If n<3, responses were excluded.

Your authority's response

Aggregated industry response for your authority (where the number next to each guideline on x-axis represents number of companies responding)

Blank (no response)

7

Company-specific questions 1i, 1ii and 1a

Before answering questions on implementation and adherence, companies were also asked on their experience in regard to a Guideline for the selected authority

Question 1aWhat is your company’s experience in regard to this Guideline for the selected authority? Select one (most recent and relevant). • From a past regulatory submission 1.1.a. If yes, give a year of the most recent submission Text box ‘yyyy’ format• Through ongoing regulatory intelligence input/local affiliate opinion• Being used to prepare for an upcoming submission• Through interactions and exchanges with the authority• No experience

If ‘no experience’ company does not answer the questionnaire but can provide additional comments (question 3).

NOTE: responses in the subsequent questions were intended to relate to company’s general experience, and not only to the single submission/experience selected. Any additional comments and/or divergences can be captured through comment boxes, for example in question 3.

Before answering the questionnaire, companies were asked the following two general questions

Question 1i: Please specify your company type, which refers to what countries/regions the company is submitting drug applications to:� Local country only� Single region� Multi-regional� Global

Question 1ii: Please specify your company’s focus for drug development:� Innovative medicines� Generic medicines� Both

8

Company-specific questions 1i, 1ii and 1a

Local country only

Multi-regional

Global

Innovative medicines

Generic medicines

Both

From a past regulatory submission

Through ongoing regulatory intelligenceinput/local affiliate opinionBeing used to prepare for an upcomingsubmissionThrough interactions and exchanges with theauthorityNo experience

Blank (no response)

1i) Company type (countries/regions the

company is submitting drug applications to)

1a) Company experience for your authority (across all 11 ICH Guidelines

surveyed) 1ii) Company focus for drug development

For those companies that had experience:

SlideID:286

27 companies provided responses for your authority (for at least one guideline)

Blank responses excludedBlank responses excluded

9

Company experience – Question 1a

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Q1 Q7 E6 E2A E2B E2D M1 M4 M3 M8 E17

Blank (no response)No experienceThrough interactions and exchanges with the authorityBeing used to prepare for an upcoming submissionThrough ongoing regulatory intelligence input/local affiliate opinionFrom a past regulatory submission

Company experience for your authority (1st pie chart from previous slide split by the 11 ICH Guidelines surveyed)

SlideID:285

Tier 1 Tier 2 Tier 3

10

Implementation status

Adequacy of implementation

(based on modifications)

Adherence to the guideline (based

on practice)

Data provision stepsThe questionnaire has been designed to assess each guideline in terms of the following three sequential concepts:

11

Implementations status – Question 1Question 1 (for Companies and Authorities)

1.1 Please provide your organisation’s view on the implementation status for the selected Guideline. Select one.� Not implemented - The process for the implementation of an ICH Guideline has not yet started. (a) No guideline

exists or b) national/ regional guideline deviating from ICH Guideline or national/regional guideline exists but the process for replacement or amendments for alignment with the ICH Guideline has not started yet.)

� In the process of implementation - The process for the implementation of the ICH Guideline has started and has reached a specified milestone. (The process can have different starting points: a) no national/regional guideline exists; the ICH Guideline defines new requirements and b) a national/regional guideline is in the process of development or c) a national/regional guideline exists and is replaced by or is amended to be in line with the ICH Guideline. Generic processes for a) non-electronic and b) electronic guidelines will be defined outlining the milestones that should be followed.)

� Implemented - The process of implementation is completed. (This term refers to the self-declaration of the regulator regarding the conclusion of the implementation process. Usually, the regulator publishes the final guideline. This could relate to both adequate or inadequate implementation of the Guideline. The adequacy of implementation will be queried in the next question.)

� Not Applicable - The implementation of a specific ICH Guideline is not applicable in this country/region. An appropriate justification is provided. (Example: A country may not have its own Pharmacopeia but references internationally recognized Pharmacopoeias. Hence, the ICH Q4B Guideline is not applicable (and does not need to be implemented).)

If ‘Not applicable’ selected in Question 1.1, respondent redirected to Question 1.1.1, and then Question 3. If ‘not implemented’ or ‘in the process of implementation’ selected in Question 1.1, respondent redirected to Question 3. If ‘implemented’, respondent asked to answer Question 1.2.

1.1.1 If ‘not applicable’, please comment (Free text comment);

12

Implementations status – Question 1

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Q1 (23) Q7 (23) E6 (21) E2A (20) E2B (20) E2D (20) M1 (20) M4 (20) M3 (18) M8 (20) E17 (17)

Prop

ortio

n of

resp

onse

s

Not implemented In the process of implementation Implemented Implementation not applicable

Your Authority's response:

Aggregated Company response for your authority:

Q1 Q7 E6 E2A E2B E2D M1 M4 M3 M8 E17

(n) = number of companies that provided a response SlideID:287If n<3, responses were excluded.


13

Modification to guideline – Question 1.21.2 Please indicate which statement best characterises your organisation’s view of the implementation of the ICH Guideline? Select one.� An unmodified ICH guideline has been implemented, where all relevant elements,

concepts and principles of the ICH Guideline are followed. This is done preferably by referring to/implementing the original ICH Guideline text and/or translating the original guideline text.

� Some modifications have been made to the original ICH guideline either by adding or altering certain elements, concepts or principles

If ‘An unmodified ICH…’ to Question 1.2, respondent redirected to Question 1.3. If ‘Some modification’ to Question 1.2, respondent redirected to 1.2.1

14

Modification to guideline – Question 1.2

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Q1 (23) Q7 (23) E6 (20) E2A (20) E2B (18) E2D (19) M1 (20) M4 (18) M3 (18) M8 (19) E17 (14)

Prop

ortio

n of

resp

onse

s

An unmodified ICH guideline has been implemented Some modifications have been made to the original ICH Guideline


Q1 Q7 E6 E2A E2B E2D M1 M4 M3 M8 E17




15

Justification of modifications – Question 1.2.1

1.2.1 Are these modifications objectively justified by the authority? (This may include in justified cases implementation of the guideline in a way that may incorporate additional information beyond those defined in the ICH Guideline in circumstances when the Guideline is too high-level and does not provide sufficient guidance. Additional information to the ICH guideline should only be included in order to provide clarity and facilitate implementation by industry, but should not increase regulatory burden.)� Yes � No

If ‘No’ to Question 1.2.1, respondent redirected to Question 2. If ‘Yes’ to Question 1.2.1, respondent asked to answer Question 1.3 (i.e. only if guideline is ‘adequately’ implemented will the respondent answer the question on adherence)

16

Justification of modifications – Question 1.2.1

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Q1 (4) Q7 (4) E6 (1) E2A (3) E2B (5) E2D (2) M1 (1) M4 (5) M3 (1) M8 (6) E17 (0)

Prop

ortio

n of

resp

onse

s

Yes - modifications justified No - modifications not justified



Q1 Q7 E6 E2A E2B E2D M1 M4 M3 M8 E17



17

If the guideline has been not adequately implemented (modifications not justified), the rationale is asked for

Rationale for inadequate implementation – Question 2

Question 2 (for Companies and Authorities)2.1. Please provide the rationale for your selection by specifying the appropriate root cause(s) listed below. Select all that apply.

If ‘not adequately implemented’ is specified in Question 1.2, the following will be displayed:

� Incorporates additional requirements beyond those defined in the ICH Guideline without objective justification in cases where clear guidance is provided

� Does not include all relevant elements, concepts and principles of the ICH Guideline and does not provide any objective justification for omitting some requirements in the Guideline

� Requires application of the Guideline for a smaller range of products than outlined in the ICH Guideline

� Other

If ‘other’, please specify

(free text comment)

18

Rationale for inadequate implementation – Question 2

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%Pr

opor

tion

of re

spon

ses

Incorporates additional requirements Omits some requirements Smaller range of products Other

Your Authority response and Aggregated Company response for your authority:

(n) = number of companies that provided a response; multiple responses per company are possible

SlideID:302If n<3, responses were excluded.


E2A E2B E2D M1 M4Q1 Q7 E6 M3 M8 E17

19

If the guideline has been adequately implemented (unmodified or modified with justification), adherence is measured:

Adherence status – Question 1.3

1.3 Please provide your organisation’s view on the adherence status for the selected Guideline? Select one.� In its practice, the regulatory authority consistently adheres to (applies) all identified relevant

elements, concepts and principles of the ICH Guideline over time (Once an ICH Guideline has been (adequately) implemented by a regulatory authority, experience is gathered on how the regulator applies the guideline in practice. Adherence leads to a stable regulatory environment and to increased sustainability. Adherence may be assessed in regular intervals.)

� Even if the guideline has been adequately implemented, it is not being applied and adhered to in practice (The regulatory authority does not in practice require industry to adhere to the guideline or does not follow the guideline when assessing the applications; e.g. is in its practice adding requirements beyond what is provided in the (implemented) ICH guideline.)

� The regulatory authority has only recently implemented the Guideline therefore it is too early to assess the adherence to the guideline due to limited experience

If ‘Even if the guideline has been adequately implemented, it is not being applied and adhered (…)’ to Question 1.3, respondent asked to answer Question 2. Otherwise respondents redirected to Question 3.

20

Adherence status – Question 1.3

0%10%20%30%40%50%60%70%80%90%

100%

Q1 (21) Q7 (21) E6 (19) E2A (18) E2B (18) E2D (19) M1 (20) M4 (17) M3 (18) M8 (19) E17 (14)

Prop

ortio

n of

resp

onse

s

Adherence No adherence Too early to assess adherence



Q1 Q7 E6 E2A E2B E2D M1 M4 M3 M8 E17



21

Rationale for lack of adherence – Question 2If the guideline has been adequately implemented but is not adhered to,

additional rationale is asked for:

Question 2 (for Companies and Authorities)2.1. Please provide the rationale for your selection by specifying the appropriate root cause(s) listed below. Select all that apply.If ‘not adequately implemented’ is specified in Question 1.2, the following will be displayed:� Incorporates additional requirements beyond those defined in the ICH Guideline without objective justification in

cases where clear guidance is provided � Does not include all relevant elements, concepts and principles of the ICH Guideline and does not provide any objective

justification for omitting some requirements in the Guideline� Requires application of the Guideline for a smaller range of products than outlined in the ICH Guideline� Other

If ‘other’, please specify (free text comment)

If ‘lack of adherence’ is specified in Question 1.3, the above will be displayed, as well as the below (i.e. all 9 options)� Other local guidelines conflict with the ICH Guideline and prevent full adherence to the Guideline� Agency process or capacity issues (agency does not have an internal process and/or resources to implement the

Guideline)� There is a general lack of understanding of the elements of the ICH Guideline by technical reviewers (the underlying

regulatory science is not understood)� Inconsistent application of the Guideline; e.g. adherence and interpretation varies by submission/review

division/reviewer� The agency does not in practice require industry to adhere to the Guideline

22

Rationale for lack of adherence – Question 2

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%Pr

opor

tion

of re

spon

ses

Incorporates additional requirements Omits some requirements Smaller range of products

Local guidelines Agency process/capacity Lack of understanding of the Guideline

Inconsistent application of the Guideline Agency does not require adherence Other

Authority and Aggregated Company response for your authority:

(n) = number of companies that provided a response; multiple responses per company are possible SlideID:303If n<3, responses were excluded.


E2A E2B E2D M1 M4Q1 Q7 E6 M3 M8 E17

23

Analysis of free text responsesIf responses from Question 1 corresponded to lack of adequate implementation or adherence, in addition to multiple choice Question 2, the organisations were asked to provide additional evidence in Question 2.2:

Organisations could also provide optional comments in Question 3

2.2 Please provide specific evidence or examples that substantiate your root cause choice(s), including details regarding any unjustified modifications made to the Guideline (OPTIONAL)

(free text comment)

Question 3 (for Companies and Authorities- OPTIONAL)Please provide any other comments you would like to make in regard to the implementation and adherence of the Guideline, including any objectively justified modifications made to the Guideline by the authority and well as future plans regarding the status of the guideline in the authority with key milestones and timelines if available.

(free text comment)

24

Analysis of free text responses• Once the quantitative slides have been analysed and shared, CIRS will carry

out a qualitative analysis of free text comments, particularly evidence from authorities and companies to support their responses

• This will focus on where there is a gap between company and agency response and/or perception that the implementation or adherence are lacking

• Free text comments will also be analysed to identify common themes across guidelines that identify need for further training

SlideID:318

25

Contact information

Prepared by the Centre for Innovation in Regulatory Science, in collaboration with the ICH and the ICH Implementation Subcommittee.

ContactDr Magda Bujar, CIRS Manager Strategic Development [email protected] Neil McAuslane, CIRS Scientific Director [email protected]

CIRS - The Centre for Innovation in Regulatory Science Limited - is a neutral, independently managed UK based subsidiary company, forming part of Clarivate Analytics (UK) Limited. CIRS’ mission is to maintain a leadership role in identifying and applying scientific principles for the purpose of advancing regulatory and HTA policies and processes. CIRS provides an international forum for industry, regulators, HTA and other healthcare stakeholders to meet, debate and develop regulatory and reimbursement policy through the innovative application of regulatory science and to facilitate access to medical products through these activities. This is CIRS’ purpose. CIRS is operated solely for the promotion of its purpose. The organization has its own dedicated management and advisory boards, and its funding is derived from membership dues, related activities, special projects and grants.

To find out more about CIRS please visit our website: http://www.cirsci.org/

26

mailto:[email protected]

mailto:[email protected]

Appendix – complete questionnaire

SlideID:315

27

Appendix – complete questionnaire

SlideID:316

28

Monitoring the Adequacy of Implementation and Adherence to ... · Implementation and Adherence to...

Documents

Transcript of Monitoring the Adequacy of Implementation and Adherence to ... · Implementation and Adherence to...