WHO GLOBAL BENCHMARKING TOOL (GBT) FOR …...1 2 WHO GLOBAL BENCHMARKING TOOL (GBT) FOR 3 EVALUATION...

1

WHO GLOBAL BENCHMARKING TOOL (GBT) FOR 2

EVALUATION OF NATIONAL REGULATORY SYSTEM OF 3

MEDICAL PRODUCTS 4

FACT SHEETS FOR REGISTRATION AND MARKETING AUTHORIZATION 5

(DRAFT REVISION VI VERSION 1) 6

(DECEMBER 2017) 7

DRAFT FOR COMMENTS 8

Should you have any comments on the attached text, please send these to: Country Regulatory Support 9 (CRS) Group , Regulatory Systems Strengthening (RSS), World Health Organization, 1211 Geneva 10 27,Switzerland; Alireza Khadem ([email protected]); fax: (+41 22) 791 4856; and to Laura 11 Brown ([email protected]) by 28 February 2018. Working documents are sent out electronically and they 12 will also be placed on the Medicines website for comment. 13

14 © World Health Organization 2017 15

All rights reserved. 16

This draft is intended for a restricted audience only, i.e. the individuals and organizations having received this draft. 17 The draft may not be reviewed, abstracted, quoted, reproduced, transmitted, distributed, translated or adapted, in part 18 or in whole, in any form or by any means outside these individuals and organizations (including the organizations' 19 concerned staff and member organizations) without the permission of the World Health Organization. The draft 20 should not be displayed on any website. 21

Please send any request for permission to: 22

Dr Alireza Khadem Broojerdi, Scientist, Country Regulatory Support, Regulatory Systems Strengthening, 23 Department of Essential Medicines and Health Products, World Health Organization, CH-1211 Geneva 27, 24 Switzerland. Fax: (+41 22) 791 4856; email: [email protected]. The designations employed and the 25 presentation of the material in this draft do not imply the expression of any opinion whatsoever on the part of the 26 World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or 27 concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent approximate border lines 28 for which there may not yet be full agreement. 29

The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or 30 recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. 31 Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters. 32

All reasonable precautions have been taken by the World Health Organization to verify the information contained in 33 this draft. However, the printed material is being distributed without warranty of any kind, either expressed or 34 implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall the 35 World Health Organization be liable for damages arising from its use. 36

This draft does not necessarily represent the decisions or the stated policy of the World Health Organization. 37

WHO global benchmarking tool rev. VI ver. 1, MA

02- REGISTRATION AND MARKETING AUTHORIZATION (MA) 1 2

The issuance of marketing authorization (also referred as product licensing or registration) is critical to 3 any National Regulatory Authority (NRA). It is a procedure of release a medical product for marketing 4 after it has undergone a process of evaluation in order to determine its safety, efficacy and quality and the 5 appropriateness of the product information. The objective of this regulatory function is to provide a 6 system which ensures that only medical products which have been duly authorized by the NRA are 7 allowed to be manufactured, imported, distributed or sold/supplied to end-users. The process of 8 assessment for marketing authorization includes the review of data on quality, safety and efficacy 9 submitted by the applicant, applying the same standards to imported and locally manufactured medical 10 products. Nevertheless, evaluation of the complex toxicological and clinical data from new chemical 11 entities requires resources and experience not always available in the NRA; therefore countries with 12 limited resources may wish to give priority to well-established products. In relation to the evaluation of 13 well-established products the NRA may prepare its own report, rely on revaluation reports prepared by 14 other national authorities or rely on decisions made by other NRA or use a combination of these 15 approaches. GMP inspection or certification should be part of the marketing authorization requirements. 16 A legal provision should exist to mandate the NRA to grant marketing authorization with limited validity 17 and also request that marketing authorization should be renewed. Mechanisms should exist to ensure that 18 information on marketing authorization applications, including authorized, suspended, rejected and 19 completed applications are published to promote transparency and information sharing amongst 20 stakeholders. Regular publication of marketing authorization decisions is helpful to procurement and 21 distribution networks and to other NRAs. Mechanisms and /or procedures should be in place to monitor 22 and ensure that all activities within the marketing authorization function are checked to reduce errors and 23 to reasonably ensure that the processes are consistent in order to generate an assured output. This would 24 lead to consistency in the regulatory performance of the marketing authorization function as well as 25 reliable outputs. 26

Indicator: MA01 Legal provisions, regulations and guidelines required 27

to define regulatory framework of registration and/or marketing 28

authorization. 29

Rationale: 30

The objective of this indicator is to assure that legal provisions, regulations and guidelines, which form 31 the backbone of the regulatory framework, ensure that medical products for human use in a particular 32 jurisdiction have been assessed by a NRA, and found of adequate quality, safety and 33 efficacy/effectiveness before registered or granted marketing authorization. The legal provisions empower 34 the NRA with the legal mandate to register and/ or grant marketing authorization but also to punish those 35 who market medical products without the proper health registration or marketing authorization. 36

Category: Legal provisions, regulations and guidelines 37

Sub-Indicator: MA01.01: There are legal provisions to hold a registration and/or 38

marketing authorization (MA) before placing the product on the market. 39

Scope: All 40

Maturity: 1 41


Requirement: 1

Legal provisions for medical product registration /marketing authorization 2

Rationale: 3

The objective of this sub-indicator is to assure legal provisions require that before a medical product is 4 marketed /or placed on the market, the product should have been registered /or granted marketing 5 authorization by the NRA. 6

Description: 7

The assessor should verify that legal provisions requiring registration and /or marketing authorization 8 before placing a medical product on the market should be in existence, enacted and implemented. The 9 legal provisions should clearly mandate that all medical products require registration /marketing 10 authorization before they are placed on the market. It should provide the list of medical products’ classes 11 (e.g. drugs, vaccines, medical devices, among others) that require registration / marketing authorization 12 before they are marketed/ sold. The legal provisions should present the applicable fines/ charges/penalty/ 13 sanctions in the event of non-compliance. The assessor should note that some medical products may be 14 imported and marketed before registration and /or marketing authorization in special circumstances such 15 as during an emergency or public health concern. The NRA should have systems in place to do active 16 monitoring of such medical products. The assessor may refer to the legal provisions on emergency 17 situations before applying the scoring. 18

Evidence to review: 19

The assessor should request for and review: 20

1. Legal provisions that mandate that all medical products require registration /marketing 21 authorization before they are placed on the market, as well as the applicable fines/ 22 charges/penalty/sanctions in the event of non-compliance. 23

2. Whether the legal provisions have been published and implemented, and are applicable to all 24 medical products. 25

3. Evidence of registration or marketing authorization, this may be the registration or marketing 26 authorization number or the registration or marketing authorization certificate. 27

Framework: 28

Structure/Foundation/Input 29

Rating Scale: 30

N: There is no evidence of legal provisions to hold a registration and/or marketing authorization (MA) 31 before placing the product on the market. 32

OI: The NRA has recently drafted legal provisions to hold a registration and/or marketing authorization 33 (MA) but it has not yet been followed. 34

PI: The NRA has legal provisions to hold a registration and/or marketing authorization (MA) and has 35 been applying it for less than two years. 36

I: The NRA has legal provisions to hold a registration and/or marketing authorization (MA) before 37 placing the product on the market and can demonstrate a proven track record of applying them. 38


Limitations and remarks: 1

Related indicators: 2

References: 3

1. 5- WHO TRS 790, Annex 6, 1990.Chapter 1.4 and 1.8 (b) 4

2. 3- WHO TRS 858, Annex 1, 1995 5

Sub-Indicator: MA01.02: There are legal provisions to withhold, suspend, and/or 6

withdraw or cancel a MA in case there is/are concerns on quality, safety or efficacy 7

issues. 8

Scope: All 9

Maturity: 1 10

Requirement: 11

Legal provisions on withholding, suspension and /or withdrawal or cancellation in MA 12

Rationale: 13

The objective of this sub-indicator is to assure that legal provisions exist mandating the NRA to 14 withhold, suspend and /or withdraw or cancel registration /marketing authorization in the event of 15 adverse findings related to the quality, safety and efficacy of medical products. The NRA is empowered 16 by legal provisions to withhold, suspend and /or withdraw or cancel a registration /marketing 17 authorization if it becomes convinced that any requirement that allowed the products to be registered is no 18 longer being met. This legal provision should also sanction those who market medical products without 19 the proper registration / marketing authorization. 20

Description: 21

The assessor should verify that legal provisions with respect to withholding, suspending and /or 22 withdrawing or cancelling a registration and /or marketing authorization in the event of uncovering 23 quality, safety or efficacy issues should be in existence, enacted and implemented. The legal provisions 24 should provide requirements on when and how to withhold, suspend and /or withdraw or cancel 25 registration /marketing authorization. 26



1. Legal provisions that mandate the NRA to withhold, suspend and/or withdraw or cancel a 29 registration / marketing authorization in the event of adverse findings related to the quality, safety and 30 efficacy of the medical products. 31

2. Regulations and /or guidelines presenting details on when and how to withhold, suspend and /or 32 withdraw or cancel registration /marketing authorization, 33

3. Whether the legal provisions have been implemented, and if possible evidence of implementation. 34

4. Whether stakeholders including applicants, wholesalers, and retailers among others, are aware of 35 the legal provisions. 36


Framework: 1


Rating Scale: 3

NI: There is no evidence of legal provisions to withhold, suspend, and/or withdraw or cancel a MA in 4 case there is/are concerns on quality, safety or efficacy issues. 5

OI: The NRA has recently drafted the legal provisions but they have not yet been followed. 6

PI: The NRA has legal provisions and has been applying them for less than two years. 7

I: The NRA has legal provisions to withhold, suspend, and/or withdraw or cancel a MA in case there 8 is/are concerns on quality, safety or efficacy issues and can demonstrate proven track record of applying 9 them. 10



References: 13

1. 2- WHO TRS 822, Annex 2, 1992 14

2. 5- WHO TRS 790, Annex 6, 1990. Chapter 1.4 and 1.8 (b) 15

Sub-Indicator: MA01.03: There are legal provisions that require demonstration of the 16

product quality, safety and efficacy prior to registration and/or marketing authorization 17

Scope: All 18

Maturity: 1 19

Requirement: 20

Quality, safety and efficacy prerequisites for registration and/or marketing authorization 21

Rationale: 22

The objective of this sub-indicator is to assure that legal provisions exist mandating the NRA to request 23 for a demonstration - technically and scientifically - that the medical product meets all quality, safety and 24 efficacy regulatory requirements prior to registration /marketing authorization. The NRA should require 25 that applicants comply fully with the legal requirements for product registration / marketing authorization. 26

Description: 27

The assessor should verify that legal provisions on demonstration of acceptable quality, safety and 28 efficacy profiles being a pre-requisite to registration / marketing authorization should be in existence, 29 enacted and implemented. The legal provisions should empower the NRA to request satisfactory 30 information on the quality, safety and efficacy of medical products seeking registration and /or marketing 31 authorization. The submission should be as per CTD format or other international acceptable format. It 32 should be supported by regulations and guidelines that provide guidance to applicants on the required 33 quality, safety and efficacy documentation. All the necessary legal documentation should be published 34 and easily accessible, as well as enforceable. 35




1. Legal provisions that empower the NRA to request for satisfactory information on the quality, 3 safety and efficacy of medical products seeking registration /marketing authorization. 4

2. Whether legal provisions are published and easily accessible to applicants and other stakeholders, 5 as well as the systems and structures in place to enforce implementation of the legal provision. 6

3. Whether legal provisions have been implemented. The assessor should verify whether MA 7 applications guidance documents provide direction on the quality, safety and efficacy aspects of the legal 8 provisions. 9

Framework: 10


Rating Scale: 12

NI: There is no evidence of existence of legal provisions that require demonstration of the product 13 quality, safety and efficacy prior to registration and/or marketing authorization. 14

OI: The NRA has recently established these legal provisions, but have not yet been implemented. 15

PI: The NRA has legal provisions and has been applying it for minimum two years. 16

I: The NRA has legal provisions that require demonstration of the product quality, safety and efficacy 17 prior to registration and/or marketing authorization and can demonstrate a proven track record of applying 18 them. 19



References: 22

1. 2- WHO TRS 822, Annex 2, 1992 23


Sub-Indicator: MA01.04: There are legal provisions and/or regulations regarding the 25

limited duration of the validity of the ma and for periodic reviews to MAs (i.e. renewals) 26

Scope: All 27

Maturity: 2 28

Requirement: 29

Registration /marketing authorization validity period and registration /marketing authorization renewal 30

Rationale: 31

The objective of this sub-indicator is to assure that legal provisions and/or regulations exist to mandate 32 the NRA to register /grant marketing authorization with limited validity and also request that the 33


registration /marketing authorization is renewed before the medical product is re-introduced onto the 1 market 2

Description: 3

The assessor should verify that the legal provisions and /or regulations with respect to the validity period 4 for registrations and /or marketing authorizations and their renewals should be in existence, enacted and 5 implemented. The legal provisions and / or regulations should mandate the NRA to register /grant 6 marketing authorization with limited validity and also request that the registration /marketing 7 authorization is renewed before the medical product is re-introduced onto the market. The legal provision 8 and /or regulation should specify the validity periods of the initial registration / marketing authorization 9 and subsequent registration / market authorization (renewals), as well as provide information on the 10 requirements for renewing registration / marketing authorization. 11



1. Legal provisions and/or regulations that mandate the NRA to register /grant marketing 14 authorization with limited validity and also request that the registration /marketing authorization is 15 renewed before the medical product is re-introduced onto the market 16

2. Legal provisions and/or regulations on the validity periods of the initial registration / marketing 17 authorization and subsequent registration / market authorization renewals. 18

3. Regulations and/or guidelines on the requirements for renewing registration / marketing 19 authorization. 20

Framework: 21


Rating Scale: 23

NI: There is no legal provisions and/or regulations regarding the limited duration of the validity of the 24 MA and for periodic reviews to MAs 25

OI: The NRA has recently drafted these legal provisions, but they have not yet been implemented. 26

PI: The NRA has legal provisions and has been applying them for less than two years. 27

I: The NRA has legal provisions and/or regulations regarding the limited duration of the validity of the 28 MA and for periodic reviews to Mas and can demonstrate a proven track record of applying them. 29

Limitations and remarks: The assessor should note that some NRAs will have regulations that mandate 30 them to grant a “One Time” registration to a medical product due to active post-market surveillance 31 (PMS) (market surveillance and vigilance)activities. The assessor may review the PMS portfolio of the 32 NRA for more information, and the scoring should be “Not Applicable”. 33


References: 35



Sub-Indicator: MA01.05: There are regulations and/or guidelines for the definition, 1

types and the scope of variations along with the required documentation 2

Scope: All 3

Maturity: 3 4

Requirement: 5

Regulations/ guidelines on variations 6

Rationale: 7

The objective of this sub-indicator is to assure that regulations / guidelines exist to provide definitions, 8 types and the scope of variations as well as the documentation requirement for each variation. Due to 9 scientific and technological advancement, new developments/approaches that may affect the requirements 10 that permitted the registration of the product or the granting of marketing authorization should be made 11 known to the NRA. 12

Description: 13

The assessor should verify that the regulations and / or guidelines with respect to the definition, types and 14 the scope of variation as well as the documentation requirements should be in existence, enacted and 15 implemented. The regulations and / or guidelines should provide details of the corresponding 16 documentation requirements for each type/class of variation as well as guidance on the scope of 17 evaluation required. 18



1. Regulations and / or guidelines that document the definition, types and scope of variations, as 21 well as the corresponding documentation requirements. 22

2. Whether the regulations and / or guidelines are published, easily accessible, implemented and 23 known to the applicant, as well as what is required of them. 24

3. Types of variations documented, the scope of evaluation required and the corresponding fees (if 25 any) for appropriateness. 26

4. Processes and procedures for submitting variations to the NRA for review, and receiving 27 feedbacks, as well as timelines. 28

Framework: 29


Rating Scale: 31

NI: There is no regulations and/or guidelines for the definition, types and the scope of variations along 32 with the required documentation. 33

OI: The NRA has recently drafted these regulations and/or guidelines but they have not yet been 34 implemented. 35

PI: The NRA has regulations and/or guidelines and has been applying them for less than two years. 36


I: The NRA has regulations and/or guidelines for the definition, types and the scope of variations along 1 with the required documentation and required evidence of implementation. 2



MA04.03 5

References: 6

1. 77- WHO TRS 978, Annex 6, 2013 7

2. 56- WHO TRS 996, Annex 8, 2016 8

3. 50- WHO TRS 993, Annex 4, 2015 9

4. 12- WHO TRS 1003, Annex 6, 2017 10

Sub-Indicator: MA01.06: There are legal provisions to cover circumstances in which the 11

routine MA procedures may not be followed (e.g. for public- health interests) 12

Scope: All 13

Maturity: 1 14

Requirement: 15

Exemptions to routine MA procedures 16

Rationale: 17

The objective of this sub-indicator is to assure that legal provisions exist to mandate the NRA to use a 18 non-routine approach/procedure in exceptional cases in the interest of public health (e.g. emergencies, 19 epidemics, among others) for registration/marketing authorization of medical products. 20

Description: 21

The assessor should verify that legal provisions, with respect to circumstances in which the routine 22 medical product registration or marketing authorization procedures may not be followed, should be in 23 existence, enacted and implemented. The legal provisions should clearly indicate instances or situations 24 when the normal or routine procedure should not be applied. It should also provide guidance on the scope 25 of evaluation, as well as the documentation required in the event of use of the non- routine approach for 26 registration/marketing authorization. In addition, if the conditional authorization is time limited, the 27 assessor should verify duration of validity in this regulation. 28



1. Legal provisions that cover circumstances / instances in which the routine marketing 31 authorization procedures may not be followed. 32

2. Regulations and /or guidelines that give clarity on the regulatory requirements for granting 33 marketing authorization to medical products through the non-routine route other than the routine 34 marketing authorization procedure. 35


3. Records of medical products that have been authorized by this route. 1

Framework: 2


Rating Scale: 4

NI: There is no evidence of existence of legal provisions to cover circumstances in which the routine MA 5 procedures may not be followed. 6

OI: The NRA has recently drafted these legal provisions but they have not yet been followed. 7

PI: The NRA has legal provisions and has been applying them less than two years. 8

I: The NRA has legal provisions to cover circumstances in which the routine MA procedures may not be 9 followed and has applied them for all medical products that are used under this condition. 10

Limitations and remarks: It is proposed that the NRA should establish a provision to clarify the role and 11 responsibilities of all involved stakeholders who are involved in the emergency. The establishment of an 12 advisory committee under emergency, to make decisions and share the responsibilities, is suggested. 13

Non-routine MA procedures are those applicable only under specific circumstances e.g. emergencies with 14 public health impact and aimed to reduce the time for registration/granting MA. Names may vary 15 according the national legislation. 16

The collaborative procedure is a procedure for collaboration between the World Health Organization 17 (WHO) Prequalification Team (WHO/PQT) and interested national regulatory authorities in the 18 assessment and accelerated national registration of WHO-prequalified pharmaceutical products and 19 vaccines. The collaborative procedure can be of particular relevance when implemented for 20 pharmaceutical products and vaccines in emergency situation. (WHO TRS 996, Annex 8, 2016) 21


References: 23

1. 1- WHO TRS 885, Annex 8, 1999 24

Sub-Indicator: MA01.07: There are legal provisions and/or regulations that define 25

regulatory requirements to approve donation medical products. 26

Scope: All 27

Maturity: 1 28

Requirement: 29

MA procedures for donation medical products approval 30

Rationale: 31

The objective of this sub-indicator is to assure that legal provisions and/or regulations exist and are 32 implemented to mandate the NRA to have adequate regulatory oversight on received medical products 33 through donation. 34

Description: 35


The assessor should verify that legal provisions and/or regulations with respect to circumstances in which 1 the medical products are received through donation exist and are implemented. The legal provision should 2 provide guidance regarding required information and documentation that should be reviewed before 3 approval of donation medical products. It should also provide guidance on the scope of evaluation as well 4 as the documentation required to show quality, safety and efficacy of medical products. 5



1. Legal provisions that mandate the NRA to have adequate regulatory oversight on received 8 medical products through donation. 9

2. Regulations and /or guidelines giving clarity on the regulatory requirements for using medical 10 products that are received through donation. 11

3. Records. 12

Framework: 13


Rating Scale: 15

NI: There is no evidence of existence of legal provisions and/or regulations to define regulatory 16 requirements to use donation medical products 17

OI: The NRA has recently drafted this legal provisions and or regulations but it has not yet been followed. 18

PI: The NRA has legal provisions and has been applying it less than two years. 19

I: The NRA has legal provisions and/ or regulations defining regulatory requirements to use donation 20 medical products and has been applied for all received medical products through donation. 21

Limitations and remarks: Use a non-routine approach/procedure in exceptional cases in the interest of 22 public health (e.g. emergencies, epidemics, etc.) to use donation medical product or grant marketing 23 authorization to a medical product should be considered. If the NRA use the same regulatory approach 24 established for review other medical products for review of donation medical products, this sub indicator 25 can score as not applicable. 26


References: 28

1. 111- GPV Policy statement. Vaccine donations. (WHO/VSQ/97.05) 29

Sub-Indicator: MA01.08: Legal provisions and/or regulations allow the NRA to 30

recognize and/or use MA-relevant decisions, reports or information from other NRAs or 31

regional and international bodies 32

Scope: All 33

Maturity: 1 34

Requirement: 35


Recognition/reliance of MA application decision /information 1

Rationale: 2

The objective of this sub-indicator is to establish whether legal provisions and/ or regulations exist to 3 allow the NRA to establish a systematic framework to recognize and/or use relevant MA decisions, 4 reports or information from other NRAs, or regional and international bodies. 5

Description: 6

The assessor should verify that the legal provisions and/or regulations allowing the NRAs to recognize 7 and/or use relevant MA decisions, reports or information from other NRAs or regional and international 8 bodies should be in existence, enacted and implemented. The legal provisions and /or regulations should 9 provide clarity on the modalities/processes and procedures to recognize and/or use relevant MA decisions, 10 reports or information from other NRAs or regional and international bodies, as well as clarity on the 11 conditions (when/how) to recognize and/or use relevant MA decisions, reports or information from other 12 NRAs, or regional and international bodies. The legal provisions and/ or regulations should also provide 13 clear directives on the scope or extent of recognition and/or use of MA decisions, reports or information 14 from other NRAs, or regional and international bodies. 15



1. Legal provisions including guidelines that permits the NRA to recognize and/or use relevant MA 18 decisions, reports or information from other NRAs or regional and international bodies 19

2. The scope /extent of recognition and /or use of relevant MA decisions, reports or information 20 from other NRAs, or regional and international bodies. 21

3. Instances or situations that permit recognition and/or use relevant MA decisions, reports or 22 information from other NRAs or regional and international bodies 23

4. List of NRAs or regional and international bodies whose relevant decisions, reports or 24 information may be used to influence a MA application decision, and whether those NRAs, or regional 25 and international bodies are aware that their relevant MA decisions, reports or information on certain 26 MAs may be used (aware of the legal provision). 27

Framework: 28


Rating Scale: 30

NI: There is no evidence of existence of legal provisions and/or regulations that allow the NRA to 31 recognize and/or use MA-relevant decisions, reports or information from other NRAs or regional and 32 international bodies 33

OI: The NRA has recently drafted this legal provisions and/or regulations or other documented producers 34 but it has not yet been followed. 35

PI: The NRA has legal provisions and/or regulations for that and has been applying it for less than two 36 years with some supported resources. 37


I: The NRA has legal provisions and/or regulations allowing the NRA to recognize and/or use MA-1 relevant decisions, reports or information from other NRAs or regional and international bodies including 2 other supporting evidence of implementation like the list of NRAs or regional and international bodies. 3

Limitations and remarks: If the NRA has adequate resources this sub indicator can be considered not 4 applicable, but if any reliance exist, the NRA should have the legal bases. 5


References: 7

1. 112- Good regulatory practices: guidelines for national regulatory authorities for medical 8 products. Working document QAS 16.686 9

Sub-Indicator: MA01.09: Specific guidelines on the quality, nonclinical/safety and 10

clinical aspects are established and implemented 11

Scope: All 12

Maturity: 3 13

Requirement: 14

Requirement on quality, nonclinical/safety and clinical aspects 15

Rationale: 16

The objective of this sub-indicator is to establish whether specific guidelines exist on the quality, 17 nonclinical/ safety and clinical requirements for registering and/or granting marketing authorization, and 18 to verify whether they have been implemented. 19

Description: 20

The assessor should verify that specific guidelines on quality, nonclinical/safety and clinical requirements 21 for registering and/or granting marketing authorization should be established and implemented. The 22 guidelines should be detailed and should provide the specific regulatory requirements for each section in 23 the quality, nonclinical/safety and clinical dossier. They should be specific on the type of application and 24 the medical product. These specific guidelines may be documents not prepared by the NRA but adopted 25 in accordance with proper process. 26



1. Guidelines detailing the specific regulatory requirements for the quality, nonclinical/ safety and 29 clinical aspects of the MA dossier. The requirements, including the content and format should comply 30 with updated international best practices such as WHO or ICH. 31

2. Whether the guidelines are specific to the quality, nonclinical/safety and clinical aspect of the 32 MA application dossier and to the type of application and pharmaceutical product, and whether they are 33 published and implemented. 34

3. Sample of MA application submissions that is complying with the published guidelines. 35


4. Whether the document submitted was appropriate for the type of product and type of application 1 (new drug application, multisource drug, and product of biological origin or chemical synthesis among 2 others). 3

5. Specific guidelines on product labelling and packaging, package insert and the summary of 4 product characteristics information pamphlet for professionals or equivalent, as well as information 5 pamphlet for patients. 6

6. All quality and safety/efficacy reports for recently approved MA applications to determine 7 whether reviews are done thoroughly and based on guidelines requirements. 8

Framework: 9


Rating Scale: 11

NI: There is no evidence of existence of specific guidelines on the quality, nonclinical/safety and clinical 12 aspects. 13

OI: The NRA has recently drafted these guidelines but they have not yet been followed. 14

PI: The NRA has these guidelines and has been applying them for less than two years. 15

I: The NRA has established and implemented specific guidelines on the quality, nonclinical/safety and 16 clinical aspects. 17



References: 20

2. 79- Quality Assurance of pharmaceuticals. WHO guidelines, good practices, related regulatory 21 guidance and GXP training materials. WHO 2017 (CD-ROM) 22

3. 54- WHO TRS 987, Annex 4, 2014 23

4. 51- WHO TRS 986, Annex 2, 2014 24

5. 12- WHO TRS 1003, Annex 6, 2017 25

Sub-Indicator: MA01.10: There are guidelines on the format and content for submission 26

of marketing authorization applications consistent with the WHO or other internationally 27

accepted standards 28

Scope: All 29

Maturity: 3 30

Requirement: 31

Format and content for MA application 32

Rationale: 33


The objective of this sub-indicator is to establish whether guidelines exist on the format and content for 1 submitting a marketing authorization application to the NRA, by considering the nature of products and 2 different medical product streams. The presented content and format should be consistent with that 3 recommended by the WHO or other internationally accepted agencies. 4

Description: 5

The assessor should verify that guidelines on the format and content of a marketing authorization 6 submission should exist and should be consistent with international best practices, such as those proposed 7 or recommended by WHO or other internationally established agencies such as ICH. It should provide 8 clarity on the format and content on all sections of the MA application submission by considering 9 specifically each medical product streams, as well as, the processes and procedures for submitting the 10 application to the NRA. The guidelines should be established and implemented. 11



1. Guidelines providing clarity on the format and content of the MA application submission, as well 14 as the processes and procedures for submitting the application to the NRA. 15

2. Guidelines should be consistent with international best practices, such as those proposed by the 16 WHO or other internationally accepted agencies. 17

3. The checklist used to ensure that MA submissions actually comply with the content of the 18 guidelines before a MA application is acknowledged by the NRA. 19

Framework: 20


Rating Scale: 22

NI: There are no guidelines on the format and content for submission of marketing authorization 23 applications. 24

OI: The NRA has recently developed or drafted these guidelines, but it has not yet been followed. 25

PI: The NRA has developed or adopted these guidelines and has been applying them for less than two 26 years, or these guidelines have not been applied for all marketed medical products. 27

I: The NRA has guidelines on the format and content for submission of marketing authorization 28 applications consistent with WHO or other internationally accepted standards including other required 29 documentation. This process is implemented and followed for all marketed medical products. 30

Limitations and remarks: In case of benchmarking of more than one medical products stream, the 31 assessor should consider and review different guidelines, as applicable. In some cases, instructions for 32 dossier compilation may be part of the main (technical) guideline. 33


There is need to add more guidance in this regard such as other international references. 35 36


References: 1

1. 12- WHO TRS 1003, Annex 6, 2017 2

2. 51- WHO TRS 986, Annex 2, 2014 3

3. 54- WHO TRS987, Annex 4, 2014 4

4. 65-WHO TRS 961, Annex 15, 2011 5

5. 52- WHO TRS 986, Annex 6, 2014 6

6. 118- ICH. The Common Technical document for registration of pharmaceuticals for human use: 7 quality. M4Q, September 2002 8

Sub-Indicator: MA01.11: There are guidelines for marketing authorization holders 9

defining the types and scopes of variations, the format and the documentation required as 10

well as specifications of the variations that are subjected to prior approval or notification. 11

Scope: All 12

Maturity: 3 13

Requirement: 14

Variation documents for MA holders 15

Rationale: 16

The objective of this sub-indicator is to assure that guidelines exist for marketing authorization holders 17 (MAHs) providing guidance on the types and scopes of variations, the format and the documentation 18 required, as well as specifications of the variations that are subjected to prior approval or notification. 19

Description: 20

The assessor should verify that the guidelines for MAHs providing guidance on the types and scopes of 21 variations, the format and the documentation required, as well as specifications of the variations that are 22 subjected to prior approval or notification should be in existence and implemented. It should include 23 directives on changes made to leaflets, package inserts, labelling (container and package), and summaries 24 of product characteristics. It should also clearly specify to the MAH, variations that would require their 25 approval before the change(s) is /are implemented, those that require just notifying the NRA before or 26 during implementation and those that do not need notifying the NRA. 27



1. Guidelines for MAHs providing guidance on the types and scopes of variations, the format and 30 the documentation required, as well as specifications of the variations that are subjected to prior approval 31 or notification. The assessor should note that the guidelines should include changes to information 32 contained in leaflets, package inserts, labelling (package and containers), and summaries of product 33 characteristics that have been previously approved. 34

2. The guidelines that specify to the MAH, variations that would require their approval before the 35 change(s) is /are implemented, those that require just notifying the NRA before or during implementation 36


and those that do not need notifying the NRA. The assessor should request for a sample of submitted and 1 processed variation application and relate it to the guidelines for the purpose of conformance. 2

Framework: 3


Rating Scale: 5

NI: There is no evidence of existence and implementation of guidelines for marketing authorization 6 holders defining the types and scopes of variations, the format and the documentation required as well as 7 specifications of the variations that are subjected to prior approval 8

OI: The NRA has recently developed or drafted these guidelines, but it has not yet been followed. 9

PI: The NRA has developed or adopted these guidelines and it is applying for less than two years or it has 10 not been applied for all marketed medical products. 11

I: The NRA has guidelines for marketing authorization holders defining the types and scopes of 12 variations, the format and the documentation required as well as specifications of the variations that are 13 subjected to prior approval and they are implemented and documented for all marketed medical products. 14

Limitations and remarks: In case of benchmarking of more than one medical products stream, the 15 assessor should consider and review different guidelines as applicable. 16


MA04.03 18

References: 19


2. 3- WHO TRS 858, Annex 1, 1995 21

Sub-Indicator: MA01.12: There are established guidelines to cover circumstances in 22

which the routine MA procedures may not be followed (e.g. for public- health interests) 23

Scope: All 24

Maturity: 3 25

Requirement: 26

Guidance on non-routine MA procedures 27

Rationale: 28

The objective of this sub-indicator is to assure that guidelines exist to provide guidance to the NRA on 29 how to process MA applications using the non-routine MA approach/procedure in exceptional cases and 30 in the interest of public health (e.g. emergencies, epidemics, others). 31

Description: 32

The assessor should verify that guidelines providing guidance to the NRA on how to manage MA 33 applications through the non-routine MA application procedures due to emergencies and /or in the interest 34


of public health are established and implemented. The guidelines should provide guidance on the critical 1 requirements that need to be reviewed by the NRA (e.g. label indication, mode of usage or application, 2 storage, GMP certificate and /or report, among others), on what has to be documented and monitored (e.g. 3 location of deployment, quantities to be deployed, identity of persons to receive and manage deployment 4 and others) by the NRA before registration of the medical products. It should be supported with SOPs 5 that provide guidance on the assessment /evaluation process as well as the scope of evaluation needed. 6 The assessor should verify if NRA will have access to experts (advisory committee) to support NRA 7 during the decision making process. The application processing flow and corresponding timelines should 8 be available. 9



1. Guidelines that provide guidance to the NRA on the application of a non-routine MA procedure 12 in emergency situation, as well as the corresponding SOPs or similar documents. 13

2. A sample application that was processed using the guidelines. The assessor should verify whether 14 the requirements in the guidelines were submitted and duly evaluated, and whether the evaluation process 15 flow and timelines were respected. 16

3. Evidence of access to experts or advisory committee in case of emergency 17

Framework: 18


Rating Scale: 20

NI: There are no guidelines to cover circumstances in which the routine MA procedures may not be 21 followed. 22

OI: The NRA has recently developed or drafted these guidelines but it has not yet been followed. 23

PI: The NRA has developed or adopted officially this guideline but has not been applied for all medical 24 products that were approved by non-routine regulatory procedures. 25

I: The NRA has guidelines to cover circumstances in which the routine MA procedures may not followed 26 and has applied them for all medical products that have been approved under this condition. 27

Limitations and remarks: There is need to verify if these guidelines have been applied in emergency 28 situation and all relevant records are available. If there was not any emergency situation after the 29 guidelines’ development, the assessor should verify that at least all required resources are available. 30


• MA04 01 32

• MA04 04 33

• MA04 07 34

• MA04 09 35

References: 36


1. 76- Guidance on development and implementation of a national deployment and vaccination plan 1 for pandemic influenza vaccines. WHO 2012 2

2. 114- Emergency response framework. WHO, 2013 3

Sub-Indicator: MA01.13: There are guidelines on the content of product information 4

leaflets, summary of product characteristics (SPC) like information, packaging and 5

labelling 6

Scope: All 7

Maturity: 3 8

Requirement: 9

Guidance on content of product information leaflets, summary product characteristics (SPC), packaging 10 and labelling 11

Rationale: 12

The objective of this sub-indicator is to assure that guidelines exist to provide guidance on the MA 13 application requirements on the content of the product information leaflets, summary product 14 characteristics (SPC), packaging and labelling. 15

Description: 16

The assessor should verify that guidelines providing guidance on the MA application requirements on the 17 content of the product information leaflets, summary product characteristics (SPC) like information, 18 packaging and labelling should be in existence and should be implemented. These should also provide 19 clarity on the information to be included in the package inserts (inserts, patient information leaflets), 20 summary of product characteristics (information leaflet for professionals or equivalent) and labelling of 21 container-packaging. 22



1. Guidelines that provide guidance on the MA application requirements on the content of the 25 product information leaflets, summary product characteristics (SPC), packaging and labelling. 26

2. Relevant documentation including forms and records 27

Framework: 28


Rating Scale: 30

NI: There is no guidelines on the content of product information leaflets, summary of product 31 characteristics (SPC), packaging and labelling 32

OI: The NRA has recently drafted this guideline but it has not yet been followed 33

PI: The NRA has developed or adopted this guideline and has been applying it for less than two years or 34 not fully followed for all marketed medical products. 35


I: The NRA has a guidelines on the content of product information leaflets, summary of product 1 characteristics (SPC), packaging and labelling and are followed for all marketed medical products. 2



• MA04.08 5

• MA05.01 6

References: 7

1. 80-WHO TRS 902, Annex 9, 2002 8

Indicator: MA02 Arrangement for effective organization and good 9

governance. 10

Rationale: 11

The objective of this indicator is to assure that there is a legal basis and relevant administrative process 12 and documentation for the organizational structure and governance that allows for the smooth 13 coordination, collaboration and exchange of information within and without the entity that is responsible 14 for registration and/or granting marketing authorization (MA). The arrangement should establish and 15 define the role and responsibilities within and without the NRA for registration/MA activities. It should 16 ensure that the NRA responsible for registration and /or MA has complete control or access to all 17 information related to produced, procured or received medical products, relevant information such as 18 GMP inspection reports, lab analysis data, collected vigilance data, registration and /or MA applications, 19 the number and identity of medical products registered and/or granted MA, on-going registration and /or 20 MA activities, new directives, newly registered products, withdrawn/suspended registration and/or MA, 21 rejected applications, etc. 22

The objective of this indicator is to establish that structures are in place at the organizational and 23 governance levels to promote effective intra- and inter NRA relationships where information traffic is 24 efficiently managed. 25

Category: Organization and governance 26

Sub-Indicator: MA02.01: There is a defined structure with clear responsibilities to 27

conduct registration and/or marketing authorization activities 28

Scope: All 29

Maturity: 2 30

Requirement: 31

Structure and responsibilities 32

Rationale: 33

The objective of this sub-indicator is to assure whether there are structures in place, with clearly defined 34 roles and responsibilities for each structural and governance level for registration and/ or marketing 35 authorization activities to ensure effectiveness, efficiency and consistency. 36


Description: 1

The assessor should verify that the defined structures with clear responsibilities to conduct registration or 2 MA activities should be established and implemented. The structure should show identity (ies) of entity 3 (ies) within the NRA that has/have the mandate for registration /marketing authorization activities. The 4 structure should be supported with regulations and/or guidelines mandating such entity (ies) to conduct 5 registration /marketing authorization activities, as well as defining the scope/extent of their 6 role/responsibilities within the NRA in relation to registration /marketing authorization activities, as well 7 as their reporting lines. The structure of the entity (ies) relative to relationships and ranks, as well as 8 information exchange / flow within and without (e.g. officers at the port of entry, officers in the post 9 market surveillance unit) the entity (ies) should be defined. 10



1. Roles and responsibility (ies) of the entity (ies) responsible for registration /marketing 13 authorization activities and their placement on the organizational chart in relation to other entities 14 involved in registration /marketing authorization –related activities 15

2. Regulation and/or guidelines mandating such entity (ies) or authority (ies) with the responsibility 16 to conduct registration /marketing authorization activities. 17

3. The document with the organizational structure or organogram. 18

Framework: 19


Rating Scale: 21

NI: There is no evidence of the defined structure with clear responsibilities to conduct registration and/or 22 marketing authorization activities. 23

OI: The NRA has recently drafted a defined structure with clear responsibilities to conduct registration 24 and/or marketing authorization activities, but it has not yet been placed. 25

PI: The NRA has a defined structure with clear responsibilities to conduct registration and/or marketing 26 authorization activities and has been established, however there is the need to improve this function by 27 providing adequate resources or more support for effective implementation and coordination with other 28 entities. 29

I: The NRA has a defined structure with clear responsibilities to conduct registration and/or marketing 30 authorization activities that is fully supported with the required resources. 31


References: 33

1. 1- WHO TRS 885, Annex 8, 1999 34

2. 12- WHO TRS 1003, Annex 6, 2017 35


Sub-Indicator: MA02.02: Documented and implemented procedures exist to ensure 1

involvement and communication with all relevant regulatory entities (assessors, quality 2

control (QC) laboratory and inspectorate) as necessary 3

Scope: All 4

Maturity: 3 5

Requirement: 6

Procedures for effective communication 7

Rationale: 8

The objective of this sub-indicator is to assure there are documented procedures and /or 9 structures/mechanisms in place to ensure proper inter- and intra-relationships between and within entities 10 involved in dossier assessment, quality control laboratory and inspectorate in order to ensure effective and 11 efficient exchange of information amongst stakeholders. 12

Description: 13

The assessor should verify the documented procedures promoting involvement and communication 14 between all relevant entities inside the NRA should be in existence and implemented, and supported with 15 adequate documentation. The procedure should provide clear directives on the regulatory divisions and 16 guidance on how to implement the operational procedures to ensure the involvement and communication 17 between relevant entities within the NRA such as dossier assessment, quality control laboratory and 18 inspectorate. It should be clear on the scope/extent of role/ responsibilities of each relevant entity (ies). In 19 addition, the procedure should be supported by information and/or documentation transfer policy to 20 ensure that directives/ information and/or documentation reach the intended recipients and feedback is 21 received. The procedures should be captured in the form of SOPs and guidelines which are approved and 22 authorized to be implemented, and with appropriate disclosure. 23



1. The registration /marketing application guidelines capturing duties/roles and responsibilities of 26 the relevant entities inside the NRA involved in registration /marketing activities and whether documents 27 are known to the relevant organizations, divisions, institutions, and/or departments. 28

2. Whether documented procedures have been implemented, and the impact of the implementation 29 has been evaluated. 30

3. Standard operating procedures (SOPs) that guide and/or inform effective communication and 31 collaboration between relevant entities inside the NRA such as the quality control laboratory, vigilance, 32 inspectorate, among others. 33

4. The feedback mechanism in the information and /or documentation transfer policy (ies) among all 34 relevant regulatory divisions. 35

Framework: 36

Process 37


Rating Scale: 1

NI: There is no evidence of documented procedures to ensure involvement and communication with all 2 relevant regulatory entities (assessors, quality control (QC) laboratory and inspectorate) as necessary. 3

OI: The NRA has recently drafted procedures to ensure involvement and communication, but it has not 4 yet been followed. 5

PI: The relevant entities inside the NRA established the procedures to ensure involvement and 6 communication, but these have not been fully incorporated neither is consistent with relevant SOPs. 7

I: The NRA has procedures to ensure involvement and communication with all relevant regulatory entities 8 (assessors, quality control (QC) laboratory and inspectorate) as necessary, including required 9 documentation and records. 10



References: 13

1. 8- WHO TRS 902, Annex 8, 2002. Chapter 5.2 (d) 14

2. 35- WHO TRS 957, Annex 1, 2010 15

Indicator: MA03 Human resources to perform registration and 16

marketing authorization activities. 17

Rationale: 18

The objective of this indicator is to assure all entity (ies) within an NRA is/are adequately resourced with 19 trained, experienced and skilled workforce, as well as empowered to fully perform their functions of 20 registering and/or granting marketing authorization to medical products. This will ensure that medical 21 products registration and/or marketing authorization processes or activities are performed in accordance 22 with international best practices. 23

The objective of this indicator is to evaluate the human resource capacity of the entity (ies) with respect to 24 the number of personnel, their composition, their skill set and experience, as well as their specific areas of 25 expertise to perform the registration and/or marketing authorization function. 26

Category: Resources (HR, FR, infrastructure and equipment) 27

Sub-Indicator: MA03.01: Enough competent staff (education, training, skills and 28

experience) are assigned to perform marketing authorization and/or registration activities 29

Scope: All 30

Maturity: 3 31

Requirement: 32

Enough number of competent human resources in charge of MA activities. 33

Rationale: 34


The objective of this sub-indicator is to assure the existing human resources for registration and/or 1 marketing authorization is adequate in terms of number, their experience and their specific competences 2 to perform all the activities along the entire registration and /or marketing authorization chain. 3

Description: 4

The assessor should verify that the human resources assigned to perform registration and /or marketing 5 authorization activities should be enough in terms of number and are competent with the requisite skills, 6 education, experience and training. There should be technical documents and /or SOPs that provide 7 guidance on the required background for MA activities taking into account the educational background, 8 skills, and experience and training requirements. 9

The assessor should verify that the NRA estimated the adequate number of staff required to effectively 10 and efficiently perform MA function and such number is actually recruited. In addition, the assessor shall 11 verify that these competency requirements are well set and maintained by the NRA Metrics and statistics 12 on the different activities performed as well as performance indicators can be used for estimating the 13 adequacy of the number of the assigned staff. The assessor should also verify that the competency of the 14 assigned staff is built, maintained and improved through induction as well as continuous on-the-job 15 training. 16



1. The number of human resources involved in each of the documented activities along the entire 19 registration and /or marketing authorization process flow for adequacy. 20

2. The systems/structures in place to ensure appropriate placement of staff with respect to 21 competence and skills, and whether the system or structures have been implemented including the records 22 of staff review. 23

3. The professional profiles of the human resources engaged in the registration and/or marketing 24 authorization activities for appropriateness of education, and skills/expertise to perform a particular 25 function along the registration and /or marketing authorization chain including the list the requisite skills 26 and training. 27

4. Recruitment plan. 28

Framework: 29


Rating Scale: 31

NI: The NRA does not have enough competent staff (education, training, skills and experience) to 32 perform marketing authorization and /or registration activities 33

OI: The NRA has recently developed the plan to recruit adequate competent staff, however, the plan has 34 not been implemented. 35

PI: The NRA has initiated the implementation of the human resources development plan, however there is 36 need to complete competency profile. 37


I: The NRA has a sustained number of competent staff (education, training, skills and experience) 1 assigned to perform marketing authorization and/ or registration activities. 2

Limitations and remarks: The adequacy and appropriateness of the number of the staff is quite 3 subjective and shall be linked to some process and/or output indicators. The assessor should consider the 4 workload, backlog, and delay in delivery based on established timeframes for estimation of staff 5 adequacy. 6


References: 8

1. 1- WHO TRS 885, Annex 8, 1999 9

2. 51- WHO TRS 986, Annex 2, 2014 10

Sub-Indicator: MA03.02: Duties, functions, and responsibilities of the staff, in charge of 11

marketing authorization and/or registration activities are established and updated in the 12

respective job descriptions 13

Scope: All 14

Maturity: 3 15

Requirement: 16

Duties, roles and responsibilities of the staff relevant to registration and/or marketing authorization 17 activities. 18

Rationale: 19

The objective of this sub-indicator is to assure clear and well defined duties and responsibilities of the 20 staff and that their job descriptions are up to date with their current duties, functions and responsibilities, 21 and adequately documented. 22

Description: 23

The assessor should verify that a current and updated structure for job description should be in place. In 24 addition, it should address staff current duties, responsibilities and the requisite competency. Such format 25 and content should be established and implemented for all staff. The procedure should be supported with 26 a guidance document that provides direction on when and how to update the information, and where the 27 information is kept for easy access. It should present the appropriate duties/responsibilities that are 28 assigned to roles for each member of the entity (ies) involved in registration and/or marketing 29 authorization activities. Thus, the professional profiles of staff are reflected in their respective roles and/ 30 or responsibilities within the NRA. There should be procedures to guide responsible persons to document 31 up to date revised duties/functions and responsibilities. In addition, procedures to guide the keeping 32 and/or documenting of up to date work schedule, and enforcing the implementation of the documented 33 guidelines /procedures should be available. 34



1. The procedure and /or guidelines that guide placement of staff members within the NRA; 37


2. The professional profiles of staff (job descriptions), and relate it to their current roles/duties; 1

3. The professional profiles of the external experts for completeness, in relation to their composition 2 as prescribed in the legal provision; 3

4. Procedures to guide the documentation of up to date duties and work schedules, and enforcing the 4 implementing of the documented guidelines /procedures; 5

5. Job description of designated staff. 6

Framework: 7


Rating Scale: 9

NI: There is no evidence of defined or establishment of duties, functions, responsibilities, respective job 10 descriptions and necessary required competencies. 11

OI: The NRA has recently draft or develop the role and responsibilities document but it has not yet been 12 in place. 13

PI: The NRA has initiated implementation of this requirement but has not defined or followed for all staff 14 or the role and responsibility document including staff job descriptions are not up to date. 15

I: The NRA has defined and established of all required duties, functions, responsibilities and respective 16 job descriptions. 17


References: 19

1. 1- WHO TRS 885, Annex 8, 1999 20

2. 4- ISO standard 9001:2015 sub-clause 5.1.1 21


4. 7- Effective Drug Regulation; 2002. Chapter 4.4 23

Sub-Indicator: MA03.03: Training plan developed, implemented and updated at least 24

once a year 25

Scope: All 26

Maturity: 3 27

Requirement: 28

Implementation of training plan 29

Rationale: 30

The objective of this sub-indicator is to assure training plan for staff exist and it is implemented, and 31 annually updated. Through the training plan, NRA can be sure that competency of staff in charge of MA 32 activities is maintained and enhanced. 33


Description: 1

The assessor should verify that the training plans are developed, implemented and updated at least once 2 every year to reflect current situation by considering education and experience of the staff. The training 3 plan should be complemented with guidelines or similar documents to guide the development and 4 implementation of training plans. The assessor should ensure planning and implementation of induction 5 training for new staff as well as continued on-the-job for staff. There should be procedures to approve 6 training plan and budget allocated for the training plan, as well as updating the plan. The plan should 7 present clearly defined training goals. This should involve learning certain topics and /or skills to address 8 deficiencies. Learning objectives, training methods/activities, documentation / or evidence of learning and 9 evaluation / assessment of training should be documented. This will confirm whether the learning 10 objectives were achieved and they have been designed to address weaknesses within the entity (ies). 11 Procedures should be in place to ensure that a training plan is developed, implemented and updated at 12 least once every year. Measuring training effectiveness is another aspect which the assessor should 13 check. The assessor should verify that there is a system in place for monitoring the implementation and 14 effectiveness of the training plan along with the skills acquired in training activities for internal and 15 external experts. 16



1. Guidelines for development, implementation and update of the training plan at least once every 19 year as well as mechanism to measure effectiveness of training. 20

2. The system/structures used to approve the training plan as well as the budget allocated to the 21 training activities for the purposes of adequacy. 22

3. The current/existing training plan/matrix of staff, and relate it to their respective job 23 descriptions. 24

4. SOP for developing and maintaining the training plan. 25

5. Collection and/or investigation of training needs. 26

6. List of trainings performed. 27

7. Example records for training activities. 28

Framework: 29

Process 30

Rating Scale: 31

NI: There is no systematic training program including training matrix/plan. 32

OI: The NRA has recently drafted or developed the training plan but there is no evidence of 33 implementation. 34

PI: The NRA has developed and initiated the training plan implementation. However, the NRA has not 35 fulfilled all required planed training and has been applying it for less than two years. 36


I: The NRA has an updated training plan developed and supported by adequate records to demonstrate 1 effective plan implementation, including induction training for new staff and routine on-the-job training 2 for recruited staff. 3

Limitations and remarks: Training plans must be updated regularly, expectedly on annual basis, but not 4 beyond biennial (once every two years). 5


References: 7


2. 8- WHO TRS 902, Annex 8, 2002. Chapter 5.2 (d) 9

Sub-Indicator: MA03.04: The NRA performs and maintains records of staff training 10

activities 11

Scope: All 12

Maturity: 3 13

Requirement: 14

Training records 15

Rationale: 16

The objective of this sub-indicator is to assure that training organized by the NRA/ responsible regulatory 17 authority (ies), and the training records are adequately documented, and maintained/kept. Staff training 18 records as an integral part of staff file is a tool for measuring and tracking staff competency and 19 development and adequacy. 20

Description: 21

The assessor should verify that the records on staff training performed/organized by the NRA exist, are 22 implemented and regularly updated. The activity should be supported with guidelines to guide the NRAs 23 to perform and maintain records of staff training activities. Procedures to document and propose staff 24 training needs as well as to allocate a budget for continuous staff capacity building/development should 25 be in place. The assessor should check there is an evaluation/ assessment mechanism to verify the quality 26 of learning, and to confirm whether learning objectives are achieved. An inventory (soft and/or hard) 27 system to document all impactful and non- impactful trainings, and members of staff who participated. A 28 system to measure /estimate impact of trainings should be established. 29



1. Guidelines or similar documents to guide the NRA to perform and maintain records of staff 32 training activities; 33

2. Training effectiveness evaluation; 34

3. The training inventory, and how the inventory is completed; 35

4. Records keeping or staff training archives and archiving system. 36


Framework: 1

Output 2

Rating Scale: 3

NI: There is no evidence that the NRA performs and maintains records of staff training activities. 4

OI: The NRA has recently planned or initiated to perform, document and keep records of staff training 5 activities, however it is not followed yet. 6

PI: The NRA has recently planned or initiated to perform, document and keep records of staff training 7 activities but it is not fully followed for all training activities or it is established for less than two years. 8

I: The NRA performs and maintains records of staff training activities. 9

Limitations and remarks: The assessor should note that some NRAs out-source training including staff 10 capacity development activities. In this case the assessor should request for the identity, as well as the 11 provider evaluation status. The assessor may request for the professional profiles of tutors or resource 12 persons used to offer training, as well as the records and archiving systems in place. 13


References: 15

1. 1- WHO TRS 885, Annex 8, 1999 16

2. 4- ISO standard 9001:2015 sub-clause 5.1.1 17



Indicator: MA04 Procedures established and implemented to perform 20

registration and/or marketing authorization 21

Rationale: 22

The objective of this indicator is to assure that procedures have been established and implemented to 23 effectively perform registration and/or marketing authorization activities. The NRA should have 24 documented procedures for registration and/or marketing authorization activities and they should be 25 implemented. The documented procedures should be efficiently designed to assist in the preparation of 26 the registration and/or marketing authorization application pack, receipt and evaluation of the pack. The 27 procedures should present an overview of the activities to be carried out, the steps to be followed, the 28 resources required, the processes to be followed to evaluate the documents submitted, the 29 interrelationship between the various documents, and the legal basis for them. The established procedures 30 will give guidance on how registration and/or marketing authorization applications are handled to ensure 31 efficiency and effectiveness of the process. 32

Category: Regulatory process 33


Sub-Indicator: MA04.01: Documented procedures/tools are implemented for the 1

assessment of the different parts of the application and for the assessment of specific 2

requirements of specific classes of medical products (quality, safety & efficacy) 3

Scope: All 4

Maturity: 3 5

Requirement: 6

Documented procedures and tool implemented 7

Rationale: 8

The objective of this sub-indicator is to assure that documented procedures/tools exist for the assessment 9 of the different parts of the registration and/or marketing authorization application and for the assessment 10 of particular requirements of specific class medical products, and that the procedure/ tool is implemented 11 during the assessment process to avoid discrimination/ bias. 12

Description: 13

The assessor should verify that documented procedures/tools are developed, made available and 14 implemented in the assessment of the different parts of the registration and/or marketing authorization 15 applications as well as in the assessment of particular requirements for specific class of medical products. 16 The procedure should be designed to be adaptable to all applications regardless of the source and product 17 class. It should provide guidance on the different parts of the MA application processing flow, including 18 receipt, screening, review, decision and others. The procedure for the assessment of registration and/ or 19 marketing authorizations should be supported with adequate SOPs or similar documents. The 20 scope/extent of the screening and review processes should be clearly documented for routine and non-21 routine MA application processing. The assessor should note that there may be other registration / 22 marketing authorizations routes for prequalified products or during emergencies where the formal 23 documented procedures and tools may not apply. 24



1. Documented procedures/tools, including SOPs developed and used in the assessment of the 27 different parts of the registration / marketing authorization applications as well as in the assessment of 28 particular requirements for specific class of medical products 29

2. SOPs used for receipt, screening, review and decision among others, of MA applications. 30

3. Evidence of use of SOP, for example by checking whether major milestones/decision points are 31 recorded for a selected recent MA. 32

4. The scope/extent of the assessment process with respect to MA application received and 33 processed through the routine and non-routine routes. 34

5. The registration application backlog and relate it to the timelines for the review process as well as 35 the number of competent officers involved in the review process. 36

Framework: 37


Process 1

Rating Scale: 2

NI: There is no documented procedures/tools for the assessment of the different parts of the application 3 and for the assessment of particular requirements of specific classes of medical products (quality, safety 4 & efficacy) 5

OI: The NRA has recently drafted or adopted the documented procedures/tools for the assessment of MA 6 but it has not yet been followed. 7

PI: The NRA has developed and /or adopted documented procedures/tools for the assessment of MA and 8 has been applying it for less than two years, or has not considered them for all marketed products. 9

I: The NRA has documented procedures/tools for the assessment of different parts of the application and 10 for the assessment of particular requirements of specific classes of medical products (quality, safety & 11 efficacy) and it has applied for all reviewed and marketed medical products. 12

Limitations and remarks: There may not be adequate time during the on site assessment to review the 13 scope/extent/consistency of the assessment processes for example between two MAs. This may be done 14 off site though review of point 4 (scope/extent of the assessment process) will likely require expert 15 assessors (dossier reviewers) from SRAs or PQ assigned reviewers. 16


References: 18

1. 6- Regulation of vaccines: building on existing drug regulatory authorities. (WHO/V&B/99.10) 19

2. 3- WHO TRS 858, Annex 1, 1995 20

3. 79- Quality Assurance of pharmaceuticals. WHO guidelines, good practices, related regulatory 21 guidance and GXP training materials. WHO, 2017(CD-ROM) 22

Sub-Indicator: MA04.02: Documented procedures are implemented to renew and/or to 23

periodically review the MAs granted 24

Scope: All 25

Maturity: 3 26

Requirement: 27

Registration /marketing authorization renewal 28

Rationale: 29

The objective of this sub-indicator is to assure that documented procedures/ tools exist and are 30 implemented to renew and /or periodically review the registration / marketing authorization granted to 31 ensure consistency and effectiveness of procedure. 32

Description: 33

The assessor should verify that documented procedures /tools to renew and /or periodic review medical 34 products registration and / or marketing authorization should be established and implemented. The 35 procedures should be supported with guidance on the requirements to be reviewed, as well as the SOPs to 36


be implemented at the various stages of the renewal and /or periodic review process. Guidance should be 1 available on the receipt of application, screening, review, decision, others. There should be a documented 2 process flow that is implemented. 3



1. Legally approved validity periods. 6

2. Documented procedures /tool, including SOPs or other technical documents for receipts, 7 screening, review, decision, issuance of registration number of certificate, others. 8

3. Process flow with responsible divisions and timelines 9

Framework: 10

Process 11

Rating Scale: 12

NI: There is no evidence of documented procedures to renew and/or to periodically review the MAs 13 granted. 14

OI: The NRA has recently drafted or adopted documented procedures to renew and/or to periodically 15 review the MAs granted but it has not yet been applied. 16

PI: The NRA has developed or adopted documented procedures to renew and/or to periodically review 17 the MAs granted and has been applying it for less than five year or has not considered yet for all marketed 18 medical products. 19

I: The NRA has documented procedures to renew and/or periodically review the MAs granted and has 20 applied them for all marketed products. 21

Limitations and remarks: The assessor should note that not all NRA’s apply the registration and/ or 22 marketing authorization renewal or periodic review system. Some NRA’s may boast of a stringent and 23 functional market surveillance system /structure with a demonstrated ability to perform its functions. In 24 this case, the assessor should score this sub indicator as not applicable. 25


Refer to PV and MC functions. 27

References: 28

1. 79- Quality Assurance of pharmaceuticals. WHO guidelines, good practices, related regulatory 29 guidance and GXP training materials. WHO, 2017(CD-ROM) 30

Sub-Indicator: MA04.03: Documented procedures are implemented for assessing the 31

applications for variation of MAs. 32

Scope: All 33

Maturity: 3 34

Requirement: 35


Documented and implemented variations assessment procedures 1

Rationale: 2

The objective of this sub-indicator is to assure that documented procedures exist, and are implemented to 3 assess variations to registrations / marketing authorizations. 4

Description: 5

The assessor should verify that documented procedures / SOP that will guide the assessor to assess the 6 various classes/type of variations for the various product classes are established and implemented. It 7 should contain procedures to receive and process the various classes/ types of variation applications. 8


The assessor should request for and review. 10

1. Procedures / SOP that guides the assessor to assess the various classes of variations. The 11 documented procedure should be medical product class specific. 12

2. Procedures to receive and process variation applications per the prescribed procedure, and 13 whether they have been implemented. The assessor should request a sample application received and 14 assessed for compliance with documented procedures. 15

Framework: 16

Process 17

Rating Scale: 18

NI: There is no evidence of documented procedures for assessing the applications for variation of MAs. 19

OI: The NRA has recently drafted or adopted documented procedures for assessing the applications for 20 variation of MAs but it has not yet been followed. 21

PI: The NRA has developed or adopted documented procedures for assessing the applications for 22 variation of MAs and has been applying it for less than two years or all submitted variations were not 23 reviewed based on that. 24

I: The NRA has documented procedures for assessing the applications for variation of MAs and has 25 implemented them for all marketed medical products. 26

Limitations and remarks: This review may also require an expert dossier reviewer. The assessor should 27 note that NRA may have different requirements and guidelines for different medical product streams. 28


• MA01.05. 30

• MA01.10 31

References: 32

1. 79- Quality Assurance of pharmaceuticals. WHO guidelines, good practices, related regulatory 33 guidance and GXP training materials. WHO 2017 (CD-ROM) 34


Sub-Indicator: MA04.04: The same criteria apply for assessing applications regardless 1

of the origin or destination for the use of the medical products (e.g. domestic, foreign, 2

public/private sector) 3

Scope: All 4

Maturity: 3 5

Requirement: 6

Same criteria for assessing registration and/or marketing authorization applications regardless of applicant 7

Rationale: 8

The objective of this sub-indicator is to assure the criteria used for assessing registration and/or marketing 9 authorization applications for medical products is the same, regardless of the source/origin or the 10 destination where the product(s) will be used. 11

Description: 12

The assessor should verify that the criteria applied to assess registration and/or marketing authorization 13 applications are the same regardless of the origin/source of the medical product and the actual application, 14 as well as where the product will be used. A set of criteria for assessment should be documented and 15 implemented for the various class of medical products application. The criteria should be documented in 16 the form of SOPs. Guidance in the form of SOP should be documented and implemented. 17



1. Whether SOPs or similar documents are in place to ensure that the same set of criteria is used to 20 assess registration /marketing authorization applications regardless of source /origin and / or destination 21 of the medical product. 22

2. Whether the staff members are aware of the single set of criteria, how to apply the criteria in their 23 assessment activities, and whether the criteria is documented, published and implemented. 24

3. Sample registration and/or marketing authorization applications from different source for the 25 purposed of verification, compliance and consistency. The assessor should compare the procedures and 26 verify whether discrepancy in terms of scientific review/assessment exists. Sample MA applications of 27 medical products coming from different sources (e.g. domestic, imported, public, private) along with their 28 assessment reports should be checked by the assessor. 29

Framework: 30

Process 31

Rating Scale: 32

NI: The NRA does not apply the same criteria for assessing applications (e.g. domestic, foreign, 33 public/private sector) 34

OI: The NRA has recently defined or drafted the same criteria or policy for assessing applications but it 35 has not yet been applied. 36


PI: The NRA has defined the same criteria or policy for assessing applications and has been applying for 1 assessing applications for less than two years or has not applied for all currently market medical products. 2

I: The NRA has the same criteria or policy for assessing applications regardless of the origin or 3 destination of medical products (e.g. domestic, foreign, public/private sector) and has applied for all 4 marketed medical products. 5

Limitations and remarks: There may not be adequate time during the on site assessment to compare the 6 scope/extent/consistency of the assessment processes for example between two MAs. This may be done 7 off site though 8

The assessor shall consider in non-producing countries, there is no record to compare the same 9 implementation of this requirement. 10


References: 12

1. 12- WHO TRS 1003, Annex 6, 2017 13

2. 51-WHO TRS 986, Annex 2, 2014 14

3. 7- Effective Drug Regulation; 2002.Chapter 4.4 15

Sub-Indicator: MA04.05: An advisory/scientific committee, including external experts 16

is involved in the review of MA applications as necessary 17

Scope: All 18

Maturity: 4 19

Requirement: 20

Advisory/scientific committee, including external experts 21

Rationale: 22

The objective of this sub-indicator is to assure that an advisory/scientific committee, including external 23 experts, is involved in the review of the registration and/or marketing authorization applications. 24

Description: 25

The assessor should verify that the use advisory/scientific committee, including external experts in the 26 review of registration and/ or marketing authorization applications is documented and supported by 27 adequate regulations. The objectives and functions of the committee, as well as the professional profile of 28 the committee members should be available and the composition of the committee should be guided by 29 appropriate regulations. There should be clearly defined role/ responsibilities of each member of the 30 committee in relation to the review of registration and/or marketing authorization. Terms of reference for 31 each member of the committee or external expert should be prepared and documented. 32



1. The composition of the advisory/ scientific committee, and whether it is in line with the directive 35 prescribed in the regulations. 36


2. The objectives and functions of the committee and the individual members as well as that of the 1 external experts. 2

3. The terms of reference for committee and its members as well as that of the external experts. 3

4. The scope /extent of their advisory/scientific contributions to the registration and/or marketing 4 authorization activities of the NRA 5

5. SOPs for the review activities, as well as their programme of work. 6

6. Minutes of the meetings that result in registration and/or granting marketing authorization. 7

Framework: 8

Process 9

Rating Scale: 10

NI: The NRA does not have access to an advisory/scientific committee, including external experts, in 11 reviewing MA applications and /or providing scientific advice as necessary. 12

OI: The NRA has recently defined the criteria and composition of the advisory/ scientific committee, but 13 the advisory/ scientific committee has not yet been established. 14

PI: The NRA has established the advisory/ scientific committee less than two year. 15

I: The NRA has access to an advisory/scientific committee, including external experts, in the review of 16 MA applications that is supported with adequate documentation and records. 17

Limitations and remarks: The assessor should note that some NRAs may not involve the 18 advisory/scientific committee, including external experts in the review of the submitted registration 19 and/or marketing authorization applications, due to the availability of internal experts with the requisite 20 competency. Some NRAs may refer only certain applications that require specialist attention to the 21 committee or external experts for their expert advice. In this case the scoring is not applicable. 22


Add sub indicator No of related sub indicator to manage conflict of interest under this sub indicator 24

References: 25

Sub-Indicator: MA04.06: There are timelines for the assessment of the applications and 26

an internal tracking system is established to follow the targeted time frames 27

Scope: All 28

Maturity: 3 29

Requirement: 30

Timelines and internal control mechanisms 31

Rationale: 32


The objective of this sub-indicator is to assure whether timelines for the assessment of registration 1 /marketing authorization applications exist, and whether the timelines are monitored by internal tracking 2 systems to ensure compliance. 3

Description: 4

The assessor should verify that the documented timelines and internal timeline tracking systems that 5 guides the registration and/ or marketing authorization application processing exist and it is implemented. 6 It should be supported by guidelines that require that MA applications should be processed/ assessed per a 7 published timeline and that the published timeline should be monitored internally for compliance. The 8 guidelines should be supported with SOPs that provide guidance on how to establish the timelines and 9 how to monitor the established timelines. It should be designed to adapt to the various classes of medical 10 products registration and/or marketing authorization applications (e.g., innovator medicines, generic, 11 IVDs, vaccines, etc.). It should be designed to address each stage of the registration / marketing 12 authorization application process flow. The system should be developed to monitor the various timelines 13 for the various registration / marketing authorization application assessment procedures (i.e., the routine 14 and non-routine procedures). The assessor should note that some NRAs will have varied timelines for 15 registration / marketing authorization assessment routes, as well as internal tracking systems to ensure 16 adherence to the timelines. In this case, the assessor should review the timelines for each route and their 17 corresponding tracking systems 18


The assessor to request for and review: 20

1. The documented registration application process flow and corresponding timelines for the various 21 registration / marketing authorization application route and products class. 22

2. The internal tracking systems for monitoring adherence to the assessment timelines. 23

3. Whether the timelines are published, easily accessible and are known to all stakeholders including 24 the applicants and assessors, as well as whether they have been implemented, and are adhered to during 25 the assessment process. 26

4. Whether a dedicated unit and /or office is responsible for instituting/implementing and 27 monitoring the assessment timelines tracking system for the purposes for compliance, and the corrective 28 measures available in the event of non-compliance. 29

Framework: 30

Structure/Foundation/Input & Process 31

Rating Scale: 32

NI: There is no timelines for the assessment of the applications and an internal tracking system to follow 33 the targeted time frames 34

OI: The NRA has recently defined timelines for the assessment of the applications but an internal tracking 35 system to follow the targeted time frames has not been implemented. 36

PI: The NRA has defined the timelines for the assessment of the applications and it has been applying an 37 internal tracking system to follow the targeted time frames for less than two year. 38


I: The NRA has timelines for the assessment of the applications and an internal tracking system to follow 1 the targeted time frames is implemented. 2



MA03.01 5

References: 6

1. 1- WHO TRS 885, Annex 8, 1999 7

Sub-Indicator: MA04.07: There is a documented mechanism, with specific registration 8

requirements, for special situations (e.g. public-health interest) 9

Scope: All 10

Maturity: 3 11

Requirement: 12

Non-routine registration /marketing authorization applications 13

Rationale: 14

The objective of this sub-indicator is to assure a documented mechanism with specific registration 15 requirements exists, and it is applied in special situations (e.g. public-health interest). 16

Description: 17

The assessor should verify that regulations and /or guidelines allowing the use of a non-routine 18 mechanism supported with specific registration requirements exist and are implemented. The regulation 19 should be supported with guidelines and SOPs that clearly specify the registration requirements needed 20 and the instances (e.g. public health emergencies, orphan medicinal products, others) where an 21 application would be considered for specific pathway under public health interest such as emergency. In 22 addition, the registration and/or marketing authorization application process flow and their corresponding 23 specific timelines should be provided, published and implemented. The SOPs for the assessment process 24 should be developed to reflect the process flow and the specific requirements. 25



1. Regulation on the establishment and implementation of mechanism, with specific registration 28 requirements, for special situations for marketing authorization applications assessment and granting MA. 29

2. Guidelines on instances/special situations where this system applies, and the specific registration 30 requirements based on the nature of the products (new products, biological, generic, other). 31

3. The application sample that was process through this route for content/format relative to the 32 prescribed requirements and adherence to published timelines. 33

4. The registration and/or marketing authorization application process flow and their corresponding 34 timelines. The timelines should demonstrate timely manner review relative to the normal route/process. 35


5. SOPs that provide guidance for the assessment process. 1

Framework: 2

Process 3

Rating Scale: 4

NI: There is no documented mechanism, with specific registration requirements, for special situations 5 (e.g. public-health interest) 6

OI: The NRA has recently drafted or adopted a documented mechanism, with specific registration 7 requirements, for special situations (e.g. public-health interest) but it has not yet been applied. 8

PI: The NRA has developed or adopted a documented mechanism, with specific registration 9 requirements, for special situations (e.g. public-health interest), and has been applying for less than two 10 years or in less than three cases. 11

I: The NRA has a documented non-routine mechanism, with specific registration requirements, for special 12 situations (e.g. public-health interest) 13

Limitations and remarks: In case of no emergency to apply this requirement, the assessor should verify 14 the implementation of this sub-indicator by considering the existence of adequate resources and 15 procedures for implementation of the non-routine mechanism. 16


MA01.11 18

References: 19

1. 6- Regulation of vaccines: building on existing drug regulatory authorities. (WHO/V&B/99.10). 20

2. 3- WHO TRS 858, Annex 1, 1995 21

Sub-Indicator: MA04.08: SPC, labelling and packaging information are approved by 22

the NRA as part of the MA procedure 23

Scope: All 24

Maturity: 3 25

Requirement: 26

SPC, packaging, and labelling information 27

Rationale: 28

The objective of this sub-indicator is to assure that product information on SPC, packing and labelling are 29 approved by the NRA as part of the registration /marketing authorization application process. The NRA 30 should have clear requirements regarding medical products information that should be provided on the 31 SPC, packaging and labels. 32

Description: 33


The assessor should verify that the regulation /guidelines stating that product information on the SPC, 1 packaging and labelling should comply with the regulations provided to cover product information on 2 medical products. The information should be assessed as part of the registration and /or marketing 3 authorization application pack assessment process, and must be found to be appropriate and satisfactory. 4 The regulation should be supported by specific guidelines and /or SOPs for assessing product information 5 on SPC, packaging, and product labels 6



1. Regulation and /or guidelines stating that product information on SPC, packaging and labelling 9 should comply with the regulations provided to cover product information on medical products. The 10 information should be assessed as part of the registration /marketing authorization application pack 11 assessment process, and must be found to be appropriate and satisfactory. 12

2. Guidelines and /or SOPs for assessing product information on SPC, packaging, and product 13 labels. 14

3. The medical products registration and / or marketing authorization application assessment reports 15 to verify whether recommendations are provided by the assessor on product information on SPC, 16 packaging, and product labels, and how the information influenced the outcome of the application. 17

4. The registration and/ or marketing authorization application processing flow for the stage or 18 phase that require review of the product information on SPC, packing and labelling, and whether the 19 assessment process is conducted by trained staff with the requisite qualification. 20

Framework: 21

Process 22

Rating Scale: 23

NI: There is no requirement of approving SPC, labelling and packaging information by the NRA as part 24 of the MA procedure. 25

OI: The NRA has recently drafted or adopted the requirements to comply with the regulations provided to 26 cover product information on medical products such as SPC, labelling and packaging information, but it 27 has not yet been followed. 28

PI: The NRA has defined the requirements to comply with the regulations provided to cover product 29 information on medical products such as SPC, labelling and packaging information and has been applying 30 it for less than two year or it has not considered for all marketed medical products. 31

I: The NRA is approving SPC, labelling and packaging information as part of the MA procedure for all 32 reviewed medical products and is applied for all marketed medical products. 33



• MA01.12 36

• MA05.01 37


References: 1

1. 80-WHO TRS 902, Annex 9, 2002 2

2. 13-Ethical criteria for medicinal drug promotion. WHO, 1998 3

Sub-Indicator: MA04.09: GMP inspection report and/or certification is considered as 4

part of the MA process. 5

Scope: All 6

Maturity: 3 7

Requirement: 8

GMP inspection report and / certificate as part of MA process 9

Rationale: 10

The objective of this sub-indicator is to assure that GMP inspection or certification is part of the 11 registration and/ or marketing authorization requirements, and whether the NRA has a legal provision to 12 request the manufacturer to demonstrate adherence to GMP in the manufacture of the medical products, 13 either through on-site verification/inspection by the NRA and /or through a satisfactory assessment of 14 GMP documentation issued by other NRAs. 15

Description: 16

The assessor should verify that the regulation and /or guidelines on the type, content and format of GMP 17 documentation to submit to the NRA that is issued by another NRA, or the processes and procedure 18 through which an on-site inspection or verification will be conducted by the NRA is established and 19 implemented. The document should be supported by supplementary guidelines/SOPs that provide 20 guidance during the on-site visits inspection/verification visits or during assessment of the submitted 21 GMP documentation. Procedures /SOPs to ensure the GMP inspections and /or GMP documentation 22 verification are part of the medical products registration /marketing authorization process, and to ensure 23 that GMP requirements are published, easily accessible, are in existence and implemented as part of the 24 registration and/or marketing authorization application process. The assessor should note that in some 25 NRAs, the GMP on-site visit/inspection may not be carried out by NRA staff but by consultants hired by 26 the NRA to perform the task. In such instances, if the competences of the hired consultants are 27 appropriate, the scoring will apply. 28



1. Legal provisions that clearly state that compliance with the principles of GMP is part of the 31 medical products registration and/or marketing authorization application requirements. 32

2. Legal provisions that clearly describes the format and content of the GMP requirements, i.e. 33 either through a satisfactory on-site verification/inspection by the NRA and /or through a satisfactory 34 assessment of GMP documentation issued by other NRA. 35

3. Guideline/SOPs to provide guidance during the on-site visits inspection/verification visits or 36 during assessment of the submitted GMP documentation for the purposes for completeness. 37


4. Guidelines and /or regulation on the type, content and format of GMP documentation to submit to 1 the NRA that is issued by another NRA, or the processes and procedure through which an on-site 2 inspection or verification will be conducted by the NRA. 3

5. Procedures that ensure that the GMP inspections and /or the verification of submitted GMP 4 documentation is/ are part of the authorization process as well as procedures that ensure that GMP 5 requirements are implemented as part of the marketing authorization application process. 6

6. Whether GMP report and /or certification is a requirement for all registration /marketing 7 authorization applications and whether risk based routine/periodic inspection is required. 8

7. A record of registration and/or marketing authorization applications submission for the purposes 9 of verification and compliance. 10

Framework: 11

Structure/Foundation/Input & Process 12

Rating Scale: 13

NI: GMP inspection report and/or certification is not required as part of the MA process 14

OI: The NRA has recently defined GMP inspection report and/or certification as requirement for MA 15 authorization, but it has not yet been followed. 16

PI: The NRA has required GMP inspection report and/or certification as requirement for MA 17 authorization and has been applying it for less than two years. 18

I: GMP inspection report and/or certification is required by the NRA as part of the MA process, and have 19 been considered in all received MA applications. 20

Limitations and remarks: The assessor should verify that all requirements are same regardless of the 21 source of products. 22


References: 24

1. 32-WHO TRS 996, Annex 4, 2016 25

2. 51- WHO TRS 986, Annex 2, 2014 26

Sub-Indicator: MA04.10: The regulations and/or guidelines for good review practices 27

(GRevP) are developed or recognized and implemented. 28

Scope: All 29

Maturity: 3 30

Requirement: 31

Guidelines/SOP for GRevP 32

Rationale: 33


The objective of this indicator is to assure that regulations and/or guidelines exist for the NRA to 1 review/evaluate the different components of registration /marketing authorization applications, and to 2 review /evaluate the specific requirements of peculiar classes of products, and that registering and / or 3 granting marketing authorization or rejection outcomes /decisions follow documented procedures for the 4 different parts of the application. It will provide consistency in the approval or rejection of application 5 through the application of defined requirements and criteria so that the resultant decision would be seen to 6 be based on documentation submitted irrespective of the assessor. 7

Description: 8

The assessor should verify that procedures and documentation on GRevP are established and 9 implemented. It should be supported by an operational manual (i.e. documented and approved SOPs for 10 the screening and/ or review of the various classes of medical products and their corresponding 11 application routes (routine or non-routine)) for the screening and review of registration /marketing 12 authorization applications. The procedure should encompass some quality control and quality assurance 13 systems or structures to ensure that actual application assessment/ review is conducted in accordance with 14 approved SOPs. The procedure should provide guidance on how to handle medical products application 15 non-conformities. 16



1. Regulation and/or guidelines to provide guidance on principles of GRevP. 19

2. SOPs for assessment of registration and/or marketing authorization application to guide assessor 20 during the screening and/or review of the different components of the various classes of medical products. 21

3. Whether the same set of principles are applied during the screening and /or review of MA 22 applications submitted through the non-routine procedure. The assessor should request for SOPs that 23 provide guidance on how to handle actual review of application through the non-routine procedure. 24

4. Procedures in place to ensure that actual application assessment/ review are conducted in 25 accordance with the principles of GRevP and the assessment reports are peer-reviewed. The assessor 26 should request for a sample of the assessment report for a particular product class, and relate it to the 27 corresponding GRevP SOPs or similar documents for compliance. 28

Framework: 29

Process 30

Rating Scale: 31

NI: There are no regulations and/or guidelines for good review practices (GRevP) 32

OI: The NRA has recently drafted or adopted good review practices (GRevP) but it has not yet been 33 followed. 34

PI: The NRA has developed or adopted good review practices (GRevP) and has been applying it less than 35 two years or has not yet reflected to other relevant SOPs and guideline for MA activities. 36

I: The NRA has regulations and/or guidelines for good review practices (GRevP) and following them in 37 all relevant activities. 38




References: 3

1. 49- WHO TRS 992, Annex 9, 2015 4

Indicator: MA05 Mechanism exists to promote transparency, 5

accountability and communication. 6

Rationale: 7

The objective of this indicator is to assure that mechanisms are in place to ensure that information on 8 registration and/or marketing authorization applications, including authorized, suspended, rejected and 9 completed applications are published to promote transparency and information sharing amongst 10 stakeholders and potential stakeholder such as retailers and patients. This approach will intend build trust 11 among stakeholders involved. 12

Category: Transparency, accountability and communication 13

Sub-Indicator: MA05.01: Web site or other official publication with SPC-like 14

information is available and regularly updated 15

Scope: All 16

Maturity: 3 17

Requirement: 18

SPC-like information available 19

Rationale: 20

The objective of this sub-indicator is to assure the NRA has the legal provision or regulation to generate 21 and publish SPC-like information, and the practice is implemented. 22

Description: 23

The assessor should verify that the website or other official publications where SPC-like information is 24 published, are in existence and regularly updated. Guidelines and /or SOPs to provide guidance on 25 preparation with respect to the content and format of the SPC-like information to be published, as well as 26 the procedure/processes for the regular update (i.e., frequency and processes) should be documented and 27 implemented. The website to host the information should be clearly documented. 28



1. Whether the SPC-like information/ document is published, and if published whether the 31 publication is in the appropriate format per the published guidelines. 32

2. Whether the website is regularly updated to reflect the current situation. The assessor should 33 review the SOP for updating the website. 34


3. Whether the publications are easily accessible, where the information is hosted, either on the 1 NRA’s or the Ministry of Health website or others. If on another website, the assessor should request 2 SOPs that provide guidance on how to remotely upload new information onto the website and /or update 3 existing information. 4

Framework: 5

Output 6

Rating Scale: 7

NI: There is no evidence of official SPC-like information publication on website or other places. 8

OI: The NRA has recently defined official SPC-like information publication, but it has not yet been 9 published. 10

PI: The NRA has developed and initiated official SPC-like information publication but the information 11 are not updated regularly or all information has not reflected yet. 12

I: The NRA has official SPC-like information publication on website or other places for all marketed 13 medical products. 14

Limitations and remarks: The assessor should note that in some instances, the NRA may not physically 15 host the website. The website may be hosted by the MoH, professional bodies, standards authority, others. 16 In all instances, it is the NRA responsibility to ensure that SPC-like information is published and the 17 website is regularly updated. The scoring is applicable. 18


• MA01.12 20

• MA04.08 21

References: 22

1. 1- WHO TRS 885, Annex 8, 1999 23

Sub-Indicator: MA05.02: Updated list of all medical products granted ma is regularly 24

published and publicly available 25

Scope: All 26

Maturity: 3 27

Requirement: 28

List of all medical products registered and / or granted marketing authorization 29

Rationale: 30

The objective of this sub-indicator is to assure that guidelines and regulations exist to guide and advise 31 the NRA to regularly update and publish the list of medical products that are registered or granted 32 marketing authorization 33

Description: 34


The assessor should verify that the register for medical products registered and /or granted marketing 1 authorization is in existence. The list of registered and /or granted marketing authorization medical 2 products should be regularly updated and published. The published information should be supported with 3 SOPs that provided guidance on how to enter information, the type of information to enter (e.g. may 4 include but not limited to the product brand name, INN, the registration or marketing authorization 5 number, the registration or marketing authorization certificate number, the validity period of the 6 registration or marketing authorization, active ingredient, manufacturer and potency/concentration, 7 others) and how to translate the information into the appropriate format. The host or website where the 8 list is published should be documented and should be easily accessible to the general public. 9



1. The published list of medical products registered and/ or granted marketing authorization. 12

2. The published information on the registered medical product for the purposes of compliance with 13 guidelines or SOPs. 14

3. Whether the list is regularly updated. The assessor should verify whether the list is current. 15

4. SOP or similar documents for updating the list. 16

5. Whether an internal register (paper or electronic version) for entering registered medical products 17 exist, and if yes, the guidelines /SOPs for completing the register and regularly updating the register. 18

6. Ease of accessibility of the published list, and the identity of the host organization, i.e. NRA or 19 MoH. 20

Framework: 21

Output 22

Rating Scale: 23

NI: The updated list of all granted MA medical products are not regularly published and publicly 24 available. 25

OI: The NRA has recently committed to publish the list of all granted MA medical products, but it has not 26 yet been followed. 27

PI: The NRA has initiated publishing the list of all granted MA medical products, however it is nor 28 regularly updated, or the list does not reflect of all granted MA medical products. 29

I: The NRA has publicly published the list of all granted MA medical products and it is regularly updated. 30

Limitations and remarks: For different products medical streams may NRA publish different list. In 31 addition it should be easy to access as well. 32

The assessor should consider that existence of the database of all granted MA medical products is not 33 adequate to fulfil this requirement, since the purpose is transparency and then the database should be 34 publicly available. 35



• MA06.01 1

References: 2

1- WHO TRS 885, Annex 8, 1999 3

Sub-Indicator: MA05.03: A summary technical evaluation report for approved 4

registration marketing authorization applications is published and available to the public 5

Scope: All 6

Maturity: 4 7

Requirement: 8

Summary technical evaluation report (Public assessment report (PAR)) available to stakeholders 9

Rationale: 10

The objective of this sub-indicator is to assure that guidelines and/ or regulations exist to allow the NRA 11 to publish and make available to the public the summary technical evaluation reports of approved, MA 12 applications. The NRA actually publish and make available to the public the summary technical 13 evaluation reports. 14

Description: 15

The assessor should verify that the summary technical evaluation reports of medical products registration 16 applications that have been approved, are published and made available to the public, and this action 17 should be supported by a regulation and / or a guideline. The content and format of the report should be 18 guided by an SOP to ensure uniformity. Procedures should be in place to monitor the generation of the 19 reports to ensure that the content and format conforms to the approved standards, and the reports are 20 regularly published and made available to the public. 21



1. Procedures in place to support and monitor the generation of the reports, to ensure that the 24 content and format conforms to the approved standards, and to ensure that the reports are regularly 25 published and made available to the public. 26

2. A sample of the report for the purposes of conformance to the published guidelines and or SOPs. 27 The assessor should verify whether the report is generated, published and made available to the public. 28

3. The website for hosting the reports, and the SOPs that provide guidance to staff in the 29 performance of their duties with respect to uploading new reports and updating existing reports. 30

Framework: 31

Output 32

Rating Scale: 33

NI: There is no evidence of publishing and publicly availability of a summary technical evaluation report 34 for approved registration and/or marketing authorization applications. 35


OI: The NRA has recently defined this requirement, but it has not yet been implemented. 1

PI: The NRA has initiated publishing summary technical evaluation report for approved registration 2 and/or marketing authorization applications but has not been applying for all evaluated medical products 3 applications. 4

I: The NRA has published and made available to public a summary technical evaluation report for 5 approved registration and/or marketing authorization applications. 6

Limitations and remarks: The assessor should note that some NRAs will not publish the summary of 7 the technical evaluation report for medical product application processed under the emergency/ WHO 8 collaborative registration /Article 58 / orphan route (abridged registration application processing 9 route/pathway). 10


References: 12


2. 3- WHO TRS 858, Annex 1, 1995 14

Sub-Indicator: MA05.xx: A summary technical evaluation report for deferred or 15

rejected registration and/or marketing authorization applications is published and 16

available to the public 17

Scope: All 18

Maturity: 4 (Optional) 19

Requirement: 20

Summary technical evaluation report (Public assessment report (PAR)) available to stakeholders 21

Rationale: 22

The objective of this sub-indicator is to assure that guidelines and/ or regulations exist to allow the NRA 23 to publish and make available to the public the summary technical evaluation reports of deferred or 24 rejected MA applications. The NRA actually publishes and makes available to the public the summary 25 technical evaluation reports. 26

Description: 27

The assessor should verify that the summary technical evaluation reports of medical products registration 28 applications that have been deferred, or rejected is published and made available to the public, and this 29 action should be supported by a regulation and / or a guideline. The content and format of the report 30 should be guided by an SOP to ensure uniformity. Procedures should be in place to monitor the 31 generation of the reports to ensure that the content and format conforms to the approved standards, and 32 the reports are regularly published and made available to the public. 33




4. Procedures in place to support and monitor the generation of the reports, to ensure that the 1 content and format conforms to the approved standards, and to ensure that the reports are regularly 2 published and made available to the public. 3

5. A sample of the report for the purposes of conformance to the published guidelines and or SOPs. 4 The assessor should verify whether the report is generated, published and made available to the public. 5

6. The website for hosting the reports, and the SOPs that provide guidance to staff in the 6 performance of their duties with respect to uploading new reports and updating existing reports. 7

Framework: 8

Output 9

Rating Scale: 10

NI: There is no evidence of publishing and publicly availability of a summary technical evaluation report 11 for deferred or rejected registration and/or marketing authorization applications. 12

OI: The NRA has recently defined this requirement but it has not yet been implemented. 13

PI: The NRA has initiated publishing summary technical evaluation report for deferred or rejected 14 registration and/or marketing authorization applications but has not been applying for all evaluated 15 medical products applications. 16

I: The NRA has published and made available to public a summary technical evaluation report for 17 deferred or rejected registration and/or marketing authorization applications. 18

Limitations and remarks: Some NRAs may not publish the report of registrations applications that were 19 rejected or deferred based on the NRA’s legal provisions. In both cases, the scoring will not apply. 20 However, it is recommended that summary technical evaluation reports for all medical product 21 applications regardless of the evaluation outcome should be published and made available to the public. 22 In this case, the scoring should be yes. 23


References: 25


2. 3- WHO TRS 858, Annex 1, 1995 27

Indicator: MA06 Mechanism in place to monitor regulatory 28

performance and output 29

Rationale: 30

The objective of this indicator is to assure that mechanisms and /or procedures are in place to monitor 31 and ensure that all activities within the MA functionality (such as receipt of registration applications, 32 acknowledgement of registration applications, process/evaluation/assessment of the various parts of the 33 applications, generation of recommendations after evaluation/assessment process, 34 approval/deferral/rejection of registration applications, issuance of registration or marketing authorization 35 numbers or certificates, publication and making publicly available the SPC-like information and summary 36


technical evaluation reports ) are subjected to quality controls (QC) and other checks to reduce errors and 1 to rationally ensure that the processes, as well as the results at the various stages/phases of the registration 2 application processing flow are consistent in order to generate an assured output. This would lead to 3 consistency in the regulatory performance of the registration and/ or marketing authorization function as 4 well as reliable outputs. 5

Category: Monitoring progress and assessing outcomes & impact 6

Sub-Indicator: MA06.01: There is a database of all the product applications received, 7

approved, rejected, suspended and/or withdrawn along with their supporting 8

documentation. 9

Scope: All 10

Maturity: 3 11

Requirement: 12

Database of all medical products application 13

Rationale: 14

The objective of this sub-indicator is to assure regulations and /or guidelines exist to request the NRA to 15 keep (up-to-date list) and manage a database of the supporting documentation of all medical products 16 registration and/or marketing authorization applications received, approved, rejected, suspended and/or 17 withdrawn. Database would significantly contribute to consistency in regulatory approach as well as 18 expand institutional memory. 19

Description: 20

The assessor should verify that the database to keep all medical products registration applications 21 received, approved, rejected, suspended and/or withdrawn, as well as their supporting documentation is in 22 existence and should be constantly updated to reflect current situation. The operation and management of 23 database should be supported with guideline and /SOPs on the type of information or documentation to 24 keep, the format, and duration of storage. Personnel with authorization to access information should be 25 documented, as well as the procedure to access information. The location of the actual database should be 26 documented. Information on the back-up database should be provided. If not available, plans or effort to 27 secure a back-up system should be presented. 28



1. Whether a database exist for keeping up to date list of supporting documentation and information 31 (such as legal information and requirements, communications, application file, bases of decision, 32 others)on all medical products registration and/or marketing authorization applications received, 33 approved, rejected, suspended and/or withdrawn. 34

2. Guidelines and/ or SOPs on the essential information to keep on the database and for how long. 35

3. Whether the database is up to date or current, and it is supported by a back-up data base. 36

4. Authorization needed to access information, and update the database, as well as procedures to 37 monitor and reflect changes. 38


Framework: 1

Output 2

Rating Scale: 3

NI: There is no database of all the product applications received, approved, rejected, suspended and/or 4 withdrawn along with their supporting documentation. 5

OI: The NRA has recently initiated and drafted the requirement to create the database but it has not yet 6 been established. 7

PI: The NRA has established a database of all the product applications received, approved, rejected, 8 suspended and/or withdrawn along with their supporting documentation but all applications have not been 9 reflected in the database or is not regularly updated or fully integrated. 10

I: The NRA has a database of all the product applications received, approved, rejected, suspended and/or 11 withdrawn along with their supporting documentation. 12



• MA05.02 15

References: 16

Sub-Indicator: MA06.02: Performance indicators for registration and/ or market 17

authorization activities are established and implemented 18

Scope: All 19

Maturity: 4 20

Requirement: 21

Performance indicator. 22

Rationale: 23

The objective of this sub-indicator is to assure that a system/mechanism/procedure exist to compel the 24 NRA to establish performance indicators along the entire registration and /or marketing authorization 25 activities chain. The performance indicators should have been established and implemented for 26 monitoring and evaluation as well as continuous improvement. 27

Description: 28

The assessor should verify that the system establishing the performance indicators for the registration and 29 /or marketing authorization activities exist and is implemented. It should provide performance indicators 30 along the entire registration /marketing authorization activity chain, and adequately justify each and every 31 indicator. The guidelines should be supported by SOPs/tools that would be used to monitor and evaluate 32 the performance indicators, and provide guidance on when and how to revise /review the indicators. 33




1. The system/mechanism/procedure compelling the NRA to implement established performance 1 indicators along the entire registration /marketing authorization activity chain. 2

2. Whether the performance indicators have been established and implemented and whether the 3 members of staff involved in the registration and/or marketing authorization function are aware of the 4 indicators, as well as the guidelines/SOPs used to monitor and evaluate performance. 5

3. The performance indicators with corresponding justifications. 6

Framework: 7


Rating Scale: 9

NI: There are no performance indicators for registration and/ or marketing authorization activities. 10

OI: The NRA has recently drafted performance indicators for registration and/ or marketing authorization 11 activities but it has not yet been reported. 12

PI: The NRA has developed performance indicators for registration and/ or marketing authorization 13 activities and has been applying it for less than two year or has not covered all critical steps. 14

I: The NRA has established and implemented performance indicators for registration and/ or marketing 15 authorization activities and are regularly reviewed, and appropriate actions or decisions are taken. 16

Limitations and remarks: There are different methodologies to measure the NRAs performance on 17 registration and/ or marketing authorization activities. In this case the assessor should verify that adequate 18 supporting documents are available. The assessor should consider that developed KPI should be 19 (SMART). 20


References: 22

1. 115- ISO 9004: 2009 23

WHO GLOBAL BENCHMARKING TOOL (GBT) FOR …...1 2 WHO GLOBAL BENCHMARKING TOOL (GBT) FOR 3 EVALUATION...

Documents

Transcript of WHO GLOBAL BENCHMARKING TOOL (GBT) FOR …...1 2 WHO GLOBAL BENCHMARKING TOOL (GBT) FOR 3 EVALUATION...