Data Collection Tools 27- Functions, Indicators & A. National regulatory system Indicator RS1: Legal framework for establishment of a regulatory system, mandate and enforcement power for each function Sub-Indicator RS1.1: Legislation defines the medicinal products for human use to be regulated Sub-Indicator RS1.2: Legislation defines the institutions involved in the vaccine regulatory system, their mandate, functions, roles, responsibilities and enforcement power Indicator RS2: Independence of the regulatory authority in decision making Sub-Indicator RS2.1: Independence of NRA from researchers, manufacturers, distributors and wholesalers Sub-Indicator RS2.2: Independence of NRA from procurement system / institutions Sub-Indicator RS2.3: Mechanism to manage conflicts of interest Indicator RS3: Recall system with mechanism to ensure proper disposition / disposal of affected lots Sub-Indicator RS3.1: Legal basis for NRA to take actions on recall, suspension, withdrawal and / or destruction Sub-Indicator RS3.2: System based on documented communication to appropriate level of the distribution system with feedback mechanism Sub-Indicator RS3.3: Mechanism for written confirmation that appropriate, batch traceable, action and / or destruction when necessary has been taken Indicator RS4: Commitment of management to implement a quality management system relevant to all regulatory functions 1 August 21 Page 1 of 23
Data Collection Tools 27- Functions, Indicators & A. National regulatory system Indicator RS4: Commitment of management to implement a quality management system relevant to all regulatory functions Sub-Indicator RS4.1: Policy statement and development plan to implement a quality management system for all regulatory functions Sub-Indicator RS4.2: Financial support to implement a quality management system Sub-Indicator RS4.3: Human resources designated and assigned to implement a quality management system Indicator RS5: Transparency and public accountability Sub-Indicator RS5.1: Information related to legislation and regulations is publicly available Sub-Indicator RS5.2: Information on decisions is accessible to the public Sub-Indicator RS5.3: Information on marketed products and authorised companies is publicly available Sub-Indicator RS5.4: Information on license withdrawal, sanctions, recalls and public health warnings is publicly available Sub-Indicator RS5.5: All publicly available information is kept up-to-date Sub-Indicator RS5.6: Appropriate mechanism for management of confidential information / material Sub-Indicator RS5.7: Code of conduct (including conflicts of interest) published and enforced Indicator RS6: Institutional development plan Sub-Indicator RS6.1: Plan developed, implemented and regularly updated Sub-Indicator RS6.2: Performance indicators established and used for monitoring progress 1 August 21 Page 2 of 23
Data Collection Tools 27- Functions, Indicators & A. National regulatory system Indicator RS6: Institutional development plan Sub-Indicator RS6.3: Training plan developed, implemented and updated Indicator RS7: Advisory / experts committees Sub-Indicator RS7.1: Documented procedures for the establishment and management of advisory / expert committees 1 August 21 Page 3 of 23
Data Collection Tools 27- Functions, Indicators & Function 1: Marketing authorization and licensing activities Indicator MA1: System for marketing authorization and licensing for manufacturing activities Sub-Indicator MA1.1: Provision for marketing authorization and licensing for manufacturing activities Sub-Indicator MA1.2: Marketing authorization required for all products for which NRA is being assessed Sub-Indicator MA1.3: License to manufacture granted on the basis of demonstrated compliance with GMP Indicator MA2: Provision for licensing for importing, exporting and distributing activities Sub-Indicator MA2.1: License to import, export and distribute granted on the basis of demonstrated compliance with GDP Indicator MA3: Quality Management System for marketing authorization and license activities Sub-Indicator MA3.1: Defined organizational chart and responsibilities to implement the Quality Management System Sub-Indicator MA3.2: Written documentation for performing marketing authorization and manufacturing license activities Sub-Indicator MA3.3: Written documentation for performing license activities for importers, exporters, wholesalers, distributors Sub-Indicator MA3.4: Management system to ensure traceability of actions (marketing authorization and manufacturing license dossiers and updates, assessment reports, meeting reports, variations etc.) 1 August 21 Page 4 of 23
Data Collection Tools 27- Functions, Indicators & Function 1: Marketing authorization and licensing activities Indicator MA3: Quality Management System for marketing authorization and license activities Sub-Indicator MA3.5: Management system to ensure traceability of actions of licensing of importers, exporters, wholesalers, distributors (dossiers and updates, assessment reports, meeting reports, variations etc.) Sub-Indicator MA3.6: Auditing system implemented and documented (internal and/or external) Indicator MA4: Human resource management Sub-Indicator MA4.1: Adequately qualified staff (education, training, skills and experience) or access to experts for assessment of quality, pre/non-clinical and clinical data Sub-Indicator MA4.2: Written procedures for selection and use of external experts Sub-Indicator MA4.3: Staff training plan developed and implemented Sub-Indicator MA4.4: Monitoring skills development after training activities Indicator MA5: Submission of Marketing authorization and manufacturing license applications Sub-Indicator MA5.1: Administrative instructions available for applicants Sub-Indicator MA5.2: Requirements on the format and content for submission of marketing authorization applications consistent with global standards available to applicants Sub-Indicator MA5.3: Requirements on the format and content for submission of manufacturing license applications (including format of the authorization) available to applicants 1 August 21 Page 5 of 23
Data Collection Tools 27- Functions, Indicators & Function 1: Marketing authorization and licensing activities Indicator MA6: Assessment of marketing authorization and manufacturing license application Sub-Indicator MA6.1: Assessment of product related information (legal part and quality) Sub-Indicator MA6.2: Assessment of clinical data (safety & efficacy) Sub-Indicator MA6.3:Assessment report prepared and used as reference for decision Sub-Indicator MA6.4: Policy with written criteria for recognition of other NRA s reports/decisions (if applicable) Sub-Indicator MA6.5: Assessment of the manufacturing license application Sub-Indicator MA6.6: Assessment of the import, export, wholesale, distribution license application Indicator MA7: GMP assessment during marketing authorization process Sub-Indicator MA7.1: GMP assessment for domestic manufacturers through dedicated inspections on the manufacture site Sub-Indicator MA7.2: GMP assessment for non-domestic manufacturers Indicator MA8: Requirements for variations to be submitted and assessed Sub-Indicator MA8.1: Written guidelines for applicants with definition of types and scopes of variations and documentation required Sub-Indicator MA8.2: Written guidelines for assessment based on type of variation Indicator MA9: Clear and comprehensive information on authorized products 1 August 21 Page 6 of 23
Data Collection Tools 27- Functions, Indicators & Function 1: Marketing authorization and licensing activities Indicator MA9: Clear and comprehensive information on authorized products Sub-Indicator MA9.1: NRA approved Summary of Product Characteristics (SPC)-like information available for all products (as basis for lot release) Sub-Indicator MA9.2: Web site or other official publication with SPC-like information is available and regularly updated Indicator MA1: Same criteria/standards for evaluation of marketing authorization applications for products regardless of the source Sub-Indicator MA1.1: Written criteria to be applied Sub-Indicator MA1.2: Written criteria to cover circumstances in which the routine licensing procedures may not have to be followed 1 August 21 Page 7 of 23
Data Collection Tools 27- Functions, Indicators & Function 2: Post-marketing activities including surveillance of adverse events following immunization (AEFI) Indicator PM1: Institutional regulations and guidelines for post-marketing surveillance including monitoring and management of adverse events following immunization (AEFI) Sub-Indicator PM1.1: Provisions for monitoring and management of adverse events following immunization (AEFI) Sub-Indicator PM1.2: Guidelines exist, are published and accessible (i.e. distributed or available when needed) to all staff involved in AEFI surveillance Sub-Indicator PM1.3: Provisions for the NRA to require the marketing authorization holder to perform a specific study of safety in the post-marketing period as necessary Sub-Indicator PM1.4: Requirements exist for manufacturer to inform NRA of any new safety issue or marketing / regulatory decisions taken in other countries based on safety and is enforced by NRA Indicator PM2: Quality Management System for post-marketing activities Sub-Indicator PM2.1: Defined organizational chart and responsibilities Sub-Indicator PM2.2: Written documentation for performing post marketing surveillance activities Sub-Indicator PM2.3: Management system to ensure traceability of post marketing activities Sub-Indicator PM2.4: Auditing system documented and implemented (external and or internal) Indicator PM3: Roles and responsibilities of the key players (immunization staff, NRA, NCL, surveillance staff, etc) Sub-Indicator PM3.1: Clearly defined and documented roles and responsibilities for NRA, NCL, national immunization program and, where relevant, disease surveillance and pharmacovigilance staff involved in AEFI monitoring and management activities 1 August 21 Page 8 of 23
Data Collection Tools 27- Functions, Indicators & Function 2: Post-marketing activities including surveillance of adverse events following immunization (AEFI) Indicator PM3: Roles and responsibilities of the key players (immunization staff, NRA, NCL, surveillance staff, etc) Sub-Indicator PM3.2: Official instructions for key players to conduct AEFI surveillance activities Indicator PM4: Human resource management Sub-Indicator PM4.1: Adequate qualified staff (education, training, skills and experience) to perform post marketing surveillance activities Sub-Indicator PM4.2: Staff training developed and implemented, including routine training/information on AEFI monitoring and management Sub-Indicator PM4.3: Monitoring skills development after training activities Indicator PM5: Routine and functional system for regular review of safety and efficacy of the vaccine product for regulatory action, including a process to review and share relevant data between key players and taking appropriate action Sub-Indicator PM5.1: AEFI data compiled and analysed/interpreted on regular (e.g. monthly) basis Sub-Indicator PM5.2: Information on serious cases and AEFI clusters and investigation reports shared between NRA, NCL, national immunization program, and disease surveillance and pharmacovigilance staff Sub-Indicator PM5.3: Formal/official communication and regular meetings among above-mentioned key players when dealing with AEFI Sub-Indicator PM5.4: Access to a relevant standing committee or group of experts for the assessment of AEFI cases 1 August 21 Page 9 of 23
Data Collection Tools 27- Functions, Indicators & Function 2: Post-marketing activities including surveillance of adverse events following immunization (AEFI) Indicator PM5: Routine and functional system for regular review of safety and efficacy of the vaccine product for regulatory action, including a process to review and share relevant data between key players and taking appropriate action Sub-Indicator PM5.5: Manufacturers are notified of significant safety and efficacy issues and kept up to date or/and at upon request Sub-Indicator PM5.6: Process to review/assess AEFI and initiate appropriate action including at the subnational level, when needed Indicator PM6: Capacity to detect and investigate significant vaccine safety issues Sub-Indicator PM6.1: Demonstrated capacity of the reporting system (active or passive, sentinel or countrywide/statewide) with satisfactory sensitivity Sub-Indicator PM6.2: Documented evidence of appropriate investigations or sufficient elements indicative of capacity to investigate Sub-Indicator PM6.3: Documented evidence of timely investigations Sub-Indicator PM6.4: Documented process and/or activities to promote timeliness of reporting and completeness of information collected Indicator PM7: Regulatory outcome regarding vaccine performance Sub-Indicator PM7.1: Evidence that corrective/regulatory action (e.g. recall, update of product leaflet, etc) are taken in case of deviation/non compliance Sub-Indicator PM7.2: NRA regularly informed of data relevant to ongoing assessment of vaccine performance 1 August 21 Page 1 of 23
Data Collection Tools 27- Functions, Indicators & Function 2: Post-marketing activities including surveillance of adverse events following immunization (AEFI) Indicator PM8: System for providing feedback on AEFI from the national to all levels Sub-Indicator PM8.1: Periodic (quarterly or yearly) feedback on AEFI including summary and specific investigation reports Sub-Indicator PM8.2: Process is established for feedback down to health facility level Sub-Indicator PM8.3: Process is established for feedback to public/community/patients/parents 1 August 21 Page 11 of 23
Data Collection Tools 27- Functions, Indicators & Function 3: NRA Lot release Indicator LR1: System for lot release Sub-Indicator LR1.1: Provision for lot release including the responsible officer to sign the regulatory lot release certificate Sub-Indicator LR1.2: Lot release applied to all vaccines Sub-Indicator LR1.3: Based as a minimum on summary lot protocol review Sub-Indicator LR1.4: Mandatory summary lot protocol as part of licensing requirements Sub-Indicator LR1.5: Acceptance policy/criteria of lot release performed by another NRA (eg. lot release certificate from the country of origin) Sub-Indicator LR1.6: Testing policy including frequency Indicator LR2: Quality Management System for lot release Sub-Indicator LR2.1: Defined organizational chart and responsibilities to implement the Quality Management System Sub-Indicator LR2.2: Written documentation for performing lot release activities available Sub-Indicator LR2.3: Management system to ensure traceability of lot release activities Sub-Indicator LR2.4: Auditing system documented and implemented (external and or internal) Indicator LR3: Human resource management Sub-Indicator LR3.1: Adequately qualified staff (education, training, skills and experience) to perform lot release Sub-Indicator LR3.2: Staff training plan developed and implemented 1 August 21 Page 12 of 23
Data Collection Tools 27- Functions, Indicators & Function 3: NRA Lot release Indicator LR3: Human resource management Sub-Indicator LR3.3: Monitoring skills development after training activities Sub-Indicator LR3.4: Adequate number of staff to implement the workplan Indicator LR4: Lot release management process Sub-Indicator LR4.1: Defined acceptance criteria for lot release Sub-Indicator LR4.2: Provision for exemption from lot release Indicator LR5: Access to product related documentation to guide particular areas of scrutiny in lot release Sub-Indicator LR5.1: Approved relevant parts of marketing authorization file and updates available to NRA staff involved in lot-release (e.g. variations) Sub-Indicator LR5.2: Access to summary of product characteristics (SPC) and reports when necessary (including GMP inspection, laboratory data, quality defects, vaccine performance issues and AEFI) Indicator LR6: Monitoring and data analysis for lot release Sub-Indicator LR6.1: Analysis of lot-to-lot consistency conducted Sub-Indicator LR6.2: Corrective action taken in case of deviation Sub-Indicator LR6.3: Regulatory action taken in case of non compliance Sub-Indicator LR6.4: Follow up and communication with involved parties including the manufacturer on issues of data quality 1 August 21 Page 13 of 23
Data Collection Tools 27- Functions, Indicators & Function 4: Laboratory access Indicator LA1: System for Quality Control (QC) Sub-Indicator LA1.1: National Control Laboratory (NCL) established or operational agreement to use external QC laboratory Sub-Indicator LA1.2: Clearly defined responsibilities for scientific/technical input during the pre and post licensing period Sub-Indicator LA1.3: NCL or QC lab involved in definition of specifications and analytical methods during assessment of MA Indicator LA2: Quality Management System for laboratory access Sub-Indicator LA2.1: Defined organizational chart and responsibilities to implement the Quality Management System Sub-Indicator LA2.2: Written documentation for performing testing activities Sub-Indicator LA2.3: Management system to ensure traceability of activities Sub-Indicator LA2.4: Auditing system documented and implemented (external and or internal) Indicator LA3: Human resource management Sub-Indicator LA3.1: Adequate qualified staff (education, training, skills and experience) to perform NCL activities Sub-Indicator LA3.2: Staff training plan developed and implemented Sub-Indicator LA3.3: Monitoring skills development after training activities Sub-Indicator LA3.4: Adequate number of staff to implement the workplan 1 August 21 Page 14 of 23
Data Collection Tools 27- Functions, Indicators & Function 4: Laboratory access Indicator LA4: Testing procedures and related documentation Sub-Indicator LA4.1: Standard Operating Procedures (SOPs) for performing testing activities Sub-Indicator LA4.2: SOPs for handling out-of-specification results including a retest policy Sub-Indicator LA4.3: Specifications and validity criteria are defined for all tests Indicator LA5: Building and equipment Sub-Indicator LA5.1: Building and premises are adequate to host NCL activities Sub-Indicator LA5.2: There are operational manuals, SOPs and log books (records of use /maintenance /calibration) Sub-Indicator LA5.3: There is a calibration and maintenance plan Sub-Indicator LA5.4: There are qualifications protocols and reports Indicator LA6: Validation policy Sub-Indicator LA6.1: Validation master plan exist, non-compendial tests have been validated and compendial tests have been verified Sub-Indicator LA6.2: Procedures for transfers of validated methods (e.g. from manufacturer) Indicator LA7: Safety programme Sub-Indicator LA7.1: The list of hazardous substances is available Sub-Indicator LA7.2: The responsible staff is designated for the safety program's management 1 August 21 Page 15 of 23
Data Collection Tools 27- Functions, Indicators & Function 4: Laboratory access Indicator LA7: Safety programme Sub-Indicator LA7.3: There is a procedure for the storage, handling and disposal for hazardous substances Sub-Indicator LA7.4: Staff immunization requirements are defined and followed Indicator LA8: Policy for use of reference standards and reagents Sub-Indicator LA8.1: System in place to establish and qualify national reference standards (in IU if appropriate) Sub-Indicator LA8.2: Appropriate handling and use of reference standards/materials Sub-Indicator LA8.3: Appropriate handling and use of reagents of assured quality Sub-Indicator LA8.4: Regular supply system for reference standards/materials Indicator LA9: Monitoring, analysis and regulatory outcome of laboratory results Sub-Indicator LA9.1: Compliance against specifications is systematically checked Sub-Indicator LA9.2: Corrective action taken in case of deviation Sub-Indicator LA9.3: Regulatory action taken in case of non compliance Sub-Indicator LA9.4: Monitoring and analysis of data trends for laboratory results Sub-Indicator LA9.5: Monitoring and analysis of data trends for reference standards/materials Sub-Indicator LA9.6: Comparison of results with those of manufacturer 1 August 21 Page 16 of 23
Data Collection Tools 27- Functions, Indicators & Function 4: Laboratory access Indicator LA1: Participation in international proficiency schemes, collaborative studies and/or inter-laboratory comparisons Sub-Indicator LA1.1: Evidence of regular participation Sub-Indicator LA1.2: Appropriate performance in proficiency schemes, collaborative studies and/or inter-laboratory comparisons 1 August 21 Page 17 of 23
Data Collection Tools 27- Functions, Indicators & Function 5: Regulatory inspections Indicator RI1: System for regulatory GMP inspections Sub-Indicator RI1.1: National GMP code equivalent with WHO GMP (TRS 92, 98, 822 or any update) or other recognized standards and published Sub-Indicator RI1.2: GMP inspection conducted for manufacture/facility licensing Sub-Indicator RI1.3: Mandate to inspect and collect samples at any vaccine manufacturing facility in the country Sub-Indicator RI1.4: Authority to access any premises and documents that are relevant for the inspection Sub-Indicator RI1.5: Enforcement power to suspend or stop production in any vaccine manufacturing facility in the country Sub-Indicator RI1.6: Provision (or written criteria) for recognition of other Inspectorates (NRAs) GMP certificates/reports/decisions (if applicable) Indicator RI2: Quality Management System for regulatory inspectorates Sub-Indicator RI2.1: Defined organizational chart and responsibilities to implement the Quality Management System Sub-Indicator RI2.2: Written documentation for performing inspection and relating activities Sub-Indicator RI2.3: Management system to ensure traceability of activities Sub-Indicator RI2.4: Auditing system documented and implemented (external and or internal) Indicator RI3: Human resource management Sub-Indicator RI3.1: Adequately qualified staff (education, training, skills and experience) to perform inspections and related activities 1 August 21 Page 18 of 23
Data Collection Tools 27- Functions, Indicators & Function 5: Regulatory inspections Indicator RI3: Human resource management Sub-Indicator RI3.2: Staff training plan developed and implemented Sub-Indicator RI3.3: Monitoring skills development after training activities Indicator RI4: Code of practices for regulatory inspection Sub-Indicator RI4.1: SOPs for conducting inspections Sub-Indicator RI4.2: Plan for inspections at appropriate intervals and based on quality risk management Sub-Indicator RI4.3: Reports of inspections available for all types of GMP inspections Sub-Indicator RI4.4: Use of team approach to ensure expertise in specific products Sub-Indicator RI4.5: Conflict of interest & confidentiality provisions for all external team members Indicator RI5: Monitoring, analysis and regulatory outcome of inspections Sub-Indicator RI5.1: Evidence that follow up process is documented and implemented Sub-Indicator RI5.2: Evidence that regulatory action is taken in case of non-compliance Indicator RI6: Inspection of distribution channels Sub-Indicator RI6.1: Provision for monitoring onward distribution as appropriate Sub-Indicator RI6.2: National GDP code equivalent with WHO GDP (TRS 937 or any update) and published Sub-Indicator RI6.3: Only authorized products are distributed 1 August 21 Page 19 of 23
Data Collection Tools 27- Functions, Indicators & Function 5: Regulatory inspections Indicator RI6: Inspection of distribution channels Sub-Indicator RI6.4: Mandate to inspect and to collect samples at any point of the distribution chain of vaccines in the country 1 August 21 Page 2 of 23
Data Collection Tools 27- Functions, Indicators & Function 6: Regulatory oversight of clinical trials Indicator CT1: System for regulatory oversight of clinical trials (CTs) Sub-Indicator CT1.1: Provision for regulation of clinical trials consistent with WHO Good Clinical Practices (GCP) Sub-Indicator CT1.2: Roles and responsibilities of all stakeholders/institutions involved in CTs are defined Sub-Indicator CT1.3: Mandate and enforcement power for NRA to inspect, suspend or/and stop CT Sub-Indicator CT1.4: Regulatory authorization for importation-release of investigational products Sub-Indicator CT1.5: GCP inspection system established and implemented Sub-Indicator CT1.6: Adverse events during CT reported Sub-Indicator CT1.7: Feedback report from sponsors or contract research organizations (CROs) Indicator CT2: Quality Management System for oversight of clinical trials Sub-Indicator CT2.1: Management system to ensure traceability of actions Indicator CT3: Human resource management Sub-Indicator CT3.1: Adequate qualified staff (education, training, skills and experience) or mechanism to ensure it Sub-Indicator CT3.2: Staff training plan developed and implemented Sub-Indicator CT3.3: Monitoring skills development after training activities Indicator CT4: Format and content for submission of clinical trials application 1 August 21 Page 21 of 23
Data Collection Tools 27- Functions, Indicators & Function 6: Regulatory oversight of clinical trials Indicator CT4: Format and content for submission of clinical trials application Sub-Indicator CT4.1: Guidelines on the format and content for submission of clinical trial application available to the applicants Sub-Indicator CT4.2: Standard Operating Procedures (SOPs) for submission of CTs application and records established Sub-Indicator CT4.3: Meetings to advise sponsors and/or principal investigators to meet regulatory requirements Indicator CT5: Scientific review of clinical trials application Sub-Indicator CT5.1: Clinical candidate material manufactured in compliance with WHO GMP requirements for investigational products and available quality data submitted Sub-Indicator CT5.2: Preclinical data submitted for clinical candidate material Sub-Indicator CT5.3: Appropriate assessment of CT protocol with respect to product and patient safety Indicator CT6: Assurance of ethical oversight Sub-Indicator CT6.1: Defined roles for Ethics Committees (ECs) at all levels Sub-Indicator CT6.2: EC responsibility for clearance and follow-up until completion of clinical trial Sub-Indicator CT6.3: Appropriate composition of Ethics Committees 1 August 21 Page 22 of 23
Data Collection Tools 27- Functions, Indicators & 1 August 21 Page 23 of 23