Topic 5: Competency Authors: Jelena Ivanovic; Marco Di Girolamo (Version 4 June 2015)

Transcription

1 SCOPE Work Package 8 - LIFECYCLE PHARMACOVIGILANCE Survey Report Topic 5: Competency Authors: Jelena Ivanovic; Marco Di Girolamo (Version 4 June 2015)

2 1 Introduction 1.1 Purpose of the document The purpose of this document is to report the outcome of survey for topic 5 Competency in the context of Work Package 8 Life cycle pharmacovigilance. Topic lead IT, active contributors IE, UK. This report summarizes the current pharmacovigilance assessors competency levels, organizational matters relative to assessment procedures in different MSs and working conclusions on the core competency requirements for the profession of pharmacovigilance assessors. This report concerns some general competency aspects, while procedure specific competency issues have been addressed in WP8 topics 2,3 and 4 (RMP, PASS, PSUR/PSUSA and referral procedure assessment). 1.2 Document Revision History Version Revision date Comments Authors 1 23/02/2015 NO 16/03/15 UK 12/03/15 IE 13/03/15 Jelena Ivanovic Marco Di Girolamo 2 14/04/2015 ES 21/04/2015 NO 21/04/2015 SE 24/04/2015 Jelena Ivanovic Marco Di Girolamo IE 24/04/2015 UK 24/04/ /05/2015 Jelena Ivanovic Marco Di Girolamo 4 09/06/2015 ES 05/06/2015 Jelena Ivanovic Marco Di Girolamo 1.3 Definitions and abbreviations Terminology ADR ATC B/R CAP CPD DIA EMA EU ISOP MS Description Adverse Drug Reaction Anatomical Therapeutic Chemical classification system Benefit/risk Centralised authorisation procedure Continuing Professional Development Develop Innovate Advance European Medicines Agency European Union International Society of Pharmacovigilance Member State 2

3 NAP NCA PASS PSUR PV RMMs RMP SOP UMC WHO WP National authorisation procedure National Competent Authority Post-authorisation safety studies Periodic Safety Update Report Pharmacovigilance Risk Minimisation Measures Risk Management Plan Standard Operating Procedure Uppsala Monitoring Centre World Health Organization Work Package 1.4 Attachments Document name Annex 1 : Questionnaire topic5.pdf Annex 1 link Annex 2: Table of training courses Annex2.docx Annex 2 link Annex 3 : List of useful literature Annex3.docx Annex 3 link 3

4 1.5 Index 1 Introduction Purpose of the document Document Revision History Definitions and abbreviations Attachments Annex 1 : Questionnaire Annex 2: Table of training courses Annex 3 : List of useful literature Index Project preconditions Background Related projects Main goal and objectives Main goal Objectives Results Questions 1-5 concerning organization Question 6-8 concerning assessors education and experience Question 9-14 concerning training policy, training materials and other useful tools for PV assessors competency Discussion and Conclusions Preliminary recommendation

5 1.6 Project preconditions Background In the context of the effective operation of the PV system in the EU, improvement of B/R assessment through lifecycle of medicines and harmonization of procedures in Member States is one of the most important aspects of the PV legislation 1 came into force in July 2012 (system implementation). In areas of health care, including PV, professionals are expected to have attained an appropriate level of competence, that when combined with their past experience and knowledge base, enables them to make decisions and take valid and efficient regulatory actions. Assessors should be competent to provide high-quality assessments including the B/R decision-making which is a complex and challenging task. It is not only restricted to evaluation of safety data, but in the process of assessment and decision making, other aspects must be considered as well (e.g. handling data heterogeneity, lack of data and/or long term outcomes in different time horizons, consistency across assessments, flexibility and ability to deal with uncertainty, ability to work in teams and to adhere to proposed timeframes, good coordination and collaboration with other participants and stakeholders in the process etc. ). Usually assessors have different backgrounds with degrees in pharmacy, medicine or other life-sciences with different experiences (including the pharmaceutical industry, contract research organizations, clinical and general practice, academia). Moreover, they are requested to develop and/or update their regulatory knowledge and competencies across a number of therapeutic areas. It is expected that there is no consistent difference in applied knowledge and assessment methodology among various European settings, however the description of the current situation in different MSs and identification of the best assessors skills and useful procedures aimed to ensure and maintain good level of assessors knowledge could be of particular interest in promotion of high-quality and harmonized assessments. Continuous education (e.g. trainings, courses, workshops, exchanges programs), availability of specific training materials (e.g. PV textbooks and scientific papers) and a curriculum in PV are key factors for the development of competency in assessment. In the context of the WP8 the appropriate training program will be identified based on the responses of all MSs to the surveys in order to help NCAs to integrate, enhance the competence and knowledge of assessors. 1 Directive 2010/84/EU and Directive 2012/26/EU amending Directive 2001/83/EC, Regulation (EU) No 1235/2010 and Regulation (EU) No 1027/2012 amending Regulation (EC) No 726/2004 and in the Commission Implementing Regulation (EU) No 520/2012. Consolidated versions of the Directive 2001/83/EC and Regulation (EC) No 726/2004 are available in EUR-Lex. 5

6 1.6.2 Related projects The work and results of topic 5 on competency could contribute to the topics 2, 3 and 4 on assessment and processes for RMPs, PASS and for PSURs and referral procedures. The findings from the WP8 topic 5 could be connected with deliverables from other SCOPE WPs, in particular linkage with WP5 - Signal Management and WP7 - Quality Management Systems could be of interest. The linkage could be obtained in the context of the following aims: for WP5 - implementation of shared understanding of good operating in signal management across the EU network; for WP7- enabling Member States to understand and assess their quality management systems for PV and to provide a report on the impact of new PV legislation on quality management systems. Currently, some other projects at EU level with complementary tasks have addressed different aspects of the PV competency framework. The EU Network Training Centre ( is aimed to create a European central platform for exchange of information and supply of regulatory and scientific training across the EU regulatory network to assure quality and to promote the common standards for assessment. The EU Network Training Centre is committed on improvement of the quality, consistency and efficiency of the work of the European Regulatory Network by promoting the harmonised application of the regulatory framework and guidelines and fostering scientifically based, pragmatic and consistent assessments, inspections and PV as well as decision making. It also will provide continuous professional development for regulatory agencies and possibly other stakeholders involved in development of regulation of medicines. Another European initiative which introduces innovative approaches in PV assessors competency framework is The Pharmacoepidemiological Research on Outcomes of Therapeutics by a European Consortium (PROTECT) project ( ). It is a public-private partnership for innovative methodologies in PV and pharmacoepidemiology coordinated by the European Medicines Agency, which has reached a crucial stage with the delivery of two databases that will offer access to important data resources for PV activities and pharmacoepidemiological studies. The goal of PROTECT is to strengthen the monitoring of the B/R of medicines in Europe by developing innovative methods such as enhancement of early detection of signals and assessment of ADRs from different data sources (clinical trials, spontaneous reporting and observational studies), enabling the integration and presentation of data on benefits and risks. 2 Main goal and objectives 2.1 Main goal 6

7 The overall goals of the work in topic 5 are to produce a status report on the existing competency levels and organizational matters in NCAs at present, to exchange experience on core skills and good practices including list of useful trainings and literature. These aspects will be promoted through a WP8 training package for assessors in order to further improve the quality, consistency and efficiency of the work of the European PV Network. 2.2 Objectives To describe competency levels and types of organization in the context of PV procedures in NCAs at present; To identify core assessor skills important for B/R assessment; To identify good practices recommended by NCAs as useful for promotion of PV procedures assessment; To promote and disseminate identified core skills, good practices and knowledge. 3 Results The survey on competency was conducted between July and November In total 25 NCAs responded and 25 completed the questionnaire although not all questions have been answered by all NCAs. When the percentage is presented below the denominator is 25 unless otherwise noted. 3.1 Questions 1-5 concerning organization Most NCAs reported that PV assessors activities in their units are organized on type of procedure basis (e.g. referrals, RMPs, PSURs etc.) and Anatomical Therapeutic Class (ATC code) basis (44% and 40% respectively). The limitation of the organization on procedural basis, reported by one NCA, is that PV and (pre-)clinical assessors work in different departments, which could complicate the communication and organization during B/R assessment. When activities are organized on the bases of ATC code, member states report that the assignment of procedures is easier and continuous monitoring and historic knowledge for safety issues is ensured. The principal limitations of this type of organization are seen in situations when the assessment of different procedures for the same ATC code products need to be assessed simultaneously. In addition, when an assessor is specialized in one ATC code, there may be limited opportunities to become involved in certain procedures (e.g. referrals) thus restricting their development of specific expertise for that procedure. Among other limits the following aspects were reported: irreplaceability in case of assessor's longer absence (depends on number of PV assessors in unit dedicated to the same ATC), workload may be distributed in an unequal manner and 7

8 where human resources are very limited it is not possible to adhere to this type of organisation. Other types of organization were reported by 10 NCAs and included distribution of procedures by: assessors availability, route of authorisation (CAP and NAP medicinal products), assessors experience, order of receipt of documentation from the front office, prioritization on the basis of the public expectations and public health importance. These criteria are adopted in some NCAs also in relation to a number of available assessors. Figures are noted in Table 1. Figure 1. Organization of PV assessors activities* 50.0% 45.0% 40.0% 35.0% 30.0% 25.0% 20.0% 15.0% 10.0% 5.0% 0.0% Anatomical Therapeutic Class (ATC) basis On type of procedure basis On additional monitoring status basis Using seriousness to determine priority On body system organ class basis Other: please specify *In some member states work is organised according to more than one factor Whether a team of assessors or a single assessor only is involved in a B/R assessment, 16 of 25 (64%) NCAs reported that the decision is made on type and complexity of the procedure, while in 5 NCAs (20%) the assessment is mainly performed by the team and in 4 NCAs (16%) is usually performed by single assessor. With regard to team composition in B/R assessments, the participation of different professionals is reported in Table 1. In most NCAs the combination of a PV and clinical assessor is most common, while the participation of staff with other expertise (quality, preclinical, statistician and pharmaco-epidemiology assessors) is reported in less than 50% of responders (total responders 22). 8

9 Specific clinical/therapeutic area expertise Experience in specific type of procedure Personal interest and expertise in specific field (e.g pharmacogenomics, special populations etc.) Other (please specify) SCOPE Work Package 8 Survey Report Topic 5 Table 1 Inclusion of different types of assessors in the team When the B/R assessment procedures is performed by a team, please select which type of assessors are included in the team: Answer Options Response (%) Response n. Quality assessor Preclinical assessor 41 9 Clinical assessor Pharmacivigilance assessor Statistician 27 6 (Pharmaco)epidemiology assessor 27 6 Other (please specify) 41 9 answered question 22 skipped question 3 When the response Other was selected, it was done to specify that in some NCAs the role of PV and clinical assessor is covered by the same person, that the inclusion of different types of assessors depends on type of the procedure and on the need for some specific expertise. Therefore the composition of the team is decided on case by case basis. Concerning additional criteria for definition of team composition, 9 of 24 (37%) NCAs apply some specific criteria. These criteria are reported in Figure 2. Figure 2 Additional criteria for definition of team composition indicated in the survey 120.0% 100.0% 80.0% 60.0% 40.0% 20.0% 0.0% With regard to involvement of external experts, 10 of 25 (40%) NCAs engage external experts as B/R assessors and 13 of 25 (54%) use them for reviewing and giving advice on scientific recommendations. In total 9 of 25 (36%) NCAs engage 9

10 experts in circumstances when a specific competency is needed that is not available within the NCA, in particular for questions related to clinical practice and specific clinical expertise or when ad-hoc advice peer review is needed. The question on whether member states had a database for the selection of external experts was answered by 21 of 25 (84%) NCAs; 18 of 21 (86%) confirmed the existence of such a database. In 5 of 17 NCAs ( 19%) the experts data basis updated on yearly basis and in 4 of 17 NCAs (15%)it is updated every 2-4 years. Not-specified or in case of changes/need answer was obtained from 8 of responding NCAs (31%). 17 of 18 NCAs with database for selection of external experts answered the question about the criteria for inclusion of experts. Useful information was obtained with regard to creation and updating of the databases: Selection of external experts could be open to the public (meeting the criteria of transparency) and using the procedure of self-nomination. This foresees compilation of a specific application form which includes applicants general information, professional details/experience, area of interest/expertise and some other relevant information. Further step in this practice is inclusion of eligible applicants in to the expert data base. The Agency / Unit Heads/assessors in charge of the various procedures could select appropriate/suitable experts from the database on the basis of experts knowledge and declaration of interest basis. Recruitment based on selfnomination could be a useful strategy to recruit sufficient number of different external experts interested in collaborating with NCAs. This procedure is described in SOP for recruitment of and signing contracts with external experts. Another good practice in the management of an experts database consists of a public procurement procedure which is organised every two years in the NCA for selection of external experts to be included in the Expert Register. A general agreement for the duration of two years has been made in one NCA with all external experts and the work with external experts has been defined in the Methodical Instructions for Working with External Experts (SOP for recruitment of and signing contracts with external experts). If no specialists have applied for a certain area, but an expertise is necessary, the NCA approaches external contacts in clinical practice /academia with a request to ensure recruitment of certain specialists. The importance of implementation of a quality systems (e.g SOPs) in this area which includes continuous prediction and determination of areas of insufficient internal expertise was highlighted by many SCOPE participating NCAs. Beside the specific expertise which is warranted in the selection of experts, some additional criteria were highlighted in the survey by NCAs in order to better define professional characteristics that are useful for this type of professional profile: possess or develop a working knowledge and understanding of the national/european medicines regulatory procedures; to be able and prepared to speak on a range of relevant issues and not just on their own areas of specialty; 10

11 to be able to operate effectively on a national expert scientific committee to be skilled communicators; to be able to assimilate complex scientific information at short notice; to be recognized by their peers and/or be Fellows of the relevant academia institutions; to have a track record of achievement in their specialty. In conclusion, the competency framework depends also on an organisational structure with defined roles and responsibilities and optimisation of personnel interaction. A good organizational skills in the context of the assessment of therapeutic products are important and these promotion should be considered during the trainings for highly competent assessors. This survey highlighted the heterogeneous organization of PV activities across NCAs in Europe mostly due to the availability of human resources. In light of that, it seems that the organization of PV activities based on assignment of ATC code is adopted in bigger NCAs with large number of human resources, while in agencies with a limited number of assessors their involvement in all PV activities for all medicinal product is adopted in order to guarantee complete coverage of all types of procedures. Most national authorities consider joint/team assessment as essential for evaluating B/R in PV procedures and the PV assessor, in collaboration with other competences, is the main resources. Considering that engagement of experts for B/R assessment in the context of different PV procedures is an adopted practice among different NCAs in Europe, the appropriate selection process and management of expert databases could play an important role in the achievement of the necessary expertise. With regard to this issue, practices adopted in some NCAs were considered useful. 3.2 Question 6-8 concerning assessors education and experience All 25 involved NCAs answered the question regarding minimum education and experience levels for Junior assessor profile. One NCA reported that no distinction between Junior and Senior Assessors is applied in that NCA and the only professional profile is Assessor (PV or Pre/Clinical). Most of the NCAs, 22 of 25 (88%) reported that they apply specific criteria to define this professional profile when recruiting assessors and during the professional progress evaluation. The most commonly reported criteria concerned educational attainment such as university degree (e.g. degree in pharmacy, medicine, human biology or similar educations) with additional criteria for other important skills (e.g. ability to write and communicate in English fluently; to work in team, to manage own workload and to adhere to proposed timelines etc.) described by NCAs and included in Table 2. In addition, 9 of 25 (41%) NCAs declared the existence of obligatory training for newly employed assessors. In this section, one NCA described more in detail the criteria necessary for maintenance of Assessor status. 11

12 University degree in biomedical sciences Working experience in specific field* High level knowledge of English Ability to work in team Ability to manage own workload and to adhere to proposed timelines Ability to perform critical analysis of scientific research findings and literature Basic knowledge of regulatory procedures Adequate knowledge and use of computer applications Obligatory training (including on the job training) Criterion Education Professional Experience Skills Training requirements SCOPE Work Package 8 Survey Report Topic 5 Table 2. Criteria concerned educational attainment specified by the NCAs N of NCAs % of NCAs *the 2 NCAs 2 years, 1 NCA 1 year, 1 NCA 10 years, 2 not NCAs specified. Two examples of requirements description for Junior Assessor profile emerged from the survey are reported below: Example 1. The level of experience of at least 12 months, achievement of practical and theoretical training performed to ensure independence in activities of different complexity, including 7 working days of courses and at least 10 Assessment Reports as Assessor of renewals, PSURs, referrals, RMPs and PASSs procedures is requested for Junior Assessors. To be appointed as an Assessor, applicant must have received a favorable opinion in at least 5 consecutive assessments over the past 10. The tutor should deliver a written opinion on the quality of each assessment report carried out by the applicant assessor by filling out the evaluation form of practical training which will be forwarded to the head of PV unit. For maintenance of Assessor status the following is necessary: attendance of 7 working days of courses per year; delivery of at least 5 safety procedure assessments in the last 12 months; integrity and professional objectivity; self-control, enthusiasm and ability to adapt; 12

13 ability to work in a team. Example 2. One NCAs highlighted utility of the skills listed by the WHO guide Practical Guidance for Conducting a Review - based on the WHO Data Collection Tool for the Review of Drug Regulatory Systems for definition of assessor professional profile ( the guidance is available on uideassessregsys.pdf) This guidance includes following criteria: Education Essential: University Degree/Qualification in pharmaceutical science or another equivalent degree/qualification; Desirable: advanced training in medicine regulation. Experience Essential: At least 5 to 7 years of experiences in medicine regulation (Drug registration; Licensing/Regulatory Inspection or Laboratory control) at national level required in applyingand interpreting the drug-related regulations and guidance; Desirable: International experience in medicines regulation particularly in developing countries. Languages Essential: Working knowledge of English or French; Desirable: Basic knowledge of the official language of the country assessed would be an asset. Competencies An assessor must; Demonstrate formulation and synthesis skills; Demonstrate adequate knowledge and use of computer applications needed in the Regulatory Assessment environment; Demonstrate investigative and analytical skills; Foster the cooperation, communication, and consensus within and among groups and meetings to accomplish a common goal; Demonstrate the ability to structure and organize work as well as to set priorities and to meet deadlines; Demonstrate technical skills in the production of written materials related to the medicines regulation. All NCAs answered the question on any other existing structured support system (e.g. buddy/mentor system) for less experienced/ newly employed assessors and 19 (76%) of these declared existence of mentoring, supervising and introduction program for new employees. Beside the specific criteria for minimum education, some additional criteria were highlighted in the survey by NCAs: 13

14 Specific duration of the period under mentoring/training Evaluation foreseen Necessary number of independent assessment reports Obligatory training (including on the job training) Best practice meetings Discussion of issues with line manager /PRAC Rapporteur/experts Peer Reviewing system SCOPE Work Package 8 Survey Report Topic 5 the first two (or more) independent assessments are reviewed by the mentor; peer review system; reference to internal SOPs ; newly employed assessor works under supervision during the first 6 months of activity; "best practice" meetings which allow assessors to share experiences; learning by doing culture; a number of planned technical meetings. The overview of structured support system for less experienced/ newly employed assessors from the information obtained in the survey is included in Table 3. Table 3. Type of structured support system for less experienced/ newly employed assessors Criterion Mentoring System Introduction program and obligatory trainings N(%) of 25 respondi ng NCAs sharing of experiences 19 (76) 9 (36) 5 (20) N of 19 NCAs 5* (+2 from Q 6a) % of 19 NCAs * Range from 3 to 12 month Only 12 (48%) NCAs reported application of particular criteria for definition of compulsory expertise and experience level for Senior Assessor profile. Beside the essential requirements such as educational level (e.g degree in biomedical sciences, some NCAs require PhD and/or specialization or evidence of an up to date specialist level of knowledge in one or more relevant scientific/clinical areas), the analysis of open-ended responses showed that the most applied additional criteria are these concerning years of experience (ranging from 6 of 12 months to 2, 3 or 5 years of experience), proven ability to individually deliver good quality assessments 14

15 (3/12) and relevant scientific and professional experience (2/12). Other criteria adopted in some settings include: proven ability to effectively manage the successful implementation of RMMs to protect public health; broader knowledge across the range of scientific activities; detailed working knowledge and experience of all relevant national and European regulations and procedures applicable to PV. Good social and cooperation skills has been strongly taken into account by the majority of NCAs. Two examples of most structured and accurate approach for definition of the requirements for Senior Assessor profile are reported below: Example 1: The level of experience of at least 24 months, theoretical and practical courses equivalent to not less than 14 working days, at least 20 Assessment Reports as Assessor of renewals, PSURs, referrals, RMPs and PASSs procedures (at least one assessment performed in autonomy) is requested for Senior Assessor. To be appointed as Senior Assessor the applicant must have received a favorable opinion in at least 5 consecutive assessments over the past 10. The tutor should deliver a written opinion on the quality of each assessment report carried out by the candidate assessor by filling out the evaluation form of practical training which will be forwarded to the head of PhV unit. For maintenance of Senior Assessor status the following are necessary: attendance of 7 working days of courses per year; delivery of at least 5 safety procedure assessments in the last 12 months. In the absence of such a requirement, the head of PV unit could decide which training must be made to maintain the qualification of Senior Assessor. Example 2: There is not a fixed number of years of experience before an assessor can be considered to meet the competency of a senior assessor. Instead assessors are expected to show they meet the following competencies: Experience of working as an accredited assessor having demonstrated a consistently high level of performance in that role. Evidence of having proactively led on and provided insight into the assessment of major or complex drug safety issues under pressure, using evidence from a number of sources to independently make appropriate recommendations or decisions. Proven ability to effectively manage the successful implementation of RMMs to protect public health. Evidence of an up to date specialist level of knowledge in one or more relevant scientific areas or broader knowledge across the range of scientific activities and detailed working knowledge and experience of all relevant national and European regulations and procedures applicable to PV. In conclusion, valuable information was obtained in the survey concerning the compulsory experience and expertise for professional profile of Junior Assessor that is requested by different NCAs. Although almost all NCAs have in place a quality 15

16 system (e.g mentoring system, introduction program, obligatory trainings and/or sharing of experiences opportunities) aimed to ensure good level of expertise for newly employed assessors, the heterogeneity of requirements, emerged from the survey, is one of the principal characteristics of the system. In addition, structured NCA approaches for the definition of skills, required knowledge, experience and expertise for a Junior Assessor profile have been identified in the survey. Limited information on the required expertise and experience for a Senior Assessor level was obtained (high number of NCAs skipped the response). This may be explained by different factors such as the existence of a unique assessor professional profile only (no distinction between assessors); limited human resources dedicated to PV procedures assessment activities; absence of formally well-defined criteria for definition of a Senior Assessor profile etc. Moreover, the criteria applied by different NCAs were highly heterogeneous. Two examples of well-defined criteria for the definition of Senior Assessor profile emerged from the survey could help NCAs to appropriately and more completely address this issue if necessary. 3.3 Question 9-14 concerning training policy, training materials and other useful tools for PV assessors competency This section of the report is dedicated to one of the most important aspects of the PV assessors competency framework. It concerns exploration of the availability of training courses for PV assessors at EU, national and international level, NCAs opinion on the most useful educational events and the availability of other useful tools beside formal trainings. In this section, some training policy criteria were explored with participating NCAs and these included the availability of a formal training program, the frequency of refresher training, the number of days of obligatory training and specific methods that NCAs apply to motivate assessors to participate in training events. The most important training courses, dedicated to B/R assessment procedures, that NCAs recommend to assessors were divided into European, National and International. In total 16 of the 24 (67%) NCAs who responded have a formal training program (which includes a list of internal and/or external trainings useful for assessors). In most NCAs the formal training program is updated annually, in sporadic cases it is updated 1-2 times a year with a possibility to add additional training on case by case basis according to the demand / availability. There is a high heterogeneity in number of days annually considered optimal by NCAs for assessors to attend in the context of an appropriate training program. The current situation is described in Table 4. 16

17 Table 4. The number of training days per year NCAs consider optimal for assessor staff to attend in the context of an adequate training program Answer Totals Days >15 For Junior 0 (0) 2 (12) 3 (18) 0 (0) 1 (6) 1 (6) 1 (6) 2 (12) 7 (41) 17 Assessor* For Senior Assessor* 1 (6) 0 (0) 3 (19) 1 (6) 2 (12) 1 (6) 3 (19) 2 (12) 3 (19) 16 answered question 17 skipped question 8 * number (%) of responders In total 18 (72%) NCAs provided a list of recommended trainings. 44 different training courses were identified; of these 12 are European, 18 are national and 14 international. The most important information on these training courses, including, name, level, duration and focus are included in Annex 1. The most important reasons for the recommendation of the listed training events are included the high quality teaching and regulatory focus of the event. In total 19 (79%) of the 24 NCAs who responded considered other tools (beside training courses) useful for assessors when assessing B/R in different PV procedures. The types of tools considered useful by NCAs are included in Table 5 and detailed list of scientific papers, textbooks and guidance useful for B/R assessment in the context of PV procedures are reported in Annex 2. Table 5. Types of useful tools other than training courses identified by NCAs Answer Options Response (%) Response n. scientific publications attendance at scientific events (e.g. meetings, conferences, workshops ) clinical guidance on assessment of ADR mentorship meetings for assessors expertise exchange consultations with medical scientific societies websites use of textbooks on PV, pharmacoepidemiology, regulatory sciences participation in international web courses and seminars Other 32 6 answered question 19 skipped question 6 17

18 Other useful behaviors/practices and tools listed by responders are: participation in National/European PV Committees and/or supporting secretariats; participation in internal/national quality assurance meetings; consultation with Key Opinion Leaders in clinical areas; consultation and comparison of different assessment reports (experiences from different procedures and different active substances). The response rate on the question concerning any specific method that NCAs apply to motivate assessors to participate in training events was high (24/25, 96 %) and half (12/24) of responders declared application of some specific methods. All NCAs which apply specific incentives for motivation of assessors participation in training events provided the type of incentives (see Table 6). Table 6. Specific methods that NCAs apply to motivate assessors to participate in training events Answer Options Response (%) Response n. Annual scores 33 4 Benefits/bonus 17 2 Appraisals 42 5 Continuing professional development (CPD) 50 6 Personal educational forms (e.g. format that allows recording 58 7 of attended trainings) Other (please specify) 50 6 answered question 12 skipped question 13 Beside the possible methods to incentivise continuous education included in Table 6, half of the responders identified one or more additional methods reported below: inclusion of continuous education as an annual objective for all assessors; revalidation for medical assessors; consider personal qualifications whenever promotions are discussed; project based training: e.g. regulatory science research projects, skills: analysis; achievement motivation; mentoring and feedback (journal clubs etc.) - a regulatory science project helps to upskill the individual; payment of all expenses related to the training. In Topic 5 survey, the NCAs opinion on an Exchange Program, intended as a voluntary organizational exchange activity between organizations (e.g NCAs, EMA, UMC etc.) was also explored. The exchange program for assessors was recognized by all NCAs as a valuable tool for promotion of close collaboration, exchange of information and harmonization of effective standards for PV procedures evaluation (assessments). Almost all (23/25, 92%) NCAs described the most important reasons considered useful in an assessors exchange program. These reasons are summarized below: communication, sharing experience and knowledge between experts (to understand better other colleagues' opinions, decision-making and prioritization methods); 18

19 for keeping assessors up to date; opportunity for involvement in every-day practice and to study various fields (e.g. databases, interfaces). the qualification of the assessors, enhancement and harmonization of the assessment quality (exchange/learning of useful and information concerning best/good practices ); cooperation with WHO or UMC: useful for more complex and global overview of world PhV and about non-eu PV; the need for practical experience (to learn about the organisation, evaluation process and the quality assurance (comparable assessment quality); exchange programs between NCAs and EMA have been recognized as useful; the most useful exchange program among assessors of NCAs ( between NCAs); the possibility for on-site training or trainer facilitation is recognized as more cost effective; very targeted short term exchanges may be of more benefit where the host agency has some specific skill or knowledge to offer taking into account resource implications and language differences for all participants. mutual exchange programs. All NCAs provided the response on methods that they apply to ensure that assessors technical/scientific and regulatory knowledge is kept up to date. These responses are included in Table 7, highlighting the importance of on-the-job training, 92% and 88% for technical/scientific knowledge and regulatory respectively. Formal training (with internal or external provider) is utilized by 18 NCAs for technical training and 17 NCAs for regulatory training, around 70% for the both types of knowledge. Table 7 Methods that NCAs apply to ensure that assessors knowledge is kept up to date technical/scientific regulatory knowledge knowledge Answer Options Response (%) Response n. Response (%) Response n. Formal training, external provider Formal training, internal On-the-job training No specific training Other methods used/ Any further comments: answered question 25 skipped question 0 1 Other: 2 Other: - Attending relevant conferences. - External trainers/speakers are invited on an 19

20 - Scientific publications - Self-guided learning through reading - Internal audits, quality assurance - Quality assurance meetings ad hoc basis - Internal audits, quality assurance - Quality assurance meetings - EMA webpage - studying guidelines - EMA assessor's training/webinar In conclusion, from the analysis of provided information it could be concluded that the possible options of training courses available to PV assessors, at the national and international level, are extensive and cover a comprehensive range of relevant topics. The existence of training with different aims and levels make a training framework accessible for different types of assessors. The analysis also highlighted that the EMA provides basic PV education for new employees and less experienced assessors with courses and webinars often procedure specific. Most of NCAs have recognized the value of these trainings, and 12 of 18 NCAs included EMA trainings in their list of recommended trainings. At international level ISOP, UMC, WHO, EMA and DIA international PV trainings courses particularly focusing on senior or more experienced assessors were reported with higher frequency. Although the number of days considered optimal by NCAs for assessors to attend training varies widely (ranging from >15 to 1-2 days per year), in most of the NCAs a formal training program exists and is updated annually. In addition, On-the-job training and formal training (with internal or external provider) for both types technical/scientific and regulatory knowledge have been recognized by NCAs as the principle methods applied to ensure that assessors knowledge is kept up to date. Appraisals, continuing professional development (CPD), personal educational forms (e.g. format that allows recording of attended trainings) were identified as the most commonly applied incentives for motivation of assessors participation in training events. An exchange program for assessors was recognized by all NCAs as a valuable tool for the promotion of close collaboration, exchange of information and harmonization of quality standards for the assessment of PV procedures. 4 Discussion and Conclusions The aims of Topic 5 survey are: a) To describe competency levels and types of organization in the context of PV procedures in NCAs at present; b) To identify core assessor skills required for profile of PV assessor; c) To identify good practices and knowledge (training courses and literature) recommended by NCAs as useful for promotion of PV procedures assessment. 20

21 a) Competency levels and types of organization The competency framework depends also on an organisational structure with defined roles and responsibilities and optimisation of personnel interaction. This survey highlighted the heterogeneous organization of PV activities within NCAs across Europe depending mostly on the availability of human resources. The organization of PV activities on the basis of ATC code was most commonly seen in larger NCAs, while in NCAs with smaller size, assessors are involved in all PV activities in order to provide complete coverage of all types of procedures. Most NCAs consider joint/team assessment as essential for evaluating B/R in PV procedures and PV assessors in collaboration with other competences are the main resource. Considering that engagement of external experts for B/R assessment in the context of different PV procedures is an adopted practice among different NCAs in Europe, the appropriate selection process and management of expert databases could play an important role in achievement of the necessary expertise. This could be of help particularly for NCAs with limited staff dedicated to PV activities to facilitate assessment process and bidding for PV assessment procedures at the EU level. With regard to this issue, practices adopted in some NCAs were considered useful. Concerning required assessor professional profile, almost all NCAs have a quality system in place (e.g. mentoring system, introduction program, obligatory trainings and/or sharing of experiences opportunities) aimed to ensure a good level of expertise for newly employed assessors. There is a wide range in the number of days considered optimal by NCAs for assessors to attend training annually (range from >15 to 1-2 days per year). In most of the NCAs a formal training program exists and is updated annually. In addition On-the-job training and formal training (with internal or external provider) for both technical/scientific and regulatory knowledge have been recognized by NCAs as the principle methods of ensuring that assessors knowledge is kept up to date. Appraisals, continuing professional development, personal educational forms (e.g. in a format that allows recording of attended trainings) were identified as the most commonly used incentives for the motivation of assessors participation in training events. The exchange program for assessors was recognized by all NCAs as a valuable tool for the promotion of close collaboration, exchange of information and harmonization of effective standards for PV procedures evaluation (assessments). b) Core assessor skills required for PV procedures assessment The heterogeneity of requirements for assessors professional profile is one of the principal characteristics of the system which has emerged from the survey. However, the most commonly reported criteria for Junior assessor professional profile reported in the survey are: - university degree in biomedical sciences, - working experience in specific field, - high level knowledge of English - ability to work in a team, - ability to manage own workload and to adhere to proposed timelines, 21

22 - ability to perform critical analysis of scientific research findings and literature, - basic knowledge of regulatory procedures, - adequate knowledge and use of computer applications, - obligatory training (including on the job training). Two detailed descriptions of NCAs approaches for definition of skills and compulsory knowledge, experience and expertise for a Junior Assessors have been identified in the survey and could be helpful for other NCAs, if necessary. Few replies on the compulsory expertise and experience level for Senior Assessor were received. This could be explained by different factors such as existence of unique assessor professional profile only (no distinction between junior and senior assessor level); limited human resources dedicated to PV procedures assessment activities; absence of formal well-defined criteria for definition of Senior Assessor profile. In addition, the criteria applied by different NCAs emerged from the survey were diverse. Two examples of criteria for definition of a Senior Assessor profile were identified in the survey that could help NCAs to address this issue if necessary. c) Training courses and literature recommended by NCAs as useful for promotion of PV procedures assessment practices The existence of trainings with different aims and levels at national and international (EU) level, provides a comprehensive, accessible training framework for different types of assessors. Most of the NCAs have recognized the value of EMA trainings, with 67% of NCAs who included them in the their training recommendations. At international level, ISOP, UMC, WHO, EMA and DIA PV trainings courses focusing on senior or more experienced assessors were identified as useful sources. The complete lists of useful trainings and literature are included in Annexes 2 and 3 of this Report. 5 Preliminary recommendations These recommendations come out from the analysis of the topic 5 survey data. Further elaboration of preliminary recommendations is foreseen in the next moths in order to provide the final recommendations for the topic 5 on PV assessors competency. The recommendation for topic 5 is complementary with recommendations for WP8 topics 2,3 and 4 that concern PV procedure specific competency (RMP, PASS, PSUR/PSUSA and referral procedure assessment). a) Competency levels and types of organization Collaboration with quality, pre-clinical, clinical assessor and experts in (pharmaco) epidemiology/statistics is of importance during the assessment of the specific PV procedures. Establishing assessment teams for assessment in the context of the specific PV procedures is encouraged. 22

23 Implementation of quality systems (e.g. SOPs describing evaluation process for main PV procedures, assessor professional profile, training programme) aimed to ensure a good level of assessment is recommended. The appropriate selection process could play an important role in ensuring of the specific expertise in the context of PV procedures assessment, thus external experts selection process thought use of specific dedicated databases is encouraged. Beside the specific expertise which is warranted in the selection of experts, some additional criteria could be pointed-out as useful: - possess or develop a working knowledge and understanding of the national/european medicines regulatory procedures; - to be able and prepared to speak on a range of relevant issues and not just on their own areas of specialty; - to be able to operate effectively on a national expert scientific committee; - to be skilled communicators; - to be able to assimilate complex scientific information at short notice; - to be recognized by their peers and/or be Fellows of the relevant academia institutions; - to have a track record of achievement in their specialty. b) Core assessor skills required for PV procedures assessment Some important characteristics necessary for assessor professional profile could be recommended: - university degree in biomedical sciences; - working experience in specific field; - high level knowledge of English; - ability to work in a team; - ability to manage own workload and to adhere to proposed timelines; - ability to perform critical analysis of scientific research findings and literature; - basic knowledge of regulatory procedures; - adequate knowledge and use of computer applications; - obligatory training (including on the job training). Implementation of quality system (e.g. mentoring system, introduction program, obligatory trainings) aimed to ensure a good level of expertise for newly employed assessors and for maintenance of assessors appropriate level of knowledge is recommended. Implementation of continuing professional development programmes, personal educational forms (a format that allows recording of attended trainings) are encouraged for the motivation of assessors participation in training events. 23

24 The exchange program for assessors as a valuable tool for the promotion of close collaboration, exchange of information and harmonization of effective standards for PV procedures evaluation (assessments) could be encouraged on EU level. c) Training courses and literature recommended by NCAs as useful for promotion of PV procedures assessment practices On the basis of NCAs input, a list of recommended trainings and literature useful for PV assessors was created and is recommended for continues education of PV assessors. 24