Quality Improvement and Performance Framework

Transcription

1 Health Education East of England Quality Improvement and Performance Framework Education Providers Developing people for health and healthcare

2 Contents Introduction... Page 6 Summary of Process... Page 12 Key Performance Indicators... Page 20 Education Provider RAG Ratings... Page 24 Annexes... Page 27 Annex 1 Assessment Surveys... Page 28 Annex 2 Student Surveys... Page 30 Annex 3 Education Provider Key Performance Indicators... Page 32 Annex 4 Guidance for RAG Ratings... Page 48 Annex 5 Dispute Resolution... Page 50 Annex 6 Aggregation and Rounding... Page 52 2 Quality Improvement and Performance Framework

3 Foreword Professor Simon Gregory Director of Education and Quality 3

4 Welcome to the Quality Improvement and Performance Framework Handbook which outlines the process for Education Providers (EPs) for 2014/15. I hope that you find it a useful guide for this year. The purpose of Health Education East of England (HEEoE) is to enhance quality services for patients by ensuring the workforce is planned, educated and trained to a high quality. The Quality Improvement and Performance Framework (QIPF) is the process by which HEEoE assures the education it commissions and delivers on behalf of Employers providing NHS commissioned care in the east of England. The Framework was launched on 01 April 2014, and brings together a number of established processes to assure and continually improve the quality of healthcare education in the east of England. This Handbook focuses on the process for reviewing EPs delivering commissioned education but is designed to be read by all stakeholders in the process, including Employers. HEEoE invests 377 million every year on commissioning a wide range of education on behalf of local and national health systems. It has a duty to ensure that the EPs delivering this education provide a high standard of credible and professional education. 4 Quality Improvement and Performance Framework

5 Glossary ARM DBS Employer EP HEC HEEoE HSBs KPI LDA MPET OH QIPF WP Annual Review Meeting Disclosure and Barring Service Organisation hosting students for practice placements (referred to contractually as Practice Placement Provider ) Education Provider Healthcare Education Contract Health Education East of England Health Service Bodies Key Performance Indicator Learning and Development Agreement Multi Professional Education and Training Levy Occupational Health Quality Improvement and Performance Framework Workforce Partnership 5

6 Introduction 6 Quality Improvement and Performance Framework

7 This Handbook provides an overview of the Quality Improvement and Performance Framework for Education Providers (EPs) and gives detailed guidance for organisations involved in delivery of the Framework. Introduction The Quality Improvement and Performance Framework The Quality Improvement and Performance Framework (QIPF) sets out HEEoE s approach to ensuring that the quality of the non-medical healthcare education commissioned via Healthcare Education Contracts (HECs) is of high quality. It provides a framework that supports world class commissioning, continually drives up quality, links payment with performance and gives assurance that education and training programmes equip staff with the values, knowledge and skills to deliver high quality care. QIPF is transparent, locally standardised and actively promotes partnership working between EPs and providers of NHS commissioned services. This Handbook provides details of process for EPs; a separate but aligned process is available for Employers reviewed as part of QIPF. The Key Elements of QIPF There are four key elements to the EP QIPF process: 1. The ongoing collection and analysis of information to support management of EPs 2. The annual triangulation of information provided by all stakeholders involved in education and training 3. Annual Review of EPs against agreed Key Performance Indicators (KPIs) 4. The development and review of plans to support continuous improvement in performance and quality. 7

8 Period of Review The period reviewed as part of QIPF is outlined below: 01 September 2013 to 31 March 2015 Evidence relating only to this period will be reviewed as part of the QIPF process, unless stated otherwise. Key issues relating to performance and quality identified outside of the period may be considered if this is deemed to impact on the Review. In subsequent years this will be aligned with the NHS financial year April to March. 8 Quality Improvement and Performance Framework

9 Stakeholder Responsibilities Employers n Provide high quality educational learning environments for all students; n Ensure robust processes are in place to supervise and support student development; n Actively participate in QIPF, including the Annual Review process; n Work with EP partners to ensure the continuous improvement of the clinical learning environment and education programmes; n Utilise the opportunity provided by QIPF to give objective and constructive feedback to EP partners; n Work with EP partners and Workforce Partnerships (WPs) to use Improvement Plans to make continuous quality improvements. Introduction Education Providers n Deliver high quality education that meets commissioning requirements and professional standards/values; n Ensure high quality data is submitted in a timely manner; n Actively participate in the QIPF, including the Annual Review process as agreed within the pre-registration contracts; n Utilise Improvement Plans to make continuous quality improvements, working closely with WPs and employers; n Ensure all identified students are aware of their responsibilities to complete student feedback; n Undertake an objective and reflective self assessment against agreed KPIs; n Utilise Improvement Plans to make continuous quality improvements, working closely with WPs and other partners. 9

10 Health Education East of England n Leads the development and improvement of QIPF with key stakeholders; n Ensures consistency and transparency of approach; n Manages the Annual Review processes, including the membership of the Review Panel (the Panel); n Benchmarks outcomes across the region and shares best practice; n Collects and collates performance data to inform the QIPF; n Supports the development of capacity and capability within locally devolved systems to manage performance and quality improvement; n Works with locally devolved systems to deliver year on year improvements in education. Via Workforce Partnerships: n Manages EPs performance; n Uses data to review performance; n Takes a leading role in the Annual Review processes; n Develops and manages Improvement Plans to deliver ongoing quality improvement and manage risk; n Provides assurance for commissioned programmes and escalating concerns; n Supports EPs and Employers to complete constructive, reflective assessment of partners and self assessments. 10 Quality Improvement and Performance Framework

11 QIPF Outcomes The aim of QIPF for EPs is to demonstrate and support high quality education outcomes to improve patient care and continuously improve the education being delivered. Each EP will receive an Annual Report detailing the outcomes of the review process, identifying areas for improvement as well as areas of good practice. The review process is shown in Figure 1. Each commissioned programme will be reviewed against the agreed KPIs and an overall Red/Amber/Green (RAG) rating assigned. The Annual Report and final outcomes will form the basis of the development and Improvement Plans, which will be jointly delivered by the EP, WP and Employers to improve performance and quality in identified areas, and drive continuous improvement. Introduction Measure Improve Triangulate Rate Review Assure Figure 1 The Quality Improvement and Performance Framework review process 11

12 Summary of Process 12 Quality Improvement and Performance Framework

13 Student Feedback: HEEoE provides link to Student Surveys to EPs and Employers (see also Annex 2: Student Surveys) When: September 2014 Student Surveys will be issued in September These will be for completion by all identified students. Further information on Student Surveys will be issued as an addendum to this document. Closing date will be confirmed in summer 2014 as outlined in Annex 2. Summary of Process Pre-Registration Student Survey EPs will receive a link to a pre-registration student survey. EPs are responsible for ensuring all identified students are aware of their responsibility to provide feedback on their NHS-funded education as defined in the education contracts, and are responsible for issuing surveys to all students currently in training at the EP on programmes commissioned via the pre-registration contract. The timeline for the process is set out in Figure 2 at the end of this section. 1. HEEoE issues advance copies of EP Self Assessments and Employer Assessments of Partners to EPs and Employers When: 26 January 2015 Advance copies of the assessments and self assessments will be issued to support both Employers and EPs with the completion of feedback. 2. HEEoE Issues EP Self Assessments and Employer Assessments of Partners to EPs and Employers (see also Annex 1: Assessment Surveys) When: 16 February 2015 Two surveys will be issued by HEEoE which will form the basis of the qualitative KPIs: 13

14 Self Assessment Survey EPs will receive an academic self assessment survey covering the ten qualitative academic KPIs. These surveys should be completed with a Self Assessed RAG rating based on the level of evidence the organisation is able to provide, demonstrating that they have complied with the criteria outlined in the KPI. Strong consideration should be given to the narrative provided by all organisations as this will be reviewed as part of the triangulation process (see stage 6 below) with HEEoE and the WPs. Comments are mandatory. Assessment of Partners Employers will receive an assessment of EP partners survey covering the ten qualitative academic KPIs. These surveys should be completed with RAG ratings for each EP from which the employer has hosted students for placement. It is the Employer s responsibility to ensure the RAG rating and supporting narrative are objective and reflective as this information will be used as evidence in assessing the EP s performance. It is a requirement that the assessment is signed off by the agreed Director with responsibility for education governance, as outlined in the Learning and Development Agreement (LDA) on behalf of the organisation. Comments are mandatory. 3. EPs complete self assessment When: 16 February 2015 to 02 March 2015 EP self assessments will be due no later than 13:00 on 02 March However, the assessment can be submitted at any point between 16 February 2015 and 02 March Employers complete assessment of EP partners When: 16 February 2015 to 02 March 2015 Employers assessments of EP partners will be due no later than 13:00 on 02 March However, the assessment can be submitted at any point between 16 February 2015 and 02 March Quality Improvement and Performance Framework

15 5. Production of scorecards and reports for Review Panel When: 16 March 2015 Scorecards are produced for each EP and made available to the Review Panel. 6. HEEoE triangulates all information submitted for discussion at the first Review Panel pre ARM meeting When: 30 March 2015 The Panel will review the EP self assessments, Employer assessments and progress with the previous year s Action Plan to allocate tentative RAG ratings for each KPI for each programme being reviewed. At this stage, where the Panel determines that additional clarification is required, this will be requested from the EP. Summary of Process 7. HEEoE requests additional evidence from EPs where required When: 30 March 2015 Additional evidence requests issued to EPs. Additional evidence must be provided no later than 09 April Panel Pre-Annual Review meeting When: 13 April 2015 Additional evidence is reviewed and considered by the Panel. Key lines of enquiry (KLEs) are agreed by the Panel in preparation for the Annual Review Meeting (ARM). 9. HEEoE issues to EPs key lines of enquiry, draft RAG ratings and composition of Panel for ARMs When: 14 April 2015 Key lines of enquiry are issued to EP one week prior to the date of its ARM, along with draft RAG ratings for each KPI and confirmation of composition of the Panel for the Meeting. Normally only KPIs where there are outstanding issues will be subject to KLEs at the ARM. 15

16 10. HEEoE processes Quantitative Information for Annual Review When: 20 April 2015 Quantitative KPIs are calculated based on the contract data submitted for 2014/15 Quarter 4. Failure to submit an accurate data return by the agreed deadline will result in all KPIs being rated Red. 11. ARM When: 20 April 2015 onwards EP ARMs will focus on KLEs developed in stage 8. Membership of the Annual Review Panel will be confirmed prior to the ARM and will include: Review Chair, Key members of the Directorate of Education and Quality & Education Commissioning Team, WP Representation, and external members including subject experts and student representation. For the 2015 Review, the Chair will be the HEEoE Director of Education and Quality. There is an expectation that each EP Dean, with appropriate senior colleagues, will attend their EP ARM. EPs are responsible for ensuring that an appropriate range of Employer partners is present at the ARM. EPs are encouraged to secure representation from the student body and service users as appropriate. When deciding on representation at the ARM, EPs should consider the overall size of the group to ensure it is practical to chair. 12. Annual Report produced and agreed When: May 2015 An Annual Report for each EP will be produced by HEEoE following the ARM. The Annual Report will be issued in draft form for the EP being reviewed, within four weeks of its ARM, to agree its accuracy. Final reports will be issued in May. 16 Quality Improvement and Performance Framework

17 13. Improvement Plan produced, agreed and managed in year When: May 2015 Improvement Plans should be developed, based on the recommendations outlined in the Annual Report, for delivery in the ten months from 01 June 2015 to 31 March Responsibility for ongoing management of the Improvement Plan will sit with the WP. Delivery against agreed Improvement Plans will be assessed as part of the following year s Annual Review process. Summary of Process 17

18 26 Jan Feb March March 2015 Student feedback reports provided 2 Weeks 1 Week 2 Weeks KPI questions issued in advance in Word document HEEoE issues EP assessments and partner assessments Surveys with EPs and Employers Deadline for assessments and partner assessments Reports produced and issued to the Review Panel Panel reviews reports Figure 2 Process Timeline 18 Quality Improvement and Performance Framework

19 30 March 2015 Improvement Plans from previous review provided by WPs 13 April April 2015 Quantitative KPIs calculated May 2015 Summary of Process 2 Weeks 1 Week First Panel Pre ARM Meeting Evidence requested KLEs to EPs Second Panel Pre ARM Meeting ARMs, 20 April onwards Outcomes from ARMs agreed 19

20 Key Performance Indicators 20 Quality Improvement and Performance Framework

21 This section provides a summary of the KPIs used for the EP QIPF process. Additional information for KPIs can be found in Annex 3. Information in relation to how both EPs and Employers should assign RAG ratings against KPIs is outlined in Annex 4. Education Provider Key Performance Indicators EPs will be reviewed against the following KPIs: R1 R2 Recruitment (Qualitative) Recruitment (Quantitative) A representative sample of senior staff from Health Service Bodies (HSBs) carries out a stock-take of recruitment and selection policy and processes, and agrees any actions with the EP annually. Variance between commissioned numbers and actual students recruited per programme (percentage). Number of starters/ number of students commissioned. A1 Course Content A representative sample of senior staff from HSBs mutually stock takes, annually reviews and agrees with the EP action required to ensure that course content and delivery is suitable for ensuring a workforce that is fit for purpose. Assurance should also be provided that curriculum content reflects NHS behaviours, values and attitudes required by healthcare professionals as defined by the NHS Constitution. P1 Partnerships The EP can assure HEEoE that it is able to effectively manage in partnership with HSBs all risks identified within practice proactively therefore minimising the impact on student learning. P2 Placement Audit A representative sample of senior staff from HSBs, as agreed with Practice Placement Providers, confirms that the EP places students within currently audited, appropriately staffed clinical areas. The EP ensures that staff supporting students have undertaken appropriate training, and offers updating and link lecturer support to practice. P3 Fitness for Placement A representative sample of senior staff from HSBs, as agreed with Practice Placement Providers, confirms that any concerns about the fitness for placement of students are being responded to in line with processes and timeframes mutually agreed by the EP and the Practice Placement Provider and that the Practice Placement Provider and the EP work in partnership to resolve any issues. Key Performance Indicators 21

22 P4 DBS/OH A representative sample of senior staff from HSBs, as agreed with Practice Placement Providers, confirms that Disclosure and Barring Service (DBS) and Occupational Health (OH) checks and any resultant actions are carried out by the EP in accordance with mutually agreed processes and communicated to Practice Placement Providers appropriately. P5 Basic Skills A representative sample of senior staff from HSBs confirms that students starting placements demonstrate basic skills, knowledge and professional behaviours as mutually agreed with the EP. C1 Review Outcomes EP confirms that over the course of the year it has reported any weakness identified by relevant reviews (e.g. Quality Assurance Agency or internal reviews) within two weeks of verbal feedback or as soon as possible, and in any case within three working days of receipt of the written report; whichever is soonest. In addition, the EP is able to confirm that an action plan has been or is being developed in partnership with Practice Placement Providers or the EP is able to confirm that no weakness was identified by any form of review over the previous year. C2 Learner Feedback The EP collects student feedback through the National Student Survey and other appropriate means including the QIPF student survey(s) and can demonstrate an audit trail showing resultant action plans and service improvements. L2 Destinations The EP can assure HEEoE that it is collecting robust information in relation to the first destinations of students, and is using this information appropriately to develop programmes. O1 Attrition Attrition as a percentage for the programme. Sum of all (Discontinuances + Withdrawals + External Transfers Out + Internal Transfers Out External Transfers In Internal Transfers In) / Sum of all Starters. 22 Quality Improvement and Performance Framework

23 L3 Outturn The percentage of completions on time from the programme against starters. L4 Standard Progression = number of students that complete on time / number of starters. The percentage of completers on standard progression against overall completers. = sum of all completers on standard progression on time / sum of all completers. Key Performance Indicators 23

24 Education Provider RAG Ratings 24 Quality Improvement and Performance Framework

25 Each KPI will attract a RAG rating which will provide a total score for each KPI. This score will be multiplied by the weighting assigned for each KPI, which can be found in Annex 3. The RAG ratings are given in the table below: RAG Description Score Red Indicates a KPI has not been met or no assurance can be provided of 1 compliance Education Provider RAG Ratings Amber Indicates a KPI has been partially met 0 Green Indicates a KPI has been met providing a robust level of assurance 1 For each EP there is a maximum score of 100 if all KPIs are rated Green, and a minimum score of 100 if all KPIs are rated Red. Overall RAG Boundaries Scores for each of the fourteen individual KPIs will be combined to give an overall score. Overall RAG boundaries for combined KPIs can be seen in the table below: RAG Score Range Red Less than 0 Amber Less than 50 Green Greater than or equal to 50 There are scenarios in which particular KPIs cannot be measured. In these cases, the maximum quantitative score will be 15, 10 or 5, and the overall RAG boundaries will be adjusted to compensate proportionally for the reduced maximum score. Example 1 A new programme which has not been running long enough to count towards attrition or have any completers. KPI R2 is measured; O1, L3, L4 are not measured. Maximum quantitative score 5, total maximum score

26 Example 2 A programme is no longer commissioned but still has cohorts running through the system. KPI R2 is not measured. n The last cohort completed in September O1 is not measured, L3 and L4 are measured. Maximum quantitative score 10, total maximum score 90. n There are cohorts active in/after Jan O1, L3, L4 are measured. Maximum quantitative score 15, total maximum score 95. The minimum score is the same value as the maximum, i.e. a programme with a maximum score of 90 will have a minimum score of 90, as set out in the table below: RAG Maximum 95 (One quantitative KPI not measured) Maximum 90 (Two quantitative KPIs not measured) Maximum 85 (Three quantitative KPIs not measured) Red Less than 0 Less than 0 Less than 0 Amber Less than 48 Less than 45 Less than 43 Green Greater than or equal to 48 Greater than or equal to 45 Greater than or equal to Quality Improvement and Performance Framework

27 Annexes Annexes 27

28 Annex 1: Assessment Surveys 28 Quality Improvement and Performance Framework

29 Both the EP Self-Assessment and Employer Assessment of EPs will be completed online. A link and specific information on how to access and complete surveys will be issued to a nominated contact in each organisation when surveys are issued. All questions in assessment surveys will be based on the qualitative KPIs being measured. An advance copy of all questions will be provided in Word format on 26 January 2015 to support completion by stakeholders; however, final submissions must be made using the online submission. Guidance on assigning RAG Ratings Evidence requirements for each KPI should be noted as outlined in Annex 3. Guidance on how RAG ratings will be assigned against evidence requirements is outlined in Annex 4. It is important that as much relevant information as possible is provided as part of the assessment surveys to support RAG ratings so as to ensure the Panel is able to consider all factors. EP Surveys Assessments and Self Assessments for EPs must provide a RAG rating for each programme offered by the EP. Programmes are rated individually, with one comment against each KPI for all programmes. Where programmes have not been rated Green, reference should be made in the commentary as to why this is the case. Comments are mandatory against all KPIs; Employers and EPs must be able to provide evidence to support any comments made. This evidence can be in the form of s, minutes of meetings or any other suitable form which justifies the comments. Annex 1 Assessment Surveys 29

30 Annex 2: Student Surveys 30 Quality Improvement and Performance Framework

31 To Follow This information will be issued as an addendum in Summer Annex 2 Student Surveys 31

32 Annex 3: Education Provider KPIs 32 Quality Improvement and Performance Framework

33 R1 Recruitment (Qualitative) A representative sample of senior staff from HSBs carry out a stock-take of recruitment and selection policy and processes and agrees any actions with the EP annually. Annex 3 Exemplar Evidence Evidence of a mutually agreed recruitment and selection plan between EP and HSBs showing actions and the outcomes of these actions. As a minimum this plan must include the following topics: n support for widening access; n promotion of equality and diversity; n ensuring candidates compatibility with the values and behaviours defined within the NHS Constitution; n management of Employer concerns in relation to candidates suitability; n clear record of interview decisions is being maintained; n active engagement of representatives of all Practice Placement Providers in recruitment and selection process; n active engagement of service users in the recruitment and selection processes; n active engagement of students in the recruitment and selection process; n demonstration of innovation and adoption of best practice in the development of marketing, recruitment and selection plans; n evidence that the EP has ensured that students understand their responsibility as NHS-funded students to provide feedback on their education experience; n demonstration of partnership working; n students are recruited in line with Schedule 13 of this agreement; n plans for ensuring that students are eligible for NHS funding; n evidence that all staff, including administrative staff, involved in the recruitment and selection of students are aware of the requirement to recruit to the values of the NHS Constitution; n a mutually agreed process for communicating and managing DBS checks; n the EP works in partnership with the WP to manage over- and underrecruitment. EP Assessment Requirements n The EP is able to provide evidence that it is fully compliant with all exemplar evidence identified. Employer Assessment Requirements n The Employer is assured that the EP is fully compliant with the exemplar evidence identified. Education Provider KPIs This KPI has an overall weighting of 8. 33

34 R2 Recruitment (Quantitative) Variance between commissioned numbers and actual students recruited per programme (percentage). This is a quantitative measure based on the contract data collection; no selfassessment or assessment by partners is required. Recruitment will be measured at 31 March 2015 and will cover all new starters reported during the NHS financial year 01 April 2014 to 31 March Recruitment will be measured against agreed commissioning plans. The recruitment KPI is a measure of the percentage extent to which the commissioned places have been achieved through recruitment and is calculated as follows: number of starters number of students commissioned For the purpose of recruitment, starters are defined as those students who: n are first-time entrants to Year 1; and n have not transferred from another healthcare programme. Information on aggregation and rounding is outlined in Annex 6. Programmes will be RAG rated as outlined below: This KPI has an overall weighting of 5. Red Amber Green Programmes with commissions of <85% or >110% <95% or >105% 95% to 105% 20 students or more Small Cohort Calculation: 3 or more students 2 students below Less than or Programmes with commissions of below target OR 3 target OR 2 equal to 1 fewer than 20 students or more students student above student above above target target target or less than or equal to 1 student below target 34 Quality Improvement and Performance Framework

35 A1 Course Content A representative sample of senior staff from HSBs mutually stock-takes, annually reviews and agrees with the EP action required to ensure that course content and delivery is suitable for ensuring a workforce that is fit for purpose. Annex 3 Assurance should also be provided that curriculum content reflects NHS behaviours, values and attitudes required by healthcare professionals as defined by the NHS Constitution. Exemplar Evidence Evidence of a mutually agreed plan between EP and HSBs for reviewing course content and delivery showing actions and outcomes of these actions. As a minimum this plan must include: n clear process for the management of concerns raised by Practice Placement Providers and/or Students about course content and delivery; n clear process for involving Practice Placement Providers in curriculum design and delivery; n curriculum content reflects behaviours, knowledge, skills and attitudes required by healthcare staff as defined within the NHS Constitution; n curriculum content and delivery are being reviewed annually in partnership with Practice Placement Providers; n demonstration of partnership working in curriculum design and delivery; n innovation in curriculum design and delivery; n evidence that academic staff developing and delivering programmes have up-to-date, relevant clinical knowledge; n evidence that academic staff developing and delivering programmes reflect the values and behaviours of the NHS Constitution; n evidence that the EP is responding to national and local priorities such as local Transformation Projects. EP Assessment Requirements n The EP is able to provide evidence that it is fully compliant with all exemplar evidence identified. Employer Assessment Requirements n The Employer is assured that the EP is fully compliant with the exemplar evidence identified. Education Provider KPIs This KPI has an overall weighting of 8. 35

36 P1 Partnerships The EP can assure HEEoE that it is able to manage all risks identified within practice effectively and proactively in partnership, thereby minimising the impact on student learning. Exemplar Evidence Evidence must be available on request from a representative sample of senior staff from HSBs confirming the KPI is being met. Acceptable evidence would be: n minutes of meetings containing clear and unambiguous reference to the confirmation that all risks are reviewed annually and that there are no risks or that risks are being managed effectively; n where risks are identified, a clear, mutually agreed action plan is available; n a signed letter from representative senior practice staff from HSBs agreeing that the KPI is being met (as above) in partnership with all stakeholders; n Operational Contract Meetings are being held regularly with representation from the EP, practice placement partners and the WP; n Strategic Review Meetings are effective and resolve those operational issues which are escalated. EP Assessment Requirements n The EP is able to provide evidence that it is fully compliant with all exemplar evidence identified. Employer Assessment Requirements n The Employer is assured that the EP is fully compliant with the exemplar evidence identified. This KPI has an overall weighting of Quality Improvement and Performance Framework

37 P2 Placement Audit A representative sample of senior staff from HSBs, as agreed with Practice Placement Providers, confirms that the EP places students within currently audited and appropriately staffed clinical areas. The EP ensures that staff supporting students have undertaken appropriate training, and offers updating and link lecturer support to practice. Annex 3 Exemplar Evidence Evidence must be available on request from a representative sample of senior staff from HSBs that confirms the KPI is being met. Acceptable evidence would be: n minutes of meetings containing clear, unambiguous reference to the confirmation; n a signed letter from representative senior practice staff from the HSB agreeing that the KPI is being met in partnership with all stakeholders; n the EP is working with local HSBs to ensure that current and future placement circuits reflect the changing NHS and can provide documentation demonstrating this. EP Assessment Requirements n The EP is able to provide evidence that it is fully compliant with all exemplar evidence identified. Employer Assessment Requirements n The Employer is assured that the EP is fully compliant with the exemplar evidence identified. Education Provider KPIs This KPI has an overall weighting of 8. 37

38 P3 Fitness for Placement A representative sample of senior staff from HSBs, as agreed with Practice Placement Providers, confirms that any concerns about the fitness for placement of students are being responded to in line with processes and timeframes mutually agreed by the EP and the Practice Placement Provider and that the Practice Placement Provider and the EP work in partnership to resolve any issues. Exemplar Evidence Evidence must be available on request from a representative sample of senior staff from HSBs that confirms the KPI is being met. Acceptable evidence would be: n minutes of meetings containing clear, unambiguous reference to the confirmation in partnership with all stakeholders; n a signed letter from representative senior staff from the HSB agreeing that the KPI is being met. A clear policy, mutually agreed and implemented by the EP and HSB, should exist regarding fitness for placement including: n cause for concern processes implemented and effective; n reporting of untoward incidents that involve students to the EP and HEEoE; n systems in place for monitoring individual students as they progress through each practice placement; n capacity of each practice placement to support students is being managed consistently; EP Assessment Requirements n The EP is able to provide evidence that it is fully compliant with all exemplar evidence identified. Employer Assessment Requirements n The Employer is assured that the EP is fully compliant with the exemplar evidence identified. n placement audit systems that record capacity and are reviewed annually. This KPI has an overall weighting of Quality Improvement and Performance Framework

39 P4 Disclosure and Barring Service and Occupational Health checks A representative sample of senior staff from HSBs, as agreed with Practice Placement Providers confirms that Disclosure and Barring Service (DBS) and Occupational Health (OH) checks and any resultant actions have been carried out by the EP in accordance with mutually agreed processes and communicated to Practice Placement Providers appropriately. Annex 3 Exemplar Evidence Evidence must be available on request from a representative sample of senior staff from providers of NHS services that confirms the KPI is being met. Acceptable evidence would be: n evidence of 100% compliance with the policy; no students ever enter placement without the necessary DBS and OH clearance; n evidence that DBS and OH clearance is communicated to all Practice Placement Providers in advance of students entering placements. For clarity, reporting should not be by exception; n minutes of meetings containing clear, unambiguous reference to confirmation the KPI is being met in partnership with all stakeholders; n a signed letter from representative senior practice staff from the HSB agreeing that the KPI is being met. EP Assessment Requirements n The EP is able to provide evidence that it is fully compliant with all exemplar evidence identified. Employer Assessment Requirements n The Employer is assured that the EP is fully compliant with the exemplar evidence identified. Education Provider KPIs This KPI has an overall weighting of 8. 39

40 P5 Basic Skills A representative sample of senior staff from HSBs confirms that students starting placements demonstrate basic skills, knowledge and professional behaviours as mutually agreed with the EP. Exemplar Evidence Evidence must be available on request from a representative sample of senior staff from HSB that confirms the KPI is being met. Acceptable evidence would be: n minutes of meetings containing clear, unambiguous reference to confirmation the KPI is being met in partnership with all stakeholders; n a signed letter from representative senior practice staff from the HSB agreeing that the KPI is being met; n students demonstrate basic skills, knowledge and behaviours which reflect professional codes of conduct and the values of the NHS Constitution; n students complete all mandatory training prior to commencement in placement; n students have a clear induction from service staff prior to commencing each placement; n record of induction held as part of the placement documentation. Students have clear placement guidelines and expectations; EP Assessment Requirements n The EP is able to provide evidence that it is fully compliant with all exemplar evidence identified. Employer Assessment Requirements n The Employer is assured that the EP is fully compliant with the exemplar evidence identified. n guidelines and expectations agreed between EP and HSB and mechanisms are in place to ensure patients are aware and consent to care being provided by students; n ensuring student awareness of a positive attitude to hearing, accepting and responding to student concerns, complaints and compliments. This KPI has an overall weighting of Quality Improvement and Performance Framework

41 C1 Review Outcomes EP confirms that over the course of the year it has reported any weakness identified by relevant reviews (e.g. QAA or internal reviews) within two weeks of verbal feedback, or as soon as possible and, in any case, within three working days of receipt of the written report; whichever is soonest. Annex 3 In addition, the EP is able to confirm that an action plan has been or is being developed in partnership with Practice Placement Providers, or the EP is able to confirm that no weakness was identified by any form of review over the previous year. Exemplar Evidence n Reports from other reviews should be available if requested; n A signed letter from the Dean agreeing that the KPI is being met; n Action plans developed in partnership with HSBs. EP Assessment Requirements n The EP is able to provide evidence that it is fully compliant with all exemplar evidence identified. Employer Assessment Requirements n The Employer is assured that the EP is fully compliant with the exemplar evidence identified. Education Provider KPIs This KPI has an overall weighting of 8. 41

42 C2 Learner Feedback The EP collects student feedback through the National Student Survey and other appropriate means including the QIPF student survey(s) and can demonstrate an audit trail showing resultant action plans and service improvements. Exemplar Evidence Action plans and collated student feedback for each cohort must be available on request and should cover the following topics as a minimum: n course content and delivery; n adequacy of preparation for placements; n placement learning experiences; n service improvements. Evidence should be available to demonstrate: n how the EP is increasing response rates; n an effective approach to interrogating the data; n effective mechanisms for feeding back to students; n process for making changes based on feedback; n how the EP is measuring effectiveness of changes made as a result of student feedback. EP Assessment Requirements n The EP is able to provide evidence that it is fully compliant with all exemplar evidence identified. Employer Assessment Requirements n The Employer is assured that the EP is fully compliant with the exemplar evidence identified. This KPI has an overall weighting of Quality Improvement and Performance Framework

43 L2 Destinations The EP can assure HEEoE that it is collecting robust information in relation to the first destination of each student, and is using this information appropriately to develop programmes. Annex 3 Exemplar Evidence The EP should provide evidence that it is: n making all reasonable efforts to collect information on students first destination after qualifying; n attempting to obtain information on first destinations from all qualifying students; n encouraging and supporting students to enter employment in healthcare within the east of England; n working in partnership with organisations providing NHS-funded care in relation to the recruitment of newly qualified healthcare professionals into the workforce. EP Assessment Requirements n The EP is able to provide evidence that it is fully compliant with all exemplar evidence identified. Employer Assessment Requirements n The Employer is assured that the EP is fully compliant with the exemplar evidence identified. Education Provider KPIs This KPI has an overall weighting of 8. 43

44 O1 Attrition (Quantitative) Attrition as a percentage for the programme. This is a quantitative measure based on the contract data collection; no selfassessment or assessment by partners is required. Attrition is measured at 31 March 2015 and includes cohorts which were active during the Quarter 4 reporting period (i.e. have a completion date on or after 1 January 2015) and have been active for at least three months (i.e. commence on or before 31 December 2014). The KPI value is produced for each programme by grouping equivalent eligible cohorts (those which have a common subject or nursing branch, level, and duration), and calculating the aggregate rate of attrition over all cohorts. The percentage attrition for a programme, based on the eligible cohorts, is given by the formula: Sum of all (Discontinuances + Withdrawals + External Transfers Out + Internal Transfers Out External Transfers In Internal Transfers In) / Sum of all Starters Definitions of the items used in the formula: Discontinuation: Withdrawal: A student who leaves by the EP decision A student who leaves by their own decision External Transfer: A student who moves to or from a different EP, with credit Internal Transfer: Starters: A student who moves between programmes or cohorts within an EP, e.g. a change of nursing branch or level, or returning after a break in study and moving to a later cohort For the purpose of attrition, starters are defined as those students who: n are first-time entrants to Year 1; and n have not transferred in from some other healthcare programme. Information on aggregation and rounding is outlined in Annex 6. RAG rating tolerances are outlined over the page for all programmes. This KPI has an overall weighting of Quality Improvement and Performance Framework

45 For programmes with 8 or more students: Programme Red Amber Green Adult Nursing More than 20% 13% to 20% 13% or Less Annex 3 Children s Nursing More than 20% 13% to 20% 13% or Less Learning Disabilities Nursing More than 30% 13% to 30% 13% or Less Learning Disabilities Nursing More than 30% 13% to 30% 13% or Less & Social Work Mental Health Nursing More than 30% 13% to 30% 13% or Less Mental Health Nursing & Social Work More than 30% 13% to 30% 13% or Less Midwifery 3-Year More than 18% 13% to 18% 13% or Less Midwifery 18-Month More than 15% 13% to 15% 13% or Less Physiotherapy More than 15% 13% to 15% 13% or Less Occupational Therapy More than 15% 13% to 15% 13% or Less Speech and Language Therapy More than 15% 13% to 15% 13% or Less Dietetics More than 15% 13% to 15% 13% or Less Education Provider KPIs Diagnostic Radiography More than 15% 13% to 15% 13% or Less Therapeutic Radiography More than 25% 13% to 25% 13% or Less Operating Department Practice More than 15% 13% to 15% 13% or Less Clinical Psychology More than 15% 13% to 15% 13% or Less Health Visiting More than 10% 7% to 10% 7% or Less Clinical Physiology More than 15% 13% to 15% 13% or Less Biomedical Science More than 15% 13% to 15% 13% or Less Oral Health Foundation Degree More than 20% 13% to 20% 13% or Less Small Cohort Calculation: Red Amber Green For programmes with fewer than 8 students 3 or more net 2 net leavers 0 to 1 net leavers leavers 45

46 L3 Outturn The percentage of completions on time from the programme against starters. This is a quantitative measure based on the contract data collection; no selfassessment or assessment by partners is required. This KPI measures the number of students who complete from a programme on time as a percentage of the number of starters for that programme. n Completion on time is defined by a student having an actual completion date no earlier than 100 days before and no later than 45 calendar days after the programme completion date. n Completion is defined as a student who qualifies from a programme, which may or may not be the same programme that they originally enrolled on. n Students who qualify with an exit award that does not make them eligible to register with a professional body will not be classed as completers. For the purpose of completions starters are defined as those students who: n are first-time entrants to Year 1; and n have not transferred from another healthcare programme. Outturn will be measured on 31 March 2015 for all programmes with a completion date between 1 January 2014 and 31 December The percentage of students who complete a programme, on time, as a percentage of the number of starters is calculated using the following formula: number of students that complete on time number of starters Information on aggregation and rounding is outlined in Annex 6. RAG Rating tolerances for this KPI are outlined in the table below: This KPI has an overall weighting of 5. Red Amber Green Programmes with commissions Less than 45% Less than 60% Greater than, or of 4 students or more equal to 60% 46 Quality Improvement and Performance Framework

47 L4 Standard Progression The percentage of completers on standard progression against overall completers. This is a quantitative measure based on the contract data collection; no selfassessment or assessment by partners is required. This KPI measures the number of students who complete on standard progression on time in a calendar year as a percentage of the total number of completers that year. Standard Progression will be measured at 31 March 2015 for all students with an actual completion date between 01 January 2014 and 31 December 2014 inclusive. The number of students who complete on standard progression on time in a calendar year as a percentage of the total number of completers that year is calculated using the following formula: Sum of all completers on standard progression on time Sum of all completers Completer on Standard Progression: A student on standard progression completes the programme of study for which they originally enrolled and qualifies with an exit award that makes them eligible to join the professional register, without re-sitting such that they would not complete within the normal timeframe. The normal timeframe includes students whose actual completion date is no earlier than 100 days before and no later than 45 days after the programme completion date. Students who enter a programme as direct entrants with Accreditation of Prior and Experiential Learning (APEL) and who continue and complete on time in that programme are included in the students who complete under standard progression transfers from nursing diploma to degree programmes, or between health visiting programme levels, are not classed as leaving standard progression, unless the cohort year, nursing branch or programme duration is also changed. Completer: A student who completes a programme of study and qualifies with an exit award that makes them eligible to join the professional register. If this is not the same programme they originally enrolled on, they are not a completer on standard progression. Completion on time: This is defined by a student having an actual completion date no earlier than 100 days before and no later than 45 calendar days after the programme completion date. RAG Rating tolerances for this KPI are outlined in the table below: Annex 3 Education Provider KPIs This KPI has an overall weighting of 5. Red Amber Green All Programmes Less than 78% Less than 82% Greater than or equal to 82% 47

48 Annex 4: Guidance for RAG Ratings 48 Quality Improvement and Performance Framework

49 To ensure a consistent regional approach, all RAG ratings will be subject to peer review and agreement prior to ARMs to ensure consistency within the information being reviewed and the outcomes being agreed. The following guidance has been provided as a tool to support the determination of RAG ratings for each qualitative KPI. Annex 4 Green Amber Red Evidence has been provided that all criteria have been met as identified in the KPI. Partial evidence can be provided of compliance with a KPI. Serious concerns are identified about one or more element included in a KPI. Assurance has been provided to the extent required in the KPI. The Review Panel is confident that the subject of the ARMs will be able to maintain performance against the KPI for the period until the next Annual Review. Evidence has been provided that the subject of the Review is making all appropriate effort to achieve the objectives outlined in each KPI. Partial assurance has been provided of compliance with a KPI. No serious concerns have been identified; however, outstanding performance has also not been identified. While not all elements of the KPI have not been met, the current level of compliance is not deemed to be unsafe, or of a quality which would impact on levels of patient care. Evidence cannot be provided of compliance with several elements of a KPI. The Review Panel is not assured of compliance with several elements of a KPI. Significant review is required to address issues outlined in the KPI. Panel is not confident that issues can be addressed within a reasonable timeframe. Guidance for RAG Ratings 49

50 Annex 5: Dispute Resolution 50 Quality Improvement and Performance Framework

51 Although a three-step dispute resolution process has been agreed for QIPF, it is expected that Steps 2 and 3 will only be used as a last resort and that the informal resolution process described in Step 1 will be attempted first. The dispute resolution steps Step 1 Step 2 Step 3 Individual EP concerns should first be raised with the WP. The WP will endeavour to find an informal resolution to the problem through discussion and mediation, involving others as necessary. If the concerns cannot be resolved at this level, the matter should be raised by either party (or both) with HEEoE. The Head of Education and Commissioning may convene an appropriately constituted panel to consider the matter further and appoint a mediator, involving others as necessary. If the EP is still not happy, the last recourse in this local process would be to HEEoE. Annex 5 Dispute Resolution 51

52 Annex 6: Aggregation and Rounding 52 Quality Improvement and Performance Framework