Specialty Selection Test (SST) Pilot INFORMATION PACK

Transcription

1 Specialty Selection Test (SST) Pilot INFORMATION PACK Page 1 of 10

2 Contents Section 1: Introduction... 2 Section 2: One-page overview... 2 Section 3: FAQs for Applicants... 3 Section 2: FAQs for LETBs/Deaneries/Specialties... 9 Section 1: Introduction As part of the Medical and Dental Recruitment and Selection (MDRS) Programme, Health Education England (HEE) is working with the devolved nations, the Work Psychology Group and academics in the University College of London Medical School to pilot a Specialty Selection Test (SST). The SST will be piloted across a number of different medical specialties, across the United Kingdom, and includes a Clinical Problem Solving Test (CPS) and a Situational Judgement Test (SJT). The purpose of this pilot is to evaluate whether these tests enhance the current methods of selection used for ST1/CT1 level recruitment. This information pack has been developed to assist with raising awareness about the SST pilot; the objectives, the approach, and how you will be involved. The pack contains a one-page overview document and Frequently Asked Questions (FAQs). Your feedback is important to us, so please don t hesitate to contact the team via at: help.sstpilot@wm.hee.nhs.uk if you have further questions. Section 2: One-page overview The one-page overview document (appendix 1) is a high-level brief that covers the implementation approach, who s involved and why we are doing this. This document was distributed during August Page 2 of 10

3 Section 3: FAQs for Applicants 1. What is the SST pilot and why are we doing this? As part of the Medical and Dental Recruitment and Selection (MDRS) Programme, Health Education England (HEE) is working with the devolved nations, the Work Psychology Group and academics in the University College of London Medical School to pilot a Specialty Selection Test (SST). The SST will be piloted across a number of different medical specialties, across the United Kingdom, and includes a Clinical Problem Solving Test (CPS) and a Situational Judgement Test (SJT). The purpose of this pilot is to evaluate whether these tests enhance the current methods of selection used for ST1/CT1 level recruitment. We are seeking doctors help to participate and provide feedback about the pilot test. The results from this pilot will enable a group of leading selection researchers to consider whether there are any advantages with introducing this test and how the results correlate with current selection methods. We will also continue to follow applicant progression for a number of years, to ensure a long-term robust evaluation. Ultimately, we are assessing whether the SST is a good predictor of a doctor s performance throughout their training and to ensure the selection process is reliable, valid, fair and cost-effective. 2. How will the SST pilot work? We plan to pilot the SST between January 2014 for applicants to specialty training at ST1/CT1 level. On 11 December 2013, all applicants for ST1/CT1 will receive an invitation to take part in the Specialty Selection Test pilot. The test will be delivered online through Pearson VUE test centres in the same way the General Practice (GP) test is run. You will be asked to choose a time from a range of time-slots and a venue from one of many Pearson VUE driving test centres across the United Kingdom (UK). These venues are located within close proximity to hospitals across the UK. The test itself will consist of a two hour and fifteen minute online test. It will be completely computer based and the questions will be developed by leading specialists according to person specifications and foundation programme curriculum. If you are already doing the Page 3 of 10

4 GP test as part of your assessment for GP or Broad Based Training (BBT) you will not be asked to do the additional test. If you are applying to a number of different specialities you will only be required to sit the test once. 3. Who is eligible to sit the SST pilot? Any doctor applying for ANY specialty training post at ST1/CT1 level is eligible to sit the test for the Specialty Selection Test pilot. All applicants will be asked to take part - especially those in smaller specialties with fewer applicants - as we need to ensure that enough trainees sit the pilot to provide enough data for a thorough assessment. 4. What will the SST pilot evaluate? The pilot will evaluate the correlations of performance in the selection test to the selection decision, and where applicable, to scores from currently used assessment tests as well. It is important to note that the test results will not form part of the actual selection process. We aim to introduce a testing method that is fairer, more consistent, offers more transparency and is less labour intensive, and so we will be assessing these components too. By taking part in the pilot, your results and feedback will enable the team to evaluate the selection process and introduce improvements to the way in which doctors are selected for specialty training posts. Comparisons to achievements in on-job performance and in professional exams will also be made at a later date as ultimately, we want to assess whether the multispecialty test results can predict how you perform during specialty training. Page 4 of 10

5 5. Will the results contribute to recruitment scores or be passed on to regulating bodies? Your results will not form part of the recruitment process nor will your results impact on your fitness to practise. This is a pilot evaluation study only and as such, individual results will not be shared with regulatory bodies. Your results will only be seen by the research team who will evaluate the correlations of performance in the selection test to the selection decision, and where applicable, to scores from currently used assessment tests as well. It is important to note that the test results will not form part of the actual selection process. 6. Which specialties are involved in the SST pilot? A detailed list will be made available shortly. Discussions have taken place with Medical Royal Colleges to be involved in and support the SST pilot, and most have confirmed their support and willingness for their applicants to be involved in this pilot evaluation. 7. Why are we piloting a Specialty Selection Test hasn t this been piloted already? Similar tests and pilots have been used before in many specialties, but not across specialties currently 7,000 out of the 12,000 doctors applying for specialty training take a Clinical Problem Solving (CPS) and a Situational Judgement Test (SJT). The test we are piloting will be similar to that used in General Practice (GP) and Public Health and similar to the specialty specific tests that have been piloted in surgery, acute specialties, medicine and other specialties. The test in GP was successfully implemented in 2007 and used to date as part of the selction process, constituting 30% of a trainee doctor s total recruitment score. In comparison with other methods of selection, these tests have been shown to be the best predictor of performance in the work place and likelihood of successful completion of specialty training. Applicants consistently rate these tests as fair and appropriate to use as sub-components of speciality selection. Each of the devolved nations is focused on the National Health Services (NHS) objective to recruit staff into the NHS with appropriate values and behaviours, as well as knowledge and skills. The Specialty Selection Test will support the need to incorporate values-based selection methods and is a possible way of meeting these requirements. Page 5 of 10

6 This pilot is an evaluation of selection processes for the benefit of our patients and you, as an NHS doctor. 8. How will I benefit from taking part in the SST pilot what s in it for me? This pilot aims to improve the way doctors are selected in the future and this is your chance to give your views on this selection process and make your voice heard! By participating in the pilot, you will also receive feedback on how you performed and go into the draw to win one of five prizes, which includes a 32GB ipad and Kindle Fire HD. As an applicant, your feedback will be viewed as very important and you will be able to have your say on how appropriate you found the selection test to be in relation to selection into specialty training, and whether you think it allowed you to demonstrate your aptitude for specialty training. This information will then be used to inform the Specialties and other stakeholders on whether these selection methods add additional value to the selection processes currently used. 9. Why will the pilot test be across multi-specialties and not specific to the specialty that I'm applying for? The pilot of the SST is designed to be broad-based due to the nature of the overlap of person specifications required by all specialties. A good example of this is the fact that all specialties require trained doctors not just to be clinically competent, but also empathic and sensitive with patients and colleagues, behaving with professional integrity, with the ability to prioritise and cope well under pressure and work effectively in a team. All doctors who have undertaken Foundation will be familiar with the Foundation Curriculum by which these skills, values and behaviours are embedded. The Specialty Selection Test does not seek to replace individual specialty specifications. Each speciality will still have its own speciality specific elements within the selection process. Page 6 of 10

7 10. Have CPS and SJT tests been used before and if so, by whom? Yes. The CPS and SJT test is successfully being used currently for applicants to General Practice (GP) (since 2007) and Public Health (since 2009). Approximately 7000 out of 12,000 applicants to ST1/CT1 training already sit the test each year; though it is currently specific to the specialty they are applying for. A multispecialty pilot (of 800 applicants) was undertaken in 2010 across several ST1/CT1 specialties. The pilot test performed well, with the results correlating well with the selection scores, but there wasn t enough involvement across all specialties to determine whether the test would be beneficial for all specialties. This is one of the reasons why we are undertaking a Specialty Selection Test pilot across a wider range of specialties. 11. Is it mandatory to take part in the SST pilot? It is not mandatory to take part in the SST pilot. If you are applying for GP specialty training or Broad Based Training (BBT) then you will be required to sit the GP selection test as it is part of the assessment centre for entry into GP training. Questions from the SST pilot will be embedded in the GP test, but will not count towards your score in the selection process. This will allow us to study how well the multispecialty questions perform compared to items in the GP test. 12. Will travel expenses be reimbursed? Yes, standard class travel expenses will be reimbursed for the SST pilot project up to the value of There is a large selection of Pearson VUE test centres, which are located within close proximity to hospitals across the UK and should be easily accessible. Page 7 of 10

8 13. Do I need to do the test if I m applying for General Practice (GP) or any other specialty? As indicated above, those applying to General Practice (GP) training and Broad Based Training (BBT) who already undertake an online test will not need to sit an additional test for the SST pilot. If you are applying to specialties other than General practice, we are seeking your support to undertake the test and we will request your participation. As such, your involvement in the pilot would be greatly appreciated. 14. Will the mark I achieve be known to anyone involved in selection into my chosen specialties? Absolutely not! Your individual results will only be seen by the research team who will be evaluating results to determine whether the SST pilot test adds value to the specialty recruitment selection process. 15. How will the security of the data about me and my results be ensured? The results and information obtained from this pilot will be anonymised and secured using the same measures that are currently in place to protect selection results for General Practice (GP). We will use secure file transfer protocols to manage your information, ensuring that your data is kept completely secure. 16. Will the SST be used in the selection process for shortlisting during 2014 specialty training recruitment? Your performance in the piloting of the SST will not in any way affect your selection scores for entry into specialty training in 2014 at ST1/CT1 level. It will not be used for shortlisting or at any other stage of the specialty training process. A poor performance in the pilot will not have any impact on your application to a specialty training post at ST1/CT1 level. Page 8 of 10

9 17. Where can I find out more information? You can contact us at help.sstpilot@wm.hee.nhs.uk for more information. The Specialty Training website will be updated regularly and the applicant guide is another useful source where you can find more information about the pilot and what is involved. Page 9 of 10

10 Appendix 1 A Specialty Selection Test is being piloted for entry into Specialty Training As part of the Medical and Dental Recruitment and Selection (MDRS) Programme, Health Education England (HEE) is working with the devolved nations, the Work Psychology Group and academics in the University College of London Medical School to pilot a selection test that includes a Clinical Problem Solving Test (CPS) and a Situational Judgement Test (SJT), for entry into Specialty Training. Known as the Specialty Selection Test (SST) pilot, this will be piloted between January 2014 for applicants to specialty training at ST1/CT1 level. The results from this pilot will enable a group of leading selection researchers to consider any advantages of introducing this test and how the results correlate with the current selection methods used. We will share these results widely with stakeholders and continue to follow applicant progression for a number of years for a long-term evaluation. Ultimately, we are assessing whether the SST is a good predictor of a doctors' performance throughout their training and to ensure the selection process is reliable, valid, fair and cost-effective. Similar tests and pilots have been used before - currently 7,000 out of the 12,000 doctors applying for specialty training take a CPS and a SJT 1. Compared to other selection methods, these tests have been shown to be the best predictor of performance in the work place and likelihood of successful completion of specialty training. The National Health Service (NHS) needs to recruit staff into the NHS with appropriate values and behaviours, as well as knowledge and skills. This principle has been incorporated into NHS mandates and organisation objectives. The Specialty Selection Test is a values-based selection method and is a possible way of meeting these requirements. Applicants will be asked for their views after completing the test on how appropriate they find the selection test in relation to selection into specialty training. Discussions have taken place with Medical Royal Colleges and most have confirmed they wish their applicants to be involved in this SST pilot. Applicants taking part in the pilot will sit the test only once, irrespective of the number of specialties they apply to. All identifying information will be kept confidential. The results will not form part of the selection process and will not be made available to selectors - it is a pilot only. Input from applicants is crucial and results will inform us about whether these selection methods add additional value to the selection processes currently used. On 11 December, all eligible applicants will receive an invitation, once they have applied to a specialty, and will be able to choose a test centre that is most convenient to them. This will be managed separately to the online application process for recruitment into ST1/CT1 level. As a member of a Local Education Training Board (LETB), Deanery or Royal College recruiting organisation, you will receive further, specific, information regarding how this process will be managed. HEE will offer support during the SST pilot from a staff member centrally within the Directorate of Education and Quality (DEQ) who will respond to applicant queries specifically related to the pilot. Frequently Asked Questions will also be available and the applicant guide will include relevant information about the pilot. For more information, please contact the team via at: help.sstpilot@wm.hee.nhs.uk. 1 General Practice (GP) and Public Health use a CPS and a SJT test for specialty selection. Specialty specific tests have also been piloted in surgery, acute specialties, medicine and other specialties. The test in GP was successfully implemented in 2007 and used to date as part of the selection process; constituting 30% of a trainee doctor's total recruitment score. Page 10 of 10