Assessing competencies evidence level, sufficiency, efficiency Richard Scott, Professional Lead

Transcription

1 Assessing competencies evidence level, sufficiency, efficiency Richard Scott, Professional

2 Assessment in the workplace: Physical Sciences Work based assessments The number of work based assessments required for each module varies for each specialism. Details are in the trainee handbook, on OneFile and the NSHCS website 1 st year - Need complete 1 DOP/OCE and 1 CBD for each rotation

3 Today s aim Assessing competencies What is evidence? Is it sufficient? How much evidence is enough?

4 Objectives What is competence and competency? Recognise what traits we are seeking to develop How assessment of competency can be applied What is suitable evidence; quantity versus quality?

5 What is competence and competency? Competence: the ability (or qualification) of an individual to do a job properly Competency: a set of knowledge, skills, experience and other attributes necessary to do the job properly Competence is the outcome and competencies are the inputs to achieve the task Synonyms: capability ability capacity proficiency accomplishment adeptness adroitness knowledge expertise expertness skill skilfulness prowess mastery resources faculties facilities talent bent aptitude artistry virtuosity savvy know-how

6 Assessment hierarchy Level 1 Awareness The trainee has been introduced to the process/procedure associated with the competency: Level 2 Performance The trainee has repeatedly performed the process/procedure (supervised) with increasing confidence: Level 3 Proficient The trainee has demonstrated repeated successful performance of the process/procedure (indirect supervision): Level 4 Competent The trainee performs the task(s) referring infrequently to their supervisor as required:

7 Assessment hierarchy: Level 1 Level 1 Awareness The trainee has been introduced to the process/procedure associated with the competency: The trainee has read all relevant SOP s, COSHH and Health and Safety and other recommended documents. The trainee has an introductory level of knowledge and understanding of the application of the process/procedure. The trainee has been shown how the process/procedure is performed and allowed to perform the task(s) under supervision. The trainee requires direct supervision. This Level may be applied for competencies and assessments undertaken on rotation in the first year.

8 Assessment Hierarchy: Level 2 Level 2 Performance The trainee has repeatedly performed the process/procedure (supervised) with increasing confidence: The trainee has a knowledge and understanding of the task(s) and is able to identify situations when they should be applied or are relevant. The trainee performs the task(s) with few or no errors and asks fewer questions related to the task. The trainee may only require indirect supervision. This Level may be applied for competencies and assessments undertaken on rotation in the first year and when performing tasks from Specialist Modules.

9 Assessment Hierarchy: Level 3 Level 3 Proficient The trainee has demonstrated repeated successful performance of the process/procedure (indirect supervision): Trainee has developed a level of knowledge & understanding of the competency that allows them to critically analyse the task(s) and outcomes produced. The trainee is able to identify potential sources of error and can correctly resolve problems that may occur. The trainee is able to successfully perform the task(s) without supervision. This Level is achievable for most procedures from the Specialist Modules.

10 Assessment Hierarchy: Level 4 Level 4 Competent The trainee performs the task(s) referring infrequently to their supervisor as required: The trainee has repeatedly demonstrated a level of knowledge, skill and aptitude of the competency to work with a level of independence but still recognising their scope of practice. They are able to demonstrate or train other staff in the processes and procedures relevant to the competency. This Level is expected for most procedures from the Specialist Modules. This Level compliments the Good Scientific Practice Domain of Clinical Leadership: Readiness for practice e.g. Shows competency at a level that is appropriate for a newly registered clinical scientist, is a safe pair of hands, dependable, trustworthy, efficient, knowledgeable about their specialism.

11 Competencies and portfolios One File defines the Competencies, or work activities, to be Undertaken and Understood for each trainee. Trainee needs to complete all of these, and create a portfolio of supporting evidence. The evidence supports the trainee s own learning as a record of what was actually done for future reference/e-portfolio for CPD/Professional Regulation purposes. Assessor needs to review, comment, and sign off the evidence.

12 Decisions on competency Consider the levels described and how they may apply differently to rotations and specialisms during the course of training. Each module has a number of assessments and these provide an opportunity to demonstrate competency. Use the detail given on OneFile under each competence where there are suggested knowledge and understanding statements. Level 1 Level 2 Level 3 Level 4 Awareness The trainee has been introduced to the process/procedure associated with the competency Performance The trainee has repeatedly performed the process/procedure (supervised) with increasing confidence Proficient The trainee has demonstrated repeated successful performance of the process/procedure (indirect supervision) Competent The trainee performs the task(s) referring infrequently to their supervisor as required

13 Finding evidence for competencies There is considerable variation in the amount and type of evidence and feedback being uploaded on e-portfolio Providing evidence is not just a tick box exercise! But, don t overload the trainees by asking for 10,000 word essays! Requires good communication between the Training Officer and trainees Why not ask other 2nd and 3rd year trainees What isn t right: My supervisor saw me do this 10 page extract from standard textbook or SOP What about? Demonstrates that the trainee undertook it, and understood it. Supervisors have a responsibility to make sure the evidence is all the trainees own work.

14 Evidence for competencies Ensure: that a competency is completed for each Learning Outcome that it applies to; that there is supporting evidence for each Learning Outcome; that they relate to the work of clinical scientists in that specialism; Evidence should show that the trainee Undertook and Understood the activity; Many competencies can be grouped together, and single pieces of evidence can be used to demonstrate their completion. Project work can be a very motivational and efficient way to complete competencies; Discuss types of evidence with your trainee; Encourage them to be innovative; Avoid essays for every competence; Use evidence wisely; Upload copies of work completed this may be local documentation; Ensure patient identifiable material is not used. Get them to do it as they go along and not all at the end

15 Evidence for competencies Make sure: John Smith John Smith that a competency is completed for each Learning Outcome that it applies to. that there is supporting evidence for each Learning Outcome

16 Competencies and Assessments The Curriculum Library The STP (and HSST) curricula are available online via the School s Curriculum Library. Work-based competencies and assessments are listed for each specialism. The principles, values and standards of behaviour and practice of healthcare scientists are contextualised through the Academy s Good Scientific Practice (GSP). GSP maps to the HCPC Standards of Proficiency which are the professional standards which every Clinical Scientist must meet in order to become registered, and must continue to meet in order to maintain their registration. STP curricula are designed to meet these Standards Competencies and assessments are mirrored in the trainees record on OneFile.

17 Example: Physical Sciences

18 Looking for evidence Documents that support the delivery of the training e.g. results and brief summary documents that evidence knowledge skill and understanding If the evidence relates to a diagnostic test: use anonymised pre and post test results during the assessment ensure the trainee understands the impact on the patient Good evidence includes: flowcharts, witness statements, trainee presentations, videos, photos reference to regulations & supporting literature Use evidence prepared for equipment validation and ISO audits as evidence on OneFile Consider a project based approach, and using one set of evidence to satisfy multiple competencies

19 Level of knowledge expected Appropriate to Master s degree level A systematic understanding of knowledge, including, where appropriate, relevant knowledge outside the field and/or discipline. A critical awareness of current problems and/or new insights, much of which is at, or informed by, the forefront of their academic discipline, field of study, or area of professional practice. M a s t e r s

20 Example of evidence: Physical Sciences Submission Competencies & Standards [[SPE104] Imaging With Non Ionising Radiation [SPE104 Competencies] Competencies [13] Make patient exposure measurements and performance testing on a range of non-ionising equipment Competencies [14] Critically appraise an emerging modality [15] Undertake online and offline analysis and interpretation of the results [5] Perform routine quality control measurements on a clinical magnetic resonance scanner Evidence: No additional evidence provided Evidence Attachments MRI Exposure Measurements - INIR-C-13 - OW v2.docx MRI QC Competencies - INIR-C-5,15 - OW v3.docx Raman Spectroscopy Emerging Modality- INIR-C-14 OW.pptx Feedback & Comments No comments provided. Declaration & Signatures Only sign here when you have finished with this submission and wish to send it to the next person. Declaration Name Signed on Date Completion I agree that the information provided here Trainee þ 31/10/ :47 is an accurate account of what has taken place. I agree that the information provided here Assessor þ 31/10/ :48 is an accurate account of what has taken place.

21 Signing off evidence: Options Approve The evidence meets the required standard. How to respond to a submission in OneFile When you receive a submission from a trainee you have 3 options: Reject The evidence doesn t meet the standard. Offer helpful feedback as to what is required. Resubmit The evidence is partly complete. Provide feedback on what additional work is required Approve Tick the competency or standard Sign the declaration Click Save and quit Reject Provide your feedback Don t tick the competency or standard Sign the declaration Click Save and quit 1 2 Resubmit Provide your feedback Click Resubmit Your trainee will be prompted to do more work on the submission

22 Giving feedback; encourage reflection Evidence supports the trainee s own learning as a record of what was actually done for future reference/e-portfolio for CPD/Professional Regulation purposes Feedback is crucial to supporting the trainee s learning. It needs to be supportive, constructive and allow 2-way communication Reflection is also a very important aspect to the trainees learning and can be used to add to evidence already uploaded and accepted. And, to cover learning and activities covered in the Clinical Experiential Learning (start of each module) I liked how you explained the expected learning outcomes Can you tell me about the actions you used to achieve this? You were clear and precise when saying the key new concepts you learnt You were able to evidence how your knowledge has changed

23 Monitoring progress Monitor the trainee s progress throughout. The School will also be doing this. Evidence of good progression will be necessary. Lack of evidence on OneFile could have implications on a trainee s ability to be allowed to progress to the next stage of the programme. Engage with the summative Mid term Review

24 Summary Use the detail in the Curriculum Library - next to each competence there are knowledge and understanding statements Good constructive feedback is essential to develop your trainee Encourage reflective practice Apply competence(y) appropriate for the level of training Remember the professional practice competences!

25 Further examples of evidence Additional theme specific examples can be found at:

26 Questions?