P22 - Assessment Tools Validation Policy and Procedure

Transcription

1 P22 - Assessment Tools Validation Policy and Procedure 1.0 Purpose Doc ID: P22_Validation_v2.6 Assessment validation refers to a process where Baxter Institute compares and evaluates assessment tools against the relevant competency standard/s to ensure, validity, reliability, fairness, flexibility and effectiveness of their: assessment methods assessment tools including RPL kits assessment procedures assessment decisions assessment outcomes The process will involve validators in professional conversation to discuss and reach agreement about qualifications/ assessments and outcomes in a particular industry and ensure that the way a unit or module is assessed, and the evidence collected is consistent with the unit or module requirements. This process aims to support the continuous improvement process and to demonstrate compliance with Standards for RTOs Clauses 1.9, 1.10 and As a result of the validation process, learning and training resources will also be reviewed and continuously improved. Baxter Institute will implement P22-Assessment tools validation policy by: 1. Implementing a plan for ongoing systematic validation of assessment practices and judgements that includes each training product on Baxter s scope of registration. 2. Reviewing, comparing and evaluating assessment processes, tools and evidence contributing to judgements made by a range of assessors against the same competency standards. 3. Documenting any action taken to improve the quality and consistency of assessment tools or practices. 4. Reviewing learning resources considering the findings from validating assessment tools. 2.0 Responsibilities 2.1 The director of studies coordinates the implementation of the validation Policy and ensures that trainers and assessors are aware of its application and monitors its compliance. 2.2 Responsibility for managing the implementation of the Validation Policy, for each qualification/unit is delegated to the course coordinator/head trainer, for that qualification. 2.3 Systematic validation of the assessment practices and judgement is undertaken by a group of one or more persons who are not directly involved in the particular instance of delivery and assessment of the training product being validated, and who collectively have: i) Vocational competencies and current industry skills relevant to the assessment being validated; ii) Current knowledge and skills in vocational teaching and learning; iii) The training and assessment qualification or assessor skill set as required. 3.0 Definitions Assessment: The process of collecting evidence and making judgements on whether competency standards have been met, to confirm that an individual can perform to the standard expected in the workplace, as expressed by the relevant endorsed industry/enterprise competency standards of a training package or by the learning outcomes of an accredited course. Assessment tool: Assessment tools (including RPL kits) provide a means of collecting the evidence that assessors use to make judgments about whether students have achieved competency. An assessment tool includes the following components: the context and conditions for the assessment; This is an uncontrolled document once it is printed Page 1 of 8

2 the tasks to be completed by the student; an outline of the evidence to be gathered from the student; the assessment criteria used to judge the quality of performance; and administration, recording and reporting requirements. External Participants: All people except those employed by the Institute to deliver and assess any units of competency from any of the qualifications in the training package or course being validated are considered external participants. Validation of assessment: Validation is a quality review process. It involves checking that the assessment tools produced valid, reliable, sufficient, current and authentic evidence to enable reasonable judgements to be made as to whether the requirements of the relevant aspects of the Training Package or accredited course have been met. It includes reviewing and making recommendations for future improvements to the assessment tool (including RPL kit), process and/or outcomes. 4.0 Requirements Pre-Assessment Validation: Pre-assessment validation is when newly developed assessment tools are validated prior to being released to assessors for the purpose of assessing students. Post- Assessment Validation: Post-assessment validation is when newly developed assessment tools have been released to assessors for the purpose of assessing students and therefore after collecting feedback is validated and moderated for on-going improvement. 4.1 Validation will occur before, during and/or after the assessment activity. Validation before assessment (pre-assessment/ One part of Quality check): Prior to implementing a new assessment tool, Baxter Institute conducts several checks (including but not limited to): Inviting people for review to confirm if the content of the tool is correct and relevant to the workplace. Industry representatives could critique the tool for its clarity, content accuracy, relevance and appropriateness of language for the learners. Reviewing the tool with other trainers and assessors, who may not be industry experts but who have current skills and knowledge in vocational training, learning and assessment including competency based training and assessment), can check whether: - the tools will address all assessment requirements of the training package or accredited course - the level of difficulty is appropriate to the unit or module - the tools will enable effective collection of evidence, and - you have provided clear instructions are which can be easily understood by the learner, third party and assessor. Trialling the tool before it is used by learners tests the effectiveness of the tool without affecting a learner. When trialling a tool, select a group of individuals who have similar characteristics to, and levels of ability of, the target learner group. The findings of the trial will help predict whether the tool would: - be cost effective to implement - be engaging to the learner, and - produce valid and reliable evidence. Validation during assessment will focus on: the actual performance being undertaken by a learner the assessment process and the role of the assessor/trainer in the process The feedback from this validation process is collected from assessors and from learners after an assessment has been conducted. All feedback will be reviewed and validated collectively at the This is an uncontrolled document once it is printed Page 2 of 8

3 scheduled Validation after assessment activities/meeting. The following forms will be used for the collection of feedback from students VT257 Post Assessment Student Feedback Form and for assessors VT252 Post Assessment Assessor Feedback Form. Validation after assessment (post-assessment) will focus on: the effectiveness of the assessment tool (including RPL kit) and the assessment process the standard of performance achieved the validity of the evidence collected reporting and record keeping consistency of outcomes as determined by assessors 4.2 During the delivery of lessons or assessments assessors/trainers must record any feedback that would be considered valuable for continuous improvement. Their feedbacks will be documented on the VT252 Post Assessment Assessor Feedback Form. The feedbacks supplied will be given a level of urgency. For low urgency cases, the feedbacks will be considered during in the next scheduled validation activities/meetings or where there is high urgency, the feedback will be actioned sooner as continuous improvement. 4.3 Each training product is validated at least once every five years, with at least 50% of products validated within the first three years of each five-year cycle, considering the relative risks of all the training products on Baxter s scope of registration, including those risks identified by the VET regulator. Steps involved in validation process: 4.4 Establishment of an assessment Validation Plan Baxter Institute implements a plan for ongoing systematic validation of assessment practices and judgements that includes for each training product on our scope of registration a Validation Plan that identifies o When assessment validation will occur; o Which training products will be the focus of the validation; o Who will lead and participate in validation activities; and o How the outcomes of these activities will be documented and acted upon All assessments tools (including RPL kits) will be reviewed over time and it is the responsibility of the director of studies to ensure that the process is followed and all assessments are reviewed. The validation plan will be updated at the commencement of each year taking into account the outcomes of assessment validation activities from the previous year. Validation process and outcomes (per qualification) will be documented on the VT254 Department Assessment Validation Summary Log. 4.5 Identification of units to be validated for the year. A plan for each qualification on scope will be developed and units selected for validation will be aligned according to the delivery schedule. 4.6 Identification of participants to be involved in the assessment validation activities. Internal and external participants will be involved in the activities. This may include teaching staff/trainers, industry representatives, vocational and assessment specialists, professional association representatives, teacher networks, or other appropriate stakeholders. The validation is undertaken by one or more person who are not directly involved in the particular instance of delivery and assessment of the training product being validated and who collectively have: - Current knowledge and skills in the industry relevant to the assessment - Current knowledge and skills in vocational teaching and learning - TAE40110 or TAESS00001 This is an uncontrolled document once it is printed Page 3 of 8

4 4.7 Inform internal assessor/staff of assessment validation requirements. Doc ID: P22_Validation_v2.6 Before each meeting, the director of studies or the course coordinator will finalize the Validation plan to detail the strategies and tools to be used. Validation participants will be briefed on the exact type of materials to be collected for the exercise and the number of qualifications/units to take place. Validation participants will be provided with a validation summary form (VT 253 Post Validation Summary forms) to identify steps in the validation process. During the meeting, minutes of meeting, hand written annotations, marked- up versions of the original, notes and other supporting evidence of the validation must be stored as listed on VT256 Validation Document Collection Checklist. All Baxter s teaching staff will be informed of the proposed validation schedule related to their industry areas. 4.8 Validation before assessment i) Initially, assessment strategies will be developed through consultation with industry to ensure that they are consistent with industry needs. Assessment tools and activities may also be validated in consultation with assessors from within Baxter. ii) Validation before rolling out assessments will focus on the following: The design of assessment activities; Assessment tools that are planned to be used; The benchmarks against which each participant will be assessed including: - the interpretation of the unit(s) of competency - the development of a common understanding of the standard to be achieved, what the competent person would look like - the assessment strategy, the overall design of the assessment process, including types of assessments to be used, location, number of assessments - assessment tools and the evidence guides - the benchmarks against which learner performance is to be assessed 4.9 Validation during assessment i) The comments of assessor/trainer are collected at the end of each assessment tools based on the following criteria: the actual performance being undertaken by a learner the assessment process the role of the assessor/trainer in the assessment process ii) These comments are viewed when conducting Validation activities. iii) Assessor/trainer may note issues they may encounter in the daily delivery of lessons or assessments. This will form part of the feedback collected for the validation activities/meetings Validation after assessment i) Validation will focus on the following: the effectiveness of the assessment tool and the assessment process the standard of performance achieved the validity of the evidence collected reporting and record keeping 4.11 Validation activities are conducted based on the validation schedule and include when changes to the training package have occurred or prior to a new qualification being placed on scope. This is an uncontrolled document once it is printed Page 4 of 8

5 4.12 Validation participants will be provided with Baxter s (Validation forms) outlining the discussion topics to measure against the key criteria below: ensure that evidence guides adequately cover all relevant information such as performance criteria, range statements and key competencies; ensure that the evidence which is collected meets the rules of evidence (validity, sufficiency, currency and authenticity); ensure that the participants are assessed against the level of competence required by the unit of competency; ensure that participants are being assessed taking into account language, literacy and numeracy skills and key competencies to the level required in the competency; ensure that assessment strategies are appropriate for the target group; ensure that there is an agreed understanding of the criteria being used to arrive at an assessment of competent ; ensure that the agreed criteria are being used consistently by all those assessing competency. ensure that relevant stakeholders (i.e. students, trainers/assessors etc.) have given feedback on the assessments and tools used so that they might be improved; 4.13 Sampling of student files: The sampling methodology will include: identifying the total number of completed assessments within the 6-month period prior to the validation meeting. Calculating the sample size using any statistically valid calculator. E.g. Selecting student files to be used as samples by o Create a list of all students who completed the assessments. Each student will be assigned a number o Use a random number generator to select the numbers randomly E.g. During the validation meetings, the selected student files will be reassessed against the criteria listed in Section 4.12 of this Policy Record keeping Form VT 254 Department Validation Log is to be maintained with dates, assessments reviewed for validation, external/internal flags, actions including who they are assigned to, outcomes and completion date. The documents collected together with the form VT254 would include the following and will be kept as evidence to support compliance with Standards for RTOs 2015 a) VT 251 Pre-Assessment Validation Summary form b) VT 252 Post Assessment Assessor Feedback form c) VT 253 Post-Assessment Validation Summary form d) VT 257 Post-Assessment Student Feedback form e) Hand written annotation relevant documents g) Original assessment tool before amendment h) Sampling report i) New version of the assessment tool with the completed amendment k) VT 254 Department Validation Log 4.15 Reporting This is an uncontrolled document once it is printed Page 5 of 8

6 Meeting agendas to be sent to attendees at least one week in advance. Actions and details to be recorded in the Validation Log. Summary of the actions and outcomes are to be recorded in Continuous Improvement Register. Major findings / issues should be consolidated (ensure any evidence of identifying any individual assessor or student is to be removed) to ensure the generic findings and actions are understood across Baxter Institute. Meeting minutes to be sent to attendees after the meeting This is an uncontrolled document once it is printed Page 6 of 8

7 4.16 Review of the system and its effectiveness Conduct internal audit to review that policies and procedures are being followed. Evaluation of strengths and weaknesses e.g. cost effectiveness, continuous improvement, impact on quality are to be carried out annually by the Director of Studies Validation Schedule Qualifications Internal Validation External Validation All qualifications within Baxter s scope of registration Progressively by assessors as student assessments are marked. Once every 4 months in line with annual schedule and unit assessment schedule. Relevant assessment tools (including RPL kits) and procedures will be reviewed within the life cycle of the qualification. External people to be involved in validation meetings at least once per year. End This is an uncontrolled document once it is printed Page 7 of 8

8 Revision History Doc ID: P22_Validation_v2.6 Version Date Description of modifications 1.0 April 2011 Original 1.2 April 2011 Schedule modified/amend to include assessment feedback section 4.2 second dot point 1.3 May 2011 Updated Moderation to distinguish between Non-Practical and Practical assessments review /10/2013 Review and updated procedure /11/2013 Revised relevant assessment to be reviewed within life cycle of qualification /11/2013 Reviewed and added explanation on the usage of form VT255 Personal Validation & Moderation Log and the required documentations to be collected as compliance evidence (section under Records ) /04/2014 To explicitly document that the continuous improvement of learning resources is performed in conjunction with assessment validation and thereby is closely related. Updated input from Industry Consultation in 4.0 Removed from VT255.Removed 4.10 Complaints and Appeals /04/2017 Revised and amended in accordance to Standards for RTOs /09/2017 Revised /11/2017 Updated the address This is an uncontrolled document once it is printed Page 8 of 8