Ensuring Quality & Comparability of RTO based assessments 7 May 2014 Shelley Gillis
Concerns with Quality and Comparability Concerns have been raised about the Quality of Assessments being undertaken by RTOs Comparability of Standards OECD (2008) Reviews of VET (Australia) NQC (2008) Industry Expectations of VET Service Skills SA (2010) VETiS Project Skills Australia (2011) Skills for Prosperity AWPA (2013) Future Focus National Workforce Development Strategy
Quality, Comparability and Standards? Quality Quality as Exceptional or as Excellence Quality as Perfection or Consistency Quality as Value for Money Quality as Fitness for Purpose Quality as Transformation Harvey & Green, 1993, Harvey 2006 Consistency of Assessment Outcomes (comparability) Achieving the same standards in qualifications issued by different RTOs
Concerns with Quality and Comparability Quality as Fitness for Purpose Concerns Skills Australia (2011) Skills for prosperity - a roadmap for VET. Recommended external validation as a mechanism to lift the quality, rigour, validity and consistency of assessment outcomes.
COAG Communique In April 2012, COAG signed up to a set of reforms to the national training system. One of the key reforms related to VET provision included: improving the confidence of employers and students in the quality of training courses, by developing and piloting independent validation of training provider assessments and implementing strategies which enable TAFEs to operate effectively in an environment of greater competition
Independent versus External No universal definition of Independent Validation Tendency for the terms external and independent to be used interchangeably Has design options implications
Independent Validation Options Consensus meetings: involves a panel (with majority of participants external to the RTO) reviewing assessment tools and outcomes as part of a group. Rigour and quality control External approaches (validation and moderation): involves site visits or central data collection of samples of assessment tools and/or judged candidate evidence to be reviewed by external assessors, specialists or panels with authority. Statistical : involves the use of externally developed and administered common assessment task(s) to review and/or moderate RTO based assessments. RTO autonomy
Approaches being piloted Consensus meetings RTO managed processes Externally managed processes External Validation Small panels of subject matter experts who are external and independent to the RTO reviewing tools through site visits and/or central collection; Satisfaction type surveys of key stakeholder groups (e.g. employers and recent graduates)
Challenges No universal definitions of the terms: Independent Industry Issues of sustainability given resource intensive Consensus and External will not necessarily ensure comparability of standards at the national level
Comparability Concerns Comparability Concerns OCED (2008) Reviews of VET (Australia) AWPA (2013) Future Focus: 2013 National Workforce Development Strategy. Recommended the use of common assessment tasks to bring standards into alignment.
OECD Review of VET in Australia In the absence of national assessment, there is no standard to ensure that a particular set of skills has in fact been acquired [and that] consistency in standards throughout Australia should be achieved through a common assessment procedure to determine whether the necessary skills have been acquired (OECD, 2008; p.36-37).
Capstone Testing A more structured solution to the ongoing concerns about quality and consistency might be to require VET students, initially in high risk VET qualifications, to complete a final mandatory assessment that is externally set and administered.a final assessment that is fully independent of the delivery RTO might go some way to restoring confidence in the quality of VET qualifications and provide a guaranteed consistency of outcomes (AWPA 2013, p.133)
Standardised assessment within TAE10 assessments for the Training and Education Training Package would ideally require a written examination focused on VET pedagogy supplemented by submission of teaching resources and a practical involving the student delivering a teaching session. However, a starting could be a written test completed on-line (AWPA, p.134).
Emerging Issues to be considered Is it possible to achieve comparability of standards through independent validation? If no, Should externally developed and administered tests be introduced for high risk qualifications? Would the introduction of externally developed and administered tests assure quality? Could quality be compromised in the pursuit of comparability? Would a combination of RTO based assessments and externally administered tasks help assure quality and comparability? Are there other options yet to be considered for assuring quality and comparability at the national level?
Associate Professor Shelley Gillis Education Policy and Leadership Unit PHONE +61 3 9035 4925 EMAIL sgillis@unimelb.edu.au Copyright The University of Melbourne 2011