Assessing competencies at higher AQF levels

Transcription

1 Assessing competencies at higher AQF levels The assessment of competencies in higher qualifications (Certificate IV, Diploma, Advanced Diploma and Graduate Certificates) presents special challenges for assessors. The traditional approaches to assessment, such as questioning and the demonstration of practical tasks, are usually not sufficient to confirm the competence of team leaders, supervisors, managers and specialist personnel. Interpreting high level competencies The Australian Qualifications Framework (AQF website) provides descriptors for each of the levels of qualifications. As you go up the AQF levels, the nature of the skills and knowledge required becomes more complex. In higher qualifications, the skills include generating designs and ideas, analysing and solving problems, leading teams and developing long-term plans. Often people working at higher levels set and monitor their own work plans and they are not closely supervised. Before you undertake assessment planning and development work, visualise what it means to be competent and develop an accurate picture of the performance required. To do this: Analyse the details of the units of competency Relate the performance requirements to actual job roles and tasks Consider the descriptors of the AQF level Reflect on how the dimensions of competence (in About Assessment > Resources) relate to jobs at the AQF level Comparing descriptors for different AQF qualifications: Certificate III The competencies enable an individual with this qualification to: Demonstrate some relevant theoretical knowledge Apply a range of well-developed skills Apply known solutions to a variety of predictable problems Perform processes that require a range of welldeveloped skills where some discretion and judgement is required Interpret available information, using discretion and judgement Take responsibility for own outputs in work and learning Take limited responsibility for the output of others Adapted from AQF Handbook (pdf 997KB), 2013 Advanced Diploma The competencies enable an individual with this qualification to: Demonstrate understanding of specialised knowledge with depth in some areas Analyse, diagnose, design and execute judgements across a broad range of technical or management functions Demonstrate a command of wide ranging, highly specialised technical, creative or conceptual skills Generate ideas through the analysis of information and concepts at an abstract level Demonstrate an accountability for personal outputs within broad parameters Demonstrate accountability for group outcomes within broad parameters Page 1 of 7

2 For a comparison of the work roles between different AQF levels, see the Appendix for Competency Profiles for workers at Certificate III and Advanced Diploma for a specific industry context. Challenges for assessing high level competencies Complex competencies may take a long time to acquire and assessors require considerable evidence in order to be able to make a judgement. Where students undertaking study at this level are not currently engaged in relevant employment, the opportunities to gather suitable evidence for assessment may be limited. Particular issues include: Many college-based students are undertaking full-time study following high school and have little or no workplace experience. The training and assessment of students who will be seeking employment after gaining the qualification needs to reflect the broad interests of industry rather than the specific requirements of a particular workplace. The demand for student work experience and work placements often exceeds the capacity for employers to provide opportunities. Also it is not feasible to practise and assess high level management and technical competencies during short placements. High level competencies are often underpinned by a substantial body of knowledge and skills that require considerable time for reflection and development. This does not always match the time pressures for students to complete units. High level competencies are often graded, adding complexity to the assessment process. Complex competencies cannot usually be directly observed in a brief assessment. Assessors need to be able to consider alternative evidence from a variety of situations demonstrating performance over time. What are relevant sources of evidence? As with the development of any assessment tool, assessors will need to: Plan, by clarifying the evidence requirements and selecting methods of collecting evidence that are both practical and cost-effective Design and develop your assessment tools, providing appropriate advice to students and assessors Perform quality checks on your tools, that is, trial, review and validate your assessments With high level competencies, it is appropriate to involve the students in the process of providing evidence of their skills and knowledge. This is often a portfolio of evidence incorporating: Project reports A journal or personal reflection on work A resume (with past experience and qualifications) Photographic or video evidence of performance Case studies Response to scenarios Training records Workplace documents The portfolio of evidence, or aspects of it, can be recorded on a blog, digital story or in an e- portfolio. This can be in a format devised by the teacher or left to the discretion of the student. The currency and authenticity of this evidence must be verified. Page 2 of 7

3 Possible sources of evidence for assessing high level competencies Adapted from Assessing competencies in higher qualifications, from the ANTA Training Package Assessment Materials Kit, 2001 Source of evidence Explanation Examples A: Evidence collected by the assessor A1: Practical demonstration observation of real work or simulation by the assessor or agreed third party video or photographic evidence of performance A2: Third party reports confirmation of consistent performance over time and a range of contexts A3: Questioning/ structured interview confirmation of candidate's application and adaptation of complex procedures ability to meet Key Performance Indicators confirmation of understanding clarification of ethics, values and attitudes review of portfolio for relevance, authenticity and sufficiency establish capacity to handle unforeseen situations, predict and evaluate B: Evidence collected by the candidate and provided to the assessor B1: Personal statement/ resume personal statement and self-assessment using relevant examples of performance resume that outlines past work experience and qualifications operating complex equipment troubleshooting equipment faults presentation at a meeting video of inducting new staff reports from: o supervisor o manager o customer o suppliers o peers Questions about: application of enterprise procedures eg OHS handling of critical incidents application of business goals in planning and evaluation reflection on personal and professional practice reasons why a particular management strategy was used self-assessment against performance criteria outline of work experience personal development activities reflections in diary, journal, blog or e-portfolio nominated referees Page 3 of 7

4 Source of evidence Explanation Examples B2: Workplace documents B3: Training records work outputs that are relevant and current and verified as the candidate s work training outcomes mapped to competency standards in the relevant Training Package B4: Case study critical analysis of performance that is mapped to competency standards in the relevant Training Package response can be on paper or in a journal, blog or e-portfolio B5: Work project completion of a negotiated task to provide evidence of the capacity to analyse, synthesise, predict and evaluate B6: Journal/diary entries B7: Testimonials/ awards record of work roles, tasks and responsibilities, contributions to team outputs, reflection on personal performance, learning reflective narrative record in on-line journal, blog or e-portfolio could include contributions to an on-line forum independent confirmation of personal performance job cards, rosters, shift records, position descriptions minutes of meetings operational plans and budgets recommendations and reports explanations of procedures, manuals calculations, flowcharts, project plans RTO qualification relevant workplace professional development equipment supplier's certificate of training design and implementation of new procedures illustration of ethical practice response to complex workplace scenario marketing strategy for new product or service quality improvement strategy designed and implemented for a work process personal organisation and time management operational planning budgeting for organisation or work section analysis of critical incidents and contingency management reflection on personal performance and development solicited letters from customers company/industry awards for innovation, excellence, achievement Page 4 of 7

5 A model for assessment of high level competencies This model involves empowering the students to play a key role in their own assessment. In many instances, the bulk of the evidence required for assessment will be provided by the students. For this to occur, the assessor and students will need to work together to develop evidence plans that identify the evidence to be provided, the best sources of this evidence and the most effective way of collecting the evidence. This model involves: An initial interview where the requirements of the units are explained and the evidence plan is developed and agreed upon between the assessors and the candidate. The candidate prepares and submits a portfolio of evidence based on the evidence plan. The evidence could be paper-based or in an on-line journal or blog, and can include photos or videos of performance. The assessor considers the content of the portfolio, confirming where the evidence meets the requirements of the units and identifying any areas where further evidence is required. A final interview to: o clarify any issues about the portfolio, including asking questions focusing on gaps. o ask questions of the student to explore additional aspects of their skills and knowledge like reflection, evaluation, problem-solving and ability to transfer skills to unfamiliar situations. The assessor will need to have a set of prepared questions as a guide. As well, the assessor will need to respond to the student s answers with appropriate probing questions to explore further any areas requiring additional evidence. o confirm the authenticity of the evidence submitted, based on targeted questioning For further information go to About Assessment > Resources: Evidence Plan Template in Checklists and Templates. Examples of completed evidence plans for two different qualifications Record of assessment outcome suitable for recording diverse evidence related to a workplace or simulated project. Example of a completed assessment record. Example of a Third party report Template. Assessing evidence from different workplaces showing how evidence for the same unit of competency might look very different for two students from different work contexts. The content in this reading is adapted from Assessing competencies in higher qualifications, from the ANTA Training Package Assessment Materials Kit, 2001 (in About Assessment > Resources Page 5 of 7

6 Appendix: Sample competency profiles for workers at Cert III and Advanced Diploma In the Training Package for laboratory workers, MSL10, job and competency profiles are provided to assist in the interpretation and application of the qualifications at different AQF levels. They are reproduced here: Job profile for Laboratory Assistants (Certificate III) Laboratory Assistants help to maintain the laboratory and undertake straightforward tasks that support the safe and efficient delivery of laboratory services to customers. They may perform straightforward sampling and testing by following set procedures and recipes and applying basic technical skills and scientific knowledge. The majority of their work involves a predictable flow of parallel or similar tasks within one work area. They may also perform a range of administrative tasks. Two examples of their work are given below. - A laboratory assistant at a dairy factory gathers samples from the milk tankers, vats and the processing line, and performs routine chemical and bacteriological tests on the samples - A laboratory assistant in a pathology laboratory receives and prepares tissue samples. Job profile for Laboratory Supervisors (Advanced Diploma) Laboratory supervisors are generally responsible for the planning, allocation of tasks, coordination, quality assurance, recording and reporting of laboratory outputs within their section. This requires significant judgement about work sequences, choice of appropriate technology and procedures to ensure that products and services meet customer expectations and are provided safely and efficiently in keeping with enterprise business plan. Laboratory technical officers apply specialist technical skills in addition to the full range of laboratory skills to specific projects with minimal supervision. Laboratory technical officers are required to apply considerable technical insight, well developed analytical skills and possess the ability to apply in-depth specialist technical knowledge to determine methods of approach from a range of possible alternatives. Under broad direction from scientists/medical staff/engineers the laboratory technical officer/supervisor accepts responsibility for the day-to-day operation of his/her work/functional area. They are often responsible for the effective implementation of operational policies and the technical training of personnel in their work area. They also contribute significantly to the development of these policies through the application of specialised technical knowledge. The work of laboratory supervisors involves frequent peak periods, multiple and competing demands and frequent interruptions. Immediate decisions are often required. They must be adaptable to deal with the demands brought about by any of a number of causes. For example: - a range of demanding clients, suppliers or contractors - changes in technology - regularly changing priorities. An example of the work of a laboratory supervisor is given below. A laboratory supervisor in a large water and sewerage utility company has been a senior technical officer for more than five years. The officer supervises technical personnel in the environmental testing section, monitors the quality of their work, oversees their training and ensures that regulatory and NATA requirements are met. The officer assists with the planning of the section's work program and advises management and customers about test schedules, results and methodology. Page 6 of 7

7 Application of dimensions of competency at different AQF levels Dimension of competency Task skills Task management skills Job role and function skills Contingency management skills Transfer skills Competency profile for Laboratory Assistant (Certificate III) clean work preparation areas and equipment dispose of waste store laboratory equipment and materials perform straightforward sampling and testing package, label, store and transport samples process and record data follow set procedures handle a predictable work flow of parallel or similar tasks work under close and regular supervision take decisions within defined limits work as part of a team recognise and report out of control conditions solve predictable problems refer complex problems to senior staff adjust to changed procedures and work situations Competency profile for Laboratory Supervisor (Advanced Diploma) plan, allocate tasks and record/report work outputs for work area conduct a wide range of complex and specialised tests determine work sequences determine appropriate methods and procedures handle frequent peak periods, multiple and competing demands and frequent interruptions work under general direction of manager take responsibility for daily operations and personnel in work area maintain and promote quality assurance, OHS and compliance requirements explain complex instructions and procedures to others provide workplace training and assessment contribute to the development of policy and procedures liaise and negotiate with outside organisations and customers provide technical information to customers define and solve complex problems by investigating, developing and testing alternatives deal with a range of demanding clients, changes in technology and changing priorities ensure services and products are provided safely and efficiently develop or adapt methods and optimise instruments for non routine samples troubleshoot processes, products and equipment deal with change in organisation, work processes and management Adapted from Assessing competencies in higher qualifications, ANTA TP Assessment Materials Kit, 2001 (in About Assessment > Resources) Page 7 of 7