Beyond Academic Credentials: A 360-degree look at the Recognition of Prior Learning

Transcription

1 Beyond Academic Credentials: A 360-degree look at the Recognition of Prior Learning Creating the New Normal : People, Tools, Process and Quality Assurance CAPLA: 2011 Fall Focus Workshop

2 Contributors Christine Nielsen Jan Robertson Jane McLaren (Project Manager) Nigel Lloyd Roman Savka Susan Simosko (Facilitator) Participants!

3 Purposes Explore assessment options to use with entry-level immigrants beyond academic credentials Identify organizational models of good practice Explore system-wide issues of quality assurance Identify strategies for creating the new normal in working with entry-level immigrants

4 Process Experientially based Interactive and explorative Outcomes oriented Flexible but need to keep to time Based on principle that everyone has something to contribute and everyone has something to learn Enjoyable!

5 Webinar: 31 October 2011 Purpose Stimulate our thinking about the use of different assessment strategies with entry-level immigrants Process Used five question to focus inputs from Jan, Christine and Susan Almost 270 people participated Outcomes Generated a list of topics and questions for inclusion in this pre-conference session Supported by The Foreign Credential Recognition Program at HRSDC

6 Getting started: Focusing on key issues What does each group want or need? Immigrants Prior to coming to Canada On arrival Entering the job market Employers Regulators Please refer to relevant case study Use Worksheet 1

7 Looking at the tools Immigrant s goal: To obtain employment that uses the knowledge, skills and abilities previously acquired in work, and through education and training

8 What do immigrants need? Clear statements of Professional entry-level expectations Occupational standards Competencies Standards of practice Framework of professional practice Other Description of the required process with timelines, fees, milestones A clear description of the options for support and/or new learning

9 What do regulators need? They need to verify that applicants meet all professional standards and can practice safely and effectively in a Canadian context Critical knowledge Skills (including language skills) Attitudes and values consistent with Canadian culture and such things as: Relevant legislation Ability to work in a team Work autonomously, etc.

10 What do employers need? They need to verify that applicants meet all organizational requirements Critical knowledge Skills (including language skills) Attitudes and values consistent with Canadian culture and such things as Willingness to learn Getting along with others Flexibility

11 A competence-based RPL model Occupational standards or competencies Skills, Knowledge, Attitudes Values Education Training Self Study Opportunities for self-assessment Preparation for Assessment Assessment Occupation Language Written exam Demonstration Case Studies Oral exam Portfolio Helpful feedback Employment, Licensure, Certification

12 Building Self-Assessment Tools: What do we need? A clear statement of the professional or occupational standards or competencies by which an immigrant can compare his /her own skills and knowledge User-friendly delivery methods Meaningful feedback that will enable people to make sound decisions about their lives We do not want to create false hope!

13 Self-assessment tools Self-directed (used by individual on his/he own) Interpretive (used with peer or mentor) Can be general or specific For immigrants considering coming to Canada we need to make sure they have enough specific information to understand provincial expectations But do not want to overwhelm them either!

14 Written or computer-based examinations Primary purpose is to assess knowledge and problem-solving ability Multiple choice Short answer Essay

15 Written or computer-based examinations Advantages Relatively easy to administer and score High reliability Can be used to complement other forms of evidence Limitations Poor validity of competence by itself (insufficient evidence) Quality depends on the quality of the items and the development of sound technical data Difficult to measure integrated knowledge and professional decision making Many immigrants not familiar with multiple choice exams

16 Demonstrations Simulation Objective Structured Clinical Examination (OSCE) Role play or other classroom-based examples Workplace Observation of candidate in employment Observation of candidate during Practice audit Internships Preceptorships Mentorships

17 Simulation Advantages Provides opportunity to observe candidate interacting with client/patients in a standardized situation Has been shown to be valid and reliable in the field of medicine, nursing and other health professions Can assess currency of skills and knowledge Limitations Costly to develop and maintain Requires extensive training of assessors Administration is complex Not necessarily a good source of evidence since many candidates have exam nerves

18 Work-place/work-based assessment Advantages Assesses candidates actual performance and the application of skills and knowledge at work Enables candidate to demonstrate problemsolving skills Provides opportunity for assessor to rate the candidate against practice standards and provide meaningful (and contextual) feedback Highly valid and high levels of inter-rater reliability possible

19 Work-place/work-based assessment Limitations Must have clear practice standards Must have well-trained assessors (auditors, preceptor, mentors, etc.) Requires a high-level of organizational commitment and resources Requires systematic and well-planned feedback strategies

20 Oral examinations Enables assessors to gather specific information from applicants and to determine their ability to think on their feet

21 Oral examinations Advantages Gives candidate an opportunity to expand on other evidence and demonstrate his/her ability, communication effective and respond to contingencies Provides assessors with direct information about the candidate Limitations Lack of validity and open to subjective scoring Requires the development of clear assessment guidelines and a coherent rating system Best used in conjunction with other assessment methods, not as a stand-alone tool

22 Portfolios of evidence Provides an opportunity for candidates to show a range of direct and indirect evidence related to their performance and/or experience Most often used for formative purposes

23 Portfolios of evidence Advantages Self-directed by candidate Enables candidate to present best self Has the potential to provide diverse and valid evidence of competence Limitations Requires explicit standards Requires the comprehensive training of assessors Does not show how processes were performed Raises issues about reliability and validity Often authenticity and currency need to be checked, e.g. through oral questioning

24 Preparation for assessment Immigrants (all learners!) need: Examples of different types of assessments with answers for written examinations and case studies and an explanation of what is being assessed Examples of portfolios with critiques and criteria for their evaluation Descriptions of assessment processes for demonstrations and oral examinations Performance criteria so applicants know on what basis they will be evaluated

25 Preparation for assessment Workshops or courses on how to take an exam, prepare for an oral assessment, or a demonstration Practice sessions Workshops or courses on how to build a portfolio of evidence linked to competencies and/or standards KEEP IT SIMPLE! KEEP IT CONCISE! Establish clear criteria for the assessment of the portfolio

26 Technical requirements of competence-based assessment Assessments must be Valid Reliable Sufficient Authentic Assessment processes must be Fair Transparent Feasible Defensible Affordable