[Year] Year] How to Develop Evaluators for the Sport of Curling. Canadian Curling Association and Coaching Association of Canada. Revised:

Transcription

1 [Year] How to Develop Evaluators for the Sport of Curling Year] Canadian Curling Association and Coaching Association of Canada. Revised:

2 Table of Contents How to Develop Evaluators for the Sport of Curling Introduction 3 Assessment vs. Evaluation 4 Principles of Evaluation in the NCCP 5 Structure of Evaluation in the NCCP 6 Competition Coach Evaluators 7 Competition Development Evaluators 8 Path to Evaluator Training 9 Process to Become an Evaluator 9 Evaluator Currency 9 Sample Evaluator Training for Competition Introduction Agenda 10 Evaluator Application Form 11 Evaluation of Competition-Introduction Coach 12 Coach Pathway for Competition-Introduction Context for the Sport of Curling - 13 Competition Coach Training and Evaluation Competencies for Competition Introduction Coach 14 Evaluation Methodology for Competition Coach 15 Challenge Process 16 Unsuccessful Evaluation Process 16 Evaluation of Competition-Development Coach 17 Training and Evaluation Competencies for Competition Development Coach 18 Coach Pathway for Competition-Development Context for the Sport of Curling 19 Proposed Timeline Two Year Training/Evaluation Process 20 Competition Development Evaluation Methodology 21 Quality Assurance 22 2

3 Introduction The job of coaching is multi-faceted and can vary depending on the sport, the type of athlete coached, and the environment in which the coaching takes place. A competent coach must acquire and demonstrate a variety of knowledge, skills, and attitudes that contribute to his or her effectiveness in a particular coaching context. Given the above, a competency-based National Coaching Certification Program (NCCP), requires that: (1) the degree to which a coach should be competent in a given context be determined as clearly as possible and (2) appropriate procedures and methods be designed and implemented to validate that a coach has achieved the standards set for certification. In a competency-based approach, it is important to identify the usefulness of both assessment vs. evaluation. The difference between the two provides a philosophical underpinning for the certification of coaches (see Page 2 - Coach Assessment and Evaluation: What s the difference?). Although the merits of assessment in promoting learning are well established, the basis for certification requires that coaches be evaluated in the specific areas that are deemed important to the NCCP and to the sport. It is also important to frame the concept of evaluation within the structure of the NCCP and the achievement of certification. The NCCP Coach Pathway proposes that coaches could move through three phases: (1) coach training, (2) coach certification, and (3) coach development. Since one of the philosophical foundations of a competency-based NCCP is that a coach, given prior learning and experience, may move directly to the certification phase in the pathway, it is critical to clearly articulate the requirements, standards, methods, and procedures that apply to the evaluation. 3

4 Assessment vs. Evaluation What s the difference? What is it? Assessment Assessment is a step in the learning process whereby the learner is informed of his or her performance or progress towards the achievement of a given outcome. Assessment is a formative process. Assessment provides information and feedback on coaching performance at a given time. It is cross-sectional and context-specific. Assessment may be objective or subjective. An Assessor: Provides feedback and information to coaches based on outcomes and criteria. Uses valid and objective assessment tools to gather information on coaching performance for a given outcome. Makes recommendations to improve coach performance. Qualities of an Assessor: May have context-specific knowledge and experience. Has some knowledge of the outcome being assessed. May be an athlete, parent, administrator, or another coach. May also function as a mentor coach. Is identified by the P/TSO, according to NCCP and NSF standards. Evaluation What is it? Evaluation is the process whereby a judgment is made on the ability of the coach to demonstrate one or more outcomes to an established standard. Evaluation is a summary process. Evaluation is cumulative and may require several assessment methods. Evaluation is objective and provides a benchmark. An Evaluator: Passes a judgment on coaching competency in a specific context based on outcomes and criteria that have a defined standard. Uses valid and objective evaluation tools that define performance for a given outcome and criterion. If external, has been identified by the NSF to conduct an evaluation of a coach and has not been involved in the training of the coach. Grants certification. Qualities of an Evaluator: Is considered an expert in the context and in the sport. Is well versed in the NCCP evaluation standards and process. Is a leader in the sport s coaching community. Is free of bias and subjectivity. Is selected and trained by the NSF, according to NCCP and NSF standards. for Certification and Evaluation in the NCCP For the purposes of certification within the NCCP, coach evaluation is based on the following assumptions: Certification should identify and confirm that the coach is capable of doing certain things deemed important in his or her context. Certification should promote favourable coaching behaviours that have a positive impact on athlete development. Certification is not about recognizing the perfect coach, but rather about acknowledging that a coach has demonstrated evidence of competence in the minimum standards identified within the NCCP. A common evaluation framework is desirable across sports for a given coaching context, but, at the same time, flexibility must exist to reflect diversity among sports. 4

5 It is possible to recognize and respect individual coaching styles while validating the presence of critical evidence. Adequate training or relevant experiences should occur before evaluation. NCCP training activities should prepare coaches adequately to meet the standards in place for certification. Evidence demonstrated during the evaluation may not reflect all of the elements or objectives identified during training activities. Principles of Evaluation in the NCCP Coach evaluation within the NCCP is based on the following principles: A competency-based NCCP is a system for evaluating the knowledge, skills, and attitudes deemed important for a given coaching context. The evaluation must reflect the framework of ethical coaching practice promoted by the Canadian sport system. NCCP certification is outcomes-based and requires evidenced-based evaluation that assumes that (1) the coach can demonstrate certain criteria and that (2) sufficient evidence has been gathered to pass judgment on the attainment of a given criterion. The evaluation process must assist the coach in achieving certification by identifying the specific evidence and descriptors that are used to judge the successful attainment of a given criterion. For certification purposes, only NCCP trained and accredited Evaluators can determine a coach s attainment of a given criterion. Competition Introduction coaches must complete a formal observation that (1) is done by an external Evaluator and (2) features coaching of athletes or participants in that specific coaching context. Evaluation requirements, procedures, and methods must be (1) administratively feasible, (2) professionally acceptable, (3) publicly credible, (4) legally defensible, (5) economically affordable, and (6) reasonably accessible. 5

6 Structure of Evaluation in the NCCP The structure for evaluation is based on a systematic approach for gathering evidence on coaching abilities. These abilities are tracked by the NCCP Database and substantiate coaching competency. The following structure based on outcomes, criteria, and evidence is proposed to establish a clear link between the key components of the evaluation system. OUTCOMES: What a coach should be able to do in specific areas deemed important for the NCCP. The outcomes provide the framework for the NCCP and identify a generic foundation that presents a consistent structure across all sports. The outcomes provide a link between evaluation and training. CRIRIA: What will be evaluated within a given outcome. Criteria are attached to each outcome. Criteria define critical coaching abilities that Curling wants to emphasize for the Competition Introduction context. In the NCCP Database, these criteria are used to track the progress of a coach toward certification and are recorded as either trained (T) or evaluated (E). EVIDENCE: What the Evaluator must see to confirm the attainment of a given criterion. Evidence can be (1) discrete, observable coaching behaviours or (2) specific elements that are involved in determining the achievement for particular criteria. 6

7 Competition Coach Evaluators Job Description for the Competition Coach Evaluator Complete an Initial Evaluator Training Session Audit/Attend a Competition Coach Training Workshop Attend any future Evaluator Update Sessions Evaluate Practice Portfolio prior to in-person or video evaluation Establish a mutually agreeable evaluation time with coach for evaluation Evaluate coaches (in-person or video) in the Competition Coach Context, provide debrief/action plan as required Follow the administrative procedures established for evaluators in their respective province/territory Potentially become a Training Facilitator (**Provincial/Territorial Rules may supersede this) Act as a resource person for coaches in their region Forward suggested amendments and modifications to the Provincial/Territorial Curling Coordinators Identify potential facilitators and evaluators Minimum Standards Level 3 Certified in the old NCCP Certified Competition Coach Minimum 5 years recent Coaching Experience 90+% score on the Make Ethical Decisions in Competition Introduction Evaluation Skill Set for the Evaluator Credibility with peers in curling High Ethical standards and Leadership skills Recognized technical and coaching expertise Good Communication Skills (Listening and Speaking) Ability to critically reflect and ask questions Ability to provide positive constructive feedback Current technical and coaching skills Thorough understanding of the NCCP Demonstrates: integrity, fairness and objectiveness Potential training in adult education 7

8 Competition-Development Evaluators Job Description for the Curling: Competition-Development Evaluator Complete an Initial Evaluator Training Session Audit/Attend a Curling: Competition-Development Coach Training Workshop Attend any future Evaluator Update Sessions Evaluate Practice Portfolio prior to in-person or video evaluation Establish a mutually agreeable evaluation time with coach for evaluation (practice and competition) Evaluate coaches in the Curling: Competition-Development Coach Context, provide debrief/action plan as required Follow the administrative procedures established for evaluators by the CCA Potentially become a Training Facilitator Forward suggested amendments and modifications to the NCCP Transition Committee Identify potential facilitators and evaluators Minimum Standards Minimum Level 3 or Competition Development Certified with additional experience* (ex. National/International Coaching experience), Level 4/5 (full or in part) preferred and/or Member of National Coaching Certification Program Transition Committee and/or Member of National Team Coaching Program. 90+% score on the Make Ethical Decisions in Competition Development Evaluation Skill Set for the Evaluator Credibility with peers in curling High Ethical standards and Leadership skills Recognized technical and coaching expertise Good Communication Skills (Listening and Speaking) Ability to critically reflect and ask questions Ability to provide positive constructive feedback Current technical and coaching skills Thorough understanding of the NCCP Demonstrates: integrity, fairness and objectiveness Potential training in adult education 8

9 Path to Evaluator Must attend an Evaluator s Training Session - Session may be conducted in person or by webinar/video training where applicable Meets the minimum requirements for training Is available for the entire training period Minimum 90% on Make Ethical Decisions Competition Development Evaluation Evaluator Certification is valid for 5 years, during which time the Evaluator must demonstrate currency. Process to Become an Evaluator 1) Attend an Evaluator Training Session a) To Gain consistency in the interpretation and application of the criteria and evidence expected in a Competition Coach context b) To demonstrate skills that are involved in the coach debrief 2) Evaluate a Practice Portfolio (pre evaluation submission)/annual Plan portfolio (Competition Development) 3) Conduct an evaluation, either in-person or video, under the supervision of a Master Evaluator Evaluator Currency To maintain currency and evaluator must do the following: Conduct a minimum of three in-person and/or video evaluations every two years, and; Attend an evaluator updating session when necessary and; Conduct an evaluation, either in-person or video, under the supervision of a Master Evaluator every 5 years. 9

10 Sample Evaluator Training for Competition Introduction Agenda Pre training activity: Review Competition Coach Evaluation Tool and Standards of Evidence Matrix Evaluators are to be provided with a current copy of all training material prior to evaluation training Time Topic 8:30am House keeping & overview of the day 8:45am Evaluation overview 9:00am Training and certification pathways 9:15 am Review of evaluation methods & tools 10:15am Break 10:30 am Portfolio evaluation 11:30 am Portfolio evaluation debrief 12:00 pm Lunch 1:00 pm In-the-field evaluation 2:00 pm In-the-field evaluation debrief 2:45 pm Break 3:00 pm Admin & logistics 4:00 pm Wrap-up 10

11 Evaluator Application Form Name Date of Birth Nom Date de naissance D/J M Y/A Address Gender Adresse Sexe City Province Ville Postal Code Code Postal Telephone Téléphone Adresse électronique Languages spoken Langues parlées Citizenship Status Citoyenneté Sport NCCP CC# No de CC du PNCE Education/Études Post-Secondary Education/Études postsecondaires Period of study/période Institution/Établissement Credit Received/Crédit obtenu d études From/Du To/Au Name/Nom Task or Level/Tâche ou Niveau Year/Année Advanced Coach Education/Formation avancée en entraînement From/Du To/Au Name/Nom Task or Level/Tâche ou Niveau Year/Année Application Checklist : Cover letter Completed Application Form Completed Preliminary Worksheet Transcript of NCCP Certification Status Make Ethical Decisions Online Evaluation - 90% or more Résumé of your coaching experience and achievements (with names and phone numbers for 2 references) Declaration I hereby certify that the information I have provided with this application is true and complete. Signature of Applicant 11

12 Evaluation of Competition-Introduction Coach Each Context has a module dedicated to explaining how the coach will be evaluated and the standards which they will be measured on. All applicable forms and tools are contained within that Module. Competition Coach See Module 10 Evaluation Logistics 12

13 Coach Pathway for Competition-Introduction Context for the Sport of Curling - Competition Coach Enter Competition Stream Here* (Training Process) Multisport Module Make Ethical Decision (MED) (3 hours) Competition Coach Training Workshop (15 hours) Understanding the NCCP Teaching and Learning Plan a Practice + Technical Skills = Strategy & Tactics Analyze Performance Coaching in Curling Coaching Athletes with Disabilities In Training Competition Coach Status: Trained Competition Stream Certification Process Enter after Training Successful completion of MED online Evaluation Successful Completion of Competition - + = Introduction Curling Evaluation Competition Coach Status: Certified Competition Stream Challenge Process** Successful completion of MED online evaluation +/- MED Training Successful Completion of Competition - Introduction + = Curling Evaluation Competition Coach Status: Certified * No prerequisites required to enter Competition Stream. ** Coach Training highly recommended by not required. 12

14 The Canadian Curling Association has determined the following for the Competition Introduction Coach: NCCP Outcomes Criteria Make Ethical Decisions Apply an ethical decision making process. Provide Ensure that the practice environment is safe Support to Implement an appropriately structured and organized Athletes in practice Training Make interventions that promote learning Plan a Practice Identify appropriate logistics for practice Identify appropriate activities in each part of the practice Design an emergency action plan Analyze Detect performance Performance Correct performance Required to be part of training (T) or evaluation (E) program T E T E T E T E T E T E T E T E T E 14

15 Evaluation Methodology for Competition Coach Outcome When Evaluated Method Notes 1. Make ethical decisions 2. Plan a Practice 3. Provide Support to Athletes in Training* 4. Analyze performance* After completion of Ethical Training Module through Multi-Sport Module (MSM) or after the coach has completed (MSM) Intro to Competition A After completion of the Curling Intro- Competition Training. After completion of the Curling Intro-Competition Training and At a time mutually determined by coach and evaluator or At a time determined by PSO when evaluations take place After completion of the Curling Intro-Competition Training and At a time mutually determined by coach and evaluator or At a time determined by PSO when evaluations take place NCCP-developed online evaluation. (must the Coaching Association of Canada (coach@coach.ca) for login information Written practice plan submitted - ed or mailed at least two weeks prior to formal observation Formal observation of implemented practice session (onsite/camp) or Video submission of coach during a practice session Formal observation of implemented practice session (onsite/camp) or Video submission of coach during a practice session This evaluation is located at Evaluation is free x 2 There is a practice evaluation on line, this is not intended to be a training Coach is encouraged to use practice plan template provided by CCA - Practice session should be minimum 1 hour based mainly on athlete skill development for delivery AND brushing. - Practice must include on-ice time. - Video submissions must follow the Video Protocol provided by the CCA. - Practice session should be minimum 1 hour based mainly on athlete skill development for delivery AND brushing. - Practice must include on-ice time. - Video submissions must follow the Video Protocol provided by the CCA. * Coach will be observed one time during their practice session for both these outcomes. 15

16 Challenge Process While it is recommended that everyone take the training, it is recognized that some coaches may already possess the knowledge required to successfully complete the Evaluation without training. e.g. Competitive athletes who have worked closely with certified coaches in the past, or coaches who have been exposed to similar training in other sports, or physical educators. For those who would like to Challenge the evaluation, the Evaluation Module will provide the coach with the standard competencies in which they will be evaluated on. Unsuccessful Evaluation Should a coach not meet the minimum standards required for the evaluation, they will be provided with a debriefing session and action plan to assist them in meeting the standards. The coach can then apply to be re-evaluated. If the coach is still not successful in their re-evaluation they will be referred back to the context s training program. 16

17 Evaluation of Curling: Competition-Development Coach Each Context has a module dedicated to explaining how the coach will be evaluated and the standards which they will be measured on. All applicable forms and tools are contained within that Module. Curling: Competition Development See Module 10 Evaluation Toolkit and Matrix 17

18 The Canadian Curling Association has determined the following for the Competition Development Coach Curling Specific Training and Evaluation: NCCP Outcomes Design a Sport Program Manage a Program Provide Support to Athletes in Training Plan a Practice Support the Competitive Experience Analyze Performance Criteria Outline program structure based on training and competition opportunities. Identifies appropriate measure to promote athlete development within own program. Integrates yearly training priorities for Curling into own program. Organizes and sequences training priorities and objectives on weekly basis to optimize adaptations. Develops a tapering and peaking program in preparation for important competitions. Manage administrative aspects of program and oversee logistics. Report on Athlete progress throughout the program. Ensure that the Practice Environment is safe. Implement methods that develop athletic abilities relevant to the sport Make interventions that enhance learning and are aimed at improving the athletes performance Make adjustments to the practice based on the athletes' response to the training task Identify logistics and appropriate background information for practice Identify appropriate activities in each part of the practice Design and sequence activities appropriately within the practice to enhance performance or create optimal adaptations Design an emergency action plan Pre-Competition: Implement procedures that promote readiness for performance. During competition: Make decisions and interventions that promote sport-specific performance. After competition: Use the competitive experience in a meaningful manner to further athletes development. Detect technical elements that have to be improved or refined to enhance performance Correct technical elements that have to be improved or refined to enhance performance Correct tactical elements that have to be improved or refined to enhance performance Detect tactical elements that have to be improved or refined to enhance performance Correct tactical elements that have to be improved or refined to enhance performance Required to be part of training (T) or evaluation (E) program 18

19 Competition Development Coach Pathway: Curling It is recommended that coaches be Level 2 or Competition Coach Certified before beginning training. Recommended Pathway for all Coaches Prerequisites Make Ethical Decisions (MED) Training* Plan a Practice 1 Design a Basic Sport Program* Sport Specific Training Analyze Technical and Tactical Performance Advanced Practice Planning Managing a Program Performance Planning Developing Athletic Abilities Prevention and Recovery Multisport Module Training Coaching and Leading Effectively* Managing Conflict* Psychology of Performance* Leading Drug Free Sport* "In Training" "Trained" Evaluation MSM Online Evaluations: 1) Comp-Dev MED 2) Leading Drug Free Sports 3) Managing Conflict Formal Observation of Coach: 1) During Practice Session 2) During Competitive Session Portfolio Submission (Annual Plan, EAP, Practice Plans, Athlete Profiles) Curling Rules Evaluation Video Analysis Task Debrief "Certified" Maintenance Certification Expires after 5 years of inactivity. To maintain certification: Coach must provide evidence of active coaching and professional development (30 points) or repeat Evaluation process. Direct to Evaluation Pathway Notes: Design a Basic Sport Program - is offered as a MSM in Intro to Competition B, Plan a Practice 1 - is offered as a MSM in Intro to Competition A or Competition Coach (Curling). (*) Each of thes e modules are offered by P/TSO. 19

20 Proposed Timeline Two Year Training/Evaluation Process Take MSM sport modules as they become available in your area: Coaching and Leading Effectively, Managing Conflict, Psychology of Performance, Leading Drug Free Sport Enter Program with MED, Design a Basic Sport Program and Plan a Practice 1 Register for Program Program Pretasks: Understanding the NCCP LTAD Review Curling for Life Wheelchair Curling Pre- Task Curling Specific Training (4.5 days), Performance Planning, Technical Skills, Analyze Performance, Manage a Program, Prevention and Recovery, Developing Athletic Abilities Preparation of Annual Plan Submission of Annual Plan for review and feedback. Implementation of plan for June to May season. Bi-Weekly Webinars: Advanced Practice Planning (2), Strategy (2) Practice Plan Submission Practice Evaluations (Video or In Person) Prerequisites January February March April May June July August September October November December Year One Take MSM sport modules: Coaching and Leading Effectively, Managing Conflict, Psychology of Performance, Leading Drug Free Sport Continue work on Portfolio, Video Analysis Assignments, Annual Planning Check in Point with Mentor Webinar Presentations of Annual Plan/Analysis, Preparation of new Annual Plan Finish any MSM Check in Point with Mentor Competition Evaluation, Portfolio Submission Check in Point with Mentor Coach Debrief January February March April May June July August September October November December Year Two 20

21 Outcome Provide Support to Athletes in Training Make Ethical Decisions Plan a Practice Analyze Performance Support the Competitive Experience Design a Sport Program Manage a Program Competition-Development: Evaluation Methodology Method of Evaluation Submission of Portfolio Including: - Team Profile - Practice Plan with Emergency Action Plan - Specific phase of the annual plan - Practice Plan with Emergency Action Plan - Pre-competition phase of the annual plan - Practice Plan with Emergency Action Plan - Competitive phase of the annual plan In Person/Video Observation - Observation of one of the submitted Practice Plans included in the portfolio involving on-ice training during one of the following phases (Specific, Pre-competition, Competitive). Must include skill development/analysis of performance Successful Completion of Online Evaluation. See Provide Support to Athletes in Training Successful Completion of Testing Successful completion of web based training/testing with a bank of videos (delivery and brushing), submission of observation and proposed corrections In Person/Video Observation Observation of practice session In Person/Video Observation Observation of competition session Submission of Portfolio Including: - Written description of pre-, during and post competition routines and demonstrates their use during the on-site competition evaluation - A list of competition specific factors which may influence performance and is able to discuss the factors with an evaluator during on-site competition evaluation - Written rules evaluation Submission of Portfolio Including: - Annual plan from month of April or May of year preceding season Submission of Portfolio Including: - Annual plan from month of April or May of year preceding season Submission of a Season Evaluation Report - At the completion of the season, a written paper/report regarding the implementation of the annual plan, including an analysis of it's effectiveness. Report should include athlete feedback. Successful Completion of Managing Conflict Online Evaluation Successful Completion of Leading Drug Free Sport Online Evaluation 21

22 Quality Assurance The Canadian Curling Association has committed to reviewing and updating all programs and resource material every three years. Revisions will include feedback from participants, P/TSO s and the NCCP Transition Committee Members. 22