SPECIALIZED PROFESSIONAL ASSOCIATION (SPA) ELCC

Transcription

1 SPECIALIZED PROFESSIONAL ASSOCIATION (SPA) ELCC 2017 Fall CAEP Conference Presented by Joan Auchter, NPBEA SPA Program Administrator Washington Hilton Hotel September 24, 2017

2 Discussion Guidelines for Preparing Your Program Report Spring 2017 SPA Update Status of the National Educational Leadership Preparation (NELP) Standards

3 Relationships and Responsibilities Among the Organizations NPBEA CAEP ELCC/ NELP SPA Council for Accreditation of Educator Preparation (CAEP) Umbrella organization that sets the Specialty Professional Associations (SPA)s requirements. National Recognition Report is one piece of evidence in the CAEP review process. The National Policy Board for Educational Administration (NPBEA) The National Policy Board for Educational Administration is a national alliance of major membership organizations committed to the advancement of school and school-system leadership. Sets initiatives to guide preparation programs in educational leadership, creates educational leadership standards, and owns the ELCC/NELP SPA. NPBEA members include AACTE, AASA, CCSSO, ICPEL, NAESP NASSP, and UCEA. The National Educational Leadership Preparation (NELP) SPA Deals with all building and district level educational leadership program reviews which provides evidence that program candidates have a strong foundation of content and pedagogical knowledge in the educational leadership program area. Program review is part of the overall accreditation process and occurs prior to the selfstudy and on-site accreditation visit. Educator Preparation Provider (EPPs) then use the results of program review as evidence to meet applicable CAEP standards.

4

5 WHO BENEFITS FROM NPBEA STANDARDS AND SPA REVIEWS? Preparation Accreditation Institutions of Higher Education/ CAEP School District Level Internship Placement/ Mentors Principal Pipeline National Organizations State Level Great Leaders in Each School School Leader Licensure

6 GUIDELINES FOR PREPARING YOUR PROGRAM REPORT

7 CAEP S ANNUAL CALENDAR OF ACTIVITIES FALL PROGRAM REVIEW CYCLE SPA Program Reports Posted in AIMS September 15 SPA Program Team Review Period Oct 15 Nov 15 SPA Audit Committee Review Period Nov 15 Jan 1 CAEP Technical Edit Period Jan 1 Feb 1 National Recognition Reports Posted February 1 SPRING PROGRAM REVIEW CYCLE SPA Program Reports Posted in AIMS March 15 SPA Program Team Review Period April 15 May 15 SPA Audit Committee Review Period May 15 July 1 SPA Tech Edit Period July 1 August 1 National Recognition Reports Posted August 1

8 Who Should Submit Program Reports? All colleges and universities that offer programs for the preparation of superintendents, principals, curriculum directors or supervisors at the master s degree, post-master s, specialist, or doctorate levels should respond to these guidelines. Program s CAEP coordinator s responsibility to keep profile updated in AIMS.

9 7 STANDARDS & ELEMENTS ARE THE FOUNDATION

10 CRITICAL RECOGNITION ELEMENTS STANDARDS- AND EVIDENCE-BASED DESIGN

11 SIX ASSESSMENTS - DISTRICT ELCC CAEPCON 2017

12 SIX ASSESSMENTS - BUILDING ELCC CAEPCON 2017

13 INFORMATION REQUIRED BY ELCC ONLY Program must include a one-page description to inform reviewers how the internship/clinical experience(s) have been designed to meet ELCC Standards 7.1 and 7.3 Assessment 4 evaluates candidate skills (ELCC 7.2)

14 Option A: Report Sections Section I. Context Specifies general program information Enter Candidate Information table online Enter information for all faculty online in the AIMS Manage Faculty Information view Pertinent faculty information imports into each program report Specifies character limits for responses to narrative questions Requires one attachment

15 Section II. List of Assessments Section III. Relationship of Assessments to Standards Section IV. Evidence for Meeting Standards For each assessment, attach one document that includes the assessment, scoring guide/criteria, data tables and a 2-page maximum narrative Section V. Use of Assessment Results to Improve the Program Describe how faculty are using the data from assessments to improve candidate performance and the program, as it relates to content knowledge, pedagogical and professional knowledge, skills and dispositions, and student learning.

16 CRITICAL RECOGNITION ELEMENTS SECTION IV: EVIDENCE FOR MEETING STANDARDS

17 Section IV: Assessment Narrative A brief description of the assessment and its use in the program (one sentence may be sufficient); A description of how this assessment specifically aligns with the standards it is cited for in Section III. Cite SPA standards by number, title, and/or standard wording. A brief analysis of the data findings; An interpretation of how that data provides evidence for meeting standards, indicating the specific SPA standards by number, title, and/or standard wording;

18 Assessment Narrative ELCC CAEPCON 2017

19 PATH TO NATIONAL RECOGNITION Section IV. Evidence for Meeting Standards For each assessment, attach one document that includes the assessment, scoring guide/criteria, data tables and a 2-page maximum narrative

20 Assessment Directions Data Charts Assessments Scoring Rubric Evaluative Criteria

21 EXPLICIT AND DIRECT ALIGNMENT ALIGNMENT ALIGNMENT

22 ALIGNMENT TO ELCC STANDARDS Definition of Alignment The concepts addressed in the ELCC standard elements are visible in the assessment and scoring guide to the same degree of depth, breadth, and specificity. Does not require exact wording of the standard element in the description of the assessment or the scoring guide. However, same concepts must be there. STANDARDS ASSESSMENT SCORING GUIDE/CRITERIA DATA TABLES

23 ASSESSMENTS WHAT develop and deliver assessments explicitly aligned to the standards that ask learners (directions) to demonstrate their knowledge and skills If you don t ask for it, you can t measure it.

24 SECTION IV: ASSESSMENT DESCRIPTION Can be the assessment tool itself or a rich description of the assessment (often the directions given to candidates for completing the assignment) Where possible, indicate standard alignment to assessment tasks (e.g. ELCC 3.1 or ELCC 3.2) so reviewers can easily find your evidence

25 2011 ELCC DISTRICT STANDARDS ELCC CAEPCON 2017

26 PART 2E: ALIGN ASSESSMENT DIRECTIONS TO STANDARDS

27 TEAM ASSIGNMENT Review your program report. Select one assessment to review. How clearly does your assessment align with the standard elements you identified? Read the assessment directions and then read the identified standard element. Are the assessment directions clearly aligned with the identified standard element? If not, how can you better align the standard element with the assessment?

28 RUBRIC OR SCORING GUIDE HOW develop scoring guides and score scales explicitly aligned to the standards and assessment to reliably measure the learner s evidence You can only measure WHAT you asked for in the assessment directions

29 SECTION IV: ALIGNING THE SCORING GUIDE/CRITERIA Design a scoring guide or a likert scale instrument that explicitly defines the criteria you will use to evaluate the degree of candidate mastery of the essential ELCC Standards concepts required in the assessment. The scoring guide instrument must evaluate a preponderance of the standard elements align to the assessment description and directions

30 2011 ELCC SCORING GUIDE (RUBRIC) STARTER CHARTS 2011 ELCC DISTRICT STANDARDS, PAGE 15

31 PART 2F- ALIGNED SCORING GUIDE ELCC CAEPCON 2017

32 TEAM ASSIGNMENT Select the same assessment you reviewed during your first assignment. Review the assessment scoring rubric. How clearly does your rubric align with the standard elements you identified in the assessment? Does the rubric clearly describe difference among levels of performance using descriptions of what a reviewer would expect to see at each level? (does not rely on subjective use of qualifiers)? Does the rubric incorporate the preponderance of standard element indicators to describe what the candidate must demonstrate for each standard element?

33 SECTION IV: DATA TABLES Data tables should relate back to what is measured in the scoring guide instrument. Report data at the standard level, not element level. Use elements level data to make case for standard quality as a whole. Initial reports must include TWO applications of data on assessments 1, 2, 3, 4, 5 and 6.

34 PART 2g DATA TABLE ELCC CAEPCON 2017

35 TEAM ASSIGNMENT Review the data table you provided for your selected assessment. Does you data table relate back to what is measured in the scoring guide instrument? Does you data table report data at the standard level, not element level? Did you separate the data tables by year and include the N?

36 Use of Assessment Results to Improve Program (Section V) In this section of the program report, the program is asked to describe how it is using the data from assessments to improve candidate performance and the program as it relates to content knowledge, pedagogical and professional knowledge, skills and dispositions and student learning. This description should not link improvements to individual assessments but, rather, it should summarize principal findings from the evidence, the faculty's interpretation of those findings, and changes made in (or planned for) the program as a result. Developing an Improvement Process It is important to include your process for evaluating data to inform program improvement, including those involved in the evaluation and the frequency and types of meetings in the process flow. Additionally, the report should indicate how specific assessment data is used as evidence for change/modifications and validation of quality. Faculty should develop a timeline for periodic review and provide findings for change/modifications. Identify and include faculty and practitioner experts as reviewers. The following template can be used. Below are some reflection questions concerning the information you are providing in this section of the report: How did you describe the steps program faculty has taken to use information from assessments for improvement of both candidate performance and the program? Is it clear that assessment evidence is used by the institution in evaluating the program, counseling candidates, and revising courses or other elements of the program? Has the institution made program changes based on assessment evidence? Educational Leadership Preparation Program Assessment Results Self-Analysis Semester Reviewers Names/Titles Data Reviewed Findings Suggested Modifications to Strengthen Program Fall Spring Summer

37 Submitting Your Program Report It s time to submit your Program Report. Below are guidelines to follow to ensure you meet CAEP requirements and facilitate a smooth reading for your reviewer. General Program Report is limited to a total of 20 attachments. Each attachment should be no longer than the equivalent of 17 text pages. Section IV: Evidence for Meeting Standards Create ONE file that includes a two-page maximum narrative, assessment, rubric, and data charts for each assessment Ensure that all sections of each assessment file are clearly labeled. Include the assessment number and name and district or building level designation. Submission Checklist Only 20 attachments Each attachment is 17 pages or fewer Only ONE file for each assessment that includes Two-page maximum narrative Assessment Rubric Data charts Label for each assessment section that includes Assessment name Assessment number District or Building designation Degree level designation

38 ELCC STANDARDS Assessment #1 (State Licensure Exam or another Measure of Leadership Knowledge) Assessment #2 (Measure of Leadership Knowledge) Assessment #3 (Leadership that coaches teachers towards better instruction, better curriculum Assessment #4 (Leadership that measures professional skills during Internship/Clin ical Practicum Experience) Assessment #6 (Leadership in Organizational Management/ Community Relations) Assessment #5 (Leadership that Creates a Supportive Learning Environment) CONTENT ASSESSMENTS PROFESSIONAL SKILLS ASSESSMENTS SKILL EFFECTS ELCC ELCC ELCC ELCC ELCC ELCC ELCC

39 NATIONAL RECOGNITION REPORTS ELCC CAEPCON 2017

40 PROGRAM REPORT DECISIONS FURTHER DEVELOPMENT REQUIRED OR NATIONALLY RECOGNIZED WITH PROBATION OR NOT NATIONALLY RECOGNIZED NATIONAL RECOGNITION WITH CONDITIONS NATIONAL RECOGNITION

41 PROGRAM REVIEWER S JOB Judge alignment of assessment description, scoring guide, and candidate data with ELCC standard elements (e.g., 2.1, 3.2, etc.). Clearly communicate strengths and weaknesses of program in relation to each ELCC standard ( ). Make a judgment with a clear and open mind. Make a judgment based on accepted criteria rather than a personal bias. Apply ethical obligations to be objective, reflective, conscientious and discrete. To write an objective evaluation about the degree to which a program aligns to the ELCC standards/elements. Submit team program report recommendations to Audit Committee.

42 REVIEWER EVALUATION RUBRIC FOR ELCC STANDARDS

43 PROGRAM DECISION IS NATIONALLY RECOGNIZED The program substantially meets (preponderance of evidence) all ELCC standards 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, and 7.0; No further submission required; program will receive full National Recognition when the unit receives accreditation. Program will be listed on the NCATE website as Nationally Recognized if the unit is already accredited Nationally Recognized pending unit accreditation if the unit is not accredited

44 PROGRAM REPORT DECISION IS NATIONAL RECOGNITION WITH CONDITIONS Substantially meets some but not all ELCC standards Response to Conditions Report must be submitted within 18 months Has two opportunities within 18 months after the decision to remove conditions. If unsuccessful after two attempts, program status changes to Not Recognized and the program will be removed from the list on the CAEP website.

45 PROGRAM REPORT DECISION IS NATIONAL RECOGNITION WITH CONDITIONS Conditions could include one or more of the following Insufficient amount of data to determine if ILCC standards are met Insufficient alignment among ILCC standards or assessments or scoring guides or data (see ELCC Standard Evaluation Rubric) Lack of quality in some assessments or scoring guides Does not meet the NCATE requirement for an 80% pass rate on state licensure tests Lacks two applications of data in initial submission

46 PROGRAM REPORT DECISION IS NATIONAL RECOGNITION WITH CONDITIONS What are your next steps? Pay special attention to the conditions to remediate listed in Part G of your report. No other conditions can be added at a later date Evaluate items listed with your faculty and determine how best to address them If unclear on how to address items, contact SPA coordinator Joan Auchter auchterj@nassp.org If questions about format or timelines, contact CAEP program review staff Banhi Bhattacharya Banhi.Bhattacharya@caepnet.org

47 PROGRAM REPORT DECISION IS NATIONAL RECOGNITION WITH CONDITIONS (continued) What are your next steps? List of dates to resubmit a Response to Conditions report is at the endo of the report Submit ONLY after your faculty has had time to fully remediate the conditions and you believe the program is ready to be recognized To submit the Response to Conditions report Read How to Submit a Response to Conditions Report on the CAEP website

48 PROGRAM DECISION IS FURTHER DEVELOPMENT REQUIRED The program does not substantially meet all ELCC standards and the ELCC standards that are not met are critical to a high-quality program and more than a few in number, or are few in number but so fundamentally important that recognition is not appropriate; The program will have two opportunities within 12 to 14 months after the first decision to attain National Recognition or National Recognition with Conditions. If the program is unsuccessful after two attempts, program status will be changed to Not Recognized and the program will be removed from the list on the CAEP website.

49 PROGRAM DECISION IS FURTHER DEVELOPMENT REQUIRED What are our next steps? Read the report carefully and pay special attention to the comments Address the items listed with your faculty and determine how best to address them If unclear on how to address comments, contact SPA coordinator Joan Auchter If questions about format or timelines, contact CAEP program review staff Banhi Bhattacharya

50 PROGRAM REPORT DECISION IS FURTHER DEVELOPMENT REQUIRED (continued) What are your next steps? List of dates to resubmit a Revised Report is at the endo of the report Submit ONLY after your faculty has had time to fully remediate the conditions and you believe the program is ready to be recognized

51 PROGRAM DECISION IS NOT NATIONALLY RECOGNIZED What does this mean? The program has exhausted opportunities to resubmit and did not reach minimal SPA expectations for Recognition with Conditions. What are our next steps? Read the report carefully and pay special attention to the comments Address the items listed with your faculty and determine how best to address them If unclear on how to address comments, contact SPA coordinator Joan Auchter If questions about format or timelines, contact CAEP program review staff Banhi Bhattacharya The program can continue to submit in subsequent semesters, but the next submission will be a completely new report.

52 NELP SPA SPRING 2017 PROGRAM CYCLE SPA REVIEW RESULTS

53 NUMBER ELCC CAEPCON 2017 TREND DATA NUMBER OF SCHOOLS AND REVIEWS PER CYCLE/YEAR, S2014 -S # Reviews # Schools S F S F S F S CYCLE/YEAR

54 2017 PROGRAM REVIEW CYCLES S17 Numbers 38 Schools 64 Reports Initial Review 37 Second Review 23 Third Review 4 Preliminary F17 Numbers 40 Schools 70 Reports Initial Review 35 Second Review 27 Third Review 10

55 Number ELCC CAEPCON 2017 TREND DATA Number of First, Second and Third Submissions by Cycle/Year, S2014- S S2014 F2014 S2015 F2015 S2016 F2016 S2017 Cycle/Year 1st Time to submit 2nd Time to submit 3rd Time to submit

56 NUMBER ELCC CAEPCON 2017 TREND DATA S F2014 S F2015 S F2016 S # Recognized # Recognized with Conditions # Recognized with Probation #Further Development Required # Not Recognized

57 S17 Recognition Decisions by Category National Recognition Recognized with Conditions Further Development Required Not Recognized 0

58 S2017 TOP 7 REASONS FOR CONDITIONS AND FURTHER DEVELOPMENT

59 PROGRAM REPORT TROUBLESHOOTING Incorrect standards are used. (Standards are out of date or intended for a different program type.) Alignment is unclear. (Report does not show clear alignment of evidence to the standard components in the assessments assessment description/directions, rubric, and data charts. Data charts are incomplete or not aggregated per NELP guidelines. Two applications of data were not provided with an initial report. Scoring rubrics are vague or do not clearly differentiate evaluative categories. Use of assessment evidence to inform program improvement is unclear. Description of the field and clinical experience does not provide sufficient information, e.g., the number of hours on-site, qualifications and training of mentors.

60 Launching and Implementing the National Educational Leadership Preparation (NELP) Standards

61 Council for the Accreditation of Educator Preparation requirements

62 NELP CAEPCON 2016 CAEP Requirements for SPA Standards (2015) 1. Must align to CAEP Principles 2. May have only 7 standards and 28 elements 3. Must be written so that each concept in an element appears in the language of the standard 4. Must include supporting explanations, research support, and describe appropriate performance assessments 5. Must include rubrics or criteria to guide reviews (must be measurable) 6. An 8th standard on the clinical experience can be requested

63 PURPOSE & GOALS FOR THE NELP STANDARDS 1. Provide leadership programs with a set of voluntary accreditation standards that guide the preparation of building and district leaders 2. Provide leadership programs with guidance about the assessment of candidate learning and program quality 3. Provide reviewers of leadership programs with criteria and guidance for assessing the effectiveness of leadership programs 4. Connect the research base on school leadership to preparation standards

64 ISLLC 2008 & PSEL 2015 CROSSWALK ISLLC Standards 2008 (ELCC 2011) PSEL 2015 (NELP 2018) S1 Vision S2 Culture of Support and Instructional Program S3 Operations, Management, and Resources S4 Collaboration with Faculty and Community S5 Ethics S6 Political, Social, Legal, Cultural Context S1 Mission, Vision, Core Values S10 School Improvement S4 Curriculum, Instruction, and Assessment S5 Community of Care and Support for Students S6 Professional Capacity of School Personnel S7 Professional Community for Teachers and Staff S9 Operations and Management S8 Meaningful Engagement of Families and Communities S2 Ethics and Professional Norms S3 Equity and Cultural Responsiveness S3 Equity and Cultural Responsiveness S8 Meaningful Engagement of Families and Communities

65 NELP FOCUS: BEGINNING LEVEL Professional Standards for Educational Leaders (PSEL) Standards Approved and Owned by NPBEA For use by states, districts and practitioners in the field 7 states adopted, others adapted One set of standards for emerging, developing and distinguished educational leaders 10 standard and 83 indicators National Educational Leader Preparation (NELP) Standards Approved and Owned by NPBEA For use by Institutions of Higher Education for preparation, CAEP accreditation and state licensure Two sets of standards designed to prepare two specific types of leaders building level and district level 7 standards and 25 components CAEP requirement

66 PSEL The Process: Identifying Beginner Level Gaining Input from the Field & CAEP Refinement, Research Mapping & Comparisons NELP

67 NELP Standards Committee Joan Auchter, NASSP, ELCC Rosemarie Young, NAESP Tom Bellamy, UW- Bothell, Principal Supervisors Monica Byrne-Jimenez, Hofstra (ELL) David Chard, SMU Dean, Deans for Impact David DeMathews, (SPED), UT-ElPaso Paul Katnik, State Education Agency, Missouri Susan Korach, University of Denver, Wallace Pipeline Glenn Pethel, AsstSup, Gwinnet County, Wallace Pipeline L. Oliver Robinson, AASA (superintendent) Pam Tucker, UVA (Evaluation Expertise) Michelle Young, Chair, UCEA, ELCC

68 NELP Assessment Committee Joan Auchter, NASSP, ELCC Don Peurach, University of Michigan (Evaluation and Improvement Science) Ellen Goldring, Vanderbilt University (Evaluation Expertise Val Ed) Gina Ikemoto, AIR (Evaluation Expertise) Paul Katnik, State Education Agency, Missouri Susan Korach, University of Denver, Wallace Pipeline Pam Tucker, University of Virginia (Evaluation Expertise) Michelle Young, Chair, UCEA, ELCC

69 Draft NELP Standards Building Leader Standards 1. Mission, Vision, and Core Values 2. Ethics and Professional Norms 3. Equity, Inclusiveness and Cultural Responsiveness 4. Learning and Instruction 5. Community and External Leadership 6. Operations and Management 7. Building Professional Capacity 8. (The Internship) District Leader Standards 1. Mission, Vision, and Core Values 2. Ethics and Professionalism 3. Equity and Cultural Leadership 4. Instructional Leadership 5. Community and External Leadership 6. Management of People, Data, and Processes 7. Policy, Advocacy and Governance 8. (The Internship)

70 What s New in the NELP Standards? Aligned to the new Professional Standards for Educational Leaders Number increased from 6 ELCC content to 7 NELP content standards New stem focuses on educational success and well-being Emphasis on supportive and inclusive school culture Identifies nine key leadership practices: developing, implementing, evaluating, collaborating, communicating, modeling, reflecting, advocating and cultivating

71 MOST FREQUENTLY ASKED QUESTION Can you please tell me whether it would be prudent to wait to align our assessments to the new NELP standards? Our SPA reports are due in University of Connecticut Our SPA report is due September 2019, which would make us responsible for the new standards. However, we re required to submit 2 administrations of data which requires us to have 24 months of data (each class is offered once a year at most). Having 18 months won t be enough time to collect data, to say nothing of the required modifications to the rubrics. What is your recommendation for how we handle this? Should we start developing rubrics/tools based on the draft standards? Kent State In the meantime, and in view of the replacement of the ELCC standards with the NELP standards for school leaders, we are convinced that the use of the NELP standards would be most beneficial to us and to our program both now and in the long run. Clearly, we are devoting a great deal of time and energy to our accreditation work. That said, it makes sense to follow standards that go into effect in January, 2018 which is only three months after our submission is due. Can you offer some guidance on this matter? We have seen a copy of the draft NELP standards thanks to a colleague who serves on the National Policy Board for Educational Administration, but would appreciate it if you could send a copy to us. Penn State We are working on syllabi for a newly approach EdS in Educational Leadership and want to ensure we are indeed using the correct standards. We should be using the District Level NELP Standards found here in draft? I did not see updated information on the CAEP SPA website, so we are trying to confirm their approval or endorsement before proceeding to far. Winthrop University At this point, we have designed new rubrics tentatively aligned with the new draft NELP standards. We are administering them beginning spring 2017 (now) so as long as the standards do not change too much, we would like to submit our reports as soon as you are ready to receive them. Even though we won't have two administrations, we would like to submit our beautiful rubrics and our new data and receive a decision of "Further Development Required" so that we do not appear in any way to be "Not Recognized" which can be problematic for us with New York State. Do you have any sense of when you would begin to welcome reports aligned with the new standards? Would it likely be fall 2018? Buffalo State College

72 CALL FOR REVIEWERS Complete application form Attach resume Letter of interest NPBEA.ORG

73 ELCC CAEPCON 2016 Resources Program Report Templates: (ELCC Standards & Report Templates): ms/tabid/676/default.aspx#elcc ELCC Standards Questions about ELCC Program Report Design Joan Auchter or Questions about AIMS systems and technical submission problems Banhi Bhattacharya or (202)

74 Questions ELCC CAEPCON 2017