Instructional Effectiveness: Proficiency Rating Scales

Transcription

1 Instructional Effectiveness: Proficiency Rating Scales Kerridan A. Kawecki, B.A. Assessment Technology, Incorporated 6700 E. Speedway Boulevard Tucson, AZ Assessment Technology, Incorporated 2016 Galileo and the Galileo logos are trademarks or registered trademarks of Assessment Technology Incorporated. All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission from the publisher. Printed in the U.S.A

2 Revised as of

3 Instructional Effectiveness: Proficiency Rating Scales Table of Contents About Proficiency Rating Scales... 1 Getting Started... 1 Number of rating scales... 1 Standards... 2 Tallied and Reporting... 2 Process... 4 Accessing Proficieny Rating Scales... 5 Class-level User... 5 School-level user... 6 District-level user... 8 Administer Proficiency Rating Scales Tips Steps Print Proficiency Rating Scale Steps Adding Comments Steps Import and View Staff Files Purpose Tips Steps Post Results Steps Monitor Progress Steps Aggregate Analysis Steps Individual Results Steps Proficiency Rating Results Scoring Details Proficiency Rating Details Staff Response Revised as of i

4 ii Revised as of

5 ABOUT PROFICIENCY RATING SCALES WHAT IS A PROFICIENCY RATING SCALE? A Proficiency Rating Scale is a Galileo tool that allows districts and charter schools to document staff members teaching performance, professional practice and leadership performance. Most states require that this component of a staff member s evaluation be based upon observations and scored based on a rubric. Galileo s Proficiency Rating Scales may be used to create rubric-based forms to document both teacher and principal observations. Additionally, Galileo s performance rating scales are easily aligned to standards such as the Interstate New Teachers Assessment and Support Consortium (InTASC) Professional Teaching Standards and the Professional Administrative Standards from the Interstate School Leaders Licensure Consortium (ISLLC). OVERVIEW OF THE PROFICIENCY RATING SCALE TOOLS School districts and charter schools have expressed the need to develop, customize, administer and manage the implementation of their own teacher and principal evaluation tools. Galileo addresses this need by giving school districts and charter schools online access to an Instructional Effectiveness Dashboard. Among other tools, the dashboard includes: A Proficiency Rating Scale Administration tool An Educator Proficiency Ratings Progress report A Proficiency Rating Results report and Staff File Import and View functionality. GETTING STARTED Assessment Technology, Incorporated (ATI) provides districts/charter schools with the opportunity to work with our Education Management Services (EMS) department in building the Proficiency Rating Scales that will be utilized by the district/charter school. Please contact ATI s Educational Management Services (EdMgtSvs@ati-online.com) or your ATI Field Services Coordinator ( ) to discuss the number of Proficiency Rating Scales you need, to which, (if any) standards your Proficiency Rating Scales will be linked, and how the results should be tallied and reported. NUMBER OF RATING SCALES It is possible to have more than one Proficiency Rating Scale created. For example, many districts/charter schools like to have their informal or spot observations placed in Galileo, as well as the summative. Typically it is the summative that is tied to the final evaluation. Revised as of

6 STANDARDS Each Proficiency Rating Scale in Galileo will be linked to standards. The standards may take many forms. They may be the InTASC, ISLLC, a propriety set of standards, or custom standards. In the event your tool is not linked to standards, ATI will place the items to be observed in as standards. TALLIED AND REPORTING The Galileo system will tally, present, and report out rating scale results through proficiency categories determined by the district/charter school. The naming and cut scores associated with each proficiency category are established when the Proficiency Rating Scales are entered into Galileo. For example they might be: Highly Effective, Effective, Partially Effective, and Ineffective Distinguished, Proficient, Basic, and Unsatisfactory The Proficiency Rating Results widget lists all of the domains and standards on a given rating scale and displays the proficiency category for each, based on how staff members were rated on all the items linked to the standards. Example 1: 2 Revised as of

7 Example 2: Example 3: Revised as of

8 PROCESS To administer a Proficiency Rating Scale the following steps must be taken: Step 1. Confirm with ATI s EMS department that the proficiency categories for each Proficiency Rating Scale have been set up. 2. Determine what sources of Evidence you would like to be available when scoring each item on the proficiency rating scale. For example, Classroom Environment, Informal Observation, Student Work, etc. 3. Determine if you will record the Proficiency Rating Scale online or offline. 4. If administering offline, print the Proficiency Rating Scale so the observation may be documented and later transcribed online. 5. Administer the Proficiency Rating Scale according to your district/charter school evaluation protocol. a. Manually score items. b. Select the appropriate Evidence for each item. c. Record anecdotal notes. Information found in Getting Started section, page 1 Getting Started section, page 1 Print Proficiency Rating Scale section, page 14 Administer Proficiency Rating Scales section, page 10 d. Add Evaluator comments. 6. Import files, as needed. Import and View Staff Files section, page Post Results to staff members. Post Results section, page Monitor the continued administration of Proficiency Rating Scales. Monitor Progress section, page Analyze Aggregate Results. Aggregate Analysis section, page 29 4 Revised as of

9 Step 10.Access Individual Results. Information found in Individual Results section, page 32 ACCESSING PROFICIENY RATING SCALES Which Proficiency Rating Scale tools a user may access will depend on what level user an individual is and what Instructional Effectiveness (IE) permissions the individual user has been given. Below is a summary of each level and the access to the various IE tools: CLASS-LEVEL USER User may see the Proficiency Rating Results widget once the results are posted. These are accessed from the Results tab on the Instructional Effectiveness Dashboard. and Revised as of

10 SCHOOL-LEVEL USER User may access the Instructional Effectiveness Dashboard s Results tab, to see their own evaluation results. If provided the User Has Access to Instructional Effectiveness Tools permission, the user can administer and import files for Class-level users at his or her school and can see the Class-level users Proficiency Rating Results. and 6 Revised as of

11 If provided the User Can Administer and View Instructional Effectiveness Results for Staff of the Same User Level permission, the user can administer and import files for Class-level users at his or her school and other School-level users at their school. This user can also see the Classand School-level users Proficiency Rating Scale Results. and Revised as of

12 DISTRICT-LEVEL USER User may access the Instructional Effectiveness Dashboard page, to see their own evaluation results. If provided the User Has Access to Instructional Effectiveness Tools permission, the user can administer and import files for Class-level and School-level users and can see the Class-level and School-level users Proficiency Results. If provided the User Can Administer and View Instructional Effectiveness Results for Staff of the Same User Level permission, the user can administer and import files for Class-level, School-level users, and other District-level users. This user can also see the Class-, School-, and District-level users Proficiency Results. (Image on next page.) 8 Revised as of

13 and Refer to A Guide for Galileo K-12: Configure Instructional Effectiveness Security Settings for permission options and how to establish such permission. Document is found in the Galileo Help Files. Revised as of

14 ADMINISTER PROFICIENCY RATING SCALES TIPS The link is available even if no ratings have been administered. The same rating scale cannot be administered to the same educator twice. STEPS School- and District-level users with the appropriate IE permissions may administer Proficiency Rating Scales by accessing two different links. 1. Click the Dashboard icon. 2. Hover over the Dashboard menu option and select Instructional Effectiveness. 3. Use the down arrow to select the desired School or to select All Schools option. 4. Click the Administration tab. 10 Revised as of

15 5. Select the Filter by User Level option School, Class, or All. Filtering by user-level will only display data for the selected user-levels in the widgets. 6. In the Administer Proficiency Ratings widget, click the Administer Proficiency Rating Scales link. 6. In the Administer Proficiency Ratings widget, click the View all link or click the bar column. 7. Proceed to the next step. 7. Click the scale icon to administer a Proficiency Rating Scale. 8. On the Educator Proficiency Ratings popup window page, select the School. 9. Use the down arrow on Staff drop-down menu to select the educator. 10.Use the down arrow on the Library drop-down menu to select rating scale s library. 11.Use the down arrow on Rating Scale drop-down menu to select the scale. Revised as of

16 12. Enter the Observation Date or use the calendar to select the date. 13. The Proficiency Rating Scale will appear. 14. Rate each item that appears. Each item will contain the following elements: a. The domain to which the evaluation item is aligned. This will show in burgundy and in all capital letters. b. The standard to which the evaluation item is aligned. c. The evaluation item that is being scored. d. The level of proficiency as determined by the evaluation rubric. The level of proficiency selected determines the number of points earned on this evaluation item. e. Link to the scoring rules. Typically, an explanation of the rubric is displayed. The Scoring Rules and Examples appear in a new window. f. Expands to open a Notes box, where the observer may document anecdotal information. g. Allows the observer to document the Evidence that supports the selected proficiency category. 12 Revised as of

17 15.For each item, click the appropriate proficiency radio button (e.g., Evident, Not Evident). 16.Use the down arrow on the Evidence field to select the evidence. 17.(optional) Click the Show notes link to open the notes box. Type in notes regarding that objective. Users typically note anecdotal information related to what was observed or document why a particular proficiency category was selected. Revised as of

18 18.Click the Hide notes link to close the notes box. 19.Repeat steps for each item on the scale. 20.Click the Save Changes button located at the top or bottom of the page to save the changes. 21. A pop-up will appear informing you of the items that have not been evaluated or confirming that all items have been scored. It is possible for an evaluator and the staff member to record comments on the Proficiency Rating Scale. To learn how to record and utilize this feature please see Adding Comments section, page 17. PRINT PROFICIENCY RATING SCALE It is possible to print the Proficiency Rating Scale for a staff member. A Proficiency Rating Scale may be printed as a worksheet or printed to serve as a record. WORKSHEET: If the Proficiency Rating Scale has not yet been completed, it can be printed as a worksheet to be completed by the evaluator offline. If used as a worksheet, the evaluator will need to enter this information into Galileo. RECORD: Once a Proficiency Rating Scale has been completed by an evaluator, it can printed, and signed by the staff member and evaluator and kept as a record. STEPS School- and District-level users with the appropriate IE permissions may print Proficiency Rating Scales by accessing two different links. 1. Click the Dashboard icon. 14 Revised as of

19 2. Hover over the Dashboard menu option and select Instructional Effectiveness. 3. Use the down arrow to select the desired School or to select All Schools option. 4. Click the Administration tab. 5. Select the Filter by User Level option School, Class, or All. Filtering by user-level will only display data for the selected user-levels in the widgets. Revised as of

20 6. In the Administer Proficiency Ratings widget, click the Administer Proficiency Rating Scales link. 6. In the Administer Proficiency Ratings widget, click the View all link or click the bar column. 7. Proceed to the next step. 7. Click the scale icon to administer a Proficiency Rating Scale. 8. On the Educator Proficiency Ratings popup window page, select the School. 9. Use the down arrow on the Staff drop-down menu to select the educator. 10.Use the down arrow on the Library drop-down menu to select the rating scale s library. 11.Use the down arrow on Rating Scale drop-down menu to select the scale. 12.To include notes when printing, place a checkmark in the Display note in report box. (Image on next page.) 16 Revised as of

21 13.Click the printer icon. 14.A printable version of the document is generated. Notice that signature and date lines are available on the printed copy enabling the copy to be signed and kept for record-keeping. ADDING COMMENTS Once an evaluation has been conducted, the evaluator may add general comments. These comments may be saved and made visible to the staff member when the Allow selected staff to view rating scale results permission is extended. There is also a place for the staff member to enter a response to evaluator comments. Once a staff member has viewed his or her results and saved his or her response, those comments will appear to the evaluator on the Proficiency Rating Scale. Both evaluator comments and staff responses will also show at the bottom of the page and will appear on any printed copies of the Proficiency Rating Scale. Revised as of

22 STEPS 1. Click the Dashboard icon. 2. Hover over the Dashboard menu option and select Instructional Effectiveness. 3. Use the down arrow to select the desired School or to select All Schools option. 4. Click the Administration tab. 5. Select the Filter by User Level option School, Class, or All. Filtering by user-level will only display data for the selected user-levels in the widgets. 18 Revised as of

23 6. In the Administer Proficiency Ratings widget, click the Administer Proficiency Rating Scales link. 6. In the Administer Proficiency Ratings widget, click the View all link or click the bar column. 7. Proceed to the next step. 7. Click the scale icon to administer a Proficiency Rating Scale. 8. On the Educator Proficiency Ratings page or popup window, select the School. 9. Use the down arrow on the Staff drop-down menu to select the educator. 10.Use the down arrow on the Library drop-down menu to select the rating scale s library. 11.Use the down arrow on Rating Scale drop-down menu to select the scale. 12.Scroll to the bottom of the page to see two comment boxes. Revised as of

24 13.The Evaluator Comments box allows the administrator to add general comments about the proficiency scale or other evaluation topics. 14.Once comments are added, click the Save Changes button. While you may see the Staff Response, only the selected staff member may add to this box. Once the selected staff member has added his or her response and saved it, it will appear here. Staff response instructions may be found on Staff Response section, page 38. IMPORT AND VIEW STAFF FILES PURPOSE This feature allows evaluators to link important documents, to each staff member s Proficiency Rating Scale. Many districts/charter schools, for example, like to build and maintain portfolios for staff members. The import and view staff file capability allows for these documents to be housed and managed online. TIPS All file types are permitted. File size is limited to 20 MB. STEPS 1. Click the Dashboard icon. 20 Revised as of

25 2. Hover over the Dashboard menu option and select Instructional Effectiveness. 3. Use the down arrow to select the desired School or to select All Schools option. 4. Click the Administration tab. 5. Select the Filter by User Level option School, Class, or All. Filtering by user-level will only display data for the selected user-levels in the widgets. Revised as of

26 6. In the Administer Proficiency Ratings widget, click the Import and View Staff Files link. 7. The Import and View Staff Files popup window opens. 8. Use the down arrow on the School drop-down menu to select the school. 9. Determine if you d like to filter using the Filter by user level (school, class, or all) before selecting a staff member. 10.Use the down arrow on the Staff drop-down menu to select the educator. ATTACH A FILE 11.Type a File description in the file description field. 12.Click the Browse button and locate the relevant staff file to upload. 13.Click the Add button. The file is now attached to this educator s file. 14.For each attached file determine if the file should be Read Only. Checking the Read Only box will allow a staff member to only view the file. Unchecking the file will allow a staff member to remove the attached file. (Image on next page.) 22 Revised as of

27 15.For each attached file determine if the file should be posted for staff to see. If the file should be posted, place a checkmark in the Post for Staff checkbox. VIEW A FILE 16.Click the binoculars icon to view the desired file. DELETE A FILE 17.Click the trash can icon for the desired file. POST RESULTS Once a Proficiency Rating Scale has been completed, and appropriate notes and comments recorded, evaluators then post the results for staff members. Posting the results for staff members will allow staff members to access and see the Proficiency Rating Scale results from the Galileo s Instructional Effectiveness Dashboard s Results tab page. and Revised as of

28 A detailed explanation of the Proficiency Rating Scale Results accessed from the Results tab may be found in the Proficiency Rating Results section, page 34. STEPS 1. Click the Dashboard icon. 2. Hover over the Dashboard menu option and select Instructional Effectiveness. 3. Use the down arrow to select the desired School or to select All Schools option. 24 Revised as of

29 4. Click the Administration tab. 5. Select the Filter by User Level option School, Class, or All. Filtering by user-level will only display data for the selected user-levels in the widgets. 6. Click the Administer Proficiency Rating Scales link Click the View all link or click the bar column. 7. Proceed to the next step. 7. Click the scale icon to administer a Proficiency Rating Scale. 8. On the Educator Proficiency Ratings page or popup window, select the School. Revised as of

30 9. Use the down arrow on the Staff drop-down menu to select the educator. 10.Use the down arrow on the Library drop-down menu to select the rating scale s library. 11. Place a checkmark in the Allow selected staff to view rating scale results field. 12. Click the Save Changes button. The staff s rating scale results will be displayed on the staff s Dashboard page. Refer to the Proficiency Rating Results section, page 34, for directions on how a staff member views their Proficiency Rating Scale results. MONITOR PROGRESS School- and District-level users with the appopriate IE permissions can monitor the progress of the administration of proficiency ratings scales. The Administer Proficiency Rating Scales widget tracks completed proficeny rating scales in the bar graph. Intially, all staff memebers list in the 0 column. As administraors complete proficiency ratings scales on staff members, the staff members will be move to the subsequent column(s). This allows administrators to know and easily access the proficiency rating scales of staff members who still need to be evulated. STEPS 1. Click the Dashboard icon. 26 Revised as of

31 2. Hover over the Dashboard menu option and select Instructional Effectiveness. 3. Use the down arrow to select the desired School or to select All Schools option. 4. Click the Administration tab. 5. Select the Filter by User Level option School, Class, or All. Filtering by user-level will only display data for the selected user-levels in the widgets. 6. An explanation of the monitoring tools found in the Administer Proficiency Ratings widget is described below: Revised as of

32 a. The total number of staff members represented in the graph. b. Graphic representation of completed Educator Proficiency Rating Scales. There will always be a 0 column representing those staff members who have no Proficiency Rating Scales completed. A column will be present in the bar graph for each Proficiency Rating Scale that needs to be completed on a staff member. As Proficiency Rating Scales are completed staff members are automatically moved into the completed columns. c. The column visually indicates the total number of staff members in a particular column. The number of staff members represented in that column appears at the top of the column. Each column provides a drill down to a list of staff members in the column and provides direct access to the Proficiency Rating Scale(s). d. This number indicates the number of Proficiency Rating Scales completed. e. Clicking the View All link displays a list off all staff members, alphabetically who are represented in the widget. 7. To complete or view a Proficiency Rating Scale for a staff member click the column, or the View All link. 28 Revised as of

33 8. Click the staff member s name to access the most recent Proficiency Rating Scale or click the desired Proficiency Rating Scale. 9. Proceed to the Administer Proficiency Rating Scales section, step #8, page 11, for further instructions. AGGREGATE ANALYSIS The Proficiency Rating Scale Results by School widget summarizes the overall Proficiency Rating Scale scores at the selected schools. District- and School-level users with the appropriate permission may drilldown within the selected school, to view a staff member s Proficiency Rating Scale Score. STEPS 1. Click the Dashboard icon. 2. Hover over the Dashboard menu option and select Instructional Effectiveness. 3. Use the down arrow to select the desired School or to select All Schools option. 4. Click the Administration tab. Revised as of

34 5. Select the Filter by User Level option School, Class, or All. Filtering by user-level will only display data for the selected user-levels in the widgets. 6. On the Proficiency Rating Scale Results by School widget, select the Rating Scale for which you wish to see results. 7. An explanation of the monitoring tools found in the Proficiency Rating Scale Results by School widget is described below: a. The Rating Scales drop-down menu. 30 Revised as of

35 b. Displays the name of the Proficiency Rating Scale, the number of staff represented, the total number of items scored on the selected Proficiency Rating Scale, and the total possible points for the rating scale. c. Each column on this table can be sorted using the green down/up arrows. d. Displays the average points earned out of total points possible. e. Displays the average percent. f. Displays the number of staff members included in the results. g. The school is a hyperlink to drill down to view staff rating scales and scores. 8. Click the School name to drill down to see individual staff scores. 9. To sort the list, click the headings. 10.To print the list click the printer icon. 11.Click the Staff Name to see individual Proficiency Rating Scale Results in the Rating Scale Results window. (Image on next page.) Revised as of

36 For information on how to interact with the individual Proficiency Rating Scale Results see the Individual Results section, page 32. INDIVIDUAL RESULTS Staff members may view their individual Proficiency Rating Scale Results from the Results tab on the Instructional Effectiveness Dashboard page. STEPS 1. Click the Dashboard icon. 2. Hover over the Dashboard menu option and select Instructional Effectiveness. 32 Revised as of

37 3. If a School- or District-level user, select the School. (The School defaults based on your Settings page.) 4. If a School- or District-level user with appropriate Instructional Effectiveness (IE) permissions, select the Results tab. IMPORT STAFF FILES 1. If an administrator has opted to Post file(s) for staff, a user may access these by clicking on the folder icon. 2. The Staff Portfolio: Import and View Staff Files window will appear. The user will see any files the administrator/evaluator has imported and opted to post. 3. By default each posted file provides Read Access only. Clicking on the binoculars icon will allow the user to view the file. 4. If a trash can icon appears by the file, then the Read Only permission has been removed from the file, allowing the user to click the trash can and remove the imported file. Revised as of

38 PROFICIENCY RATING RESULTS If the administrator/evaluator has posted the Proficiency Rating Scale Results, then the staff member may access the results by navigating to the Results tab and selecting the Rating Scale in the Proficiency Rating Scale widget. The results of the Proficiency Rating Scale will appear: a. Total points earned on the rating scale out of total points possible. b. Date that the Proficiency Rating Scale was administered. c. The proficiency category designated for the domain as set up by the district/charter school. d. The domain that is being tallied and reported on. e. The proficiency category of the individual standards associated with each item. f. The standards that make up the domain (d). The standards are directly aligned to the items scored on the Proficiency Rating Scale. The actual items on the scale may be viewed by clicking the Proficiency Rating Scale details. 34 Revised as of

39 SCORING DETAILS If the administrator/evaluator has posted the Proficiency Rating Scale Results, then the staff member may access the scoring details by navigating to the Results tab and selecting the Rating Scale in the Proficiency Rating Scale widget. Then they may click the Scoring Detail link. The scoring details will appear: a. The staff member s name. b. The Proficiency Rating Scale selected. c. The total points earned out of the total points possible. d. Domains and the total points earned out of the total points possible for a domain. Please note, it is possible to have more than one goal make up a domain. (Image on next page.) Revised as of

40 e. The standard and the total points earned out of the total points possible for a particular standard. f. The item scored and the total points earned out of the total points possible for an item. g. Printer icon that allows the scoring details to be printed. PROFICIENCY RATING DETAILS If the administrator/evaluator has posted the Proficiency Rating Scale Results, then the staff member may access the Proficiency Rating Scale Results Details by navigating to the Results tab and selecting the Rating Scale in the Proficiency Rating Scale widget. Then they may click the Proficiency Rating Details link. This interface allows the user to see the interface as evaluators see it, including how each item was scored and which proficiency category was marked for each item. 36 Revised as of

41 The proficiency rating details will appear: a. The date the Proficiency Rating Scale was administered. b. Provides the ability to print the Proficiency Rating Scale. The printed version of the scale has signature and date lines for record keeping. c. Checking the Display note in report check box includes notes in the printed version of the report. d. The domain of the standard and item being assessed. e. The rubric used to score the Proficiency Rating Scale. f. The standard being assessed. g. The item being evaluated and scored. h. The proficiency category documented for the item. This determines the number of points earned for this item. i. Clicking Show Notes will allow the user to see any notes recorded by the evaluator. j. Shows the Evidence used to document the assigned proficiency category for the item. Revised as of

42 Scrolling down to the bottom of the page will show general Evaluator Comments, and a place for Staff to respond. For instructions on responding, see the Staff Response section, page 38. STAFF RESPONSE If the administrator/evaluator has posted the Proficiency Rating Scale Results, then the staff member may access the Proficiency Rating Scale Results details and record a response. This is done by: 1. Navigating to the Results tab and selecting the Rating Scale in the Proficiency Rating Scale widget. 2. Click the Proficiency Rating Details link. 3. Scroll to the bottom of the page. 4. Simply type in your response 5. Click the Save button. 38 Revised as of