Assessment: Assessment Unit Planning

Transcription

1 Assessment: Assessment Unit Planning Program Review - Office of Institutional Effectiveness Discipline / Department Mission Statement: The Office of Institutional Effectiveness provides leadership for college-wide research activities, assessment of student learning and institutional outcomes, planning, accreditation, enrollment management, pathways, and other activities directly impacting institutional effectiveness at the College. The Office of Institutional Effectiveness works with a large number of faculty, administrators, and staff throughout the College and District. Creating and maintaining collaborative partnerships is key to being effective as an assessment, research, and planning leader. Mission: The Office of Institutional Effectiveness integrates program review, assessment, and research to enable the College to report on and analyze the effectiveness of its programs and services. The Office facilitates the use of strategic inputs to inform planning, decision-making, and policy development at the College. Staffing: RCC s Office of Institutional Effectiveness is staffed by one full-time Dean and one full-time Director. The office also shares an administrative assistant with the Office of the President. Upcoming Circumstances or Planning Considerations: In , the Office of Institutional Effectiveness will be realigned and placed under the Vice President for Planning and Research. At that time, the Office's structure will be reviewed and potentially modified. Effectiveness and Engagement Analysis: Primary Group Served: College Leadership: The Office of Institutional Effectiveness provides substantial support for the Strategic Planning Process including input for the Strategic Plan, Educational Master Plan, Enrollment Management Plan, Accreditation, and Program Review and Plan. The Dean, Institutional Effectiveness serves as the administrative co-chair of the Institutional Effectiveness Leadership Council, the Assessment Committee, and the Program Review Committee. The Dean also plays a substantial role in leading equity efforts and planning college-wide. Faculty: The Office of Institutional Effectiveness has a primary role of providing data and coaching faculty for assessment, program review and planning, and enrollment management. The Office also performs ad hoc analysis regarding program effectiveness. Grants: The Office of Institutional Effectiveness supports grant proposals including working closely to craft grants and identify ways that data can be used to support grant proposals. Operationally, the Office of Institutional Effectiveness supports on-going assessment to improve student success, access, and organizational effectiveness and efficiency. It is important to note that both quantitative and qualitative methods are used to assess instructional and non-instructional efforts. The OIE plays a foundational role in encouraging and supporting the use of metrics to support assessment and decision make at all levels of the organization. To be most effective and useful, data must be accompanied by support in how to use the data meaningfully. Effectiveness and Engagement Analysis: Effectiveness: The Office of Institutional Effectiveness does not currently have a formal method for evaluating effectiveness. The informal method involves discussing the Office's support to the internal customers. As the office grows, we can discuss ways to more effectively evaluate and prioritize the office's activities. Effectiveness and Engagement Analysis: Org Chart and Meetings: See attached documents for visual of the role the Office of Institutional Effectiveness plays in the context of IE-LC. Effectiveness and Engagement Analysis: Communication: The Office of Institutional Effectiveness Dean serves as the administrative co-chair of the Institutional Effectiveness Leadership Council, the Program Review Committee, and the Assessment Committee. The Dean also attends many other on-going meetings. Via this method, the Office learns what projects are on-going and what support can be offered. The office also encourages on-going communication about assessment and 11/07/2017 Generated by Nuventive Improve Page 1 of 10

2 equity as part of these conversations. Effectiveness and Engagement Analysis: Technology & Equipment: The Office of Institutional Effectiveness serves as an administrator for TracDat. TracDat is the online system used to manage assessment and Program Review. As part of this administrative duty, the Office provides on-going training and technical support for this system. The system is fairly userfriendly. The Office of Institutional Effectiveness does need additional support from a data-entry perspective for TracDat because the system does not work with CurricuNet -- our curriculum management system. Any changes to curriculum, courses, SLO's, etc. must be manually updated in TracDat. Effectiveness and Engagement Analysis: Challenges: The Office of Institutional Effectiveness needs a Business Intelligence System and SharePoint These pieces of technology would better enable the Office to focus on data analysis instead of data collection and manipulation. The Office also needs an analyst who can perform data manipulation and technically support the Dean and Director. Initiative/Project Details: THEME: CA GUIDED PATHWAYS Initiative directly supports RCC's efforts at redesigning student experiences using the CA Guided Pathways framework. (PUT SPECIFICS ABOUT YOUR INITIATIVE HERE). Target:. Initiative/Project Details: 3 Grants Allocation Integration of Equity, SSSP, and other Planning RCC has several initiatives which are grant-funded. Beginning in Fall 2017, these initiative have to be reported on systematically in a combined way. The Office of Institutional Effectiveness will need to provide substantial support for the completion of this document as well as the evaluation of these programs' effectiveness. Initiative/Project Status: In Progress Year(s) Implemented: , Target: Completed, combined report Revise and envision structure for plan. This will be completed in conjunction with an Innovation and Planning mini-grant. (Active) Initiative/Project Target: Defined structure with integration of all aspects of involved grants and plans. Implementation Timeline: Winter 2017 Related Documents: RCC Planning Grant Proposal.docx Grant Visuals.pptx Plan has been revised to 3 Allocations. The OIE is participating in the conversation and structure of these programs Completed plan including supporting data and analysis (Active) Implementation Timeline: Fall /15/ : Institutional Effectiveness - Integrate research, assessment, and program review to enhance understanding of student learning. 11/07/2017 Generated by Nuventive Improve Page 2 of 10

3 Initiative/Project Details: THEME: CURRICULUM AND ENROLLMENT MANAGEMENT This initiative might include steps to evaluate, assess, modify, and monitor course offerings to better align the curriculum with student needs and FTES targets. Put your initiative details here. Target:. Initiative/Project Details: Enrollment Management Planning Support The Office of Institutional Effectiveness has been charged with providing on-going detailed support to the Instructional Deans and Department Chairs as they plan and revise their enrollment throughout the academic year. Initiative/Project Status: In Progress Year(s) Implemented: , Date Started: 09/01/2016 Target: Automated report which can be accessed by Deans and Department Chairs Work with the District Office to create an effective and automated method for the Deans and Department Chairs to use to track their enrollment versus targets. One the dashboard is updated, the Office must work with the Deans and Department Chairs to train on the dashboard's use and definitions. (Active) Initiative/Project Target: Completed, automated, report at the Department and Discipline level. This report should be produced by the District Office and included as a tab on the Enrollment Management Dashboard. Implementation Timeline: In place by April /22/2017 Conclusion: Target Partially Met With the new VP, this role has substantially increased. The office is working to support it, but would like to hire a new analyst to help with the workload Continue working with the VPAA to develop appropriate tracking and reporting templates. The OIE is also creating a "data warehouse" of EMD's by day to allow for tracking of wait list changes and enrollment changes over time. This project is requiring a substantial amount of excel support including reporting on a weekly -- and often daily -- basis. To be able to fully support the VPAA's needs, the OIE needs to hire an analyst with strong excel capabilities. While not a fulltime position, this analyst's time can be split between Enrollment Management, Strong Workforce reporting, and OIE reporting for grants. (Active) Initiative/Project Target: Continue supporting VPAA. Hire analyst. Implementation Timeline: Fall post for the analyst position. Resources Needed Analyst -- part time for enrollment management, but full-time position will be split to support Strong Workforce reporting, Enrollment Management supporting, and other OIE duties. - The VPAA needs dedicated support for enrollment management supporting. Year of Request: Resource Category: Human Resources - Classified Staffing Projected Cost : Disciplines / Departments sharing Cost of Resource: Grant funded (transformation grant and strong workforce grant) Support from District IT Office - Significant modification of the Enrollment Management Dashboard is needed to produce this 11/07/2017 Generated by Nuventive Improve Page 3 of 10

4 report. Resource Category: Other - Staff Development/Guest Speaker/Conference/Etc Projected Cost : 0 Disciplines / Departments sharing Cost of Resource: District Office Support 1.2: Student Success - Consistently use data to make decisions and to understand and support evolving student needs. 2.1: Student Access - Ensure that all students have equitable access to the college s programs, courses, and services. Initiative/Project Details: THEME: LEARNING ENVIRONMENT IMPROVEMENT This initiative might include long-term plans to assess, evaluate, modify, and monitor the department's learning environment. Examples include lab effectiveness, curriculum support (Greenhouse, Native California Gardens), etc. Put your initiative details here. Target:. Initiative/Project Details: Program Review and Planning including Process Assessment The Office of Institutional Effectiveness provides a substantial amount of data as well as coaching at all levels of the college as the units complete their Program Review and Plan and annual updates. Additionally, the Dean chairs the Program Review Committee which has been charged with assessing and revising the template and process in preparation for the March 2017 cycle. Initiative/Project Status: In Progress Year(s) Implemented: , , , , , Date Started: 01/05/2015 Target: Readily available data and trained faculty and completed assessment report and updates Complete the revised program review and plan template as well as establish a process. (Active) Initiative/Project Target: Template in TracDat as well as a working document for training and trained faculty who have entered the data into TracDat. Related Documents: RCC Strategic Planning Report Card Feb 2016 Feb2wlm 2pm.docx Template was completed and most units completed the review Work with the Program Review Committee to revise the TracDat Program Review and Planning template. Provide Fall 2016 data in time for the March 2017 data updates. (Active) Initiative/Project Target: Winter 2017 Strategic Plan update as well as individualized course and student success data. Updated TracDat Template. Updated and documented assessment of Program Review and Planning process. 06/22/ /07/2017 Generated by Nuventive Improve Page 4 of 10

5 Implementation Timeline: Winter 2017 data sets built and loaded onto TracDat. March 2017 coaching for Program Review and Plan. March 2017 Resources portion of Plan added into TracDat. Final evaluation of Program Review and Planning Process documented and reported out to EPOC by May /22/2017 Conclusion: Target Partially Met Process was evaluated and revised. There is still some work which needs to be done -- particularly for resources and for faculty prioritization. The Program Review Committee is working on that in the Fall Continue to revise the template and process. Particular focus for this year will be the resource allocation process and the faculty prioritization process. For the resource allocation, the OIE will work with the Program Review Committee as well as Technology Committee and Financial Resources Committee to create processes. (Active) Initiative/Project Target: Processes in place for faculty prioritization and resource allocation. Implementation Timeline: Both processes should be drafted and reviewed by the appropriate LC's in Fall 2017 for implementation and training for the next PRaP cycle. Resources Needed SharePoint 2013 Technology Manager - In order to fully implement the PlanningPoint portion of TracDat, RCC needs to activate SharePoint While the District owns this Microsoft product, it has not been enabled. Because of this, the integration of PlanningPoint (Program Review) with relevant data is clunky and time-consuming. Resource Category: Human Resources - Classified Staffing Projected Cost : Disciplines / Departments sharing Cost of Resource: District IT position shared District-wide. 3.2: Institutional Effectiveness - Integrate research, assessment, and program review to enhance understanding of student learning. Initiative/Project Details: SLO and PLO Assessment Results entered into TracDat and used to inform Program Review and Planning The Office of Institutional Effectiveness has the role of facilitating, tracking, and reporting on SLO, GE, and PLO assessment. Working closely with Riverside City College s Assessment Coordinator, assessment at all levels of the College is one of the primary responsibilities of the Dean, Institutional Effectiveness. The Dean serves as a non-voting representative on the Riverside Assessment Committee (RAC). The RAC meets monthly, sharing assessment best practices as well as assessment challenges. This committee is also responsible for coordinating assessment at the department level. One of RAC s major initiatives for will be to include non-instructional departments in assessment conversations and activities. The Office of Institutional Effectiveness serves as the administrative coordinator of assessment activities. TracDat is the software the College uses to collect and report on these activities. One significant Assessment initiative coordinated by the Dean, Institutional Effectiveness for was the revision and update 11/07/2017 Generated by Nuventive Improve Page 5 of 10

6 of the Riverside Assessment Plan. This plan was completed in September Initiative/Project Status: In Progress Year(s) Implemented: , , Target: Annual Report with completed assessment activities Created a standard report from TracDat with SLO's and PLO's. This report was regularly distributed and used to inform assessment. Also created a systematic approach to training for TracDat use. (Active) 84% of SLO's scheduled to be assessed in were assessed In coordination with the PLO workgroup, facilitate the creation of a systematic template for PLO assessment and complete planned PLO assessment for (Active) Initiative/Project Target: Completed PLO template and completed PLO assessment Implementation Timeline: January 2017 for completed template and May 2017 for completed PLO assessment. 06/15/ /15/2017 PLO templates and pilot completed or well underway. Several program-level assessments were completed in (Math ADT, Kinesiology Cert, English ADT, Nursing, FTV, Music, ADJ Cert) Continued implementation of PLO assessment (Active) Initiative/Project Target: Continued completion of PLO assessment including results entered into TracDat and used to inform Program Review and Planning. Resources Needed Data Entry Support - CurricuNet does not communicate electronically with TracDat. All curriculum changes must be manually entered into TracDat. The Office of Institutional Effectiveness needs approximately 80 hours of administrative support each summer to update TracDat with all curriculum changes. Resource Category: Human Resources - Classified Staffing Projected Cost : : Student Success - Increase student engagement, learning, and success by offering a comprehensive and flexible curriculum, including clear pathways for achieving certificates, degrees, and transfer-ready status. 1.2: Student Success - Consistently use data to make decisions and to understand and support evolving student needs. 2.1: Student Access - Ensure that all students have equitable access to the college s programs, courses, and services. 3.2: Institutional Effectiveness - Integrate research, assessment, and program review to enhance understanding of student learning. 11/07/2017 Generated by Nuventive Improve Page 6 of 10

7 Initiative/Project Details: District Strategic Plan Support Work with "Team A" and "Team B" in supporting the District Strategic Planning Process. As part of this process, it will also inform RCC's development of Benchmarks and a Balanced Scorecard. Initiative/Project Status: In Progress Year(s) Implemented: , Date Started: 05/01/2017 Target: Plan in place Initiative Category: Category 2: Initiatives which need resources to be implemented but include one or more of these considerations Begin providing baseline data and context for both teams. Work with team leaders to craft a "plan" to meet the deadlines. (Active) Initiative/Project Target: Team A's target is to have the assessment and evaluation piece completed by September. Team B's plan is to have a recommendation for Benchmarks as well as a draft completed by September. Implementation Timeline: Meet in May, June, and over summer to continue planning Continue working with the teams in Fall 2017 to craft the "A" and "B" portions, providing context and information for the subsequent planning. (Active) Initiative/Project Target: Sections A and B completed by October 2017 Implementation Timeline: Sections A and B completed by October 2017 Initiative/Project Details: COMPLETED / DISCONTINUED. Target:. Initiative/Project Details: Strategic Plan Annual Update Each January, the Office of Institutional Effectiveness will produce a report updating the metrics for RCC's Strategic Plan. This plan must include disaggregated data to inform discipline, department, and Unit Program Review and Planning (PRaP). Additionally, the Office must provide technical support for the PRaP updates in TracDat. Finally, the Office must provide substantial input and support to the IE-LC as they evaluate the Strategic Planning and PRAP processes in this first cycle. Initiative/Project Status: Discontinued Year(s) Implemented: , , , , , Date Started: 01/05/2015 Target: Annual Report produced in time for February FLEX Day Event Produce metrics and analysis for plan update (see Winter 2016's document in Document Repository). Produce disaggregated data to be used at the discipline level as part of the PRaP annual update. Provide technical training and support for TracDat and provide substantive support for the IE-LC as they evaluate the process. (Active) Implementation Timeline: January 2017 produce updated data and reports. February 2017 provide updates at FLEX. March 2016 provide at least 6 open training sessions for TracDat support. April/May 2017 provide significant technical and planning support to the IE-LC as they evaluate this first Strategic Planning cycle. Related Documents: RCC Strategic Planning Report Card Feb 2016 Feb2wlm 2pm.docx 06/22/ /07/2017 Generated by Nuventive Improve Page 7 of 10

8 Completed -- winter KPI Update was completed and distributed in February. Resources Needed Business Intelligence System Support Needed - Most of the processes for tracking and reporting the Strategic Planning KPI s are manual. Data must be collected and combined from sources including MIS files, National Student Clearinghouse files, Colleague files, and other stand-alone reports into useable and meaningful formats. This is time-consuming and also introduces opportunities for errors. As an effort to create a more effective and efficient data collection and reporting structure, the Dean, Institutional Effectiveness is working with the Deans for the other 2 Colleges, the Dean of Planning and Institutional Research at the District, and the District Information Technology staff to develop a data warehouse or a datamart and a Business Intelligence Reporting System. A Business Intelligence system would not only allow more efficient data analysis by the Office of Institutional Effectiveness, but will also provide reports to instructional and non-instructional departments. While there are some promising Business Intelligence solutions, selecting and implementing the best choice is time consuming and requires substantial IT support. IT is working with Institutional Effectiveness but it is likely that a solution will not be implemented in Until a Business Intelligence system is established, the OIE will have to do a large amount of manual data manipulation using SPSS and Excel to create the data linkages enabling reporting and analysis of course and student success. Resource Category: Technology - Hardware Projected Cost : Disciplines / Departments sharing Cost of Resource: Shared District-wide 3.2: Institutional Effectiveness - Integrate research, assessment, and program review to enhance understanding of student learning. Initiative/Project Details: Completion Counts through Pathways Evaluation The Office of Institutional Effectiveness has been given a primary task of tracking and evaluating all aspects of the Completion Counts through Pathways. See attached document for detailed plan. Initiative/Project Status: Discontinued Year(s) Implemented: , , , , , Date Started: 09/01/2016 Target: Detailed, tracking and reporting system for students enrolled in the Completion Counts through Pathways Develop baseline tracking for students in the four different pathways in the college including college preparedness. (Active) 11/07/2017 Generated by Nuventive Improve Page 8 of 10

9 Conclusion: Target Not Met Monitored and assessed -- realize target was too ambitions and reset target. 03/03/ Create detailed evaluation plan for Pathways and identify data needs and data collection gaps. Conduct focus groups to provide qualitative information as well as quantitative regarding student support experiences at the College. (Active) Initiative/Project Target: Detailed planning document Implementation Timeline: Fall 2016 for draft and Winter 2017 for identification and evaluation of data gaps. Related Documents: RCC Equity Focus Groups SOW DRAFT.docx Completion Counts through Pathways Evaluation Plan.docx Implementation by Design Article.pdf Completed -- this project has been discontinued because it is now included in the 3 grants allocation Begin tracking and reporting on students in robust manner including working with Counseling and Student Services personnel to document and analyze co-curricular support. (Active) Initiative/Project Target: Structured, systematic, reportable database updated regularly Continue tracking and reporting students (Active) Initiative/Project Target: Annual report produced as well as continual updating of database Continued tracking and reporting (Active) Initiative/Project Target: Annual report and update of database. Resources Needed 06/22/2017 Data Analyst - The Office of Institutional Effectiveness needs a data analyst to support this initiative. This specialist position would provide technical data manipulation and report writing expertise. This position would be similar to the positions currently filled in both Moreno Valley and Norco College Offices of Institutional Effectiveness. Resource Category: Human Resources - Management Staffing Projected Cost : RP Group Focus Group Support - The RP Group has developed a focus group protocol which can be used to specifically inform the systems and structures supporting RCC's students. This group will conduct targeted focus group interviews in March Year of Request: Resource Category: Other - Staff Development/Guest Speaker/Conference/Etc Projected Cost : Disciplines / Departments sharing Cost of Resource: Equity Workgroup Technical Support from District Office - RCC's Office of Institutional Effectiveness needs to be able to periodically query results from Colleague. Census data including gender, race/ethnicity, student admit status. Enrollment by course at least twice per semester, course grade at end of the semester. Total units and GPA as of the end of the semester. This information is available via MIS files but not until about 2 months after the semester ends. RCC needs support from the District Office to get this data sooner to be able to inform the curricular and programmatic discussions. Resource Category: Other - Staff Development/Guest Speaker/Conference/Etc Projected Cost : 0 11/07/2017 Generated by Nuventive Improve Page 9 of 10

10 Disciplines / Departments sharing Cost of Resource: Support from District Office 1.1: Student Success - Increase student engagement, learning, and success by offering a comprehensive and flexible curriculum, including clear pathways for achieving certificates, degrees, and transfer-ready status. 1.2: Student Success - Consistently use data to make decisions and to understand and support evolving student needs. 3.2: Institutional Effectiveness - Integrate research, assessment, and program review to enhance understanding of student learning. 11/07/2017 Generated by Nuventive Improve Page 10 of 10