Office of Institutional Effectiveness and Advancement AUP for Administrative Units

Transcription

1 AUP for Administrative Units Strategic Initiative Office of Institutional Effectiveness & Advancement Strategic Initiative Type: Annual Update Plan Date: Strategic Initiative Description: Introduction Introduction and Certification Name Ruben Arenas Bryan Ventura Laura Cruz-Atrian Adrian Shadaram Maribel Carbajal-Garcia Jeffrey T. Pang Role Acting VP of Instruction, LAS (Dean of OIEA) Associate Dean Research Analyst Assistant Research Analyst Senior Office Assistant Office Assistant Program Description Department or Unit Mission Statement Please enter your mission below. The serves the campus community and external stakeholders by improving teaching, learning, and service through data-driven decision making, cutting-edge research, authentic assessment, and professional development. November 02, :14 PM Page 1 of 7

2 Program Assessment SWOT Analysis Strengths and Opportunities (Optional) Please describe your department or unit's strengths and opportunities. Are there things that your department or unit does better than other departments/units within or outside of the college? What do our students see as your strengths? Are you offering unique programs or courses to our students? Are there interesting trends that your department or unit can take advantage of (including technological, policy, or social trends)? Where does your department or unit excel? What opportunities or initiatives does your department or unit wish to pursue? Strengths Use of predictive modeling - our office is the recognized leader in the District, and perhaps in the region for applying predictive modeling to college problems. In particular, the college has a robust model for predicting section-level enrollments that is used in conjunction with department chairs to guide scheduling. Local server housing "cleaned" district data - OIEA deployed a local server to merge data from the legacy SIS system with data from the PeopleSoft system. Due to this work, OIEA can maintain most data functions during the transition to PeopleSoft, including the elumen data loads for learning outcomes. This work is not entirely complete at this point. Commitment to professional development - the Office provides on-going professional development, and time for this professional development, to the research staff to develop skills in coding, data analytics, and machine learning. This professional development has made our research staff some of the most valuable in the District in terms of skills. Close working relationship with other units within OIEA (PD, LAO, DE, Title V). Close working relationship with all campus Departments (Academic and Student Services) - OIEA meets with Academic Departments at least twice per year to discuss course scheduling in the context of meeting student demand and providing for completion. OIEA regularly meets and attends Student Services committees to provide data support and assistance on state initiatives and local program evaluations. Vast knowledge of local (PeopleSoft) and Statewide Data Systems (LaunchBoard, LMI) - OIEA has experts in both PeopleSoft and state LMI data that is shared internally as well as externally via workshops on using LMI to develop educational programs. Opportunities Continue professional development to keep pace with research and evaluation industry trends - the space in which OIEA works is constantly changing with respect technology, and in order to keep pace we must continue to allow our employees access to appropriate professional development activities. Continue to use predictive analytics in college decision-making - although we have made significant progress in enrollment modeling, we would like to now move towards student success and retention modeling, especially regarding tools for ensuring that students receive appropriate November 02, :14 PM Page 2 of 7

3 services before they need them. Continue to develop web-applications for data dissemination and increased access. Develop staff planning skills through participation in various state and college initiatives. Weaknesses and Threats (Optional) Please describe your department or unit's weaknesses or threats. What can your department or unit improve? What do students see as your weaknesses? What factors are keeping your students from enrolling, persisting, or succeeding in your courses or programs? What obstacles are you facing (these could be technological, policy, or social obstacles)? What are similar departments or units at other colleges doing to be successful? Weakness: Ongoing staff turnover - given the strengths of our staff, turnover will remain a weakness for the office. Recently, the Dean has stepped in as the Acting VP of Instruction in Liberal Arts and Sciences, one researcher moved up to Associate Dean, and one assistant researcher left to the pursue a Ph.D. Given that the Office now oversees Research, Learning Assessment, Professional Development (and Title V), and Distance Education, it is necessary to backfill positions in order to meet the increasing workload demands. Lack of replacement for separated staff - as of this writing, no replacements have been made for separated staff or staff who have moved into other roles. This seriously hinders the ability of the Office to meet its mission or the College mission. Threats: Many of the new state initiatives require intensive data analysis to inform planning - things like Guided Pathways will require significant time from OIEA. It is unclear how this time will be made available without letting go of or deprioritizing other activities. Uncertainty of replacement positions being filled due to budgetary concerns. PeopleSoft implementation has resulted in data quality uncertainty and access - the accuracy of some data in PeopleSoft is questionable, and without significant time to review and test this data (a few weeks) we may not be reporting accurate figures in some areas. Professional Development activities have stopped due to increasing data requests and project initiatives - while PD is a strength of the Office, this strength is being threatened due to overwhelming workload (further due to a lack of replacement of separated/promoted staff). When workload becomes overwhelming, "non-essential" activities like professional development cease to be a priority. Administrative Unit Outcomes Participation Progress Please discuss your unit or department's participation in the development, revision, or assessment of your Administrative Unit Outcomes (AUOs). Please briefly describe your AUOs and what they measure. Currently, OIEA's AUOs are as follows: 1. Accessibility of research and data on the Office of Institutional Effectiveness and Advancement (OIEA) website. 2. Research and data provided for college planning purposes. 3. Support and availability in guiding decision-making and planning. 4. Quality of data, research, and interpretations provided to all campus constituents. November 02, :14 PM Page 3 of 7

4 These AUOs measure the thematic components of our office. However, after several discussions regarding the functions and future direction of our office, we have decided to rewrite our AUOs to improve outcome clarity, and address both the specific data-related services provided by our department and the functional use of our analyses in college planning and decision-making. Once the revisions have taken place, we will plan to assess our new AUOs. Program Plan Program Plan Update Goals Name Benchmark Benchmark Descriptor By Fall 2017, create an online and face-to-face training component focused on expanding use of data and the understanding of college core functions. Completed The goal, as written, has been met, though possibly not within the original time frame specified. (100%) By Fall 2017, integrate the newly developed online data system into the planning and program review processes. By Fall 2018, develop a data collection framework and an analysis workflow for providing college stakeholders with the necessary data to plan, implement, and assess various state and/or college initiatives (e.g. Guided Pathways, Integrated Planning, Multiple Measures, TOP Code Alignment, AB 705, Common Assessments, etc.). Completed Some Progress The goal, as written, has been met, though possibly not within the original time frame specified. (100%) Initial discussions and planning meetings have been held to discuss and plan for the goal. An individual or committee has been made responsible for goal. (33%) By Spring 2019, develop, test, and implement a menu of live, Significant Progress Some, but not all, changes have been put in place with respect to November 02, :14 PM Page 4 of 7

5 Name Benchmark Benchmark Descriptor web-based course, program, scheduling, and cohort outcomes data applications using open-source software to provide open data access for planning and evaluating student success. Significant Progress the goal. If necessary, resources have been deployed to meet the goal. (67%) Please briefly describe any difficultities in achieving stated goals carried over from a previous Annual Update. Some previous goals were not carried forward from the previous AUP due to the ambiguity of the goals. Some of our previous goals weren't measureable and lacked time-boundedness. In particular, goals linked to SSSP and the Student Equity Plan will need to be reworked in a future AUP to better define the support OIEA will provide to these important initiatives. In general, high turnover in staffing continues to slow down progress towards meeting goals. On the other end, the rate at which the State and college generate new initiatives also makes it difficult to gain traction on evaluation programs for initiatives. Please briefly comment on goals completed within the last year. The Office was able to make significant progress on most of it's goals. Goals that required the office to develop online data tools (web applications and elumen), expand the use of data, and increase participation have been achieved and will be revised for further progress monitoring. Specifically, the Office has developed two web-based applications (in beta-testing phase) for course scheduling and program completions (used for this AUP). The course scheduling application is being used during bi-annual meetings with Department Chairs to assess historical, current, and future enrollments. These data tools have or will be made available to the college community for ongoing usage. We completed two major goals present in the last AUP. The first goal involved creating online and face-to-face training components to support the understanding of data and college functions. These was completed via a variety of trainings on using LMI data to develop educational programs, using OIEA's program completion tool, creating mission statements and goals, and an AUP walkthrough. The materials for these trainings are available online. The second completed goal involved integrating our program completions app into the program review process. The program completions app is an integral part of the AUP this year. Recommended Actions Goal: By Fall 2018, develop a data collection framework and an analysis workflow for providing college stakeholders with the necessary data to plan, implement, and assess various state and/or college initiatives (e.g. Guided Pathways, Integrated Planning, Multiple Measures, TOP Code Alignment, AB 705, Common Assessments, etc.). Research the various initiatives and determine which statewide metrics may be applicable. Consult with colleges in the district/region regarding data collection process, analysis, and report related to a particular initiative. Determine baseline data needed for each initiative, including state data requirements and any November 02, :14 PM Page 5 of 7

6 additional data needed to help our college stakeholders make informed decisions. Develop a menu of data collection/analyses available given the district/college data system. Utilize the expertise of researchers to create additional data collection instruments/options (qualitative) for supplementary analyses. Integrate the quantitative data needed for initiatives into web-based, live data applications. Work with the Program Review and Viability Committee to integrate key performance indicators from state initiatives into the Annual Update Plan and Program Review, if applicable. Attend state-sponsored workshops on initiatives to gather additional insight into data report and required analyses. Goal: By Spring 2019, develop, test, and implement a menu of live, web-based course, program, scheduling, and cohort outcomes data applications using open-source software to provide open data access for planning and evaluating student success. Determine "frequently used or requested" data to feature in web-based applications, including course-level data, cohort progression, and achievement milestone data. Discuss the most appropriate and clear ways to display analyses and data to "non-researchers", including implementing the "3-clicks rule" which requires users to see the data they need within 3-clicks on a webapplication. Work with faculty, staff, and administrators to beta-test applications for "bugs" and functional errors. Build the capacity of the research and office staff through local research-centered conferences (RP Group) and technical skill-building conferences (R Studio Conference). Develop a series of workshops to train students, faculty, staff, and administrators on appropriate use and interpretation of the data available on each web-based application. Program Review Responses Program Review Recommendation Responses November 02, :14 PM Page 6 of 7

7 # Recommendation Progress to Date 1 2 That the program collaborate with the Vice Presidents and other units on campus to clearly define roles in the grant management process. That the program develop a cycle of outreach to other departments and units to ensure that they are aware of the services available to them aside from assistance with standard college processes. RETIRED RETIRED 3 That the program collaborate with the Learning Assessment Office to restore their reports on TracDat. The program s work on Administrative Unit Outcomes is currently at the level of proficiency. As of October 2017, all AUOs that were previously in TracDat are now in elumen. Further, the office will work to enter data from last year's AUO assessment into elumen. November 02, :14 PM Page 7 of 7