Research Brief: Action Step Assessment Outcome Utilization within the Resource Allocation Process

Transcription

1 OFFICE OF PLANNING, RESEARCH, AND ACCREDITATION Research Brief: Action Step Assessment Outcome Utilization within the Resource Allocation Process Jennifer Tucker Klein Date Created: 6/16/2016 Last Updated: 6/16/2016

2 Executive Summary: Saddleback College s strategic plan Goal IV asserts that, Saddleback College will strengthen long-term financial health and institutional effectiveness through integrated planning and resource allocation. Objective 4.1 under Goal IV aims to, Systematically assess the effectiveness of planning and resource allocation district-wide. Under this objective, the college has put forward several action steps to make progress to reach this vision. One of the key actions steps is 4.1.1, which is to, Refine the college process to ensure verified and consistent data and verifiable assessment outcomes are used to substantiate resource allocation. This report outlines the ways in which the college understands use of data in the current resource allocation process. Progress to Date: In the summer term of 2016 the Office of Planning, Research and Accreditation deployed an electronically-based survey to 119 key individuals involved with the resource-planning process. The survey (Appendix A) consisted of six questions using a combination of forced-choice, multiple answer, and open-ended items. The list of key individuals obtaining the survey included college administrators and directors whose s were associated with a submitted resource allocation request for the year. All responses were recorded anonymously to ensure privacy of individual records. Of the 119 potential participants, a total of 63 responses were collected (53%) over a three-week period. Results: The majority of respondents reported using evidence of assessment in their allocation process. Q1. Did your resource allocation request(s) include evidence of assessment (i.e. PR/AURs, SLO/AUOs, inform data)? Yes 50 (79%) No 4 (6%) I Don t Know 9 (14%) Total 63 (100%) Saddleback College - Office of Planning, Research, and Accreditation 1

3 Many respondents openly noted that data helped derive conclusions about demand and/or areas of need in the resource allocation process. Positive examples of data utilization include confirmation that data informed growth in certain areas. Some improvement could be made for individuals who cited data sets or data points without a reflection on what the implications of the data were, in terms of informing the decision-making process about resources. Q2 Please use the area below to describe the ways your assessment data informed your resource allocation process. Increased crowding in the clinical areas resulted in the need to replace some clinical time with simulation on campus so we requested more equipment for simulation Assessed resource needs were determined by analyzing resource deficiencies and how addressing those needs can increase department performance. Safety, productivity, student success and/or business continuity support Through the AUR process. It explains/justifies the need. But also see the comment below (more explanation options are needed) Created a task list to electronically measure the efficiency of completing projects. It will also be used to assess the variety of data requests. Finally, it will help determine appropriate workload ratios. Assisted in the development of items that needed to be requested in the resource allocation process. All information taken from PR was used to justify need during the allocation process and how it would affect student success and institutional effectiveness. Help is gauging the request and the needs I didn't use assessment data. I only used PR page reference and narrative reference. The request was for equipment. Helped determine needs of the department Used market demand as evidence of need. Used as references to show need. We look at the data to see if equipment is being used by students. Used LMI data and presented findings to counseling staff to support new software request I really don't know how it informed the process because I wasn't privy to the discussion and decision. It might not have been used at all. We referenced PR because it was required. It was intended to show that the need was on going. We have identified a need for an increased number computers in a lab, so that we can enroll/serve more students. The data analysis revealed areas that were growing and areas that were in decline, so we made adjustments to our requests. I don't remember Helped to select the items needed. Program review Document provided by staff The specific programs requesting resources used outcome data, enrollment data, as well as input from their Advisory Boards as part of their request for resources. The type of data needed was not available from research office (i.e. was not related to any datasets we currently manage). It confirmed growth we knew we had experienced. Student and employee survey, outside research agency, recommendations from the CCCCO, PR/AUR, etc. The assessment data was included in the Program Reviews submitted by the two departments making the request. Saddleback College - Office of Planning, Research, and Accreditation 2

4 Q3. IF YES from Q1, Please check off all of the types of evidence included in your resource allocation request(s): Total Yes from Q1 50 PR/AUR Page Reference 29 (58%) PR/AUR Narrative Reference 27 (54%) PR/AUR Data Set (inform) 10 (20%) How SLO/AUO improves with more resources 11 (22%) Other-Specify 4 (8%) Individuals who noted Other types of evidence used for resource allocation cited using other sources of data that were not available through the research office. Examples of external data include building condition assessment reports, safety data, and business continuity reports. Of the individuals reporting use of data in the resource allocation process, some (26%) were unsure of how assessment data could inform resource allocation requests. Another 22% of individuals either skipped this item or did not think that assessment data could help inform resource allocations. Q4. IF YES, from Q1, Do you think that assessment data could help inform your resource allocation request(s)? Total Yes from Q1 50 Yes 27 (54%) No 6 (12%) I Don t Know 13 (26%) Skipped Item 5 (10%) Q4 addressed the extent to which respondents would like more training in integrating assessment data with resource allocations. For this item, more than half of the individuals surveyed indicated wanting training. Such findings speak to the importance of continuing an institutional effectiveness plan for the college. Q5. IF YES, from Q1, Would you like more training on integrating assessment data with your resource allocation request(s)? Total Yes from Q1 50 Yes 26 (52%) No 10 (20%) I Don t Know 9 (26%) Skipped Item 5 (10%) Saddleback College - Office of Planning, Research, and Accreditation 3

5 Q6. Comments about Assessment Data in Resource Allocation Processes: A way to more directly identify needs and then to report if needs were met through the resources that were allocated by somehow re-measuring it All items approved through the resource allocation process should he purchased independent of workday challenges. Processes should be put in place to ensure all approved items are actually purchased/acquired. Given that student success and instruction is our priority, we should consider looking at a way to prioritize facilities requests specifically from M&O separately form instructional facilities requests as a way to make sure that our M&O systems, equipment and infrastructure needs are not taking a back seat to instruction when prioritizing available funding? I do not know what we are talking about I don't think I have any resource requests because I may not have heard about the "open window" to submit requests. I think we need more training into TracDat. It becomes cumbersome to me to handle all of the different areas and retrieve the entered information it is a very formal and cumbersome process to attach the data to the request. The program is so bad - it does not save your work, unless everything is completed on the page. You might spend 20 min typing everything up, but if even one little thing is missing, you have lost it all. Very, very frustrating. Also, it does not allow to elaborate at all, only quote a page in PR. For a non-specialist, an explanation/memo window is needed. It would have been nice to have completed outcomes from prior years to inform planning and resource allocations. more training would be good Much more information is needed for faculty to answer these survey questions. This is a poorly made survey. My department at our level, don't hear nothing if that!! No. Just the process. Still no answer regarding our request. Not involve except related to my own department NO--thank you for doing this survey. It should prove very helpful as we move forward. Our resource allocation process is good... but could be strengthened by making strong links to evidence. Most AURs and PRs are more narrative in nature than derived directly from data. Part of the issue may be that many of us are still trying to figure out what key questions we need answered in order to substantiate our request. The next big "evidence gap" is on the backend... affirming that the resource decision/investment achieved the desired impact we thought it would achieve. We rarely double back and access the effectiveness of our resource decisions. Thank you for your help. The entire resource allocation process needs to be revised to align more with instruction related requests, faculty support, and student support programs. The prior AUR data was not collected, thus it was difficult to ascertain what could be improved. However, in the AUR, assessments were created so that data will be able to be used and inform the next iteration. This will enable a reflective process whereby resource allocations are informed by departmental data. The Resource Allocation process should be simplified/accessible and more transparent. There is still much confusion about the relationship between Program Reviews and Resource Allocation Process. Too complicated and time consuming Unclear how assessment data is relevant to some requests, but with training this might become more apparent. We don't hear anything about that! Saddleback College - Office of Planning, Research, and Accreditation 4

6 APPENDIX A Resource Allocation Survey In order to support progress on our strategic plan, we are conducting a very short survey about the resource allocation process. This survey is confidential and will only take 5 minutes to complete. Please complete the survey as it relates to your submitted resource requests for the allocation year. If you have any questions about this survey, you may contact the Director of Planning, Research and Accreditation, Jennifer Klein, at jklein26@saddleback.edu or Please complete this survey no later than June 15, Thank you for contributing to our strategic planning process! Q1 Did your resource allocation request(s) include evidence of assessment (i.e. PR/AURs, SLO/AUOs, inform data)? Yes (1) No (2) I don't know (3) Q2 Answer If Did your resource allocation requests include evidence of assessment (i.e. PR/AURs, SLO/AUOs, inform data)? Yes Is Selected Q3 You indicated that you used assessment data in your resource allocation request(s). Please use the area below to describe the ways your assessment data informed your resource allocation process. Answer If PR/AUR page reference Is Selected Q4 Please check off all of the types of evidence included in your resource allocation request(s): PR/AUR page reference (1) PR/AUR narrative reference (2) PR/AUR data set (inform) (3) Specific reference to how your SLO/AUOs would improve if more resources were provided (4) Other (5) Q6 Do you think that assessment data could help inform your resource allocation request(s)? Yes (1) No (2) I don't know (3) Saddleback College - Office of Planning, Research, and Accreditation 5

7 Q7 Would you like more training on integrating assessment data with your resource allocation request(s)? Yes (1) No (2) I don't know (3) Q8 Any other comments related to the resource allocation process as it relates to including assessment data? Q9 Thank you for taking the time to complete this survey! Saddleback College - Office of Planning, Research, and Accreditation 6