Institutional Effectiveness Guide and Template for Administrative and Student Support Services

Transcription

1 1 Institutional Effectiveness Guide and Template for Administrative and Student Support Services The purpose of this document is to provide suggestions for developing an assessment plan and a template for reporting to be used annually by administrative and student support service units. The document reflects guidelines for designing and tailoring assessment plans aligned with the needs and purposes of various administrative and educational support units required for SACSCOC institutional effectiveness reporting. SACSCOC Core Requirements: 2.5 The institution engages in ongoing, integrated, and institution-wide research-based planning and evaluation processes that (1) incorporate a systematic review of institutional mission, goals, and outcomes; (2) result in continuing improvement in institutional quality; and (3) demonstrate the institution is effectively accomplishing its mission. (Institutional Effectiveness) The institution identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results in each of the following areas: (Institutional Effectiveness) Administrative support services Academic and Student support services Research within its mission, if appropriate Community/public service within its mission Developing Administrative and Student Support Unit Assessment Plans and Reports The principal aim of developing an assessment plan is to clarify the focus and content of your plan, which should be based upon the needs of your administrative or student support unit. Approach to developing the plan should be determined by the mission of the unit. Below are a specific guide and a template to assist in developing an effective assessment plan and institutional effectiveness report. The guide aligns with the SACSCOC TTU reporting template.

2 2 Helpful hints: Build an initial roadmap for the reviewer to follow. Charted overviews, policy outlines, and, summary of goals and assessments. Organize narrative by key terms in Institutional Effectiveness guidelines. (Ongoing, Integrated, Institution-wide, Researched-Based, Systemic, Accomplishing Mission, Continuing Improvement 2.5 & 3.3.1) The IE Reviewers must see Practice, Policy, and Product (examples). Reviewers look for numbers, percentages, and comparative and longitudinal data. Combine direct and indirect measures. Use multiple assessments in each area. (Researched based. 2.5) Documentation must be ongoing and systematic. A minimum of 2 cycles should be included when comparing measures and making changes. (Ongoing. Systematic. 2.5) It is also important to include proof of analysis and integration of data and changes. Meeting minutes, agendas, discussions. This shows leaders have shared, discussed, analyzed, and acted upon the results. (Analysis. Integrated ) Highlight sections pointing to proof. (Evidence of Improvement ) Common Mistakes: No overview or clear roadmap to guide the evaluator Multiple formats in the documentation Confusion with traditional/nontraditional services and on/off campus services Inconsistent terminology throughout document Poorly align assessments with goals Mismatch between unit documentation and information in catalog or website Not enough focus on Modifications and Continuous Improvement or Program Changes/Actions due to Assessment Try to cover academic jargon or instruct on what IE is and is not Are not specific enough or too specific Write too much to cover the lack of substance Confuse personnel evaluation with department evaluation Attribute lack of consistency to prior format, method, or person Fail to close the loop: modification come from nowhere and are not tied to assessment results; No assessment results are cited (No results = no use = no improvement = no compliance); Nothing done about assessment results cited List only a summary of improvements: must include the why

3 3 Institutional Effectiveness Template Comments in italics are not part of the template, but are meant to guide you through the process. Academic Year: Administrative/Student Support Unit: Submission Date: Contact (Person submitting this report): I. Definition of Support Service Unit: Describe the mission, vision, and purpose of the Support Unit. The mission is a broad statement that reflects the aim of the administrative or educational support unit. Mission statement should: Be concise Be linked directly to TTU s mission statement Include the major functions of the unit II. Goals and Objectives (List Objectives if applicable): 1. List Each Goal a. List each objective related to respective goal The goals of administrative and educational support units should be aligned to the goals of the institution. Goals are broader/general assertions that describe the overarching long-term intended objectives of the unit. Goals may or may not be measurable and usually need to be further developed as separate/distinct objectives, that, when measured appropriately, provide evidence of how well you are accomplishing your goals. a) Example of a goal (administrative unit): To improve TTU s budget process b) Example of a goal (student support unit): To provide leadership skills to students and student leaders Define Intended Objectives (Administrative and Student Support Units) Objectives are measurable statements that describe the expected or intended quality relative to timeliness, responsiveness, accuracy, etc. Objectives often describe how well a unit intends to function or improve its functioning or the services provided. The important question in this section is: What is the office trying to accomplish?

4 4 Client satisfaction is the most common type of administrative objective. Less common, but equally acceptable administrative objectives are those that focus on the ability of clients once services have been provided (e.g., staff s ability to process a requisition after training has been provided). Administrative-level objectives must be: Linked to the university goals and mission statement Realistic in terms of the resources and support currently available within the unit Measurable feasible to collect accurate quantitative and/or qualitative data usable for making improvements to process or unit Example of administrative objective: Expand use of purchasing cards (P-Card) for commodities and services utilized by the university Define Intended Objectives (Student Support Units) Student Support Objectives are statements that target the intended knowledge, abilities, values and attitudes students are meant to be able to demonstrate after participating in a given activity and/or after using services within an educational student support unit. Student Support objectives must be specific to the unit and measurable. Example of a Student Support objective: Students participating in Learning Communities will express higher levels of personal interaction and satisfaction with the learning environment in the residence halls than other resident students III. Assessments (Related to Goal/Objectives above): Name of the Assessment Tool (List the assessment time frame) List the related Goal/Objective numbers. Use an assortment of well-matched assessment types related to respective goals. Method of Assessment refers to the measures your unit intends to use to reach the objectives (e.g. surveys, focus groups). It is important that the methods chosen enable the unit to evaluate its effectiveness in terms of services offered. A primary objective of assessment is to illuminate strengths and weaknesses in your unit that that can lead to improvement. List the timeframe that the assessment tool was implemented and data was collected.

5 5 IV. Rationale for Goals, Assessments, and the Process of Data Analysis: Describe each Assessment Tool (Use points 1-4 in the paragraph below) Describe each assessment tool and the process of periodic review of assessments used. Evaluators are looking for use of multiple assessment methods. Description of your assessment tool(s) should include (1) baseline data/information which focuses on the current state of the unit regarding a given objective whereas Performance Target refers to where the unit would like to be. State the Method of Assessment and Performance Targets. The statement(s) should (2) briefly state how you will define success based on the defined goals and performance targets. It is the unit s responsibilities to (3) make a compelling case as to why the sampling and assessment findings are an appropriate representation of the institution s services. Describe the assessment instrument that was used and why it was selected. (4) How was the data disseminated and analyzed throughout the unit to make modifications? Example: (1) Customer Satisfaction Survey will indicate that more than 80% of respondents are satisfied with our services. (2) Successful results will indicate a 15% increase from our last implementation. (3) Sampling includes all students that used our services from The Customer Satisfaction Survey incudes questions concerning Overall satisfaction with outcome, process, accessibility; Caring and helpfulness of institutional staff; Institutional (time) response to problem; Awareness of services offered; Fair and unbiased treatment; Commitment to solving issues; and Adequacy of services. These variables are important to success in this unit. (4) Results are discussed with coordinators and service staff at bimonthly meetings to define areas and strategies where improvements can be made. V. Results (Use current results compared to past/benchmark results if applicable): Highlight the name of the Assessment Tool (List Goal/Objective numbers) and present results This contains a highlighted section that includes evidence for improvement. Insert graph, tables, and charts that provide mature data for your decisions of goals and objectives, and for improvement. Please report the data results here. Discussion of the results is in the next section. This section requires actual data/information based on the previously stated objectives, methods of assessment and performance targets. The data gathered must be analyzed and formatted in a way that it provides useful information for identifying strengths and weaknesses and improving processes and services. Benchmarking Tips: The goal of benchmarking is to provide a standard for measuring, and to help identify where opportunities for improvement may reside. Setting benchmark goals with assessment data should be done so carefully. If you say that you will increase performance by specific percentage points within a time frame, chances are that you will experience a ceiling effect in the data over time. A better way to make benchmark comparisons with your data would be to compare to a rolling three year average, or compare to the national mean results of the assessment tool of applicable. This should be taken into consideration when setting your assessment benchmarks in relation to your service goals.

6 6 VI. Modifications and Continuing Improvement: Program Changes due to Assessments For Goals/Objectives #, #, & #- Describe changes made. Link to Assessment Data: Describe the link between modifications to strategies and the assessment results you reported. This section, commonly referred to as closing the loop, requires an action plan of the unit based on the achieved results. Administrative and educational support units are required to review assessment results with a focus toward uncovering services or parts of services that need to be improved. Since the express purpose of assessment is to use what is learned in the process to improve performance of units, it is important that annual assessment report contain a description of the actions or steps that were taken in response to the assessment results to improve the unit s services (e.g., enhance services, add personnel, enhance technology, etc). Discuss evidence of improvement, based on analysis of assessment results, as opposed to a plan for improvement for each goal/objective mentioned above. Note: It s okay to say that no modifications are needed at this time, but prove you looked at the data to determine this. Describe actual changes and why they were made, based on the described assessment above. You can mention upcoming plans for changes, but you should have already discussed actions that have already been implemented as a result of your data reported in the above section. Highlight the use of assessment results to improve services. The Modification and Continuing Improvement section should be the main focus of your report. It should be extensive and concise. This section is the main reason for reporting institutional effectiveness, and should stand out in the report. Suggested Codes for Categorizing Use of Results: Code Revised service Revised process Implemented new policy Implemented new process Informed budget Changed assessment Changed criteria Consultant Create/modify i nstruction Development/training Other Description Modified way a service is offered, modified frequency of service, etc. Changed reporting form or process, changed tabulation process New policy to improve service New process not simply a change in existing one Requested fiscal or human resources Developed and implemented a new assessment method or modified one Modified criteria for success Engaged someone to study and recommend changes Changed workshop, training session in response to outcome assessment Provided staff development or training Any other use of results not described above

7 7 VII. Improvements to Assessment Plan Discuss any changes made to the department s assessment plan. Include any new tools or modifications to existing tools here. Contact Academic Affairs for help: Dr. Theresa Ennis tennis@tntech.edu Dr. Sharon Huo xhuo@tntech.edu Resources: Principles of Accreditation: Foundations for Quality Enhancement (2011) Through the Eyes of an Institutional Effectiveness Evaluator (2012) Dr. Marila Palmer, SACSCOC Summer Institute Presentation, Atlanta, GA.