Evidence-Based Management: Techniques and Strategies for Using Assessment for Achieving Overall Institutional Effectiveness

Transcription

1 Evidence-Based Management: Techniques and Strategies for Using Assessment for Achieving Overall Institutional Effectiveness Sean McKitrick Vice President Middle States Commission on Higher Education Philadelphia, PA December 5,

2 Learning Goals Review what is expected of institutions in demonstrating compliance with Standard 7 (Institutional Assessment) Discuss institutional assessment information that is commonly used by campuses, and some themes associated with such use alignment, integration, meaningfulness, and usefulness Suggest techniques that campuses successfully use in institutional assessment to engage in periodic and useful evidence-based management 2

3 Standard 7 Institutional Assessment: The institution has developed and implemented an assessment process that evaluates its overall effectiveness in achieving its mission and goals and its compliance with accreditation standards. 3

4 4

5 In a nutshell An accredited institution is expected to possess or demonstrate the following: Identification of institutional and unit goals and their inter-relationships with one another and with the institution s mission Collection of information related to those goals Evidence of sharing of results among appropriate stakeholders Use of data for documenting success and improving programs and services A process which informs institutional and unit planning A process that is periodically evaluated

6 Assessment Infrastructure Integration Sustainability and Planning Validity and reliability Usefulness Cost-effectiveness and efficiency 6

7 Integration Institutional and unit goals are clear. Institutional and unit outcomes have something to do with one another. Avoidance of silos--sometimes this starts with a strategic plan. 7

8 Sustainability Cost-effective Organized Efficient Periodic, not episodic 8

9 Validity and Reliability Accurate Directly related to goals they assess Reliable--consistently applied over time and used with confidence by key stakeholders Periodic, not episodic 9

10 Useful Discussions lead to recommendations to enhance performance with regard to the goals. Recommendations affect resource allocation decisions. Multiple stakeholders can describe how a regular, systematic process has been used in their own contexts, vertically and horizontally. 10

11 Cost-Effective & Efficient Is it organic or imposed? Can you describe, at least qualitatively, the "returns on investment" that have resulted from your assessment efforts? Or, do your choices of assessment approaches exhaust key people such as faculty, staff, and administrators? 11

12 Tips for Successfully Navigating the Institutional Assessment Standard 12

13 Tip 1: It is a Good Idea to Organize a Meta- Strategy First and Share with Your Supervisor Periodically for Support and Advice

14 Planning a Process 14

15 Tip 2: Think You Can Go It Alone Without A Provost/VP Willing to at Least Consider What Assessment Information Says? Maybe But Not Likely

16 Value of Leadership An interested and invested VP or Dean makes a big difference when it comes to closing the loop Assessment professionals (student learning assessment, institutional assessment, a combination thereof, or all) are not the policy makers Overall success in closing the loop is not within the domain of the person/office assigned with assessment tasks At the time of an accreditation visit, this is usually quite apparent Documenting this via memos, signatures, etc., is usually a good idea.

17 Tip 3: Don t Let People Convince You that Electronic Portfolio Systems are the Only Convenient Substitutes for Closing the Loop

18 Electronic Portfolio Systems Think of the purpose if it is to facilitate conversations and produce assessment information that is useful (and is worth the expense), GREAT IDEA But if you think electronic portfolio systems will ensure that such conversations and recommendations will occur, THINK AGAIN If the purpose is to store information aligned with standards, this is not closing the loop. Still, use MS Excel, wikis, etc., to share documentation and to demonstrate that you are closing the loop

19 Tip 4: To Demonstrate that you Close the Loop, Document (Within Reason)

20

21 Tip 5: Mundane Documentation Might Provide More Powerful Evidence than Dramatic Evidence

22 Dramatic and Mundane Ways to Close the Loop Dramatic Mundane Faculty, staff, and administrative discharges Curriculum overhauls New programs New buildings Institutional initiatives Adjustments in lesson plans Follow-up studies Organization of task forces Highlighted pre and post changes in course syllabi Discussions with service providers

23 Tip 6: Periodic Reviews Are GREAT Ways to Document Closing the Loop

24 Periodic Reviews Important to have a guidelines document that explicitly indicates that self-study should have assessment in it and that reviewers should consider assessments when making recommendations Count number and percentage of recommendations not enacted, being considered, and enacted

25

26 Tip 7: Linking Budgeting and Planning Based on Assessment Can Be Documented

27

28 Tip 8: The Process Takes Time

29 From establishment of system to implementation takes time to develop 3-5 years at a minimum

30 Assessment Techniques A Few Campus-Based Examples 30

31 Using Dashboards Backed Up With Meeting Minutes or Other Documents 31

32

33

34 Assessment on a Dime: IPEDS Feedback Report 34

35 Documenting Use of Assessment 35

36 Assessing Assessment 36

37 Questions?