Seton Hall University BACCALAUREATE SOCIAL WORK PROGRAM ASSESSMENT OF STUDENT LEARNING OUTCOMES. LAST COMPLETED ON July 2, 2015

Transcription

1 Seton Hall University BACCALAUREATE SOCIAL WORK PROGRAM ASSESSMENT OF STUDENT LEARNING OUTCOMES LAST COMPLETED ON July 2, 2015 Form AS4 (B) Duplicate and expand as needed. Provide table(s) to support self -study narrative addressing the accreditation standards below. This form is used to assist the COA in the evaluation of the program s compliance with the accreditation standards below: The program provides summary data and outcomes for the assessment of each of its competencies, identifying the percentage of students achieving the benchmark The program uses Form AS 4 (B) and/or AS4 (M) to report assessment outcomes to its constituents and the public on its website and routinely up-dates (minimally every 2 years) these postings All Council on Social Work Education programs measure and report student learning outcomes. Students are assessed on their mastery of the competencies that comprise the accreditation standards of the Council on Social Work Education. These competencies are dimensions of social work practice that all social workers are expected to master during their professional training. A measurement benchmark is set by the social work programs for each competency. An assessment score at or above that benchmark is considered by the program to represent mastery of that particular competency. COMPETENCY Demonstrate Ethical and Professional Behavior Engage Diversity and Difference Advance Social Justice and Human Rights Engage in Practice- Informed Research and Research-Informed Practice Engage in Policy Practice* Engage with Individuals, Families, Groups, Organizations, and Communities Assessment Individuals, Families, Groups, Organizations, and Communities Intervene with Individuals, Families, Groups, Organizations, and Communities Evaluate Practice with Individuals, Families, Groups, Organizations, and Communities* COMPETENCY PERCENTAGE OF STUDENTS ACHIEVING BENCHMARK BENCHMARK 85% 94% 85% 91% 85% 79% 85% 75% 85% 67% 85% 89% 85% 89% 85% 89% 85% 87% Form modified to reflect data collection conducted using the 2015 EPAS. Full assessment report attached. * Prepared prior to the Final approval of EPAS, thus 1 Practice Behavior is lacking data for identified Competencies.

2 Seton Hall University BSW Program Evaluation Prepared by: Matthew Corrigan, PhD, LMSW, CASAC July 2, 2015 In Social Work education, a competency-based model is employed. The accrediting body, the Council on Social Work Education (CSWE), has identified a core of 9 basic competencies, along with several measurable Practice Behaviors to operationalize these Competencies. Table 1 below shows which Social Work core course each Competency/Practice Behavior is being primarily evaluated in. This is followed by a full listing of the Competencies and the Practice Behaviors by course. Each Practice Behavior for each Competency is measured by 3 instruments. The first is a classroom evaluation, completed by faculty teaching coursework. The Practice Behaviors are spread throughout the curriculum and represent Course Goals. These Course Goals are measured with assignments at the end of a given semester. The second measure for each Practice Behavior is from direct observation from a Field Instructor, with each Practice Behavior being evaluated. Finally, there is an exit survey for each of the Senior Years given in the Senior Research Seminar. This exit survey is a self-efficacy measure, asking students to rate their proficiency on each Practice Behavior. When combined as described below, we believe that this provides a comprehensive picture of whether or not our program is graduating students that are competent entry level practitioners. By measuring each Practice Behavior with three measures from different sources, we feel confident that we will be able to determine competency, or what specific constituency and which specific Practice Behaviors, are experiencing difficulty. The first cut of analysis will be to combine all measures. As is shown below, all measures require data to be presented on a 5 point scale (see Appendix A for data Collection Sheets). The benchmark set is that of a minimum of 85% of students will meet at least a 4 on the 5 point scale for each competency. If this benchmark is not met for a given Competency, further analysis will be undertaken to see which Practice Behaviors and which measures are proving deficient, as a way of guiding program renewal. The data produced will also give us the opportunity to assess the measures we are using. For example, if the student self-efficacy measures tend to be relatively high or low, as compared with the teacher or field instructor measures, we will examine the validity of those measures. However, as we see consistency across measures, we can assume a level of validity and look to identify potentially deficient instruction or field practice opportunities for a given Practice Behavior, or potentially a Competency.

3 Table 1 - Competency/Practice Behavior by Course Comp/PB a. X X d. X X e. X X 2a. X X 3a. X X 4a. X X 5a. X X 6a. X X 7a. X X X d. X X 8a. X X X d. X X e. X X 9a. X X CSWE 9 Core Competencies 1. Demonstrate Ethical and Professional Behavior 2. Engage Diversity and Difference 3. Advance Social Justice and Human Rights 4. Engage in Practice-Informed Research and Research-Informed Practice 5. Engage in Policy Practice 6. Engage with Individuals, Families, Groups, Organizations, and Communities 7. Assessment Individuals, Families, Groups, Organizations, and Communities 8. Intervene with Individuals, Families, Groups, Organizations, and Communities 9. Evaluate Practice with Individuals, Families, Groups, Organizations, and Communities

4 Practice Behaviors by Course SOWK 3201 Social Problems and Programs 3a. apply their understanding of social, economic, and environmental justice to advocate for human rights at the individual and system levels 3b. engage in practices that advance social, economic, and environmental justice SOWK 3511 Behavior & Environments 2a. apply and communicate their understanding of the importance of diversity and difference in shaping life experiences in their practice at the micro and macro levels 2b. present themselves as learners and engage clients and constituencies as experts of their own experiences 2c. apply self-awareness and self-regulation to manage the influence of personal biases and values in working with diverse clients and constituencies 6a. apply knowledge of human behavior and the social environment and practice context to engage with clients and constituencies 7b. apply knowledge of human behavior and the social environment, person-in-environment, and other multidisciplinary theoretical frameworks in the assessment of data from clients and constituencies* 8b. apply knowledge of human behavior and the social environment, person-in-environment, and other multidisciplinary theoretical frameworks in interventions with clients and constituencies* * only till SOWK 3512 is taught SOWK 3910 Research Methods 4a. use practice experience and theory to inform scientific inquiry and research 4b. engage in critical analysis of quantitative and qualitative research methods and research findings 4c. use and translate research findings to inform and improve practice, policy, and service delivery SOWK 4911 Theory & Practice II 7a. collect, organize, and critically analyze and interpret information from clients and constituencies 7c. develop mutually agreed-on intervention goals and objectives based on the critical assessment of strengths, needs, and challenges within clients and constituencies 7d. select appropriate intervention strategies based on the assessment, research knowledge, and values and preferences of clients and constituencies SOWK 3301 Social Policy Analysis 5a. assess how social welfare and economic policies impact the delivery of and access to social services

5 5b. critically analyze and promote policies that advance human rights and social, economic, and environmental justice SOWK 3611 Theory & Practice I 1a. make ethical decisions by applying the standards of the NASW Code of Ethics, relevant laws and regulations, models for ethical decision-making, and additional codes of ethics as appropriate to context 1b. use reflection and self-regulation to manage personal values and maintain professionalism in practice situations 1c. demonstrate professional demeanor in behavior; appearance; and oral, written, and electronic communication 1d. use technology ethically and appropriately to facilitate practice outcomes 1e. use supervision and consultation to guide professional judgment and behavior 6b. use empathy, self-regulation, and interpersonal skills to effectively engage diverse clients and constituencies SOWK 4912 Theory & Practice III 8a. implement interventions to achieve practice goals and enhance capacities of clients and constituencies 8c. use inter-professional collaboration as appropriate to achieve beneficial practice outcomes 8d. negotiate, mediate, and advocate on behalf of clients and constituencies 8e. facilitate effective transitions and endings that advance mutually agreed-on goals 9a. select and use appropriate methods for evaluation of outcomes 9b. critically analyze, monitor, and evaluate intervention and program processes and outcomes 9c. apply evaluation findings to improve practice effectiveness at the micro and macro levels SOWK 4812 Field II All Practice Behaviors to be evaluated SOWK 5111 Student Self-Assessment - All Practice Behaviors to be evaluated

6 Outcomes and Analysis As shown in Table 2 below, students have been evaluated as having met the Benchmark score of 85% for six of the 9 Competencies (# 1, 2, 6, 7, 8, & 9). We feel confident that are students are competent in these skills and no further analysis is currently needed. The benchmark was not met for Competencies 3, 4, & 5 (with respective % of student scores of 79%, 75%, and 67%). The ranges for these scores between Practice Behaviors are as follows: 3 (79%-80%); 4 (72%-78%); and 5 (67%-67%), with no Practice Behavior reaching the Benchmark of 85%. There was greater variation among the data sources. For Competency 3, the range averages were 78%, 73%, & 86% (Course, Field, Self); Competency 4 showed 53%, 75%, 87% (Course, Field, Self); and Competency 5 had 59%, 70%, & 88% (Course, Field, Self). With the Student Self-Evaluations showing consistently and significantly higher ratings, these were ratings were summed for all competencies. For the 3 Competencies determined to not meet the Benchmark, the Field Evaluation fell between the Course Instructor Evaluation and the Student Self- Evaluation. However for the 6 judged to meet the Benchmark, the Field Evaluation was lower than both other measures. Table 2 Outcomes Comp/PB Course Number 4+/n Field Number 4+/n Self-Evaluation Number 4+/n Total Number 4+/n % at Benchmark 1a. 28/36 11/13 28/29 b. 33/36 12/13 29/29 c. 28/36 12/13 29/29 d. 34/36 12/13 26/29 e. 34/36 12/15 29/29 Comp. Total 157/180(87%) 59/67(88%) 141/145(97%) 357/380 94% 2a. 28/30 9/13 28/29 b. 28/30 8/13 29/29 c. 28/30 9/13 29/29 Comp. Total 84/90(93%) 26/39(67%) 86/87(99%) 196/216 91% 3a. 30/39(77%) 16/22(73%) 25/29(86%) 71/90(79%) b. 31/39(79%) 16/22(73%) 25/29(86%) 72/90(80%) Comp. Total 61/78(78%) 32/44(73%) 50/58(86%) 143/180 79% 4a. 9/17(53%) 16/22(73%) 26/29(90%) 51/68(75%) b. 8/17(47%) 11/13(85%) 27/29(93%) 46/59(78%) c. 10/17(59%) 16/22(73%) 23/29(79%) 49/68(72% Comp. Total 27/51(53%) 43/57(75%) 76/87(87%) 146/195 75% 5a. 46/78(59%) 15/22(68%) 26/29(90%) 87/129(67%) b. 46/78(59%) 16/22(73%) 25/29(86%) 87/129(67%) Comp. Total 92/156(59%) 31/44(70%) 51/58(88%) 174/258 67% 6a. 27/30 8/13 28/29 b. 34/36 8/13 28/29 Comp. Total 61/66(92%) 16/26(62%) 56/58(97%) 133/150 89% 7a. 29/33 8/13 26/29 b. 28/30 15/22 28/29 c. 32/33 8/13 27/29 d. 32/33 8/13 27/29

7 Comp. Total 121/129(94%) 31/48(65%) 108/116(94%) 260/293 89% 8a. 30/31 8/13 26/29 b. 27/30 15/22 27/29 c. 30/31 12/15 26/29 d. 30/31 8/13 27/29 e. 30/31 8/13 27/29 Comp. Total 147/154(95%) 51/76(67%) 134/145(92%) 332/375 89% 9a. 29/31 8/13 26/29 b. 30/31 15/22 26/29 c. 30/31 8/13 26/29 Comp. Total 89/93(96%) 31/48(65%) 78/87(90%) 198/228 87% Analysis Two potential instrumentation issues are apparent. First, the Student Self-Evaluations are higher than either the Course Instructor or Field Evaluation on most measures, though not all. The inconsistent nature of the Field Evaluations, particularly the non-alignment with the Course Instructor measure for the satisfied Competencies, as compared with the not satisfied Competencies, warrants further examination, particularly since the final course grades given to students in their Field courses more closely matches the Instructor Evaluations than the Field Evaluation measures used in the analysis. The 3 Competencies that failed to meet the Benchmark are closely matched to specific courses, with all Practice Behaviors for the respective Competencies being measured in a particular course. Competency 4 - Engage in Practice-Informed Research and Research-Informed Practice, is primarily taught and measured in SOWK 2910 Research Methods in Social Work. Similarly, Competency 3 - Advance Social Justice and Human Rights, is taught in SOWK 2201 Social Problems and Solutions, and Competency 5 - Engage in Policy Practice, is addressed in SOWK 2301 Social Policy Analysis. This evaluation cycle was the first time that SOWK 2910 has been taught, as this requirement had previously been met outside of the Social Work curriculum. During this cycle, all 3 courses were at the 2000 level and thus were not exclusive to Social Work majors, as Social Work minors and other students with other majors were able to enroll. Actions Taken/Planned The elevated Student Self-Evaluation issue will be monitored in future evaluation cycles, and if consistent, we will consider differential weighting of the measures. To address the Field Evaluation inconsistencies, the Social Work program faculty will undertake a training session for Field Instructors to be certain that this framework is understood by all Field Instructors. To address the issue of non-social Work majors being included in evaluation of the curriculum, the three core Social Work courses that were previously at the 2000 level listed above are now designated as 3000 level courses. This will allow only Social Work majors to take these courses.

8 The Research Competency had previously failed to meet the Benchmark during the prior evaluation, thus precipitating the development of this course. However, this evaluation cycle is the first time that it has been taught. The Practice Behaviors taught and evaluated in this course will be reviewed next year, in between the full two year evaluation cycle, and further action will be taken if indicated. In addition to the course level change described above, another modification will be made to the Policy sequence (SOWK 3201 & SOWK 3301). Policy faculty have proposed, and the Program approved, a switch in the order of the courses. In this new alignment, students will first be given tools to analyze social policies, prior to learning about the policies themselves. This is thought to be a more logical progression. The Practice Behaviors taught and evaluated in these courses will be reviewed next year, in between the full two year evaluation cycle, and further action will be taken if indicated.