Systems Appraisal Feedback Report. Completed in Response to a Systems Portfolio Submitted by

Transcription

1 Systems Appraisal Feedback Report Completed in Response to a Systems Portfolio Submitted by 1116 LINCOLN LAND COMMUNITY COLLEGE August 28, 2017 TEAM CHAIR LINDA WELLBORN TEAM MEMBERS KATHLEEN DEDEYN BLAKE FAULKNER PAMELA HUMPHREY TONYA NELSON Higher Learning Commission

2 1 Contents I. Reflective Overview... 2 II. Strategic Challenges Analysis... 5 III. AQIP Category Feedback... 6 IV. Criteria for Accreditation Evidence Screening... 8 V. Quality of the Systems Portfolio VI. Using the Systems Appraisal Feedback Report APPENDIX A: Stages in Systems Maturity APPENDIX B: AQIP Category Feedback APPENDIX C: Criteria for Accreditation & Core Component Evidence Screening... 69

3 2 I. Reflective Overview Upon completing its review of the Institutional Overview and Category Introductions included in the Systems Portfolio, the Systems Appraisal team formulated its understanding of the institution, the institution s mission, and the constituents served. This understanding is conveyed in the following Consensus Reflective Statement. Additional team insights are also summarized here in relation to the six AQIP Pathway categories. Reflective Overview Statement Lincoln Land Community College (LLCC) is a public, nonprofit, comprehensive community college with an open admissions policy. LLCC s main campus is located in Springfield, Illinois but serves 4,115 square miles and includes all or a portion of 15 counties in central Illinois. Additional instructional locations are in Beardstown, Jacksonville, Litchfield/Hillsboro, and Taylorville communities as well as three additional locations in Springfield. LLCC offers four degrees designed for transfer Associate of Arts, Associate of Science, Associate of Engineering Science, and Associate of Fine Arts, and a nontransferable Associate in General Education. Students are able to earn AAS degrees in 31 careertechnical programs and approximately 62 certificate programs. The College is approved to offer programs via distance education; some associate and certificate programs are delivered through a combination of online and hybrid offerings. LLCC s mission is To provide district residents with quality educational programs and services that are accessible, affordable, and responsive to individual and community needs. Fall 2016 total headcount was 5,744 (41.8% full-time; 58.2% part-time). 63.1% of students were under 24 years of age and 36.7% were over the age of % of the student body is Caucasian; 17.2% are minority. 58% of the student body is female; 42% male. As of fiscal year 2017, the College employed 613 full- and part-time faculty and staff (122 full-time faculty delivered 54% of student credit hours; 194 part-time faculty delivered 46%). LLCC has two collective bargaining units: the Faculty Association, representing both full-time and adjunct faculty, and the Facilities Services Union. State budget and pension reform shortfalls have impacted stability for all Illinois higher education institutions. Even though only 8.9% of the College operating revenues come from state appropriations, absence of these appropriations over an extended period, coupled with enrollment decline (17% from spring 2013 to spring 2016) could be a challenge for LLCC. LLCC s Foundation kicked off a $2.5-million endowment campaign in September The Grow Beyond campaign surpassed the $1 million milestone in October By May 2017, the endowment campaign had exceeded the $3 million milestone and stood at $3,123,985.

4 3 The 2013 Systems Appraisal identified LLCC s strategic issues as: aggregating SLO measures at the institutional level; stronger connections to businesses that hire LLCC graduates and the program SLOs; alignment of co-curricular program with SLOs; analysis of needs of the various stakeholder groups; and methods for measuring cultural change from CQI efforts. This is LLCC s second Systems Portfolio. Category Summary Statements 1. Helping Students Learn: Lincoln Land is still in the process of addressing strategic issues identified in the 2013 Systems Appraisal related to teaching and learning, including student learning and program outcomes assessments. Following LLCC s participation in the HLC Assessment Academy in 2007, the College developed general education student learning outcome assessment processes. Assessment processes for determining and assessing program learning outcomes (PLO s) have begun with ongoing implementation to occur in phases over the next three to five years. LLCC indicates that co-curricular assessment is still inconsistent, but systemic assessments in co-curricular areas are planned to become more prominent in the academic year. General education assessment is recognized as systematic but changes in the general education assessment process have led to a year void of results. 2. Meeting Student & Other Key Stakeholder Needs: Lincoln Land uses Acuplacer to identify appropriate placement for incoming students and the Community College Survey of Student Engagement (CCSSE), the Noel Levitz Student Satisfaction Inventory (SSI) and locally developed survey instruments to identify student needs. The College uses IPEDS data to benchmark its targets for completion, persistence, and retention. LLCC has many collaborations and partnerships within the community, industry, and state government with employers, clinical supervisors and internship providers and has strong ties to K-12 schools as well as state universities and local colleges. An advisory committee provides feedback regarding stakeholder needs and satisfaction. Recent improvements in Meeting Students and Stakeholders Needs include participation in the HLC Retention and Persistence Academy, completing a 2015 Action Project, Dynamic Stakeholder Inventory Process, to better identify external stakeholders and contractual partners needs, implementing the Maxient system to enhance its student complaint management/monitoring processes. developing a first-year experience course, starting a TRIO Student Support Services (SSS) program, and launching an Open Door mentoring program. 2. Valuing Employees: All processes related to recruiting, hiring, and evaluating employees are

5 4 clearly established in Board Policy and are consistently applied by members of the College. LLCC has long established processes for utilizing employee survey data to inform decision making and continuous improvement and has recently switched survey tools to the PACE instrument to enable benchmarking opportunities with other institutions. While Human Resources coordinates training and development for staff, the Center for Academic Innovation and e-learning coordinates professional development activities for faculty. The College recognizes an opportunity to improve professional development, and a Strategy Forum initiative to design a training program to assist employees transitioning to supervisory roles has been proposed for FY2018. Recent improvements include the addition of employee wellness and recognition programs. 3. Planning and Leading: Lincoln Land s mission, vision, values and goals are reviewed on a 5-year cycle with any changes requiring Board approval. Planning and processes for determining institutional priorities are linked to the College s six goals; processes for linking departmental missions to the institutional mission are in place. The PACE survey is utilized to measure the effectiveness of LLCC in ensuring the mission is articulated and understood broadly across the institution. The Action Research Project to implement a new strategic planning process completed in 2013 continues to be utilized by LLCC. Forward looking improvements identified include a review of the effectiveness of the current shared governance structure and communication flow. 5. Knowledge Management & Resource Stewardship: Lincoln Land has invested significantly in enhancing its data analysis capabilities. The College uses Ellucian Colleague ERP/SIS and ZogoTech data warehouse to provide robust, accurate, and user-friendly information and data. The systems enable departments and teams to access reports and trends to inform daily operations and strategic planning through the Institutional Research (IR) website. The Illinois Community College Board system enables LLCC to benchmark its performance within the state. Membership in the National Community College Benchmarking Project (NCCBP) enables the College to benchmark its performance nationally. Planned improvements include developing and implementing an online data request system to track, prioritize and collect user satisfaction data and an Action Project to develop a data framework. 6. Quality Overview: The College s general commitment to quality led to its application to AQIP in

6 LLCC s commitment to its six Strategic Goals, which flow from the institution mission and vision, helps assure alignment among overall planning and improvement processes, Systems Portfolios, Action Projects, Quality Check-Ups, and Strategy Forums. The president and cabinet select and oversee improvement projects that are consistent and align with the strategic goals. There appears to be opportunity for improvement in further developing results in assessing the culture of quality at LLCC and in developing assessment plans for faculty and staff. II. Strategic Challenges Analysis In reviewing the entire Systems Portfolio, the Systems Appraisal team was able to discern what may be several overarching strategic challenges or potential issues that could affect the institution s ability to succeed in reaching its mission, planning, and overall quality improvement goals. These judgments are based exclusively on information available in the Systems Portfolio and thus may be limited. Each item should be revisited in subsequent AQIP Pathway reviews, such as during the comprehensive evaluation in Year 8. Strategic Challenge: As identified in the 2013 Systems Appraisal Feedback Report, collecting and measuring direct measures remains a strategic challenge. The institution provides minimal evidence of assessment of student learning outcomes especially at the program level. This is a strategic and clearly an accreditation issue related to Core Component 4B. Strategic Challenge: A strategic concern is whether LLCC has demonstrated commitment to fulfilling AQIP pathway responsibilities in Category 2. A review of Category 2 indicates it is not evident that the College has made strong efforts to address strategic issues of concern noted in the 2013 Systems Portfolio Appraisal Feedback Report. Specifically, the 2013 review team recommended the College should improve prioritization of partnership effectiveness and collect and analyze more direct measures in most of the categories. Strategic Challenge: LLCC appears to have managed its way through the financial turmoil and uncertainty in state funding over the past several years. However, it does appear that there may be some early signs of decline in professional development satisfaction among staff and administrators, as well as in professional development funds available over the past several years, due to funding shortfalls, which may be having negative impact on people. Financial restraints could become a strategic issue if state funding continues on the course it has been on over the past several years.

7 6 Strategic Challenge: As noted throughout the Category review, a lack of internal targets identified for measures related to Category 3 may inhibit fulfilment of the Criteria for Accreditation and progress on LLCC s quality journey. Internal and external targets were missing from almost every category. Strategic Challenge: There is no mention of how dual-credit teachers are vetted, and no indication the College maintains records to document employees are current in their academic disciplines as required by Core Component 3.C.4. III. AQIP Category Feedback As the Systems Appraisal team reviewed the Systems Portfolio, it determined the stages of maturity of the institution s processes and results. These stages range from Reacting to Integrated and are described in Appendix A. Through use of the maturity stages and its analysis of the institution s reported improvements, the team offers below summary feedback for each AQIP Pathway category. This section identifies areas for further improvement and also possible improvement strategies. In addition to the summary information presented here, Appendix B conveys the team s specific feedback for all Process, Results, and Improvement items included in the institution s Systems Portfolio. The summary feedback below, and the detailed feedback offered in Appendix B, is based only upon evidence conveyed in the Systems Portfolio. It is possible that the institution has additional information on specific processes, results and improvements that was not included in the Systems Portfolio. In such instances, the institution should plan to provide this evidence in a future AQIP Pathway review process. Category 1: Helping Students Learn LLCC appears overall to be systematic to aligned in its processes related to helping students learn, however results are reacting to systematic. LLCC has developed learning outcomes at the course, the program, and the institution levels and has begun collecting measurement results. The processes are relatively new and being revised as necessary. Program outcomes and measures are less developed. The program review process appears to be thorough, however the newer program quality and viability rubric appears to need further development. As LLCC continues developing its program review and assessment processes, the institution would benefit from identifying targets for achievement as well as external benchmarks. LLCC might engage in more interpretation of data and assessment results. Furthermore, complementing the strong work that it has done in establishing co-curricular goals and objectives within its student services division with a fully implemented assessment program would be beneficial to LLCC. The College has the opportunity to develop measurement processes for ethical academic behavior.

8 7 Category 2: Meeting Student & Other Key Stakeholder Needs While many of the processes for Category 2 are aligned to systematic, many of the results are still in the reacting to systematic stage, with measures related to students the most mature. LLCC has identified the majority of areas in need of improvement and has initiatives in place to advance these areas. Initiatives to improve the measurement of effectiveness of its efforts in this area will provide LLCC the ability to make better data informed decisions that will drive continuous improvement in this area. The Dynamic Stakeholder Inventory Process (DSIP) Action Project was launched in spring The project will establish a regular, periodic review process to ensure continuing effectiveness of the relationships. LLCC joined the Persistence and Completion Academy in 2017 and anticipates new Action Projects will be developed. Category 3: Valuing Employees Many of the processes within Valuing People are between aligned and systematic. However, many of the results are reacting to systematic. While processes for hiring, evaluating and supporting employees are systematic, it is not clear that the processes themselves are regularly assessed for effectiveness. The College has the opportunity to develop more direct measures within this category. The ICCB and NCCBP provide valuable comparison information. There is a clear lack of internal targets and benchmarks; establishing such targets and employing additional measures will move the institution forward in maturity. Category 4: Planning and Leading Category 4 is generally strong with processes that are primarily systematic and many moving toward aligned. Results, however, tend to be reactive. The general lack of internal targets makes it difficult to identify opportunities for improvement within the various sub-categories. Throughout the portfolio, LLCC relies heavily on PACE data. Although this data is extremely valuable, LLCC may benefit from identifying some objective, direct measures of its processes. Some of the processes are relatively new; as LLCC works through the cycle and evaluates these processes it has the opportunity to advance further in maturity. Category 5: Knowledge Management & Resource Stewardship While LLCC describes multiple systems and processes for knowledge management and resource planning, minimal data are provided as evidence the College is committed to the AQIP process or using or measuring effectiveness of those processes. The College is encouraged to review the Action Project Directory to learn how other AQIP institutions are measuring effectiveness in this category.

9 8 Category 6: Quality Overview Overall LLCC appears to be operating at systematic level of maturity in its processes to ensure a culture of quality. Results, however appear to currently be at a reacting level which is somewhat concerning considering the length of time LLCC has been in AQIP. The institution is encouraged to continue to develop the quality culture through engagement, information and public celebrations of quality outcomes. LLCC has an opportunity to systematically evaluate the processes to ensure they are effective and current. IV. Criteria for Accreditation Evidence Screening The Systems Appraisal team screened the institution s Systems Portfolio evidence in relation to the Criteria for Accreditation and the Core Components. This step is designed to position the institution for success during its comprehensive evaluation in Year 8. In order to accomplish this task, HLC has established linkages between the Systems Portfolio s Process and Results items and the Criteria s Core Components. Systems Appraisal teams have been trained to conduct a soft review of the Criteria and Core Components for Systems Portfolios completed in the third year of the AQIP Pathway cycle and a more robust review for Systems Portfolios completed in the seventh year. The formal review of the Criteria and Core Components for purposes of reaffirming the institution s accreditation occurs only in the eighth year of the cycle and is completed through the comprehensive evaluation, unless serious problems are identified earlier in the cycle. As part of this Systems Appraisal screening process, teams indicate whether each Core Component is Strong, clear, and well-presented ; Adequate but could be improved ; or Unclear or incomplete. When the Criteria and Core Components are reviewed formally for reaffirmation of accreditation, peer reviewers must determine whether each is Met, Met with concerns, or Not met. Appendix C of this report documents in detail the Appraisal team s best judgment as to the current strength of the institution s evidence for each Core Component and thus for each Criterion. Institutions are encouraged to review Appendix C carefully in order to guide improvement work relative to the Criteria and Core Components. Immediately below, the team provides summary statements that convey broadly its observations regarding the institution s present ability to satisfy each Criterion, as well as any suggestions for improvement. Again, this feedback is based only upon information contained in the institution s Systems Portfolio and thus may be limited. Criterion 1. Mission: The current Mission, vision and values were adopted by the Board of Trustees in A process is in place to review, and revise as appropriate, the mission, vision and goals of the College every five years in light of current needs of the district workforce, program components, and societal needs. The mission

10 9 vision and core values are intentionally integrated into the institutional planning process. Six strategic goals flow from the mission which is to provide district residents with quality educational programs and services that are accessible, affordable and responsive to individual and community needs. Criterion 2. Integrity: Ethical and Responsible Conduct Integrity and ethical standards are reflected in board policy based upon Illinois laws and mandates. Additional policies regarding harassment and discrimination, fraternization, nepotism, academic freedom, sexual harassment, and non-discrimination along with division-specific policies and procedures that embed integrity into everyday operations of the College s financial, academic, personnel and auxiliary functions are found in the institutional Policy Manual. The College presents itself clearly and completely to the public with regard to programs, requirements, faculty, staff, costs, control and accreditation relations through multiple channels, including but not limited to the College website, catalog, Forward magazine, and program-specific publications. Board roles and responsibilities are set forth in the Policy Manual and include (1) studying legal responsibilities; (2) defining and regularly reexamining the College mission; and (3) insisting the President keep all Board members fully informed about vital issues facing the College and Board. There is not enough evidence to show ethical performance in accordance with polices or enforcement and outcomes of any incidences of policy violations. There is not enough evidence regarding resolution of complaints or effectiveness of communication channels. Criterion 3. Teaching and Learning: Quality, Resources, and Support Degree programs are designed to meet specialized accreditation and licensure standards and are approved by the Illinois Community College Board and Illinois Board of Higher Education. Six general education student learning outcomes were developed in 2007 by the Assessment Committee and were reviewed by workgroups in Outcomes include Critical Thinking, Cultural and Global Awareness, Information Fluency, Communication, Quantitative and Scientific Reasoning, and Technology Competency. Course and program outcomes are linked to at least one of the general education student learning outcomes; a College-wide syllabus template ensures the six outcomes are embedded within courses and programs. Although in process, there is a lack of appropriate data analysis and utilization of data throughout the Portfolio which makes it difficult to discern how the College learns from its initiatives and applies the results and CQI methodology to its quality improvement journey.

11 10 Criterion 4. Teaching and Learning: Evaluation and Improvement Academic programs undergo systematic review that prescribes an in depth examination of various quality indicators every five years. An abbreviated annual review is also conducted. LLCC is in critical jeopardy related to Core Component 4B. At this point only 20 programs have clearly stated goals for student learning and effective processes for assessment of student learning and achievement of learning goals; another 26 are emergent, and 34 are foundational. Therefore, only 20% of LLCC programs are assessing program outcomes. The College provides minimal to no evidence it has used information gained from assessment to improve student learning. Criterion 5. Resources, Planning, and Institutional Effectiveness LLCC s budget is sufficient to support operations. Budgeting and planning flow from the College strategic plan which is based on the College mission and values. Shared governance structure ensures employees across the College are engaged in governance; seven governance teams report to the Shared Governance Council. A well-developed strategic planning process, which includes the Board of Trustees, ensures the College allocates resources in alignment with mission and priorities. There is a lack of participation of external stakeholders and students in the planning process. V. Quality of the Systems Portfolio Overall, Lincoln Land Community College presented a thorough Portfolio. Many of the processes described are new with limited results reported. There was very little interpretation or analysis of data and minimal or no explanation of how data led to improvements. Some explanations and data were unclear. Examples of confusing sections are: 1R1 Results for the CCSSE were clearly articulated in tables and provided trending over time. The summary of GEAR results were unclear as the percentage meeting standards provided ranges but no explanation of what the data meant. It may be beneficial to LLCC to provide stronger descriptors around results for easier interpretation by end users. 1R3 It appears there are processes in place that utilize the Program Viability rubric results to inform insights and actions through the College s Program Improvement matrix. Results of the program viability review are presented, but it is not clear what the labels mean (e.g., critical, warning, etc.) and what action will be taken for those programs, with the exception of those with a rating of warning. 3R1 The summary of results provide trend lines related to employee retention over several years. From information provided, it is difficult to ascertain what data are gathered and whether other processes are in place; it is unclear how the results offered indicate continuous quality improvement.

12 11 It is difficult to determine if a lower % is good or bad in the NCCBP benchmarking table (3.2). Clarification in notes may be beneficial in future for accurate interpretation by external reviewers. 4I4 It was difficult to discern what data from 4R4 led to the initiative to develop fraud risk training for budget manners. Examples of connecting results with process improvements would better demonstrate the full cycle of continuous improvement expected of institutions in the AQIP Pathway. 5R2 LLCC provides results on multiple measures related to the effectiveness of its fiscal resource management. However there are no measures provided in relation to the effectiveness of its processes for ensuring appropriate physical and technological resources. As LLCC has been under financial constraints due to uncertainty in state funding this has been essential to ensure ongoing fulfillment of its mission. However, providing a broader and more well-rounded set of results in relation to its physical and technology resources would help the institution ensure that these critical resource components are being addressed effectively. VI. Using the Systems Appraisal Feedback Report The Systems Appraisal process is intended to foster action for institutional improvement. Although decisions about specific next steps rest with the institution, HLC expects every institution to use its Feedback Report to stimulate improvement and to inform future processes. If this Appraisal is being completed in the institution s third year in the AQIP Pathway cycle, the results may inform future Action Projects and also provide the focus for the institution s next Strategy Forum. In rare cases, the Appraisal completed in the third year may suggest either to the institution itself or to the Commission the need for a mid-cycle (fourth year) Comprehensive Quality Review. If this Appraisal is being completed in the institution s seventh year in the cycle, again the results may inform future Action Projects and Strategy Forums, but more immediately they should inform institutional preparation for its comprehensive evaluation in the eighth year of the cycle when the institution s continuing accredited status will be determined along with future Pathway eligibility. Institutions are encouraged to contact their staff liaison with questions.

13 12 APPENDIX A Stages in Systems Maturity: Processes Reacting Systematic Aligned Integrated The institution focuses on activities and initiatives that respond to immediate needs or problems rather than anticipating future requirements, capacities, or changes. Goals are implicit and poorly defined. Informal procedures and habits account for all but the most formal aspects of institutional operations. The institution is beginning to operate via generally understood, repeatable, and often documented processes and is prone to make the goal of most activities explicit, measurable, and subject to improvement. Institutional silos are eroding and signs of coordination and the implementation of effective practices across units are evident. Institutional goals are generally understood. The institution operates according to processes that are explicit, repeatable and periodically evaluated for improvement. Processes address key goals and strategies, and lessons learned are shared among institutional units. Coordination and communication among units is emphasized so stakeholders relate what they do to institutional goals and strategies. Operations are characterized by explicit, predictable processes that are repeatable and regularly evaluated for optimum effectiveness. Efficiencies across units are achieved through analysis, transparency, innovation, and sharing. Processes and measures track progress on key strategic and operational goals. Outsiders request permission to visit and study why the institution is so successful. Stages in Systems Maturity: Results Reacting Systematic Aligned Integrated Activities, initiatives, and operational processes may not generate data or the data is not collected, aggregated, or analyzed. Institutional goals lack measures, metrics, and/or benchmarks for evaluating progress. The monitoring of quality of operational practices and procedures may be based on assumptions about quality. Data collected may not be segmented or distributed effectively to inform decision-making. Data and information are collected and archived for use, available to evaluate progress, and are analyzed at various levels. The results are shared and begin to erode institutional silos and foster improvement initiatives across institutional units. The tracking of performance on institutional goals has begun in a manner that yields trend data and lends itself to comparative measures in some areas. Measures, metrics and benchmarks are understood and used by all relevant stakeholders. Good performance levels are reported with beneficial trends sustained over time in many areas of importance. Results are segmented and distributed to all responsible institutional units in a manner that supports effective decision-making, planning and collaboration on improvement initiatives. Measures and metrics are designed to enable the aggregation and analysis of results at an institutional level. Data and information are analyzed and used to optimize operations on an ongoing basis. Performance levels are monitored using appropriate benchmarks. Trend data has been accrued and analyzed for most areas of performance. Results are shared, aggregated, segmented and analyzed in a manner that supports transparency, efficiency, collaboration and progress on organizational goals. Measures and metrics for strategic and operational goals yield results that are used in decision-making and resource allocations.

14 13 APPENDIX B AQIP Category Feedback Category 1: Helping Students Learn Category 1 focuses on the design, deployment and effectiveness of teaching-learning processes (and the processes required to support them) that underlie the institution s credit and non-credit programs and courses. 1.1: Common Learning Outcomes Common Learning Outcomes focuses on the knowledge, skills and abilities expected of graduates from all programs. The institution should provide evidence for Core Components 3.B., 3.E. and 4.B. in this section. 1P1 Describe the processes for determining, communicating and ensuring the stated common learning outcomes, and identify who is involved in those processes. This includes, but is not limited to, descriptions of key processes for the following: Process Aligning common outcomes to the mission, educational offerings and degree levels of the institution Determining common outcomes Articulating the purposes, content and level of achievement of the outcomes Comments on Process Maturity and Improvement Systematic: The general education philosophy was developed in 2007 and used to craft the six general education student learning objectives (SLOs). The SLOs are incorporated across the curriculum using a master course syllabus, and are addressed in all transfer degrees as well as career-technical courses. LLCC could improve maturity in this category by creating a plan to regularly evaluate processes used to align SLOs to the mission and to various aspects of the curriculum. Aligned: The SLOs were developed in 2007 and have been reviewed twice, in 2013 and 2016, therefore the process is clearly repeatable. Additionally, every three years the General Education Assessment by Rubrics (GEAR) process is evaluated to determine the relevancy and appropriateness of each outcome. Systematic: The original general education scoring rubrics were published in 2009 for use in assessing student proficiencies of the general education SLOs. In , diverse faculty teams were again assembled to review and revise the College s original rubrics. Each general education rubric defines the expected outcome and identifies the criteria as well as the proficiency level which faculty consider when applying the rubric to student artifacts.

15 14 Incorporating into the curriculum opportunities for all students to achieve the outcomes Ensuring the outcomes remain relevant and aligned with student, workplace and societal needs Designing, aligning and delivering co-curricular activities to support learning Selecting tools, methods and instruments used to assess attainment of common learning outcomes Assessing common learning outcomes Aligned: The state of Illinois has determined the number of credits in General Education each of the six degree types must contain. CurricUNET, an electronic source available to community colleges in Illinois for course and program documentation, is used to ensure program outcomes are linked to at least one general education goal. Systematic: After a full cycle of general education assessment, the SLOs were reviewed by diverse groups of faculty and academic deans. Suggested revisions were based on current standards in the outcome area, the standards of professional organizations, and the needs of students in today s society. This approach helps ensure that outcomes remain relevant and aligned with student, workplace and societal needs. Once this process is repeated and evaluated for improvement it will become more aligned. Systematic: LLCC has begun taking steps to develop outcomes for the Student Services division based on the Council for the Advancement of Standards in Higher Education (CAS) standards. The work began in 2016, and the six learning domains and associated outcome statements were approved in spring It is important to also plan and assess the delivery of these activities. Reacting: The Academic Assessment Steering Committee, in consultation with the Academic Assessment team, is responsible for selecting assessment tools and outcomes to be measured. LLCC does not provide details of how the selection of these tools are determined and what outcomes are to be measured. Systematic: A three-year cycle is used to assess the six outcomes. Courses for the current outcomes are identified through CurricUNET and faculty are recruited to submit student artifacts. These samples are scored by other faculty using the GEAR rubric. Results are returned to the original faculty member, who is asked to provide feedback on changes made based on the results. The process was revised after the first 3-year cycle with future revisions expected. Once the process has been repeated and used by more faculty it should move into a more aligned process. 1R1 What are the results for determining if students possess the knowledge, skills and abilities that are expected at each degree level? The results presented should be for the processes identified in 1P1. All data presented should include the population studied, response rate and sample size. All

16 15 results should also include a brief explanation of how often the data is collected, who is involved in collecting the data and how the results are shared. These results might include: Results Summary results of measures (including tables and figures when possible) Comparison of results with internal targets and external benchmarks Interpretation of results and insights gained Evaluation of Results and Systems Improvement Systematic: Results for the CCSSE were clearly articulated in tables and provided trending over time. The summary of GEAR results were unclear as the percentage meeting standards provided ranges but no explanation of what the data meant. It may be beneficial to LLCC to provide stronger descriptors around results for easier interpretation by end users. Systematic: External benchmarks and internal trending were provided for CCSSE data but internal targets (goals) were not provided and may be beneficial to LLCC to develop. GEAR results did not articulate the expected standard for each SLO. Based on the narrative and data provided it would appear there are standards but it may be beneficial if LLCC could articulate more clearly what these standards are and how they are established. Reacting: While LLCC provides some interpretation of the CCSSE data, it is not clear by whom the results are used or how the results are used to make improvements. The GEAR process revisions do not yet allow for insights in the results. 1I1 Based on 1R1, what process improvements have been implemented or will be implemented in the next one to three years? Evaluation of Improvement Efforts The GEAR assessment process is still developing and LLCC realizes more communication and training with faculty are needed. The College has implemented several initiatives to improve the process of assessing general education and cocurricular goals but has not matured to a level of consistently using data and analysis to drive improvement of student learning. As a second-cycle AQIP institution it is a concern that only six faculty members indicated they would make material or curricular changes based on their results in the GEAR process. That seems a very small percentage of participating faculty. 1.2: Program Learning Outcomes Program Learning Outcomes focuses on the knowledge, skills and abilities graduates from particular programs are expected to possess. The institution should provide evidence for Core Components 3.B., 3.E. and 4.B. in this section.

17 16 1P2 Describe the processes for determining, communicating and ensuring the stated program learning outcomes and identify who is involved in those processes. This includes, but is not limited to, descriptions of key processes for the following: Process Aligning program learning outcomes to the mission, educational offerings and degree levels of the institution Determining program outcomes Articulating the purposes, content and level of achievement of these outcomes Ensuring the outcomes remain relevant and aligned with student, workplace and societal needs Comments on Process Maturity and Improvement Systematic: The Illinois Articulation Initiative assures alignment of transfer outcomes among state institutions. The Curriculum Team reviews new courses and revisions to assure alignment with the College mission and degree levels. Career and technical courses align with outside accrediting requirements and standards. More details on these processes; who is involved, how often, and actions, would help the institution mature in this area. Systematic: Faculty discipline groups, under the guidance of their academic deans and departmental assessment coordinators, propose program SLOs. When developing program outcome statements, faculty consider the course outcomes for existing program courses as well as course and program outcomes at senior institutions in the region. Additional considerations may include requirements of accrediting bodies and other externally imposed regulations or Program Advisory Committee recommendations. It is important that the College designs course outcomes to meet PLOs rather than use course outcomes to define PLOs. Outcomes must proceed through a thorough review process before acceptance. However, due to the rapid rate of change in all industries it may be beneficial to LLCC to consider a program review cycle shorter than the current 5-year cycle and to develop more robust processes for determining specific current workforce needs as well as those emerging that will be required in the future. Systematic: Program learning outcomes are documented in CurricUNET, and all courses in a program are linked to the program outcomes. By policy, approved course learning outcomes are articulated in all general education and career technical education course syllabi. It is unclear how level of achievement of these outcomes is articulated. Systematic: The Academic Effectiveness Manager meets with all Departmental Review teams annually as part of the Academic review process to review program outcomes. Program Advisory Committees also provide input regarding the current skills needed in the workforce and other job-related information. Transfer faculty regularly meet with faculty from other institutions to

18 17 Designing, aligning and delivering co-curricular activities to support learning Selecting tools, methods and instruments used to assess attainment of program learning outcomes Assessing program learning outcomes create articulation agreements, thus ensuring the acceptance of LLCC coursework when a student transfers. However, it may be beneficial for LLCC to consider a shorter program review cycle and more robust structure for determining workplace and societal needs as the pace of change in workforce needs is, and will continue to accelerate. Systematic: Work to design and align co-curricular activities and outcomes to support learning began in The College sets forth a four-step process for ensuring alignment of co-curricular activities with PLOs. Student service outcome statements and program-level outcomes are mapped to divisional outcomes to ensure alignment and support of divisional and program missions. Assessment is in the early stages with only one year of results from departments. These results are publicized on LLCCs Institutional Effectiveness dashboard. It appears that groundwork has been laid to help support the assessment of effectiveness in student services programming which to date appears to be in early stages of maturity. Systematic: Appropriately, content area faculty select tools, methods and instruments used to assess attainment of PLOs including licensure exams, accreditation standards and in-house constructed instruments. A specific process for selecting instruments is not set forth, particularly in areas where there are no accreditation guidelines or licensure requirements. While it is important to continually review assessment tools and methods to obtain trend data, it is important to be consistent in instruments and methods of assessment. Systematic: Program assessment reports are completed each year and examined in depth every five years by program faculty. The reports include program outcomes, assessment process, results obtained, and changes that will be made based on the data. While the process tends to be Aligned, the fact that not all programs currently engage in the assessment process makes this Systematic in maturity. Although there are several strong examples of programs (particularly programmatically accredited programs in healthcare) that have strong program outcome assessment processes in place, there remains a large number of programs (34), that at the time of this Portfolio, had not initiated assessment of program outcomes. This was noted as a concern in LLCC s last Systems Appraisal and does not appear to have been fully addressed.

19 18 1R2 What are the results for determining if students possess the knowledge, skills and abilities that are expected in programs? The results presented should be for the processes identified in 1P2. All data presented should include the population studied, response rate and sample size. All results should also include a brief explanation of how often the data is collected, who is involved in collecting the data and how the results are shared. These results might include: Results Overall levels of deployment of the program assessment processes within the institution (i.e., how many programs are/not assessing program goals) Summary results of measures (including tables and figures when possible) Comparison of results with internal targets and external benchmarks Interpretation of assessment results and insights gained Evaluation of Results and Systems Improvement Reacting: LLCC has 34 programs for which the institution has not initiated assessment of program outcomes and another 26 in the emergent category. Only 11 programs have fully developed assessment processes and 9 have exemplary processes. Therefore, only 25% of programs have usable assessment processes. Having participated in the AQIP pathway since at least 2009, it would be expected that program assessment results within the institution would be mature by now. Systematic: Tables reporting licensure pass rates and trends were presented. One example of a PASR was presented. LLCC is currently searching for an online repository for program results, which may make data analysis and reporting more meaningful. Summary results from the follow-up survey were also reported indicating employment and satisfaction with the program of study; these are more indirect measures. Systematic: Results from CTE surveys for three years with comparisons to other Illinois institutions were presented. Healthcare licensure pass rates with national comparisons were presented. It may be beneficial, particularly in internal comparisons, for LLCC to establish protocols for establishing internal goals/targets for improvement and measures of effectiveness in attaining those goals/targets. Reacting: Although a couple examples were provided in results that indicate some narrow insights, there does not appear to be a clear process for aggregating results, analyzing results, determining if an intervention is required, implementing the intervention, and then remeasuring to determine if it was effective. 1I2 Based on 1R2, what process improvements have been implemented or will be implemented in the next one to three years?

20 19 Evaluation of Improvement Efforts The 2013 Systems Appraisal indicated shortcomings in assessment of program outcomes at that time. A 2015 Action Project launched to design a system that aggregates performance data on student learning outcomes at the program level to provide evidence students meet program goals/outcomes has developed some improvements to the assessment processes. However, overall as reflected in this Portfolio, progress has been slow as currently 34 programs have not initiated assessment plans. It appears that there is a only small core of faculty participating in assessment programs and LLCC must work to increase faculty buy-in and involvement to achieve improvement goals. 1.3: Academic Program Design Academic Program Design focuses on developing and revising programs to meet stakeholders needs. The institution should provide evidence for Core Components 1.C. and 4.A. in this section. 1P3 Describe the processes for ensuring new and current programs meet the needs of the institution and its diverse stakeholders. This includes, but is not limited to, descriptions of key processes for the following: Process Identifying student stakeholder groups and determining their educational needs Identifying other key stakeholder groups and determining their needs Developing and improving responsive programming to meet all stakeholders needs Comments on Process Maturity and Improvement Systematic: The state of Illinois determines key purposes and college stakeholder areas. Input from community and business partnerships, information gathered and analyzed, and targeted outreach inform program needs. Table 1.10 outlines various student groups and methods used to determine need. The institution may be missing an opportunity to serve a more diverse group including categories such as age, veteran status, parents, single parents, unemployed, fulltime workers, ethnic distinctives, ESOL populations, etc. Systematic: While LLCC has identified a number of other key stakeholders and their needs (Table 1.11), it is not clear whether the processes are regularly reviewed to ensure they are still effective; doing so would help move this to an Aligned maturity. Aligned: A new program proposal includes researching potential student demand, market demand for graduates, transferability, staffing potential, facility needs, offerings of other organizations, and optimal curricular design. In addition, CTE programs consult with Program Advisory Committees as part of the proposal process. The new program proposal goes through an extensive review process. New courses are typically developed by faculty within their area of expertise and may have been initiated through the program review

21 20 Selecting the tools, methods and instruments used to assess the currency and effectiveness of academic programs Reviewing the viability of courses and programs and changing or discontinuing when necessary process. As pace of change increases and expectations of relevance by all stakeholders continues to rise, reviewing frequency of processes may help LLCC create further competitive advantage. Reacting: It is unclear from the response who besides academic administrators are involved in selecting tools and methods to assess currency and effectiveness of academic programs and how these tools and methods are determined and selected. Systematic: Review of the viability of courses and programs occurs primarily via an annual program review process and more in-depth five-year Academic Program Review. A program s quality is defined through quality indicators and program assessment activities; viability is defined through a number of criteria outlined in Table Internal leadership committees guide the process. Additionally, faculty may initiate course or program changes. 1R3 What are the results for determining if programs are current and meet the needs of the institution s diverse stakeholders? The results presented should be for the processes identified in 1P3. All data presented should include the population studied, response rate and sample size. All results should also include a brief explanation of how often the data is collected, who is involved in collecting the data and how the results are shared. These results might include: Results Summary results of assessments (including tables and figures when possible) Comparison of results with internal targets and external benchmarks Interpretation of results and insights gained Evaluation of Results and Systems Improvement Systematic: Results indicate that LLCC is consistently active in revising, adding and withdrawing programs and courses; these results are articulated clearly through charts and graphs. The recent addition of a process for assessing program viability, that is categorized and monitored, appears to be an advancement that has the potential to help inform program viability and relevance throughout the 5-year review cycle. It is not clear whether stakeholder groups and needs are assessed. Reacting: Besides the numbers of programs in each viability rating, which is currently under study and revision, no comparison data was presented. The College has the opportunity to articulate internal targets and provide clear expectations for programs to advance in maturity of this area. Systematic: It appears there are processes in place that utilize the Program Viability rubric results to inform insights and actions through the College s Program Improvement matrix. Results of the program viability review are presented, but it is not clear what the labels