Detailed Assessment Report 2011-2013 Research & Analytical Services As of: 11/04/2013 02:55 PM CDT (Includes those Action Plans with Budget Amounts marked One-Time, Recurring, No Request.) Mission / Purpose Goals Research & Analytical Services (RAS) professionals lead informed and collaborative planning and decision-making at all levels of the College through research design, data collection and analysis, reports of findings, and recommendations for performance excellence in the accomplishment of the comprehensive mission. RAS is the official reporting office for federal IPEDS and regional SACS data, and also conducts the Institutional Review Board for the College. G 1: Inform STC Leaders of Relevant Findings from Research Provide current and relevant information and recommendations to College leaders based on timely research. G 2: Provide Comprehensive Feedback from Stakeholders Ensure cyclical collection of data from all college stakeholders, thorough analysis of the data, and distribution of findings and recommendations for College decision support. G 3: Prevent Risks to Human Subjects in Research Review all research proposals conducted at STC or with STC employees or students to mitigate risks to human subjects. G 4: Support Continuous SACS Accreditation Provide information and support for any and all SACS Accreditation efforts of the College. G 5: Promote Teamwork in Decision Support for Student Success Promote teamwork to provide timely relevant decision support services for student success. Other Outcomes/Objectives, with Any Associations and, Targets, Findings, and Action Plans O/O 1: Excellence in Research Develop a team of highly skilled researchers and research staff and a system of timely, accurate, comprehensive, and applied educational research that provide strategic information and recommended actions for use with College administrators and decision-makers to promote excellence in student learning. 2008.3.5 Significantly improve student learning and student success benchmarks, including student participation, persistence and graduation/transfer rates. 2008.3.6 Expand initiatives throughout STC to identify and eliminate barriers to course and program completion. 2008.4.1 Continually adapt programs and services to effectively meet changing community and regional needs and expectations. 2008.6.7 Strengthen methods for effective planning, evaluation, and assessment. Finding (2011-2013) - Met STC Departmental Professionalism Survey was administered in May-June of 2013. RAS overall satisfaction rate was 97% (69% Very Satisfied, 28% Satisfied) based on survey findings. M 2: Institutional Performance Improvement Student learning and success will be improved through strategic use of data analyses, reports and research findings. Source of Evidence: Benchmarking of learning outcomes against peers Researchers will demonstrate understanding of and engagement with institutional performance objectives and improvement efforts by identifying and contributing individual and collaborative findings for use in the development of strategic interventions.
Finding (2011-2013) - Met RAS research team continuously works with administration, faculty and staff to assist them measure student success. For example, RAS played a key role in the development of the new strategic plan, by providing student success and institutional performance data to the taskforce and its sub-committees. RAS researchers have also helped compile and integrate both external environment data and internal feedback during the strategic planning process. RAS also periodically presents and discusses accountability data with the college leaders. As performance funding has become the reality in the state of Texas, RAS has monitored student success points data and funding calculations and informed the college community about its implications. M 3: Stakeholder Voices Heard Stakeholders (students, faculty, staff, administrators, community members) voices will be heard, documented, analyzed and summarized for feedback to stakeholders themselves and appropriate decision-makers. Source of Evidence: Administrative measure - other Research designs will incorporate use of all available stakeholder survey data. Finding (2011-2013) - Met RAS continues to listen to all stakeholders' voices in the design and execution of quantitative and qualitative research studies. Examples include Departmental Professionalism Survey, ISP/AS Employee Engagement Survey; Puente pre-post surveys, GOT program survey, Student Textbook Survey, Library Service Quality Focus Groups, Security Study, Dual Enrollment Faculty Focus Groups, among several others. In all these cases RAS researchers received input from different stakeholders throughout the STC community in the form of meetings and presentations. Furthermore, RAS researchers are instilled as permanent members on various committees and taskforces around the college to better hear the voices of stakeholders in a continuous fashion. O/O 2: Pro-Active Research Agenda Collaborate with data management team and other front line units to provide multiple views/forms of business intelligence information which will serve as foundation for pro-active research to support institutional decision-making. 2008 2008-2012 Strategic Plan 2008.5.7 Engage in inter-departmental partnerships and cross-functional teams to eliminate silos and facilitate student success. 2008.6.7 Strengthen methods for effective planning, evaluation, and assessment. Finding (2011-2013) - Met According to the STC Departmental Professionalism Survey, 97% of respondents were satisfied with services received from Research and Analytical Services. Furthermore, RAS continued to listen to internal stakeholders, scan the external environment, do literature reviews, follow key journal publications, and attend conferences to set the pro-active research agenda at the college. The following is a list of high priority areas where statistical analyses have been completed or are ongoing (not a comprehensive list): 1. LibQUAL+ survey 2. SENSE survey 3. CCSSE survey 4. GOT program, student persistence and performance 5. Dual Enrollment 6. Enrollment progress 7. Departmental survey 8. ATD Annual Reflection, student success indicators 9. Student textbook survey 10. Puente program Additionally, the following qualitative research studies have been completed or are ongoing (not a comprehensive list): 1. Library service quality focus groups 2. Security study 3. Childcare study 4. PR student focus groups 5. Progress Meter and Off-Track Notifications student focus groups 6. Dual Enrollment faculty focus groups
M 2: Institutional Performance Improvement Student learning and success will be improved through strategic use of data analyses, reports and research findings. Source of Evidence: Benchmarking of learning outcomes against peers At least one new research-findings-initiated intervention for performance improvement will be documented for each item on the pro-active research agenda. Finding (2011-2013) - Met RAS played a key role in the Texas Completes (formerly Completion by Design - CBD) initiative, from identifying two target student populations, to engaging stakeholders and to designing the ideal pathway to completion. The pathway is designed to eliminate leakage points and increase the number of graduates and timely completions. Currently, RAS research team is playing a support role in integration Texas Completes initiatives with the new Strategic Plan and with initiative implementation plans. M 4: Timely Facilitation of Surveys Surveys will be purposefully scheduled and conducted within scheduled time-lines. Additionally, the results from surveys will be analyzed and findings disseminated within a month of data collection close date. Source of Evidence: Efficiency Survey research schedule will be integrated with pro-active research agenda in order to support the research agenda with timely survey data and findings. Finding (2011-2013) - Met The timing and frequency of large scale surveys are aligned with the data needs of stakeholders for decision making and IE reporting. Examples include, but are not limited to: CCSSE, SENSE, and Departmental Survey. A survey calendar has been created and communicated. Ad hoc survey requests for smaller studies are performed as needs arise and communicated to RAS researchers. Continue monitoring timeliness of survey administration The survey calendar is set to meet the stakeholders' needs. As these data needs can change, the calendar will be updated periodi... O/O 3: Strategic Collaboration and Communication Collaborate with IEA and IT to design and implement an effective business intelligence system for data extraction, integration, presentation and analysis with relevant findings at the finger-tips of decision-makers and stakeholders in support of timely decisions and strategizing for the promotion of student learning and success. 2008.3.5 Significantly improve student learning and student success benchmarks, including student participation, persistence and graduation/transfer rates. 2008.4.1 Continually adapt programs and services to effectively meet changing community and regional needs and expectations. 2008.5.5 Empower employees to make appropriate and timely decisions. Finding (2011-2013) - Met Collaboration and continuous communication utilized at every step of the research process, from defining the initial research purpose and questions to the final report creation. RAS staff attends meetings with stakeholders; make regular presentations on research findings and summary of external reports. When asked if RAS department engages in inter-departmental partnerships, 97% of RAS clients said yes (78% Yes, consistently; 19% yes, sometimes). In response to RAS departmental professionalism which definitely includes strategic collaboration and communication, 100% of clients said that RAS exhibits high levels of professionalism.
M 2: Institutional Performance Improvement Student learning and success will be improved through strategic use of data analyses, reports and research findings. Source of Evidence: Benchmarking of learning outcomes against peers Feedback from clients who request RAS to present and make recommendations regarding specific findings from the data will indicate sufficient understanding of the data to apply it to specific performance improvement strategies Finding (2011-2013) - Met Findings by RAS researchers have been regularly presented at PDC and PAS for discussion and review of institutional performance. Some examples are strategic planning external and internal data, THECB accountability report, Student Success Points, IPEDS annual report, and so on. RAS researchers also present data and findings requested by task forces, such as GOT Program Steering Committee, Dual Enrollment Task Force, and so on. M 3: Stakeholder Voices Heard Stakeholders (students, faculty, staff, administrators, community members) voices will be heard, documented, analyzed and summarized for feedback to stakeholders themselves and appropriate decision-makers. Source of Evidence: Administrative measure - other Decision-maker input at all levels of collaboration will be heard and considered when applying research findings. Finding (2011-2013) - Met RAS staff has continually consulted with various college entities in order to get decision-maker input at all levels. Collaborative meetings have been held in order to hear stakeholder voices and to take these into consideration when applying research findings. Examples of this include, but are not limited to: IS&P/AS, Puente, CSI, GOT, CS4H, New Mathways Project, Texas Completes, Strategic Planning, Technology Enhanced Advising. M 5: SACS Compliance & Interim Reports All reports submitted to SACS via RAS will be accurate and timely. Source of Evidence: Evaluations SACS Reviewers, whether off-site or on-site will be satisfied that STC has met compliance requirements in all OIRE assigned areas of SACS documentation. Finding (2011-2013) - Met SACS compliance is being continuously maintained. No annual SACS reports were due. Ad-hoc reports are provided as requested for ongoing compliance to the Office of Curriculum and Accreditation. Provide data and ad-hoc reports for the college community Established in Cycle: 2009-2011 Coordinate with the Office of Accreditation to produce any data or ad hoc reports as required by SACS. O/O 4: Survey Research Design and Data Collection Design, schedule, and facilitate a comprehensive and continuous survey research data collection to periodically listen to the voices of all stakeholders regarding the College's effectiveness in accomplishing its mission, and also respond to adhoc survey requests for specific targeted populations. 2008.3.6 Expand initiatives throughout STC to identify and eliminate barriers to course and program completion. 2008.4.2 Transform the regional economy through rapid response, workforce training, and resourceful partnerships in order to compete in evolving and emerging global markets. 2008.5.5 Empower employees to make appropriate and timely decisions. 2008.6.7 Strengthen methods for effective planning, evaluation, and assessment.
Finding (2011-2013) - Met According to the STC Departmental Professionalism Survey, 97% of respondents were satisfied with services received from Research and Analytical Services. M 2: Institutional Performance Improvement Student learning and success will be improved through strategic use of data analyses, reports and research findings. Source of Evidence: Benchmarking of learning outcomes against peers Survey research data from each primary survey on the annual survey schedule will be integrated into research designs and analyses to identify and communicate known factors impacting performance improvement. Finding (2011-2013) - Met CCSSE and SENSE survey findings are being used as spring board for in-house follow-up studies (qualitative and quantitative). For example, in the area of support for learners, RAS researchers work with Centers for Learning Excellence (CLE) staff to correlate CLE and Supplemental Instruction (SI) with student performance. Similarly, student-faculty interaction data are being monitored and are available to deans and faculty so they can triangulate with student evaluations of the faculty. M 3: Stakeholder Voices Heard Stakeholders (students, faculty, staff, administrators, community members) voices will be heard, documented, analyzed and summarized for feedback to stakeholders themselves and appropriate decision-makers. Source of Evidence: Administrative measure - other 80%+ of stakeholders will be listened-to each biennium. Finding (2011-2013) - Met RAS conducts large scale surveys that are relevant to all stakeholders, such as CCSSE, SENSE, Departmental Survey, and Strategic Plan Feedback Survey. In addition, RAS listens to each stakeholder group's individual needs and creates surveys to serve their unique needs. Examples include Puente survey, GOT program survey, Progress Meter survey, JagCard survey, CLE survey, Website survey, etc. M 4: Timely Facilitation of Surveys Surveys will be purposefully scheduled and conducted within scheduled time-lines. Additionally, the results from surveys will be analyzed and findings disseminated within a month of data collection close date. Source of Evidence: Efficiency 90% of surveys will be conducted on schedule. Finding (2011-2013) - Met All surveys on the calendar and other surveys requested ad hoc have been completed on-time. The survey calendar is updated and announced on the Factbook. Currently, we are revising Graduate Follow-Up and Employer Satisfaction surveys. Continue monitoring timeliness of survey administration The survey calendar is set to meet the stakeholders' needs. As these data needs can change, the calendar will be updated periodi... O/O 5: Protection of Human Subjects from Research Risks Collaboration with principle investigators on research projects to provide for the protection of human subjects through professional communication and training, as well as thorough review, revision and approval of their proposed research to be conducted at STC or involving the College in any way.
2008.1.3 Foster student engagement through an understanding of diverse student needs and preferences. 2008.2.4 Build a greater understanding and appreciation of the community college philosophy of open access throughout the region. 2008.3.6 Expand initiatives throughout STC to identify and eliminate barriers to course and program completion. Finding (2011-2013) - Met As part of its efforts to keep clients satisfied with RAS Service of reviewing and approving IRB proposals, IRB applications/proposals were approved in a timely manner. A list of approved proposals is available from the department. According to the STC Departmental Professionalism Survey, 97% of respondents were satisfied with services received from Research and Analytical Services. M 9: Review IRB Proposals All IRB proposals submitted to RAS will be reviewed and evaluated in a timely manner. Source of Evidence: Administrative measure - other Zero complaints from principal researchers regarding the timely review of their submitted proposals. Finding (2011-2013) - Met All IRB proposals received have been processed in a timely manner. As a result, no complaints from principal researchers regarding the timely review of their submitted proposals have been received. Continue tracking IRB requests Continue tracking all IRB requests: Requestor, project title, time received, time completed, status. O/O 6: SACS Compliance & Interim Reports SACS Compliance Support and Interim Reporting. 2008.4.1 Continually adapt programs and services to effectively meet changing community and regional needs and expectations. 2008.5.7 Engage in inter-departmental partnerships and cross-functional teams to eliminate silos and facilitate student success. 2008.6.2 Validate excellence through accreditations, external awards, recognitions, and publications. Finding (2011-2013) - Met Although no SACS related compliance or interim reports were due, there is always continuous environmental scanning to stay abreast of the latest on topics of interest to accreditation, student learning, and student success. M 4: Timely Facilitation of Surveys
Surveys will be purposefully scheduled and conducted within scheduled time-lines. Additionally, the results from surveys will be analyzed and findings disseminated within a month of data collection close date. Source of Evidence: Efficiency Timeliness of survey research schedule will satisfy SACS Compliance requirements. Finding (2011-2013) - Met SACS compliance is being continuously maintained. No annual SACS reports were due. Ad-hoc reports are provided as requested for ongoing compliance to the Office of Curriculum and Accreditation. Continue monitoring timeliness of survey administration The survey calendar is set to meet the stakeholders' needs. As these data needs can change, the calendar will be updated periodi... M 5: SACS Compliance & Interim Reports All reports submitted to SACS via RAS will be accurate and timely. Source of Evidence: Evaluations Zero Recommendations from Off-site or On-site SACS Reviews. Finding (2011-2013) - Met No SACS Reviews were conducted during this reporting cycle. Provide data and ad-hoc reports for the college community Established in Cycle: 2009-2011 Coordinate with the Office of Accreditation to produce any data or ad hoc reports as required by SACS. O/O 7: Promote team communication and mutual recognition by holding weekly staff meetings Promote team communication and mutual recognition by holding weekly staff meetings. 2008.5.1 Set guidelines for institutionally defined expectations of professional, productive, teamoriented behavior. 2008.5.3 Foster a respectful, professional, and secure environment. 2008.5.5 Empower employees to make appropriate and timely decisions. 2008.6.2 Validate excellence through accreditations, external awards, recognitions, and publications. M 6: Employee Engagement Survey ISP Employee Engagement Survey items: Q1. I know what is expected of me at work. Q4. In the last 7 days, I have received recognition or praise for doing good work. Q7. At work, my opinion seems to count. Source of Evidence: Activity volume 85% agreement rate. Finding (2011-2013) - Partially Met Q1. I know what is expected of me at work: 100% agreement rate (6 out of 6) Q4. In the last 7 days, I have received recognition or praise for doing good work: 83% agreement rate (5 out of 6) Q7. At work, my opinion seems to count: 100 agreement rate (6 out of 6) Professional Development and Team Building Weekly staff meetings Selection of appropriate professional development opportunities annually for each team member Details of Action Plans for This Cycle (by Established cycle, then alpha) Cross-functional teams with data owners Form cross-functional work teams to collaborate with data owners to refine data and reporting needs. Established in Cycle: 2009-2011 Implementation Status: Planned Responsible Person/Group: RAS Director and Staff Provide data and ad-hoc reports for the college community Coordinate with the Office of Accreditation to produce any data or ad hoc reports as required by SACS. Established in Cycle: 2009-2011 Implementation Status: Planned Measure: SACS Compliance & Interim Reports Outcome/Objective: SACS Compliance & Interim
Reports Strategic Collaboration and Communication entire college community. This plan will be aligned with the institutional community plan that will be developed as part of the ATD plan feedback and recommendation. The RAS communication plan will ensure timely delivery of research briefs, study reports, and presentations. Measure: Institutional Performance Improvement Outcome/Objective: Excellence in Research Pro-Active Research Agenda Strategic Collaboration and Communication Survey Research Design and Data Collection Measure: Stakeholder Voices Heard Outcome/Objective: Excellence in Research Strategic Collaboration and Communication Survey Research Design and Data Collection Continue monitoring timeliness of survey administration The survey calendar is set to meet the stakeholders' needs. As these data needs can change, the calendar will be updated periodically to determine which surveys to keep, which to eliminate, and which to add. For example, currently, the Graduate Follow-Up and Employer Satisfaction surveys are being revised. Measure: Timely Facilitation of Surveys Outcome/Objective: Pro-Active Research Agenda SACS Compliance & Interim Reports Survey Research Design and Data Collection Continue tracking IRB requests Continue tracking all IRB requests: Requestor, project title, time received, time completed, status. Measure: Review IRB Proposals Outcome/Objective: Protection of Human Subjects from Research Risks having representation in steering committees and taskforces, RAS will strive for 90% or higher satisfaction rate. Measure: Client Satisfaction with RAS Services Outcome/Objective: Excellence in Research Pro-Active Research Agenda Protection of Human Subjects from Research Risks SACS Compliance & Interim Reports Strategic Collaboration and Communication Survey Research Design and Data Collection Professional Development and Team Building Weekly staff meetings Selection of appropriate professional development opportunities annually for each team member Measure: Employee Engagement Survey Outcome/Objective: Promote team communication and mutual recognition by holding weekly staff meetings