Improving Customer Service Using Survey Data: Framework for Developing Initiatives University of Texas at El Paso

Transcription

1 Improving Customer Service Using Survey Data: Framework for Developing Initiatives University of Texas at El Paso SCUP 2010 SOUTHERN REGIONAL CONFERENCE

2 Overview Background about Planning UTEP Business Affairs Using Survey Data Representation & Limitation of Data Sample Exploration of Key Metrics Exploration of Diagnostic Metrics Development of Strategies Strategy Refinement Summary and Lessons Learned

3 Planning at UTEP Realized growth in Key Metrics 12 years of consecutive growth in total enrollment From 14,677 in Fall 1998 to 22,106 in Fall % increase in degrees awarded From 2,131 AY99-00 to 3,968 in AY % increase in research expenditures From 28 million in FY00 to 56 million in FY09 Focused on Outcomes Tracking key metrics Understanding factors that affect key metrics Applying knowledge to action continuous improvement process

4 Planning within Business Affairs All divisions (3) and units (15) develop strategic operating plans yearly mission and vision statements measureable goals and objectives Based on the Balanced Scorecard framework, which focuses on four key perspectives Financial Customer Internal Processes Employee Learning and Growth

5 Customer Service Survey Project Involves all campus units that provide services to students Generates data about usage, satisfaction, and other operational data Administered during Survey Week CIERP supports data collection, analysis, and strategy development

6 Understanding Survey Data

7 Identify questions that need to be answered using data 1. Where is the greatest impact? 2. Are levels of usage as expected? 3. Are levels of satisfaction as expected? 3. Where is it possible to effect a change in satisfaction?

8 Ensure sample is representative All students enrolled at UTEP in Spring 2009 were invited to participate. Completed Surveys: 2,179 Number of students who began survey: 2, % of the UTEP population responded VPBA Survey Respondents 12% 17% 22% 34% 12% 3% 1% Spring Enrollment 14% 15% 20% 32% 15% 2% 1% Freshmen Sophomores Juniors Seniors Master's Doctoral Post-bacc

9 Determine Margin of Error What claims can we make about the data? All students: +/- 2.0% estimated margin of error* Undergraduates: +/- 2.0% estimated m.e. Graduate Students: +/- 5.0% estimated m.e. *95% confidence interval: if survey were administered 100 times, in 95 of those times, the results will fall within +/- 2.0 percentage points of the results shown here.

10 Understand the Limitations of Data Margin of error for subpopulations could be higher Some services may target or be used by a smaller subpopulation margin of error will be larger for those services Small changes (up or down) from year-to-year may occur within the margin of error, due to chance. Survey data will not show the whole picture Use data in context Other Resources and Insights

11 Key Metrics: Usage and Satisfaction

12 Reported Use of Auxiliary Services Bookstore (2179) University Food Services (2179) Parking & Transportation (2179) Miner Metro (2179) Miner Gold Card (2179) Office of Special Events (2179) 93% 75% 57% 39% 25% 18% 7% 25% 44% 61% 75% 82% 0% 20% 40% 60% 80% 100% Yes No

13 Auxiliary Services: Customer Satisfaction with Programs and Services Question: Rate your overall satisfaction with. Office of Special Events (398) Miner Metro (840) Miner Gold Card Office (543) University Food Services (1619) Bookstore (2027) Parking & Transportation (1225) 92% 92% 92% 87% 82% 65% 5% 8% 7% 12% 18% 34% 0% 20% 40% 60% 80% 100% Satisfied/Very Satisfied Dissatisfied/Very Dissatisfied Don't Know

14 Diagnostic Metrics and Selected Program Results

15 Bookstore Overall Satisfaction 82% of respondents indicated being Satisfied or Very Satisfied with the Bookstore. 100% 80% 60% 40% 82.0% 20% 17.6% 0% Satisfied/Very Satisfied Dissatisfied/Very Dissatisfied 0.4% Don't Know

16 Diagnostics: The Detailed Q s about a Service or Program The University Bookstore anticipates my needs. The University Bookstore is responsive to my needs. The University Bookstore staff is courteous. The University Bookstore staff is knowledgeable. Check-out at the University Bookstore is problem-free. The University Bookstore website fulfills my needs. The University Bookstore website is easy to work with. I am satisfied with the variety of products offered at the University Bookstore. Strongly Disagree Disagree Agree Strongly Agree Don t Know Total Count % by Row 3.8% 18.1% 58.8% 16.8% 2.5% 100.0% Count % by Row 3.1% 15.5% 60.0% 18.7% 2.7% 100.0% Count % by Row 2.1% 7.8% 58.0% 31.5% 0.6% 100.0% Count % by Row 2.0% 11.5% 57.5% 26.3% 2.8% 100.0% Count % by Row 6.5% 21.3% 50.9% 18.1% 3.2% 100.0% Count % by Row 4.7% 15.1% 43.9% 15.8% 20.5% 100.0% Count % by Row 3.6% 12.5% 45.0% 17.8% 21.1% 100.0% Count % by Row 5.9% 16.8% 53.6% 22.3% 1.3% 100.0%

17 All Bookstore Comments If you have specific concerns with the Bookstore not covered in this survey, please provide your comments or suggestions. Comment Category % Number Costs 42.1% 188* Customer Service and Policies 18.4% 82 Selection of Merchandise 13.2% 59 Lack of Books 8.7% 39 Buyback Prices 6.3% 28 Information Management 6.0% 27 Thank-you s/ Notes of Satisfaction 2.4% 11 Excitement about New Bookstore 2.0% 9 Requests for More Hours when Open 0.4% 2 * Note that the number of students who mentioned cost is higher than the number who reported being dissatisfied and mentioned cost.

18 A Deeper Look at the Bookstore Data Total number dissatisfied: 355 (18% of 2,027 respondents to the question) Comment Category Percent Number Costs 22.0% 78 Customer Service and Policies 9.3% 33 Selection of Merchandise 5.1% 16 Lack of Books 8.7% 39 Buyback Prices 3.4% 12 Information Management 2.5% 9 Excitement about New Bookstore 0.6% 2 Requests for More Hours when Open 0.3% 1

19 Major findings Insights from survey Focus on improving customer service Focus on improving products Look for new markets Focus on showing value Insights from focus groups Look for ways to reduce costs / increase value (e.g. text book rental program) Offer more products (e.g., event-specific apparel)

20 Strategies Implemented Adjusted products Changed products to meet regional needs of southwest desert region Expanded price-points Improved textbook policies and procedures Increased products offered on Website from 50 to over 200 items Developed new markets Student Alumni Group Alumni Association Athletics Improved Customer Service Modified training Modified operations

21 Results After Implementation 100% Have you had any contact with the UTEP University Bookstore during the last 12 months? 93.2% 92.0% 100% 90% 80% Rate your overall satisfaction with the University Bookstore 75% 70% 19.1% 29.0% 60% 50% 50% 40% 25% 30% 62.9% 57.6% 20% 10% 14.3% 9.9% 0% Yes 0% 3.3% 2.7% Strongly Disagree Disagree Agree Strongly Agree

22 Results After Implementation The University Bookstore is responsive to my needs The University Bookstore staff is knowledgeable 90% 90% 80% 80% 70% 18.7% 24.6% 70% 26.3% 32.7% 60% 60% 50% 50% 40% 40% 30% 60.0% 57.7% 30% 57.5% 52.7% 20% 20% 10% 15.5% 10.7% 10% 11.5% 9.0% 0% 3.1% 3.1% % 2.0% 1.5% Strongly Disagree Disagree Agree Strongly Agree Strongly Disagree Disagree Agree Strongly Agree

23 Results After Implementation Check-out at the University Bookstore is problem-free The University Bookstore website fulfills my needs 90% 70% 80% 60% 70% 60% 18.1% 26.5% 50% 15.8% 20.1% 50% 40% 40% 30% 30% 20% 10% 21.3% 50.9% 51.8% 13.6% 20% 10% 15.1% 43.9% 13.3% 41.7% 0% 6.5% 3.5% % 4.7% 3.8% Strongly Disagree Disagree Agree Strongly Agree Strongly Disagree Disagree Agree Strongly Agree

24 Insights Insights from Survey Improve website increase online sales and products Increase market Shape perception Insights from Secret Shoppers High response time for small group of customers

25 Strategy Refinement Adjust products Continue to expand produces Continue to increased products offered online Increase visibility of website Develop new markets Focus on developing bookstore as a community destination point during evenings and weekends Improve Customer Service Identify periods of high wait-times using secret shoppers

26 Using Data for Continuous Improvement Recognize Data Limitations Explore Key Metrics Explore Diagnostic Metrics Develop Strategy to Affect Change Modify Strategy Based on Outcomes

27 Lessons Learned Focus on adequacy of sample rather than response rate Use data to develop strategies Focus on understanding impact of initiatives and outcomes Generate insights by exploring strategies across many units and institutions

28 Contact Information Roy Mathew, PhD Associate Vice President and Director, Center for Institutional Evaluation, Research and Planning Victor Pacheco, PhD Assistant Vice President for Business Affairs, Business Affairs

29

30 The Balanced Scorecard: The Four Perspectives The Balanced Scorecard views an organization from four key Perspectives. Performance metrics are categorized in the following four key areas: Financial Customer Internal Processes Employee Learning and Growth Financial Perspective To Succeed Financially, how should we appear to our stakeholders? Customer Perspective To Achieve our vision, how should we appear to our customers? Learning& Growth Perspective To achieve our vision, how will we sustain our ability to change and Improve? Internal Process Perspective To Satisfy our Stakeholders and customers, what business processes must we excel at?