Actionable Metrics for Continuous Improvement: Balanced Scorecard Survey Tools

Transcription

1 Actionable Metrics for Continuous Improvement: Balanced Scorecard Survey Tools Alexis Naiknimbalkar, CSU Chancellor s Office Angela Song, UC San Diego

2 Key topics for today 1. Introductions 2. Quick view of the Balanced Scorecard 3. The UC San Diego Surveys and Analytics Program 4. Overview of the Surveys 5. How the Survey results are used to drive improvements 6. The CSU UC Collaboration Journey begins! 7. Next steps

3 The CSU-UC collaboration Background The collaboration UC San Diego s surveys By using these standard questions, it enables benchmarking and sharing of best practices between Universities and departments

4 First A quick history about UCSD s Balanced Scorecard

5 What is the Balanced Scorecard? History: Developed in 1993 by David P. Norton and Robert Kaplan More than half of major companies in the US, Europe and Asia are using balanced scorecard approaches Harvard Business Review rates the Balanced Scorecard as one of the most influential business ideas of the past 75 years. Purpose: A framework for performance measurement, communication, and management tied to strategy Provides a roadmap on where we should focus energies, priorities and resource based on a holistic view of the organization. Four Perspectives: Financial/Stakeholder Internal Processes Innovation/Growth Customer

6 beyond The Survey Program s Evolution at UC San Diego Business Affairs units begin using the balanced scorecard approach Customer Satisfaction Survey Student Satisfaction Survey Staff Climate Survey Surveys go online Surveys expand to all campus UC San Diego inducted into Balanced Scorecard Hall of Fame Key Performance Indicators identified and benchmarked Mission and vision aligned with strategy UC San Diego VC Business Affairs First study on diversity and staff satisfaction Scalable advanced analytics and reports Customer service, department outreach, and efficient internal processes are an expectation and norm 2012 UC San Diego s 1st strategic plan Surveys are fully scalable Surveys provide actionable data for continuous improvement initiatives and in support of the campus strategic plan UC San Diego All VC Areas Best practices and benchmarking opportunities UC San Diego, UC Irvine (OIT), UC Riverside (BAS), CSU Chancellor s Office (BAF)

7 LETS TALK ABOUT SURVEYS FOR A MOMENT: Why surveys?

8 Surveys are fun! 5 most bizarre survey finds ❶ ❷ ❸ ❹ ❺ From: oddee.com

9 5 most bizarre survey finds ❶ 23% thought MP3 was a Star Wars robot ( Vouchercloud.net)

10 5 most bizarre survey finds ❷ 51% of surveyed Americans think stormy weather "affects" Cloud Computing (-Wakefield Research, 2012)

11 5 most bizarre survey finds ❸ 1 in 4 Americans thinks the Sun goes around the Earth (-NSF, 2012)

12 5 most bizarre survey finds ❹ Average Americans think they're smarter than the Average American (-YouGov, 2014)

13 5 most bizarre survey finds ❺ Survey found that most Americans (75%) don t trust Survey Results (-Kantar data investment management poll, 2013)

14 Surveys allow a way to understand people s attitudes, feelings, and behaviors Do I think this is a good place to work? Will I do my best to contribute or will I just coast? How will I talk about it to others inside and outside the university? Do I feel valued? Do I feel I am making a difference? Am I feeling engaged and inspired or bored and just clocking my time? Does X department even care if they are helping or getting the way of my work?

15 Why survey in a university setting? We should know we are helping to support the mission of the university. Did we help a researcher spend more time in his or her lab (i.e., rather than track down policies and fill out confusing forms)? Did we help a faculty member concentrate their efforts toward their student s learning (i.e., rather than hours spent on processing administrative paperwork)? Did we enhance a student s overall experience (i.e., rather than stressing them with red tape, hoops, and waiting in lines)? Did we help an administrator focus on their most mission critical work (i.e., rather than getting bounced around from desk to desk to get an issue resolved)? How would a leader know if he/she is meeting these needs?

16 ITS ALL ABOUT THE DATA

17 Psychometrics getting good data Not METRICS

18 Psychometrics getting good data from people (because we cant read their minds!) Psychometrics is a field of study concerned with the theory and technique of psychological measurement. Construct and validate assessments instruments (questionnaires, tests, personality tests) Statistical measurement theory Psychometric research involves two major tasks: 1) construct instruments that are valid 2) develop procedures to measure

19 Methodology Design Customer Satisfaction Survey Standard 8 rating questions for all services with up to 5 customized questions per service area 1. Overall satisfaction 2. Understands my needs 3. Accessible 4. Responsive 5. Resolves issues 6. Knowledgeable/professional/courteous, etc. 7. Effective use of Blink (info sharing website) 8. Moving in positive direction Stop, save, and finish later Confidential responses All staff and faculty invited Message: Help us help you fulfill the mission of the university!

20 Methodology Design Survey Tested for internal reliability, conducted Factor Analysis 53 questions measure 4 dimensions: 1. UC San Diego overall 2. Department effectiveness (diversity, mission) 3. Supervisor effectiveness 4. Employee effectiveness Equity, diversity, and inclusion questions for comparisons Message: Are you a satisfied UCSD employee? Would you recommend others to work here? Regressions analysis to predict what drives this outcome Yes, it really is Anonymous Participation rate: 100% for some areas, 56% overall

21 Annual Campus Survey Overview Faculty and Staff Customer Satisfaction 21 years of data Departments voluntarily opt-in to be rated 80 service units/programs currently evaluated 3,773 Verbatim comments and suggestions for improvement One burning question 36% responded with varying participation per rated unit Student Customer Satisfaction 20 years of data Departments voluntarily opt-in to be rated 44 student service units/programs currently evaluated 20,351 Verbatim Comments One burning question 4884 undergraduate and graduate students participated (ie employee engagement) 18 years of data This is a campus wide anonymous survey for staff invited by the Chancellor Eight VC areas units participate 824 Verbatim Comments Who made a difference this year? 56% responded campus wide. Participation within departments reach up to 100%

22 Congratulations! Here are your survey results. Good luck!

23 Analytics? Sure! Here you go.

24 Not cool.

25 ITS ALL ABOUT making meaning of THE DATA

26 Customer Satisfaction Survey Reports: Who, What, Where, When, Why Trend analysis: When over time Heat maps for easy identification of Where should we dig deeper Correlational analysis to identify drivers of satisfaction and start the conversation of Why are the scores varying What are the basic descriptive statistics Drill downs to know which departments to focus your outreach or study best practices: Who needs attention

27 Customer Satisfaction Survey heat map quickly identify strengths and opportunities

28 A picture can say a thousand words Impact analysis: These are the items where people are saying, I am not as happy about these things and they are also very important drivers of my satisfaction here Descriptive statistics Arrows indicate positive or negative movement and statistical analysis inform you of significant trends

29 Interpreting the impact analysis report I am happy about these things BUT they are NOT things that are as important to me in impacting my level of satisfaction here This quadrant shows the survey items that are rated high (above the mean score of 3.73) and also found to be highly related to satisfaction (above the mean correlation of.42). I am happy about these things AND they are important to me in impacting my satisfaction here. Keep it up! But no need to spend too much effort here. Keep it up! Don t change anything All is well. I am not as happy about these things BUT they are NOT things that are as important to me in impacting my level of satisfaction here I am not as happy about these things AND they are also things that are important to me in impacting my level of satisfaction here Keep an eye on these if they move into Primary Opps Focus on these things to address! Correlation means relationship. The closer to 1.0, the stronger the relationship between satisfaction and the survey item

30 Now what?

31 ITS ALL ABOUT what you do with THE DATA

32 The Survey Accountability Loop REDEFINE SURVEY QUESTIONS Step I Deploy survey and obtain feedback (ratings and comments) (Oct) IDENTIFICATION OF NEEDS AND PROGRAM PRIORITIES EVALUATION AND RECOGNITION FOR ACTIONS TAKEN Step VI Realign with strategic goals and/or course correct (Jul Sept) Step II Identify themes and opportunities (Dec) Step V Did changes result in goal attainment? Communicate impact and share results with Senior Leadership (June) Step III Opportunities to Action: Develop and implement action plans and set goals (share with team and Sr. Leadership) (Jan Feb) PROJECT AND PROGRAM IMPLEMENTATION Step IV Follow up and assess performance and impact of action plans (Feb-May) RESPONSE PLANNING, GOAL SETTING

33 Case Study: Health Sciences Development: We Heard You Risk Scenario: Based on FY13 survey results, Health Sciences Development identified 12 opportunities for improvement Data-Driven Action Taken: Sr. Leadership Team developed and executed comprehensive strategies to address each opportunity under the We Heard You campaign Outcome: FY14 survey results revealed significant score improvements AND the largest fundraising year for Health Sciences in the history of UC San Diego!

34 Case Study: We Heard You Areas of Opportunity

35 Case Study: We Heard You Data-Driven Action The Senior Leadership Team worked together to develop comprehensive strategies that addressed each of the identified areas. These strategies were branded under the theme We Heard You. Over the past year, the Senior Leadership Team executed each strategy while periodically reminding the entire department that We Heard You.

36 Case Study: We Heard You - Outcomes

37 Case Study: We Heard You - Outcomes

38 Case Study: We Heard You - Outcomes

39 Case Study: We Heard You - Outcomes Quote from the Sr. Executive Director of Health Sciences Development: While we know our work is not complete, the Staff@Work survey has provided a roadmap to help guide our entire team toward improved results, least of which is accomplishing the largest fundraising year for Health Sciences in the history of UC San Diego.

40 ITS ALL ABOUT using THE DATA to make a positive impact

41 A few examples of actions taken as a result of staff and customer survey data Creation of a Professional Development and Training Program in Business and Financial Services which has resulted in career advancement and salary increases of an average or 21% for participants, 8 graduate level degrees and 20 professional certifications Dining enhanced their menu choices to include healthier and vegan options Housing improved lounge and shared living spaces in response to student feedback Facilities Management instituted a client response system to more quickly address customer requests Transportation offered specific commuting alternatives per the feedback received Campus Shuttle brought back a shuttle route after hearing the feedback from customers Equipment Management created a new inventory process to alleviate the burden on departments and resulted in successful inventory of approximately 80 campus buildings and 6,000 pieces of equipment with minimal intrusion into research or operational processes Procurements created a Department Outreach program to address the specific needs of targeted customers Career Services Center updated their Port Triton system to make their search feature more user friendly for students seeking quality internships The BFS STRIVE Leadership Development Program was created and implemented to support development of highpotential employees in the department through mentorship. The program increased diversity, spurred career growth, encouraged professional development and fostered mentorships for the participants. It is identified as a University best practice in succession planning For more examples, quotes from leaders, and impact, visit

42 ITS ALL ABOUT THE DATA so you can make meaningful interpretations of what is important to people based on their attitudes and feelings and take actions to make a positive impact the mission of your university

43 Back to our collaboration with CSU

44 The timeline for launching CSU Chancellor s Office Business and Finance Staff@Work Survey Approximate Pre-Launch Time: 8-10 weeks Sponsor meeting (CSU and UCSD) Project approval, internal communications rollout (CSU) Working session and official kickoff (CSU and UCSD) Develop hierarchy, lists, marketing (CSU) Build customized survey questions and application, testing (UCSD) Deploy survey (UCSD) Analyze results (UCSD) UCSD Delivers results to CSU 4 hours 8 weeks 4 hours 4/15/15 6/15/15 6 weeks 4 weeks 2 weeks 4 weeks 8/31 survey open 8/25 Survey announcement 10/2 survey closed 10/15/15 initial results 11/15/15 final reports

45 CSU Chancellor s Office plan Deploy the Customer Satisfaction Survey to customers in the Chancellor s Office and 23 CSU campuses Identify metrics for the other two perspectives of the BSC Financial/Stakeholder Internal Business Processes Now have benchmark capability with UCSD on the same survey questions year over year to collaborate and partner on best practices

46 Appendix: Example screenshots of the Survey and Faculty and Staff Customer Satisfaction Survey

47 Questions? Contacts: Alexis Naiknimbalkar, Angela Song,

48 Some screenshots of the Survey Instrument

49 Welcome Page English or Spanish

50 Your department name pops up if you hover over Know what you are rating & where you are in the survey

51 Anonymous comments Highlight a colleague or leader that s made a difference!

52 Conduct and behavioral questions for VC Equity, Diversity, Inclusion initiatives

53 Choose to opt out from further reminders

54 Thank you gift for participation!

55 Some screenshots of the Faculty and Staff Survey Instrument

56 See slide 4 and 5 for snippets of the FAQ and Dept Descriptions

57 Let us know if you would like this to be a specific person or your general mailbox

58

59

60

61

62

63

64

65 Once you submit your survey, you cannot change responses. You are automatically returned to this page. You can also come back here to print you coupon. You can click out of the survey to our webpage to see recent Actions Taken and other information This is the coupon that everyone receives for a gift item at the Bookstore. Each coupon is barcoded and unique

66 This is the promotional banner that will be posted in three different locations on campus

67 to our VC contacts and departments in preparation for the launch

68 The communication kit we provided departments to help encourage response participation

69

70

71 Note: reminders are only sent to people who have not submitted their survey

72