Customer Satisfaction Research 2018 Q3 Results October 22, 2018

Transcription

1 Florida Rehabilitation Council Division of Vocational Rehabilitation Florida Department of Education Customer Satisfaction Research 2018 Q3 Results October 22, 2018 Brian Robertson Mark Noyes Jotham Illuminati

2 Consumer Survey Survey Methodology Contents Customer Experience Core Metrics Problems Experienced by VR Customers and Feedback Understanding the Process if You Disagree with VR About Your Services Satisfaction with Outside Vendors Current Employment Status Area Profiles and FY Data 2

3 2018 Q3 Survey Results 3

4 Current Quarter Survey Methodology Data collection took place from July 5, 2018 through October 5, Over the course of data collection, 750 customers completed surveys: 377 customers with open cases 202 customers - closed successfully 171 customers - closed unsuccessfully. 4

5 5 Current Quarter Response Rates APPOR Response Rate (RR3) 23.6% % AAPOR Respondent Cooperation Rate (COOP3) 80.4% AAPOR Respondent Refusal Rate (REF3) 7.1% AAPOR The American Association for Public Opinion Research. Response Rate The response rate is the number of completed interviews divided by the number of eligible reporting customers in the sample. Cooperation Rate This represents the proportion of all cases interviewed out of all eligible customers ever contacted. That is, the percent of identified respondents that ended up completing the interview. Refusal Rate The refusal rate represents the proportion of all cases in which the respondent refused to be interviewed, or broke off an interview, out of all respondents that were contacted and spoken with.

6 Customer Experience Core Metrics 6

7 Florida VR Customer Experience Dashboard Measure 2018 Q Q Q3 Overall Satisfaction and Expectations Experience with Services Provided by VR Experience with VR Staff and Counselors Communications with VR Staff Customer Control and Involvement Outcomes and Meeting Goals Satisfaction with Current Employment Satisfaction with Outside Vendors All things considered, would you tell your friends with disabilities to go to the VR program for help? (% Yes) Have you experienced any problems with VR or the services they have provided to you? (% No)

8 Overall Satisfaction and Expectations: Q Q Q Q Q Q Q Q Q Q3 Q01: Overall, how satisfied are you with the Florida VR s program? Q33: How satisfied would you say you are with the services provided by VR? Q35: Please rate how well VR services met your expectations. 8

9 Experience with Services Provided by VR: Q Q Q Q Q Q Q Q Q Q3 Q09: How satisfied are you with the choice of services? Q11: How satisfied are you with the choice of service providers? 9

10 Experience with VR Staff and Counselors: Q Q Q Q Q Q Q Q Q Q3 Q19: How helpful is the staff of VR in helping you achieve your VR goal? Q29: The VR staff treats me with dignity and respect. 10

11 100 Communications with VR Staff: Q Q Q Q Q Q Q Q Q Q3 Q13: How satisfied are you with the information you were given about the choices you had? Q15: How satisfied are you with how long it took your counselor to answer your questions and address your concerns? Q21: How easy is it for you to contact your VR counselor? 11

12 Customer Control and Involvement: Q Q Q Q Q Q Q Q Q Q3 Q05: How satisfied are you with your involvement in your VR experience? Q07: How satisfied are you with your choice of a vocational goal? 12

13 Outcomes and Meeting Goals: Q Q Q Q Q Q Q Q Q Q3 Q25: The VR services I am receiving will help me become more financially independent. Q27: The VR services I am receiving will help me become more independent, in general. Q31: The VR staff will help me reach my job goal. 13

14 Satisfaction with Current Employment: Q Q Q Q Q Q Q Q Q Q3 Q51: If working, thinking about your current job, how satisfied are you with what you are doing? Would you say you are...? 14

15 Satisfaction with Outside Vendors: Q Q Q Q Q Q Q Q Q Q3 Q46: How satisfied were you with the services provided by OUTSIDE VENDOR? 15

16 100 Recommend VR: All things considered, would you tell your friends with disabilities to go to the VR program for help? (% reporting yes) Q Q Q Q Q Q Q Q Q Q3 16

17 Experienced Problems: 100 Have you experienced any problems with VR or the services they have provided to you? (% reporting no) Q Q Q Q Q Q Q Q Q Q3 17

18 Problems Experienced by VR Customers and Feedback Customers were asked if they experienced problems specifically with VR or the services they received from VR and reasons for dissatisfaction. 18

19 16% of customers experienced problems with VR or the services they provided. 100% Q40: Have you experienced any problems with VR or the services they have provided to you? (% Yes) 80% 60% 40% 20% 0% 18% 17% 17% 20% 18% 18% 16% 9% 2016 Q Q Q Q Q Q Q Q3 19

20 The Most Commonly Mentioned Problems (2018 Q3): Counselor was not helpful or supportive 29% % Counselor did not return calls, s or follow up 22% Had to leave multiple messages before getting a call back 18% Services offered by VR were not effective 17% Listen to customer, understand needs, wants, ability 16% Counselor would not listen, dismissed concerns 13% 20

21 30% of customers reporting problems indicated that VR worked to resolve their problem(s). 100% Q42: If you expereinced problems, did VR work to resolve this problem? (% Yes) 80% 60% 40% 20% 35% 32% 37% 17% 26% 36% 6% 30% 0% 2016 Q Q Q Q Q Q Q Q3 21

22 Customer Feedback Customers were asked to provide additional feedback when they were not positive in their assessment. Feedback was collected throughout the survey in response to 17 questions. Customers could provide more than one response. The following table summarizes the most common responses from 483 customers that provided additional feedback. 22

23 Customer Feedback Better communication needed 42% % Changing counselors, switching too much, causes problems 36% Counselor did not return calls, s or follow up 34% Counselor was not helpful or supportive 21% Get voic , never answers the phone 19% Counselor would not listen, dismissed concerns 16% Didn't receive job search help 14% 23

24 Understanding the process if you disagree with VR about your services Customers were asked if they were aware of the options available if they disagreed with VR s decisions about the services they did or did not receive. 24

25 72% of customers were aware they could contact their VR counselor in cases of disagreement. 100% nq44a: Are you aware of the following option if you disagree with VR's decision about your services - Contacting your VR Counselor? (% Yes) 80% 60% 82% 81% 77% 70% 67% 75% 69% 72% 40% 20% 0% 2016 Q Q Q Q Q Q Q Q3 25

26 60% of customers were aware they could contact the VR Unit Supervisor. 100% nq44b: Are you aware of the following option if you disagree with VR's decision about your services - Contacting the VR Unit Supervisor? (% Yes) 80% 60% 70% 66% 62% 62% 57% 66% 61% 60% 40% 20% 0% 2016 Q Q Q Q Q Q Q Q3 26

27 52% of customers were aware they could contact the VR Ombudsman Office. 100% nq44c: Are you aware of the following option if you disagree with VR's decision about your services - Contacting the VR Ombudsman Office? (% Yes) 80% 60% 40% 53% 56% 49% 45% 45% 49% 43% 52% 20% 0% 2016 Q Q Q Q Q Q Q Q3 27

28 52% of customers were aware they could contact Disability Rights Florida, Client Assistance Program. 100% NQ44e: Are you aware of the following option if you disagree with VR's decision about your services - Contacting Disability Rights Florida, Client Assistance Program? (% Yes) 80% 60% 40% 59% 51% 51% 50% 50% 50% 52% 20% 0% 2017 Q Q Q Q Q Q Q3 28

29 64% of customers were aware they could appeal the decision(s) they disagreed with, if necessary. 100% NQ44d: Are you aware that you may appeal the decision, if necessary if you disagree with VR's decision about your services? (% Yes) 80% 60% 67% 69% 63% 62% 58% 58% 57% 64% 40% 20% 0% 2016 Q Q Q Q Q Q Q Q3 29

30 Satisfaction with Outside Vendors Customer were asked to provide feedback on the services they received from outside vendors. 30

31 64% of customers with an outside vendor were satisfied with the services provided. 100% Q46: How satisfied were you with the services provided by this outside vender? (% Satisfied) 80% 60% 63% 69% 70% 67% 65% 62% 72% 64% 40% 20% 0% 2016 Q Q Q Q Q Q Q Q3 31

32 11% of customers using an outside center reported problems with their vendor. 100% Q48: Did you experience any problems with this outside vendor? (% Yes) 80% 60% 40% 20% 0% 18% 7% 14% 12% 13% 15% 12% 11% 2016 Q Q Q Q Q Q Q Q3 32

33 The Most Commonly Mentioned Vendor Problems (2018 Q3): Did not receive or received little help from Vendor 47% % Received no help in reaching plan or goals 35% Vendor no help or help not effective 31% Vendor would not listen, dismissed concerns 25% Need more guidance, support, explanation 22% Get voic , never answers the phone 11% 33

34 Current Employment Status 34

35 37% of current and former customers are working full or part time. Q50: Finally, these last few questions ask about what you are currently doing. Are you currently...? Working full time, that is, more than 35 hours per week % 13% Working part time 24% Currently looking for a job 28% In school or receiving job training 34% Volunteering my time 2% Keeping house 3% Currently unable to work 7% 35

36 80% of working customers were satisfied with their current job. 100% Q51: Thinking about your current job, how satisfied are you with what you are doing? (% Satisfied) 80% 79% 78% 75% 76% 75% 83% 81% 80% 60% 40% 20% 0% 2016 Q Q Q Q Q Q Q Q3 36

37 Most Commonly Mentioned Reasons for Being Dissatisfied with their Current Job: Q52: If Dissatisfied with Job: Why do you say that? (Multiple responses accepted) (n=26) Job does not provide benefits, does not provide good benefits % 68% Job is physically demanding 32% Not a career move, only temporary job 26% Bored with job, been at job too long, burnt out, need change 16% Dissatisfied with aspects or parts of job 16% 37

38 Area Profiles FY

39 What are the area profiles? The area profiles contain data for each area, broken out to represent the customers for that area. They provide the core metrics and the individual items in the survey. The area profiles provide data aggregated over the course of the fiscal year. Quarters 3 and 4 of 2017 and quarters 1 and 2 of 2018 Data has been reweighted to reflect the population of VR over this time span Some data from the ACS about the general population of each area is also included Note: Area 7 was not formed until 2017 Q4. All surveys are attributed to the area the customer used at the time of the survey.

40 What is included? Each page of the area profiles includes a different type of information. Page 1 covers basic demographic information about the population of the area over the fiscal year, including a small excerpt about the general population. Page 2 details some of the individuals items asked in the survey. Percentages listed are always the percentage of positive responses. Page 3 details problems and feedback to open ended questions throughout the survey, including the most commonly reported problems. Finally, there is an insert page which details the core metrics for the area.

41 Fiscal Year Data By combining the data for this span of time, MDR is able to produce more sensitive measurements of the data. This highlights some differences between groups Number of items significantly more positive Number of items significantly less positive Area Area Closed successfully 15 0 Closed unsuccessfully 0 17

42 Core Metrics by Area State Area 1 Area 2 Area 3 Area 4 Area 5 Area 6 Area 7 Overall Satisfaction and Expectations Experience with Services Provided by VR Experience with VR Staff and Counselors Communications with VR Staff Customer Control and Involvement Outcomes and Meeting Goals Satisfaction with Current Employment All things considered, would you tell your friends with disabilities to go to the VR program for help? (% Yes) Have you experienced any problems with VR or the services they have provided to you? (% No) 93% 94% 90% 94% 93% 94% 94% 92% 83% 85% 78% 83% 84% 86% 82% 85%

43 Fiscal Year Findings The highest measures in every area were always the metrics which make up the Experience with Staff and Counselors domain. State Area 1 Area 2 Area 3 Area 4 Area 5 Area 6 Area 7 Q19 Q29 How helpful was the staff of VR in helping you achieve your VR goal? [% Helpful] The VR staff treated me with dignity and respect. [% Agree] 89% 91% 85% 88% 89% 89% 92% 87% 96% 96% 95% 97% 96% 96% 96% 96% Lower measures were generally related to outcomes and communications with staff.

44 Area 1 Highest measure: How helpful was the staff of VR in helping you achieve your VR goal? [% Helpful: 91%] Lowest measure: How satisfied were you with how long it took your counselor to answer your questions or address your concerns? [% Satisfied: 77%] Most common problem: Had to leave multiple messages before getting a call back [35%]

45 Area 2 Highest measure: How satisfied were you with the information you were given about the choices you had? [% Satisfied: 85%] Lowest measure: How satisfied were you with how long it took your counselor to answer your questions or address your concerns? [% Satisfied: 75%] Most common problem: Counselor did not return calls, s or follow up [37%]

46 Area 3 Highest measure: How easy was it for you to contact your VR counselor? [% Easy: 87%] Lowest measure: The VR services I received helped me become more financially independent. [% Agree: 78%] Most common problems: Did not receive employment, VR could not find me a job [30%]

47 Area 4 Highest measure: How satisfied were you with the information you were given about the choices you had? [% Satisfied: 86%] Lowest measure: The VR services I received helped me become more financially independent. [% Agree: 76%] Most common problems: Did not receive employment, VR could not find me a job [32%]

48 Area 5 Highest measure: How satisfied were you with your involvement in your VR experience? [% Satisfied: 87%] Lowest measure: How satisfied were you with how long it took your counselor to answer your questions or address your concerns? [% Satisfied: 78%] Most common problem: Did not receive employment, VR could not find me a job [31%]

49 Area 6 Highest measure: Overall, how satisfied are you with Florida VR's program? [% Satisfied: 89%] Lowest measure: Thinking about your current job, how satisfied are you with what you are doing? [% Satisfied: 75%] Most common problem: Counselor was not helpful or supportive [28%]

50 Area 7 Highest measure: Overall, how satisfied are you with Florida VR's program? [% Satisfied: 85%] Lowest measure: The VR services I received helped me become more financially independent. [% Agree: 76%] Most common problem: Counselor did not return calls, s or follow up [31%]

51 Final Note: Survey Revisions MDR is currently working with FL VR and the FRC to implement survey changes for the new period of the survey. These changes have no yet been finalized. Feedback from various stakeholders needed. Once the changes are finalized, MDR will implement them and begin reporting on them immediately.

52 Thank You I d be happy to answer any questions you might have.