WORKERS COMPENSATION DIVISION SURVEY

WORKERS COMPENSATION DIVISION SURVEY by Steve Stuart FEBRUARY OREGON SURVEY RESEARCH LABORATORY UNIVERSITY OF OREGON EUGENE OR 973-5245 541-346-824 fax: 541-346-526 Internet: OSRL@OREGON.UOREGON.EDU World Wide Web: http://darkwing.uoregon.edu/~osrl Introduction The Workers Compensation Division of the State of Oregon (Workers Comp) provides a wide variety of essential services to Oregon s citizens. As part of Workers Comp s efforts to evaluate the quality of its services, they contracted with the Oregon Survey Research Laboratory (OSRL) to conduct a survey on how callers to the Appellate Review Unit (), Benefit Consultation Unit (), and Preferred Worker Program () assess the quality of automated menus and staff attributes such as attitude, knowledge, and timeliness. Working closely with Workers Comp, OSRL planned, pre-tested and implemented a telephone survey of 23 respondents. This report summarizes the survey methodology and results. Survey Methodology Survey Instrument The broad goals of the survey were to obtain valid and reliable information from callers to Workers Comp on the quality of services provided to them during those calls. In designing the survey instrument, OSRL used a multi-path approach which included: drawing from OSRL s survey archives and professional networks for questions related to Workers Comp s needs; creating original survey questions with the assistance of Workers Comp s staff; and pre-testing individual questions and the entire survey instrument with members of the survey population. Oregon Survey Research Laboratory Page 1

The survey instrument was programmed into OSRL s computer-aided telephone interviewing (CATI) system and further pretested. A facsimile of the survey instrument is provided in Section 2 of this documentation. All interviews were completely confidential, and human subjects approval was obtained. Sample and Data Collection Interviewer training was conducted on October 28, 1999; see Section 3 for interviewer instructions. Interviewing was conducted from 9: AM until 9: PM, Monday through Sunday, until the target call unit sample size of 15 was met for the Preferred Worker Program (15), almost achieved for the Benefit Consultation Unit (94), and resolved for the Appellate Review Unit (31). Altogether, OSRL interviewers made 1,5 telephone calls to complete 23 interviews between October 29, 1999 and January 27,. Up to calls were made to each valid telephone number provided to OSRL by Workers Comp. Callers to Workers Comp had an equal chance of being selected. The net response rate was 72% and the refusal rate was 3%; see Section 5 for the sample and response rate report. The average length of the interviews was 8 minutes. Survey sampling errors are calculated to assist data users in assessing how much confidence to place in a particular survey result. Large random samples reduce sampling error, while select samples such as this survey create a larger error margin. Results for survey questions in which there is low variability also have less sampling error; for example, a variable with a 5/5 proportional split has wider confidence intervals than a variable with a 5/95 proportional split. For this study of 23, the sampling error for all call units combined, when the entire population of Oregon adults is used, is +6.46 percentage points on a variable with a 5/5 proportional split (at the 95% confidence level). For a variable with a 5/95 proportional split, the sampling is +2.82 percentage points. Note, the entire adult population of Oregon is not part of the universe of callers to Workers Comp and the sampling error is in fact unknown, but much smaller then the figures given here. Survey Results The presentation of survey results is organized around the subject areas identified on page 1. Readers of this report may refer to the 37 banner-style tables in Section 6 for more detail. In the banner tables, the contents are cross-tabulated by a wide range of demographic information. The banner data include counts and percentages for each question overall, and counts and percentages for each row and column of the crosstabulation. See Section 5 for instructions on how to read banner tables. Overall Caller Satisfaction Before discussing the importance of and satisfaction with individual call characteristics, it is important to first get a feel for overall call satisfaction. To measure the level of satisfaction, callers were asked to rate their overall experience, telephone service, whether Oregon Survey Research Laboratory Page 2

the telephone service met their expectations, and Workers Comp s ability to provide assistance. Through the course of the survey, callers were asked to rate their overall call experience satisfaction. Eighty-seven percent of the respondents were either satisfied or very satisfied. Those results did not vary noticeably by call unit, though the Preferred Worker Program did log the highest percentage of very satisfied callers. See Chart 1 below. Overall Satisfaction 5 3 1 46.1 46.847.6 38.7 45.2.9.4 Very Satisfied Satisfied Not Very Satisfied Chart 1 9.7 6.1 6.5 6.4 5.7 5.7 5.3 4.8 Not at All Satisfied When asked to rate overall satisfaction with the telephone service provided by Workers Comp, fully 51% of callers responded that it was excellent. These results did vary by which unit callers were accessing: 6% of callers to the Preferred Worker Program (), 48% of Appellate Review Unit () callers, and 43% of Benefits Consultation Unit (). See Chart 2 below. Telephone service rating 7 6 5 3 1 6 51.3 48.4 42.6 45.7 36.538.7 27.6 7.8 6.5 7.4 8.6 3.5 6.5 4.3 1.9 Excellent Good Fair Poor Chart 2 Callers were also asked to judge how well the telephone service met their expectations. Twenty-five percent of all callers, and 3% of those calling the call unit, felt the telephone service exceeded their expectations. Roughly two-thirds of the callers had their Oregon Survey Research Laboratory Page 3

phone service expectations met by the call units. It is important to remember that the content of each caller s expectations is not quantified in this question, and thus is relative to each individual s service needs. See Chart 3 below. How well phone service met expectations 8 7 6 5 3 1 67 6.9 61.3 55.2 29.5 24.8 25.8 19.1 13 12.9 13.8 12.4 Exceeded Met Failed to Meet Chart 3 The last overall satisfaction question measured any change in callers opinions of Workers Comp s ability to provide them assistance. Again, the was most successful in changing people s opinions for the better, with over 1/3 of their callers expressing an improved opinion of s ability to provide assistance. The was least successful at changing opinions for the better. However, they were the best at holding opinions steady and the least likely to cause opinions to decrease. See Chart 4 below. Change in opinion of Workers' Comp's ability to provide assistance 6 5 3 1 55.3 5 45.2 46.7 3.932.3 34.3 26.6 16.1 1.4 8.5 1.5 Improve Same Decrease Chart 4 Itemized Importance/Satisfaction Attributes Oregon Survey Research Laboratory Page 4

To better understand what importance callers place on staff and call characteristics, and how Workers Comp is performing on those items, six sets of importance/satisfaction questions were asked. There were question sets covering customer service characteristics, staff knowledge, helpfulness, timeliness, the automated call system, and general issues. The survey was programmed so each question was tailored to reference the particular call unit accessed. Customer Service When asked to identify the most important customer service characteristic, from a list of five choices, the results were somewhat evenly distributed. The characteristic most important to the greatest percentage of callers (28.3%) was that staff members displayed a genuine interest in their issues. Least frequently cited by callers as most important (7.8%) was staff members taking personal responsibility for their call. See Chart 5 below. Importance - Customer Service Characteristics 5 3 1 32.3 28.3 26.6 28.6 Display genuine interest 38.7.9 9.7 8.5 6.7.4 24.5 22.9 21.3 19.1 18.1 18.1 18.1 16.1 7.8. Follow up if necessary Take care of all needs in one call Treat with respect Take personal responsibility Chart 5 Callers were then asked to rate their satisfaction with the characteristic they felt was most important. Of all callers, 86.9% responded that they were either very satisfied or satisfied with the customer service characteristic they ranked most important. When broken down by characteristic, it is revealed that the most important characteristic to the greatest number of callers also had the highest level of satisfaction. Conversely, the least important characteristic had the lowest (relatively speaking) satisfaction level. Thus, Workers Comp call units seemed to be successfully matching performance with importance when addressing their clients customer satisfaction needs. See Chart 6 below. Oregon Survey Research Laboratory Page 5

Satisfaction - Customer Service 1 8 6 86.9 92.3 89.4 88.6 Display genuine interest Take care of all needs in one call Treat with respect 81.3 Follow up if necessary 72.2 Take personal responsibility Chart 6 Knowledge The next group of staff characteristics callers were asked to rate related to staff knowledge. Over % of all callers listed staff s ability to answer their questions, or refer them to someone who could, as most important. There was no differentiation in ranked importance by call unit. See Chart 7 below. Importance - Knowledge 5 3 1.9 43.6 41. 32.3 27.829. 27.7 27.6 25.8 18.7 21. 13.8 8.3 9.7 12.8 3.8 Ability to answer questions or refer Provide clear answers callers have confidence in Chart 7 Understand questions, concerns, problems Know the Workers' Comp law When asked how satisfied they were with Workers Comp s performance on their most important knowledge characteristic, 87% of callers responded very satisfied or satisfied. Variability between characteristics was fairly small, though staff providing clear answers callers have confidence in did fall off almost 6% from the next lowest. Since that characteristic is most important to 27% (second overall) of the callers, Workers Comp may need to address that mild shortfall. See Chart 8 below. Oregon Survey Research Laboratory Page 6

Satisfaction - Knowledge 92 9 88 86 84 82 8 78 86.8 89.5 Know the Workers' Comp law 88.4 88.3 Understand questions, concerns, problems Ability to answer questions or refer 82.8 Provide clear answers callers have confidence in Chart 8 Helpfulness The third bank of characteristics related to actual help given by staff. Again, callers were asked to identify the characteristic most important to them. In this set of options, % chose staff ability to provide clear answers they had confidence in. It is interesting to note that the same characteristic was tested in the knowledge group, with only 27% choosing it as most important. It is also interesting to note the large group of callers (32.3%, remembering that had the smallest sample size) who ranked staff s ability to stay impartial and non-judgmental as most important. See Chart 9 below. Importance - Helpfulness 5 3 1 39.641.9.4 38.1 37.1 3 27.7 Provide clear answers callers have confidence in 12.9 Help identify options 32.3.4.3 15.2 Impartial and nonjudgmental 6.1 6.5 7.4 4.8 Help make decisions Chart 9 When asked to rate their satisfaction level for this bank of issues, callers were most satisfied with staff s ability to stay impartial and non-judgmental. They were least satisfied with staff s help identifying options. Workers Comp may need to bolster their service in this area, as it is the most important help 3% of the callers needed, with the lowest relative satisfaction. See Chart 1 below. Oregon Survey Research Laboratory Page 7

Satisfaction - Helpfulness 9 88 86 84 82 8 78 86.9 89.4 89 Impartial and nonjudgmental Provide clear answers callers have confidence in 85.7 Help make decisions 82.6 Help identify options Chart 1 Timeliness The fourth bank of characteristics that Workers Comp was interested in testing dealt with the timeliness of service on the telephone. Approximately one-third (32.2%) of all callers found access to a live person on demand to be the most important piece of timely call management. That trend held true across all call units. The only major blip in the rankings came from the call unit, where 23% of callers, almost 9% more than the overall sample figure, ranked receiving a prompt call-back if needed as most important. Importance - Timeliness 3 1 32.232.33.933.3 Access to a person (versus recording) on demand 23.525.824.521.9.9 21.322.9 12.9 Call answered in a timely fashion Only need to make one call to get needs met Chart 11 22.6 13.9 16. 9.5 7.8 1.5 3.2 6.4 Receive a prompt callback if needed On hold a reasonable amount of time Overall satisfaction for this bank of questions was higher than for any other set of characteristics (93.4%). Callers seemed most pleased with Workers Comp s prompt phone answering (96.3%). However, once on the line, only 83.3% of all callers were satisfied with the length of time they were left on hold. When viewing the gap between importance and satisfaction, however, Workers Comp seems to be doing fine on a relatively unimportant issue. Workers Comp is doing incredibly well at providing satisfactory timeliness on the two most important issues. See Chart 12 below. Oregon Survey Research Laboratory Page 8

Satisfaction - Timeliness 1 95 9 85 8 75 93.4 96.3 95.9 Call answered in a timely fashion Access to a person (versus recording) on demand Chart 12 91.7 9.6 Only need to make one call to get needs met Receive a prompt callback if needed 83.3 On hold a reasonable amount of time Automated Call System The fifth category tested by Workers Comp judged importance of and satisfaction with automated call system characteristics. Of the five options presented, almost half the callers (49.1%) chose the ability to use a toll-free number as the most important call system component. There was no variation between call units on this point. See Chart 13 below. Importance - Automated Call System 6 5 3 1 49.1 38.7 52.1 49.5 18.7 29. 21.3 13.3 17.1 12.6 9.7 8.5 7.8 6.5 8.5 7.6 4.3 5.7 3.2 3.2 Able to use a toll-free number Operator assistance is readily available Call is routed correctly through automated options The automated instructions and options are clear Transfers from the introduction recording to another Chart 13 Satisfaction with call system features was basically inversely related to the percentage of callers who ranked it as most important. Thus, the lowest percentage of callers was satisfied with their ability to use a toll-free number, the most important call system characteristic. However, even the lowest satisfaction isn t bad, with more than 92% of those who rated this issue at least satisfied. See Chart 14 below. Oregon Survey Research Laboratory Page 9

Satisfaction - Automated Call System 1 98 96 94 92 9 88 93.9 1 Transfers from the introduction recording to another 96.6 Call is correctly routed through automated instructions 94.4 The automated instructions and options are clear 93 92.9 Operator assistance is readily available Able to use a toll-free number Chart 14 General Issues The final bank of importance/satisfaction options touched on more abstract concepts. Overall, 42% of the callers ranked getting the information they wanted or needed as most important. Over half of the callers (56.4%) ranked this concept highest. The lowest percentage of callers (12.2% overall) ranked being treated respectfully most important, though callers outpaced the other call units here as well (14.9%). See Chart 15 below. Importance - General Issues 6 5 3 1 56.4 42.2 34.3 25.8 35.5 37.1 28.3 16. 22.6 14.8 15.2 11.7 12.2 9.7 14.9 1.5 Callers get the information they want, or need Callers get adequate person-to-person help Callers get served in an acceptable amount of time Callers are treated respectfully Chart 15 Caller satisfaction in getting the information they wanted or needed did not match the importance level placed on it. Most important by a large margin (13.9% overall), the concept ranked a distant third in satisfaction, with less than 9% very satisfied or Oregon Survey Research Laboratory Page 1

satisfied. The concept most important to the fewest callers garnered the highest relative satisfaction. See Chart 16 below. Satisfaction - General Issues 1 95 9 85 92.9 1 98.5 88.7 88.2 8 Callers are treated respectfully Callers get adequate personto-person help Chart 16 Callers get the information they want, or need Callers get served in an acceptable amount of time Conclusions The results show callers to Workers Comp are generally satisfied with the service and services they receive. However, as noted in the Automated Call System and General Issues issue banks, Workers Comp needs to make sure service improvements are mapped to areas that callers deem important. Otherwise, resources will be wasted by oversatisfying relatively unimportant needs. Results may be further tailored to meet the needs of callers to different call units. For example, callers seem to have a relatively greater need for non-judgmental, helpful, live operator interactions where those operators take personal responsibility for the call. That need is being met fairly well, but may be improved upon. callers are apparently more information driven providing them clear, concise advice and direction will go a long way to meeting their needs. callers, like the callers, require person-toperson help. However, unlike callers, callers appear driven by a desire to get the necessary information in a timely manner, to avoid the need for further calls. To meet these needs, Workers Comp should maximize operator effectiveness in routing calls, referrals, and fully answering questions the first time through. Oregon Survey Research Laboratory Page 11