Content analysis of support service quality experiences

Transcription

1 Content analysis of support service quality experiences W.H.E. Bun9nx, M.A. Herps, J. Tan, S. Kuppens, L. Curfs contact: Maastricht University Governor Kremers Center & June 2015 AAIDD annual congress, Louisville (KY)

2 Introduc9on Method Results Conclusion and discussion

3 Introduc9on Persons with ID oten require significant support from professional service organiza9ons in order to maintain and enhance their quality of life. The support organiza9on s performance is pivotal to pursuing its mission and effec9vely contribute to their users well- being. User perceived support quality is an important factor of customer sa9sfac9on and a strategic element for con9nuous quality improvement management. Ques9ons: How to develop a valid and reliable method for assessing relevant percep9ons of professional support performances? What do stakeholders consider important issues in support service performance? What do stakeholders appreciate? [strengths - posi9ve experiences à maintain!] What do stakeholders consider issues for improvement? [weaknesses - nega9ve experiences improve!]

4 Method How to develop a valid and reliable method for assessing relevant percep9ons of support organiza9on performances? Cliënt (C) Family (F) Direct support staff (DSS) Support = interac9ve process partnership involving 3 actors 3 primary stakeholders in the support service encounter 3 relevant perspec9ves Family = oten formally (legal representa9ve) and emo9onally involved as cliënt of the support organiza9on

5 Method How to develop a valid and reliable method for assessing relevant percep9ons of support organiza9on performances? 3 dimensions of supports quality (1) Outcomes: content or outcome related ac9vi9es aimed at enhancing the cliënt s Quality of Life (2) Facilitators: condi9ons and instrumental ac9vi9es necessary for delivering supports (3) Rela9onships: service rela9onships in terms of interac9ons between users and staff Percep9ons = subjec9ve measure of supports organiza9on performance (no objec9ve standards)

6 Method How to develop a valid and reliable method for assessing relevant percep9ons of support organiza9on performances? 3 dimensions of service quality (1) Outcomes: support content ac3vi3es aimed at enhancing the cliënt s Quality of Life 1 OUTCOME (QoL) (Schalock et al., 2008) 1.1 Personal development 1.2 Self- determina9on 1.3 Interpersonal rela9ons 1.4 Social inclusion 1.5 Rights 1.6 Emo9onal well- being 1.7 Physical well- being 1.8 Material well- being

7 Method How to develop a valid and reliable method for assessing relevant percep9ons of support organiza9on performances? 3 dimensions of service quality (2) Facilitators: condi3ons and instrumental ac3vi3es necessary for delivering supports 2 FACILITATORS 2.1 Staff competences 2.2 Team collabora9on 2.3 Individual support plan 2.4 Staff availability 2.5 Security 2.6 Informa9on 2.7 Con9nuity 2.8 Management (VGN, 2007/2013 & Dutch Inspectory Disability Care, 2008)

8 Method How to develop a valid and reliable method for assessing relevant percep9ons of support organiza9on performances? 3 dimensions of service quality (3) Rela3onships: service rela3onships in terms of customer values 3 RELATIONSHIPS 3.1 Responsiveness 3.2 Trust 3.3 Communica9on 3.4 Commitment 3.5 Empathy (Zeithaml, V.A. et al., 2006/2014)

9 Method How to develop a valid and reliable method for assessing relevant percep9ons of support organiza9on performances? This project: 2 open ques9ons for collec9ng subjec9ve experiences On the basis of your experiences of supports as received (C&F) from / delivered (DSS) by the supports team : (1) What do you value most about the supports that are offered to you (C) / your rela9ve (F) / your users (DSS)? What do you see as 'good' or 'strengths what do you like in the supports? [POSi9ve issues] (2) What do you see as 'weak' or things that should be improved what do you not like? [NEGa9ve issues] Please, answer in your own words. Consider the support received / offered in the past 6 months.

10 Method How to develop a valid and reliable method for assessing relevant percep9ons of support organiza9on performances? This project: 2 open ques9ons for collec9ng subjec9ve experiences Ques9ons discussed in interviews with users (independent interviewers). Ques9ons presented to Family and DSS in a paper or internet based survey. What was done with the answers? 1) Feedback to the user s ISP 2) Summarized in Quality Improvement Cards per team as input for team improvement ac9on planning 3) Each individual answer was coded in terms of the 3 dimensional / 21 domain validity framework (inter- rater reliability: Kappa between.88 and.94) 1 OUTCOME (QoL) 1.1 Personal development 1.2 Self- determina9on 1.3 Interpersonal rela9ons 1.4 Social inclusion 1.5 Rights 1.6 Emo9onal well- being 1.7 Physical well- being 1.8 Material well- being 2 FACILITATORS 2.1 Staff competences 2.2 Team collabora9on 2.3 Individual support plan 2.4 Staff availability 2.5 Security 2.6 Informa9on 2.7 Con9nuity 2.8 Management 3 RELATIONSHIPS 3.1 Responsiveness 3.2 Trust 3.3 Communica9on 3.4 Commitment 3.5 Empathy

11 Example of a Quality Improvement Card User s feedback on POS and NEG issues à Family feedback on POS and NEG issues à DSS own feedback on POS and NEG issues à

12 Method This project: Surveys in 15 support service organiza9ons with mixed campus and community based models. All users involved in long term 24 hour residen9al support. Respondents: samples of users (1/3) most significant family contact for all users all DSS Response rates: users 65-94% - family 54-78% - direct support staff 39-64% Respondents Statements Posi9ve Nega9ve Total Users 722 1,681 1,138 2,819 Family 2,652 6,066 4,559 10,625 Direct support staff 1,990 5,110 4,714 9,824 Total 5,364 12,857 10,411 23,268

13 Results 1. The 3 dimensions of service quality are differently appreciated by users, family and DSS % , , ,9 34, ,9 25,9 23,1 19,7 10 9,7 0 Clients Family Direct support staff OUTCOME (QoL) FACILITATORS RELATIONAL Distribu9on of statements of each group with respect to quality dimension (Total number of statements is 100% for each group of respondents) (Shown = sum of POS and NEG statements à importance of the dimension)

14 Results 1. The 3 dimensions of service quality are differently appreciated by users, family and DSS (Posi9ve and nega9ve statements show rather similar than complementary profiles) 0,0 20,0 40,0 60,0 80,0 100,0 dss NEG POS NEG NEG POS users family family dss users POS dimension group OUTCOME (QoL) FACILITATORS RELATIONAL Frequency of appearance of the 3 dimensions in posi9ve and nega9ve statements of each group (Shown = rela9ve distribu9on of the 3 dimensions in each group for POS and NEG statements)

15 Results 2. Posi9ve and nega9ve statements oten show in the same domain Users most frequently express posi9ve experiences with support in the area of social inclusion (ou9ngs and meaningful (day)ac9vi9es). Users most frequently express nega9ve experiences in the area of material well- being (housing - maintenance). users Posi9ve Nega9ve 0,0 5,0 10,0 15,0 20,0 25,0 OUTCOME (QoL) Personal development Self- determina9on Interpersonal rela9ons Social inclusion Rights Emo9onal well- being Physical well- being Material well- being FACILITATORS Staff competences Team collabora9on Individual support plan Staff availability Security Informa9on Con9nuity Management % (Total number of statements is 100% for POS and NEG each) RELATIONAL Responsiveness Trust Communica9on Commitment Empathy

16 Results 2. Posi9ve and nega9ve statements oten show in the same domain Family very frequently expresses posi9ve experiences with DSS commitment to the user s well- being. Concern is more oten expressed in the areas of material well- being (housing maintenance - cleaning), staffing levels and physical well- being (personal care). Family Posi9ve Nega9ve OUTCOME (QoL) Personal development Self- determina9on Interpersonal rela9ons Social inclusion Rights Emo9onal well- being Physical well- being Material well- being FACILITATORS Staff competences Team collabora9on Individual support plan Staff availability Security Informa9on Con9nuity Management 0,0 5,0 10,0 15,0 20,0 25,0 30,0 35,0 % RELATIONAL Responsiveness Trust Communica9on Commitment (Total number of statements is 100% for POS en NEG each) Empathy

17 Results 2. Posi9ve and nega9ve statements oten show in same domain 0,0 5,0 10,0 15,0 20,0 25,0 OUTCOME (QoL) Personal development Self- determina9on Interpersonal rela9ons Social inclusion Rights Emo9onal well- being Physical well- being Material well- being % DS staff frequently reports posi9ve experiences in the domains of commitment to the users well- being, staff competences and team collabora9on. DS staff frequently expresses dissa9sfac9on with staffing levels (9me available for individual user contact), team collabora9on and management (bureaucracy). DSS Posi9ve Nega9ve (Total number of statements is 100% for POS en NEG each) FACILITATORS Staff competences Team collabora9on Individual support plan Staff availability Security Informa9on Con9nuity Management RELATIONAL Responsiveness Trust Communica9on Commitment Empathy

18 Results 3. Importance of service quality domains 7 top ranking domains in total number of statements (POS + NEG) accoun9ng for 80% of statements 0,0 2,0 4,0 6,0 8,0 10,0 12,0 14,0 16,0 18,0 20,0 % Social inclusion 18,3 Material well- being 17,9 users Emo9onal well- being Physical well- being 9,0 10,1 Commitment 8,9 Self- determina9on 7,6 Interpersonal rela9ons 7,6

19 Results 3. Importance of service quality domains 7 top ranking domains in total number of statements (POS + NEG) accoun9ng for 68% of statements 0,0 2,0 4,0 6,0 8,0 10,0 12,0 14,0 16,0 18,0 20,0 % Commitment 18,0 Material well- being 10,8 Family Communica9on Physical well- being Social inclusion Staff availability Emo9onal well- being 9,3 8,1 7,9 6,9 6,8

20 Results 3. Importance of service quality domains 7 top ranking domains in total number of statements (POS + NEG) accoun9ng for 74% of statements 0,0 2,0 4,0 6,0 8,0 10,0 12,0 14,0 16,0 % Team collabora9on 15,2 Staff competences 13,7 DS staff Management Staff availability Commitment 10,2 10,8 11,9 Social inclusion Empathy 5,8 6,3

21 Results 4. Profiles of posi9ve and nega9ve experiences show resemblance across organiza9ons however, differences in frequencies per domain do exist between organiza9ons and within the same organiza9on across 9me.

22 Conclusion and discussion Open ques9ons about posi9ve and nega9ve experiences with respect to support delivery by professional support teams result in useful data for evalua9ng and improving supports quality. At the individual level, responses can be used in the ISP process to improve individual support effec9veness. At the team level, responses of each group (C, F, DSS) and in each category (POS- NEG) can be summarized for use in team quality improvement Ac9ons ( Quality Improvement Cards ). At the organiza9on level, profiles of strengths and weaknesses per domain and profile comparisons can be used in the context of service quality improvement policy and prac9ce.

23 Desired quality level A C P D Present quality level A C P D Assurance

24 Conclusion and discussion The theore9cal framework of 3 dimensions and 21 domains is validated in QoL literature, field norms of the Dutch Inspectory and Associa9on of Service Providers, and in the service quality model of Zeithaml et al. (SERVQUAL). It has shown prac9cal value in categorizing and analyzing spontaneous responses to open ques9ons about service quality experiences in support organiza9ons. Profiles of domain strengths and weaknesses can be used in support service organiza9on comparison both between different organiza9ons and over 9me within the same organiza9on. Profiles demonstrate an organiza9on s perceived strengths and weaknesses. Knowledge of these profiles may be useful for marke9ng and communica9on purposes. Knowledge about user s, family and DSS quality percep9ons may contribute to quality improvement and improvement of communica9on with stakeholders.

25 Literature - Bun9nx, W. & Benjamins, C. (2010). Content analysis of client, family and staff percep9ons of service quality. Journal of Applied Research in Intellectual Disabili9es, 23 (5), Cobigo, V., Mar9n, L., Lysaght, R., Lunsky, Y., Hickey, R.& Ouelle{e- Kuntz, H. (2014). Quality Improvement in Services for Adults with Intellectual and Developmental Disabili9es: Guiding Principles. Journal on Developmental Disabili2es, Volume 20, Number 2, Lighter, D.E., & Fair, D.C. (2004). Quality Mangement in Health Care. Principles and Methods. Boston: Jones and Bartle{ publishers. - Gehandicaptenzorg Nederland (VGN) (2007). Visiedocument Kwaliteitskader Gehandicaptenzorg. Utrecht: VGN et al. - Inspec9e Gehandicaptenzorg (2008). Overzicht Kwaliteitsindicatoren Gehandicaptenzorg. Den Haag: IGZ - Schalock, R.L., Bonham, G.S., & Verdugo, M.A. (2008). The conceptualiza9on and measurement of quality of life: Implica9ons for program planning and evalua9on in the field of intellectual disabili9es. Evalua2on and Program Planning, 31(2), Zeithaml, V.A., Bitner, M.J., & Gremler, D.D. (2006), Services Marke2ng: Integra2ng Customer Focus across the Firm. New York: McGraw- Hill. [Alan Wilson, Zeithaml, Mary Jo Bitner, Dwayne Gremler (2012). Services Marke2ng: Integra2ng Customer Focus Across the Firm / Kindle Edi9on]