Analyzing the Relationship between Organizational Culture and HMIS Use among Homeless Service Providers

Transcription

1 Analyzing the Relationship between Organizational Culture and HMIS Use among Homeless Service Providers December 2009 By Courtney Cronley, PhD UNIVERSITY OF TENNESSEE COLLEGE OF SOCIAL WORK OFFICE OF RESEARCH AND PUBLIC SERVICE

2 Acknowledgements This study was funded by a Doctoral Dissertation Research Grant from the United States Department of Housing and Urban Development s Office of University Partnerships. The author would like to thank the following groups for their participation in and cooperation with this research: the East Tennessee Coalition to End Chronic Homelessness; the Michigan Coalition Against Homelessness; Holland/Ottawa County Continuum of Care (CoC); Out-Wayne Counties CoC; and Pontiac/Oakland County CoC. Contact The author can be reached at ccronle1@utk.edu to answer any questions regarding the content of this report. A full description of the methodology and results is available through the dissertation abstracts in the Hodges Library at the University of Tennessee, 2

3 Table of Contents Executive Summary..4 Research Report Introduction...5 Background....5 Research Questions Design. 7 Results 9 HMIS Use... 9 Organizational Culture 12 HMIS Use and Organizational Culture 15 Limitations Summary and Recommendations.. 17 References

4 Executive Summary The United States Department of Housing and Urban Development (HUD) has required federally-funded homeless service providers to participate in their homeless management information systems (HMIS) since As of now, though, no one has examined how and to what extent these technologies are being used. Theory and research suggest that the technology dissemination is contingent upon the organizational culture in which new resources are being used. This study represents the first empirical analysis of HMIS use and explores the cross-level relationship between staff members HMIS use and organizational culture. Staff members at 26 homeless service providers completed the Organizational Social Context (OSC) survey. Individual scores were aggregated to determine the organizational culture of each organization. Data on HMIS use, measured as the number of times that an individual attempted to log on to the system, were collected from 142 individuals. Results suggest that organizational proficiency is related to HMIS use and is moderated by gender. The rate of log on attempts for male staff members increases in organizations with higher levels of proficiency. Moreover, organizational culture results revealed that the sample reported significantly higher levels of organizational proficiency, rigidity, and resistance, compared to a national sample of children s mental health providers. The study concludes with the recommendation that policy makers view HMIS implementation as an ongoing, cyclical process of interactions among the organizational social context, the software, and the researchers developing the technology. 4

5 Introduction This study represents the first attempt to measure empirically the use of homeless management information systems (HMIS). It stems from the Department of Housing and Urban Development s (HUD) 1999 mandate requiring all federally funded homeless service providers to participate in an HMIS (HUD, 2008). To date only one study has assessed HMIS implementation (Gutierrez & Friedman, 2005). This study was not empirical, however. It was based on the authors observations and self-reports from staff members at the organizations. Considering the funding that HUD has allocated for this project and the expected implications of its use, it is critical that we conduct systematic and objective assessments of the extent to which HMIS are being used by organizations. Background Research demonstrates that use of new technologies, such as information management systems and electronic referral systems, in human services can significantly improve service provision and client outcomes (McCoy & Vila, 2002). Studies show, though, that new technologies frequently are under or mis-used in organizations (Carrillo, 2005, 2007; O Looney, 2004; McCoy & Vila; Herie & Martin, 2002). Reasons for this include poorly designed technology, limited technical competence, and leadership that do not support change (Carrilio, Packard, & Clapp, 2003). This study considered how frequently homeless service providers were using their HMIS and what factors were related to use. It relied on three theories to explain 5

6 technology use in organizations: diffusion of innovations; sociotechnical theory; and organizational effectiveness. Table 1 describes each of the theories. Table 1. Theory base Theory Diffusion of Innovations (Rogers, 2003) Sociotechnical theory of organizational effectiveness (Trist & Bamforth, 1954) Organizational culture theory (Schein, 1992) Question How do new ideas spread among people? What is the relationship between technology and the social context in which it is used? Do shared values, beliefs, and expectations develop in the organization s social context and guide employee behavior? Both the theories of diffusion of innovation and sociotechnical organizational effectiveness consider how new ideas are adopted and what makes their adoption successful. Diffusion of innovations argues that new ideas, such as the use of cell phones, spread among people through social networks. One person tells another, who tells another, etc. Sociotechnical theory argues that the use of new technology, specifically in organizations, relies on a goodness of fit between the social context and the technology. Leadership and work practices must accommodate a new technology for it to be used, regardless of its effectiveness. Organizational culture theory provides a conceptual bridge between diffusion of innovations and sociotechnical theory. People in organizations behave according to established values and expectations as well as shared history and leadership. New ideas move into organizations to the degree that organizational leaders are aware of and introduce the innovations. Staff members in organizations whose cultures support these innovations, those that value proficiency and are willing to take risks with new 6

7 ideas, are most likely to use new technologies. Table 2 identifies the three main constructs of organizational culture that were measured in this study: rigidity, proficiency, and resistance. Table 2. Organizational culture constructs Construct Rigidity Proficiency Resistance Definition The degree to which organizations observe set policies and procedures for work processes. The degree to which organizations value staff competency and strive to provide the best possible services to clients. The degree to which organizations invite change. Research Questions The study sought to answer the following two questions: 1) Does organizational culture influences staff members HMIS use? 2) Do individual characteristics (e.g. gender) interact with organizational culture to influence staff members HMIS use? Design Four Continua of Care (CoC) in two states, Michigan and Tennessee, participated in the study. Figure 1 shows the geographic distribution of the data collection. A total of 26 homeless service providers and 142 staff members were sampled. Data were collected in two-waves, 1) January May, 2008, and 2) January May, Two variables were measured: organizational culture and HMIS use. Organizational culture was assessed as an organizational-level variable, meaning that each organization had a single score. HMIS use was assessed as an individual-level variable, meaning that each individual had a different score. Figure 2 shows the 7

8 Figure 1. Distribution of data collection sites. The blue dots represent participating organizations. structure of the sample. The analysis involved examining the relationship between organizational culture scores and staff members HMIS scores. Data were collected in two waves, 1) January May, 2008, and 2) January May, Two variables were measured: organizational culture and HMIS use. Organizational culture was assessed as an organizational-level variable, meaning that each organization had a single score. HMIS use was assessed as an individual-level variable, meaning that each individual had a different score. Figure 2 shows the structure of the sample. The analysis involved examining the relationship between organizational culture scores and staff members HMIS scores. The purpose was to determine if an organization s culture score influenced how frequently a staff member used an HMIS. Would staff members in organizations that scored high on organizational proficiency be more likely to use the HMIS than staff members in organizations scoring low or average? The study also assessed the 8

9 Nested Sample CoC 1 CoC 2 4 Organizations 8 Organizations 13 HMIS Users 34 HMIS Users CoC 3 CoC 4 5 Organizations 7 Organizations 44 HMIS Users 51 HMIS Users Figure 2. The sample consisted of 142 HMIS users, nested within 26 organizations, nested within four CoC. Each organization had an organizational culture score, and each staff member within each organization had an HMIS use score. variables separately. It examined organizational culture across the sample to see if it varied substantially among providers. In addition, the study considered how frequently staff members were attempting to use the HMIS and if this frequency was similar across organizations. Results HMIS Variability in Use The median number of times that staff members attempted to log on to the HMIS was 47 times per year. Results suggest, however, that there is a large amount of variety in how the organizations surveyed are using their HMIS. The number of times that staff members attempted to log on to an HMIS ranged from 2 to 719 in one year. Looking at use by organizations, one can group them into three main categories: 1) regular, 9

10 proportionate use; 2) irregular use; and 3) irregular, disproportionate use. Select organizations reflecting these profiles are shown in Figures 3, 4, and 5. The user ID, on the right, vertical axis, denotes individual staff members, in the organization. The months are identified on the horizontal axis, beginning with 2 for February. The count, on the left, vertical axis, denotes the number of times each month the staff member attempted to log on to an HMIS. Figure 3. Regular, proportionate use: A large number of staff members licensed to use the HMIS and many of them attempting to log on to the HMIS at least once every month. 10

11 Figure 4. Irregular use: A small number of staff members licensed to use the HMIS and attempting to log on infrequently and inconsistently. Figure 5. Irregular, disproportionate use: A single staff member licensed to use the HMIS and attempting to log on inconsistently. 11

12 There was also variability among the CoC in organizational use of HMIS. Aggregate organizational use of an HMIS ranged from a median of 33 in CoC 2 to a median of 337 in CoC 3. Figure 6 shows the total number of times that staff members attempted to log on to an HMIS in each CoC. Figure 6. Pie graph showing the total number of times that staff members attempted to log on to the HMIS, by CoC. Results show great disparity in attempts from a low of 817 times in CoC 1 to a high of 6,106 times in CoC 4. Organizational Culture Variability across Organizations Organizational culture results are reported as T-scores derived from a national sample of children s mental health providers (n = 100). A T-score of 50 means that an organization s culture shows average levels of proficiency, rigidity, and resistance compared to the national sample. One standard deviation is 10 points higher or lower than 50. Figure 7 shows that the homeless service providers in this study are markedly different from the national sample. These organizations report average levels of rigidity and resistance that are more than a full standard deviation above the mean (M =

13 T Score Rigidity Proficiency Resistance Figure 7. A graph comparing the sample of homeless service providers (in blue) to a national sample of children s mental health providers. It shows that the sample is markedly different from the national sample. and 64.11, respectively) and a proficiency level that is nearly one standard deviation above the mean (M = 58.11). Among the homeless service providers, however, there is a great deal of variability, particularly in proficiency scores, which ranged from a T-score of to Figure 8 show the two extreme organizational profiles found in the study. The model organization shows a high proficiency score and low resistance score. The least constructive organization shows the inverse relationship with a low proficiency score and high resistance score. 13

14 T Score T Score T Score Rigidity Proficiency Resistance 20 Rigidity Proficiency Resistance Figure 8. Examples of extreme organizational culture profiles. The organization on the left shows a model organization for disseminating innovations. The level of resistance is low, and the proficiency is high. The organization on the right shows a challenging site for innovation. Levels or rigidity and resistance are high while expectations of worker competency, proficiency, are low. Results also showed differences in organizational culture profiles among CoC. Figure 9 graphs the average T-scores of all organizations in each of the four CoC. There was an 8.33 point difference between CoC 3, in which organizations reported the highest average rigidity score (M = 63.86) and CoC 2, where organizations showed the Rigidity Proficiency Resistance Figure 9. The figure above compares average organizational culture profiles for all four CoC graphed against the normative sample of children s mental health providers. 14

15 T Score T Score lowest average rigidity score (M = 55.53). Similarly, there was an 8.54 point difference in average resistance scores between the highest in CoC 3 (M = 67.88) and the lowest in CoC 2 (M = 59.34). HMIS Use and Organizational Culture Figures 10, 11, and 12 compare select organizations HMIS use with their organizational culture profiles. The blue line indicates the select organization. The dark green line represents the sample average Rigidity Proficiency Resistance Figure 10. This organization show regular, proportionate use of the HMIS. Compared to the sample average (green line), the organization reports lower levels of rigidity and resistance (blue line) and a higher level of proficiency Rigidity Proficiency Resistance Figure 11. The organization above shows irregular use of the HMIS. One staff member never uses the HMIS and other staff members do not access the system for entire months at a time. The organizational profile shows resistance that is almost two standard deviations above the mean. 15

16 Log on Attempts T Score Rigidity Proficiency Resistance Figure 12. A single staff member uses the HMIS, suggesting disproportionate use, and the use is sporadic and irregular, swinging between a low of less than 10 times a month to a high of nearly 40 times. The corresponding organizational culture profile shows a level of resistance that is nearly two standard deviations above the mean. Results also showed that one component of organizational culture, proficiency, affected staff members use of an HMIS, although this effect was moderated by gender. Specifically, in organizations with high levels of proficiency, men were more likely to use an HMIS. Figure 13 shows this interaction between gender and proficiency Female (0) Male (1) Proficiency Figure 13. Men s use of an HMIS increases as organizational proficiency increases, while women s use is unaffected. Interestingly, the increase for men doesn t occur until organizations show markedly higher levels of proficiency compared to the average organization. 16

17 Limitations This study shared many of the limitations common to studies of organizations. Every effort was made, however, to minimize the effects of limitations to the accuracy of the results and interpretation. Primary limitations include: Sample the small, purposive sample limits the results to the organizations and communities included in the study. Measurement the study relied on log on attempts to measure HMIS use. This measurement may not measure the variety or the substantive ways in which staff members interact with an HMIS. Data collection the study did not collect individual-level data about staff members familiarity with technology. Factors such as education, job title, and technical training may affect an individual s use of an HMIS, above and beyond the organizational culture. Summary and Recommendations Results from this study can be summarized in Figure 14. Dissemination of the HMIS into homeless service providers has been an ongoing, cyclical process moving from initial efforts by HUD to disseminate the technology to service providers adopting it, to front-line practitioners who are expected to use the system in daily activities. HUD designed HMIS to act as real-time data bases that case managers use as they serve clients. As organizations adopt the technology and staff members use it, this research shows that not all organizations are using the data base in the real-time. Possible explanations for this finding may be that organizations are maintaining dual record keeping systems with paper assessments that they transfer to an HMIS. Alternatively, 17

18 Figure 14. The diagram shows the dynamic interplay among different factors that contribute to dissemination of innovations into organizations. they may be allocating HMIS work to a single staff member who receives data from other staff members and enters everything en masse. Policy makers and researchers have a responsibility to assess the implementation and consider ways in which the technology or the methods of dissemination should be altered to maximize successful use of HMIS as tools of practice for service providers. The results of the study point to two major policy area recommendations: 1) Organizational culture influences how organizations are able to adopt and implement an HMIS. a. Implement organizational culture change interventions aimed at reducing resistance and rigidity while emphasizing proficiency. b. Provide long-term funding that supports technical assistance for the organizational change. 18

19 c. Act patiently. Organizational change and full implementation of technology are long-term, non-linear processes that face numerous setbacks and challenges. 2) Organizations are using the HMIS differently and often in ways contrary to the original design. a. Provide organizational specific training that supports the unique service environments of each provider. b. Monitor implementation through more implementation evaluations to ensure fidelity to the original design. 19

20 References Carrilio, T. (2005). Management information systems: Why are they underutilized in the social services? Administration in Social Work, 29(2), Carrilio, T. (2007). Using client information systems in practice settings: Factors affecting social workers use of information systems. Journal of Technology in Human Services, 25(4), Carrilio, T. E., Packard, T., & Clapp, J. D. (2003). Nothing in nothing out: Barriers to the use of performance data in social service programs. Administration in Social Work, 27(4), Gutierrez, O. & Friedman, D. H. (2005). Managing project expectations in human services information systems implementations: The case of homeless management information systems. International Journal of Project Management, 23. Herie, M. & Martin, G. W. (2002). Knowledge diffusion in social work: A new approach to bridging the gap. Social Work, 47(1), McCoy, H. V. & Vila, C. K. (2002). Tech knowledge: Introducing computers for coordinated care. Health and Social Work, 27(1), O Looney, J. (2005). Social work and the new semantic information revolution. Administration in Social Work, 29(4), Rogers, E. M. (2003). Diffusion of innovations (5 th ed.). New York: Free Press. Schein, E. (1992). Organizational culture and leadership. San Francisco: Jossey-Bass. Trist, E. L. & Bamforth, K. W. (1951). Some social and psychological consequences of the longwall method of coal-getting. Human Relations, 4, The United States Department of Housing and Urban Development. (2007). The Annual Homeless Assessment Report to Congress. Washington D.C.: Author. The University of Tennessee College of Social Work 224 Henson Hall Knoxville, TN