Office of Shared Accountability. February David J. Bernstein, Ph.D. Evaluation Specialist. Cynthia Loeb Logistics Specialist. Julie Wade Analyst

Transcription

1 Summary of Results from the Survey of Work Environment: Non-School-Based Employees Office of Shared Accountability February 2003 David J. Bernstein, Ph.D. Evaluation Specialist Cynthia Loeb Logistics Specialist Julie Wade Analyst

2 SUMMARY OF RESULTS FROM THE SURVEY OF WORK ENVIRONMENT: NON-SCHOOL-BASED EMPLOYEES Evaluators: David Bernstein, James Griffith, Julie Wade, and Cynthia Loeb BACKGROUND This report provides results of the Survey of Work Environment completed in the spring of 2002 by Montgomery County Public Schools (MCPS) staff in non-school locations. Permanent staff members that worked in non-school-based locations districtwide in the spring of 2002 were surveyed, with the exception of most employees with the classification of director or other higher-graded managers. Survey results describe employee perceptions of specific areas of the work environment in non-school-based offices and departments of MCPS. Survey results are presented for non-school-based permanent employees in the district who responded to the survey. The intended use of survey results is by managers and staff for planning purposes. What Was the Survey Purpose? In response to the school district s strategic plan, Call to Action, the Survey of Work Environment was developed to collect information regarding the work processes and functions for purposes of accountability and improvement. Typically, in studies of organizations, surveys of organizational members serve to assess functions and processes found in the organization. Perceptions of work environment obtained from staff members serve as measures of work functions and processes. What Was Asked? METHODOLOGY The survey consisted of positively worded items or statements representing seven broad work environment content domains: work environment, management environment, social environment, physical environment, training and development, customer satisfaction, and job satisfaction. Domains were identified by reviews of published research in areas such as employee opinion surveys, job satisfaction, and organizational effectiveness. School district research staff met with various managers of non-school-based employees, union representatives, and employees to review and to adapt and/or develop survey items that would be used to assess identified domains. In some cases, broad domains were subdivided into specific areas related to the broad domain. For example, the management environment included supervisory support of employees; employee empowerment and innovation; goal clarity; and safety. Insofar as possible, broad domains, specific areas, and individual items were developed to be consistent with the school-based Staff Survey

3 of School Environment. Assessment of reliability and validity showed adequate internal reliability and adequate construct validity for the Non-School-Based Staff Survey of Work Environment (see Appendix 2). Who Was Surveyed? All 3,557 MCPS employees who had permanent employee status, were listed on the MCPS Human Resource Information System (HRIS) in February 2002 with a work position below the director level (with a few exceptions), and did not receive their pay voucher in a school were sent a survey. Most surveys were distributed to employees at their work locations through the MCPS Pony and returned to the Office of Shared Accountability (OSA) by Pony. Employees who did not have one defined work location in an MCPS office, such as bus operators and other employees, were mailed a survey to their homes. These employees returned the completed surveys in postage-paid envelopes to the OSA. How Representative Are Results? Of the 3,557 non-school-based employees surveyed, 1,215 staff members or 34.2 percent completed questionnaires. The percentage of returned surveys (response rate) among the offices ranged from 19 percent to 62 percent. The lowest response rate (19 percent) was observed in the Department of Transportation, where many staff members are permanent employees that work part-time. The completion rate for staff other than transportation employees was 42.9 percent. Characteristics of responding staff members were examined in relation to those of all non-school-based staff members, and some differences were observed. Table 1 (see page 3) shows the percentage of staff represented in the responding group and among all non-school-based staff in terms of gender, race/ethnic identification, and number of years employed by MCPS. Differences in the demographic characteristics between the responding group and the universe of non-school-based staff were under representation of African American staff, and overrepresentation of white staff, and an average of one more year of employment with MCPS in the responding group than among all nonschool-based staff. Office of Shared Accountability Survey of Non-School-Based Employees Page 2

4 Table 1 Composition of the Universe and Respondent Sample 1 (Note: Percentages reflect subgroup as a percentage of the universe or respondent sample.) Universe Respondent Sample Districtwide Response Rate % of % of N n Respondent Universe Sample Number of surveys sent out/surveys returned 3, % 1, % with department identified Number of respondents not identifying dept. NA NA % Total surveys sent out/surveys returned 3, % 1, % Race Identification African American % % American Indian 18.5% 5.5% Asian American % % Hispanic % % White 2, % % Number of surveys sent out/surveys returned 3, % 1, % with race identified Number of respondents not identifying race NA NA % Total surveys sent out/surveys returned 3, % 1, % Gender Male 1, % % Female 2, % % Number of surveys sent out/surveys returned 3, % 1, % with gender identified Number of respondents not identifying gender NA NA % Total surveys sent out/surveys returned 3, % 1, % Years Working in MCPS Mean (SD) number of years 13.6 (9.9) 14.6 (9.8) Number of surveys sent out/surveys returned 3, % 1, % with years in MCPS specified Number of respondents not specifying years MCPS % Total surveys sent out/surveys returned 3, % 1, % 1 The universe is defined as permanent staff members that worked in non-school-based locations districtwide in the spring of 2002, with the exception of most employees with the classification of director or other higher-graded managers. The respondent sample is the number of non-school-based employees that responded to the survey. Office of Shared Accountability Survey of Non-School-Based Employees Page 3

5 RESULTS How Responses Are Reported. Non-school-based employees were asked to rate their agreement or disagreement with specific survey items using a four-point response scale: Strongly Agree, Agree, Disagree, and Strongly Disagree. The survey results present the average percentage of respondents who agreed (i.e., marked Strongly Agree or Agree ) with sets of work environment-related items having similar content, which are referred to as specific areas. The percentage of agreement with a series of positively worded statements about the work environment is deemed to reflect positively or less positively on staff perceptions of a specific area. The degree of agreement with items within a specific area and/or a broad domain is deemed to reflect positively or less positively on that element of the work environment. Interpreting Response Values or Percentages. Responses of all non-school-based employees are presented in the District Summary below, and detailed in Appendix 3. The overall values of sets of items of similar content (specific areas) are examined districtwide. This approach allows an overview of results for all MCPS non-schoolbased staff on staff perception of the work environment. Districtwide, using level of agreement with positive statements about the work environment, areas of strength and areas that may need improvement can be identified. District Summary In the districtwide summary of survey results (see Figure 1), sets of items of similar content having high and low values relative to other specific areas were identified. Districtwide areas of strength included broad areas such as customer satisfaction and job satisfaction. Areas identified as needing improvement included the physical working environment and specific training and professional development needs of employees. Areas of Strength Specific areas having high values (relatively greater agreement) included: Customer Satisfaction (average 89 percent agreement). These items rated employee perceptions about meeting customer needs. Items receiving the highest agreement include: customers were satisfied with the services provided by the work group (94 percent); customers can easily reach someone in the work group for help (92 percent); the work group responds quickly to requests for information (90 percent); and customers are satisfied with the completeness of services provided (90 percent). Job Satisfaction (average 86 percent agreement). Items in this specific area receiving highest agreement included: employees like the work that they do (94 percent) and take pride in working for MCPS (92 percent). Office of Shared Accountability Survey of Non-School-Based Employees Page 4

6 Figure 1 Summary of Districtwide Results by Specific Area Non-School-Based Staff n = 1,215 Percentage Agreement Customer Satisfaction Job Satisfaction Goal Clarity Supervisor Support of Employment Teamwork and Cooperation Staff Recognition Employee Empowerment/Innovation Staff Orientation Environmental Areas Safety Work Quality Facilities Training and Development Goal Clarity (average 84 percent agreement). Items in this specific area receiving highest agreement included that employees understand how their job contributes to achieving MCPS goals (92 percent) and employees know what is expected of them in their jobs (91 percent). Supervisory Support of Employees (83 percent agreement). Items in this specific area receiving highest agreement included: supervisor lets me know what is expected of me (87 percent); supervisor treats me with respect (91 percent); supervisor considers my suggestions (87 percent); and supervisor sets high standards for performance for the work group (87 percent). Areas for Improvement Specific areas having relatively low levels of agreement compared with other specific areas were identified as areas that might require improvement. These included: Training (64 percent agreement). Items in this specific area receiving lowest percentage agreement included: development of training plan to improve job Office of Shared Accountability Survey of Non-School-Based Employees Page 5

7 skills (43 percent); performance evaluation that helps improve job skills (64 percent); satisfaction with current training received (65 percent); and available training enabling career advancement (65 percent). Work Unit Facilities (65 percent agreement). Items in this specific area receiving lowest percentage agreement included satisfaction with: air quality (49 percent); air temperature (47 percent); and repairs made quickly (64 percent). Districtwide results for all of the specific areas, as well as the individual survey items in each specific area, are presented in Appendix 3-1. Appendix 3-2 includes a summary of districtwide results by specific area. The Survey of Work Environment for Non-School-Based Employees provides a starting point for assessing the effectiveness and climate of the workplace for MCPS staff working outside the schools. Districtwide results suggest that, overall, non-school-based employees feel high levels of satisfaction and pride in their work. Certain areas may need attention districtwide because those specific areas had relatively lower levels of agreement with positive statements about the work environment,. Specific attention might need to be focused on the physical environment (facilities), as well as opportunities for training and development. Staff within each work group can examine their results in order to set priorities for improvements in the work environment. Use of the Survey Results Questions to be asked by the user of this survey results report include: Which of the specific areas do employees report relatively strong levels of agreement with positive statements about the work environment? These areas might be considered strengths for the district. Which of the specific areas do employees report relatively strong levels of disagreement with positive statements about the work environment? These areas might be considered areas needing improvement. MCPS staff should review and discuss the survey results, and consider using this year s districtwide survey results as a benchmark against which future results may be compared. About the Appendices The appendices provide detailed information that may be of interest to readers. Appendix 1 includes a copy of the survey instrument. Appendices 2 through 2.3 include analysis of the validity of the survey. Appendix 3-1 includes districtwide percentage agreement with items in the survey. Appendix 3-2 includes a districtwide summary by specific work environment area. Office of Shared Accountability Survey of Non-School-Based Employees Page 6

8 Appendix 1. Questionnaire. Office of Shared Accountability Survey of Non-School-Based Employees Page 7

9 Office of Shared Accountability Survey of Non-School-Based Employees Page 8

10 Office of Shared Accountability Survey of Non-School-Based Employees Page 9

11 Office of Shared Accountability Survey of Non-School-Based Employees Page 10

12 Appendix 2. Reliability and Validity of the Survey. Reliability of the specific areas, or scales of the survey was assessed with Cronbach alpha coefficient. Results presented in Appendix 2-1 report the consistency of responses among staff for items in each specific area. These results and the item-total correlations for each specific area showed that ratings obtained from respondents had adequate internal reliability. Appendix 2-1. Reliability of Specific Areas Specific Work Environment Area Number of Items Cronbach Alpha Range of Item-Total Correlations Work Environment (Quality) Supervisor Support of Employees Employee Empowerment / Innovation Goal Clarity Safety Teamwork and Cooperation Staff Orientation Staff Recognition Facilities Training and Development Customer Satisfaction Job Satisfaction Validity of the survey or, the extent the survey items actually measured what they were intended to measure was assessed using factor analysis and correlation. Results of the factor analysis showed that responses given by staff corresponded fairly well with the arrangement of survey items into specific areas. In other words, staff rated items of similar content in like ways, providing some evidence of the construct validity of the survey. Appendix 2-2 (see next page) lists the factors and the specific areas represented by each. Office of Shared Accountability Survey of Non-School-Based Employees Page 11

13 Appendix 2-2. Factor Arrangement of Survey Items Within Specific Areas Factor 1: Supervisory Support, Empowerment, Training and Development Factor 2: Teamwork and Cooperation, Staff Recognition Factor 3: Customer Satisfaction Factor 4: Job Satisfaction Factor 5: Training and Development Factor 6: Work Unit Facilities (items relating to repairs and available materials) Factor 7: Quality Work Factor 8: Physical Environment (items relating to physical space, air quality and temperature) Factor 9: Employee Empowerment / Innovation (items relating to job involvement, Questions 30, 31, and 32) Factor 10: Safety Factor 11: Goal Clarity In addition, given the vast research on job satisfaction and work performance, it would be expected that staff who rated their work environments favorably would also report higher levels of job satisfaction and supervisory rating of their job performance. Indeed, the relation of specific areas correlated significantly and positively with job satisfaction and self-reported supervisory rating of job performance. Appendix 2-3 details the relations of specific areas to job satisfaction and self-reported supervisory rating of performance. Office of Shared Accountability Survey of Non-School-Based Employees Page 12

14 Appendix 2-3. Relations of Specific Areas to Job Satisfaction and Self-reported Supervisory Rating of Performance Intercorrelations Specific Area Work Quality Supervisor Support Empowerment / Innovation Goal Clarity Safety Teamwork Staff Orientation Staff Recognition Facilities Training Customer Satisfaction Job Satisfaction Job Performance Note. N ranged from 815 to 1,141 due to missing responses for some items in specific areas. All correlations are statistically significant, p <.001, two-tailed. Office of Shared Accountability Survey of Non-School-Based Employees Page 13

15 Appendix 3-1. Districtwide Survey Results: Percentage Agreement with Items (based on individual staff responses; overall n=1,215) % BROAD AREA/SPECIFIC AREA/SURVEY ITEM (Item #) Agree WORK ENVIRONMENT Work Environment Quality 74% 1. Overall quality of work done is very good (10). 92% 2. Responsible for services provided (6). 89% 3. Ensure services meet high standards (4). 88% 4. Quality placed above other work objectives (5). 81% 5. Established procedures to prevent/correct mistakes (8). 78% 6. Given enough support to do work (3). 73% 7. Track how services affect satisfaction of others (7). 73% 8. Seek information to improve services provided (11). 71% 9. Extra duties shared equitably (1). 65% 10. Trained to deal with people who are angry/upset about services (9). 58% 11. Few interruptions to work (2). 45% MANAGEMENT ENVIRONMENT Supervisory Support of Employees 83% 1. Treats me with respect (18). 91% 2. Lets me know what is expected of me (12). 87% 3. Considers my suggestions (19). 87% 4. Sets high standards for performance of work group (20). 87% 5. Competent to do his/her job (25). 85% 6. Available when needed (15). 84% 7. Easy to talk to when have a problem about work (24). 84% 8. Evaluates me accurately (13). 83% 9. Openly talk about work-related issues without worry (16). 82% 10. Encourages work group to talk about work-related issues (22). 82% 11. Supported when others make unreasonable demands (17). 80% 12. Good understanding of my daily activities (21). 79% 13. Successful in getting work group to work together (23). 79% 14. Provides feedback on my work (14). 67% Office of Shared Accountability Survey of Non-School-Based Employees Page 14

16 Appendix 3-1, continued. Districtwide Survey Results: Percentage Agreement with Items % BROAD AREA/SPECIFIC AREA/SURVEY ITEM (Item #) Agree MANAGEMENT ENVIRONMENT Employee Empowerment/Innovation 80% 1. Job offers me challenges (31). 88% 2. Variety in work I do (30). 86% 3. Job makes good use of my skills and abilities (32). 83% 4. Work group willing to try new ways to do things (27). 79% 5. Efforts to involve others in work group in decisions (29). 77% 6. Encouraged to come up with new, better ways of doing things (26). 76% 7. Efforts to involve me in decisions (28). 74% Goal Clarity 84% 1. Understand how job contributes to achieving goals of MCPS (38). 92% 2. Know what is expected of me in my job (36). 91% 3. Goals of work group are clear (33). 88% 4. Goals of MCPS are clear to me (37). 84% 5. Kept informed about matters relating to my job (34). 74% 6. Job description defines my job tasks and responsibilities (35). 73% Safety 78% 1. Feel safe from personal harm and injury in workplace (41). 82% 2. Workplace is a safe place to work (40). 79% 3. Personal things are safe in my workplace (39). 73% SOCIAL ENVIRONMENT Teamwork and Cooperation 81% 1. I get help and support from work group (42). 91% 2. Feel comfortable sharing ideas with other work group members (46). 85% 3. Members of work group work well together (44). 82% 4. Members of work group respect one another (43). 79% 5. Members of work group are open to feedback about performance (45). 76% 6. Enough time to coordinate work with other work group members (47). 74% Staff Orientation 80% 1. Newly assigned employees receive help/encouragement (50). 89% 2. Efforts to make new work group members feel welcomed (48). 87% 3. Procedures to orient new work group members to jobs (49). 63% Staff Recognition 81% 1. Work group recognizes the work I do is important (53). 87% 2. People who receive services tell us we are doing a good job (54). 83% 3. Work group members recognize accomplishments of others (52). 80% 4. Supervisor lets me know when I am doing a good job (51). 74% Office of Shared Accountability Survey of Non-School-Based Employees Page 15

17 Appendix 3-1, continued. Districtwide Survey Results: Percentage Agreement with Items % BROAD AREA/SPECIFIC AREA/SURVEY ITEM (Item #) Agree PHYSICAL ENVIRONMENT Work Unit Facilities 65% 1. Materials and equipment in good condition (61). 78% 2. Enough materials and equipment to do my job (60). 73% 3. Enough space in workplace to do work (55). 71% 4. Restrooms kept clean and well-supplied (58). 71% 5. Timely repairs (59). 64% 6. Satisfied with air quality in workplace (56). 49% 7. Satisfied with air temperature in workplace (57). 47% TRAINING AND DEVELOPMENT Training 64% 1. Given opportunities to improve job skills (64). 74% 2. Performance evaluation identifies areas of strength and needing improvement 73% (67). 3. Training available for career advancement (63). 65% 4. Satisfied with training received to do my job (65). 65% 5. Performance evaluation process improves job skills (66). 64% 6. Developed training plan to improve my job skills (62). 43% CUSTOMER SATISFACTION Customer Satisfaction 89% 1. Customers satisfied with services provided by work group (74). 94% 2. Customers can easily reach someone in work group for help (68). 92% 3. Respond quickly when customers make requests (70). 90% 4. Customers satisfied with completeness of services provided (72). 90% 5. Customers receive services they expect on time (71). 87% 6. Customers can easily reach right person in work group (69). 86% 7. Come up with new and better ways to do things for customers (73). 86% JOB SATISFACTION Job Satisfaction 86% 1. Like the work I do (78). 94% 2. Proud to work for MCPS (80). 92% 3. Get a lot of satisfaction from work (77). 85% 4. Satisfied with job (79). 84% 5. Look forward to going to work each day (75). 81% 6. Recommend workplace as good place to work (76). 80% Office of Shared Accountability Survey of Non-School-Based Employees Page 16

18 Appendix 3-2. Percentage Agreement with Specific Area: Districtwide Results Survey of Work Environment: Non-School-Based Staff, Montgomery County Public Schools Number of Respondents 1,215 Percentage Agreement Specific Work Environment Area Work Quality Supervisor Support of Employment Employee Empowerment/Innovation Goal Clarity Safety Teamwork and Cooperation Staff Orientation Staff Recognition Facilities Training and Development Customer Satisfaction Job Satisfaction Average Percentage Agreement Non-School-Based Staff n = 1,215 Percentage Agreement Work Quality Supervisor Support of Employment Employee Empowerment/Innovation Goal Clarity Safety Teamwork and Cooperation Staff Orientation Staff Recognition Environmental Areas Facilities Training and Development Customer Satisfaction Job Satisfaction Office of Shared Accountability Survey of Non-School-Based Employees Page 17

19 OFFICE OF SHARED ACCOUNTABILITY Dr. Wesley L Boykin, Director 850 Hungerford Drive Rockville, Maryland (301) Dr. Jerry D. Weast Superintendent of Schools Dr. Frieda K. Lacey Chief of Staff