Assessing Employer Satisfaction: a test of several survey techniques

Transcription

1 Assessment & Evaluation in Higher Education ISSN: (Print) 69-97X (Online) Journal homepage: Assessing Employer Satisfaction: a test of several survey techniques Raymond H. Phillippi & Trudy W. Banta To cite this article: Raymond H. Phillippi & Trudy W. Banta (99) Assessing Employer Satisfaction: a test of several survey techniques, Assessment & Evaluation in Higher Education, 9:, -, DOI: 0.080/ To link to this article: Published online: Aug 006. Submit your article to this journal Article views: 8 Citing articles: View citing articles Full Terms & Conditions of access and use can be found at

2 Assessment & Evaluation in Higher Education, Vol. 9, No., 99 Assessing Employer Satisfaction: a test of several survey techniques RAYMOND H. PHILLIPPI Research Associate, Oak Ridge National Laboratory TRUDY W. BANTA Vice Chancellor for Planning and Institutional Improvement, Indiana University-Purdue University Indianapolis ABSTRACT Assessing employer satisfaction with the preparation their employees receive in institutions of higher education is a matter of growing importance world-wide as students, governments and the public seek tangible evidence of educational quality through workforce competitiveness. Three techniques for surveying college and university graduates and employers were conducted over the space of three academic years for the purpose of comparing the effectiveness of these methodologies. Response rates obtained indicate that gaining permission for employer contact directly on a questionnaire mailed to graduates can be as effective as more costly phone contacts, and is more effective than two successive mailings to graduates. The results also suggest that in general employers give relatively undifferentiated positive ratings of employee characteristics unless forced by the method of questioning to differentiate among items. The study provides evidence that a mailed survey methodology for graduates and their employers can be effective if the graduate's permission is obtained on the initial questionnaire and if employers are forced by the survey instrument's format to differentiate among listed job characteristics and performance ratings. As colleges and universities around the world make more self-conscious efforts to improve their effectiveness and demonstrate their accountability for public trust and funding, interest is increasing in surveying employers to gather their opinions and suggestions concerning the preparation of graduates. With grant support from the Fund for the Improvement of Postsecondary Education (FIPSE), faculty at the University of Tennessee, Knoxville formed a consortium of seven colleges and universities for the purpose of developing a mail survey for employers of graduates. This survey methodology was pilot-tested during the first year of the three-year project, and several shortcomings were noted. The employer questionnaire and a companion instrument for graduates were modified, and two more pilot studies were performed during the second and third years of the study. While the initial trial of the survey took place on five campuses,

3 R. H. Phillippi & T. W. Banta funding constraints limited the second pilot to two of the institutions and the third to only one, the University of Tennessee, Knoxville. The purpose of this paper is to describe the experience of the one institution that was able to pilot test this methodology in all three years. It provides a critique both of the revised questionnaires and of the modifications made in administration procedures. The result is a cost-effective survey methodology that can be used by two-year colleges of further education as well as by universities to assess employer satisfaction with graduates. Literature Review This review of relevant research investigates the use of employer surveys in both two-year colleges and universities. Studies are examined for their perspectives on survey methodology and attendant response rates, the latter being a primary indicator of survey quality. Several questions guide the review. Will employers be asked questions about individual graduates or about graduates in general? What method of questioning will be used mail survey, telephone interview or face-to-face interview? What kind of followup will be conducted? Experience of Two-year Institutions Two-year colleges in the USA serve the dual mission of preparing some students to enter universities to continue their education while preparing others for immediate employment in skilled and technical fields. In keeping with their emphasis on job preparation, two-year institutions have more extensive experience in surveying employers than have four-year institutions. At least two institutions report having used what might be called a 'broadside' approach to surveying sending questionnaires to employers only, without identifying any particular graduates. A survey of community college employers in New York yielded a 7% response rate when placement records were used to obtain the names of employers (Francis & Jones, 976). St Louis Community College at Forest Park identified employers from responses to a survey of graduates (Kapraun & Nienkamp, 985), a technique that yielded an employer response rate of 9%. In an attempt to obtain more specific information about program quality, many two-year colleges have linked their surveys of employers to named graduates. This approach necessitates obtaining the graduate's permission to question the employer, and the additional step lowers generalizability because those who grant permission may differ in important ways from those who do not. Institutions have used various techniques to resolve these problems. Some colleges have obtained permission as part of a mailed survey of graduates (Gell & Jones, 975; Gell et al., 976; Head, 990; Montemayor et al., 986). This technique has generally yielded permission rates varying from to 0%. In another approach, Lorain County Community College mailed postcards to graduates briefly explaining the purpose of the survey and asking for permission to survey the graduate's employer (Isbell & Jonas, 976). This method yielded a 6% rate of return. Finally, one institution surveyed its graduates first, then sent a letter requesting the graduate's permission to collect information from his/her employer (University of Hawaii, 977). This technique yielded a permission rate of 9% lowest of the methods in current use.

4 Assessing Employer Satisfaction 5 A seemingly more productive approach to increasing response rates involves the use of telephone interviews with former students. Using this method, Johnson County Community College obtained an 80% permission rate (Conklin, 990), highest of all techniques described in the current literature. Regardless of the method used to obtain permission, employer response rates have been relatively high (Case, 986; Conklin, 990; Head, 990; Maryland State Board for Community Colleges, 986). Employers respond at the rate of 60 to 90% when asked by institutions of higher education for their opinions of specific employees. Response rates are highest when the survey mailing includes a copy of the graduate's signed statement granting permission for the employer to respond (Head, 990). The literature concerning methods employed by two-year institutions indicates that 'broadside' surveys of employers of graduates elicit response rates of 9 to 7%, while surveys that identify specific employees attract higher rates of 60 to 90%. However, surveys that ask about a specific employee require that person's permission. This additional step reduces the employer pool because only to 8% of the graduates typically give their permission. This permission rate can be increased if contact is made by telephone rather than by mail, but the expense of a telephone interview may force those who use this technique to ask fewer questions than they would include in a mailed questionnaire. Experience of Universities The literature contains less about the experience of universities in using employer surveys than is the case for two-year institutions, and, in general, the techniques used by universities have yielded lower response rates. For instance, universities identifying employers through placement records have obtained response rates approximating 8% (Atkins & Kent, 988; Knoblauch & German, 989). By seeking permission to contact employers through a telephone interview with graduates, one institution obtained permission from 7% of its graduates and a response from 78% of their employers (Lucas, 98). Employer Responses Surveys of employers generally produce positive findings, regardless of the type of institution or the survey methodology applied. Employers rate most employee attributes as important and most specified employees as good or very good. In fact, employer responses show little variance and appear to be the result of a response set (tendency to respond generally in a particular way positively or negatively) rather than carefully considered rankings. Summary and Research Questions The literature suggests that the most effective way to obtain graduates' permission for employer contact is through a telephone survey, and the least effective way is through a second mailing after an initial response to a survey of graduates. In addition, published studies reveal that high percentages of employers respond to college-initiated surveys, but their generally positive employee ratings show little variance. However, these techniques have not been systematically compared by applying different instruments and methods to the same population. Moreover, despite the growing interest in learning how

5 6 R. H. Phillippi & T. W. Banta employers assess the workforce skills of college and university graduates, very few articles on this topic have appeared in the literature on higher education. This study undertook a systematic comparative analysis. Over the space of three years, three different methods were used to survey the same population graduates of a single university. The investigators sought to determine the best methodology for gathering employer opinion a valuable source of assessment information for institutions of higher education. First Administration Methods The Tennessee Higher Education Commission requires each public institution of higher learning to survey a segment of its graduates every other year using a common survey instrument. The results are used in a program of performance funding that determines the amount of an annual state budget supplement for the institutions (Banta, 988). The first administration of the survey for graduates used in this study occurred as part of the performance-funding regimen. As required, the state-approved questionnaire (see Appendix ) was mailed to all individuals who had graduated two years previously from the University of Tennessee, Knoxville (UTK). The questionnaire was accompanied by a letter of explanation and a stamped return envelope. After 0 days, a reminder postcard was sent to non-respondents. Finally, 0 days after the initial mailing, a second letter, copy of the questionnaire and return envelope were sent to all individuals who still had not responded. This methodology produced a response rate of 5%. Tennessee's performance funding requirements do not permit alterations in the standardized process for surveying graduates. Thus the request for permission to contact employers that was needed for the FIPSE-supported study could not be included. Immediately following receipt of each graduate's response, a separate letter was sent to solicit permission to contact employers. As the data in Table I indicate, % of the respondents who were employed full time gave this permission. The employer survey designed for the FIPSE-assisted study is unique in the literature in that the items were developed by faculty at two colleges of further education and four universities; thus the instrument can be used at two-year institutions as well as universities. The items were intended to elicit employers' opinions about the knowledge and skills their employees need and the extent to which the institution's graduates exhibit needed traits and skills (Banta et al., 99). The initial mailing and follow-up methodology described for the survey of graduates was used also for the employer survey. This procedure yielded a very high response rate 9%. However, there was little variance among the responses of given employers. Discussion The core of the employer survey instrument used in the first as well as subsequent administrations for this study is a set of items that ask respondents to rate on a three-point scale, ranging from not important to very important, the importance of characteristics of the graduate's job and then to rate on a four-point scale, ranging from poor to excellent, the performance of the graduate on these same characteristics. Examples of the characteristics include: planning projects, speaking effectively, writing effectively, being dependable and on time, and using computers. Of the 99

6 Assessing Employer Satisfaction 7 TABLE I. Response rates for the first administration Group No. % Respondents to the alumni survey 56 5 Respondents employed full time 7 78 Full-time employed who gave permission 55 Employers who responded 6 9 (6 respondents X items) possible responses to these items, 66 or 67% of them were s (very important in terms of importance to the job and good in terms of performance). A response rate that is so uniformly positive is of little value to institutional decision makers. In summary, the methods employed in the first administration yielded response rates for graduates and for employers that were acceptable, at least as judged against findings in current literature. However, too few graduates gave permission to contact their employers when they were asked by means of a second letter. Moreover, those employers who responded to the survey designed for them tended to rate all identified characteristics as very important to the graduate's job and the graduate as good in all of these areas. The research team concluded that if the information from the employer survey were to be useful to college and university personnel, revisions in the methodology were required. Second Administration Methods The second year of the FIPSE project was not one in which the state required a survey for graduates. Thus the UTK graduates surveyed in that year were asked on their own questionnaire for permission to query their employers. The instrument mailed to employers was nearly identical to that sent previously. Mailing and follow-up procedures were the same as in the first year. Table II contains the response rates achieved during the second year of the study. They are very similar to those obtained in the first year except for the permission rate. Twice as many graduates gave permission to contact their employers when asked at the time they completed their own questionnaire. Eliminating the second mailing apparently improved substantially the percentage of graduates willing to give permission for employer contact. The response rate for employers again was quite high 9%. However, employers' responses continued to show little individual variance. Of the 860 possible responses to the common items, 5 or 6% were assigned the rating of (very important to the graduate's job or good performance by the graduate). Discussion Changing the method of obtaining permission to contact the graduate's employer did result in an improved permission rate. However, the employer survey instrument continued to evidence the lack of response variance noted in the first administration. Apparently, the lack of employer variance was not the result of an unacceptably low

7 8 R. H. Phillippi & T. W. Banta TABLE II. Response rates for the second administration Group No. % Respondents to the alumni survey 50 5 Respondents employed full time 0 76 Full-time employed who gave permission 7 Employers who responded 6 9 graduate permission rate, which might be assumed to indicate that only satisfied employees had granted permission. Rather the lack of variance seemed to stem from something inherent in the questionnaire design. The investigators determined that the employer questionnaire should be modified for the third administration in a way that might increase response variance. Third Administration Methods In the third year of this study, changes in survey methodology were instituted for securing permission from graduates and for querying employers. Since it was again a year when the state required an alumni survey, permission requests could not be mailed directly. As questionnaires were received from graduates employed full time, these respondents were contacted by telephone with the permission request. The employer instrument subsequently administered by mail (See Appendix ) was modified in an attempt to increase the response variance. The response format for the section including the job characteristics described earlier was modified to eliminate the Likert scales used in previous surveys. In place of those scales, the researchers used a forced-choice format. One characteristic that seemed duplicative of another was eliminated. The remaining 0 characteristics were divided into two groups. The respondent was instructed to divide 00 points between the 0 characteristics in each group such that the point assignment would represent the relative importance of each characteristic to the graduate's job. A second 00 points were to be divided on the basis of the relative performance of the graduate on each characteristic. The employer was thus forced to choose among the characteristics; saying all were important was made more difficult. Table III presents the response rates obtained in the third survey administration. The permission rate secured through phone contact is similar to that obtained through the secondary mailing in the first year, and much lower than the primary mail contact yielded in the second administration. Nevertheless, this rate is dependent upon how it is calculated. The rate could be considered the percentage of those who were actually reached by phone. In the time available for the permission phase, 76 graduates were contacted, and 0 or 80% of them granted the university permission to contact their employers. This may well be the calculation method for the percentages reported for telephone permissions in the literature reviewed (Conklin, 990; Lucas, 98). This method was not used here, however, because it is not directly comparable to that computed for mailed surveys: it does not take into consideration those who were called but could not be reached by phone.

8 Assessing Employer Satisfaction 9 TABLE III. Response rates for the third administration Group No. % Respondents to the alumni survey 5 5 Respondents employed full time 90 7 Full-time employed who gave permission 0 Employers who responded The employer component of this third administration yielded a lower, but still highly acceptable, response rate 80%. The forced-choice method apparently produced this decline; a small but significant number of respondents wrote to complain about the time it took to complete the questionnaire. The forced-choice format did, however, produce greater variance among employer responses. The most prevalent response to the common items was an assignment of 0 points, which is exactly the number of points one would give if they were dividing the points evenly among the items. But, of the 700 possible responses (76 respondents X 0 questions), 0% were 0s. This percentage represents a decline of more than 0 points in the proportion of common responses assigned by employers in previous administrations. The forced-choice method apparently reduced the proportion of common responses and thus increased the variance within a given set of ratings. Discussion The changes made for the third administration of the surveys made a difference both in the permission rate for graduates and in the variance among employer responses. The low permission rate obtained by telephoning was very similar to that obtained through a secondary mailing technique, but not nearly as high as that produced by mailing the permission request directly, along with a companion questionnaire for the graduate. Contacting graduates by phone does not seem to improve their overall permission rate because it is so difficult to make direct contact with them. Requiring employers to distribute points across items instead of using the traditional Likert scale does seem to improve the response variance. Although this improvement apparently comes at the expense of some reduction in the employer response rate, employers generally seem prepared to respond when their opinions are solicited by institutions of higher education, and a high proportion will return even a questionnaire that forces them to consider their responses carefully. Conclusion This unusual opportunity to conduct a three-year comparative study of survey methods for graduates and employers has produced several findings that could emerge only from a carefully conducted longitudinal investigation. This research suggests that the most cost-effective method for obtaining graduates' permission to contact their employers is to couple the request with a questionnaire designed for the graduates themselves or some other direct mailing. Phone contact could improve the chances of obtaining permission, but only if the institution has accurate phone numbers and is willing to devote the resources necessary for making follow-up calls to those who were not available on the first attempt to reach them.

9 0 R. H. Phillippi & T. W. Banta High response rates indicate that employers are interested in sharing with colleges and universities their opinions concerning the employment preparation needed and being received by graduates. Employers will, however, tend to respond positively to most items typically asked of them by colleges and universities if the methodology allows it. Therefore, a forced-choice response format is recommended when asking employers about the relative importance of job skills or the performance of graduates. While most institutions think first about gathering data from employers using survey techniques, this study reveals problems with using mailed surveys exclusively. Using the best methods for obtaining permission and questioning employers, one could only hope to reach about one-third of the employers of the graduates in a given sample. Almost certainly this is too small an employer sample to be representative. Thus individual and group interviews with employers should be carefully considered as supplements to responses gathered by mail. Employers' opinions about an institution's graduates can provide valuable information for improving college and university programs and services. The information can assist faculty as they demonstrate to external authorities the quality of their programs and services and contemplate the need for improvements. To be most useful, the information must be current and decision makers must be able to make some discriminations among the factors being assessed. For this to occur, the methodology must be planned and implemented carefully. While no approach is without flaws, a combination of mailed surveys and face-to-face contact holds promise for producing the most reliable information. Notes on Contributors RAYMOND H. PHILLIPPI is senior research assistant in the Risk Analysis Section of the Health Sciences Research Division of Oak Ridge National Laboratory. TRUDY W. BANTA is professor of higher education and vice chancellor for planning and institutional improvement at Indiana University-Purdue University Indianapolis. Correspondence: 55 N. Lansing St. A0 0 Indianapolis, IN60-896, USA. At the time this article was written, Banta was the director and Phillippi a research associate at the Center for Assessment Research and Development at the University of Tennessee, Knoxville. REFERENCES ATKINS, C. P. & KENT, R. L. (988) What do recruiters consider important during the employment interview? Journal of Employment Counseling, 5, pp BANTA, T. W. (988) Assessment as an instrument of state policy, in: BANTA, T. W. (Ed.) Implementing Outcomes Assessment: Promise and Perils. New Directions for Institutional Research, 59 (San Francisco, Jossey-Bass). BANTA, T. W., PHILLIPPI, R. H. & LYONS, W. (99, Winter) Critique of a method for surveying employers, AIR Professional File, No. 7. CASE, C. (986) Employer follow-up. Responses of employers of former telecommunication technology students (San Bruno, CA, Skyline College). ERIC Document No. ED897. CONKLIN, K. A. (990, May) Assessment of institutional effectiveness: Career student outcomes, paper presented at the 990 annual forum of the Association for Institutional Research, Louisville, KY. ERIC Document No. ED698. FRANCIS, J. B. & JONES, G., III (976) Education and employment. Comprehensive follow-up study of

10 Assessing Employer Satisfaction two-year college graduates in New York state. Phase II (Buffalo, State University of New York, Department of Higher Education). ERIC Document No. ED567. GELL, R. L. & JONES, R. F. (975) The employers II: A survey of employers who have hired career program graduates of Montgomery Community College (Rockville, MD, Montgomery College, Office of Institutional Research). ERIC Document No. EDI 997. GELL, R. L., JONES, R. F. & LOWE, A. M. (976) The employers III: A survey of employers who have hired career program graduates of Montgomery Community College (Rockville, MD, Montgomery College, Office of Institutional Research). ERIC Document No. ED8050. HEAD, R. B. (990) Employer survey results for the PVCC graduating class of , Research Report No. -90 (Charlottesville, VA, Piedmont Virginia Community College, Office of Institutional Research and Planning). ERIC Document No. ED78. ISBELL, F. T. & JONAS, S. (976) Engineering technologies program evaluation (Elyria, OH, Lorain County Community College). ERIC Document No. ED8595. KAPRAUN, E. D. & NIENKAMP, R. L. (985) Employer follow-up survey: Employer assessment of 98-8 Forest Park graduates, Final report (Forest Park, MO, Saint Louis Community College). ERIC Document No. ED KNOBLAUCH, W. A. & GERMAN, G. A. (989) Survey of firms/agencies employing Cornell graduates with bachelors degrees in applied economics and business management, Cornell Agricultural Economics Staff Paper No (State University of New York, Ithaca, Agricultural Experiment Station at Cornell University). ERIC Document No. ED77, published as work-in-series at Cornell University. LUCAS, J. A. (98) Follow-up of occupational students enrolled at Harper College, 98-98, () (Palatine, IL, William Rainey Harper College, Office of Planning and Research). ERIC Document No. ED 500. MARYLAND STATE BOARD FOR COMMUNITY COLLEGES (986) Employer evaluation of Maryland community college graduates. Results for four surveys: 978, 980, 98, and 98 graduates (Annapolis, MD, Maryland State Board for Community Colleges). ERIC Document No. ED757. MONTEMAYOR, J. J., HARDIN, K. & REED, J. F. (986) Employer follow-up, Spring 986, Research report (Glendale, AZ, Glendale Community College, Office of Research and Development). ERIC Document No. ED8588. UNIVERSITY OF HAWAII (977). A proposal for surveying the employers of community college graduates [and] survey of employers of community college graduates, Student Flow Project, report no. [and] (Honolulu, HI, Office of the Chancellor for Community Colleges, Educational Services Division, University of Hawaii). ERIC Document No. ED85. Appendix. The University of Tennessee, Knoxville Employer Survey In this questionnaire, you will be asked about the job performance of a particular employee, the UT Knoxville graduate whose name is listed in the accompanying letter. Again, this employee has given us permission to seek this evaluation. Please be assured that in no way will your identity or that of the employee be reported in connection with answers you provide here.. Overall, how satisfied do you think this employee is with the type of work he/she is doing on the job? Very Dissatisfied Dissatisfied Satisfied Very Satisfied. Overall, how satisfied do you think this employee is with his/her pay? Very Dissatisfied Dissatisfied Satisfied Very Satisfied. Overall, which of the following best describes this employee's job performance? Poor Fair Good Excellent. Would you hire this employee again for the same position, now that you have worked with him/her? Definitely Not Probably Not Probably Definitely

11 R. H. Phillippi & T. W. Banta 5. Would you recommend this employee for a promotion, now that you have worked with him/her? Definitely Not Probably Not Probably Definitely 6. Overall, how do you think this employee's general education (obtained in courses in social science, science, math, and English) compares with that of other employees in comparable positions? Poor Fair Good Excellent 5 No Opinion 7. Overall, how does the employee's quality of education in his/her area of specialization (or major) compare with that of other employees in comparable positions? Poor Fair Good Excellent 5 His/Her Major Not Relevant in This Job 8. Overall, to what extent does this employee possess the characteristics you would expect from a college graduate? Does Not Meet My Expectations Meets My Expectations Exceeds My Expectations 9. Please assess in two ways the following traits in reference to the position held by this employee. How important is each trait to successful job performance? Compared to others, please rate this employee's performance. NOT IMPORTANT a. Planning projects b. Speaking effectively c. Writing effectively d. Reading effectively e. Listening effectively f. Defining problems g. Solving problems h. Understanding written information i. Processing and interpreting numerical data j. Working with computers k. Applying job-related technical skills l. Thinking creatively m. Working cooperatively n. Leading others o. Adjusting to new job demands p. Being dependable and on-time q. Working under pressure r. Working with minimal supervision s. Handling several tasks at once t. Making decisions under pressure u. Working with people from diverse backgrounds SOMEWHAT IMPORTANT VERY IMPORTANT POOR FAIR GOOD EXCELLENT

12 Assessing Employer Satisfaction 0. How many people do you supervise directly?. How long have you supervised this employee?_. Did you attend the University of Tennessee, Knoxville? No Attended, But Did Not Graduate Graduated From UTK. What is your gender? Male Female Thank you very much for assisting us. If you would like to add any comments, please use the space below. Please return your completed questionnaire in the enclosed postage-paid envelope. Appendix. The University of Tennessee, Knoxville Employer Survey In this questionnaire, you will be asked about the job performance of a particular employee the UT Knoxville graduate whose name is listed in the accompanying letter. Again, this employee has given us permission to seek this evaluation. Please be assured that in no way will your identity or that of the employee be reported in connection with answers you provide here. Please circle the number of your response.. Overall, how satisfied do you think this employee is with the type of work he/she is doing on the job? Very Dissatisfied Dissatisfied Satisfied Very Satisfied. Overall, how satisfied do you think this employee is with his/her pay? Very Dissatisfied Dissatisfied Satisfied Very Satisfied. Overall, which of the following best describes this employee's job performance? Poor Fair Good Excellent. Would you hire this employee again for the same position, now that you have worked with him/her? Definitely Not Probably Not Probably Definitely 5. Would you recommend this employee for a promotion, now that you have worked with him/her? Definitely Not Probably Not Probably Definitely 6. Overall, how do you think this employee's general education (obtained in courses in social science, science, math, and English) compares with that of other employees in comparable positions? Poor Fair Good Excellent 5 No Opinion 7. Overall, how does the employee's quality of education in his/her area of specialization (or major) compare with that of other employees in comparable positions? Poor Fair Good Excellent 5 His/Her Major Not Relevant in This Job 8. Overall, to what extent does this employee possess the characteristics you would expect from a college graduate? Does Not Meet My Expectations Meets My Expectations Exceeds My Expectations 9. Please assess the job skills and personal characteristics listed on this and the following page in reference to the position held by this employee. In the column marked importance assess how important each skill or trait is to successful job performance in this employee's position, and in the column marked performance assess how good this employee's performance is compared to others holding the same type of job. In each column, you have a total of 00 points to allocate among 0 job skills on this page and 0 personal characteristics on the following page. The points you choose to assign to each skill or characteristic represent its relative importance to the job or the relative performance of the employee. For example, if an item reflects something that is very important to the job, you may want to assign it a score of 0 or 0. An item that is twice as important as another should receive twice as many points. You may assign 0 points to an item, or you may (although this is highly unlikely) assign 00 points. The total number of points in each column must total 00.

13 R. H. Phillippi & T. W. Banta BEFORE ASSIGNING POINTS WITHIN A COLUMN, PLEASE READ ALL OF THE ITEMS TO BE RATED. JOB SKILLS IMPORTANCE TO PERFORMANCE OF a. Planning projects b. Speaking effectively c. Writing effectively d. Reading effectively e. Listening effectively f. Defining problems g. Solving problems h. Understanding written information i. Processing and interpreting numerical data j. Working with computers JOB PERFORMANCE (Allocate 00 Points) THIS EMPLOYEE (Allocate 00 Points) TOTAL POINTS Please follow the same procedures for rating Personal Characteristics as you did for rating Job Skills. JOB SKILLS a. Applying job-related technical skills b. Working cooperatively c. Leading others d. Adjusting to new job demands e. Being dependable and on-time f. Working- under pressure g. Working with minimal supervision h. Handling several tasks at once i. Making decisions under pressure j. Working with people from diverse backgrounds IMPORTANCE TO JOB PERFORMANCE (Allocate 00 Points) PERFORMANCE OF THIS EMPLOYEE (Allocate 00 Points) TOTAL POINTS How many people do you supervise directly?. How long have you supervised this employee?. Did you attend the University of Tennessee, Knoxville? No Attended, But Did Not Graduate Graduated From UTK. What is your gender? Male Female Thank you very much for assisting us. If you would like to add any comments, please use the space below or a separate sheet of paper. Please return your completed questionnaire in the enclosed postage-paid envelope.