Service Ability Inventory. Telephone Communications. Administrator s Manual S.A.I.-T.C. Developed by J. M. Llobet, Ph.D.

Transcription

1 Service Ability Inventory S.A.I.-T.C. Telephone Communications Developed by J. M. Llobet, Ph.D. Administrator s Manual

2 Table of Contents G. Neil Testing Products: An Investment in Your Company s Future Use of Testing Products As Tools Legal Aspects of Test Use and Administration Testing Products and Adverse Impact Federal Laws Title VII The Americans with Disabilities Act Record-Keeping Requirements State and Local Laws Test Selection and Follow-Up Procedures Selection Monitoring Validation Scoring The Importance of Telephone Communication Skills in the Workplace Selecting Applicants with Effective Telephone Skills Determining Employee Telephone Skill Levels Description of the S.A.I. T.C Using Job Analysis to Justify Use of Test Interpretation and Use of Test Scores Norms Developing Company-Specific Norms Discussing Results of the S.A.I. T.C Service Ability Inventory Telephone Communications i

3 Table of Contents (cont.) Validity and Reliability Validity Study # Validity Study # Reliability Adverse Impact To ensure that you are obtaining the full benefits available to you from the use of G. Neil HR Assessments products, please read all information contained in this manual carefully. By using this testing product, you are acknowledging that you have read and understand the general guidelines provided in this manual, and that if you have any specific questions, you have referred them to a competent testing and/or legal expert for advice. G. Neil and the test developer do not accept liability for any unlawful use of this product. ii

4 G. Neil Testing Products: An Investment in Your Company s Future The decision to use testing products in the employment process is one that can be very beneficial to your company in many ways. A well-designed, properly validated test, when used in conjunction with other employment screening tools, can save your company from investing training resources in an applicant who is not suited to perform the job for which he or she was hired, and, as a consequence, can help protect your company from negligent hiring lawsuits. Each G. Neil test has been researched and developed by our own in-house staff of testing professionals, which includes experienced industrial psychologists and attorneys. Use of Testing Products As Tools Validity studies of the testing products we offer have shown them to be predictive of job performance and therefore quite useful during the selection process. It is important to remember that tests should be used in conjunction with other, equally important employment screening tools such as criminal background checks, work histories and employer references to present a balanced picture of the particular job candidate. Only when used in coordination with one another will you be able to truly determine a fit between the candidate and the particular job for which he or she is applying. Employment tests, as defined in this manual, can be of several different varieties, including trustworthiness or integrity tests, skills-oriented tests and personality tests. Each test can focus on one of these elements, or may include several different components, testing a variety of factors. Choosing the proper test product for your needs is a key factor in making your selection process more effective. Legal Aspects of Test Use and Administration Although employment tests have been in use for more than 40 years, their use became more prevalent after the passage of the Employee Polygraph Protection Act (EPPA) of 1988, which made it illegal for most private employers to use polygraph examinations as a routine pre-employment screening tool. Employment tests that are not prohibited by the EPPA are designed to give the employer a legal way to gauge an employee s job-related skills and personality traits as an alternative to the polygraph test. Whereas the polygraph test is designed to monitor an applicant s physiological reaction to certain questions, the employment tests seek to gain information on the job candidate through a series of questions designed to measure certain job-related attributes. Today, the use of employment tests continues to increase. Many of the country s largest corporations use these types of screening devices on a regular basis and have found great success in using them to hire and promote the best candidates. Testing Products and Adverse Impact A common misperception of these tests is that they all tend to discriminate against certain classes of applicants, in violation of state and federal laws against discrimination in employment decisions. In fact, this is not the case. While there is evidence of poorer test performance by some members of protected classes on some skills tests that include language and mathematical components, the use of such tests is still justified as long as the skills assessed by the tests are essential for the successful performance of one or more of the job s key functions. In addition, researchers have found no evidence that well-constructed personality tests discriminate on any unlawful basis. Service Ability Inventory Telephone Communications 1

5 However, it is incumbent upon employers who use test products to constantly monitor selection procedures to ensure that no adverse impact is occurring in the overall selection process. Adverse impact is defined as a situation in which there is a substantially different rate of selection in hiring, promoting or other employment decisions that works to the disadvantage of members of a race, sex or ethnic group. If adverse impact does occur, the employer needs to be able to demonstrate the job relatedness of the selection process. For further guidance in this area, read the Test Selection and Follow-Up Procedures section of this manual. Federal Laws There are federal laws and regulations governing the use of selection tools, such as employment tests, insofar as they have any adverse impact on the employment opportunities of protected classes of individuals. Some of the more subtle aspects of these laws as they apply to the selection process are discussed in the section of this manual entitled Using Job Analysis to Justify Use of Test and Its Sections (Legal Implications). Title VII Title VII of the Civil Rights Act of 1964 (Title VII), covering employers with 15 or more employees, prohibits discrimination in employment decisions on the basis of race, sex, color, religion and national origin. Title VII authorizes the use of any professionally developed ability test provided that such test, its administration or action upon the results is not designed, intended or used to discriminate on any unlawful basis. In 1971, the United States Supreme Court, in Griggs v. Duke Power Co. (401 U.S. 424), adopted the standard that employer practices that had an adverse impact on minorities and were not justified by a business necessity violated Title VII. Congress amended Title VII in 1972, adopting this legal standard. As a result of these developments, the government sought to produce a unified governmental standard on the regulation of employee selection procedures because the separate government agencies had enforcement powers over private employers and each used different standards. This resulted in the adoption of the Uniform Guidelines on Employee Selection Procedures (Guidelines), codified at 29 CFR Part 1607, which establishes a uniform federal position in the area of prohibiting discrimination in employment practices on the grounds of race, color, religion, sex or national origin, and applies to all public and private employers covered by Title VII, Executive Order 11246, the State and Local Fiscal Assistance Act of 1972, the Omnibus Crime Control and Safe Streets Act of 1968, and the Intergovernmental Personnel Act of

6 Highlights of the Guidelines include: Provision of a uniform set of principles governing use of employee selection procedures that is consistent with applicable legal standards. Setting out validation standards for employee selection procedures generally accepted by the psychological profession. The Guidelines do not require a validation of the selection device unless there exists evidence of adverse impact. It is important to note also that compliance with the Guidelines does not remove the affirmative action obligations for test users, including federal contractors and subcontractors. The Americans with Disabilities Act The Americans with Disabilities Act (ADA) provides that an employer shall not conduct a medical examination or make inquiries of a job applicant as to whether such applicant is an individual with a disability or as to the nature or severity of such disability (42 USC Sec (d)(2)(A); see also 29 CFR Sec ). Inquiries into a person s disabilities are prohibited at the pre-offer of employment stage, except in a very narrowly defined situation where the applicant has voluntarily disclosed a medical condition requiring accommodation. The ADA protects disabilities, not a characteristic that an employer may consider to be a personal flaw or undesirable aspect of an applicant s personality. The ADA does not prohibit inquiries into such personality attributes as propensity for honesty, ability to get along with others, organizational skills or management skills, to name a few examples. No question or series of questions designed to elicit information about a person s mental impairment (as defined by the ADA), or questions that would even tend to elicit such information, should appear on a testing product. Each of G. Neil s testing products has been carefully reviewed under this standard, in order to avoid any conflict with the ADA guidelines. Record-Keeping Requirements Various federal laws require employers to retain tests and test results for at least one year from the date the test is administered or from the date of any personnel action relating to the testing, whichever is later. State and Local Laws Due to the wide variety, complexity and ever-changing nature of state laws, it is impossible to summarize each state s requirements in this brief overview. If you are unfamiliar with the state and local laws governing the use of screening devices applicable in your locale, consult with a qualified labor law attorney or testing specialist who may provide competent guidance on this topic. Service Ability Inventory Telephone Communications 3

7 Test Selection and Follow-Up Procedures Selection Generally, when selecting a test or any other selection tool, you should choose one that has been specifically designed to measure the skills or traits necessary for the position in question. It is recommended that a thorough job analysis be performed to determine the links between job functions and the attributes the testing product is designed to measure. Monitoring Monitor your selection process to ensure compliance with all applicable federal, state and local laws, checking your selection process for evidence of adverse impact. This should be conducted on a continual basis. G. Neil testing products include testing logs that can be used to record each test taker s scores, as well as other important data that may be used to compute your own test norms and adverse-impact statistics. Validation Should your monitoring results indicate that adverse impact is occurring in the selection procedures, you should determine in which component of the selection process this is happening. If the use of a certain testing product is found to be the cause, you will need to conduct a validation study of the test. Qualified testing professionals may be contacted to help in conducting a validity study. These professionals will be able to help determine if the test is the cause of the adverse impact and whether or not the test is focusing on a bona fide occupational qualification for the job. In some instances, tests that in some contexts may be considered discriminatory may be lawful to use in others, as long as the test is focusing on a bona fide occupational qualification. Scoring Cut-offs and suggested pass or fail scores are not provided with these tests. Instead, norms and, in some instances, average test scores for various levels of job performance are provided. This information is provided for the elements the test is designed to measure. This information is a result of the testing universe used in the validation studies performed by G. Neil and is for demonstrative purposes only. Test results should always be interpreted, along with other information gathered through your selection process, to ensure that you get a complete picture of the job candidate or employee. It is recommended that you administer the test to your current employees so that you may develop your own company-specific norms for test performance. These norms can then be used as benchmarks during your testing and selection process. 4

8 The Importance of Telephone Communication Skills in the Workplace Not only is the telephone the most vital link between your company and the rest of the world, it also becomes your good will ambassador, every time it rings. That s why telephone communication skills are so important for all staff members, from your department managers to the part-time receptionist. Employees with effective telephone communication skills have the power to: Increase customer satisfaction when handling questions or complaints Drive up sales when customers phone in orders Create lasting business relationships with clients and vendors Companies that identify and hire employees with successful telephone skills ensure a high degree of overall success. Employees who are helpful and personable, even under stressful conditions, build your company s image and cultivate customer loyalty. Consequently, telephone communication skills are especially critical for customer sales and service jobs such as account representatives, customer service, receptionists, secretaries, and telemarketing. The Service Ability Inventory for Telephone Communications (S.A.I. T.C.) measures four personality areas critical to successful telephone operations. 1. Customer Relations measures people skills which tell how effectively your applicants will handle service and sales opportunities and customer complaints, and how pleasantly they deal with people. 2. Stress Management predicts whether applicants will be flexible and patient during high-stress, demanding times such as peak call periods and when speaking with difficult customers. 3. Helping Disposition shows how willing your applicants will be to assist your customers and clients and how responsive they will be to changing situations. 4. Team Skills shows how well your applicants will cooperate with others to solve problems and work toward a common goal. Selecting Applicants with Effective Telephone Skills Some individuals are more service-oriented than others, especially when using the telephone and are, therefore, more suited for positions requiring a high degree of telephone communication. The S.A.I. T.C. provides you with the opportunity to objectively assess the telephone service orientation of job applicants. Research has consistently shown that the S.A.I. T.C. is an accurate predictor of an individual s service orientation (See Validity and Reliability section). Incorporating this assessment into your hiring process should significantly increase the accuracy of your hiring decisions and, therefore, provide an effective and cost-efficient means of building your customer-oriented team. Service Ability Inventory Telephone Communications 5

9 Determining Employee Telephone Skill Levels In addition to assisting you in hiring telephone service-oriented individuals, the S.A.I. T.C. can also help you identify your current employees service skill levels. The S.A.I. T.C. may be administered to current employees whose jobs require telephone service skills. Based upon the test scores, you can identify strengths and weaknesses within your current employee group. Those employees scoring low on the S.A.I. T.C. could be offered coaching or training to help increase service awareness and help develop telephone service skills. With minimal cost and effort, the S.A.I. T.C. helps you select telephone service-oriented job applicants as well as identify the strengths and weaknesses of your current workforce in this area. 6

10 Description of the S.A.I. T.C. The S.A.I. T.C. provides a reliable measurement of an individual s ability to be serviceoriented, particularly in those jobs that require interaction with others using the telephone. The test consists of 40 questions concentrating in the areas of customer relations, stress management, helping disposition and team orientation. Although the test is untimed, most people complete it in less than 30 minutes. The questions that make up the S.A.I. T.C. were developed based on information gathered from a review of training materials concentrating on proper telephone etiquette, an extensive review of the psychological literature that focuses on service orientation and the prediction of job performance, and from interviews with human resource professionals, managers of call centers, and telephone operators (e.g., customer service representatives, outbound telemarketing representatives, secretaries, and receptionists). Each question was written specifically with the employment setting in mind, unlike most other clinically based personality assessment instruments. Tests designed specifically for the employment setting are viewed by applicants as more job-relevant (face valid) than those developed for clinical assessment and are, therefore, less likely to be questioned in terms of their relevance to the position in question. Service Ability Inventory Telephone Communications 7

11 Using Job Analysis to Justify Use of Test and Its Sections (Legal Implications) From a legal standpoint, if a test is to be used for selection or promotion purposes, it is important that users of the test take the necessary steps to establish a clear linkage between the job tasks and the occupational environments measured by the test. This relevancy should exist to meet the principles outlined in the Uniform Guidelines on Employee Selection Procedures (1978) and other federal government employment-related legislation, such as Title VII of the Civil Rights Act of 1964, the Civil Rights Act of 1991, and the American with Disabilities Act of The tasks that are crucial or essential to the job in question should first be identified. Then, the occupational environment that matches the job in question can be determined. This process should be carefully documented to justify the appropriateness of the test administered in the employee selection process. The following are examples that indicate the relationship between job tasks or behaviors that require one or more of the characteristics assessed by the S.A.T. T.C. Task S.A.I. T.C. Provides personalized service to each customer. Attends to the needs of customers ensuring their satisfaction. Handles multiple tasks in an efficient and appropriate manner. Answers customer calls in a courteous and professional manner. Works well with others to achieve a common goal. Customer Relations Helping Disposition Stress Management Customer Relations Team Skills 8

12 As a general guideline for compliance with federal discrimination and disability laws, test users should not subject test takers to any adverse employment decision based on a test result, unless the test result and other factors considered in the decision-making process reveal that the person does not possess qualifications that are crucial or essential to the job in question. To illustrate, if a test taker performs poorly on a test section designed to measure inspection skills, and inspection skills are not crucial or essential to the position for which the test taker is being considered, the test result should not serve as a basis for excluding the test taker from the position. Similarly, if a test result indicates that a test taker is unable to perform certain physical tasks that are not crucial or essential to the job position at issue, the test taker should not be excluded from that position on the basis of the test result. 1 Test users can avoid this type of scenario altogether by carefully identifying the tasks that are essential to the job position at issue, and administering only those tests or test sections that are appropriate and relevant to the position's requirements. Tests or test sections measuring proficiency in the English language also should be administered in accordance with these principles. Thus, if spelling, grammar, vocabulary, or reading comprehension skills are not essential to a job position, a test taker should not be subjected to an adverse employment decision based on poor test results in those areas. Requiring employees or applicants to be fluent in English may constitute national origin discrimination in violation Title VII of the Civil Rights Act if the requirement is not justified by business necessity or directly related to job performance. There are some limited exceptions to this rule for jobs involving dangerous work requiring a heightened understanding of written or verbal safety instructions in English, or service positions that require significant communication in English with the public. Test users should consult with an attorney before subjecting any test taker to an adverse employment decision on the basis of English language deficiencies. 1 If the test taker's ability to perform a particular physical task is essential to the job position at issue, the Americans with Disabilities Act may require the test user to provide certain accommodations to facilitate the test taker's performance of the task at issue. Test users should consult an attorney before making any adverse employment decision based upon a test taker's physical inability to perform a task measured by a test result. Service Ability Inventory Telephone Communications 9

13 Interpretation and Use of Test Scores To help you hire the best individual for your organization, the S.A.I. T.C. scores should be used in conjunction with other applicant information (e.g., the applicant s work history, references, and skills assessments). A high test score indicates that the applicant/employee is likely to demonstrate behaviors indicative of a service-orientated individual. The S.A.I. T.C. questions cover four important, service-related areas. Each area is defined below: Customer Relations: The degree to which a person deals effectively with customers, showing empathy when appropriate and making every reasonable effort to satisfy the customer. Stress Management: The ability to work effectively under stressful, work-related situations. Helping Disposition: The extent to which an individual is willing to help others. Team Orientation: The ability to work well with others to achieve a common goal. When interpreting scores, examining each section score may reveal the applicant s strengths or potential weaknesses. 10

14 The bar graph below presents the average S.A.I. T.C. total scores (i.e., adding the response values of the four test sections together) by job performance level for employees who participated in the validity studies presented in this manual. The results presented here indicate that, in general, the higher the S.A.I. T.C. total score, the likelier it is that an individual is a service-oriented professional (i.e., the individual has excellent relations with customers, has good interpersonal skills, manages stress effectively, is flexible, has a helping disposition, and is team oriented). The lower the score, the less likely it is that the individual possesses these characteristics. Average S.A.I. T.C. Test Score by On-The-Job Service Level Average S.A.I. T.C. Test Score Low Average On-The-Job Service Level High Norms When evaluating applicants/employees, norms provide a point of reference regarding the relative test score of each applicant/employee. Norms are the average scores or distribution of scores obtained from the study sample. These score patterns can be compared to your own applicants /employees test scores to better define their performance on the S.A.I. T.C. Tables 1 to 5 present the distribution of scores for each test section and the associated percentile rank for the employees who have participated in S.A.I. T.C. validity and norm studies. The percentile rank is the percentage of applicants/employees in the sample who obtained scores lower than the corresponding test score. For example, when reviewing Table 1, it can be said that an applicant/employee obtaining a score of 155 scored in the 87th percentile. This means the applicant/employee scored higher than 87% of the applicants/employees in the norm sample. Service Ability Inventory Telephone Communications 11

15 Table 1 S.A.I. T.C. Total Score Raw Score Corresponding Percentile Continued on next page 12

16 Table 1 (cont.) S.A.I. T.C. Total Score Raw Score Corresponding Percentile or less 1 Average Score 139 Standard Deviation Number of Participants 163 Service Ability Inventory Telephone Communications 13

17 Table 2 Customer Relations Section Raw Score Corresponding Percentile or less 1 Average Score 47 Standard Deviation 5.50 Number of Participants

18 Table 3 Stress Management Section Raw Score Corresponding Percentile or less 1 Average Score 36 Standard Deviation 5.01 Number of Participants 176 Service Ability Inventory Telephone Communications 15

19 Table 4 Helping Disposition Section Raw Score Corresponding Percentile or less 1 Average Score 28 Standard Deviation 4.15 Number of Participants

20 Table 5 Team Skills Section Raw Score Corresponding Percentile or less 1 Average Score 29 Standard Deviation 3.55 Number of Participants 176 Service Ability Inventory Telephone Communications 17

21 Developing Company-Specific Norms You can use the information in Tables 1 to 5 and the bar graph in this section as a guide when evaluating job candidates; however, it is strongly recommended that you collect and validate your own test data. The applicant/employee pool in your organization may differ from the study sample presented in this manual. Factors such as geographic location, business type and job responsibilities may have a significant effect on test scores. One way to develop your own norms and benchmarks is to administer the S.A.I. T.C. to your current employees. This will allow you to compare the scores of your top performers with those of your less productive employees. The information can then serve as a guide during your applicant evaluation process. In addition, if you can establish and document that, in general, high scorers on the test are also your more service-oriented employees, this can serve as an initial step in establishing the validity of the S.A.I. T.C. within your organization. If you do administer the S.A.I. T.C. to your employees for the purpose of establishing company-specific norms, make sure your employees understand that the results of your study will be used for norm development only and that their employment status will in no way be affected by their scores. The EEOC and the Uniform Guidelines on Employee Selection Procedures recommend that job analyses be performed in conjunction with validation studies to determine the jobrelatedness of each test and other selection tools used throughout the hiring process. It is the employer s responsibility to periodically monitor its employment screening process to ensure that it is fair and valid. 18

22 Discussing Results of the Service Ability Inventory for Telephone Communications Your company should develop a procedure so that the applicant can be told what the next step in the hiring process is, regardless of his/her score on the S.A.I. T.C. or any other assessment tool. Emphasize that the S.A.I. T.C. is only one of the criteria used to determine if the applicant is a good match for the position. Remind the applicant that there are many people applying for the same position and that each applicant will be considered based on how all of his/her qualifications and experience match the position s requirements. Some interviewers may be tempted to look for a quick or easy reason to tell the applicant why he/she was not selected. Blaming a test may seem like a plausible reason, but it is no comfort to the rejected applicant and should not occur. The fact is, the reason to hire or not to hire should never be based solely on any single test score. It is the interviewer s responsibility to review all of the information gathered from the various tools used during the hiring process such as the job application, the interview, reference checks and other tests to form the decision on the applicant s appropriateness for the position. The issue is, and should always be, whether there is an appropriate job fit between job and applicant. Using the S.A.I. T.C. is only one part of the information you need to make a decision. The other important part is knowing what else is required and desired in the employee filling the position and effectively using all the sources available to you to make the best decision. This will ensure an effective selection process that offers a more comprehensive view of the applicant and results in hiring the best employee for your organization. The employer assumes full responsibility for the proper use of the S.A.I. T.C. as mentioned in this manual. This includes establishing its job-relatedness to the position in question. If you have any questions about the proper use of employment tests, contact G. Neil or an employment testing specialist. Service Ability Inventory Telephone Communications 19

23 Validity and Reliability Effective applicant/employee evaluation procedures need to be valid and reliable. Validity can be defined as the extent to which the instrument measures what it is supposed to measure. In other words, validity can be conceptualized as whether or not there is a relationship between test scores and job performance. Reliability refers to how consistent the test is at measuring what it is supposed to measure. The research studies described next have been conducted to demonstrate the validity and reliability of the S.A.I. T.C. The validation method used to examine the S.A.I. T.C. s predictive validity is known as concurrent, criterion-related validation methodology. A professionally conducted concurrent, criterion-related validation study is an appropriate means of test validation, as described by the federal government s Uniform Guidelines on Employee Selection Procedures. Essentially, this approach requires that the test be administered to current employees and, concurrently, data on the performance of these employees be gathered. If the test is valid, one would expect a statistically significant correlation between individual test scores and job performance. In other words, those employees scoring high on the test would be those who also perform best on the job; those who do poorly on the test would be those likely to receive poor performance evaluations. 20

24 Validity Study #1 The S.A.I. T.C. was administered to 85 customer service representatives from a national direct marketing company. The study participants job responsibilities included handling telephone product orders and assisting customers with product or service-related questions. The supervisors of these employees were asked to rate the study participants on six work-related behaviors. These six work behaviors are presented below: Very Very Low Level Average Level High Level 1. Customer Relations Deals effectively with customers. Makes every effort to ensure customer satisfaction. Is well-liked by customers. 2. Interpersonal Skills Gets along well with others. Has great people skills Stress Management Works effectively under stressful work-related situations. Demonstrates patience and stress tolerance during times of conflict with customers and work in general Flexibility Adapts well to change. Has little trouble reprioritizing tasks when necessary Helping Disposition Is more than willing to help staff, coworkers and/or customers. Goes out of his/her way to help those in need Team Player Works well with others to achieve a common goal. Cooperates in all phases of work relationships Factor analysis of these six dimensions resulted in one service-related factor. Therefore, an overall performance rating was calculated for each study participant by adding the performance ratings of these six dimensions and dividing by six. This average rating will be referred to as Overall Job Performance in the following tables. Table 6 provides the significant correlations between the S.A.I. T.C. section scores, total score, and the performance ratings collected. As a point of reference, the average correlation coefficient for the standard interview has been found to be Hunter, J. E. & Hunter, R. F. (1984). Validity and Utility of Alternative Predictors of Job Performance. Psychological Bulletin, 96, Service Ability Inventory Telephone Communications 21

25 Table 6 Correlation Between the S.A.I. T.C. Scores and Work-Related Performance Validity Study #1 S.A.I. T.C. Work-Related Validity Significance Section Behavior Coefficient Level Customer Relations Customer Relations.24 p< Interpersonal Skills.22 p< Helping Disposition.29 p< Team Skills.34 p< Overall Job Performance.27 p< Stress Management Interpersonal Skills.24 p< Flexibility.23 p< Helping Disposition.36 p< Team Skills.29 p< Overall Job Performance.29 p< Helping Disposition Customer Relations.23 p< Interpersonal Skills.24 p< Stress Management.21 p< Flexibility.34 p< Helping Disposition.39 p< Team Skills.37 p< Overall Job Performance.35 p< Team Skills Helping Disposition.25 p< Team Skills.25 p< Overall Job Performance.21 p<.061* 82 Total S.A.I. T.C. Customer Relations.24 p< Score Interpersonal Skills.27 p< Flexibility.25 p< Helping Disposition.40 p< Team Skills.39 p< Overall Job Performance.34 p< N Note: N equals the number of participants in the analysis. *Correlation coefficient approaches.05 level of statistical significance. These correlations indicated that, in general, those individuals who scored high on the four sections that make up the S.A.I. T.C. received high ratings by their supervisors on numerous, important job-related behaviors, as well as on overall job performance. Those employees who scored low on the test received lower performance ratings. 22

26 Validity Study #2 In a separate study similar to Validity Study #1, the S.A.I. T.C. was administered to 92 employees from two separate companies. One company was a Fortune 500 insurance company. Study participants from this organization were customer service representatives whose primary job function was to handle insurance claim telephone calls. The other company was a collections agency. The jobs included in the study for the collections agency were credit consultant and recovery specialist. Supervisory ratings of on-the-job performance were collected for each study participant using the same rating form as the one used in Validity Study #1. Again, the study participants scores were statistically compared to the supervisors performance ratings using correlation analysis. Table 7 presents the significant correlation coefficients between test scores and performance ratings. Table 7 Correlation Between the S.A.I. T.C. Scores and Work-Related Performance Validity Study #2 S.A.I. T.C. Work-Related Validity Significance Section Behavior Coefficient Level Customer Relations Stress Management.36 p< Flexibility.25 p< Helping Disposition.34 p< Team Skills.28 p< Overall Job Performance.32 p< Stress Management Interpersonal Skills.22 p< Stress Management.31 p< Overall Job Performance.22 p< Helping Disposition Helping Disposition.27 p< Team Skills Customer Relations.23 p< Interpersonal Skills.22 p< Helping Disposition.24 p< Overall Job Performance.24 p< Total S.A.I. T.C. Stress Management.35 p< Score Flexibility.24 p< Helping Disposition.33 p< Team Skills.25 p< Overall Job Performance.32 p< N Note: N equals the number of participants in the analysis. The results obtained in Validity Study #2 were consistent with Validity Study #1. That is, in general, those individuals who scored high on the four sections of the S.A.I. T.C. were rated high by their supervisors on numerous, important job-related behaviors, including overall job performance. Those individuals who scored low on the test were rated low on performance by their supervisors. Service Ability Inventory Telephone Communications 23

27 Reliability In addition to validity research, studies have been conducted to assess the internal consistency or reliability for the S.A.I. T.C. That is, to what degree do the items in the test measure the same construct. The reliability coefficient obtained in this effort was.86 which indicates a high level of test-item consistency. The results of the validity and reliability studies presented in this manual offer strong evidence that the S.A.I. T.C. is a valid and reliable predictor of service-related behaviors. Once you have established the requirements of the job, incorporating the S.A.I. T.C. into your selection process should help you find the best-person job match. Understanding an applicant s personality and how it relates to the job in question is critical to finding the right fit and enhancing the effectiveness of your selection process. 24

28 Adverse Impact To determine if the S.A.I. T.C. could have an adverse effect on members of a protected class (e.g., minorities), the average scores for 41 African American and 103 white study participants were examined. The results of this comparison revealed that the average score obtained for these two groups was identical (i.e., 139). This result suggests that if this test were used for selection purposes, it is unlikely that there would be any adverse impact. This notion is consistent with the review of the personality testing literature that concludes: there is no evidence that well-constructed personality inventories systematically discriminate against any ethnic or national group (H. Hogan, J. Hogan & B. W. Roberts, 1996). 2 Even though the average score comparison discussed above suggests that the use of the S.A.I. T.C. would not be likely to have an adverse effect on the hiring rates of minorities versus non-minorities, it is always recommended that you periodically monitor your selection process to ensure that it continues to be fair and valid. Based on all of the validity, reliability and adverse-impact research presented in this manual, it appears that in addition to providing a sound, reliable and job-related basis for making employment decisions, the S.A.I. T.C. can also enhance equal employment opportunities by increasing the objectivity, standardization and job-relatedness of the selection process. To order the Service Ability Inventory for Telephone Communications or any other HR Assessments product, or if you have any questions, call toll-free or visit our website at 1 Hogan, H., Hogan, J. & B. W. Roberts (1996). Personality Measurement and Employment Decisions. American Psychologist, Vol. 51, No. 5, Service Ability Inventory Telephone Communications 12/00 25