Hiring is Testing. (Even If You re Not Calling It a Test ) Geoff Burcaw. Hilary Ricardo. Senior Consultant. Senior Consultant

Transcription

1 Hiring is Testing (Even If You re Not Calling It a Test ) Geoff Burcaw Senior Consultant Hilary Ricardo Senior Consultant

2 Hi! Introductions Geoff Burcaw Senior Consultant Hilary Ricardo Senior Consultant

3 Introduction Hiring is Testing (Even If You re Not Calling It a Test )

4 Poll Question Which of these are you using to hire personnel? MQ screening T&Es Interviews Multiple-choice exams Performance tests (Select all that apply.)

5 Agenda What Constitutes a Test? Common Selection Tools and Their Pitfalls Legal and Professional Standards

6 Agenda Potential Outcomes If Tests Do Not Meet Proper Standards How to Ensure Tests Adhere to Professional Standards and Laws Recommendations for Using Common Tests

7 What Constitutes a Test? An employment test is any procedure used to make an employment decision. - Richard Tonowski, EEOC Office of General Counsel A test is any method used to make an employment decision. - Uniform Guidelines on Employee Selection Procedures (1978)

8 What Constitutes a Test? Tests include traditional knowledge and ability tests as well as inventories, subjective procedures, and projective instruments. - US Department of Labor, Employment and Training Administration A test includes any investigation of character, personality, education, and experience. - CA Government Code, Title 2

9 Common Selection Tools and Their Pitfalls Minimum Qualifications (i.e., MQs) Screening/Resumé Screening Defined The amount and type of work experience, education, training, and/or licensure one is required to have to be considered for a job May be reviewed through an automatic screening process May include social media screening Pitfalls Can unintentionally discriminate and create disparate impact on protected groups (e.g., requiring a degree for a position that does not actually require it can be socioeconomic discrimination) Can cause unintentional bias MQs may be outdated (e.g., physical standards), irrelevant, or not created scientifically May screen out qualified candidates

10 Common Selection Tools and Their Pitfalls Training and Experience (i.e., T&E) Evaluation Defined Pitfalls Used to assess previous experience, education, and/or training (self-report, selfrating, narrative) Evaluated against benchmarks Easy to develop and may apply to multiple entry-level occupations Supplemental application is a narrativebased T&E Applicant inflation or distortion can threaten validity Self-inflators at the top of the list with more honest applicants at the bottom of the list May result in discrimination or screen out qualified candidates

11 Common Selection Tools and Their Pitfalls Interview Defined Pitfalls Used to assess domains/competencies May be used to assess job knowledge Often used at two different points in the hiring process (selection and hiring) Typically uses a standard set of questions; however, many hiring managers develop different questions for different candidates Lack of standardization Biases may be introduced (e.g., stereotyping, halo effect, contrast effect, similar-to-me effect) Unintentional discrimination may occur Hiring managers typically have a lack of understanding/training about proper interviewing

12 Common Selection Tools and Their Pitfalls Why are these so commonly used rather than more robust testing? Quicker and easier to develop and administer Lack of internal resources (e.g., staff, money, expertise) Determined by Civil Service Rules, CBAs/MOUs, policy, etc. Belief that there is nothing to test

13 Poll Question What challenges have you experienced? Lack of: Time Expertise Staff Money Organizational support (Select all that apply.) What other challenges have you experienced?

14 Legal and Professional Standards Overview Those involved in hiring personnel at any level should be familiar with all applicable federal and state laws and regulations. Federal laws include: Title VII of the Civil Rights Act of 1964 Age Discrimination in Employment Act of 1967 Americans with Disabilities Act of 1990 All assessment tools used to make employment decisions, regardless of their format, level of standardization, or objectivity, are subject to professional and legal standards. - U.S. Department of Labor Merit System Principles (5 USC 2301), Principle #1: Recruitment should be from qualified individuals from appropriate sources in an endeavor to achieve a workforce from all segments of society, and selection and advancement should be determined solely on the basis of relative ability, knowledge and skills, after fair and open competition which assures that all receive equal opportunity.

15 Legal and Professional Standards Uniform Guidelines on Employee Selection Procedures (Federal Register, 1978) Created by Equal Employment Opportunity Commission Department of Labor Department of Justice Civil Service Commission (OPM) Broadly adopted by state and local governments and civil service commissions (as well as courts, including the U.S. Supreme Court). Developed to assist employers in complying with civil rights laws related to the use of selection procedures A selection procedure is any procedure used to make a decision regarding hiring, promotion, demotion, referral, retention, licensing and certification, and selection for training or transfer. Requires evidence that a selection procedure be related to the job. A selection procedure may not meet this standard when Assessing KSAs that are not needed for performing the job at a minimallyacceptable level Requiring education or experience above that which is needed Assessing KSAs that can be learned on the job

16 Potential Outcomes If Tests Do Not Meet Proper Standards Loss of time and money Poor job fit High turnover Lawsuit

17 How to Ensure Tests Adhere to Professional Standards and Laws Overview Job Analysis Process Develop Items Requirements Determine Rating Leading Performance Scale Exam Plan Development Overview Job-Relatedness the test measures something important and needed at entry to the job Reliability the test produces consistent results over time Validity the degree to which a test actually measures what it claims to measure o Different types of validity o Content validity is most common

18 How to Ensure Tests Adhere to Professional Standards and Laws Overview Job Analysis Process Develop Items Requirements Determine Rating Leading Performance Scale Exam Plan Development Requirements A job analysis is needed to identify important work behaviors required for successful job performance while carrying out job-related tasks. Any selection tool/test must assess the KSAs needed for performing those important work behaviors

19 How to Ensure Tests Adhere to Professional Standards and Laws Overview Job Analysis Process Develop Items Requirements Determine Rating Leading Performance Scale Exam Plan Development Job Analysis Process 1. Literature review 2. Job Audits/Interviews with subject matter experts (SMEs) 3. Develop task and KSA statements with SMEs 4. Develop the task and KSA rating method 5. Collect task and KSA ratings 6. Data analysis 7. Assess the task/ksa relationships 8. Prepare a technical report

20 How to Ensure Tests Adhere to Professional Standards and Laws Overview Job Analysis Process Develop Items Requirements Determine Rating Leading Performance Scale Exam Plan Development Exam Plan Development 1. Gather a panel 2. Review the job analysis data 3. Discuss examination plan options 4. Weight the final selection process

21 How to Ensure Tests Adhere to Professional Standards and Laws Sample Selection Options Matrix Overview Develop Needs Items Determine Job Rating Leading Analysis Performance Process Scale Conclusion

22 Recommendations for Using Common Tests MQ/Resumé Screening More experience is not automatically better Use SMEs and job analysis results to help determine years and type of experience Train staff who review MQs to eliminate unintentional bias Work with SMEs to identify key phrases and words to input into computerized screener T&E Evaluation Avoid point-based selfratings Build in methods of reducing response distortion Suggest narrative supplemental applications and score using behaviorallyanchored rating scales (BARS) or other objective criteria All rating criteria should be based on tasks and KSAs and SME input Interview Utilize the job analysis results and SMEs to develop structured, standardized, and jobrelated questions for exam and hiring interviews Create benchmark objective scoring (BARS) with SMEs Conduct interviewer/ rater training for all personnel involved

23 Conclusion Any tool used to make an employment decision is considered a test. This includes common tools, such as MQ screening, T&Es, and interviews. Consider all testing options. Use for a specific purpose when warranted (reduce candidate counts) Don t rely on any one tool. Use a wholistic approach to selection. They can be developed better to be more effective, fair, and defensible. This can be achieved by conducting a job analysis, developing an exam plan, using SMEs, and choosing the appropriate testing method(s).

24 Thanks! Any Questions? Please contact us by