Developing, Validating, and Analyzing Training, Education & Experience (TEE) Requirements

Transcription

1 Developing, Validating, and Analyzing Training, Education & Experience (TEE) Requirements October 24, 2013 Presented by Jim Kuthy, Ph.D. Heather Patchell, M.A. Visit BCGi Online If you enjoy this webinar, Don t forget to check out our other training opportunities through the BCGi website. BCGi Standard Membership (free) Monthly webinars on EEO compliance topics EEO Insight Journal (e-copy) COMPare Lite (limited comp analysis software) BCGi Platinum (paid) Membership ($199/year) Includes validation/compensation analysis books Compensation Power Analysis Tool COMPare Lite (full license discounted) EEO Insight Journal (e-copy and hardcopy) Members only webinars and training and much more 2 HRCI Credit BCG is an HRCI Preferred Provider CE Credits are available for attending this webinar Only those who remain with us for at least 80% of the webinar will be eligible to receive the HRCI training completion form for CE submission 3

2 About Our Sponsor: BCG Assisted hundreds of clients with cases involving Equal Employment Opportunity (EEO) / Affirmative Action (AA) (both plaintiff and defense) Compensation Analyses / Test Development and Validation Published: Adverse Impact and Test Validation, 2 nd Ed., as a practical guide for HR professionals Editor & Publisher: EEO Insight an industry e-journal Creator and publisher of a variety of productivity Software/Web Tools: OPAC (Administrative Skills Testing) CritiCall (9-1-1 Dispatcher Testing) AutoAAP (Affirmative Action Software and Services) C 4 (Contact Center Employee Testing) Encounter (Video Situational Judgment Test) Adverse Impact Toolkit (free online at AutoGOJA (Automated Guidelines Oriented Job Analysis ) COMPare: Compensation Analysis in Excel 4 Developing, Validating, and Analyzing Training, Education & Experience (TEE) Requirements October 24, 2013 Presented by Jim Kuthy, Ph.D. Heather Patchell, M.A. This presentation is offered for information purposes only and should not be considered legal advice.

3 Contact Information Jim Kuthy, Ph.D. Principal Consultant (800) x 239 Heather Patchell, MA Consultant (800) x A Little About the Presenters Jim holds Masters and Doctorate Degrees in Industrial & Organizational Psychology Heather holds a Masters Degree in Psychology More than twenty-five combined years of experience in the employment selection field Designed and/or validated selection devices for dozens of employers, including conducting validation studies that have been successfully defended in court or passed review by federal agencies Jim has taught Psychology and Business-related courses at the University of Akron and California State University, Sacramento and Heather is the Director of the BCGi Institute for Workforce Development 8 What are Training, Education & Experience (TEE) Requirements? Training requirements posted by employers as hiring or promotion criteria. Certifications, credentials, training institution completions, licensing requirements Educational requirements, including: Degrees, specific course requirements, etc. Experience requirements used as hiring or promotion criteria, including: Closed systems (e.g., time-in-grade requirements) Open systems (inside/outside experience counts)

4 Why is this Topic Important to HR/EEO Professionals? Why do I need to know about this topic? TEE screens are both highly effective and typically overlooked selection tools Invalid TEEs can be discriminatory TEE/MQ requirements can frequently exhibit adverse impact What are the key essentials I need to know about this topic? Invalid TEEs are sitting ducks for plaintiffs or enforcement groups Valid TEEs can be readily established What are the consequences surrounding these issues? TEEs that are not based on job standards will unfairly screen out qualified workers, and may exhibit adverse impact The start up cost of a validation court case is frequently between $30k and $80k Presentation Overview Overview & Background Uniform Guidelines Criteria Regarding TEE Requirements Professional Standards Regarding TEE Requirements Using TEE Requirements in Open Selection/ Promotional Processes Using TEE Requirements in Closed Selection/ Promotional Processes TEE Fundamentals Some Basic Questions to ask about your Training, Education, and Experience Requirements Are they: Objective? Uniformly applied to all applicants? Likely to differentiate between qualified and unqualified candidates? Clearly job related? Provided in such a way that all applicants will have an equal opportunity to demonstrate that they possess the desired KSAPC or experience? Scored using a system that is consistent and reliable?

5 Guidelines and Principles of Valid and Defensible TEEs For many HR professionals, TEE requirements boil down to a judgment call Internal vs. external experience? Informal training vs. formal classroom training? Years of experience vs. depth of experience? Relevance of experience? Management preference? How do you sort what s important from what s not? Guidelines and Principles of Valid and Defensible TEEs Federal Uniform Guidelines (Section 14C2) A job analysis must be conducted Work behaviors must be documented o Identify important/critical job duties/tasks Tasks/duties are linked to Knowledge, Skills, Abilities, or Personal Characteristics (KSAPCs) Measure KSAPCs linked to important/critical work behaviors that form the content of the job or directly measure indicators of previous work behaviors (i.e., previous experience) see Content Validation Process Other KSAPCs Job Duties Operationally defined KSAPCs Selection Devices (e.g., application form, tests, interviews)

6 Guidelines and Principles of Valid and Defensible TEEs Federal Uniform Guidelines (Section 14C6) Level or amount of TEE requirement must be specified Linkage between job content and the content/level of TEE Requirement Linkages must be specific to KSAPCs (which are linked to important/critical behaviors/duties and/or affiliated work products) and/or to important/critical job duties TEE requirements do not need to exactly duplicate the duties of the job Guidelines and Principles of Valid and Defensible TEEs 2003 Society for Industrial and Organizational Psychology (SIOP) Principles (p. 23) Similar to Uniform Guidelines Requires more than a superficial resemblance (similar job titles are not enough) Instead, evaluate similarities between content of experience and content of work See Using TEE Requirements in Open Selection/Promotional Processes A quick note on Checkbox or Point- Method TEEs: Vague requirements vs. the need to cover all possible contingencies How do you measure level? Are you willing to pre-list all possible ways of attaining experience and take a firm stance on exactly how much is required?

7 Using TEE Requirements in Open Selection/Promotional Processes Steps for developing a valid TEE rating system: 1. Conduct a job analysis that addresses all Uniform Guidelines requirements 2. Identify appropriate KSAPCs for measurement. Consider using a Selection Plan for this purpose. 3. Develop a TEE survey that provides applicants with enough space to describe their relevant experience and education for each KSAPC selected for evaluation Knowledge, Skill, Ability or Personal Characteristic [KSAPC from Job Analysis] Training (including professional, military, or other) (Applicant completes) Experience (professional or volunteer) (Applicant completes) Education (list by course title) (Applicant completes) Did you receive any grades, awards, credentials, performance ratings, or commendations? Describe. How long was the training, experience, or educational course? If applicable, what levels were attained? When was it completed? Using TEE Requirements in Open Selection/Promotional Processes 4. Develop a scoring system for each survey. Consider the job analysis data when evaluating the worth of various KSAPCs. 5. Have two raters score the surveys using a rating system for each item (e.g., a 1-5 scale). Raters should review the job analysis and have a consistent scoring taxonomy. 6. Final score should be calculated and used on a pass/fail, banding, or rank ordered fashion based on reliability of the raters, adverse impact, and other factors

8 Sample TEE Rating Guidelines Criteria Explanation Relevancy Achievement Time Extent Date How relevant is the TEE description? How well is it linked to the target KSAPCs? How well does the TEE resemble job requirements? What level of achievement or proficiency is indicated (if any)? Were any credentials or awards attained? How much time did the applicant spend completing the training, experience, or education? How frequently was it repeated? To what extent was the TEE acquired? To what level? At only a baseline? Mastery? Expert? When was the TEE acquired? How recently? Was it acquired recently enough to still be relevant? Using TEE Requirements in Closed Selection/Promotional Processes Validation requirements are slightly relaxed in closed, internal only promotional processes TEEs should still be objective, fair, and valid When there is a negotiated labor contract, straight line seniority can be used with very little liability exposure However, seniority alone is not always the best predictor of future performance Using TEE Requirements in Closed Selection/Promotional Processes When there is only one feeder job: Employers may desire to set a time in grade experience requirement Using number of hours is recommended to address the issue of part time workers. Use Job Experts to calculate the average minimum required hours o Reduce this value by one Standard Error of the Mean or SEM (SEM = S.D. Ratings/Square Root(# of Raters 1) to adjust for sampling error

9 Using TEE Requirements in Closed Selection/Promotional Processes When there are several feeder jobs (or many functional areas within one job): Develop a valid Work History Evaluation (WHE) Assign weights to time in various feeder positions based on their relative importance to performing the target job Calculate the minimum and maximum hours in the selection procedure. Steps for Developing a WHE for a Closed Promotional Process 1. Create a WHE development survey to determine the following: Feeder Job/Functional Area Importance Weight Minimum Required Hours Maximum Hours Job 1 or Experience Area Job 2 or Experience Area Job 3 or Experience Area Job 4 or Experience Area Steps for Developing a WHE for a Closed Promotional Process 2. Convene a panel of Job Experts and for the target position. (Usually 7-10) 3. Provide the panel members with the Job Analysis data 4. Give the panel the survey you created in Step 1 5. Calculate the Mean and S.D. of all ratings, discarding those ratings S.D. above or below the mean for each

10 Steps for Developing a WHE for a Closed Promotional Process 6. Calculate a final weight and minimum/maximum** value for each feeder position or functional area. Use these weights to calculate scores for applicants: % of Possible Hours * Weight = Score Why Minimum and Maximum Hours? Too new to know what you don t know Meaningful Training, Experience, or Education More experience doesn t help Time Duration of Experience Minimum Maximum Weighting TEEs Each KSAPC or experience area on the Survey can be unit weighted (with each counting equally) or weighted based on the relative importance of each TEE requirement to the job. Weights for each KSAPC or experience area on the Survey can be developed by asking a panel of Job Experts to assign 100 points among the KSAPCs, and then averaging the results. The job analysis data should be considered by the Job Experts when assigning point values; however, it is not necessary that the point ratings are in agreement with the job analysis data.

11 Example Job A Weight = 10% Min. Hours = 250 Max Hours = 1500 Applicant Hours = 1250 % Possible Hours * Weight = Score ( )/( ) * 10% =? 1000/1250 * 10% =? 80% * 10% = 8% Further Explanation We re interested in what % of the Range of hours someone has in that Job or Functional Area. Max Hours(1500) Min. Hours(250) = Range(1250) We do not include hours below the minimum requirement (250) or above the maximum (1500) in the calculation This leaves us with 1000/1250 * Weight (10%) 1000 is 80% of 1250, so that becomes 80% * 10% or 8% Steps for Developing a WHE for a Closed Promotional Process 7. If applicants can have work experience in a feeder position or functional area that has changed substantially, a minimum recency factor can also be included.

12 Training, Education, and Experience Requirements vs. Basic Qualifications TEEs: Detailed assessments of a person s relevant training, experience, and education. Usually scored on a predetermined scale by multiple raters Can be used to rank order applicants BQs: A basic, minimum requirement necessary for a person to be considered for a job Generally used as pass/fail screens before the general assessment of the candidate begins Candidates who do not meet BQs are not considered Internet Applicants for record-keeping purposes Summary TEEs would be considered selection devices under the federal Uniform Guidelines if they are considered when making employment decisions TEEs can and generally should be validated to insure they are fair, job-related, and consistent with business necessity There are a number of different factors to consider when evaluating TEEs, such as the weights (i.e., relative importance), minimum/ maximum levels, as well as recency/staleness Questions?

13 Resources Adverse Impact and Test Validation: A Practitioner's Handbook by Daniel A. Biddle, Ph.D. Purchase online at Adverse Impact and Test Validation Book Series Webinars Recordings available online for all BCGi Platinum Members Webinar slides available online to all BCGi Standard & Platinum Members BCGi Membership Free Standard Membership Premium Platinum Membership About Our Sponsor: Biddle Consulting Group (BCG) BCG is an HR firm dedicated to providing the highest products and services related to Equal Employment Opportunity (EEO), Affirmative Action and Employee Selection. BCG s Consulting Services Affirmative Action Plan Outsourcing Compensation Analysis EEO/AA Litigation Support (Plaintiff and Defendant) Job Analysis Test Development and Test Validation BCG s Software Products Adverse Impact Toolkit AutoAAP affirmative action plan development software AutoGOJA job analysis software C4 call center testing software COMPARE compensation analysis software CritiCall dispatcher/call-taker personnel selection software ENCOUNTER soft skills video situational judgment testing software OPAC office skills testing software TVAP Test Validation & Analysis Program software