WorkKeys Innovations: A Holistic Solution

Transcription

1 WorkKeys Innovations: A Holistic Solution presented at the 2007 Michigan WorkKeys Conference Steve Robbins, AVP, Applied Research, ACT, Inc.

2 Overview Why we should care about combining cognitive- and personality-based measures WorkKeys Pyramid for Success The National Career Readiness Certificate Plus and Personal Skills Assessments Differential test strategies along the entire continuum of employment 2 2

3 What We Know from Research Literature on Incremental Test Validity Cognitive and foundational skill tests rule Adverse impact issues can be ameliorated with specific tests: Task analysis Job-specific tests Combinational use of cognitive and noncognitive tests Personality and career tests add incremental validity 3 3

4 General vs. specific test effect sizes Training Outcomes Work Performance GMA.54 (.55*).62** Math.48 (.48*).52** Reading.44 (.40*).35** * Brown, Le, & Schmidt (2006) **Salgado et al. (2003) 4 4

5 Combining Personality & Cognitive Ability Tests Level of correlations are low: GMA x C =.02 GMA x ES =.17 Math x C = -.15 Math x ES =.17 Reading x C = -.05 Reading x ES =.11 (Ackerman & Heggestad, 1997) 5 5

6 Personality Tests Add Value Validity Estimates: Training Outcomes Old Validity New 1 Validity Work Performance Old Validity New 1 Validity GMA Conscientiousness Emotional Stability From Schmidt et al. (2007) using indirect range restriction 6 6

7 Combining Personality & Cognitive Ability Tests Creating opportunity for incremental validity especially as criteria vary Correlations between general cognitive ability and personality tests and measures of job performance in Project A Cog Pers Both Criteria Core technical proficiency General Soldiering Proficiency Effort and Leadership Personal Discipline Physical fitness and military bearing (McHenry, Hough, Toquam, Hanson, & Ashworth, 1990) 7 7

8 Why Now? Market Need: SHRM National Study Body of research informs how to optimize Personal Skill Assessments ACT strength in cognitive & noncognitive assessment (John Holland, VP Research in 1960 s) Solution-focused approach 8 8

9 Businesses want integration of Cognitive & Personality Constructs SHRM: Applied Skills and Basic Knowledge: Combining and Ranking For new entrants with a two-year college/technical school diploma, applied skills are four of the top five very important skills in combined ranking with basic knowledge and skills. Rank Skill % 1 Professionalism/Work Ethic Teamwork/Collaboration Oral Communications Critical Thinking/Problem Solving Reading Comprehension Written Communications English Language Ethics/Social Responsibility 70.6 Casner-Lotto, J. & Barrington, L. (2006) 9 9

10 Solutions are Need Driven: Continuum of Employment Purpose Needs Purpose Needs Pre- Selection -Screen people in most cost effective way -Find honest/dependable employees Coaching/ Development -Identify other jobs that an employee can fit -Develop employees for future company needs -Save time in the screening process -Employees identify areas of improvement Recruitment -Identify people who fit the work environment -Identify people with skills that match the job Succession/ Leadership Planning -Identify candidates for toplevel positions -Develop employees for future needs of company -Retain top performers Selection -Select employees with skills that best fit the job -Save time in selection process -Select people in most costeffective way -Find honest/dependable employees Training/ Development -Identifying work readiness -Identify basic workplace skill levels -Educating about career planning -Job Placement -Certifying employees 10 10

11 WorkKeys Assessment Solutions: Pyramid for Success Ensure work and training readiness (WorkKeys Foundational Skills) Give a snap shot of strengths and areas of improvement across key response tendencies or domains ( Talent ) 11 11

12 Pyramid for Success FIT Match individual interests/values to work environment Enhance job persistence & satisfaction Develop Talent pool to meet needs TALENT Benchmarking for selection Coaching & Development Compound Indices: Sales, Managerial, Leadership, Safety PERFORMANCE General Work Performance: Productivity, Absenteeism, Complaints about conduct Safety and Risk Reduction FOUNDATIONAL SKILLS Job Analysis identifies the skills and skill levels needed to be successful on the job Assessments show the current skill levels of an individual Training helps individuals and employers correct skill gaps 12 12

13 National Career Readiness Certificate The WorkKeys system assesses foundational skills, allowing individuals to seek a certificate, and when necessary to fill in the gaps Silver Certificate Goal Skill gap 2 Worker s Skill Level 1 Applied Math Reading Locating Information 13 13

14 National Career Readiness Certificate Plus A flexible solution to meet state and system needs Georgia example Talent for coaching & development Alterable variables allow for intervention & guidance 14 14

15 It was about here, wasn t it, Ed, when you came on board as sales manager? Harvard Business Review. March p

16 Page 1 of 3 Examinee: Examinee ID: Carefulness Cooperation Creativity Talent Assessment Alvin C. Tracey XXXXX7890 WorkKeys Talent Assessment measures personal and workplace behaviors and attitudes. This report is designed to help identify examinee strengths and weaknesses in order to ensure success in the workplace. WORKEYS TALENT ASSESSMENT PROFILE Percentile Rank: Approximate percent at or below score Employer Report Report for: Abbatoir Industries Site: Iowa City, IA Test Date: 4/12/07 Talent Score Report Discipline Drive Goodwill Influence Optimism Order Savvy Sociability Stability Capitalize on Individual Strengths Percentile 90 Scale Definitions Carefulness: Tendency to think and plan carefully before acting. This individual s responses suggest that he or she is cautious, deliberate, and pays close attention to detail in the workplace. Responders at this score level tend to think carefully before acting or speaking. They always consider the consequences of their actions, and their decisions are usually well thought-out Order: Tendency to be well organized as well as keeping the work area neat and tidy. The examinee s responses suggest that he or she is well-organized, and consistently keeps physical surroundings neat and tidy. Individuals who respond at this score level are always methodical in their manner and maintain a structured professional environment. Stability: Tendency to maintain composure and rationality in situations of actual or perceived stress. This individual s responses suggest that he or she maintains his or her composure even when faced with highly stressful situations. Individuals who respond at this score level tend to remain calm and even-tempered in their conduct, and they feel confident in their ability to handle the pressure and stress of working under deadlines.. 85 Discipline: Tendency to begin tasks and complete them without becoming distracted or bored. The examinee s responses suggest that he or she commits to work duties until they are complete. Individuals who respond at this score level take responsibility and can always be relied upon to get their work done on time. They are not easily distracted, and always persist through challenges until the task is done

17 TALENT Development Worksheet Scales Score Skills to Develop Timeline Method Notes Cooperation Discipline Goodwill Influence Optimism Savvy Give and Take Self- Management Self-monitoring Time Management Selfmanagement Establishing Goals Working toward goals Resource Management Trusting & Vulnerability Positive thinking regarding others Relational appropriateness Public speaking Persuasion skills/influence tactics Negotiation skills Ways to provide feedback Manage expectations Substitute negative thinking Locus of control Sensitivity & appropriateness Self-monitoring Assimilation of verbal content Interpretation of non-verbal cues 17 17

18 Behavioral Scales, Targets, and Representative Behaviors Cooperation Tendency to be likable and cordial in interpersonal situations. Relational Give & Take Effectively negotiates solutions to problems Is able to compromise in order to facilitate task completion Distinguish people from tasks Effectively works with other persons with differing opinions Does not let interpersonal differences interfere with task performance Listening Skills Focuses on the person communicating Asks questions to build understanding Restates or paraphrases to demonstrate understanding Communication Disagree without being critical Directs differing opinions towards the task, product, or process rather than the person Voices opinions/viewpoints that contrast with the majority opinion Voices opinion/provides feedback to others in respectful manner 18 18

19 Discipline The tendency to be responsible, dependable, and follow through with tasks without becoming distracted or bored. Behavioral Scales, Targets, and Representative Behaviors Planning Goal Setting Resource Mgt. Time Management Shows up to work on time Makes a daily to do list Self-management Sets goals for performing a new task Keeps track of task performance Establishing Goals Sets goals which are clear Sets goals which are specific Sets goals which are achievable Sets goals which are challenging Has a written set of goals Working toward goals Reviews goals on a regular basis Monitors progress in goal achievement Effectively deals with setbacks and challenges Self-discipline and motivation Consistently completes tasks of projects started Displays confidence in his or her own abilities Avoids distraction and remains focused on work tasks Asking for assistance 19 Is not afraid to ask for clarification if something is unclear Identifies and utilizes appropriate support resources 19

20 Behavioral Scales, Targets, and Representative Behaviors Influence The tendency to impact and dominate social situations by speaking without hesitation and often becoming a group leader Assertiveness Communication Relational appropriateness Is confident when interacting with others Uses language/style of speech that is appropriate for communicating with a particular person or group Public speaking Is comfortable speaking before a large group of people Maintains audience interest when making presentations Speaks clearly during presentations Maintains adequate eye contact when addressing the audience Persuasion skills/influence tactics Is able to show others the value of her position on an issue Uses evidence and logic to influence others Negotiation skills Proactively resolves conflict among individuals or groups Effectively negotiates solutions to problems Conducts negotiations in a professional and respectful manner Provides feedback Provides positive feedback when warranted Delivers constructive feedback in a respectful manner Providing feedback that is consistent Provides feedback that is useful 20 20

21 Mean Scores A Comparison of Talent Scale Scores for Computer Programmers vs. Sales Representatives & Sales Managers Talent Scales Computer Programmers Sales Representatives & Managers 21 21

22 22 22

23 Examinee: Examinee ID: Fit Assessment Alvin C. Tracey XXXXX7890 Employer Report WorkKeys Fit Assessment measures the fit between an examinee's work-relevant interests and values and the characteristics of occupations. Worker satisfaction and commitment are associated with greater fit. Examinee-specified primary occupation is in BOLD. Fit Index for Occupations Specified by Examinee Percentile Rank: Approximate percent at or below score Human Resources Managers 56 Report for: Abbatoir Industries Site: Iowa City, IA Test Date: 4/12/07 1 of FIT Score Report Training and Development Managers Human Resources Managers, All Other Purchasing Managers Transportation, Storage, and Distribution Managers 86 Fit scores are based on the examinee's responses to the Interest Inventory and Work Values Inventory. Fit Index for Related Occupations Occupations specified by the examinee, as well as related occupations, are ranked by Fit Index. Examinee-specified occupations are in BOLD. Code Title Fit Index Level of Fit Transportation, Storage, and Distribution Managers 86 High Administrative Services Managers 80 High Sales Managers 79 High Transportation Managers 77 High Storage and Distribution Managers 69 Moderate Training and Development Managers 61 Moderate Human Resources Managers 56 Moderate Purchasing Managers 56 Moderate Financial Managers, Branch or Department 55 Moderate Compensation and Benefits Managers 54 Moderate Human Resources Managers, All Other 54 Moderate Financial Managers 53 Moderate Advertising and Promotions Managers 51 Moderate Marketing Managers 50 Moderate Public Relations Managers 50 Moderate Treasurers and Controllers 42 Moderate Industrial Production Managers 29 Moderate Computer and Information Systems Managers 27 Moderate Crop and Livestock Managers 23 Low Nursery and Greenhouse Managers 14 Low 23 23

24 Examinee: Code Fit Assessment Alvin C. Tracey Examinee ID: XXXXX7890 Top 10 Occupations Ranked by Fit Insurance Sales Agents Administrative Services Managers Real Estate Sales Agents Sales Managers Title Sales Agents, Financial Services Loan Counselors Transportation, Storage, and Distribution Managers Appraisers, Real Estate Employer Report Real Estate Brokers 79 Middle School Teachers, Except Special & Vocational Educ of 3 Report for: Abbatoir Industries Site: Iowa City, IA Test Date: 3/30/07 The top 10 occupations for the examinee, ranked by Fit Index, are shown below. This is based on all of the occupations in the WorkKeys Fit database. Examinee-specified occupations, if any, are in BOLD. Fit Index Employer Report: Top 10 Occupations and Interest Inventory Results Interest Results The Interest Inventory obtains scores on six scales. The examinee's score profile and highest scales are shown. Interest Score Profile Standard Score (20-80) Administration & Sales: Persuading, influencing, or motivating others via sales, management, etc. Business Operations: Maintaining accurate/ orderly files/accounts via systematic procedures. Technical: Working with tools, instruments, machines, etc. Science & Technology: Studying phenomena through research, reading, etc. Arts: Expressing oneself via painting, singing, writing, etc. Social Service: Helping or serving others via teaching, counseling, etc Highest Interest Scales Administration & Sales Science & Technology Business Operations 24 24

25 Examinee: Examinee ID: Fit Assessment Work Values Results Employer Report Examinee's highest 5 and lowest 2 work values are shown. The definition of each value, as it appears in the Work Values Inventory, is provided. Variety Highest Values Using many different types of skills in my work. Public Contact Interacting with customers (as in sales) or the public (as in police work). Authority Telling people what to do; controlling the behavior of others. Precision Alvin C. Tracey XXXXX7890 Being exact or very accurate in the work I do. 3 of 3 Report for: Abbatoir Industries Site: Iowa City, IA Test Date: 3/30/07 Employer Report: Work Values Inventory Results Helping People Improving the lives of others through activities such as teaching, physically assisting, or mentoring. Lowest Values Social Status Being looked up to by others in my company or my community because of my job. Physical Activity Moving around in my work by walking, bending, lifting, etc. For more information go to

26 Selection Solutions Reducing Risk Task Competence through WorkKeys and job profiling General Work & Safety Increasing Tenure Task Competence through WorkKeys and job profiling Fit Getting the Right Person Talent Benchmarking Past Work Performance Record 26 26

27 If they staged a slowdown, how would we know? Harvard Business Review. March p

28 Coaching & Development Solutions Leadership Development Talent Fit Teamwork Talent 28 28

29 Return on Investment Approximations under Various Scenarios Selection % 10% 25% 50% Candidate Success % Selected Success % 25% 54% 50% 80% 25% 45% 50% 72% 25% 36% 50% 64% Cost per failure Notes: Selection % = the percentage of the candidate pool selected for hire, Candidate Success % = the percentage of the candidate pool that would be successful if hired, Selected Success % = the percentage of the selected candidate pool that will be successful, Cost per failure = the average cost for each unsuccessful employee relative to each successful employee, ROI per 100 candidates = the average return on investment for the selection procedure assuming a $15 fee per candidate. ROI per 100 candidates $5,000 $13,000 $10,000 $27,500 $5,000 $13,500 $10,000 $28,500 $5,000 $23,500 $10,000 $48,500 $5,000 $26,000 $10,000 $53,500 $5,000 $26,000 $10,000 $53,500 $5,000 $33,500 $10,000 $68,

30 Final Thoughts Adverse Impact may be reduced when combining tests Still recommend multiple hurdles approach Incremental Validity Research underway: Combination of Task & Personality measures Differential work outcomes ROI x Solution 30 30

31 References Ackerman, P. L., & Heggestad, E. D. (1997). Intelligence, personality, and interests: Evidence for overlapping traits. Psychological Bulletin, 121, Brown, K. G., Le, H., & Schmidt, F. L. (2006). Specific aptitude theory revisited: Is there incremental validity for training performance? International Journal of Selection and Assessment, 14, Casner-Lotto, J. & Barrington, L. (2006). Are they really ready to Work? Society for Human Resource Management. McHenry, J. J., Hough, L. M., Toquam, J. L., Hanson, M. A., & Ashworth, S. (1990). Project A validity results: The relationship between predictor and criterion domains. Personnel Psychology, 43, Robbins, S., Allen, J., Casillas, A., Peterson, C., & Le, H. (2006). Unraveling the differential effects of motivational and skills, social, and self-management measures from traditional predictors of college outcomes. Journal of Educational Psychology, 98, Rotundo, M., & Sackett, P. R. (2002). The relative importance of task, citizenship, and counterproductive performance to global ratings of job performance: A policy-capturing approach. Journal of Applied Psychology, 87, Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C., & de Fruyt, F. (2003). International validity generalization of GMA and cognitive abilities: A European community meta-analysis. Personnel Psychology, 56, Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, Schmidt, F. L., Shaffer, J., & Oh. I. (2007). Reassessing the Relative Importance of Cognitive Ability and Personality in Job Performance and Training Performance: Some Surprising New Research Findings. Paper presented at the 2007 ATP conference, Palm Springs, CA. Feb

32 Incorporating Foundational and Soft Skill Assessments For questions regarding this presentation or for further information contact: Steve Robbins at or Gary Nolan at or 32 32