Evaluating the use of psychometrics

Transcription

1 Evaluating the use of psychometrics It is now readily accepted that the effective use of psychometric instruments can add value to an organisation. When used in selection, in conjunction with a competency framework and structured interview process, they enable companies to select more accurately those people who will perform best in the role. What is less clear is how companies can communicate the impact of these benefits in financial terms that can be easily articulated at senior management and board levels. In the current economic climate, companies need to control their budgets tightly. HR professionals with a responsibility for their department s expenditure inevitably feel this pressure. They are increasingly called upon to justify their spending and must present a convincing case. Ideally the figures they present will demonstrate the impact of their allocation of resources on bottom line business performance. This is a difficult task, but for those who can rise to the challenge successfully there is another advantage they stand to win board level support for their work. A major part of any HR function is the recruitment and development of employees. The selection process represents a significant commitment of resources, one aspect of which is the use of psychometric instruments. These enable companies to measure accurately important skills that would not otherwise be apparent at interview, such as the ability to make critical business decisions. If the financial effectiveness of systematic selection processes can be measured, their value will be more readily recognised and support for HR initiatives strengthened. Shortening the odds Any selection method has a margin of error and selecting the right person for the role every time is an unrealistic goal. The use of psychometric instruments however increases the validity of the selection process by identifying those candidates who are likely to be most suited to the position. The relationship between candidates performances during the selection process and their subsequent performance within the role can be broken down into four categories. These are known as: True Positives, False Positives, True Negatives and False Negatives: True Positives These candidates perform well during the recruitment process. Once in the job they continue to perform well, making their assessment during the selection procedure an accurate prediction of their future performance. This category will ideally contain the highest proportion of candidates. Diagram 1: Distribution of candidates when valid selection methods are used False Positives These candidates give a good performance during the recruitment process and seem ideal for the role. Once in the job, however, their performance falls below expectations. People in this category stand to cost the company the most amount of money through a lack of productivity. Additionally they may need to be replaced and the selection process repeated. This category needs to contain a low proportion of candidates for the selection procedure to remain cost-effective.

2 True Negatives These candidates appear unsuited to the position during the recruitment process. If selected they go on to perform poorly in the job. The selection process has therefore been valid in accurately identifying people unsuited to the role. False Negatives These candidates give a poor performance during the recruitment process and seem unsuitable for the position. In fact if they were to be selected they would perform well. This category needs to contain a low proportion of candidates to prevent high calibre candidates being turned away. Without the use of valid selection tools, the number of false recruitment decisions can equal or outweigh the number of good recruitment decisions: There will always be a proportion of False Positives and False Negatives, but using valid selection methods, such as psychometric tools, reduces the number of candidates who fall into these categories. Proven to be an accurate predictor of job performance, psychometric instruments increase the proportion of True Positives and True Negatives, as shown in Diagram 2. Diagram 2: Distribution of candidates when psychometric tools are used In most selection situations, it is preferable for as many candidates as possible to be in the True Positives category. Sometimes, however, there may be a large number of candidates in the True Negatives category, for example, when there are very large numbers of applicants for relatively few jobs. When the number of True Negatives is larger than expected, this could be an indication that the calibre of the applicant pool was unexpectedly low. Possible causes for this are if advertising for the role or the short-listing of candidates are inappropriate or set at too low a level. In cost-benefit terms, False Negatives and False Positives both add to the cost of selection. False Positives will perform less well in the job, reducing productivity and incurring other costs, and may need to be replaced. False Negatives represent wasted selection costs and, because potential high performers have not been recruited, the loss of future productivity and income. In order to keep the numbers of false selection decisions small, it is important to use tools that are likely to predict job performance well. Diagram 3 illustrates the relative accuracy of different selection methods in predicting overall job performance. Psychometric tools are among the most accurate methods; when their use is supported by a competency-based approach that allows the integration of other objective and valid tools such as structured interviews and workplace simulations, the prediction of job performance can be even more effective. Balancing the costs Selecting the wrong person for a role is costly, as is the cost of finding a replacement. The costs of a typical selection process are shown in Diagram 4. Balancing the extra cost of using psychometric tools against this potential expenditure clearly highlights the financial benefits of psychometric instruments, which typically represent only 4% of the selection process costs.

3 Whilst they do require an initial outlay in qualification training and materials, the ongoing costs are minimal, making them more cost-effective the longer and more often they are used. This contrasts with, for example, the interview, where there are significant hidden costs such as the time spent interviewing, which do not decrease on an ongoing basis. Further insight can be gained by balancing the expenditure on psychometric instruments against the long-term effect a successful candidate will have on company performance. To determine this, the following factors need to be considered: Cost of recruiting Recruitment costs include: time spent developing a job competency framework; advertising for the role; involvement of recruitment agency; time spent short-listing candidates; any psychometric instruments used; time spent interviewing candidates. The more candidates that are involved, the more the investment multiplies, and it may be sensible to use more cost-effective methods such as psychometric tools at earlier stages in the selection process. Impact of job role on company performance Every job role impacts on company performance. With some jobs, this impact is more direct. For example, if someone is recruited into a call centre role and creates a bad impression to clients over the phone, then the reputation and, in turn, performance of the company will suffer. Conversely, someone in a business development role who is effective at winning new business will directly benefit the bottom line performance of the company. The greater the potential impact of the role on company performance, the more value is gained from recruiting the right candidate. In turn, a poor selection decision may cost the company dearly in terms of their reputation and performance. Diagram 3: Accuracy of different selection methods Information adapted from M. Cook, Personnel Selection and Productivity (1988) and I. Robertson and M. Smith, Journal of Occupational and Organisational Psychology (74, , 2001) Length of tenure in role The longer a suitable candidate remains in their role, the more

4 financial value will be gained from their recruitment. If they only stay in the role for a short time, the company needs to invest in another selection process. This represents an additional investment of resource and a waste of the initial outlay. From the perspective of the candidate, if they are in a job to which they are ideally suited they will have a substantially higher level of job satisfaction. This affects both the length of time they are likely to stay in the role and their level of motivation. In turn this is likely to impact positively on both individual and organisational performance. Personality questionnaires have been shown to predict tenure and job satisfaction as well as job performance. Ability to distinguish between candidates Any selection tool should be able to distinguish between candidates in two ways: 1) The tool should be accurate and precise. Any differences in results between candidates should be real and not simply random or due to errors or inaccuracies. This is known as reliability, and psychometric tools have consistently been shown to be more reliable than other common selection methods. 2) The differences in results between candidates should reflect differences in their ability to perform in the job. This is known as criterion-related validity. As shown in Diagram 3, psychometric tools do predict job performance well, provided that the right tool has been chosen, for example, via a careful analysis of the attributes required for the job. Differences in results do therefore relate to differences in ability to perform well in the job. The importance of getting this right is one reason why most instruments are only sold to those who have been trained in their use. Estimating the cost-benefit of psychometric instruments Companies can examine the effect of a selection process on company performance by analysing how well a candidate performs in the role and hence how much they contribute to productivity (this will increase as they stay in the role longer), and balancing this against the investment in their recruitment. Research across a range of organisations suggests that the amount an employer can save, per employee recruited, per year, is: (Validity of the test) multiplied by (Calibre of those selected) multiplied by (Standard deviation of job performance) minus (Cost of the selection) divided by (Proportion of applicants selected) This formula, known as the utility equation, may look complex, but it has successfully been used by HR professionals in order to justify the use of psychometric instruments and other objective assessment methods. For example, it is regularly employed by the HR team of one major airline because of the severe financial pressures that exist in the post-9/11 marketplace. Calculating the cost benefits of psychometric instruments The utility equation can be used to calculate the financial benefit to an organisation of using psychometric instruments as part of the selection process. An example of this is selecting for a position with a salary of 40,000. It is possible to calculate the financial benefit achieved by using a selection system of unstructured interviews that have a validity of 0.18, compared to using personality questionnaires and ability tests that have a combined validity of Using the utility equation and a conservative estimate of the extent to which this position will impact on company productivity, it can be predicted that using psychometric tools would increase the organisation s productivity by 6,720 compared to using unstructured interviews. This calculation assumes that the candidate remains in the job for one year; if they remain for three years the productivity increase can be as much as 20,160. The use of psychometric tools will distribute higher proportions of candidates into the True Positive and True Negative categories and make it far easier to select suitable employees. By having high calibre employees, companies can reduce the need for further and unnecessary

5 expenditure on additional recruitment and reap financial rewards in the form of improved productivity. By working with the information outlined here, anyone with responsibility for justifying HR expenditure will be able to present tangible benefits of using psychometric instruments. The benefits they identify will help HR to be recognised as a contributor to the company s efficiency and productivity. Pre-selection 3,000 Preparation of job description, preparation and placement of advert, processing of applications, and short-listing candidates. Selection 850 Preparation for selection process, travel expenses, conducting interview, and reviewing selection process. Post-selection 150 Following up with candidates, making a job offer and negotiating terms, and preparing a contract. Psychometric instruments 150 Administering, scoring and providing feedback on a personality questionnaire and an ability test for each of the three candidates at the second interview stage. These costs are based on recruiting for the position of a recently qualified professional with approximately three years work experience. They account for all the costs involved from identifying the role that needs to be filled to confirming the employment of a candidate. They include the time spent in administration and management time, and assume twenty applications are received, with five candidates invited to a first interview and three to a second interview. The costs of qualification training for the psychometrics have not been included, as the return on investment for these should be considered over a long-term period. Diagram 4: The costs of a typical selection process