CHAPTER 8 PERFORMANCE APPRAISAL OF A TRAINING PROGRAMME 8.1. INTRODUCTION

Transcription

1 168 CHAPTER 8 PERFORMANCE APPRAISAL OF A TRAINING PROGRAMME 8.1. INTRODUCTION Performance appraisal is the systematic, periodic and impartial rating of an employee s excellence in matters pertaining to his present job and his potential for a better job. This HRM activity measures skills and accomplishments of an employee. Also it works as a strategic concept because it is dealing with organizational mission, vision and goals. A better performance system can help the employer to give a better quality product service to the society through productive employees. Nowadays, there are many organizations that are relying on employees for success and competitiveness. Accordingly, performance appraisal has come to play an indispensable role in helping organizations to reach their goals of productivity. Apart from evaluating individuals any activity can also be evaluated. For example, whenever any training programme is conducted, the management may plan to determine the effectiveness of such a programme and ways in which similar programmes can be improved. There are three specific reasons for such an evaluation (1) To justify the existence and budget of the training department by showing how it contributes to the organizations objectives; (2) To decide whether to continue or the training programme or not; (3) To gain information on how to improve future training programmes. Normally the appraisal, provides new insights into what is being assessed or new information that was not anticipated as an outcome of the exercise. As it is known, job control, human resource management (HRM) practices, structural factors and workforce characteristics are supposed to be the four broad determinants of performance appraisal system. By taking into consideration, one of these four determinants i.e the HRM practices, we can identify some of its components which are significant for our analysis like worker training, job redesign and joint consultative committees. The effectiveness and efficiency of worker training programmes will be better decided only by the trainees who participate in such programmes. It is therefore obvious that any appraisal of a training programme can be effective, if done through performance appraisal of employees.

2 169 The performance scores of the employees who participated in a training programme can be calculated and those scores can form the basis for the evaluation of the programme. To compute those scores, there are many appraisal methods such as rating scales, ranking comparisons, 360 degree feedback, 720 degree feedback, management by objectives and so on. Obviously no method can claim that it has an integrated approach in performance appraisal. Hence human resource managers should select an appraisal method which is most suitable to their organizations. For example, The rating scales method which is the simplest and the most popular technique, can help us understand the aspect of how the performance scores are computed. The typical rating scale system consists of several numerical scales, each representing a job-related performance criterion such as job knowledge, quality of work, productivity, commitment to safety, innovative, initiative, attitude, cooperation and the like. Each criterion is ranked as Excellent, Exceeds expectations, Meets expectations, Needs improvement or Unsatisfactory. In simple words it can be Very good, Good, Average, poor or Very poor. They are allotted scores ranging from 5 (Very good) to 1 (Very poor). The rater checks the appropriate performance level on each criterion and then computes the employee s total numerical score, which is taken as his performance score WORK PLAN Now the objective is to evaluate the effectiveness and efficiency of a training programme by studying the impact of the programme on the trainees. Philosophically speaking, statistics can be seen as an inductive method; on the basis of a number of observations general conclusions are drawn. Modeling, on the other hand, is a deductive approach wherein, on the basis of known behavior of components, conclusions for specific models are drawn. Note, however, that the behavior of the components is often obtained through statistics. Hence statistics definitely plays an important role in almost any modeling project. Different examples are discussed using statistical methods. In the first two models a small sample of employees who attended the training programme, is used to evaluate the programme.

3 170 Let us consider an organization that has some branches. To compare the impact of a training programme on the employees of those branches, a small sample is taken from each branch and analysis is done using ANOVA technique in example 3. In the fourth example, if ANOVA technique shows that the means of difference of performance scores are not equal, Tukey s HSD method is explained to find out that mean which is different from others. This is done when sample sizes are equal. When they are not equal Tukey-Kramer method is used for that purpose. Finally a reference is made to explain how Data Enveloping Analysis has been used in performance appraisal APPLICATION OF PAIRED t-test In this chapter, application of different statistical techniques in the field of performance analysis is being discussed. As the first case, this section describes the application of paired t-test to study the impact of a training programme on the employees. Suppose, a sample of n employees are selected from the organization and their performance appraisal scores are calculated. Then they are given the training required. When they return from the training, again their performance appraisal score are obtained. Now the problem is to find out whether, in general, the training given leads to any improvement in employee s knowledge and skill. This is done on the basis of their performance appraisal scores. We can use the results from that sample to draw conclusions about the impact of the training. There comes the application of paired t- test, which is used to compare the performance scores before training with those after training. The paired t-test procedure is briefly explained now. If x i s are the performance scores before training, y i s are the scores after training the difference d i = y i x i (i = 1, n) are calculated. making sure we

4 171 distinguish between positive and negative differences. Then mean difference =. And the standard deviations of the differences s are calculated. Then the standard error of the mean difference SE( ) is given by. Now the null hypothesis that the true mean differences of performance scores is zero is to be tested, using the t statistic T = (8.1) Since this statistic follows a t-distribution with (n-1) degrees of freedom, t distribution table can be used to compare our value for T with the table value. Based on this we can draw our conclusion whether the training has helped the participants to improve their performance. But before applying the t-test it should be verified whether the samples are randomly drawn from normally distributed populations with unknown population variances. If such assumptions cannot be made, we may try non parametric methods. Thus the paired t-test examines if the mean of the differences (effect of training) is discernable from zero (no effect). Also pairing is a good idea when we expect greater variation between the pairs when compared to variation within a pair APPLICATION OF INDEPENDENT SAMPLES t-test In the previous section, an organization with only one branch has been considered. The paired t-test is also applicable when any organization, though it has many branches, analyses the impact of training on the performances of the employees for one branch after another. There are two sets of scores which can be paired and hence are dependent. Suppose the organization plans to compare the performance improvement of the employees due to training, considering two branches at a time. A training programme is arranged for the employees of those two branches. Both before and

5 172 after completion of the training performance appraisal is carried out for them and mean of the differences of the performance scores are obtained. The management is interested in finding which branch is more benefited by the training i.e. for which branch mean difference of score is more. For that purpose two small samples not necessarily of equal sizes, one from each branch may be taken and the mean difference of performance scores of their employees are calculated. Then the independent sample t-test is applied for the comparison sake. Now it is assumed that both population i.e. performance scores for all the employees who participated in the training programme are normally distributed or nearly normal. Let 1 and 2 be the mean difference scores for all the trainees of the two branches. We are typically interested in the difference between 1 and 2. Then we consider Null hypothesis H 0 : 1 = 2 Alternate hypothesis H a : 1 < 2 or 1 > 2 As always with hypothesis testing, the claim is about the population i.e. all the trainees of the two branches but it will be tested using sample data. Each group is just a sample of the difference of scores (before and after training for the same sample) that would have been collected if every member of the population has been considered, Now the test can be explained. Let, be the mean difference scores of the two samples, n 1, n 2 be the sample sizes and s 1, s 2 be the sample standard deviations. Then 2 = (8.2) s SE( ( 8.3) The test statistic for the independent t-test is given by

6 173 t = ( 8.4) The test statistic value is compared with the t distribution value for (n 1 + n 2 2) degrees of freedom By using independent sample t test it is tested whether the null hypothesis is accepted or not. If it is accepted, it can be concluded that the training has the same impact on the employees of both the branches. Otherwise it will be known which branch has greater advantage of the training. Thus with the help of this statistical test we determine the probability that two populations are the same with respect to the variable tested ANOTHER APPLICATION OF INDEPENDENT SAMPLES t-test In the previous section when one training programme is conducted, its impact on two branches is analysed, with the help of independent sample t-test. Now another application of the same test in a different situation can be seen. Here also two samples of employees are considered. But both of them are taken from the same branch. Also instead of one type of training, the management will conduct two types of training. Then t-test is applied to find out whether there is any difference in the effectiveness of the two training methods. This situation can be explained with the help of an example. In an organization, an application of the test of the difference in small sample means arises. New employees are expected to attend a three-day seminar, to learn about the company.at the end of the seminar, they are tested to measure their knowledge about the company. The traditional training method has been lecture and a question-and-answer session. Management decided to experiment with a different training procedure which involves new employees for two days by using video cassettes. Now company

7 174 manager would like to know whether there is any difference in the effectiveness of the two training methods. For that purpose one group of 15 newly recruited employees is selected to take the three day seminar (Method A) and a second group of 12 new recruits for the twoday video cassette method (Method B). Assume that the test scores for two groups are normally distributed and that the population variances are approximately equal. The test scores of the two groups are as follows. Training Method A Training Method B We have = 47.73, = 56.5, = , = Using the formulae (8.2), (8.3), (8.4) we get the observed value of t to be 5.20 while the table value of t for = 0.05 is 2.06 Thus the null hypothesis that population means are equal is rejected and the management concludes that there is significant difference in the effectiveness of the training method. Upon examining the means, it is understood that training method B is better. Thus we have seen application of independent samples t-test to two different situations. In the first case impact of one training programme on employees is studied while in the second case the effectiveness of two training methods is analyzed. If needed instead of new recruits, existing employees can also be considered in the second application while new recruits cannot replace the existing employees in the

8 175 first case as new recruits cannot be appraised before training. Also in the first application our concern is about the employees performance while it is about the training methods in the second. After discussing the application of the statistical technique t-test, next technique to be discussed is ANOVA test APPLICATION OF ANOVA TEST The independent samples t-test is limited to comparing the impact of a training programme on two branches only. Suppose an organization with more than two branches is to be considered. Then we are in need of a technique to compare more than two groups. Somebody may suggest conducting multiple t-test. That means, if 4 populations (i.e. branches) are to be compared and tested two at a time, it takes 6 tests(= 4 c 2 ) to analyze the hypothesis between all possible pairs. But conducting multiple t- tests can lead to severe inflation of the Type I error rate (false positives) and hence not recommended. In such a situation, it is suggested to use ANOVA technique which uses data from all groups to estimate standard errors which can increase the power of the analysis, without increasing Type-I error rate. The one-way ANOVA (Analysis Of Variance) uses F statistic to test if all groups have the same mean. So in our case it can be applied to analyse whether the impact of a training programme is the same on the employees of all the branches. By ANOVA, inferences about means are made by analysing variances. The t-test is considered a special case of the one-way ANOVA. When comparing mean difference of performance scores of two branches, the t-statistic is the square root of the F-statistic of ANOVA (i.e. F = t 2 ) Like so many of other inference procedures ANOVA also has some underlying assumptions which should be in place in order to make the results of calculations completely trustworthy. They include (i) Subjects are chosen via a simple random sample (ii) Within each group/population, the response variable is normally

9 176 distributed (iii) While the population means may be different from one group to the next, the population standard deviation is the same for all groups. For the organization which is presently under consideration also, those assumptions are taken to be valid. Even otherwise fortunately ANOVA is somewhat robust (i.e. results remain fairly trustworthy despite mild variations of these assumptions). Suppose there are n branches for the organization. The problem to be discussed is whether the impact of the training is the same on all the branches or not. Let,.. be the mean difference performance scores for those n branches. Then the null hypothesis is that H 0 : = =..= (i.e.) impact is the same on all the branches. The alternate hypothesis is that atleast one mean difference performance score is different from atleast one other mean difference performance score. The hypothesis is tested with the F statistic at 5% or 1% significance level. The analysis is done, as explained in any book on statistics. Also an example of ANOVA can be seen in a later section. Then conclusion can be drawn accordingly about the impact of the training on the employees of the branches APPLICATION OF TUKEY S HSD TEST ANOVA technique is particularly useful in testing hypothesis about the differences of means in multiple groups because ANOVA utilizes only one single overall test. Sometimes the researcher is satisfied with conducting an overall test of differences in groups. However when it is determined that there is an overall difference in population means, it is often desirable to go back to the groups and determine from the data which pairs of means are significant

10 177 Such pairwise analyses can lead to the buildup of the Type 1 experimental error rate. Fortunately several techniques referred to as multiple comparisons have been developed to handle this problem. Multiple comparisons are to be used only when an overall significant difference between groups has been obtained by using the F values of the analysis of variance. This is called A posteriori or post hoc pairwise comparison. The two multiple comparison tests to be seen here are Tukey s HSD (Honestly Significant Difference) approach and Tukey-Kramer procedure. Tukey s HSD test is for designs with equal sample sizes and Tukey Kramer procedure is used for situations in which sample sizes are unequal. Tukey s HSD test was developed by John W.Tukey and presented in The method uses the studentized range distribution. Suppose we have r independent observations y 1, y 2,.. y r from a normal distribution with mean and variance 2. Let be the range for the set i.e. maximum minus the minimum. Now suppose that we have an estimate s 2 of the variance 2 which is based on degrees of freedom and is independent of the y i s. Then the studentized range is defined as = Using the number of treatment levels, the value of mean square error, the sample sizes and a table value of q, the HSD determines the critical difference necessary between the means of any two treatment levels, for the means to be significantly different. Once the HSD is computed, the researcher can examine the absolute value of any or all differences between pairs of means to determine whether there is a significant difference. The formula to compute a Tukey s HSD test is

11 178 HSD = Where MSE mean square error n Sample size N Total number of observations Level of significance q Critical value of the studentized range distribution (from Table) Many text books have tables of this distribution A copy of the table is attached at the end of this thesis. Tukey s method is a single step multiple comparison procedure and statistical test. It can be used on raw data or in conjunction with an ANOVA (Post-hoc analysis) to find means that are significantly different from each other AN EXAMPLE FOR THE APPLICATION OF TUKEY S HSD TECHNIQUE TO PERFORMANCE ANALYSIS Now an organization with more than 2 branches is to be considered. For example let there be 3 branches. Employees from all the three branches are given training To objective of the management is to find out the impact of training on the employees of the branches. For that purpose, performance scores of those employees are calculated both before the training and after that.let there be 3 branches Then a small sample of size n ( say 5 ) is taken for each branch and their difference of performance scores are computed. Let them be given as follows.

12 179 Difference of performance Scores I Branches II III Branch I Branch II Branch III Total n mean T = 125 N = 15 = 8.33 Before applying Tukey s method, we have to see whether it is needed i.e. we have to find out whether the means of the difference of performance scores of the 3 branches are equal or not. H 0 : = = H a : Atleast one of them is different from others. Then ANOVA technique is applied and the ANOVA table is as follows.

13 180 Table 8.1 ANOVA Table Sum of Squares Degree of freedom Mean Square Between Error F = 64.87/1.63 Total Table value of for =0.01 is 6.93 Thus > 6.93 So the null hypothesis is rejected i.e. atleast one mean difference of performance score is different from others. The management should understand that because job knowledge, quality of work, productivity etc. may differ with performance score, the differences may call for different organizational approaches in each branch. The significant F value says that the difference between the means of difference of performance scores is relatively greater than the difference of performance scores within each branch. Now for the same problem, because the sample sizes are equal, Tukey s HSD test can be used to complete multiple comparison test between branches I and II, II and III and III and I For c = 3, N-c = 12, = 0.01 the table value of q is 5.04, MSE = 1.63 (already calculated in ANOVA)

14 181 HSD = 5.04 = 2.88 Using this value of HSD, the difference between the means for any 2 branches can be examined. Any of the pairs of means that differ by more than 2.88 are significantly different at = Here are the differences for all three possible pairwise comparisons. - = = = = = = 3.4 All the differences are greater than the value of HSD which is Thus the mean differences of performance scores between any and all the pairs of branches are significantly different. Thus Turkey s HSD approach is specifically for comparing branch means in an ANOVA setting. Accordingly the impact of a training programme on employees is studied here through performance scores APPLICATION OF TUKEY-KRAMER PROCEDURE FOR TRAINING PROGRAMMES Tukey s HSD was modified by C.Y. Kramer in the mid 1950 s, to handle situations in which the sample sizes are unequal. The modified version of HSD is referred to as Tukey-Kramer procedure. The formula for computing the significant difference with this procedure is similar to that for the equal sample sizes, with the exception that the mean square error is divided in half and weighted by the sum of the inverses of the sample sizes under the root sign. HSD =

15 182 Where n r Sample size for r th sample n s Sample size for s th sample To see how this approach can be used to study the impact of a training programme on employees of different branches of an organization, we illustrate the following example. Let us have 4 branches for the organization. As in the previous example, here also 4 small samples are taken, one for each branch. But the sample sizes are not equal. The performance scores are calculated before and after the training. They are as follows Mean Table 8.2 =

16 183 Table 8.3 Degree Sum of Mean of Squares Square freedom F Between Error Total Table value = 3.10 For the above problem q = 3.96, c = 4, N = 24, = 0.05 For I and II critical difference = 3.96 = Actual Difference = - = = Because this result is less than the critical difference , there is no significant difference between the mean difference performance scores of I and II. In the same way we find the critical differences for other pairs. The values can be tabulated Table 8.4

17 184 For the pairs I & IV and II & IV since the actual difference is less than the critical difference there is no significant difference between the means. But for the other 3 pairs i.e. I & III, II & III and III & IV, the difference is significant. Hence we can conclude that the impact of the training on the pair of branches I and II, I and IV and II and IV are more or less the same. Thus, statistical methods such as paired t-test, independent samples t-test, ANOVA and Tukey s tests are useful in determining the impact of a training programme on the employees of an organization. Accordingly, the training programme is also evaluated. In the next section, application of Data Envelopment Analysis (DEA), a mathematical technique in performance appraisal is discussed in brief APPLICATION OF DATA ENVELOPMENT ANALYSIS (DEA) DEA method has been established in 1987 by Charnes, Cooper and Rhodes, which was later modified into one of the scientific managerial methods of performance evaluation. This method applies to rational evaluation of the decision maker units (DMU), utilizing mathematical programming. Balanced Scorecards (BSC) is a managerial tool including some criteria and evaluation and arranged as set of cards. These criteria will include all organization aspects, related to important managerial approach and will assist the top management, enabling them to obtain a widespread vision to their position and organization. One of the most important problems in proper performance evaluation is organization performance appraisal, considering its strategy and targets. In their article [13] Fereydon Rahnamay Roodposhti and others have described comprising methods utilizing DEA and BSC to acquire most precise and secure evaluation of organization without any personal interference and also performance evaluation implementation through a comprising method which carries the entire path onto the main target in phasing algorithm mode. Assuring about mathematical model to evaluation of DEA, the performance evaluation will be easily implementable.

18 185 Already the methodology to compare the performance of a number of units is elaborately discussed in Chapter 2. Thus, that article gives an analytical method for organization performance evaluation using DEA and BSC techniques CONCLUSION In today s competitive business world, it is understood that organizations can compete with their rivals only by innovating and improving the performance of their employees. Improvement in performance can be measured by performance apprasisal.appraisal help develop individuals improve organizational performance and feed into organizational planning. Thus in this chapter application of statistical methods have been discussed to find whether there has been an improvement in the efficiency or effectiveness of related human resources development programs. It may be difficult, however if not impossible, to attribute the results of organizational performance to an employee appraisal program since an appraisal program is only one of many systems and processes that affect organizational outcomes.