Is Training Effective? Evaluation of transfer factors with FET in the Public Administration in Spain

Transcription

1 Is Training Effective? Evaluation of transfer factors with FET in the Public Administration in Spain Authors, organisation affiliation and position: Dr. Pineda-Herrrero, Pilar AssistantProfessor, Autonomous University of Barcelona BA.Espona, Berta Researcher, Autonomous University of Barcelona MA.García, Natalia Researcher, Autonomous University of Barcelona BA.Ciraso, Anna Researcher, Autonomous University of Barcelona MA. Quesada-Pallares, Carla PhD Student, Autonomous University of Barcelona Abstract In this paper are presented the results of the ETAPE study, whose aim is to evaluate the effectiveness of continuing training in public administration in Spain. To evaluate that, the FET model was applied to show which factors influence the transfer of training to the workplace. FET-ETAPE questionnaire (Pineda, Quesada & Ciraso, 2011) is one of the instruments of the methodology of this study, which combines quantitative data analysis with qualitative method in a non-simultaneous way, that is, in different phases. The most important results of this study are, on the one hand, a methodological and scientific progress that makes possible to predict learning of participants in training, the intent to transfer of this learning and the application in the workplace; and, on the other hand, the factor that influences the most the transfer is orientation towards job s requirements. Keywords: continuing training, public administration, transfer of training, factors of transfer 1. Introduction Currently, continuing training plays an important role in human resource policies, since it is a key strategy for organizations to achieve their objectives and to enable workers professional development within the organization. Today continuous learning is essential.

2 In a situation of rapid economic, political, and technological changes, training has been increasingly considered as a key element in the development of human resources management, and in the changes in the workplace (Bossaert, 2008). Companies and institutions invest many resources in training their employees, but rarely know to what extent the training is effective. To solve this problem, it is necessary to conduct an evaluation of the training to verify the effectiveness and cost of training, in terms of transfer of learning. The relevance of transfer consists of evaluating to what extent the results obtained during training (learning) contribute to the development of professionals in the organization, i.e. the extent to which training participants apply the acquired learning in their workplace. For this reason, this study focuses on the issue of evaluating the transfer of training, specifically, on the continuing training of workers in the Spanish public administration. 2. Theoretical background The term "transfer" was first coined in 1988 (Baldwin and Ford, 1988). These authors defined it as the degree to which participants apply the knowledge, skills, and attitudes acquired in the context of occupational training. In order for the transfer takes place, the learned behaviour must be generalized to the context of work and must be maintained in the workplace for certain period of time (Pineda, 2002). For these reasons, the evaluation of transfer of training becomes a priority to know the effectiveness of training within the work context. However, an exhaustive process of evaluation of transfer requires many human and financial resources due to the difficulty of measuring changes caused by training in the workplace. Because of this, several authors raised the possibility of evaluating transfer indirectly, through the factors that influence the applicability of learning to the workplace. Such is the case of the models of Baldwin and Ford (1988), Noe (1988), Rouiller and Goldstein (1993), Thayer and Teachout (1995), Holton (1996, 2005), Burke and Hutchins (2008), among others. From the above models, the model of Holton (version 2005) stands out as the transfer factor model that includes more elements. It groups them into three dimensions: motivational factors, contextual factors, and capacity factors. The studies of the EFI-GIPE research team from the Autonomous University of Barcelona are focused on evaluating the effectiveness of continuing education from the perspective of the implementation of training in the workplace, that is, from the perspective of transfer. These studies have been done from two points of view (Pineda, Quesada & Ciraso, 2011):

3 Direct evaluation of the transfer of training. It consists of creating specific instruments to measure the actual transfer of learning in the workplace (Pineda et al., 2010; Moreno, Quesada, & Pineda-Herrero, 2010, Pineda et al., 2011a, 2011b). Indirect evaluation of the transfer of training. It involves the search for alternatives to the measurement of transfer in order to avoid the cost and difficulty of direct evaluation. The diagnosis of the factors that influence the transfer is the best way to do the indirect assessment. By detecting barriers and facilitators of transfer in an organization, we can predict whether or not there will be transfer and introduce the needed corrections. Thus, factors act as predictors of transfer, allowing an indirect evaluation of it (Pineda et al, 2010; Pineda-Herrero, Quesada, & Espona, 2011; Quesada, 2010, Pineda-Herrero, Quesada, & Stoian, 2011). These studies allowed Pineda, Moreno and Quesada (2010) to develop a transfer factor model for the Spanish context in order to measure the variables that could facilitate or hinder the transfer of learning in the workplace. It is called Evaluation of Training Transfer (ETF). It was administered to 1044 workers in Spain (Pineda, ed. 2010). From its validation six factors emerged (see Figure 1). Figure 1. The ETF factor model empirically validated. Source: Adapted from Quesada, Espona & Pineda (2011) The results allowed the validation of the instrument and the analysis of the factors that determine the transfer in the Spanish context. The model also showed a significant relationship with transfer and a medium-high predictive potential.

4 Thanks to the results of the application of the ETF in 2010, the research of Quesada (2012) focused on developing the Exploratory Model of Transfer Influential Variables (EMTIV) in training for the Public Administration of Catalonia, it was developed and implemented a more complex model on the Factors for Transfer Evaluation in the Spanish Public Administration, i.e. the FET-ETAPE model. The aim of this paper is to present the results of the ETAPE study conducted during 2011 in which the FET model was applied to evaluate the transfer of continuing training received by workers of the Spanish public administration. 3. Methodology The study was undertaken primarily to evaluate the effectiveness of continuing training in public administration in order to improve it. The question that guided the research was: what factors influenced the transfer of training to the workplace in the context of the Spanish public administration? We hypothesized that these factors were related to three dimensions: participant, training, and organization Method The methodological approach was mixed and non-simultaneous, which combined the collection of quantitative and qualitative data at different stages. In a first qualitative phase, we performed a review of the literature on factors that may influence the transfer of training. This first approach was validated with the factors responsible for the training of each of the five Spanish regions that participated in the study. The second phase of the study was quantitative and consisted of the application of four questionnaires to a sample of 1527 employees of the public administration of the five selected Spanish regions. These questionnaires were: (1) the Initial Questionnaire of Skills (CIP), which evaluated the initial degree of mastery of the skills to be trained. It was applied before starting the training. (2) The Factors for Transfer Evaluation Questionnaire (FET-ETAPE), which measured the factors influencing the transfer of training to the workplace. It was applied just before completion of training. (3) The Deferred Transfer Questionnaire (CTd), which was applied between 2 months and 2 ½ months after completion of training to assess the degree of application of learning to the workplace. The Final Questionnaire of Skills (CFC), which was applied along with the CTd, to assess the degree of improvement in the mastery of skills of training and the use of these skills after completing the training process Sample To define the sample, we conducted a non-probabilistic multistage sampling (Hernández, Fernández-Collado & Baptista, 2008) according to various criteria selected from the characteristics of our research. The first criterion was related to the content of training. Thus, we selected three content areas: technological,

5 legal, and social skills. The second criterion was temporary, that is, to select the training to be evaluated we chose the ones held during the months of April, May, and June From here, the third criterion was to choose to follow the training according to its modality: classroom or elearning Techniques The FET-ETAPE questionnaire (Pineda, Quesada & Ciraso, 2011) consists of 59 items, which must be valued on a 5-point Likert scale. The items allow to assess the 8 factors of the FET-ETAPE model as well as the achieved learning (5 items) and the intent to transfer (4 items). The factor satisfaction with training consists of 8 items and refers to the participant s reaction to the training and to the role of the trainer. (Tannenbaum et al., 1991, Ford & Kraiger, 1995; Holladay & Quiñones, 2003). The accountability consists of 8 items and refers to the degree to which the manager asks the participant evidence of changes resulting from the implementation of training (Baldwin, Magjuka & Loher, 1991, Ford et al., 1992, Burke & Baldwin,1999, Clarke, 2002; Russ-Eft, 2002; Chiaburu & Marinova, 2005). On the other hand, the factor orientation towards job s requirements, consisting of 7 items, is defined as the relationship or similarity of training and materials with the needs of participants work position and the development of their careers (Baldwin & Ford, 1988; Rouiller & Goldstein, 1993; Ford & Kraiger, 1995; Alliger et al., 1997; Ruona et al., 2002, Holladay & Quiñones, 2003, Lim & Morris, 2006). The factor environment opportunities for application is composed by 10 items and refers to elements external to the participant that may influence the application of the learning acquired in training to the workplace (Noe & Schmitt, 1986, Tannenbaum et al., 1991, Holton, Bates, & Ruona, 2000; Clarke, 2002;Gaudine & Saks, 2004). The motivation to transfer (5 items) refers to the involvement and the desire of participants to apply the training to their workplace (Ford et al., 1998, Holton, Bates & Ruona, 2000). The internal locus of control, consisting of 5 items, refers to the participant s perception that his/her success or failure in the transfer of training to his/her workplace depends on him/her (Noe & Schmitt, 1986, Tannenbaum et al. 1991, Saks, 1995; Chiaburu & Marinova, 2005). Finally, the factors peer s support (4 items) and manager s support (3 items) refer to the degree to which partners (Xiao, 1996; Chiaburu & Marinova, 2005) and managers (Ford et al., 1992, Burke & Baldwin, 1999, Russ-Eft, 2002; Chiaburu & Marinova, 2005), respectively, support the participants during the process of application of learning to their workplace. In the last phase of qualitative type, we conducted two groups of dicussion. The first group was comprised of participants in training from just one region. The second group was comprised of those responsible for training and managers with experience in training from all the regions. These discussion groups allowed us to better interpret the results obtained with the questionnaires, after a descriptive analysis, mean comparison analysis or ANOVA, and multiple regressions with SPSS.

6 4. Results The results presented below refer to the sample G1 of the study, consisting of training participants who completed the questionnaire FET-ETAPE (transfer factors) and the CTd (deferred transfer). It was composed of 1142 workers from the Spanish public administration. As shown in the following figures, they were mostly women (66%). Among the job titles, skilled workers (33%) and technicians (35%) are highlighted. The management positions group is the least numerous (3%). The sample was balanced according to the content area, with a slight majority of classroom training compared to the elearning modality. The average age of the surveyed participants was 43 years. Figure 2. Distribution of participants by sex. Source: Authors. Figure 3. Distribution of participants by job tittle. Source: Authors. Men Women Figure 4. Distribution of participants by training s content area. Source: Authors. Figure 5. Distribution of participants by training delivery. Source: Authors. The following figure displays the results of descriptive analysis of the transfer factors. The data are placed in a 5-point Likert scale. Values below 2 are considered a barrier to transfer. Values between 2 and 3 may be a risk of barrier. Values between 3 and 4 represent a weak transfer facilitator. Finally, the values above 4 represent a strong facilitator. It is noted that only two factors, satisfaction with training and motivation to transfer, obtain results that place them as strong facilitators of transfer. Meanwhile, accountability, with an average value of 2.60, is positioned as a potential risk barrier to transfer. The other factors are weak facilitators.

7 Figure 6. Descriptive analysis of transfer factors. Source: Authors Figure 7 shows a description of the analysed training results, i.e., the intent to transfer and the achieved learning, which were evaluated using the FET- ETAPE instrument, and the transfer perception from the CTd questionnaire. As noted, the first two variables get a high score, while the perceived transfer is closer to 3 with a medium-high result. This result could indicate that although participants have learned during training and are willing to transfer, they do not believe they have applied learning in their workplaces to the extent one would expect. Figure 7. Descriptive analysis of training results. Source: Authors

8 In order to establish causal relationships among transfer factors and the hypothetical independent variables achieved learning and intent to transfer, towards the perceived transfer, we used simple and multiple regressions. First, and after checking that none of the assumptions of the regression were infringed, all variables were introduced, considering the achieved learning as dependent variable. The resulting adjusted R 2 of the model was However, it was observed that some variables were not significant in T. The analysis was repeated excluding intent to transfer, environment opportunities for application, internal locus of control, and manager s support. We obtained the model presented in Table 8. Table 8. Multiple regression towards achieved learning. Source: Authors Independent variables B SE B ß (Constant) Satisfaction with training ** Accountability * Orientation towards job s requirements ** Motivation to transfer ** Peers support to transfer * Also, we conducted a multiple regression of factors on the variable intent to transfer, introducing all the factors and the variable achieved learning. A model emerged that explained 67.4% of the variance, but accountability, peer s support, manager s support, and achieved learning were not significant in T. We then proceeded with other multiple regressions, excluding non-significant variables, and emerged a model with a satisfactory adjusted R 2 (0.673) of 5 factors, as shown in the table below. Table 9. Multiple regressions towards intent to transfer. Source: Authors Independent variables B SE B ß (Constant) Satisfaction with training ** Orientation towards job s requirements ** Environment opportunities for the application ** Motivation to transfer ** Internal locus of control ** We also performed a multiple regression of all factors towards the perceived transfer. A model emerged that explained 33% of the variance, but we found that 3 factors were not significant: motivation to transfer, peer s support, and manager s support. Excluding these factors, we obtained a model with an adjusted R 2 of 0.329, as shown in Table 10. This implies that the developed

9 model can explain almost 33% of the variance of the transfer. This percentage may be considered appropriate in social sciences (in the absence of similar studies in our context, it is assumed that the R 2 obtained indicates a large effect, following the advice of Cohen, 1988). Table 10. Multiple regressions towards perceived transfer. Source: Authors Independent variables B SE B ß (Constant) Satisfaction with training ** Accountability ** Orientation towards job s requirements ** Environment opportunities ** Internal locus of control ** Finally, we analysed the simple regressions among the independent scales achieved learning, intent to transfer, and perceived transfer, using the latter as dependent variable. In Table 11, we can observe that the three relationships were significant at 99%. Table 11. Simple regressions among independent scales. Source: Authors Independent variables Dependent variables ß R 2 Achieved learning Intent to transfer,49**.24 Achieved learning Perceived transfer,37**.14 Intent to transfer Perceived transfer,37**.16 By integrating the models and the results of simple regressions, we obtained the general model presented below. Regarding the factors that are specifically related to the transfer results, it appears that the orientation towards job s requirements has a higher coefficient, and therefore has a greater weight. The other independent scales, i.e. achieved learning and intent to transfer, also have a relatively high impact in the transfer. In the general model, we noticed all the factors that emerged in the factor analysis, except manager s support. It is observed that the factor that has a higher coefficient, and therefore a greater weight, is the orientation towards job s requirements. It also has a statistically significant relationship with the three separate scales and with the satisfaction with training. The other independent scales, i.e. achieved learning and intent to transfer, also have a relatively high impact in the transfer.

10 Figure 12. General model of transfer factors. Source: Authors It is emphasized that the regression coefficient between satisfaction with training and the intent to transfer is negative, which indicates an inverse relationship. In discussion groups with participants in training, their managers, and those responsible for training, this phenomenon was related to participation in training due to personal interests and not to the needs of the workplace or the organization. Therefore, a course may obtain a high degree of satisfaction. However, there may be no intended application because the contents are not linked to the employment situation. This phenomenon would occur mainly in the free training, in which the worker is enrolled by his own will, rather than on courses that are strategic for a particular job. The relationship between accountability and achieved learning is another inverse one. This could be due to the organizational culture of the Spanish public administration, where workers perceive accountability as a process of control and monitoring only. This would adversely affect the learning of the trainees. To analyse these results and make suggestions for improvement, it is useful to consider the analysis of comparison of means between factors, according to the content area and the type of training. To this end, we conducted one-way ANOVAs with the overall sample of the ETAPE study, which consisted of 1527 valid cases. In the locus of control there were significant differences according to the content area (technological, legal, social skills), F (2, 1523) = 23.89, p =.000. The Tukey test indicated that the group of participants from the legal field training (M = 3.40, 95% CI [3.61, 3.74]) obtained significantly higher values on this factor, compared

11 with the group of participants in the technological areas (M = 3.68, 95% CI [3.61, 3.74], p =.000), and in social skills (M = 3.69, 95% CI [3.62, 3.75], p =.000). In the intent to transfer variable we also observed statistically significant differences, F (2, 1522) = 10.99, p =.00, with the same pattern. The Games- Howell post-hoc test indicates that in participants in the legal area (M = 4.00, 95% CI [3.94, 4.06]) the values are significantly higher than those in the technological area (M = 4.16, 95% CI [4.11, 4.21, p =.00]) and in social skills (M = 4.16, 95% CI [4.11, 4.22], p =.000). These differences could be related to the characteristics of training in the area of legal content, as reported in the discussion group with participants in training. Often, legal training is related to innovations in the administrative procedure whose implementation does not depend on oneself, but on all those involved in the process and on their willingness to promote change. Possibly, the locus of control may be displaced more easily from the outside, because people do not clearly perceive their own control in the transfer. This could also be influencing the intent to transfer and inhibiting it. In the case of accountability on the implementation of training, we also detected significant differences between the three content areas, F (2, 1522) = 10.60, p =.000. Instead, as indicated by the Tukey test, the mean in the area of legal content (M = 2.72, 95% CI [2.65, 2.78]) is significantly higher than the mean of training in technology (M = 2.51, 95 % CI [2.44, 2.58], p =.000) and the mean of training in social skills (M = 2.57, 95% CI [2.51, 2.63], p =.003). Although the factor in this content area is less than a score of the facilitator of the transfer (2.72). These differences could be related to the reasons leading to enrol in training courses and, on the other hand, the characteristics of the various training, which lead to receive different considerations by the trainees managers. Furthermore, as expressed by some people in both groups of discussion, social skills training seem less consistent with more ethereal knowledge, workers tend to enrol in them due to personal interest rather than due to a need of the workplace or the organization and, therefore, no accountability process is carried out. We also detected significant differences in the factor peer s support, according to the content area, F (2, 1522) = 6.05, p =.002. The Games-Howell test informs us that in the case of social skills (M = 3.23, 95% CI [3.18, 3.29]) the trainings have less support compared to the legal content area (M = 3.39, 95% CI [3.32, 3.48], p =.001) and to technological area (M = 3.35, 95% CI [3.28, 3.41], p =.029). This difference may be related to an argument put forward by the participants in a discussion group. They indicated that social skills are a new training field towards which peers can probably have more resistance and mistrust, because it is unfamiliar. In addition, participants in training, managers, and training technicians said that certain envy might arise, especially when training involves time off work and it is not perceived as useful. This aspect can be seen also in the manager s support factor, where differences were significant (F [2, 1523] = 5.23, p =.005) among legal trainings (M = 3.52, 95% CI [3.44, 3.60]) in relation to the area of

12 social skills (M = 3.33, 95% CI [3.25, 3.41], p =.002), according to the application of the Games-Howell test. Also by content area, there are differences in achieved learning (F [2, 1523] =19.46, p =.000). The Tukey test indicates that the factor is significantly higher in training with technological content (M = 4.24, 95% CI [4.19, 4.29]) than the training with legal content (M = 4.07, 95% CI [4.01, 4.12], p =.000) and with social skills (M = 4.02, 95% CI [3.96, 4.07], p =.000). And finally, we find that the motivation to transfer is 0.10 higher in trainings in the technological area (M = 4.32, 95% CI [4.28, 4.36]) than in trainings in the legal area (M = 4.22,95% CI [4.17, 4.26], p =.002). The Tukey test confirmed that this difference is statistically significant. Regarding the training modalities, there are significant differences in factors satisfaction with training, accountability, orientation towards job s requirements, and environment opportunities for application (see Table 13). All differences favour the classroom modality over elearning. Table 13. Significant differences in the factors by type of training (classroom or elearning). Source: Authors Factor F df df2 p Satisfaction with training 27, Accountability Orientation towards job s requirements 5, Environment opportunities for the application The qualitative information gathered through discussion groups confirmed that, in general, classroom training often results in more favourable reactions than elearning. Many participants agreed that the latter, although it is more flexible, is more difficult because it requires greater discipline and self-regulation. ELearning is often chosen for reasons of balancing family and other commitments that would not allow to meet the attendance requirement of classroom training. However, it is assumed that there are components of the relationship with the trainer or with peers that affect the learning process, and cannot be replaced. Moreover, despite the greater flexibility of online courses, some participants noted that they could cause problems because the participant does not have time allotted for training. This may be one reason why the satisfaction with elearning is lower. A final explanation for this result, also based on qualitative data, is that there are workers who request specific classroom training but since there are no places, they are assigned to an elearning modality. This could lead to discontent that would influence the results of this factor.

13 Another significant difference (F statistic) that we detected by type of training refers to the accountability, i.e lower in elearning. This result can also be interpreted with the information from discussion groups. According to some participants in training, classroom courses are more evident because the workers are absent from the workplace, and it is easier that when they come back a manager ask them explanations on the training and subsequently on its implementation. However, in the case of elearning, they might not even know what is being done, which explains the general lack of established processes of accountability. The factor orientation towards job s requirements also shows significant differences (F statistic) by modality, placing classroom training 0.09 above elearning. According to the perceptions of participants in discussion groups, this result could be related to the number of students in the courses. In classroom courses the group is smaller, so it would be easier to personalize the training, adjusting it to the needs of participants and their specific jobs. Finally, we detect a significant difference (Welch statistic) in the factor environment opportunities for the application, which had a mean lower (0.09) in elearning trainings. 5. Discussion The evaluation of the transfer of training aims to measure its effectiveness in the improvement of the performance and professional functioning of organizations. This study demonstrates that the FET instrument allows to measure the factors of the transfer of training in a complete and exhaustive way, and provides an indirect measurement of transfer through its predictive capacity. The application of the FET instrument in the Spanish public administration shows that the possibilities of transfer are positive given the high satisfaction and motivation of the participants. The achieved learning and the intent to transfer also have high scores. However, there are factors that can clearly be improved. These include the factor accountability, which appears as a barrier. This may be due to the limited tradition in the Spanish work culture (especially in the public environment) of accounting for investments in training. Analyses have shown that the factors identified can readily predict learning of participants in training, the intent to transfer these learning, and its application in the workplace. This result constitutes an important methodological and scientific progress. Given the complexity of measuring training results directly in the workplace, the possibility of indirect assessment through factors provides a revolutionary new approach. The FET model makes it possible something so far unattainable: the evaluation of transfer with a single easy to apply questionnaire, which also provides very suggestive results for improving the effectiveness of training.

14 In the predictive model we found, which explains 33% of the variance of the transfer perceived by the participants, emerge key factors for achieving effective training: orientation towards job s requirements, which is the factor with more weight in transfer; satisfaction with training; workers perceived control of the implementation of learning in the workplace (internal locus of control); environment opportunities for application; and accountability to the manager on the results of training at work. Results allow to state that training will be effective, that is, it will produce transfer to the workplace, if it has the aforementioned factors with high scores. The general model we have developed suggests that in order to predict the results of transfer and act on the items that can be improved, we do not have to evaluate the learning achieved by participants of training and their intent to apply it in their workplace. Instead, the evaluation of the factors themselves can predict the extent to which public employees apply the skills acquired in their professional performance. This prediction of transfer has important implications because it allows to make improvements to increase the transparency and effectiveness of training. The factor that most influences the transfer is orientation towards job s requirements, which refers to the linking of training with the participant s employment situation; this factor can be improved with a rigorous analysis of participants needs for each training, or choosing materials, exercises and activities related with labour reality of workers. An effective continuing training is the best help that any professional can have on the current context of economic uncertainty and job loss. Guidelines of this kind, which come from the evaluation of the results obtained with the FET instrument, can greatly help organizations to improve the effectiveness of their training. Theses are the goals of our research: assessing training in an easy and accessible way, and helping organizations to improve the effectiveness of their training. 6. References Alliger, G.M., et al. (1997). A meta-analysis of the relations among training criteria. Personnel Psychology 50(2), Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: A review and directions for future research. Personnel Psychology, 41(1), Baldwin, T.T., Magjuka, R.J., & Loher, B.T. (1991). The perils of participation: Effects of choice of training on trainee motivation and learning. Personnel Psychology, 44, Bossaert, D. (coord.) (2008). Training and human resource development in the European Union member states. New practices and innovative trends. Maastricht: European Institute of Public Administration. Burke, L. A., & Hutchins, H. M. (2008). A study of best practices in training transfer and proposed model of transfer. Human Resource Development Quarterly, 19(2),

15 Burke, L.A., & Baldwin, T.T. (1999). Workforce training transfer: A Study of the effect of relapse prevention training and transfer. Human Resource Management, 38(3), Chiaburu, D.S., & Marinova, S.V. (2005). What predicts skill transfer? An exploratory study of goal orientation, training self-efficacy, and organizational supports. International Journal of Training and Development 9(2), Clarke, N. (2002). Job/work environment factors influencing training effectiveness within a human service agency: Some indicative support for Baldwin and Fords transfer climate construct. International Journal of Training and Development 6(3), Cohen, J, (1988). Statistical Power Analysis for the Behavioral Sciences. 2nd. edit., Hillsdale, N.J.: Erlbaum. Ford, J.K., & Kraiger, K. (1995). The application of cognitive constructs and principles to the instructional systems model of training: Implications for needs assessment, design, and transfer. In C.L. Cooper & I.T. Robertson, ed., International Review of Industrial and Organizational Psychology 10, Ford, J.K., Quiñones, M.A., Sego, D.J., & Sorra, J.S. (1992). Factors affecting the opportunity to perform trained tasks on the job. Personnel Psychology 45(3), Ford, J.K., Smith, E.M., Weissbein, D.A., Gully, S.M., & Salas, E. (1998). Relationships of goal orientation, metacognitive activity and practice strategies with learning outcomes and transfer. Journal of Applied Psychology 83, Gaudine, A.P., & Saks, A.M. (2004). A longitudinal quasi-experiment on the effects of posttraining transfer interventions. Human Resource Development Quarterly 15(1), Hernández, R., Fernández-Collado, C & Baptista, P. (2008). Metodología de la investigación, 4ª ed. México: McGraw-Hill. Holladay, C.L., & Quiñones, M.A. (2003). Practice variability and transfer of training: The role of self-efficacy generality. Journal of Applied Psychology 88(6), Holton, E. F., Bates, R. A., & Ruona, W. E. A. (2000). Development of a generalized learning transfer system inventory. Human Resource Development Quarterly, 11(4), Holton, E. F.,III, & Kirkpatrick, D. L. (1996). The flawed four-level evaluation model. Human Resource Development Quarterly, 7(1), Holton, E. F.,III. (2005). Holton's evaluation model: New evidence and construct elaborations. Advances in Developing Human Resources, 7(37), Lim, D.H., & Morris, M.L. (2006). Influence of trainee characteristics, instructional satisfaction, and organizational climate on perceived learning and training transfer. Human Resource Development Quarterly 17(1), Noe, R. A. (1996). Trainees' attributes and attitudes: Neglected influences on training effectiveness. Academy of Management Review, 11(4), Noe, R.A., & Schmitt, N. (1986). The influence of trainee attitudes on training effectiveness: Test of a model. Personnel Psychology 39,

16 Pineda, P. (2002). Gestión de la formación en las organizaciones. Barcelona: Ariel. Pineda, P., Quesada, C., & Ciraso, A. (2011). Evaluating Training Effectiveness: results of the FET Model in the Public Administration in Spain. 7th International Conference on Researching Work and Learning. Shanghai: Normal University of China. Pineda, P., Úcar, X., Moreno, V. & Belvis, E, (2011a). Evaluation of teachers continuing training in the early childhood education. Teacher Development, 15, Pineda-Herreno, P., Belvis. E., Moreno, V., Duran-Bellonch, M. & Úcar, X. (2011b). Evaluation of training effectiveness in the Spanish health sector. Journal of Workplace Learning, 23. Pineda-Herrero, P., Quesada, C. & Stoian, A. (2011). Evaluating the e-learning efficacy in Spain: a diagnosis of learning transfer factors affecting e-learning. Procedia-Social and Behavioral Sciences Journal, 30. Pineda-Herrero, P., Quesada, C., & Moreno, M.V. (2010). The ETF, a new tool for Evaluating Training Transfer in Spain. 11th International Conference on Human Resource Development: Human Resource Development in the Era of Global Mobility. Pécs: University of Pécs. Quesada, C., Pineda-Herrero, P., & Espona, B. (2011). Evaluating the efficiency of the leadership training programs in Spain. Procedia-Social and Behavioral Sciences Journal, 30, Rouiller, J. Z., & Goldstein, I. L. (1993). The relationship between organizational transfer climate and positive transfer of training. Human Resource Development Quarterly 4(4), Ruona, W. E. A., Leimbach, M., Holton, E. F.,III, & Bates, R. (2002). The relationship between learner utility reactions and predicted learning transfer among trainees. International Journal of Training and Development, 6(4), Russ-Eft, D. (2002). A typology of training design and work environment factors affecting workplace learning and transfer. Human Resource Development Review, 1(1), Saks, A.M. (1995). Longitudinal field investigation of the moderating and mediating effects of self-efficacy on the relationship between training and newcomer adjustment. Journal of Applied Psychology 80, Tannenbaum, S.I., Mathieu, J.E., Salas, E., & Cannon-Bowers, J. A. (1991). Meeting trainees expectations: The influence of training fulfillment on the development of commitment, self-efficacy, and motivation. Journal of Applied Psychology 76(6), Thayer, P. W., & Teachout, M. S. (1995). A climate for transfer model. Armstrong Laboratory/Human Resources. Xiao, J. (1996). The relationship between organizational factors and the transfer of training in the electronics industry in Shenzchen, China. Human Resource Development Quarterly 7,