The Learning Transfer System Inventory (LTSI) translated into French: internal structure and predictive validity

Transcription

1 International Journal of Training and Development 11:3 ISSN The Learning Transfer System Inventory (LTSI) translated into French: internal structure and predictive validity Christelle Devos, Xavier Dumay, Michel Bonami, Reid Bates and Elwood Holton III The Learning Transfer System Inventory (LTSI, Holton et al., 2000) considers 16 factors likely to influence the transfer of training to the workplace. The purpose of this study is to translate the Learning Transfer System Inventory into French and to examine (1) the internal structure of the translated instrument; and (2) its predictive validity. The Learning Transfer System Inventory was administered to 328 participants from six companies during the week following the end of a training program. The transfer questionnaire was filled in by 106 of those participants 1 3 months later. The results showed that a principal component analysis reveals a factor structure very similar to the original structure: the 16 original factors are replicated. Second, seven factors display statistically significant correlations with transfer: learner readiness, motivation to transfer, transfer design, opportunity to use, transfer-performance expectations, performance-outcomes expectations and performance self-efficacy. Comparisons with four similar previous studies allow us to draw directions for future research on the instrument. Christelle Devos, University of Louvain, Department of Psychology, Place Cardinal Mercier, 10, 1348 Louvain-la-Neuve, Belgium. christelle.devos@uclouvain.be, 9600 Garsington Road, Oxford OX4 2QD, UK and 350 Main St., Malden, MA 02148, USA. Validity of the French version of the LTSI 181

2 Introduction One of the biggest challenges regarding training is the transfer of the newly learned skills to the workplace. According to the practitioners, less than 20 per cent of the skills and knowledge acquired in training are used on the job. Why is transfer so low? Beyond Kirkpatrick s work (see Kirkpatrick, 1998), who considered learning as the unique antecedent of transfer, and increasingly during the last 20 years, researchers have worked at identifying variables likely to foster transfer (for a review, see Baldwin & Ford, 1988; Ford & Weissbein, 1997; Salas & Cannon-Bowers, 2001). Empirical support was found for variables such as supervisor support (e.g. Facteau et al., 1995), peer support (Bates et al., 2000), role ambiguity, negative change, job stress (Bennett et al., 1999), intrinsic and extrinsic incentives (Facteau et al., 1995), training reputation (Facteau et al., 1995), relevance of training content (Bates et al., 2000), goal-setting posttraining intervention (Gist et al., 1990), self-efficacy, locus of control and conscientiousness (Colquitt et al., 2000). However, these factors have only recently been considered together as a unified set of variables, i.e. as a transfer system (e.g. Holton et al., 2000; Tracey & Tews, 2005). Until recently, research on transfer kept considering factors assumed to affect transfer independently from each other and overlooked the relations between them. Only a small number of studies investigated the relations that these variables could have through moderation or mediation effects (e.g. Colquitt et al., 2000; Smith-Jentsch et al., 2001). Some comprehensive taxonomies of these factors have been suggested, but they remain mainly at the conceptual level and have not been empirically tested yet (e.g. Baldwin & Ford, 1988). Nevertheless, an instrument containing an exhaustive and non-redundant set of factors affecting transfer, whose internal, discriminant and predictive validity would have been tested, besides being a valuable diagnostic instrument for practitioners, would be a valuable basis to understand the various processes that occur around training transfer how do these variables interact with each other, to what extent are they independent, how is it possible to model their influence on transfer. Holton et al. (2000) designed a questionnaire whose purpose is to investigate this system of variables the Learning Transfer System Inventory (LTSI). The aim of this study is to contribute to the improvement of this instrument. The Learning Transfer System Inventory The purpose of the Learning Transfer System Inventory (LTSI, Holton et al., 2000) is to assess the transfer system, i.e. all factors in the person, training and organization that influence transfer of learning to job performance (pp ). With this aim, the instrument measures 16 dimensions likely to influence training transfer; 11 specific factors, which relate to the particular training program the trainee was attending, and five general factors, which are likely to influence any training program conducted. The name, definition and one item of the factors are presented in Table 1. Development and internal structure of the LTSI The instrument was developed trough three successive steps. First, on the basis of extended interviews, Rouiller and Goldstein (1993) designed a questionnaire of transfer climate and tested its content validity and its predictive validity toward transfer. Second, Holton et al. (1997) brought some modifications to this instrument, assessed its dimensionality, found a 9-factor structure, and called it the Learning Transfer Questionnaire (LTQ). Third, on the basis of an empirical and theoretical literature review, Holton et al. (2000) further developed this instrument, from which emerged the LTSI. These authors modified the item composition of some scales, added seven new constructs to the questionnaire and divided the items into general versus specific scales. Holton et al. (2000) administered the LTSI to 1616 trainees. They conducted first- and second-order factor analyses. First-order factor analysis revealed a clean 11-factor structure among the specific scales and a 5-factor structure among the general scales. 182 International Journal of Training and Development

3 Table 1: The Learning Transfer System Inventory: definition of the scales and example of items Factor Definition Item examples Specific factors Learner readiness Motivation to transfer Positive personal outcomes Negative personal outcomes Personal capacity for transfer Peer support Supervisor support Supervisor sanctions Perceived content validity Transfer design The extent to which individuals are prepared to enter and participate in training. The direction, intensity and persistence of effort toward utilizing in a work setting skills and knowledge learned. The degree to which applying training on the job leads to outcomes that are positive for the individual. The extent to which individuals believe that not applying skills and knowledge learned in training will lead to outcomes that are negative. The extent to which individuals have time, energy and mental space in their work lives to make changes required to transfer learning on the job. The extent to which peers reinforce and support the use of learning on the job. The extent to which supervisors-managers support and reinforce use of training on the job. The extent to which individuals perceive negative responses from supervisors-managers when applying skills learned in training. The extent to which trainees judge training content to reflect job requirements accurately. The degree to which training has been designed and delivered to give trainees the ability to transfer learning on the job. Before the training I had a good understanding of how it would fit my jobrelated development. I get excited when I think about trying to use my new learning on my job. Employees in this organization receive various perks when they utilize newly learned skills on the job. If I do not utilize my training, I will be cautioned about it. My workload allows me time to try the new things I have learned. My colleagues encourage me to use the skills I have learned in training. My supervisor set goals for me that encourage me to apply my training on the job. My supervisor opposed the use of the techniques I have learned in training. What is taught in training closely matches my job requirements. The activities and exercises the trainers used helped me know how to apply my learning on the job. Validity of the French version of the LTSI 183

4 Table 1: Continued Factor Definition Item examples Opportunity to use General factors Transfer effortperformance expectations Performanceoutcomes expectations Openness to change Performance selfefficacy Performance coaching The extent to which trainees are provided with or obtain resources and tasks on the job enabling them to use training on the job. The expectation that effort devoted to transferring learning will lead to changes in job performance. The expectation that changes in job performance will lead to valued outcomes. The extent to which prevailing group norms are perceived by individuals to resist or discourage the use of skills and knowledge acquired in training. An individual s general belief that he is able to change his performance when he wants to. Formal and informal indicators from an organization about an individual s job performance. The resources I need to use what I learned will be available to me after training. My job performance improves when I use news skills that I have learned. When I do things to improve my performance, good things happen to me. People in my group are open to changing the way they do things. I am confident in my ability to use newly learned skills on the job. After training I get feedback from people about how well I am applying what I learned. Second-order analyses revealed two macrofactors among the specific scales (job utility and rewards) and one macrofactor among the general scales (climate). Further, cross-cultural studies were conducted in order to test the generalization of the structure. Chen et al. (2005), Khasawneh et al. (2006) and Yamnill (2001) translated the LTSI into Thai, Chinese and Arabic respectively, and investigated the factor structure of these questionnaires in Thailand, Taiwan and Jordan, respectively. These analyses revealed similar structures to those in the USA, although some slight differences did appear. Predictive validity of the LTSI Several studies have provided evidence of the predictive validity of the factors contained in the LTSI. First, Bates et al. (2000) found that, after controlling for learning and motivation to transfer, a training design variable (content validity), supervisor support variables (supervisor sanctions, supervisor support) and co-worker support variables (peer support, change resistance) produced statistically significant increments in explained variance in ratings of job performance. These findings are consistent with other research confirming the relationship between social support in the workplace and learning transfer from training (Cromwell & Kolb, 2004; Facteau et al., 1995; Montesino, 2002; Smith-Jentsch et al., 2001; Warr et al., 1999). 184 International Journal of Training and Development

5 Second, some scales of the LTSI, whose predictive validity have not yet been tested directly, refer to constructs that have received empirical support in the literature. Colquitt et al. (2000) and Dumay (2004) found a statistically significant relationship between self-efficacy and transfer. Bennett et al. (1999) supported the influence of workload, which is similar to the personal capacity for transfer factor. Facteau et al. (1995) found a statistically significant effect for intrinsic and extrinsic incentives, which are similar to the scales transfer-performance expectations and positive personal outcomes, respectively. Third, no studies specifically investigated the relation between transfer and the remaining constructs assessed in the LTSI, i.e. learner readiness, transfer design, performance coaching, negative personal outcomes and performance-outcomes expectations. However, a study by Bates and Khasawneh (2005) examined the combined effects of several of these factors, including transfer-performance expectations, performance-outcome expectations, performance self-efficacy, performance coaching and openness to change and found them, as a group, to be predictive of organizational innovation. In addition, Facteau et al. (1995) supported the role of compliance and training reputation, which are complementary to learner readiness. Similarly, transfer design can be considered similar to goal-setting and self-behavioural management techniques, whose influence on transfer has been empirically supported (Gist et al., 1990; Richman-Hirsch, 2001). Finally, performance coaching can be considered as similar to supervisor support, whose influence on transfer was empirically supported. Most of the factors of the LTSI are therefore likely to be related to transfer. The present research The internal structure of the LTSI that was revealed in the USA was globally replicated in Asia (Thailand and Taiwan) and in the Middle East (Jordan). Yet, some differences with the original structure occurred in each of these countries. More research was therefore needed in order to identify more clearly the strengths and weaknesses of the questionnaire in order to improve it. Furthermore, although previous studies suggested that transfer system is a cross-cultural construct, the LTSI had not yet been tested in a Western European culture. Consequently, we translated the LTSI into French and investigated its factor structure in Belgium in order to test whether or not the original structure would replicate. Second, although some of the constructs of the LTSI had been found to predict transfer in various studies, the predictive validity of the whole instrument had not yet been tested. We therefore used the LTSI to predict effective transfer of training to the workplace one to three months after the training program. Method Procedure and sample The LTSI has been completed at time 1 by 328 trainees (response rate = 41 per cent), within an average of 6.8 days after the last day of the training program. The mean age was 39.4, and 58.6 per cent of this sample were men. Trainees were from six different organizations located in the French-speaking community of Belgium. One was a government organization (n = 188), and the five others belonged to the private sector (n = 140) in order for the study to be representative of these two sectors. Four of these five companies were international groups (two human resource consultancy companies, one manufacturing company and one distribution group). The fifth one was a Belgian socially-oriented company of health insurance. The questionnaire on training transfer was completed at time 2 by 106 of the 328 initial participants (response rate = 31 per cent). Their mean age was 39.5, and 58.5 per cent were men. Over 40 different training programs were included in the study. They focused on personal development (e.g. stress management), technical skills (e.g. computer pro- Validity of the French version of the LTSI 185

6 grams) or technical knowledge (e.g. insurance legislation). The mean length of the training programs was 2.7 days, and 47 per cent were only 1 day long. A researcher ed the LTSI to the participants within the week following the last day of the training (time 1). This period was the most appropriate because some scales focused on the time just after training (e.g. motivation to transfer), whereas other scales required that the trainees were back at their job (e.g. supervisor support). Participation was voluntary and trainees were asked to send back the completed questionnaire by mail or directly to the researchers. A reminder was sent to the trainees 1 week after the first . We used the same procedure for the questionnaire on training transfer, which was sent to the participants 1 3 months after the last day of the training program (time 2). The questionnaires were anonymous and a code was used for the matching of the two questionnaires. Instruments The Learning Transfer System Inventory The LTSI contains 68 items that measure 16 dimensions likely to influence transfer. Answer scales were Likert-type scales ranging from 1 (strongly disagree) to 5 (strongly agree). We translated the instrument by a forward-backward process (see Chen et al., 2005). First, two bilingual translators translated independently the LTSI into French. We compared and synthesized these two translations and developed a French version of the LTSI (FLTSI). A bilingual committee of two other researchers checked the accuracy of the FLTSI. Each problematic item was discussed and revised until agreement was reached. Second, a third bilingual translator translated the FLTSI back into English. The two English versions (the original and the back-translated one) were finally sent to the original authors of the LTSI, who pointed some differences of meaning, which were consequently corrected. Transfer of training The training transfer questionnaire (nine items; a=0.91) was a self-report measure of the extent to which the participants were transferring the skills and knowledge learned in training to their workplace. It was based on Facteau et al. s (1995) and Xiao s (1996) transfer questionnaires. Three items measured transfer directly (e.g. I use this training in my job whenever I have the opportunity to do so ) and four items measured transfer indirectly, i.e. the consequences of transfer (e.g. The quality of my work has improved after using the new skills I learned in training ). Participants were asked to answer on a 5-point response scale ranging from 1 (strongly disagree) to 5 (strongly agree). A principal component analysis revealed a single-component structure explaining 60.5 per cent of the variance. The mean of the loadings was 76.8 and five of the nine items loadings were larger than Data analysis The dimensionality of the LTSI was investigated by a principal component analysis with an oblique rotation (Oblimin). Because the Listwise option was used, participants with missing data were removed and this analysis therefore considered 298 participants only for the specific factors and 302 for the general factors. However, this number of subjects was sufficient to run a component analysis because the ratio between the number of subjects and the number of items was 4.4:1, which is higher than the ratios recommended by some authors: 3:1 (Guadagnoli & Velicer, 1988; West, 1991) or 2:1 (Foster, 2001; Kline, 1994). Principal component analysis (PCA) was chosen over exploratory common factor analysis (EFA) because the objective of this study was more on data reduction than the identification of a small number of fundamental underlying constructs that cannot be measured directly (Everitt, 1999). According to Gorsuch (1997), EFA is the general case of which component analysis is a special case, and the only difference between the two approaches is that EFA considers that a part of the variance of each item is not 186 International Journal of Training and Development

7 accounted for by the latent factors. In this view, EFA is preferable to PCA. However, Fabrigar et al. (1999) stress that it is a mistake to believe that PCA is a type of EFA, when in fact these procedures are different statistical methods designed to achieve different objectives (for a discussion, see Bentler & Kano, 1990; McArdle, 1990; Mulaik, 1990). Therefore, because the purpose of this study was more on determining linear combinations of the measured variables that retain as much information from those variables as possible (PCA), rather than to understand the structure of correlations among measured variables by estimating the pattern of relations between the common factor(s) and each of the measured variables whose content in not necessarily linked to the content of the latent constructs (EFA), the principal component approach was more appropriate. Second, oblique rotation was chosen over orthogonal rotation because the first approach constrains the factors to be uncorrelated, whereas the second approach does not restrict intercorrelations between the factors (Gorsuch, 1997). Several strengths of this approach can be highlighted. First, there is substantial theoretical and empirical basis for expecting psychological constructs to be correlated with one another in reality (Fabrigar et al., 1999). Specifically, because they all relate to a more general concept of transfer system, the items of the LTSI are not only expected to correlate with the other items of their scale, but are also likely to correlate with items from other scales of the instrument. Oblique rotation is therefore a more accurate and realistic representation of how constructs are likely to be related (Fabrigar et al., 1999; Rennie, 1997). Second, whereas orthogonal rotation restricts the factors to be uncorrelated, oblique rotation does not require them to be correlated. If the solution with the best simple structure involves orthogonal factors, an oblique rotation will provide estimates of the correlations among factors that are close to zero and produce a solution that is quite similar to that produced by an orthogonal rotation. However, in situations in which the best simple structure is a solution with correlated factors, oblique rotation will produce solutions with correlated factors (Fabrigar et al., 1999; Gorsuch, 1997). Third, oblique rotation provides more information than orthogonal rotation because it provides the estimates of the correlations between the factors. Further, the existence of substantive correlations among factors suggests that higher factors may exist, which would be overlooked by orthogonal rotation (Fabrigar et al., 1999). Finally, because the purpose of this article is to compare the factor structure that will emerge in Europe with those previously revealed in other countries, we used the method that was adopted in these studies, which was an oblique rotation. In order to assess the predictive validity of the LTSI, we used correlation analyses because they offered the clearest presentation of the results. To that aim, we computed the mean of the items composing each scale on the basis of the results that emerged from the component analysis. Results Internal structure of the LTSI Before computing the factor analyses, we tested the Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO). The values of that index were 0.87 for the specific scales and 0.85 for the general scales, which indicates that the data was appropriate for exploratory factor analyses. Common factor analyses revealed an 11-factor structure among specific scales and a 5-factor structure among the general scales. They represent the 16 factors originally revealed in the US. The 11 specific factors explained 68.8 per cent of the variance. The five general factors explained 59.8 per cent of the variance. The pattern and structure coefficients of the items are presented in Table 2 (specific factors) and Table 3 (general factors). The correlations between the components are presented in Tables 4 and 5. Ten of the factors were exactly the same as in the original questionnaire, and six displayed little differences. Three of the 68 items (items 18, 41, and 43) displayed pattern coefficients below 0.40 and were therefore removed. Item 61 went from the factor Validity of the French version of the LTSI 187

8 Table 2: Communalities, pattern coefficients and structure coefficients of the specific components Items Communalities Supervisor support Transfer design Negative outcomes Learner readiness Capacity transfer Content validity Peer support Opportunity to use Motivation Supervisor sanction Positive outcomes (0.90) (0.89) (0.84) (0.83) (0.83) (0.79) (0.85) (0.78) (0.76) (0.77) (-0.49) (0.66) (-0.41) (0.81) (0.41) 0.64 (0.77) 0.20 (0.43) (0.67) (-0.45) (0.43) 0.48 (0.62) 0.38 (0.53) (0.87) (0.78) (0.24) (0.78) (0.68) (-0.32) (0.86) (0.85) (-0.55) (-0.37) (-0.87) 188 International Journal of Training and Development

9 (-0.88) (-0.82) (-0.83) (-0.86) (-0.84) (0.43) (-0.71) (0.84) (0.80) (-0.36) (-0.61) (-0.82) (-0.80) (-0.30) (-0.77) (-0.83) (0.82) (0.72) (-0.22) 0.61 (0.68) (0.82) (0.46) 0.59 (0.69) (0.50) 0.57 (0.63) (0.40) (-0.42) 0.23 (0.21) 0.22 (0.34) (-0.60) (-0.44) (-0.37) (-0.21) (-0.34) 0.39 (-0.52) (-0.50) Note: The loadings below 0.20 were omitted. Pattern coefficients refer to the unique relationships between each factor and each observed variable (i.e. uncontaminated by overlap among factors). Structure coefficients refer to the correlation between the factors and the variable. These coefficients are presented in brackets. Validity of the French version of the LTSI 189

10 Table 3: Communalities, pattern coefficients and structure coefficients of the general components Items Communalities Performance-outcomes Openness to change Self-efficacy Transfer-performance Performance coaching (0.76) 0.02 (0.17) (0.77) (0.76) (0.72) 0.24 (0.49) (0.67) (-0.42) (-0.64) (0.84) (0.82) (-0.74) (0.68) (0.61) (-0.68) (0.78) (0.80) (0.75) (0.73) (-0.83) (-0.82) (-0.79) (0.40) (-0.67) (0.79) (0.81) (0.54) 0.63 (0.75) Note: The loadings below 0.20 were omitted. 190 International Journal of Training and Development

11 Table 4: Correlation matrix of the specific components (1) Supervisor support (2) Transfer design (3) Negative outcomes (4) Learner readiness (5) Capacity for transfer (6) Content validity (7) Peer support (8) Opportunity to use (9) Motivation (10) Supervisor sanction (11) Positive outcomes Validity of the French version of the LTSI 191

12 Table 5: Correlation matrix of the general components (1) Performance-outcomes (2) Openness to change (3) Self-efficacy (4) Transfer-performance (5) Performance coaching performance coaching to the factor performance-outcomes expectations, and item 42 went from the factor content validity to the factor transfer design. Reliability analysis indicated Cronbach alphas that ranged from 0.64 (supervisor sanctions) to 0.93 (supervisor support). Ten of the scales had an alpha superior to Finally, the correlations between the components were low to moderate. Motivation to transfer was highly correlated with transfer design (r = 0.41) and content validity (r = 0.28); and these two components were also correlated (r = 0.29). Supervisor support was highly negatively correlated with peer support (r =-0.35). The other components share lower correlations. Predictive validity of the LTSI Table 6 presents the correlations and the part of variance explained between the factors of the LTSI and training transfer. Seven factors showed statistically significant correlations with transfer, four of these being significant at the 0.01 level, and two being significant at the level: learner readiness (r = 0.26, p < 0.01), motivation to transfer (r = 0.43, p < 0.001), transfer design (r = 0.39, p < 0.001), opportunity to use (r = 0.25, p < 0.01), transfer-performance (r = 0.26, p < 0.01), performance-outcomes (r = 0.23, p < 0.05) and self-efficacy (r = 0.26, p < 0.01). Discussion The purpose of this study was to rigorously translate the LTSI into French and to investigate the internal structure and predictive validity of the translated instrument. A stream of research had been successively working at the development of the LTSI and this study brought this process one step further. First, the initial factor structure of the LTSI was successfully replicated within the Belgian sample: the 16 original factors did emerge. Only four of the 68 items were removed because they did not reach satisfactory pattern coefficients, and two items loaded on a different scale than the original one. These results complement the three previous studies that investigated the internal structure of the LTSI in the USA, Thailand and Taiwan. A comparison and synthesis of this stream of research is presented in Table 7, which allows for a sound identification of the strong and weaker scales and items. Second, seven of the LTSI factors are strong predictors of future transfer of training. These dimensions are (1) the extent to which individuals are prepared to enter and participate in training (learner readiness); (2) the motivation to transfer the new learning (motivation to transfer); (3) the degree to which training has been designed and delivered to give trainees the ability to transfer learning on the job (transfer design); (4) the expectation that effort devoted to transferring learning will lead to changes in job performance (transfer-performance expectations); (5) the availability of the necessary resources (opportunity to use); (6) the expectation that changes in job performance will lead to valued outcomes (performance-outcomes expectations); and (7) the degree of self-efficacy concerning transfer (self-efficacy). 192 International Journal of Training and Development

13 Table 6: Correlations between the LTSI components and transfer (1) Learner readiness (2) Motivation 0.26** (3) Positive outcomes 0.18** 0.16** (4) Negative outcomes 0.26** 0.28** 0.58** (5) Capacity transfer * 0.11* 0.10 (6) Peer support 0.37** 0.41** 0.34** 0.47** 0.10 (7) Supervisor support 0.17** 0.36** 0.34** 0.51** 0.13* 0.52** (8) Supervisor sanction ** 0.27** (9) Content validity 0.35** 0.43** 0.29** 0.35** 0.17** 0.45** 0.34** (10) Transfer design 0.27** 0.60** ** ** 0.26** ** (11) Opportunity to use 0.14* 0.22** ** 0.19** 0.21** -0.25** 0.29** 0.27** (12) Transfer-perf. 0.30** 0.43** 0.14* 0.23** 0.18** 0.36** 0.22** ** 0.46** 0.18** (13) Perf.-outcomes 0.16** 0.28** 0.49** 0.46** ** 0.57** ** 0.27** 0.23** 0.31** (14) Openness 0.18** * 0.35** 0.25** -0.27** 0.18** 0.14* 0.30** 0.14* 0.32** (15) Self-efficacy 0.29** 0.30** 0.14* 0.16** 0.18** 0.17** 0.14** ** 0.20** 0.24** 0.33** 0.21** 0.12* (16) Coaching 0.23** 0.30** 0.35** 0.38** ** 0.58** ** 0.24** 0.19** 0.29** 0.55** 0.32** 0.20** (17) Transfer 0.26** 0.43** ** 0.25** 0.26** 0.23* ** 0.15 R Note: n = 106 for the correlations between the LTSI scales and transfer. n = 328 for the intercorrelations between the LTSI scales. * p < ** p < LTSI = Learning Transfer System Inventory, Transfer-perf. = transfer-performance, Perf.-outcomes = performance-outcomes. Validity of the French version of the LTSI 193

14 Table 7: Synthesis of the four cross-cultural studies that investigated the internal structure of the LTSI Factors Item composition of the factors Comparison Belgium/USA USA Taiwan Thailand Belgium Factor status Specific factors Learner readiness 1, 7, 8, 9 (0.73) 1, 7, 8, 9 (0.73) 7, 8, 9 (0.58) 1, 7, 8, 9 (0.81) Idem + Motivation to transfer 2, 3, 4, 5 (0.83) 2, 3, 4, 5 (0.83) 1, 2, 3, 4 (0.76) 2, 3, 4, 5 (0.85) Idem + Positive outcomes 6, 11, 12 (0.69) 6, 11, 12 (0.69) 6, 11 (0.60) 6, 11, 12 (0.73) Idem + Negative outcomes 10, 14, 15, 16 (0.76) 10, 14, 15, 16 (0.79) 10, 12, 14, 15, 16 (0.84) 10, 14, 15, 16 (0.82) Idem + Capacity for transfer 13, 17, 18, 19 (0.68) / 19 (n/a) 13, 17, 19 (0.68) Item 18 missing +/- Peer support 20, 21, 22, 23 (0.83) 20, 21, 22, 23 (0.89) 20, 21, 22, 23 (0.8) 20, 21, 22, 23 (0.85) Idem ++ Supervisor support 24, 25, 26, 28, 29, 30 (0.91) 24, 25, 26, 28, 29, 30, (0.89) 24, 25, 26, 28, 29, 30 (0.85) 24, 25, 26, 28, 29, 30 (0.93) Idem ++ Supervisor sanctions 27, 31, 32 (0.63) 27, 32 (0.69) 27, 32 (0.66) 27, 31, 32 (0.64) Idem +/- Content validity 33, 34, 35, 41, 42 (0.84) 33, 34, 35, 36, 37, 38, 39, 33, 34, 35, 41, 42 (0.84) 33, 34, 35 (0.86) Items 41 and 42 missing +/- 40, 41, 42, 43 (0.93) Transfer design 36, 37, 38, 39 (0.85) / 36, 37, 38, 39 (0.79) 36, 37, 38, 39, 42 (0.86) Addition of item 42 + Opportunity to use 40, 43, 44, 45 (0.70) 44 (n/a) 44, 45 (0.58) 40, 44, 45 (0.70) Items 43 missing +/- General factors Transfer-performance 47, 48, 51, 53 (0.81) 48, 51, 53 (0.83) 47, 48, 51, 52, 53 (0.81) 47, 48, 51, 53 (0.81) Idem + Performanceoutcomes Openness to change 55, 56, 57, 58, 59, 60 (0.85) 46, 49, 50, 52, 54 (0.83) 49, 50, 52, 54, 61 (0.80) 49, 50 (0.63) 46, 49, 50, 52, 54, 61 (0.81) 55, 56, 57, 58, 59, 60 (0.80) 55, 56, 57, 58, 59, 60 (0.77) 55, 56, 57, 58, 59, 60 (0.82) Addition of item 61 +/- Idem ++ Self-efficacy 62, 63, 64, 65 (0.76) 62, 63, 64, 65, 68 (0.84) 62, 63, 64, 65 (0.77) 62, 63, 64, 65 (0.77) Idem + Performance coaching 61, 66, 67, 68 (0.70) / 61, 66, 67, 68 (0.61) 66, 67, 68 (0.74) Missing item 61 +/- Non-selected items / 13, 17, 18, 19, 31, 45, 46, 47, 66, 67 5, 13, 17, 18, 31, 40, 43, 46, 54 18, 41, 43 Note: The numbers in the parentheses are the Cronbach alphas values. Bold numbers represent reverse-scored items. The cutoff for item retention was 0.40 in the USA, Taiwan and Belgium, and 0.35 in Thailand. This table is inspired from Cross-cultural construct validation of the Learning Transfer System Inventory in Taiwan, by H.-C. Chen (2003), Unpublished doctoral dissertation, Louisiana State University. Reprinted with permission of the author. LTSI = Learning Transfer System Inventory. 194 International Journal of Training and Development

15 The integration of these two sets of results factor and predictive analyses lead to several considerations. The significant predictors of transfer can be categorized into four groups. The first group of variables focuses on the training program itself. Hence, transfer was found to be strongly predicted by learner readiness and transfer design. These factors explain 6.8 and 15.2 per cent of the transfer variance, respectively, and would still display a statistically significant correlation if n = 60, and n = 26, respectively. The first association complements Facteau et al. s (1995) findings that training reputation and compliance are related to transfer. The association between transfer design and effective transfer is suggested for the first time in this study, and complements the stream of research that investigates the influence of goal-setting and behavioural self-management on transfer (Gist et al., 1991; Marx, 1982; Richman-Hirsch, 2001). From a practical point of view, these findings show that training can be fostered if the trainees know what to expect from the training (e.g. have a clear idea of the expected outcomes of the training and of how the program is supposed to affect their performance), and that the trainers give lots of examples that show the trainees how they can use their learning on the job and make them feel confident about that. These findings are important for the training managers because these factors are more controllable than work environment- and individual-related factors. It shows that, although the influence of training-related factors have been overlooked in the scientific literature, the design of the training program has a strong influence on transfer. The second group of variables related to transfer are the factors based on Vroom s (1964) VIE (Valence-Instrumentality-Expectancy) model: transfer-performance expectations and performance-outcomes expectations. This study is the first to explicitly support the relevance of Vroom s model with regard to transfer, i.e. that participants will use their new skills if they expect that transfer will result in first-order outcomes (e.g. performance improvement), which are perceived to lead to valued second-order outcomes (e.g. the rewarding of a high performance). This model should therefore be used in more future studies on training transfer. Two other factors related to rewards and transfer consequences were found to be non-statistically significant predictors of transfer: positive personal outcomes and negative personal outcomes. A possible explanation for this result is the very low occurrence of financial rewarding for transfer in the companies where the study was run. Further, the relevance of the scale negative personal outcomes could be further considered because this construct is mainly pertinent for training programs that lead to obligatory use of the new skills (e.g. security training, training on a new computer program, etc.), which is the case for only a small proportion of training programs. However, although not statistically significant because of the small n, these two factors display some correlation with transfer (r = 0.18 and 0.15, respectively). Further, when an orthogonal rather than an oblique rotation is run, these two factors merge and predict transfer significantly. These two factors are therefore likely to have a substantial impact on transfer in workplaces where they are common practices. These findings refer to more organizational features than the first group of factors. They stress the importance of a sound needs analysis, that insures that the training skills will be useful for the workers and enable them to do their job better and increase their productivity. Further, they are related to the organizational culture and reward system. It is by acting at this level that the workers will feel that the organization does really value their performance and that people who get rewarded are the ones that do something to deserve it. However, although these factors have their roots in dimensions that go far beyond training, these dimensions need to be addressed during the program, for example by showing how the new skills learned can be used on the job in order to improve one s performance. The third group of variables deals with individual variables: motivation to transfer and performance self-efficacy. Motivation to transfer was the most significant predictor of transfer and it explained 18.5 per cent of the variance of transfer one to three months after training. Its influence on transfer confirms Dumay s (2004) findings. The role of self-efficacy confirms Colquitt et al. s (2000) meta-analysis. Individual variables seem to play an important role in training transfer, possibly as mediators of training- and environment-related variables. Hence, when using Baron and Kenny s (1986) methods Validity of the French version of the LTSI 195

16 to measure mediation and Sobel tests, one can observe that the scales motivation to transfer and self-efficacy mediate the relation between the other statistically significant variables and transfer. Although these factors refer to the individuals feelings, they are determined by organizational and training-related dimensions. For example, selfefficacy can be enhanced during the training through the four sources of self-efficacy highlighted by Bandura (cf. Bandura, 1997): making trainees master the newly taught skills, providing them with opportunities to observe someone who manages the transfer of the skills on the job, providing trainees with encouraging feedback and drawing their attention to positive physiological cues. Surprisingly, this study did not support the influence of social support variables on transfer. Although not trivial, e.g. supervisor support (r = 0.17), these correlations are lower than what can be observed in the empirical literature. These findings join van der Klink et al. (2001), who found no significant influence of supervisor support on transfer either. Nevertheless, we do not wish to advocate in this direction in regard to the numerous studies that found a positive relationship between social support variables and transfer (e.g. Montesino, 2002; Smith-Jentsch et al., 2001; Warr et al., 1999). Two explanations for this non-significant result can be considered. First, informal interviews showed that social support regarding training and performance-coaching in general is not a common practice in the companies in which we ran the study, and is therefore likely to barely influence employees behaviour. Second, maybe the level of aggregation was not appropriate. Hence, a higher level of analysis would possibly be more appropriate. For example, Tracey et al. (1995) found little predictive power for transfer climate and continuous learning culture considered at the individual level, whereas the relationship was stronger when the unit level of analysis was considered. Moreover, although some of these scales replicated exactly across studies (supervisor support, peer support, openness to change), others deserve some specification (cf. Table 7). For example, the factor supervisor sanction displayed low Cronbach alphas across studies and sometimes contained only two items. Further studies should therefore develop additional items to develop this scale. Finally, the extent to which trainees are provided with resources enabling them to use training on the job (opportunity to use) was a statistically significant predictor of transfer (r = 0.25), whereas the extent to which individuals have time, energy and mental space in their work lives to make changes required to transfer learning on the job (personal capacity for transfer) was the least correlated factor (r = 0.02). Since this last factor failed to replicate well across studies, it should be reinforced by additional items in further studies. Study limitations This study suffers from several limits, which should be overcome in future research. First, the measure of transfer was self-reported. Having an objective measure of transfer is improbable unless the transfer consists in, for example, the use of a computer program whose use can be recorded. Therefore transfer can be assessed either by the person himself, or by an external observer (e.g. supervisor, peers, subordinates, customers). Although the measure would increase its validity if coming from different sources, several studies have showed that the assessment of transfer by the participant and observers often varied greatly (Harris & Schaubroeck, 1988) and one could reasonably think that the one who owns the most knowledge about its behavior is the person himself. Nevertheless, in order to minimize potential effects of self-report bias, we identified steps that were suggested in the literature and incorporated them into our research design. First, participants had the opportunity to answer the survey anonymously. The questionnaires were ed directly to the participants, who sent them back to the researchers by or by post. They did not need to mention their name because an anonymous code was used for the matching of the two questionnaires. Second, since the participants were free to complete a printed version of the questionnaire, they could take it back home and answer it in total privacy so that they had control over their progress through the instrument (Richman et al., 1999). Third, we 196 International Journal of Training and Development

17 administered the LTSI and the transfer questionnaire at two different points in time in order to avoid contamination of the transfer measure. A second limitation is that we used the 68-item version of the LTSI, which was designed by Holton et al. (2000), although Chen et al. (2005) used an enlarged version of the questionnaire (89 items). Since the questionnaire was sent by and that its completion was voluntary, the first version was preferred to the longer version in order to maximize the number of participants. Furthermore, it was relevant, at this stage of development of the instrument, to synthesize the existing research on the internal structure of the 68-item version of the LTSI. Future research will therefore be able to rely on this synthesis and work at improving the scales that were identified as weak. We presented exploratory factor analyses rather than confirmatory factor analyses. Hence, the objective we were pursuing in the investigation of the internal structure of the questionnaire was to analyze the repartition of the items on the factors in order to compare the emerging pattern of results with previous validation studies. Because the LTSI is still in its developmental phase, the aim of this study was more to analyze the behavior of each single item rather than to confirm the replication of a well-established structure. In this way, we contributed to identify the strengths and weaknesses of the questionnaire in order to stimulate its development. However, when an improved version of the instrument is developed (the authors of the questionnaire are currently working on it), confirmatory factor analyses will be needed to test the replicability of its internal structure. Implications for practice and future research This study brought several contributions to the understanding of training transfer. First, it contributed to the development of the LTSI. From a practical perspective, such instruments help practitioners to establish a diagnosis of the transfer environment, which enable them to identify possible explanations of transfer problems, to assess potential weaknesses before conducting major training interventions, to target interventions designed to enhance transfer, etc. (Holton et al., 2000). In Europe specifically, companies are becoming more and more aware of the importance of the effectiveness and the return on investment that can be obtained from training. The assessment and improvement of transfer, which is the link between individual learning and organizational growth, is therefore becoming a top priority for many human resource managers. Further, a French version of the LTSI is likely to be of interest for the whole French community, ranging from Quebec to France and North Africa. From a research standpoint, this study contributes to the development of cross-cultural instruments with good psychometric properties, which stimulate international research designs, and allow the running of studies in various part of the world with psychometrically solid tests and to compare the results obtained. Further, a sound instrument of the transfer system encourages researchers to model the relations between the relevant variables, which will lead to a better understanding of the influence processes around training transfer. Second, this study provided a deeper insight into the antecedents of training transfer and identified new relevant variables with a longitudinal methodology. Indeed, this study complemented a stream of research that has developed during the last 20 years with the objective of identifying the dimensions practitioners can act on in order to foster transfer. This study outlined several research directions; it highlighted the relevance of training variables, which are often neglected in empirical research (Devos & Dumay, 2006), suggested the relevance of Vroom s (1964) model with regard to transfer, and brought some nuance with regard to social support. Further, whereas data in most correlational studies on transfer are collected at one point in the time only, this research based its conclusion on a longitudinal design, which allow for more confidence in the results. The results obtained in this study are very encouraging and future studies are required to improve the LTSI in various ways. First, the content of the LTSI could be further improved on the basis of the set of validation studies that have been run on the Validity of the French version of the LTSI 197

18 LTSI. The definition and item composition of several scales could be further specified (content validity and transfer design, personal capacity to transfer, supervisor sanctions and opportunity to use) and the relevance of other scales could be further considered (e.g. personal negative outcomes). Furthermore, some additional scales could be considered, for example, goal-setting/self-behavioral management techniques (Gist et al., 1991), training reputation (Laroche & Haccoun, 1999), and a broader operationalization of the opportunity to use (Ford et al., 1992). Third, future studies should design a shortened version of the questionnaire. The 16 first-order factors of the LTSI could be considered as mini-scales (Gorsuch, 1997). The second-order factors that were identified in the LTSI (Holton et al., 2000) would than be considered as first-order factors containing several mini-scales. Finally, future research should investigate the validity of the LTSI with structural equation modeling analyses. On the one hand, confirmatory factor analysis is needed to confirm the internal structure of the questionnaire. On the other hand, path analysis should be used to test the predictive validity of the transfer system, and to investigate the mediation effects between the constructs. References Baldwin, T. T. and Ford, J. K. (1988), Transfer of training: a review and directions for future research, Personnel Psychology, 41, 1, Bandura, A. (1997), Self-Efficacy: The Exercise of Control (New York: Cambridge University Press). Baron, R. M. and Kenny, D. A. (1986), The moderator-mediator variable distinction in social psychological research: conceptual, strategic and statistical considerations, Journal of Personality and Social Psychology, 51, 6, Bates, R. A. and Khasawneh, S. (2005), Organizational learning culture, learning transfer climate and perceived organizational innovation in Jordanian organizations, International Journal of Training and Development, 9, 2, Bates, R. A., Holton, E. F. III, Seyler, D. L. and Carvalho, M. A. (2000), The role of interpersonal factors in the application of computer-based training in an industrial setting, Human Resource Development International, 3, 1, Bennett, J. B., Lehman, W. K. and Forst, J. K. (1999), Change, transfer climate and customer orientation, Group and Organizational Management, 24, 2, Bentler, P. M. and Kano, Y. (1990), On the equivalence of factors and components, Multivariate Behavioral Research, 25, 1, Chen, H. -C. (2003), Cross-Cultural Construct Validation of the Learning Transfer System Inventory in Taiwan. Unpublished Doctoral Dissertation, Louisiana State University, Baton Rouge. Chen, H. -C., Holton, E. F. III and Bates, R. A. (2005), Development and validation of the Learning Transfer System Inventory in Taiwan, Human Resource Development Quarterly, 16, 1, Colquitt, J. A., LePine, J. A. and Noe, R. A. (2000), Toward an integrative theory of training motivation: a meta-analytic path analysis of 20 years of research, Journal of Applied Psychology, 85, 5, Cromwell, S. E. and Kolb, J. A. (2004), An examination of work-environment support factors affecting transfer of supervisory skills training to the workplace, Human Resource Development Quarterly, 15, 4, Devos, C. and Dumay, X. (2006), Les facteurs qui influencent le transfert: une revue de la littérature [Factors likely to influence transfer: a literature review], Savoirs, 12, Dumay, X. (2004), Le transfert des acquis de la formation: Un modèle des facteurs d influence [Transfer of training: a model of influential factors], Gestion 2000, 21, 2, Everitt, B. S. (1999), Making Sense of Statistics in Psychology: A Second-Level Course (Oxford: Oxford University Press). Fabrigar, L. R., Wegener, D. T., McCallum, R. C. and Strahan, E. J. (1999), Evaluating the use of exploratory factor analysis in psychological research, Psychological Methods, 4, 3, Facteau, J. D., Dobbins, G. H., Russell, J. E. A., Ladd, R. T. and Kudisch, J. D. (1995), The influence of general perception of the training environment on pretraining motivation and perceived training transfer, Journal of Management, 21, 1, Ford, J. K. and Weissbein, D. A. (1997), Transfer of training: an update review and analysis, Performance Improvement Quarterly, 10, 2, Ford, J. K., Quinones, M. A., Sego, D. J. and Sorra, J. (1992), Factors affecting the opportunity to perform trained tasks on the job, Personnel Psychology, 45, 3, Foster, J. J. (2001), Data Analysis Using SPSS for Windows Versions 8 to 10: A Beginner s Guide (London: Sage). 198 International Journal of Training and Development