The Labor Market Impacts of. Adult Education and Training in Canada

Transcription

1 The Labor Market Impacts of Adult Educaton and Tranng n Canada Shek-wa Hu Department of Economcs Unversty of Western Ontaro shu@uwo.ca Jeffrey Smth Department of Economcs Unversty of Maryland smth@econ.umd.edu Revsed Fnal Draft We thank Human Resources Development Canada for fnancal support and Statstcs Canada for support n data access. We thank Luce Glbert for her support, encouragement and patence throughout the completon of ths report. We also thank the partcpants n the HRDC Adult Educaton and Tranng Survey Workshop n Ottawa for helpful comments. Any errors are, sadly, our own.

2 Abstract In ths report, we use the data from the Adult Educaton and Tranng Survey (AETS) 1998 to estmate the mpact of partcpatng n adult educaton and tranng on the employment and earnngs of Canadans. We apply methods that assume selecton on observables, ncludng both standard regresson-based methods and propensty score matchng methods. We also apply methods based on nstruments or excluson restrctons, ncludng standard nstrumental varables estmaton and the well-known Heckman bvarate normal selecton estmator. These methods am to deal wth selecton on unobservables. We fnd that none of the methods we examne produce plausble estmates of the mpact of adult educaton and tranng, although the methods that assume selecton on observables produce more reasonable estmates than those that assume an nstrument or excluson restrcton. Based on the results of our analyss, we suggest mprovements n the AETS data that would make t a better tool for estmatng the labor market mpacts of adult educaton and tranng. 1

3 1. Introducton Evaluatng the mpacts of adult tranng and educaton s of great value for a number of reasons. To the polcymaker, nformaton on the labor market effects of publcly fnanced adult educaton and tranng has obvous mplcatons once placed nsde a coherent cost-beneft framework. Smlarly, nformaton on the labor market effects of self-fnanced and employer provded adult educaton and tranng provdes nformaton on the extent to whch exstng polces to subsdze or tax such tranng (or the earnngs ncrements t leads to, f any) lead ndvdual decson makers away from the socally optmal level of partcpaton n these actvtes. For scholars, nformaton on the mpacts of adult educaton and tranng provdes nsght nto how ndvduals and frms accumulate human captal, as well as sheddng lght on questons of poltcal economy, credt constrants on ndvduals that may prevent them from makng ndvdually and socally optmal nvestments n human captal, and theores of under-provson of tranng n the labor market. The lterature dstngushes publcly fnanced or provded tranng, especally that for the unemployed or for workers reenterng the labor force, from that provded by frms to ther employees. There are several reasons for dong so, ncludng the fact that the populatons recevng the two types of adult educaton and tranng have qute dfferent characterstcs, as well as the fact that the content and duraton of the tranng tends to dffer substantally. We follow that dstncton n our emprcal work below and n our bref lterature revew here. A large lterature exsts on the labor market effects of government employment and tranng programs. Table 25 of Heckman, LaLonde and Smth (1999) lsts lterally dozens of 2

4 such studes from numerous countres around the world. In the Unted States, numerous studes have made use of random assgnment methods to produce hgh qualty, credble estmates of the mpacts of programs focused on job search assstance, classroom tranng and wage subsdes. Table 22 of Heckman, LaLonde and Smth (1999) provdes a partal lst of such studes. The publshed evaluaton record for such programs n Canada s much thnner. The federal and provncal governments commsson most Canadan studes for nternal use. Few ever see the lght of day outsde the government and even fewer are ever subject to peer revew. Two notable exceptons are Park, et al. (1996) and the wdely cted Self-Suffcency Project, summarzed n Mchalopoulos, et al. (2000). Rddell (1991), Smth and Sweetman (2001) and Warburton and Warburton (2002) analyze and crtque the evaluaton of publc adult educaton and tranng programs n Canada. Fortunately, the types of programs and populatons served n the Unted States are smlar enough that ther evaluatons provde as useful benchmark to whch to compare our fndngs n ths study. The exstng lterature on the effects of employer-provded tranng s much thnner. There are several reasons for ths. Frst, governments are, qute reasonably, wllng to spend a lot more money evaluatng ther own programs than evaluatng those of prvate frms. Second, good data on the recept of employer-provded tranng s hard to come by. Even when a large survey contans questons relatng to employer provded tranng, there are strong ssues of measurement error documented n, e.g., Barron, Berger and Black (1997). Thrd, whle ndvduals typcally partcpate n government fnanced tranng only once, or only at rare ntervals when they are unemployed, employer-provded tranng often contnues throughout the lfecycle. Tranng 3

5 epsodes are often short, on the order of days or weeks, and there are often multple spells wthn a year. As documented some below and n Hu and Smth (2002a), these patterns characterze employer provded tranng n Canada. These features of employer-fnanced tranng mply the need for longtudnal rather than cross-sectonal data and also make t dffcult to know how to code partcpaton whether n terms of ncdence, hours, epsodes and so on. The Brtsh studes by Arulampalam, Booth and Elas (1997) and by Blundell, Dearden and Meghr (1996) that have attempted to use panel data to study the labor market effects of such tranng have both wrestled wth these ssues n depth. Heckman, Lochner, Smth and Taber (1997) and Carnero, Heckman and Manol (2002) summarze the evdence from the lterature that attempts to evaluate employer-fnanced adult educaton and tranng. One common fndng s qute hgh estmated effects, whch are generally attrbuted to a falure of the avalable data to completely control for the assumed selecton of more able and more motvated employees nto tranng wthn frms. We dd not fnd any studes along these lnes usng data from Canada. In ths paper, we estmate the mpacts of partcpaton n adult educaton and tranng usng the data from the 1998 Adult Educaton and Tranng Survey (AETS). The AETS s a supplement to the Canadan Labor Force Survey (LFS), and as a result ncludes all of the nformaton on labor market actvty and demographc characterstcs ncluded n the LFS. Our analyss has two prmary goals. The frst s to begn to fll the vod n the lterature n regard to the labor market effects of adult educaton and tranng n Canada. The second s to determne 4

6 the value of the AETS as a data source for use n evaluaton. The second goal s not trval because the prmary focus of the AETS survey nstrument s on documentng the types and extent of partcpaton n adult educaton and tranng, as well as provdng a vehcle for studes of the determnants of partcpaton n such adult educaton and tranng, such as Hu and Smth (2002a). The actve lterature on econometrc methods for evaluatng the mpacts of treatments such as adult educaton and tranng ncludes a varety of alternatve estmaton strateges. See Heckman, LaLonde and Smth (1999) and Angrst and Krueger (1999) for recent summares. The basc problem addressed by all of these estmators s the general absence of data on randomly assgnment treatments. In the absence of random assgnment, we have observatonal data, whch has the fault that the observed varaton n treatment, n our context the observed varaton n partcpaton n adult educaton and tranng, comes from the (assumed) optmzng choces of ndvduals. Indvduals have nformaton that the analyst does not, and have characterstcs that the analyst does not observe. As a result, smple comparsons of the labor market outcomes of partcpants and non-partcpants combne the effects of partcpaton wth dfferences due to non-random partcpaton. These dfferences lead to selecton bas. The lterature offers two wde classes of estmators to deal wth ths problem: those that assume suffcent nformaton n the data to mostly correct for systematc dfferences between partcpants and non-partcpants, and those that assume the absence of such nformaton but nstead assume the presence of a varable (an nstrument or excluson restrcton) that affects partcpaton but not outcomes n the absence of partcpaton. We utlze two evaluaton 5

7 strateges drawn from each of these broad classes. In the frst class, we use standard regressonbased methods (the so-called Barnow, Can and Goldberger (1980) estmator) as well as recently developed propensty score matchng methods. In the second class, we use nstrumental varables methods as well as the wdely known Heckman (1979) bvarate normal selecton estmator. In each case, our specfcatons buld on what we learned about the determnants of partcpaton n adult educaton and tranng n Hu and Smth (2002a). As dscussed, for example, n Heckman, LaLonde and Smth (1999) and Smth (2000), each of these dfferent estmators makes dfferent assumptons about the processes that generate partcpaton n adult educaton and tranng as well as employment and earnngs, the two labor market outcomes we examne. At most, the assumptons of one of the estmators we consder matches the data and nsttutonal context we examne here. Our purpose n examnng all of them s to allow the data to nform us, n part, regardng whch estmaton strateges seem most plausble n the AETS context, and also to allow the fndngs to suggest ways n whch the AETS data mght be mproved for the purposes of mpact estmaton. The mpact estmates we obtan from all of the econometrc methods we apply prove dsappontng. The mpact estmates for tranng fnanced by employers are much too large, whle those for tranng fnanced by the government are often negatve. Theoretcal arguments based on expected rates of return, as well as comparsons wth alternatve estmates n the lterature that use better data (n the case of government fnanced tranng, often expermental data), cast strong doubt on the estmates obtaned here. These poor results hold for all of the estmators we examne, but the results from the nstrumental varable and Heckman bvarate 6

8 normal estmators prove the least credble. Senstvty analyses ndcate that these poor results are robust to modest changes n the specfcaton we estmate, leadng us to conclude that the prmary problem wth the estmates les n the data rather than n the methods. Put smply, the AETS data lack crtcal elements necessary to produce credble mpact estmates. The remander of our study proceeds as follows. Secton 2 descrbes the 1998 AETS data that we use, and defnes our measures of tranng partcpaton and labor market outcomes. Secton 3 descrbes the non-expermental estmaton methods we employ to estmate the effects of adult educaton and tranng on partcpants earnngs and employment. Secton 4 presents our mpact estmates, and ndcates why they are problematc n lght of the exstng theoretcal and emprcal lterature. Secton 5 brefly summarzes our suggestons for ways to make the AETS a better tool for estmatng mpacts; these suggestons are elaborated on n our companon paper Hu and Smth (2002b). We conclude n Secton 6 wth a summary and some conclusons regardng adult educaton and tranng n Canada. 2. Data 2.1 The AETS data The data we use come from the 1998 Adult Educaton and Tranng Survey (AETS) master fle. The AETS 1998 s the sxth n a seres of smlar surveys desgned to measure partcpaton n adult educaton and tranng, defned as educaton and tranng that occurs after the concluson of formal schoolng. The objectves of the survey are to measure partcpaton rates, determne the role of employers n adult educaton and tranng partcpaton and provson, and to dentfy 7

9 barrers to adult educaton and tranng. Statstcs Canada collected the AETS data on behalf of Human Resources Development Canada. The AETS s a supplement to the Labor Force Survey (LFS). The LFS has an overlappng panel desgn. Each month a new random sample of the LFS populaton cvlans ages 15 and over s drawn. Each such sample s called a rotaton group. Each rotaton group s of roughly equal sze, and each one remans n the LFS for sx consecutve months, at whch pont they are no longer followed but nstead replaced by a new rotaton group. The AETS was admnstered to fve of the sx rotaton groups n the January 1998 and March 1998 Labor Force Surveys. 1 The 1998 AETS (hereafter just AETS) conssts of fve modules, desgnated A to E. The questons n Module A collect background nformaton on the respondent. The module also asks whether the respondent receved any tranng or educaton wthn the prevous year. Respondents who ndcate that they dd not receve any educaton or tranng skp the followng three modules, B, C and D, and proceed drectly to module E. The questons n Module B ask about the detals of any tranng or educaton leadng to formal certfcaton of some sort. The AETS calls such educaton and tranng tranng programs. The questons n Module C ask about the detals of any educaton or tranng not leadng to formal certfcaton, but stll ntended for career development. The AETS calls such educaton and tranng tranng courses. The questons n Module D concern courses taken for hobby, recreatonal, or personal development reasons. We omt the courses reported n Module 8

10 D from our analyss due to our focus on tranng related to labor market outcomes. In each of Modules B, C, and D, the survey collects nformaton on up to fve dfferent courses or programs. The nformaton collected on each course or program ncludes the feld of study, the locaton, the provder, the teachng medum, the duraton, whether or not the tranng was completed, who pad for the tranng, and what employer support was provded (f any). The survey also collects nformaton on respondents reasons for takng the tranng, expectatons regardng the tranng, and opnons of the tranng s usefulness. All of the questons n Modules B, C and D refer to educaton and tranng actvtes undertaken n All respondents complete Module E. Ths module collects nformaton on labor market outcomes n 1997 for whch data are not collected on the LFS. Ths ncludes nformaton on the man job n 1997 f t dffers from that at the tme of LFS completon n Module E also collects a varety of demographc nformaton ncludng characterstcs of the respondents parents and the respondents mmgraton and dsablty statuses. To supplement the nformaton collected n Module E, the labor force nformaton collected on the LFS s attached to the record of each AETS respondent. In addton, for respondents who report (n Module A) that they dd not partcpate n any educaton and tranng n the prevous year, Module E ncludes a seres of questons that seek to determne why they dd not do so. The household response rate to LFS s 94.8%, whle 85.2% of LFS respondents responded to the AETS. Ths mples a respectable overall AETS response rate of 80.8%. The 1998 AETS has a total of 33,410 respondents. In order to restrct our attenton to those who have 1 The data from March 1998 conssts solely of respondents resdng n Quebec. Ths addtonal 9

11 completed ther formal schoolng, we further restrct the sample to persons 25 to 64 years of age who are not full-tme students at the tme they complete the LFS. Table 1 shows how these restrctons result n basc analyss samples of 10,748 males and 12,418 females. The samples actually used n some analyses are smaller due to tem non-response on partcular covarates. 2.2 Defnng our measures of tranng recept As noted n the ntroducton, the lterature on adult educaton and tranng (hereafter we often just call t tranng ) dstngushes publc and prvate tranng for a number of reasons. As, e.g., Hu and Smth (2002a) show for Canada, the populatons that receve publc and prvate tranng dffer substantally. Publc and prvate tranng also tend to dffer n ther content and n the nature of ther provders. Thus, we dstngush between publc and prvate tranng n ths study. The AETS dstngushes between employer supported tranng and non-employer supported tranng. The AETS nterprets employer support very broadly, to nclude such thngs as unpad tme off for tranng. In contrast, we feel that someone recevng unpad tme off from hs or her employer to partcpate n a government-sponsored tranng program should be desgnated as recevng government tranng, rather than prvate tranng. Thus, we adopt an alternatve defnton that (necessarly) reles on the nformaton avalable n the AETS but nstead focuses on who pad for the tranng. In partcular, we defne three mutually exclusve categores: employer or unon fnanced tranng, self-fnanced tranng, and government or other fnanced tranng. The frst category conssts of any tranng pad for, n sample was drawn due to the severe ce storm n Quebec n January

12 whole or n part, by an employer or a unon. Ths category domnates the others n the sense that tranng pad for by both an employer and the government, or by the employer and the ndvdual, s counted only n ths category. The second category, self-fnanced tranng, ncludes any tranng pad for solely by the respondent, along wth tranng provded free of charge. Ths category may nclude some tranng where the government subsdzes (n whole or n part) the tranng provder, dependng on whether or not the respondent recognzes ths subsdy n ther survey response. It may also nclude tranng that receves ndrect subsdes n the form of tax credts, transportaton assstance, chldcare allowances or exempton from job search requrements. The fnal category s a resdual category that ncludes tranng the respondent reports as exclusvely funded by the government or other sources, such as relatves. The vast majorty of the tranng n the thrd category s reported as funded exclusvely by the government. As already descrbed, the AETS dstngushes between programs and courses based on whether or not the tranng leads, or s ntended to lead, to formal certfcaton. Hu and Smth (2002a) show that partcpaton n programs and courses are not mutually exclusve n the AETS data, although the vast majorty of partcpants partcpate n only one or the other. In addton, they provde some evdence of dfferences n the determnants of partcpaton for programs and courses. However, gven our relatvely small sample szes, and gven that the dstncton between programs and courses reles on subjectve judgements by the respondents, we combne the two types of tranng n defnng our measures of treatment. Hu and Smth (2002a) show that whle some AETS respondents report recevng multple tranng spells, whether programs or courses, n 1997, the vast majorty of partcpants report only 11

13 a sngle spell. Thus, n our analyses the treatment varables consst of ndcator (dummy) varables for recept of employer/unon tranng (courses, programs or both), self-fnanced tranng (courses, programs or both) or government/other tranng (courses, programs or both). Table 2 provdes descrptve statstcs for the three treatment measures, as well as the underlyng dstrbutons n terms of courses and programs for each of the three fundng sources. The values n Table 2, as well as all the other descrptve nformaton n Tables 3, 4 and 5, are weghted usng the weghts provded by Statstcs Canada. To get a sense of how much tranng the treatment represents, Table 3 presents descrptve statstcs on the dstrbuton of hours wthn reported tranng spells. The top panel ndcates the mean duraton, as well as the 25 th, 50 th (medan) and 75 th percentles of the dstrbuton for the combned sample of tranng programs and courses. The nformaton s presented separately for men and women and, wthn these groups, both overall and separately by the fnancng source for the tranng. Four man fndngs emerge from Table 3. Frst, employer fnanced tranng generally has much shorter duratons than government and self-fnanced tranng. Ths holds for both men and women and for both courses and programs. Second, as expected gven ther defntons, tranng programs tend to have much longer duraton than tranng courses, although the two dstrbutons do have non-trval overlap. Thrd, the mean duratons show a remarkable smlarty between men and women. Fourth, the data reveal a huge amount of heterogenety n the ntensty of the treatment whose mpacts we seek to measure. To take just one example, government fnanced tranng for men has a mean duraton of hours, but the 10 th percentle duraton s but 12 hours, whle the 90 th percentle s over 1300 hours. 12

14 A number of the tranng spells n the data reman n progress at the tme of the AETS ntervew n Persons wth a spell of tranng n progress at that tme, whch s when our outcomes are measured, are ncluded n the descrptve statstcs but omtted from the mpact analyses. 2.3 Defnng our outcome varables Studes of the mpact of educaton and tranng typcally focus on ther effect on employment and on earnngs. We care about employment because the employed generally support themselves, rather than relyng on the taxpayer through unemployment nsurance or socal assstance. Thus, gettng people employed represents a goal of many government tranng and assstance programs. At the same tme, for conventonal cost beneft analyss, earnngs provde a more natural outcome measure. In addton, earnngs reflect dfferences n hours of work and rates of pay between jobs. All else equal, the government (and the tranee!) would prefer that government fnanced or subsdzed tranng result n full tme jobs wth hgher rates of pay rather than parttme jobs wth lower rates of pay. We follow the lterature by defnng two outcome measures, one related to employment and one related to earnngs. The frst outcome measure s employment at the tme of the respondent s LFS ntervew n The second s the respondent s usual weekly earnngs at all jobs as of ther LFS ntervew n Table 4 reports the means (and, for earnngs, the standard devatons) of these varables for the full samples of men and women, and condtonal on 13

15 partcpaton n each type of tranng (employer/unon fnanced, self-fnanced, and government/other fnanced) or recevng no tranng n As dscussed n detal n Hu and Smth (2002b), these outcome varables have the very mportant drawback that they are measured no more than 12 (or 15 n the case of the March 1998 respondents) months after the completon of the tranng whose effect we seek to measure. In some cases, the lag may be only a month or two, or the tranng may not yet even be complete. As a result, the outcomes may not fully pck up the earnngs and employment effects of tranng, partcularly f t takes some tme to fnd a job followng completon of the tranng. Recent evdence from the Calforna GAIN program presented n Hotz, Imbens and Klerman (2000) suggests that government fnanced human captal acquston may have a payoff that does not fully appear for a couple of years after the completon of tranng. 3. Evaluatng the Labor Market Effects of Educaton and Tranng In ths secton, we lay out a model of labour market outcomes and partcpaton n tranng. We then descrbe the assumptons requred under dfferent econometrc methods of estmatng the mpact of tranng on outcomes. In consderng alternatve evaluaton methods, we are lmted by the fact that the AETS s essentally a cross-sectonal survey. The AETS collects nformaton on each respondent only once. Whle the data contan nformaton on total annual earnngs for the year 1997, whch s the perod durng whch the tranng t measures takes place, ths earnngs measure s not comparable to the weekly labor earnngs n 1998 measure obtaned n the Labor Force Survey. Moreover, 14

16 for most longtudnal estmaton strateges, we want data on the outcome pror to, rather than at the same tme as, the tranng whose mpact we seek to estmate. The lack of precse nformaton on the tmng of tranng durng 1997 further lmts any attempt at usng longtudnal methods. Thus, the data compel us to restrct our analyss to cross-sectonal evaluaton methods. We consder two pars of related methods. The frst par of methods reles on the assumpton that the data contan nformaton on all of the mportant factors affectng both labor market outcomes and partcpaton n tranng. The lterature refers to ths assumpton as selecton on observables. We consder both parametrc regresson and non-parametrc matchng estmators that buld on ths assumpton. The second par of methods allows for selecton on unobservables. Both methods requre the presence n the data of an nstrument (or excluson restrcton). An nstrument s a varable that affects partcpaton but not outcomes, other than through ts effect on partcpaton. Credble examples of such varables are dffcult to come by n ths context; we examne the performance of multple canddate nstruments n the emprcal work presented n Secton 4. As methods are not our prmary purpose, our dscusson s short and focuses on the man ponts. Further detal on all of the methods we consder appears n Angrst and Krueger (1999) and Heckman, LaLonde and Smth (1999). 3.1 A model of labor market outcomes and partcpaton n tranng The standard human captal earnngs functon (see, e.g., Becker, 1964, or Mncer, 1974) forms the bass of the outcome models we use to estmate the mpact of tranng on earnngs and employment usng the AETS. Assumng a lnear functonal form, we have the outcome equaton, 15

17 Y = β + β X β X + δ T δ T + ε, t 0 1 1t K Kt 1 1t J Jt t where Y t denotes the outcome of nterest for person n perod t (earnngs or employment n our case), X, k = 1,..., K denote factors such as years of schoolng and experence, and kt T, j = 1,.., J are ndcators for recept of dfferent types of tranng. Henceforth, gven that we jt have only cross-sectonal data, we drop the t subscrpt. For smplcty later on, we defne Y (, 1, ) 1 = Y1 X T = ε to be the observed outcome wth tranng and Y0 = Y( X, T = 0, ε) to be the observed outcome wthout tranng. Now consder the partcpaton equaton. 2 For smplcty, assume for the moment only a sngle type of tranng, so that the partcpaton choce conssts of takng tranng or not, and that the tranng s avalable only n a sngle perod. Let Y * ( X, T = 1) denote the expected, dscounted present value of earnngs assocated wth tranng. Smlarly, let Y * ( X, T = 0) denote the expected, dscounted present value of earnngs assocated wth not takng tranng. Now let CW ( ) denote the expected, dscounted present value of the costs assocated wth takng tranng, where W, whch may nclude elements of X, denotes factors that vary the cost of tranng among persons. Such factors may nclude age, exstng human captal, famly characterstcs, ndustry, occupaton, job tenure, frm sze, regon, and so on. Assumng lnearty, ths gves the tranng cost functon: C = + W + + W + u, γ 0 γ γ L L 2 See Hu and Smth (2002a) for more theoretcal dscusson of the tranng partcpaton decson, 16

18 where W,..., 1 W L are the ndvdual elements of W. Rsk-neutral ndvdual wll take the tranng f and only f, Y ( X,T = 1)-C(W ) > Y (X,T = 0). * * A (very) small amount of algebra allows us to defne the expected, dscounted gan (or loss) from tranng as H ( X, W) = Y ( X, T = 1) Y ( X, T = 0) C( W). * * * We do not observe H * ( X, W ), because we do not observe the counterfactual expected earnngs wthout tranng for persons who take tranng or the counterfactual expected earnngs wth tranng for persons who do not take tranng. What we do observe n the data s the decson of whether or not to take tranng. We can wrte ths decson n the form of a bnary choce model, wth T = 1 for persons who take tranng and T = 0 for those who do not: T > = 0 otherwse. * 1 f H ( X, W) 0; Assumng that Y * ( X, T ) s a lnear functon of X and T (as above), that CW ( ) s a lnear functon of W, and that the unobservable components of both have normal dstrbutons centered at zero, yelds a reduced form probt model of partcpaton. The generalzaton to ndvduals who are not rsk-neutral s straghtforward. Smply change the dscounted earnngs streams above to dscounted utlty streams. Equally straghtforward s the generalzaton to multple types of tranng, so long as we contnue to as well as detaled emprcal evdence from the AETS. 17

19 assume that the tranng takes place n only one perod an assumpton consstent wth our crosssectonal data (but not, of course, wth realty). In ths case, there are multple possble earnngs, or utlty, streams, wth one for no tranng and one assocated wth each avalable type of tranng. Each ndvdual chooses the tranng acton assocated wth the maxmum of these dscounted values. Now consder some mplcatons of ths smple model of partcpaton and outcomes for the mpact estmaton undertaken n ths model. Ths s a model of ratonal tranng partcpaton. Indvduals partcpate n tranng when they expect, ex ante, that the benefts wll exceed the costs. Ths feature of the model has several mplcatons. Frst, t suggests a strong pror belef that the mpacts of tranng, partcularly of prvate tranng (publcly fnanced tranng s sometmes taken for other reasons), wll have a postve mpact on labor market outcomes. Negatve mpact estmates wll rase suspcon and can be consdered an nformal specfcaton test of sorts. The second mplcaton of ratonal behavor n the context of ths model relates to nstrument selecton. Indvduals decdng whether or not to take tranng are weghng the costs and benefts of dong so. Good nstruments wll be varables that affect the costs and benefts of takng tranng wthout affectng outcomes n the absence of tranng. Examples of possble nstruments suggested by ths lne of reasonng nclude varables specfcally related to costs, such as dstance to the local tranng center, and varables related to varaton n the mpacts of tranng. The thrd mplcaton of ratonal behavor relates to heterogeneous mpacts. If some ndvduals gan more (or gan at all) from tranng and others gan less, we would expect that f 18

20 ndvduals can predct ther gans to some extent, those we observe takng tranng wll have larger mpacts from t than those we do not. Ths has mportant mplcatons for polces that seek to ncrease partcpaton n tranng, as t suggests care n generalzng estmated mpacts of tranng to populatons not presently observed to take t. There may be a reason they are not dong so. Next, consder the ssue of selecton bas n the context of the smple model. Some varables affect both partcpaton n tranng and outcomes n the absence of tranng. If we fal to take condton approprately on these varables when estmatng the mpact of tranng, our estmates of the mpact of tranng wll be based as the tranng ndcator wll proxy for the mssng varables that affect both tranng and outcomes. Two standard examples of such varables are ablty and motvaton. Both ablty and motvaton lkely have a postve effect on both earnngs and partcpaton n prvate tranng, whch mples a postve bas n the estmated mpact of tranng f they matter and we fal to condton n them. The same s lkely true for publcly fnanced tranng, although one could make arguments n the other drecton. If we observe the relevant varables that affect both partcpaton n tranng and outcomes n the absence of tranng n our data, then we have what Heckman and Robb (1985) refer to as selecton on observables. In ths case, ncludng these varables approprately usng the methods dscussed n Secton 3.3 wll suffce to solve the selecton problem. If we do not observe the relevant varables, then n terms of the model these unobserved factors result n a correlaton between the error terms n the outcome and partcpaton equatons, so that 19

21 corr( ε, u ) 0. In the case, we requre methods for selecton on unobservables, whch we dscuss n Secton 3.4. These methods typcally requre an nstrument or an excluson restrcton, whch, n terms of our model, s a varable that belongs n W but not n X. Fnally, consder the relatonshp between ths smple, statc model and the underlyng dynamc process of tranng partcpaton over the lfecycle. As shown n Becker (1964), t makes sense to take tranng when young rather than old, as young people have a longer perod over whch to realze the labor market benefts of ther tranng. Ths dynamc aspect of the tranng partcpaton decson can be captured n the statc model by ncludng age as a determnant of tranng. Another lfecycle ssue concerns repeated partcpaton n tranng. Emprcally, we observed ndvduals takng both publcly fnanced tranng (see, e.g., Trott and Baj, 1993, for the U.S.) and prvate tranng (see, e.g., Blundell, Dearden and Meghr, 1996 for the U.K.), more than once. Suppose these repeated nstances of tranng are not ndependent, but nstead are postvely correlated, perhaps due to unobserved dfferences n tastes for tranng. In ths case, the mpact of current tranng we estmate may also nclude the mpact of past tranng epsodes we do not observe. To the extent that tranng has the postve effect that theory suggests t should, ths would bas our mpact estmates up, f we nterpret them strctly as mpacts of the tranng we observe n the AETS. 3.2 Parameters of nterest 20

22 In a world n whch ndvduals have heterogeneous mpacts from tranng, t s mportant to consder precsely what the parameter of nterest s n evaluatng the mpact of tranng. 3 To keep the dscusson smple, we agan assume for the moment only a sngle tranng type, wth mpact ( = Y Y ) for person. The extenson to multple tranng types s straghtforward. We δ 1 0 consder three possble parameters of nterest and brefly dscuss the relatonshps among them. The average treatment effect s, smultaneously, the effect of tranng on a randomly selected person n the populaton of nterest or the mean effect on all persons n the populaton of nterest. It s defned as ATE = E( δ ). Ths parameter s of nterest n cases where a populaton wll be requred (or nduced) to partcpate n tranng. The most common treatment effect parameter n the lterature s the so-called treatment on the treated parameter. Ths parameter measures the mean mpact of tranng on those observed to receve t n the data. In term of our notaton, t s gven by TT = E( δ T = 1) Ths parameter s of nterest f we want to perform a cost-beneft analyss on tranng currently beng receved, whether prvately or publcly funded. 3 See Heckman, Smth and Clements (1997) and Heckman and Smth (1998) for extended dscussons of heterogeneous treatment effects. For an mportant early dscusson, see Björklund and Mofftt (1987). 21

23 The fnal type of parameter of nterest conssts of varous treatment effects measured at the margn. If we have a bnary nstrument, the we can defne local average treatment effects (LATEs), as n Imbens and Angrst (1994). The LATE s the mean effect on those persons who change partcpaton status when the nstrument changes value. It assumes a monotonc response, so that persons do not, for example, become more lkely to partcpate when move farther away from a tranng center. Each dfferent nstrument mples ts own LATE, and the LATEs for two dfferent nstruments may dffer substantally dependng on the mpacts realzed by the persons each nstrument nduces to partcpate. If the nstrument s a polcy varable, such as the tuton for the tranng, then the LATE may be of great polcy polcy nterest. If we have a contnuous nstrument, we can defne margnal treatment effects (MTEs) as n Heckman and Vytlacl (2001). The margnal treatment effect they defne s the effect on the person just ndfferent to partcpatng at ther current value of the nstrument. That s, the margnal person s one for whom H * ( X, W ) = 0. Heckman and Vytlacl (2001) show that all of the other common treatment effect parameters can be wrtten as partcular ntegrals of such MTEs. Fnally, we can defne other margnal effects not necessarly related to nstruments. If we buld a new tranng center n a depressed town, then we can defne the mpact on the persons who choose to partcpate n the presence of the new tranng center who dd not partcpate before when they had to travel to the next town. Ths treatment effect we refer to as a margnal average treatment effect (MATE). It s not a LATE, because the varaton (the new tranng center) s not an nstrument, due to the placement of t n a depressed town where, presumably, 22

24 outcomes n the absence of tranng are lower. From the dscusson, the polcy nterest n partcular MATEs s clear. In ths paper, we look only at the mpact of treatment on the treated for dfferent types of tranng. We do so for several reasons. Frst, although t s the parameter most often examned n the lterature, estmates for both publcly fnanced and prvately fnanced tranng reman somewhat controversal, especally the latter. Second, none of the nstruments we examne arse from varaton n polcy, whch s typcally necessary for LATEs to be of nterest. Fnally, as no one s proposng makng ether publc or prvate tranng mandatory, potental nterest n the ATE parameter n ths context s small. 3.3 Cross-Sectonal Evaluaton Methods Based that Assume Selecton on Observables In ths secton, we consder two methods based on selecton on observables. That s, both methods assume that we observe n the data all the man factors that affect both partcpaton n tranng and outcomes n the absence of tranng. The most common (and, whch s not unrelated, the smplest) method for evaluatng the mpact of tranng reles on standard regresson methods. For smplcty, we frst consder the case of one tranng type and a common effect of tranng. The extenson to multple tranng types s straghtforward; we dscuss the extenson to heterogeneous tranng mpacts below. Now suppose that X ncludes the standard varables one ncludes n the human captal model, such as prevous schoolng and experence (or ts proxy, age). But suppose that there 23

25 reman other factors, not ncluded n the standard human captal model but avalable n the data, whch affect both outcomes n the absence of tranng and partcpaton n tranng. Geographc locaton s a potental example here. These latter varables represent a subset of denote the unon of ths subset of but W wth W. We let X. Under these assumptons, we have that E( ε X, T) 0, E( ε Z, T) = 0. (1) Barnow, Can and Goldberger (1980) (hereafter BCG ) frst derved ths motvaton for estmatng the mpact of tranng usng standard regresson methods but wth a rch set of covarates rch enough to make the outcome equaton error term condtonally mean ndependent of tranng. As dscussed n Heckman and Robb (1985) and Heckman and Smth (1996), n a world wth heterogeneous mpacts, the error term n the outcome equaton now mplctly ncludes the person-specfc component of the mpact for persons who receve tranng. That s, the error term ncludes the dfference between the mean mpact of treatment on the treated and the ndvdual tranee s mpact from tranng as well as the unobserved component of the outcome n the absence of tranng. In the BCG set-up, the only major change s n nterpretaton. The coeffcent on the tranng ndcator now just estmates the mean mpact of treatment on the treated; under the common effect world t was also an estmate of the average treatment effect. Z 24

26 Lke the standard regresson estmator, matchng assumes selecton on observables. However, rather than assumng a functonal form for the outcome equaton, matchng drectly compares the outcomes of traned and untraned persons wth the same (or smlar) values of those varables thought to nfluence both partcpaton n tranng and outcomes n the absence of tranng. Matchng has two advantages relatve to the regresson estmator just dscussed, and one dsadvantage. The prmary advantage s that t s sem-parametrc. No functonal form assumpton from the outcome equaton s requred to mplement the estmator. In standard regresson analyss, even f you have the correct covarates, you can stll get based estmates f you assume the ncorrect functonal form say by falng to nclude needed hgher order or nteracton terms. The second advantage s that you can match on varables that are correlated wth the error term n the outcome equaton. Ths s the case because matchng only requres that the mean of the error term be the same for tranees and non-tranees wth gven values of the condtonng varables, not that t be zero. In notaton, t requres that E( ε Z, D = 1) = E( ε Z, D = 0), but t does not requre, as regresson does, that both terms equal zero. Ths s a weaker assumpton than assumpton (1) above. See Heckman, Ichmura and Todd (1997, 1998) for further dscusson. The dsadvantage of matchng s that, f the lnear functonal form restrcton mplct n regresson based analyss n fact holds n the data, then falng to mpose t reduces the effcency of the estmates. Put dfferent, f the outcome equaton really s lnear, mposng lnearty wll lead to smaller standard errors on the mpact estmates. 25

27 The condtonal ndependence assumpton (CIA) that justfes matchng s gven by: Y T Z. 0 Ths assumpton mples the balancng condton mentoned n the precedng paragraph. The CIA states that, condtonal on Z, the varables affectng both partcpaton n tranng and outcomes n the absence of tranng, partcpaton n tranng s unrelated to outcomes n the absence of tranng. Put dfferently, whatever selecton nto tranng takes place, wthn groups defned by the same values on all the varables n Z, partcpaton s unrelated to what would happen f the person dd not take tranng. Thus, overall, tranees may have better or worse labor market prospects than non-tranees, but condtonal on Z, ther expected labor market outcomes are equvalent. Two techncal detals deserve note. Frst, ths s the verson of the CIA that justfes usng matchng to estmatng the mean mpact of treatment on the treated; a stronger verson, whch also apples to Y 1, s requred for estmatng the average treatment effect. Second, the varables n Z may not be factors that may be altered by the tranee n antcpaton of takng tranng (or by a non-tranee n antcpaton of not takng tranng). See Lechner and Mquel (2002) for more on ths latter pont and Heckman, LaLonde and Smth (1999) for more on the former pont. The problem wth matchng drectly on Z s that any set of covarates that plausbly satsfes the CIA s gong to be of relatvely hgh dmenson. Even f all the elements of Z are dscrete, the number of dstnct combnatons becomes large very rapdly, leadng to the problem of empty cells values of Z for whch we observe partcpants but no correspondng non- 26

28 partcpants to provde the estmated counterfactual. Smply omttng tranees n cells wth no non-tranees s not a very satsfyng soluton. Equally unsatsfyng are the varous ad hoc cell combnaton algorthms used n some of the evaluatons of the Comprehensve Employment and Tranng Admnstraton (CETA) program n the U.S. These evaluatons are surveyed and referenced n Barnow (1987). Ths problem of the potental absence of non-tranees to provde the estmated counterfactual for tranees wth certan values of the condtonng varables s called the support problem. The support s a statstcal term meanng the set of values for whch a densty functon s non-zero; that s, t s the set of values of a varable that you mght observe wth postve probablty. Along wth the CIA, the second man assumpton underlyng matchng s the support condton, gven by Pr( T = 1 Z ) < 1 for all possble values of Z. Ths condton states that for all values of the condtonng varables, some persons wll not partcpate. Even f ths condton holds n the populaton, t may sometmes fal n fnte samples. A more general verson of the support condton s requred to estmate the average treatment effect; see the dscusson n Heckman, Ichmura, Smth and Todd (1998). Regresson-based methods, such as the BCG estmator, mplctly solve the support problem through the lnear functonal form assumpton. The functonal form assumpton flls n where the data are absent. Ths fact reveals another advantage of matchng; t hghlghts the support condton and makes t clear whether the results obtaned were generated by the data, or whether the counterfactuals nstead depend heavly on the lnearty assumpton. 27

29 The lterature has converged on an alternatve soluton to the curse of dmensonalty and the related support problem. Rosenbaum and Rubn (1983) showed that f you can match on Z ; that s, f Z satsfy the CIA, then you can also match on PZ ( ) = Pr( D= 1 Z). Ths quantty s the probablty of partcpaton or propensty score. Ths helps solve the problem because PZ ( ) s a scalar just a real number between zero and one, rather than a vector. The lterature contans a number of dfferent methods of actually mplementng propensty score matches. These nclude nearest neghbor matchng (wth and wthout replacement), cell matchng, kernel matchng and local lnear matchng. These methods are all consstent n the sense that, as the sample sze becomes arbtrarly large, they all gve the same answer because n an arbtrarly large sample, all of them rely only on comparsons of tranees and non-tranees wth equvalent values of PZ ( ). Detaled dscussons of the varous methods can be found n Heckman, Ichmura and Todd (1997), Heckman, LaLonde and Smth (1999) and Smth and Todd (2003). In ths paper, where we are concerned wth substance rather than methods, we confne ourselves to nearest neghbor matchng wth replacement, but vary the number of nearest neghbors. Consder frst just one nearest neghbor. Nearest neghbor matchng wthout replacement goes through the treated (tranee) observatons one by one and, for each one, fnds the non-tranee wth the nearest (n absolute value) estmated propensty score. That non-tranee becomes the nearest neghbor match for the current tranee and may not be matched to any other tranees. Nearest neghbor matchng wth replacement proceeds n the same fashon, but allows a gven non-tranee to be used as the match for more than one tranee. Matchng wth replacement 28

30 reduces the average dstance (n propensty scores) between each tranee and hs or her matched non-tranee. Ths should reduce bas. The cost s that f some non-tranees are re-used, ths wll ncrease the varance of the resultng estmate. Deheja and Wahba (1999) clearly llustrate the problem wth matchng wthout replacement when the number of comparson non-tranee observatons wth hgh probabltes of tranng s less than the number of tranees wth hgh probabltes of tranng (as t usually s for obvous reasons). In ths case, you get a lot of bad matches. To avod ths, we match wth replacement. The formula for the (sngle) nearest neghbor estmator s gven by 1 n { T = 1} n{ T= 1} n{ T= 0} ( Y w Y ), 1 j 0 j = 1 j= 1 where the sum s over tranees, the j sum s over the non-tranees, n { T = 1} s the number of tranees, n { T = 0} s the number of non-tranees, and where w j 1 f j = arg mn{ P( W) P( Wj) }; = 0 otherwse. The generalzaton to the case of multple nearest neghbors, each recevng equal weght, s straghtforward. Varyng the number of nearest neghbors n the estmaton allows us to trade off between the bas and varance n our estmator. Consder swtchng from usng one nearest neghbor to construct the counterfactual for each observaton to usng two nearest neghbors. The average dstance (n terms of propensty scores) between each tranee and the non-tranees used to construct hs or her estmated counterfactual mean necessarly ncreases. At the same tme, the 29

31 number of observatons used to construct the counterfactual ncreases, whch reduces the varance of the estmator. The optmal number depends on the densty of non-partcpants. For example, f there are not many more non-partcpants than partcpants, there s lttle gan to usng addtonal neghbors. We experment wth one, two, and fve nearest neghbors n our emprcal work. The norm n the economc lterature that employs matchng s to use bootstrappng methods to estmate the standard errors, for reasons lad out, e.g., n Heckman, Ichmura and Todd (1997,1998). For reasons of tme and of computng convenence, we nstead report here estmates that do not take account the varance components resultng from the estmaton of the propensty score or from the matchng tself. Thus, our estmated standard errors are lkely downward based. 3.4 Cross-Sectonal Evaluaton Methods that Assume Selecton on Unobservables We consder two (related) methods that attempt to deal wth selecton on unobservables. In ths settng, we beleve that we do not have all of the varables that affect both partcpaton and outcomes n the absence of partcpaton n our data. But, we beleve that we have a varable, an nstrument or excluson restrcton, whch affects partcpaton but does not affect outcomes other than through ts effect on partcpaton. The frst of these estmators s the Heckman (1979) bvarate normal selecton estmator. Ths estmator assumes that the error terms n the partcpaton and outcome equatons have a jont normal dstrbuton, and that the selecton bas results from a non-zero correlaton between 30

32 the two error terms. When the outcome varable s bnary, ths model corresponds to a bvarate probt model. Techncally, the Heckman (1979) model s dentfed solely based on the jont normalty assumpton, and no excluson restrcton s requred. Extensve experence n the lterature n the form of both tral and error and Monte Carlo studes ndcates that, n practce, an excluson restrcton s requred to ensure the stablty of the model. The lterature also reveals that the performance of the bvarate normal estmator depends crtcally on the valdty of the normalty assumpton. Smulaton results n Heckman, LaLonde and Smth (1999) show that t also depends on havng a strong excluson restrcton that s, that the varable ncluded n the outcome equaton but not n the partcpaton has a substantvely mportant effect on partcpaton. Puhan (2000) summarzes much of the methodologcal lterature on the performance of the bvarate normal estmator. The bvarate normal model s often referred to as the Heckman two-step model, because of the smple two-step procedure to estmate t outlned n Heckman (1979). However, estmaton n two-steps s neffcent. As many common software packages (e.g., Stata) now nclude routnes to jontly estmate the partcpaton and outcome equatons, t makes sense to do so, and to drop ths name for the estmator. There are both common effect and heterogeneous effect versons of the Heckman model; Björklund and Mofftt (1987) frst lad out the latter verson. For smplcty, we nterpret our results n terms of the common effect model. The nstrumental varables (IV) estmator also deals wth selecton on unobservables. The smplest way to thnk about how the IV estmator works s n terms of mplementng t by dong two stage least squares. In the frst stage, the endogenous varable, n our context partcpaton n 31