Impacts of Aid-Funded Technical Assistance Programs: Firm-Level Evidence from the Indonesian Foundry Industry

Transcription

1 Impacts of Aid-Funded Technical Assistance Programs: Firm-Level Evidence from the Indonesian Foundry Industry Yasuyuki Todo July 2008 Abstract This study examines the effect of Japanese aid-funded technical assistance programs in the Indonesian foundry industry funded, applying difference-in-differences propensity score matching estimation to a unique firm-level dataset. The major finding is that the average effect of the aid programs on the change in the reject ratio is negative and significant, suggesting that these programs help local participant firms improve their technology. However, the effect of the programs is limited to their participants and does not spill over to non-participants. In addition, technical assistance programs provided by the local counterpart of aid after the completion of the aid programs do not seem to improve technology of participants on average. Keywords: propensity score matching, impact evaluation, technical assistance, foreign aid JEL classifications: F35, O12, O33 This research was conducted as part of the Economics of Foreign Aid project undertaken at the Research Institute of Economy, Trade and Industry (RIETI). The author would like to thank RIETI for providing us with the opportunity to conduct this research. The author is also grateful to Keitaro Aoyagi, Masahisa Fujita, Seiro Ito, Hisaki Kono, Hyeog Ug Kwon, Toshiyuki Matsuura, Yasuyuki Sawada, and seminar participants at RIETI and Nihon University for helpful comments and suggestions, and Toru Homma, Katsuhiko Ozawa, Akira Shibuya, Toshinobu Sugimura, Etsuji Yoshimura, and particularly Muhammad Furqon and Yohan Endro Susanto for their help in collecting data. Financial support by the Grant-in-Aid for Scientific Research (no ) from the Japan Society for the Promotion of Science is gratefully acknowledged. The opinions expressed and arguments employed in this paper are the sole responsibility of the author and do not necessarily reflect those of RIETI, the Japan International Cooperation Agency, the Ministry of Economy, Trade and Industry of Japan, the Institute for Research and Development of Metal and Machinery Industries of Indonesia, or any institution the author is related to. Graduate School of Frontier Sciences, the University of Tokyo, Kashiwanoha, Kashiwa, Chiba Japan ( yastodo@k.u-tokyo.ac.jp; URL: 1

2 1 Introduction Technological progress is the major source of economic growth (Romer, 1990). In the case of less developed countries (LDCs), technology diffusion from developed countries is the most important channel of technological progress (Grossman and Helpman, 1991, chs. 11 and 12), and technological barriers between national borders often prevent LDCs from income growth (Klenow and Rodriguez-Clare, 1997; Parente and Prescott, 2000; Caselli, 2005). Therefore, a number of empirical studies have examined practical channels of technology diffusion to LDCs. Recently, due to the widespread availability of firm-level data for LDCs and development of microeconometrics, many of those studies use firm-level data. For example, Javorcik (2004) and Todo and Miyamoto (2006) among many others find knowledge diffusion to LDCs is facilitated by technology spillovers from foreign direct investment, whereas Blalock and Gertler (2004), Van Biesebroeck (2005), and Amiti and Konings (2007) find that international trade promotes such diffusion. This paper focuses on technical assistance programs funded by foreign aid as an alternative channel of technology diffusion and estimates effects of technical assistance programs funded by Japanese aid on the technology level of participant firms, using a unique firm-level dataset for the Indonesian foundry industry. This study is motivated by three strands of literature. First, since the publication of a seminal paper by Burnside and Dollar (2000), impacts of foreign aid have been extensively examined using country-level data. Burnside and Dollar (2000) find that aid has positive impacts on growth of GDP per capita in countries implementing good policies, while this is not the case for countries implementing poor policies. However, subsequent studies such as Hansen and Tarp (2001), Easterly, Levine, and Roodman (2004), and Roodman (2007) find that the finding of Burnside and Dollar (2000) is not robust to different datasets, specifications, or estimation methods. Therefore, whether foreign aid stimulates economic growth of LDCs is still an open question from the perspective of country-level empirics. Some of the existing studies on the aid-growth nexus examine impacts of technical assistance in particular. For example, Gounder (2001) using time-series data for Fiji and Kohama, Sawada, and Kono (2003) using cross-country data disaggregate foreign aid into various types including technical assistance and find that technical assistance improves growth. Sawada, Matsuda, and Kimura (2007) also find that technical assistance facilitates technology transfer to LDCs and improves the growth rate of total factor productivity (TFP), using a method developed by Benhabib and Spiegel (2005). While these existing 2

3 studies use country-level data, this study contributes to the literature by looking at firmlevel evidence. To the best of the author s best knowledge, this is the first firm-level study on econometric evaluation of the impact of technical assistance programs by foreign aid. The use of micro-data allows one to investigate issues which are difficult to examine using macro-data, for example, what types of technical assistance program are more effective, or whether technology achieved by participants of aid programs spills over to non-participants. The second strand of literature related to this study is that on impact evaluation of development programs using micro-data, which has proliferated recently in development economics. Notable contribution has been made by Abhijit V. Banerjee and his colleagues in the Abdul Latif Jameel Poverty Action Lab (J-PAL), who claim that impacts of development programs should be evaluated by randomized trials (see, for example, Banerjee, 2007) and have indeed implemented and/or evaluated a number of programs in LDCs using randomized trials (Miguel and Kremer, 2004; Chattopadhyay and Duflo, 2004, among many others). The present study is in line with those studies in that this study uses firm-level micro-data to evaluate technical assistance programs of Japanese foreign aid. Note that the existing studies mostly evaluate impacts of programs related to education, health, poverty reduction, and micro-finance using household-level data. Evaluation of technical assistance programs using firm-level data is new to this literature. However, this study does not utilize randomized trials despite Banerjee s argument. One reason is that I collected data after the completion of aid programs implemented without any randomized trial. One might be able to engage in a program in the pre-program period and implement randomized trials. However, randomized trials in technical assistance programs to individual firms are not easy to implement due to ethical and political reasons. In Banerjee (2007), Ian Goldin, F. Halsey Rogers, and Nicholas Stern point out and Banerjee himself admits that randomized trials are not necessarily possible in all areas of development policies. Technical assistance to individual firms may be one of these areas. Third, this study relies on the literature on propensity score matching estimation. Without randomized trials, standard estimators, such as ordinary least squares (OLS), of the effect of a program may be biased due to self-selection to the program. For example, if potentially high-growth firms are likely to participate in a technical assistance program, the OLS estimator of its effect tends to be positive, reflecting the selection bias. To correct for such selection biases, this study combines propensity score matching and difference-in- 3

4 differences estimation, as suggested by Heckman, Ichimura, and Todd (1997, 1998). Recently, the difference-in-differences propensity score matching estimation is widely applied to non-experimental data for LDCs. For example, Arnold and Javorcik (2005) examine the impact of foreign acquisition on productivity in Indonesia, whereas van de Walle and Mu (2007) investigate impacts of construction of roads in rural areas in Vietnam. The Indonesian foundry industry is an interesting case to examine effects of technical assistance by foreign aid, since the Japanese government intensively implemented several types of program in the industry in recent years, including provision of technical consultations and training courses to local firms. The Japanese government was willing to develop local foundry firms which produce parts and components, partly because Japanese multinational enterprises in the downstream industries in Indonesia, such as the electric machinery and automobile industries, could benefit from the development of parts suppliers. To examine effects of these intensive Japanese aid programs, I collected data from a unique survey that covered 200 firms, or most firms in the industry except for very small family-operated firms. Applying the difference-in-differences propensity score matching estimation to the data, this study finds that the average effect of various types of technical assistance program funded by Japanese aid on the change in the reject ratio of products, a technology indicator, of participant firms is negative and statistically significant. Our results indicate that by participating in the aid programs, firms can reduce the reject ratio by 13 percent-16 percent. The size of the reduction is equivalent to what the average non-participant firm can achieve in six years. This evidence suggests that the aid programs help local participant firms improve their technology level. However, the effect of the aid programs is limited in two ways. First, technology improvement is limited to the participants of the programs and does not spill over to nonparticipants. Second, technical assistance programs provided by the local counterpart of aid after the completion of the aid programs do not seem to improve technology of participants on average. This evidence indicates that technology transfer to the local counterpart for longer-term effects, which is in fact a major objective of the aid programs, is unsuccessful. The rest of the paper is organized as follows. Section 2 describes technical assistance programs by Japanese aid in the Indonesian foundry industry. Section 3 explains empirical methodologies for impact evaluation, whereas Section 4 presents details of the data used. 4

5 Estimation results are shown in Section 5, and Section 6 provides conclusions and policy implications. 2 Technical Assistance Programs Funded by Japanese Aid There are several types of technical assistance program funded by Japanese development aid in the Indonesian foundry industry. Most notably, the Project on Supporting Industries Development for Casting Technology (hereafter, the SIDCAST project) was implemented from 1999 to 2004 by the Japan International Cooperation Agency (JICA), a public institution in charge of technical assistance to less developed countries, funded by the Japanese government. The SIDCAST project was a joint project with a local counterpart, the Institute for Research and Development of Metal and Machinery Industries (MIDC), 1 a public institution located in Bandung near Jakarta under the Ministry of Industry and Trade of the Indonesian government. 2 The ultimate goal of the SIDCAST project is to improve the technology and skill level of private firms in the Indonesian foundry industry. The targeted technologies and skills cover most stages of foundry, including wood pattern-making, casting, molding, sand preparation, melting, pouring, and testing and inspecting for quality control. To achieve this goal, JICA granted machinery and equipment worth about three million US dollars, sent eight engineers and technicians for two-four years and 61 for less than six months. These engineers and technicians provided technical assistance programs to local firms, using the machinery and equipment. Additionally, Japanese engineers taught technologies and skills to local engineers of MIDC so that it could on its own improve the technology level of local firms after the completion of the SIDCAST project. Technical assistance programs of the SIDCAST project took the following three forms. First, the project carried out 192 one-day visits to 71 local firms. In each visit, expertised engineers of the project directly provided technical consultation to workers of the firm. Second, the project held 18 training courses in MIDC or in a particular firm. 3 The period 1 MIDC stands for Metal Industry Development Center, an old name of the institute. Since this abbreviation is more widely used in practice, I also use it in this paper. 2 The description on the SIDCAST project in this section is based on MIDC and JICA (2002), JICA (2004), and the author s interviews with managers of JICA, MIDC, and private firms in the Indonesian foundry industry. 3 There were three more training courses for engineers of the government and universities. 5

6 of each course ranges from three days to three months, whereas its average is 20 days. The number of participants in a training course varies from two to 60 with an average of Third, six one-day seminars were held in or near MIDC. The average number of participants in these seminars was 160. All three types of technical assistance program above were offered by both Japanese engineers sent by JICA and local engineers of MIDC. However, Japanese engineers often played more significant roles than local instructors, particularly in the case of consultation visits (JICA, 2004, p. 14). In addition to the SIDCAST project, training courses are provided by the Association for Overseas Technical Scholarship (AOTS), a public institution closely related to the Ministry of Economy, Trade, and Industry (METI) of Japan. A major objective of the AOTS is to provide training to engineers and managers of LDCs. These training programs are in part funded by the Japanese government through its foreign aid. In AOTS training programs, local engineers and managers of LDCs, including Indonesia, are trained in the AOTS training center in Tokyo for a period from nine days to 13 weeks and further trained in private firms in Japan for several months. Finally, Japanese engineers provided technical consultations directly to local firms under two programs. Under the Expert Service Abroad Program of the Japan Overseas Development Corporation (JODC), more than 200 Japanese engineers are sent to LDCs each year, mostly in Asia, of which about a quarter go to Indonesia. Most of the cost of this program is financed by Japanese aid through METI. The other is JICA s Senior Volunteers Program, in which more than 400 retired and pre-retired Japanese in many fields of work, not limited to engineering, are sent to LDCs around the world each year. The share of Indonesia in the total number of Senior Volunteers is about six percent. In both cases, each Japanese expert provides technical assistance in local institutions for a period of up to two years. 3 Empirical Methodology 3.1 Problems in and solutions to impact evaluation The central aim of this paper is to identify the causal effect of participation in the technical assistance programs by Japanese aid on the technology level of local participant firms. Let D it be a dummy variable indicating firm i s participation in any technical assistance program mentioned in the previous section in year t. The outcome variable, an indicator of the technology level, of firm i in year t + s (s 0) is denoted by Y i,t+s (D it ), which depends 6

7 on whether or not the firm participated in the programs. Then, the effect of program participation in year t on the outcome in t + s is given by Y i,t+s (1) Y i,t+s (0). The major difficulty in examining this effect is that Y i,t+s (0) is not observable if firm i participates in the technical assistance programs, or if it is in the treatment group, while Y i,t+s (1) is not observable if firm i does not participate in the programs, or if it is in the control group. Therefore, existing studies on impact evaluation often estimate the average effect of treatment on the treated (ATT), defined as AT T = E(Y i,t+s (1) Y i,t+s (0) D it =1,X i,t 1 ) = E(Y i,t+s (1) D it =1,X i,t 1 ) E(Y i,t+s (0) D it =1,X i,t 1 ), (1) where X i,t 1 denotes pre-program characteristics of firm i in year t 1. The first term on the right-hand side of equation (1), E(Y i,t+s (1) D it =1,X i,t 1 ), can be estimated by the average outcome of the treated observations. However, problems arise when we estimate the second term, E(Y i,t+s (0) D it =1,X i,t 1 ), since Y i,t+s (0) represents the outcome level that firm i would have achieved if the firm had not participated in the program and thus is counterfactual. Using experimental data, one can estimate this by averaging outcomes of non-treated observations (i.e., Y j,t+s (0) where D jt = 0). However, given non-experimental data, as in the case of this paper, the characteristics of each firm affect its decision on participation in technical assistance programs. Therefore, characteristics of non-participant firms are likely to be different from those of participant firms, and thus the average of outcomes of non-participants is a biased estimate of the mean of counterfactual outcomes of participants if they had not participated in the program. To overcome this difficulty, Rosenbaum and Rubin (1983, 1985) develop propensity score matching (PSM) estimations, and Heckman, Ichimura, and Todd (1997, 1998), and Heckman, Ichimura, Smith, and Todd (1998) extend them. In PSM estimations, each participant in the program is matched with a non-participant that has a similar probability of participation according to its pre-program characteristics. From the average of the matched observations in the control group, one can obtain a reasonable estimate of the mean of counterfactual outcomes of participants if they had not participated in the program. Accordingly, a PSM estimator of the ATT can be generally written as PSM = 1 Y i,t+s (1) W (P (X i,t 1 ),P(X j,t 1 )) Y j,t+s (0), (2) N j I0 i I 1 7

8 where I 1 and I 0 are respectively the treatment and the matched control group, and N is the number of observations in the treatment group. P (X) represents the propensity score, or the probability of participation in the program given X, andw is a weight determined by the distance between propensity scores of the treated and the matched control observations. In addition, when panel data are available, as in the case of this paper, one can employ a difference-in-differences (DID) PSM estimator of the ATT proposed by Heckman, Ichimura, and Todd (1997, 1998). The DID-PSM estimator is defined as DID PSM = 1 ΔY i,t+s (1) W (P (X i,t 1 ),P(X j,t 1 )) ΔY j,t+s (0), (3) N j I0 i I 1 where ΔY i,t+s Y i,t+s Y i,t 1,orΔY i,t+s is the (s+1)-period difference in Y. An advantage of the use of the DID-PSM estimation is that it can eliminate time-invariant effects on the outcome variable that are not correlated with covariates, X. Heckman, Ichimura, and Todd (1997, 1998) and Smith and Todd (2005) find that DID estimators perform better than matching estimators without using DID. 3.2 Practical procedures for the DID-PSM estimation To obtain the DID-PSM estimator of the impact of the technical assistance programs of Japanese aid given the dataset in hand, I first examine how participation in the programs is determined, using a probit model. The covariates used in the probit estimation are as follows: the log of output per worker measured by the total weight of products and its square; the log of the number of workers and its square; the share of workers with a high-school degree or higher in the total number of workers; the share of foreign workers; a dummy variable that indicates whether the firm receives technical assistance through other channels, for example, from foreign firms and universities; and region and year dummies. Output per worker is an indicator of firms productivity, whereas the number of workers indicates firms scale. Since I find these two variables have non-linear effects on participation in technical assistance programs as shown later, their squares are incorporated as additional covariates. The shares of educated workers and foreign workers are also potential determinants of participation, since those may be correlated with firm productivity which further influences the participation decision. Moreover, since educated and foreign workers can obtain information on the technical assistance programs more easily, their presence in a firm may lead to a larger propensity of the firm s participation. Since participants in technical assistance programs 8

9 unrelated to Japanese aid are also likely to participate in the programs funded by Japanese aid, I include the dummy for participation in technical assistance by other institutions as a covariate. 4 Based on the propensity score from the probit estimation, I employ two alternative matching methods to create the matched control observations: caliper and kernel matching. In both methods, I impose a common support condition and drop observations in the treatment group whose propensity score is higher than the maximum or lower than the minimum score among observations in the control group. In the case of caliper matching, each observation in the treatment group is matched with a control observation that has the closest propensity score to the treated observation s score within the maximum score distance, or the caliper. In this study, the caliper is set at In the case of kernel matching, each treated observation is matched with the weighted average of all control observations in the common support region. More specifically, the weight function W in equation (3) is given by G (P (X j,t 1 ) P (X i,t 1 )) /a n W (P (X i,t 1 ),P(X j,t 1 )) = k I 0 G (P (X k,t 1 ) P (X i,t 1 )) /a n where G is the Epanechnikov kernel function and a n, the bandwidth, is set at I match treatment observations with control observations in the same year, following Arnold and Javorcik (2005). In the case of evaluation of impacts of a job training program, Heckman, Ichimura, and Todd (1997) find that matching estimates perform well when participants and non-participants reside in the same local labor market. Therefore, they argue that geographic mismatches should be avoided in matching estimation. In the case of this paper, time, rather than geographic, mismatches may be more substantial, since the data of this paper contain a six-year period as explained in detail later and technical assistance programs were provided throughout the period. Therefore, the time restriction is imposed in this study. Since firms in the Indonesian foundry industry are clustered in four regions, 4 Another potential covariate is the amount of capital stock. Although I do not have data on the amount of capital stock, I do have data on the maximum capacity of furnace used in each firm. The dataset also includes information on the type of furnace: traditional furnaces called tunki, cupola furnaces, or electric furnaces. However, I found that either the maximum capacity or the type of furnace does not have any significant effect on participation in the aid programs when it was included in the probit estimation. 5 Another widely-used kernel is the Gaussian kernel. In addition, a generalized version of kernel matching, called local linear matching, is proposed by Heckman, Ichimura, and Todd (1997, 1998). According to Fan (1992), an advantage of local linear estimators over kernel estimators is better adaptation to different data densities. I experimented with Gaussian kernel matching and local linear matching and found qualitatively the same and quantitatively similar ATT estimates as in the case of Epanechnikov kernel matching. However, I also found that these types of matching led to a failure in balancing tests, explained below. Therefore, the benchmark estimation employs Epanechnikov kernel matching. 9

10 geographic mismatches may also lead to a bias in matching estimation. However, because of a small sample size in this paper, matching participant firms with non-participants within the same year and region may lead to bad matches or a very small sample after matching. Therefore, this paper allows geographic mismatches. After the matching, the treatment and the control group should have similar characteristics in the pre-program period. To check whether this is the case, I employ two types of balancing test. First, a simple t test is used to examine whether the mean of each covariate differs between the treatment and the control group after matching. In addition, following Girma and Görg (2007), the Hotelling s T -squared test is performed to jointly test the equality of the mean between the two groups for all covariates. Second, I run probit using the sample after matching and compare the pseudo-r 2 with that obtained from the probit estimation using the sample before matching. In addition, a likelihood-ratio test is performed to test whether all the estimated coefficients from the after-matching probit estimation are zero. These tests are proposed by Sianesi (2004). If matching is successful, the after-matching probit should have no explanatory power so that the pseudo-r 2 should be low and the estimated coefficients should be close to zero. Given that the treatment and the control group pass the balancing tests, I compute the DID-PSM estimator using equation (3). To take the advantage of the panel data for this paper which cover a six-year period from , the length of years between treatment and impact evaluation (s in equation [3]) is set at either 0 or 1. 6 By so doing, I can examine contemporaneous effects of the technical assistance program as well as its a-yearafter effects. The standard error of the DID-PSM estimator is obtained by bootstrapping based on 100 replications, following Smith and Todd (2005). Most existing studies use bootstrapping standard errors for PSM estimators, since multiple steps in PSM estimation, including estimation of propensity scores and matching procedures, lead to larger variation in PSM estimators than standard estimators with only one step. 7 6 A larger s significantly lowers the number of observations. Therefore, I did not use s greater than one. 7 In practice, these estimation procedures are performed using Stata s commands based on psmatch2 of Leuven and Sianesi (2003) and bootstrap. 10

11 4 Data 4.1 The firm-level survey and dataset To quantitatively examine whether technical assistance programs funded by Japanese aid improve the technology level of local participant firms, I conducted a unique firm-level survey in the Indonesian foundry industry in cooperation with MIDC, 8 from November 2006 to May A questionnaire was mailed to 200 firms in the industry, and MIDC staff visit each firm to collect responses directly from them. According to an MIDC manager, those 200 firms cover almost all firms in the industry, except for very small family-operated ones. These firms are clustered in four regions: western Java including Jakarta, Bekasi, and Bandung, 10 Klaten in Central Java Province, Surabaya, Mojokerto, and other cities in East Java Province, and Medan in Sumatra Province. Foundry products they produce are mostly parts and components of machinery, electric machinery, and automobiles, ranging from simple products such as pulleys and levers to more advanced such as crank shafts and cylinder heads. Among the 200 firms surveyed, 150 firms responded so that the response rate is 75 percent, a high rate for this type of firm-level survey. The data collected by the survey include information on outputs, inputs, technology indicators, and participation in technical assistance programs related and unrelated to Japanese aid during the six-year period Since some firms were established after 2000 or lack information in earlier years, the raw dataset from the firm-level survey contains 659 firm-year observations for which necessary information is available. Our primary indicator of firm-level technology is the reject ratio, or the share of rejects in the total number of products, which is widely recognized as a measure of firm-level technology in the foundry industry. 11 A disadvantage of the reject ratio as a measure of the technology level is that if firms improve their technology level and hence produce more advanced foundry products, the reject ratio may not improve despite the technological 8 I cooperated with MIDC since it is closely linked with the industry, so that firms are more likely to respond to the survey if MIDC engages in the survey. An obvious disadvantage of the cooperation with MIDC is that since MIDC implemented the SIDCAST project, responses of private firms can be biased when MIDC engages in the survey. 9 The survey period was prolonged due to severe flooding in Indonesia in the early West Java is a name of a province in Indonesia, in which Jakarta is not included. Note that in the present paper, western Java is differently defined from West Java and includes Jakarta and West Java. 11 A potential alternative measure is total factor productivity (TFP), but since information on the amount of capital stock is not available, it is impossible to construct TFP from the dataset. Since most firms surveyed are small- and medium-scale enterprises (the median number of employees is 36), they are often unaware of the value of their own capital stock. 11

12 improvement. Therefore, the technological improvement inferred from changes in the reject ratio is likely to be underestimated. Although the majority of firms provided detailed numbers for the reject ratio, some reported very rough numbers, such as 5 or 10 percent for all years surveyed. I drop those firms from the dataset. When firms use more than one kinds of metal as materials, they report the reject ratio for each kind of metal. In that case, I constructed the weighted average of the reject ratio, using as the weight the share of each kind of metal in the total weight of products. 12 Since I perform matching using pre-program characteristics and employ DID estimation using post-program outcomes, each observation for the estimation should contain information for multiple years. For simple presentation, let an observation in year t consist of data on (1) whether or not the firm participate in any type of technical assistance program in year t, (2) the reject ratio in year t + s, the post-program period where s, either 0 or 1, denotes the length of years between program participation and evaluation, and (3) the reject ratio and other firm-level characteristics, such as the number of workers, in year t 1, the pre-program period. Observations for which any information among (1) (3) above is missing are dropped. I also drop observations in the starting year, 2000, since no pre-program firm characteristics are available. 13 In addition, all observations in 2005 are dropped for two reasons. First, when evaluating effects one year after the aid programs finish, or when s =1, I naturally drop observations in 2005 since no outcome data are available for Second, when I evaluate contemporaneous effects of the aid programs, or when s = 0, I also drop observations in 2005 since the SIDCAST project, the major technical assistance programs funded by Japanese aid in the industry, was completed in These data management processes lead to a sample of 85 firms and 285 observations for estimation. 4.2 Descriptive statistics Among the 285 observations in the sample, 93 participated in one or more technical assistance programs funded by Japanese aid, including consultation visits, training courses, and seminars of the SIDCAST project, training courses of AOTS, and technical services provided by JODC engineers and JICA s Senior Volunteers, as shown in panel A of Table A better measure of production for the weight may be sales. However, sales for each kind of metal are not available. 13 However, data for 2000 are used as pre-treatment characteristics of observation in

13 Among those, 88 participated in the SIDCAST project, of which 65 received consultation visits and 62 and 42 participated in training courses and one-day seminars, respectively. 27 Japanese senior volunteers worked in the industry, whereas the number of participants in AOTS or JODC was small. This same table suggested that many firms participate in more than one program in the same year. For example, out of 65 observations that received consultation visits, 42 and 31 participated in training courses and seminars, respectively, in the same year. Out of 27 observations that received technical services from Senior Volunteers, 23 participated in more than one program of the SIDCAST project. The same panel also shows the distribution of participants and non-participants in terms of the four clusters of the foundry industry in Indonesia. The panel indicates that participants of technical assistance programs funded by Japanese aid were heavily concentrated in Central Java Province: 67 out of 144 observations in Central Java, or 47 percent of all observations in the region, participated in the project, while only 26 out of 141 in other regions, or 18 percent, participated. For most types of program, this geographical concentration in Central Java is found, except for the AOTS and JODC programs, for which participants were mostly located in western Java. The number of participants and non-participants by year is presented in panel B of Table 1, showing that the degree of technical assistance does not vary much over time, although the aid programs seem to be the most active in The table shows figures for 2005, although the SIDCAST project was completed in In the case of 2005, participants in the SIDCAST project indicate participants in technical assistance programs provided by the local counterpart, MIDC, on its own without the help of Japanese engineers. Therefore, the fact that there were 18 participants in 2005 indicates that MIDC is actively providing technical assistance to private firms after the completion of the foreign aid project. 14 Table 2 provides summary statistics used in estimation. The reject ratio varies substantially across observations, ranging from 0.1 percent to 30 percent. However, it declines over time on average, and the mean of the first difference in the log of the reject ratio, or the annual growth rate of the ratio, is 2.5 percent. Another measure of firm-level technology is sales per worker, but this is not available for many firms due to high confidentiality of data on sales in general. Using a subsample in which sales data are available, I also find that sales per worker substantially varies across observations and grows over time on aver- 14 Note, however, that when estimating impacts of technical assistance programs funded by Japanese aid, I exclude programs provided by MIDC in

14 age. The number of workers is 79 on average, whereas its median is 36 and it is less than 100 in 230 out of the 285 observations. These observations indicate that the firms in the sample are mostly small- and medium-scale enterprises. The average share of workers with a high-school degree or higher is 4 percent, suggesting that the education level of workers in the industry is quite low. Foreign workers are absent in most firms, and their average share is only 0.1 percent. About 20 percent of firms participated in technical assistance programs unrelated to foreign aid and provided by other institutions including private foreign firms and local universities. 5 Estimation Results 5.1 Average effect of all types of aid program In the benchmark estimation, I define treatment as participation in any technical assistance program funded by Japanese aid. In other words, although there are various types of program as described in Section 2 and each of these types may have a different impact in size, I estimate the average effect of all the aid programs. This is for two reasons. First, the number of participants is small when I estimate the impact of each type, in particular in the case of the AOTS, the JODC, or the Senior Volunteers program (see Table 1). Second, many firms participated in more than one program in the same year, as shown in Section 4.2. Therefore, it is not easy to distinguish between the effect of the various types of aid program. Following the empirical strategy described in Section 3, I first perform a probit estimation to obtain propensity scores for matching, using participation in any aid program as the dependent variable. The estimation results presented in column 1 of Table 3 indicate the following. 15 First, output per worker measured by the weight of products, a measure of productivity, has an inverted-u shaped effect on the probability of participation in the aid programs. Up to a certain level of productivity, the propensity for participation rises with the productivity level. However, beyond the threshold level of productivity, more productive firms are less likely to participate in the programs. Second, the impact of the number of workers, a scale indicator, is also inverted-u shaped. In general, the larger the firm, the greater the propensity of participation, whereas this relation does not apply when firms 15 For later use, column 2 shows probit results using participation in the SIDCAST project as the dependent variable. 14

15 are very large. Third, the share of educated workers has a positive impact, supporting the presumption that educated workers have greater access to the aid programs. Fourth, the effect of the share of foreign workers is positive but insignificant. 16 Finally, the dummy for participation in technical assistance programs provided by other institutions is positively correlated with the probability of participation in the programs funded by Japanese aid. As the second last row of Table 3 indicates, pseudo R squared from the probit estimation, 0.37, is sufficiently high for the matching purpose. These results indicate that firm-level characteristics of the treatment and the control group are substantially different from each other. Therefore, I confirm that by simply comparing the average outcome of the treatment group with the average of the control, it is impossible to distinguish between effects of the aid programs and effects of other characteristics, such as the firm size and technical assistance by other institutions. Therefore, based on the propensity score from the probit estimation, I create a new control group using caliper or kernel matching, so that the treatment group and the new control group after matching have similar characteristics. Then, the average of outcomes of the matched control group can be a good estimate of the mean of counterfactual outcomes of treated observations if they had not received the treatment. After matching, I perform two types of balancing test, as described in Section 3.2. One is a simple t test and the Hotelling s T -squared test, to check the similarity of firm characteristics between the two groups. The other is a pseudo-r 2 test and an LR test to check whether probit estimation for participation on the sample after matching has no explanatory power. The results of the t tests presented in Table 4 indicate that the mean of each covariate for the treatment group and its mean for the control group are sufficiently similar in all cases after matching, although these are substantially different in most cases before matching. In addition, the Hotelling s tests suggest that differences in firm characteristics between the treatment and control groups are jointly insignificant after matching. Table 4 also indicates that the pseudo-r 2 is very low after matching, while it is high before matching. According to the p value of the LR statistic, the hypothesis that all the estimated coefficients are zero cannot be rejected. 17 These results suggest that 16 Heckman, Ichimura, Smith, and Todd (1998) and Smith and Todd (2005) suggest that PSM estimators are sensitive to the selection of covariates in propensity score estimation. Therefore, I experimented with probit estimation without using the share of foreign workers and found that the results are very similar to the benchmark results. 17 Note that the number of observations in the case of kernel matching is larger than that in the case 15

16 both caliper and kernel matching are successful, and that there is no systematic difference between the treatment group and the control group after matching. Using the treatment group and the matched control group, I now construct the differencein-differences propensity score matching (DID-PSM) estimator of the average treatment effect on the treated (ATT) from equation (3). The results for the two types of time length between treatment and evaluation, 0 or 1 year, are presented in Table 5. Note that since the outcome variable is the log of the reject ratio and DID estimation is employed, what is compared is the growth rate of the reject ratio between the treatment and the control group. For reference, OLS estimators using the sample before matching are present in columns (1) and (4), indicating no significant effect of the aid programs. However, this is not the case after matching, as shown in columns (2), (3), (5), and (6) of Table 5. In all cases, regardless of whether caliper or kernel matching is used, or whether the time length between participation and evaluation is 0 or 1 year, the effect of participation in the aid programs on the growth of the reject ratio is negative and statistically significant at the 5- percent level. 18 The comparison between the OLS estimator and the DID-PSM estimators implies that potentially low-growth firms were chosen as participants in the aid programs. On average, firms participating in the aid programs reduce the reject ratio by 13 percent- 15 percent (not percentage points) in the year of participation and further reduce it by 1 percent-3 percent in the next year. Since the average reduction rate of the reject ratio for non-participant firms is 2.5 percent, 19 the size of the reduction due to participation in the aid programs is equivalent to what the average non-participant firm can achieve in six years. The results suggest that the technical assistance programs by Japanese aid indeed improved the technology level of local participant firms on average and that the effect was not negligible in size. 5.2 Effect of the SIDCAST project As explained earlier, estimating the effect of each type of aid program separately is difficult, since participants in various types of program overlap in the dataset. However, this subsection ventures to focus on the effect of participation in the programs of the SIDCAST of caliper matching, since in the latter treated observations are dropped if they do not have untreated observations within the caliper. 18 To check the robustness of the results, I perform OLS estimations on the after-matching sample, using as independent variables post-program characteristics at the firm level and region and year dummies in addition to the treatment dummy. The results are very similar to those for ATT. 19 Table 2 shows that the average for all firms is also 2.5 percent. 16

17 project, which covered the major aid programs in the industry, for further analysis. First, I estimate the average effect of participation in any of the three types of program (consultation visits, training courses, and seminars) of the SIDCAST project. In this case, the control group for matching consists of observations which do not participate in any type of aid program including the AOTS, the JODC, or Senior Volunteers program. The results from probit estimation shown in column 2 of Table 3 are very similar to the previous results shown in column 1. Using the propensity score from the probit estimation, the matched control group is constructed, whose characteristics are sufficiently similar to the participants in the SIDCAST project, according to balancing tests. 20 Using the treatment group and the matched control group, I estimate the average effect of participation in the SIDCAST project on the change in the reject ratio, finding it negative and significant at least at the 10-percent level (panel A of Table 6). This finding, similar to the finding on the average effect of all aid programs, is not surprising, since 88 out of the total of 93 participants in the aid programs funded by Japanese aid participated in the SIDCAST project (Table 1). I further distinguish between the three types of program of the SIDCAST project and estimate the average effect of each type. Panels B-D of Table 6 indicate that consultation visits and training courses are effective in lowering the reject ratio, while participation in seminars does not have a significant impact. The results should be interpreted with caution, since participants in one program often participate in other programs so that the estimate effect of participation in one type of program does not necessarily reflect the true effect of the individual program. Instead, the estimated effect of a program should be viewed as the average effect of the program when combined with other programs to the average extent. However, despite this caution, the difference between results on consultation visits and training courses and those on seminars is noticeable, since the difference is likely to stem from differences in intensity of technical assistance among programs. Note that each seminar was only one day long, while the average period of training courses was 12.5 days, as shown in Section 2. Although each consultation visit was also one day long, technological diagnosis provided in consultation visits was specific to individual firms while seminars provided more general information to many participants. In addition, several firms were chosen as target firms in the program and repeatedly received consultation visits within a year (JICA, 2004). Therefore, the difference among the three types of technical assistance programs suggests 20 The results of the balancing tests are, for the sake of brevity, not presented but available upon request. 17

18 that technology transfer requires lengthy and intensive training or consultation. 5.3 Intra-region technology spillovers A disadvantage of the analysis above is that if new technologies and skills achieved by participant firms spill over to non-participants, the effect of the aid programs is underestimated. In the extreme case in which technology perfectly spills over across firms, there would be no difference in the technology level between participants and non-participants, and hence the effect of the aid programs on technological progress could not be detected by the DID-PSM estimation employed in this paper. Therefore, whether the effect of the aid programs spills over to non-participants is an important issue when considering the overall effect of the programs. To examine this issue, I focus on intra-region spillovers, i.e., spillovers within each of the four clusters of foundry firms. Technology diffusion is often geographically localized so that inter-region spillovers, or spillovers across regions, are less likely to take place, as Jaffe, Trajtenberg, and Henderson (1993) and Branstetter (2001) suggest. Intra-region spillovers can be examined particularly in the case of this study, since participants in the aid programs were heavily concentrated in Central Java (Table 1). If intra-region spillovers in fact took place, firms in Central Java which did not participant in any aid program should have improved their technology level to a greater extent than non-participants in other regions. To check if this is the case, I perform OLS estimation using a sample of non-participant observations to estimate the effect of the dummy variable for firms in Central Java on the two-year difference in the log of the reject ratio (or the growth rate of the reject ratio for a two-year period). Non-participants are defined as firms that do not participate in any aid program over the preceding three years. The result presented in column 1 of Table 7 indicates that the effect of the dummy is positive and significant at the 10-percent level. This evidence implies that despite the presence of many participant firms in Central Java, technological improvement of non-participant firms in that region is smaller in size than that of non-participants in other regions. However, this finding does not necessarily reject the intra-region spillover hypothesis, since characteristics of non-participant firms in Central Java and other regions may differ. For example, if technological progress of non-participants in Central Java is potentially very slow, it is still possible that they receive technology spillovers from participants in the same 18

19 region but still improve their technology slower than non-participants in other regions. Therefore, I apply the propensity score matching estimation previously used to this analysis, defining treatment as locating in Central Java. Thus, the treatment group is firms in Central Java that did not participate in any aid program, while the control group is non-participant firms in other regions. Using the balancing tests as before, I find in the before-matching sample that non-participants in Central Java employed less educated workers and participated in technical assistance programs by other institutions more frequently than non-participants in other regions. However, these differences between the treatment and the control group become statistically insignificant after matching. The results from the PSM estimation presented in columns 2 and 3 of Table 7 indicate that the difference in the change in the reject ratio between non-participants in Central Java and those in other regions is not significant, rejecting the intra-region spillover hypothesis. In other words, the effect of the technical assistance programs funded by Japanese aid is restricted to participants in the programs, and hence the estimate of the average effect of the aid programs in Section 5.1 is unlikely to be undervalued. 5.4 Effect of technical assistance by the local counterpart of aid programs As explained in Section 2, the SIDCAST project funded by Japanese aid was completed in 2004, but Indonesian engineers of the local counterpart of the project, MIDC, continued to provide technical assistance to local firms on their own without the help of Japanese engineers. In fact, one of the major objectives of the SIDCAST project is to train local engineers of MIDC so that they can provide their own technical services to local firms. Since the dataset includes information on whether local firms participated in programs provided by MIDC in 2005, it is possible to estimate the effect of MIDC s own programs. However, since the number of participants in MIDC s programs in 2005 is small, 18 as shown in Table 1, focusing on these participants in particular leads to a very small sample, and thus results from the sample may not be reliable. Therefore, I instead estimate the average effect of all technical assistance programs during the period , including those of the SIDCAST project from 2001 to 2004 and those of MIDC in By comparing this average effect with the average effect of programs of the SIDCAST project obtained in the previous subsection, I can infer the average effect of MIDC s own programs. 19