INCORPORATING INFORMATION COMPLEXITY INTO REGRESSION-BASED FREIGHT GENERATION MODEL SELECTION

Transcription

1 INCORPORATING INFORMATION COMPLEXITY INTO REGRESSION-BASED FREIGHT GENERATION MODEL SELECTION Paper No Words: (5, *250) = 7,471 by Hyeonsup Lim Department of Civil and Environmental Engineering The University of Tennessee, Knoxville 311 JD Tickle Building, Knoxville, TN Phone: (865) hlim4@utk.edu Shih Miao Chin, PhD (Corresponding Author) Center for Transportation Analysis Oak Ridge National Laboratory 2360 Cherahala Boulevard, Knoxville, TN Phone: (865) chins@ornl.gov Ho-ling Hwang, PhD Center for Transportation Analysis Oak Ridge National Laboratory 2360 Cherahala Boulevard, Knoxville, TN Phone: (865) hwanghl@ornl.gov Lee D. Han, PhD Department of Civil and Environmental Engineering The University of Tennessee, Knoxville 319 JD Tickle Building, Knoxville, TN Phone: (865) lhan@utk.edu

2 Lim et al ABSTRACT Freight transportation is vital to the growth of national economy and advancement of the wellbeing of America. The transportation community has been engaging in comprehensive and inclusive transportation planning, design, and implementation. Freight demand modeling is an essential part of the overall transportation development strategy for the future. It enables officials, policy-makers, and transportation analysts to have a comprehensive understanding of current and future transportation needs. Consequently, alternatives that address national transportation demands can be generated, and freight development strategies can be formulated to guide future transportation investments. This paper presents a paradigm shifting freight-demand model formulation. Instead of using the same functional equation form and a similar set of independent variables for all industry sectors, this study generates different models, with different functional forms and various sets of available independent variables, for each industry sector. Data from three Commodity Flow Surveys, which consists of state-level origin-destination movements of goods for two-dozen industry sectors, are utilized to showcase the practice of constructing state-level industry-based freight demand models and selecting the best models using the Bozdogan s index of information complexity approach. This paper suggests that freight models should be tailored to individual industry sectors, i.e., use different functional equation form with different independent variables for each given industry sector. The resulting freight demand models enable transportation professionals to: Disaggregate the model s geographic resolution from state (or metropolitan) level to county level, Update annual provisional freight data for the intermediate years between CFS surveys, and Forecast long-term freight movements KEY WORDS: Freight Generation Model, Production, Attraction, Industry Type, Model Selection, and Information Complexity (ICOMP)

3 Lim et al INTRODUCTION Background Transportation is the lifeline of the nation, connecting people and employment, supporting industries and economy, facilitating the delivery of vital goods and services. Freight transportation is the connection of these gigantic supply chains that tie the economy of this nation together. Recognizing the significance of freight transportation, the U.S. Department of Transportation (DOT) has been heavily engaged in comprehensive and inclusive transportation planning, designing, and implementation. Such processes enable elected officials, policy-makers, and transportation analysts to have a comprehensive understanding of current and future transportation needs. It also allows stakeholders to generate alternatives to address state level transportation demands and to develop strategic programs that guide transportation investment decisions. In order to collect the desperately needed transportation movement information, the Bureau of Transportation Statistics (BTS) took on the responsibility of initiating and administrating the quinquennial Commodity Flow Survey (CFS) since The CFS is the only publicly available survey of data on goods movements, which is the primary source for providing national and state-level data on domestic freight shipments by establishments in mining, manufacturing, wholesale, auxiliaries, and selected retail and services trade industries. The latest release of the CFS is for The study described in this paper utilizes three years (2002, 2007, and 2012) of CFS state-level data for two-dozen industry sectors in an effort to enhance the current state of the art regressionbased freight demand models. This research is a paradigm-shifting endeavor on a model selection. Traditionally, a transportation modeler would use a same functional form and a similar set of independent variable(s) to formulate a freight model that has similar freight shipping characteristics. Instead, in this study, a group of different models, based on three different functional forms and six independent variables, is generated as the impending candidate freight models for each industry sector. The three functional forms investigated are linear or logarithmic, ton or value, and freight production or freight attraction. In addition, the six independent variables evaluated include annual payroll, number of employees, number of establishments, population, number of registered vehicles, and number of registered trucks. Considering the full combinations of the three functional forms and the six independent variables, a set of 1,016 impending candidate freight models is created for each industry sector. Information criterion is employed to calculate a performance index for each set of the 1,016 impending candidate freight-demand models. Four best freight models with the highest performance index, in terms of total freight production and attraction in tons and value, are then selected. Here, the performance index is calculated based on expected entropy in the data by considering both "goodness-of-fit" and model complexity simultaneously. A model complexity involves both the quantity of independent variables and the covariance between the independent variables.

4 Lim et al Literature Review The freight generation model, i.e., estimating the production and attraction of freight movement, is the first process in the four-step model, and its result affects the rest of the processes. In aspects of freight generation modeling approaches, many studies use regression models with a consideration of cross-classification [1-9]. The cross-classification model, also referred to as a category analysis model, assumes that the freight generation rate (or parameter estimates of regression model) of a certain category (e.g., industry, commodity, or mode) is the same. The advantages of the regression model are on its simplicity, convenience, and ability to provide statistical inferences. The link function of the regression used is typically linear or logarithmic [1, 2, 5-11], except in a few studies [12]. Among the reviewed papers, only Chin and Hwang chose different link functions by industry based on R-square, while other studies employed only one of the two forms for all of their classification types [6]. According to findings from a literature review of the regression model approaches, the linear model seems to be more straightforward, while the logarithmic function fits relatively better, but not conclusively. Moreover, economic activities are the major factors used for developing freight generation models in almost all of freight modeling studies [1-18]. The primary economic variables, described in Federal Highway Administration (FHWA) Quick Response Freight Manual II, include employment, industry type, and population. They also set the zero intercept, assuming that freight movement is always related to one of the economic activities [13]. Chin and Hwang, and Oliveira-Neto et al., which are base studies of this paper, pointed out that annual payroll alone explained roughly 80~90% of variance in the model for most of industry types [6, 7]. Bastida and Holguin-Veras (2009) tested more than 190 variables at company level and stated that industry type, commodity type, facility type, number of employees, and total sales are found to be statistically significant [8]. Ranaiefar et al. and Novak et al. showed the substantial improvement of model structure by incorporating spatial autocorrelation or supply chain of commodity [2, 9]. In spite of all these efforts on the model developments, few studies discuss the model selection criteria. Although most of the studies have different parameter estimates by industry type or have the industry type as a categorical variable, the variable selection or link function of their model remained the same. In other words, the variable selection is identical regardless of the difference in industry type. Thus, the industry type changes only the intercept or the slope of certain parameters in the model, not affecting the variable selection. Specifically, Chin suggested a set of industry-specific models using either linear or power (logarithm) regression selectively based on R-square, but the same variable set was used for the all industry types [6]. Objective of This Study The goal for this research is to formulate a set of state-level industry-based freight demand models and select the best model using the Bozdogan s index of information complexity (ICOMP) approach.

5 Lim et al Specifically, the modeling efforts are to: Formulate a set of state-level regression-based freight demand models for each industry sector (e.g., linear or logarithmic, with or without intersect, etc.). Generate freight demand models for different sets of business activity measurements (e.g., number of establishment, employment size, annual payroll, etc.). Generate the production and attraction, in dollar values and tons, for U.S. domestic goods movements by industry sector. Utilize the ICOMP to determine the best functional form for each industry sector. Apply models developed using prior-year CFS data to estimate the production and attraction of subsequent CFS, e.g., use the 2007 CFS-based model to estimate the production and attraction of the 2012 CFS. Calculate error measurements (e.g., Mean Absolute Percentage Error, or MAPE) between the observed numbers and the estimates of the target year. Due to the word-limitation on papers imposed by the publisher, the estimation of trip generation and attraction rates has been eliminated from this paper. PUBLIC DATA SOURCES Commodity Flow Survey [19] As abovementioned, the CFS is the only publicly available survey data for commodity flows in the United States. It provides data on the types, origins and destinations, values, weights, modes of transport, distance shipped, and ton-miles for commodities shipped. To facilitate the development of industry-based freight demand models, three special tabulations were made available by the U. S. Census for the three CFS years (2002, 2007, and 2012). These state-level origin-destination movement of goods (tons and value) between states are available by industry sectors. The CFS covered 28 industry sectors in 2007 and 27 industry sectors in 2002 and The industry code 511 (publishing industries) and 551 (management of companies and enterprises), covered in 2007, were not included in 2002 and 2012 data respectively. Also, the shipments of industry code 454 (Non-store Retailers) and 493 (Warehousing and Storage) were partially covered in 2012, which does not represent the whole shipments at the 3-digit North American Industry Classification System (NAICS) level. By eliminating these four industries, this study considers 24 industry sectors covered in all three CFS years, as shown in Table 1. County Business Patterns [20] The County Business Patterns (CBP) is an annual series of economy data at county level published by the U.S. Census Bureau. It contains information on the number of establishments, employment, first quarter payroll, and annual payroll. The CBP covers most of the industries, from 2- to 6- digit NAICS codes, which is useful for analyzing the economic activities in small areas and examining the economic changes over time. The CBP data is used in this study as quantitative measures for industry sector activities.

6 Lim et al TABLE 1 NAICS 3-Digit Industry Codes Included in This Study 1997 NAICS (2002 CBP Data) 2002/2007 NAICS (2007/2012 CBP Data) Description (based on 2012 NAICS) Mining (except Oil and Gas) Food Manufacturing Beverage and Tobacco Product Manufacturing Textile Mills Textile Product Mills Apparel Manufacturing Leather and Allied Product Manufacturing Wood Product Manufacturing Paper Manufacturing Printing and Related Support Activities Petroleum and Coal Products Manufacturing Chemical Manufacturing Plastics and Rubber Products Manufacturing Nonmetallic Mineral Product Manufacturing Primary Metal Manufacturing Fabricated Metal Product Manufacturing Machinery Manufacturing Computer and Electronic Product Manufacturing Electrical Equipment, Appliance, and Component Manufacturing Transportation Equipment Manufacturing Furniture and Related Product Manufacturing Miscellaneous Manufacturing Merchant Wholesalers, Durable Goods Merchant Wholesalers, Nondurable Goods Input-Output Accounts [21] Note that the CFS is a shipper-based survey, i.e., information on industry sector of the shipper is collected, thus known. However, there is no information on the establishment receiving the shipments. Additional effort is necessary in order to transform shipper-based information to the receiving end. These transformations make use of the Make and Use tables from the Input- Output (I-O) Accounts data compiled and released by the Bureau of Economic Analysis (BEA). The BEA I-O data shows how industries interact, by providing inputs to and outputs from each other. The standard tables include a make table and a use table. The make table represents the commodities produced by each industry, while the use table describes the inputs to industry production and the commodities consumed by final users. Auxiliary Data Besides the three major data sources discussed above, this study also applied data from other sources, particularly when considering additional variables and practical applications of the model in forecasting freight generations. These data sources include Census population data [22] and vehicle registration (both all vehicles and trucks) [23].

7 Lim et al VARIABLE SELECTION CONSIDERATIONS The authors abide to three principles in selecting data for the model parameters and independent variables. That is, the data sources should be (1) publicly available, (2) geographically disaggregated at county level, and (3) released on a regular basis (e.g., annually). Unfortunately, no data sources could be identified to allow for a county-level response variable. Nonetheless, the model developed under this study can be used to estimate the county level production and attraction, since all input parameters used were publicly available at the county level. As an example, Table 2 lists the two dependent variables (dollar value and ton) and the six independent variables, with their associated means and standard deviations, for the mining industry (NAICS 212). Note that states with zero volumes in shipments were excluded from the analysis because log-transformation is only applicable on positive numbers. TABLE 2 Summary Statistics of Variables (for the Production Model of NAICS 212) Variable (name in the model) Data Source 2002 (N=50) 2007 (N=50) 2012 (N=48) Mean Std. Mean Std. Mean Std. Shipments in million dollars (Value)* CFS 947 1,084 1,714 1,969 1,863 2,185 Shipments in thousand tons (KTons)* CFS 66,631 70,857 72,762 84,787 60,159 68,432 No. of Employees (Emp) CBP 3,559 4,123 4,255 4,351 3,982 4,705 Annual Payroll (AP) CBP 168, , , , , ,566 No. of Establishments (Est) CBP Population (Pop) Census 5,741,041 6,354,887 6,013,136 6,656,950 6,407,039 7,111,855 No. of Registered Vehicles (Veh_All) FHWA 4,587,649 5,000,806 4,940,942 5,725,031 5,190,303 5,197,664 No. of Registered Trucks (Veh_Truck) FHWA 1,857,927 1,897,135 2,208,967 2,498,168 2,723,559 2,648,096 * Value and Tons are the dependent variables in the model. OVERVIEW OF FREIGHT GENERATION MODELS Although there are several sophisticated modeling approaches, the authors chose to consider either linear or logarithmic function in building the desired freight generation models, by following the Keep It Simple Stupid principal. Most freight models work best when they are kept simple. Therefore, simplicity should be a key goal in model design and unnecessary complexity should be avoided. Moreover, simple freight models are more sensitive, between the input and the output data, allowing users to grasp the underlying causality and relationships among variables more easily. In this regard, Chin s 2007 study is used as a template for the planning, design, and building freight models in this study. Regression Model

8 Lim et al Regression is a statistical process for estimating the causal relationships among variables. It includes many techniques for modeling and analyzing a dependent variable base on several independent variables. Regression analysis helps transportation analysts to understand how a typical value of the performance measures changes when one or more of the freight activity variables are varied, while all other independent variables are held fixed. Many regression techniques for carrying out analysis have been developed. Familiar methods such as linear regression and ordinary least squares regression are parametric, in that the regression function is defined in terms of a finite number of unknown parameters that are estimated from the data. Production Model The freight production model is used to estimate freight activity (in terms of million dollars and thousand tons) that could be produced (i.e., made) from each state. Chin and Hwang, and Oliveira-Neto et al., used annual payroll of the given industry as their only input variable [6, 7]. Under this study, a vector X, representing the multivariate set of input variables, instead of a univariate variable, is incorporated in the model design and construction. Mathematically, the linear model can be expressed as: P ijt = α jt + β jt X ijt + ε ijt (1) β jt X ijt = β 1jt x 1ijt + β 2jt x 2ijt + + β njt x nijt (2) Where, P ijt is the freight production of industry j in origin state i in year t; X ijt is the set of input variables (e.g., annual payroll, employment, and etc.) of industry j in origin state i in year t; α jt and β jt are the parameter estimates of linear regression model for industry j in year t. The logarithmic regression model can be obtained simply by taking a log-transformation on both sides, which results in the form of log (P ijt ) = α jt + β jt log (X ijt) + ε ijt (3) P ijt = exp (α jt + ε ijt)x ijt β jt (4) Where, P ijt is the freight production of industry j in origin state i in year t; X ijt is the set of input variables (e.g., annual payroll, employment, and etc.) of industry j in origin state i in year t; α jt and β jt are the parameter estimates of logarithmic regression model for industry j in year t.

9 Lim et al Attraction Model On the other hand, the freight attraction model estimates freight activity (in terms of million dollars and thousand tons) that could be received (or used) by each state. The link functions of attraction models are the same as those applied in the production models. The difference of the attraction model is that the economic input variables need to be adjusted using information from the I-O make and use tables from the BEA. The basic adjustment is to sum up all industries that use a certain commodity (i.e., resources for the industry) to calculate the attraction of the given commodity in a specific region. Specifically, the adjusted input variable, X ijt, can be calculated by a linear combination of a share of industry, q, signified as ω jqt, that consumes the commodities produced by the originated industry j, and the input variables of industry q in state i for year t. These can be mathematically expressed as X ijt = ω jqt X (5) iqt q ω jqt = m jct u (6) cqt c Where, X ijt, is the transformation coefficients and calculated by a linear combination of the share of industry q, signified as ω jqt, that consumes the commodities produced by the originated industry, j, and the input variables of industry q in state i for year t. Make (m jct ) and use (u cqt ) are simply the percentage value of commodity c made and used by the intermediate industry sector, j and q, respectively. More details of these Make and Use tables can be found in Chin and Hwang, and Oliveira- Neto et al. s papers [6, 7]. MODEL SELECTION AND VALIDATION METHOD Index of Information Complexity Information criteria model selection is a modern branch of statistical analysis. It draws its basis from entropy and information theoretic concepts to analyze massive data in identifying a model characterized by information complexity. This methodology calculates expected uncertainty by taking into account both the disciplinary between the estimations and the observations, and the complexity of the model simultaneously. Model complexity involves functional form, the number of independent variables and the interaction between the independent variables. Based on it, each industry could have a different set of variables and link function. One of the well-known information criterion is the Akaike Information Criterion (AIC), but its penalty term 2k (where k is number of parameters), does not correct the bias of maximized loglikelihood, ln L(θ k). Thus, many researchers have suggested alternatives such as Schwarz

10 Lim et al Bayesian Information Criterion (SBC or SBIC). The SBIC penalizes more severely for larger samples, but does not capture the covariance between factors. To overcome these issues, Bozdogan has suggested an unbiased information criterion, ICOMP, using the estimated inverse- Fisher information matrix and accounting for the collinearity between the factors and dependence among the parameter estimates [24]. Where, ICOMP = 2 ln L(θ k) + s ln tr ( I 1 (θ k) ) ln I 1 (θ k) (7) s θ k is the parameter estimates of parameter set k I 1 (θ k) is the inverse of Fisher information matrix s = rank (I 1 (θ k)) (8) Then, the final model is determined as the one with the lowest ICOMP for each industry, j, considering the all-possible combinations as an impending candidate set. ICOMP j = min k S j ICOMP jk S j = {all possible combinations of models} (9) This paper describes a study that employs ICOMP approach to evaluate and select models. For each link function, the variable set that minimizes the ICOMP is selected as a best model. The final model is selected to be the link function with the highest R-square. The R-square is used for determining the link function because the log-likelihood of model is not directly comparable due to the transformation of scale. Modeling and Validation Procedures Generally, the procedure of getting from the model construction to its validation follows these steps: 1) set the input variables for the base year; for attraction model, one would adjust the variables using the BEA s I-O make and use tables; 2) create an impending candidate set of models for all combinations using the input variables; 3) select the best model for each industry based on ICOMP; 4) apply the base year model (e.g., 2002) to estimate the production and attraction of the target year (e.g., 2007); and 5) calculate error measurements between the observed numbers and the estimates of target year The model selection is a process for determining the final model that has the minimum ICOMP among the set of impending candidate models. The impending candidate sets consist of all possible combinations of variables, including an intercept term. For ease of discussion, the model selection and validation procedures are described in more detail using an example in the following section.

11 ... Lim et al An Example on Model Selection The case presented in Table3 is to develop a production model for the mining industry, except oil and gas (NAICS 212) in Since there are seven parameters, including six variables and an intercept term, the number of possible combinations is 127 (=2 7-1) for each link function, total 127 2(link function) 4(production/attraction in values and tons) = 1,016 for each industry; note that a model must have at least one parameter. Sorting the all cases in ascending order by ICOMP, the models with a logarithmic transformation are labeled as 1, 2, to 127 as shown in the leftmost column in Table 3; where model #1 is the best with the lowest ICOMP value. As shown in Table 3, the annual payroll (variable AP ) seems to be essential in the model since it is included in all of the top ten impending candidates. The intercept term is also included in the all of top ten impending candidates. Adding the number of establishments (Est) to the first model (i.e., model #1) increases the R-square only by The equation (10) as listed below represents the final production model for this industry, i.e., model #1 in Table 3. The ratio of the productions from two states is expected to be the ratio of their corresponding annual payroll to the power of 1.07, as shown in equation (11). For instance, if state A s annual payroll is 10% higher than state B, then state A s production will be expected to be about 11 % (= ) higher than the state B. log(p i) = log (AP i ) (10) P B/P A = (AP B /AP A ) 1.07 (11) TABLE 3 Example of Model Selection (2012 NAICS 212 Production Model (Logarithm), in Values) Model # Intercept Emp AP Est Pop Veh_All Veh_Truck R 2 ICOMP Mean Absolute Percentage Error To evaluate the model performance, this study used the Mean Absolute Percentage Error (MAPE), which is one of most commonly used measures in estimating forecast/projection errors. The absolute term in the MAPE prevents the positive and negative errors from cancelling out each other, and the relative error takes an advantage of comparing forecast accuracies for data in different scales. Mathematically, denoting the observed production by P ijt and the estimated production amount by P ijt for an industry j, the MAPE can be calculated with the equation:

12 Lim et al MAPE jt production = 1 N P ijt P ijt P ijt i Under this study, data from a selected base-year (e.g., 2002), by industry, is used to build the model. The model is then applied with data from another year (e.g., 2007) and its performance is evaluated with the MAPE measure. RESULTS Final Model By repeating the process of constructing a list of models similar to those in Table 3 for all industries, the final set of industry-specific freight generation models can be produced. Table 4 illustrates the final freight production models (in dollar values), for all industries in year Of the twenty-four industries in this study, 20 models take the log-transformation form with the other four models select the simple linear regression form. The R-square for all models are at or above 0.9. As shown in Table 4, 17 industries include the intercept term in its final model, while the AP is the most often selected variable (15 of the 24 models) among the considered variables. An additional brief discussion of results, on the production and attraction models, is presented in the Summary of Results section below. TABLE 4 Final Freight Production Model by Industry (2012, in Millions of Dollars) (12) NAICS Model Intercept Emp AP Est Pop Veh_All Veh_Truck R Log Log Log Log Log Log Log Log Log Log Log Log Log Linear Log Log Linear Linear Log Log Log Log Log Linear

13 Lim et al Effects of Model Complexity The final 2012 production model of the mining industry (NAICS 212), in log-form, is used as an example to illustrate the effect of including additional variables in the model. In Figure 1, the horizontal axis shows the additional variable that was added cumulatively from the left to the right. For instance, the term +Pop means a model that includes three variables: AP, Emp, and Pop. Three measurements of ICOMP, R-square, and MAPE are shown in different scales on the vertical axis. As shown in Figure 1, the model with only AP was selected as a best model with the lowest ICOMP among all alternative sets. Although additional variables increase the R-square slightly, the more complex model does not always yield a lower MAPE. This implies that an over-fitting may result in a lack of transferability of the model in forecasting FIGURE 1 Effect of Model Complexity (2012 Production Model, NAICS 212) Summary Tables 5 and 6 present a summary of comparisons on the three approaches discussed in this paper, for production and attraction models, respectively. Namely, the models are: Model 1: the approach that uses a simple linear regression with AP as the variable, Model 2: the log-transformation of both the independent and dependent variables, and, Model 3: considers all possible sets using the six input variables and determines the best set of models by industry using ICOMP. Based on the two evaluation sets (i.e., 2007-flows estimated with 2002-model and 2012-flows estimated with 2007-model), Model 3 (ICOMP models) outperformed Model 1 and Model 2 for 90% of industries in the production estimation and 91% in the attraction estimation. However, Model 3 does not guarantee that all ICOMP models are superior (higher MAPE) to Model 1, and Model 2.

14 Lim et al Regarding the production value (million dollars) for Model 3-estimated 2012 flows, all 24 industry-specific models (i.e., model 3, under 2012 in Table 5) yield the MAPE lower than 40%, while alternative models have less industries with an equivalent MAPE. Even though Model 3 reduced overall MAPE, estimated freight production by weight (tonnage) is inferior to production by value. TABLE 5 Model Comparisons by MAPE (Production Models) Y in Million Dollars Y in Thousand Tons NAICS Model Model Model Model Model Model Model Model Model Model Model Model 1 2 3* 1 2 3* 1 2 3* 1 2 3* ) Model 1: Linear Model with Intercept and Annual Payroll 2) Model 2: Log-Log Model with Intercept and Annual Payroll 3) Model 3*: Industry-Specific Model selected by ICOMP (Proposed Model) Similarly, attraction models are shown in Table 6, most of the Model 3 (in values) estimated 2012 freight flows, except NAICS 324, yield the MAPE lower than 70%, while alternative models have fewer industries with a MAPE greater than 70%. As for production models, freight attraction estimated based on shipment weight (in tons) is inferior to those by shipment values. There are two industries in 2007-estimated freight flow and three in 2012-estimated freight flows that have an attraction MAPE (in tons) greater than 100%. One of the reasons for having such a high MAPE might be attributed to the changes of productivity in the given industry over time [7]. It could also be possible that is due to certain uniqueness in the characteristics (or supply chain) of such industries. Further investigations might be necessary to confirm that. In addition to the index of information complexity,

15 Lim et al alternative methodologies, such as agent-based model, simulation, and other more complex models could be considered in future studies of this kind. TABLE 6 Model Comparisons, MAPE (Attraction models) Y in Million Dollars Y in Thousand Tons NAICS Model Model Model Model Model Model Model Model Model Model Model Model 1 2 3* 1 2 3* 1 2 3* 1 2 3* ) Model 1: Linear Model with Intercept and Annual Payroll 2) Model 2: Log-Log Model with Intercept and Annual Payroll 3) Model 3*: Industry-Specific Model selected by ICOMP (Proposed Model) Figure 2 displays information on average of the MAPEs of production and attraction models for 2007 and 2012 (as presented in Tables 5 and 6). Clearly, Model 3 outperforms (with lower average MAPE) Models 1 and 2 in all industry sectors. According to 2007 production model (in values), the average MAPE of Model 3 is 27%, which is lower than Model 1 average by about 109% and Model 2 average by 3%. For the 2012 freight value estimated by production model, the average MAPE of Model 3 is 22%, which are lower than Model 1 by 83% and Model 2 by 3%. The MAPE of the Model 3 production model in tons are greater than that of in values, around 73% in both 2007 and Similarly, the average MAPE of attraction models generated by Model 3, in freight values, is 40% for estimated 2007 flows and 56% for estimated 2012 flows, which are lower than the average MAPE of Model 1 and Model 2. Like the production models, the MAPE of Model 3 attraction model in tons is greater than that of in values, 65% in 2007 and 71% in A possible reason

16 Lim et al might be that several input variables, such as the annual payroll, which is associated with economic activity, is more sensitive to monetary values than the weights of freight shipments. Production Model Average of Mean Absolute Percentage Error (%) Value Attraction Model Tons Value Tons FIGURE 2 Model Performance Comparisons with Average of MAPE CONCLUSION AND DISCUSSION This study compared the proposed model selection by ICOMP with two alternative approaches, which use only annual payroll and a predefined link function. It found that most of the models (industry-specific) take a log-transformation form and include annual payroll as a significant predictor. The results from this study showed that the model should be different by industry type. The use of ICOMP reduces the MAPE for models, on average, as compared to the naïve model selection approaches. The key contributions of this study are summarized below: Employs the information complexity to choose the best set of industry-specific models Uses data from multiple years to validate models, thus considering temporal effects Reveals that the model selection should be different by industry type Empowers transportation analysts to disaggregate the model s geographic resolution from metropolitan, rest of state, or state level to county level Enables freight transportation practitioners to carry out shorter term (annual provisional) updates and long term (from 2020 through 2045 in 5-year intervals) freight forecasts

17 Lim et al Although this study used relatively simple regression models to identify the effects of the model selection. More sophisticated freight transportation demand models can further improve the model performance by including additional variables or incorporating more complex functional forms, such as synthetic correction [1] and spatial autocorrelation [9]. Furthermore, the authors anticipate that several applications can directly benefit from the freight demand models developed from this research, including: Disaggregation of the model s geographic resolution from metropolitan, rest of state, or state level to county level, Generation of annual provisional freight data for intermediate years between CFS surveys, and Estimation of long-term forecast of national freight movements. While the data availability has always been one of the key concerns in freight demand modeling, the authors have identified the limitations of this study and provided suggestions for future research in the following areas: Currently, the response variables in the models are based on the CFS survey data, which is provided only every five years at the national, state, and metropolitan area levels. Although the study used simple model approaches, the input variables for attraction model still need to be adjusted using data from the BEA s make and use tables. The MAPE for several industries are relatively high, especially those involving tons, thus a further research to improve this area is needed. Temporal and spatial effects need further investigation. Exploring alternative freight modeling approaches, such as supply chain model, agentbased model, and simulation-based model, could be examined along with the proposed model selection. Other information criteria, such as AIC, SBIC, Deviance Information Criterion (DIC), and Focused Information Criterion (FIC), could also be tested and utilized selectively depending on the model approach. As a final note, this study provides production and attraction models in values and tons, but not the trip generation rates. To obtain the trip generation rates, additional analyses, such as estimation of the unit value or the unit ton by transportation mode, will be required.

18 Lim et al. 18 REFERENCES Holguín-Veras, J., et al., Transferability of freight trip generation models. Transportation Research Record: Journal of the Transportation Research Board, 2013(2379): p Ranaiefar, F., et al., Geographic scalability and supply chain elasticity of a structural commodity generation model using public data. Transp. Res. Rec, : p ZHAO, Y.-c. and Y.-w. SUN, The Research of the Relationship Between the Freight Volume Per GDP and the Urban Industrial Structure [J]. Technology & Economy in Areas of Communications, : p Lawson, C., S. Nampoothiri, and O.J. Peters, A freight data architecture application at the local level using commodity flow survey data. Transportation Research E-Circular, 2011(E-C158). 5. Campbell, S., et al. Comparison between industrial classification systems in freight trip generation modeling. in Transportation Research Board 91st Annual Meeting Chin, S.-M. and H.-L. Hwang, National Freight Demand Modeling-Bridging the Gap between Freight Flow Statistics and US Economic Patterns. 2007, Oak Ridge National Laboratory (ORNL). 7. Oliveira-Neto, F., S. Chin, and H.-l. Hwang, Aggregate Freight Generation Modeling: Assessing Temporal Effect of Economic Activity on Freight Volumes with Two-Period Cross-Sectional Data. Transportation Research Record: Journal of the Transportation Research Board, 2012(2285): p Bastida, C. and J. Holguin-Veras, Freight generation models: comparative analysis of regression models and multiple classification analysis. Transportation Research Record: Journal of the Transportation Research Board, 2009(2097): p Novak, D.C., et al., Nationwide freight generation models: A spatial regression approach. Networks and Spatial Economics, (1): p Meersman, H. and E. Van de Voorde, The relationship between economic activity and freight transport. Freight transport modelling. Bingley, Emerald, 2013: p Harris, G.A., et al. Development of a Freight Database for Use in Allocating Freight Traffic to Sub-State Traffic Zones. in Submitted to the 89th Transportation Research Board Annual Meeting, Washington, DC Citeseer. 12. Garrido, R.A. and H.S. Mahmassani, Forecasting freight transportation demand with the space time multinomial probit model. Transportation Research Part B: Methodological, (5): p FHWA, U.S.D.o.T., Quick Response Freight Manual II. 2007(FHWA-HOP ).

19 Lim et al Doustmohammadi, E., et al., Comparison of Freight Demand Forecasting Models. International Journal of Traffic and Transportation Engineering, (1): p De Jong, G., et al., Recent developments in national and international freight transport models within Europe. Transportation, (2): p Chow, J.Y., C.H. Yang, and A.C. Regan, State-of-the art of freight forecast modeling: lessons learned and the road ahead. Transportation, (6): p De Jong, G., H. Gunn, and W. Walker, National and international freight transport models: an overview and ideas for future development. Transport Reviews, (1): p Pendyala, R., V. Shankar, and R. McCullough, Freight travel demand modeling: synthesis of approaches and development of a framework. Transportation Research Record: Journal of the Transportation Research Board, 2000(1725): p Commodity Flow Survey. U.S. Census Bureau, accessed on June County Business Pattern. U.S. Census Bureau, accessed on June Input-Output Accounts Data. Bureau of Economic Analysis, accessed on June Population Estimates. U.S. Census Bureau, accessed on June Highway Statistics Series. Federal Highway Administration, accessed on June Bozdogan, H., Akaike's information criterion and recent developments in information complexity. Journal of mathematical psychology, (1): p