Process and Data Needs for Local Calibration of Performance Models in the Pavement-ME

Transcription

1 Process and Data Needs for Local Calibration of Performance Models in the Pavement-ME By Syed Waqar Haider, Ph.D., P.E. (Corresponding Author) Assistant Professor syedwaqa@egr.msu.edu Wouter C. Brink Graduate Research Assistant brinkwou@egr.msu.edu Neeraj Buch, Ph.D., FACI Professor and Chairperson buch@egr.msu.edu Karim Chatti, Ph.D. Professor and Acting Assoc. Dean for Research chatti@egr.msu.edu Department of Civil and Environmental Engineering Michigan State University, Engineering Building 428 S. Shaw Lane, Room 346, East Lansing, MI Phone: ; Fax: Prepared for presentation at the 94 th Annual Transportation Research Board Meeting and for publication in the Journal of Transportation Research Record November 1, 214 No. of figures = 11 2 = 2,7 word equivalents No. of tables = 6 2 = 1, word equivalents Text = 3,2 words Total = 7,4 words

2 Haider, Brink, Buch and Chatti ABSTRACT The local calibration of the performance models in the Pavement-ME is a challenging task, especially if the data are limited. This paper summarizes the local calibration process for flexible and rigid pavements for the state of Michigan. However, other agencies can learn from the steps needed to accomplish a more streamlined local calibration. The local calibration process includes several sequential steps. Statistical sampling concepts were used to determine the adequate number of pavement sections for a robust calibration. The next step is to identify the candidate pavement sections in the PMS database based on the pavement type, age, geographical location, and number of performance data collection cycles. Subsequently, the final set of pavement sections is selected based on the distress magnitude over time. It is important to categorize the selected pavement sections based on the measured distresses (i.e., poor, normal and good performing pavements) because the local calibrated models are typically used to predict normal pavement performance at the design stage. For the selected pavement sections, the as-constructed input variables are collected from the construction records. However, when such input information is unavailable, the best estimates are used to represent MDOT pavement design and construction practices. Lastly, the typical steps for local calibration (verification, calibration and validation) by using various resampling techniques are demonstrated for the rutting (flexible) and transverse cracking (rigid) models. The different techniques are compared by using the standard error of the estimate (SEE). The SEE of a technique shows how much of the variance is explained by the model. The main advantage of using repeated resampling is to quantify the variability associated with the model predictions and parameters. The quantification of the variability will also help in determining more robust design reliability in the Pavement-ME. 2

3 Haider, Brink, Buch and Chatti INTRODUCTION The local calibration of the performance models in the new mechanistic-empirical pavement design guide is a challenging task, especially due to the lack of needed data. Pavement sections should be selected to represent the local conditions for a state. Generally, the local pavement management systems (PMS) database is utilized to extract historical performance data for the sections selected for local calibration. For the selected set of pavement sections, the data requirements include (a) a wide range of inputs related to traffic, climate, design and material characterization, (b) a reasonable extent and occurrence of observed performance data over time. The NCHRP 1-4B guide documented the recommended practices for local calibration of the Pavement-ME performance models (1). The guide outlines the significance of the local calibration process and the general approach for local calibration. The calibration process is used to (a) confirm that the models can predict pavement distress and smoothness without bias, and (b) determine the standard error associated with the transfer functions. The standard error estimates the scatter of the data around the line of equality between predicted and measured distress. It should be noted that the local calibration process only applies to the transfer functions or statistical models in the Pavement-ME. Further, the mathematical models within the Pavement-ME are assumed to be accurate and depict a correct simulation of the real-world conditions. The calibration-validation process depends on the number of pavement sections selected. The in-service pavements data are used to establish calibration coefficients such that the overall standard error of the estimate between the predicted and measured performance is minimized. The model validation procedure is used to demonstrate that the calibrated model can produce accurate predictions of pavement distress for sections other than the ones used for calibration. The reasonableness of the validation process depends on the bias in the predicted values and the standard error of estimates (SEE). This paper highlights the process for local calibration for flexible and rigid pavements in the state of Michigan. The main objectives are to (a) outline the overall process of the local calibration; (b) highlight various challenges related to the data needs, and (c) documents ways to address some of these data challenges. The Pavement- ME performance models and various approaches used to calibrate each are briefly presented below. BACKGROUND The local calibration of the performance prediction models are performed by changing the calibration coefficients in each model. These coefficients are adjusted individually or estimated through minimizing the error between the predicted and measured distress. Table 1 summarizes the flexible and rigid pavement performance prediction models, their corresponding transfer functions, and model calibration coefficients. The detailed performance prediction models can be found elsewhere (2). For closed-form model calibration, two methods are generally used: (a) an analytical process for linear models, and (b) a numerical optimization technique for non-linear models. In both methods, the model constants are determined by minimizing the error between measured and predicted distress values. Further, two types of models are used in the Pavement- ME for performance prediction: (a) structural response models, and (b) transfer functions. The former are derived from analytical solutions based on engineering mechanics (e.g., linear elastic solution to determine stress, strain and deformation for flexible pavements) while the latter are empirical in nature and relate the pavement response or damage to distresses over time. The local calibration process deals with the transfer function for predicting distresses. Among empirical 3

4 Rigid pavements Flexible pavements Haider, Brink, Buch and Chatti transfer functions, two different calibration approaches may be required depending upon the nature of the distress being predicted: (I) model that directly calculates the magnitude of surface distress, and (II) model that calculates the incremental damage index rather than actual distress magnitude. In the first approach, the pavement response parameter is used to compute the incremental distress in a direct relationship. The approaches for each model are summarized in Table 1. Approach I indicates that the local calibration is performed without running the software each time. Alternatively, approach II requires software execution each time the coefficients are adjusted. Table 1 Performance models and transfer functions calibration approach (2) Pavement type Performance measures Fatigue cracking bottom up Fatigue cracking top down Approach I II Performance models and transfer functions FC Bottom 1 C4 * * C C o 1 C Bottom 6 1 C Lg DI e C1C2 top 1 1 e FC Top Log DI 1r 2r 2r k3r 3r HMA ( ) ( ) 1 ( ) 1 k k p HMA p HMA hhma rkzr HMA n T Rutting n Base k h o e p( soil) s 1 s1v soil r Thermal ktt Log EHMA m cracking A 1 IRI IRI C RD C FC C TC C SF IRI Transverse cracking Transverse joint faulting o 1 2 Total 3 4 CRK 1 1 C DI BU / T D C 4 F TCRACK CRK Bottomup CRKTop down CRK Bottomup CRKTop down 1% Fault m Fault m i i1 2 i C3 4 ( i1 ti 1) DEi Fault FAULTMAX Faul FAULTMAX FAULTMAX C DE LogC i 7 m j1 j EROD C6 C6 EROD P2 WetDays C12 curli ng (1 C. ) ( ) ps FAULTMAX Log Log.2 C C C FR C C C FR IRI IRI IRI C CRK C SPALL C TFAULT C SF I DATA NEEDS FOR LOCAL CALIBRATION Local calibration of the performance prediction models requires a vigilant selection of in-service pavement sections that represent local pavement design, construction practices, and performance. These pavement sections should represent all current pavement types and rehabilitation types 4

5 Haider, Brink, Buch and Chatti which are constructed by the State Highway Agencies (SHAs). The project selection criteria adopted in this study to identify and select pavement sections were established prior to the local calibration effort and are briefly discussed below (3). The process for identifying and selecting pavement sections consists of the following steps: 1. Determine the minimum number of pavement sections based on the statistical requirements 2. Identify all available in-service pavement projects constructed after Extract all pavement distresses from the customized database for all identified projects 4. Evaluate the measured performance for all the identified projects. Establish a refined list of the potential projects which exhibit multiple distresses with adequate magnitude. Projects with multiple distresses simplified the input data collection process. Determine the Minimum Number of Pavement Sections The first step for project identification and selection consists of determining the adequate number of pavement sections for local calibration based on statistical needs. The NCHRP 1-4B (4) suggests a method to determine the minimum number of sections for each performance measure. The minimum number of sections is calculated using Equation (1) and the results are summarized in Table 2 for each performance measure Z n where: n = Minimum number of pavement sections = Performance threshold e t = Tolerable bias Z global model SEE 2 et /2 Table 2 Minimum number of sections for local calibration in Michigan Performance Model Global calibrated SEE Z 9 2 MDOT identified threshold N (required number of sections) Flexible Pavements Alligator cracking (%) Thermal cracking Rutting (in) IRI (in/mile) Rigid Pavements Transverse cracking (%) Joint faulting (in) IRI (in/mile) n= minimum no. of samples required for 9% confidence level (1)

6 Haider, Brink, Buch and Chatti Identification and Selection of Pavement Sections In collaboration with MDOT, the pavement sections were identified to represent all current construction practices (i.e., typical new and rehabilitation pavement types). The reconstruct and rehabilitation types for flexible pavement include: new reconstruct; crush and shape; HMA overlay over existing HMA, rubblized PCC, and intact PCC. For rigid pavements, new reconstruct JPCP and unbonded concrete overlay were considered. The pavement sections to be included in the study were selected to represent the following criteria: Site factors: The site factors addressed the various regions in the state, climatic zones and subgrade soil types. Traffic: Three traffic categories were selected; less than 1 AADTT, 1 to 3 AADTT; and more than 3 AADTT. The three levels were selected based on pavement class, trunk routes, US routes and Interstate routes. Thicknesses: The range of constructed HMA, PCC and overlay thicknesses. Open to traffic date: The information is needed to determine the performance period. As built cross-section: Includes details of the existing structure and the overlay. Pre-overlay repairs performed on the existing pavement (such as partial and/or full depth repairs, dowel bar retrofit). Material properties of both the existing and the new structure. Initially, a total of 223 pavement projects (flexible and rigid) were identified based on the pavement age (constructed after 1992). Later, the measured performance data were extracted from the PMS and Sensor databases for all the 223 pavement projects. The projects that had very low levels of measured distress or no available data were excluded from the final list. As an example, Figure 1 shows the geographical distribution of the initial and revised number of identified projects based on measured performance evaluation for freeway projects. The extracted performance data from the PMS were analyzed to evaluate the following temporal attributes: Increasing trend (i.e., positive progression of distress over time) Decreasing trend (i.e., negative progression of distresses over time which may happen because of maintenance history, or measurement errors) Flat line (i.e., no progression over time) Not enough data (i.e., inadequate measured performance over time) No trend (i.e., high variability among different measurement cycles) If a project showed an increasing trend for any of the performance measures it was included in the final list. Projects that had insufficient performance data, showed no time series trend, or showed a consistent flat line (no growth over time) with minimal magnitude were also eliminated. The total number of increasing trends were determined from the final project list and compared to the minimum number of sections needed for each performance model. The final revised list of selected projects was sent to MDOT to acquire the necessary construction, design, and material related inputs. Table 3 summarizes the number of pavement sections which meet the above mentioned performance criteria for different distresses. 6

7 Haider, Brink, Buch and Chatti Table 3 Final number of pavement sections for the local calibration Performance measure Acceptable Total available Flexible pavements Alligator cracking Longitudinal cracking 128 (37) 129 (4) Rutting 129 (33) 129 (4) IRI 127 (4) 129 (4) Rigid pavements Transverse cracking 18 (13) 18 (13) Joint faulting 33 (16) 33 (16) IRI 29 (1) 29 (1) Note: The values in parenthesis represent number of rehabilitated pavement sections (a) Initial HMA projects (b) Revised HMA projects 199 (c) Initial JPCP projects (d) Revised JPCP projects Figure 1 Geographical location of identified new HMA and JPCP projects 7

8 Frequency Frequency Frequency Title Frequency Title Haider, Brink, Buch and Chatti Performance Data for Local Calibration A thorough investigation was performed to determine the extent of distress for all the selected pavement sections for the local calibration. The calibration process involves comparisons of predicted and measured performance for each of the selected sections. For a robust local calibration, the distress magnitudes should cover a reasonable range (i.e., above and below threshold limits for each distress type). Therefore, the distress magnitudes for all projects were summarized to determine their ranges. In the case of insufficient number of pavement sections were available for any pavement type, the LTPP pavement sections located in Michigan and the surrounding States were considered to further increase the sample size of the calibration database. Figure 2 shows an example of observed surface rutting distribution for flexible pavements. The maximum rut depth ranged from.1 to. inches. It can be seen that only a few pavement sections reached the rutting threshold of. inch and corresponding age of the sections ranged from 4 to 18 years. Similarly, Figure 3 presents distributions of transverse cracking and age for the selected JPCP sections. The transverse cracking for all projects ranged from to 8% slabs cracked while nine sections exceeded the distress threshold of 1% slabs cracked. The age at maximum transverse cracking ranged from 4 to 16 years Rutting (in.) (a) Distress magnitude (b) Age distribution Figure 2 Selected HMA non-freeway project rutting data Percent slabs cracked (a) Distress magnitude 1% 8% 6% 4% 2% % 1% 8% 6% 4% 2% % (b) Age distribution Figure 3 Selected JPCP project transverse cracking data Further, the measured performance for each of the selected pavement section was compared with the expected pavement performance criteria to identify if any section exhibit unexpected performance over time. The FHWA expected pavement performance criteria for each performance measures was adopted in this study (, 6). Any pavement section that exceeds the threshold performance limit, especially at an early age (premature), was a cause for concern for the local calibration because of the significant difference expected between predicted and Age at maximum distress (years) Age at max distress (years) 1% 8% 6% 4% 2% % 1% 8% 6% 4% 2% % 8

9 Rutting (inch) Percent slabs cracked Haider, Brink, Buch and Chatti measured performance. Several projects exceeded the expected normal performance threshold based on the performance criteria. Figure 4 shows examples of the measured rutting and transverse cracking for all flexible and rigid pavements. The rutting performance followed the conventional trends and fewer projects exceeded the threshold limits while several pavements exceeded the threshold limits for transverse cracking in the rigid pavement population. The causes for early age distress are important in determining whether the projects should be included in the calibration or not. Such decisions are made based on if there were any construction, or material related issues encountered at the time of construction..6. Good Poor Pavement age (years) Pavement age (years) (a) Rutting (b) Transverse cracking Figure 4 Comparison of measured performance with expected performance Subsequently, a list of projects which exceed the expected pavement performance were identified and sent to MDOT for further review. After reviewing the identified sections, it was concluded that not enough information was available to determine why these sections were performing unexpectedly. As a result, some of the rigid pavement sections were not excluded from the local calibration where sufficient evidence was available to explain the abnormal performance. For example, the built-in curling was considered to explain higher cracking observed on several rigid pavements. It should be noted that such measures were only adopted due to the limited availability of rigid pavement sections, otherwise abnormal performing pavements should be excluded from the calibration dataset. As-Constructed Inputs for the Selected Pavement Sections Subsequent to pavement project selection and performance evaluations, the parallel activity consisted of collecting the input data for all the selected pavement sections in order to characterize the in-service pavement sections in the Pavement-ME. The cross-section, traffic and material input data are needed to characterize the as-constructed pavements. The accuracy of the data directly impacts the performance prediction. The Pavement-ME uses a very large number of inputs to characterize a pavement. Furthermore, the hierarchical structure of the Pavement-ME provides three levels of inputs for many of the important input parameters. The input data collection efforts used to characterize a pavement in the Pavement-ME can be very time consuming. In order to reduce input data collection time, the most sensitive inputs which affect 9

10 Construction materials Cross-section (new and existing) Traffic Haider, Brink, Buch and Chatti the pavement performance predictions were given priority. The sensitive inputs which affect the performance prediction for new and rehabilitation design of rigid and flexible pavements were determined and can be found elsewhere (7-11). Based on the results of these studies, focusing on the sensitive inputs significantly reduced the amount of time spent to collect input data. Additionally, the best available input level was used for the selected pavement sections. The general process for collecting the as-constructed input data, the details regarding the source of the data, issues and observations related to the data, and the final selection of the input values can be found elsewhere (3). Table 4 summarizes the inputs and corresponding levels for the available data used in this study. Table 4 Summary of input levels and data source Input Input level Input source AADTT 1 MDOT Historical Traffic counts TTC 2 Cluster analysis ALS Tandem 2 Cluster analysis HDF 2 Cluster analysis MDF 3 MDOT traffic characterization study AGPV 3 MDOT traffic characterization study ALS single, tridem, quad 3 MDOT traffic characterization study HMA thickness 1 Project specific HMA thicknesses based on design drawings PCC thickness 1 Project specific PCC thicknesses based on design drawings Base thickness 1 Project specific base thicknesses based on design drawings Subbase thickness 1 Project specific subbase thicknesses based on design drawings Binder type 3 Project specific binder and mixture gradation data obtained from data collection HMA HMA mixture aggregate Project specific binder and mixture gradation data 3 gradation obtained from data collection Pseudo Level 1- MDOT HMA mixture characterization Binder type 1 study HMA mixture aggregate Pseudo Level 1- MDOT HMA mixture characterization 1 gradation study PCC Strength (f' c, MOR) 1 Psuedo Level 1 - project specific testing values CTE 2 MDOT CTE report recommendations Base/sub base MR 2 Recommendations from MDOT unbound material study Subgrade MR 2 Soil specific MR values - MDOT subgrade soil study Soil type 1 Location based soil type - MDOT subgrade soil study Climate 1 Closest available climate station Note: 1. Level 1 is project specific data, pseudo level 1 means that the inputs are not project specific but the material properties (lab measured) corresponds to similar materials used in the project 2. Level 2 inputs are based on regional averages in Michigan 3. Level 3 inputs are based on statewide averages in Michigan 1

11 Haider, Brink, Buch and Chatti LOCAL CALIBRATION PROCESS The calibration process includes different data subsets (options) and statistical techniques. The different data subsets (options) are combinations of reconstruct, rehabilitation and LTPP pavement sections. The main objective for considering different options is to verify if different calibration coefficient are required for different datasets. The options with different dataset combinations considered in the study are as follows: Option 1: MDOT reconstruct sections only Option 2: MDOT reconstruct and rehabilitation sections Option 3: MDOT reconstruct, rehabilitation, and LTPP sections Option 4: MDOT rehabilitation sections only The performance prediction models are locally calibrated by minimizing the sum of squared error between the measured and predicted distresses by using the following statistical techniques: a. No sampling (include all data) b. Traditional split sampling c. Repeated split sampling d. Bootstrapping e. Bootstrapping validation The different sampling techniques (a to d) are used to determine the best estimates of the local calibration coefficients and the associated standard errors. The use of these techniques is considered because of data limitations, especially due to limited sample size for rigid pavements, and to utilize a more robust way of quantifying model standard error and bias. The split sample bootstrapping technique (e) was used to validate the local calibrated performance prediction models in the Pavement-ME. The following performance models in the Pavement-ME were locally calibrated for Michigan conditions. Flexible pavements o Fatigue cracking o Rutting o Transverse (thermal) cracking o IRI Rigid pavements o Transverse cracking o Faulting o IRI The Pavement-ME software (version 2..19) was executed using the as-constructed inputs for all the selected pavement sections and the predicted performance was extracted from the output files. The measured and predicted distresses over time were compared. These comparisons indicate the adequacy of global model predictions for the measured distresses on the pavement sections. Generally, the predicted and measured performance should have a one-to-one (4 degree line of equality) relationship in the case of a good match. Otherwise, biased and/or 11

12 Predicted total rutting (inch) Predicted transverse cracking (% slabs cracked) Haider, Brink, Buch and Chatti prediction error may exist based on the spread of data around the line of equality. As a consequence, local calibration of the model is needed to reduce the bias and standard error between the predicted and measured performance. The above mentioned steps can be accomplished by performing the following process (1, 3). 1. Compare the globally calibrated model predictions with measured performance. 2. Perform hypothesis tests between the measured and predicted performance. If any of the null hypotheses are rejected, follow step 3 otherwise no local calibration is needed. 3. Adjust local calibration coefficients to minimize the standard error between predicted and measured performance, and compare the measured and predicted performance. 4. Perform hypothesis testing again based on the locally calibrated coefficients and determine if the model accuracy has improved. If not, identify the possible sources of bias such as outliers in the measured performance data or improve the accuracy of input data and continue local calibration process until the standard error of the estimate is lower than the globally calibrated model.. Accept or reject the local calibration coefficients based on the results from step 4. A few examples of the local calibration results (for Option 2 only) demonstrating the process are summarized below for flexible and rigid pavement performance prediction models. Global Model Verification The globally calibrated rutting (flexible) and transverse cracking (rigid) models were verified by comparing the predicted and measured performance. The model adequacy was tested by comparing the standard error of the estimate (SEE) and bias of the global model and by performing the hypothesis tests mentioned above. The first hypothesis test determines if there was a statistically significant difference between the predicted and measured performance. The second and third hypothesis tests indicates if the intercept and slope of the linear line between measured and predicted performance is similar to zero and one, respectively. A zero intercept and slope of one indicate that no bias exists between the predicted and measured performance (12-14). Figure shows the comparison between the measured and predicted performance for the global rutting and cracking models. Based on the results, it was concluded that the global models significantly over-predict surface rutting and under-predicts measured cracking SEE 14.3 % slabs cracked Bias.83 % slabs cracked C4 1 C SEE.343 in Bias.322 in r1 1 s1 1 1 sg Measured total rutting (inch) (a) Surface rutting Measured transverse cracking (% slabs cracked) (b) Transverse cracking Figure Comparison between measured and predicted transverse cracking (global model) 12

13 Haider, Brink, Buch and Chatti Local Model Recalibration The different sampling techniques mentioned above were used to determine the best estimates of the local calibration coefficients and the associated standard errors. First, the entire dataset including all the selected pavement sections were used to calibrate performance prediction models. Since all of the pavement sections were included in the calibration effort, no validation of the locally calibrated model was performed. Second, a traditional split sampling technique was utilized. In this method, 7% of the pavement sections were randomly selected for local calibration, and the remaining 3% were utilized for validation. Split sampling can indicate how well the calibrated model can predict pavement distress for pavement sections that are not included in the calibration dataset. Generally, SEE and bias from the validation should be similar to those of locally calibrated model. However, the split sampling technique might not give reasonable results when using limited sample sizes. In order to address the concerns of limited sample size, the split sampling technique was used repeatedly (1 times) to obtain distributions of the calibration and validation parameters (i.e., SEE, bias, calibration coefficients). Based on these distributions, a mean, median, and confidence intervals for each parameter was estimated. The confidence interval determined through repeated sampling provides an indication of the calibration parameters variability. The final resampling technique considered was bootstrapping. For a dataset of sample size N, B mb f b samples of size N was randomly selected with replacement. The model parameters were estimated for B number of bootstraps. Similar to the split sampling approach, bootstrap samples were drawn from the entire dataset. The model was calibrated for each bootstrapped sample dataset and the SEE, bias, and calibration coefficient parameters are estimated. The process was repeated for B number of bootstraps to obtain distributions for each parameter. Figure 6 shows the flow diagram for calibration process using both bootstrapping and repeated split sampling for the performance models. The rutting and tranverse cracking models were calibrated using Option 2 dataset as a demonstration in this paper. The emphasis of this demonstration is to outline the differences between various statistical techniques and how those affect the calibration coefficients and associated variability when using limited number of pavement section for local calibration. Entire dataset (No sampling) In this procedure, the rutting and transverse cracking models were recalibrated using all of the available MDOT pavement sections. Figure 7 shows the comparison between the measured and predicted rutting and transverse cracking for the locally calibrated models. The SEE, bias, and model coefficients are also shown in the figure. Based on the results, SEE reduced from 14.3 to 8.43 percent slabs cracked, and the bias reduced from.83 to.37 percent slabs cracked. The C 4 and C coefficients were adjusted from 1 and to.24 and -1.67, respectively. Split sampling and repeated split sampling The split sampling technique was utilized to calibrate both models. Seventy (7) percent of the pavement sections were used for calibration, and the remaining thirty (3) percent were used for model validation. The comparison between the measured and predicted transverse cracking is summarized in Figure 8. The results show that the calibrated model SEE reduced from to 6.74 percent slabs cracked and the model bias reduced from -4.6 to.6 percent slabs cracked when compared with the global model. The C 4 and C coefficients were adjusted to.19 and as a result of local calibration. The model validation results show that the SEE and bias are 13

14 Predicted total rutting (inch) Predicted transverse cracking (% slabs cracked) Haider, Brink, Buch and Chatti higher than the calibrated model. Generally, the calibration and validation SEE, and bias should be similar in magnitude when calibrated models are developed based a larger sample size. Original Dataset N = number of pavement projects for calibration Sample from original dataset Pavement sections 1, 2, 3, 4,, 6, 7, 8 Pavement sections 1, 2, 3, 4,, 6, 7, 8 C4 Bootstrap sampling with replication Split sample Bootstrapped sections 1, 2, 4,, 3, 8, 8,3...n Calibration set 7% of pavement sections Validation set 3% of pavement sections C Local Calibration Calibrate model using bootstrapped samples Repeat process and record results to create distributions Determine mean, and confidence intervals for each distribution Select final coefficients SEE Determine calibration coefficients Calculate model parameters (SEE, Bias, R 2 ) (hypothesis testing) Figure 6 Repeated sampling calibration procedure Bias SEE 8.43 % slabs cracked Bias.37 % slabs cracked C4.24 C SEE.87 in Bias.13 in r1.98 s sg Measured total rutting (inch) (a) Surface rutting (b) Transverse cracking Figure 7 Local calibration results for the no sampling Measured transverse cracking (% slabs cracked) 14

15 Predicted total rutting (inch) Predicted transverse cracking (% slabs cracked) Predicted total rutting (inch) Predicted transverse cracking (% slabs cracked) Predicted total rutting (inch) Predicted transverse cracking (% slabs cracked) Haider, Brink, Buch and Chatti SEE % slabs cracked Bias 4.6 % slabs cracked C4 1 C SEE.33 in Bias.331 in r1 1 s1 1 1 sg Measured total rutting (inch) (a) Global model total rutting Measured transverse cracking (% slabs cracked) (d) Global model transverse cracking SEE 6.74 % slabs cracked Bias.6 % slabs cracked C4.19 C SEE.8 in Bias.13 in r1.92 s sg Measured total rutting (inch) (b) Local model total rutting Measured transverse cracking (% slabs cracked) (e) Local model transverse cracking SEE % slabs cracked Bias.84 % slabs cracked C4.19 C SEE.91 in Bias.28 in r1.92 s sg Measured total rutting (inch) Measured transverse cracking (% slabs cracked) (c) Validation for total rutting (f) Validation for transverse cracking Figure 8 Split sampling local calibration results 4 2 1

16 Frequency Percentage Frequency Percentage Frequency Percentage Frequency Percentage Haider, Brink, Buch and Chatti The split sampling technique only considers a random selection of 7 percent of the pavement sections. However, if multiple split samples are taken, the SEE, bias, and the model coefficients will vary for each realization. Therefore, the results of a split sample may not indicate an accurate representation of all the sections on average, especially when the sample size is limited. In order to determine a better estimate of the calibration coefficients, SEE and bias, the split sampling technique was performed 1 times and named repeated split sampling. The results of the local calibration using repeated split sampling are shown in Figure 9. The frequency distributions for SEE, bias, C 4 and C indicate the variability of each parameter due to repeated split sampling. Average SEE and bias, and 9% confidence intervals based on the results are summarized in Table for both calibration and validation datasets. The validation results showed a slightly higher SEE and bias than the calibration dataset; however, these values are much lower than for the case of a single split sample, especially for bias. Table Repeated split sampling results for calibration and validation sets (average values) Parameter Calibration Lower CI Upper CI Validation Lower Upper CI CI SEE Bias C C (a) SEE (b) Bias (c) C (c) C Figure 9 Repeated split sampling frequency distributions calibration set (d) 16

17 Frequency Percentage Frequency Percentage Frequency Percentage Frequency Percentage Haider, Brink, Buch and Chatti Bootstrapping Bootstrapping is the final resampling technique considered to recalibrate both models. The difference between split sampling and bootstrapping is that the latter method does not split the original dataset. The bootstrap samples are selected randomly with replacement from the total number of the selected pavement sections. In this method, 1 bootstrap samples were used to recalibrate the cracking model. Figure 1 illustrates the parameter distributions for the 1 bootstrap calibrations. The average values and the 9% confidence intervals for SEE, bias, C 4 and C are summarized in Table 6. These results show that the SEE is slightly lower while bias is slightly higher than repeated split sampling. Table 6 Bootstrap sampling calibration results summary (1 bootstraps) Parameter Average Value Lower CI Upper CI SEE Bias C C (a) SEE (b) Bias (c) C4 (d) (c) C Figure 1 Bootstrap sampling calibration results (1 bootstraps) 17

18 Haider, Brink, Buch and Chatti RELIABILITY OF THE PERFORMANCE MODELS The reliability of the performance models assumes that the expected percentage of distress is approximately normally distributed. The likely variation of distress around the expected level estimate can be defined by the mean predicted distress and a standard deviation. The standard deviation is a function of the error associated with the predicted distress and the data used to calibrate the model. The procedure to estimate the parameters of the error distribution consists of the following steps: 1. Group all the data by the level of predicted distress. This can be accomplished by identifying the distribution bins based on the magnitude of the predicted distress 2. Group the corresponding measured distress data in the same distribution bins 3. Compute descriptive statistics for each group of data i.e. mean and standard deviations of predicted and measured distress 4. Determine relationship between the standard error of the measured and mean predicted distress. For example, the following equation shows the relationship between measured standard deviation and the mean predicted transverse cracking s. CRK 6467 (2) e( CRK ). Adjust the mean cracking for the desired reliability level by using the following relationship: where; Ĉ = Predicted cracking at reliability C = Mean predicted cracking S e = Standard deviation of cracking Z = Standard normal deviate 2 Cˆ C Se Z (3) Figure 11 shows an example of rutting and cracking performance prediction for a flexible and rigid pavement section each using bootstrap local calibration coefficients and its corresponding reliability relationships. The results show that the surface rutting predictions by local models are much lower as compared to the global models. The locally calibrated models show better predictions when compared with the measured performance on this particular pavement section. 2 18

19 Predicted rutting (inch) Predicted transverse cracking (% slabs cracked) Haider, Brink, Buch and Chatti 1.8 Global % Global 9% Local % Local 9% Measured rutting 1 8 Global % Global 9% Local % Local 9% Measured cracking Age (years) Age (years) (a) Rutting (a) Transverse cracking Figure 11 Impact of local calibration on the design reliability SUMMARY OF FINDINGS The paper summarizes the local calibration process of flexible and rigid pavements for the state of Michigan. However, other agencies can learn from the overall steps needed to accomplish more streamlined local calibration. The local calibration process includes several sequential steps as described in the paper. The study also demonstrated the use of various resampling techniques to recalibrate the Pavement-ME performance models. The importance of the resampling procedure is highlighted, especially when the dataset for calibration contains limited sample size. The following is the summary of the local calibration process: The local calibration of the performance models in the Pavement-ME is a challenging task, especially if the data is limited. The pavement section selections process should consider pavement type, age, geographical location, and number of performance data collection cycles. The selected set of pavement section should reflect the following data requirements: a. Adequate number of sections for each performance model, b. Wide range of inputs related to traffic, climate, design and material characterization, c. Reasonable extent and occurrence of measured performance data over time. Before performing the local calibration, all the selected pavement sections should pass the test of reasonable performance, especially at early ages. If abnormal performance is observed in any of the sections, those can either be eliminated from the calibration dataset or can be included when the abnormal behavior can be explained through the input variables. However, it is still important to meet the requirement for the minimum number of sections. The local calibrations for all performance prediction models for flexible and rigid pavement can be performed for multiple datasets (reconstruct, rehabilitation and a combination of both) and by using robust statistical techniques (e.g. repeated split sampling and bootstrapping). 19

20 Haider, Brink, Buch and Chatti The main advantage of using repeated resampling is to quantify the variability (i.e., CI) associated with the model predictions and parameters. In addition, for the limited data set these techniques will help in reducing the SEE and bias for the calibrated model. The quantification of the variability will also help in determining more robust design reliability in the Pavement-ME ACKNOWLEDGEMENTS The authors would like to acknowledge the Michigan Department of Transportation (MDOT) for funding the study. REFERENCES 1. Quintus, H. L. V., M. I. Darter, and J. Mallela, "Local Calibration Guidance for the Recommended Guide for Mechanistic-Empirical Design of New and Rehabilitated Pavement Structures," AASHTO, "Mechanistic-Empirical Pavement Design Guide: A Manual of Practice: Interim Edition," American Association of State Highway and Transportation Officials Haider, S. W., N. Buch, W. Brink, K. Chatti, and G. Y. Baladi, "Preparation for Implementation of the Mechanistic-Empirical Pavement Design Guide in Michigan - Part 3: Local Calibration and Validation of the Pavement-ME Performance Models," Michigan Department of Transportation Final Report RC-19, NCHRP Project 1-4B, "Local Calibration Guidance for the Recommended Guide for Mechanistic-Empirical Pavement Design of New and Rehabilitated Pavement Structures," Final NCHRP Report 29.. Rauhut, J. B., A. Eltahan, and A. L. Simpson, "Common Characteristics of Good and Poorly Performing AC Pavements," Federal Highway Administration, FHWA-RD , December Khazanovich, L., D. M, B. R, and M. T, "Common Characteristics of Good and Poorly Performing PCC Pavements," Federal Highway Administration, FHWA-RD , Buch, N., K. Chatti, S. W. Haider, and A. Manik, "Evaluation of the 1-37A Design Process for New and Rehabilitated JPCP and HMA Pavements, Final Report," Michigan Department of Transportation, Construction and Technology Division, P.O. Box 349, Lansing, MI 4899, Lansing, Research Report RC-116, Kutay, E. and A. Jamrah, "Preparation for Implementation of the Mechanistic-Empirical Pavement Design Guide in Michigan: Part 1 - HMA Mixture Characterization," Michigan Department of Transportation, 42 West Ottawa, P.O. Box 3 Lansing, MI 4899, Lansing, Research Report ORBP OR1-22, Schwartz, C. W., R. Li, S. Kim, H. Ceylan, and R. Gopalakrishnan, "Sensitivity Evaluation of MEPDG Performance Prediction," NCHRP 1-47 Final Report, Harsini, I., W. Brink, S. W. Haider, K. Chatti, N. Buch, G. Baladi, and E. Kutay, "Sensitivity of Input Variables for Flexible Pavement Rehabilitation Strategies in the M- E PDG," presented at ASCE T&DI, Congress Airfield and Highway Pavements, Los Angeles, California, 213, pp

21 Haider, Brink, Buch and Chatti Brink, W., I. Harsini, S. W. Haider, K. Chatti, N. Buch, G. Baladi, and E. Kutay, "Sensitivity of Input Variables for Rigid Pavement Rehabilitation Strategies in the MEPDG," presented at ASCE T&DI, Congress Airfield and Highway Pavements, Los Angeles, California, 213, pp Mallela, J., L. Titus-Glover, H. V. Quintus, M. I. Darter, M. Stanley, and C. Rao, "Implementing the AASHTO Mechanistic-Empirical Pavement Design Guide in Missouri Volume II: MEPDG Model Validation and Calibration," Missouri Department of Transportation Glover, L. T. and J. Mallela, "Guidelines for Implementing NCHRP 1-37A M-E Design Procedures in Ohio: Volume 4 - MEPDG Models Validation & Recalibration," Ohio Department of Transportation Mallela, J., L. Titus-Glover, S. Sadasivam, B. Bhattacharya, M. I. Darter, and H. L. V. Quintus, "Implementation of the AASHTO Mechanistic-Empirical Pavement Design Guide for Colorado," Colorado Department of Transportation