Pavement Maintenance Performance and Thresholds Analysis Using DEA and ROC Methods

Transcription

1 Pavement Maintenance Performance and Thresholds Analysis Using DEA and ROC Methods Jyh-Tyng Yau Researcher Architecture and Building Research Institule, Taiwan Tel: Fax: Jianxiong Yu Ph.D., P.E. Aztec Engineering, Inc. 625 Fair Oaks Avenue, Suite 101, South Pasadena, CA Tel: Fax: Jyh-Dong Lin Professor Department of Civil Engineering, National Central University, Taiwan Tel: Fax: Shih-Chi Lo Researcher Architecture and Building Research Institule, Taiwan Tel: Fax: ABSTRACT This study uses the DEA (Data Envelopment Analysis) and ROC curve (Receiver Operating Characteristic) methods to analyze the factors that affect the performance of asphalt concrete overlays on flexible pavements in Ohio. The ROC method is used to compare the relative importance of the parameters and to determine the appropriate maintenance timing. In order to satisfy the binary data form that is required when ROC is performed, the treatment results are considered as either effective or ineffective. The DEA is employed to help define the effective treatment from ineffective ones. DEA analysis was performed to distinguish between relatively good and poor performing sections according to PCR values of six successive years. Furthermore, the PCR threshold value for pavement maintenance and optimal overlay thickness can be obtained. The results show that maintenance timing is the most important factor effecting flexible pavement with low traffic and overlay thickness is most important one in high traffic group. The appropriate PCR value of maintenance timing is 71.5 for low traffic and 70 for high traffic group. The appropriate quantity of thickness added should be 2.1 inches for high traffic group. BACKGROUND The deterioration of pavement condition over time involves complex interactions between a pavement s structure, climate, traffic loading and previous maintenance works. Selection of repair alternatives and proper timing is often determined by how a pavement is expected to perform based on empirical knowledge. Because of the complex interaction of the various factors involved, quantify the effect of various factors on pavement performance is the subject of many past and current research studies. [1][2] The factors that have been identified to potentially affect pavement performance can be grouped into seven categories: pavement materials, weather, soil, region, traffic, repair type, and pavement condition at time of repair. To manage a pavement network systematically requires the knowledge of the key factors that affect maintenance performance. Also, it would be beneficial to determine the type and timing of repair treatment.

2 Yau, Yu, Lin and Lo Page 2 INTRODUCTION OF ANALYSIS METHODS Data Envelopment Analysis (DEA) DEA was developed by Charnes et al. [6,7,8,9]who employed a mathematical planning model to measure the technical efficiency frontier based on the concept of the Pareto optimum. DEA is a performance measurement technique which is recognized as a valuable decision support tool for managerial control and organizational diagnosis. The Data Envelopment Analysis (DEA) approach [7,8,9] provides a means for assessing relative efficiencies of Decision Making Units (DMUs), with minimal prior assumptions on input-output relations in these units. Such relative efficiencies can be evaluated among a group of a single period or in a sequence of periods. Alternatively, efficiencies can be evaluated with respect to known performance by competitors or measured against predetermined goals or standards.[5] It has been applied successfully in various managerial contexts. The mathematical programming of DEA is particularly adept at estimating multiple input or multiple output production processes. It is a linear programming based method designed to assess the relative efficiencies of decision making units (DMUs). DEA applies a linear programming technique that converts multiple inputs and multiple outputs into a scalar measure of relative productive efficiency to construct a frontier based on the sample. DMU efficiency measurement is defined as each DMU s mathematical position relative to the frontier of best performance established by the ratio of weighted sum of outputs to weighted sum of inputs. DMUs on the frontier are efficient, while DMUs inside the frontier are inefficient. With the assumption of no random error, all deviations from the estimated frontier are attributed to inefficiencies. The following 6 1 shows the relationship between efficient frontier, efficiency score, four inefficient units, three efficient units and projection points. The units depicted along the vertical axis and abscissa indicate the relative quantity of output produced by each unit s input (y1/x, y2/x). The seven black points represent seven separate DMUs. Points A and C, in particular, produce comparably efficient performance, while point B is located along the same mathematical frontier as A and C. If points A, B and C are all considered as 100% efficient in terms of the DEA model, then the curve connecting A, B and C is the so-called efficient frontier. Therefore the four remaining DMUs effectively surrounded by the efficient frontier are all deemed as relatively inefficient while the three DMUs specifically located along the efficiency curve are deemed to be efficient. Data Envelopment Analysis effectively separates efficient DMUs from their inefficient peers. From a competitive standpoint, inefficient DMUs should routinely benchmark themselves against more efficient peers to improve performance. The efficient frontier line in the DEA model is recognized simply as an envelopment line to separate efficient DMUs from their inefficient peers. The essential characteristic of the CCR ratio construction (developed by Charnes, Cooper and Rhodes[8]) is the reduction of the multiple-output-multiple-input situation(for each DMU) to that of a single virtual output and a single virtual input. For a DMU, the ratio of this single virtual output to single virtual input provides a measure of efficiency that is a function of the multipliers. This ratio, which is to be maximized, forms the objective function for the particular DMU 0 being evaluated, so that symbolically: Max h j u, v 0 ) s r= 1 = m i= 1 u r v x i y rj ij 0 0 (.(1) Of course, without additional constraints, the above model is unbounded. The additional set of constraints (one for each DMU) reflects the condition that the ratio of virtual output to virtual input of

3 Yau, Yu, Lin and Lo Page 3 every DMU should be less than or equal to unity. The mathematical programming problem for the CCR (input-oriented) ratio form is: Max h j s r= 1 = m i= 1 u r v x i y rj ij (2) s t. s r= 1 = m i= 1 u r v x i y rj ij (3) u r, v i ε > 0, i = 1, 2,, m; r = 1, 2,, s; j = 1, 2,, n where yrj=amount of output r from unit j, xij=amount of input i to unit j, ur=the weight given to output r vi=the weight given to input i, n=the number of units, m=the number of inputs, s=the number of outputs, ε=a small positive number. It is important to note that DEA calculations only produce relative efficiency measures. The relative efficiency for each DMU is calculated in comparison to all the other DMUs. DEA calculations are designed to maximize the relative efficiency score for each DMU. DEA produces an empirical production surface that, in economic terms, represents the revealed best-practice frontier -- i.e., the maximum output obtainable from any DMU in an observed population, given a predefined level of inputs. DEA presents some extremely useful features and advantages (1) It characterizes each DMU by a single efficiency score; (2) By projecting inefficient units on the efficient envelop, it highlights areas of improvements for each single DMU; (3) It facilitates making inferences on the DMUs general profile; (4) The possibility of handling multiple inputs and outputs stated in different measurement units; (5) The focus on a best-practice frontier, instead of on population central-tendencies. Every unit is compared to an efficient unit or a combination of efficient units. The comparison, therefore, leads to sources of inefficiency of units that do not belong to the frontier; (6) No restrictions are imposed on the functional form relating inputs to outputs; Receiver Operating Characteristic (ROC) curve Receiver Operating Characteristic (ROC) curve method was applied extensively in many area especially in medical science.[10,11] Two basic measures of diagnostic accuracy are sensitivity and specificity. Their definitions are best illustrated by a 2 2 contingency table(table 1), or decision matrix, where the

4 Yau, Yu, Lin and Lo Page 4 rows summarize the data according to the true condition status of the patients and the columns summarize the test results. We denote the true condition status that S2 if the condition is present and S1 if the condition is absent. Test results indicating the condition s presence are called positive; those indicating its absence, negative. We denote positive test results as R2, negative test results as R1. The total number of patients with and without the condition is, respectively, N A and N N ;The total number of patients with the condition tested positive and negative is, respectively, n11and n12; and the total number of patients without the condition tested positive and negative is, respectively, n21and n22.the total number of patients in the study group, N, is expressed as N = n11+ n11+ n21+ n22. The sensitivity ( or true-positive rate ) of a test is its ability to detect the condition when it is present. We write sensitivity = n11/(n11+ n12). The specificity ( or true-negative rate) of a test is its ability to exclude the condition in patients without the condition. We write specificity =22/(n21+ n22).figure 2 gives the illustration. In 1971, Lusted described how a method used often in psychophysics could be adopted for medical decision making. This method overcomes the limitations of a single sensitivity and specificity and provides a summary measure associated with single sensitivity and specificity pairs by including all of the decision thresholds [12]. A Receiver Operating Characteristic (ROC) curve is a method of describing the intrinsic accuracy of a test apart from the decision thresholds. Since the 1970s, it has been the most valuable tool for describing and comparing diagnostic procedures. The ROC curve is defined as a plot of the sensitivity (or true-positive rate) (TPR) on the vertical axis versus the one minus specificity (or false-positive rate) (FPR) on the horizontal axis. Each point on the graph is generated by a different decision threshold. If we choose k different points of cut probability, then we have k 2 by 2 tables that can form k points on the plane. Let k be infinite, the points can be connected as a smooth curve named ROC curve and the two pairs of values then rise monotonically. It is a path in the unit square, from the lower left corner to the upper right corner. In fact, it can be viewed as a cumulative distribution function. It is concave and above diagonal line. Typical ROC curve are illustrated in Figure 3. For a useless marker test, the normal and abnormal will have similar distributions, and the corresponding values of sensitivity and one minus specificity will be essentially equal in each point. The ROC curve will be a straight line at 45 angles. Figure3 gives the illustration. The Area under the ROC Curve (AUROC) Although the area under the ROC curve (abbreviate AUROC) is not the only measure of a test s accuracy, it is preferred over simple estimates of sensitivity or specificity because the area under the curve incorporates both of theses measure of accuracy and accounts for the inherent trade-offs between them as the decision criterion changes [12,13]. Define the variables X and Y as the random variable T given D = 1, and the random variable T given D = 0, respectively. So the population sensitivity can be expressed as: Se( c ) = P( T = 1 D = 1) = P( X C ), and the population specificity can be expressed as: Sp( c ) = P( T = 0 D = 0) = P( Y C ) Therefore, the area under the ROC curve can be formulated as the form: θ = 1 0 = 1 0 = Se( c ) d(1 Sp( c )) P ( X c ) dp( Y c ) P γ = P( X Y ), ( X c ) f ( c ) dc

5 Yau, Yu, Lin and Lo Page 5 C : the cut off point here f y (C)is the probability density function of Y The ROC curve area can take values between 0.0 and 1.0. An ROC curve with an area of 1.0 consists of two line segments: (0, 0)-(0, 1) and (0, 1)-(1, 1). Such a test is perfectly accurate because the sensitivity is 1.0 when the FPR is 0.0. The practical lower bound for the ROC curve area is 0.5. The (0, 0)-(1, 1) line segment has an area of 0.5; it is called the chance diagonal. Diagnostic procedures with ROC curves above the chance diagonal have at least some ability. The closer the curve to the (0, 1) point (left upper corner), the better the test will be. DATA ANALYSIS Climate and traffic volume are two major factors that affect the deterioration of flexible pavements. In the area being researched the highest average annual temperature is 55.7 and the lowest is For precipitation, the highest is 48.1 inches and lowest is 31.8 inches. Most of the pavement sections locate in an area where the average annual temperature is around 50 and the precipitation is about 40 inches. Snowfall amount varies from 16 inches to 99 inches. To eliminate the effect of extreme high or low snowfall on the analysis, sections where snowfall is higher than 80 inches or lower than 30 inches are removed. One of the main purposes of this research is to evaluate whether the prior pavement condition, i.e., PCR-1, has an obvious effect on the pavement performance. If it does, what is the appropriate maintenance threshold? It is known that traffic volume has important effect on pavement performance. To evaluate the effect of prior pavement condition, differences in traffic volume have to be considered. In this research, data are classified into two groups, H and L. Group H has an annual ESAL(Equivalent Single Axle Load ) of greater than 2.5 million. The average ESAL in this group is 3.7 million and the average ADT(Average Daily Traffic) in Group H is The Group L has an annual ESAL equal or lower than 2.5 million. The average ESAL is 1.1 million and average ADT is Performance of asphalt concrete overlay on flexible pavements is analyzed. Each sample contains all following information: PCR-1 Pavement Condition Rating before treatment PCR0 Pavement Condition Rating immediately after treatment PCR1 Pavement Condition Rating 1 year after treatment PCR2 Pavement Condition Rating 2 year after treatment PCR3 Pavement Condition Rating 3 year after treatment PCR4 Pavement Condition Rating 4 year after treatment PCR5 Pavement Condition Rating 5 year after treatment PCR6 Pavement Condition Rating 6 year after treatment ADT Annual Average Daily Traffic ESAL Annual Average Equivalent-Single-Axle-Load Thickness The thickness of overlay System Interstate Route, US route or State Route Priority Highway priority classification: Priority, Urban or General Temperature Average annual temperature in Snowfall Average annual snowfall in inches Because a few of samples do not have complete information for following 4 variables, we only analyze the sample which has the information. Time Since Major Time since last major rehabilitation in years Age At Repair Time since last rehabilitation in years Age At Next Repair Time to the next repair in years Repairs Since Major Number of repairs since last major rehabilitation

6 Yau, Yu, Lin and Lo Page 6 This research evaluates the effectiveness of maintenance based on pavement sections that have at least 6 years of pavement condition. Though longer analysis period can be used, the number of samples with complete information decreases. The longer the period, the fewer the number of samples can be obtained. In addition, nearly 90% of the pavements in the data set receive some treatment within 6 to 10 years. To analyze the influence of several factors on pavement maintenance performance and optimal maintenance thresholds, the relatively effective and ineffective maintenance were distinguished. Pavement that received relatively effective of treatment can be determined using the Data Envelopment Analysis (DEA) based on its PCR0, PCR1, PCR2.PCR6.These seven attributes were regarded as the outputs of each DMU. When the inputs of all DMU are all assumed to be equal, the effectiveness score of each DMU can be calculated and the relative effectiveness determined. In this phase, we only focus on measuring overall maintenance performance according to PCR values. As regards the factors leading to the performance, we analyzed using ROC method in next chapter. In this research, EMS (Efficiency Measurement System, Version 1.3) was used to analyze individually the efficiency of two groups. In Group L, among 143 samples there are 44 samples are relatively efficient while the remaining 99 samples are not. In Group H, among 99 samples there are 37 samples are relatively efficient while the remaining 62 samples are not. ANALYSIS RESULTS After carrying out DEA analysis, basically we can convert a ranking scale or a continuous measurement into a binary outcomes: efficient and inefficient. Then the plotting and analysis of ROC curves were performed with the commercial software SPSS10.0 to determine maintenance thresholds and effective variables ROC Analysis Results of Group L Among snowfall, precipitation, temperature, ADT and ESAL, only snowfall has representative effect on pavement performance. The area under ROC curve (Figure 4(a)) is With 95% confidence interval, the lower limit of the area is and the upper limit is with a standard error of 0.52 and a significance of This result shows the thicker the snowfall, the worse the pavement performance is. Other 4 variables have no significant effect on pavement performance statistically (Table 2(a)). Analysis on PCR-1, Thickness, System and Priority shows PCR-1 has a strong significance value of 0 and the area under ROC curve reaches The 95% confidence interval is (0.590, 0.793). This confirms that for same treatment the pavement condition before the treatment dominates the 6-year performance after the treatment. The higher the prior PCR, the better the performance is. On the contrary, a delay to a treatment, i.e., to treat a pavement until it reaches lower PCR, will decrease the efficiency of the treatment. This fact brings up the need to determine an optimal maintenance threshold. This problem can be solved by the ROC analysis. The area under the curve represents the relativity of the variable to the research objective. On the curve, each point means a different variable value and its corresponding sensitivity (1-sigualarity). The most ideal case is a point with both sensitivity and singularity of 1. The point locates on the top-left corner of figure 4(b). In reality, the efficiency of pavement performance obeys probability density distribution. This means that even a pavement can be treated at a very high prior PCR, the efficiency of the treatment could still be very low. On the other hand, even a pavement was treated at a very low PCR, the efficiency of the treatment could still be high, though the possibility is low. Any point on the figure could have possible adverse case. Therefore, the point on the ROC curve that is nearest to the point (1, 0) is the optimal threshold. In this research, when PCR-1 is 71.5, the sensitivity is and the singularity is The distance to point (1, 0) is This point is the separating point whether a treatment is efficient or not. Pavements in analysis have 3 system types: IR (Interstate Route), SR (State Route) and US (Urban Street). For analysis convenience, they are assigned a value of 3, 2 and 1 respectively. Similarly, pavements have three different priority levels: P (Priority), U (Urban) and G (General). Each priority

7 Yau, Yu, Lin and Lo Page 7 level is represented by a value 3, 2 and 1 respectively. Analysis results (Table 2(b)) show that the effect of priority on performance efficiency is not significant. However, route system has a significant effect on performance efficiency, IR is the best and US is the worst. The reason could be that different route system has different pavement property, such as different material of subbase and base, different pavement structure and so on. Priority level is determined mainly by traffic volume but not pavement structure and design. For a given group, the traffic has already been classified. Therefore, it has no significant effect on the efficiency of the performance. Results also show, for low traffic Group L, the thickness added has no significant effect on the efficiency of the performance. Because not all samples have all Time Since Major, Age At Repair, Age At Next Repair and Repairs Since Major information, analysis will only be performed on completed cases to find out the relationship between these variables and the efficiency of performance. Research results (Table 2(c)) show only Age At Next Repair has significant effect on the efficiency of treatment performance. The area under ROC curve is 0.71 and the significance is The result is reasonable. Though Age At Next Repair should not affect the efficiency of the previous treatment, it confirms that pavements that had performed better received treatment in a later time. Meanwhile, this result also shows that pavement engineers had made correct treatment decisions. To low traffic group, the efficiency is still highly correlated with PCR-1. However, PCR-1 has a very low correlation with either Age At Repair or Repairs Since Major. One reason of this might be that most of the previous treatments were either minor or preventive maintenance. The effect of each individual PCR (PCR0 PCR6) value on the efficiency of the treatment performance is analyzed using ROC curves. From the curves in Figure 4(c) and the area in the Table 2(d), it can be seen that PCR1 has the most significant effect on the treatment performance and then PCR2 has a less significance. The effect decreases with time. It shows that the efficiency of a treatment can be observed from the initial several years. With time going on, the difference diminishes. ROC Analysis Results of Group H Snowfall, precipitation, temperature, and ESAL have no significant effect on treatment efficiency. Their significances are all greater than Only ADT has representative effect on pavement performance. The reason is that in this group most of the pavement sections have good treatment efficiency are interstate routes. Interstate routes have higher traffic volume.(shown in table 3(a) and figure 5(a) ) Analyses on PCR-1, Thickness, System and Priority shows both PCR-1 and Thickness have a strong significant effect on treatment efficiency. Both of them have an area under ROC curves of 0.72 and the threshold of PCR is 71. The sensitivity is 0.73 and the singularity is The distance from this optimal point to (0, 1) is The threshold of Thickness is 2.13 and its sensitivity is 0.83, singularity is This point is 0.34 far from (0,1). ( shown in table3(b) and figure5(b) ) The analysis results show that the relationships between Time Since Major, Age At Repair, Age At Next Repair, Repairs Since Major and the treatment efficiency are opposite to the result of Group L. The lower the Age At Repair, the more efficient the treatment is. The higher the Repairs Since Major and Time Since Major, the more efficient the treatment is. They are all statistically significant. The areas under ROC curves for Repairs Since Major and Age At Repair are and respectively. The significances are all 0. The reason is that these two variables affect PCR-1. The area under ROC curve of the variable Age At Next Repair is This means the better the efficiency the longer the next treatment will be. However, this relationship is not statistically significant.(shown in table3(c) ) It can be seen from Table3(d) and curves in Figure 5(c) that the effect of individual PCR value (PCR0 PCR6) on the overall efficiency increases with time until year 4. The area under ROC curve is This result differs from that of low traffic Group L. The table 4 indicates that the different of PCR value between group L and group H. CONCLUSION AND DISCUSSION For the low traffic group, Snowfall, Thickness, prior PCR (PCR-1) and route System have significant effect on treatment efficiency. PCR-1 is the one that has the most significant effect. The area under ROC

8 Yau, Yu, Lin and Lo Page 8 curve is The treatment efficiency also affects the next treatment time. Other variables are not statistically significant. For the high traffic group, the variables that have significant effect on treatment efficiency are Thickness, PCR-1, System, Time Since Major, Age at Repair, Repairs Since Major and ADT. Among snowfall, precipitation, temperature, ADT and ESAL, only snowfall has representative effect on pavement performance to low traffic group. For high traffic group, the only one that has significance is ADT. However, ADT is highly correlated with System. Interstate routes have better efficiency and high traffic volume. For the whole state, temperature and precipitation do not vary significantly, so they don t have effect on performance statistically. For ESAL, it is categorized into groups. Therefore, ESAL has no significant effect on treatment efficiency in each group. In addition, ADT and ESAL are all correlated with route system. Though higher-level routes could have a higher traffic, it does not mean they should have worse pavement quality. Among Thickness, PCR-1, System and Priority, PCR-1 and System have significant effect on treatment efficiency for both groups. Thickness only has effect on high traffic group. Priority has no significant effect on both groups. The maintenance PCR threshold is 71.5 for low traffic and 70 for high traffic group. Thickness added should be 2.1 inches for high traffic group. For high traffic group it seems unreasonable that the better the efficiency the longer the Time Since Major. In fact, though the time since major is long, the number of other treatment during this period is also larger. This can be confirmed by the fact that Repairs Since Major is highly correlated with treatment efficiency. The other reason is because pavement sections with better efficiency have more interstate routes (51%) while pavement sections without efficiency have less interstate routes (13%). Interstate routes have more repairs since last rehabilitation or initial buildup. Research also shows that the Age At Repair has a significant effect on the treatment efficiency. However, the above three variables have no significant effect on the treatment efficiency for low traffic group. It can be concluded that the time factor has very less effect on the treatment efficiency. The linear correlation coefficient of Age At Repair and PCR-1 can confirm the above deduction. For high traffic group, the R 2 is For low traffic group the R 2 is 0.13 and it means though the Age At Repair is short the PCR-1 could still be poor. One possible reason is that the previous treatment was just a crack sealing. Crack sealing cannot improve the PCR significantly. Therefore, the treatment efficiency of low traffic group is only affected by PCR-1 but not Age At Repair. Variable Age At Next Repair has significant effect for low traffic only. If a treatment is efficient then the Age At Next Repair is also long. For high traffic group, the Age At Next Repair is about 8.9 years for efficient sections and 8.2 years for inefficient sections. However, the result is not statistically significant because efficient sections have more interstate routes and higher traffic. For low traffic group, the PCR of the first year, PCR1, has the most significant effect on the efficiency of treatments. The effect deteriorates with time. For high traffic group, the PCR of the fourth year, PCR4, has the most significant effect on the efficiency of treatments. This may because the effect of accumulative traffic loads is significant for high traffic group. In high traffic group, the distribution of the areas under the ROC curves of PCRs is obviously different REFERENCES 1. Labi, Samuel.; Sinha, Kumares C. Effectiveness of Highway Pavement Seal Coating Treatments. Journal of Transportation Engineering v. 130 no1 (Jan./Feb. 2004) p Labi,Samuel.; Sinha, Kumares C. Measures of Short-Term Effectiveness of Highway Pavement Maintenance. Journal of Transportation Engineering v. 129 no6 (Nov./Dec. 2003) p Shahin, M.Y. Pavement Management for Airports, Roads, and Parking Lots. Chapman and Hall, New York, Ohio Department of Transportation. Pavement Design and Rehabilitation Manual. ODOT, Columbus, Ohio, Farrell, M. J. The measurement of productive efficiency, Journal of the Royal Statistical Society, Series A, Vol.120, 1957, Banker, R. D., A. Charnes and W. W. Cooper. Some models for estimating technical and scale

9 Yau, Yu, Lin and Lo Page 9 efficiencies in data envelopment analysis. Management Science, Vol. 30, No. 9, 1984, Charnes, A., W. W. Cooper, A.Y. Lewin and L.M. Seiford. Data Envelopment Analysis: Theory, Methodology, and Applications. Kluwer Academic Publisher, Boston, Charnes, A., W. W. Cooper and E. Rhodes. Measuring the efficiency of decision making units. European Journal of Operational Research, Vol. 2, 1978, Charnes, A. and W. W. Cooper, Programming with linear fractional functionals. Naval Research Logistics Quarterly, Vol. 9, 1962, Obuchowski, N. and McClish, D. Sample size determination for diagnostic accuracy studies involving binormal ROC curve indices. Statistics in Medicine 16: DeLong, E., DeLong, D., and Clarke-Pearson, D. Comparing the areas under two or more correlated receiver operation characteristic cures: A nonparameteric approach. Biometrics 44: 1988, Metz, C. E. Basic principles of ROC analysis. Seminars in Nuclear Medicine, 1978, Metz, C. E. Some practical issues of experimental design and data analysis in radiolgic ROC studies. Invest. Radiol. 24: 1989, True Condition Status TABLE Contingency Table Test Results Positive (R 2 ) Negative (R 1 ) Total Present (S 2 ) n 11 n 12 N A Absent (S 1 ) n 21 n 22 N N Total n +1 n +2 N TABLE 2(a) ROC Analysis Results of Group L Area Std. Error Asymptotic Asymptotic 95% Sig. Confidence Interval Test Result Variable(s) Lower Upper Bound Bound Snowfall Ave ADT Temperature Precipitation Ave ESAL TABLE 2(b) ROC Analysis Results of Group L Area Std. Error Asymptotic Asymptotic 95% Confidence Sig. Interval Test Result Variable(s) Lower Bound Upper Bound PCR Thickness System Priority TABLE 2(c) ROC Analysis Results of Group L

10 Yau, Yu, Lin and Lo Page 10 Time Since Major No. of Samples with Efficienc y No. of Samples without Efficiency Area Std. Error Asymptotic Sig. Asymptotic 95% Confidence Interval Lower Bound Upper Bound Age At Repair Age At Next Repairs Repairs Since Major TABLE 2(d) ROC Analysis Results of Group L Area Std. Asymptotic Error Sig. Asymptotic 95% Confidence Interval Test Result Variable(s) Lower Bound Upper Bound PCR PCR PCR PCR PCR PCR PCR

11 Yau, Yu, Lin and Lo Page 11 TABLE 3(a) ROC Analysis Results of Group H Area Std. Asymptotic Asymptotic 95% Confidence Error Sig. Interval Test Result Variable(s) Lower Bound Upper Bound Snowfall Ave Esal Ave ADT Temperature Precipitation TABLE 3(b) ROC Analysis Results of Group H Area Std. Asymptotic Asymptotic 95% Error Sig. Confidence Interval Test Result Lower Variable(s) Bound Upper Bound System Priority PCR Thickness TABLE 3(c) ROC Analysis Results of Group H Time Since Major Age At Repair Age At Next Repairs Repairs Since Major No. of Samples with Efficiency No. of Samples without Efficiency Area Std. Error Asymptotic Sig. Asymptotic 95% Confidence Interval Lower Bound Upper Bound

12 Yau, Yu, Lin and Lo Page 12 TABLE 3(d) ROC Analysis Results of Group H Area Std. Asymptotic Asymptotic 95% Error Sig. Confidence Interval Test Result Variable(s) Lower Bound Upper Bound PCR PCR PCR PCR PCR PCR PCR TABLE 4 Compare PCR Value between Group L and Group H PCR0 PCR1 PCR2 PCR3 PCR4 PCR5 PCR6 Efficient Group L Inefficient Average Efficient Group H Inefficient Average FIGURE 1 Efficient frontier.

13 Yau, Yu, Lin and Lo Page 13 Normal Abnormal Specificity Sensitivity Threshold C FIGURE 2 Sensitivity and specificity Sensitivity ROC curve Chance curve Specificity FIGURE 3 Typical ROC curve.

14 Yau, Yu, Lin and Lo Page 14 ROC Curve Source of the Curve Reference Line Ave Esal Precipitation Sensitivity Temperature Ave ADT Snowfall 1 - Specificity Diagonal segments are produced by ties. FIGURE 4(a) ROC curve of group L ROC Curve Source of the Curve Reference Line Priority Sensitivity System Thickness PCR Specificity Diagonal segments are produced by ties.

15 Yau, Yu, Lin and Lo Page 15 FIGURE 4(b) ROC curve of group L. ROC Curve Source of the Curve Reference Line PCR 6 PCR 5 PCR 4 PCR 3 Sensitivity PCR 2 PCR 1 PCR Specificity Diagonal segments are produced by ties. FIGURE 4(c) ROC curve of group L. ROC Curve Source of the Curve Reference Line Precipitation Temperature Sensitivity Ave ADT Ave Esal Snowfall 1 - Specificity Diagonal segments are produced by ties. FIGURE 5(a) ROC curve of group H

16 Yau, Yu, Lin and Lo Page 16 ROC Curve Source of the Curve Reference Line Thickness Sensitivity PCR-1 Priority System 1 - Specificity Diagonal segments are produced by ties. FIGURE 5(b) ROC curve of group H. ROC Curve Source of the Curve Reference Line PCR 6 PCR 5 PCR 4 PCR 3 Sensitivity PCR 2 PCR 1 PCR Specificity Diagonal segments are produced by ties. FIGURE 5(c) ROC curve of group H.