COMPARISON OF LOGISTIC REGRESSION MODEL AND MARS CLASSIFICATION RESULTS ON BINARY RESPONSE FOR TEKNISI AHLI BBPLK SERANG TRAINING GRADUATES STATUS

Transcription

1 International Journal of Humanities, Religion and Social Science ISSN : Volume 2, Issue COMPARISON OF LOGISTIC REGRESSION MODEL AND MARS CLASSIFICATION RESULTS ON BINARY RESPONSE FOR TEKNISI AHLI BBPLK SERANG TRAINING GRADUATES STATUS Prima Adityawardani 1, Ni Wayan Surya Wardhani 2, and Maria Bernadetha Theresia Mitakda 2 1 Master Student of Statistics Department, Brawijaya University, Malang, Indonesia; And 2 Lecturer of Statistics Department, Brawijaya University, Malang, Indonesia Abstract: Logistic regression and Multivariate Adaptive Regression Splines (MARS) can be used as analytical methods for non-linear data with binary response variable. Both of these analyzes can be used for object classification. The existence of multicollinearity is not allowed in logistic regression and MARS, since it causes a large variance tend to logistic regression and improper placement of knots on MARS so that the resulting model is not accurate. Principal Component Analysis (PCA) is used to fixed the multicollinearity and PCA for a combination of discrete and continuous variables called PCAMIX. The new variables formed from PCAMIX are not correlated so that they can be used in the classification of observations based on logistic regression and MARS model. Classification accuracy testing uses accuracy, Noise Signal Ratio (NSR) and similarity testing of two proportions. The results of the analysis shows that the factors that affect the Teknisi Ahli BBPLK Serang Training graduates status who directly work in industry is the factor of intelligence and physiological. In addition, the classification test indicates that there is no difference of classification accuracy generated logistic regression and MARS. Keywords:logistic regression, MARS, multicollinearity, PCAMIX, classification I. Introduction Simple or multiple linear regression is a model used to describe the relationship between predictor variables and response variable. However, sometimes linear regression with Least Square Method is less suitable for the analysis since of Gauss-Markov assupmtions infraction, such as categorical response variable. To solve this problem can be used logistic regression (M.H. Kutner, C.J. Nachtsheim, J. Neter and W. Li, 2004) and Multivariate Adaptive Regression Splines (MARS) (J.H. Friedman, 1991). Logistic regression is a statistical analysis method that describes the relationship between the categorical response variable and the categorical or continuous predictor variables (D.W. Hosmer and J.R. Lemeshow, 2000). Logistic regression model with response variable that has two categories is known as binary logistic regression. While MARS is an approach to model nonparametric regression introduced first by Friedman (J.H. Friedman, 1991). This model is well used when predictor variables are large and non-linear (J. Munoz and A.M. Felicimo, 2004). Multicollinearity is not allowed in logistic regression and MARS. This will cause the value of variance tend to be large as the level of collinearity among the predictor variables increases in the logistic regression (A.M. Aguilera, M. Escabias and M.J. Valderrama, 2006). Similarly, in MARS analysis, if the predictor variables are correlated, then the MARS forward procedure places the knots in less precise position so that the model is less accurate (R.D. De Veaux and L.H. Ungar, 1993). Principal Component Analysis (PCA) is suggested to fixed multicollinearity among predictor variables in logistic regression and MARS by Aguilera (A.M. Aguilera, M. Escabias and M.J. Valderrama, 2006). PCA for mixed 14

2 qualitative and quantitative variables called PCAMIX (S.A. Jaramillo, M.O. Munoz and F.I.Z. Diaz, 2016). Application of logistic regression and MARS with PCAMIX to fixed multicollinearity used to determine the factors that allegedly affect the quality of Teknisi Ahli BBPLK Serang Training graduates who are accepted to work in industry. Then compare the two models to find out which model has better performance in terms of classification. Measurement of classification performance using accuracy (M.P. Kuhnert, K. Anh Do and R.McClure, 1999), Noise Signal Ratio (NSR) (O. Zambrano, C.M. Rocco S. and M. Muselli, 2007) and similarity test of two proportions. II. Material And Methods This study uses data of , that is about Teknisi Ahli Training BBPLK Serang graduate status for training duration time 2 years, with variables: Teknisi Ahli training graduate status (Y), 1 (directly work) dan 0 (indirectly work). If students graduate and work in August 2 years later, then it is said to directly work. Gender (X 1 ), 1 for male dan 0 for female, Types of student education prior to training (X 2 ), categorized 1 for SMK dan 0 for SMA, Residence location after graduation training (X 3 ), 1 (urban) dan 0 (rural), Graduate training GPA (X 4 ), Age of student while passing training (X 5 ), Average of Ujian Akhir Nasional (UAN) score (X 6 ). The analytical procedures are: first perform multicollinearity testing using VIF (Varince Inflation Factor) and if the data shows multicollinearity, PCAMIX is performed as follows: 1. Calculate the quantification matrix S j 2. Calculate the matrix S 3. Performs an eigen value decomposition of S Maximize the qudratic form of W: W (q q) = X (q n) S (n n) X (n q) s.t X X = I q, so as to obtain the eigen equation: W (q q) λi q = 0 The results of this equation is eigenvalue λ 1, λ 2,, λ q that satisfy λ 1 λ 2 λ q Calculate the variance of the l-th component given by x l Sx l where x l denotes the l-th column of X (l = 1,2,, q). 5. Calculate the matrix C of the squared loadings of the k variables on the l components with c jl = n 1 x l S j x l, where c jl denotes correlation coefficients ratio the variable j and the component l for qualitative variables and c jl is squared correltion coefficients the variable j and the component l for quantitative variables. Second, establish a logistic regression model based on PCAMIX results and perform simultaneous parameter test (G test statistics) and partial (Wald test statistics). Third, establish MARS model of 15

3 PCAMIX results by combining BF (maximum basis function is 2-4 times as many predictor variables), MI (maximum interaction of 1-3) and MO (number of observations between knots), where MO is formulated (J.H. Friedman, 1991): k is the number of predictor variables and n is the number of observations. Determine the best MARS model based on the smallest GCV (Generalized Cross Validation) (J.H. Friedman, 1991): Then perform simultaneous test (F test statistics) and partial (t test statistics) to model parameter. From both models will be obtained factors that affect the Teknisi Ahli training graduates status who directly work in industry. Fourth, calculate sensitivity, specificity, accuracy and NSR based on classification table of each model. Then choose a better model by comparing the accuracy value, NSR and testing the two proportions similarity (R.E. Walpole, R.H. Myers, S.L. Myers and K. Ye, 2007). PCAMIX and logistic regression analysis performed on RStudio with the package PCAmixdata and LOGIT, then MARS performed on SPM8. III. Result and Discussion Multicollinearity detection among predictors variables based on VIF is presented in Table 1. Table1. VIF Predictor Variables VIF Conclusion Graduate training GPA (X 4) Multicollinearity Age of student while passing training (X 5) 1.01 There is no multicollinearity Average Ujian Akhir Nasional (UAN) score (X 6) Multicollinearity Table1 shows that there is multicollinearity among graduate training GPA and average of UAN score variables based on VIF > 10. Then perform PCAMIX analysis and derive eigenvalue, variance proportion and cumulative proportion in Table 2. Table 2.Eigenvalue, Variance Proportion and Cumulative Proportion Component Eigenvalue Variance Proportion (%) Cumulative Proportion (%) KU KU KU KU KU KU

4 Table 2 shows that the eigenvalue greater than 1 are the first, second and third components, in which these three components are able to explain 71.36% of predictors variance and will be used in logistic regression and MARS analysis. In addition, we obtain loading coefficients as in Table 3. Table 3.PCAMIX Loading Coefficients Predictor Variables KU 1 KU 2 KU 3 X X X X X X The first component (KU 1 ) is called intelligence factor due to the X 4 (graduate training GPA) and X 6 (average of UAN score) variables as the largest contributor of variance. While the second and third components are X 2 (types of student education prior to training), it is called interest factor; and X 5 (Age of student while passing training), it is called physiological factor. Logistic regression results shows that the significant variables affecting the training graduates status who directly work are first component (KU 1 ) and third component (KU 3 ). The logistic regression coefficient model estimation can be seen in Table 4. Table 4.Logistic Regression Coefficients Estimation Variables Coefficient Standard Error Wald Test Statistic p-value Constant KU * KU * Logistic regression model is: or in natural logarithmic transformation form is: Classification based on logistic regression model according to Table 5. Table 5.Two Class Classification Confusion Matrix for Logistic Regression Model Observation Result Predicted Result Total y = 0 y = y =

5 Total Table 5 provides information on the sensitivity, specificity and accuracy of the logistic regression model classification results by 72%; 52.17% and 62.50%. The percentage of logistic regression model overall classification is 62.50%. MARS model formation by combining BF between 2 and 4 times the predictor variables, MI is 1 to 3 and MO according to equation (1) is 5, so that there are 9 models combination. Table 6 presents GCV values of 9 models. Table 6.GCV for Every Combination BF, MI and MO BF MI MO GCV Table 6 shows that MARS model with the combination of BF=6, MI=1 and MO=5 has the smallest GCV value among the 9 MARS models,which is MARS model for graduate status BBPLK Serang is: Classification results by MARS model are presented in Table 7. Table 7. Two Class Classification Confusion Matrix for MARS Model Observation Result Predicted Result Total y = 0 y = 1 y = y = Total

6 Table 7 provides information on the sensitivity, specificity and accuracy of the MARS model classification results by 65,33%; 69,57% and 67,36%. The percentage of MARS model overall classification is 67,36%. Accuracy and NSR calculation results is presented in Table 8. MARS accuracy level is higher than logistic regression, that is 67,36%. Similarly, when viewed from NSR, MARS classification performance is better than logistic regression since it has smaller NSR. Table 8. Accuracy and NSR Logistic Regression and MARS Model Model Accuracy (%) NSR (%) Logistic Regression MARS In addition, a similarity test of two proportions was performed to determine the exact classification performance between logistic regression and MARS model. Based on the test statistic Z = 0,865 < Z 0,05 2 = 1,96, then H 0 is accepted. It can be said that there is no difference in the accuracy of classsification by logistic regression and MARS model. IV. CONCLUSION Training graduates intelligence factor (GPA and average of UAN score) and physiological factor (age) affect the training graduates to directly or indirectly work after graduating training. The accuracy of classification by logistic regression and MARS model based on similarity test of two proportions can be said there is no difference, although MARS accuracy and NSR value is more than logistic regression in Teknisi Ahli BBPLK Serang training graduates which is directly or indirectly work in industry. REFERENCES A.M. Aguilera, M. Escabias and M.J. Valderrama Using Principal Components for Estimating Logistic Regression with High-Dimensional Multicollinear Data. Computational Statistics and Data Analysis, 50, D.W. Hosmer and J.R. Lemeshow Applied Logistic Regression. New Jersey: John Wiley & Sons. J. Munoz and A.M. Felicimo Comparison of Statistical Methods Commonly Used In Predictive Modelling. Journal of Vegetation Science, 89. J.H. Friedman Multivariate Adaptive Regression Splines. The Anals of Statistics, 19 (1), M.H. Kutner, C.J. Nachtsheim, J. Neter and W. Li Applied Linear Statistical Models. New York: The McGraw Hill Company Inc. M.P. Kuhnert, K. Anh Do and R.McClure Combining Non-Parametric Models with Logistic Regression: An Application to Motor Vehicle Injury Data. Computational Statistics and Data Analysis, 34. O. Zambrano, C.M. Rocco S. and M. Muselli Estimating Female Labor Force Participation Through Statistical and Machine Learning Methods: A Comparison. Computational Intelligence in Economics and Finance, 2,

7 R.D. De Veaux and L.H. Ungar Multicollinearity: A Tale of Two Nonparametric Regressions. Artificial Intelligence and Statistics, 89, R.E. Walpole, R.H. Myers, S.L. Myers and K. Ye Probability and Statistics for Engineers and Scientist. New Jersey: Pearson Education International. S.A. Jaramillo, M.O. Munoz and F.I.Z. Diaz Principal Component Analysis for Mixed Quantitative and Qualitative Data. EAFIT University, Department of Mathematical Sciences. Medellin: EAFIT University. 20