Regression Analysis I & II

Transcription

1 Data for this session is available in Data Regression I & II Regression Analysis I & II Quantitative Methods for Business Skander Esseghaier 1

2 In this session, you will learn: How to read and interpret a regression output how much does the regression explain (R-Square) is the regression significant? (F-Test) is any coefficient significant? (T-Test) How to deal with some key issues you may be confronted with when using regression models multicollinearity categorical independent variables (dummy variable regression) How to identify the most important variables in a regression analysis 2

3 Meddicorp Meddicorp sells medical supplies to hospitals, clinics and doctors offices Company currently markets in 3 regions of US: South, West and Midwest Meddicorp management is concerned with the effectiveness of a new bonus program for its sales force Management wants to know if there is a relationship between sales and bonuses in

4 What Does the Data Says? SALESPERSON SALES $ BONUS $

5 Measures of Association Scatter plot It describes the relationship between two variables graphically Relationship between Sales Perfromance and Bonus Received Bonus in $ Sales in $

6 How Strong is the Association? Relationship between Sales Perfromance and Bonus Received B o n u s i n $ Sales in $

7 Measures of Association SALESPERSON SALES $ BONUS $ Advertising Sales in S The two variables must have the same number of observations (paired variables)

8 Measures of Association Covariance and Correlation Measures that summarize the strength of the linear relationship between the two variables numerically Cov(X,Y) = n i= 1 (X X)(Y Y) i n 1 i Corr(X,Y) = Cov(X,Y) ss X Y Excel: COVAR(data1,data2) Excel: CORREL(data1,data2) The two variables, say X and Y, must have the same number of observations (paired variables)

9 Measures of Association The advantage that the correlation has over covariance is that the correlation is always between -1 and +1. Corr(X,Y) = -1 negative linear relationship Corr(X,Y) = +1 positive linear relationship Corr(X,Y) = 0 no linear relationship All other values of correlation are judged in relation to these three values.

10 Measures of Association Covar(X,Y) = Corr(X,Y) = 0.91 Covar(X,Y) = Corr(X,Y) =

11 Measures of Association Covar(X,Y) = Corr(X,Y) = 0.20 Covar(X,Y) = Corr(X,Y) =

12 The Basic Idea Behind Regression Suppose you believe Salary is related to Education, Experience & Gender Measure of Association Corr (Salary, Education) Hypothesis testing Gender discrimination? Corr (Salary,Experience) Average Salary for Males versus Average Salary for Females 12

13 The Basic Idea Behind Regression If we believe Salary is related to Education, Experience and Gender, can we come up with some weighted sum of Education, Experience and Gender that will help us predict Salary with as little error as possible Salary = b0 + b1*education + b2*experience + b3*gender Goal of regression come up with the weight b0, b1, b2, b3 that will predict Salary with the least possible error using the variables Education, Experience and Gender Gender discrimination context Suppose b3 is significantly different from zero, then we can more conclusively say that there is possible gender discrimination, because we have separated out the effects of education and experience 13

14 Uses of Regression Describing and understanding relationships Sales = 20, Price Advertising Forecasting and predicting a new observation what will sales be if I change advertising or price? Adjusting the independent variables how much should I change advertising or price? 14

15 Back to Meddicorp Meddicorp management is concerned with the effectiveness of a new bonus program for its sales force There seems to be a relationship between sales and bonuses in 1999 But sales should be adjusted for other factors for example, if Meddicorp advertised heavily, then that would increase sales so sales should be corrected for advertising expenses to get the true effect of sales force performance 15

16 Meddicorp Regression Regression Analysis: SALES versus ADV, BONUS The regression equation is SALES = ADV BONUS T-Stats P-Value for 2-tailed test Predictor Coef SE Coef T P Constant ADV BONUS Analysis of Variance R-Sq = 85.3% R-Sq(adj) = 84.0% R-Square Source DF SS MS F P Regression Residual Error Total F-Test 16

17 What is the R-Square Here? Y Y Y X X X R-Sq = 1 R-Sq < 1 R-Sq << 1 17

18 And Here? Y Y Y X X X R-Sq = 1 R-Sq < 1 R-Sq << 1 18

19 What About Here? Y Y Y X R-Sq = 1 R-Sq = 1 X R-Square is meaningless X 19

20 What if R-Square = 0? Does it imply that there is no relationship between the variables? 20

21 No Linear Relationship Y There is a relationship between the variables but not a linear one: R-Sq = 0 X 21

22 Types of Non-Linear Relationships Y Y X X 22

23 No Relationship Between the Variables Y No Relationship R-Sq = 0 X (but R-Sq = 0 No Linear Relationship only!!) 23

24 Meddicorp More Variables Suppose Meddicorp believes that sales of a representative could be a function of: its market share; and the sales of the largest competitor in its territory What would be your research hypothesis for these variables? 24

25 Meddicorp Interpret Output Regression Analysis: SALES versus ADV, BONUS, MKTSHR, COMPET The regression equation is SALES = ADV BONUS MKTSHR COMPET Predictor Coef SE Coef T P Constant ADV BONUS MKTSHR COMPET Not significant R-Sq = 85.8% R-Sq(adj) = 82.9% Analysis of Variance Source DF SS MS F P Regression Residual Error Total

26 Meddicorp In Search for More Predictors Meddicorp wonders if sales could be better explained if they not only included current bonus, but also last year s bonus of the sales person 26

27 Including Last Year Bonus to Explain Sales Regression Analysis: SALES versus ADV, BONUS, LAST YR BONUS The regression equation is SALES = ADV BONUS LAST YR BONUS Predictor Coef SE Coef T P Constant ADV BONUS LAST YR No longer significant R-Sq = 85.4% R-Sq(adj) = 83.3% Analysis of Variance Source DF SS MS F P Regression Residual Error Total

28 Meddicorp Regression Regression Analysis: SALES versus ADV, BONUS The regression equation is SALES = ADV BONUS T-Stats P-Value for 2-tailed test Predictor Coef SE Coef T P Constant ADV BONUS Analysis of Variance R-Sq = 85.3% R-Sq(adj) = 84.0% R-Square Source DF SS MS F P Regression Residual Error Total F-Test 28

29 Multicollinearity Caused by high correlation between the RHS variables Correlations: SALES, ADV, BONUS, LAST YR BONUS SALES ADV BONUS ADV BONUS Too High LAST YR Cell Contents: Pearson correlation P-Value 29

30 Detecting and Correcting Multicollinearity Detecting Multicollinearity Look at Correlations Large F-Stat, but poor T-Stat Correcting Multicollinearity Keep only one of the highly correlated variables 30

31 More on Detecting Multicollinearity Look at Variance Inflation Factors (VIF) set this Option in Minitab when estimating regression look at largest VIF VIF=1 indicates no multicollinearity VIF >5 is considered very bad 31

32 Meddicorp Regression Regression Analysis: SALES versus ADV, BONUS The regression equation is SALES = ADV BONUS T-Stats P-Value for 2-tailed test Predictor Coef SE Coef T P Constant ADV BONUS Analysis of Variance R-Sq = 85.3% R-Sq(adj) = 84.0% R-Square Source DF SS MS F P Regression Residual Error Total F-Test 32

33 Correcting for Multicollinearity Regression Analysis: SALES versus ADV, BONUS, LAST YR BONUS The regression equation is SALES = ADV BONUS LAST YR BONUS Predictor Coef SE Coef T P VIF Constant ADV BONUS LAST YR S = R-Sq = 85.4% R-Sq(adj) = 83.3% Analysis of Variance Source DF SS MS F P Regression Residual Error Total Source DF Seq SS ADV BONUS LAST YR

34 Modeling Categorical Variables Meddicorp believes that sales are often a function of a region s attractiveness and bonus should correct for that is bonus related to sales beyond regional attractiveness? How to include regions in the regression? South, West, Midwest Create dummy variables 34

35 Including Dummies Regression Analysis: SALES versus ADV, BONUS, South, West, Midwest * Midwest is highly correlated with other X variables * Midwest has been removed from the equation The regression equation is SALES = ADV BONUS South West Predictor Coef SE Coef T P Constant ADV BONUS South West R-Sq = 94.7% R-Sq(adj) = 93.6% Can use only n-1 dummy variables if there are n categories; if you use all n it creates multicollinearity 35

36 Assessing the Importance of the Variables Standardized coefficient for Xi when regressed on Y standardized coeff = unstandardized coeff * stdv of Xi / stdv of Y standardized coeff are also called beta coefficient Compute in Excel if you like compute std deviation for regulars variables using the stdev(.) compute std deviation for dummy variables using the stdeva(.) Warning: Compute betas only for significant coefficients 36

37 Most Important Variables for Meddicorp Regression Analysis: SALES versus ADV, BONUS, South, West, Midwest The regression equation is SALES = ADV BONUS South West Predictor Coef SE Coef T P Constant ADV BONUS South West Beta Coef R-Sq = 94.7% R-Sq(adj) = 93.6% 37

38 Takeaways T-stats > 2 and p-values < 0.05 indicate significance at 5% level of a variable in a regression implies that the variable can explain variation in the dependent variable With large number of independent variables, don t put them all at one shot in the regression if there are high correlations among some variables, pick one of these for the regression to avoid multicollinearity If there are n categories in a categorical variable, use n-1 dummy variables Compute standardized (beta) coefficients to check importance of variables 38