The Big Picture: Four Domains of HLM. Today s Discussion. 3 Background Concepts. 1) Nested Data. Introduction to Hierarchical Linear Modeling (HLM)

Transcription

1 Center for dvanced Study of Teaching CSTL and Learning Introduction to Hierarchical Linear Modeling (HLM) The Big Picture: Four Domains of HLM Part 1 Specifying Equations Part I: Concepts and Equations Understanding Concepts Interpreting Results ndy Mashburn June 5-6, 2009 Language metaphor Running nalysis Part 2 Today s Discussion 3 Background Concepts Nested data Multiple levels of analysis Variance Specifying Equations Regression HLM 3 Background Concepts Examples in Educational Research 3 Background Concepts 1) Nested data 2) Multiple levels of analysis 3) Variance 1) Nested Data Data structure in which smaller units are clustered in larger units Prevalent in educational research Multiple l individuals id within multiple l settings 20 students within each classroom; 100 classrooms Multiple settings within multiple settings 50 schools within each district; 20 districts Multiple observations within multiple people t least 3 observations of each child; 300 children ny combination of these scenarios 1

2 2) Multiple Levels of nalysis What data do you have that are nested? Research with nested data: Can ask questions at multiple levels Can be analyzed at multiple levels Ex. SES chievement Definition: a descriptive stat; the average of the squared deviation scores about their mean Equation: s 2 = (X mean) 2 N 3) Variance COMPUTING VRINCE raw score mean Diffscore Difference score squared sum of difference score=600 average difference score=120 Summary: General conceptual premises of HLM 1. Nested data are prevalent in education research 2. Nested data can be analyzed at multiple levels 3. Nested data have multiple sources of variance Implications for nalysis 1. Equations for regression analyses only specify one source of variance 1. This loses information 2. Violates an assumption of the linear model Specifying Regression Equations 2. Logic of HLM 1. In equations, specify multiple sources of variance 2. Include predictors at multiple levels 2

3 3 Equations for Regression Equation 1: With one piece of information, Y, for each child, we specify the following equation: Y i = B 0 + r i B 0 = the average of Y for all children r i =the error term (variance σ2) Equations for Regression Equation 2: With two pieces of information (X and Y) for each child, we specify the following equation Y i = B 0 + B 1 (X1) + r i B 1 =the association between X and Y B 0 = the average of Y when X is zero r i =the error term (variance σ2) This equation can be represented as a line Linear Regression e competence = * age R-Square = 0.02 Yi = B0 + B1 (X1) + ri age competenc Regression with Nested Data The equation specifies one source of variance Nested data have multiple sources of variance nalysis of nested data at a single level only causes problems Setting-level analyses Problem: Lose information about within group variation, tend to find relations that may be different than at individual level Individual-level analyses Problem: Violation of assumption of independence of observations Specifying HLM Equations Logic of HLM Individuals within the same group are more similar than individuals in different groups Variation in Y has a group and an individual g p component HLM equations specify multiple sources of variance Predictors can be added at the group and individual level

4 Four Sets of Equations in HLM Estimating multiple sources of variance The Unconditional Model HLM Equation 1: Estimating Multiple Sources of Variance The Unconditional Model Setting effects on individuals Intercepts as Outcomes Cross-level interactions Slopes as outcomes Describing and predicting growth Combined: B 0j Grand mean Variability between classes Variability between children within classes HLM Equation 2: Setting Effects on Individuals HLM Equation 3: Cross-Level Interactions The unconditional model: B 0j Combined: The unconditional model: Combined: dd level 1 and level 2 predictors: B 0j + B 1j (child) B 0j + γ 01 (setting) j dd level 2 predictors: New Combined: B 0j + γ 01 (setting) j + γ 01 (setting) j Grand mean Coefficient for Variability between Variability between Setting predictor classes children within classes Specify between-classroom variability in B 1j B 1j = γ 10 + u 1j Predict between classroom variability in B 1j with setting-level predictor B 1j = γ 10 + γ 11 (setting) + u 1j Final Equations: + B 1j (child) B 0j + γ 01 (setting) j B 1j = γ 10 + γ 11 (setting) + u 1j HLM Equation 4 Representing Growth as a Straight Line Individual Growth 60 Linear Regression Example: 3 or more assessments of letter-word skills from beginning of PK to end of 1 st Grade LW Raw Fall 40 LW Raw Fall = * time R-Square = 0.90 This can be represented as a line: a straight line a squiggly line time 4

5 HLM Equation 4: Representing Growth as a Line HLM Equation 4: Multiple Sources of Variance in a Line Y ti = µ 0i + µ 1i (time) + e ti Y ti = µ 0i + µ 1i (time) + e ti µ 0i = the intercept of the line, the child s developmental status at time zero µ 1i =the slope of the line; the expected rate of change in development associated with each unit increase of time e ti =error term, unique effect associated with person, has a variance Slope of child s line Intercept of child s line Grand mean intercept µ 0i = B 00 + r 0i µ 1i = B 10 + r 1i Grand mean slope Children s variability around intercept Children s variability around slope dd child predictors to explain variance in level 2 intercept and slopes General pplications of HLM Educational Measurement School/Classroom/Teacher Effects on Individuals Examples in Educational Research chievement Gaps Children s and Teachers Development over Time Experimental Tests of Classroom Interventions Meta-analysis Examples in your field where HLM has been used? Examples in your field where HLM can be used? Break! 5

6 2 Specific Examples Study 1: Variance in Beer Preferences Study 2: Preschool Quality and Children s Development Variance in Beer Preferences Study 1 Study Method: Sample: bunch of people at my house Procedures: Brought a six pack of one of four categories of Beer Dark, Medium, Light, Specialty Setting up the rating process Measures: Rating on 1-4 scale for 4 dimensions Taste, Smell, Color, and Overall (range 4-16) Data: 198 Valid Tastes 38 Different Beers 4 Different Categories Study 1 Research Questions Study 1 Results Which beer has the highest rating? How much variance in beer ratings is there? Between tastes? Between beers? Between categories of beer? Which is the best beer? Between Beers Leinenkugel s Sunset Wheat was the best You can buy it at Food Lion and Giant Between Categories Specialty: 12.4 Light: 10.8 Medium: 10.7 Dark: 10.4 Within me I really liked the Lindeman Lambic Framboise But it s really expensive Study 1 Equations Three sources of Variance in Ratings Level 3: Y tbc = π 0bc + e ibc π 0bc = B 00c + r 0bc B 00c 0 + u 00c Y tbc 0 + e ibc +r 0bc +u 00c Variance Source Study 1 Results Estimated Variance Percentage of Variance ICC Within Beer (eibc) 8.5** 81.00% Between Beer (r0bc) 1.81* 17.20% Between Category (u00c) % Total

7 Study 1 Limitations Study 2 Confusing anchors to the beer rating scale No random assignment of tastes Selection bias The Ripski Effect Drunk grad student entered ratings His beer won He knowingly brought the beer that won the previous year Preschool Quality and Children s Development Study Method 2,439 4 year olds in 671 state pk classes Pre and post-test assessments of academic, language and social skills 3 assessments of quality NIEER index ECERS-R CLSS instructional support and emotional support Study 2: Research Question 1 Study 2: Equations To what extent is each of three methods of evaluating pre-k quality associated with children s academic, language and social development? B 0j + B nj (pretest, child characteristics) B 0j + γ 01 (Quality) + (other setting) Study 2 Results None of the nine indicators of high quality suggested by NIEER, individually or in combination, was associated with children s development The ECERS-R was associated with children s language development Instructional interactions predicted academic and language skills, emotional interactions predicted teacherreported social skills. Policies, program development, and professional development efforts that improve teacher-child interactions can facilitate children s school readiness. Summary 7

8 Conceptual Summary Nested data are prevalent in education research Nested data can be analyzed at multiple levels Nested data have multiple sources of variance Equation Summary Equations express specific hypotheses that can be represented as a line Regression analysis equations specify variance at only one level of analysis HLM equations can specify variance at multiple levels HLM equations can include predictors at different levels to account for different sources of variance around means and around slopes What if I don t use HLM with Nested Data? Giamartino & Wandersman (1983) Influence of organizational climate on work attitudes at individual level Florin, Giamartino, Kenny, & Wandersman (1990) Reanalysis using HLM Good Resources Raudenbush & Bryk (2002). Hierarchical Linear Models: pplications for Data nalysis Methods. Singer and Willett (2005). pplied Longitudinal Data nalysis: Modeling Change and Event Occurrence. Center for dvanced Study of Teaching CSTL and Learning Introduction to Hierarchical Linear Modeling (HLM) Part II: nalysis and Interpretation Running nalysis in HLM Software ndy Mashburn June 5-6,

9 Teacher and Classroom Characteristics ssociated with Teachers Ratings of Children s Competencies Method Sample: 711 children in 210 classrooms (~ 4 children per class) Measures: Child outcomes: spring teacher ratings of the child s social behaviors Child Problem Behaviors Child Social Competence Child-level predictors: Boy, age, mother s ed, poor Classroom/teacher level predictors: years experience, hours per day Study ims 1. Estimate within-teacher and betweenteacher variance in teachers ratings of children s competence 2. Examine associations between teachers ratings of children s competence and child and teacher/classroom characteristics Research Question 1: Estimating Multiple Sources of Variance Research Question 2: Child and teachers ratings of children s competence and child and teacher/classroom characteristics Combined: B 0j B 0j +B 1j (individual) B 0j + γ 01 (setting) Combined: + γ 01 (setting) + B 1j (individual) Grand mean rating Variability between raters--tau Variability within raters Sigma 2 Grand mean rating Coefficient for Class/teacher predictor Coefficient for child predictor Variability between raters (tau) Variability within raters (Sigma 2) 5 Step Process of Using Software 5 Step Process of Using HLM Software 1. ccess software 2. Structure SPSS files 3. Import SPSS files 4. Communicate with Software 5. Report and Interpret Results 1. ccess HLM software: 2. Structure SPSS files in the way that is appropriate for HLM: Child and setting characteristics are in separate data files Files are linked by a common setting-level ID 9

10 3) Import SPSS files into HLM 4) Communicate with Software 1. Choose Make new MDM File, Stat Package Input. Select MDM type dialog box opens 2. Select HLM2 and click OK. Make MDM - HLM2 dialog box will open. 3. Select SPSS/Windows from the Input File Type drop-down box. 4. Click Browse in the Level-1 Specification section to open an Open Data File dialog box. Open the level-1 SPSS data file. 5. Click Choose Variables to open up the Choose Variables - HLM2 dialog box. Choose the ID and variables by selecting the appropriate boxes. Click OK to return to the main dialog box. 6. Select options for missing data in this section. If you have missing data, select Yes and Making MDM. This is only for level-1. You cannot have missing data in the level-2 data file. 7. Select either measures within persons or persons within groups depending on your data structure. We have persons within groups, so make sure this is selected. 8. Click Browse in the Level-2 specification section to open an Open Data File dialog box. Follow the same process as you did for the level-1 file. 9. Enter a name for the MDM file in the MDM File Name box (upper-right hand corner of dialog box). The file name must end with the extension.mdm. 10. Click Save mdmt file in the MDM template file section to open a Save MDM template file dialog box. Enter a file name. HLM will add the.mdmt extension for you. This file will contain all of your input information. 11. Now click on Make MDM button. screen with descriptive statistics will appear briefly while HLM is merging the two files. When it goes away click on Check Stats to display your level-1 and level-2 descriptive statistics. Unconditional Model Identify level-1 outcome variable Set Basic Model Settings Review Other Settings Run nalysis View Output 5) Report and Interpret Results Research Question 1 4) Communicate with Software Intercept γ00 Total Variance U0j (tau) + rij (sigma 2) Between Rater Variance U0j (tau) ICC U 0j (tau) / u 0j (tau) + rij (sigma 2) dd Level 1 and Level 2 predictors Choose centered or uncentered Teacher-Child Rating Scale Competence Problem Behaviors Run nalysis View Output 5) Report and Interpret Results Research Question 2 Child Characteristics ge Boy Mother s education (years) Family is poor Percentage of variance (σ 2 ij ) explained Teacher and Classroom Characteristics Years of experience Length of school day (hours) Percentage of variance (τ 0j ) explained Competence Problem Behaviors B SE B SE Homework Download student version of HLM Find a study with a nested data structure Prepare the data file for use with HLM 10

11 Tomorrow Review of Concepts and Equations Group project using HLM State research questions Specify equations Import data to HLM Run HLM Interpret Results Demonstrate SS Proc Mixed 11