GUIDE TO WEB-RESOURCES FOR MULTILEVEL MODELING AND

GUIDE TO WEB-RESOURCES FOR MULTILEVEL MODELING AND ANALYSIS 3rd Edition Revised September 2005 These resources have been put together by Kelvyn Jones, Myles Gould and SV Subramanian. It derives from the questions we are frequently asked when teaching introductory multilevel courses. The listing of resources is designed as an organized meta-site, whereby other resources for multilevel modeling on the web can be accessed. We list a specific resource only once, so you may have to search through the document to find what you want. To that end we have put it on the web as a single document and you can then use Find or Search facility on your browser to scan the entire document. General The best general site is the Centre for Multilevel Modelling at: http://www.mlwin.com/ The ESRC National Centre for Research methods is to be found at http://www.ncrm.ac.uk/ while the Lemma node of the NCRM (Learning Environment for Multilevel Methodology) is to be found at http://www.ncrm.ac.uk/nodes/lemma/about.php A powerpoint presentation on its scientific mission is given at http://www.ncrm.ac.uk/events/documents/learningenvironmentformultilevelmethodologyandapplications- JonRasbash.ppt The multilevel mailing list is also a key general resource as it is searchable; it represents many years of accumulated questions and answers: http://www.jiscmail.ac.uk/lists/multilevel.html Another vital resource is provided by the UCLA Academic Technology Services who maintain data and worked examples in a number of different software packages for a number of different multilevel textbooks: http://www.ats.ucla.edu/stat/mlm/default.htm Book and other downloads that accompany books A taster of Goldstein s classic text in its 3 rd edition on multilevel statistical models (Goldstein H, 2003, Multilevel statistical models, London, Arnold Publishers) is available at: http://www.ioe.ac.uk/hgpersonal/multmodels-edition3/multilevel_statistical_models-third_edition.htm A previous version of this classic can also be downloaded at http://www.ats.ucla.edu/stat/examples/msm_goldstein/default.htm Supplementary material for Tom Snijders and Roel Bosker textbook Snijders T, Bosker R, 1999 Multilevel analysis: an introduction to basic and advanced multilevel modeling, London, Sage, including updates and corrections, data sets used in examples, with set-ups for running the examples in MLwiN and in HLM, and an introduction to MLwiN) can be found at: http://stat.gamma.rug.nl/snijders/mlbook1.htm Supplementary material for Joop Hox s textbook Hox J, 2002, Multilevel analysis: techniques and applications, Mahwah, NJ, Lawrence Erlbaum can be found at: http://www.fss.uu.nl/ms/jh/mlbook/leabook.htm and also at http://www.ats.ucla.edu/stat/examples/ma_hox/default.htm The complete content of Hox J, 1995, Applied multilevel analysis, Amsterdam: TT-Publikaties can be downloaded at http://www.fss.uu.nl/ms/jh/publist/amaboek.pdf at Hox s website at: http://www.fss.uu.nl/ms/jh/publist/pubenjh.htm Version September 2005 multilevel 1

Supplementary material to Skrondal, A. and Rabe-Hesketh, S. (2004). Generalized Latent Variable Modeling: Multilevel, Longitudinal and Structural Equation Models. Chapman & Hall/CRC. can be found at http://www.gllamm.org/ Supplementary material to Garrett Fitzmaurice, Nan Laird, James Ware (2004) Applied Longitudinal Analysis Wiley is to be found at http://biosun1.harvard.edu/~fitzmaur/ala/ Supplementary material to Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence by Judy Singer and John Wilett (2003) Oxford University Press, can be found at: http://gseacademic.harvard.edu/~alda/ For those interested in the analysis of spatial data, there is Andrew B. Lawson, William J. Browne, Carmen L. Vidal Rodeiro (2003) Disease Mapping with WinBUGS and MlwiN, Wiley and it associated website http://www.maths.nott.ac.uk/personal/pmzwjb/dm.html To read a comparison of multilevel modeling with traditional approaches to running ANOVA, regression, and logistic regression with memories/events being "nested" within people/testing session see Wright DB, 1998, Modeling clustered data in autobiographical memory research: the multilevel approach, Applied Cognitive Psychology, 12, 339-357 at: http://www.sussex.ac.uk/users/danw/pdf/multil.pdf To keep up to date with developments in the field have a look at the downloadable Multilevel Newsletters: http://www.mlwin.com/publref/newsletters.html References to published work on multilevel modelling List of references to material that uses multilevel modeling can be found at Jung-Ho Yang s: http://128.248.232.90/archives/mchb/michep1999/handouts/multlev.pdf and the Centre for Multilevel Modelling has a very extensive and growing list of references: http://www.mlwin.com/publref/references.html Wolfgang Ludwig-Mayerhofer s annotated references on multilevel modeling: http://www.lrz-muenchen.de/~wlm/wlmmule.htm There is a structured list of references (based on different types of model) at the HLM website http://www.ssicentral.com/hlm/references.html#r7 Software in general If you want to compare the different packages that are available for multilevel modeling, detailed comparisons are being developed at: http://www.mlwin.com/softrev/index.html If you want to see how a particular model can be fitted in particular software, there are the developing resources at UCLA http://www.ats.ucla.edu/stat/mlm/default.htm For those wishing to analyze longitudinal data, software instructions in a wide range of programs is provided by UCLA to accompany the textbook Singer JD, Willett JB, 2003 Applied longitudinal data analysis: modeling change and event occurrence, New York, Oxford University Press, at: http://www.ats.ucla.edu/stat/examples/alda/ Version September 2005 multilevel 2

Training associated with software A growing amount of web-based (or at least down-loadable) training materials are being developed. We have chosen to organize this section by the particular software that is being used, and rather arbitrarily separated commercial software from the freeware that follows aml HLM MLwiN Mplus SAS SPSS Can be used to fit a range of multilevel models but has specific features for fitting multi-process or simultaneous equation models to hierarchical data where predictor variables may be non-random or endogenous, and other types of models used by economists such as a multilevel Heckman selection models: http://www.applied-ml.com/product/multiprocess.html The official site gives guidance at http://www.ssicentral.com/hlm/examples.html There is very good introductory material on how to set up the models by Information Technology Services at the University of Texas http://www.utexas.edu/its/rc/tutorials/stat/hlm/ Jason Newsom s Multilevel Regression course that uses HLM, but covers a lot of other ground too (eg Distinguishing between random and fixed: variables, effects, and coefficients; comparison of estimators, and kinks to SPSS Mixed) http://www.upa.pdx.edu/ioa/newsom/mlrclass/default.htm You can download a version of the software, data and training manuals from TRAMMS (Teaching Resources and Materials for Social Scientists): http://tramss.data-archive.ac.uk/software/mlwin.asp The manuals are a course in themselves http://www.mlwin.com/download/manuals.html The substantial enhancement of the MCMC procedures in MLwiN is discussed in full in 'MCMC Estimation in MLwiN' which is to be used with the version 2.0 of the program: http://www.mlwin.com/download/mcmcman20.pdf This software allows structural equation modeling, multilevel modeling and mixture modeling; the home site has training downloads and examples: http://www.statmodel.com/mplus/examples/webnote.html Judy Singer has a pdf download that shows how to fit multilevel models in PROC MIXED; it is very well written: http://gseweb.harvard.edu/~faculty/singer/ UCLA has implemented the Singer example in other software (eg R\Splus; HLM. MlwiN, SPSS): http://www.ats.ucla.edu/stat/paperexamples/singer/default.htm C.J. Anderson has a lot of material for his course online at: http://www.ed.uiuc.edu/courses/edpsy490ck/ Data and SAS related material are available for Applied Longitudinal Analysis by Garrett Fitzmaurice, Nan Laird, James Ware at http://biosun1.harvard.edu/~fitzmaur/ala/ The code and data to fit the models contained in SAS System for Mixed Models (1996) by RC Littell, GA Milliken, WW Stroup, and RD Wolfinger, is to be found at: http://ftp.sas.com/samples/a55235 A useful discussion of the Linear Mixed Models procedure in SPSS Advanced Models is to be found at: (search under Linear Mixed Effects): http://www.spss.com/downloads/papers.cfm while a HTML down-loadable tutorial based on a set of case studies is to be found at http://www.spss.com/downloads/papers.cfm?productid=00035&name=spss_base&dltype=de mo John Painter provides a clear guide on how to fit multilevel models using SPPS mixed http://www.unc.edu/~painter/ Another brief demonstration of SPSS Mixed in action is to be found at http://step.psy.cmu.edu/materials/spss/mixed.doc Version September 2005 multilevel 3

Freeware There are a number of programs that are available at low or nil cost; some of these are general (like R), others are more specific but can have special features that make them particularly attractive; we have tried to identify these special features below. We have also pointed to some appropriate training resources. BAYESX BUGS GeoBUGS GLLAMM MIX R has a number of distinctive features including handling structured (correlated) and/or unstructured (uncorrelated) effects of spatial covariates (geographical data) and unstructured random effects of unordered group indicators. It allows non-parametric relationships between the response and the predictors (generalized additive models) and does this for continuous and discrete outcomes, it can manipulate and display geographical maps: http://www.stat.uni-muenchen.de/~lang/bayesx/bayesx.html Bayesian inference Using Gibbs Sampling is really a flexible language that allows the fitting of a very wide range of models using MCMC methods; this is a very rich site developed by the MRC Biostatistics Research Unit in Cambridge which has lots of freely down-loadable software and detailed manuals http://www.mrc-bsu.cam.ac.uk/bugs A number of courses using BUGS have been put online, a listing is given at http://www.mrc-bsu.cam.ac.uk/bugs/weblinks/webresource.shtml Peter Congdon has written two books based around BUGS (Bayesian Statistical Modelling, and Applied Bayesian Modelling) data and programmes are available for both books at ftp://www.wiley.co.uk/pub/books/congdon/ is an add-on to BUGS that has been developed by a team at Imperial College to fit spatial models and produce a range of maps as output: http://www.mrc-bsu.cam.ac.uk/bugs/winbugs/geobugs.shtml this software usefully undertakes multilevel latent class and factor analysis, adapative quadrature to derive the full likelihood with discrete and normal response, and has facilities for fitting nonparametric models in which the distribution at the higher level can be non-normal (you need STATA to run this software; preferably STATA 8) ; this software is particularly useful for the models listed above, but can be slow to converge. This site also a rich one with growing number of downloads of lectures and papers showing how the approach can be used in practice http://www.gllamm.org/ These are a set of stand-alone programmes that fir a number of specific models including mixedeffects linear regression, mixed-effects logistic regression for nominal or ordinal outcomes, mixed-effects probit regression for ordinal outcomes, mixed-effects Poisson regression, and mixed-effects grouped-time survival analysis. They have a common interface, and importantly they calculate the likelihood directly so allowing comparison of the change in deviance for nested models. The are versions for Windows as well as for PowerMac and Solaris http://www.uic.edu/~hedeker/mix.html R is complete system for statistical computation and graphics, it can be seen as an Open Source implementation of the S language which in turn underlies the S-Plus software. It is distributed freely under the GNU General Public License and can be used for commercial purposes. It operates across a very wide range of platforms. The latest version and documentation can be obtained via CRAN, the Comprehensive R Archive Network http://cran.r-project.org/ R/ S Normal-theory models are fitted in R using lme and nlme functions described in full in Mixedeffects models in S and S-PLUS' by J. C. Pinheiro and D. M. Bates (2000), there is an additional support for this book at http://cm.bell-labs.com/cm/ms/departments/sia/project/nlme/ for discrete responses there is the function glmmpql which is discussed in the 4th edition of Modern applied statistics with S W. N. Venables and B. D. Ripley; the book also covers normal theory models; there is online support for the book at; http://www.stats.ox.ac.uk/pub/mass4/ Jeff Gill maintains a website that provides help, tutorials and references for those who want to use R http://psblade.ucdavis.edu/ Version September 2005 multilevel 4

Useful macros and other software PreML Diagnostics PINT OD DismapWin PROC TRAJ There is a very useful utility written so as to export an SPSS file into a MLwiN worksheet, it is down-loadable from Tom Snijders webpage: http://stat.gamma.rug.nl/snijders/preml.inc Tom Snijders homepage contains a set of MLwiN macros for producing diagnostics and for fitting a social network model http://stat.gamma.rug.nl/snijders/mlnmac.htm For determining appropriate required sample sizes and power in a two-level model; there is a manual http://stat.gamma.rug.nl/snijders/multilevel.htm#progpint is another program for power analysis and optimal design, it has excellent graphical output, you will need to read the published papers by Steve Raudenbush and Xiao-Feng Liu http://sitemaker.umich.edu/group-based a manual has now appeared at Steve Raudenbush s personal website http://www-personal.umich.edu/~rauden/ is a public domain software for the statistical analysis of epidemiological maps; it allows the analysis of unobserved heterogeneity using mixture models; the program offers a Poisson regression approach which links disease and exposure data http://www.medizin.fu-berlin.de/sozmed/dismapwin.html is a SAS procedure, written by Bob Jones, that fits a discrete mixture model to longitudinal data, and thereby implements Nagel s group trajectory model; a very useful site for this type of model with downloads of papers is http://www.andrew.cmu.edu/user/bjones/ Websites maintained by individuals Douglas Bates who developed the LME and NLME functions in R and S-plus has a website at: http://www.stat.wisc.edu/~bates/bates.html Bill Browne (who has made major contributions to the MCMC component of MLwiN) has a large number of down-loadable papers at: http://www.maths.nott.ac.uk/personal/pmzwjb/bill.html David Draper s home page has a lot of material about the Bayesian approach to hierarchical models: http://www.cse.ucsc.edu/~draper/ Tony Fielding has useful material on ordered categorical variables, endogeneity and instrumental variables including MLwiN macros, at http://www.economics.bham.ac.uk/people/fieldingt.htm Andrew Gelman has lots of downloadable papers and presentations on multilevel modelling with a strong Bayesian flavour http://www.stat.columbia.edu/~gelman/ Harvey Goldstein, who is the instigator of the MLwiN software has a number of down-loadable papers at his personal website: http://www.mlwin.com/hgpersonal/index.html Don Hedeker who has been behind the MIX set of programs has lecture transparencies and class notes on longitudinal analysis at: http://tigger.uic.edu/~hedeker/ Joop Hox s webpage has papers, programs and lectures to download at: http://www.fss.uu.nl/ms/jh/ Alastair Leyland s (who has extensively used multilevel modelling in public health) details are available at Version September 2005 multilevel 5

http://www.msoc-mrc.gla.ac.uk/staff/biography/leyland.html Bengt Muthen who is the developer of Mplus which allows multilevel factor analysis has a site at: http://www.gseis.ucla.edu/faculty/muthen/muthen3.htm Jason Newsom s multilevel page has discussion of topics like centering, and how to distinguish between fixed and random effects http://www.upa.pdx.edu/ioa/newsom/mlrclass/default.htm David Rogosa s hompage has useful link s to his course Education 260 on Popular Methods (including multilevel modeling,and causal inference) and Education 351 on Longitudinal analysis http://www.stanford.edu/~rag/ Jon Rasbash who has written most of the code for MLwiN has down-loadable papers at: http://www.mlwin.com/team/jon.html Steve Raudenbush s LAMMP website has publications and pre-prints and links to the projects he is currently working on: http://www-personal.umich.edu/~rauden/ Tom Snijders homepage: http://stat.gamma.rug.nl/snijders/multilevel.htm Neil Spencer s website includes MLwiN muticategory macros with additions for probit link, conditional mean scoring and instrumental variable estimation http://www.herts.ac.uk/business/staff_public/nhspencer_public/research/ Fiona Steele has a number of down-loadable papers particularly on multilevel event history analysis and multiprocessor variables at http://www.mlwin.com/team/fiona.html Subramanian s research papers on using multilevel models in social epidemiology and health as well training resources related to the concepts and application of multilevel statistical methods can be found at http://www.hsph.harvard.edu/faculty/svsubramanian.html Examples of multilevel modeling The First Oxford Research Methods Festival held in July 2004 has an a number of examples using multilevel modeling; one session is introductory and includes slides by Kelvyn Jones introduce the ideas, and a presentation by Ian Plewis shows that the level-1 individual coefficients can change substantially when a higherlevel context is taken into account http://www.ccsr.ac.uk/methods/festival/programme/fri/am/c/ In a more advanced session, Jon Rasbash shows the complexity of non-hierarchical models that can now be estimated for family and household structures, while Fiona Steel considers partnership transitions and fertility http://www.ccsr.ac.uk/methods/festival/programme/thu/pm/a/ For an interesting discussion about what multilevel models can (and cannot do) see the interchange at http://www.stat.columbia.edu/~cook/movabletype/archives/2005/01/brouhaha_about.html For the use of multilevel models in social network analysis, see http://stat.gamma.rug.nl/snijders/socnet.htm Tutorials in MCMC estimation MCMC estimation is increasingly being used to estimate complex models; there are number of sites with really helpful resources to get you started: Version September 2005 multilevel 6

Simon Jackman s Estimation and Inference via Markov chain Monte Carlo: a resource for social scientists: http://tamarama.stanford.edu/mcmc/ Jeff Gill s homepage is a mine of information in this area, it includes some down-loadable chapters from his 2002 book Bayesian Methods for the Social and Behavioral Sciences which is to be thoroughly recommended: http://psblade.ucdavis.edu/ There is a useful website for David Spiegelhalter, Keith Abrams and Jonathan Myles (2003) Bayesian approaches to clinical trials and health-care evaluation, Wiley; it contains downloads for the examples that use WinBugs and Excel worksheets that allow simple analysis of odds-ratio and hazard ratio models on the basis of normal priors and likelihoods http://www.mrc-bsu.cam.ac.uk/bayeseval Sujjit Sahu s tutorial on MCMC: http://www.maths.soton.ac.uk/staff/sahu/utrecht/ There is a lot of background material on MCMC in 'MCMC Estimation in MLwiN' which is an additional manual for version 2 of the software http://multilevel.ioe.ac.uk/download/manuals.html Harold Lehmann Bayesian Communication webpage prototypes Bayesian analysis on-line http://www.hopkinsmedicine.org/bayes/primarypages/index.cfm A Brief Introduction to Graphical Models and Bayesian Networks is to be found at: http://www.ai.mit.edu/~murphyk/bayes/bayes.html For software to determine sample-size requirements using prior opinion see Lawrence Joseph s Bayesian software site http://www.medicine.mcgill.ca/epidemiology/joseph/ To keep up to date in this area, you can visit the MCMC preprint service: http://www.statslab.cam.ac.uk/~mcmc/ Causal analysis General sites Christopher Winship s Counterfactual Causal Analysis in Sociology website provides a good introduction to developments in this area http://www.wjh.harvard.edu/~winship/cfa.html Harvard School of Public Health PROGRAM ON CAUSAL INFERENCE in Epidemiology and Allied Sciences http://www.hsph.harvard.edu/causal/index.htm Personal websites with material on causal analysis Judea Pearl s home-page has a large number of downloads of lectures and papers http://bayes.cs.ucla.edu/home.html Guido Imbens homepage http://ideas.repec.org/e/pim4.html David Harding s homepage http://www.wjh.harvard.edu/~dharding/ Version September 2005 multilevel 7

Software for causal analysis with observational data for R-based matching software which uses a wide range of techniques see Gary King s site http://gking.harvard.edu/matchit/ there is a SPPS syntax file for propensity scoring is available at John Painter s site http://www.unc.edu/~painter/spsssyntax/propen.txt facilities in R for Multivariate and Propensity Score Matching Software written by Jasjeet Sekhon http://jsekhon.fas.harvard.edu/matching/ and SAS program for propensity score matching is available at http://www.rx.uga.edu/main/home/cas/faculty/propensity.pdf and Stata programs for ATT estimation based on propensity score matching http://www.sobecker.de/pscore.html Multilevel and causal analysis The Columbia group on Bayesian statistics, multilevel modelling, causal inference, and social networks have a site at http://www.stat.columbia.edu/~sam/multilevelmodeling/ There are pre-prints and publications Steve Raudenbush s (search for causal) http://www-personal.umich.edu/~rauden/publication.htm Fiona Steele has a number of down-loadable papers on endogeneity and multi-processor models at http://www.mlwin.com/team/fiona.html Tony Fielding has material on endogeneity and instrumental variables including MLwiN macros, at http://www.economics.bham.ac.uk/people/fieldingt.htm Courses on multilevel modelling The multilevel website maintains a list of courses using MLwiN http://www.mlwin.com/support/workshop.html The Michigan-based Summer Program in Quantitative Methods of Social Research usually has one or more courses http://www.icpsr.umich.edu/training/summer/ There is a two-week long course by Kelvyn Jones and Myles Gould consisting of 10 half-days at The Essex Summer School http://www.essex.ac.uk/methods/ On a two year basis there is a week-long course by Kelvyn Jones and Subramanian at the Swiss Summer School (based in Lugano) http://www.unige.ch/ses/sococ/ss/ There is a unit at Brussels in December taught by Kelvyn Jones http://www.kubrussel.ac.be/english/index.htm Lemma will be providing introductory and advanced training on a regular basis http://www.ncrm.ac.uk/nodes/lemma/training.php Version September 2005 multilevel 8