Sex differences on the WISC-R in Belgium and The Netherlands

Transcription

1 Available online at Intelligence 36 (2008) Sex differences on the WISC-R in Belgium and The Netherlands Sophie van der Sluis a,, Catherine Derom b, Evert Thiery c, Meike Bartels a, Tinca J.C. Polderman a,d, F.C. Verhulst d, Nele Jacobs c, Sofie van Gestel c, Eco J.C. de Geus a, Conor V. Dolan e, Dorret I. Boomsma a, Danielle Posthuma a a Department of Biological Psychology, VU University Amsterdam, Van der Boechorststraat 1, 1081 BT Amsterdam, The Netherlands b Department of Human Genetics, University Hospital Gasthuisberg, Katholieke Universiteit Leuven, B-3000 Leuven, Belgium c Association for Scientific Research in Multiple Births, B-9070 Destelbergen, Belgium d Department of Child and Adolescent Psychiatry, Erasmus MC- Sophia, Dr. Molewaterplein 60, 3015 GJ, Rotterdam, The Netherlands e Department of Psychology, FMG, University of Amsterdam, Roeterstraat 15, 1018 WB, Amsterdam, The Netherlands Received 6 May 2006; received in revised form 15 January 2007; accepted 17 January 2007 Available online 22 February 2007 Abstract Sex differences on the Dutch WISC-R were examined in Dutch children (350 boys, 387 girls, age years) and Belgian children (370 boys, 391 girls, age years). Multi-group covariance and means structure analysis was used to establish whether the WISC-R was measurement invariant across sex, and whether sex differences on the level of the subtests were indicative of sex differences in general intelligence (g). In both samples, girls outperformed boys on the subtest Coding, while boys outperformed girls on the subtests Information and Arithmetic. The sex differences in the means of these three subtests could not be accounted for by the first-order factors Verbal, Performance, and Memory. Measurement invariance with respect to sex was however established for the remaining 9 subtest. Based on these subtests, no significant sex differences were observed in the means of the first-order factors, or the second-order g-factor. In conclusion, the cognitive differences between boys and girls concern subtest-specific abilities, and these sizeable differences are not attributable to differences in first-order factors, or the second-order factor g Elsevier Inc. All rights reserved. Keywords: Intelligence; Sex-differences; Multi-group covariance and mean structure analysis; Measurement invariance 1. Introduction Sex differences on the WISC-R have been studied in the WISC-R standardization samples of the USA, Scotland, The Netherlands, and China, and in data from Mauritius, New Zealand, and Belgium (e.g., Born & Lynn, 1994; Dai & Lynn, 1994; Grégoire, 2000; Jensen & Reynolds, 1983; Lynn & Mulhern, 1991; Corresponding author. address: s.van.der.sluis@psy.vu.nl (S. van der Sluis). Lynn, Riane, Venables, Mednick, & Irwing, 2005). The results are largely comparable across countries. Consistently, large differences favoring girls are reported regarding the subtest Coding (effect sizes about.5), and large differences favoring boys are reported regarding the subtest Information (effect sizes about.35). In addition, girls sometimes outperform boys on the subtest Digit Span, but these differences are usually small and statistically insignificant. Boys score slightly higher than girls on all other subtests, and even though these differences are sometimes statistically significant, /$ - see front matter 2007 Elsevier Inc. All rights reserved. doi: /j.intell

2 S. van der Sluis et al. / Intelligence 36 (2008) the differences are often small, with effect sizes ranging between.00 and.20. In all these studies, WISC-R subtest scores and factor scores have been compared directly between boys and girls. Yet it has never been established whether the factor structure of the WISC-R is actually comparable or measurement invariant across sex (see below). The interpretation of group differences in subtest- or factors scores may be complicated greatly if the underlying factor structure differs between the groups. That is, if a test battery does not measure the same construct(s) in different groups, then group differences in test scores representing first or higher order factors are difficult to interpret. The aim of the present study is to find out whether the WISC-R is measurement invariant across sex in children before comparing subtest and factor scores between boys and girls. The factor structure underlying the WISC-R has been studied in clinical and non-clinical samples (e.g., Anderson & Dixon, 1995; Burton et al., 2001; Donders, 1993; Huberty, 1987; Kush et al., 2001; Meesters, van Gastel, Ghys, & Merckelbach, 1998; Wright & Dappen, 1982). Principal component analyses (PCA, e.g., Born & Lynn, 1994; Lynn & Mulhern, 1991; Rushton & Jensen, 2003), exploratory factor analyses (EFA, e.g., Dolan, 2000; Dolan & Hamaker, 2001; Kush et al., 2001), and confirmatory factor analyses (CFA, e.g., Burton et al., 2001; Dolan, 2000; Dolan & Hamaker, 2001; Keith, 1997; Kush et al., 2001; Oh, Glutting, Watkins, Youngstrom, & McDermott, 2004) have yielded either a two factor ( Verbal and Performance ), or a three factor solution ( Verbal, Performance, and Memory, also known as Freedom from distractibility ). In these models, general intelligence ( g ) was either operationalized as the first principal component (PCA), or as a second-order factor (CFA). Given the assumption that these latent factors underlie the performance on the level of the subtests, one question of interest is whether the observed sex differences at the level of the subtests are a function of differences in g,or of differences on the level of the broad primary factors of intelligence (e.g., Verbal intelligence, Performance intelligence and Memory). However, it may also be the case that the subtest differences are not attributable to common factor differences, but rather are a manifestation of differences in the specific ability that the subtest taps. If boys and girls differ with respect to the mean on a given subtest, and this difference cannot be explained by the mean differences on the latent factor, which is supposed to underlie performance on the subtest, then the subtest may be viewed as biased with respect to sex. The term bias does not imply that the observed mean difference is not real, rather the term, as used here, implies that the mean difference on the subtest is greater or smaller than that expected on the basis of the latent factor mean difference. According to this definition, the term bias refers to the subtest as an indicator of the common factor, which the subtest is supposed to measure. For example, it has been established that the Information subtest of the WAIS is biased with respect to sex. Specifically, the male advantage on this subtest, which is supposed to measure general knowledge, is too large to be accounted for by the common factor Verbal Comprehension (e.g., Dolan et al., 2006; Van der Sluis et al., 2006). The difference is not indicative of a difference with respect to Verbal Comprehension. However, it may well be indicative of a true male advantage in general knowledge. Establishing the exact nature of an observed (subtest) mean difference is important in the light of theories, in which sex differences are attributed to latent mean differences (e.g., a difference in Verbal Comprehension, or a difference in g). In previous studies aimed at identifying the source(s) of the sex differences, PCA was mostly used to investigate sex differences on the factors underlying intelligence. Sex differences were evaluated by calculating weighted linear combination of the subtests means, where the subtests factor loadings served as weights (e.g., Born & Lynn, 1994; Jensen & Reynolds, 1983; Lynn, Fergusson, & Horwood, 2005; Lynn & Mulhern, 1991; Lynn, Riane, et al., 2005). The general finding of these studies is that boys score higher on the Verbal and Performance factors, while girls score higher on the Memory factor. With respect to general intelligence, operationalized as the first principal component, boys usually score higher than girls, but effect sizes are often small (about.10), and the difference is not always statistically significant. When expressed on the conventional IQ-scale with a mean of 100 and standard deviation of 15, these sex differences range from 1 to 6 IQ points (e.g., Lynn, Fergusson, et al., 2005; Lynn, Riane, et al., 2005). All these results are however based on samples with a broad age-range (6 16 years), and it remains to be seen whether the factor structure of the WISC-R, and the effects reported for the (factor) means, are stable across age. One obvious problem concerning this PCA-based method of studying sex differences is that sex differences on the level of the weighted means of the observed subtest scores may be due to one or just a few of many subtests. For example, boys may outperform girls on the Verbal factor only because they outperformed girls on the subtest Information, while their performance on the other verbal subtests may even be inferior. In that case, it

3 50 S. van der Sluis et al. / Intelligence 36 (2008) is more accurate to conclude that boys outperform girls with respect to a specific cognitive ability, i.e., general knowledge, rather than suggesting that boys have higher Verbal intelligence. Stated succinctly, this method does not explicitly address the structure of the observed mean differences. The use of PCA to study group differences is however characterized by several other disadvantages. Because PCA, in contrast to EFA or CFA, is based on a data transformation rather than an explicit statistical model, it does not generally include statistical testing or explicit model fitting. As a consequence, goodness of fit is not evaluated, i.e., the question of whether the model provides a reasonable description of the structure of the data is not addressed. In addition, an explicit statistical procedure to test whether the factor structure underlying the given test battery is comparable across groups is not conducted. The comparability of the factor structure across groups is however of utmost importance if one wishes to make meaningful comparisons between the subtest scores or factor scores of different groups. Furthermore, within the context of PCA, competing hypotheses are not compared statistically (e.g., are sex differences present on the level of the primary factors of intelligence or rather on the level of the observed subtests only?). Finally, PCA is not suited for modeling measurement error in the subtest scores, which is sure to exist. An alternative method for testing group differences within the context of factor models is multi-group covariance and means structure analysis (MG-CMSA; Sörbom, 1974; Little, 1997; Widaman & Reise, 1997). This method provides a comprehensive, model-based means to investigate the main sources of group difference. MG-CMSA allows one to evaluate and compare the fit of alternative models, which correspond with competing hypotheses. The advantages of MG- CMSA have been studied and discussed in detail, and MG-CMSA has repeatedly been shown to be superior to other methods used to study group difference (e.g., method of correlated vectors, the Schmidt Leiman procedure, PCA) with regard to, among things, its flexibility and the facility to test (competing) hypotheses explicitly (see e.g., e.g., Dolan, 2000; Dolan & Hamaker, 2001; Dolan, Roorda, & Wicherts, 2004; Lubke, Dolan, & Kelderman, 2001; Lubke, Dolan, Kelderman, & Mellenbergh, 2003; Millsap, 1997). Previously, MG-CMSA was used to study ethnic group difference in intelligence (e.g., Dolan, 2000; Dolan & Hamaker, 2001; Dolan et al., 2004; Gustafsson, 1992), the Flynn-effect (Wicherts et al., 2004), and sex differences on the WAIS (Dolan et al., 2006; Van der Sluis et al., 2006). In the present study, we used MG-CMSA to investigate sex differences on the Dutch WISC-R in Dutch and Belgian children of limited age-range (9 13 years old). Below, we outline the MG-CMSA modeling procedure that we used to investigate the sources of sex differences on the WISC-R. Both firstand second-order factor models are fitted, with the second-order factor representing g. The results are presented for Dutch and Belgian subjects separately, and are discussed in the light of previous studies. 2. Method 2.1. Subjects Dutch sample The Dutch data constitute a combination of two datasets: data that were previously used in a study of the genetic and environmental contributions to the development of individual differences in intelligence (Bartels, Rietveld, van Baal, & Boomsma, 2002), and data that were used to establish the extent to which the phenotypic correlation between working memory speed and capacity is of genetic origin (Polderman et al., 2006). All Dutch subjects were recruited from the young Netherlands Twin Register (NTR, Boomsma, 1998; Boomsma et al., 2002; Bartels, Beijsterveldt, Stroet, Hudziak, & Boomsma, in press). Since 1986, the majority of parents with multiple births in The Netherlands receive a brochure and a registration form from the NTR. Registration is voluntary, and about 40% of the parents register their twins with the NTR. Information from questionnaires, blood group, and DNA polymorphisms (genetic markers) was used to assign zygosity to same-sex twins (Rietveld et al., 2000). For this study, data were available from 368 twin pairs (77 monozygotic male pairs, 100 monozygotic female pairs, 67 dizygotic male pairs, 62 dizygotic female pairs, and 62 opposite sex twins), and, due to missingness, one single twin. As in most of the twin studies, the percentage of MZ twins in this sample (48%) is somewhat higher than in the overall population ( 33%) due to self-selection bias. The sample included 350 boys and 387 girls (737 subjects in total). For all twins in this study, level of parental occupation was assessed at age 10 of the twins. Occupational level was rated on a 5-point scale, ranging from manual labor to academic employment. Paternal occupational level was used, or maternal occupational level in case paternal information was not available. In comparison to the Dutch population (Centraal Bureau voor de Statistiek, 2002), the level of occupation of the

4 S. van der Sluis et al. / Intelligence 36 (2008) parents of the twins participating in this study was somewhat higher: the percentages observed in the present study and the Dutch population are: 1% and 6% (manual), 15% and 26% (lower), 42% and 40% (middle), 30% and 19% (higher), and 11% and 9% (academic). Paternal and maternal educational level did not differ between the boys and girls in this sample (z=.19, ns, and z=.20, ns, respectively). With the exception of one twin pair aged 10.9 years old, the age of all subjects ranged between 11.9 and 12.9 years at the time of testing. The youngest twin pair at 10.9 years old was not removed from the sample as it did not constitute an outlier in any aspect. Sex differences with respect to age were absent (t(735) b 1, ns) Belgian sample The Belgian subjects were recruited from the East Flanders Prospective Twins Survey (EFPTS), a population-based register of twins in the province of East Flanders, Belgium (Derom et al., 2002; Loos, Derom, Vlietinck, & Derom, 1998). Since 1964, EFPTS collects information on the mother, the placenta and the child of 98% of all multiples born in the province. Zygosity of all twins was determined through sequential analysis based on sex, foetal membranes, umbilical cord blood groups (ABO, Rh, CcDEe, Mnss, Duffy, Kell), placental alkaline phosphatase and, since 1982, DNA fingerprints. Unlike-sex twins and same-sex twins with at least one different genetic marker were classified as DZ; monochorionic twins were classified as MZ. For all same-sex dichorionic twins with the same genetic markers a probability of monozygosity was calculated. All subjects, whose data are used in the present study, participated in an ongoing study on cognitive ability in twins aged 7.5 to 15 years old. This sample was shown to be representative for gender, birth weight, and gestational age. As in most of the twin studies, the MZ twins were slightly over represented (42%) due to self-selection biases. Comparison of the 663 twins with known IQ scores with the twins who refused to participate in the study (n=204) revealed that, in the non-participating group, parents with a lower educational level and twins who attend special schools tend to be over represented. Part of the data (only complete same-sexed twin pairs with known IQ scores) was previously used to study the effect of chorion-type on the estimation of the heritability of intelligence (Jacobs et al., 2001). The present dataset comprises a subsample (agerange between 9.5 and 13 years at time of measurement) of the above-mentioned study. The sample consisted of 370 boys and 391 girls (761 subjects in total). Data were available from 374 complete twin pairs (83 monozygotic male pairs, 76 monozygotic female pairs, 44 dizygotic male pairs, 63 dizygotic female pairs, and 108 opposite sex twin pairs), and 13 single twins. Sex differences with respect to age were absent (t(759) =1.73, ns) Tests In both Dutch and Belgian samples, psychometric IQ was measured with the following 12 subtests of the Dutch WISC-R (Van Haasen et al., 1986): Information (INF), Similarities (SIM), Arithmetic (AR), Vocabulary (VOC), Comprehension (COMP), Picture Completion (PC), Picture Arrangement (PA), Block Design (BD), Object Assembly (OA), Mazes (MA), Coding (CO), and Digit span (DS). In the Belgian data missingness was absent. The Dutch data, in contrast, included systematic missing data. Specifically, for reasons of efficiency, only 6 out of 12 subtests (namely SIM, AR, VOC, BD, OA, and DS) were administered to the 354 Dutch subjects (165 boys, 189 girls) who previously participated in the study by Polderman et al. (2006). Some additional missingness occurred due to procedural errors, but this percentage was very small (.2%). Due to this systematic omission of subtests in part of the Dutch sample, missingness can not be considered completely at random (MCAR, e.g., Schafer & Graham, 2002) in the total Dutch sample (p b.01 for Little's MCAR test performed across families and for boys and girls separately). In the following exploratory and confirmatory factor analyses, raw data Maximum Likelihood estimation was employed to accommodate the missingness, and use all available data. Raw data ML estimation has been found to provide better parameter estimates than conventional methods, such as listwise or pairwise deletion and mean imputation, even if data are not missing (completely) at random (e.g., Tomarken & Waller, 2005) Statistical analyses Measurement invariance and model fitting strategies To study sex differences in the means and covariances within the common factor model, multi-group confirmatory covariance and means structure analysis (MG- CMSA) was used. Before sex differences with respect to the latent common factors can be examined, we first need to establish whether the WISC-R is measurement invariant with respect to sex. Measurement invariance with regard to sex implies that the distribution of the observed scores on a subtest (y i ) given a fixed level of the

5 52 S. van der Sluis et al. / Intelligence 36 (2008) latent factor (η), depends on the score on the latent factor η, and not sex, i.e., f [ y i η,sex]=f [ y i η] (Mellenbergh, 1989). Given normally distributed data, measurement invariance can be defined in terms of the means and the variances of the y i given η. With respect to the means, measurement invariance implies that the expected value of subjects i on subtest y depends only on the latent factor score η, and not on sex, i.e., E[y i η,sex]=e[y i η]. Within the common factor model, to establish measurement invariance one needs to establish whether the relation between the observed subtest scores and the underlying latent factors is the same in boys and girls (Meredith, 1993). Measurement invariance can be established through the imposition of a series of specific constraints on the model parameters over groups, i.e., across sex in this case (Meredith, 1993). First of all, the subtests should load on the same factors in both boys and girls, i.e., the measurement model should be the same in both sexes (also called configural invariance). Subsequently, the function relating the observed subtest scores to the latent factors can be considered identical for boys and girls if the following parameters can, to reasonable approximation, be considered equal across sex: a) the factor loadings of the observed subtests on the latent factors, b) the intercepts (note that the factor means are allowed to differ across groups), and c) the residual variances, i.e., the variance in the observed subtest scores that is not explained by the latent common factors. If these constraints prove tenable, the WISC-R may be considered to be measurement invariant with respect to sex, and in that case, individual differences and group differences on the level of the observed subtests can be interpreted in terms of differences on the common factors. The model fitting strategy that follows from the above described equality constraints is described in detail in Van der Sluis et al. (2006). Below we give an overview of this model fitting procedure, and we refer to Appendix A of Van der Sluis et al. (2006) for a description of this procedure in matrix notation First-order factor models First, we fitted the least constrained model, model F 1, that tests for configural invariance (Horn & McArdle, 1992; Widaman & Reise, 1997). Configural invariance implies that the configuration of factor loadings (and correlated residuals, if any) is the same across sex, but the exact values of these parameters are allowed to differ across groups. In this model, the observed means of the 12 subtests are estimated freely in boys and girls, i.e., we do not yet introduce a constrained model for the mean structure. Note that in model F 1, we fixed the variances of the latent factors to 1 in both boys and girls. This is a standard identifying constraint in factor analysis (e.g., see Bollen, 1989). Subsequently, we tested for metric invariance (Horn & McArdle, 1992; Widaman & Reise, 1997) by constraining the factor loadings to be identical across sex. We denote this model F 2. Identical factor loadings are a prerequisite for a meaningful comparison between boys and girls with respect to the latent common factors, i.e., if the factor loadings of the subtests on the latent factors are not identical across sex, we cannot be sure that the latent factors are identical, and thus comparable, across sex. If the constraints introduced in model F 2 do not result in a significant deterioration of the model fit compared to model F 1, metric invariance is considered tenable. Note that these equality constraints on the factor loadings render fixation of the factorial variance in both group superfluous, so in model F 2, the factor variances remain fixed to 1 in the boys, but are estimated freely in the girls. Next, we test for strong factorial invariance (Horn & McArdle, 1992; Meredith, 1993; Widaman & Reise, 1997) by introducing a restrictive structure for the means. We denote this model F 3. In model F 3, the intercepts in the regression of the observed variables on the common factors are constrained to be equal in boys and girls, while the means of the factors are estimated. We thus introduce a constrained model for the means structure. Note that for reasons of identification, it is not possible to estimate the factor means in both groups (Sörbom, 1974). We chose to fix the factor means to zero in the boys, and estimated freely in the girls. Modeled as such, the boys function as a reference group, and the factor means in the female group are calculated as deviations from the factor means of the boys. If the fit of model F 3 is not significantly worse than the fit of model F 2, the assumption that the expected values of the observed subtest scores depend not on sex but only on the latent factor scores, is considered tenable, i.e., E [ y i η,sex]=e[ y i η]. Model F 3 thus embodies the test whether the latent common factors can account for the observed mean differences between boys and girls on the level of the subtests. Boys and girls can be compared meaningfully with respect to their first-order factors means only if model F 3 holds. If model F 3 is not tenable, one or more of the mean differences between boys and girls on the level of the observed subtest scores cannot be accounted for by the first-order factors. As explained above, subtests are considered biased with respect to sex if observed difference on these tests cannot be attributed to differences on the level of the primary factors of intelligence. We next test for strict factorial invariance (Horn & McArdle, 1992;Meredith, 1993; Widaman & Reise, 1997) by constraining the residual variances to be equal across sex. We denote this model F 4. If model F 4 is

6 S. van der Sluis et al. / Intelligence 36 (2008) tenable, in comparison to model F 3, we conclude that all differences between boys and girls with respect to the means and the covariance structure can be accounted for by sex differences in the first-order factors. Note that the tenability of model F 4 is not a prerequisite for the comparability of boys and girls with respect to the observed means, or with respect to the means of the firstand second-order latent factors. To this end, F 3 suffices Second-order factors A second-order factor (model S 1 ), i.e., the model including general intelligence g as a second-order factor, was introduced in either model F 3 or F 4, depending on the tenability of the constraints introduced in model F 4. Depending on the number of first-order factors, this hierarchical factor model is either equivalent to the first-order factor model (in the case of 3 firstorder factors), or it tests whether all relations between the first-order factors can be explained by 1 secondorder factor (in the case of 4 or more first-order factors). At this point, the second-order factor loadings, i.e., the loadings of the first-order factors on the second-order factor, are allowed to differ across sex, and the means of the second-order factors are fixed to zero in both groups, while the first-order factor means are fixed to zero in boys, and freely estimated in girls (as in models F 3 and F 4 ). For reasons of identification, the variance of the second-order factor is fixed to 1 in both groups. In model S 2, the second-order factor loadings are constrained to be equal across sex. Like in model F 2, these equality constraints on the factor loadings allow one to freely estimate the variance of the second-order factor in one of the groups (in our case the girls). With model S 2 we thus test whether the factor loadings of the first-order factors on the second-order factor are equal in boys and girls. In model S 3, the first-order factor means are constrained to be equal across sex. Given our present parameterization, this involves fixing the first-order factor means differences to zero in the girls. The second-order factor mean is then fixed to zero in the boys, and estimated freely in the girls (analogous to model F 3 ). In this model, the mean difference between boys and girls are described entirely in terms of mean differences on the second-order factor, i.e., in g. IfmodelS 3 is tenable (in comparison to model S 2 ), we conclude that the mean differences between boys and girls on the level of the observed subtests can be accounted for completely by differences in g. If model S 3 does not fit the data, we conclude that the differences between boys and girls at the level of the first-order factors are not (or not completely) attributable to difference between boys and girls in g. In the final model, model S 4, we constrain the second-order factor means to be equal across sex (i.e., we fix the second-order factor mean difference to zero). If model S 4 fits as well as model S 3, we conclude that boys and female do not differ with respect to g. A significant deterioration of the fit as a result of this constraint indicates the presence of sex differences in g Estimation and model fit Both the Dutch and Belgian data were gathered within families. The focus of this paper, however, is on gender differences, and not on the correlations among family members, which are undoubtedly present as cognitive abilities are known to be quite heritable (e.g., Bartels et al., 2002; Daniels, Devlin, & Roeder, 1997; Posthuma et al., 2002). Treating within-family data as if they are independently distributed observations results in incorrect standard errors and incorrect χ 2 goodness of fit values, while the point estimations of parameter estimates remain unbiased (e.g., Rebollo, de Moor, Dolan & Boomsma, 2006). All factor analytic analyses were therefore performed in Mplus, version 4 (Muthén & Muthén, 2005), which computes corrected standard errors and Satorra Bentler scaled χ 2 -tests, taking into account the dependence of observations. Competing hypotheses, represented by different nested models (where the nested model is the more restricted model), can be compared through a weighted χ 2 -difference test developed especially for the comparison of the Satorra Bentler scaled χ 2 s(satorra, 2000). The more restricted model is accepted as the preferred model, if its fit is not significantly worse than the fit of the less restrictive model, i.e., if the χ 2 -difference test (henceforth the χ 2 diff ) is not significant. Below, we will not report scaled χ 2 - values for each model separately, as these are not 2 informative, rather we report weighted χ diff tests for the comparison between competing models. Given the large sample sizes, and the number of tests that were required to compare all ensuing models, we chose an α of.01. To evaluate the fit of the ensuing models to the data, the root mean square error of approximation (RMSEA), and the comparative fit index (CFI) were used (e.g., Bentler, 1990; Bollen & Long, 1993; Jöreskog, 1993; Schermelleh-Engel, Moosbrugger, & Müller, 2003). The RMSEA is a measure of the error of approximation of the covariance and mean structures as implied by the specified model to the covariance and mean structures in the population. As a measure of approximationdiscrepancy per degree of freedom, this fit index favors more parsimonious models. Generally, good fitting models are thought to have RMSEA b.05, although simulation studies by Hu and Bentler (1999) showed

7 54 S. van der Sluis et al. / Intelligence 36 (2008) that a cut-off criterion of.06 can be used as well. Here we adopt the following rule of thumb: a RSMEA of.05 or less indicates good approximation, RMSEA between.05 and.08 indicates reasonable approximation, and RMSEA greater than.08 indicates poor approximation (Browne & Cudeck, 1993; Schermelleh-Engel et al., 2003). The CFI is based on the comparison of the fit of the target model (i.e., the user-specified model) with the fit of the independence model (i.e., a model in which all variables are modeled as unrelated). Like the RMSEA, the CFI favors more parsimonious models. CFI ranges from zero to 1.00, and values N.90 or.95 are usually taken as indicative of adequate model fit (e.g., Hu & Bentler, 1999; Schermelleh-Engel et al., 2003). When testing for measurement invariance, the scaled χ 2 statistic was used to compare the fit of the competing models, while the RMSEA and the CFI were used only to check that the general fit of the ensuing models was still acceptable. In addition, modification indices were used to detect local misspecifications in the models. The modification index of a constrained parameter (i.e., fixed to a given value or subject to an equality constraint) expresses the expected drop in overall χ 2, if the constraint on the parameter is relaxed. 3. Results All analyses were performed on the standardized subtest scores, which have a mean of 10 and SD of 3 in the population Preliminary analyses Table 1 contains means and standard deviations of all 12 standardized subtest scores, reported separately for Dutch and Belgian boys and girls. As a measure of effect size, Cohen's d is also reported, which is calculated as the difference between the mean of the boys and the girls (μ boys μ girls ) divided by the pooled standard deviation, so that positive (negative) d's denote male (female) advantage. Most effect sizes were small (b.3 ), and medium effect sizes (between.3 and.6 ) were only observed with respect to INF and AR (favoring boys) and CO (favoring girls). It is possible that the differences between boys and girls are indicative of differences between families in, for example, socioeconomic status (SES), rather than of genuine sex differences. It is impossible to measure all variables on which families might differ, but comparing boys and girls who grew up in the same family environment provides a powerful check of the possible influence of family background. Opposite sex twin pairs are therefore of special interest. If the sex differences that are observed across families remain significant within families, then these differences are more likely to represent real differences between boys and girls. However, if these between family differences diminish, or even disappear, within families, the between family differences are more likely to relate to environmental differences. (Note that the opposite twin design does not imply perfect matching of boys and girls; e.g., Table 1 Means (M) and standard deviations (SD) for the Dutch and Belgian boys and girls on the 12 WISC-R subtests Netherlands Belgium Boys Girls Boys Girls M SD N M SD N d M SD N M SD N d INF SIM AR VOC COMP PC PA BD OA CO MA DS Scores are standardized scores (in norm sample, M=10, SD=3). Note. d is Cohen's measure of effect size d, defined as (M boys M girls /σ pooled ). INF =Information, SIM =Similarities, AR= Arithmetic, VOC= Vocabulary, COMP =Comprehension, PC =Picture Completion, PA =Picture Arrangement, BD=Block Design, OA=Object Assembly, MA=Mazes, CO=Coding, DS=Digit span.

8 S. van der Sluis et al. / Intelligence 36 (2008) systematic differences may exist in the way that boys and girls are treated). To more closely examine the mean differences, paired t-tests were performed on the data of the Dutch and Belgian opposite sex twin pairs (Table 2). In both the Dutch and Belgian samples, boys scored significantly higher on INF than their female sibling, while girls scored significantly higher on CO than their male sibling. These results are consistent with the effect sizes in Table 1, and likely to represent genuine differences between boys and girls. In addition, Dutch boys scored higher on AR, VOC and COMP than their female siblings. These latter results are consistent with the intermediate effect sizes for the Dutch sample as presented in Table 1. In sum, the results of the paired t-tests correspond to the differences observed between boys and girls in the total sample, and the sex differences are therefore not likely to be the result of between family differences in factors like SES. The mean differences between boys and girls on the level of the observed subtest scores, and the relation with the underlying primary factors of intelligence are further examined using MG-CMSA Exploratory factor analyses Because the reported patterns of factor loadings vary across studies, exploratory factor analyses (EFA) were carried out first to establish the pattern of factor loadings in Dutch and Belgian boys and girls separately. The Table 2 Paired t-tests for Dutch and Belgian opposite sex twins Netherlands Belgium (N pairs =62) (N pairs =108) t df p t df p INF SIM AR VOC COMP PC PA BD OA CO MA DS Note. Positive t-values indicate male advantage; negative t-values indicate female advantage. INF = Information, SIM = Similarities, AR = Arithmetic, VOC = Vocabulary, COMP = Comprehension, PC = Picture Completion, PA = Picture Arrangement, BD = Block Design, OA = Object Assembly, MA = Mazes, CO = Coding, DS = Digit span. Table 3 Results exploratory factor analyses, separately for Dutch and Belgian boys and girls Netherlands Boys Girls (N=350) (N=387) V P M V P M INF SIM AR VOC COM PC PA BP OA CO MA DS Belgium Boys Girls (N=370) (N=391) V P M V P M INF SIM AR VOC COMP PC PA BD OA CO MA DIG INF=Information, SIM=Similarities, AR=Arithmetic, VOC=Vocabulary, COMP=Comprehension, PC=Picture Completion, PA=Picture Arrangement, BD=Block Design, OA=Object Assembly, MA=Mazes, CO=Coding, DS=Digit span, V=Verbal factor, P=Performance factor, M=Memory factor. exploratory factor solution was followed by an oblique rotation (Promax; see Lawley & Maxwell, 1971), using normal theory maximum likelihood estimation (ML). In both Dutch and Belgian samples, solutions with one factor or with two correlated factors were inadequate, while the solution with three correlated factors proved reasonable in terms of goodness of fit and interpretability. Table 3 contains the loadings of the 12 subtests on the three correlated common factors reported separately for Dutch and Belgian boys and girls. Factor loadings in bold print were considered substantial in all subsamples. This empirically established pattern of factor loadings is largely similar to factor solutions

9 56 S. van der Sluis et al. / Intelligence 36 (2008) Table 4 Fit statistics for the Dutch models CFI RMSEA χ 2 diff F 1 Configural invariance F 1a Configural invariance+residuals OA and BD correlated F 1a vs. F 1 : χ 2 diff(2)=26.84, pb.001 F 2 Metric invariance F 2 vs. F 1a : χ 2 diff(11)=9.43, ns F 3 Strong factorial invariance F 3 vs. F 2 : χ 2 diff(9)=65.66, pb.001 F 3a Strong factorial invariance, bar INF, AR and CO F 3a vs. F 2 : χ 2 diff(6)=10.75, ns F 4 Strict factorial invariance F 4 vs. F 3a : χ 2 diff(13)=12.80, ns S 1 Introduction 2nd order factor S 1 is identical to F 4 S 2 Metric invariance 2nd order factor S 2 vs. S 1 : χ 2 diff(2)=5.34, ns S 3 Strong factorial invariance 2nd order factor S 3 vs. S 2 : χ 2 diff(2)=1.79, ns S 4 Strict factorial invariance 2nd order factor S 4 vs. S 3 : χ 2 diff(1)=4.20, ns (p=.04) reported in previous papers (e.g., Burton et al., 2001; Dolan, 2000; Dolan & Hamaker, 2001; Keith, 1997; Kush et al., 2001; Meesters et al., 1998; Oh et al., 2004), with the first factor representing the Verbal factor, the second factor the Performance factor, and the third factor the Memory factor (also known as Freedom from Distractibility: a mix of memory and speed). This configuration of factor loadings was subsequently used in the confirmatory MG-CMSA, with the bold factor loadings estimated freely, and all other factor loadings fixed to zero Confirmatory factor analyses Dutch sample The results and fit statistics of the MG-CMSA on the Dutch data are presented in Table First-order factor models. In model F 1 we tested for configural invariance: a factor model with three correlated factors was fitted in boys and girls separately, with INF, SIM, AR, VOC and COMP loading on the Verbal factor, OA, BD, PC and PA loading on the Performance factor, and AR, BD, CO, MA and DS on the Memory factor. All these factor loadings, which were estimated separately in the two sexes, were significant in both boys and girls. The modification indices (MIs) showed however that the fit of this baseline model could be improved substantially by allowing the residuals of OA and BD to correlate (MI=20 in boys, and MI=18 in girls). Note that such minor modifications of the Wechsler-model are not uncommon, and this specific link between OA and BD has been established before (e.g., Arnau & Thompson, 2000; Dolan et al., 2006; Ward, Axelrod, & Ryan, Fig. 1. First-order factor model for Dutch sample, where the λ's denote the regressions of the 12 subtests on the three factors, the Ψ 's denote the correlations between the factors, and the ε's denote those parts of the variances in the subtests that are not predicted by the factors, i.e., the residual variances. VERB=Verbal factor, MEM=Memory factor, PERF=Performance intelligence, INF=Information, SIM=Similarities, AR=Arithmetic, VOC=Vocabulary, COMP=Comprehension, PC=Picture Completion, PA=Picture Arrangement, BD=Block Design, OA=Object Assembly, MA=Mazes, CO=Coding, DS=Digit span.

10 S. van der Sluis et al. / Intelligence 36 (2008) ). Addition of these parameters to model F 1a in female and male samples resulted in a significant improvement of the fit (χ 2 diff (2)=26.84, p b.001). Model F 1a, which is depicted in Fig. 1, will serve as the baseline model for all subsequent analyses. As the configuration of factor loadings and correlated residuals was identical in boys and girls, configural invariance across sex was established in the Dutch sample. In model F 2 we tested for metric invariance by constraining the 14 factor loadings to be equal across sex, while the variances of the three common factors were freely estimated in the girls, and fixed to 1 in the boys for reasons of identification. These constraints did not result in a significant deterioration of the fit, compared to model F 1a (χ 2 diff (11)=9.43, ns), so the factor loadings can be considered identical across sex. Strong factorial invariance was tested in model F 3 by constraining the intercepts to be equal across sex, while the factorial means were fixed to zero in the boys, and estimated as latent mean differences in the girls. The fit of this model was however significantly worse than the fit of model F 2 (χ 2 diff (9)=65.66, pb.001), implying that not all mean sex differences on the level of the subtests can be accounted for by differences on the level of the first-order factors. In view of the MIs, in model F 3a, it was decided to constrain all intercepts to be equal across sex, except the intercepts of INF, AR, and CO. Model F 3a did not fit significantly worse than model F 2 (χ 2 diff (6)=10.75, ns). This means that strong factorial invariance was established for 9 of the 12 subtests. The sex differences on the subtests INF, AR and CO were too large to be accounted for by the first-order factors. So, in the sense discussed above, these three subtests may be viewed as biased within the common factor model. In all subsequent models, the subtest means of INF, AR, and CO were therefore estimated freely in each group, thereby effectively eliminating these subtests from the means model, while all other subtest means remained constrained to be equal across groups. Note that subtests that are biased with respect to their means can be retained in the model without consequence because, once these subtests' means have been relaxed (i.e., allowed to vary over sex), these indicators no longer contribute to the model for the means. Strict factorial invariance was tested in model F 4 by constraining the residual variances plus the correlated residuals to be equal across sex. The fit of model F 4 was not significantly worse than the fit of model F 3a (χ 2 diff (13)=12.80, ns), so the (correlated) residuals could be considered identical in boys and girls. The factor correlations and the factor means of this model are presented in Table 5. We find practically no sex Table 5 Correlations between the first-order factors Verbal, Performance, and Memory for Dutch boys (below diagonal) and girls (above diagonal), and the means and standard deviations for boys and girls on the firstorder factors Verbal Performance Memory Verbal Performance Memory Boys (N=350) Mean SD Girls (N=387) Mean SD Effect size Note. The means of the girls should be interpreted as deviations from the means of the boys, and were not significantly different from those of the boys (as tested in model S 3 ). difference with respect to Memory (.01, s.e.), and small differences with respect to Verbal (.26, s.e.) and Performance (.19, s.e.). Fixing the first-order factor mean differences to be equal across sex did not result in a significant deterioration of model fit (χ 2 diff (3)=6.49, p=.09), i.e., boys and girls did not differ significantly with respect to their means on the first-order factors. We note however, that the missingness present in the Dutch data may have reduced the statistical power to detect small factor mean differences between the sexes. In sum, full measurement invariance was not tenable as the sex differences on INF, AR and CO were too large to be accounted for by the first-order factors. Partial measurement invariance was however tenable for the remaining 9 subtests, and all small (and non-significant) sex differences as observed on the level of these subtests could be described as (non-significant) differences on the level of the first-order factors. Although no significant mean differences were observed between boys and girls on the level of the first-order factors, a more parsimonious model for the means may identify a significant effect for sex. We therefore proceed in studying sex differences with respect to the secondorder factor g Second-order factors models. In model S 1, g, was introduced as a second-order factor for general intelligence. However, as there were only three firstorder factors, this model with three first-order factors loading on 1 second-order factor was statistically equivalent to the model without a second-order factor, in which the first-order factors were simply correlated. The fit of model S 1 was thus identical to the fit of model F 4. Model S 1 is illustrated in Fig. 2.

11 58 S. van der Sluis et al. / Intelligence 36 (2008) Fig. 2. The hierarchical factor model, where the λ's denote the regressions of the 12 subtests on the three first-order factors, the γ's denote the regressions of the first-order factor on the second-order factor g, and the ζ's and ε's denote those parts of the variances in the subtests and first-order factors that are not predicted by the first-order factors and the second-order factor, respectively. Note that the model is identical for the Dutch and Belgian sample, except that in the Belgian sample age-effects were regressed out on the level of the subtests (not drawn here for convenience). g =factor for general intelligence, VERB=Verbal factor, MEM=Memory factor, PERF=Performance intelligence, INF=Information, SIM=Similarities, AR=Arithmetic, VOC=Vocabulary, COMP=Comprehension, PC=Picture Completion, PA=Picture Arrangement, BD=Block Design, OA=Object Assembly, MA=Mazes, CO=Coding, DS=Digit span. In model S 2, the second-order factor loadings were constrained to be equal across sex, and the variance of the second-order factor was fixed to 1 in the boys (for reasons of identification) and estimated freely in the girls. The fit of model S 2 was not significantly worse than the fit of model S 1 (χ 2 diff (2) =5.34, ns), i.e., the factor loadings of the three first-order factors on g are identical across sex. In model S 3, all first-order factor means were constrained to be zero in both boys and girls, while the mean of the second-order factor g was constrained to zero in boys for reasons of identification, and estimated freely in girls. The fit of the model did not deteriorate significantly as a result of these constraints (χ 2 diff (2)=.79, ns), meaning that the sex differences with respect to the means of the first-order factors could be accounted for by the second-order factor. Finally, model S 4, in which the second-order factor means were constrained to be identical for boys and girls (i.e., fixed to zero in both groups), did not fit the data 2 significantly worse than model S 3 (χ diff (1)=4.20, p=.04). So we conclude that boys and girls do not Table 6 Fit statistics Belgian sample CFI RMSEA χ 2 diff F 1 Configural invariance F 1a Configural invariance+residuals OA and BD correlated F 1a vs. F 1 : χ 2 diff(2)=48.01, pb.001 F 2 Metric invariance F 2 vs. F 1a : χ 2 diff(11)=4.89, ns F 3 Strong factorial invariance F 3 vs. F 2 :χ 2 diff(9)=112.90, pb.001 F 3a Strong factorial invariance, bar INF, AR and CO F 3a vs. F 2 : χ 2 diff(6)=17.05, ns F 4 Strict factorial invariance F 4 vs. F 3a : χ 2 diff(13)=29.81, pb.01 F 4a Strict factorial invariance, bar INF F 4a vs. F 3a : χ 2 diff(12)=23.31, ns S 1 Introduction 2nd order factor S 1 is identical to F 4a S 2 Metric invariance 2nd order factor S 2 vs. S 1 : χ 2 diff(2)=7.86, ns S 3 Strong factorial invariance 2nd order factor S 3 vs. S 2 : χ 2 diff(2)=4.21, ns S 4 Strict factorial invariance 2nd order factor S 4 vs. S 3 : χ 2 diff(1)b1, ns