Multidimensionality. Multi-dimensionality & Factor analysis. Adding new dimensions. Adding new dimensions. Adding new dimensions. What is a dimension?

Transcription

1 Multi-dimensionality & Factor analysis Adding dimensions: Compressing dimensions: Factor analysis Intelligence testing and g Adding new dimensions Most tests measure single dimensional constructs We have already seen one case of dualdimensionality in IQ It is easy to imagine extending into a third dimension e.g. Gould s football, correlating growth of three body parts Adding new dimensions Mathematically, there is no need to stop at three dimensions Correlations, Euclidean distance, orthogonality [to be explained] etc. are all well-defined in higher dimensional spaces Informally, it is easy to imagine high dimensional spaces e.g. Think of the qualities that make up a good car, or a good apartment, or a good mate: We have no trouble saying that there are many dimensions that matter in these cases Adding new dimensions What is a dimension? Visually, it is not so difficult either at least for a few more dimensions Imagine using color for the 4th dimension, and size of point for the fifth dimension [etc. etc.] Dimensions are information = they are differences that make a difference D2 D3 = Size D4 = Colour D1 1

2 What is a factor? A factor is a dimension Correlation and information What is the relation between a correlation and information/probability? A significant correlation between x and y tells us that x contains information about y Another way of saying this is that x and y are not independent, in the sense we used the term in discussing probability: if we know something about x o, we have a probability > 0 of knowing something about y Correlation and information However, note that the dependence may be roundabout or even spurious if it depends on relations between features that are very common or just accidental Examples: In American towns and cities, the correlation between the number of churches and the number of violent crimes is about 0.85: Why? Income and homelessness are positively correlated: Why? Outside temperature and rape rates are positively correlated: Why? Relations between dimensions Any relation between any two dimensions (which means: any correlation) can be expressed in terms of two orthogonal (=rightangle) components By definition, these two components that have nothing to do with each other, each containing no amount of the other dimension If you know something about one dimension, you know nothing about an orthogonal dimension If knowing something about one dimension does give you information about another, then they are not orthogonal = they are correlated Relating two dimensions A concrete example Dimension 2 Pure vertical component Pure horizontal component Digit span score Pure vertical component Pure horizontal component Dimension 1 Picture arrangement score 2

3 Pure dimensions If two dimensions are orthogonal, they are pure dimensions, wholly separable from one another (independent) No information about one is contained in the other By the same token, when two dimensions are not orthogonal, they are contaminated by at least one common component (they are not independent) Information about one is contained in the other i.e. Gould: 14 independent bone measurements reduce to the single dimension of 'size' The need for orthogonality Dimensions which are orthogonal are independent = whatever happens on one dimension has no effect on what happens on the other dimension (or, equivalently, knowing the value on one dimension provides no information about value along the other dimension or they correlate with r = 0). Example: Beauty and intelligence are orthogonal (if not, the world sure is unfair!); height and weight are not orthogonal The theoretical true dimensionality of a thing (the number of things we need to know to know ~everything that thing) is the value of ~all orthogonal dimensions relevant to that thing- and that may be less than the number of apparent dimensions Information and orthogonality B Dimension 2 A Here we know something about Dimension 2 (the value of B) when we know the value along Dimension 1(the value of A)- precisely because the dimensions are not orthogonal. Dimension 1 Information and orthogonality B Dimension 2 A This is mathematically equivalent to saying that the dimensions are correlated. (For the curious, the cosine of the angle = r) Dimension 1 Example: Four correlations of 4 subtest results with 2 factors Subtest 2 Subtest 1 Subtest 3 Factor B' Factor B Subtest 4 ' We rotate the dimensions here to get 2 different orthogonal dimensions. 3

4 Factor B F1: General intelligence (g) Non-verbal subtests Verbal subtests F2: Verbal load Test results as factors Note (to repeat) that this eliminates any hard distinction between the identified dimensions (say, the labels on a graph of one subtest against another) and alleged real factors In other words: Beware of falling in love with the axes labels on a graph These are just factors that some has chosen to graph: They are no more (or less) true than a rotated graph in which abstract factors now defined the axes, and text scores are vectors (regression lines) on the graph One hard idea to grasp is that the differences here are merely notational = no data or information is gained or lost through rotation, and the labels are just a descriptive convenience Correlation matrix Factor analysis We can represent the relationship between many dimensions with a correlation matrix Dimensions Factor analysis is a method for reducing a correlation matrix with a large number of dimensions to (usually) their orthogonal dimensions, called principal components In other words, it is a method of dimensional reduction It looks for contamination of one dimension by others (= inter-correlations), and tries to re-present the matrix without that contamination An everyday example Perhaps it seems odd to talk about reducing dimensions in the abstract, but we do it all the time in our lives Expression of personal preference is an everyday example of such dimensional reduction, as when we ask If you had pick just three qualities to look for in your next car, what would you pick? If you wanted a black-haired Italian boyfriend, or a brown-eyed African girlfriend, you d have pretty good odds of reducing the nationality/body part preferences to one dimension What is a factor? Factors are not Platonic 'true' objects gifted to man from the benevolent inhabitants of the Heavens A factor is a mathematical construct, containing abstracted information about the inter-relations between other information/dimensions It is no better than those measures: GIGO "The 'factors', in short, are to be regarded as convenient mathematical abstractions, not as concrete mental 'faculties', lodged in separate 'organs' of the brain. Cyril Burt,

5 What is a factor? Mathematically, a factor is a (usually but not necessarily) linear combination of correlations between the items (questions or subtest scores in our case) in the analysis Review: What does it mean to be a linear combination? The factors are produced produced by adding in portions of the variance accounted for by some dimensions (for example, subtests or questions) and subtracting out portion of the variance accounted for by others e.g. F1 = ax + by + cz, if X,Y,Z are correlations in the input table, and a,b,c are (positive or negative) real number coefficients The factor table The principal components are taken out in an ordered fashion, from those accounting for the most variance (strong loading or weighting = high correlation) to those accounting for the least (weak loading or weighting = low correlation) They are presented in a factor table FACTORS QUESTIONS A B C D Item Item Item Are factors real? Again, remember the danger of reification: Just because a factor can be extracted does not necessarily mean it has any relevant reality Some principal component can always be extracted so long as most correlations are in the same direction In modern factor analysis, the axes are also rotated to see if one rotation is better able to account for the data than another. Because different rotations lead to different descriptions, Gould warns against reifying any particular description Spearman's g " 'When I use a word', Humpty Dumpty said, in rather a scornful tone, 'it means just what I choose it to mean - neither more nor less.' " Lewis Carroll Through the Looking-Glass Spearman (1904) invented factor analysis as a way of studying correlations between mental test scores He called the first principal component (accounting for about 50-60% of variance) g = general intelligence People have argued ever since whether g is real 5

6 F1: General intelligence (g) Non-verbal subtests Verbal subtests Non-verbal subtests Verbal subtests Factor B F2: Verbal load Factor B F1: Performance IQ F2: Verbal IQ Cyril Burt's Error Intelligence is an abused concept Burt cut off 80% of people tested from access to higher education, entirely on the basis of a single test Questions Is a factor more or less 'real' than a 'traditional' construct? How can we evaluate the reality of any given factor/dimension? What advantage is there is discussing psychometric results in terms of non-orthogonal factors? Should people's lives be planned around abstract mathematical constructs? Should there be legal limits on what constructs can be defined? 6