The statistical analysis of compositional data: Exploratory analysis

Transcription

1 preliminaries descriptive statistics stardisation biplot balance-dendrogram The statistical analysis of compositional data: Exploratory analysis Prof. Dr. Vera Pawlowsky-Glahn Prof. Dr. Juan José Egozcue Ass. Prof. Dr. René Meziat Instituto Colombiano del Petróleo Piedecuesta, Santer, Colombia March 20 23, 2007

2 preliminaries descriptive statistics stardisation biplot balance-dendrogram the starting point It is sometimes considered a paradox that the answer depends not only on the observations but on the question: it should be a platitude. Harold Jeffreys the starting point is (or should be) always a question the second step is proper sampling to answer the question the next step is checking for zeros in the data then one can proceed with exploratory analysis

3 preliminaries descriptive statistics stardisation biplot balance-dendrogram the starting point It is sometimes considered a paradox that the answer depends not only on the observations but on the question: it should be a platitude. Harold Jeffreys the starting point is (or should be) always a question the second step is proper sampling to answer the question the next step is checking for zeros in the data then one can proceed with exploratory analysis

4 preliminaries descriptive statistics stardisation biplot balance-dendrogram the treatment of zeros case : the part with zeros is not important for the study the part should be omitted case 2: the part is important, the zeros are essential divide the sample into two or more populations, according to the presence/absence of zeros case 3: the part is important, the zeros are rounded zeros use imputation techniques

5 preliminaries descriptive statistics stardisation biplot balance-dendrogram centre total variance centre of a compositional dataset of size n = closed geometric mean n g = C [g, g 2,..., g D ], with g i = total variance = measure of total dispersion totvar[x] = 2D D D i= j= j= x ji [ var ln x ] i x j /n other summary statistics: min, max, Q, median, Q3

6 preliminaries descriptive statistics stardisation biplot balance-dendrogram example: Kilauea Iki data Compositional Descriptive Statistics SiO2 TiO2 Al2O3 Fe2O3 FeO MnO MgO CaO Na2O K2O P2O5 Center Min Max Q Median Q total variance: check for unreasonable values, due to errors or outliers

7 preliminaries descriptive statistics stardisation biplot balance-dendrogram variation matrix [ var T = var var ln x [ ] ln x 2 x x ] [. ] ln x D x [ ] var ln x x 2 [ ] var ln x 2 x 2 [. ] var ln x D x2 [ ] var ln x x D [ ] var ln x 2 x D... [. ] var ln x D xd T is symmetric T has zeros in its diagonal neither the total variance nor any single entry in T depend on the closure constant κ rescaling has no effect

8 preliminaries descriptive statistics stardisation biplot balance-dendrogram variation array E E [ ] ln x x 2. [ ] ln x x D [ ] var ln x x 2 [ ] var ln x x D var [ln x D [ ] E ln x D x D x D ] upper triangle: variances of simple logratios lower triangle: means of simple logratios

9 preliminaries descriptive statistics stardisation biplot balance-dendrogram example: Kilauea Iki data Compositional Descriptive Statistics SiO2 TiO2 Al2O3 Fe2O3 FeO MnO MgO CaO Na2O K2O P2O5 Center Min Max Q Median Q Variation Array SiO2 TiO2 Al2O3 Fe2O3 FeO MnO MgO CaO Na2O K2O P2O5 SiO TiO Al2O Fe2O FeO MnO MgO CaO Na2O K2O P2O Means Variances

10 preliminaries descriptive statistics stardisation biplot balance-dendrogram centring a dataset consider a sample X with centre g, compute X g = ˆX, plot ˆX notes: the centred sample ˆX will gravitate around the barycentre helpful to visualise data in a ternary diagram variation array total variance do not change perturbation transforms straight lines into straight lines gridlines compositional fields can be included in the graphical representation without the risk of a nonlinear distortion

11 preliminaries descriptive statistics stardisation biplot balance-dendrogram example: Kilauea Iki data TiO Al2O3 SiO2 SiO2 TiO Al2O3 TiO Al2O3 SiO2

12 preliminaries descriptive statistics stardisation biplot balance-dendrogram stardisation consider a sample X with centre g total variance s 2 T, compute (/s T ) X g = X s, plot X s notes: the stardised sample X s will gravitate around the barycentre will have unit total variance helpful to visualise data in a ternary diagram variation array total variance change (lower triangle is identically zero; logratio variances are different)

13 preliminaries descriptive statistics stardisation biplot balance-dendrogram example: Kilauea Iki data SiO2 SiO2_st TiO2 Al2O3 TiO2_st Al2O3_st SiO2_st

14 preliminaries descriptive statistics stardisation biplot balance-dendrogram the compositional biplot: a graphical display a biplot represents simultaneously the rows columns of any matrix by means of a rank-2 approximation construction: consider the centred matrix X g = ˆX compute Z = clr( ˆX) its singular value decomposition Z = Ldiag(k, k 2,..., k s )M, k i = λ i with λ > > λ s the s positive eigenvalues of ZZ or Z Z, L the eigenvectors of ZZ, M the eigenvectors of Z Z

15 preliminaries descriptive statistics stardisation biplot balance-dendrogram the compositional biplot: a graphical display the best rank-2 approximation in the least squares sense is l l 2 l 2 l 22 ( ) ( ) k 0 m m Y = 2 m D.. 0 k 2 m 2 m 22 m 2D l n l 2n with a proportion of variability retained = λ + λ 2 s i= λ i

16 preliminaries descriptive statistics stardisation biplot balance-dendrogram the compositional biplot: a graphical display representation: write Y = GH with n n l n l2 g l2 n l22 G =.. = g 2. n ln n l2n g n H = ( k m n k m 2 n k 2 m 2 n k 2 m 22 n k n m D k 2 m 2D n ) = ( h h 2 h ) D plot g, g 2,..., g n = row markers = projections of samples plot h, h 2,..., h D = column markers = projections of clr-parts

17 preliminaries descriptive statistics stardisation biplot balance-dendrogram example: Kilauea Iki data Cumulative proportion explained: 0,7 0,9 0,98 K2O Al2O3 CaO P2O5 TiO2 Na2O FeO SiO2 MnO MgO Fe2O3

18 preliminaries descriptive statistics stardisation biplot balance-dendrogram interpretation of a compositional biplot () Cumulative proportion explained: 0,7 0,9 0,98 FeO K2O SiO2 MnO Al2O3 CaO P2O5 TiO2 Na2O MgO Fe2O3 origin O: center of the dataset vertices h j : one for each part case markers g i : one for each sample rays Oj: join of O to a vertex j; links jk: join of two vertices j k

19 preliminaries descriptive statistics stardisation biplot balance-dendrogram interpretation of a compositional biplot (2) Cumulative proportion explained: 0,7 0,9 0,98 FeO K2O SiO2 MnO Al2O3 CaO P2O5 TiO2 Na2O MgO Fe2O3 [ jk 2 var ln x ] [ j ; Oj 2 var ln x ] j x k g(x) subcompositional analysis: select vertices

20 preliminaries descriptive statistics stardisation biplot balance-dendrogram interpretation of a compositional biplot (3) Cumulative proportion explained: 0,7 0,9 0,98 FeO K2O SiO2 MnO Al2O3 CaO P2O5 TiO2 Na2O MgO Fe2O3 [ ] jk il intersect in M cos(jmi) corr ln x j x k, ln x i x l jk il at a right angle possible zero correlation [ cos(jmi) 0 corr ln x j, ln x ] i 0 x k x l

21 preliminaries descriptive statistics stardisation biplot balance-dendrogram example: Kilauea Iki data Cumulative proportion explained: 0,7 0,9 0,98 FeO K2O SiO2 MnO Al2O3 CaO P2O5 TiO2 Na2O MgO y2 2,2 2,0,8,6,4,2 Scatterplot of y2 vs y,0 Fe2O3 0,8 0,9,0,,2,3,4,5,6,7 y FeO Fe 2 O 3 MgO Na 2 O possibly [ corr 2 ln FeO, ln MgO ] 0 Fe 2 O 3 2 Na 2 O in fact: Pearson correlation = -0,49; P-Value = 0,567

22 preliminaries descriptive statistics stardisation biplot balance-dendrogram example: Kilauea Iki data Cumulative proportion explained: 0,7 0,9 0,98 FeO K2O SiO2 MnO Al2O3 CaO P2O5 TiO2 Na2O MgO y2 2,2 2,0,8,6,4,2 Scatterplot of y2 vs y,0 Fe2O3 0,8 0,9,0,,2,3,4,5,6,7 y FeO Fe 2 O 3 MgO Na 2 O possibly [ corr 2 ln FeO, ln MgO ] 0 Fe 2 O 3 2 Na 2 O in fact: Pearson correlation = -0,49; P-Value = 0,567

23 preliminaries descriptive statistics stardisation biplot balance-dendrogram interpretation of a compositional biplot (4) Cumulative proportion explained: 0,7 0,9 0,98 FeO K2O SiO2 MnO Al2O3 CaO P2O5 TiO2 Na2O MgO Fe2O3 coincident vertices x j, x k redundant: [ jk 0 var ln x ] j 0 x j constant x k x k collinear vertices one-dimensional variability compositions plot along a compositional line

24 preliminaries descriptive statistics stardisation biplot balance-dendrogram example: Kilauea Iki data Cumulative proportion explained: 0,7 0,9 0,98 FeO K2O SiO2 MnO Al2O3 CaO P2O5 TiO2 Na2O MgO TiO2 Fe2O3 Al2O3 CaO coincident vertices: TiO 2, Al 2 O 3, CaO, Na 2 O, P 2 O 5, CO 2 redundant information

25 preliminaries descriptive statistics stardisation biplot balance-dendrogram example: Kilauea Iki data Cumulative proportion explained: 0,7 0,9 0,98 FeO K2O SiO2 MnO Al2O3 CaO P2O5 TiO2 Na2O MgO MgO Fe2O3 SiO2 Na2O collinear vertices: MgO, SiO 2, Na 2 O compositional line

26 preliminaries descriptive statistics stardisation biplot balance-dendrogram building an orthonormal basis strategy: define a sequential binary partition (SBP) criteria: expert knowledge new theory related to the question previous steps (variation matrix, biplot)

27 preliminaries descriptive statistics stardisation biplot balance-dendrogram SBP for Kilauea Iki data order FeO Fe 2 O 3 MgO SiO 2 TiO 2 Al 2 O 3 CaO Na 2 O K 2 O P 2 O 5 MnO balance : separates {FeO, Fe 2 O 3 } from the other parts balance 2: separates FeO from Fe 2 O 3 balance 3: separates MgO from the remaining parts balance 4: separates SiO 2 from the remaining parts etc.

28 preliminaries descriptive statistics stardisation biplot balance-dendrogram summary statistics for balances of Kilauea Iki data Data in the sample 7 Number of parts Total Variance Computed balances 0 coord.num. x* x2* x3* x4* x5* x6* x7* x8* x9* x0* mean coord S^2 centre std. coord var. coord min. coord max. coord x* x2* x3* x4* x5* x6* x7* x8* x9* x0* Prob

29 preliminaries descriptive statistics stardisation biplot balance-dendrogram balance-dendrogram for Kilauea Iki data Na2O K2O P2O5 MnO Boxplot scale = (-2,2) FeO Fe2O3 MgO SiO2 TiO2 Al2O3 CaO Na2O K2O P2O5 MnO boxplot-scale: ( 4, +4)

30 preliminaries descriptive statistics stardisation biplot balance-dendrogram balance-dendrogram for Kilauea Iki data Na2O K2O P2O5 MnO Boxplot scale = (-2,2)

31 preliminaries descriptive statistics stardisation biplot balance-dendrogram balance-dendrogram biplot for Kilauea Iki data Cumulative proportion explained: 0,7 0,9 0,98 FeO K2O SiO2 MnO Al2O3 CaO P2O5 TiO2 Na2O MgO Fe2O3 FeO Fe2O3 MgO SiO2 TiO2 Al2O3 CaO Na2O K2O P2O5 MnO

32 preliminaries descriptive statistics stardisation biplot balance-dendrogram correlation matrix for balances of Kilauea Iki data Coordinate Correlation Matrix coord.num. x2* x3* x4* x5* x6* x7* x8* x9* x0* x* x2* x3* x4* x5* x6* x7* x8* x9*

33 preliminaries descriptive statistics stardisation biplot balance-dendrogram scatterplot for balances of Kilauea Iki data Matrix Plot of x*; x2*; x3*; x4*; x5*; x6*; x7*; x8*; x9*; x0*,00,25,50,5 2,0 2,53,2 3,4 3,60,5 0,6 0,7 2,35 2,40 2,452,6 2,7 2,8,5,6,7 0,50 0,75,00 0,0 0,2 0,4 0,8 0,6 x* 0,4 x2*,50,25,00 2,5 x3* 2,0,5 3,6 x4* 3,4 3,2 x5* 0,7 0,6 0,5 2,45 x6* 2,40 2,35 x7* x8* 2,8 2,7 2,6,7,6,5,00 x9* 0,75 0,50 x0*