A SAS PROCEDURE FOR HIERARCHICAL CLUSTERING USING TRANSACTION FLOW TABLES

Transcription

1 PFPHC: A SAS PROCEDURE FOR HERARCHCAL CLUSTERNG USNG TRANSACTON FLOW TABLES Dant@l M. Chilko and p. B. Slater Statistics and Computer Science Department West Virginia University Morgantown, West Virginia ABSTRACT CHLKO J D. M., and P. ll StATER PFPHC: A SAS procedur~ for hierarchical clustering using transaction flow tables. pp N Strand. R. H. (ed.), Proe., Third Annual Conference of the SAS Users Group nternational. SAS nstitute, nc., Raleigh. North Carolina. 318 pp. Transaction flow tables. such as internal migration, input-output. trip distribu~ion, trade, and occupational mobility matrices. are frequently available to social scientists. They often assume large dimensions. A two-stage procedure for effectively summarizing this fo~ of data structure has been implemented in SAS. n the first stage t the table is adjusted 50 that all its row and column sums are equal. n the second, the units of the adjusted table, which 1s viewed as an asymmetric similarity matrix, are hierarchically clustered using the directed graph analogue of the single linkage method. NTRODUCTON nternal migration, input-output, trip distribution. trade j and occupational mobility tables are important forms of social science data. These tables are all square matrices with the same corresponding row and column labels~ The ij~entry of any such table is the number of items that have been transferred from the ith unit to the jth unit over a giv@n time period. Matrices with these characteristics have been termed transaction flow tables. Several procedures have been developed to provide insights into the structural relationships between the units--geographic areas, occupations, industries, etc.--of transaction flow tables. n one, the table is reg~rded as the capacity matrix of a network (Slater~ 1976a). The maximum possible flow of a single commod~ty between each ordered pa1r of units is found. As~ sociated wlth each such flow is a partition of the units into two groups. f this dichotomy does not separate just a single unit from all others, insightful information is provided. Such results can be used to define nodal regions on the basis of internal migration tables. Any such region has the property that fewer people leave (enter) it than leave (enter) a distinguished member of it, its node. A FORTRAN program for the maximum flow - minimum cut algorithm. the major step in nodal regionalization, is available (Nijenhuis and Wilf, 1975). n this paper, we describe an algorithm for functional regionalization (Slater, 1976b; Leusmann and Slater, 1977). The algorithm has been developed and implemented as a local SAS procedure (PFPRC). t consists of two stages: double-standardization of the flow table ~nd its subsequent hierarchical clustering. DOUBLE-STANDARDZATON nstead of using the recorded flows themselves as input to the clustering routine~ the flows are first adjusted by the iterative proportional fitting procedure (PFP) so that the total out-flow and total in-flow associated with all units are Lhe same. Rows and columns are alternately scaled so that all their sums ate eqnal. f the recorded flows are all positive, this double-standardization can be accomplished with mathematical certainty. However~ under sorne conditions of sparsity, converg-ence will not occur (Sinkhorn and Knopp; 1967)~ The ij-entry of the adjusted table is a measure of the functional relationship between i and j. f there is no flow from 1 to j, the ij-entry of the adjusted table will be zero. f the cnly flows from i go to j, the ij-entry will equal the prescribed row and column sum, here tsken to be 1. The adjusted table can be regarded as a similarity (linkage) table. in which confounding effects of origin and destination sizes have been eliminated. The interaction structure of the original table, given by the cross-product ratios, is preserved by the PFP. ntra-unit phenomena have typically not been taken into accouct in the use of the SAS procedure~ PFPHC. Zeros are entered on the diagonal of the flaw table and remain unchanged by the PFP. Larger areal units would tend to have larger dibgonal entries in the adjusted table- if the original diagonal elem~nts were available and not nullified--and consequently smaller offdiagonal entries. PFPHC does possess an option for non-zero diagonal analyses, however. HERARCHCAL CLUSTERNG The hierarchical clustering algorithm used is consequently founded on the properties of directed graphs; in particular, those of strong 255

2 components. (A strong component in a directed graph is a set of vertices between any two members of which there exists a path of one-way links.) For a symmetric n x n table, the proce~ clure reduces to the familiar single linkage method. though single linkage always requires (0-1) clustering steps) while the Btrong component based procedure may have fewer than (n-l). nitially, in the algqrithm, there are no direct~d links connecting the points or units. Directed links are then inserted in order of their decreasing magnitude in the adjusted table. f the ij-entry is the largest, an arrow is first drawn. conceptually, from i to j. f the kl-entry is the second largest, an arrow is constructed from k to, and so forth. As the procedure con~inue8, points unite into strong components (clusters). (The reachabi1ity matrix of the digraph is employed as the appropriate data structure to store this information~) A directed link, the insertion of which results in the unification of two or more strong components is called a critical link. (The adjusted flow v~lua associat~d with a critical link ib referred to as a threshold., Eventually, all units lie in one strong component~ The process is hierarchical in nature. There are several ways to represent this clustering process. n PFPHC, a rooted tree diagram (dendrogram) is employed. Clusters that are particularly significant in that they either (l) retain their identity through a wide range of thresholds or (2) are of relatively large size for the thresholds over which they exist can be discerned from the dendrogram. This clustering procedure can be viewed as the fitting of a hierarchical structure to an asymmetric matrix of linkages. The hierarchy obtained can be represented by another matrix of the s~e dimensions~ The ij-entry of it is the value of the lowest threshold at which a cluster containing both the i and j units exi6ts~ ~esidua1s of the fitted matrix froru th~ adjustad matrix are the distortions that result from representing the flow table in hierarchical form. (A robust multidimensional scaling procedure for examining the matrix of residuals is presently under development.) The co~re1ation between the entries in the estimated table and the adjusted table is termed th~ cophhnetic correlation coefficient. t is an indicator of the degree of resemblance between the original table and the hiera~chy imposed upon it. The SAS PFPHC procedure, for this twostage standardization and clustering algorithm, produces a "historyr! of the clustering process and a dendrogram by default. t has options to print the input flaw table. the adjust~d flow table~ the estimated flow table, and a table of n~gative residuals. (Negative residuals are printed since the residuals themselves are predominantly negative~) APPLCATON OF PFPHC Figure 1 is a map of Switzerland showing its 25 local governmental units. (Codes for these units are given in Table 1.) A transaction flow table of the number of people who lived in one unit in 1965, and in anoth~r in 1970, was analyzed by PFPHC. A dendrogram (Figure 2) wab produced. The critical links ane thresholds are listed in Table 2. (The results for the analysis of a Swiss migration table were essentially identical to these.) At the level of.068~ the value of the linkage from Ticino (T) to Neuchatel (NE), a tenmembe~ western section. which had itself formed at united with a fifteen-member eastern region, born at.093, to form the nation itself. Prominent sub clusters can be discerned in the dendrogram. The two half cantons of Appenzell nner-rhoden CAl) and Appenzell Ausser-Rhoden (AR) were formed from the canton of Appenzell in the year at which point they ~ere separated because of religious differences. (These two half cantons form an enclave in St. Gallen, with 'Which they join at a threshold of.173.) This analysis reassembles them, as well as the other two pairs of half cantons. Basel Landschaft (EL) and Basel-Stadt CBS) were formed from the cantor. of Basel in 1833, and Obwalde.n C'Above the Forest") (OW) and Nidwalde:n C1Below the. Forest'!} (Nt') from the -canton of Unterwalden. The contiguous German-speaking, predominantly Protestant cantons of Schaffhausen (BH) and Thurgau (TG) are also paired. Geneve borders only Vaud~ so it is plausible that those two cantons are joined. The three pairs of half cantons enumerated above. particularly those forming BaB~~ have long r8ng2s of existence, indicative of their significance as groups. Basel is the most peripheral group in the western cluster~ and Unterwalden, tdgether with Luzern, the most peripheral in the eastern region. Table 1. Codes for Swiss local governmental units AG Aargau A Appenzell 1. Rh. All Appenzell A. Rh. BE Bern BL Basel-Landschaft BS 13asel-Stadt FR Fribourg GE Geneve GL Glarus GR Graubunden Ll; Luzern NE Neuchatel N" Nidwalden 01.' Obwalden SG St. Gallen SH Schaffhausen SO Solothurn SZ Sc.hwyz TG Thurgau T Ticino UR Uri \E Vaud vs Vala1s ZG Zug ZH Zurich 256

3 Table 2. Critical Links and Thresholds Critical Link Threshold Al + AR.374 BS + BL.339 OW + NW.305 GE + VD.201 TG + SH.182 A + SG.173 NE -T VD.169 GE + VS.159 LU + OW.158 UH+ SZ.158 FH+ VU.144 SO + BE.135 ZG + SZ BE + NE TG + SG.12l AG + SO.119 T+ GR.118 ZH + SH.112 SZ + GL.108 GR + GL.104 SH + GL.102 BL + SO.094 UR + Nl HE.068 REFERENCES Leusmann, C.~ and P+ E. Slater A functional regiona1ization program based on the standardization and hierarchical clustering of transaction flow tables. Computer Applications. (in press). Nijenhuis, A.; and H. Wilf Combinatorial Algorithms. Academic. Press, New York~ Sinkhorn, R., and p~ Knopp Concerning non-negative matrices and doubly stoc.hastic matrices. Pacific Journal of Mathematics 21, Slater, P. B. 1976a. A multiterminal network flow analysis of an unadjusted Spanish interprovincial migration table. Environment and Planning A 8: Slater~ P. B+ 1976b. A hierarchical regionalization of Japanese prefectures using 1972 interprefectural migration flow6. Regional Studies 10,

4 Figure 1. Map of Swiss local govermnental. units. A.Rh..Rh. Graubunden 258

5 Figure 2. Hierarchical Regionalization of Swiss local gover.runental units. f, 4~ 35 3B 25 2B 15 B 5 V~RABLE AR A SG SH ZH GR T UR 52 ZG GL 0\'1 NW LV BS BL VD GS NE vs FR BE so AG _ _" :r 259