A scalable Approach to Sizeindependent

Size: px

Start display at page:

Download "A scalable Approach to Sizeindependent"

Elfreda Turner
6 years ago
Views:

1 Research Dublin Dept. of Computer Science Rutgers A scalable Approach to Sizeindependent Network Similarity Michele Berlingerio Tina Eliassi- Rad Danai Koutra Christos Faloutsos WIN, September 28 th - 29 th 2012, NYU Stern School of Business

2 Why network similarity? (1) 2 x- fer learning 1 Graph Database: clustering Danai Koutra (CMU) - danai@cs.cmu.edu 2

3 Why network similarity? (2) 3 Anomaly detec:on: diﬀerent models? 4 Discon:nuity Detec:on Day 1 Day 2 Day 3 Day 4 Day 5 Danai Koutra (CMU) - danai@cs.cmu.edu 3

4 RoadMap Problem Defini:on NetSimile Experiments Applica<ons Conclusions Danai Koutra (CMU) - danai@cs.cmu.edu 4

5 Network Similarity: DeHinition INPUT: 2 anonymized networks GIVEN: node IDs NOT GIVEN: side- info class labels 1 2 OUTPUT: structural similarity score Danai Koutra (CMU) - danai@cs.cmu.edu 5

6 Network Similarity: Extension INPUT: set of anonymized networks 2 1 k OUTPUT: pairwise structural similarity scores and.8 s 12 = 0.1 s 13 = 0 s 5k = 1 Danai Koutra (CMU) - danai@cs.cmu.edu 6

7 Required Properties P1. eﬀec:veness size- independence intui<veness interpretability P2. scalability Danai Koutra (CMU) - danai@cs.cmu.edu 7

8 RoadMap Problem Defini<on NetSimile Experiments Applica<ons Conclusions Danai Koutra (CMU) - danai@cs.cmu.edu 8

9 NetSimile: overview E ego, N ego 1 Feature Extrac:on 2 Feature Aggrega:on 1 n Ε[X 2 ] Ε[X] Comparison similarity metric Danai Koutra (CMU) - danai@cs.cmu.edu 9

Step 1: Feature Extraction egonet Local and egonet features: #

clustering coefficient They satisfy all the constraints! avg.

of neighbors edges in egonet outgoing edges from egonet # of

10 Step 1: Feature Extraction egonet Local and egonet features: # of neighbors Why these features? clustering coefficient They satisfy all the constraints! avg. # of neighbors neighbors avg. clustering coeff. of neighbors edges in egonet outgoing edges from egonet # of neighbors of egonet F G = Danai Koutra (CMU) - danai@cs.cmu.edu 10 n o d e s n features

Step 2: Feature Aggregation 5 aggregators median n mean Why

distribu<ons constraints! kurtosis median mean 1 2. s.d. features 1 2 3.

11 Step 2: Feature Aggregation 5 aggregators median n mean Why these aggregators? o d. standard devia<on Moments e They satisfy the s. skewness of feature effectiveness + scalability n distribu<ons constraints! kurtosis median mean 1 2. s.d. features skewness 1 n, median, kurtosis single signature vector per network Danai Koutra (CMU) - danai@cs.cmu.edu 11

features) Similarity Scores s12 s13 G1 Why

2 sg2 d Pi Qi s1k canberra( P, Q ) = i=1 s23

near s2k ① sensitive to small changes - cosine

② normalizes the absolute- difference euclidean

12 Step 3: Comparison Networks 1 3 k... Signature Vectors (aggr. features) Similarity Scores s12 s13 G1 Why Canberrasdistance? 2 sg2 d Pi Qi s1k canberra( P, Q ) = i=1 s23 Canberra Pi + Qi distance tried: 0 sg3 Also near s2k ① sensitive to small changes - cosine similarity. ② normalizes the absolute- difference euclidean distance... - hypothesis tesdng:. of the individual comparisons. Mann- Whitney,. Kolmogorov- Smirnov sgk sk-1,k Danai Koutra (CMU) - danai@cs.cmu.edu 12

13 Required Properties: NetSimile P1. effec:veness size- independence intui<veness interpretability avoids the node- correspondence problem P2. scalability LEMMA The run<me complexity for genera<ng NetSimile s signature vectors is linear on the number of edges in the input networks: k # nodes O( f n j + f n j log(n j )) j=1 Danai Koutra (CMU) - danai@cs.cmu.edu 13

14 RoadMap Problem Defini<on NetSimile Experiments Applica<ons Conclusions Danai Koutra (CMU) - danai@cs.cmu.edu 14

15 Experiments: Data 30 real- world networks dblp mul<ple synthe<c networks Barabási- Albert Forest Fire Erdös- Rényi Wa[s- Strogatz querylog Oregon AS Danai Koutra (CMU) - danai@cs.cmu.edu 15

16 Experiments: intuitiveness + interpretability of NetSimile homogeneity in colors Observa:on: NetSimile gives be[er and more intui<ve graph clusters than the EIG method (eval- based compe<tor method). Danai Koutra (CMU) - danai@cs.cmu.edu 16

Experiments: NetSimile and node-overlap bigger overlap smaller distance Hypothesis: bigger node overlap => greater similarity Implicit Assump<on: networks are from the same

17 Experiments: NetSimile and node-overlap bigger overlap smaller distance Hypothesis: bigger node overlap => greater similarity Implicit Assump<on: networks are from the same domain Observa:on: The lower the NetSimile score (greater similarity), the higher the normalized node intersec<on of the input networks. Danai Koutra (CMU) - danai@cs.cmu.edu 17

18 Application: Discontinuity Detection in Yahoo! IM 1. Microso^ offers to buy Yahoo!. 2. New features for flickr were announced. nodes: IM users edges: communica<on events Danai Koutra (CMU) - danai@cs.cmu.edu 18

19 RoadMap Problem Defini<on NetSimile Experiments Applica<ons Conclusions Danai Koutra (CMU) - danai@cs.cmu.edu 19

20 Conclusions Novel approach: signature vector for each graph (summariza<on) NetSimile: effec<ve size- independent, intui<ve, interpretable scalable Applicability to a variety of problems Danai Koutra (CMU) - danai@cs.cmu.edu 20

21 Thank you! hbp:// Danai Koutra (CMU) - danai@cs.cmu.edu 21

22 Experiments (2): Are we measuring size? Observa:on: NetSimile is not measuring size there is no correla<on between extracted features and network size. Danai Koutra (CMU) - danai@cs.cmu.edu 22

23 Advantages of NetSimile size- invariant scalable LEMMA The run<me complexity for genera<ng NetSimile s signature vectors is linear on the number of edges in the input networks: k # nodes O( f n j + f n j log(n j )) j=1 avoids the node- correspondence problem Danai Koutra (CMU) - danai@cs.cmu.edu 23

Final Project - Social and Information Network Analysis

Final Project - Social and Information Network Analysis Factors and Variables Affecting Social Media Reviews I. Introduction Humberto Moreira Rajesh Balwani Subramanyan V Dronamraju Dec 11, 2011 Problem