DSC 201: Data Analysis & Visualization

Transcription

1 DSC 201: Data Analysis & Visualization Aggregation Dr. David Koop

2 Selection & Highlighting Selection: a user action (mouse, keyboard) on items, links, etc. Selection types: single vs. multiple, contiguous vs. non-contiguous Feedback is important! How? Change selected item's visual encoding - Change color: want to achieve visual popout - Add outline mark: allows original color to be preserved - Change size (line width) - Add motion: marching ants 2

3 Selection & Highlighting Selection: a user action (mouse, keyboard) on items, links, etc. Selection types: single vs. multiple, contiguous vs. non-contiguous Feedback is important! How? Change selected item's visual encoding - Change color: want to achieve visual popout - Add outline mark: allows original color to be preserved - Change size (line width) - Add motion: marching ants 2

4 line). Although it is just an example, using it you can already ask some quite interesting questions. If I am going to gamble whether Nasdaq 100 will gain or lose tomorrow what is my chance? Is Friday or Monday the most unlucky day for investors? Is spring better than winter to invest? Can you find the outliers? When did the outliers occur? Public data source: PiTrading.com. Try it out or check out these other examples. Juxtaposition and Coordinated Views US Venture Capital Landscape 2011 (choropleth chart, bubble chart) Major Canadian City Crime Stats (bubble overlay, bar chart, line chart) List of Community Contributed Examples Simple, Specific Chart Examples (source) Nasdaq 100 Index 1985/11/ /06/29 Yearly Performance (radius: fluctuation/index ratio, color: gain/loss) 150% 100% Index Gain % 50% 0% -50% % -150% -2,000-1,500-1, ,000 1,500 2,000 Index Gain Days by Gain/Loss Quarters Day of Week Days by Fluctuation(%) Loss(46%) Gain(53%) Q4 Q1 Mon Tue Wed 2,000 1,500 1,000 Q3 Q2 Thu Fri , % -20% -15% -10% -5% 0% 5% 10% 15% 20% 25% 7,000 6,000 5,000 [ Monthly Index Average Monthly Index Move 3

5 Multiple Views All Subset None Same Redundant Overview/ Detail Small Multiples Multiform Multiform, Overview/ Detail No Linkage [Munzner (ill. Maguire), 2014] 4

6 Superimposition 80 Temperature (ºF) 70 Austin 60 New York San Francisco October November December 2012 February March April May June July August September [M. Bostock, 5

7 Flattening the Sphere? [USGS Map Projections] 6

8 Choropleth Map: What are Marks and Channels? [M. Ericson, New York Times] 7

9 What are the Marks and Channels? [M. Ericson, New York Times] 8

10 House Races: More Geographic Data? [New York Times, 2010] 9

11 House Races: Maps Aren't Always Best [NYTimes] 10

12 Assignment 4 Visualization using Tableau Year as a Dimension 11

13 Aggregation: Histograms Very similar to bar charts Often shown without space between (continuity) Choice of number of bins - Important! - Viewers may infer different trends based on the layout Weight Class (lbs) [Munzner (ill. Maguire), 2014] 12

14 Boxplots Show distribution Single value (e.g. mean, max, min, quartiles) doesn't convey everything Created by John Tukey who grew up in New Bedford! Show spread and skew of data Best for unimodal data Variations like vase plot for multimodal data Aggregation here involves many different marks [Flowing Data] 13

15 Data Analysis Scenarios Often want to analyze data by some grouping: - Dogs vs. cats - Millennials vs. Gen-X vs. Baby Boomers - Physics vs. Chemistry Compute statistics based on those groupings - max, min, median Perform your own type of transformation (top-k, spread, etc.) Create visualizations for each group 14

16 Split-Apply-Combine Coined by H. Wickham, 2011 Similar to Map (split+apply) Reduce (combine) paradigm The Pattern: 1. Split the data by some grouping variable 2. Apply some function to each group independently 3. Combine the data into some output dataset The apply step is usually one of : - Aggregate - Transform - Filter [T. Brandt] 15

17 Split-Apply-Combine [W. McKinney, Python for Data Analysis] 16

18 Splitting by Variables.(sex).(age) name age sex John 13 Male Mary 15 Female Alice 14 Female Peter 13 Male Roger 14 Male Phyllis 13 Female name age sex John 13 Male Peter 13 Male Roger 14 Male name age sex Mary 15 Female Alice 14 Female Phyllis 13 Female name age sex John 13 Male Peter 13 Male Phyllis 13 Female name age sex Alice 14 Female Roger 14 Male name age sex Mary 15 Female [H. Wickham, 2011] 17

19 Apply+Combine: Counting.(sex).(age).(sex, age) sex value age value sex age value Male Male 13 2 Female Male Female 13 1 Female Female [H. Wickham, 2011] 18

20 In Pandas groupby method creates a GroupBy object groupby doesn't actually compute anything until there is an aggregation or we wish to examine the groups Choose keys (columns) to group by 19

21 Aggregation Operations: - size() - mean() - sum() May also wish to aggregate only certain subsets - Use square brackets with column names Can also write your own functions for aggregation and pass then to agg function - def peak_to_peak(arr): return arr.max() - arr.min() grouped.agg(peak_to_peak) 20

22 Optimized groupby methods Function name count sum mean median std, var min, max prod first, last Description Number of non-na values in the group Sum of non-na values Mean of non-na values Arithmetic median of non-na values Unbiased (n - 1 denominator) standard deviation and variance Minimum and maximum of non-na values Product of non-na values First and last non-na values [W. McKinney, Python for Data Analysis] 21

23 Iterating over groups for name, group in df.groupby('key1'): print(name) print(group) Can also.describe() groups 22

24 Example: Tipping Data tipping.ipynb 23