What about streaming data?

Transcription

1 What about streaming data? 1

2 The Stream Model Data enters at a rapid rate from one or more input ports Such data are called stream tuples The system cannot store the entire (infinite) stream Distribution changes over time How do you make critical calculations (in real time) about the stream using a limited amount of (secondary) memory? 2

3 General Stream Processing Model Streams Entering... 1, 5, 2, 7, 0, 9, 3... a, r, v, t, y, h, b... 0, 0, 1, 0, 1, 1, 0 time Ad hoc Queries Standing Queries Processor Output Limited Working Storage Archival Storage 3

4 Applications In general, stream processing is important for applications where New data arrives frequently Important queries tend to ask about the most recent data, or summaries of data Mining query streams Google wants to know what queries are more frequent today than yesterday Mining click streams Yahoo wants to know which of its pages are getting an unusual number of hits in the past hour Mining social network news feeds Look for trending topics on Twitter, Facebook 4

5 Sliding Windows A useful model of stream processing is that queries are about a window of length N the N most recent elements received (or last N time units) Interesting case: N is still so large that it cannot be stored in memory, or even on disk Or, there are so many streams that windows for all cannot be stored 5

6 Sliding Windows: Single Stream q w e r t y u i o p a s d f g h j k l z x c v b n m q w e r t y u i o p a s d f g h j k l z x c v b n m Past Future Window size = 5 Step = 21 6

7 Example: Averages Stream of integers Window size = N Standing query: What is the average of the integers in the window? For the first N inputs average = sum/count Afterward, when a new input i arrives Update average: average + (i j)/n where j is the oldest value in the window 7

8 Let s start simple: Counting bits Problem Given a stream of 0 s and 1 s Answer queries of the form how many 1 s in the last k bits? where k N Obvious solution Store the most recent N bits (i.e., window size = N) When a new bit arrives, discard the N +1 st bit Real Problem Slow need to scan k bits to count What if we cannot afford to store N bits? E.g., we are processing 1 billion streams and N = 1 billion Estimate with an approximate answer N = 6 8

9 Example: Exponentially Increasing Blocks 9

10 How Good Is This? Store O(log 22 N) bits per stream (<< N bits) O(log N) counts of log N bits Gives approximate answer How accurate? If 1 s are evenly distributed, never off by more than 50% Otherwise error can be unbounded! E.g, all the 1 s are in the unknown block! 10

11 DGIM* Method Store O(log 22 N) bits per stream (<< N bits) O(log N) counts of log N bits Gives approximate answer Never off by more than 50% Error factor can be reduced to any fraction > 0, with more complicated algorithm and proportionally more stored bits Trick: Block size is based on count of 1s instead of fix sized blocks *Datar, Gionis, Indyk, and Motwani (Stanford) 11

12 DGIM: Timestamps Each bit in the stream has a timestamp, starting 1, 2, Record timestamps modulo N (the window size), so we can represent any relevant timestamp in O(log 2 N) bits If N = 1M, then we need 20 bits 12

13 DGIM: Buckets A bucket is a record consisting of The timestamp of its right (most recent) end O(log 2 N) bits The number of 1 s between its beginning and end Number of 1 s must be a power of 2 O(log 2 log 2 N) bits 13

14 Representing a Stream by Buckets Either one or two buckets with the same power of 2 number of 1 s Buckets do not overlap in timestamps Buckets are sorted by size (# of 1 s). Earlier buckets are not smaller than later buckets Buckets disappear when their (right) end time is > N time units in the past O(log 2 N) buckets 14

15 Updating Buckets When a new bit comes in, drop the last (oldest) bucket if its (right) end time is prior to N time units before the current time If the current bit is 0, no other changes are needed 15

16 Updating Buckets If the current bit is 1: Create a new bucket of size 1, for just this bit End timestamp = current time If there are now three buckets of size 1, combine the oldest two into a bucket of size 2 If there are now three buckets of size 2, combine the oldest two into a bucket of size 4 And so on 16

17 Example 17

18 Querying To estimate the number of 1 s in the most recent N bits: Sum the size of all buckets but the last size means the number of 1 s in the bucket Add half the size of the last bucket NOTE: We do not know how many 1 s of the last bucket are still within the wanted window 18

19 Example: Bucketized stream How many 1 s? ½*16 = 36 19

20 How good is the scheme? Suppose the last bucket has size 2 r Assuming 2 r 1 (i.e., half) of its 1 s are still within the window, we make an error of at most 2 r 1 Since there is at least one bucket of each of the sizes less than 2 r, the true sum is at least r 1 = 2 r 1 actually 2 r as the first bit of the last bucket is always 1 Thus, error is at most 50% 20

21 Extensions How many 1 s in the last k where k < N? Find earliest bucket B that overlaps with k Number of 1 s is the sum of sizes of more recent buckets + ½ size of B How about counting integers (sum of the last k elements)? 21

22 Counting positive integers Stream of positive integers We want the sum of the last k elements Amazon: Avg price of last k sales Solution If you know all integers have at most m bits (i.e. integer value upto 2 m ) Treat each of the m bits of each integer as a separate stream Use DGIM to count 1s in each bit stream The sum is = 22

23 Filtering data streams Consider a web crawler It keeps, centrally, a list of all the URLs it has found so far It assigns these URLs to any of a number of parallel tasks These tasks stream back the URLs they find in the links they discover on a page It needs to filter out those URLs it has seen before 23

24 Filtering data streams Recall: Each element of data stream is a tuple Given a list of keys S, determine which tuples of stream are in S Naïve solution: Hash table But suppose we do not have enough memory to store all of S in a hash table E.g., we might be processing millions of filters on the same stream 24

25 Naïve solution Given a set of keys S that we want to filter Create a bit array B of n bits Initialize to all 0s Choose a hash function h with range [0,n) Hash each member of s S to one of n buckets, and set that bit to 1, i.e., B[h(s)] = 1 Hash each element a of the stream and output only those that hash to bit that was set to 1 Output a if B[h(a)] == 1 25

26 Naïve solution S = {A1, A2, A3} H(A1) = 7 H(A2) = 0 H(A3) = Bit array H(a1) = 1. Since bit at position 1 is 0, drop a1. H(a2) = 7. Since bit at position 7 is 1, output a2. 26

27 Naïve solution False positives? Will invalid item passed through the filter? If the item is NOT in S, it may still be output Why? Collision False negatives? Will valid items be filtered out? If item is in S, we surely output it 27

28 Analysis on False Positives: Throwing darts If we throw m darts into n equally likely targets, what is the probability that a target gets at least one dart? In our case: Targets = bits/buckets Darts = hash values of items Equals 1/e as n 1 ( 1 1/n ) n (m/n) m = n(m/n) = 1 e m/n Prob some target X not hit by a dart Prob at least one dart hits target X 28

29 Analysis: Throwing darts Fraction of 1 s in the array B = probability of false positives = 1 e m/n Example: 10 9 darts, targets Fraction of 1 s in B = 1 e 1/8 = Lower than 1/8 = (why?) 29

30 Bloom filter Consider: S = m; B = n Use k independent hash functions h 1,, h k Initialization Set B to all 0 s Hash each element s S k times Run time using each hash function h i,set B[h i (s)] = 1 (for each i= 1,.., k) When a stream element with key x arrives If B[h i (x)] == 1 for all i= 1,.., k then declare that x is in S That is, x hashes to a bucket set to 1 for every hash function h i (x) Otherwise, discard the element x 30

31 Bloom filter S = {A1, A2, A3} H1(A1) = 7 H1(A2) = 0 H1(A3) = 2 H = {H1, H2} H2(A1) = 2 H2(A2) = 1 H2(A3) = Bit array H1(a1) = 1 H2(a1) = 3 Since bit at position 3 is 0, drop a1 H1(a2) = 7 H2(a2) = 5 Since bits at positions 5 and 7 are both 1, output a2 (potentially a false positive) 31

32 Bloom filter: Analysis What fraction of the bit vector B are 1 s? Throwing k mdarts at n targets So fraction of 1 s is (1 e km/n ) But we have k independent hash functions and we only let the element x through if all k hash element x to a bucket of value 1 So false positive probability = (1 e km/n ) k 32

33 Bloom Filter Bloom filters guarantee no false negatives and use limited memory Great for pre processing before more expensive checks Suitable for hardware implementation Hash function computations can be parallelized Is it better to have 1 big B or k small B s? It is the same: is (1 e km/n ) k vs (1 e m/(n/k) ) k But keeping 1 big B is simpler 33

34 So far, We have looked at some operations on data streams when memory is limited Counting bits Counting integers Filtering streams 34

35 Online (Web) advertising: Streaming queries, dynamic answers 35

36 Multi billion dollar industry! Online advertisements 36

37 Banner Ads 37

38 Pay Per Impression Pay per 1000 impressions (PPM): advertiser pays each time ad is displayed Models existing standards from magazine, radio, television Main business model for banner ads to date Untargeted to demographically targeted Low clickthrough rates Low ROI for advertisers Banner blindness : effectiveness drops with user experience Barrier to entry for small advertisers Contracts negotiated on a case by case basis with large minimums (typically, a few thousand dollars per month) 38

39 Pay Per Click Pay per click (PPC): advertiser pays only when user clicks on ad Common in search advertising Middle ground between PPM and PPA Does not require host to trust advertiser Provides incentives for host to improve ad displays 39

40 Sponsored (Performance based) advertising Advertisements sold automatically through auctions: Advertisers bids (indicating value for clicks) on search keywords Low barrier to entry Increased transparency of mechanism Keyword bidding allowed increased targeting opportunities When someone searches for that keyword, a mechanism selects the best ad to display Advertiser is charged only if the ad is clicked on Example: Google s Adwords 40

41 Ads vs. search results 41

42 42

43 Algorithmic Challenges Interesting problems What ads to show for a search? If I m an advertiser, which search terms should I bid on and how much to bid? Not our focus 43

44 Scenario A stream of queries arrives at the search engine q1, q2, q3, Several advertisers bid on each query When query q i arrives, search engine must pick a subset of advertisers whose ads are shown Goal: maximize search engine s revenues Clearly we need an online algorithm! 44

45 How to maximize revenue? Greedy solution: Initial GoTo model (First price auction) Advertise based on highest bidders Upon a click, advertiser is charged a price equal to his bid Used first by Overture/Yahoo! Complications Each ad has a different likelihood of being clicked Advertiser 1 bids $2, click probability = 0.1 Advertiser 2 bids $1, click probability = 0.5 Clickthrough rate measured historically 45

46 How to maximize revenue? Google model: Second price auction Advertisers ranked according to bid and clickthrough rate (CTR) Use the expected revenue per click Upon a click, advertiser is charged minimum amount required to maintain position in ranking 46

47 The Adwords Innovation Advertiser Bid CTR Bid * CTR A $1.00 1% 1 cent B $0.75 2% 1.5 cents C $ % cents Click through rate Expected revenue 47

48 The Adwords Innovation Advertiser Bid CTR Bid * CTR B $0.75 2% 1.5 cents C $ % cents A $1.00 1% 1 cent Instead of sorting advertisers by bid, sort by expected revenue! 48

49 More complications Each advertiser has a limited budget Search engine guarantees that the advertiser will not be charged more than their daily budget Solution: Ensure budget is not exhausted too quickly unnecessarily 49

50 Simplified model (for now) All bids are 0 or 1 Each advertiser has the same budget B One ad per query All ads are equally likely to be clicked Let s try the greedy algorithm Arbitrarily pick an eligible advertiser (bid 1) for each keyword 50

51 Bad scenario for greedy Two advertisers A and B A bids on query x, B bids on x and y Both have budgets of $4 Query stream: x xxxy yyy Worst case greedy choice: B BBB Optimal: A AAAB BBB Competitive ratio min all possible inputs Revenue greedy /Revenue opt =½ 51

52 Adwords Problem Given A set of bids by advertisers for search queries A click through rate for each advertise query pair A budget for each advertiser (say for 1 day, 1 month, ) A limit on the number of ads to be displayed with each search query, N Respond to each search query with a set of advertisers such that The size of the set is no larger than N Each advertiser has bid on the search query Each advertiser has enough budget left to pay for the ad if it is clicked 52

53 BALANCE algorithm [Mehta, Saberi, Vazirani, and Vazirani] For each query, pick the advertiser with the largest unspent budget Break ties arbitrarily but deterministically A. Mehta, A. Saberi, U. Vazirani, V. Vazirani, Adwords and Generalized On line Matching, Journal of the ACM (2007). Conference version appeared in IEEE Symposium on Foundations of Computer Science (2005). 53

54 Example: BALANCE Two advertisers A and B A bids on query x, B bids on x and y Both have budgets of $4 Query stream: x xxxy yyy BALANCE choice: A B A B BB Optimal: A AAAB BBB Competitive ratio = ¾ For BALANCE with 2 advertisers 54

55 Analyzing BALANCE Consider simple case: two advertisers, A 1 and A 2, each with budget B (assume B 1) Assume optimal solution exhausts both advertisers budgets BALANCE must exhaust at least one advertiser s budget If not, we can allocate more queries Assume BALANCE exhausts A 2 s budget 55

56 Analyzing Balance B Queries allocated to A 1 in optimal solution Queries allocated to A 2 in optimal solution A 1 A 2 Opt revenue = 2B BALANCE revenue = B+y x B Goal: Show that y B/2 y A 1 A 2 x BAL: B+B/2 = 3B/2 BAL/OPT = 3/4 Competitive Ratio = 3/4 56

57 Analyzing Balance B Queries allocated to A 1 in optimal solution Queries allocated to A 2 in optimal solution A 1 A 2 Case 1: BALANCE assigns at least B/2 blue queries to A1. So, y B/2 x y A 1 A 2 x B Case 2: BALANCE assigns more than B/2 blue queries to A 2 At that time, A 2 s unspent budget must have been at least as big as A 1 s That means at least as many queries have been assigned to A 1 as to A 2 At this point, we have already assigned at least B/2 queries to A 2. S, y B/2 57

58 General Result: > 2 advertisers Worst competitive ratio of BALANCE is 1 1/e = approx Interestingly, no online algorithm has a better competitive ratio! 58

59 General version of problem So far, all bids = 1, all budgets equal (=B) Arbitrary bids, budgets Consider query q, advertiser i Bid = x i Budget = b i BALANCE can be terrible Consider two advertisers A 1 and A 2 A 1 : x 1 = 1, b 1 = 110 A 2 : x 2 = 10, b 2 = 100 Suppose we see 10 instances of q BALANCE always selects A 1 and earns 10 Optimal earns 100! Competitive ratio: 0.1!! Generalized BALANCE scheme with competitive ratio of 1 1/e Modify BALANCE to be based on fraction of unspent budget, and some others. 59

60 Actual problem is even more sophisticated Upon each search, Interested advertisers are selected from database using keyword matching algorithm Budget allocation algorithm retains interested advertisers with sufficient budget Advertisers compete for ad slots in allocation mechanism Upon click, advertiser charged with pricing scheme CTR updated according to CTR learning algorithm for future auctions 60

61 Conclusion Data streaming applications are becoming increasingly important Online algorithms are needed We have looked at data stream processing in general, and web advertising in particular 61

62 Acknowledgements The slides is adapted from numerous sources (not all listed here): Jure and Ullman (Stanford) Bergemann (Yale) 62