CSE 2331/5331. Topic 3: Prob. Analysis Randomized Alg. CSE 2331/5331

Transcription

1 Topic 3: Prob. Analysis Randomized Alg.

2 Expected Complexity Probabilistic method: Given a distribution for all possible inputs Derive expected time based on distribution Randomized algorithm: Add randomness in the algorithm Analyze the expected behavior of the algorithm

3 First Example What is worst case time complexity? What is expected / average time complexity?

4 Expected Running Time Expected / average running time EEEE nn = II Pr II tt II Pr II = probability of event I tt II = running time given event I To analyze, need to assume a probabilistic distribution for all inputs

5 SeqSearch Alg Expected running time = nn Pr K A t K A + Pr AA ii = KK tt AA ii = KK ii=1 If we assume Pr K A = 0 All permutations are equally likely implies Pr AA ii = KK = 1 nn Then expected running time = ii 1 nn cccc = Θ nn

6 Remarks For probabilistic analysis An input probabilistic distribution input model has be assumed! For a fixed input, the running time is fixed. The average / expected time complexity is for if we consider running it for a range of inputs, what the average behavior is. Randomized algorithm No assumption in input distribution! Randomness is added in the algorithm For a fixed input, the running time is NOT fixed. The expected time is what we can expect when we run the algorithm on ANY SINGLE input.

7 Randomized Algorithms

8 Expectation X is a random variable The expectation of X is EE XX = II Pr XX = II II E.g, coin flip Linearity of expectation: EE XX 1 + XX 2 = EE XX 1 + EE XX 2 Conditional expectation: EE XX = EE XX YY) Pr YY + EE(XX NNNNNN YY 1 Pr YY

9 Linearity of Expectation EETT 1 nn = nn EETT 2 nn + cn EETT 2 nn = expected running time for func2 What is EETT 1 nn?

10 Conditional Expectation EETT 1 nn = Pr heeeeee EETT 2 nn +(1 Pr heeeeee ) EETT 3 nn + cc EETT 2 nn = expected running time of func2 EETT 3 nn = expected running time of func3 What is the expected running time of func1?

11 A Randomized Example Worst case complexity? Expected case?

12 Running Time Analysis Worst Case: TT nn = Θ(nn 2 ) Expected running time: EEEE nn = nn ii=1 Pr kk = ii (ccii 2 ) = nn 1 ii=1 nn (ccii2 ) = cc nn ii=1 nn ii 2 = Θ nn 2

13 Another Example

14 Running time analysis Worst case: TT nn = nn ii=1 ccii 3 2 = Θ nn 2.5. Expected running time: Lines 4 8 takes time: cckk 3 2 Let XX ii denote the running time for the i-th iteration of outer forloop (i.e, from line 3 8) EE XX ii = ii qq=1 PPPPPPPP kk = qq iiii ttttt iiiii iiiiiiiiiiiiiiiiii TTTTTTTT llllllllll 3 8 kk = qq = ii 1 qq=1 ii ccqq 3 2 = cc ii ii qq=1 qq 3 2 = Θ(ii 1.5 ) Hence we can assume that EE XX ii = cc ii 1.5 EEEE nn = EE nn ii=1 XX ii = nn ii=1 EE XX ii = nn ii=1 cc ii 1.5 = Θ nn 2.5

15

16 Time Analysis Worst case: TT nn = Θ nn 2 Expected running time: EEEE nn = Pr kk < nn EEEEEEEEEEEEEE(kk < nn) + Pr kk nn EEEEEEEEEEEEEE kk nn = nn nn ccnn2 + 1 nn nn (cc nn lg nn) EEEE(nn) = Θ(nn 1.5 )

17 A more complicated variation.

18 Analysis Worst case: TT nn = Θ(nn lg nn) Expected case analysis (1) EEEE nn = Pr kk < nn EEEE llllllllll 4 7 kk < nn + Pr kk nn EETT llllllllll kk nn = nn nn nn ii=1 Pr kk = ii kk < nn EETT llllllllll 4 7 kk = ii + Pr kk nn EETT llllllllll kk nn = 1 nn nn ii=1 1nn ccii2 + 1 nn (cc nn lg nn) nn = Θ( nn) + Θ nn lg nn = Θ(nn lg nn)

19 Analysis Expected case analysis (2) EEEE nn = nn ii=1 Pr kk = ii EETT llllllllll 4 7 kk = ii + Pr kk nn EETT llllllllll kk nn nn 1 = ii=1 nn ccii2 + 1 nn (cc nn lg nn) nn = Θ( nn) + Θ nn lg nn = Θ(nn lg nn)

20 An Example with Recursion

21 Worst case: T n = T n 1 + cn = Θ nn 2 Expected running time EEEE nn EEEE nn = cccc + 1 EEEE nn 1 2 Solving this we have EEEE nn = 1 ET n 1 + cn 1 EEEE nn + ccn EEEE nn cccc EEEE nn 2cccc EEEE nn = OO nn EEEE nn = Θ nn

22 Another Example with Recursion

23 Analysis Worst case: TT nn = cc + TT nn 1 + TT nn 2 TT nn = Ω 2 nn/2! Expected running time: EEEE nn = cc EEEE nn 1 + EEEE nn 2 EEEE nn cc + 1 ET n 1 ET n cc 2 2 EEEE nn 2cc EEEE nn = O(1) EEEE nn = Θ 1!

24 Another Example

25 Analysis Worst case: TT nn = 2TT nn 2 + cccc = Θ nn lg nn Expected running time: EEEE nn = 1 4 EEEE nn 2 + EEEE nn 2 + cccc = 1 2 EEEE nn 2 + cccc EEEE nn 1 EEEE nn + cccc 1 EEEE nn cccc 2 2 EEEE nn = OO nn EEEE nn = Θ nn

26 Example 5

27 EEEE(nn) = expected running time of FFFFFFFFF on nn values

28 Bounding EEEE(nn) PPPP kk < nn 4 = 1 4 ; PPPP nn 4 kk 3nn 4 = 1 2 ; PPPP kk > 3nn 4 = 1 4 ;

29 Upper Bound

30 Upper Bound cont. EEEE nn cccc EEEE nn (EEEE nn 4 + EEEE 3nn 4 )

31 Lower Bound In the best case, kk = nn 2 Hence Putting upper and lower bound together, we have: EEEE nn = Θ nn llll nn

32 Example 6

33 Recurrence for Func6 EEEE(nn) = expected running time of FFFFFFFF6 on nn values

34 Bounding ET(n)

35 Upper Bound

36 Lower Bound Whatever kk is, EEEE(kk) is at least cccc. Hence EEEE(nn) = Ω(nn) Putting upper and lower bounds together, we have EEEE(nn) = Θ(nn)