Anexinet s Business Intelligence Practice. Transforming Data into Insight

Transcription

1 Anexinet s Business Intelligence Practice Transforming Data into Insight 1

2 Bob Blackburn Sr. BI Consultant with 2

3 Why Agile BI? 3

4 Topics Waterfall vs Agile Modifying Agile Converting Team to Agile Agile Data Modeling 4

5 Agile vs Waterfall 5

6 Waterfall 6

7 Waterfall Pros Easy to measure against Know estimated cost up front Safe standard to follow Resources are often interchangeable Cons Relies on initial Requirements Testing is left until the end Hard to change direction Rewards over optimism. Price to Win 7

8 Agile 8

9 Agile Pros Allows for changes Re-prioritize often Collaboration over design Continuous testing Cons Harder to predict timeline and budget. Needs a strong team Needs significant time from business during project. 9

10 How is a BI project different then a typical agile OLTP project? 10

11 Web Architecture 11

12 Data Warehouse Architecture 12

13 Breadth of complexity Extracting Cleansing Integrating Transforming 13

14 Depth Conflicting data definitions Business Rules High data volumes 14

15 Modifying Agile 15

16 Modifying Agile 16

17 Agile Adaptations Creating another layer of requests under user stories 17

18 Agile Adaptations Creating another layer of requests under user stories Adding additional team roles 18

19 Agile Adaptations Creating another layer of requests under user stories Adding additional team roles Organizing the team into a pipeline of delivery activities 19

20 Agile Adaptations Creating another layer of requests under user stories Adding additional team roles Organizing the team into a pipeline of delivery activities Conducting two lead-off iterations 20

21 Agile Adaptations Creating another layer of requests under user stories Adding additional team roles Organizing the team into a pipeline of delivery activities Conducting two lead-off iterations Splitting user demos into two stages 21

22 Agile Adaptations Creating another layer of requests under user stories Adding additional team roles Organizing the team into a pipeline of delivery activities Conducting two lead-off iterations Splitting user demos into two stages Maintaining many small managed test data sets 22

23 Additional BI Roles Project Architect business model. Balanced tech solution Data Architect logical/physical model Business/System Analyst Processing patterns, transform rules, STTM System Tester Validation 23

24 Roles by Sprint Sprint Project Arch Data Arch Developer Testing -1 A+ 0 B A+ 1 C B A 2 D C B A 3 D C B 24

25 Converting to Agile 25

26 Converting to Agile 26

27 Converting to Agile BI: Stage 1 Time Box and Story Points See how much working code you can get done in a sprint. Introduce estimation, product charts, burn down and velocity Most likely fall into Waterscrum or Scrummerfall Can take 2 sprints 27

28 Converting to Agile BI: Stage 2 Pipelined delivery May be hard to keep everyone productive at first Colocation can now be seen as a benefit Introduce Pipelined development when productivity drops. When quality of hand offs reaches a high level, move to next stage Time: 2 4 Sprints 28

29 Converting to Agile BI: Stage 3 Developer Stories and current estimates Transition to developer stories Manage Sponsor s expectations. Change in estimation Develop new current estimate with buildup of experience. Time: 1 2 Sprints 29

30 Converting to Agile BI: Stage 4 Manage Development Data and TDD Eliminate data churn TDD Time: 2 4 Sprints 30

31 Converting to Agile BI: Stage 5 Automatic and Continuous Integration Fail Fast Can be used as a short cut to Agile if implemented early (stage 1) Time: 2 4 Sprints Conversion Time: 8 16 Sprints 31

32 Where Agile gets its speed Aspect Self-Organized Teams Direct Acceleration Y 80/20 Specifications Y Avoids time-consuming Mistakes Colocation Y Y Better Estimates Y Y Test-driven Development Coding starts early, feedback sooner Y Y Frequent review of deliverables Automated Testing Paying off tech debt early Y Y Y 32

33 Agile Data Modeling 33

34 Agile Data Modeling Focus on Business process rather then reports Avoid data dependency Collaborative modeling engages stakeholders JEDUF Just Enough Design Up Front Automated Testing and CI support Agile methods 34

35 7 Ws 7 W s Data Example Who People & Organizaions Employee, Customer What Things Product, Service When Time Date, Calendar Where Locations Store, Address Why Reasons & Causality Promotion, Weather How Transactions, Status Order ID (deg dim), Status How Many Measures, KPIs Revenue, Quantity (Facts) 35

36 Interview Who does what? When Where How Many Why How 36

37 Create examples Customer Product Buys Sells On at/from/to with/for for in/using Sales Person Sales Date Location Quantity Reason Manner Who What Who When Where How many Why How Joe Bottle Mike 5/11/15Store 123 2Walk in Order Sally Helmet Dan 5/12/15Store 456 1Coupon 321 Order? Jersey Karen 5/12/15Store 123 2Walk in Order 37

38 First Pass Logical Model 38

39 Modeling Dimensions What identifies each Customer/Product/(Dim)? What do you want to report on? What do you want to group on? Can Customer have more then one address? Is there a lookup table for this code? Mandatory? Missing? 39

40 Query Model Country Category Year Company Group City Quarter Type SubCategory Region Zip Month Brand Territory Customer Product Order Date Sales Person Promotion 40

41 Second Pass 41

42 Adventure Works Reseller Sales Fact 42

43 Additional Reading 43

44 44