BI with Best-Practice Architectures and Data Models

Transcription

1 BI with Best-Practice Architectures and Data Models Business Analytics Insight October 2015 Bart De Soete

2 2 AGENDA 1. Timeline 2. Normalized modelling 3. Dimensional modelling 4. Data Vault 5. How it all comes together 6. References

3 3 USG Professionals A strong foundation with years of experience

4 Timeline Relational modeling (E.F.Codd) Bill Inmon began discussing Data Warehousing Barry Devlin and Dr Kimball release Business Data Warehouse Bill Inmon popularizes Data Warehousing Dr Kimball popularizes Star Schema Dan Linstedt begins R&D on Data Vault Modeling Dan Linstedt releases first 5 articles on Data Vault Modeling 2012 : Dan Linstedt announces Data Vault : Dan Linstedt releases Data Vault 2.0 specs

5 Normalized modelling - Bill Inmon Definition : A data warehouse is a subjectoriented, integrated, time-variant and nonvolatile collection of data in support of management s decision making process - Stay as close as possible to the operational model - Only adding historization

6 Normalized modelling - Bill Inmon Advantages - Easy ETL to populate the model - Prevents redundancy Disadvantages - Difficult to understand for business people - Not resilient to change every small change in the source system impacts the model

7 Dimensional modelling Ralph Kimball Definition : A data warehouse is a copy of transaction data specifically structured for query and analysis. Data warehouse is the conglomerate of all data marts within the enterprise.. Information is always stored in the dimensional model - Focus on query and analysis through facts and dimensions - Integration through conformed dimensions

8 Dimensional modelling Ralph Kimball Advantages - Understandable for business users - Build specifically for reporting and analysis - Designed to handle historical data Disadvantages - Complex ETL due to integration of business rules - Ever growing development cycles - Limited enterprise view

9 Data Vault Dan Linstedt Definition 1.0 : The Data Vault is a detail oriented, historical tracking and uniquely linked set of normalized tables that support one or more functional areas of business. It is a hybrid approach encompassing the best of breed between 3rd normal form (3NF) and star schema. Definition 2.0 : A system of business intelligence containing the necessary components needed to accomplish enterprise vision in Data Warehousing and Information Delivery

10 Data Vault Dan Linstedt Data Vault 1.0 is focused on modelling, Data Vault 2.0 is a complete package Advantages - Flexible and scalable - Based on Hub & Spoke - Integration with NoSQL Hadoop - Pattern based ETL development Disadvantages - Explosion of tables - Difficult to query

11 How it all comes together - To choose is to lose - Use best of all breeds - Divide and conquer

12 How it all comes together Enterprise Service Bus Sales Finance Staging EDW Data Vault Business Vault Information HR Raw

13 How it all comes together Enterprise Service Bus Sales Finance Staging EDW Data Vault Business Vault Information HR Raw

14 How it all comes together Enterprise Service Bus Sales Finance EDW Data Vault Business Vault Information HR Raw - Move to real-time BI by replacing Staging with ESB

15 How it all comes together Enterprise Service Bus Sales Finance EDW Data Vault Business Vault Information HR Raw Virtual layer - Move to real-time BI by replacing Staging with ESB - Virtualize Business Vault and Data (Powerful hardware needed)

16 How it all comes together Enterprise Service Bus Sales Finance EDW Data Vault Business Vault Information HR Raw Virtual layer - Move to real-time BI by replacing Staging with ESB - Virtualize Business Vault and Data (Powerful hardware needed) - Move business rules to ESB to allow business to take ownership

17 Reference material Inmon : Kimball : Data Vault : Data Modelling zone :

18 Thank You