The Road to Data Warehouse Utopia

Transcription

1 The Road to Data Warehouse Utopia

2 Utopia is a name for an ideal community or society, taken from the 1516 book written by Sir Thomas More describing a fictional island... possessing a seemingly perfect socio-politico- legal system... Utopia is sometimes used in reference to an unrealistic ideal that is impossible to achieve. Source: Wikipedia

3 Dystopia or anti-utopia is the vision of a society in which conditions of life are miserable and characterized by poverty, oppression, war, violence, disease, pollution, nuclear fallout and/or the abridgement of human rights, resulting in widespread unhappiness, suffering, and other kinds of pain. Source: Wikipedia

4 Dystopia Utopia

5 The Reality of the Road

6 The Reality of the Road

7 About Catapult BI Specialising in BI & Data Warehousing Strong business, risk and ROI focus Large Corporate & Government customer base Gold Sponsor AAIR DWSIG Forum 2008 Cognos Partner of the Year (Qld/NT) Our team has significant higher education experience (BI) and student admin systems

8 1. Doing Growth 2. Are we there yet? 3. The data is wrong! 4. Borrrrring! The Potholes 5. Why can t I get what I need? 6. Pressures of Demand 7. The Silver Bullet Syndrome 8. Who is looking at my data

9 Doing Growth It is clear a data warehouse can have a massive influence and strong business returns to every moderately complex organisation.. You MUST grow it or it will die. How do you nurture it and achieve growth?

10 Managing Growth Understand it s a journey, not a destination Get with the Program Include additional data sources Be ready to change

11 Are we there yet? Slide removed from presentation BI environments are always chaotic, have unpredictable performance, mixed workloads If your BI environment matures in the usual development path, it will inevitably get even MORE complex You MUST assume that performance will become a routine / chronic problem What planning have you made to handle this in your environment?

12 Managing Performance Understand the drivers of performance Manage end user expectation Flexible Architecture Archive Shorter ETL update cycles Design to Manage

13 The data is wrong! Question. End user gets their report, and you ve always found this person constantly on your case. They say The data is wrong.. and You know he s wrong.. what do you do? If you can t respond to this rapidly you ve lost the battle

14 Managing Data Quality Reconciliation integrity reporting Data lineage Undertake data quality assessment Use an automated tool (profile & manage) Design to Manage

15 Borrrring! Q. What do you do when you have rolled out a series of KPIs in reports/dashboards but within weeks no-one is looking at them? Q. What do you do when your dashboard goes past the RED zone? If your tool can t help you manage in adverse situations back through the RED zone it s also irrelevant?

16 Managing relevance KPIs for today Stay in tune with the business Pursue problem areas & work them! Deep analytical capability

17 Why can t I get what I need? Do you have issues of who s really in control of the DWH? Are there bottlenecks in getting anything done, gatekeepers frustrating you, what I need isn t in the DWH, too hard to use? The purpose of BI is enable us to better manage and control our organisations, yet we are often not in control of managing our BI programs! Many organisations will have a BI steering committee or control group. Q: Do you know that they truly understand what BI is all about?. and do you truly believe that they are committed to the BI program?

18 Managing direction Executive & business buy-in BI Program Management Promote promote promote!

19 Pressures of demand As your organisation matures, it is natural for progression from simple reporting to deeper analytical requirements. Are you ready for your architecture to change with new data, data source changes, data quality evolution (not always for the better), user needs changing, and data volumes escalating at fearful rates? What will you do when there is a business need for the warehouse to be used for things that it wasn t intended for? E.g. feeder to other application systems, audit, defacto CRM which might impact the whole dynamics of the DWH environment!

20 Managing demand pressures Be Proactive Archiving Adaptive architecture Bottlenecks in development methodology Design to Manage

21 The Silver Bullet Syndrome A successful BI project is determined more by how you go about the projects, governance, data issues, fit to real business needs, your skill and experience. Good tools won t guarantee success, however mediocre or poor tool choice will ensure problems even failure! Q: If you have a problem with a tool, what do you do? How good are your local support networks, how responsive is your vendor?

22 Tool tips Tool(s) that will grow with your business We have a problem! BI tools and performance Train users in both tool AND the data

23 Who is looking at my data! Slide removed from presentation The desire for data confidentiality and privacy, and sensitivity to our own failings & faults means data security in a DWH environment is extremely important. Q: Do you have clones of your DWH production data for development & test environments? Perhaps these environments might be more prone to security breaches and inadvertent data transmission? Q: What would happen if name and address details of students or staff inadvertently leaked? Do you need to do a risk assessment on possible data breaches? Q: Do you need to test for security tightness?

24 Managing data confidentiality Risk assessment on potential data breaches Test for security tightness Keep security models simple De-identify sensitive data in non-production databases

25 DESIGN TO MANAGE Design your DWH to take account of the many constraints and expected bottlenecks and problems that WILL occur in a real world data warehouse.

26 Where are you on the continuum? Dystopia Utopia

27

28 Questions?