Data Governance and Data Quality Stewardship 1
Agenda Discuss Data Quality and Data Governance Considerations for future technical decisions 2
Intelligence Portal Embedded InfoApps Hot Social Bad Feedback Predictive Analytics Sentiment and Word Analytics Search Location Analytics Mobile Write-Back Data Discovery Reporting Dashboards Casting and Archiving Active Technologies High-Performance Data Store Integrity Data Quality Data Governance Master Data Management Integration Batch ETL Real-Time ESB Legacy Systems Applications Relational/Cubes Big Data Columnar/In Memory Unstructured Social Media Web Services Trading Partners 3
4
Which ones are bigger? Tell yourself they are the same, it doesn t matter! 5
Data Value Chain Data Business Value Connect Move Fix Relate Govern Business Intelligence Data Batch Profile Master Data Monitor Dashboards Application E-Business Legacy SaaS Big Data Transactional Event Driven SOA Automate Orchestrate Clean Enrich DQ Firewall Organize Synchronize 360 View Visualize Alert Remediate Report History Analytics Ad Hoc Reports Enterprise Search Mobile Visualize Predictive Social Intelligence Performance Mgt Integration Integrity Intelligence
Anyone Facing these Challenges? Difficult to produce accurate customer count Not clear who owns data Different answers for the same question LOB manage their own data in Excel Duplicate data across systems Resources tied up researching and fixing data issues Errors in processing data due to incomplete data No single version of the truth 7
Why Information Management Barriers to Information Management Address disparate, dirty and timeliness of data Data Spread Across Too Many Apps and Systems Multiple Versions of the Truth Data Not Timely Enough 60% 64% 67% Data Not Clean Enough To Use Technology Not Able to Meet Needs 58% 57% Source: Ventana Research Information Management Benchmark Research
Data needs to be treated as a Business Asset Gartner has researching the concept that information is an under-managed, underutilized asset because it's not a balance sheet asset. Gartner 25% of critical data is flawed Not ALL data should be managed equally 9
10 How can we think about Data Quality? Potential Energy = m * h * g There will not be a test after the talk!
Data Quality Potential of Data How can we think about Data Quality? Time Optimal Data usage gives continual energy to your organization 11
Data Stewardship enables Data Quality What are common aspects of Objective Data Quality? Available Usable Reliable Accessible Defined Consistent Authorized Recognized Accurate Timely Structured Complete Efficient Auditable 12
Data Stewardship enables Data Quality What are common aspects of Subjective Data Quality? Trust Understandability Interpretability Objectivity Timeliness Relevance 13
Benefits of Data Quality Business Function Benefit of Data Quality, Governance, and MDM Finance and Corporate Enhanced and Accurate Reporting Efficient Planning and Budgeting with increased granularity Enhanced ability for regulatory compliance Improved decision making based on accurate data Sales, Marketing and Customer Service Single View of Customer (increases customer satisfaction) Enable better interaction with customers across touch points Ability to cross sell and up sell products and services Accurate Install Base information HR Improved productivity with efficient processes Reduced errors IT Reduced staff for data cleansing tasks Improved productivity of standards based application development
Remediation and Data Stewardship Data stewardship is the management and oversight of an organization's data assets to help provide business users with high-quality data that is easily accessible in a consistent manner. How is it maintained? How is it used? How is it consumed? 16
Success in Data Quality relies on the harmony of People Process Technology
The Harmony is supported via Data Governance Data Governance is supported via Remediation People Process Technology
People People Process Technology
Element of Success: Data Governance Start with a realistic goal Provide a plan Be clear with roles & responsibilities Marketing! Marketing! 20
Information Management Maturity Quadrant Where is your organization? Sporadic Early Siloed Executed Wide-spread Replicated Systemic Governed
5 Roles of a Data Steward Lead Promote and drive practical governance guidelines throughout an organizations data ecosystem Map Work with the business to understand data needs and find better or more efficient processes. Then, the steward can craft appropriate processes to the data uses. Define Be an Expert Advocate Per Forrester, A data steward should drive the implementation and enforcement of requirements for services including, but not limited to, data management, data brokering, customer data integration and data appends. To further drive organizational consistency, the data steward should participate in the vendor evaluation process as necessary to ensure compliance. A data steward must keep up-to-date on changes to data-related legislation for external business implications as well as internal communications and compliance. Along with daily roles, the data steward should be a point person on the organization s data evolution. Meaning as new data requirements arise, they must advocate that data governance best practices continue to be utilized. 22
Success in Data Quality relies on the harmony of People Process Technology
Deciding on the right Process Which Factors to consider Current data skills Company culture Data reputation Current opinion on data ownership Maturity of KPI Culture Reusability of data. 24
Allocating Resources to the Stewardship Process Data Stewardship can be broken into 5 different patterns of allocating the remediation process responsibilities By Subject Area By Function By Business Process By System By Project 25
Stewardship by Subject Area Each Data Steward is in charge of their own subject area. One is in charge of customer and another is in charge of product. Product Location Customer Data Management & Data Governance Processes Vendor 26
Stewardship by Subject Area Pros Boundaries are clear Subject Knowledge Grows over time Cons Focus may be at expense of broader business benefits (Customer Retention for example). Size differences of Domains. Might be difficult to tie Data Steward back to business initiatives. 27
Remediation by Function Each Data Steward focuses on their line of business or department. Such as Marketing or Finance. Finance Sales Customer Service Logistics Marketing Business Rules & Standards ERP CRM Inventory FMS 28
Remediation by Function Pros Bounded by the organization means easier to establish definitions and rules. Will be business-savvy and familiar with the data s context They know the team Cons Multiple data stewards in different departments may be managing and manipulating the same data. The nature of this model means that data stewards are rarely motivated to collaborate across functional boundaries Functional data stewardship won t work in companies that have prioritized enterprise-class single view initiatives or consolidation programs. 29
Remediation by Business Process Each Data Steward is assigned to a single business process. For example Sales or Enrollment. Start End Sales Start End Enrollment Start End Procurement Start End Reporting Data Management & Data Governance Processes Tip : For very mature data-driven organizations 30
Remediation by Business Process Pros Extension of exiting processes Success measurement is more straightforward The process oriented model is a very effective way to entrench data stewardship. Cons Data ownership is more difficult to assign. A broader data governance program is critical for managing such situations. Business constituents can get confused. Consistency around similar types of data. In this model, data stewardship is only as effective as the company is clear about its processes. 31
Remediation by System Data Steward is assigned to the system that they manage the data for. Such as SAP ERP or Salesforce. ERP CRM Inventory Data Management & Data Governance Processes FMS Tip : This may have caused some of the data quality issues in the first place. 32
Remediation by System Pros IT can take a leadership role Drives from a Bottom Up approach Assigning multiple data stewards at once is more realistic: each core system will have a data steward becomes an established practice. Cons Business people may equate data ownership with data stewardship, thus assuming stewardship to be an IT issue Data stewards can become myopic as they maintain the integrity of the data on their systems A systems orientation doesn t ensure data sharing or reconciliation. 33
Remediation by Project Data Steward is assigned to a project that they will manage the data for. Can be assigned through the PMO office. Examples are a Data Warehouse Implementation or ERP Migration. Project Management Office Data Management & Data Governance Processes Tip : This can be the fastest way to introduce the role to the organization 34
Remediation by Project Pros Speed! It is part of the Project and most organizations can add that as part of a process easily. Start with Project then Grow Clear definition of success Cons Project implies ending Are skills in house? 35
Success in Data Quality relies on the harmony of People Process Technology
Changing the landscape 37
Data Integration, Quality, and Mastering Typical Historical Approach End State MDM hub Data warehouse Partner interface Operational systems BI/analytics app Quality process 38
Data Integration, Quality, and Mastering Agile Approach to MDM, Data Quality, and Data Integration End State MDM hub Data warehouse Partner interface Operational systems BI/analytics app Quality process 39
Traditional in Transition to Modern More use cases EII IoT Hadoop Virtual DW Data Lake Fewer use cases Point-to-point Integration OLTP Traditional OLAP Data warehouses Data marts Streaming Modern 40
The Evolution of Integration Hand Coded Integration ETL Messaging Bus EAI ESB Hadoop-Based Integration 41
We Have Some Pretty Simple Problems According to a May 2015 Gartner Survey 26% are deploying Hadoop, 11% in 12 months, 7% in 24 months 49% cite trying to find value as their biggest problem 57% cite the Hadoop skills gap as their biggest problem To summarize Companies are investing in Hadoop, but not sure why Companies are investing in Hadoop, but don t know how to use it 42
Big Data Under the Control of Master Data Hadoop can be: Staging area for application data Source for mastered subjects Source for transactional subjects Master Data Repository Golden Records Master data can: Provide context to Hadoop data Establish trust in big data Guide extraction of Hadoop data Hadoop
Want More Information? 44