Geiger - 1
It is important to note that data quality while critically important to business intelligence is also critical to the operations of the enterprise. Data Quality Management covers all operational systems, ODS, and BI components as well as the processes behind their construction and usage. Similarly, master data management (MDM) also depends on data quality. Data profiling provides an understanding of the existing environment so that appropriate steps can be taken to meet quality expectations. Geiger - 2
Geiger - 3
Geiger - 4
Geiger - 5
Deming s first point is to create constancy of purpose for improvement of product and service. The thrust of this point is that we need to move from a short-term focus to a long-term focus. The implication for data management is that we need to treat data management as a program and be willing to make the foundational investments that will pay-off over time. These investments include establishing a strategy, deploying a governance structure, creating and enforcing standards, developing models top-down, understanding the quality of existing data, and committing to continuously improving data quality. Geiger - 6
The planning process must be driven top-down. The data management strategy sets the direction and becomes the basis for the tactical and operational plans. These in turn drive the actions so that the highest priority tasks are performed within the context of the overall program. Geiger - 7
Deming s second point is to adopt the new philosophy. The thrust of this point is that we should establish an environment that relies on our ability to prevent problems rather than relying on an environment that fixes them. The implication for data management is that we need to establish repeatable practices for all aspects of data management and that these practices need to address the full data lifecycle. Geiger - 8
Deming s third point is to cease dependence on mass inspection. The thrust of this point is that while inspections prior to delivery may be needed, their purpose is not to create quality. Quality must be created throughout the process. The implication for data management is that we need to ensure that all development and data management steps include activities that verify the quality of the product before it moves to the next step. Further, the person performing the activity is ultimately responsible for the quality. Techniques include self-inspection, peer reviews, audit and control points, testing, reconciliation, etc. Geiger - 9
Deming s fourth point is to end the practice of awarding business on price tag alone. The thrust of this point is that awarding business to the low-cost bidder is not necessarily a good decision. That practice leads to too many suppliers for similar products, product variability, and supplier turnover. The implication for data management is that we need to evaluate factors beyond price when we look for products and services. There are benefits to be gained by limiting the number of suppliers and building strategic longterm relationships with them. There are also significant costs that are encountered when we have multiple suppliers and products essentially providing the same capabilities. Geiger - 10
Deming s fifth point is to improve constantly and forever the system of production and service. The thrust of this point is that quality is not a one-time effort. It requires a vigilant effort to constantly look for ways to eliminate waste and improve quality. The implication for data management is that we need to be on the lookout for wasted resources. Examples include the inefficient processes used to find, validate, reconcile, verify, and fix data that does not meet quality expectations as well as uncontrolled redundant data, dormant data, and processes that move data that is not ultimately used. At the heart of the program is an approach that strives to continuously improve quality. Geiger - 11
This is a continuous process. Ideally, one would start with planning. In most companies, something is already in place and is operating. If nothing proactive is done, these companies begin with doing. Some companies recognize that they have data quality issues in their source systems, and as they embark on their business intelligence initiatives, they try to understand their data through data profiling. These companies begin with checking. Some companies wait till problems occur and then deal with them through acting. Unless these companies move onto the planning step, they are destined to continue to react to problems with no improvement in the environment. Geiger - 12
The Fishbone and Pareto Diagrams can be used in combination as well Geiger - 13
Measuring the result is often the simple part. The difficulty is in determining what the results mean. If control charts are used, then the relationship of the value to the target, upper control limit, and lower control limit is significant in pointing the direction of the analysis. For example, a value that is within the target and below the lower control limit (with low being good) indicates that performance was better than should have been expected from the process and that something unusual probably happened outside of our process. This warrants investigation as it may identify actions for improving the environment in a way that can ultimately improve the results from the process. Geiger - 14
Deming s sixth point is to institute training. The thrust of this point is that people must be properly equipped to perform their jobs. The implication for data management is that we need to understand that instruction manuals are not sufficient. We need to ensure that data creators, maintainers, and users understand the related business rules and how to apply the tools at their disposal. In addition, data users need education to help them understand how they can best leverage the available information for corporate advantage. Geiger - 15
Deming s seventh point is to institute leadership. The thrust of this point is that the manager s job is to lead not to micromanage or use punishment as an incentive. The implication for data management is that we need to the key stakeholders the CIO, data stewards, and data management leaders must lead by example. In addition, once the staff is properly trained and understands how to perform the job, the leaders must provide latitude for them to do so. Geiger - 16
Deming s eighth point is to drive out fear. The thrust of this point is that people often don t fully understand their job and are afraid to ask. We must build an environment in which people feel secure. The implication for data management is that we provide a ready reference of information about the environment, train people on its use, and then provide any needed assistance. Another application of this principle is to ensure that people are not afraid to express ideas. Within the data management world, we need to ensure that when people have concerns about the data or the way its managed, they are encouraged to espouse their thoughts with no fear of repercussions. Geiger - 17
Deming s ninth point is to break down barriers between staff areas. The thrust of this point is that people throughout the organization must work together to achieve the best results. Managing data requires horizontal cooperation throughout the organization. The implication for data management is that we need to pursue and promote partnerships. Information Technology and the business units need to work together effectively Business units need to cooperate and share information to achieve an enterprise perspective Different levels of management must understand each other s needs Different groups within Information Technology must support each other. Geiger - 18
Deming s tenth point is to eliminate slogans, exhortations, and targets for the workforce. The thrust of this point is that management must recognize that its job is to provide an environment in which workers can succeed. Slogans imply that the workers could perform better if they tried harder, even if the proper environment does not exist. The implication for data management is same. The leaders must provide an environment that promotes cross-functional teamwork to arrive at acceptable data definitions, business rules, and quality expectations. Merely promoting that our goal is one version of the truth without the appropriate governance, education, and other tools to do so is not appropriate. Geiger - 19
Deming s eleventh point is to eliminate numerical quotas. The thrust of this point is that numerical quotas stress task completion but not quality. For example, if customer service representatives are appraised based on the number of calls handled, they may strive to shorten calls rather than properly addressing the issue. The implication for data management is that we need to stress usable definitions rather than the number of definitions completed. The same holds true for other data management products. Geiger - 20
Deming s twelfth point is to remove barriers to pride of workmanship. The thrust of this point is that the workers inherently want to do a good job and that management actions and emphasis areas can inhibit their ability to do so. The implication for data management is that latitude needs to be given to stewardship and other teams to accomplish their objectives. The stewardship team needs to learn how it can best operate to meet the goals. Geiger - 21
Deming s thirteenth point is to institute a vigorous program of education and retraining. The thrust of this point is that management jobs change over time, and people need to continuously acquire new knowledge and skills to deal with the changing environment. The implication for data management is same. As the data management role evolves both its team members and its customers need to understand the new capabilities that are being provided and any improvements in the tools set that is available. Effective metadata management and distribution is a major component in providing the needed information as it evolves. Technical Metadata t - metadata t that t describes the physical structures in the CIF and the detailed processes that move and transform data in the environment. Business Metadata - metadata that describes the data structures, data elements, business rules, and business usage of data in the CIF. Administrative Metadata - metadata that describes the operation of the CIF, including audit trails, performance metrics, data quality metrics, and other statistical metadata. Geiger - 22
Deming s fourteenth point is to take action to accomplish the transformation. The thrust of this point is that it is not enough to have a set of principles. Management must initiate actions to ensure that the principles are implemented. The implication for data management is same. The leaders, through the governance structure, must provide an environment in which the data management principles can be effectively pursued. A set of meaningful guiding principles helps ensure that the direction is clear. Geiger - 23
Geiger - 24
Geiger - 25
Geiger - 26