57% WHAT OF DATA SCIENTISTS HATE ABOUT THEIR JOB. Understanding and Optimizing Data Quality Assurance. By Clint Eagar

Transcription

1 WHAT 57% OF DATA SCIENTISTS HATE ABOUT THEIR JOB Understanding and Optimizing Data Quality Assurance By Clint Eagar

2 Introduction According to a recent study by CrowdFlower, data scientists are spending a hefty 60% of their time cleaning and organizing data, and 57% of these data scientists regard this task as the least enjoyable part of their work. By the time these scientists get to actually analyzing data for patterns, they end up spending only 9% of their time doing so. The need to clean data indicates a data quality problem, affecting anyone working with data or data collection technologies, including developers, analysts, digital marketers or decision-makers. So what do you do? 2

3 Table of Contents DIRTY DATA 5 THE SYMPTOMS 6 THE DATA QUALITY ASSURANCE JOURNEY 7 BUSINESS REQUIREMENTS/DOCUMENTATION 9 TECHNOLOGY DEPLOYMENT 10 QUALITY ASSURANCE 11 MONITORING & MAINTENANCE 12 THE DATA QUALITY ASSURANCE CYCLE 13 WHAT IS AUTOMATED DATA GOVERNANCE? 14 BE SURE OF YOUR DATA QUALITY 15

4 About the Author Clint proclaims to know a little bit about a lot of things. With more than 20 years experience in customer service and technical consulting and implementation, Clint has become an expert at building, optimizing and marketing websites. Passionate about data quality, Clint was one of the earliest employees of ObservePoint, joining the young company to help support, test, develop and manage the OP product. Clint is the guy everyone counts on to get things done. If you yell for Clint, he always comes running and always delivers. 4

5 Dirty Data It s easy to want to treat symptoms instead of diagnose the cause. Because it s usually an analyst or data scientist that discovers and deals with bad data, it may be tempting to try to solve the issue at the analysis stage. But if you re asking How can I make it faster to clean up data? then you re asking the wrong question. Things only need to be cleaned when they re dirty. Data isn t inherently dirty, not when collected correctly.? The question you need to be asking is, How did my data get dirty? 5

6 The Symptoms What exactly do I mean by dirty data? As explained in another other ebook, Why You Should Audit Your Web Tags, dirty data is a result of broken tags and broken business processes. The product of these broken assets is data inflation, data loss and data leakage. Data Inflation Data inflation is caused by data being collected more than once for the same measurement. This doesn t just mean duplicate entries in your CRM, it means your analytics data can be inflated giving you false perceptions of performance. Data Loss When you fail to thoroughly deploy tags across your site or mobile app, you re missing out on data, giving you an incomplete view of your users and giving your users a fragmented experience. Data Leakage The unauthorized transmission of data to third-parties or former team members caused by unsanctioned tags or outdated security measures can put your data and company at risk. These are serious threats to the integrity of your data, as well as to the decisions and operations that run on that data. 6

7 a The Data Quality Assurance Journey Bad data is usually indicative of a lack of transparency and accountability between data governance leaders and data stakeholders, and can be aggravated by pressure on developers and QA professionals to release new updates at an ever-increasing rate. It s important to identify at what point data gets lost, duplicated or leaked. Or rather, at what point data governance personnel fail to validate data collection processes. Here are the underlying business processes (the data quality assurance journey) that a data governance steward should be monitoring to ensure data is collected correctly: 1 Business Requirements Product owners/project stakeholders provide requirements for how analytics and digital marketing technologies are to be formatted in a digital asset. This includes variable and tag documentation (often known as a solution design reference or SDR). Many times measurement of a new feature is an afterthought, but should be part of the initial conversation. Technology Deployment 2 Developers deploy a TMS container code, and then TMS controllers deploy the required technologies into the digital asset using the TMS solution. Quality Assurance 3 QA professionals validate the implementation against the original business requirements, ensuring backend and front-end technologies are functioning as expected. Monitoring & Maintenance 4 Stakeholders continually ensure the digital asset functions as intended, fixing bugs and errors in the code base as users expose unique use cases. 7

8 ? Where do data end users like analysts and data scientists fit into the data quality assurance journey? Data end users often fulfill the role of TMS controller, responsible for deploying code to meet their own analytical needs. They can also help close the loop on data collection, providing feedback to business requirements personnel about potential events to track. These end users should just be able to trust the data, minimizing their role as data janitors and becoming data gurus, analyzing, mining and modeling data. However, they still spend much of their time cleaning up the data before they can perform their primary business function. Going about data quality assurance this way is inefficient, tedious and often frustrating for everyone involved. There is a better way. 8

9 Business Requirements Role: Provide requirements for digital marketing and analytics technologies to be included in the digital asset, including variable and tag documentation. To achieve your digital marketing and analytics objectives, you need a blueprint to understand how to architect your technology stack. This begins with identifying business requirements, which answer the following questions: What do we want to know about visitors to our website and mobile app? These business requirements are then translated into tag and variable documentation. The purpose of tag and variable documentation often known as a variable strategy or solution design reference (SDR) is to help maintain consistency across digital assets, as well as provide a baseline to perform quality assurance against. An SDR is typically a spreadsheet with the following information about each technology: How can we better help our visitors get value from our products? Tag Status Use Case Variable Variable Description What events do we want to measure? active or inactive what is being captured a prop, evar or event quick explanation of the variable How will we measure/capture this? Example what you expect the value to look like When to set page load, click or other Where to set which pages Additional Notes additional notes The SDR is meant to be living and accessible. Main Takeaway: Tag documentation needs to be fluid, but also needs to be a firm baseline to validate your technologies against. SDR maintenance is often neglected as technologies are changed and deployed. As digital assets are added and changed, so should the SDR. But SDR stakeholders often struggle to keep up with the rate at which technologies are changed, and thus lose their baseline to validate their technologies against when there is an error. The result is data governance personnel becoming blind to what stakeholders were hoping to accomplish with their analytics implementations. 9

10 Technology Deployment Role: Deploy TMS container code, and then deploy the required technologies into the digital asset using the TMS solution. With the advent of tag management systems, the responsibility of deploying tags has been passed from developers to data end users, such as analysts, marketers and data scientists. As TMSs have powerful rules engines and syntax validation, these personnel have been able to quickly deploy the tags that are most relevant to their business objectives. Consequently, organizations that use tag management experience a serious bump in data quality. But while tag management systems have streamlined and simplified tag deployment, there still remains gaps for human error. Tag management is not a substitute for tag validation. Misconfigured tags, conflicting JavaScript code, or tags deployed outside of a TMS can result in data quality issues issues that cannot be detected or reported by a TMS. Tag deployment via TMS should still go through rigorous quality assurance testing processes. Main Takeaway: TMSs are extremely helpful, but still prone to human error, and do not provide feedback on data quality issues such as broken tags or failed compliance rules. 10

11 Quality Assurance Role: Validate the implementation against the original business requirements, ensuring back-end and frontend technologies are functioning as expected. Unless an issue with a technology emerges during migration from staging to production or somehow crops up unexpectedly in the production environment then it falls under QA s purview to discover and resolve beforehand. This will ensure technologies work as needed. But even though it is best practice for quality assurance teams to preemptively resolve broken tags before production, unfortunately some broken tags proceed into production unaddressed. And it may not be QA s fault. Here s why: DevOps and QA are under increasing pressure to continuously release new features, technologies, web pages, app versions, etc. The release early, release often mantra is causing software release cycles to shrink, and QA professionals are feeling the pressure of finding and resolving bugs in less time. Unfortunately, these same QA professionals often are only equipped to respond to rapid release cycles with slow manual spot-checking methods one survey showed that while 67% of teams deploy new builds weekly, only 26% of teams have more automated testing than manual. Consequently, bugs can end up slipping through the system into production, undetected until they present a problem, either for overwhelmed monitoring & maintenance personnel, or exhausted/jaded data scientists and analysts. Main Takeaway: Quality Assurance teams are expected to deliver better products in less time, and yet often have to perform quality assurance through inefficient, manual spot-checking methods. 11

12 Monitoring & Maintenance Role: Continually ensure the digital asset functions as intended, fixing bugs and errors in the code base as users expose unique use cases. The role of monitoring & maintenance personnel is to continuously verify that the tracking technologies (tags) and critical user paths on your website or mobile app are functioning as expected. To do this manually, these postproduction QA professionals have to replicate user activity on the front-end production layer to look for bugs, as well as monitor back-end mechanisms to ensure functionality when new components are added to the mix. So they should be the ones to catch those final mistakes, right? Well, yes. But that can be easier said than done. As a company grows, and its website or mobile app matches that growth, then the number of digital assets post-production QA engineers have to monitor will just keep adding up. QA professionals often don t have scalable monitoring processes to catch errors, relying on manual tag validation methods. To put it simply, they re in over their heads, and are required to limit themselves to what they feel are the most important digital assets, leaving the rest out to dry. Main Takeaway: Monitoring & Maintenance have an ever-increasing load of digital assets to monitor, and scaling up their manual efforts is a challenge. 12

13 The Data Quality Assurance Cycle Up until now, we ve been treating the data quality assurance journey as a linear process. But at it s finest, the data quality assurance journey is best understood as a cycle: Monitoring/Maintenance Quality Assurance Data Quality Assurance Cycle Requirements/Documentation Technology Deployment Optimal, efficient data governance is forwardlooking when analysts and data scientists don t have to struggle to clean data, they can focus more on closing the loop with business requirements stakeholders, expanding the analytical capability of their tag implementations.? But how do you get there, when your company is facing serious challenges with data governance? In the face of a mountain of inefficient data governance processes, what you need is automated data governance. 13

14 What Is Automated Data Governance? Automated data governance solutions crawl your website or mobile app, firing code and validating tag presence/performance against predefined business compliance rules. This can save data governance personnel mountains of time, increasing efficiency and building confidence in data quality. 1 2 Automated Production Monitoring Enterprise-grade data governance tools can be configured to regularly audit large portions of your website or app, ensuring all technologies are working as intended. These solutions can also replicate traffic along critical user paths through your website or mobile app, validating the functionality of the tags that should be present along the journey. Automated Quality Assurance In addition to keeping track of the tag performance in a production environment, data governance solutions can also audit digital assets in a staging environment, assisting QA pros as they prepare to push the software into production. QA engineers can also create compliance rules to validate against their implementation, ensuring the digital asset holds up to business requirements. Automated Data Governance 3 4 Technology Deployment Assistance In addition to compliance-based audits and critical path monitoring, some data governance solutions provide tag debuggers to allow developers and TMS users to spot-check the performance of tags on a web page without having to read through any code. Enhanced SDR Documentation Data governance solutions dramatically simplify the process of generating and maintaining tag and variable documentation by scanning your website or mobile app and providing reports on all tags and variables found on the digital asset. These reports can also be used to augment your library of compliance rules to reflect your company s business requirements. 14

15 Be Sure of Your Data Quality ObservePoint s Data Quality Assurance is the premier data governance software for automating quality assurance, production monitoring and compliance validation. The ObservePoint platform relieves the pressure on data governance stakeholders by automating processes that, when performed manually, are highly monotonous, inefficient and prone to human error in order to monitor and validate your analytics and marketing tags with efficiency, confidence and accuracy. REQUEST A SAMPLE AUDIT to see what ObservePoint can do for you. 15