The Shift to Quality Engineering

Transcription

1 The Shift to Quality Engineering Bart Gourley 2016 Accenture Testing Symposium February 2016

2 Today s Discussion Industry/market pressures and the quality challenge Trending responses for testing organizations Structural transformations (enabling a quality factory) Rollout of new roles Shifts to new/different tools What s next? Copyright 2016 Accenture All rights reserved. 2

3 Industry/Market Pressures And the Quality Challenge Industry change in is disrupting internal quality process, putting some testing organization at risk if they fail to adapt. Industry Drivers of Change Growth of new and potentially disruptive B2C IT products in historically stable markets Uber and Shared Car Services Cable and Content Streaming Services Theaters and Direct to On Demand Challenges for Testing Organizations Stakeholders: Integration with development, business, and customer partners within the quality workflow often requires continuous and active collaboration through non-it traditional methods (e.g. waterfall vs. agile/lean) Methodologies: Structured multi-speed IT test methodologies (e.g., internal industrialized agile, waterfall, extreme, hybrid approaches) are required where they often do not exist. Continued downward cost pressures Budget alignment with product market performance and value Increasing business agility requirement pressures from dynamic client demands Crucible for Transformation Market feedback rates have increased (app store, real-time surveys, etc.) Focus: Engineering for quality is replacing testing for quality. QA is asked to delivery transparency into root causes of quality impacts both up and downstream from testing (e.g., shift left and shift right). Skills: New/Non-traditional QA roles and skills (e.g., SDET, Techno testers, Quality Czar, etc.) are required. Value: The challenging economic environment retains a focus on higher value SDLC activities. However, innovation has been slow to respond to creating more value at lower costs. Copyright 2016 Accenture All rights reserved. 3

4 Quality organizations are seizing the moment to evolve not simply to survive, but to elevate their value. Challenges What actions are others taking? What benefits are they seeing? Stakeholders Methodologies Focus Skills Value Test organizations are moving away from being external to the development team and towards being part of the engineering team through the staffing of roles with new skills (e.g., SDET, QA engineer, etc.) Test organizations are shifting left and right to influence and to sometimes control quality across the delivery. Centralized test organizations are shifting away from Testing Center of Excellences towards Quality Center of Excellences and are held accountable for more than defect leakage Test organizations often need to reorganize from a structure and capability perspective to support the new engineering models and/or requirements Tighter integration with internal and external (market) stakeholders Increased perceived value by stakeholders Decreases in UAT defect leakage and improvement in market feedback Increased transparency into the true drivers of quality (e.g., development code readiness green builds, requirement readiness, and crowd acceptance) Copyright 2016 Accenture All rights reserved. 4

5 Structural Transformations > Enabling Quality Engineering Industry is redefining how software is developed, tested, & delivered. Lean engineering is rapidly evolving the QA space. The intersection of DevOps, Traditional QA, Development, and Market Feedback Leaders in the space envision the SDLC as a lean process that, like any other manufacturing process, has waste that should be reduced to optimize for cost, time to market, and market relevance. Shifting left and right removes traditional test stages and gates and blends quality measures into each function of the engineering process. Core Concepts Quality is a by-product of the engineering process, not a point-in-time function Quality activities are deeply embedded into release engineering (DevOps), product performance (business/market definition and success monitoring), and technical development Quality organizations and their activities are lean at the core and scale variability for execution Common Traits of Mature Quality Factories Technical level quality activities (TDD, BDD, etc.) are formalized and SDETs (or the equivalent function) are prolific supporting these functions Quality oversight and control is injected into each part of the SDLC Copyright 2016 Accenture All rights reserved. 5

6 Structural Transformations > Example Run State A holistic quality engineering approach places more testing and automation left in the delivery workflow. Copyright 2016 Accenture All rights reserved. 6

7 Structural Transformations > New Role Definition Establishing a Quality Czar and SDET capability is a journey that typically starts with identifying muti-functional/ utility resources and ends with those resources engineering portions of the production solution*. - Quality Czar: SDLC experienced resource that has a mixed skill background of PM, QA, Development, and Release Engineering. The resource provides governance and input into all engineering activities to drive for quality outcomes - SDET: technical testing resource that frequently provides low level automation and manual testing services within a sprint team. - QA Analyst: functional testing resource that responsible to provide behavior and/or business transaction validation of the solution within an integration team. SDET Journey Example Forming Norming & Storming Performing The SDET term is applied to any non-development/test resource that is providing testing within the development team. Typical Environmental Factors/Prerequisites: None * Conceptual and for discussion only The SDET term is applied to any engineering resource that is providing low-level automation and test coverage that is stored within the source code repository. Typical Environmental Factors/Prerequisites: Development and development integration leveraged in testing The SDET term is applied to any engineering resource that is providing low-level automation and test coverage that is stored within the source code repository, providing coverage and execution integration into the continuous delivery solution, that collaborates with integration level validations from external teams and that delivers within an agile methodology structure. Typical Environmental Factors/Prerequisites: Continuous Integration solution is leveraged Copyright 2016 Accenture All rights reserved. 7

8 Structural Transformations > New Role Definition Example role functions within an DevOps driven development workflow Test driven development: Developer cuts feature branch, writes test case, tests, writes code, tests and submits for review Creates automated and manual tests for continuous validation activities Creates and integrates unit tests into code base Once code review is approved, Git with Atlassian Stash is used to merge the code update with development branch Bamboo begins automated build & deploy process Gulp provides Automated task scripting Automated Smoke Testing (Build Validation) JS Hint, JSRC, HTMLLINT rand other code quality run and perform analysis Istanbul Gulp runs automated Karma behavior with Jasmine assertions and test reporting with Istanbul Gherkin Behavioral driven development: Automation tester writes Gherkins scripts that become inputs for drafting unit test cases. Cod e artifacts, including applications are stored/retrieved in NPM and Bower JSON used to integrate with Rest API Creation and execution of E2E, full integration, performance, crowd, and other testing. Test results are sent to SonarQube server for analysis Analysis of testing results and determination of followup / action items Feedback is passed to developers QA Testers test on multiple devices (Automated Regression (selenium) + Manual Functional) Code is deployed to environments by Bamboo via Docker Code is packaged & readied for deploy by Gulp Quality Czar SDET Activity Developer Activity Copyright 2016 Accenture All rights reserved. 8 QA Analyst Activity Automated Activity

9 Structural Transformations > Shifts to new/different tools Tooling needs to support a quality engineering workflow blend functional traditional and technical development tools Continuous Delivery * Conceptual and for discussion only Architecture Relevant Tooling landscapes are evolving to support a structure of core+. The core is centrally supported where as the + is supported only within the scrum or application stack. Tooling focus is on supporting the continuous and often automated delivery and analysis of quality measurements. Cloud Enabled Tooling is more frequently ondemand and cloud supported either through SaaS or frameworks that leverage dynamic infrastructure Workflow Tooling scope and breath looks at the entire SDLC and promotes adoption of requirement capture and traceability to deployment performance. Example: BDD Cucumber & JBehave. Copyright 2016 Accenture All rights reserved. 9

10 Questions?