The Quality Quotient Insightful Strategies for Quality IT Project Releases. October 24, 2015

Transcription

1 The Quality Quotient Insightful Strategies for Quality IT Project Releases October 24, 2015

2 History On September 9, 1947, the Mark II technical team of Harvard University found a moth in the wiring of the computer system. Computer scientist and US Navy Rear Admiral Grace Hopper removed the moth, and taped it in the logbook where she made an entry: "Relay #70 Panel F (moth) in relay... First actual case of bug being found." 2

3 Quality Defined Quality is a perceptual, conditional, and somewhat subjective attribute and may be understood differently by different people. The degree to which the product/service was produced correctly, is reliable, maintainable or sustainable. How good or bad something is. Quality is when expectations are either met or exceeded (depending on how high your expectations were!) 3

4 Perceptions of Quality Journey from perception to reality is unique - it includes scientific methods and emotional connects 4

5 Quotients, Quotients, Quotients Examples of Quotients: IQ TQ MQ XQ PQ LQ QQ Intelligence Quotient How bright you are Technical/Operational Quotient How able are you to get things done Motivational Quotient How driven are you to achieve and grow The experience Quotient How many of the requisite kinds of experience you ve had People Quotient How well you handle yourself at work with others (sometimes referred to as EQ) Leaning Quotient - How deftly you adopt new skills, behaviors and beliefs Quality Quotient How well you incorporate strategy, capabilities, process, talent and testing rigor 5

6 Quality Quotient Formula Quality includes two key parameters: Quality of Design and Quality of Conformance Because of the two perspectives on quality, the de-facto definition of quality needs to be enhanced to reflect practitioner and empirical views where q + Q + S = Quality Quotient q = operationally limited to product s defect rate and reliability Q = includes product quality, process quality and customer satisfaction S = establishing the right strategies, capabilities, heuristics and organizational talent 6

7 Top Challenges Diverse Technologies and evolving Application landscape Agility to meet Business Strategy and Cost Governance Organizational Maturity and Change Management Process & Standards Vendor Governance (outsourcing partners, software products) Culture of Innovation Talent / Skill Mix 7

8 Use Case Very Large Investment Fund Accounting Project Quality was the #1 priority for the project team Four years in the making High complexity Large Integration testing effort Several new teams created Total Defects = 5600; almost 2700 found by Model Office Vendor defects accounted for 17% with 44% of those classified as critical / high severity This project was perceived as extremely successful due to high quality 8

9 Key Attributes of Quality Quotient 1. Strategy and Planning 2. Engage Early, Test Early, Test Often 3. Talent & Skill Mix 4. Process and Tools 5. Influential Communication 9

10 1. Strategy and Planning With 15% increase in planning efforts, test execution efforts can be reduced by 25 30% How are we influencing Scope - business acceptance, white box view, defined roles getting agreements! Are our Test Objectives aligned to SMART principles Is my Quality Strategy aligned How is my Test Strategy governing Product and Release level - See the Forest through the Trees How am I instilling Heuristics and Innovation How can we.. culture Are the Tools supplementing the life cycle Are we incorporating Lessons Learned? 10

11 Sample Test Objectives Test Objectives Test Level Objective Requirements Execution Traceability Root Cause Metric 100% of functional, non functional requirements are testable, assessed for impact, have full test coverage and are approved by stakeholders 100% completion of high and medium test cases and 80% of low priority (time to market driven project). Include Defect Detection % - 2/3 rd defects found in 1/3 rd of test execution, % of acceptable defects by phase 100% of requirements are risk rated (assigned a risk score); all high risk requirements are traced to test cases By test level (requirements, code, design, data, etc. defects found in each test level) use a guidepost to determine expectation. Industry examples: 20% requirements, 30% design, 35% coding, 15% other Automation Percent of automated testing (API vs Non API, Regression), failure rates, # sprints behind Non Functional Performance results meet X% variance to baseline, % improvement, % of x-priority SLA/SLO met per non functional test type 11

12 2. Engage Early, Test Early, Test Often Early Engagement Influence ecosystem and multifaceted testing needs Vendor assessment and proof of concept phases Contract phase Test Early Influence requirements (functional / non-functional) - strengthens collaboration between Quality, Business and Dev teams Test design techniques integrated with requirements Testability, traceability and impact assessment for business process integration testing Test Often In Sprint API automation Integrated Day In The Life as well as business regression testing Integrated Continuous Integration/Continuous Delivery (Jenkins/Hudson) 12

13 MindMap for pair-wise testing Sample Techniques Use cases / Test Scenarios / Data reqs Technique/Too l (Pairwise Algorithm, etc..) Unified view of disparate entities 13

14 Skills Talent You don t just need testers.. 3. Talent & Skill Mix BA Tester SDET Strategic Accountable Analytical Interpersonal and politically savvy Problem solver Sixth Sense extra sensory perception Influential Communicator (written, verbal, visual) Strategic sees the forest Business Domain knowledge Developer, ETL and Data testing skills Project Management Your natural aptitude Your ability that comes from your knowledge, practice or aptitude We know that our best people are scattered everywhere but we don t know exactly who knows what. 14

15 HOW? Skill Development Plans Setup Knowledge Sharing & adoption Dedicated Knowledge Management (KM) focus group - KSM and SDP (Knowledge Silo Metrics, Skill Development Plans) Test life cycle integration with KM framework, standards adaptation Injection of New Roles & Tools Introduction of Test Architect, Technical Program Manager, Test Data Architect, SDET s (Software Developer Engineer in Test) Impact assessment, business regression, RBT, Pairwise Injection of new skills Enabling non-ui Automation with SDET skills, up-skill to develop hybrid testers Performance Engineers Test Data Architects Faster Smarter Better 15

16 4. Process and Tools (Prescriptive) Standards lightweight with guiding principles Move from Silo department functions to a culture of embedded/integrated mindshare Process focus on objectives, allow innovation with some balance Define table stakes (i.e. IT Controls) Tools Process and Engineering (continuous build and release, code coverage, etc.) Continuous Improvement We cannot have islands of excellence in a sea of slovenly indifference to standards - John W. Gardner 16

17 Scorecard Maturity Levels A transformation maturity framework, focused on behavior and practices, Level -1 Basic Does not meet criteria for level 2 incorporating CMMi, TMMi framework standards Level 2 Operational Example Quality is governed by Test Approach which is base-lined, approved and continuously refined Level 3 Insightful Example Enablement of upstream defect prevention, refined using test optimization techniques Level 4 Influential Example Quality approach is a key influencer, focus on expected business outcomes and strongly advocated Level 5 Aspirational Example Quality influences IT and business process with a Best in QM view 17

18 5. Influential Communication Quality does not stop at defect prevention Pre-Production Quality Post Production Quality Architecture and Design Build stability (First Pass Yield), root cause patterns, corrective actions, defect rejections Quality gates Metrics driven reporting with dashboards Operational readiness Production checkouts 30 / 60 / 90 day user acceptance Reminders on cost to value Large transformation programs may have overlapping Integration phases Go/No Go recommendations 18

19 1. Strategy and Planning Applied Example Well Defined Scope, roles and responsibilities agreement Test objectives matured over time (defined well with vendor) Heuristics matured over time, leveraged lessons learned 2. Engage Early, Test Early, Test Often Leveraged mindmaps, well defined acceptance criteria, static testing and impact assessment Traditional and API automation, maturing operational readiness, production checkouts Business flow and Day in the Life testing 3. Talent & Skill Mix Large Investment Fund Accounting Project Good use of talent, quality efficiencies helped drive integration strategy 4. Process and Tools Leveraged maturing test and development standards and process 5. Influential Communication Excellent communication throughout (vendor trips, business, dev, integration, architecture, EPMO), good use of stage gates 19

20 Conclusion Quality IS in the eye of the beholder. You have a unique perspective that can influence the perceptions of quality. Start with a shared understanding of the objectives to align teams to the common vision. Put the time and effort into the Quality Strategy Find your own Quality Quotients that work for your organization Disrupt! Quality Quotient is not about tools or people or changing technology landscape, it is about U and I. QUALITY 20