"Test Automation Success: Choosing the Right People and Process"

Transcription

1 W8 Track 4/28/2010 1:45 PM "Test Success: Choosing the Right People and Process" Presented by: Kiran Pyneni Aetna, Inc. Brought to you by: 330 Corporate Way, Suite 300, Orange Park, FL

2 Kiran Pyneni As senior QA manager at Aetna, Kiran Pyneni is responsible for improving test organization efficiencies through automation. Kiran leads a team of innovative automation enablers responsible for driving automation across Aetna. Prior to joining Aetna, Kiran had several roles: software developer, senior tester, testing architect, QA lead, and delivery manager for Fortune 500 organizations. Kiran was an invited speaker at the HP Software Universe 2009 USA Conference speaking about automation testing.

3 Test Success: Choosing the Right People & Process Kiran Pyneni, Manager Aetna, Inc. Agenda Company Background Team Objectives Our Journey Strategy Team Delivery Team Benefits of Teams Strategy Lifecycle Benefits of Process Integration Metrics Of Our Success Coverage, Efforts Automated, Utilization & Cost Avoidance Questions 1

4 Aetna Background Aetna is one of the nation's leading diversified health care benefits companies, serving members with information and resources to help them make better informed decisions about their health care. Products and management Programs: Aetna offers a broad range of insurance and employee benefits products. The first national, full-service health insurer to offer a consumer-directed health plan, Aetna continues to lead the way with its Aetna HealthFund2 line of products, including HSA, HRA and RRA options. Aetna offers a wide array of programs and services that help control rising employee benefits costs while striving to improve the quality of health care, such as case management; disease and patient safety programs; integrated medical, dental, pharmaceutical, behavioral health and disability information. Aetna provides members with access to convenient tools and easy-to-understand information that can help them make better-informed decisions about their health and financial wellbeing. 3 Team Improves efficiencies and quality at our organization by automating consistent and repeatable manual tasks using industry standard technologies / tools / frameworks. Standards & Best Practices Reports & Metrics Team Assessments & Designs Development & Maintenance 2

5 Objectives Objectives Automating Right Tasks Sustaining and Growing Suite Utilization of existing automation Our Journey Decentralized Approach Centralized Approach Benefits Easy identification of automation opportunities Higher collaboration between application & automation teams Challenges development has become expensive Usage of different tools & solutions Standards & Best practices are not shared across applications Benefits Standardization of automation tool set at enterprise level Sharing of coding best practices, templates and standards Cost of automation development is lowered Challenges Slower automation coverage progression Lower automated scripts utilization Funding Challenges 3

6 Our Journey Cont d. Challenges & Solutions Challenges Identification of automation opportunities Prioritization of automation projects based on ROI Cost of automation Implementation of optimized designs Challenges Early completion of maintenance for automation scripts Funding for automation projects Collaboration between automation & business teams Solution Structuring automation as strategy & delivery team Solution Create automation life cycle process and integrate with company's overall software development life cycle Team Structure Team Strategy t Team Delivery Team 4

7 Strategy Team Definition: To provide automation strategic direction and identify potential new automation opportunities Characteristics: Project management experience Various automation frameworks/tools knowledge Ability to manage multiple projects in parallel Very strong written and verbal communication skills Very strong leadership and analytical skills Activities: Process & Best Practices Assessment & Strategies Design Solutions & Techniques Estimations & Planning ROI Cost/Benefit Analysis Reports & Metrics Tool Technical Support & Analysis Delivery Team Definition: To develop, deliver, and maintain automation solutions Characteristics: Understanding d of programming g principles p and best practices Knowledge of various automation tools Knowledge of multiple programming languages Strong communication skills Activities: Scripts Development Scripts Maintenance Coding Standards Scripts Planning & Execution Day to Day interactions w/end User 5

8 Benefits of Teams Strategy Benefits Easy identification of automation opportunities Prioritization of automation projects based on ROI Improved automation design Optimized the cost of automation Clearly defined roles & responsibilities Better communication process Optimized automation project planning Lifecycle Phases Activities Evaluation Understand functionality of the project / application Understand testing processes Understand target environments Understand test data details Verify automation tool compatibility Identify opportunities Preparation Identify the resources needed. Understand the requirements with the help application demos. Create automation framework / design Identify timelines and milestones Training required for automation tools Tool installation if required Access & Environment Setup Generation Create Business Process Test (BPT) components Parameterize test data in BPT components Review components for quality and correctness String the BPT components together to create a test script Develop automation scripts for BPT components Identify test data to conduct unit & acceptance testing G1 G2 G3 Execution & Acceptance Unit testing of automation Deliver automation to application team Execute automation by application team Validation of ROI Retrospection Sign off Deliverables Preliminary Estimations Test Strategy Business Case - ROI Selected automation tool Detailed Estimations Go, No Go Project Plan Framework documentation Re-usable components, Portability, test data grid design, etc. Business Process Test (BPT) components Scripts Test Data Tested automation scripts Automated test execution results Lessons Learned Sign Off 6

9 Team Collaboration Strategy Team Delivery Team Evaluation Preparation Generation Execution & Acceptance Integration of Lifecycle & Software Development Lifecycle Evaluation Feasibility/Analysis Preparation Design Generation Implementation Execution & Acceptance Testing & Maintenance 7

10 Benefits of Process Integration Benefits Improved understanding on automation process across the entire organization Better collaboration between business and automation teams Funding challenges are removed Improved on-time delivery of automation scripts Helped to promote the automation across entire organization Formal escalation process developed Initialized the creation of automation reports & metrics Coverage Regression Testing Year % Q % Q % Q Q % 52% 8

11 Manual Testing Efforts Automated 13,803 hrs 8,570 hrs 10,200 hrs 11,750 hrs 7,047 hrs Year-2008 Q Q Q Q Year As of 2008 As of only efforts MR1 Total Efforts Automated 7047 hrs hrs 6756 hrs Utilization & Cost Avoidance Utilization 150% 134% 130% 87% 45% Year-2008 Q Q Q Q Cost Avoidance 10,810 hrs 9,618 hrs 9,867 hrs 7,200 hrs Q Q Q Q

12 Slide 17 MR1 Deleted "I&T" from chart Matt Rosenfeld, 2/1/2010

13 Questions 10