INTEGRATED TEST AUTOMATION PLATFORM -Innovative approach to

Transcription

1 Contents HU A U T H O R : R A G HUN A T H R A N E & V A I S H A L I M A N E R a g h u n a t h. R a n l n t i n f o t e c h. c o m ; V a i s h a l i. M a n l n t i n f o t e c h. c o m INTEGRATED TEST AUTOMATION PLATFORM -Innovative approach to deliver measurable benefits from Automation Implementation LARSEN & TOUBRO INFOTECH PVT LTD., Gate No. 5, Tech Tower, Saki Vihar Road, Powai, Mumbai Abstract Current Market Scenario & Challenges faced by Automation Practices Integrated Software Test Automation Platform from Assurance to Confirmation Case Conclusion and way ahead References About the Authors Appendix White Paper Submitted for STC -2013

2 Page 2 of Abstract : Innovative approach to deliver measurable benefits from Automation Implementation Most of the Automation Practices and Testing CoEs are busy building frameworks, tools or best practices to reduce cost and improve quality. While this approach works fine in providing assurance to the clients, it might still fall short to provide confirmation, measurable real time benefits and customer intended results from Automation implementation. High level study indicates that the gap which exists in meeting customer expectations is due to multiple factors such as improper alignment of customer business objectives with test automation efforts; unsynchronized efforts; lack of comprehensive approach; ignorance of important aspects like people, process and governance; and lack of appropriate metrics and KPI specific to automation for ensuring effective control. Developing an integrated Test Automation Platform is a proposed solution to meet these gaps and enable Automation implementations to provide confirmation to the customer. The paper discusses the proposed approach, its key features and its phase-wise development and implementation. 2.0 Current Market Scenario& Challenges faced by Automation Practices 2.1 Software Testing Trends: Poor quality of software applications increases cost, impacts revenue and negatively affects the reputation of an organization. Automation testing is the best way to increase effectiveness, efficiency and coverage of testing. It is a key area of focus by clients to improve time-to-market and meet deadlines. Below are some current Software Testing trends captured by the World Quality Report [1] through customer interviews across the globe. SOURCE World Quality Report Figure 1: Software Testing Trends

3 Page 3 of Current Test Automation Trends The following figure shows the current trends in Automated Software Quality (ASQ) assurance market which are driving Automation Test Practices across the globe to spend efforts in building capabilities aligned with these trends: Figure 2: Test Automation Trends Although the above efforts will help in enhancing Test Automation and giving Benefit Assurance to the customer, they lack a centralized approach which inculcates and orchestratesall these silos efforts. 2.3 Gap in Customer Expectation from Test Automation The below figure illustrates the gaps between customer s expectations and the actual results obtained by implementing Test Automation. Figure 3: Gaps in Customer Expectations

4 Page 4 of Integrated Software Test Automation Platform from Assurance to Confirmation The above gaps confirm the need of developing an integrated test automation platform which will centralize the silo efforts of Automation Test Practices. This will ensure that these efforts are well directed in order to gain tangible benefits and achieve customer goals. The below figure illustrated the transition from Traditional Test Automation Practice to Integrated Test Automation Platform. It depictstwo different scenarios which can be compared with a typical demand-supply matching. Customer s Automation objectives form the Demand side whereas the outcomes of Test Automation Practices form the Supply side. Figure 4: Traditional Test Automation Practice v/s 3.1 Features of : The Platform should comprise of three tightly coupled components - Process Framework, Innovation Group (administrating solution accelerators, tools & alliances) and Dashboard (with test automation specific KPIs aligned with business) The platform should be perfectly compatible with the Project Environment which enables Just in Time supply meeting customer objectives Outside Support The platform should keeptrack of evolving customer demands and industry trends and prepare a roadmap for the new development. Every new development should be well-thought-out

5 Page 5 of 14 withdifferent components perfectly in tandem and compatible with each other. Every new development should be released as a new version of the test automation platform. The platform should also simultaneouslyoffer flexibility to plug-in the new innovations or sub components in any of the three layers i.e. Process, Dashboard and Innovation. However, in order to maintain integrity, a proper impact analysis should be done while making changes in each component to ensure that the changes are reflected in other components of the platform as well. 3.2 Key Benefits: The following table shows the benefits delivered to various stakeholders through the implementation of Integrated Test Automation Platform: Test Automation Practice Customer Business &Sales Compatibility Clear cut roadmap for development Synchronized efforts Reduced wastage of Automation Practices by 10-15% Alignment of Test Automation objectives with Customer Business objectives Dashboards for better predictability Continuous improvement & better control One stop solution instead of depending on multiple solutions Clarity Shift of focus from technical tasks to other activities to realize benefits SOURCE Internal Test Automation Practice Budget Table 1: Benefits of 4.0 Case - L&T Infotech has developed and implemented an which is in-line with the above needs and offers a one stop solution covering process, innovation and governance. 4.1 Key Features: Integrated platform for all test automation project needs, delivering cost, quality & time benefits Focuses beyond functional test automation Optimum test automation solution through Test Automation Process Framework o icap (Innovation Centre for Automation Practice) - Innovation group administrating solution accelerators, tools & alliances o - Dashboard with test automation specific KPIs aligning with Continuous assessment of Test Automation Aligns and caters to all phases of the standard Automated Test Lifecycle Methodology (ATLM)

6 Page 6 of 14 TM : Test Automation Platform 4.2 Stages in Developing the Integrated Test Platform Stage 1: Current Inventory Stock & Gap Analysis The first step is to have an inventory of tools, utilities, process and framework in place to carry out gap analysis of current inventory. Given below is a snapshot showing the initial inventory stock with its gap analysis. Figure TM : Test Automation Platform

7 Page 7 of 14 Stage 2: Preparing Blue Print for development of Integrated Platform After gap analysis, a Blue Print is prepared for the Integrated Platform v1.0 covering all the components - process, dashboard, innovation and their alignment with each other. Stage 3: Development of Individual Components Figure TM : Test Automation Platform Development of individual components includes the following: a.) Process Component The first step in building/enhancing the process component is to lay down the objectives behind it. The following table lists down the points to be considered in order to achieve a process which is comprehensive, aligned with standards and provides continuous improvement. Objective Comprehensiveness Alignment Continuous improvement, lean process oriented Checklist Are there standards, templates, guidelines etc. for each stage? Does the process covers end-to-end automation right from Test Automation Objective/Feasibility Check to Test Suite development to Run and Maintenance phase? Does the Automation framework covers different project and methodology types (Agile, Waterfall etc.)? Is the Automation Test Process aligned with overall standard test process? Is Automation Test Process aligned with external standards such as ATLM? Does my process provide continuous improvement? Is this continuous improvement quantitative in nature which analyses and adjusts as per the requirement?

8 Page 8 of 14 towards achieving customer goals Is there a holistic assessment framework in place to define the percentage of automation required? Am I getting benefitted from the same? Which tool is to be used? Is this automation feasible? Which areas to improve? Table 2: Checklist for Building/Enhancing Process The below diagram depicts the diagram (process part Key features of the Process part are as follows: Figure TM : Test Automation Platform Comprehensive & lean process customized for Test Automation Aligned with Automated Test Life Methodology (ATLM -Holistic assessment of your Test Automation Scenario and provide Test Automation Transformation Roadmap or Improvement plan Readily available templates and guidelines for every stage of automation from Decision to Automate till Automation Maintenance

9 Page 9 of 14 b.) Governance Component Similar to the process part of the, it is important to lay down the Governance objectives for building a Governance component. Objective Aligned and Linked towards business objectives Checklist 1. Do I have Automation specific metrics? 2. Do I have single dashboard for multiple projects/lobs irrespective of the test management tool or excel template used? 3. Is my metricsaligned with Business objectives and answersthe CXO/CETO level questions such as - percentage Test Automation covered, effectiveness of Test Automation and $Value saved. Table 3: Checklist for Building/Enhancing Governance The given below is the real time dashboard that is customized for each project by getting the customer preferences and its business objectives. It helps comparing target Automation with the actual automation achieved. Figure 9: Checklist for Building/Enhancing Governance

10 Page 10 of 14 c.) Innovation Component aligned with all other components The objectives of the Innovation component part of Test Integration Platform are listed in the below table: Objective Orchestration of Innovation efforts Achieving real time benefits Built-In Innovation (Building Innovation into the Platform rather than working in Silos) Checklist Are all my Testing Practicesefforts and Innovations well-orchestrated? Do I have roadmap for building different Tool Sets and utilities? Are my efforts in-line with roadmap and balanced or am I developing utilities on the fly? Are my Innovations just the show case to attract the customer or to achieve real time benefits? Are my innovations meant for reducing the manual efforts in any area of testing or just meant to enhance automation of functional testing? Do I have road map for using Testing Practice and Expert help todirectly deliver benefits to customer projects and portfolios? Are there any defined efforts towards achieving above points or they are just ad hoc? Table 4: Checklist for Building/Enhancing Innovation Given below is the snapshot and list of Innovation Components and mapped to the process. The list is not exhaustive. Stage 4: Implementation & Benefits Table 5: Process-Innovation Mapping Benefits to the Customer: The following figure illustrates successful implementation for a large Bank. It covers the challenges faced by the customer before the implementation the implementation process and the benefits realized by the customer implementation.

11 Page 11 of 14 Figure 10: Customer Benefits Benefits for Automation Practice: Figure 11: Checklist for Building/Enhancing Governance

12 Page 12 of 14 The figure above indicates the immediate benefits realized by Automation Practice by moving towards Integrated Test Platform. Challenges faced during implementation Challenge Challenge Phase Type of Challenge Consolidation of good practices/templates from different accounts to create generic process templates Benefit Tracking - Calculation of real time ROI for customer Reluctance to modify process & templates when customer has its own process/template Development Development& Implementation Implementation Process Technical People Reluctance to move from existing process to a new one Implementation People Short term view- Difficult to get buy in for long term benefits Implementation People Table 6: Challenges faced Implementation 5.0 Conclusion and way ahead Importance of Test Automation cannot be denied in years to come and onus lies on the automation practices to think differently and innovatively to bring benefits of Automation closer to the customer, thus moving from assurance to confirmation. The is the proposed approach for taking automation closer to the customer business needs. The Platform can also be a revolutionary way to re-organize Automation Practices way of working and their efforts to reduce wastage by moving towards synchronization. This can be compared with moving from Smart Feature Phones to Smart Phones in Mobile Industry, in which the Mobile OS like ios, Android are developed and different Applications are plug in to the same and Mobile Companies are spending efforts in building and upgrading the Mobile OS as platform. The Integrated Test Automation Platform is synonymous innovation in Test Automation. We can also foresee a future of building standards for this platform which is agreeable across industry so that there could be a common platform and architecture (like SOA etc.) for Test Automation and all other players including customers are left to customize it or add plug-ins to this platform to meet their needs. This will help in standardization as well as in improving maintenance efforts. Though there are different challenges in the nascent stage, the case study of initial implementation has a great potential of providing benefits for both Automation Practice and customer. It is always advisable to leverage this new way of thinking and overcome the challenges.

13 Page 13 of References [1] Capgemini, Sogeti, HP, World Quality Report [2] Thomas E. Murphy, Nathan Wilson (11 July 2013), Magic Quadrant for Integrated Software Quality Suites, Gartner, G [3] L&T Infotech Customer Survey [4] Internal Budgetary Analysis [5] [6] s_in_the_ever_changing_technology_scenario_ramesh_krish.html [7] Testing -2013, Trends and Benchmark Report, Switzerland 7.0 About the Authors Raghunath Rane works with L&T Infotech as Sr. Project Manager. He has 14+ years of extensive experience in Delivery and Business Development in Software Testing Arena and has managed large Testing Engagements in the past.he holds most of the Testing and Project Management certifications viz. PMP, CSQA,CSTE, ISTQB and Six Sigma Yellow Belt. He is an active contributor in Testing CoE and Automation Practice operations and has presented white papers in the Testing and Project Management Conferences in the past. Vaishali Mane works with L&T Infotech as Sr. Manager and heads the Automation Test Practice. Having more than 15 years of Testing and Development experience, she is very passionate about designing and developing innovative tools and utilities. With a strong technical expertise, Vaishali has successfully led the innovation group in building multiple innovative tools which have become strong differentiators for L&T Infotech Testing Services.

14 Page 14 of 14 Appendix ASQ Automated Software Quality ATLM Automated Testing Lifecycle Methodology Testing CoE Testing Centre of Excellence KPI Key Performance Indicator