testing white paper November 2013 TO ERR IS HUMAN THAT S WHERE TESTING COMES IN

testing white paper November 2013 TO ERR IS HUMAN THAT S WHERE TESTING COMES IN

TO ERR IS HUMAN THAT S WHERE TESTING COMES IN Technology has never been more important to a company s success than it is today. Software applications are behind almost every business system and process, from mundane to mission critical. That means more demand for applications, with less time to build them. IT s central role also means that other factors are adding to the demands placed on application development. Competitive lines are being drawn based on information and software-based solutions, a factor that Haresh Patel, a business planner for CSC s Testing services, says is ratcheting up the pressure on application developers. Innovation through applications is one of the primary means by which companies create competitive advantage today, Patel says. That means, when a competitor reaches the market with a new software-based offering, companies feel a great need to get to the market at the same time, or very close. Trends like mobility and smart devices have opened up entirely new product and service delivery channels for companies. Customers have become accustomed to new levels of information access and self-service through lightweight, mobile applications. These new channels also use various operating systems, development practices and form factors. Developers must ensure the applications work effectively on a broader range of systems than before. The bring-your-own-device (BYOD) movement has employees expecting anywhere, anytime information access in the workplace just as they do outside work. This growing dependence on and higher expectations of applications means that it s never been more important to ensure their performance, reliability and security. The cost of application failure is higher than ever. This paper examines the costs tied to application failure and the common reasons many companies continue to experience high defect rates. It examines the role that independent software testing can play to reduce defects and improve quality and speed for software product development. The Cost of Mistakes Mounts Quickly As the role of information-based solutions rises in organizations, so too does the cost of mistakes. App defects found after release can come from a number of sources. Architectural or structural issues may lead to poor performance or a mismatch between user requirements and delivered capabilities. These issues are the most severe, and the most costly to remedy. Failure to ensure alignment of performance requirements and infrastructure can lead to apps that are frustratingly slow and do not gain user acceptance. Poor coding practices such as a lack of error checking contribute to poor data quality and often create security issues that aren t realized until an incident occurs in the field. All of these factors can be accelerated by the simple rush to get things shipped, says Patel. Most applications are being developed in a short amount of time, and that typically leads to more defects going into the field. It s ironic that in the rush to get things into the market, the defects that are present means it will take longer to get a product to market. The haste to get things shipped inevitably leads to costly fixes. 1 TO ERR IS HUMAN... THAT S WHERE TESTING COMES IN

Vimal Kohli, general manager for Testing services in the Americas at CSC, says the cost to fix problems adds up faster than a company realizes. Defects that aren t caught during product development and testing are very expensive to fix later. Studies show that the cost to fix bugs after an application is released can be 30 to 40 times more than defects identified during development, Kohli says. The need to issue updates can create more costs, he adds. When you take steps to correct deep-rooted issues, you can introduce new problems. That creates new delays, adds to user frustration and leads to even higher costs. As the number of releases grows, the cost of support increases as well. the cost to fix bugs after an application is released can be 30 to 40 times more than for defects identified during development Why Companies Struggle A misaligned culture Despite a growing dependence on software-based systems, application testing remains largely unchanged in many organizations. Manish Tomar, general manager for Testing services for CSC globally, says the role of testing and its impact on the cost of quality isn t well understood. Many CIOs and IT managers still think of testing as an after-the-fact once application development is done. What they don t realize is, seeing it as an after-the-fact increases their cost of quality. Kohli and Patel say that a number of internal and external factors contribute to this situation starting with how the testing function is viewed by other teams in the development process. Testing is viewed as a box you have to check on the way to getting an application shipped, Kohli says. The testing group is viewed as the last obstacle to shipping, not as an integral part of the process. Patel agrees. Testing faces a number of challenges in many companies. They re often strained for resources. They don t have sufficient authority to manage the quality of applications they test. They simply aren t treated well by the rest of the organization. Structural issues contribute to this schism between development and testing. They are organizations with competing agendas, Patel says. The development team is pressured to get products into the field sooner so they can get on to the next product. If they don t understand the full cost of defects that are present in the applications they release, they won t make the connection between more testing, higher quality, and faster development. In some cases, Patel says, companies simply don t realize what they re missing. Many enterprises are not aware of the different types of testing they should be doing, aren t aware of the methodologies, don t realize what processes they could put into place that will help them improve their application quality. Complexity of solutions and the environment Even when a testing team has authority equal to its responsibility and is respected by its peers, challenges remain. Project complexity is a significant issue for many companies. Large projects often include more than one business unit within a company, multiple vendors, and multiple sets of development tools and platforms, all involved at different points in the development process. Kohli says that type of environment breeds issues. 2 TO ERR IS HUMAN... THAT S WHERE TESTING COMES IN

When business units and IT are involved at varying level in the development cycle, with different vendors in and out of the process, and many different products involved, that is a complex environment for a small testing organization to grapple with, he says. Environmental complexity contributes to another fault common in many testing organizations: a lack of standard tools and practices, which results in unevenly applied testing procedures and inefficient resource use. In the end, Kohli says, there is no unified process nor a single gatekeeper charged with a responsibility to get the product right, the first time out. Today, product lifecycles in mobile development are measured in months, not years. When your product is ready to go, who s making sure it won t break your business? Who ensures it will perform the way your customers expect? And who can certify that it won t open any security holes and make you vulnerable to attack? Often, the answer is no one. New trends increase skills and infrastructure needed for testing All the trends fueling growth in application development and the rise of IT in the organization increase the demands on the testing function. Cloud computing s flexibility and performance are encouraging many companies to adapt applications to a virtualized environment. Meanwhile, the spread of smart devices and mobile broadband have created an explosion of demand for self-service mobile applications. You have all these new requirements for application development and testing, and they are on top of the existing requirements, Kohli says. You cannot take your eye off your traditional model, which is a growing market. But it s an absolute requirement that you meet these other demands. That means new skills and infrastructure. Today, product lifecycles in mobile development are measured in months, not years. And each new device release is accompanied with new features and form factors. Certifying application performance on even a small number of devices requires specific knowledge of the hardware and its operating environment. In a similar vein, Kohli says, the push to move applications to the cloud raises questions that should be answered by specialized testing. Is this application well suited to a cloud-based architecture? It s not an automatic fit for every application. If it s virtualized, will it perform at an acceptable level? he says. Applications that rely on entirely new constructs such as social media and big data applications require specialized skills to design tests that return accurate results. These applications raise many questions that rely on accurate testing for answers, Kohli says. How do you extract information? How do you incorporate unstructured data into your business intelligence? How do you make sure that you re able to get the meaningful information out of it? The Value of Independent Testing Companies that have a newfound appreciation for testing are revisiting its role in the application development process. After comparing in-house capabilities with other options, a growing number are concluding that independent testing capabilities result in higher software quality, faster time to market and lower overall costs. 3 TO ERR IS HUMAN... THAT S WHERE TESTING COMES IN

Tomar says CSC s independent testing service helps companies achieve markedly improved results because it offers more than a test lab. CSC helps companies integrate testing at points throughout the development process. We become part of the application development process, and we look at things from a product quality perspective, not just a testing service perspective, Tomar says. In practical terms, CSC s testing service helps companies develop their own abilities, supplementing them as needed with specialized skills and tools. The addition of independent testing helps companies establish a process that improves quality over a long term basis. The implementation of best practices, benchmarks and metrics leads to a more mature and professional testing capability inside the company. Strict quality checks at all stages of development lead to earlier defect detection. Kohli says these steps can be directly responsible for significant savings. When you have the right testing structure in place, you can measure your success at each phase. This means you can quickly identify a failure and diagnose it before it goes further. That makes it much easier to identify the root cause and eliminate the defect. The further it goes, the more it costs to identify and fix, Kohli says. Specialized testing also eases the burden imposed by competitive pressures, complex environments and the pressures imposed by new technologies. Tools that automate lifecycle quality management and standardize processes and interfaces help to dramatically reduce the time required to test applications. Patel says independent testing addresses limitations created when developers are asked to also take on a testing role. Testers have a different skill set than developers. They develop skills and knowledge that help them test more effectively and identify defects readily. They understand how to implement new testing products and methodologies. You don t get that when you convert a developer into a tester. That s a benefit of an independent testing facility, Patel says. New technologies and testing automation advances support many types of engagement models. That means companies can find the testing model that best fits their business requirements and budget. We work with clients on many different levels, whether we are supplementing their testing capabilities, helping them implement testing into development or doing it all for them, Patel says. We have the flexibility with models like Testing-as-a-Service that helps them strike the right balance between what the business needs while providing them with a predictable cost. Over the longer term, Tomar says, the service provides value that endures long after an application has shipped. Independent testing creates a culture of application quality that leads to improvements in every product a company delivers. We are pleased to play a role in helping many companies realize that value year over year. 4 TO ERR IS HUMAN... THAT S WHERE TESTING COMES IN

Worldwide CSC Headquarters The Americas 3170 Fairview Park Drive Falls Church, Virginia 22042 United States +1.703.876.1000 Asia, Middle East, Africa 20 Anson Road #11-01 Twenty Anson Singapore 079912 Republic of Singapore +65.6221.9095 Australia 26 Talavera Road Macquarie Park, NSW 2113 Australia +61(2)9034.3000 Central and Eastern Europe Abraham-Lincoln-Park 1 65189 Wiesbaden Germany +49.611.1420 Nordic and Baltic Region Retortvej 8 DK-2500 Valby Denmark +45.36.14.4000 South and West Europe Immeuble Balzac 10 place des Vosges 92072 Paris la Défense Cedex France +33.1.55.707070 UK and Ireland Region Royal Pavilion Wellesley Road Aldershot, Hampshire GU11 1PZ United Kingdom +44(0)1252.534000 About CSC CSC is a global leader in next-generation IT services and solutions. The company s mission is to enable superior returns on our clients technology investments through best-in-class industry solutions, domain expertise and global scale. For more information, visit us at www.csc.com. 2013 Computer Sciences Corporation. All rights reserved. Creative Services 8158-21 11/2013