Platform Transformation Testing A Case Study. Dave Pavey Big Cat Test Manager and Freelance Test Consultant

Transcription

1 Platform Transformation Testing A Case Study Dave Pavey Big Cat Test Manager and Freelance Test Consultant davepavey@blueyonder.co.uk

2 Big Cat Test Manager A Great Job Title?

3 Today s Talk What is Platform Transformation and Project Big Cat? What was the test approach? What types of issues did we find? Project results? Summary and what can you learn? Your questions

4 Project Big Cat A Technical Transformation 47 HBOS Legacy Life and Pensions systems dating back to External interfaces 400+ Batch processes Users Area Code Base Software Infrastructure Database Hardware From Lion Proprietary Lion (Linc based) 8.1 million lines of code Delphi, Client Server 2000 SQL Servers: 32 bit, Proprietary Load Balance, Comms Servers To Big Cat C# 16.8 million lines of code.net, Browser, Client Server 2003 SQL New Servers: 64 bit, Edge Server Balance, no Comms Servers

5 Test Approach Testing is the process by which we explore and understand the status of the benefits and the risk associated with release of a software system. J Bach 1999 To do this well, as testers we must. Understand the context Understand all stakeholder requirements and their risk perspectives Use Management & Test Standards, but also apply our knowledge and skills

6 Big Cat Context Original Lion Project Legacy systems converted from Unisys Platform to Lion in 2003/04 Systems expected to be used for run-off business only few system changes anticipated old code produced a large number of functional and conversion defects project took longer than planned serious performance issues & business backlogs after go-live systems have been stable and robust in recent years The New Project Decision to use legacy systems as strategic platform Significant change programme expected Strategic development platform Original Lion Infra-structure needs updating Partnership with Asysco - proposition to develop a conversion process from Lion to C#.NET Resulting systems will have been converted twice

7 Stakeholder perspectives

8 Some initial views IT Management The project is primarily about business risk avoidance and strategic alignment of systems. It requires a quality driven approach Customer Services Few benefits for us, but if it goes like last time it will be a disaster, very nervous Asysco A project of significant strategic importance Developers Will it be maintainable, new skills are required Group Technology Hardware work orders, logistics, new technologies, we have other group priorities Testers Too much to test, it will never work, no system experts, little documentation The I&I Board Do not mess it up stronger words may have been used!

9 Test Approach Risk Based Traditional Function Testing Approach Rejected; takes too long too expensive Would not be effective insufficient knowledge to predict results Business agreed that old system functionality and performance were acceptable Risk based, Black box, Parallel Run Approach Looking for differences New system issues logged, but not what we are looking for Every batch program and on-line screen run at least once Interfaces and gate-keeper sign-offs are key

10 Test Approach - Performance Online Existing performance was unknown so new targets defined 16 key on-line business processes mapped E2E Further padding to simulate all other concurrent transactions Batch Full-volume copies of live data and inputs used Live copies taken on longest run dates Environments New hardware ordered early, used as pre-production test environment for SAT and EPT Cross system data compatibility addressed by using copies of live data

11 Test Approach - Organisation End-to-end Test Manager, responsibilities include; Approach strategy coordinate cross-team activity coordinate cross phase test activity Phased Approach; Alpha, Link Component Test (LCT) System Acceptance Test (SAT), Early Performance Test (EPT) Business Acceptance Test (BAT), Implementation Modelling Operations Acceptance Test (OAT), Release Acceptance Test (RAT) Single code base-line used to prove code conversion Final tests reconvert latest code and regression test

12 Test Approach - Schedule Quality driven approach Delays and bottlenecks inevitable so plan accordingly New code and script converter Full code-set conversions, migration and deployment after each fix release New hardware New environments Complex data alignment

13 Key Issues found Test Environment set-up 64-bit compatibility Initial slow batch impacted SAT schedule and threatened later phases Some old systems used machine dates Cross system data compatibility - used copies of live data to overcome this Handling of duplicate records (old systems relied on record sequence) Meaning of leading spaces, zeros, blanks and nulls was inconsistent across systems Temptation of testers and users to over-do-it Volume of defects over 1000 raised post alpha

14 Project Results System delivered on schedule HBOS aligned with strategic goals.net and C# Growth and capability via industry standards No down time in first 3 months Minimal functional issues Improved batch performance Asysco have a proven Unisys to.net transformation tool AMT-VS implementation seen as a huge success by all stakeholders

15 Summary and what can you learn? Invest time in identifying interfaces and their owners Test scope must address Context Risks Stakeholder concerns Look for transformation issues, not problems with the existing system Don t attempt to change the system at the same time Test Team is part of a partnership work together Plan, monitor, review, adapt to circumstances (then let everyone know)

16 Questions