SAP - Redefining software testing. Sanujeet Puhan SAP Technical Architect

Transcription

1 SAP - Redefining software testing Sanujeet Puhan SAP Technical Architect

2 Agenda Overview Application softwares Enterprise applications Current trends Test Approach Applications vs Processes Risk perspective Critical questions Methods and Tools Business blueprint Technical Bill of Material Impact Analysis Test plan simulation

3 Application softwares Custom application, supports specific functions/services Standard application, can be combined to provide specific functions/services Systems Applications Products Enterprise application, provides range of standard enterprise functions and services

4 Enterprise Applications A whole bunch of domains FI,MM,SD many unused Tightly integrated Change means side-effects Vendor specific Data model Client has limited view

5 SAP Testing situation Source: ASUG Test influence council member survey 2010

6 Customer software vs Standard erp software 1(4) Focus Custom software Thinking in terms of Applications Focus on - Bug-free operation, - Ergonomics of user interface, - Compact solutions Standard software Thinking in terms of Business Processes Focus on - Quick deployment, - Robust business processes, - separation and standardization How it matters for testing Major tests are: - Unit testing (code quality) - Whitebox testing (coverage) - Exploratory testing (exception) Most tests are: - Integration testing - Blackbox testing - Regression testing

7 Customer software vs Standard erp software 2(4) Development Custom software Coding is a major activity. Typical cycle is: Design Build Test Standard software Coding is only for enhancements. Typical cycle: Design Configure Test Deploy Deploy How it matters for testing Since coding is a major activity, Testing involves developers. Needs good data quality. Testing requires business process experts.

8 Customer software vs Standard erp software 3(4) Change control Custom software Changes triggered by - new user requirements - obsolescence Standard software Changes triggered by - User requirements - Obsolescence - Vendor s strategy How it matters for testing Since customer is in control, Frequency of testing is less. As vendor updates must be applied, Regression Testing is frequently done.

9 Customer software vs Standard erp software 4(4) Dependencies Custom software Standard software Self architected, meaning Dependencies between modules are usually well-known. Predefined architecture, meaning Many dependencies maybe unclear. How it matters for testing Test scope can be decided with good accuracy. Test scope can become large, to account for hidden dependencies.

10 A typical test scenario Test bench Px P1 P2 P5 P3 P4 Test everything, it is safest - too vast - too intricate C1 C3 C4 C6 C2 C5 But Test all feasible?

11 Risk based test approach Identify critical processes Risk = Probability of failure * Cost of failure Test bench Px P1 P2 P5 P3 P4 P1 P2 P5 C1 C3 C4 C6 P3 P4 C2 C5 Px But, what is the actual probability of failure?

12 Critical Questions How to find processes with actual risk, NOT assumed risk adapt test plan to resource constraints

13 If process and code can be linked Px P1 P2 P5 P3 P4 C1 C3 C4 C6 C2 C5 Then we know which processes are actually impacted

14 An optimization example Scenario Coverage (%) Effort (min) Choice S S S S S1 S2 S3 S4 Test bench Px P1 P2 P5 Px P3 P4 P2 P5 C1 C3 C4 C6 P3 P4 C2 C5 8 steps, 6 affected by code changes Links also indicate test effort

15 How to: Identify processes with actual risk process /* code Impact analysis Test effort Adapt test plan to actual resource constraints Optimization Test coverage

16 A real life scenario: New SAP Enhancements are applied, which changes many objects

17 The need of a test strategy in SAP

18 Business Blueprint SAP s approach to unify processes, applications and systems

19 Technical bill of materials Links process to underlying objects

20 Impact Analysis Determining which processes are affected

21 Boundary setting Select a meaningful subset of packages

22 Dashboarding Status and other aids for project management

23 Test Scope Optimization Approach: All changed objects should be tested at least once. Optimization based on: Test coverage Test effort Business priority

24 Actual Optimization Simulations with Coverage and Test effort

25 Final checks Identify / add anything missing

26 Benefits Precise insight of change impact Better risk management for business Safeguard test case obsolescence with TBOMs Timely identification of gaps in test scope Multivariate optimization Reduced effort for a requirement based test plan NEXT STEP: Generate test plan, automate, assign testers, create learning maps

27 Thank you!