How TPI evaluation improve the grid computing at Intel. Amar Aiman - QualiTest Antebi Nir - Intel

Transcription

1 How TPI evaluation improve the grid computing at Intel Amar Aiman - QualiTest Antebi Nir - Intel

2 Agenda A few words about grid Intel Grid computing TPI process description Current status for Intel Proposed target for Intel Action items to reach the target Summary 2

3 What is Grid? Simple definition of computational Grid: Applying the resources of many computers in a network to a single problem at the same time Academic definition of computational Grid (Source: Ian Foster - What is the Grid? A Three Point Checklist) 1) Coordinates resources that are not subject to centralized control 2) Using standard, open, general-purpose protocols and interfaces 3) Deliver nontrivial qualities of service 3

4 Intel Grid In house development 400M jobs/year ~60,000 CS (most are dual CPU, and moving to dual core) ~30,000 jobs running remotely at any moment ~9000 machines in the biggest pool ~6000 active users in the last quarter. ~1500 concurrent. ~110 pools in ~35 design center ~10 Virtual Pools 7 main Platform (Redhat, SLES, Sun, HP, Debian, Windows, MacOS + some other flavors per demand ) 4 4

5 TPI Process Description 5

6 TPI Testing Process Improvement Systematic procedure for testing process evaluation originated by Sogeti Embeds many years of experience from numerous sources Provides one common language to describe the status and improvement plan of the testing process 6

7 20 Key Areas Test strategy Life cycle model Moment of involvement Estimating & planning Test design techniques Static test techniques Metrics Test tools Testing environment Office environment Commitment & motivation Testing functions & training Scope of methodology Communication Reporting Defect management Testware management Test process management Evaluation Low level testing 7

8 Scoring 8

9 TPI Current status 9

10 Information collection process We held a number of interviews and meetings with the relevant people We reviewed documents and forms examples used in the company We reviews proprietary and commercial tools to improve the process 10

11 current status 11

12 current status In general, the testing process is functioning and providing results However, many areas are at a lower than expected level and far from optimal. The fact that the products are for internal usage does not create a standard working environment. 12

13 Proposed target for Intel 13

14 Proposed targets 14

15 Proposed target Based on the evaluation process performed, the proposed targets are the optimal objectives for Intel. Those are achievable goals in reasonable terms of time and cost. They bring Intel to the competency level of testing to optimal for the group functioning (business) model. Once this is achieved, further improvement is possible. 15

16 Legend Critical Major Important Action Item 16

17 1. Test strategy Adapt a risk based systematic and algorithmic testing management policy that relays on product documentation, traceability, dependencies, metrics, historic analysis, recent changes, human factor estimation, etc Done (adopt ReqPro, pro-chain, Bug tracking and enhancment tool) Load testing need to be performed by testing not done Redesign test cases to perform end to end testing and aggregate many feature testing in single test case on going (working with the customer to document end to end usage model). 17

18 2. Life cycle model Development should delivery per release initial builds with the major features and then add only minor and lateral features Done(dev team release version every few weeks). Add a integration and sanity test phase between development and testing (done developer release a version for QA only if pass regression). 18

19 3. Moment of involvement Involve testing in the process for the initial phases of requirements specification to ensure that the requirements are concrete, measurable, and testable. Add testing requirements as part of the release requirements to be executed - QA team envolve in the process and get all requirement documents from the dev team Move the responsibility over STD writing to the testing group STD moved to QA responsability 19

20 4. Metrics Map problematic modules, bug generating software engineers, average bugs/day/software engineer, bug life span, average repair time for a bug, average work effort investing in detecting a bug and in solving it etc. Those metrics an be put to very good use in optimizing the testing work process, better planning and estimating work effort, etc partial Add metrics in all aspects and use them to determine trends for a continuous improvement process - partial 20

21 5. Test automation This item is at the adequate level but we have a few notes on it. In terms of complying with the model, Intel received al the points. However, using Perl as the automation tool is not the optimal choice and reconsideration of the tool needs to be performed Done, starting using another automation tool for GUI testing We expect the automation to be performed in line with the STD writing and manual testing planning/execution. 21

22 6. Test functions and training Recruit testing engineers with education and prior professional experience in testing. Create a new employee training plan with close coaching. Create training, promotion and retention plan for testing staff Increase the efficiency and effectively of the testing workforce by better training and planning the work. 22

23 7. Defect management Change the defect management to comply with industry standard (all bugs/fixes go through testing, customer report bugs to testing for documentation and reproductions, testing closes all bugs after retest, no releases sent to the by development directly to the field, development does not close bugs, etc.) All fixed bugs are assumed faulty until retested and closed by testing - Done Define a formal definition for bug severity and priority and according handling procedure - Done 23

24 Summary 24

25 Summary Following the analysis process, we came up with a plan that calls for the initiation of a testing process improvement that will substantially increase the quality of the testing group and of the Intel product as a whole. To achieve this, the action items should be prioritized (as already mentioned in the document) and used to build a long term work plan. Results can be seen for this matter in a few months We thank Intel for the cooperation during the compilation of this document and wish success in its swift implementation. 25

26 Thank You