An Early Defect Elimination Best Practice. Duvan Luong, Ph.D. Operational Excellence Networks

Transcription

1 An Early Defect Elimination Best Practice Duvan Luong, Ph.D. Operational Excellence Networks

2 Peer Review and Inspection Definition: Peer Review is the methodical examination of software work products by the producers peer to identify defects and areas where changes are needed. The purpose of Peer Review is to remove defects from the software work product early and efficiently. Peer Review practice includes: Inspection (formal procedure) Review (semi-formal procedure) Desk-check (informal procedure) 3/27/2012 2

3 Inspection Process Summary Reschedule No Entry criteria met? Yes 1 Planning 2 Kickoff 3 Preparation 4 Logging Meeting 6 Rework 7 Follow-up Preparation Done? Yes Exit criteria met? Yes No No Reschedule Re-Inspect 3/27/2012 3

4 Benefits Inspection implementation has resulted in the following tangible benefits Reduction in Customer defects backlog and less defects escaped to Customers. Multi-million dollars saving in the cost of find and fix defects. Organization has a fully operational Inspection facts/statistics based improvement process. 3/27/2012 4

5 Inspection Implementation Focus Training Provide Tools and Support Set Policy Plan and assign resource for deployment Results tracking and feedback for continuous improvement 3/27/2012 5

6 Success Factors for Inspection Implementation Train 100% of all Engineers in Inspection Common Language Common Expectations Heightened awareness Moderators assigned and trained for all teams (1 moderator for 5 Engineers vs. 1 moderator for 8-10 Engineers in Industry) Trained moderators/engineers are the key factors that ensure the smooth and effective implementation of Inspection. 3/27/2012 6

7 Inspection Tools Dedicate laptops and projectors for use in Inspection. Develop the Logging tool to facilitate the recording of Inspection data Create online help to improve the tool use-ability. Tools ensure the effectiveness of process implementation Build defect classification into the tool to enable the use of Inspection data for later analysis of the process/product quality. Defect classification enables later process effectiveness analysis 3/27/2012 7

8 Inspection Policy and Process Include Inspection Plan in the project development checkpoint review. Inspection Champion identified for each section to decide the Inspection requirements for mandatory deliverables. All new and changed code require Inspection. Inspection data is tracked and published at team and company level. These policy and process should be set and enforced in the Organization to fully commit to the implementation of Inspection. 3/27/2012 8

9 Monthly Inspection Metrics Inspection Effort (hours) Defects Found Defect Discovery Rate Month 1 Month 2 Month 3 Month 4 Month 5 Month 6 Month 7 Month 8 Month 9 Month 10 Month 11 Month 12 Company dedicates resources for Inspection implementation so it can find many defects early 3/27/2012 9

10 Inspection Process Effectiveness (first period) Defect Introduced by Phase Number of Introduced Defects Number of Found Defects Requirements Design Coding Maintenance Development Phase User I/F Test S/W Standards Software I/F Req/Spec Other Module Design Logic Descr. Logic Locks Hardware I/F Functionality Functional Desc. Error Checking Data Handling Computation Defect Found during Code Inspection 3/27/

11 Inspection Process Effectiveness (second period) Defect Introduced by Phase Number of Introduced Defects Number of Found Defects Investigation Design Construction Verification Maintenance User I/F Test S/W Standards Software I/F Req/Spec Other Module Design Logic Descr. Logic Locks Hardware I/F Functionality Functional Desc. Error Checking Data Handling Computation Defect Found during Document and Code Inspections 11

12 Inspection Process Effectiveness (third period) Defect Introduced by Phase Number of Introduced Defects Number of Found Defects Investigation Design Construction Verification Maintenance User I/F Test S/W Standards Software I/F Req/Spec Other Module Design Logic Descr. Logic Locks Hardware I/F Functionality Functional Desc. Error Checking Data Handling Computation Defect Found during Document and Code Inspections 12

13 Total Defects Found Profile 100% Monthly Defects Found Distribution Inspection R&D System Test Customer 100% 80% 80% Defects Distribution % 60% 40% 20% 60% 40% 20% 0% Month 1 Month 2 Month 3 Month 4 Month 5 Month 6 Month 7 Month 8 Month 9 Month 10 Month 11 Month 12 0% Company development productivity increased with more defect discovered and fixed early in the development life cycle with Inspection 13

14 Defect Backlogs Management Statistics Internal Defect Backlogs Customer Defect Backlogs Defect Backlogs Q1Y1 Q2Y1 Q3Y1 Q4Y1 Q1Y2 Q2Y2 Q3Y2 Q4Y2 Q1Y3 Q2Y3 Q3Y3 Q4Y3 Q1Y4 Q2Y4 Q3Y4 Q4Y4 Q1Y5 Q2Y5 Q3Y5 Q4Y5 14

15 Example of Savings with Inspection Engineering Hours Engineering Hours Inspection cost to find Cost if found during maintenance Cost to Find 1 st period 2 nd period 3 rd period Cost to Fix Engineering Hours 0 Inspection Period Defect find savings Defect fix savings total savings Inspection resulted in multimillion Dollars Savings 1 st Period 2 nd Period 3 rd Period Cost to fix where found by Inspection Cost if fixed during maintenance 3/27/

16 Summary The implementation of Inspections resulted in Higher Customer Satisfaction with less escaped defects. Effective product development process with significant improvement in early defect detection and prevention. Productivity improvement with savings in backend product maintenance effort. 3/27/