Leveraging Quality For Competitive Advantage

Transcription

1 Leveraging Quality For Competitive Advantage (A Presentation by The Boeing Company) John D. Vu Ph.D Technical Fellow The Boeing Company BOEING is a trademark of Boeing Management Company.

2 Global Demand For IT Workers 15 M 10 Million shortage 12 M 9 M 6 M 3 Million shortage Global Demand for IT workers 3 M 1 M 0.5 M Supply: number of Bachelor Degrees in IT in global schools Source: International IT education conference 2005 Page 2

3 Global Shortage Of IT Workers Competitive global economy creates more demand for IT skilled workers and demand has exceeded supply. Business must hire workers globally to fill this gap or subcontract to reduce future impact Current IT subcontracting market: 45 percent to India 18 percent to Canada and Europe 18 percent to China 8 percent to Russia and E. Europe 6 percent to S. America 5 percent to S. E. Asia (Singapore, Malaysia, Philippines) Source: Gartner Group 2005 Page 3

4 Shortage Of IT Workers = Business Opportunities Projected shortage of 3 million information technology (IT) workers in 2010 and 10 million IT workers in 2020 globally; both India and China are also experiencing shortage of IT workers but have plans to train more up to 3 million to fill this gap by 2012 IT Subcontracting is a USD $100 Billion market in 2004 and could reach USD $200 Billion by 2010 IT Workers in China and India are pushing their regional economies to a new high level India s IT business continues to grow 20 percent annually with revenue of USD $48B annually Source: Gartner Group 2005 Page 4

5 Current Opportunity Most major subcontracting deals, estimated at USD $118 billion in annualized contract value (ACV), are set to expire between Approximately one quarter of these deals have already been renewed, based on confirmation by key suppliers in 2006, but the majority of these contracts, valued at $88 billion, have not New subcontracting deals, estimated at USD $98 billion in contract value are projected in the next ten year Many countries and suppliers are positioning themselves aggressively for this business Source: Gartner Group 2005 Page 5

6 Factors Affecting Client s Choice Cost Access to Low-cost Labor Pool, Attrition Levels Infrastructure Quality Skills Telecom, Power and Office-space cost, Transportation conditions Quality Commitment, Process maturity, Other Certifications Education Systems, Training, Communication Knowledge Experiences, Domain expertise, Specialization Customization Source: CMU subcontracting Benchmarking study 2004 Page 6

7 Change In Determinant Factor With Time Revenue Client s Domain Knowledge Skills Infrastructure Low Cost Labor Quality Delivery Time Source: CMU subcontracting Benchmarking study 2004 Page 7

8 Market Outlook: Low Cost Competition The large pool of highly skilled workers in low-wage countries is growing quickly, fueled by a strong increase in information technology graduates with average growth > 5.5 percent annually The number of degrees awarded in information technology jumped from 15 percent to 45 percent in Philippines, Malaysia, Indonesia in past five years Currently India accounts for nearly 30 percent of IT professionals while China has 11 percent of global workers The share of degrees awarded in business management jumped from 18 percent to 31 percent in Russia, and from 16 percent to 36 percent in Poland in just five years Source: CMU subcontracting Benchmarking study 2004 Page 8

9 Low Cost Subcontracting Suppliers India: Low Cost Provider with experiences Indian salaries are rising > 15 percent per year High employee turn-over > 21 percent Shortage of skilled workers China: Low Cost Provider with strong infrastructure Language barrier (English) Project management needs improvement Shortage of skilled workers Eastern Europe: Low Cost Provider with Western culture Growing market segment due to young workforce Good command of English language Strong government support (Romania, Poland, Czech Republic and Hungary) Philippines: English-speaking low-cost provider Alternative Asia destination Good English skills throughout its population Source: CMU subcontracting Benchmarking study 2004 Page 9

10 What Worked and Did Not Worked Subcontracting results are mixed: The Gartner/Dataquest survey found that: 1) 82 percent report moderate satisfaction 2) 37 percent report canceling contract 3) 55 percent report renegotiating contract The 2006 Information Week s study found: 1) 57 percent of contracts met obligations 2) 25 percent of contracts were cancelled due to poor performance 3) 14 percent of contracts were not fully satisfactory Study also found many low-cost suppliers failed to perform due to poor quality, not meeting clients schedule and functionality Page 10

11 Expand Market Beyond Cost Many global companies that subcontracted have gone beyond cost as the main factor and are seeking quality and domain expertise in specific areas that can meet their needs Improving quality and meeting client s business needs in both financial and technical are the key success factors in expanding subcontracting business Vietnam s IT subcontracting companies need to focus on both quality and understanding of client s needs to capture this market Vietnam s IT industry needs to improve education and training in higher value skills that are in high demand today and in the future Source: CMU subcontracting Benchmarking study 2004 Page 11

12 Software Quality Software quality is becoming increasingly important. Software is now using in many applications and software defects has caused serious damage and even physical harm. Defects in financial applications can be very costly but defects in software that fly airplanes, drive cars, operate machines, or run manufactories can be very dangerous. Software suppliers that do not have a good quality management systems are risking their business in a highly competitive world. Many global clients are now consider quality as the key factor in selecting software suppliers. Page 12

13 Cost of Software Defects Requirement Analysis Requirements Definition $1 Design $5 BUGS Coding $20 Relative cost cost factor factor to to find find and and fix fix defects at at each each phase phase of of the the Software Development Life Life Cycle Cycle Source: Source: Boehm, Boehm, B., B., Software Software Engineering Engineering Economics, Economics, Prentice-Hall, Prentice-Hall, Testing $50 Maintenance $100 Page 13

14 Find/Fix Cycle Testing found a defect Defect reported Defect being investigated and verified Defect fixed and test Development includes fix in next baseline New release to SQA SQA performs regression testing This cycle can take from hours per defect Using an average of $15 per hour for Software Engineering time... Cost per defect is: 30 * $15 = $450 For a product with 100 defects, the cost is: 100 * $450 = $45,000 Page 14

15 Quality Issues Low quality product = Higher cost Low quality product = Take longer time Low quality product = May not meet schedule Low quality product = May not have all features Low quality product = May not meet customer s needs Low quality product = Unhappy customer Low quality product = Bad for business Almost half of software subcontracting contracts did not meet quality requirements and end up costing subcontracting suppliers their business Many customers are willing to pay higher prices for higher quality products Customer satisfaction = key factor to success in global subcontracting business Page 15

16 Improve Quality = Focus on Process The quality of a software product is governed by the quality of the processes used to develop and maintain it To improve quality of the product, one must improve the quality of the processes that create the product Page 16

17 Software Process A process is a sequence of steps performed for a given purpose A software process is a set of activities, methods, practices, that people employ to develop and maintain software and the associated products (e.g. project plans, design documents, code, test cases, and user manuals) Note: a process is what people do - not what documentation says they should do Source: The Capability Maturity Model: Guidelines for Improving the Software Process, Addison-Wesley, 2005 Page 17

18 Context Customer needs and Market trends Software Development and Maintenance Process Software Product Business Goals, Objectives Process Improvement Implementation plans and measures Software Development and Maintenance Process: The development and maintenance of products that meet the needs of customers and markets Software Improvement Process: Application of technology and disciplines to improve software process to meet business goals and objectives Page 18

19 Capability Maturity Models Integration (CMMI) Created at Software Engineering Institute (SEI), a federally funded research lab at Carnegie Mellon University (CMU) as: Frameworks for process improvement based on state-of-the-art software and systems engineering practices. A benchmarking model to compare capability of software organizations. A model tool used to select software and systems contractor by U.S. Department of Defense (DoD). History: Software CMM (1989); System CMM (1994); People CMM (1996); CMM Integration (CMMI) 1999; IPD CMM (1999) SA-CMM (2002), CMMI Acquisition (2006) Page 19

20 Global CMMI Usage Capability Maturity Model Integration (CMMI) is widely used worldwide: Over 3500 appraisals conducted ( ), more than half were outside the U.S. CMMI in regions around the world: Asia/Pacific Western Europe Eastern Europe S. America 35 Australia Source: CMU s Benchmarking Study Page 20 Maturity Profile Worldwide Level 1 Level 2 Level 3 Level 4 Level 5

21 The CMMI Maturity Levels Focus on continuous process improvement of the business, organization, and projects Process characterized for the business where critical processes are measured and controlled Process characterized for the organization and is proactive Defined Optimizing Quantitatively Managed 2 1 Process characterized for projects and is often reactive Process unpredictable, poorly controlled, and reactive Initial Managed Page 21

22 The Capability Maturity Models The CMMI have become a de-facto standard for assessing and improving software processes SW-CMM (earlier versions) 1988 SW-CMM V CMMI V CMMI V CMMI V Process Improvement using CMMI is > 16 years Page 22

23 Failure To Improve Industry data found that 72 percent of organizations report little or no success in software improvement after an assessment 83 percent of organizations abandon their improvement efforts in the first three years 57 percent of organizations that abandoned improvement efforts did restart their improvement later Less than one percent of organizations claimed success or reported any improvement data * SEI Software Industry Benchmarking Study 2004 Page 23

24 Why Do So Many Improvement Efforts Fail? 1. Over-emphasis on passing an appraisal but not much focus on making quality improvement happen 2. Focus on maturity levels without measurable objectives 3. Lack of a skilled personnel to manage quality improvement activities for business advantages 4. Confusion between terminology and actual practices 5. Implementation of improvement solutions poorly managed Source: SEI Software Industry Benchmarking Study 2004 * Software Industry Benchmarking Study 2001 Page 24

25 Lessons Learned Appraisal consumes a lot of effort Training Logistics and Preparation Artifacts, documentation and reviews Interviewing Briefing Appraisal is expensive Hiring qualified lead assessor Training and tools (templates and documents) Consulting fees and follow-on activities How many appraisals are needed? Several attempts to achieve a maturity level? The need to buy tools and templates to pass the appraisal Page 25

26 Some People Believe that... The goal of process improvement is the maturity level Many organizations confuse maturity level with improvement objectives Level 2 by 2002 Level 3 by 2003 Level 4 by 2004 or else! Some organizations believe the maturity level is the miracle that can make business happen CMMI Page 26

27 However, The Fact Is... Maturity levels are meaningless if they cannot be explained in terms of business objectives Profit and Loss? Customer Satisfaction? Operating Costs? Time to Market? Productivity? Business Operational Efficiency? Balanced Scorecard Revenue? Quality? Market Share? Skills and Knowledge? Page 27

28 The Need To Link Improvement To Business Value Organizations need to understand that the goal of improvement must be based on business needs, NOT maturity levels. Management must communicate process improvement in terms of Business Value: Increase Quality Reduce Cost Reduce Cycle Time Increase Productivity Increase Customer Satisfaction Increase Employee Satisfaction If process improvement is not directly aligned with business goals and objectives, it has no value and should not be allowed to continue. Page 28

29 Some People Believe that... Just document all CMMI practices and select few projects to represent the organization for the appraisal then claim the entire organization has achieved a high level of maturity Many organizations believe CMMI is a requirement, and they must document everything according to this book Many organizations are interested in a maturity level and doing just the minimum to pass an appraisal This approach will not improve the business or help the organization to succeed in a highly competition world Page 29

30 However, The Fact Is... Appraisal is more than just reviewing documents. Processes must be practiced, used, measured, continuously improved to achieve business goals and objectives. Process improvement is about solving critical business problems, NOT achieving meaningless maturity levels Appraisals must focus on both process compliance and organizational performance The appraisal method requires organizations to disclose the scope of the appraisal, and all improvements must be measured as part of the business value stream Page 30

31 Some People Believe that... Just fill in this template -- improvement will come! The CMMI requires documentation These templates are your document Many organizations are buying CMMI templates to substitute for process documentation Having documents according to a CMMI book does not reflect your company s processes or the way your company does business Documentation alone will not lead to improvement. All processes must be used, measured, verified and institutionalized Page 31

32 However, The Fact Is... Process improvement is about changing the way people develop software by practicing a disciplined engineering approach to meet the organization s business goals. Managers must identify their workforce s competencies and provide training to improve the way the organization develops software Managers must facilitate and coordinate improvement activities and measure improvement benefits Skills and knowledge are key assets to improving the business Page 32

33 Context Scope of Improvement 5 Area Of Management Enterprise 4 Business Value Product Line Management Business Unit/Domain 3 Cost/Cycle Time Improvement Process Management Organization Quality Improvement Product Management 2 Project Schedule Improvement Project Management 1 Page 33

34 Page 34

35 Context Scope of Improvement 5 Area Of Management Enterprise 4 Business Value Product Line Management Business Unit/Domain 3 Cost/Cycle Time Improvement Process Management Organization Quality Improvement Product Management 2 Project Schedule Improvement Project Management 1 Page 35

36 Comparison Between Level 1 and 2 Level 1 Level 2 Ad-hoc process Undocumented practices Schedule is fixed Guesstimates No measurement Reactive mode No facts and data Disciplined process Documented practices Schedule is negotiable Project estimates Project measurement Get out of reactive mode Facts and data being collected Page 36

37 Overview Of Level 2 Requirements Management Approved Requirements Process and Product Quality Assurance Commitments Approved Requirements Commitments Approved Requirements Project Planning Commitments Commitments Project Monitoring and Control Allocated Requirements Review Progress Configuration Management Supplier Agreement Management Measurement and Analysis Page 37

38 What Is Software Project Management? Definition: The application of knowledge, skills, tools, and techniques to software project activities in order to meet customer s needs and expectations Meaning: Complete the software project: On time Within budget According to plan According to customer s requirements Page 38

39 Software Project Management Phases 1) Initiating To gain an understanding of all aspects of the project To define roles and responsibilities To determine completion and success criteria 2) Planning To determine the project scope To define actions required to achieve project objectives To determine resources needed for the project To define the means to control the execution of the project 3) Executing To manage the process defined to deliver the product To effective mitigate project risks To measure project performance To manage project performance To manage project integration with others Page 39

40 Software Project Management Phases 4) Controlling To track progress against the plan To take corrective actions when there are deviations from the plan 5) Closing To deliver software product to customer To conclude the business relationship in the context of the project Page 40

41 Project Planning Project plans are the basis for executing a project. The project manager's ability to control a project is highly dependent on the accuracy and completeness of the plan The project plan development the most important part of the project A good project plan includes estimates, schedules, resources, manpower, skill requirements, and risk management The plan should also include how configuration management, quality assurance, progress and earned-value tracking, and requirements management will be accomplished Key personnel skills and key technologies should also be identified along with management reserves for high-risk elements of the plan Page 41

42 Project Planning Process Initial Requirements (Scope, Functionality Schedule, Budget) Decompose requirements Estimate Size Re-Negotiate Estimate Resource Estimation Estimate Schedule No Is Schedule Meeting Customer s Requirements yes Estimates Page 42 Actuals Tracking Actuals vs Estimates Develop software

43 Improve Project Estimates The utilization of historical data will improve project performance by reducing the variation in project estimates. Better estimates will improve project schedules. Better schedules will improve project management. Better management will improve project performance in terms of increase quality and reduce costs. Better performance will improve customer satisfaction. Satisfied customers will improve relationships. Better relationships will improve the business. Source: Lessons Learned From projects Page 43

44 Improvement In Project Estimates Software Estimates (Actual vs Planned) Schedule Variation + 22 percent percent +26 percent -125 percent Utilize Historical data for all estimates +20 percent -24 percent +12 percent -18 percent +4 percent -7 percent CMMI Level 4 and 5 Over estimates Under estimates Source: CMU Data collected from 180 software projects Page 44

45 A Project Plan A Project Plan should include the following: The project s chosen software life cycle A list of products to be developed (Deliverables) Schedules Estimates (cost, size, effort, etc.) Facilities, support tools, hardware Project risks -- identify, mitigate Plans must be based on estimates In creating estimates for size, effort, cost, schedule, etc: Use historical data, where available Document assumptions and estimates Page 45

46 Project Monitoring and Control Provide understanding of the project s progress so that appropriate corrective actions can be taken when the project s performance deviates significantly from the plan Meaning Monitor progress against project plan: Actual vs. planned (cost, effort, schedule, milestones, etc.) Progress reviews (earned value, return on investment, etc.) Monitor risks (data, skills, turnover etc.) Analyze issues Take corrective action Track corrective action to closure Page 46

47 Monitoring and Control Project Monitoring Project manager compares progress of product development against the project plan and discusses variances with project team to identify problems and initiate a search for possible solutions Manage Changes to Scope Project manager gives visibility to changes in the scope. Change requests are used to define and justify changes in scope Control issues Problems that prevent the project team from working or being successful are controlled by project manager Project reviews Allow for the approach and process being followed by the project team to be evaluated and changed, if needed Page 47

48 Monitoring and Control Technical reviews Project manager validates the technical decisions and directions adopted by the project Documentation control Performed by Configuration Management function Product delivery control Performed by Changed/Release Management function Lessons Learned... Use Wrap-Up activities at the end of the project to produce Lessons learned and reassign personnel and equipment Page 48

49 Project Requirements The key to the success of any project is the establishment of project requirements; goals and outcomes are derived from requirements and communicate expectations between customer and developer On successful projects, requirements are clearly understood by user and developer. Additionally, requirements must be complete, correct, precise, consistent, relevant, feasible, manageable, testable, traceable, and free of design detail Requirements must be controled. Project managers who run successful projects control requirements and ensure there is a common understanding of requirements between customer and developer Page 49

50 Successful Tips: Freeze requirements before planning a project If requirements are changed or added, replan the project For organizations that maintain software and develop periodic software updates, freeze the set of requirements for each software release and provide a mechanism to allow a new high-priority requirement to be added to the current development baseline Page 50

51 Requirements Management Manage the requirements of the project and identify inconsistencies between those requirements and the project s plans and work products Meaning 1. Obtain requirements from customer 2. Document requirements 3. Control requirements changes 4. Maintain bi-directional traceability of requirements 5. Identify inconsistencies between project work and requirements Tips: Requirements must be documented If requirements change, the changes must be documented and all resulting necessary changes in other documents must be tracked and verified Page 51

52 Some People Believe That... The requirements are not clear, but we can work out the details later Most software requirements are ambiguous and not well communicated Many engineers want to start the project quickly and not take time to work out the detailed requirements Customer Page 52 Engineer

53 The Fact Is... Project Manager must work cooperatively with customer to establish and maintain a requirements baseline to ensure all requirements are expressed in simple, unambiguous terms, quantified and verifiable Requirements must be: Manager 1. Documented 2. Clear and concise 3. Understood 4. Testable 5. Usable Page 53

54 Configuration Management Establish and maintain the integrity of work products using configuration identification, configuration control, configuration status accounting, and configuration audits Meaning Identifying configuration items/units Systematically controlling changes Establish baselines Control changes Maintaining integrity and traceability of the configuration throughout the software life cycle Reporting accurate status to all stakeholders Page 54

55 The CM Disciplines Configuration Management (CM) Configuration Management Planning Change Management Baseline and Integrity Control Status Accounting Configuration Identification Configuration Audits and Review Page 55

56 Configuration Management Planning Discipline Question Answer What CM activities are All CM activities are identified in performed? The CM plan and included in the project schedule Identification What is my software The software consists of the configuration? following items..., each of which has its own version designation... Change How are changes The steps are... Management managed? Integrity and How do you baseline The baseline content is... Traceability and control changes? The version is. Status accounting What changes have The current configuration been made and when? was changed as follows... Audits and Review How do you make sure The CM process is reviewed changes are correct and for compliance per scheduled the process is followed? reviews and outside audits Page 56

57 Change Management Flow Change Change Board Analyst Approve/Reject Analyze Estimate Recommend Change Board Programmer Authorize Implement CM Librarian Control/Storage Tester/QA Change Board Verify Release Approve Authorize Baseline Page 57

58 Change Control Board (CCB) Change Board needs following information to make decision for each change: Size (How big is the change?) Complexity (Is the change within a component or involve others?) Schedule/Priority (When do you want it?) Impact (Consequences of this change?) Cost (Potential cost or cost saving by making this change?) Severity (How critical is the change?) Relationship to other changes (Superceded or invalidate others?) Test (Any additional test required?) Resource (How many people needed for this change?) Benefits (Advantages by implementing this change?) Page 58

59 Perform CM Audits Audit should be performed periodically to ensure the CM practices and procedures are consistently followed. Internal audit (performed by CM staff) External audit (performed by QA) CM audit must be planned and scheduled (in CM plan) CM audit must be performed before every major baseline change CM audit team must be trained and NOT involved in tasks being audited CM audit must verify that all changes to baseline are implemented properly CM audit must verify that the CM process is being followed appropriately CM auditing is continuous throughout the life of a project Page 59

60 Process and Product Quality Assurance Provide staff and management with objective insight into processes and associated work products Meaning Objectively evaluate processes, work products, and services against the applicable process description, standards, and procedures Identify non-compliance issues Provide feedback to project staff and managers Ensure non-compliance issues are addressed The value of QA is that it provides an independent view of project activities, process, and product Page 60

61 Quality Assurance Responsibilities Review all plans (project plan, CM plan, testing plan, etc.) for completeness Participate in project and phase reviews as indicated in project plan Review all test plans for compliance to standards Review a sample of all test results for compliance to plans Audit CM performance to determine compliance to standard Participate in organization establishment of standards, procedures, processes Page 61

62 Measurement and Analysis Develop and sustain a measurement capability that is used to support management information needs Meaning 1. Specify the objectives of measurement and analysis 2. Define the measures, data collection and storage, analysis techniques, reporting, and feedback 3. Implement project measurement 4. Provide objective results to support management decisions and corrective actions Without data, quality is just talk Page 62

63 Measurement Process Identify Scope Objectives Purpose Measures Define Procedure Measures Collection Analysis Feedback Collect Data Record and Store data Revise and Validate Extract Improve Process Evaluate Progress Evaluate process Recommend Take Actions Analyze Data Prepare Data Chart Report Recommend Page 63

64 Example of Project Measures 1. Defect: Pre-released Post released 2. Efforts: Plan vs. actual 3. Cycle time: Cycle time per development phases Cycle time per each product release 4. Project estimates: Size, effort, time (plan vs. actual) 5. Customer satisfaction 6. Employee satisfaction Page 64

65 Context Scope of Improvement 5 Area Of Management Enterprise 4 Business Value Product Line Management Business Unit/Domain 3 Cost/Cycle Time Improvement Process Management Organization Quality Improvement Product Management 2 Project Schedule Improvement Project Management 1 Page 65

66 Moving From Level 2 to Level 3 At level 2, the focus is on project management At level 3, the emphasis shifts to consistent processes across the organization Assets are gathered across the organization Standard processes are defined based on assets The organization supports projects by establishing: An organization s standard processes An organization s common measurements An organization s required training program Page 66

67 Comparison Between Level 2 and 3 Level 2 Level 3 Stabilizing the environment Project management Processes are repeatable Project management discipline Historical data is collected used Project measurements Skills and knowledge exist Some training is available Establish standard processes Integrated project management Processes are shared Standardize project management Historical data is analyzed and Project and organization measurements Skills and knowledge are trained Some training is required Page 67

68 CMMI Level 3 At level 1: The organization invests resources primarily in code and testing, but not project management resulting in poor quality and significant rework At level 2: The organization begins to take control by investing resources in project management functions so projects can be developed more efficiently and predictably At level 3: The organization organizes better by focus on standardized processes, where all projects share best practices for more consistency and better performance By level 3, All projects should be able to show measurable improvements in both schedule performance and quality improvement Page 68

69 Implement Level 3 Processes Level 3 focuses on process standardization across the organization; therefore, a sequence of logical steps must be initiated to identify assets, organized into standards and applied to all projects to achieve maximum results: 1) Organization Process Focus (OPF) Set up infrastructure to collect assets for improvement 2) Organization Process Definition (OPD) Define organization standards based on assets 3) Organizational Training (OT) Train people to use organization standards 4) Integrated Project Management (IPM) Apply organization standards in all projects Get all stakeholders working together as a team Page 69

70 Implement Level 3 Processes 5) Requirement Development (RD) Establish product requirements 6) Risk Management (RSKM) Identify and mitigate risks 7) Decision Analysis and Resolution (DAR) Evaluate alternatives and select solutions 8) Technical Solution (TS) Apply engineering methods and tools to solve problems 9) Product Integration (PI) Assemble work products from components 10) Verification (VER) Conduct informal and formal reviews to verify work products 11) Validation (VAR) Perform formal testing to validate work product Page 70

71 Organizational Process Focus Plan and improve organizational processes based on a thorough understanding of the current strengths and weaknesses of the organization Meaning 1. Establish goals and objectives 2. Appraise the organization s processes 3. Identify the organization s weaknesses and strengths 4. Establish improvement plan 5. Implement improvement plan 6. Collect organizational process assets 7. Incorporate process-related experiences into the organizational standard processes Page 71

72 CMMI and Business Goals Business goals are Key drivers Organization s Business Goals CMMI is a guide CMMI Current Process Capability Plan is based on appraisal results Improvement Tasks Task Process Improvement Plan Implementation Training Reviews Data are used to verify improvement results Measurements Page 72

73 Organizational Process Definition Establish and maintain a usable set of organizational process assets Meaning 1. Establish Standard Processes 2. Establish Life Cycle model description 3. Establish tailoring criteria and guidelines 4. Establish the organization s measurements repository 5. Establish the organization s Process Asset Library Page 73

74 Organization s Standard Processes The organization s standard processes are the means by which the organization expresses requirements that all projects development processes must follow The objective of the standard is to establish commonality across the organization s projects, in order to ensure consistency among projects, support common measurements, reduce waste, and redundancy by standardization and continuously improve its process The standard is the high level of abstraction where it contains all the primary processes and rules that all projects are expected to follow The standard establishes common terminology to support better communication between managers, customers, users, and developers Page 74

75 Organizational Process Assets A collection of practices, entities, maintained by an organization, for use by projects in developing, maintaining, implementing, and measuring their practices Process assets include: The organization s standard processes Descriptions of life cycles approved for use Guidelines and criteria for tailoring standard processes Measurement repository (historical database) Process Assets Library (PAL) Page 75

76 Organization s Measurement Repository A central repository for organization s measurement data Contains: Actual measurement data (the numbers) from individual projects The related information needed to understand the measurement data and apply it to new projects This is where planning and replanning data for the organization as a whole are kept Page 76

77 Process Assets Library (PAL) A repository where Assets used on past projects are kept It also contains: Lessons learned reports, example documents Templates In general, the PAL contains documents that can be used as models or examples for future projects Assets = Artifacts with data Page 77

78 Documentation The project plan documents all considerations and tailoring aspects of the organization s standard processes into the project s defined process where it is used to manage specific project Managing according to a plan: The project plan is used to manage the project Process Management: Tailoring is a key element in the organization s ability to develop reasonable and efficient project plans for use in managing and controlling projects according to a standard process Page 78

79 Organizational Training Develop the skills and knowledge of people so they can perform their roles effectively and efficiently Meaning: 1. Identify training needs for every project 2. Establish strategic training plan 3. Develop training capability 4. Deliver the training 5. Establish training record 6. Assess training effectiveness Page 79

80 Training Needs and Plan Project Tasks Critical Skills Training Needs Develop/Procure training Organization s Training Plan Training Opportunities Skills Training Training Effectiveness Feedback Page 80

81 Providing Training Organization Training Plan Measuring Training effectiveness Tracking Training Individual career development plan Individual career development plan Individual career development plan Page 81

82 Integrated Project Management Establish and manage the project and the involvement of the relevant stakeholders according to an integrated and defined process that is tailored from the organization s set of standard processes Meaning: 1. Tailor the standard process into the project s defined process 2. Plan the project by using organization assets 3. Integrate plan 4. Manage project according to project plan 5. Implement project 6. Provide data to measurement repository 7. Contribute to Process Asset Library Page 82

83 Organizational Processes and Project Processes Organization s Standard Processes Life Cycle Tailoring Guide Process Assets Requirements Project Defined Process Process Architecture Project Plan Activities Page 83

84 Integrated Workflow Customer Requirements Establish System Requirements Monitor Work and Resolve Issues Document The IT Plan Coordinate Critical Dependencies Review Work products Manage Intergroup issues Conduct Technical Interchanges Page 84

85 Product Integration Assemble the product from components and ensure that the integrated product functions properly Meaning: 1. Determine the process for integration 2. Define integration procedures and criteria 3. Review interface descriptions for completeness 4. Manage interfaces 5. Assemble product components 6. Evaluate the integrated product 7. Package and deliver the product Page 85

86 Product Integration Prepare for Product Integration Integration Plan Ensure Interface Compatibility Technical Solution Assemblies Subassemblies Assemble Product Components and Deliver the Product Page 86

87 Risk Management Identify potential problems before they occur, so that risk mitigation can be implemented across the life of the project to reduce impacts on achieving objectives Meaning: 1. Determine risk sources and categories 2. Define risk parameters 3. Establish risk mitigation strategy 4. Identify and analyze risks 5. Evaluate, categorize and prioritize risks 6. Develop risk mitigation plans 7. Implement risk mitigation plans Page 87

88 Risk or Issue? Risk is something that has not yet occurred Requires preventive action?????? Issue is something already occurred Requires corrective action Preventive action is cheaper than corrective action Page 88 CC ppt

89 Risk Management Process Identify risk What can go wrong? Analyze risk How are things going? Communicate and track risks How likely is it? What would be the consequences? Assess handling options What can we do about it? Plan and perform mitigation Risk Management is a continuous, disciplined process for managing uncertainty What is needed to reduce the risk? Page 89

90 Technical Solution Design, develop, and implement solutions to satisfy customer s requirements Meaning 1. Develop solutions and selection criteria 2. Define product-component solutions 3. Perform Make, buy, or reuse analysis 4. Design the product component 5. Design the interfaces 6. Implement the design 7. Develop final product that meet requirements Page 90

91 Decision Analysis and Resolution To analyze decisions using a formal evaluation process that identifies alternatives against established criteria Meaning: 1. Provide a formal process to evaluate alternatives 2. Establish guidelines for decision making analysis 3. Evaluate alternatives (requirements, functions, configurations) 4. Balance considerations (cost, performance, productivity, testability,compatibility, supportability, etc) 5. Select final solution Page 91

92 Engineering = Balanced Trade-Off Performance requirements Technical parameters Productivity Cost Schedule Reliability Maintainability Page 92

93 Verification Ensure that work products meet their specified requirements Meaning 1. Select work products for verification 2. Establish verification procedures and criteria 3. Conduct Peer reviews 4. Analyze Peer review data 5. Perform verification 6. Analyze verification results and identify corrective action Page 93

94 Validation Demonstrate that a product fulfills its intended use when placed in its intended environment Meaning 1. Select product for validation 2. Establish the validation environment 3. Establish validation procedures and criteria 4. Perform Validation 5. Analyze validation results Page 94

95 Context Scope of Improvement 5 Area Of Management Enterprise 4 Business Value Product Line Management Business Unit/Domain 3 Cost/Cycle Time Improvement Process Management Organization Quality Improvement Product Management 2 Project Schedule Improvement Project Management 1 Page 95

96 Quantitative Process Management Using data to maximize benefit of software development process 1) 1) Project Performance Size Size variances trend (Plan vs. vs. Actuals) Effort variances trend (Plan vs. vs. Actuals) Defect density trend Schedule variances trend... 2) 2) Organization Performance Number of of projects following the the standard process Milestone completion Requirements changes Organization schedule variance... These measurements help help the the organization determine where problems exist exist to to allow for for correction as as early as as possible Page 96

97 Software Quality Management Using data to manage products and services to achieve business goals 1) 1) Project performance data data Defect detection within limits Other quality attributes (Reliability, Usability, etc.) etc.) Cost/effort within limits Schedule performance within limits... 2) 2) Organization performance data data Number of of projects performing within limits Milestone completion Requirements changes within limits... These measurements help help the the organization determine where Product problems may may exist exist to to fine fine tune tune the the standard process Page 97

98 Statistical Process Control Measurement Program Project Performance Measurements Organization Performance Measurements Product-Line Measurements Project Performance Measures Project Performance Metrics Deployment Progress Measurements Process Penetration Metrics Product/Process Correlation Metrics Cycle time Effort Size Defect Core measures are in red Normalized cycle time Effort schedule variance Cost of rework Normalized defect/phase Deployment Progress Measures # of recommendations Deployment status Percent of project using processes Deployment Progress Metrics Percent of recommendations deployed Alignment to Product-line Goals and Objectives Page 98

99 Correlation Between QPM and SQM Analysis of of process data data and and product data data help help the the organization to: to: Determine the the effectiveness of of the the standard process Forecast trends Maximize investment Identify skills skills needed Plan Plan budget expenditures Prepare for for problem resolution Measure customer // user user satisfaction Measure employee satisfaction Identify conflict between groups Quantify return on on investment (ROI) Page 99

100 Example: Improve Project Estimates The utilization of historical data will improve project performance by reducing the variation in project estimates Better estimates will improve project schedules Better schedules will improve project management Better management will improve project performance in terms of increase quality and reduce costs Better performance will improve customer satisfaction Satisfied customers will improve relationships Better relationships will improve the business Page 100

101 Improvement In Project Estimates Schedule Variation + 22 percent percent Software Estimates +26 percent -125 percent Source: CMU Data collected from 180 projects (Actual vs Planned) Utilize Historical data for all estimates +20 percent +12 percent -24 percent -18 percent +4 percent -7 percent CMMI Level 4 and 5 Over estimates Under estimates Page 101

102 Improvement Benefits Source: CMU Data collected from 180 projects Page 102

103 Improvement Benefits Source: CMU Data collected from 180 projects Page 103

104 Product Lines: Maximize Benefits Organization must leverage existing components to produce systems or services that meet the demand of the business A product line is a set of components that share a common, managed set of features, satisfying the specific needs of a particular market segment that are developed from a common set of core assets in a prescribed way By integrating existing components into new products, the enterprise can speed up time to market, reduce cost, increase productivity, quality, and gain significant advantage in the highly competitive business world Page 104

105 Reduce Cost = Increase Software Reuse Percent of Software reuse ? 10 percent 25 percent 58 percent 64 percent Level Level 1 Level Level 2 Level Level 3 level level 4 level level 5 Source: CMU Data collected from 180 projects Page 105

106 Context Scope of Improvement 5 Area Of Management Enterprise 4 Business Value Product Line Management Business Unit/Domain 3 Cost/Cycle Time Improvement Process Management Organization Quality Improvement Product Management 2 Project Schedule Improvement Project Management 1 Page 106

107 Behavior Of Higher Levels Organization 1. Organization can demonstrate business benefit from process improvement (improvement trends or results) 2. Organization must know which projects follow (or not) the standard processes 3. Organization s processes must be verified independently that they are used and controlled at project level 4. Day-to-day decision making must be made based on facts and data (where appropriate) 5. Business goals are prioritized and intergroup conflicts are resolved according to a business strategy and plan Page 107

108 Business Scorecard Internal Perspective What must we be good at? How well are we meeting our business goals? How satisfied are our employees? Speed and Efficiency Growth and Acquisitions Employee satisfaction Innovation and Learning Capability Perspective What do we need to drive future performance? Skills and Knowledges Financial Perspective How do we look to shareholders? Customer Perspective How do they perceive us? How satisfied are they? Have their needs been met? Reduce Cost, Reduce Cycle time Increase Quality Increase Customer Satisfaction Page 108

109 Defect Prevention Project teams analyze defects and determine their causes Project teams evaluate organizational processes to prevent known types of defects from recurring Organization uses defect data to forecast defect trends and set quantitative goals Organization monitors defect trends to improve product quality Page 109

110 Formal Review Benefit Ratio Rework Effort Before Formal Review After Implemented Formal Review 8 percent 1 percent 12 percent 3 percent 19 percent 4 percent Reduce 31 percent in rework Req. Design Code Test Post-Release Formal Review increased effort by 4 percent decreased rework by 31 percent Cost: Benefit ratio is four percent : 31 percent or 1: 7.75 Page 110

111 Technology Change Management Organization identifies new technologies Organization follows a process to evaluate the new technologies Organization selects new technologies based on a business strategy and cost-benefits data Organization adopts new technologies according to a defined technology transfer process Organization measures and monitors adopted technologies and continuously improve its technology change management process Page 111

112 Process Change Management Organization management continuously monitors all improvement activities and makes needed changes to optimize its return-on-investment Organization management continues to broaden the scope of improvement across the product line Organization management encourages people to be involved in the business and empowers them to make changes as needed Improvement occurs in both incremental improving of the process and by technological breakthrough by innovation Page 112

113 And Finally. Page 113

114 Has Your Organization Experienced: Page 114

115 Industry Data On Improvement Process improvement can reduce costs an average of 35 percent per year Some organizations have sustained cost improvements of 55 percent or more per year Process improvement can improve quality an average of 40 percent per year Some organization have sustained quality improvements of 70 percent or more per year Source: James Herbsleb et al, Benefits of CMM based Software process improvement CMU/SEI 94-TR-13, 1994 Page 115

116 Industry Data On Improvement Organization Results Organization Results Results Boeing information systems Estimates within 20 percent, $6 million saved in 1 year CSC 65 Percent reduction in error rates CSC 65 percent reduction in error rates Hewlett Packard SESD ROI 9: 1 Hewlett Packard SESD ROI 9: 1 Hughes $2 Million annual reduction in cost overruns Hughes 2 million annual reduction in cost overruns IBM Toronto 90 Percent reduction in defects, 80 percent reduction IBM Toronto 90 in percent reworkreduction in defects, 80 percent reduction in rework Motorola GED GED 2.3X 2.3X productivity Productivity increased increased ROI ROI 6.7: 16.7: 1 Raytheon ROI ROI 7: 7: 1 1 Siemens percent Percent reduction in defects in product defects Telcordia Reduce defect defect 90 percent, 90 percent, ROI ROI 6.5: 16.5: 1 Source: CMU s Benchmarking Study 2004 Page 116

117 CMMI Performance Improvement CMMI Defect Productivity Rework Time Variance ROI % + 15% - 28% + 12% 160% - 16% % + 42% - 32% + 35% 85% + 80% % + 75% - 78% + 85% 32% + 176% % + 95% - 96% + 93% 18% + 420% Source: CMU CMMI Study Defect: # Error/KSLOC; Productivity: KLOC/Man-Month; Rework: Effort of fixing defect/total project Time; Project Estimate variances: Planned vs. Actuals; Cycle Time: Time response to customer change request; ROI: Labor cost saving/cost of improvement Page 117

118 What s Next You have seen the global market opportunity and how quality can be a key discriminator in getting business The Boeing Company would like to help Vietnam improve its information technology industry and be a key player in the global market People Working Together Page 118

119 119