Carnegie Mellon University Process Maturity Profile Software CMM 2004 Year End Update March 2005 We could not produce this report without the support of the organizations and lead appraisers who reported their appraisal results to the SEI SM. Our many thanks for their continuing cooperation with our data collection and analysis efforts. CMMI Appraisal Program The is a federally funded research and development center sponsored by the U.S. Department of Def ense and operated by Carnegie Mellon University CMM and Carnegie Mellon are registered in the U.S. Patent and Trademark Office by Carnegie Mellon University SM SEI is a service mark of Carnegie Mellon University March 2005 Process Maturity Profile - Page 1
Outline Introduction Current Status Community Trends Organizational Trends Summary This briefing is based solely on the number of appraisals reported to the SEI. It cannot be used as a source for an exact indicator of all organizations in the world using SEI models or appraisal methods nor can it be used for certification or verification purposes. The SEI does not certify organizations. The SEI only licenses SEI partners and SEI authorized lead assessors to conduct appraisals. Neither the SEI nor any other organization is a certifying authority of the results from an appraisal. Therefore requests to the SEI to provide information to be used for this purpose can not be fulfilled. Information provided by the SEI, such as this briefing, is to demonstrate use of its products and services only. Organizations performing source selection or verification should consider performing an evaluation appraisal. For more information on appraisal methods and for a directory of authorized lead appraisers who can perform them, visit: http://www.sei.cmu.edu/managing/app.directory.html March 2005 Process Maturity Profile - Page 2
Introduction - 1: Purpose and Source Characterize the process maturity of the software community This briefing uses information from appraisals using the SW-CMM in: CMM -Based Appraisals for Internal Process Improvement (CBA IPIs) and Software Process Assessments (SPAs) and Standard CMMI Appraisal Method for Process Improvement (SCAMPI SM ), v1.1 Please visit: http://www.sei.cmu.edu/sema/profile_about.html for additional information or to answer questions you may have about this briefing March 2005 Process Maturity Profile - Page 3
Introduction - 2: Briefing Contents This briefing includes three primary sections: 1. Current Status - Snapshot of the software community based on the most recent appraisal, since 2000, of reporting organizations 2. Community Trends - Global distribution of appraisals - Growth in the number of appraisals performed - Shifts in the maturity profile over time 3. Organizational Trends - Analysis of Key Process Area (KPA) satisfaction - Time to move up in maturity based on change in maturity level and time between appraisals March 2005 Process Maturity Profile - Page 4
Section 1: Current Status Appraisals conducted from 2000 through December 2004 and reported to the SEI by January 2005 1,940 organizations 894 participating companies 10,038 projects 58.7% Non-USA organizations Please refer to: Terms Used in this Report on page 29 March 2005 Process Maturity Profile - Page 5
Maturity Profile by All Reporting Organizations 100% 90% 80% % of Organizations 70% 60% 50% 40% 30% 42.1% 30.4% 20% 10% 10.4% 7.8% 9.3% 0% Initial Repeatable Defined Managed Optimizing Based on most recent appraisal, since 2000, of 1,940 organizations reporting a maturity level rating March 2005 Process Maturity Profile - Page 6
Reporting Organization Categories Commercial/In-house 77.8% DoD/Fed Contractor 18.5% Military/Federal 3.7% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% % of Organizations Based on 1,940 organizations reporting organization categories March 2005 Process Maturity Profile - Page 7
Maturity Profile by Reporting Organization Categories 100% 90% 80% % of Organizations 70% 60% 50% 40% 43.0% 35.1% 58.3% 43.7% 30% 27.9% 20% 16.7% 16.7% 10% 0% 10.3% 9.7% 10.5% 8.3% 5.8% 6.9% 5.6% 1.4% Initial Repeatable Defined Managed Optimizing Commercial/In-house DoD/Fed Contractor Military/Federal Based on most recent appraisal, since 2000, of 1,940 organizations reporting categories and a maturity level rating March 2005 Process Maturity Profile - Page 8
Organization Type Based on Primary Standard Industrial Classification (SIC) Code Business Services 35.0% Retail Trade 0.7% Wholesale Trade 0.7% Transportation, Communication, Electric, Gas and Sanitary Services 6.6% Public Administration (Including Defense) 8.8% Finance, Insurance and Real Estate 13.0% Services 45.8% Furniture And Fixtures 0.1% Engineering & Management Services 10.4% Manufacturing 24.2% Chemicals And Allied Products 0.4% Primary Metal Industries 0.4% Services, Nec 0.2% Health Services 0.1% Amusement & Recreation Services 0.1% Instruments And Related Products 6.1% Electronic & Other Electric Equipment 5.4% Transportation Equipment 6.1% Printing And Publishing 0.5% Fabricated Metal Products 1.0% Industrial Machinery And Equipment 4.4% Based on 818 organizations reporting SIC code. For more information visit: http://www.osha.gov/oshstats/sicser.html March 2005 Process Maturity Profile - Page 9
Organization Size Based on the total number of employees within the area of the organization that was appraised 501 to 1000 4.8% 1001 to 2000 1.9% 2000+ 1.2% 25 or fewer 11.8% 301 to 500 7.4% 201 to 300 9.5% 201 to 2000+ 24.8% 26 to 50 18.3% 1 to 100 53.3% 101 to 200 22.2% 51 to 75 13.0% 76 to 100 9.9% Based on 1,902 organizations reporting size data March 2005 Process Maturity Profile - Page 10
Maturity Profile by Organization Size Based on the total number of employees within the area of the organization that was appraised 100% 90% 80% The number of organizations for the 2000+ category is small whic h inflates the maturity level bars. Please see page 9 and take this into account. The purpose of this chart is to indicate that all size categories contain most, if not all, maturity levels. % of Organizations 70% 60% 50% 40% 30% 20% 10% 0% 13.3% 65.3% 18.2% 1.3% 1.8% 11.5% 53.2% 27.0% 11.7% 42.1% 32.0% 5.7% 2.6% 6.1% 8.1% 11.7% 37.8% 31.9% 9.6% 9.0% 8.7% 37.4% 34.5% 10.9% 8.5% 8.3% 36.1% 31.1% 9.4% 15.0% 9.2% 29.8% 36.2% 38.5% 40.5% 29.7% 17.0% 20.9% 14.3% 22.0% 16.2% 7.8% 4.4% 8.1% 5.4% 27.3% 22.7% 31.8% 9.1% 9.1% 25 or fewer 26 to 50 51 to 75 76 to 100 101 to 200 201 to 300 301 to 500 501 to 1000 1001 to 2000 2000+ Initial Repeatable Defined Managed Optimizing Based on 1,902 organizations reporting size data and a maturity level rating March 2005 Process Maturity Profile - Page 11
USA and Non-USA Reporting Organization Categories Commercial/In-house 24.4% 53.4% DoD/Fed Contractor 13.2% 5.3% USA Non-USA 3.7% Military/Federal 0% 10% 20% 30% 40% 50% 60% % of Organizations Based on 802 U.S. organizations and 1138 Non-USA organizations reporting organization categories March 2005 Process Maturity Profile - Page 12
Maturity Profile by All Reporting USA and Non-USA Organizations 100% 90% 80% 70% % of Organizations 60% 50% 40% 30% 48.6% 37.5% 27.3% 32.6% 20% 10% 17.1% 5.7% 4.1% 10.4% 2.9% 13.8% 0% Initial Repeatable Defined Managed Optimizing USA Non-USA Based on 802 U.S. organizations and 1138 Non-USA organizations reporting categories and a maturity level rating March 2005 Process Maturity Profile - Page 13
Section 2: Community Trends CBA IPIs, SPAs and SCAMPI v1.1 using the SW-CMM conducted since 1987 through December 2004 and reported to the SEI by January 2005 3,590 appraisals 3,042 CBA IPIs 484 SPAs 64 SCAMPIs 2,746 organizations 1,036 participating companies 663 reappraised organizations 15,128 projects Please refer to: Terms Used in this Report on page 29 March 2005 Process Maturity Profile - Page 14
Countries Where Appraisals have been Performed and Reported to the SEI Argentina Australia Austria Barbados Belgium Brazil Canada Chile China Colombia Costa Rica Cyprus Czech Republic Denmark Egypt Finland France Germany Greece Hong Kong Hungary India Indonesia Ireland Israel Italy Japan Korea, Republic of Latvia Lithuania Malaysia Mauritius Mexico Morocco Netherlands New Zealand Norway Pakistan Peru Philippines Poland Portugal Puerto Rico Russia Saudi Arabia Singapore South Africa Spain Sweden Switzerland Taiwan Thailand Turkey Ukraine United Arab Emirates United Kingdom United States Uruguay Venezuela Vietnam March 2005 Process Maturity Profile - Page 15
Country Carnegie Mellon University Number of Appraisals and Maturity Levels Reported to the SEI by Country Number of Appraisals Maturity Level 1 Reported Maturity Level 2 Reported Maturity Level 3 Reported Maturity Level 4 Reported Maturity Level 5 Reported Country Number of Appraisals Argentina 10 or fewer Malaysia 10 or fewer Australia 36 Yes Yes Yes No Yes Mauritius 10 or fewer Maturity Level 1 Reported Maturity Level 2 Reported Maturity Level 3 Reported Maturity Level 4 Reported Austria 10 or fewer Mexico 25 Yes Yes Yes Yes Yes Barbados 10 or fewer Morocco 10 or fewer Belgium 12 Yes Yes Yes No No Netherlands 24 Yes Yes Yes No Yes Brazil 28 Yes Yes Yes Yes Yes New Zealand 10 or fewer Canada 79 Yes Yes Yes Yes Yes Norway 10 or fewer Chile 19 Yes Yes Yes No Yes Pakistan 10 or fewer China 243 Yes Yes Yes Yes Yes Peru 10 or fewer Colombia 10 or fewer Philippines 11 No Yes Yes No Yes Costa Rica 10 or fewer Poland 10 or fewer Cyprus 10 or fewer Portugal 10 or fewer Czech Republic 10 or fewer Puerto Rico 10 or fewer Denmark 10 or fewer Russia 10 or fewer Egypt 10 or fewer Saudi Arabia 10 or fewer Finland 10 or fewer Singapore 23 Yes Yes Yes No Yes France 142 Yes Yes Yes Yes Yes South Africa 10 or fewer Germany 62 Yes Yes Yes No No Spain 26 No Yes Yes No Yes Greece 10 or fewer Sweden 10 or fewer Hong Kong 10 or fewer Switzerland 10 or fewer Hungary 10 or fewer Taiwan 10 or fewer India 387 Yes Yes Yes Yes Yes Thailand 15 Yes Yes Yes No No Indonesia 10 or fewer Turkey 10 or fewer Ireland 10 or fewer Ukraine 10 or fewer United Arab Israel 30 Yes Yes Yes No No Emirates 10 or fewer Italy 33 Yes Yes Yes No Yes United Kingdom 139 Yes Yes Yes No No Japan 149 Yes Yes Yes Yes Yes United States 1947 Yes Yes Yes Yes Yes Korea, Republic of 75 Yes Yes Yes Yes Yes Uruguay 10 or fewer Latvia 10 or fewer Venezuela 10 or fewer Lithuania 10 or fewer Vietnam 10 or fewer Maturity Level 5 Reported The organization maturity profile, page 8, can be created for certain countries using the Interactive Maturity Profile available in the Software Engineering Information Repository Web site. Please visit: http://seir.sei.cmu.edu. If you are not already a member of the SEIR web site, please register to join. Registration and use of the site are free. March 2005 Process Maturity Profile - Page 16
Number of SW-CMM Appraisals Reported to the SEI by Year 600 500 Number of Appraisals 400 300 200 100 0 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 First Appraisal Reappraisal Based on 3,590 appraisals conducted through Dec 2004 and reported to the SEI by Jan 2005 March 2005 Process Maturity Profile - Page 17
Number of SW-CMM Appraisals Reported to the SEI by Organization Categories and Year 500 450 400 Number of Appraisals 350 300 250 200 150 100 50 0 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 Commercial/In-house DoD/Fed Contractor Military/Federal Based on 3,588 appraisals reporting organization categories March 2005 Process Maturity Profile - Page 18
Trends in the Community Maturity Profile 100% 90% % of Organizations 80% 70% 60% 50% 40% 30% 20% 80.3% 64.4% 37.3% 32.2% 27.5% 24.4% 12.1% 21.9% 33.9% 35.8% 37.1% 37.0% 12.0% 20.2% 20.9% 23.1% 25.5% Year 1987-1991 1992-1996 1997-2001 2002 2003 2004 Orgs 132 635 1,660 1,996 2,419 2,746 10% 0% 7.6% Initial Repeatable Defined Managed Optimizing 1.4% 4.5% 5.9% 5.9% 6.2% 0.3% 4.2% 5.2% 6.4% 6.8% Based on a cumulative view of the most recent appraisals of organizations up through the year indicated. This accounts for the difference from the figures on page 5. March 2005 Process Maturity Profile - Page 19
Section 3: Organizational Trends Appraisals conducted through the SEI by January 2005 1,839 Key Process Area (KPA) profiles and reported to - satisfaction of KPAs by maturity level for organizations appraised at levels 1 and 2 663 reappraised organizations December 2004 - accounting for 1,517 appraisals - although some organizations conducted multiple reappraisals, only the first and latest appraisals were used in creating the charts on pages 23 & 24 Please refer to: Terms Used in this Report on page 29 March 2005 Process Maturity Profile - Page 20
Key Process Area Profiles - 1 Organizations Appraised at Level 1 Repeatable RM SPP PTO SSM SQA SCM Defined OPF OPD TP ISM SPE Software Subcontract Management (SSM) is not applicable/not rated in many assessments. Please take that into account when interpreting its Fully Satisfied rating. IC Managed Optimizing PR QPM SQM DP TCM Fully Satisfied Rated PCM 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% % of Appraisals Based on 576 CBA IPI and SCAMPI with SW-CMM appraisals reporting a maturity level rating March 2005 Process Maturity Profile - Page 21
Key Process Area Profiles - 2 Organizations Appraised at Level 2 Defined OPF OPD TP ISM SPE IC PR Managed QPM SQM Optimizing DP TCM Fully Satisfied Rated PCM Based on 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 1263 % of Appraisals CBA IPI and SCAMPI With SW-CMM appraisals reporting a maturity level rating March 2005 Process Maturity Profile - Page 22
Maturity Level of First and Latest Appraisal 100% 90% 80% 70% % of Organizations 60% 50% 40% 30% 40.9% 39.7% 25.9% 41.5% 20% 7.2% 10.9% 7.8% 16.1% 0.8% 9.2% 10% 0% Initial Repeatable Defined Managed Optimizing First Latest Based on 663 reappraised organizations using their first and latest appraisal March 2005 Process Maturity Profile - Page 23
Reappraisals Change in Maturity Level Level 3 to 5 5.6% Level 4 to 5 7.1% Moved Down 2.7% Level 3 to 4 2.9% Level 2 to 5 2.1% Level 1 to 2 21.0% No Change 12.7% Level 1 to 1 5.3% Level 2 to 2 4.7% Level 3 to 3 1.7% Level 4 to 4 0.5% Level 5 to 5 0.6% Level 2 to 4 3.8% Level 2 to 3 27.6% Level 1 to 3 11.8% Level 1 to 5 0.8% Level 1 to 4 2.1% Based on 663 reappraised organizations using their first and latest appraisal March 2005 Process Maturity Profile - Page 24
Time to Move Up 100 Number of months to move to next maturity level 75 50 30 Recommended { time between appraisals 18 0 Time Period of Initial Appraisal 39 40 Pre-1992 20 25 23 19 20 1992 to Present 25 13 13 All (1987 to Present) Largest observed value that is not an outlier 75th Percentile Median 25th Percentile Smallest observed value that is not an outlier Level Orgs 1 to 2 2 to 3 25 12 1 to 2 2 to 3 3 to 4 4 to 5 170 255 60 62 1 to 2 2 to 3 3 to 4 4 to 5 195 267 62 62 March 2005 Process Maturity Profile - Page 25
Summary of Current Status Commercial and in-house organizations reporting most appraisals Of U.S. organizations, the services and manufacturing industries reported most appraisals Half of the organizations reporting size have 100 or fewer software personnel More Non-USA appraisals reported than U.S. in this 5 year snapshot March 2005 Process Maturity Profile - Page 26
Summary of Organizational Trends - 1 The total number appraisals for 2004 is less than 2003. It is the first time decrease since 1996 Software Subcontract Management is the least frequently satisfied level 2 KPA among organizations* appraised at level 1 Integrated Software Management is the least frequently satisfied level 3 KPA among organizations* appraised at level 2 Higher maturity has been reached among those organizations reporting reappraisals March 2005 Process Maturity Profile - Page 27
Summary of Organizational Trends (continued) - 2 For organizations that began their CMM-based SPI effort in 1992 or later, the median time to move from: maturity level 1 to 2 is 20 months maturity level 2 to 3 is 19 months maturity level 3 to 4 is 25 months maturity level 4 to 5 is 13 months Note: The median time to move from maturity level 1 to 2 dropped from 22 month to 20 month March 2005 Process Maturity Profile - Page 28
Terms Used in this Report Appraisal Company - The appraisal methods used in this report are the Software Process Assessment (SPA), CMM-Based Appraisal for both Internal Process Improvement (CBA IPI) and Standard CMMI Appraisal Method for Process Improvement (SCAMPI). We do request and receive other Software CMM-based appraisals such as Software Capability Evaluations (SCE) and Interim Profiles. As our sampling size of these other methods increase, they will be reported here. - Parent of the appraised entity A company can be a commercial or non-commercial firm, for-profit or not for-profit business, a research and development unit, a higher education unit, a government agency, or branch of service, etc. Organization - a.k.a. Appraised entity The organization unit to which the appraisal results apply. An appraised entity can be the entire company, a selected business unit, units supporting a particular product line or service, etc. Non-USA - Appraised entity whose geographic location is not within the United States. The parent of the appraised entity may or may not be based within the United States. March 2005 Process Maturity Profile - Page 29
Report your Appraisal Results to the SEI The briefing is only possible due to the cooperation of organizations and individuals sending in their appraisal results to the SEI In order to provide this information and service in the future, it will depend on this continued cooperation Please visit: http://www.sei.cmu.edu/sema/report.html for forms, information, and instructions on how to report appraisals to the SEI March 2005 Process Maturity Profile - Page 30
Contact Information Please visit: http://www.sei.cmu.edu/sema/profile_about.html and review the information provided before contacting: SEI Customer Relations (412) 268-5800 SEI FAX number (412) 268-5758 Internet Address customer-relations@sei.cmu.edu Mailing Address Customer Relations Carnegie Mellon University Pittsburgh, PA 15213-3890 March 2005 Process Maturity Profile - Page 31