t Professor of Operations Management and Operations Research at INSEAD, Boulevard de Constance, Fontainebleau Cedex, France.

Transcription

1 1995 SOFTWARE BEST PRACTICE QUESTIONNAIRE - ANALYSIS OF RESULTS by S. DUTTA* S. KULANDAISWAMY** and L. N. VAN WASSENHOVEt 96/53/TM Associate Professor of Information Systems at INSEAD, Boulevard de Constance, Fontainebleau Cedex, France. ** MBA Research Graduate at INSEAD, Boulevard de Constance, Fontainebleau Cedex, France. t Professor of Operations Management and Operations Research at INSEAD, Boulevard de Constance, Fontainebleau Cedex, France. A working paper in the INSEAD Working Paper Series is intended as a means whereby a faculty researcher's thoughts and findings may be communicated to interested readers. The paper should be considered preliminary in nature and may require revision. Printed at INSEAD, Fontainebleau, France.

2 1995 SOFTWARE BEST PRACTICE QUESTIONNAIRE Analysis of Results Soumitra Dutta Selvan Kulandaiswamy Luk Van Wassenhove* Research Initiative in Software Excellence Technology Management Area INSEAD Fontainebleau, France *Names of authors are mentioned in alphabetical order

3 TABLE OF CONTENTS INTRODUCTION 4 LEGEND 5 DEFINITIONS AND METHODOLOGY 6 SECTION 1- ORGANISATIONAL ISSUES 7 SECTION 2 - STANDARDS AND PROCEDURES 9 SECTION 3 - METRICS 11 SECTION 4 - CONTROL OF THE DEVELOPMENT PROCESS 13 SECTION 5 - TOOLS AND TECHNOLOGY 15 SECTOR ANALYSIS 17 SIZE ANALYSIS 18

4 EXECUTIVE SUMMARY This report presents an analysis of the results of the EC Software Best Practice Questionnaire enclosed with the project proposals submitted during the ESSI Call ending in March There was a strong response to the questionnaire and 463 valid responses were obtained from 17 different countries and 33 different sectors. The aim of the EC Best Practice Questionnaire was to collect data on software engineering and management best practices within European organisations. A "best practice" is defined as a management practice that is widely recognised as excellent and is recommended by most practitioners and experts in the field. The EC Best Practices questionnaire was designed such that each question addressed a particular best practice. The questionnaire had 42 questions divided into five sections: organisational issues, standards and procedures, metrics, control of the development process and tools and technology. Filling out the questionnaire that was enclosed as part of the Information Package was mandatory for project proposals to be accepted for the evaluation process. Based on the collected data, an analysis of best practices has been performed along various dimensions. In particular, the frequency of adoption of different best practices across countries has been studied. The variations in best practice adoption across sectors and company size (in terms of number of software related employees) have also been analysed. The analysis of the data has yielded many interesting results. At an aggregate level, about half of all responding companies have adopted most of the best practices defined in the questionnaire. For a large percentage of the best practices surveyed, the adoption rate was between 40% and 60%. Though the uniformity of adoption within individual organisations cannot be observed from the questionnaire, we can infer an average level of awareness of leading edge software management practices within European organisations. It should be noted that the structure of the questionnaire (only yes-no answers) and the fact that it accompanied Call for Proposals could have led respondents to portray a scenario more optimistic than reality. There are distinct areas where European companies exhibit either very high or very low adoption rates. For example, a large percentage of the responding companies manage their project staffing, planning and tracking proficiently. In contrast, only very few companies use formal estimation and quality assurance methods. The levels of best practice adoption across countries shows significant variation. For instance, in the context of the usage of tools and technology, the levels range from a high of 55% for the UK to a low of 25% for Sweden. The UK implements more best practices in the categories of organisational issues, standards and change control than any other European country while France follows the most number of best practices in the categories of metrics and use of tools. Some countries such as Greece have very low levels of software management best practices adoption overall. While these variations across countries are to be expected, it points to the need for a greater emphasis on the sharing of knowledge and the transfer of best practices across countries. Best practices also vary greatly across sectors. The Aircraft and Spacecraft sector comes out ahead of all other sectors. The Telecom Products and Electronic Engineering and related Technical Consultancy sectors come out in second and third positions respectively (ahead of Software Consultancy and Supply, Data Processing and Related Services). A comparative study of software management best practice adoption by companies with differing numbers of employees involved in software yielded interesting results. Large organisations adopt more best practices than smaller ones. We observed this phenomenon of higher adoption rates within larger companies uniformly across all best practices included in the questionnaire. This is probably indicative of the fact that larger companies are more likely to have implemented the best practice somewhere within their organisations (as mentioned earlier, the uniformity of the implementation cannot be deduced from the questionnaire). 3.

5 Introduction This report presents an analysis of the data received in response to the Software Best Practice Questionnaire enclosed with the project proposals submitted during the ESSI Call ending in March The questionnaire (see Appendix) is divided into two parts: (A) (B) General Information Section 1 - Organisational Issues (8 questions) Section 2 - Standards and Procedures (13 questions) Section 3 - Metrics (8 questions) Section 4 - Control of the Development Process (6 questions) Section 5 - Tools and Technology (7 questions) The questions in part (B) require a response of the form "Yes" or "No". For example, one of the questions in Section 1 is "Does each software project have a nominated software project manager?". Some questions also have a "Not Applicable" response category. Responses were received from companies in 33 sectors and 17 European countries. Of the responses received, 463 were found to be valid. The maximum number of responses were received from Italy, Spain and Germany. The chart below gives a distribution of responses by country. Number of valid responses received from each country g p4 4s 40.0 E I I I ).,. =..0 re 73' - 4S 0, 09 0 a 74 ' C r / ) i 4 i 16 8 MI "A' 4.

6 The maximum number of responses were received from the Software Consultancy and Supply, Telecom Products and Industrial Process Control Systems sectors. No responses were received from companies in 13 sectors. Seven sectors had only one responding company each.the chart below outlines the distribution of responses by sector. Number of valid responses received from each sector LEGEND Fl - Finance and Insurance EP Industrial Process Control IT - Telecom Products Systems EE - Electrical Engineering & ME - Mechanical Engineering AS - Aircraft & Spacecraft Technical Consultancy & Technical Consultancy IS - Software Consultancy and IE Electronic Engineering & EQ - Machinery, Electrical & Supply Technical Consultancy Optical instruments 5.

7 Definitions and Methodology We define a "best practice" as a management practice that is widely recognised as good and that is recommended by most practitioners and experts in the field. The structure of the questionnaire is such that each question addresses a specific best practice. We define the "best practice level" for a company as the percentage of best practices that the company follows. The analysis is conducted along several lines. Each section of the questionnaire is analysed individually and the following analyses are presented: an analysis that outlines the overall extent of adoption of the practices in the section, a country-wise analysis focusing on which countries have the highest practice levels and a presentation of the average practice levels for each country. The responses are then analysed on sector lines and on the basis of the size of their software organisation. Hungary, with a single response, has been omitted from country-wise analyses. Sectors in which only one response or no responses were received have been omitted from the sectorwise analysis. 6.

8 Section 1- Organisational Issues The first section of the questionnaire deals with software management issues pertaining to the organisation. The results reveal that about half of the companies have well established management practices relating to organisational issues., Most companies About half of the companies Appoint a project manager for each project Have a separate quality assurance function Have a change control function for each project Train all newly appointed managers incorporate customer input throughout the project Ensure availability of non-software resources Maintain awareness of the latest software engineering technology While 91% of the companies surveyed nominated project managers for each project, only 41% had a separate quality assurance function. Only 42% of the companies train their project managers to familiarise them with in-house practices. Customer input is ensured by 60% of the companies. This is surprising when we see it in the context of one of the results of the ESI Survey on Software Engineering Practices in Europe [ESI] which indicates that improving customer satisfaction is the most critical aspect for the organisation according to more than 85% of the software managers. Distribution by country and sector The table below outlines the countries and sectors with high and low adoption levels of various practices. Management practice Countries with Countries with Sectors with Sectors with high adoption low adoption ' high adoption low adoption Appointing project managers All countries ' All sectors for each project Software quality assurance function with independent reporting line France, UK, Finland Belgium, Spain, Greece Aerospace,, Telecom Mechanical Engineering, Electr. & Optical Establishing a change control function UK, Austria, Ireland Training new project Ireland, Finland managers Maintaining awareness of new Ireland, Finland, development technologies Netherlands Ensuring user input at all Switzerland, UK, stages of the project Finland France, Sweden, Greece Norway, Austria, Denmark Sweden, Belgium, Portugal Austria, Belgium, Spain Aerospace, ' Telecom Aerospace, Telecom, Finance Telecom All sectors, especially Mechanical Engg. Finance, Electrical & Optical, Process Control All other sectors All other sectors The UK, while having the best practices overall, has a surprisingly low 37% when it comes to training project managers and maintaining awareness of the state-of-the-art tools and techniques. Belgium and Greece, with 18% and 20% respectively, exhibit low adoption in establishing a separate quality assurance function. The Aerospace sector maintains high levels of best practices except in attempting to keep abreast of new technologies, where they show a low 27% adoption rate. Mechanical Engineering companies are especially efficient at ensuring user input at all stages with an 89% adoption. 7.

9 Software Best Practice Questionnaire - Anal ysis of Results Organisational issues - Best Practice levels for various countries and sectors Section 1 contains 8 questions, each of which addresses a best practice for organisational issues. We find the best practice level' by calculating the percentage of the best practices a company follows. To calculate the best practice level for a country, we find the average of the best practice levels for all the companies in the country. For example, if we wish to find the best practice for France, we first find the percentage of best practices each French company follows in Section 1 of the questionnaire. Then we average across all the French companies to find a best practice level for France. There is a wide variation in best practice levels between countries. To find the best practice level for a sector, we follow a similar approach whereby we average across a particular sector instead of a country. The following bar graphs give a ranking of the countries and sectors for best practices pertaining to organisational issues. Organisational issues - Best practice levels for various countries A - Austria E - Spain I - Italy P - Portugal B - Belgium F - France IRL - Ireland S - Sweden D - Germany FIN - Finland N - Norway SF - Switzerland DK - Denmark GR - Greece NL - Netherlands UK - United Kingdom Organisational issues - Best practice levels for various sectors Fl - Finance and Insurance EE - Electrical Engineering & Technical Consultancy IS - Software Consultancy and Supply IP - Industrial Process Control IT - Telecom Products Systems ME - Mechanical Engineering AS - Aircraft & Spacecraft & Technical Consultancy IE - Electronic Engineering & EQ - Machinery, Electrical & Technical Consultancy Optical instruments I See Introduction for a definition of best practice level 8.

10 Section 2 - Standards and Procedures The second section of the questionnaire deals with software management issues pertaining to the existence of standardised processes for estimation, coding, quality assurance etc. The results reveal that most companies have established project assessment and tracking methods. However, quality assurance procedures are Most companies About half of the companies Few companies Assess viability, risks and Ensure that subcontractors Have formal methods of benefits before committing follow formal processes estimating software size to a project Conduct inspection and Conduct periodic reviews walkthroughs at each stage of the status of projects Use formal methods to Have common coding estimate schedule and cost standards for projects Plan testing before coding Perform independent testing A high 76% of companies conduct risk assessment and 86% conduct periodic reviews of project status. In the arena of estimation, only 26% of the companies have formal methods for software size estimation. The number of companies using formal procedures for effort and schedule estimation is higher at 51%. About 48% of the companies perform independent testing by quality assurance and 51% of the companies plan testing before coding. This shows that, though 47% of European companies use no formal standards and only 5% of the software is developed according to formal quality specifications [ESI], at least half of the European companies do have some kind of quality process. i Distribution by country and sector The table below outlines the countries and sectors with hi gh and low ado p tion levels of various practices. Management practice Countries with Countries with Sectors with Sectors with high adoption low adoption high adoption low adoption Assessment of risk and viability of projects Finland, UK, Belgium Greece, Austria Switzerland, All sectors except Electr. & Optical Periodic review of status UK, Netherlands, Spain Sweden, Portugal All sectors except Electr. & Optical Walkthroughs and inspections of design and code UK, Switzerland, Finland Portugal, France, Spain, Belgium Aerospc, Telecom, Electronic Engg All other sectors Application of common coding UK, Switzerland, Belgium, Denmark All other sectors Electrical Engg standards Austria Formal procedure for All, especially All sectors estimation of size Formal procedure for estimation of effort, schedule and cost Finland, Austria, Switzerland Formal procedures for hand-off UK, France from one group to another Planning testing prior to coding Testing by an independent quality assurance group UK, Switzerland, Ireland UK, Ireland, France Sweden, Portugal Sweden, Belgium, Germany Finance, Telecom Belgium, Aerospace Denmark, Finland Portugal, Belgium, Aerospace Greece, Austria Spain, Belgium, Italy Electronic Engg, Telecom, Aerospace Electronic Engg, Electrical Engg. Electr & Optical Process Control Electr. & Optical, Electrical Engg, Mechanical Engg. Electrical & Optical, Electrical Engg, The UK exhibits a strong adoption of most practices relating to the usage of standards and procedures. 96% of UK companies assess the risk and viability of projects. 81% of UK companies apply common coding standards. The usage of formal procedures for the estimation of size is very low in European companies, 9.

11 especially in Swedish and Portuguese ones (0% of respondents from each of these two countries). The Electrical Engineering sector exhibits particular weaknesses in the implementing procedures and standards. It exhibits only a 20% adoption in the practice of common coding standards and walkthroughs and a 10% adoption in using formal procedures to estimate program size. All Aerospace companies surveyed followed formal methods of hand-off between groups and commenced test planning prior to coding giving them a 100% adoption in these areas. Standards and Procedures - Best Practice levels for various countries and sectors Using the definition of the best practice level found in the introduction, we calculate the best practice level for a company as the percentage of best practices that a company follows. We include only the questions in Section 2 for this analysis. To find the best practice level for a country, we average the best practice levels for all companies in that country. To find the best practice level for a sector, we average the best practice levels for all companies in that sector. The following bar graphs give rankings of the countries and sectors for best practices pertaining to standards and procedures. Standards and Procedures - Best practice levels for various countries A - Austria E - Spain I - Italy P - Portugal B - Belgium F - France IRL - Ireland S - Sweden D - Germany FIN - Finland N - Norway SF - Switzerland DK - Denmark GR - Greece NL - Netherlands UK - United Kingdom Standards and Procedures - Best practice levels for various sectors FI - Finance and Insurance EE - Electrical Engineering & Technical Consultancy IS - Software Consultancy and Supply IP - Industrial Process Control Systems ME - Mechanical Engineering & Technical Consultancy IE - Electronic Engineering & Technical Consultancy IT - Telecom Products AS - Aircraft & Spacecraft EQ - Machinery, Electrical & Optical instruments 10.

12 Section 3 - Metrics This section of the questionnaire addresses best practices pertaining to software metrics. We observe that the companies surveyed collect data about errors found, but do not analyse them for their causes. In general, tracking and performance metrics are used more widely than quality and estimation metrics. Most companies About half of the companies Few companies Log post-implementation Use version control methods Use size of software problems and track their Use actual versus estimated measures in their estimation resolution results of previous projects process while estimating new ones Perform a root cause Use "earned-value" project analysis of errors detected tracking Gather statistics of errors Compare actual against expected performance found in each stage of the project Weaknesses in the area of estimation using previous data are again observed. While 55% of the organisations use previous records for scheduling, only 21% of them use data from previous projects for size estimation. Only 30% of the companies perform a root cause analysis of the errors discovered. Distribution by country and sector The table below outlines the countries and sectors with high and low adoption levels of various practices. Management practice Countries with Countries with Sectors with Sectors with high adoption low adoption high adoption low adoption Record and feedback of estimated versus actual efforts Sweden, Netherlands, Greece, Ireland, Italy All sectors except Electronic Engg. into estimation process Belgium Record and feedback of size All countries Aerospace All other sectors into estimation process Analysis of sources of errors Finland All other countries All sectors Gathering statistics of testing All countries All sectors efficiency Comparison of estimated and France, Austria, Ireland, Aerospace, actual performance levels Sweden Netherlands, Spain Electronic Engg. Logging of problem reports All, especially UK All sectors except Austria, Portugal Finance Electrical & Optical Sweden and Portugal have a null adoption of size estimation and testing efficiency measurement. However, it comes out among the top three countries in performance estimation and comparison with actual levels and Portugal comes out among the top three countries in the logging and tracking of post-implementation software problems. The ESI survey [ESI] indicates that the penetration of quality programs like ISO 9001 is more than 50% in Germany, the UK and Italy but less than 10% in Spain and France. However, we observe that in quality procedures such as the analysis of errors and measuring testing efficiency, Germany exhibits very low levels (24% and 7% respectively). Besides, these levels are lower than those in France (34% and 14%, respectively). None of the Electrical and Optical and Electronic Engineering companies that we surveyed gathered statistics of testing efficiency giving them a 0% adoption rate. All Aerospace and Electronic Engineering logged their problem reports, giving them a 100% adoption rate. The Electrical & Optical sector showed weaknesses in other areas too, with only 8% of companies recording and feeding back size into the estimation process and 25% comparing estimated and actual performance levels. 11.

13 Metrics - Best Practice levels for various countries and sectors The calculation of best practice levels is much the same as in Sections 1 and 2. To find the best practice level for a country, we simply find the best practice levels for the companies in that country and average across all the companies. To find the best practice levels for a sector, we find the best practice levels for companies in the sector and then average across all companies. The following bar graphs give a ranking of the countries and sectors for best practices pertaining to Metrics. Metrics - Best practice levels for various countries A - Austria E - Spain I - Italy P - Portugal B - Belgium F - France IRL - Ireland S - Sweden D - Germany FIN - Finland N - Norway SF - Switzerland DK - Denmark GR - Greece NL - Netherlands UK - United Kingdom Metrics - Best practice levels for various sectors FI - Finance and Insurance EE - Electrical Engineering & Technical Consultancy IS - Software Consultancy and Supply 11:' - Industrial Process Control IT - Telecom Products Systems ME - Mechanical Engineering AS - Aircraft & Spacecraft & Technical Consultancy IE - Electronic Engineering & EQ - Machinery, Electrical & Technical Consultancy Optical instruments 12.

14 Software Best Practice Questionnaire - Anal ysis of Results Section 4 - Control of the Development Process This section of the questionnaire addresses best practices pertaining to procedures and mechanisms for change management. Most companies About half of the companies Few companies Assign the task of Gain sign-off from all parties estimating schedules for changes to managers who involved before revising project plans directly control the project Have procedures for controlling changes to code, requirements, design etc. Ensure that every function is tested Perform regression testing following each change About 55% of European companies gain sign-off from all parties involved before revising project plans and an equal number have established processes for controlling changes to code, requirements and designs. Only 29% perform regression testing routinely. However, other quality assurance processes are followed: 51% of the companies ensure that every function is tested. Distribution by country and sector The table below outlines the countries and sectors with high and low adoption levels of various practices. Management practice Countries with Countries with Sectors with Sectors with high adoption low adoption high adoption low adoption Obtaining signoff from all parties before changing plans Switzerland, UK Sweden, Portugal All sectors except Electr. & Optical Procedures for controlling changes to requirements, designs and documentation Procedures for controlling changes to code Regression testing following each change Ensuring testing of every procedure UK, Denmark, Austria, Portugal Netherlands, Austria, UK Ireland, UK, Greece Belgium, Sweden and Finance Aerospace, Electronic Engg, Mechanical Engg Finance Sweden Aerospace Finance All countries Aerospace All other sectors Netherlands, Finland, Belgium Aerospace Electrical & Optical, Finance Sweden exhibits particular weaknesses in the area of change management with only 25% of the companies getting signoff on changes to plans, a low 38% of companies with processes for controlling changes to code, requirements and specifications. Except in the area of regression testing, the UK exhibits very clear strengths: 85% of the companies have established methods for controlling changes to code, specification and requirements. Unlike in the other sections, Greece ranks third in the control of the software process. All aerospace companies surveyed have procedures for controlling changes to requirements, designs, documentation and to code. Their adoption rates in these areas are therefore 100%. The Electrical & Optical and Finance sectors show particularly low adoption in the domain of testing. Only 12% of Finance sector companies perform regression testing following every change, while no company (i.e. 0% adoption) in the Electrical & Optical sector performs regression testing. With respect to ensuring testing of every procedures the adoption rates are 17% for the Electrical & Optical sector and 29% for the Finance sector. 13.

15 Control of the development process - Best practice levels for various countries and sectors Section 4 has 6 questions which address the control of the development process. To determine the best practice level for a company in this section, we first determine the best practice levels for each company in this country. Then, we average across all companies in the country to obtain a best practice level for the country. Similarly, to find the best practice level for a particular sector, we average across all companies in the particular sector. The following bar graph gives a ranking of the countries for best practices pertaining to the control of the development process. Control of the development process - Best practice levels for various countries A - Austria E - Spain I - Italy P - Portugal B - Belgium F - France IRL - Ireland S - Sweden D - Germany FIN - Finland N - Norway SF - Switzerland DK - Denmark GR - Greece NL - Netherlands UK - United Kingdom Control of the development process - Best practice levels for various sectors Fl - Finance and Insurance EE - Electrical Engineering & Technical Consultancy IS - Software Consultancy and Supply IP - Industrial Process Control Systems ME - Mechanical Engineering & Technical Consultancy IE Electronic Engineering & Technical Consultancy IT - Telecom Products AS - Aircraft & Spacecraft EQ - Machinery, Electrical & Optical instruments 14.

16 Software Best Practice Ouestionnaire - Analysis of Results Section 5 - Tools and Technology This section of the questionnaire addresses best practices pertaining to the utilisation of tools and technology for management and development activities. Most companies About half of the companies Few companies Use software tools for Use design notations like Use software tools in project planning, estimating and scheduling SADT in product design Use prototyping methods for tracing code to design to requirements development Use automated testing Use a data dictionary for tools systems design and development Use software tools for tracking status of subroutines in the development library The usage of software tools is fairly high for planning and scheduling, but very low (26%) in forward and backward tracing of requirements through to designs and code. As in other sections, the use of automated tools in quality assurance related activities is low at 30%. The overall levels of usage of tools is fairly low when we view this in light of the fact that the ESI survey [EST] revealed that tools were considered by 92% of the managers as one of the most important requirements for improving productivity. Distribution by country and sector The table below outlines the countries and sectors with high and low adoption levels of various practices. Management practice Countries with Countries with Sectors with Sectors with high adoption low adoption high adoption low adoption Usage of tools for tracing forwards and backwards through requirements, design and code All countries, especially Finland, Ireland and Portugal All sectors Usage of design notation like SADT Usage of automated testing tools Finland, France, Denmark, UK, Austria All other countries, especially Sweden All countries, especially Switzerland Aerospace Aerospace Usage of prototyping methods Belgium, Ireland, Netherlands Sweden, Italy, Spain Electrical Engg Usage of software tools for All, especially All, especially project planning, estimation Belgium, Denmark Aerospace and and scheduling and Switzerland _Telecom Electrical & Optical, Finance Electr. & Optical, Mechanical Engg, Software consulting Electrical & Optical Swiss companies exhibit a very low 7% usage of automated testing tools. Finnish companies, while showing higher usage of design notations like SADT, have a surprising 0% usage of tools in tracing requirements to design and code. While Aerospace companies have a high 91% usage of design notation and automated testing tools, usage of these tools and techniques is only 8% in the Electrical & Optical sector. The highest usage (80% adoption) of prototyping is found in the Electrical Engineering sector. 15.

17 Software Best Practice Ouestionnaire - Analysis of Results Tools and Technology - Best practice levels for various countries and sectors As in the other sections, we calculate the best practice level for a company using the definition given in the introduction. Specifically, for this section, we calculate the percentage of the 7 best practices that a company follows. To find the best practice level for a country, we simply average across all the companies that belong to the country. To find the best practice levels for a sector, we similarly average across all the companies that belong to the sector. The following bar graphs give a ranking of the countries and sectors for best practices pertaining to the usage of tools and technology. Tools and Technology - Best practice levels for various countries A - Austria E - Spain I - Italy P - Portugal B - Belgium F - France IRL - Ireland S - Sweden D - Germany FIN - Finland N - Norway SF - Switzerland DK - Denmark GR - Greece NL - Netherlands UK - United Kingdom Tools and Technology - Best practice levels for various sectors Fl - Finance and Insurance EE - Electrical Engineering & Technical Consultancy IS - Software Consultancy and Supply IP - Industrial Process Control IT - Telecom Products Systems ME - Mechanical Engineering AS - Aircraft & Spacecraft & Technical Consultancy 1E - Electronic Engineering & EQ - Machinery, Electrical & Technical Consultancy Optical instruments 16.

18 Software Best Practice Ouestionnaire - Analysis of Results SECTOR ANALYSIS Until this point, we have performed a sector analysis for each section of the questionnaire. Here, we calculate the overall or aggregate best practice levels for various sectors. We first calculate the best practice level for each company considering all the questions in the five sections of the questionnaire. To calculate the best practice level for a particular sector, we find the average of the best practice levels for all the companies in the sector. For example, if we seek to find the best practice level for the Telecom Products sector, we first find the percentage of best practices each Telecom company follows in all sections of the questionnaire. Then we average across all the Telecom companies to find a best practice level for the Telecom Products sector. There is a wide variation in best practice levels between sectors. The following bar graph gives a ranking of the sectors for best practices. Quite predictably, the Aircraft and Spacecraft sector came out ahead of all the others with an adoption of 75% of the best practices. The Telecom Products and Electronic Engineering and related Technical Consultancy sectors come out in second and third positions respectively (ahead of Software Consultancy and Supply, Data Processing and Related Services). The Machinery, Electrical and Optical Instrument sector comes out last with a 36% adoption of best practices. Best practice levels for various sectors Legend FI - Finance and Insurance EE - Electrical Engineering & Technical Consultancy IS - Software Consultancy and Supply IP - Industrial Process Control IT - Telecom Products Systems ME - Mechanical Engineering AS - Aircraft & Spacecraft & Technical Consultancy IE - Electronic Engineering & EQ - Machinery, Electrical & Technical Consultancy Optical instruments 17.

19 SIZE ANALYSIS Distribution by size of software organisation (no. of software employees) To calculate the best practice level for companies of a particular size, we follow a method similar to the one done for the sector analysis. We find the average of the best practice levels for all the companies in the size range. For example, if we wish to find the best practice level for companies with less than 10 software employees, we first find the percentage of best practices each company with less than 10 software employees follows in all sections of the questionnaire. Then we average across all these companies to find a best practice level for companies with less than 10 employees. We found that the size of the software organisation influences best practice levels. Software organisations with less than 10 employees, on average, followed about 42% of the best practices, while a software organisation with more than 500 employees, on average, followed about 59% of the best practices. The interesting phenomenon is that we observe a monotonic increase in the average number of best practices with increasing size. Thus, it appears that software organisations with a larger number of employees tend to employ more best practices than software organisations with smaller numbers of employees. Best practice levels for companies with various numbers of software employees.0 80% - a? g 70% 60% 271 so% 41 40% a' 30% 20% ts. o 10% - 0 0% 42% 45% 47% 50% 58% 59% < > REFERENCES 1. [ESIJ "Software Engineering Practices in Europe", European Software Institute, Bilbao, Spain, December