Session 6.1. Software Engineering

Transcription

1 Session 6.1 Software Engineering

2 elements to software production: the enabler the software is developed upon, the software being developed, the team working on the software, the people managing the software development effort, and the environment where the development activities are carried out. The cause categories in the order of occurrence frequency were also derived from the research work findings: (1) management-related causes, (2) product-related causes, (3) technologyrelated causes, (4) organization-related causes, and (5) personnel-related causes. We found that managementrelated, personnel-related, and organization-related causes were more prevalent than technology-related and product-related ones. This is in agreement with Blackbum and Genuchten s findings [3], [4]. To date, few research works have been conducted to identify solutions or effective solutions to software schedule slippage. Some research works conducted industry surveys to identify the actions that are the most frequently adopted to overcome the schedule slippage problem [l], [13]. Some research works studied the approaches to improve the productivity [3], [5]. Another research work [12] studied approaches to reduce the cycle time. The common solutions used in these research works are: (1) using software development tools; (2) improving the software design for concurrency; (3) hiring efficient and experienced designers or developers; (4) training; (5) reducing or preventing employee tum-over; (6) improving the software development process; and (7) using software quality assurance. The majority of these solutions are able to improve the project schedule to certain extent. Due to the complex correlation among various project factors, such as the causes of the schedule slippage, the project status, and the development environment, a solution that is effective for one project may not be effective for another. In addition, some solutions may be difficult to be adopted by a project due to its specific project characteristics. Therefore, it is more practical and effective to selectively apply solutions to a delayed project. 3. Our Research The surveys conducted by the related work are becoming out of date since the majority of them were carried out in the early 90 s. In addition, very few of those research works discussed solutions to schedule slippage, or how to identify effective solutions for delayed projects. In order to obtain more current, sufficient, and precise information, and derive a consistent approach to identify effective solutions for delayed projects, this research conducted an industry survey between 1998 and Research Approach The goal of this research to facilitate software organizations to make educated decisions on identifying effective solutions for delayed projects. In order to reduce the response time and obtain sufficient data, we used a short and check-off-the-boxes type of questionnaire and consulted software project managers, software developers, software quality personnel, and survey designers on the questionnaire format, content, and structure. Two experienced software managers from Computer Integrated Manufacturing Development Group, Motorola Inc., one experienced software managers from Space Systems and Services Division, Motorola Inc., and one experienced software manager from Bull Inc. were interviewed. By experienced, we mean the software manager has been working on the same type of project in the same environment for more than four years. They made suggestions on the survey format, content and approach. These managers viewed a project from a high level and had an overall understanding of the project and the development environment. Their feedback better prepared the large-scaled survey phase. Software managers, project leaders, software developers, software quality personnel, and software testers were surveyed from AT&T, Bull, China Academy of Railway Science, Cvg-grp Inc., Foursquare Solutions Inc., Honeywell, Hughes, IBM, Informix, Intel, Lucent Technology, Mediaone Inc., Motorola Inc., NASA Software Group, Nortel, SAP, Sybase, and T4G. Software professional organizations surveyed include IEEE computer society chapters, ACM chapters, Software Project Management Network, Software Process Improvement Network, SEN, Arizona State University computer science graduate network, Boston Computer Society, Utah Computer Society, and British Computer Society. 3.2 Research Results Forty-eight responses have been received and analyzed. The projects surveyed were developed between 1992 and Most of them were developed in the client server environment or the distributed environment. The majority of the projects were developed with C/C++ as the computing language in a Software Engineering Institute (SEI) Capacity Maturity Model (CMM) level 2 or 3 organization using the waterfall model. Twenty out of 374

3 the forty-eight projects were delivered on time at the end Causes of Software Schedule Slippage and Solutions (Our Survey). Figure 1 plots the occurrence frequency of each causes in the surveyed projects. It is evident that the causes having occurred most frequently are: (1) incomplete or late requirements, (2) tools too late, (3) changing requirements, (4) inaccurate schedule estimation, (5) changing design or implementation, (6) insufficient people, (7) inexperienced people, (8) customer interface with higher complexity than expected, (9) lack of software development process, insufficient software quality assurance, (1 1) communication problems, and (12) weak leadership. This list of major causes is different from the list of causes derived from the related work. Some explanations are: (1) Our survey is more recent that the ones conducted by the related work; (2) Some project factors are becoming more influential on the schedule than before due to the fast evolving software industry; and (3) The projects are becoming larger and more complex. For example, twenty-eight out of the fortyeight surveyed projects had the duration longer than ten months; the average project duration was longer than one year; and late requirements or changing requirements occurred frequently in the surveyed projects due to the project complexity. As a result, this survey ranked late requirements and changing requirements higher than the related work. Another example is the communication problem. Thirty-six surveyed projects were developed in the distributed and clienvserver environment. Twenty-two surveyed projects were delayed because of the third party tools. As a result, the communication problem occurred frequently in the surveyed projects and this survey ranked the communication problem higher than the related work. Causes not discussed by the related work were also identified, such as using too many new tools, slow hardware, additional functions being added without user requests, lack of domain expertise, people turn-over, lack of subcontract management experiences, failure to involve the vendor, organizational infrastructure changes, and unstable development environment. The identification of these causes assists software organizations to study schedule slippage in more detail and better plan for and manage schedule risks. For example, a development team can work closely with its customers, conduct requirement reviews, and sign off requirement documents to minimize late or changing requirements and thus prevent schedule slippage.! m f B 2a 5 15 b? 10 2 b 0 Causes of software schedule slippage (our survey) Totalnumber of E t s delayedbqthe cazi LTech. I Product-related A L Personnel1 L Management-related 1 L Org. 1 Figure I. Causes of software schedule slippage (our survey) 375

4 - -~ ~ -~ Solubons to software schedule slippage (our survey) B 35 Total number of propcts using Ihe soluhon * P"----~--* q~x r i ;; g p 10 I5 - Figure 2. Solutions to software schedule slippage (our survey) Figure 2 plots the occurrence frequency of the solutions used in the surveyed projects. It is evident that the common solutions include: (1) working overtime; (2) code reuse; (3) test reuse; (4) reprioritizing tasks, (5) obtaining hardware with better performance; (6) postponing features to next revision; (7) maintaining software development or testing tools; (8) having appropriate amount of accurate and up-todate documents; and (9) using reviewslinspections. This list of common solutions is different from the list of the common solutions derived from the related work. For example, working overtime is lately becoming a prevalent approach to resolve schedule slippage. It is inexpensive and its impact on the schedule is immediate and evident. Code reuse and test reuse are becoming popular since object-oriented tools and methodologies have been invented in the past decade. Descoping the project and re-negotiating the schedule tend to be effective once agreed upon by the customer. Quality assurance and process improvement techniques are used too, but not as popular as working overtime or codeltest reuse since their effects on the schedule are not as immediate or evident. Some solutions mentioned by the related work were not used in the projects surveyed, such as specification reuse, reducing code complexity, reducing design complexity, allocating additional resources beforehand, increasing job responsibility, improving the work environment, and having appropriate team size. The reasons can be: (1) The related work identified both the solutions to prevent schedule slippage and the solutions to resolve schedule slippage. The solutions to prevent schedule slippage are project planning activities, while the solutions to resolve schedule slippage are project control activities. This survey identified the effective solutions to resolve schedule slippage, which are only project control activities. (2) Some solutions were not commonly used in the surveyed projects because they led to extra expenses and their results were not immediate or evident. For example, the solutions such as increasing job responsibility and improving the work environment were either expensive or their results were not immediate, thus they were rarely used in the surveyed projects. (3) Some solutions were not commonly used in the surveyed projects also because the cases where they could be used were rare. For example, it was 376

5 very rare to have reusable specifications and schedule slippage was normally detected after the requirement phase, thus the solution specification reuse was not commonly used as a solution to schedule slippage The Knowledge Base. In the questionnaire, each interviewee or surveyee was asked: (1) if each solution was used in the surveyed project: to overcome the delay and if it was effective; and (2) to list the reason why the team took or did not take the each solution. The answers to these two questions indicated the correlation between the causes, the project status, the development environment, and the solutions. The correlation was stored as rules in a knowledge base, as shown in Table 1. An example is explained as follows. Rule 3 has four preconditions. Only the precondition that the project has long duration is discussed as the other preconditions are intuitive. The rule shows that training shall be considered only when the project duration is long enough. Since it takes time for a developer to be trained and become productive, the delayed project will not be able to take advantage of the productivity increase with a short development cycle, especially if the project duration is shorter than the learning curve. The knowledge base provides an intuitive method for a software manager to identify effective solutions to bring a delayed project on schedule in a timely manner without having much management experience. If the project information matches the statement, the statement illustrates the solution that shall be applied to the delayed project. The term effective means the solutions derived from the knowledge base are likely to have positive impact on the schedule within the time frame allowed Knowledge Base Validation. The performance of a knowledge base can be assessed by comparing the knowledge base result with expert opinions [14]. This may not seem wholly satisfactory, but since the goal of this research developing the knowledge base is to transfer schedule slippage management expertise to a knowledge base available to less experienced software managers, the knowledge base s ability to emulate expert decisions is the only practical measure of its performance. Two well qualified and reputable individuals with good credentials were identified. Expert 1 is a software director from Raytheon with more than fifteen years experience in the software industry. He has supervised maximum of 450 software people and managed projects with budget of $15M. Most important, he has successfully delivered a number of software projects on time. Expert 2 is a software project leader from Air Force with more than twelve years experience in the software industry. He has supervised maximum of 15 software people and managed projects with budget of $1.3M. Neither expert was involved in the knowledge base formation process. Selecting experts not directly involved in the knowledge base formation process removes possible biases formed during the knowledge base development. Selecting multiple experts rather than a single expert adds more credibility to the validation. In order for the knowledge base to be fairly evaluated, the test scenarios for the knowledge base and the experts to examine need to be representative and comprehensive. The following steps were used to create representative and comprehensive test scenarios: (1) Studied all of the seventeen solutions in Figure 2 and grouped them into four subsets. These four solution subsets are natural groups. The solutions not contradicting to each other were grouped into one solution subset. For example, the solution increasing schedule pressure and the solution working overtime were grouped into subset 2 since the developers would feel schedule pressure if working overtime. (2) Selected one representative project from all of the surveyed projects. This research had the first hand information of the selected project and the characteristics of the project were given by the surveyee in sufficient detail. (3) Adjusted the characteristics of the selected project to create test project 1, 2, 3, and 4, such that the knowledge base would recommend solution subset 1 for project 1, solution subset 2 for project 2, solution subset 3 for project 3, and solution subset 4 for project 4, respectively. (4) Described the characteristics of project 1,2,3, and 4 in sufficient detail in order for the knowledge base and the experts to make deterministic decisions on the solutions. (5) Purposely elaborated the project characteristics to partially satisfy the preconditions of some rules. As a result, experts not paying much attention would likely make mistakes. This would increase the difficulty of the test and add credibility to the validation. The knowledge base performance is satisfactory in practical life. Out of the twenty-one and twenty-three solutions the knowledge base and the experts chose, respectively, twenty solutions were agreed upon. The major disagreement the experts had with the 377

6 knowledge base was whether to train the inexperienced manager or to replace him. One remark the experts made was that management training and management replacement were different degrees of reaction to solve the same problem. The experts preferred to use management replacement because it was considered more decisive and less risky. The knowledge base preferred to use management training because: (1) The manager with little management experience caused the project to be delayed; (2) The project had a long duration, thus there was sufficient time for the inexperienced manager to become trained and apply the new management skills to the delayed project; (3) The inexperienced manager would be given an opportunity to learn management skills and work more effectively next time. Table 1. Rules in the knowledge base Rules Rule 1 Rule 2 Rule 3 Rule 4 Rule 5 Rule 6 Rule 7 Rule 8 Rule 9 Rule 10 Rule 11 Rule 12 Rule 13 Rule 14 Rule Description (No enough schedule pressure) (Increase schedule pressure) (Budget available) AND (Insufficient experienced people) AND (Low communication overhead) AND (Before coding phase) AND (Project has a long duration) AND (Easy to find job applicants) (Hire good designeddevelopers) (Budget available) AND (People without required skills) AND (No training planned) AND (Project has long duration) (Use training) (Budget available) AND (Tools available in the market) AND (The use of tools not planned) AND (Tool training available) AND (Project has a long duration) (Use tools) (Developers able to work overtime) AND (Low overtime) (Work overtime) (Budget available) AND (Slow hardware) AND (Hardware caused the delay) (Get better hardware) (SEI level 2 or lower) AND (Constant customer change) (Have appropriate amount of documentation) (SEI level 2 or lower) AND (High people turn-over) (Have appropriate amount of documentation) (Customer makes constant change) AND (No review planned) AND (Project has long duration) AND (Before coding phase) (Use reviews with the customer, development team, Test team, QA) (Budget available) AND (Insufficient people) AND (Project has long duration) AND (Low comm. overhead) AND (Easy to find potential job applicants) (Hire more staff) (Budget available) AND (High people turn-over) AND (Easy to find job applicants) (Hire more staff beforehand) (Budget available) AND (Project has long duration) AND (Weak management) AND (Management caused the delay) (Management training) (High turn-over) AND (Able to control turn-over) AND (Insufficient people) (Reduce people turn-over) (High people turn-over) AND (Able to control turn-over) AND (People without required skills) (Reduce people turn-over) 378

7 Rule 15 Rule 16 Rule 17 Rule 18 Rule 19 Rule20 Rule 21 Table 1, cont. (Project has reusable code) AND (Code reuse not planned) (Reuse code) (Project has reusable test) AND (Test reuse not planned) (Reuse test) (Some functions can be postponed) AND (Customer agrees) AND (Before test phase) (Postpone the functionality) (Functions can be postponed) AND (Customer agrees) AND (Before test phase) (Reduce functionality) (Budget available) AND (Other team has margin in schedule) AND (Project has long duration) (Re-assign some tasks to another team) (Budget available) AND (Other team has margin in schedule) AND (Little training needed) (Re-assign some tasks to another team) (Budget available) AND (Organization has bonus program) (Money motivation) 4. Conclusion This research used an industry survey as a vehicle to identify the major causes of schedule slippage and the common solutions to resolve schedule slippage for today s software projects. In order to assist software organizations to selectively apply effective solutions to delayed projects, the research derived the correlation between the causes of the schedule slippage, the project status, the development environment, and the solutions from the survey responses. A knowledge References D. Phan, Information Systems Project Management, Ph.D. dissertation, University of Arizona, Tucson, A. Jenkins, et al., Empirical investigation of systems development practices and results, Info. Mgmt., vol. 7, pp , J. Blackburn and G. Scudder, Improving Speed and Productivity of Software Development, IEEE Trans. ons03. Eng., vol. 22, no. 12, pp , December M. Genuchten, Why is Software Late?, IEEE Trans. on Sofl. Eng., vol. 17, no. 6, pp , June K. Maxwell, et al., Software Development Productivity of European Space, Military, and Industrial Application, IEEE Trans. on SoJ. Eng., vol. 22, no. 10, pp , October R. Glass, Software Runaways, Prentice Hall, New Jersey, A. Cole, Runaway Projects -- Cause and Effects, Software World (UK), vol. 26, no.3, KPMG, base has been built to store the correlation and facilitate software managers to make responsive decisions that have positive impact on the schedule. The performance of the knowledge base was evaluated by industry software experts. Compared to empirical researches, which attempted to provide approaches during both project planning and through project control activities, our approach focuses on real time, dynamic project control activities, which we believe is more cost effective and efficient. R. Banker, et al., A Model to Evaluate Variables Impacting the Productivity of Software Maintenance Projects, Mgmt. Sci., vol. 37, no. 1, pp. 1-18, January D. Card, et al., Evaluating Software Engineering Technologies, IEEE Trans. on SOB. Eng., vol. 13, no. 7, July B. Kitchenham, Empirical Studies of Assumptions that Underlie Software Cost-Estimation Models, Info. andsop. Tech., vol. 34, no. 4, April H. Thambain and D. Wilemon, Criteria for Controlling Projects According to Plan, Project Mgmt., pp , June J. Tvedt, An Extensible Model for Evaluating the Impact of Process Improvements on Software Development Cycle Time, Ph.D. dissertation, Arizona State University, Tempe, T. Abdel-Hamid and S. Madnick, Lessons Learned from Modeling the Dynamics of Software Development, ACM, vol. 32, pp , December A. Kidd, Knowledge Acquisition for Expert Systems, Plenum Press, New York,