A Study of Japanese Software Process Practices and a Potential for Improvement Using SOFL

Transcription

1 A Study of Japanese Software Process Practices and a Potential for Improvement Using SOFL Sirin Bekbay and Shaoying Liu Department of Computer Science Faculty of Computer and Information Sciences Hosei University, Tokyo, Japan {sirin, sliu}@k.hosei.ac.jp Abstract The goal of this paper is to examine the Japanese experience with the software development process, the challenges they face and how formal engineering methods, in particular SOFL (Structured Object-oriented Formal Language), can help overcome these problems. We will also recommend additional management tools and documents that can aid organizations in achieving a higher CMM rating through the use of SOFL. Keywords: software process, quality assurance, CMM, software development methodology 1 Introduction Software quality is becoming increasingly important as our society becomes ever more dependent on computers. The required levels of quality are unlikely achieved without giving proper attention to the software development process. According to the Software Engineering Institute (SEI), the software process is the set of activities, methods, and practices that guide people in the production of software. An effective process must consider the relationships of the required tasks, the tools and methods, and the developers skills, training and motivation. SEI has setup and is a strong advocate of the Capability Maturity Model (CMM). CMM is used to evaluate an organization s software process maturity and has become the world s de-facto standard for software process assessment and improvement [13]. While software engineering research, education, and This work is supported in part by the Ministry of Education, Culture, Sports, Science, and Technology of Japan under Grantin-Aid for Scientific Research on Priority Areas (No ) and a Visiting Fellowship of Hosei University. practice in America and Europe remain strong, they are a relative weak area in Japan, compared with its strong electronic and automobile industry [7]. Most produced software is application software in Japan, and recent accidents in Banking [1], Air Control [3], andairticketingsystemshaveraisedaseriousquestion on how the Japanese software industry manages the quality of its software processes and final products. In order to answer this question, we have conducted a study of the existing reports on Japanese software process practices and interviewed several companies to find out the state of practice in Japan. Our survey result shows that while most companies emphasize the importance of software process, the levels of enforcing concrete measures to ensure the quality of processes and the final products vary greatly. Large companies tend to put more effort on the software process quality than small ones. We have also found through our survey that the most frequently cited problem with software development lies in the area of requirements management, including capturing, understanding, verifying, and validating the requirements. It seems that whether accurate and complete requirements can be captured is a key factor to determining the success of software projects. We believe that formal engineering methods, such as SOFL (Structured Object-oriented Formal Language) [12][11], can help improve the situation in industry, based on our past experience of applying SOFL to model and develop some information systems. SOFL provides a comprehensible notation and effective method resulting from integration of data flow diagrams (DFD), Petri nets, VDM-SL (Vienna Development Method Specification Language), and Object- Oriented programming paradigm for constructing both user requirements specifications and designs. It also provides a rigorous software process in which several

2 techniques are employed to ensure the quality of software development. Both evolution and refinement are used as the transformation approach between each phase; a three-tiered approach is used to create informal, semi-formal and formal specifications; and rigorous review and testing are adopted to verify and validate specifications, designs, and programs. In this paper, a theoretical analysis is conducted to determine the CMM level that organization is able to attain by following the SOFL engineering method. Since SOFL does not address the organization-related management activities required to achieve a high CMM rating, a proposal on how to improve SOFL to satisfy theserequirementsisalsosuggested. The remainder of the paper is structured as follows. Section 2 discusses the importance of having a software process in place to facilitate the improvement of the quality of software. Section 3 provides a theoretical analysis of SOFL against CMM standards. Section 4 offers the framework for the documentation that is required to help organizations using SOFL reach a higher CMM level. Finally, section 5 summarizes the findings and suggests additional areas of study that can be elaborated in the future. 2 Study of Japanese Software Industry In this section, we examine the Japanese software industry s quality and process issues by analyzing the historical data reported in literature and compiling the results of our findings made during our interviews with several companies in Japan. In recent history, when it comes to manufacturing goods, Japanese products are synonymous with quality and reliability. The reason behind their success lies in the kaizen philosophy. It represents all activities required to improve the quality of an organization and impacts all business areas. This culture of continuousimprovementhasbeeninstilledinmanyjapanese organizations since the early 1950s [8]. As with Deming s philosophy, quality involves everyone, at all levels in the organization. In order to effectively implement the kaizen way of thinking, training, operating methodologies and company-wide participation are necessary. This kaizen-type approach, although initially applied to the manufacturing environment, is also applicable in the software environment. When it comes to developing software, Japanese organizations maintain their strong belief on quality [15]. In particular, the Japanese software buyers will tend towards purchasing a product that is bug free as opposed to one that has greater functionality [4]. Furthermore, unlike in North America where organizations Billion Yen Year Figure 1. IT Services Market in Japan rush to deliver new products to the market and deal with any defects via maintenance releases, in Japan, this approach is not advised. To satisfy the quality features in software, a software process is required. A software process is a set of integrated development and management activities implemented by an organization for software development. All things considered, quality and process are tightly correlated. The process concept is one of the key elements in the Japanese software industry as more and more organizations are implementing software process practices [10]. The Japanese IT industry has been in expansion mode, as shown in Figure 1. Figures released by the Ministry of Economy, Trade and Industry (METI) indicate that the total annual sales in this sector in 2001 were 13.6 trillion yen, an increase of 17.4% from the previous year. This marks the seventh consecutive year of steady rise [9]. Furthermore, out of the five major business categories, customized software was the predominant leader capturing 49.6% of annual sales. With the growth in this industry, there is ever the more reason to place special importance on the development processes. Many Japanese organizations use a software development process analogous to those of large American companies with some variations. The majority of these variations are related to people management due to cultural differences. An organization s process must take into consideration the technical and non-technical factors of software development. Although the nontechnical issues associated with people management are an important facet and should not be overlooked, they are not in-line with the objectives of this paper and hence will not be further analyzed. Our findings gained through interviews with several companies in Japan have shown a rather consistent conclusion with those available in literature: industry emphasizes the importance of software process and quality. However, we have also found that efforts

3 made and measures taken by large and small companies are quite different. Below we describe the key findings from our interviews categorized into several subsections. The details of the interview result are given in Appendix A. 2.1 Process A general observation noted in all visited organizations is the presence and usage of a software development model. Ad-hoc software development is not the norm. Models, ranging from waterfall to spiral to rapid prototyping, are used during development. Some, where off-the-shelf products do not meet the organization s criteria, internally develop tools for software development process and management. Some organizations document their methodology in company handbooks where clear definition of the phases along with detailed process steps and actions required at the quality level are noted. Reviews of their process and quality standards are conducted in a regular fashion. This recurring examination is what promotes continuous process improvement. 2.2 Phases The generic phases in the development process are the Requirement Gathering and Analysis, Design, Implementation, Testing, Delivery and Maintenance. While every model can be decomposed into various phases, there is a split between what is considered as being the more crucial phase. Some believe requirements gathering while others view the design phase as the most important. Others believe the testing phase is most significant since rigorous testing is imperative prior to delivering high-quality applications. People tend to notice faults more frequently than some required features. A case in point occurred with the Mizuho bank merger in It went on-live operations following the biggest bank merger, and the final system did not work. The major cause of this failure was related to the fact that thorough testing was not performed [1]. In tracing back the events, a design of the new system was not made. Instead due to scheduling issues, patchwork was done and no system testing conducted. This represents two major phases in the software development process that were not carried out. This event damaged the reputation of this so-called Mega Bank of Japan. There is a general consensus on the most challenging phase. Organizations claim that requirements capturing and management is the most difficult task. It is difficult to agree upon when a requirement set has been fully defined. One approach used consists in capturing, organizing and recording requirements following interview and brainstorming sessions. Requirements capturing and verification must be done in an efficient fashion to ensure completeness. It is also important to verify and validate early in the development cycle. Beingabletoprovideformalverificationtodetermine whether or not all requirements were correctly understood and designed would be greatly beneficial. One organization used such formalization to create a requirement model and specify the requirements formally. They found that although they spent more time in the analysis phase, the overall time spent on testing and post-release support was less comparable to other similar projects. Measurements taken by the organization throughout the development process showed an overall improvement in the quality of the final product. 2.3 Training Training of employees is an important item. This relates back to the kaizen philosophy where if the employee has some basic quality improvement knowledge, the ensuing work will be that much better. Almost all visited companies not only train their new employees on the organization s policies, but also on their software development processes. Although the training timeframe has decreased over the years, it is still prevalent since it is part of Japan s lifelong educational system [15]. Re-education is also provided on a regular basis. 2.4 Management involvement The trend in the larger organizations is to aggressively pursue high quality standards activities. Unlike in the 1950s when upper managements priority was not focused on quality and minimizing any product deficiencies, but rather on inspection after production [8], today, some managers are directly involved in quality improvement activities, namely in establishing an effective software development process program. In one case, the head of the program is lead by the company s Managing Director. All managers are encouraged to participate in the program. This involvement further stimulates the acceleration of quality activities. 2.5 Organization structure Organization structure varies from companies where some have an independent group responsible to ensuring that all quality related activities are conducted. As for the actual development, the larger organizations have separate departments for design, inspection and test activities.

4 2.6 CMM The Capability Maturity Model (CMM) is the brainchild of the Software Engineering Institute of Carnegie Mellon University. It represents a 5-stage model, that helps in the evaluation of an organization s software process maturity [13]. CMM Level 1 (initial level ) does not have an established method of developing software, hence no pre-defined KPAs are available. Level 2(repeatable level) requires that the fundamental software principles are established and project procedures are created. Level 3 presents a defined level since the administrative processes along with all the criteria for developing and maintaining software throughout the organization are recorded. Level 4 is a managed level, requiring that quantitative quality and productivity targets be established and collected for the software process and product. An organization at Level 5 (optimizing level) continuously strives to improve its software process because it has the ability to recognize its weak points and resolve them in a proactive fashion. In order to remain competitive in this global economy, being awarded a quality certificate is advantageous. Japanese organizations have either already obtained ISO certification or are striving to attain a higher CMM level. A special committee has been setup to oversee and promote software process improvement activities in Japan [2]. CMM, a very generic methodology and originating from the U.S., has been officially translated into Japanese. The number of organizations receiving CMM appraisals has risen over the past few years [6]. There is a certain prestige associated with attaining a higher echelon. In the Japanese culture, perception is everything. In the working environment, great emphasis is placed on how an individual or organization is perceived in the eyes of others. This mentality is still pertinent in society. 3 Theoretical link between SOFL and CMM 3.1 SOFL and Level 2 practices SOFL addresses to a certain degree the requirements management KPA, which entails creating a familiar understanding between the customer and the requirements of the software project. The transition from informal to semi-formal specification is done (via evolution) with customer interaction in order to clarify any possible ambiguities. The further shift from semiformal to formal specifications via refinement clearly eliminates any uncertainties and formalizes the specifications. Each transition involves rigorous review activities to ensure conformity with the requirements. Although not explicitly stated, the requirements are noted in a formal document to ensure that none are forgotten. However, documents such as the software development plan are not supported in the SOFL methodology. When it comes to the software project planning KPA, the SOFL methodology does not cover the development of estimation on the work activities to be carried out, the required resources or the schedule. These activities represent management responsibilities. Nevertheless the SOFL model, with its various distinct phases, makes up part of the required activities. Software project tracking and oversight KPA consists in reviewing the project progress from different angles and monitoring the risks. This KPA does comprise of organizational requirements since there is the need to have policies in place to direct and control the software development activities. The following KPA software subcontract management is a less common industry requirement, hence is only applicable to organizations that do subcontracting. Aside from the selection criteria of the contractor, the goals of this KPA are similar to those of requirements management because they necessitate continual communication between both parties and the requirements on the tasks to be performed by the contractor must be noted. Organizational policies on contractor management are required at this level. There is a good association between SOFL and the software quality assurance KPA. High quality is addressed by the use of formalism. Rigorous reviews by different individuals and specification testing further query the integrity and quality of the software. However there are also organizational-level issues that need to be considered, namely rules for resolving the deviations found in the conducted activities. SOFL does not fully and explicitly address the software configuration management KPAs, which consists in ensuring that all work is baselined and any required changes are traced in a managed fashion. The foundation for having such capability is present but the management mechanism is not. 3.2 SOFL and Level 3 practices The organization process focus KPA is weak in the SOFL model. There is no indication on having a dedicated independent software engineering process group (SEPG) to coordinate process definition, improvement and deployment activities. The nonexistence of software process assessment or improvement activities along with organization-wide learning

5 processes are the main reason for this ill compliance. Only the use of software process assets is satisfied to some extent. On the other hand, SOFL deals with the organization process definition KPA. This is satisfied at a project level rather than at an organizational level. SOFL has its own development process. The documented process enables repeatability of the strong points and minimizes those common problems. However, centered upon software engineering processes, SOFL does not consider any organizational infrastructure problems, but its methodology can actually be partially regarded as the organization s standard process. Theobjectiveofthe training program key process area is to acquire skills that will allow the employees to carry out their tasks in a successful fashion. Without training, an organization may not be able to successfully apply the validation and verification steps required in the SOFL methodology. To satisfy this KPA, there are many textbooks on formal engineering methods available for public use, however a training plan should be created and maintained in an organizationwide policy. The integrated software management KPA is partially addressed by SOFL. It has as goal to tailor the organization s standard software process to meet the specific project s needs and coordinate the project with the rest of the organization. Given the process specified by SOFL, it is possible to tailor it to satisfy the needs and requirements of a specific project. The SOFL model satisfies the software product engineering KPA since it comprises of distinct phases and it uses a three-tiered approach to creating specifications. The requirements are analyzed, the design is created and the code implemented, and the testing is conducted at all levels. The SOFL methodology allowsforanefficientandeffectiveapproachtodealwith customer requirements and validate their accuracy in a rigorous fashion. The intergroup coordination KPA emphasizes management level activities which are not directly applicable in the SOFL methodology. In order to comply with this KPA, the coordination aspect is to be taken into account during the project planning where all affected individuals roles and responsibilities are to be noted in the project plan. However, the additional sub-activities involved in this KPA are satisfied by the SOFL methodology since they consist of activities found in the Requirements Management and Software Product engineering KPAs. This KPA is also about communicating with the customer, documenting commitments and resolving conflicts. These are achieved with the SOFL process where verification and validation ensure that the requirements are properly implemented. Peer reviews KPAs entail methodical inspections. Rigorous reviews of the specifications and the testing of each are conducted in the SOFL environment, although there is no explicit indication that it should be done with different individuals. 3.3 SOFL and Level 4 practices The quantitative process management KPA focuses on the statistical aspect of process control by establishing strategic objectives for the quality of the product. SOFL does not deal with this facet in a complete fashion since it is up-to the organization to setup policies and measurement programs that determine the acceptable baselines. The software quality management KPA centers on the quantitative measures of quality for the software product. Some of the activities include describing quality goals and setting up plans to meet the quality goals. A software measurement database and a repository with software process related documentation are not part of the SOFL methodology. Quality is implemented in the SOFL methodology by the use of verification and validation. However more formal quantitative activities is not part of its method. 3.4 SOFL and Level 5 practices The defect prevention KPA s purpose is to detect the source of defects and find ways to avoid any reoccurrence of these types of faults. The continuous verification and validation of the specifications during the early stages of the development process along with rigorous testing methods in the testing phase are just a few of the defect-preventive items supported by the SOFL methodology. Continuously going through refinement stages until non-deterministic specifications are eliminated is another attribute potentially covering this key process area. Although not tackled in a statistical fashion by means of causal analysis, these can be interpreted as partially covering this KPA. However, the organizational level activities are not supported by the SOFL language and method, in particular the training in defect prevention approaches and the formation of a team to manage the defect management activities. The objective of technology change management is to present new technologies via tools or methods into the organization in a methodical fashion with goal to improve the quality of the software and increase

6 productivity. The SOFL methodology itself can be considered as a new technology because of its formalization approach, which may be a facet not present in the organization s current development process. Organizational activities, including the formation of a group responsible to evaluate the new technologies along with their piloting efforts, are not covered in the SOFL methodology Process change management KPA is not supported in the SOFL since it deals mainly with organization issues namely setting up process improvement goals and incentive programs aimed at encouraging employees to partake in process improvement activities. 4 Framework for organization-related documentation for SOFL The approach used to calculate the overall maturity level of a process as stated by SEI is fairly rigorous. To satisfy a level, all goals of the key process area at one level along with all from the lower level must be fully met. Due to the inability to comply with the organization-level KPAs, the SOFL methodology as such is rated at level 2 even-though it satisfies a number of KPAs at levels 3 and 5. The organization and resource management sub-group of CMM is an important area that should not be overlooked. A key element of the organization management activities consists in the creation of and compliance to organization-wide written policies. This section proposes an initial framework for the documentation that will help strengthen the project management aspect based on the SOFL method and language. The following information is compiled from the suggested actions found in the [14] document. The framework includes the required documents and activities necessary to conform to CMM. It is recommended to have detailed information on documents such as project plans, training plans, risk management plans, configuration plans, technology change management plans, tailoring guides and subcontractor selection guides. Furthermore, additional activities must be included, namely procedures on tracking the progress of the project and process, software process assessment abilities, defect prevention management, organization-wide software process management activities, metrics collection abilities and subsequent measurement and process databases. In this section, a brief description on some of the proposals is provided. Additional templates for the documents and activities can be found in [5]. Figure 2. Project Planning 4.1 Proposal A: Creation of a project development plan. With respect to the project planning and project tracking KPAs, even though the planning aspects of the software product such as estimates, deliverables and such are not precisely defined in SOFL, they are essential items that must be considered during the contract deliberation period. A proposed document with the required information and procedure is shown in the Figure 2. Due to spacing limitations, only the major headings are noted. The project organization section displays the various roles and responsibilities of each individual in the project. This information is applicable and essential in most CMM KPAs. The management process section indicates the various required plans throughout the project cycle. The project tracking and control point out the schedule and pertinent milestones. A risk management strategy explains how to deal with the risks related to the project. For example, having a system that generates daily reports will enable the project manager to make informed decisions and act quickly when unexpected problems arise. The technical plans show the various tools that will be used along with the project acceptance criteria. The template for this document can be incorporated in the SOFL toolkit that is currently under development. To greatly facilitate the task, this toolkit will guide the owner on how the document should be completed by providing brief descriptions under each section. 4.2 Proposal B: Collection of review data Review activities are required to ensure high quality products. A verification criteria in CMM involves

7 Figure 3. Review Form Template (partial) reviews of the artifacts by senior managers. In SOFL, the review methodology is covered as a result of the validation and verification activities, however, they do not necessitate any management involvement and do not include any defect traceability or quantitative measurement capabilities. In Figure 3, a review template has been created and its complete form is omitted for the sake of space. Review record should indicate some basic yet specific items. To capture quantitative concerns, the total number of defects, total review effort time and participants list is required. The above mentioned plans and templates can be included in the SOFL toolkit. This upgraded toolkit can support the management activities currently lacking in the SOFL method. Amongst its intended capabilities is the ability to manage the requirements, create a work breakdown structure and potentially generate daily reports. The toolkit will encompass the management related concepts into one manageable application and simplify the traceability aspect of the required documents. 5 Conclusions Quality is essential in software. More and more organizations are placing greater emphasis on quality issues. In order to develop quality software it is necessary to follow a pre-defined process. According to some Japanese organizations, ensuring requirements conformity can be considered as the most challenging activity in the development process. Formalization can greatly simplify the task. This paper shows that by using SOFL, requirements verification becomes a more structured task and its validation using rigorous proof creates a more reliable product. Furthermore from the theoretical analysis conducted in the previous sections, it can be concluded that SOFL provides a strong basis for implementing good quality software development projects and has the necessary quality practices suggested by CMM. SOFL and CMM complement each other. Both emphasize the importance of having a process, one that is documented and followed, and where quality assurance is an essential part of the software development process. However, improvements are required to the SOFL methodology to conform to the organization and resource management sub-group of CMM. The proposals introduced in the last section could be included during the organizational implementations of SOFL and hence would lead the way to making the SOFL methodology a more CMM-compliant process. Future work will focus on the tools support for the proposed management documentation framework, and application of SOFL and the framework to industrial software projects. Acknowledgements We would like to express our gratitude to Professor Akira Onoma for his kind help in introducing companies for the industry survey. We would also like to thank Fumiko Nagoya and Nobuhiko Kawasaki for their various helps in using facilities and joining many discussions for the improvement of our research. References [1] Anonymous, Mizuho Bank Suffers Rough Start Due to Computer Glitch, Foreign Press Center/Japan, April 19, [2] Anonymous, Japan Standards Association [3] Anonymous, Air control breakdown causes flight problems nationwide Japan Times, March 02, [4] Anonymous, Canadian Success Stories in the Japanese Software Market, Impact Consulting [5] S. Bekbay, Japanese Organizations Experience with TQM: a focus on the software development process with a potential for improvement using SOFL, Hosei University International Fund Foreign Scholars Fellowship Reports, Vol. 9 (to appear). [6] Carnegie Mellon University, Process Maturity Profile of the Software Community 2002 Mid-Year Update August 2002.

8 [7] Lorraine M. Duvall, A Study of Software Management: The State of Practice in the United States and Japan, Journal of Systems Software, Vol. 31, No. 2, November 1995, pp [8] James R. Evans, William M. Lindsay The Management and Control of Quality, 4th Edition, South-Western College Publishing [9] Japan Information Technology Services Industry Association website, [10] Inoue Katsuro, Aoyama Mikio A Status Report on Research and Development of Software Process in Japan, Software Process Newsletter No. 1, September [11] Shaoying Liu, Jeff Offutt, Chirs Ho-Stuart, Yong Sun, Mitsuru Ohba, SOFL: A Formal Engineering Methodology for Industrial Applications, IEEE Transactions on Software Engineering, Vol. 24, No. 1, January, 1998, pp [12] Shaoying Liu, Developing Quality Software Systems Using the SOFL Formal Engineering Method, Proceedings of 4th International Conference on Formal Engineering Methods (ICFEM2002), LNCS Springer-Verlag, Shanghai, China, October 21-25, [13] Mark C. Paulk, Bill Curtis, Mary Beth Chrissis, and Charles V. Weber, Capability Maturity Model, Version 1.1, IEEE Software, Vol. 10, No. 4, July 1993, pp [14] Mark C. Paulk, Charles V. Weber, Suzanne M. Garcia, Mary Beth Chrissis, Marilyn Bush, Key Practices of the Capability Maturity ModelSM, Version 1.1, Technical Report CMU/SEI-93-TR-025 ESC-TR , 93.reports/pdf/tr25.93.pdf Feb [15] Denji Tajima, Tomoo Matsubara, Inside the Japanese Software Industry, Computer, March 2000, pp Appendix A The industry survey results are summarized in Table 1. CLASSIFICATION OF FINDINGS General Information Organization Structure Process Model Internally Developed Process Tools Longest phase Important phase Challenging Phase Reviews Tracking of defects Training Management Involvement CMM assessment FINDINGS Medium to large scale organizations, local and international presence 67% have a specific process group or a dedicated process manager All use a specific development model: waterfall, spiral, rapid prototyping 50% develop their own software process tools 17% of which develop their own requirements gathering tool. At some point, all use external tools during the development 17% use some kind of formal methods for the specifications 17% point towards the requirements phase 33% indicated the analysis phase 50% mentioned the implementation phase 50% indicated requirement gathering & analysis 33% said the design phase 17% mentioned testing phase Requirements capturing and management All conduct reviews throughout the development cycle 83% track defects All provide project training 83% give process training 67% indicate that upper management is involved in the process activities 17% have been assessed at level 2 17% have been assessed at level 4 33% have been assessed at level 5 33% have not been assessed but are ISO compliant and are striving for levels 2 & 3 Table1.Industrysurveyresults