An Agile CMM. Erik Bos and Christ Vriens

Transcription

1 An Agile CMM Erik Bos and Christ Vriens Philips Research, Prof. Holstlaan 4 (WAA01), 5656 AA Eindhoven, The Netherlands {Erik.Bos,Christ.Vriens}@philips.com Abstract. This paper describes the process for developing software in a highly volatile environment. The process is based on extreme Programming (XP) and Scrum, to combine engineering practices with management directives based on the Capability Maturity Model (CMM) Level 2. In December 2003 our department, Software Engineering Services (SES), was successfully certified for CMM Level 2. The assessors were especially impressed with the transparent, easily accessible and uniform project information. We describe which XP and Scrum practices we apply in the different Key Process Areas (KPA) of CMM and which activities were added. Also, the result of the assessment and recommendations of the assessors are listed. 1 Introduction CMM [1] gives guidelines for developing quality software in the context of large projects and organizations. Following CMM involves documenting and reviewing; documenting requirements, decisions, meetings, risks, plans and effort spent on software development. XP and Scrum [2,3] focus on a less bureaucratic way of developing quality software by focusing on good engineering practices and a human centered process. At first glance these two approaches seem far apart. Our department has existed since mid 2000 and currently consists of 8 permanent staff members and 20 experienced software contractors. We develop software in small teams: two to six software engineers sometimes joined by researchers and led by a team leader. The team leader acts as an XP coach/scrummaster, making sure the defined way of working is followed. Due to the size and the self-organizing character of the teams, the team leader also contributes by developing software. The projects (both prototypes and products 1 ) are characterized by fixed time of delivery (e.g. an exhibition or a demonstration), short time frame (2-6 months), and vague, minimal and changing requirements. The projects need to quickly show whether ideas of researchers are technically and commercially feasible. If so, the results are transferred to the Philips product divisions to form the basis of a new product. This calls for quality code and adequate documentation for users of the software outside of the team. A combination of good engineering practices and good enough documentation would be ideal. 1 Example: the Audio Fingerprinting project delivered a solution for music recognition combined with related music content delivery for mobile phones and was developed using our Agile process. See C. Zannier et al. (Eds.): XP/Agile Universe 2004, LNCS 3134, pp , Springer-Verlag Berlin Heidelberg 2004

2 130 Erik Bos and Christ Vriens Our approach was to take the best of CMM and the agile methodologies and combine them. We strived for CMM certification to benchmark our Software Engineering Services (SES) department and get recognition of the rest of Philips as a professional software development organization, which makes it easier to transfer our deliverables to the product divisions. An independent CMM assessment validated the software development process in December Several references were found comparing and combining XP and CMM [4,5,6], but no references were found of actual assessments being performed on departments running Agile projects. In the following sections the KPAs 2 are listed and we show which agile practices we ve incorporated in which KPAs. We also describe the results of the assessment and the recommendations of the assessors. 2 Requirements Management In SES, software is developed in iterations of a calendar month. For shorter projects (e.g. 3 month projects) or projects where the requirements can change within a month, an iteration length of two weeks is used. At the beginning of the iterations, an iteration meeting takes place with the whole team of developers and the (onsite) customer, usually a researcher. The customer describes the requirements. We enter them in a user story database. We developed this database to allow easy access to the user stories and to have an overview of which user stories are scheduled for which iteration (see figure 1). Next, the team and the customer discuss Problem Reports (PR) and Change Requests (CR), which were entered in the bug database during system testing. The customer determines which PRs and CRs need to be solved in this iteration. They are then estimated and entered as user stories. At the end of the meeting the user stories are prioritized and assigned to an iteration. The number of user stories assigned depends on the velocity 3 in the previous iteration and on the availability of the developers in the coming iteration. This makes the amount of work, which can be delivered in the iteration, repeatable (CMM Level 2: The Repeatable Level). At the end of the iteration, the user stories are demonstrated to the customer and are either accepted or rejected. 3 Project Planning The user stories are split-up into tasks, which are estimated by the whole team. The planning game is comparable with the Delphi method [3], which is referenced in [1, page 78] as a suitable way to determine estimates. 2 For a full description of the KPAs see reference [1]. 3 Velocity is defined as: the amount of work finished in the last completed iteration. See [2] for a more elaborate explanation of its usage.

3 An Agile CMM 131 Fig. 1. Part of Web page showing the user stories and tasks for a particular iteration together with their initial and current estimations and currently spent effort Usually there are just enough user stories provided by the customer to fill two iterations, so planning ahead for more than two months is not necessary. Some of the projects require a release planning, which is made by spreading the user stories over the next iterations, based on the velocity of the last, completed iteration and the availability of the developers. The release plan is extracted from the iteration deliverables. These deliverables are written down in the Project Management Plan (PMP), based on our only document template containing the documented procedure the CMM requires. The PMP is agreed upon by all involved persons and stored in the project archive. 4 Project Tracking Every day the software engineers update the task(s) they worked on by entering the spent hours and a new estimation for the task. So the project planning is accurate on a day-to-day basis. Remaining tasks are discussed in the stand-up or Scrum meeting every morning. XP and Scrum only care about the new estimates for the tasks at hand. In the CMM this is also valued, since the engineers themselves are responsible for their estimations and may update them. Project Tracking in CMM also asks for keeping track of the hours spent, so we can see if our estimates were correct and learn from the historical data we build up this way.

4 132 Erik Bos and Christ Vriens From the data entered we draw an inverted burn-down chart, shown in figure 2. Fig. 2. Burn-down chart showing amount of planned work (in days), unallocated time and the amount of work feasible to deliver in the 4-week iterations The horizontal axis at the top of the chart shows the month and starting day of the weeks. At the bottom of the chart the week numbers are listed. The iterations in this project are 4 weeks long. The chart shows for each day the remaining planned effort in the corresponding iteration in ideal engineering days. The following situations can be distinguished (all present in the chart in figure 2): The planned effort equals the possible effort as calculated from the velocity of the previous iteration, which is indicated by columns that are complete horizontally hatched Less effort is planned than calculated from the velocity; this difference is indicated by the vertically hatched part at the end of the column More effort is planned than calculated; this difference is indicated by the black areas at the end of the column The day on which the chart is drawn is indicated by a one-day wide bar from the top to the bottom of the chart (here between week 16 and 17). In the remainder of the current iteration (week 17 and 18), the black area indicates the amount of work to be completed on each particular day in order to be able to burn-down the planned effort. In the future iteration ranging from week 18 to 22, no work is planned in yet, This is indicated because all columns are vertically hatched and their height indicates the amount of effort that could be planned based on the velocity of the last completed iteration. Everybody interested in the progress of the project (customers, senior management and the engineers themselves) have access to the project information (as required by the CMM).

5 An Agile CMM Configuration Management Most of our projects use a CVS archive to store the source code and documentation. Bigger projects have an automatic, continuous build and test script running. Next to that, the projects have a stress test exercising the application(s) to trap e.g. multithreading issues in the code. Although we assign a configuration manager, a consistent archive is the team s responsibility. When the build fails, for any reason, it s a first priority to get it fixed again. This alleviates the burden normally put on the configuration manager: the software developers integrate only code in the archive that doesn t crash the system. Documents (e.g. overview documents or API descriptions) have a promotion status (draft, proposal, accepted) and are also stored in the project archive. The task of the configuration manager is to tag the archive at the end of the iteration and describe, in a change log or in the iteration meeting minutes, which new functionality is contained in the tag. 6 Quality Assurance CMM defines that Quality Assurance needs to be performed by an independent QA group. We implemented this by assigning the team leader of one project as the QA Officer for another project (lead by a colleague team leader), thereby guaranteeing independence of the QA officers. The QA group consists of all so-assigned QA officers. Every month, the QA officer assesses the project and checks if the agreed way of working (e.g. XP practices) is followed. His report is sent to the whole team, the customer and to senior management and is stored in the project archive. Senior management is involved when major non-compliances are not resolved in time and if escalation is necessary. The QA officer also facilitates the retrospective [7] on the projects. The results of the retrospective are stored in the project archive. Every month the team leader writes a status report. It lists the risks identified on the project and their mitigation. It also, briefly, lists the accomplishments and difficulties encountered on the project. This report is sent to the whole team, the customer and senior management and stored in the project archive. 7 Subcontract Management Since we don t subcontract projects, this KPA was neither implemented, nor assessed by the assessors. 8 Customer Satisfaction The CMM, like the Agile methods, highly values the satisfaction of the customer. To measure this we ask our customers to fill in an enquiry.

6 134 Erik Bos and Christ Vriens Fig. 3. Results of the latest Customer Satisfaction survey on a scale of 0 to 10 (higher is better) The results show that most customers are highly satisfied about our way of working. In one project the customer was not satisfied. His complaint was that he couldn t see the end of the project from progress forms and wanted detailed specifications and project schedules. After escalation to senior management, it was decided not to continue our Agile way of working on this project. This is exactly the way a problem like this should be handled according to the CMM. 9 Staff Satisfaction The CMM and the Agile methods require motivated people. To measure the motivation we have a survey filled in by SES staff to express their satisfaction about the way of working and other issues not related to the project. Large deviations are discussed with the staff members and usually result in actions to alleviate issues Training The CMM requires people to be trained. Most of the experienced software engineers have no experience working in an Agile manner. Most are trained in big processes and making large designs upfront. To get all software engineers on the same XP knowledge level, we regularly organize XP Immersions on-site. Customers are invited to join this course. This makes it very efficient to set up Agile teams. We described the way of working in a (very thin, 2 pages) Life Cycle Description. It briefly describes XP and Scrum and contains references to the standard Agile books. 4 One remark was that no bonding was felt by the contractors with the SES department, because most of the contacts are on a project base. This resulted in a monthly lunch meeting with a technical lecture.

7 An Agile CMM 135 Fig. 4. Results of the latest Staff Satisfaction survey on a scale of 0 to 10 (higher is better) 11 CMM Assessment Results In December 2003, a team of two Philips Certified CMM assessors evaluated SES. They conducted 21 interviews with managers, project leaders, customers and software engineers. The following strong points were identified: ƒ Good process deployment approach has led to a disciplined Way-of-Working. ƒ The web-site is a major asset in transparency of processes and projects, and deployment of the defined Way-of-Working. ƒ Staff expresses pride on the Way-of-Working. ƒ Effective guidance by SES of new customers in introducing the 'Way-of-Working'. ƒ Good process focus! The Way-of-Working is continuously being enhanced. Attention points are: ƒ The process policy is not explicitly described (e.g. do s and don ts because of XP). ƒ Reviews with overall project management are not applicable in most cases. Where it is applicable it is not done. ƒ Reviews by senior management have little attention for the effectiveness of the process execution. ƒ There are few measurements on the process currently in place. ƒ There is a great potential in the data already collected.

8 136 Erik Bos and Christ Vriens 11.1 Requirements Management Observation. Acceptance criteria are sometimes not explicitly defined. Good practice. Frequent communication with the customer about requirements and their implementation, resulting in very good mutual understanding of the requirements Software Project Planning Observations. Sizes of work products are not estimated. Good practice. Highly detailed Work Breakdown Structure (WBS). Unfortunately the estimates are not supported by use of historical data (which are largely available). Remarks. Management of critical computer resources is generally not applicable due to the nature of the projects. Project plan review by senior management could have more attention for feasibility of the results agreed Software Project Tracking and Oversight Observations. Sizes of work products are not tracked. Good practices. Use of burn-down charts. Daily recording of effort spent. High frequent interaction on progress. Remark. Except for effort tracking, it is difficult to see what the status of the project is (e.g. status-overview of deliverables) Software Quality Assurance Observations. The involvement of the QA officer in the preparation of the PMP can be improved (e.g. to share his experience). No structured independent check on the activities of the QA officers. Remark. The QA-role can have higher impact by more presence in the project (e.g. by visiting stand-up and iteration meetings) Software Configuration Management Observations. No standard CM reporting in place. Baseline audits introduced only recently. Checklist used is still draft. Good practices. Continuous builds using automated unit tests and logging of build and test results. Remark. Planning and definition of baselines could be improved Conclusions of the Assessment SES has defined and implemented an elegant way to reach the goals of Software CMM Level 2. The XP process with CMM flavor is adequate, given the nature and sizes of the projects.

9 An Agile CMM Recommendations Take your right to say no : don t work with customers who don t play their roles as required by the process. Upgrade PMP to ask more attention for identification and handling of dependencies and dealing with open source software. Define an explicit approach with respect to measurements: ƒ Choose, do, analyze, and modify according to gained insight ƒ Impressive collection of historical data is excellent basis to derive and use metrics Examine other (Philips) coding standards to upgrade the SES standards (addressing language-specific do s and don ts). Consider improving project status reporting (e.g., status of deliverables, earned-value charts). Behave like a learning organization: ƒ Analyze retrospectives over projects to derive trends at organizational level ƒ Classify historical data, and use historical data in estimating future projects ƒ Tailor processes based on good practices encountered in projects meeting specified criteria ƒ Transfer what has been learned in one project to other projects The score on the KPAs for CMM level 2 are depicted in the figure below. Fig. 5. Scoring of the Level 2 KPAs for the SES department. For CMM level 2 certification all KPAs must be scored higher than 80% 12 Conclusion This paper showed that, while working Agile, with a little extra effort, also a CMM Level 2 certification could be obtained. The CMM level 2 KPAs and an Agile way of

10 138 Erik Bos and Christ Vriens working overlap, because both focus on basic project management. The focus on people in Agile and the organizational focus of CMM are nicely complementary. It is shown that it s not necessary to define thick procedures or write piles of documentation to prove you are working on CMM Level 2. As Watts Humphrey states in the preface of [1]: you need to look at what people are actually doing. Acknowledgements Thanks to Hans Naus, Vincent Ronteltap, Wim van de Goor, Mike Cohn and Wilko van Asseldonk for reviewing this article. References 1. M.C. Paulk et.al.: The Capability maturity Model: Guidelines for Improving the Software Process, Addison-Wesley, Reading MA (1995). 2. K. Beck: Extreme Programming Explained: Embrace Change, Addison-Wesley, Boston (2000). 3. K. Schwaber, Mike Beedle: Agile Software Development with Scrum, Prentice Hall, Upper Saddle River, NJ (2002). 4. M.C. Paulk, Extreme Programming from a CMM Perspective, in IEEE Software, vol. 18, no. 6, pp (2001). 5. J.R. Nawrocki et.al.: Comparison of CMM Level 2 and extreme Programming, in Proceedings of ECSQ 2002, pp (2002). 6. F. Paulisch, A. Volker, Agility Based on a Mature Foundation, Proc. Software Engineering Process Group Conference SEPG 2002 (2002). 7. Norman L. Kerth: Project Retrospectives, Dorset House (2001).