Reaching CMMI Level 2: Challenges, Missteps, and Successes

Transcription

1 Reaching CMMI Level 2: Challenges, Missteps, and Successes Rich Cyran & James Cusick Wolters Kluwer, Corporate Legal Services 111 Eighth Avenue New York, NY {richard.cyran; ABSTRACT Developing a process that supports the requirements of the CMMI (Capability Maturity Model Integration) and meets business goals can be challenging and brings with it its own risks. At Wolters Kluwer (WK) Corporate Legal Services (CLS) a goal was set to develop, document, and deploy a process that could bring the organization to Level 2 on the CMMI framework and improve established processes at the same time. To do this required over two years and the contributions of dozens of participants. Along the way a variety of challenges were overcome, missteps made, and successes were achieved. This paper reviews the progress we have made in our process improvement initiative and the practical steps taken to achieve Level 2 in a commercial organization. KEY WORDS Software process, process engineering, software engineering, Capability Maturity Model, Capability Maturity Model Integration, project management, configuration management, software quality. 1. Introduction Adopting the Software Engineering Institute s (SEI) Capability Maturity Model Integration (CMMI) [1] requires preparation, planning, determination, and experience with process improvement. At Wolters Kluwer (WK) Corporate Legal Services (CLS) we carried out the development and formalization of a full life-cycle set of processes and deployed them in multiple organizations. This process definition and deployment followed the CMMI model and we recently achieved a Level 2 rating on the 5 level scale as provided by a SCAMPI (Standard CMMI Appraisal Method for Process Improvement) assessment [2]. Doing so required more than two years of effort by a diverse team that led us through many challenges that are documented here. In this paper we describe the starting point and goals of the organization and the specific steps taken to define and deploy the required process maturity to achieve Level 2. We also note mistakes made along the way and key success factors. 2. Getting Started Wolters Kluwer (WK) is a Netherlands based international publisher and information services provider with operations around the world. Wolters Kluwer is organized into Business Units which then control operating companies. The experience documented here focuses on global development teams managed from the New York based Corporate Legal Services Division which manages six corporations. The practices described are grounded on numerous projects performed primarily in the United States and India. The US locations include multiple Manhattan sites and facilities in Houston, Hartford, and San Francisco. The starting point for the organization more than two years ago was a set of software teams that were using a loosely documented set of practices with conflicting lifecycle models. There were project plans with few templates. There were some requirements documents commonly used but no defined approach to developing them. There were some general process steps defined in PowerPoint presentations inherited from departed consultants. There was a shared vision on how to run releases but this was largely communicated verbally. The organization was generally successful in estimating using a detailed approach based on historical data on effort related to required work products. The bottom line was that releases were being produced but for anyone entering the organization it was unclear how this was done in any defined manner. The CTO of the organization set a goal to improve process maturity and to utilize the CMMI in doing so. Since the CTO was sponsoring the initiative, there was strong management support and high visibility to jump start this initiative. These two factors are crucial in any process improvement initiative. The initiative was proposed to improve on cycle time and quality as well as providing better managerial control of projects. This effort coincided with the beefing up of the PMO (Project 1

2 Management Office) and was in fact managed from there. Being managed out of the PMO, meant that this initiative was managed as a project with a plan, schedule, weekly status reporting and other project deliverables. Being managed as a project would also set the path for the initiative. At the outset, there was limited understanding in the organization of what was involved to reach Level 2. A 12-month stretch target was established without detailed analysis on what was required to reach that level. Formation of a process group in the PMO became the driver for the initiative. A first step was the formation of the SEPG (Software Engineering Process Group) comprised of cross- functional representatives from the companies involved. The CTO, the sponsoring executive, nominated his direct reports who in some cases appointed delegates. The composition of the SEPG included representatives from the PMO, QA and Engineering groups This is turn brought with it its own set of complicating circumstances but nonetheless, the SEPG began meeting regularly. Once the team was organized we began focusing on the process development needs within the context of the CMMI framework. The initial work focused on the collection, analysis, and compression of the five diverse lifecycles in use throughout the varying organizations and major projects going on at the time in CLS. No two lifecycles in the three organizations were exactly alike. Different teams in the same organization used different methodologies. The 5 lifecycle models were reviewed, compared and then collapsed into one unified model with six phases: Concept, Analysis, Design, Construction, Test, and Implementation. Negotiating sessions were then held to map work activities to these newly agreed to phases. However, the proof of the design came when the lifecycle model was eventually deployed to get actual feedback on what made sense. There was extensive discussion around terminology and the conversion from existing models to the new standard model. The new standard was a modified waterfall lifecycle. The CMMI framework was reviewed, a gap analysis conducted, and the team created a project plan to start developing procedures under the framework. A variety of loosely defined procedures that needed concrete documentation already existed. In some cases we had to research best practices to develop new suggested procedures. These procedures were reviewed, published, and deployed as they became available. It was expected that the project teams would use the procedures as they were approved by the SEPG and posted to the Process Asset Library. 3. Challenges Getting the effort off the ground and keeping it moving forward posed numerous challenges. The first challenge was that processes were in fact followed within each separate organization but they were not always documented. This forced the SEPG to hunt for documentation, interview people on what they did, and author new documentation so as to generate the appropriate procedural artifacts. A second major challenge was resistance to change. The SEPG needed to engage people fully to overcome this, soliciting input relentlessly, and allowing people to feel as if they owned the new process. The resistance to change in such an initiative should never be underestimated. An additional challenge was the diverse organizations in different companies with different histories, different locations, and different cultures. These differences led to varying degrees of support and involvement as business goals varied and limited integration at the business level drove a separation at the technical level. This led to different priorities. Some groups preferred to be cowboys and some did not, and this required changing tactics and communications, which was not always successful. Often, process needed to take a back seat to projects, and as a result, one of our organizations needed to be de-scoped from the eventual SCAMPI, as they did not meet all the hurdles required. Strong personalities also inserted themselves into the effort, sometimes pushing things along and sometimes dragging down progress. A learning curve for many participants had to be climbed. The concepts around what a formal process looked like compared to what they were used to had to be worked through. In some cases people were receptive to taking on more process and in other cases they were not. Navigating through the issues raised the time required in order to reach consensus and commitment. A SPI (Software Process Improvement) project manager was brought in from the outside, which was both a negative and positive. The project manager did not know the company or the people and had to win relationships without an established internal track record. Further, the manager did not have local experience in problems faced by typical projects, came from a different organizational culture and had to adapt to local practices, diverse geography, and limited face time with people in remote locations. Though the SPI Manager was unfamiliar with CLS, he was familiar with the CMM and had managed a successful CMM Level 2 implementation at a major financial services company. Though the landscape was different, similar challenges existed. The reliance on participating over the phone makes it a challenge to communicate and limits ad hoc integration. Additionally, informal communications and a mechanism to share best practices did not exist. 2

3 A key factor in getting started was the relative small number of people experienced with CMM outside the core team. A few folks on the core team of the SEPG had strong backgrounds in process improvement and maturity in applying the CMM. Reaching out to the rest of the team and to the larger organizations required extensive education of the staff to familiarize them with the structure of process. We also had to develop a sense of process and process improvement among all the staff, which took an investment in training materials and training hours. There were also two organizational issues we had to overcome in getting started. First we had to overcome perceptions of earlier unsuccessful process initiatives there had been some less directed efforts towards process improvement that had not been brought to fruition. Also, organizational changes occurred which added to the challenges. Mid-way through the effort the outsourcing of the infrastructure group caused the need for the revision of multiple processes and process artifacts. Finally, we encountered numerous issues with specific CMMI process areas. Notably, Configuration Management proved to be a struggle to get to an approach that fit the needs of the organization. We made several attempts at getting this right and each step led to a misstep requiring us to rework the process documentation and training packages again and again. Measurement and Analysis was also a challenge in that limited metrics were in place and we needed to start almost from scratch in this area. With SAM (Supplier Agreement Management) there was no vendor management function. Generally, interpretation of the model can be subjective; using our consultant lead appraiser to help us navigate these waters was very beneficial. Our lead appraiser was able to provide suggestions that were both beneficial to our business and satisfied the CMMI. We ended up with a very pragmatic approach to SAM that is both straightforward and useful. All of these varied challenges were worked through by the team over a period of more than two years. Some of the attempts at solving these issues worked and others did not. In the next section we share some of the things that did not go as planned. 4. Missteps As with any project, our SPI initiative hit some snags. There were some small glitches and some major difficulties to overcome. Below we try to chronicle some of the challenges that stand out in hopes that others working on a CMMI process improvement initiative can benefit from the problems we encountered. Resistance to change proved to be a major issue and we made mistakes in underestimating the amount of effort necessary to overcome it. Secondly, getting people involved was difficult. We held a Q&A session, for example, and no one showed up. Only when we offered prizes did involvement increase. In the beginning there wasn t a feeling of ownership of the new process by the organization. People referred to the process as the SEPG s process not the organization s own process. We could have been more proactive in reaching out to the community in more ways early on. SEPG team flux and levels of dedication often caused delays. This was understandable since SEPG members were fully allocated to projects and the process improvement initiative was not their primary responsibility. People missed meetings since development projects took greater priority. This caused the level of participation to vary. Some personnel changes during the life of the effort had mixed impacts on progress, some positive and some negative. At times had trouble making commitments stick. Generally we were unable to hold people accountable to get things done as outside priorities superseded commitments. Finally, not all SEPG members were process evangelists but were more passive members. A more active drive towards getting commitments and forcing them to closure could have helped. Not all the right people were involved at all times. Sometimes an adversarial stance was taken between the SEPG and those outside the team, and this required rework in some cases. Getting processes reviewed and accepted (by the right people) required a long time and we did not always succeed in getting buy-in on some key procedures from all stakeholders in advance. Better inclusion of all parties could have helped prevent this from happening. SEPG members, already process champions, needed to be greater advocates and communicators of the process changes to their constituencies. However, in a large organization it is hard to keep everyone equally informed! We also underestimated the amount of time required to review and approve processes, procedures and other work products. Review sessions often overran their one hour time frames and required additional review sessions and multiple follow-on reviews. It was not known that members would get hung up on certain minutia. We did not have past history to look back on. We could have placed better rules around reviews including prep time and standard durations. In general, communications could have been improved. We published several newsletter issues, for example, but could have been more consistent in producing them to keep people informed. Aside from the newsletter we also made announcements and maintained a web site, but we found people were still not always up to speed on what we were doing. It 3

4 is possible that we could have also found other ways to keep people informed such as informational meetings, status reports, and by getting more of the organization involved in developing the process artifacts themselves. Not getting additional resource(s) earlier (e.g., process consultants) slowed down the effort; the lesson here is to ramp up faster. The process team could have done a better job in carrying out onsite visits to remote locations. Of course budgetary concerns always exist but visits may have prevented one site from dropping out of the assessment. There was also an inability to leverage work done in other WK business units. There were some CMM and CMMI initiatives going on elsewhere in the organization and we found it difficult to get lessons learned or any reusable assets from them. 5. What Was Done Right While we ran into some problems we also ended up doing a few things right. The major factors in leading us to success are presented below. Initially we created a dedicated process group and began staffing it with experienced personnel. This was a key decision and commitment to the effort that paid dividends by having available resources to tackle the many tasks associated with the work. This group provided the drive behind the initiative and ensured the adherence to sound process improvement approaches. The group also provided guidance on the CMMI to process authors and the organization as a whole. Shortly into the initiative we also engaged a Lead Appraiser who played a dual role as a process consultant. This brought in crucial guidance to the effort from an outsider and helped us stay focused on what mattered most in reaching our assessment goals. Our Lead Appraiser was brought in on an ad hoc basis throughout the process and we were able to rely on him as a mentor and advisor. He was able to offer suggested practices that were both suitable for our business and satisfied the practices of the CMMI. The core SEPG team was very experienced with the CMM and in some cases with the CMMI as well. The core team required little ramp up, and they set the pace for the rest of the SEPG members and the organization. They had been through process design and deployment activities as well as having worked in organizations that had attained high process maturity assessments on the CMM ladder. This experience and sophistication helped set goals and kept the effort balanced and progressing. We took the pragmatic approach of starting from a broad foundation by creating a unified lifecycle. We then documented what was currently done, mapped these activities to the new lifecycle and worked from established practices. As gaps began arising we took on the challenge of writing required new procedures and building out the process incrementally taking one phase at a time. In rapid order we were able to build on our foundation a full lifecycle phase gate process including detailed process flows, procedures, and templates. Throughout the process of developing the organization s software development process, we constantly reviewed the process against the CMMI framework requirements. This ensured that we maintained good adherence to the model in the eventual products. Training was provided incrementally for SEPG members. Most took the Intro to CMMI course and some took additional courses. This training was both necessary and essential to make sure everyone had a full understanding of the model and the assessment approach. We also built a spirit of learning within the SEPG. Some SEPG members joined process groups, attended SPIN (Software Process Improvement Network) meetings, and otherwise looked to benefit from other experiences that related to process improvement. We actively shared papers, documents, and web sites that could help each other in understanding some aspect of process or related technology. The decision by management to decide on a CMMI course as opposed to a CMM course was also a key enabling factor. Though the pursuit of a CMMI benchmark rating was a more difficult challenge, the benefits realized as well as the foundation to achieve a higher level of maturity justified the choice of the integration model. Additionally, the CMMI was becoming the industry standard and the CMM was going to be sunseted. A useful technique we developed was to run some meetings as full day sessions off site in order to focus the team and create an atmosphere of taking a fresh look at the artifacts under development. These days were sometimes grueling but we accomplished a lot when we were sequestered in this manner. Additionally, an outside consulting agency performed periodic reviews as sanctioned by the WK process team and these reviews gave us additional external objective course corrections. Getting all work products available on the internal intranet to get them publicly visible so as to collect feedback and to support institutionalization was also a winning tactic. The website containing the Process Asset Library is easily accessible. Making it easy for the staff was a primary concern. This enabled IT staff in the organization to begin to look at the web site and read over work in progress and provide feedback. 4

5 Communications to the organization was done often and on different channels, from newsletter to s to meetings. As stated earlier, we learned that greater communication is clearly better. Keeping the executive sponsor fully informed, and with adequate visibility into the process initiative was not only required but was essential to the success of the endeavor. 6. Thoughts on Going Forward Now that we have reached our first major goal of bringing two companies up to Level 2 we are looking beyond to new goals and better ways to reach them. The first principle we often repeated comes from Vince Lombardi who explained his strategy for winning football games as three yards and a cloud of dust. By this he meant that if the team gained three yards for each of the four allowed attempts and did this consistently they would always end up in the end zone. We like to repeat this phrase to remind ourselves that tactics lead to the realization of strategy. Staying focused on the next 3 yards can win the game. The next three yards will focus on the maintenance of the SDP (our process), refining and optimizing it, staying on top of process change requests, and constantly evolving the process. We need to be acutely aware that refinement and optimization are always necessary. The next thread of execution will be to work with the one organization that dropped out of the Level 2 assessment and enable them to reach Maturity Level 2. Next, we are developing a plan to attain Level 3 for our current Level 2 organizations. Additionally, we will work with the several organizations in our business unit that are at a lower level of maturity and help them carefully pick the right facets of our software development process that will yield the most value to their smaller yet fast growing businesses. Finally, we plan to keep promoting process and creating a vehicle for exchanging best practices across organizations within Wolters Kluwer. This will be based on need and bang for the buck, again, with a cloud of dust. 8. Acknowledgements There was a large cast of people who made this work possible. The entire CLS management team from our CTO to all of his Directors played a major leadership role in getting us to our goals. The CLS SEPG members contributed over a long period of time to pull the process together and get us through the assessment. Numerous people from the Wolters Kluwer team including Chris Layer, Dan Focazio, and Mike Antico provided essential assistance. Our outside team of consultants including Mike D Ambrosa of BAE Systems and Tom Cagley of the David Consulting Group guided us throughout this work. Finally, process adopters in the organization made the process come to life and are thanked for their dutiful efforts to make this real. 9. References [1] Chrissis, Mary Beth, Konrad, Mike, Shrum, Sandy, CMMI : Guidelines for Process Integration and Product Improvement, Addison-Wesley Professional; 1st edition, February 24, [2] Ahren, D., et. al., CMMI(R) SCAMPI Distilled : Appraisals for Process Improvement, Addison-Wesley Professional, March 2, Conclusions Process improvement, when done well, brings a variety of benefits to an organization. We have seen improved clarity around project execution, standardization of work products, and a common language and consistency within multiple organizations. We have had some stumbles along the way but we have done an equal number of activities effectively enough to reach our major goals. In sharing both our wins and losses we hope that others will benefit by perhaps advancing with less friction. 5