Chapter 2. Deadlines and Project Management

Transcription

1 Chapter 2 Deadlines and Project Management This chapter is in draft form. Please use the information and pass all feedback to help@processgroup.com. Thank you The Process Group Version 3 Page 1 of 42

2 Introduction...3 Intuitive and counter-intuitive thinking an analogy...3 Intuitive Thought Topic 1: The market demands that we over commit; we must win business...5 Belief 1: Managers believe that their job is to commit the organization to please the customer....5 Belief 2: Project managers and engineers think they must take scope and deadline requests from management almost verbatim and run as fast as they can to please the manager....7 Belief 3: Project managers and engineers don t believe that they need to sell their proposal. Take this option or give us more resources, is considered sufficient...11 Topic 2: Everything must be priority number one...13 Belief 4: The players in Camp One believe that they will eventually be able to recover the project and succeed Topic 3: Moving from Camp One to Camp Two...16 Intuitive Thought 2:...18 Topic 1: Baseline current requirements and plan to manage change...18 Topic 2: Mitigate continual surprises...21 Risk management to get ahead of the problem list...21 Small batch principle to catch changes and problems on a small scale...25 Inspection of defective documents and code...25 Lessons learned to avoid repeating common or severe problems...27 Topic 3: Track project progress and take corrective action...32 Compare the remaining available effort with the estimate...32 Compare current progress with the critical path...33 Compare current project size with estimated size...37 Compare work complete with work planned The Process Group Version 3 Page 2 of 42

3 Introduction Chapter 2 - draft Project planning and establishing achievable commitments continue to challenge software projects. Approximately 60% of software professionals that we meet complain about being overcommitted, under-resourced and unable to negotiate deadlines, scope and resources. The remaining 40% have little difficulty establishing achievable commitments with their management team and customer. During our observations since 1989, we see two distinct camps. Camp One believes that, It is not realistic to make realistic commitments. Camp Two believes that, It is not realistic to make unrealistic commitments. Here we describe the common intuitive thinking of Camp One and the counter-intuitive solutions found in Camp Two. Intuitive and counter-intuitive thinking an analogy Before we describe the thought process behind making commitments, read the following short profiles of two wine sellers and determine which one you would buy from and why? Jim: Jim is a wine salesman in the Crafty Corner Restaurant, a fancy establishment in a large city. Jim enjoys selling wine and uses the following sales process: 1. Mention to each customer that the finest wines in the restaurant are the two most expensive wines on the wine list. 2. Always tell each customer that the wines in the middle of the wine list had a bad year last year and should not be considered. 3. Explain that the least expensive wines are really not worthy of the patron s time, but are included to make the list complete. Jane: Jane is a wine sales person at the Panache Restaurant, a fancy establishment in a large city. Jane s sales process is: 1. Mention to each customer that the first two wines on the list (the most expensive) are excellent, but the third and fourth listed taste equally as good and are considerably less expensive than the first two. 2. Tell each customer that the wines in the middle of the list are very good, except for number 12, which actually had a bad year last year (the truth). 3. Explain that the least expensive wines at the bottom of the list are grocery-store quality, except for the one at the very bottom that tastes cheap. 4. Offer the customer a sample of any of the wines before purchase. 5. Give each customer the one-page review by Wine Critique magazine. With which wine salesperson would you want to do business? We have yet to meet someone that prefers Jim over Jane. Why? Because Jane is focused on the customer s agenda rather than hers, and when she puts herself in their shoes, she knows that customers want to be provided with credible information, educated about possibilities, and undersold rather than oversold. Initial intuition might tell us that sales are maximized by Jim s approach, i.e., sell more expensive wines. Counter-intuition would lead to Jane s approach, since that is the way individuals want to be treated. This treatment, in-turn, creates repeat business The Process Group Version 3 Page 3 of 42

4 Background to Overcommitment Chapter 2 - draft Overcommitment is certainly not new and not unique to software development. The following summaries help explain why people overcommit. Summary 1: Under extreme pressure, software managers often make a guess rather than a plan. When the guess is drastically low, as it usually is, chaos ensues. Intuitive commitments occasionally are accurate, but generally only when the project scale and function are similar to the guesser s experience. When the going gets rough, there is a strong temptation to believe in magic. Some savior may appear, or a new technology may be the answer. Since this hope is often an excuse for not planning, it merely postpones the problems. The scale of software projects follows an escalating cycle: - Programs generally take far more code than expected. - As the programs become larger, new technical and management issues come up. - Since these are unlike previous experience, they are a surprise. - As the scale increases, the surprises continue, but at a dramatically increased cost. Even after a higher maturity level is reached by an organization, new management, increased competition, or new technical challenges put pressure on the process. With the maturity of our field and the general lack of training for our people, this often means that organizations under pressure revert to the Initial [overcommitment] process. [Watts Humphrey, Managing the Software Process (1989)] Summary 2: The root causes of overly optimistic schedules are deep and manifold. - There is an external, immovable deadline such as the date of a computer trade show, change in tax laws, or Christmas shopping season. - Managers and customers refuse to accept a range of estimates and make plans based on a single-point best case estimate. - Managers and developers deliberately underestimate the project because they want a challenge or like working under pressure. - The project is deliberately underestimated by management or sales in order to submit a winning bid. - Developers underestimate an interesting project in order to get funding to work on it. - The project manager believes that developers will work harder if the schedule is ambitious and therefore creates the schedule accordingly. - Top management, marketing, or an external customer wants a particular deadline, and the project manager can t talk them out of it. - The project begins with a realistic schedule, but new features are piled on to the project, and before long the project is running under an overly optimistic schedule. - The project is simply estimated poorly. [Steve McConnell, Rapid Development (1996)] 2005 The Process Group Version 3 Page 4 of 42

5 Intuitive Thought 1 Topic 1: The market demands that we over commit; we must win business Camp One, the group briefly referred to above, is a group that believes that the only way to succeed as a development organization is to over-commit to the customer. Among the phrases used to explain their position are: - We agree to crazy schedules because we are customer focused. - We must promise more than we can do so that we can win sales. - It s not realistic to make realistic commitments - We tell management personnel what they want to hear. - The business climate makes us over-commit. Customer satisfaction is cited as the primary motive for these actions. Camp One s behavior is the result of having one or more of the following underlying beliefs (intuitive thoughts): - Belief 1: Managers believe that their job is to commit the organization to please the customer. - Belief 2: Project managers and engineers think they must take scope and deadline requests from management almost verbatim and run as fast as they can to please the manager. - Belief 3: Project managers and engineers don t believe that they need to sell their proposal. Take this option or give us more resources, is considered sufficient. - Belief 4: The players in Camp One believe that they will eventually be able to recover the project and succeed. Belief 1: Managers believe that their job is to commit the organization to please the customer. Pleasing the customer by committing the organization to whatever the customer requests is a risky approach to running a business. The credibility of each commitment becomes suspicious when too many quick promises are made to the customer. If the commitments are unachievable, credibility plummets. When you receive a request, don t assume that the person making the request needs a response immediately. He or she might need it by the end of the day, or by the end of the month. Ask how much time you have to work on a proposal and determine how solid your commitment can be, given the time available to work on it. For example, if the request is simple and you can commit to a solution, commit. If there is some research needed, state that you can have a tentative commitment by the end of the day and a firm commitment in three days. For very complex or large requests, explain the steps needed to develop a proposal and establish dates when estimates, ranges and a final commitment can be provided. If there are many deliverables in the commitment, commit to the pieces of the project that are known and that can be estimated. Establish subsequent deadlines when the remaining pieces can be defined and promised The Process Group Version 3 Page 5 of 42

6 If your company has sales personnel (or an equivalent business analyst or manager who interfaces with the customer), send engineering representatives on sales calls with them, particularly if there is a history of sales personnel making unachievable commitments on behalf of engineering. Joint visits enable customer requirements to be communicated in parallel to both parties, minimizing communication and commitment problems later. It is easy to resist the suggestion of sales and engineering working jointly on a client visit. We hear complaints such as, There is no time for both company representatives to visit the customer, and The customer might become confused from two opinions. However, the opposite of these comments is also true. If the addition of the engineering representative adds four more hours of company time to a sales call, then only one mistake of four hours has to be saved anywhere downstream for the additional time to be valuable. Once the sales process works correctly, engineering representatives can be reserved for high-risk requests that require the expertise of sales and engineering together. Managing commitments Use a commitment process to manage expectations between parties and clarify the steps an initial request goes through to become a final commitment. Without such a process, projects and organizations can easily over commit and mismanage expectations, resulting in chronic crises. An example process is shown in Figure 1 [based on Humphreys88]. Step 1: The project team determines high-level product needs (scope of work) from customer and marketing input. Note: For a small request, a short document might be adequate to capture the customer s needs. For a large request, initial requirements are collected and a plan is established to refine the requirements, either in one phase or through several iterations. Step 2: The project team develops an initial estimate and project plan to determine what is feasible. Note: For a large project, state that two estimates will be provided, one on date X, and one on date Y. The first will be a range based on the current understanding of the requirements; the second based on a baselined set of requirements. For iterative projects, state that each iteration consists of a requirements, estimation and commitment phase. Step 3: The project team meets with management, marketing, customers and related groups to determine a) whether the product, or change request, is feasible, b) there is agreement to the resource, cost and schedule estimates, and c) the risk is acceptable. Note: This might be a brief meeting, or phone call. The purpose is to visibly agree to the work, estimates and risks. Step 4: A commitment is made OR further negotiation is held. Figure 1 Example commitment process To be effective, the commitment process must reach the top of the organization (Humphrey 1989). This requires that: 2005 The Process Group Version 3 Page 6 of 42

7 - All commitments for future software delivery are personally made by the organization s senior executive. - These commitments are made only after successful completion of a formal review and concurrence process. - An enforcement mechanism ensures that these reviews and concurrences are properly conducted. The senior executive s personal involvement is what motivates the entire commitment process. The people know they must justify their recommendations in a visible process and that poor work will likely be exposed. The process for managing commitments should not be lengthy or complex. Use the example described as a basis for your project or group. Typically, one page or less is sufficient to explain how a group will derive sound commitments. Belief 2: Project managers and engineers think they must take scope and deadline requests from management almost verbatim and run as fast as they can to please the manager. The reaction of the project manager and team to jump when the manager says Jump is probably the weakest link in the organization s ability to handle commitments. The quick reaction of the team to agree to any and all commitments handed to them, regardless of feasibility and risk, is usually the team s first large error. The underlying intuitive thought is, I can keep my job if I play along and agree to whatever I am asked to do; I can always seek forgiveness later. Changing the way a team reacts to commitment requests might be the toughest behavior to change since it is supported by ego and deeprooted beliefs that quick and pleasing reactions make us look the most capable. What is missing is the realization that the person requesting a commitment wants a sound commitment, not any commitment. When project managers and team members internalize this, their reaction to complex requests changes from, Yes, sure we can do that to, The team will develop a firstpass estimate range, assumptions, risks and schedule options. When is the latest time by which you need a sound commitment? Providing complete data Making sound commitments requires a project team or organization to accompany any estimate with assumption, risk and resource availability data. Assumptions: Assumptions are statements regarding any part of the project that help define the commitment. Example assumptions are: All the code must be developed from scratch, A new operating system version is required to implement the desired functionality, and Component B will arrive January 16 th. Assumptions can refer to resource availability, dependencies, items that are not in scope and expected quality levels. If any assumption is false, the commitment itself might be in jeopardy. Risks: Risk assessment enumerates potential technical, schedule or resource problems that might be encountered by the project. When identifying risks, also consider the assumptions that accompany the commitment. Assumptions are statements that need to be true to enable the commitment but, in reality, might become false. An example risk management plan is shown in Figure 2 [Sakry01] 1. 1 A risk management process is described later in Chapter The Process Group Version 3 Page 7 of 42

8 Risk Items (potential future problems derived from a brainstorm session) New operating system might not be stable. Communication problems over system issues and dependencies. We may not have the UI requirements right. Requirements may change late in the cycle. Database S/W might be late. Likelihood of Risk Item Occurring (1-10) Impact to Project if Risk Item Does Occur (1-10) Priority (Likelihood x Impact) Actions to Reduce Likelihood of Risk Occurring Test OS 2 more weeks Develop system interface document Build prototype of UI of screens #2 and # Get sign off on top 10 requirements. Establish change review board Check with supplier. Key people might leave Make sure Jim is happy with current workload and travel schedule. Actions to Reduce Impact if Risk Does Occur Identify 2nd OS. Test with 2 nd OS. Add replan milestone to realign the team's schedule. Limit initial product distribution. Ship core 5 features and limit initial product distribution to validate approach. Make application compatible with a substitute DB. Earmark Fred as a backup and cross train. Who Should Work on Actions Joe Cathy Lois Cecil Joe Pete Chapter 2 - draft When Status Should of Actions Actions be Complete 3/3/XX 5/6/XX 6/6/XX 6/7/XX 7/7/XX 7/8/XX Figure 2 Example risk management plan Resource availability: The deadline you can achieve will be based on specific availability of resources. When you discuss a commitment with a customer or manager, you might have an idea of the resources required. If you do, verify that they are available to meet this commitment, given their current workload. If you do not have a clear idea of the estimate or resource availability, state that, and develop action items to investigate further. Tailor your planning response based on the complexity and magnitude of the request; a trivial request can have a quick response; a complicated request will have a comprehensive response. Respond consistently with a statement that meets the requesters need, not one that immediately puts you in a nowin situation. Intuitively, you might think that the manager needs a response right now, this instant. But in most cases they don t; that was just your assumption. What the manager really wants is a credible supplier (you) who can respond with considered and feedback and a sound commitment The Process Group Version 3 Page 8 of 42

9 You might believe that you cannot choose any response other than the one you have historically chosen. However, realize that your customer and manager will change their reaction based on the way you respond to the commitment request. When asked for a commitment: - Enumerate credible options to the request and develop alternatives that the requester has not considered. - Provide planning data that can be examined for options regarding approach and errors in assumptions. - Communicate project risks and suggested mitigation actions. - Explain your conclusions regarding deadline and cost options using your data. The following example illustrates one team s reaction to a commitment request. The group in this example is a large IT division of a multi-national product manufacturer. The IT group provides computer services for company sales forecasts, sales transactions and marketing events. The development team supporting a very old legacy mainframe system was tasked with moving the sales applications to a commercial database product. The team had identified 60 business processes currently supported by the legacy system and each one was required in the new system. The team had been committed to move all 60 processes within an impossible timeframe with little or no involvement from the team members. This was the fourth time that the team had been challenged to complete this work, with three previous failures. Before this last attempt, the team members attended two of our workshops, one on requirements development and one on project planning. In the requirements workshop, they took their ambiguous paragraph-styled requirements and rewrote them as clear single sentences. They expanded the highest risk requirements into detailed Use Cases and set requirement priorities using the example format shown in Figure 3 [Wiegers03]. These priorities were shared with their management for feedback. This resulted in the elimination of half of the requirements from the first release of the project. In the planning workshop, the team estimated the project effort using the Wideband Delphi technique [Wiegers00]. In the Delphi process, the team creates an initial project task list, followed by individual preparation to derive estimates and underlying assumptions. The team members then come together to share their data. Individual estimates are kept anonymous while task descriptions and assumptions are publicly shared and refined. The process results in a clear task list, assumptions and estimate range for the work. The output is then reviewed with management. Here is the story from the team for the requirements and planning phases. The analytical nature of the requirements development and prioritization process provided a thorough evaluation of all options. As we reviewed the analysis with key decision makers, we educated them about the potential solutions for each requirement and the benefits, penalties, costs and risks of each solution option. They participated with us in the evaluation process, providing valuable insight into the analysis and ensuring buy-in with the results. When we were complete, the analysis clearly showed the best strategies going forward The Process Group Version 3 Page 9 of 42

10 Require ment Benefit (B) if Present (1-9) Penalty (P) if Absent (1-9) Total Value (V=B+P) Value % (V%) Relative Cost (C) to Implement (1-9) Cost % (C%) Relative Risk (R) to Implement (1-9) Risk % (R%) Priority (V% / C%+R%) Reqt % 1 5% 1 7% 1.83 Reqt % 3 15% 2 13% 1.25 Reqt % 5 25% 3 19% 0.41 Reqt % 7 35% 3 19% 0.28 Reqt % 4 20% 7 44% 0.16 Total % % % -- Figure 3 Example format for setting requirements priorities Since the key decision makers all participated in the analysis process, they agreed with its outcome and they were willing to abandon prior strategic decisions, even ones they had personally made. Their involvement in the analysis provided a win-win method for all involved to leave behind failed strategy decisions without losing face. Involving management was a counter-intuitive approach for the team. With this critical win in our pocket, we then improved our planning skills to give us a fresh start with our new strategy. From the outset of the planning session, we were challenged to think differently. We had managed projects for years using a very disciplined process. We were experienced in project planning and work estimation, and we were quite good at it. The fact that we rarely completed projects on time was obviously due to management, work culture, and other things out of our control! During the Wideband Delphi process, the team agreed to a high-level Work Breakdown Structure (WBS) followed by each individual estimating the tasks without discussion with the other team members. This already was different. Normally, we would sit around a table as a team and work through the WBS, estimating as we went, until we had a full project schedule. Everyone on the team would be there, everyone would participate, we would have the right information, correct? However, this time, when we first came back to the table with our individual Delphi estimates, we were all over the map. Way off. Not even close. What happened next was even more puzzling we discussed our assumptions and task definitions, but we couldn t talk about specific estimates or formulas. What we didn t realize at the time was that we were level setting. In our previous group session, a few people typically dominated the discussion. As time rolled on, those not heavily involved became bored. Now, our individual preparation in the Delphi process enabled critical and unique assumptions to be surfaced. When we came together we combined them to establish a comprehensive and clearer answer. The hardest discussion (but most critical) of the planning phase was the review of our planning data with senior management after the team session. The output from the high-level plan did not match their expectations. However, we now understood what it would take for a successful implementation The Process Group Version 3 Page 10 of 42

11 Management pressured the team to reduce their estimates, to conform to their expectations for delivery, but the team did not budge. Having followed the process, we had detailed assumptions behind each estimate, which we were willing to review, but not willing to arbitrarily change to meet a date. The team s decision to obtain good data provided the level of detail that senior management needed to change their expectations to meet our estimates. Throughout the project, we continued to use the planning and requirements techniques. We utilized the requirements prioritization process again to determine how we would satisfy each detailed requirement. We revisited the planning process to develop our WBS for the detailed implementation plan. Unlike the previous four attempts, this time the team implemented the agreed-to functionality, on time and under budget! Belief 3: Project managers and engineers don t believe that they need to sell their proposal. Take this option or give us more resources, is considered sufficient Customers and managers want sound data on what is possible, so that they can succeed in their goals. The project team uncovers project details and communicates insight that neither the customers nor managers have. This information is then packaged and communicated in a way that leads the customer to the right solution for all parties. Even if the customer or manager was initially demanding, abrupt or bluesky in their request, it is still in everyone s best interest to package and sell achievable options. Statements of, We never have enough resources, usually indicates that the team has over committed. This might be due to an unavoidable oversight in the complexity or scope of work, the lack of sound estimation and planning, poor negotiations to establish an achievable commitment or inadequate replanning for significant project changes. The most common reason for a chronic lack of resources is inadequate estimation and planning prior making the first commitment. When this occurs, adding resources to a group that is chronically over committed rarely fixes the over commitment problem. A group that promises more than it is able to deliver can continue to do so regardless of how many people it has. The solution to this problem consists of scoping and estimating the work to be done, and learning negotiation skills to match what can be done with the work request. Once these skills are internalized, more resources can be requested to achieve more work. When communicating project options, consider the following: Know by how much you are over- or under- committed: Compare the total number of effort-days required to complete the project (i.e., your effort estimate) with the number of effort-days you have available to spend (i.e., number of work-days between today and the deadline x number of people assigned to the project x the availability of each resource (e.g., 50% or 80%). If your estimate is 20% more than the resources you have, you might be able to recover by working one weekend day each week, or 1.6 additional hours each day. If your work estimate is 20-30% more, then you can potentially recover and meet the deadline by working additional hours again, hiring more labor or increasing the availability of each resource. If your estimate is approximately equal to the available effort, minimize delays and actively manage risks and the critical path to have a chance of meting the deadline. If your estimate is more than 20% less than the available resources consider using less staff, reducing the deadline or 2005 The Process Group Version 3 Page 11 of 42

12 increase scope. If your estimate is more than 30% larger, then a significant re-plan is required using some of the options described below. Understand what the deadline is based upon: Understand what is driving the current deadline. Is it a stake in the ground that a manger established to keep people focused? Is it based on a client s need to go live with a system, or to avoid the delay of the next release? Unless you know the details behind the expected deadline, you can only guess what can be changed, negotiated or dropped to achieve it. Propose an incremental delivery: Once you know what is important to the customer and manager, and how the system will be used, suggest smaller increments that can be delivered earlier. Identify with the customers the most important features that help them perform their work tasks. Split the requirements into at least three buckets that reflect the requirements priority (e.g., bucket one contains features that must be included in this release; bucket two contains features that must be included but can wait for a later release; bucket three contains features that would be nice to have if they can be included). [See appendix A for more on setting requirements priorities.] Check your assumptions with others: Detailing the project- and- task-level assumptions provide you with more potential options to make your deadline achievable. Include technical, schedule, resource availability and quality assumptions. For example, you might have been assuming that version 16 of module X must be used, but in fact, version 15 is simpler and adequate. You might have assumed that you have to wait until your sister division completes the hardware so that you can test, but in fact, the customer is on vacation during August, and you could use their hardware at no cost. Write your assumptions down and share them with others to see what options appear 2. Examine the critical path of the schedule: The critical path is the longest calendar path through the schedule and the one that defines the deadline. Some tasks can be performed more quickly to bring the deadline in, and other tasks that appear to be important can be accelerated with no reduction in the deadline. Calculate the critical path using a scheduling tool and determine if reducing delays or adding more resources to the critical-path tasks will finish the project more quickly. Clarify resource availability: All schedules assume the availability of specific resources (for example, Jane must be available 80% for tasks 1 through 20, and 30% for tasks 21 through 30). An interruption in Jane s availability adds risk to the schedule completion date. Clarify what availability is needed to attain the desired deadline. Take action to reduce chronic interruptions if team members cannot spend the required time on their tasks. If one resource is performing three or more project tasks in parallel, assume that a non-productive tax must be paid when switching between tasks (it takes a finite time to put one task down and pick up the next). In this situation, schedule the person to complete one task before starting the next, or reduce the number of parallel tasks performed at any one time, thereby reducing the tax. Look specifically at the Critical Path for examples where resources are being shared. If a resource is performing two parallel tasks, and one of them is on the Critical Path, the whole schedule is being delayed. Consider having the resource complete the Critical Path task first, and then work on the other task. 2 Do this on paper, not just swirl some random thoughts around in your head quickly and say, Yes, I ve done that! 2005 The Process Group Version 3 Page 12 of 42

13 Define the customer s quality expectations: Is the customer expecting a proof-of-concept prototype, a completely error-free system, a product with known defect density (e.g., no more than 2 defects per thousands of lines of code), specific reliability (e.g., mean time between failure is no more than 5 days), or a mature product with all import and export features. Each quality definition corresponds to a different set of project tasks, effort and duration. Check whether you are building a Rolls Royce when a Toyota Corolla was desired. Simplify your solution: Look at your project and identify areas that have been over-complicated or over-engineered. Evaluate each item from the end-user s perspective; i.e., if you didn t have this complication, would the user ever know or care? Consider simplifying algorithms, database structures, complicated graphics, wizards, operating system versions and features that are unnecessary for project success. What could you simplify to achieve the deadline? Investigate process changes that could relieve deadline pressure: Process changes can make your project complete earlier. However, each process change brings with it a change in risk that must be assessed. For example, a team waiting for an approval at the end of one project phase could start the next phase early and assume the risk of rework if unforeseen changes are introduced during the approval process. A team could also perform inspections (Peer Reviews) on 50 percent of the code base while assuming the risk of defects being undetected in the remaining 50 percent. Testing could be reduced to final acceptance testing while replacing selected unit testing with inspections. Determine sections of the project that should be purchased or built: Look at the project for areas that can be delegated to others to save time, money or risk. Can the team use standard libraries that are pre-tested? Are there sections of the project that can be adequately completed by a thirdparty at similar cost? Are there components that have been delegated that should be brought back in-house because of the available expertise within the company? When you communicate project data to managers and customers, state how the data were derived (e.g., Wideband Delphi process, historical data, intuition, fit to please!) and your confidence in the estimate (e.g., high, medium, low). Your confidence level indicates how likely this number is to change and whether further planning is necessary. If all options have been investigated, decide whether you want to commit to the request. If you do, write the decision down (including the scope, your estimates, assumptions and identified risks). Don t make this agreement verbally allowing everyone to conveniently forget the quagmire you are foreseeing. Politely send a copy of the agreement and data to all stakeholders so that the decision is visible. Obtaining and communicating planning data is not only key to setting stakeholder expectations; it changes how stakeholders react to the data. Communicating Take this estimate or else! elicits a limited set of reactions, none of which are usually pleasant. Communicating the planning data described above elicits a constructive reaction that results in a project team with a negotiated scope, schedule and risk 3. Topic 2: Everything must be priority number one When your project has more work than can be done in the timeframe allowed, take the lead and establish priorities for the project, don t suffer waiting for someone to take action. One example of this was during the writing of our first book. 3 Project teams do not need permission to perform this level of planning and communication. It is a choice available to all The Process Group Version 3 Page 13 of 42

14 After many years of work, we submitted our draft manuscript to the publisher for review. Five anonymous reviewers were identified to critique it. The anonymous reviews were sent back to us with instructions to address the findings by deadline established in our contract. The reviews contained a total of 125 change requests, some very good, some inconsistent with each other, and some we did not agree with. There was no indication from the publisher regarding the priority of the changes and we knew we did not have enough time to make all the changes by the deadline. Our intuitive (initial) thought since this was our first book was that we should go out of our way to make every last change by the deadline. Our counter-intuitive (subsequent) thought was to re-look at the goals of the book and develop our own priority system and see which changes we could drop. We defined a simple three-bin scheme and placed each change request into a bin (see Figure 4). Along side each change was an estimate of the total effort required to make the change The Process Group Version 3 Page 14 of 42

15 Original Order of Review Comments Reviewer Comment Planned Corrective Action Priority of Change 1 While Chapter 1 and Chapter 2 seem to be well developed, Chapter 3 and Chapter 4 seem to be incomplete. In my opinion the manuscript is not ready for publication yet, but is an excellent start, and with further development will be a good reference for SEPGs and other process champions. The following comments are recommended improvements for Chapters 1 and 2, and what I think is missing in Chapters 3 and 4 Two options a) Merge chapters 3 and 4. The topics in ch 3+4 are smaller than ch 1+2. Merging these 2 chapters would make the book look less unbalanced. b) Add more to chapters 3 and 4. However, we don t think we should add stuff just to make the chapters larger. Need to define what is incomplete about ch 3+4. We assume the changes listed regarding ch3 will make ch 3 complete. Time Esti mate to Fix (Effo rt- Hrs) Resou rces requi red (Pers ons) A Total Wor k (Effo rt- Hrs) Priority A = Changes required to achieve the book's goal. Priority B = Changes that improve book readability. Priority C = Changes that have little impact on the book s goal. No one would ever know if this change was dropped. Summary Effort for Priority A/B/C Changes The total effort for all changes labeled A is 180 hours (including 48 hours to find/repair defects). The total effort for all changes labeled A+B is 275 hours (including 48 hours to find/repair defects). The total effort for all changes labeled A+B+C is 302 hours (including 48 hours to find/repair defects). Figure 4 Example priority scheme and evaluation of change requests At the end of the table we listed the total effort for each category and the additional time required to inspect (peer review) the work prior to resubmission. These numbers were then compared to the available time we had in our calendars between receiving the change requests and the publishers desired deadline of October 12th. Our decision was to complete the first two categories and request that the deadline be extended by one month. We told the publisher: 2005 The Process Group Version 3 Page 15 of 42

16 Our preference is to make all of the A, B and C changes, which we believe would significantly improve the book. If we were to make all of these changes, we could have the edited manuscript completed by October 19 th based on the dedicated time we have reserved for this book. If we were to make all of the A and B changes (and not C), we could have the edited manuscript completed by October 12th. We think it would be unwise to only make the A changes. Please let us know your thoughts. In this example the publisher advised us to proceed with option two that allowed for their preferred deadline of October 12 th. What is important for this example is that, in the absence of any guidance, we created and used a priority scheme to manage the work and showed the customer how we could meet their goals with an approach agreeable to both sides. We did not ask permission to do this; we just assumed we could. Not enough time to plan? A common reaction after learning a planning process is that it takes too much time, and anyway, we have to react now! But too much time compared to what? You only have to prevent one error of a similar duration invested in planning anywhere downstream in the project and you have paid for the planning time invested. Prevent two errors, and you have doubled your money! For a small commitment request, take 15 minutes to clarify the request with the requester. Spend 15 minutes and write down the tasks and estimates required to complete the commitment. Brainstorm risks with achieving the commitment and capture on a risk list. Discuss the results with your stakeholders and consider achievable options that meet the requester s need. For a larger request, select a small team and spend between one and two hours in each of the steps to develop a first-pass schedule. Belief 4: The players in Camp One believe that they will eventually be able to recover the project and succeed. Over committing and assuming that recovery will be possible is a risky approach for a team. Recovery is possible if two conditions exist. First, the team is able to complete the estimated hours of remaining work by the deadline (assuming the estimate is somewhat accurate). Second, there are no delays in the completion of the tasks on the Critical Path of the schedule. For example, if the remaining work is estimated to be 600 hours and this is determined two weeks prior to the deadline, each team member would have to work 12 hours each day with no break 4. Assuming breaks and time to reply to is needed (say, two hours per day), the team would have to be at work 14 hours each day for two weeks. Add in support time to fix bugs in other product releases and there is no time for sleep! This project scenario is almost unrecoverable since it assumes a specific availability of the resources and that no other problems will occur. Recovery, consisting of re-planning, is accomplished by revisiting the negotiation points described under Belief #3. Topic 3: Moving from Camp One to Camp Two Camp One can continue for years before its behavior and results change, either because customers become used to schedule and quality problems, or because the team comes through at the end of the project with heroic efforts. Usually, neither the organization or customers are happy. The current economic climate is commonly used to support Camp One s position. If the economy is booming, it is declared that there is no time for practices such as planning and risk management. 4 5 people x 10 days x 12 hours/day = 600 effort-hours The Process Group Version 3 Page 16 of 42

17 Intuition says, We are successful without those practices, or We are too busy making money, we don t have time for process. If the economy is in a slump, it is declared that one must underbid and overpromise to Get in the door with the customer, or Tell the customers what they want to hear. When the economy is booming, activities such as planning and risk management enable a company to maintain a rapid rate of production and prevent problems that could causes major setbacks (such as releasing poor quality products and over committing the organization to breaking point). When the economy is sagging, success is dependent on distinguishing your organization s performance from your competitor s. Success is not achieved by over-promising (and therefore under delivering) to the customer. Regardless of the economy, customers always want dependable suppliers that deliver quality work. Planning matches your commitments and capability with the customer expectations. Risk management keeps you out of trouble by actively identifying and mitigating downstream problems that could occur. It is almost impossible (but not totally) to convince Camp One that its own intuitive thinking is a large cause of its problems. Suggestions that the customer s needs should be better understood, or that estimates should be created and risks assessed before commitments are made, are seen as luxurious or theoretical approaches. Camp One is sure that This is the way we must work, and perceives that Camp Two has an easy life in an easy industry. The route from Camp One to Camp Two is to match the organization s current problems with the solutions offered by counter-intuitive thinking and practice them until they become intuitive [Potter02]. This transition can occur proactively or reactively. The proactive approach requires some faith, thought and the realization that these skills are learned by doing them, not solely by reading a book. This can be accelerated with the introduction of a new senior manager that insists on raising the bar; someone typically from Camp Two. The common alternative is to delay change until the organization sinks under the weight of overcommitment and there are no longer any alternatives other than to plan and make sound commitments that customers can trust The Process Group Version 3 Page 17 of 42

18 Intuitive Thought 2: Chapter 2 - draft We must run fast because we are already behind. Topic 1. Baseline current requirements and plan to manage change Topic 2. Mitigate continual project surprises Topic 3. Track project progress and take corrective action Topic 1: Baseline current requirements and plan to manage change [Author note: paste corrected text from newsletter article for this section] Requirements management is a well established series of steps for tracking and evaluating change requests. However, in 40% of the companies we have observed, managing requirements changes is perceived as luxurious and theoretical, something only performed by people in snail-paced industries with plenty of time on their hands! Here we describe the common intuitive thinking that prevents teams from implementing effective requirements management and some of the variations in implementation to make requirements management effective. There are two common beliefs that cause people not to establish an effective requirements management system. First, the project members believe that there is no time to evaluate requirements changes; it is believed to be quicker just to make the change. Second, the impact of the change to other groups and other parts of the system is assumed to be negligible. These intuitive thoughts are risky and lead to unexpected schedule delays, poor communication among impacted area and quality problems that can be time consuming and difficult to fix. The lack of requirements management can cause the requirements change problem to snowball. For example, if you respond to a request by saying, No problem, we will have that change request complete in the first release, you might be interpreted as implying that all change requests can be accomplished with no impact. The requester can choose to hear this implicit message loud and clear and dream up other changes that would be cool to have. Premature agreement to change requests can appear to be intuitively correct because it is customer focused, however, it can leave your team buried in unachievable commitments and risk that the customer not achieving his or her goals. Your requirements management solution does not have to be excessively formal, documentationcentric or slow. Design the solution robust enough to set clear expectations on managing requirements changes among the team, customers and management personnel. Involve effected groups in the creation or review of the process to work out difficulties. Establish your requirements management process by answering these questions: 1. What is the objective of the requirements management process? 2. What are the current set of requirements and are they baselined (defined and agreed upon)? 3. Who will collect requirements changes? a. Select a project member, analyst or manager. 4. Who needs to evaluate and agree to requirements changes before they are made? 2005 The Process Group Version 3 Page 18 of 42

19 a. E.g., Customer champions, managers, team members and impacted system representatives 5. How will the requirements be stored and labeled so that there is no confusion regarding the current state of the requirements? a. E.g., Place requirements document under version control or use a requirements management tool. 6. How will impact to schedules be evaluated? 7. How will approved schedule and requirement changes be communicated to affected parties? Here is an example process created from answering the previous questions. 1. Objective: Requirements changes will be collected, assessed, and agreed to jointly, based on the estimated effort to complete the change, the risk to project success, and the desired release schedule. 2. The current requirements definition will be stored in the file: product-requirements-vern.doc 3. All requirements changes will be ed to change-request@sc.com. a. The project manager will collect all requests received in the file, reqs-change-logversionn.doc. All requirements changes that are not submitted to changerequest@sc.com will be ignored. 4. The Requirements Review Board (RRB) will evaluate all requirements changes weekly. The RRB will consist of the development team, customer champions and product management. a. The change request list will be ed to the team prior to the session. Each requirement will be evaluated as major or minor. The team will discuss the major items and review the minor items. b. More frequent RRB sessions can be held to process changes within one week. c. The RRB will decide on each change using the status labels defined in step The requirements will be stored in the file, product-requirements-vern.doc. The version number (N) will be incremented to reflect a new published version. a. Each requirement will be labeled, Approve now, Approve for subsequent release, Rejected, or Evaluate further ) 6. The project manager will assess the schedule impact of all minor changes with no further required discussion. The RRB will evaluate all other changes with respect to risk, cost (effort) and schedule. 7. All new requirements changes and revised schedules will be ed to the product team and reviewed in the bi-weekly team status meeting. Notice in the example process there is a fast track. Changes that can easily be evaluated as minor can pass through with an appropriate label. The RRB can then deal with the changes that need attention. You might decide all change requests that are either low risk, or take less than one-half-day to implement are minor. However, if the impact cannot be easily determined, then it should proceed to the RRB for a more accurate evaluation The Process Group Version 3 Page 19 of 42

20 Establishing a requirements management process is not only key to setting stakeholder expectations; it changes how managers and customers make requests. Customers and managers know conceptually that all changes have an associated cost, but they might have internalized this since previous requests were accepted with the response, No problem, we can do that. When they know that the impact of requirements changes will be assessed and communicated, they are more careful in the changes they request. Different implementations for different situations Design the requirements management process to suit the complexity of the problem you have. Use the process you have designed and frequently make changes to match your situation. If you suggest to all players that the change control process should be dumped, and they agree, you don t have an effective process yet! Consider the following implementations when you implement requirements management: 1. A simple change log and control board For most projects located in one location, a simple text file or spreadsheet under version control is adequate to record requirements and changes. The control board might consist of the whole project team, or representatives of development, marketing, test and requirements, along with the project manager. When there are requirements changes, the control board meets (in person or electronically) to decide on the changes. 2. Involving cross-functional teams on the control board Cross-functional teams should be represented on the board when the requirements changes impact other teams and systems. For example, changing one field in a database might be good for the database team, but it could cause application teams to spend weeks of additional time upgrading and testing their programs. If there is a group that would be upset with you if a change were made secretly, then that group should be represented on the control board! A text file or spreadsheet might still be adequate to record requirements and associated changes, but thought is needed regarding where the file is stored, who can change it, and how impacted areas are informed of new versions. A web page might be a good solution with automatic notification of changes. For very large systems, you might have several control boards that look after unique sets of requirements (for example, one board for user interface changes, one for changes to hardware, and one for changes that impact middle-ware programs). The boards could operate individually with representatives of each board being a member of the other boards to verify the impact of specific changes. 3. Using a requirements management tool Requirements management tools are effective for storing requirements and making them accessible to other groups [2]. Requirements tools assist you in storing additional information regarding each change, and enforcing the change control process. Tools should be considered once requirements are well written and managed. At this point, the needs for a tool are understood and sufficient data exist to make the tool a valuable investment The Process Group Version 3 Page 20 of 42

21 Requirements change management is the filter that verifies that customer expectations are managed and that the project in investing its scarce resources wisely. Resistance to requirements management is addressed when the relevant stakeholders have their needs captured and addressed by the process. Implementing requirements management is not conceptually difficult, but it does require thought regarding who should be involved, where data are kept, and how the status of changes should be communicated. It also needs someone to lead they way and set the example. Topic 2: Mitigate continual surprises Once projects have started, it is easy for them run off course and into never ending problems. These include technical surprises, inaccurate estimates, resource turnover, customer complaints, components that refuse to work, scope creep and defects found in existing and new work. When you have problems on your project, consider the following solutions: - Risk management to get ahead of the problem list - Small batch principle to catch changes and problems on a small scale - Inspection of defective documents and code - Lessons learned to avoid repeating common or severe problems Risk management to get ahead of the problem list Problems that occur on a project were risks just prior to them becoming problems. The chances are, someone in the organization probably knew about the risk prior to it becoming a problem. In the light of a never-ending stream of problems, risk management is a simple and inexpensive method of staying ahead of the stream. [Author note: Put the following risk management process in appendix, leave it here, or move it earlier to the first occurrence of the risk Figure?] Risk management is similar to preventive health care and insurance for your project. It involves identifying risks (making potential problems visible), analyzing those risks, managing them, and reviewing them over time. When risk management techniques are used, you can prevent problems and anticipate others, making your project run smoothly [Boehm89, Potter01]. The risk process described here is simple, effective and typically takes 90 to 120 minutes for projects with 12 to 24 person-months of effort. Projects smaller than 12 person-months can take less time. You can control the length of the session by controlling the scope you pick. Most sessions take less than two hours. There are six steps in our risk management process. A description of the process follows. Step 1: Determine the Scope of the Risk Session The scope of the risk session can be the requirements for the complete project, the next product release or the implementation of a series of bug fixes The Process Group Version 3 Page 21 of 42

22 Step 2: Select the Team and Moderator Chapter 2 - draft Invite individuals who can suggest risks that may prevent the successful completion of the project. Invite the project team, your stakeholders (for example, software developers, software quality analysts, and managers), people who have been on similar projects, and experts in the subject area. Limit the group size to nine people to keep the discussion focused. Assign a moderator to keep the session on track. The moderator explains the risk process to new team members unfamiliar with the scope of the risk session or the risk management process. Step 3: Identify Risks Risks are potential problems, ones that are not guaranteed to occur. A brainstorming session is an effective way of identifying risks. The session typically takes 15 to 30 minutes. During the session, people call out potential problems that they think could cause the project to fail. When performing risk identification, team members often start by listing known problems. Known problems are not risks. If this occurs, just move them to a problem list and concentrate on future potential problems (ones not currently occurring or not guaranteed to occur). During the brainstorm, consider the following items: - Weak areas such as unknown technology; for example, new languages, tools, a new operating system, and design methods that are new to the team - Items that are critical to the project, such as the timely delivery of a vendor s component, a translator to provide a migration path for customers, and the development of a new algorithm. - Problems that have plagued past projects, such as loss of essential staff, incorrect requirements, and changing priorities. Once you have created a list, work with the group to remove duplicate items. Step 4: Analyze Risks The first task in analyzing risks is removing ambiguities and ensuring that each team member understands each risk item. Risks such as Lack of customer buy-in, and People might leave the project, are ambiguous. In these cases, the group may decide to split the risk into smaller specific risks such as Customer X could decide that the new release is not beneficial, Test expert might leave, and Web master might get pulled off the project for one month. The second task in analyzing risks is enumerating the primary consequence if the risk occurred. This discussion allows the team members to understand each other s perspectives of the risk and to be better prepared to assign an impact number during the next task. The primary consequence of each risk is captured in the Consequence column (see Figure 2). The last task sets priorities and helps you determine where to focus your risk mitigation efforts. Some of the identified risks are unlikely to occur and others may not be serious enough to worry about. For example, if there is a risk of a key person leaving, you may decide that it would have a large impact on the project, but that it is not very likely The Process Group Version 3 Page 22 of 42

23 Set priorities by agreeing as a team how likely each risk item is to occur, using a scale from 1 to 10 (where 1 is very unlikely and 10 is very likely). Then rate how serious the impact would be if the risk did occur, using a scale from 1 to 10 (where 1 is a small impact and 10 is a very large impact). To use this numbering scheme, first select the items that rate 1, then select the items that rate 10, and then rate the remaining items relative to these boundaries. The priority of each risk item is the product of the two values: likelihood multiplied by impact. This priority scheme helps push the big risks to the top of the list and the small risks to the bottom. Figure 2 gives an example. An alternative rating process is to use absolute numbers that reflect impact and likelihood. For example, using a time or cost impact assessment of each risk item would lead you to a more precise rating, assuming you have accurate data. Similarly, using historical probability data (instead of a likelihood number) would also increase your precision. If you don t have any historical data (and most people don t), use the rating system provided. The intent of the rating system is to focus you on a few critical risk areas. For that, a precise rating system is not required. Now that the group has assigned a priority to each risk, select a few items to manage. Some project teams pursue a few of the risks, whereas others choose to work on all of them. Start by selecting the top three risks, or the top 20 percent, based on the priority calculation. Step 5: Plan to Mitigate The first way to manage risk is by reducing the likelihood of the risk occurring. For example, some project teams clarify requirements by writing use cases [Wiegers03], increase test resources by borrowing customer networks when they are on vacation to, or start product evaluation early by obtaining a demonstration copy of a vendors component. For some risks, consider eliminating the likelihood altogether by changing the decision that caused the risk. For example, the risk The new operating system (OS) might not work only exists because of the decision to use the new OS. Changing this decision will change its likelihood to zero and effectively eliminate it. An alternate option that may have less inherent risk is to make the application not be solely dependent on an untested OS, or delay that specific feature for a later release when the OS is stable. The second way to manage risk is by taking action to reduce the impact if the risk does occur. This involves starting the contingency in preparation for the risk. For example, to prepare for the potential loss of a key person, ensure that other team members become familiar with that person s work, or identify backup resources that can take over should the risk occur. Similarly, if a vendor s component is likely to be buggy, require that you run a serious of inspections with their development team to find defects prior to receiving the component. Further examples are shown in the risk management plan for the top three risks (Figure 2). The right side of the table includes additional columns for assigning and tracking the action items. Once you have developed risk management actions, decide which of them you are going to pursue. We suggest that you start by implementing a few actions for each selected risk. Focus on actions that reduce the likelihood of the risk, but also consider actions that provide the project with a contingency plan in case you are unable to prevent the risk. You may decide not to implement some actions because they are too costly. If the likelihood of the risk has not been reduced by the first action from the column labeled Actions to Reduce Likelihood of Risk Occurring, implement the next action from the column The Process Group Version 3 Page 23 of 42

24 Using the Risk Information Chapter 2 - draft The risk information is only useful if acted upon. There are four primary ways to integrate the risk information into your project schedule: 1. Move high-risk actions earlier or later in the schedule. Based on the risk information you have enumerated, change the sequence of the task in your schedule by moving high-risk actions earlier or later. In Figure 2, one of the identified risks was the learning curve of the new requirements management tool. The team can address this risk by ensuring that the tool is purchased and made operational earlier than originally planned. Alternatively, the team could delay purchase of the tool until adequate training is available. 2. Add more detail to the areas of the action plan that have the highest risk. High-risk areas can also be addressed by adding more detail to the related part of the schedule. For example, clarifying ambiguities in the requirements definition has less risk if it is broken up into more detail. The following subtasks help reduce risk: - Identify user classes (distinct types of users among the user population) - Identify business (high-level) requirements. - Enumerate critical user tasks that the product must support (i.e., use case names). - Write use cases to define details of each user task. - Determine quality attributes (e.g., requirements for performance and interoperability with other systems). - Inspect the requirements document with selected user champions and the project team to find defects. 3. Add more time to the areas of the action plan that have the highest risk. The highest risk areas of any project will most likely take longer than planned. These delays can have a ripple effect on the remaining project milestones. Adding more time to the highest risk areas is one way to counter this problem. The final project deadline would only be impacted if these high-risk improvement actions were on the critical path (the longest path through the schedule). 4. Add risk mitigation actions to the project schedule. During the risk session, team members propose mitigation actions for reducing risk likelihood and impact. Add the ones selected for implementation to the overall schedule. Step 6: Plan for Periodic Risk Review Reviewing your risks periodically will provide visibility on how well mitigation is progressing. During these reviews, determine whether any likelihood or impact numbers need revising, if new risks have been discovered, or if any risk is no longer relevant. Many people incorporate risk review into other regularly scheduled project reviews. You may decide to repeat the complete risk process if significant changes have occurred on the project. Significant changes include adding new scope, changing the target platform of the product, or changing project team members The Process Group Version 3 Page 24 of 42

25 Small batch principle to catch changes and problems on a small scale The small batch principle [Figure 5] is a technique to complete a part of work your work on a small scale to check for problems prior to finding that problem on the remainder of the work. Errors can then be found and fixed early. Changes in the scope can be comprehended as they occur causing less rework on previous completed work. Figure 5 The small batch approach allows for defects and rework to be done on a small scale For example, if you had 50 routines of code to write in a new language, write one or two of them and complete testing as far as you can before writing and testing the remainder of the code. Many unforeseen problems in using the new language can be found and resolved in a small scale with one small batch. Other examples include completing small batches of testing, translating legacy databases and inspecting project work products (code and documents). In each case, the small batch provides a cycle of learning on an inexpensive scale so that adjustments can be made. The small batch principle can either result in completed work product ready for a subsequent phase (e.g., release of a requirement to design, or code module to test) or it can be applied to dummy work to practice a skill (e.g., transmitting the text Hello World, with new wireless hardware across a city). For example, this book and our previous one were created using the small batch principle. Once an outline was established describing the topics to be covered, sections were selected to be published in trade journals and our newsletter. Taking sections through the cycle from beginning to end through all the required editing and proofing, allowed us to test the waters on a small scale. Feedback from these articles was used to refine the remaining copy. Inspection of defective documents and code Inspection is a time-tested method for quickly and thoroughly findings defects in any work product (code or document) [Humphrey89, Wiegers03]. It can be applied to any work product to find defects or to a sample of work products to determine how many defects might be remaining in the ones not examined. The life cycle shown (Figure 6) is for small incremental releases of functionality. Inspections have been inserted at specific points to find defects early The Process Group Version 3 Page 25 of 42

26 Requirements Design & Detailed Prototype Design Code Product Test Release & Manufacture Inspect Inspect Inspect Inspect Inspect Rework Errors Figure 6 Potential places in a lifecycle where work products are inspected for defects The inspection process consists of a team of 3-6 people focused on finding defects in any type work product. The inspection process has the following steps: - The work product owner selects a work product to inspect. This is usually 10-page section of text (equivalent to 6000 words) or a 300 lines of code (not including comments). For teams with large numbers of existing or new work products, start with a small percentage by focusing on the following sections: - The most critical section to the program's operation or project goal completion - The most used section in the product (e.g., the Open and Save commands, the installation guide) - The most costly section if defects were to exist - The most error-prone section - The least well known section (e.g., work product could have been created by a contractor) - The most frequently changed section - The work product owner invites a team of four to six peers of suitable technical background to be on the inspection team. - The work product owner invites a moderator to facilitate the session. The moderator is previously trained in the inspection process to enable the process to run efficiently without conflict among the team members. - A kickoff session is held for new inspectors to bring them up to speed regarding the inspection process. - The team is notified during the kickoff session of the suggested preparation time needed to look at the work product and defect severity scheme to use (e.g., major, minor). - The team members work alone on preparation over the course of a few days to find defects in the work product. - The inspection meeting is held a few days after the kickoff meeting. Team members briefly describe and then log each unique defect found in preparation. Inspectors are encouraged to find additional defects during the session. - After all defects are logged, rework is performed offline by the work product owner. Rework can be verified by selected audits, or via testing for code The Process Group Version 3 Page 26 of 42

27 [Author note: ref to a watts ch15 process and inspection data] Lessons learned to avoid repeating common or severe problems If you want to avoid recurring problems and surprises, conduct a lessons learned session and identify corrective actions to permanently lock in improvements. Lessons learned data comes from interviewing individuals across the organization or using a brainstorming session a specific project team. You can conduct a lessons learned session at any time; however, three specific times are particularly useful: when a significant project milestone has been reached, when a serious problem has occurred, or when a chronic problem reoccurs. You can use the agenda in Figure 7 to determine lessons learned and related corrective actions. As a rule-of-thumb, break the session into segments of two hours or less to avoid team fatigue. When using group interviews, construct the groups to encourage uninhibited discussion. Invite people who are willing to be frank and candid. Select a good objective facilitator, ideally someone not in charge of the project being discussed. Lessons learned agenda 1. Clarify the scope of the session. 2. Determine strengths (what went well). 3. Determine areas for improvement. 4. Set priorities. 5. Determine corrective actions. Figure 7 Lessons learned agenda. Here is an example of this process for a project. 1. Clarify the scope of the session. Capture the lessons learned for project X, requirements through first product delivery. 2. Determine strengths (what went well). Lesson 1: Lesson 2: Market requirements are initially defined, reviewed, and base-lined, resulting in a Product Requirements Document. Customer visits have clarified product requirements. Replanning meetings manage important requirements changes and address inconsistencies between requirements and schedule The Process Group Version 3 Page 27 of 42

28 Lesson 3: Lesson 4: Lesson 5: Chapter 2 - draft Initial schedules are based on effort estimates. There is a master high-level WBS and task completion and commitments are reviewed weekly and in re-planning sessions. Check-ins of code files to the integration branch are considered by the development lead to reduce risk related to coordination of dependencies, especially where there are many dependencies. System testing is robust and detects more defects than in prior releases. 3. Determine areas for improvement. Lesson 6: Lesson 7: Lesson 8: Lesson 9: Lesson 10: Lesson 11: Lesson 12: Requirements are not consistently updated (either at all or in a timely manner) allowing new quality-of-finish requirements from Marketing to be overlooked prior to test planning. Inadequate definition of project data to be collected, how to collect it, or how to roll it up, resulting in reduced visibility and inefficient program/project management (e.g., task granularity, dependencies, absolute task status). There are no defined techniques for effort estimation leading to some critical-path tasks being seriously under-estimated. Risk assessment and management is not actively performed at the team or system level, leading to preventable project-level problems and management surprises. Actual effort to achieve tasks is not tracked making future estimation inaccurate. Inter-project dependencies are not consistently identified or managed, resulting in too many surprises. Common code files are routinely edited by separate project teams in parallel, leading to time-consuming and error prone merges. 4. Set priorities. You can determine lessons learned priorities using the rating scales defined in Figure 8. Use a scale of 1 through 10 (1 = low benefit, 10 = high benefit) to estimate the relative benefits of maintaining each strength and implementing each area for improvement. Rating each item relative to the other items on the list allows you to order the list in the absence of hard benefit data. If you have data, such as the time or money saved by adopting the lesson, then substitute this data for the relative benefit number. Similarly, Figure 8 uses a scale to estimate the relative costs of maintaining each strength and implementing each area for improvement. If it exists, substitute hard data for the relative cost rating. The last step in prioritization is to order the lessons by the project phase in which they would be adopted. Figure 8 shows an example of this prioritization scheme with the last column stating the phase where the lesson would be considered The Process Group Version 3 Page 28 of 42

29 # Lessons Learned (Strength [S] or Area for Improvement [I]) Relative Benefit of Maintaining / Implementing Lesson Learned (1-10) Relative Cost of Maintaining / Implementing Lesson Learned (1-10) Priority (Benefit /Cost) Corrective Actions The Lifecycle Phase Where This Corrective Action Should be Used 1 Market requirements are initially defined, developed, reviewed, and base-lined, resulting in a Product Requirements Document. Customer visits have clarified product requirements [S]. 2 Replanning meetings manage important requirements changes and address inconsistencies between requirements and schedule [S]. 3 Initial schedules are based on effort estimates. There is a master highlevel WBS [S] No change Requirements No change Implementing: Bimonthly replanning sessions after initial planning phase No change Planning 4 Check-ins of code files to the integration branch are considered by the development lead to reduce risk related to coordination of dependencies. [S]. 5 System testing is robust and detects more defects than in prior releases [S] Continue checking changes to the integration branch to find errors. Long-term, delegate the activity to team members as part of the configuration control board Continue system testing. Next time, peer review test procedures to eliminate redundant tests, prior to testing. Implementing Testing 6 Requirements are not consistently updated (either at all or in a timely manner) allowing new quality-offinish requirements from Marketing Create a separate section in the product requirements document called Quality-of-finish 2005 The Process Group Version 3 Page 29 of 42 Implementation

30 to be overlooked prior to test planning [I]. 7 Inadequate definition of project data to be collected, how to collect it, or how to roll it up, resulting in reduced visibility and inefficient program/project management (e.g., task granularity, dependencies, absolute task status) [I]. 8 There are no defined techniques for effort estimation leading to some critical-path tasks being seriously under-estimated [I]. 9 Risk assessment and management is not actively performed at the team or system level leading to preventable project-level problems and management surprises [I]. 10 Actual effort to achieve tasks is not tracked making future estimation inaccurate [I]. 11 Inter-project dependencies are not consistently identified or managed, resulting in too many surprises [I]. 12 Common code files are routinely edited by separate project teams in parallel, leading to time-consuming and error prone merges [I]. Update the product requirements document before all re-planning sessions Define metrics to collect and capture metric definitions on a spreadsheet. Update spreadsheet at least weekly for all project teams within the programs. Meet weekly to verify data consistency Learn Wideband Delphi estimation process Implement risk assessment session on current program immediately. Communicate risk data in weekly status report and bi-monthly planning session. Revise risk list weekly in project team meeting Have team members report actual time expended per phase during the team weekly meeting. Record data in schedule tracking tool Peer review the schedule with the project team members and identity dependency errors Eliminate concurrent editing of code. Determine the sequence separate groups should edit the code. Filter all code change requests through the CCB. Chapter 2 - draft Weekly Planning and replanning Implementation Implementation Planning Implementation 2005 The Process Group Version 3 Page 30 of 42

31 Figure 8 Prioritization Scheme for Lessons learned Chapter 2 - draft 2005 The Process Group Version 3 Page 31 of 42

32 5. Determine corrective actions. If you have many lessons learned, consider developing detailed corrective actions only for the lessons that have the highest priority. Corrective actions include tasks to maintain a strength that is already in place and tasks to prevent the reoccurrence of a negative lesson. For example, lesson 12 in Figure 8 says, Common code files are routinely edited by separate project teams in parallel, leading to time-consuming and error prone merges. Tasks to address this include, eliminating the need for concurrent edits by determining the changes to the code that are required, defining the optimum sequence for those edits to avoid the need for parallel edits, and using a configuration control board to manage the changes. After establishing corrective actions, look for opportunities to define and capture these actions that cause them to be consistently performed. For example, add a checklist to the planning phase to ensure that the sequence of code edits is pre-planned, thereby avoiding situations where code needs to be edited by different teams at the same time. Failure to bake the improvement into the way the project team operates will increase the risk of repeating the same error in the future. Topic 3: Track project progress and take corrective action When your project is running, it is critical to know where you are, what work remains and a prediction of when you plan to finish. Tracking progress lets you know if you are likely to complete the work required by the desired deadline, provides visibility to detect problems early, and gives you data with which to negotiate changes. In this section we provide examples of how companies track progress and the corrective actions they take. Tailor these examples to fit your needs or use them as a starting point to generate your own tracking and corrective action ideas.5 Four measurements can help your team examine how well the project progressing: Compare the remaining available effort with the estimate Compare current progress with the critical path Compare current project size with estimated size Compare work complete with work planned Compare the remaining available effort with the estimate Earlier in chapter, we suggested that you compare the effort estimate of the project with the amount of available resources to determine if the project is overcommitted. Repeat this calculation as the project progresses since resource availability might have changed, new tasks (and associated effort) discovered, and initial estimates found to be inaccurate. Here is an example calculation for a project with half of its calendar days remaining and two resources reassigned at that time to other projects. Initial project effort estimate = 1,000 effort-days (10 people, 100 calendar days each) 5 When you are ready for more detailed information on project metrics, look at [Jalote, McConnell, Humphries PSP] 2005 The Process Group Version 3 Page 32 of 42

33 Re-estimate of work remaining on the 50 th day = 600 effort-days Chapter 2 - draft Availability of resources from today = 8 people for 50 calendar-days = 400 effort-days Amount the project is overcommitted by = = 200 effort-days In this example, corrective action is required since there is no obvious way to recover 200 effortdays simply by working a few weekends. (To recover, each person would have to work 25 additional days over the 50 remaining calendar days. This would require each person to work 12 hours for every 8 hours planned, without a break for 50 days. This also assumes that few mistakes are made, which is unlikely, and that no one else leaves the project.) Corrective actions would include the following options: - Understand what need the deadline is based upon and determine if it is acceptable for the deadline to slip. - Check your assumptions with others looking for options that might exist. For example, can the system work only with database version X and not the competitor s database. - Examine the critical path of the schedule. Verify if any other dependency is late, giving you more time. - Clarify resource availability. Ask whether other resources can become available (from inside the company or contracted from outside). - Clarify the customer s quality expectations and clarify whether the product is being overengineered for the current release. Do you really need that web-publish feature? - Identify sections of the product that could be simplified and still meet customer expectations. Involve all project team members in this analysis since they will more likely have this insight. - Investigate process changes that could relieve deadline pressure: For example, could the unit test procedures be peer reviewed (inspected) to identify redundancies prior to test, or could the upcoming customer approval sessions be simplified by replacing the presentation materials (being created by the team) with the customer logging into to the network and reviewing the current design document and schedule, followed by a shorter meeting with the customer to answer questions? - Determine sections of the project that should be purchased or built: What work has been delegated that could be done quicker in-house? What work is in-house that should be delegated? - Propose an incremental delivery with lower-priority features can be delayed. Admitting and communicating that a significant re-plan is required takes much courage. As you debate when and how to tell management and the customer about the project s current status, remember that the earlier you say something, the more time you and they have to take corrective action. Telling them the day prior to the deadline is the surprise they definitely do not want. Compare current progress with the critical path The critical path is the longest calendar path through the schedule and comprises of the sequence of tasks that have no slack [ref]. A slip in any one of these tasks will cause a ripple effect on the final 2005 The Process Group Version 3 Page 33 of 42

34 deadline. In the schedule example [Figure 9], tasks 1, 3, 6, 7 and 8 are on the critical path. The other noncritical-path tasks can slip a considerable amount before becoming critical-path tasks. Task 1 6*40; Jim, Task 3 6 person wks (4 weeks) Task 2 5*40; 5 person wks (20 weeks) 24*40; Task 4 27*40; Jim, Jane, Pete 75% 27 person wks (12 weeks) Task 5 12*40; 40%, 12 person wks (10 weeks) Task 6 15*40; Jane@40% Jim, Pete@80% 24 person wks (36 weeks) 15 person wks (7.5 weeks) Task 7 18*40; Jane@40% Jim, Pete@80% 18 person wks (9 weeks) = Critical Path 66.5 calendar weeks Task 8 24*40; J, J, 24 person wks (10 weeks) Task name Effort hours (#weeks x #hours per week) Resource names and availability Person-weeks to complete task (Calendar weeks to complete task) Figure 9 Example schedule showing the critical patch Identify the critical path of your schedule in a scheduling program such as Microsoft Project. As the project progresses, check that the tasks on the critical path are on track. If the completion of a critical path task is late compared to the original prediction, then consider the following corrective actions: - Adjust resource availability. Tasks can usually be completed quicker if the time allocated to them is increased. In Figure 2, Fred s work on Task 2 can be delayed until the completion of Task 3, and his allocation of 66% on Task 3 increased to 80%. This would cause a six-week reduction in the duration of the task. Since Task 3 is on the critical path, the final deadline would come in by six weeks also. - Reduce or eliminate transitions between tasks. Focusing Fred on one task at a time not only reduces the deadline in this case, it reduces the time spent transitioning between Task 2 and task 3. The transition time comprises of the time spent repeatedly putting one task down and picking up the next, the learning curve to become productive at a task, potential travel between locations, and errors made due to interrupted focus The Process Group Version 3 Page 34 of 42

35 - Reduce or eliminate disruptions: Reducing or eliminating disruptions also increases resource availability. In the course of a normal day, team members are usually interrupted continuously via scheduled and unscheduled meetings, customer support, support of non-related projects and . While these activities represent real-life that goes on, they can be reduced, and sometimes eliminated with some thought and discipline. For example: - Buggy products that consume time can be fixed once and for all. - Multiple meetings that cover similar ground can be merged. - Meeting durations can be halved with better preparation and meeting moderation. - Recurring project status briefings to customers and managers can be replaced with less frequent briefings at major milestone and periodic reports. - Team members can hide and be unavailable half of each day (similar to being unavailable via family vacation or surgery, where hiding is expected). - Team members can decline to be in the loop on non-related project and distribution lists. (Do you really need to be ed when every network server goes down and comes up!) - Employees can avoid checking and cell phones for N hours each day (similar to being on an airplane, playing basketball and being the focal point during a surgery). - Reduce the number of parallel releases: one source of the transition tax is trying to work on too many releases at the same time. Writing code for the current release, while fixing the one stuck in test, at the same time as fixing the one in the field, while working on the design for the next one, in parallel to working on the requirements for the one after that sounds appealing, but is a symptom of poor management and learned-helplessness. Look at your project and select two releases to work on in parallel, for example the current release and the next. If there are numerous bugs in the released product, take time out now to clean it up (unless of course if you enjoy the constant stream of complaints and impossible situation of having to factor in changes to multiple code branches!) You might be tempted to say, But we must work on 100 releases at the same time, that s what the market demands! No, you fell into it. You can chose to work in two releases well, or 100 poorly. It s just a choice. - Manage suppliers that are on the critical path: If you have suppliers, the critical path can show which ones will cause a ripple effect if they are late. Suppliers in the schedule that do not link to a critical path task can be late with no immediate impact. Use this information to identify suppliers that need more thorough oversight, and ones where you can be more flexible. In the example in Figure 10, supplier A can be late by 24 weeks before they impact the critical path. Supplier B cannot be late any amount without causing a ripple effect. If you have supplier dependencies connected to the critical path, consider the following actions: - Contact them periodically to verify that their promised delivery date is still true. - Request a copy of their development plan (or schedule) to gain visibility into work progress. 6 6 If the supplier is competent, it is likely that they will want to showoff some aspects of their planning and scheduling ability. Poor supplier performance and refusal to share their plans might indicate a lack of ability to plan. In subsequent contracts, make the early delivery of a project plan a contractual requirement The Process Group Version 3 Page 35 of 42

36 - Obtain early or demonstration copies of deliverables to verify that they are on the right track. Early deliveries allow opportunity for feedback and additional recovery time for correction. - Request a risk management plan, or participate with them in developing a risk management plan (similar to the risk management activity discussed earlier in Figure 2). - Develop contingency plans in case the supplier is late or does not deliver at all (e.g., alter your product to work without certain features, make it work with previous releases of the supplied components, or identify a possible second source of the component). Supplier A Task 1 6*40; Jim, Task 3 6 person wks (4 weeks) Task 2 5*40; 5 person wks (20 weeks) 24*40; Task 4 27*40; Jim, Jane, Pete 75% 27 person wks (12 weeks) Task 5 12*40; 40%, 12 person wks (10 weeks) 24 person wks (36 weeks) 15 person wks (7.5 weeks) Supplier B Task 6 15*40; Jane@40% Jim, Pete@80% Task 7 18*40; Jane@40% Jim, Pete@80% 18 person wks (9 weeks) = Critical Path 66.5 calendar weeks Task 8 24*40; J, J, 24 person wks (10 weeks) Figure 10 Using the critical path to determine which suppliers to monitor the closest - Manage risks that impact the critical path tasks. Use the risk management process described earlier in this chapter to identify risks that impact the critical path. The whole critical path, or individual tasks on the critical path, can be assessed as risks and actions taken to reducing the likelihood and impact of the risk. - Re-establish scope or deadline: A typical threshold for re-planning occurs when the time lost is 15-20% of the total schedule or the effort expended is 20-35% more than the last estimate [Jalote p122]. At these points, recovery is usually wishful thinking and it is time to escalate the problem The Process Group Version 3 Page 36 of 42

37 Compare current project size with estimated size Chapter 2 - draft [Author note: edit size section with march05 article changes] Tracking project size attributes provides an early warning system for problems with project progress, project growth and stability. Size data can be used to validate the likely completion date of projects and provide the team and management with a visual picture of the project s magnitude. Here, we provide some examples of how companies track size. When you are ready for more detailed information on project metrics, look at the books in the reference section of this article. In some texts, such as the Software Engineering Institute s Capability Maturity Model Integration (CMMI), size is referred to as a Work Product Attribute. This is a generic term that can be applied to any work product, including intermediate work products (such as a design document) and final work products (such as code and assembled hardware). When learning about size attributes, it is tempting to assume that My group is different, and that no size metric is applicable. It is also common for software developers to assume that size must mean lines of code (LOC), and that LOC is never an applicable metric. If you have this filter, know that size is simply a parallel view into the amount of work a project has to do, one that can be used to estimate effort and monitor progress. The absolute accuracy of the metric is not as important, as long as the metric changes proportionally in value when the amount of project work changes. Five example size tracking metrics are summarized here. Each one provides a different window into the project. They are, requirements count, requirements completed, requirements growth, cumulative requirements changes and units coded. Requirements count The unique requirements of a project can be counted, and changes to that count plotted over time. A prerequisite to counting requirements is to split ambiguous paragraphs of requirements into unique requirement statements. The precision of the count is not as important as knowing the magnitude of the requirements to be implemented and observing changes to the count over time. If the requirements are not uniform in scope, assign a complexity or weighting factor to make the requirements count larger when requirements are complex. In Figure 11, the requirements count line starts at 60 requirements, and increases late in the project. At this time, a re-plan is likely to be needed The Process Group Version 3 Page 37 of 42

38 Figure 11 Tracking requirements metrics, based on [SEI92] Two other examples of metrics to count requirements are the use case points approach [Jalote02] and counting the number of screens on a user interface. The use case points metric is based on the number of use cases, their complexity and factors including the stability of the requirements and the experience of the team. The metric number is then multiplied by 20 person-hours to derive an initial project effort estimate. This number, and the effort expended on the project, is then tracked as the project progresses. The second metric uses the number of screens in a user interface. A count of the screens provides a rough order of magnitude of project effort (e.g., 1 screen, 10 screens, 100 screens). If the effort required to complete the project increases when the number of screens increases, then the screen count is a suitable size metric. In the beginning of the project, the metric is used to validate the effort required by multiplying the number of screens by the average effort required per screen. If the number of screens changes, one can expect the effort or schedule to change also. If the screens have a wide variation in complexity, assign a weighting factor. Requirements growth Monitor changes in scope by counting the revised number of requirements (the Requirement count in Figure 11.) This line shows when there might be a schedule risk if more requirements are added to the project. The project might be able to absorb some new requirements, but at some point, the work associated with these requirements will exceed the established schedule and budget. The line labeled TBDs (To Be Defined) is a count of the placeholders in the requirements document where requirements need to be clarified. Initially, this can be included in the initial count. As the project progresses, this number should decrease, showing that the ambiguity of the project is decreasing The Process Group Version 3 Page 38 of 42

39 Requirements completed Progress can be tracked by counting the number of requirements that have been completely implemented (see the line labeled Requirements completed on the graph in Figure 11). Although this is a gross measure of progress, it is one that shows how far along the project is from a customer and releasereadiness view. The validity of the metric is dependent on the definition of a requirement being complete. A complete requirement might be one that has been successfully tested. Cumulative requirements changes The line labeled Cumulative requirements changes monitors the volatility of the project requirements. A project team should expect requirements to change as they are clarified. However, many changes late in the project add the risk of the team not being able to complete any feature, or only having limited time to test the features that are complete. This line can also indicate a situation where there are so many changes, that the team is effectively starting over with a new set of requirements. Units coded Units coded is one example metric that provides better visibility when software is being developed. (Similar metrics for other phases would be designs complete, tasks complete, tests complete and defects repaired.) In the graph in Figure 12, the project initially estimates that there will be 60 units of code to create. As the project executes, the number of modules actually complete (here, complete is defined as peer reviewed, tested and under configuration management) is tracked on the graph. Tracking is kept simple in this example by assuming that a straight line reflects the desired progress between the start and end of the project. Planned Actual Now Units Coded Total units to be coded Re-plan based on current performance. More resources could have been added. Original baseline New baseline 10 Actual Week Figure 12 Simple trend graph showing overall project size and progress for the coding phase At week three, the team members notice that their rate of progress (see the line labeled Actual) is approximately half of the initial planned rate. This causes a re-plan and a new baseline. The graph can also be used as an early warning system of potential schedule problems. For example, if the team members discover new code that needs writing, this new estimate can be added to 2005 The Process Group Version 3 Page 39 of 42

40 the graph. The change can be seen in context to the overall project work estimate, and any upwards or downwards trend can be detected. The downside to a trend graph is that it assumes all work is equal in difficulty and duration. The Earned Value concept discussed later addresses this deficiency. Selecting metrics You will not always be able to have one size metric that is valid for the duration of the project. Specific size metrics might be more useful by project phase. For example, you might measure the number of requirements baselined (requirements that have been defined and agreed upon) during the requirement phase, and the number of repaired test defects during the testing phase. Compare work complete with work planned Earned value tracking is another way to evaluate project progress. It establishes a relative value for every task and credits that value when the task is complete [Humphrey95]. The earned value system provides a common value scale for every task, regardless of the type of work involved. The total hours to do the entire project are estimated and every task is given an earned value based on its estimated portion of this total. Partially completed tasks are not given a partial credit. The earned value is only given when the task is completed. If the task is half-done, it contributes nothing. If you have a task that is so big you want some intermediate measures, break it into subtasks. The only requirement is that every task have explicit completion criteria. When these criteria are met, the earned value for that task is credited. The graph in Figure 14 shows a simple example of earned value based on the schedule milestones shown in Figure 13. Task Effort Planned PV EV SPI = EV/PV 2005 The Process Group Version 3 Page 40 of 42 AV CPI = EV/AV Budget Week Start Set Up Get Specs Design Output Plan Tests Code Code Unit Test Integrate Beta Test Total 31 Figure 13 Example Earned Value Table The line labeled Budget shows the total number of days effort (31) expected for this project. The line labeled Planned Value (PV) shows the prediction of the project s progress over time. From the Figure it is clear that the tasks Set Up and Get Specs should have completed by the end of week 2. The task Design Output, starting in week 3, has an effort of 10 days and is due to be complete by week 4. Since there is no partial credit given, at week 4 the PV line jumps to 15.

41 The line labeled Actual Value (AV) represents the number of hours actually being spent on the project each week. It gives a view of the amount the team is under- or over-committed. The line labeled Earned Value (EV) reflects the tasks that have been complete. In this example, the project is stalling on the second task. Since the AV line is increasing (which means the team is active on this task), then we can assume that they have hit a technical difficulty or that the task has been underestimated. The column in Figure 13 labeled SPI (Schedule Performance Index) shows the task completion rate of the project. In the beginning, SPI is calculated as 1.0, which means that the project is executing tasks at the planned rate. As soon as the project falls behind, the SPI index drops below 1.0. Here, SPI has dropped to 0.6 and is not shifting up, even though more hours are being spent on the task. This is a significant problem, and one that is worthy of a replan. SPI is a useful check of the current schedule speed of the project. Anything over 1.0 is good (but might be unsustainable). The column labeled CPI shows how much additional effort is being expended over time compared to the effort planned. On the task labeled Setup, the project spent two thirds more effort than planned. The estimates for the second and third tasks are less accurate. All three graphs combined mean that the project is overcommitted from the outset and will likely not be able to recover without a replan. Figure 14 Example earned value graph 2005 The Process Group Version 3 Page 41 of 42