Importance of Stable Velocity in Agile Maintenance

Transcription

1 2013 XXIV International Conference on Information, Communication and Automation Technologies (ICAT) October 30 November 01, 2013, Sarajevo, Bosnia and Herzegovina Importance of Stable Velocity in Agile Maintenance Samir Omanovic, Emir Buza Department for Computer Science and Informatics Faculty of Electrical Engineering Sarajevo, Bosnia and Herzegovina Abstract Agile maintenance is the best choice if you want to keep step with your customer needs. It is a result of trying to respond to customer change requests with the high efficiency. High involvement of the customer in the maintenance process is good but also can have negative effects. Change in the behavior of the customer can influence the execution of the change management process or cause the change of the release plan, etc. All that can destabilize normal maintenance velocity and lead to a chaotic relationship with the customer, if not controlled or prevented. This paper describes problems in agile maintenance caused mostly by the change of the customer behavior at the beginning of the economic crisis. It also presents results of the analysis of these problems and recommendations how to identify them and how to prevent them. Keywords agile maintenance; development velocity; software engineering; agile software development I. INTRODUCTION Agile software development proved its power in practice on many small and middle size projects (example success stories [1][2][3]). Later the agile idea is introduced in the maintenance [4][5]. Our practical experience showed that the agile maintenance is the best choice if you want to keep step with your customer needs. Agile maintenance is a result of trying to respond to customer change requests with the high efficiency. High involvement of the customer in the maintenance process is good but also can have negative effects. Change in the behavior of the customer can influence the execution of the change management process or cause the change of the release plan, etc. All that can destabilize normal maintenance velocity and lead to a chaotic relationship with the customer, if not controlled and prevented. Here are described problems in agile maintenance caused mostly by the change of the customer behavior at the beginning of the economic crisis. Results of the analysis of these problems and recommendations how to identify them and how to prevent them are also presented. II. AGILE METHODS The agile software development principles can be found in the Manifesto for agile software development [6]. Manifesto emphasizes: high customer involvement, software, responding to change, individuals, and communication. This means that people, software and business are in the forefront, and that documentation, processes, tools, plans and contracts are in the background. This is very important for understanding the agile philosophy. Agile methods are trying to fix delivery time and resources while number of functionalities is variable. This is opposite comparing to plan-driven development where functionalities are fixed while delivery time and resources are variable. There are many agile methods. Some of the most popular are: Scrum [7], Extreme Programming (XP) [8], and Dynamic System Development Method (DSDM) [9]. If we want, in short [10], to describe Scrum as an example of agile method then we can say that it is a framework for completing complex projects. Prioritized features list for the product (wishes of a product owner - person that has a vision of what they wish to build and that he or she is able to convey that vision to the scrum team [11]) is called a product backlog. Sprint is an iteration of software development. The software development team takes a small chunk from the top of the product backlog and creates a sprint backlog and a sprint plan. The team within the planned time (usually 2-4 weeks) completes its work and assesses the progress on daily meetings (daily scrum). The ScrumMaster - a person that is something like a coach of the team, helps the team to do best work they possibly can - keeps the team focused on their goal. At the end of the sprint, the team should pack its work in a potentially shippable product that can be handed to the customer or be a base for the next sprint. At the end of the sprint review and retrospective are performed. Next sprint can be started after that. One feature on the product or sprint backlog can be seen as a description of that feature from the customer s perspective. It is named a user story [12]. Effort estimation and resource allocation are created for each of user stories, during sprint planning. Also, for each user story is possible to create a specific test which proves that user story is properly implemented. This is important as a reference of what customer expects to get at the end. Sprint backlog can be seen as a list of features that will be included in a next version of the product. When planning sprints execution, one of the most important parameters is a velocity how much product backlog effort a team can handle in one sprint [13]. If this parameter does not change too much in sprints (is stable) then that means that the team, sprints planning and sprints execution are well aligned. III. AGILE MAINTENANCE A. Agile in Maintenance is Possible? Maintenance is traditionally seen as a plan driven activity [14] and the idea of being agile in that does not sounds well. But each day there is more people that accept the idea of the agile maintenance as something that has some advantages /13/$ IEEE

2 [4][5] comparing to traditional approach. Our experience showed that applying the agile maintenance is not so complicated and that main advantage is in a quick response to customer needs (fast aligning with customer s plans and his vision). B. How to be Agile in Maintenence Maintenance can be seen as an endless agile project whose product backlog is changing constantly new user stories are added on the list and implemented user stories are removed from the list. User stories are submitted and prioritized by the customer (Product Owner role; have a business vision). In maintenance, implementation of each user story is not unconditionally, since the customer must approve the price of the implementation. Set of user stores with the top priorities are implemented in a one scrum and a new release is created as a result. In that way change management process becomes agile. Communication with the customer is intensive and the customer is highly included in a development process. This means that people, software and business needs are in the forefront. Most of the changes of software in the maintenance are not highly interrelated and are mostly nice to have features rather than must have features. This means that it is not complicated to change release plan even during sprint execution. IV. CASE STUDY A. Change Management Process Customer support [15] is based on maintenance contract which also defines a change management process [16]. Maintenance contract with our customer defines the help desk support of 90 hours per month (for small reconfigurations, database support, simple report changes, and other support that do not require software change longer than 2 engineer-hours), defines the price of the engineer-hour and defines the change management process as presented on fig. 1 for all change requests (adaptive and corrective) not included in the help desk support. Change management process is initiated by the customer when he submits a change request (CR). Submission is performed using the change management system (for example [17] or [18]). Submitted CR must contain name, detail description (user story), who is initiator at the customer side, date of submission, priority and expected delivery date. This actually forms the product backlog. Change management system notifies the customer support department which accepts the CR and performs preliminary analysis of the CR. If CR sounds well then customer support assigns a developer that is available at the moment and has the best knowledge needed for the implementation of the given CR. If CR is not enough described, or there is some other CR with the same request, or the change is not necessary since there is a needed functionality but the customer did not know that, then CR is rejected and closed. Assigned developer to the CR will receive the notification about that and then start different analysis. For most of small to middle size CRs one developer will do all activities until the CR closure. Only for large CRs or those which requires specific knowledge, more than one developer is included in solving them. At the first, the developer will analyze the assigned CR and if necessary he will communicate with the customer to clarify the description for better understanding. He must answer the question: What should be done? After that, the developer will analyze how to implement the change. Here he must answer the question: How to change the software? Every change of one software module can have an influence on other software modules and cause their failure or improper behavior. For that reason, the developer must analyze the impact of that change [19] on other software modules in the system. After all analysis the developer estimates necessary effort [16] for that CR. That effort directly defines the price (cost) of the implementation of that CR. The overhead [16] costs are included in the price of the engineer-hour defined in the maintenance contract. For corrective change requests price for the customer is zero, but the change management system must keep record of real costs for later analysis. In the case of adaptive CR, the customer will analyze the price and may decide to reject that change because it is expensive. When customer approves the CR cost estimation then developer will receive a notification to start implementation. Developer implements the CR and after that he tests that implementation. In the case that it fails on tests he repeats steps of implementation and testing. After passing tests the developer will prepare a validation platform and instructions for the customer that will perform the validation [20]. Customer is obligated to finish validation within 15 days otherwise CR is treated as valid and is automatically a candidate for closing. Customer performs the validation to check does implemented solution meets his requirements specified in the CR. When the customer confirms that the CR implementation is valid, then that change is included in the new release of the software system. Here is very important agile flexibility and communication with the customer. In the case that CR implementation does not pass the validation then it is not included in the new release of the software system. Customer pays implementation of the adaptive CR when he confirms that the CR implementation is valid, or when do not finish validation within 15 days, or when he realize that the CR is not well prepared by his side. B. Change Management History Maintenance contract that includes the change management process as is described above is signed in year Until the beginning of the economical crisis in year 2009 change management process was very stable and both sides were very satisfied. By analyzing the history of change requests it is possible to present following indicators: On average 10 adaptive CRs per month were implemented with average effort estimation of 20 engineer-hours per one CR 90% of CRs passed the first validation.

3 Fig. 1. The change management process and issues identified in the analyses.

4 Objective Estimated Achieved Before I Time Ta Time Tb Time Tc Contingency After II Time T1 Time T2 Time T3 Recapitulation (I: T a <T b ) (II: T 1 >T 2 ) (I: T a <T b & T b >T c ) (II: T 1 >T 2 & T 2 <T 3 ) Fig. 2. Time estimation before and after beginning of the economical crisis. 9% of CRs passed the second validation. For 1% of CRs customer decided that the CR was not correctly formulated or there were more than two validations. Absolute deviation of the effort calculated after the closure of the CR and the effort estimated was less than 10%. On average for all CRs it was near zero. Customer satisfaction was high, since his business was benefiting from implementing CR. Developers were not too much pressured by deadlines or decreasing costs. C. Customer Bahvior Change Customer s business is growing and requires introducing new methods and technologies to increase profit or preserve current positive trends. When the economical crisis started in year 2009 the customer decided to cut expenses. First they introduced the balanced scorecard [21] in the strategic planning and management. They analyzed business processes and decided to make some of them more automated; they started more frequently to change their plans and made many other changes in their behavior. Consequences of that were: Increased number of CRs (they wanted more automation). Increased size and complexity of CRs (they wanted to replace human workers in some processes with the software and assign those workers to other processes; more unstructured problems). Customer started to bargain over cost estimations. etc. D. Reaction on the Customer Behaviour Change At the beginning there was no reaction. After few months it was noted that developers are working more and that results are not good as before more complaints from the customer. It was decided to analyze this situation. The analysis compared the change management history data and CRs that are initiated after the beginning of the economical crisis. It was noted the following: On average 15 adaptive CRs per month with average effort estimation of 30 engineer-hours per one CR. This means increased number of CRs and increased size of them. Increase in size is mainly caused by

5 defining CRs on large (fundamental) processes and by increasing the complexity of CRs since the customer tried to automate activities where humans were involved. Only 60% of CRs passed the first validation. This means that it is decreased the quality of the development process and that developers are losing time inefficiently. 25% of CRs passed the second validation. This caused that more and more CRs were not delivered on planned time. 10% of CRs were validated more than two times. Previous and this item caused that the customer had the feeling that the software is the reason why he cannot fulfill his plans. For 5% of CRs customer decided that the CR was not correctly formulated. This just added more chaos in the relationship with the customer. Deviation of the effort calculated after the closure of the CR and the effort estimated was between 0% and 20%. On average this means losing approximately 10% more time comparing to the previous period. Figure 2 shows the identified reasons for this more iteration mainly in the analysis and testing. Customer satisfaction was moderate, since his business plans were not fulfilled as he expected. Developers were highly pressured by deadlines and decreasing costs. E. Influence on Change Management Process The change management process was not executed as it should be. Figure 1 show some of the most important points in this process that were identified as potential places for corrective actions (see comments in yellow boxes). We identified the following issues that require some corrective actions: Issue #1: Increased number of change requests and decreased quality of change descriptions. We realized that the customer is trying to cut his expenses by replacing humans in some processes. That is why he needs more automated processes and that is why there are more requests. Usually, in the initial software development, activities that are too complex for the automation are left to be executed by humans. Decreased quality of the change description is caused by the fact that they are describing complex activities more complex than before (activities that involve user experience; more unstructured problems). For example, in the normal process there is a person that controls printed documents (given prices, discounts, calculation, alignment with the electronic version in the system etc.) since that document is an equivalent to the legal contract and it is very important that there is no any mistake. If some document is not correct or not aligned with the version in the system then that person changes the document in the system, prints a new document if necessary and documents the error. The customer requested to change this process and replace this person by the automated control. Description of this request is not so easy. It is easy to describe what a creator of document should be able to select or enter (constraints) in one field of the document unrelated to the document context, but it is hard to describe constraints on one field in the context of the document. In that case the user story can be very complex and not cover all aspects of the process. Issue #2: Pressure to reduce costs resulted in reducing analysis. Developers were aware of the economical crisis that the customer is trying to reduce costs. Since they have a very good relationship with the customer and wanted to keep that, they started to decrease their estimates by reducing or eliminating contingency time [16] in the estimation (see fig. 2) which was usually between 10% and 30%. As a result, they actually compensated that by decreasing the time of analysis and testing. A consequent of that was more unsuccessful validations. All that resulted at end in more time spent than before. Issue #3: Pressure to reduce costs resulted in reducing tests. This issue is caused by same reasons as issue #2, but is separated since it happens at the end of the development process and should be seen as separate point where some corrective action can be applied. Issue #4: Increased number of unsuccessful validations. This is actually a consequence of the increased complexity of CRs and reduced analysis and testing (previous issues), but is separated as a potential point for corrective actions. Issue #5: During the validation customer realizes that CR is not correct and needs to be changed. Although this issue is more related to the customer, it is necessary to try to decrease these problems so that the customer returns his high satisfaction and feeling that software adequately supports his business. This issue can be improved with corrective measures for issue #1. Issue #6: Development did not understand CR properly. Customer support and developers were not aware of the increased complexity of CRs and they applied the same amount of analysis as before. This caused that user stories do not cover all aspects of the process. Usual mistake was that user did not specify some constraints or specified them imprecisely. This was usually detected during the validation. As a consequence of that there was some arguing with the customer who did mistake. This issue is very important for the good relationship with the customer. It is better to accept that development should clarify these things then to say that the customer is wrong and break the relationship with the customer. Issue #7: For larger CRs the customer negotiates about the cost estimation (not in a normal protocol). In his trying to cut expenses the customer changed his

6 behavior in the point where he was deciding about CR cost estimation approval. He started to ask for discounts in the way that he ask developers to decrease their estimation. This was especially emphasized with larger CRs. In trying to satisfy customer developers sometimes decided to do that. They were not aware that by doing that they cause other problems. End Users Requirements Time (respect agile principles, see fig. 4). In either case, the customer is not satisfied and developers are under pressure. Functionality Traditional Fixed Resources Time Variable Agile Fig. 4. Agile vs traditional principles (based on [23][24]). Functionality Resources Items Determining priority of requirements Sprint - 1 Sprint - N Before I After II Items Days? Days Fig. 3. Selection of user stories for the sprint backlog. F. Influence on Agile Maintenance Aspects The identified issues can be analyzed in the context of the agile maintenance principles. In that case following problems can be identified: Sprint planning [22] is affected. Involvement of the customer in the agile maintenance is intensive. The customer by setting priorities and deadlines has a high influence on forming the sprint backlog more user stories in the sprint backlog than is normal. Figure 3 shows the process of selecting user stories that will be included in one sprint. When sprint backlog is overloaded then you have a choice to delay the delivery break timeboxing (break agile principles [23][24], see fig. 4) or not deliver all functionalities Fig. 5. Burndown chart before and after beginning of the economical crisis (based on [25]). Burndown chart [25] shows that it is not possible to finish all user stories in the sprint backlog. Possibility is to increase the number of developers (resources), but that also is not aligned with agile principles (see fig. 4). Usual burndown chart before and after beginning of the economical crisis is presented on fig. 5. It shows that before the crisis the team was not under pressure and that there was a free time in the sprint. This was good for the team. For the company this means that the team could do more but this time was less than 10% of the total sprint time and was tolerated. Figure 5 also shows that after the beginning of the crisis the team was overloaded and was unable to finish all user stories in the sprint backlog. This brings a question

7 how to get out from this situation. It is not a good solution to break agile principles in trying to solve this. Velocity [26][27] in the agile software development is a term used to illustrate the rate of the progress for the team or group of teams. It represents the number of user stories in one sprint, usually expressed as number of story points [27] in one sprint. This parameter can be observed on the burndown chart. The biggest problem in planning is predicting the team s velocity. One of the easiest approaches is to base prediction on the history data - yesterday's weather method [27]. Before the economical crisis the development velocity was stable, but after the beginning of the economical crisis velocity was destabilized. When the velocity was destabilized then the velocity prediction for the next sprint became problematic predictions were inaccurate. All above mentioned aspects of the agile maintenance are interrelated and affected. Many other aspects can be discussed also all of them are more or less affected, but it is not necessary to discuss them in the context of this paper. The most important parameter, from our perspective, is the velocity. The velocity is the parameter that can be easily observed and can be the first indicator of problems. G. How to Stay Agile in Maintenance The case study shows us on agile maintenance aspects that it is easy to break agile principles in the case of a disturbance initiated by the customer, since the customer is highly involved in the change management process. On the other side, involvement of the customer is important, not only for the agile maintenance process, but also for the overall relationship with the customer and his high satisfaction. So, the main question is how to preserve the agile principles, customer satisfaction and be able to adapt to huge disturbances generated from the customer side? In each business relation there are interests of both sides. But, the quality of the maintenance process is a mutual interest. The customer wants to have all necessary functionalities on time and with the proper characteristics to support his business and gain him a profit. Interest of the software development company is to be able to satisfy customer s needs and be reasonably paid for that. What happens in the crisis is that the customer emphasizes his interest and that the development company is trying to absorb all problems generated by that. The first key factor in preserving agile principles in the maintenance process is that the manager of the software development company understands importance of balancing interests of both sides we named it agile maintenance balancing (see fig. 6). When the customer generates disturbance that affects the agile maintenance principles then the manager should analyze the problem and decide what to change in the maintenance contract with the customer. In this case, the customer had to confirm that he will need more support. A new employee is included in the development team and included in the next scrum planning, based on that. That helped to stabilize development velocity. It is very important to understand that amount of support per month should not vary too much and must not exceed limits that team can handle. Fig. 7 shows a tolerable area on the burndown chart in the agile maintenance. Customer Business planning Fig. 6. Agile maintenance balancing. Items Days Fig. 7. Burndown chart that shows a tolerable area in the agile maintenance. The second key factor is to be involved more in the customer s planning process. When the customer analysis his business processes and decides to change some things he is not aware of the complexity of that change from the software engineering point of view, and that is why he puts in his plans unrealistic priorities and deadlines. In this case it is very important that the software development company behaves as a consultant to the customer so that he makes plans that are more realistic. This consultancy is not a part of the change management process but it can be part of the help desk support. If customer s plans are realistic then there will not be any disturbance generated from the customer side. The customer will be highly satisfied since his plans are achieved. CONCLUSION Software Development Company Sprint planning Balancing planning, interests and involvement into the planning of the other side. Customer satisfaction is the most important factor for the manager of the software development company, but this can be a trap if manager is not aware that customer s behavior change cause an imbalance in the relationship. Developers will be the first who will detect this problem through the velocity instability, as is described in the above case study. It is very important that developers communicate this problem to the manager in the way that manager

8 understands that balance in the relationship with the customer must be established and preserved. Returning to balanced relationship with the customer will stabilize development velocity. Stable development velocity is the most important process indicator for developers in agile maintenance. Balanced relationship with the customer means that the amount of the support per month should not vary too much and must not exceed limits that team can handle (see fig. 7). The team must have a possibility for working under the same pressure with small fluctuations of ±10%. It is necessary that the customer accepts higher involvement of the consultants from the software development company in his business planning related to the software change process. The customer must be aware that the alignment of the business planning and sprint planning is mutual interest that will help in preserving stable development velocity and high customer satisfaction. REFERENCES [1] J. Coyle, Agile Software Development and Project Risk Management, [Online], Available: April [2] K. Petersen, C. Wohlin, The effect of moving from a plan-driven to an incremental software development approach with agile practices - An industrial case study, Springer - Empirical Software Engineering, 2010, Volume 15, Issue 6, pp [3] D. Sato, D. Bassi, M. Bravo, A. Goldman & F. Kon, Experiences Tracking Agile Projects: an Empirical Study, Springer - Journal of the Brazilian Computer Society, 2006, Volume 12, Issue 3, pp [4] S. Darbha, Can Support and Maintenance Become Agile?, [Online], Available: April [5] M. Pronschinske, You can t be Agile in Maintenance?, [Online], Available: [6] Manifesto for Agile Software Development, [Online], Available: [7] Home of Scrum, [Online], Available: [8] Extreme Programming, [Online], Available: [9] DSDM Consortium, [Online], Available: [10] The Scrum Framework in 30 Seconds, [Online], Available: [11] Learning Scrum - The Product Owner, [Online], Available: [12] Scrum User Stories, [Online], Available: [13] Glossary of Scrum Terms, [Online], Available: [14] Software Maintenance, [Online], Available: [15] Customer support definition, [Online], Available: [16] I. Sommerville, Software Engineering, 9th ed., Addison-Wesley, 2010, pp , [17] Serena Business Manager, [Online], Available: [18] Bugzilla, [Online], Available: [19] KAIST SE LAB, Change impact analysis - what, why, how?, [Online], Available: [20] Software Verification and Software Validation What s the Difference?, [Online], Available: [21] What is the Balanced Scorecard?, [Online], Available: orecard/tabid/55/default.aspx [22] Sprint Planning, [Online], Available: [23] What Is DSDM?, [Online], Available: [24] Product Software Engineering in the Medical Equipment Technology Industry -- Full Text, [Online], Available: [25] D. Kocurek, Understanding the Scrum Burndown Chart, [Online], Available: [26] V. Duarte, What is Velocity? (in agile software development), [Online], Available: [27] P. Bielicki, Predicting the team's Velocity: yesterday's weather method, [Online], Available: