A decision modelling approach for analysing requirements configuration trade-offs in time-constrained Web Application Development

Transcription

1 A decision modelling approach for analysing requirements configuration trade-offs in time-constrained Web Application Development Sven Ziemer 1, Pedro R. Falcone Sampaio 2 and Tor Stålhane 1 1 Department of Computer and Information Science, Norwegian University of Science and Technology, N-7491 Trondheim, Norway 2 School of Informatics, University of Manchester PO Box 88, Manchester M60 1QD, UK {svenz stalhane}@idi.ntnu.no, Pedro.Sampaio@manchester.ac.uk Abstract Release planning is a key phase in a time-constrained web development process, enabling stakeholders to find an appropriate trade-off between time-to-market and quality/functional characteristics of the delivered artifact. Despite the importance of conducting release planning activities, many web development projects still employ ad-hoc methods for deciding on the next release to be developed. In this paper, we discuss an effective release planning and configuration method underpinned by a decision modelling framework aimed at supporting small development teams to analyze, prioritize requirements, and to find a candidate release configuration that can be developed within the time, quality and functionality constraints relating to the project. The method is value-based, taking into account the stakeholders belief in what is regarded as the most valuable release configuration. The method is also consensus-driven, seeking to promote a consensual agreement with regard to the next release to be implemented. 1 Introduction When developing software systems it is important to meet the stakeholders needs and expectations. Usually, there is a considerable investment at stake and the stakeholders are keen to maximise their return on investment. Many stakeholders also want to realize part of the total value at stake in connection with the software system in the early phases of the development project and not wait until the project is completed in order to benefit from the system properties. One way of meeting stakeholders expectations earlier is to deploy the system as successive functionality chunks, as it is done in incremental development. For every new chunk that is delivered, some part of the system s value will be returned to its stakeholders. In addition, early feedback can be collected and necessary changes can be implemented. Planning for the next release thus becomes an important and complex activity. The complexity stems from the task of finding the optimal release that can be developed within the time constraints and that represents the highest possible value in return for the stakeholders. Web applications that are developed in competitive environments put a strong emphasis on timely development. Hence, it is important to find the best possible release consisting of both new features and/or improved features that satisfy customers and that also address competitive forces that may arise in connection with the application context, e.g., a release that covers all functionalities available at a competitor s web site. Release planning is often done ad-hoc [8]. In interviews with Norwegian small and medium enterprises developing web applications we have found that development practises are informal, ad-hoc and rush-to-market [9]. This makes it difficult to develop a release plan strategy. One reason for this is the lack of information and time to identify and analyse candidate releases, or even to have a detailed analysis of the requirements that should be contemplated at each stage. As these companies are depending on the domain knowledge of their developers and other stakeholders, the discussions between stakeholders are based on their experience and opinions. In many cases, these opinions can be conflicting as each stakeholder has his own interest and responsibility. In order to make important decisions, like the next release(s), it is important to find the right balance between several options advocated by the stakeholders. The remainder of this paper discusses an effective release planning and configuration method underpinned by a decision modeling framework aimed at helping small development teams to analyze and prioritize requirements, and to find a candidate release configuration that can be devel- 1

2 oped within the time, quality and functionality constraints relating to the project. The method is value-based, taking into account the stakeholders belief in what is regarded as the most valuable release configuration. The method is also consensus-driven, seeking to promote a consensual agreement with regard to the next release to be implemented. The paper is organised as follows: A description of the problem of finding the next release is given with some detail in section 2 and an example is presented in section 3. Section 4 looks at how stakeholders can express their beliefs about return on value using qualitative information. This approach is then used in section 5 to analyse, negotiate and decide about requirements and configurations. Related work is presented in section 6 and conclusions are given in section 7. ID S1 S2 S3 S4 S5 Description and Interests Product director. Wants a stable website, that runs smoothly and that has a positive impact on increasing the sales. Marketing director. Wants a website with a high newness and innovation factor and a short time-to-market. Wants to announce special prices on the website every week. IT manager. Wants a solution that is easy to maintain and extend. Prefers stable versions of tools and technologies that are to be used. Provider of payment transactions. Wants visibility and reliable services. Customers. Wants easy access to the online shop, safe payment transactions, reliable transport (shipment) and access to product specs. 2 The problem Developing software in a competitive environment puts a strong emphasis on time-to-market. There are usually more requirements to implement than time available for the next release. Thus, only a subset of the requirements can be implemented. Deciding on the content of releases is a nontrivial task. For a web application it is important to offer features that attract new users and that at the same time satisfies the existing ones. Still, deciding on the content of the next release is a decision that often is taken on the fly, without time for an in-depth analysis. Based on our experience, working with small and medium enterprises in the Websys project, we have observed that this decision is made either by the strongest stakeholder, who is likely to prioritise his own interests, or by the development team, excluding other stakeholders that might have a say in this decision. Two fundamental questions follow in connection to the issue of deciding on the content of the next release to be developed: Who should be involved in this decision? There are several options about who to involve. The decision can be taken by a single stakeholder such as the project manager or the marketing director, by the project manager and the developers or by all major stakeholders. Involving all major stakeholders is the preferred approach in an environment where a considerable amount of information is handled informally, enabling a comprehensive assessment of all relevant variables to the decision and also sharing the risks linked to the consequences of the decision. What should this decision be based on? In the literature we find a multitude of approaches and variables used for deciding on what will be contemplated in the next release, such as customer feedback, return of investment, stakeholder weights and strategic urgency. Another important approach, which is discussed in this paper, focuses on the stakeholders perceived return on value associated with a release. This is a criterion every stakeholder can relate to, as it expresses his expected gain from a release. Figure 1. The stakeholders to the online shop The problem to solve then is how to find the most favourable release for all stakeholders. The stakeholders will assess each candidate release and express their perceived return on value for it. This is based on each stakeholders subjective belief. The goal is to find a consensus and to balance the interests from the stakeholders. The following design principles guide the approach discussed in this paper: It must be simple so that it can be understood intuitively by all stakeholders, it must take into account all perspectives and stakeholder information available (which in most cases are the stakeholders opinions and beliefs), and it must be effective in converging to a consensual decision with regard to the next release. The approach proposed is a decision making framework based on the return on value of candidate releases in which a strong emphasis is placed in promoting a consensual agreement among stakeholders. 3 Example An E-commerce system In this section we discuss an example of a small electronic shop, based on one of the Norwegian enterprises involved in the Websys project. The small shop for electronic equipment launched an information only website in It allows customers to browse a product catalogue and to find information and prices on products. Lately, the company faced competition from two online retailers and from a local shop that started an online shop. The sales report for the last year shows that the shop is loosing customers to the online shops. To respond to this challenge from the competitors the shop decides to start an online shop itself. The goal of starting a new sales channel is to take back customers from the competing online shops, and also to win over new customers. In the process of starting the online shop one had to iden-

3 Req Description Req Description R1 R2 Improve Browse functionality, add product specs for every entry. Add product to cart. R11 R12 Browse through a list of special offers. Personalise product list. Show the user products he may be interested in, based on previous sales. R3 Place order. Requires that a user logs in. Edit contents of shopping cart, place orders. R13 Maintain catalogue. Add, change or delete items from the systems catalogue. R4 R5 Register user, establish user profi le. Profi le contains name, username, password, and shipment and payment details. Change user profi le. R14 R15 Browse orders. For every order, check if the products are available. Manage orders. Change orders, split orders, etc. R6 Cancel order, if order still is pending. R16 General design makeover. R7 Specify a wish list. Q1 Performance R8 Browse past orders. Q2 Reliability R9 Track shipment. Q3 Security R10 Search for a product. Q4 Usability Figure 2. Requirements tify the stakeholders that should have an influence over the contents of the new website, and to make some organisational changes. The development work was done by a third party development company. A manager for the online shop had to be appointed, who was responsible for all contact with the development company. A new position that is responsible for all administration work for the online customers had to be created. An important decision was the selection of a service provider for internet payment transactions. The number of stakeholders involved in this project, increased substantially compared to the first system. A detailed description of the stakeholders is given in figure 1. The stakeholders arranged a workshop both to identify a strategy for the online shop and to elicit the requirements. The requirements identified are shown in figure 2. A development scenario The marketing director is requesting that the new version of the company website has to be deployed within six weeks. In his view this is crucial in order to provide an online offer that exceeds all features provided by the competitors. The competitors are also in the process of changing their online sites, therefore adhering to the six week time-to-market plan is essential. The first estimate from the development team concludes that it will take at least 12 weeks to implement all the new requirements. The project manager is asked to produce a list of what can be developed within six weeks. He comes up with three candidate configurations, that each can be implemented within six weeks. In order to have an online shop requirements R2 R6 are mandatory. The configurations are shown in figure 5 and the details on how they where identified are given in section 5. 4 Value assessment In the example presented in section 3, only a subset of the requirements specification can be implemented within the given time-constraints. We call such a subset a configuration. A configuration consists of a number of functional and non-functional requirements that collectively have a meaningful impact on advancing the implementation of the overall business goals (e.g., a configuration that implements a new shopping cart) and that can be developed within the given time constraints. Each candidate configuration represents a potential software release. There is more than one candidate configuration in our example and a decision has to be made on which configuration to implement. As mentioned earlier, in a situation where a considerable amount of information is not available in writing but only as tacit and informal knowledge, this decision should be taken by all the stakeholders together to include all relevant information. This information would typically express the stakeholders beliefs and opinion about value, cost, benefit, risk, assessment of competition, etc. We will in this paper focus on the return on value. One way of expressing this tacit information on the fly is the use of attitude measurement. One widely used approach is the summated rating or Likert scale [7]. We use a similar approach, where each stakeholder assess the perceived return on value of a requirement using a single scale from 1 to 5: 1 no value, 2 little value, 3 some value, 4 high value, and 5 very high value. We want to clarify some assumptions before describing our approach. The assessment represents the perceived return on value for each stakeholder and is subjective in its na-

4 ture. We assume, however, that the stakeholders understand their business and the development project. The projects where the approach is applicable are typically conducted by teams of two to five developers and have tight timeconstraints (typically between two and 24 weeks). Therefore, an assessment has to be based on every stakeholders judgement. The expressed belief represents each stakeholders expert judgement, which is based on their best information, knowledge and intentions. We recognise that any stakeholder who wants to misinform the other stakeholders to influence the decision making process in his favour could easily do so, but then, this would also be possible when basing the decision on hard facts. Start Requirement elicitation Requirement estimation and assessment Identifying configurations Configuration assessment Decision on a configuration 5 Analysing, negotiating and deciding about configurations Consensus No In this section we present the overall release decision process, starting with requirement elicitation and resulting in a consensus based decision on which configuration to implement as the next release. This process encompasses five stages, which are described below and shown in figure 3. Requirement elicitation The requirements are elicited using a elicitation method or practise. In our example a workshop was arranged. The result from this stage is a list of requirements, including both functional and nonfunctional requirements (figure 2). Requirement estimation and assessment The next stage is to estimate and assess each requirement. This involves the project manager (for the first part) and all stakeholders (for the second part). The result of this activity is shown in figure 4. For each requirement, the project manager is responsible for providing a time estimate, any dependencies on other requirements and an urgency flag, that, if set, indicates that a requirement has to be included into the next release. All stakeholders assess each requirement using the scale as shown in section 4. They also write a short justification explaining their choice for each requirement. This justification will be used during the Decision on a configuration stage in case no configuration emerges as a clear collective winner. Also, for each requirement the mean value of all stakeholder assessment is calculated. For our example, the result of this stage is shown in figure 4. Note that the mean value is not shown. The total time to develop all requirements is 133 days. With two developers, the maximum effort possible within the time constraint is 60 days. Requirements R2 R6 have to be included in every candidate configuration to get a meaningful release. Identifying candidate configurations Using the information from the previous stage, a number of candidate configurations are identified. There is no rule about how many candidate configurations should be identified, but as a rule Yes Decision made Figure 3. The configuration decision process of thumb there should be 3 7 candidate configurations. To simplify the decision scope, there are some guidelines for identifying candidate configurations: 1. Sort the requirements according to some sorting criterias. For every criteria applied there will be a separate requirement list, which in turn can be used in the subsequent step, thus producing more than one candidate configuration. Common for all sorted requirement lists is that requirements with the urgency flag set have to come first. Two sorting criterias are (1) the mean value of all stakeholder assessments for a requirement, and (2) the number of high scores from the stakeholders. 2. Include into a configuration the requirements in the sorted order, until the estimated totals equals the specified time-constraint or until the addition of still another requirement will exceed this constraint. 3. For each requirement added, include also all requirements this requirement is depending on. 4. In case that not all requirements with the same sorting criteria can be included into a candidate configuration due to the time-constraint, additional candidate configuration can be identified by including requirements that have the same ranking and that are no yet included in any configuration for the given sorted requirement list.

5 Req Urgency Estimat Dependencies Stakeholder S1 S2 S3 S5 R1 4 d R R2 Yes 3 d R3 Yes 10 d R R4 Yes 8 d R5 Yes 3 d R R6 Yes 7 d R R7 10 d R R8 5 d R R9 3 d R R10 10 d R11 4 d R12 15 d R3, R R13 2 d R14 2 d R R15 4 d R3, R R16 20 d Q1 7 d Q2 3 d Q3 8 d Q4 5 d d Stakeholders: S1: Product manager, S2: Marketing director, S3: IT manager, S5: Customer Figure 4. Return on value assessment The three configurations identified in our example are shown in figure 5. The first two lines show the requirements included and the effort estimate. Configuration assessment When the candidate configurations are identified, they have to be assessed in the same way as the requirements where assessed. Again, stakeholders write a short justification explaining their choice for each configuration. Also, for each configuration the mean value of the stakeholders assessment is calculated. The configurations can be ranked by sorting them on the mean value. The configuration with the highest mean value is the one with the highest return on value according to the stakeholders assessment. The result of this stage is shown in the two last lines of figure 5. Configuration 3 has the highest median value, followed by configurations 2 and 1. There should, however, not be any automatism in choosing the configuration with the highest rank as the next release. The information that leads to this rank is meant to support a discussion between the involved stakeholders in order to reach a consensus based decision. Decision on a configuration The purpose of this stage is to find a consensus based decision on which configuration should be implemented in the next release. The result of the configuration assessment can end with a clear cut configuration that emerges as the winner or with two or more configurations ending up with the same ranking. In both cases, a consensus on which configuration to implement as the next release, is needed. If there is a winner from the configuration assessment, it is straightforward to reach a consensus. There should be strong arguments for not choosing the winning configuration. When there is no consensus on which configuration to choose, the stakeholders should first present their written justification to gain more insight into the other stakeholders assessment and arrive at a consensus. When no consensus can be reached at this stage, the configuration assessment step has to be repeated. A Delphi inspired process [5] will be used to guide this reconciliation. To arrive at a consensus concerning the consequences of the possible actions is the main problem in a trade-off situation. The whole decision process will break down if no agreement on a configuration that have an acceptable return on value for all stakeholders, can be reached. The Delphi feed-back process is used to arrive at a common understanding. As shown in figure 3 this means that the stage Configuration assessment will be repeated. The process broadly runs as follows: 1. Each stakeholder assesses return on value for each configuration, and gives his results in writing. The coordinator sums up all the assessments again in writing and distributes them to all the stakeholders. 2. After having read the other stakeholders opinions, each stakeholder gives a new assessment. This assessment is accompanied by a rationale explaining why his assessment differs from that of one or more of the other stakeholders. This new assessment is given to the coordinator. Step 2 is repeated until a consensus is reached. 6 Related Work There are a number of methods supporting software releases decisions, such as [8], [4], [3], [6] and [2]. These methods differ both with respect to the stakeholders involved (all major stakeholder, project manager and developers, customers and users), the objectives (Return of Investment, value of requirements, customer satisfaction and benefit of features to customers) and the applied mechanisms (Optimization, Stakholder prioritiszing, AHP). The

6 Config Req s: R1 R6, R10, Q3, Q4 R2 R6, R7, R9, Q2 Estimat: 58 d 60 d 57 d R1 R6, R10, R11, Q3 Value: S1: 3 S2: 2 S1: 3 S2: 3 S1: 3 S2: 4 S3: 3 S5: 4 S3: 4 S5: 4 S3: 4 S5: 4 Mean: Figure 5. Assessment of configurations approach presented in this paper involves all stakeholders, and is assessing candidate configurations rather than requirements, thereby reducing the complexity of the decision process. Value-based software engineering acknowledge the fact that software systems are not developed in a value-neutral environment. A general theory for Value-Based Software Engineering is given in [1]. The approach presented in this paper uses qualitative measures to express the perceived return on value. 7 Conclusions and further work This paper described a consensus-driven and value based release planning approach that can help small development teams to analyze, prioritize requirements, and decide on the next release in a scenario of time-constrained web application development. The approach is underpinned by a qualitative decision modelling framework that employs scales similar to Likert scales and Delphi techniques to identify release configurations and support stakeholders in the process of assessing, negotiating and choosing the next project release. The approach was developed based on our experience in working with small and medium Norwegian enterprises in the Websys project. In the context of Websys, companies manifested their interest in a systematic (to avoid the blind rush to market ) and inclusive (to cater for the less hierarchical nature of SMEs) approach for deciding on system releases, taking into account the multiple stakeholder perspectives of value attached to system requirements. We are currently experimentally assessing the framework with the consortium of SMEs involved in Websys and preliminary results indicate that the approach has a positive impact in fostering consensus and increasing project commitment between stakeholders. The simplicity of the approach also helps in avoiding typical analysis paralysis situations that arise in connection with complex decision models and/or highly democratic decision processes. References [1] S. Biffl, A. Aurum, B. Boehm, H. Erdogmus, and P. Grünbacher, editors. Value-Based Software Engineering. Springer-Verlag Berlin Heidelberg, [2] M. Denne and J. Cleland-Huang. The incremental funding method: Data-driven software development. IEEE Software, 43(14):39 47, [3] H.-W. Jung. Optimizing value and cost in requirement analysis. IEEE Software, 15(4):74 78, [4] J. Karlsson and K. Ryan. A cost-value approach for prioritizing requirements. IEEE Software, pages 67 74, September/October [5] H. Linstone and M. Turoff. The Delphi Method - Techniques and Applications. Addison Wesley Publishing Company, Inc., [6] D. A. Penny. An estimation-based management framework for enhancive maintenacne i commercial software products. In 18th International Conference on Software Maintenance (ICSM 2002), Maintaining Distributed Heterogeneous Systems, 3-6 October 2002, Montreal, Quebec, Canada, [7] C. Robson. Real World Research. Blackwell Publishers, [8] G. Ruhe and M. O. Saliu. The art and science of software release planning. IEEE Software, pages 47 53, November/December [9] S. Ziemer and T. Stålhane. Web application development and quality - observations from interviews with companies in norway. In Proceedings of Webist 2006, 2006.