AGILE IN GOVERNMENT IT AN ANALYSIS OF THE BARRIERS TO THE PROCUREMENT OF AGILE DELIVERY SERVICES

Transcription

1 INTRODUCTION AGILE IN GOVERNMENT IT AN ANALYSIS OF THE BARRIERS TO THE PROCUREMENT OF AGILE DELIVERY SERVICES 29 June 2011 This Note is written in response to a request from the PASC for an insight into what changes are required to the public procurement process to enable greater use of agile development. We do not believe that agile development is a panacea for the problems of Government IT. However, it may be able to help. For this reason, we have slightly amended the scope of this note to ask: What needs to be done differently in the public procurement process to enable more appropriate use of agile development? In our view, there are two key areas that need to be addressed in order to enable appropriate use of agile development. (i) (ii) The implementation of an Evolutionary Contract Model that reflects the principles of Agile and Lean. The adoption of an Agile Procurement Strategy that enables public sector buyers to select and engage suppliers rapidly, with low transaction costs, whilst complying with the Procurement Regulations. The barriers to agile development cannot be effectively removed unless each of these areas is addressed. In this Note we put forward a proposal for a contract model and a procurement strategy, each of which complements the other, and which we believe work well together. We have also identified some specific areas in the Public Contract Regulations 2006 that constrain Agile Procurement. 1

2 THE SCOPE OF AN AGILE PROJECT A successful agile development project requires a range of skills and services to be available in a timely and cost effective way. A typical agile development project will require most or all of the elements set out in the diagram below: Looking at each of these elements in turn: Retained Customer Capabilities. An informed customer must retain various capabilities to initiate, manage and deliver an agile project in a way that delivers value to the business. In our view these should be part of the core competency of the public sector body (the "PSB"), rather than bought in from the private sector. Typically, these capabilities account for 15-20% of the total cost of an agile project. Agile Delivery Services. This is a reference to the various services required to undertake an agile development, from project management through to requirements analysis, development, test and integration. These can be either provided in-house or procured from a third party supplier, although they are generally procured externally by most PSBs. Typically, these services account for 30-45% of the total cost of an agile project. Platform Services. These services comprise a range of enabling services, including authentication, identity management, transaction processing, hosting; as well as end-user infrastructure and support of software and hardware. Whilst these services are essential for the delivery of an agile project, it is more efficient for them to be provided as platform services with the costs shared across multiple projects and programmes, and they are generally procured externally. Typically, these services account for 30-40% of the total cost of an agile project. This Note only considers the procurement of Agile Delivery Services. However, we recognise that the procurement strategy for an agile development project also needs to identify and make use of the optimum procurement route for Platform Services. Agile projects can only be successful if they are also implemented in conjunction with the development of the required Retained Customer Capabilities, and governance that is adapted to reflect the nature of agile projects. 2

3 THE EVOLUTIONARY CONTRACT MODEL The issue Traditional contracts are unsuited for software or product development projects. They are based on defined process control with great emphasis placed on plans, requirements and estimates that are created at the start of the project. This assumes that development projects are predictable, but sadly they are not. There is a huge amount of complexity and uncertainty surrounding a development project, particularly in IT. The level of complexity and uncertainty has increased in recent years as the pace of change increases and technology becomes more sophisticated and all-pervasive. The only way in which traditional contracts can adapt to change is by means of the change control mechanism. However, instead of facilitating change, the change control mechanism actually serves to restrict change. The process of documenting changes is time-consuming, consumes valuable resources, can be expensive to implement, and adds no real value to the project. It simply attempts to keep the contract in step with the pace of change. A possible solution - the Evolutionary Contract Model (i) The background to the Evolutionary Contract Model The Evolutionary Contract Model has been developed by Susan Atkinson from gallenalliance Solicitors together with Gabrielle Benefield from the Scrum Foundation, and with input from some of the leading organisations (notably emergn Corporation) and thought leaders in this space. This model is suitable for use with any agile methodology. It draws upon the principles of Scrum, XP, DSDM Atern and Evo as well as Lean thinking. (ii) The nature of the Evolutionary Contract Model The Evolutionary Contract Model is a goal-oriented contract for the provision of services. Unlike a traditional contract for services, it is not based on defined process control but on empirical process control as advocated by Scrum. This is a form of control driven by experience and experimentation. The fact that there are no detailed plans and specifications in the contract means there is no need for a change control mechanism. Instead, the customer's desired features of the solution are captured in a central repository which takes the form of an evolving prioritised queue of work (the "EPQ"). The EPQ does not have contractual status. However, the EPQ must be within the scope of the contract, and the ability to change it is regulated by the contract. The EPQ may be, and should be, amended and refined by the customer throughout the life of the project. It is this ability of the customer to make changes to the EPQ at any point in time that is key to building flexibility into the development process. In order to achieve empirical process control there must be visibility, inspection and adaptation. To achieve this, (a) the work involved in the development project is decomposed into a number of small, prioritised, concrete deliverables; (b) the project is broken down into short fixed-length iterations at the end of each of which completed deliverables can be inspected by the customer; 3

4 and (c) the project team must be able to self-organise which in turn means that it must be small and cross-functional. Essentially, under the Evolutionary Contract Model the project is broken down into a series of iterations during which the customer has full transparency of the development process and can inspect the process for correct operation and results. At the end of each iteration the customer can adapt the process as needed to achieve the desired outcome. The customer should be able to make the adjustment as quickly as possible to minimise further deviation. Delivery of an agile project is measured in terms of delivery of quantifiable value to the stakeholders, within the parameters of the scope of the project. Working software is the primary measure of progress. Measures such as the completion of tasks, work in progress or the production of documents are not relevant to the delivery of value. Quantified value is delivered incrementally to the stakeholders from more-or-less the start of the project. This can be achieved because, firstly, the customer is responsible for prioritising the order in which increments of the solution are delivered at the end of each iteration and, secondly, at the end of each iteration the supplier provides quantified metrics showing the progress of work and the extent to which the customer's desired outcomes have been achieved. These metrics inform both of the parties where to focus the development activities and how they may improve the process. One of the key objectives of the Evolutionary Contract Model is to delegate responsibility and accountability for the project management to the team. The customer must be represented on the team but the role of the customer is quite different from that of the supplier: the customer provides the domain expertise and the supplier provides the technical expertise. Accordingly, neither the EPQ nor the project-specific details form part of the contract. This means that there is far less administration surrounding management of the contract, making it a more attractive model for SMEs. The Evolutionary Contract Model only differs from a traditional contract in terms of how the solution is delivered. There is no reason why, for example, provisions regarding the treatment of intellectual property rights, data protection, assignment and so on should be treated any differently. (iii) The structure of the Evolutionary Contract Model In terms of the project-specific details, the contract contains merely a succinct and high level description of the scope of the project in terms of the customer's vision for the project, the desired outcomes and the constraints. This is for a couple of reasons: firstly, all parties should be aligned in terms of the goals of the project and, secondly, more granular planning is considered to result in waste given that detailed specifications are likely to become obsolete relatively quickly. The contract structures the project as several discrete phases: (a) the Discovery Phase, (b) the Calibration Phase, (c) the Delivery Phase and (d) the Deployment Phase. Initially the parties only make a firm commitment to conduct the Discovery Phase. The contract is structured as a 4

5 framework under which the customer may (but is not obliged to) place statements of work to initiate the Calibration Phase and each of the releases in the Delivery Phase and the Deployment Phase. The ability of the customer to develop the solution in stages, without an upfront commitment to the project as a whole, greatly reduces the probability of significant cost overruns and provides strong incentives for the supplier to deliver value early. During the Discovery Phase the customer works with the supplier to validate the outline business case for the project. This involves identifying the stakeholders and modelling their requirements, modelling the constraints, assessing the associated risks, mapping the solution and context modelling. As part of the solution mapping, the solution is broken down into constituent solution subsets, each of which is to be delivered as a separate release. The end of the Discovery Phase provides an ideal point for Gateway review of the project. If the business case is validated, the parties move into the Calibration Phase. During the Calibration Phase the parties work on a small amount of key functionality of the solution to inform their decisions regarding the high level architecture of the solution and to calibrate the velocity of the team. This in turn enables the supplier to create more accurate estimates for the purposes of scheduling and cost. The parties also develop the rules of governance for the project. Once the team and the solution have reached a certain level of maturity, the parties move into the Delivery Phase. During the Delivery Phase the solution is designed, built and tested as a series of modules, each of which will build upon the modules previously delivered. The Delivery Phase may comprise anything from 3 to 20 separate releases. Whichever charging model is used, it is important that there is some form of incentive linked to the delivery of value to the stakeholders. More advanced teams are looking to link charges directly to a quantifiable measure of value. 5

6 THE AGILE PROCUREMENT STRATEGY The issue Current public sector procurement strategies are based on the traditional contract model and have as their central objective the need to reduce risk to the fullest extent possible prior to the award of contracts that commit HM Government to the full investment in a new capability. The OGC Gateway Review process, and the guidance issued to PSBs encourage Senior Responsible Officers (SROs) only to submit business cases for approval when they have met the highest possible standard of maturity. These strategies are appropriate for Platform Services, where the specification is relatively stable and service volumes are very high. PSBs have tended to put in place long term contracts to offset the high entry and transition costs. However, a fundamental principle of Agile and Lean is to fail early, and fail often. "Software is nothing else than codified and explicit knowledge." 1 So, the best approach for generating knowledge is through experimentation with small, rapid try-it, test-it, fix-it cycles. 2 Rather than seeking approval when the project business case has reached a given standard of quality, agile projects seek approval when a fixed time period has elapsed. Agile business cases have a built-in presumption that, if insufficient value has been delivered by a given date, the project will be cancelled and resources diverted to other initiatives that appear more fruitful. The contrast between the two approaches could not be more stark. A possible solution the Agile Procurement Strategy The Agile Procurement Strategy is based on the 'Fail Early, Fail Often' principle. Buying organisations must maintain access to an eco-system of suppliers that enables them to rapidly appoint suppliers to support individual projects. Equally, they must also have the ability to rapidly terminate contractual arrangements if a particular project fails (whether due to a supplier default or for any other reason). The supplier eco-system must cover the full range of services, from Platform Service providers through to Agile Delivery Service Providers. In the private sector, large scale systems integrators often provide Platform Services and integration of the software with existing systems, with a range of SMEs to provide specialist resources to support the development phase of the project. However, public procurement is more constrained than private sector procurement. An Agile Procurement Strategy for PSBs must meet the following criteria: (a) (b) It must be compliant with the EU procurement directives. It must significantly reduce the time taken to select suppliers and place contracts when compared to current procurement routes. 1 'Agile Software Development with Scrum' by Ken Schwaber and Mike Beedle, This is a variation on the iterative four-step management process of 'Plan, Do, Check, Act' as popularised by Dr W Edwards Deming, who is considered by many to be the father of modern quality control. 6

7 (c) (d) It must reduce transaction/overhead costs for all parties. It must be supported by procedural changes in the way that ICT projects are defined and delivered, to enable more flexible, rapid decision making. We propose that PSBs could engage an equivalent eco-system of suppliers for the procurement of Agile Delivery Services by means of one or more framework-based Electronic Marketplaces. Each framework should have the following characteristics: Its scope is limited to a defined ICT category, potentially in line with those used for existing Buying Solutions frameworks. Only one service provider is selected - the Market Manager. The role of the Market Manager is to: establish and maintain an electronic trading environment for suppliers to provide in-scope services, and for public sector buyers to publish requirements; provide central administrative services including registration, prequalification of suppliers for different sizes of opportunity, process management, provision of security vetting services for market participants, marketing and promotion of the marketplace; to maintain and apply Market Rules: standard terms and conditions and contracting procedures that enable the rapid selection and engagement of individual suppliers to support specific requirements. The Market Manager recovers its costs through the application of a mark-up to the services procured through the framework. Crucially, the Market Manager must be a neutral party: it must not be involved in the delivery of the services. This removes the inherent conflict of interest in the operation of current framework agreements, where a large systems integrator has a strong incentive to propose its own services in preference to those of its subcontractors. Electronic marketplaces are not new - a number have been established within the public sector in the course of the last decade. They provide an electronic trading environment in which buyers can advertise their requirements to a pre-selected group of suppliers, with the majority of the steps in the purchase to payment process automated. The Zanzibar platform provides an example of the efficiencies that can be achieved through this approach. However, the existing electronic marketplaces have tended to focus on improving the efficiency of existing supply arrangements, rather than as a mechanism for enabling new providers to enter the marketplace. A framework based Electronic Marketplace could deliver a fast, efficient mechanism for the delivery of agile development projects, and also meet the parallel government objective of extending the involvement of SMEs in the delivery of government IT. 7

8 THE PROCUREMENT CONTRACT REGULATIONS 2006 The background The Procurement Contract Regulations 2006 ("PCR") permit the use of Dynamic Purchasing Systems ("DPS"), e-auctions and framework agreements to facilitate the more rapid placing of contracts. Each of these mechanisms is suitable for the award of contracts in support of agile development projects, and should be used in preference to either the Negotiated or Competitive Dialogue procedures. Electronic auctions and dynamic purchasing systems In the private sector the use of e-auctions and DPS is a well established mechanism for placing small to medium sized contracts for software development. For example, and have demonstrated that this is a viable option. The PCR also allows the use of e-auctions and DPS when the goods or services in question meet certain criteria. In relation to e-auctions Regulation 21 of the PCR states: (3) The contracting authority shall not hold an electronic auction to precede the award of a public services contract or a public works contract having as its subject matter intellectual performance, such as the design of works. (4) The contracting authority may only hold an electronic auction to precede the award of a contract when the contract specification can be established with precision. (5) The contracting authority shall base an electronic auction on (a) (b) price alone where the contract is to be awarded on the basis of the lowest price; or price or the values of quantifiable elements of tenders indicated in the contract specification, where the contract is to be awarded on the basis of the offer which is the most economically advantageous in accordance with regulation 30(1)(a). (i) The issue Regulation 21 of the PCR limits the ability of PSBs to use e-auctions for agile projects for the following reasons: (a) (b) (c) Regulation 21(4) requires the contract specification to be established "with precision". However, Agile and Lean thinking advocate that the project is initially described at a high level, and that the specification of the solution evolves and matures within the scope of the project. Regulation 21(3) prohibits the use of e-auctions for contracts whose subject matter is "intellectual performance". Arguably, for this reason e-auctions are not permitted for software development. Regulation 21(5) only permits a contracting authority to base the auction on elements of the tender (whether price or quality). However, the experience of the majority of e- auction sites is that one of the principal factors determining customer choice of a particular supplier is historic feedback on that supplier's performance - a measure which, by definition, cannot be provided impartially by the supplier in question. 8

9 (ii) Proposed amendments to the PCR Thresholds It should be clarified that e-auctions can be used for software development projects. The selection and award criteria for tenders should explicitly enable PSBs to take account of direct feedback from previous customers in evaluating bidder suitability, regardless of whether these were included in the bidder's tender: 3 Contracts placed under e-auctions should allow for staged commitment, comprising an initial contract specification, which would be fixed in the same way as are set out in Regulation 21(4). Commitments to further contract specifications should be permissible, subject to the supplier having satisfactorily delivered the initial contract specification. The objective of the thresholds is to maximise the level of visibility and participation by the market in all public sector opportunities. Contracts below the threshold may be awarded by PSBs without reference to the PCR, although they must still follow general Treaty of Rome principles such as non-discrimination, transparency, and equality of treatment. Contracts above the threshold may only be awarded after a compliant procurement process, using one of the procedures set out in the PCR. (i) The issue In practice, the thresholds are set so low that they have the opposite effect to the intended objective: they artificially restrict competition and access to the public sector market by SMEs. This is for the reason that the thresholds do not take into account the real cost to market participants of bidding on each opportunity, particularly in the IT category, where the requirements are complex and difficult to specify in advance. Recent evidence presented to the PASC shows that a buyer's transaction costs associated with running a typical Competitive Dialogue procedure for the award of an IT contract are in the order of 1 million. Bidder costs will be of a similar order of magnitude. When faced with these transaction costs, it does not make sense for the minimum threshold to be set at 220,000 Euros for the value of the resulting contract. Public sector buyers are forced to aggregate their requirements into large contracts (far higher than the threshold), and only large companies have the resources to bid on them. Regulation 19(9) of the PCR regarding framework agreements already recognises this issue and makes some allowance for PSBs to set a suitable timescale and invite tenders from a number of framework suppliers (one of the most common methods used for the procurement of IT systems). However, the award criteria have to be the same as those set out in the original framework competition, which restricts the level of discretion to select the most suitable supplier at the time of the tender being submitted. 3 Whilst this Note focuses on IT projects in particular, this principle is also applicable to other categories of spend. 9

10 (ii) Proposed Amendments The relevant threshold for the application of the full procurement procedures should be set by reference to: the nature of the goods and services being sought; and the transaction costs associated with the procurement process. In the case of IT services, this would result in the threshold being significantly higher - enabling PSBs to adopt selection and award processes that are more proportionate in terms of the ratio of transaction cost to contract value. Submission Details This Note was written by Kelvin Prescott, Director at Newbury Management Consultants, and Susan Atkinson, Legal Director at gallenalliance Solicitors. The authors would like to acknowledge contributions to this Note from the Agile Delivery Network ( 10