The background here is the fact that the SWAN Programme was the first major pan-public

Transcription

1 B/16/105 NSS Formal Board Meeting Friday, 4 November 2016 SWAN Lessons Learned Purpose The paper attached is to share key lessons learned from the procurement phase of the SWAN broadband networking services contract managed by NSS on behalf of SG and a range of public sector wide partner agencies. Recommendation The Board is asked to note the Lessons Learned and to provide advice and guidance on the sharing of these findings both within NSS and with the various broader SWAN stakeholder groups Timing There are no major timing considerations. This paper is being produced at the end of the NHS rollout phase to collect key learnings to date as requested by Board at April 16 meeting. This paper is focused on the key lessons from the procurement / initial rollout phase. Background The background here is the fact that the SWAN Programme was the first major pan-public a guaranteed Vanguard partnership commitment had not previously been tried and it is acknowledged that new models of working would need to be developed to support the procurement and ongoing management of the contract and associated services. It is important therefore that key lessons learned are captured and made available to future similar exercises / initiatives. Engagement There has been ongoing engagement with PCF and CLO within NSS on the delivery of the SWAN programme as well as the core of the delivery team coming from NSS IT. The NSS Board has been kept well appraised of status, progress, risks and issues associated with the programme since the original commission back in Andy Robertson Director of IT: Tel: andy.robertson@nhs.net Page 1 of 8

2 Scottish Wide Area Network (SWAN) Procurement Lesson Learned This Paper reviews the SWAN procurement and identifies possible improvements which could be implemented in future large scale IT procurements. The lessons learnt are categorised under ten headings. Governance Any procurement which is carried out on behalf of more than one Public Sector Entity will require a governance structure that clearly shows the role of each entity (supplier vs. customer), what the responsibilities of each party are and how decisions will be taken (including dealing with matters such as quorum and voting aspects) as well as the reporting and accountability lines up the governance structures and horizontally. A great amount of time in this project was spent in discussions between the various entities participating in agree a Memorandum of Understanding (MoU) which determined the governance structure, project procurement cost apportionment and approach. These discussions highlighted the different approach to governance taken by the various Partners. For example, Councils were keen on the adoption of voting rights while entities in NHS Scotland more commonly used a consensus system of decision making. It is recommended that, at the outset of any future Pan Scottish Public Sector Procurement, the parties agree the terms of an MoU setting out the basis upon which they have agreed to come together to undertake the procurement and detailing the governance arrangements to apply to the decision making process. All this should happen in advance of placing the OJEU advert and embarking on the procurement process. In any event it is vital that all Partners, but particularly the lead procuring organisation and ultimate Contract Authority, be given an escalation route through the defined governance channels to an appropriately empowered arbitration body / board should the risk profile become unacceptable to any one Partner. In the case of SWAN this was the SWAN Programme Board. It would also have been beneficial to have from the outset provided a clear indication of who would be the Contracting Authority (NSS vs. Scottish Government in the case of SWAN) so that the relevant organisation in question could take planned action as soon as possible in mitigating inherent risks associated with being the Contracting Authority. In the case of SWAN, the decision about who would ultimately be the Contracting Authority was made only a matter of weeks before contract signature, by which time a number of positions and required roles and responsibilities had be drafted in the contractual schedules and formed part of the final bids, which might not have been acceptable would another party have ultimately assumed responsibility as Contracting Authority. This could have resulted in the Contracting Authority assuming more risk that it preferred which, whilst beneficial to others, should not necessarily be repeated in future. Page 2 of 8

3 With all the above taken into consideration, haven ensured a rigorous governance structure on SWAN is probably the single largest success factor. The SWAN Vanguard Partnership held together throughout the procurement, bought a network services integrator everyone could use, would be scalable to the rest of the Public Sector, despite wide, varying and at times conflicting requirements across the organisations. Project Funding SWAN always was the first major initiative to be launched as part of the Digital Public Services Strategy (DPSS) by Scottish Government. As such it came as somewhat of a surprise that SWAN had to be completely self funding by those that use it and will use it in the future. Due to this self funding there were periods throughout the procurement were extremely protracted debates in meetings took place around costs and cost apportionment methodology. Still to this day a tremendous amount of time and effort is expelled in discussing costs with the next wave of Tranche 2 Customers. In future procurements very careful consideration should be given to self funding initiatives. A significant amount of the project budget which was spent was done so on external financial advice to develop and finally agree upon the cost apportionment model. Whilst impossible to define exactly how much, significant project funds could have been saved if Scottish Government agreeing to pay for the Shared Services or at the very least the Shared Service Set-Up centrally. This would have created time and cost savings in potential cross subsidisation across organisations in the cost apportionment methodology. Self funding also requires a significant anchor tenant to be prepared to pay for the fixed costs in the initial set-up stages of the rollout. This is a particular arduous task for the initial anchor tenant adopter, where there may not yet be the economies of scale of several organisations sharing this fixed costs. An anchor tenant may not always be possible in future procurements, particularly in an environment where funding is reducing across the Public Sector. Having Scottish Government help fund SWAN would have developed a much cleaner base platform from which Scottish Government could develop their Shared Services agenda and make it much more attractive to the next wave of Tranche 2 Customers, some of which are seriously struggling to join SWAN on affordability grounds. It would also give Scottish Government a much greater chance of ensuring that SWAN long term does become the Public Sector Scottish Wider Area Network, rather than only the Scottish Wider Area Network for those Public Sector organisations who can afford to join it. Output Based Specification (OBS) and Invitations to Bid A great deal of time was spent on producing an agreed Output Based Specification (OBS) and this was time well spent, but again the amount of effort involved in landing an agreed version capturing all P Page 3 of 8

4 requirements, which at times were conflicting, should not be underestimated. However, having produced this OBS, there was an attempt to keep the OBS up to date with each Partner s changing requirements for circuits. Much time was allocated to adjusting the list of circuits required in order to ensure that the list was as accurate as possible. circuits increased and decreased during the procurement process together with the capacity and technology of such circuits. In retrospect, it would seem more sensible to have captured a snapshot of the circuits required, in terms of number, capacity and technical requirements, and assessed bids against that requirement in the procurement process. On this basis, a Preferred Bidder could have been chosen and the circuits requirement appropriately updated to reflect the position at the time of award of the new contract. In a contrasting view, it is conceivable that by the OBS being kept up to date with customer demand, end requirement rather than a historical out of date snapshot from a procurement model. Going forward, more thought needs to be devoted to the balance between the effort (time + cost) in keeping the OBS requirement up to date vs. the benefits which might be achieved in getting a true final requirement price. An early decision on the final role of the OBS should also be made i.e. should only be incorporated into the various contract schedules or should the OBS response document always be a part of the contract in addition to the incor response into the Contract can be viewed as a safety measure to allow the actual Bidder response potentially to be used for clarification of the actual text entered into the contract schedules if ambiguous or inadvertently absent from the final contract. Procurement Process The procurement process itself needs to be very tightly managed. This was the approach taken in SWAN. The same amount of time was allocated for meetings with each Bidder, in a rotating fashion week by week so that e.g. Bidder A was not always first up every Tuesday morning. This was especially took hold which would have been to the advantage of Bidders toward the end of the week, but disadvantage of Bidders at the start of the week. By constantly changing who went first in any given week of dialogue, this ensured that on average there was no inherent advantage or disadvantage between Bidders. In addition a strictly controlled process for submission of questions by bidders and answers from NSS was adopted. This included not only peer reviews by Partners on answers as appropriate, but also complete logging in Excel of submitted question, by who and when along with corresponding answer and when. Detailed agendas for each dialogue meeting were also issued and used. It is clear that a number of dry runs on the final bid are required and that contract documentation should be drafted and issued early in the process to allow the final bid to be submitted against agreed contractual terms and conditions. It is Page 4 of 8

5 important that all Partners allocate sufficient resource to enable them to appropriately participate in the Dialogue and the MoU should address this. It is important that the Project Team is able to rely fully on Partners making sufficient resources available whenever they are required. This was not always the experience in the SWAN project and having on occasion to re-visit decisions made in the absence of one or more Partners caused frustration to Partners who were in attendance along with inevitable delay as the same subject had to be discussed once again. There was also on occasion a balance to be struck between decisions made by a Partner which was questionable from a procurement perspective. Any future procurement needs to be very mindful of the tricky balance that needs to be struck between customer need / requirement and external / subject matter expertise procurement advised best practice. This is particularly relevant against an increasing backdrop of Bidders challenging procurement decisions explained and detailed further below. Gateway Reviews The call for OGC Gateway Reviews was not in the control of the Project remit. As a result, Gateway Reviews were initially viewed with suspicion by the Project Team. However, it became clear after the first Gateway Review that these were very helpful and provided an oversight which helped NSS improve the procurement process. In fact, some of the best ideas came from the Gateway reviews. The timing of Gateway Reviews, this often meant that the Reviews were not undertaken at optimum times. If they had been under the control of the Project Board, they would have been scheduled at more appropriate times to coincide better with the procurement timetable and take place at key points in the process. On reflection it is seen as immensely beneficial for large scale, complex, high risk procurements to undergo formal Gateway Reviews. NSS Oversight NSS was represented at the Project Board but, unusually, was both a procurement service provider as well as the key Partner. In these circumstances, NSS required to balance the interests of the various Partners and the risk it would and did assume as the procurement team lead, contracting authority, central team within the SWAN Target Operating Model (TOM) with overall responsibility for the Service Management of the service and key Partner. It could be considered that these were conflicting interests, in respect of which, NSS could not act in an unbiased manner. In the event, NSS is confident that it managed and balanced these interests well and delivered an excellent service for Partners. However, this often resulted in NSS assuming more risk than it preferred which, whilst beneficial to other Partners, should not necessarily be repeated in future collaborative procurements. It was therefore vital to ensure NSS always had access, like all Partners, to the escalation method as mentioned in the above governance Page 5 of 8

6 breached what NSS was ultimately prepared to underwrite. During the court proceedings raised by BT a group comprising senior level officers was convened by NSS. This group provided high level decision making and oversight in relation to these proceedings. This worked well in bringing NSS Directors and senior advisers together and coordinating decisions and actions. In retrospect, it is considered that it may have been worthwhile, on a general basis, to replicate this type of high level oversight on the SWAN procurement and on future large scale, complex and high risk procurements going forward. Benefit may have been derived from a three monthly meeting of this type of group with the NSS Procurement Project Team. Central Legal Office (CLO) Role There is no doubt that, in terms of legal services, CLO has great experience in procurement and projects of this nature and should always be the first choice of legal advisers by NSS. However, a project the size and complexity of SWAN, progressed using the competitive dialogue procedure, is resource intensive to varying degrees throughout the procurement process. For significant periods it required a team of solicitors to be wholly engaged and at other times still required significant legal input. Albeit that CLO did not act in the SWAN project as a result of vires issues, given the size of its Contracts Team, CLO would have had difficulty adequately resourcing the SWAN project. External legal advisers, with sufficient experience, resource and access to facilities (meeting rooms, secretarial support, etc.) to deliver the project, were appointed. Whist it is considered that these advisers performed understanding of the public sector. In retrospect, the Project Team considers that, notwithstanding that external legal advisers were appointed, CLO should still have had significant involvement by joining the project team, or at the very least the Project Board, thereby ensuring that the external legal advisers are managed by an In future projects CLO should take this role and also undertake reviews of all procurement and contract documents. Procurement Oversight Scottish Government Procurement participated at Programme Board level and reviewed a number of documents, providing useful input. However, it is considered that, rather than simply undertaking an overview role, in any future public sector procurement project, combining the procurement expertise from multiple sources, working together to undertake the project would deliver significant benefit. Potentially this would deliver the benefit of combined expertise, experience and best practice from multiple parts of the Public Sector. Page 6 of 8

7 Incumbents In any procurement where there is an incumbent service provider, it is important that the procurement process is structured and undertaken in a manner which minimises any inherent advantage they may have. This requires that sufficient time is allocated to the procurement process to allow other bidders to gain knowledge and understanding of the requirements so as insofar as possible. Challenges to procurement processes have increased significantly over the last few years. The SWAN project was subject to challenge which delayed the award of contract and resulted in loss of the benefit of the new contract being derived from an earlier procurements should be undertaken within a timescale which mitigates the effect of such delays. Legal Challenge All procurement processes face the risk of legal challenge and this was understood by the Procurement Project Team, the Partners, the Project Board and the Programme Board. As stated above, following issue of standstill letters advising bidders of the decision on award of the SWAN contract, BT raised court proceedings against NSS alleging breach of the Public Contracts (Scotland) Regulations 2012 (Regulations) in respect of this project. The grounds of claim covered a wide range of matters and BT sought to have the decision on award overturned. In terms of the Regulations, the serving of court proceeding, automatically suspends any award of contract being made. NSS rejected the complaints and, following advice from Counsel, defended the action. NSS sought an order from the Court to lift the automatic suspension and, in addition, contended that the action raised by BT was time barred in that it had not been raised within the timescales required under the Regulations. Early hearings on these issues resulted in a high level review of the merits of the BT case. At these hearings, the Court issued an order lifting the suspension, rejected the NSS arguments on time-bar and indicated that it did not consider there was merit in the majority. However, the Court indicated that it considered that there was a case to answer on the issue of transparency of scoring regime and that a full proof on the merits of this issue was required. The result of these decisions was that NSS could proceed to make an award of contract, as the suspension was lifted, but would still be at risk of paying damages if the Court found in favour of BT at the subsequent proof on transparency. This would have resulted in the action continuing for some time with both parties incurring further expense. In subsequent discussions both parties recognised the benefits of avoiding further litigation. Accordingly, NSS reached a commercial agreement with BT which resulted in BT abandoning the court proceedings thereby obviating the need for a proof. Page 7 of 8

8 These proceedings required consideration by the Project Team, not only of the merits of the action, but also the best strategy for its defence. The options considered were: 1. NSS could seek an order from the Court to lift the automatic suspension, effected by the court proceeding on an award of contract being made 2. NSS could seek to defend the action without seeking an order to have the suspension lifted; or 3. NSS could roll back the process, reissue amended evaluation documents and rescore the bids accordingly, thereby exposing NSS to risk of challenge by other bidders, most notably Capita who were the winning bidder in the first instance. The lessons to be learnt on the drafting of scoring and evaluation methodologies are clear and these will be applied in any future procurement. However, the volume, size and complexity of the type and nature of the documents produced in the SWAN project should not be underestimated. Given the prevalent approach by bidders to challenge any procurement in which they are not successful, evidenced by the significant increase in recent years in the number of procurement challenges before the Courts, it must be accepted that large complex procurements will need to be undertaken in the knowledge that they will always be at risk of challenge. To this effect it is vital that a comprehensive document management filing structure be in place so that key information can quickly and accurately be recalled as and when required. This is something that was employed from the outset with rigour on SWAN and endorsed by the entire NSS core project team and should be seen as the baseline benchmark for future procurements. Page 8 of 8