EXECUTIVE SUMMARY 1. RECOMMENDATION FOR ACTION

Size: px
Start display at page:

Download "EXECUTIVE SUMMARY 1. RECOMMENDATION FOR ACTION"

Transcription

1 EXECUTIVE SUMMARY 1. RECOMMENDATION FOR ACTION The Vision Implementation Group (VIG) is invited to take note of the progress of the ESS Vision 2020 SERV project and to endorse the proposed roadmap for the last phase of the project. 2. BACKGROUND AND BRIEF HISTORY Investment in IT systems to support the collection, production and dissemination of statistics represents a significant share of statistical organizations budgets in terms of financial and human resources. Reusing software components across organizations can bring significant cost reductions and efficiency gains by avoiding the need to "reinvent the wheel" each time an IT application is needed for statistical production. As ESS business processes become increasingly standardized, software components (hereafter IT services) could be designed and operated in ways that allow their reuse and sharing across ESS members. The objective of the SERV project is to create the conditions for developing and sharing IT services in the ESS. The scope of the project includes the set-up of governance aspects of and the development of IT services to be shared among the ESS members. Given the priorities stemming from the ESS Vision 2020, the project focuses on the development of IT services needed for the implementation of the ESS Vision 2020 portfolio. The ESSC agreed on the business case of the SERV project in February 2015 with three phases (see Annex 1): The first phase 1 ( ) focused on providing the necessary technical infrastructure for developing IT services to be shared in the ESS. The second phase ( ) focuses on the development and deployment of IT services according to priority areas and on the development of the governance for shared IT services. The third phase ( ) will focus on further development and deployment of IT services. The second phase is under way with many deliverables being already available. The first version of the guidelines and governance documents are available, the list of priority services for the ESS to implement was analysed and produced with the central infrastructure hosting its first services. The example services have been delivered and their reuse by third-party NSIs is providing valuable lessons learnt. The Service Catalogue static version is delivered with advanced functionalities planned. 3. POLICY CONTEXT The SERV project is part of the ESS Vision 2020 Implementation Portfolio and it links to two of the key areas of the ESS Vision 2020: "Identifying user needs and cooperation with stakeholders", "Efficient and robust statistical process". 1

2 By using collaborative tools available in the ESS, the project allows gathering user needs expressed by ESS members and facilitates their cooperation by enabling sharing of IT services within the ESS and the broader statistical community. The use of shared and standard IT services will lead to increased efficiency and contribute to reduce the investment cost for new IT systems development. 4. CONSEQUENCES FOR NATIONAL STATISTICAL INSTITUTES Sharing IT services will help ESS members to optimize their investments by profiting from the developments made by other partners. This can be achieved by the use of an IT service hosted by other members of the ESS or its reuse within the organization's IT systems. The first services are providing real life examples of the benefits achievable. 5. OUTSTANDING ISSUES Delay in the implementation of the service catalogue has been identified due to contracting issues, therefore the delivery of the advanced version of the catalogue has been postponed to H Baselines for the maintenance, support and licensing of services are proving to be difficult to establish. No issues were identified for other deliverables of the project. The nature and impact of the issues identified are not jeopardising the expected outcome and benefits of the project but may lead to a delayed deployment. Additional IT services will be developed and deployed during third phase of the project based on the needs of the ESS Vision 2020 implementation portfolio. 6. RISK ASSESSMENT Coordination with business projects and insufficient participation from ESS members have been identified as major risks for the project. The lack of coordination with ESS Vision 2020 projects would result in low business value of project outcomes. To mitigate this risk, close coordination with the ESS Vision 2020 project portfolio will continue, with a view to defining priorities for new IT services investment needs. The lack of participation of Member States in the last phase would undermine the deployment of project deliverables. Several instruments have been put in place to facilitate the involvement of Member States. A Steering Group and a Task Force are operational and a second ESSnet as well as grants have been planned to involve ESS members in the deployment of IT services. 7. NEXT STEPS The last phase of the project will concentrate in deploying the deliverables of the first two phases by having ESS members follow the developed governance and use the available services. Additional services will be implemented to facilitate the adoption of shared services in statistical productions processes. Following its completion, the project team, with the advice of the Steering Group, will elaborate follow-up actions for the continuation of the sharing services initiative. 2

3 1. PURPOSE OF THE DOCUMENT ESS VISION 2020 SERV PROJECT: STATUS REPORT The purpose of this document is to present to the Vision Implementation Group (VIG): How the objectives set out for the SERV project translate into actionable deliverables; How the deliverables provide benefits to the ESS partners; A roadmap of past and future milestones of the project until its closure. 2. WHY THE SERV PROJECT? In February 2015, the ESSC agreed to launch the Shared Services project ("SERV"). The goal was to create the environment to allow sharing IT services 1 in the ESS and to define the priorities and areas where new IT services should be developed. This project supports mainly the following key areas of the ESS VISION 2020: "Identifying user needs and cooperation with stakeholders" The project facilitates cooperation, by enabling sharing of IT functionalities among ESS members. It identifies the needs for shareable services at ESS through surveys. "Efficient and robust statistical process" The project will lead to increased process efficiency by enabling commonly used IT services and standardization. Re-using IT services can reduce the investment cost for new IT systems in the ESS. The SERV project implements the ESS Statistical Production Reference Architecture (which is part of the ESS EA Reference Framework) which provides guidance for the development and the sharing of statistical production systems and components in the ESS. It builds on the principles defined in the Common Statistical Production Architecture (CSPA) project 2 sponsored by the High- Level Group on the Modernisation of Statistical Production and Methods of UNECE. The purpose of the SERV project was defined as follows in the business case 3 : The project will provide guidelines for describing statistical services, recommendations for setting up appropriate governance for sharing of services across processes and among ESS and Eurostat, as well as advice on available products (including open source) to be used to set up service oriented architecture in an organisation. The SERV project is structured in three phases 4 : The first phase 1 ( ) focused on providing the necessary technical infrastructure for developing IT services to be shared in the ESS. 1 Software component implementing statistical business function that can be reused in the statistical process of more than one organization. 2 The description of CSPA v1.5 can be found on the following web page: 3 The business case of project SERV is available here: Business case v0.7.pdf 4 A more detailed timetable can be found in Annex 1 (page 10) of this document 3

4 The second phase ( ) focused on creating guidelines, governance, templates and example services (selected according to the priority areas of the Vision) to foster the sharing and reuse of IT services in the ESS. The third phase ( ) will focus on the deployment of the governance set up in the previous phase and the lessons learnt, and will further develop and roll out IT services in the statistical production environments of the NSIs. Additional deliverables will be introduced to expand those provided in the business case. Ultimately the objective of the ESS Vision 2020 SERV project is to allow ESS partners to adopt and benefit from the sharing and reuse of services in their production processes providing an adequate governance, a service catalogue and a set of initial services covering all key steps of the GSBPM. 3. MAJOR PROJECT DELIVERABLES The SERV project materialises the objectives set out above in two main deliverables: i. Governance. An important aspect for optimizing the IT investment in the ESS and to benefit from the development and adoption of shared services is to define and adopt a governance model to help the ESS to share the IT investment priorities. The project has developed the following aspects: a. Guidelines and "how to" documents to support the creation of shared services (based on UNECE's CSPA standard). b. Guidelines for license selection and a set of funding models to help organisations developing IT services to select the best model according to their capacity and needs. These models will explore different possibilities like "pure" open source sharing code without support (re-usability of code and libraries within the ESS), community based development and support (re-usability of products, shared development and support) or license based models (central support for services based on shared funding by users of the services). c. Methodology for the identification of priorities for IT investment for the development of shared services, based on periodic surveys to IT Directors and yearly workshops to maintain the selection of priority services for the ESS. d. A service catalogue with a governance to assess the compliance of the services to CSPA before admission and to manage the status from their inception to their end of life using allowing users to know at a glance if a service is usable or not. It will act as an entry point for the market place where providers' and users' needs can meet. ii. Services. The sharing and reuse requires a critical mass of services to enable efficiency gains in organisations needing to modernise their IT systems. The SERV project is providing services to share or reuse as well as "Lessons Learnt" documents to speed up future implementations by gathering the challenges and benefits. It will also build on its own results and deliver services to cover the spectrum of the GSBPM first layer. The implementation phase delivered 7 services which are being reused both at Eurostat and ESS partners level. The deployment phase will deliver another batch of services and reuse cases. Through a prioritisation mechanism established in the governance, these candidate services will be revaluated to make sure the best value services are selected at ESS level. 4

5 4. BENEFITS OF THE SERV PROJECT Cost rationalisation Sharing software components will help ESS members to optimize IT investment by allowing the collaboration at the different stages of software creation. The pooling of resources, from the design to the implementation, testing and support phases, will permit the ESS partners to produce and have access to more solutions with better quality at reduced costs. Standardisation The use of different tools across the ESS members when producing statistics can lead to incoherence because of different methodologies, technologies, versions thereof. As ESS business processes become increasingly standardized, software components could be designed and operated in ways that allow their reuse and sharing across ESS members. Integration The replication or use of shared services ensures the use of common technologies, methodologies and versions thereof ensuring better quality and time to market for European statistics. It will also lead to better quality of the tools overall thanks to collaboration between ESS members when identifying issues or addition of new features agreed upon from the start. 5. SERV COOPERATES WITH OTHER PROJECTS OF THE ESS ESS Vision 2020 ESBRs ESS partners are working on the development of the national Statistical Business Registers (SBRs) and the EuroGroups Register (EGR 2.0) under the scope of the ESS VIP project European System of Interoperable Business Registers (ESBRs). The objective is to provide ESS statisticians with adequate statistical frames to produce relevant and consistent business statistics at national and European level. Building on the recent achievements of the project, and following extensive consultation with Member States, the ESBRs project team is organising pilots on interoperability that will illustrate how Member States can make the best use of the outputs of the ESBRs project at national and European level. The ESBRs interoperability pilots constitute a key aspect of the ESBRs business case, as approved by the ESSC in To achieve its objectives, IT statistical services will be used. The SERV project assists in defining the "Identification Service" which will be a key tool for the business users. It also provides assistance in the context of the interoperability pilots with ESDEN in the use of existing service on a secure connection (e.g. SDMX-RI). ESS Vision 2020 ESDEN ESDEN is connecting ESS members through secure networks, thus bringing higher security to the transmission of data instead of using a regular internet connection. These secure networks now constitute the highways over which statistical information can travel safely between organisations of the ESS. In connection with ESBRs, SERV provides the assistance in using services on the secure transport provided by ESDEN. As a result SERV will provide an infrastructure to host the services in 5

6 coordination with ESDEN to host the selected services of EGR and allow the interoperability pilots to take place. Validation Rules and Services (follow-up actions of ESS Vision 2020 Validation) Following the ESS Vision 2020 Validation project, with its validation handbook and the different validation scenarios that could be implemented by the ESS partners, the actions around "Validation Rules and Services" have been put in place to move forward and implement the results. SERV publishes CSPA compliant services for validation (Structure and Content Validation) allowing NSIs to test the validation scenarios and methodological handbook in production conditions (PT, AT and SE are using the services in this context). SERV assists the countries with architectural advice on service oriented architecture and on the different ways of connecting to the existing services. ESS Vision Supporting Framework ESS Enterprise Architecture By implementing the Statistical Production Reference Architecture (SPRA) which is part of the ESS EA Reference Framework, SERV contributes to its continuous improvement and deepening by developing more specific artefacts that will be integrated in the SPRA. Furthermore, these two projects contribute to facilitating and encouraging NSIs to adapt their architecture to take benefit of shared development in the ESS. 6. CONSEQUENCES FOR NATIONAL STATISTICAL INSTITUTES Sharing IT services will help ESS partners to optimize their investments by finding synergies and sharing IT investments among them. The governance structures created during the project will help the ESS members to create a priority map on IT services that can be helpful to foster partnership and cooperation. This priority map will be periodically reviewed by using surveys to the IT directors and regular workshops aiming to create a marketplace where producers of IT services can join forces and meet demands of IT services consumers. Based on this priority map, organisations could build IT services using the guidelines and different funding and support schemas delivered by the project and these services can be published in the service catalogue that will contain up to date information of the already published services. In addition, countries will start benefiting from the services that the project is making available in fields like validation or those that have been developed for the ESS VIP projects. 7. WHAT HAS ALREADY BEEN DONE Results of the first phase During the first phase of SERV, the project set up the technical environment for hosting the IT services, including the creation of a service catalogue and the set-up of the infrastructure to deploy IT services in the ESS. The project has also contributed to the ESS Vision 2020 portfolio, in assisting Validation and ESBRs projects. The governance structure of the project has been created (Steering Group and Shared Services Task Force) and with their support and in consultation with other ESS bodies (DIME/ITDG and VIG) the project has identified the priority areas that should be targeted for the development of IT services. The latter include: 6

7 Implementation of the ESS Vision 2020, with a focus on software components supporting Validation and ESBRs; Coverage of integrated production processes in order to allow ESS members to use IT services for the end-to-end statistical production chain: The goal will be to identify services that can be used to implement one of the high level steps defined in the GSBPM standard (collection, editing, production and dissemination) and make it reusable by the ESS members. A list of IT services based on these priorities has been identified to be implemented in the second phase of the project. The main high-level deliverables of phase 1 are summarized in the table in Annex 2. They were all delivered, with the collaboration with other ESS Vision projects being a continuous activity throughout the project. In this phase, the Member States have been actively involved in the project by participating both in the Steering Group and in the shared services Task Force. Results of the second phase During the second phase of SERV, the project executed an ESSnet which delivered the technical guidelines, governance of the IT shared services, recommendations on using Open Source to build solutions for statistical production. Three example services were developed and used in a production environment by the ESSnet (see Annex 3), providing valuable lessons learnt. A new version of the services catalogue including a governance workflow to manage the integration and follow the status of the services has been developed. Its content will be periodically reviewed to ensure that the content is up to date. Eurostat also delivered four new and adapted services to be shared with the statistical community (see Annex 3). In this phase actions have been put in place which will be kept beyond the end of the project to ensure sustainability. Yearly, an updated list of candidate services based on an ITDG survey and aligned with the Investment Intention Survey of UNECE will be produced hence keeping the ESS strategy on services up to date. At the same frequency a workshop will be organised to foster collaboration and present a market place where people can informally discuss and exchange to try and match their needs for new services implementations. Funding models have been discussed ranging from the "as-is" open source approach to full fledge licensing. In a parallel context, a licensing matrix is being developed to help ESS partners and others to make the right choice when providing their software for sharing. 7

8 8. DEPLOYMENT OF DELIVERABLES The table below is a proposed list of deliverables for this last phase which will be reviewed by the different ESS bodies with a view to be endorsed by the VIG followed by the ESSC in November ID Name Description Estimated D 3.1 Updated governance guideline documents ESS and Building on the experience acquired during the last phase, refined version of the guidelines allowing for sharing of statistical services across processes and among partners in the ESS and Eurostat (Commission) as well as of the recommendations for governance of statistical services; including the setup of an (ESS) service certification committee and maintenance of the content of an ESS service catalogue. June 2019 D 3.2 New Services Statistical Implementation or adaptation of new statistical services with compliant service documents and lessons learnt to be added to the service catalogue Iterative delivery until June 2019 D 3.3 Enhanced features for the Service Catalogue D 3.4 NSIs using shared services available in the catalogue D 3.5 Governance structure for the sustainability beyond the project phase New features to be added to the service catalogue for improved lifecycle management of statistical services from definition to implementation and until end of life Durable implementation into their statistical production for at least one statistical domain. It will considered a success criteria that at least 5 NSIs make use of available services in the service catalogue. Setting up the governance and work packages of a Centre of Excellence or other sustainable instrument to support the process phase of the Sharing and reuse of statistical services initiative Iterative delivery until June 2019 December 2019 December WHAT WILL BE DONE NEXT / ROADMAP In the second phase of the project, the survey launched to IT Directors by the ESSnet allowed gathering strategic priorities in the implementation of services for the third phase of the project. This phase of the project will deploy the results obtained in the second phase. It will: Use the ESS guidelines and governance as well as follow the service priorities, Implement new services and integrate them in the service catalogue, 8

9 Continue by helping NSIs to use these and other existing services in their statistical production process. Use the ESS guidelines and governance The second phase of the project has delivered guidelines and governance documents which follow international standards (CSPA, GSBPM, ), to foster and facilitate sharing and reuse, which NSIs willing to participate and take advantage of the Sharing Services initiative should follow when implementing new services. In the third phase NSIs will be able to start using them. The selection process of services to implement will follow the governance and take into account the priorities defined by the ITDG survey and the ultimate goal of the project, while grants will be put in place where the use of the service guidelines will be made mandatory. This approach will allow for refining those documents in a continuous improvement process until the end of the project. Implement new services and integrate them in the service catalogue Building on the deliverables of the second phase (service catalogue, governance documents, SOA infrastructure, lessons learnt), new services will be implemented to deploy the use of these deliverables. They will be added to the service catalogue and publicised, enhancing the goal on the fostering of collaboration. An ESSnet is foreseen for these implementations enhancing the collaboration between ESS partners. This approach to the deployment of the outcomes of the project will also allow meeting the ultimate goal by selecting services taking into on their position in GSBPM and covering the full layer 1 of the standard. Continue by helping NSIs to use these and other existing services in their statistical production process In order to enhance the collaboration and to reap the benefits of the services made available, the third phase of the project will launch grants to encourage NSIs to implement shared services in their statistical production process thus standardising and quickening the ESS statistical production overall and allowing for streamlined and more economical implementations. During this third phase, the project will also provide means to keep the ESS strategy up to date for the implementation of services by organising regular surveys and regular workshops with a "boot camp" and a "market place" to help matching their needs for shared services, identifying potential synergies, assessing their maturity to benefit from shared development, choosing the best solution among the different options and guiding in the adaptation of their architecture where needed. Involvement of Member States SERV follows a community based approach for European cooperation. The ESSnet "Sharing common functionalities in ESS" (ESSnet SCFE) provided key deliverables of the project like the definition of the governance, the service guidelines and the implementation of some of the services proposed in the second phase of the project. For the third phase, the countries are expected to make use of existing services and to create new services following the investment priorities identified and the guidelines and governance model proposed during the second phase of the project. An ESSnet and other instruments will be made 9

10 available to support the involvement of the Member States. The Steering Group is a continuous body of the project and the Task Force has been approved to continue its work ensuring ESS partners are beefing involved throughout the life of the project. 10. SERV NEXT MILESTONE Following the third phase of the project, which will be focused on deploying the project deliverables, including in terms of capacity building, across the ESS Member States, the project will come to an end in A closing report will be presented to the VIG with proposed steps and governance to transition into the process phase of the Sharing Services initiative in the ESS and ensuring its continuity beyond the project phase. 10

11 ANNEX 1 - MASTER PLAN The update master plan of the project is defined as follows: ANNEX 2 PHASE 1 DELIVERABLES Creation of the service catalogue Creation of ESS Infrastructure for hosting central services Support for VISION projects Projected December 2015 ESTAT status Delivered Description The catalogue is a repository of IT services, with documentation and operational guidelines on software components. It is hosted and maintained by EUROSTAT. Status and impact The service catalogue is in production and used by various statistical organizations. date December 2015 ESTAT Status Delivered Description The infrastructure is used to expose software components to be shared by ESS members. Status and impact The infrastructure is configured and available; the first IT service has been deployed by Eurostat. Projected Continuous ESTAT Status Ongoing Description Ensure the alignment of ESS Vision 2020 projects with the Service Oriented Architecture guidelines. Status and impact Advice on IT architecture was provided to the Validation project and a first IT service (for structural validation of data sent to Eurostat by ESS members) has been deployed in relation with that project. Coordination work was also started with ESBR and DIGICOM. 11

12 Governance structures and work packages for phase II Projected December 2015 ESTAT Status Delivered Description Setting up the project governance by creating and launching a Task Force, a Steering Group and an ESSnet Status and impact The Task Force was created and had its first meeting in October 2015; The Steering Group was created and had its first virtual meeting in December 2015; The ESSnet was created and launched. It started its work in January ANNEX 3 PHASE 2 DELIVERABLES New version of the service catalogue Example services (see Annex 4) Governance Support for VISION projects Projected December 2015 ESTAT status Static version 06/2017 Complete Planned 12/2017 Description The catalogue is a repository of IT services, with documentation and operational guidelines on software components. It provides a workflow for assessment and lifecycle management. It is hosted and maintained by EUROSTAT. Status and impact The service catalogue is in production and used by various statistical organizations. date December 2017 ESSnet Status Iterative until 12/2017 Description Seven services are planned to be delivered by the ESSnet and ESTAT and inserted into the service catalogue. Status and impact 6 services planned were delivered.1 service not planned was delivered, 1 service planned will not be delivered on time. Projected December 2017 ESSnet Status First version 01/2017 Update 12/2017 Description Guidelines to manage IT services to be adopted in the ESS as well as recommendations for governance of IT services; including the setup of an (ESS) service certification committee and maintenance of the ESS services listed in the service catalogue. Open source handbook Status and impact First deliveries received 12/2016. Updates and last documents on track. Projected Continuous ESTAT Status Ongoing Description Ensure the alignment of ESS Vision 2020 projects with the Service Oriented Architecture guidelines. 12

13 Status and impact Advice on IT architecture was provided to the Validation project and a first IT service (for structural validation of data sent to Eurostat by ESS members) has been deployed in relation with that project. Coordination work was also started with ESBR and DIGICOM. ANNEX 4 LIST OF SERVICES IMPLEMENTED IN PHASE TWO In the first phase of the project, the priorities for the development of services (ESS Vision 2020 and coverage of GSBPM activities) were identified. Based on these priorities, 6 new IT services have been developed in the second phase of the project. Name Actor Main function Priority Area Validation services Eurostat Data validation by Member States before the data is sent to Eurostat. The services include content validation, according to validation rules defined for the different statistical domains and validation of the structure of the files. ESS Vision 2020 (Validation) Transformation Service Eurostat Perform data conversion between different file types (from CSV to SDMX-ML and vice versa) to support standardisation on data collection and validation. ESS Vision 2020 (Validation) Time Series ESSnet Perform seasonal adjustment based on the functionalities offered by JDemetra+ GSBPM (Process) Questionnaire Generation ESSnet Automated generation of questionnaires (web, PDF) from their DDI description. GSBPM (Design) Metadata Dissemination ESSnet Access to different types of information about statistical concepts and classifications (list of items, correspondence tables, etc.) GSBPM (Disseminate) SDMX Data Dissemination Eurostat Dissemination of statistics according to the SDMX standard. It can be used to plug it in a dissemination database and to generate automatically data in SDMX-ML format for the dissemination and data sharing. GSBPM (Disseminate) SDMX-RI Eurostat Directly expose data and metadata stored in a database, in SDMX-ML format via queries. GSBPM (Disseminate) These services will be added to the service catalogue once ready to be deployed. In addition, the ESSnet will ensure communication and support in first use of the IT services in the ESS. Note 1: The Identification Service planned from the ESBR project will probably not delivered in the context of the second phase. Note 2: The SDMX-RI and SDMX registry services were adapted by Eurostat to be CSPA compliant while they were not initially planned. 13