ABN AMRO Transforms with CICD to Accelerate Software Delivery and Improve Security. DevOn Summit Utrecht 14 th Mar 2018 Stefan Simenon

Transcription

1 ABN AMRO Transforms with CICD to Accelerate Software Delivery and Improve Security DevOn Summit Utrecht 14 th Mar 2018 Stefan Simenon

2 Introduction Stefan Simenon Head of IT Tooling & Software Development Tel: Studied Physics & Information Technology 20+ years IT experience in various roles Currently responsible for Tooling, Software Quality & CICD in ABN AMRO IT department Speaker at several software conferences like Jenkins World, XebiaLabs DevOps Leadership Summit, AllDayDevOps, Software Quality conferences

3 ABN AMRO ABN AMRO is a leading bank with an operating income of EUR 8588 million 22,000 employees servicing retail, private and corporate finances worldwide Headquartered in Amsterdam 5,000 associates working in IT 300+ agile teams

4 Challenges Faced at ABN AMRO Long lead time for software delivery Software quality issues found at a late stage Many manual handovers and approvals Code merging happening at a late stage Inefficient cooperation between DEV and OPS Big non-frequent releases to Production

5 Financial services market is growing fast, on multiple fronts

6 Agile & DevOps transition Waterfall Traditional Enterprise, Agile teams Full Agile enterprise CICD / DevOps Full DevOps enterprise

7 The case for faster response to client needs is clear Continuous Integration Continuous Delivery Continuous Deployment Produce automated builds and detect errors as soon as possible, by integrating and testing all changes on a regular (daily) basis. High frequency delivery of a tested functional piece of software that can be deployed to production rapidly. Fully automated process including deployment to production without human interaction.

8 CICD program: Set-up PAVE THE WAY Automated production release Tooling Infra prerequisites Integration Pipelines Mature in UT/ST Move to ET MAKE IT HAPPEN Change Management Mindset & Behaviour Simplify processes Coaching for agile teams Extend technologies Front end, Java Start CICD in UT/ST

9 CICD program: Approach CICD is not only about tooling but mainly mindset & behaviour, a changed Way of Working and process improvements. The project organisation is set up into a cluster with a central and a decentralized orientation. 1. Pave the way: set up the conditions for the teams to get working. 2. Make it happen: the actual decentral CI/CD implementation within the teams. Agile teams will be supported once the right tools are available, so start with Java/Front End/BPM TIBCO. Strong alignment across DEV, OPS and SECURITY departments We know other large companies which need 3-8 years, and changed their approach along the way. Therefore we keep the overall stages in mind, but plan for the coming three months. Focus on learning and improving instead of long term planning.

10 Pave the Way Results so far (1) JIRA agile toolset defined and implemented Standard Way of Working defined and roll out in progress From 2000 to users in 2,5 years All tools required for Continuous Integration implemented and rolled out Various Continuous Integration pipelines defines and implemented Pipelines and their integrations are continuously improved and extended Tooling for release and deployment management selected: XL Release and XL Deploy Release & Deployment management WoW defined and roll out in progress Test & Production environments for XL Release and XL Deploy delivered, installation process has been fully automated Standard CD pipeline for Java/WebSphere, Open Banking and IIB delivered and connected to standard CI pipeline. VSTS selected and implemented for applications based on MicroSoft technology > 100 applications onboarded for automated deployments > 500 XL Release users

11 Pave the Way Results so far (2) SonarQube for code quality, HPE Fortify for secure coding, Nexus Life Cycle for OSS library management Governance to manage software quality setup and roll out in progress Build breakers defined and roll out in progress Tools implemented to enable automated testing Test Service Virtualization rolled out Automated test data management and governance implemented and roll out in progress Automated Test framework defined and implemented Mainframe tools upgraded to latest versions Identified strategy to clean unused components and activities to recompile programs based upon latest Cobol compiler 6.1. This will lead to improved memory usage and less MSU usage. Mainframe pipeline based on Compuware TOPAZ, ISPW, Jenkins and SonarQube in progress.

12 Midrange Build & Delivery pipeline: orchestration Release management Build Static secure code Continuous Delivery Zero touch platforms Code quality scans Package Build artefacts Deployment Build & Unit Tests Continuous Integration Develop Test data mgmt Test environment (ST) Acceptance environment (ET) Production environment (PRD) Source code ATAF Test suites

13 Pipelines within ABN AMRO Java Siebel Mobile Power Centre/ ETL IIB Mainframe Tooling Front End BPM/ TIBCO MicroSoft CoTS

14 Standard CI pipelines within ABN AMRO and build breakers Code quality scan N Developer triggers build Check out project from SCM Build project and execute unit tests Secure coding scan Y Publish Deployable artifact Dependency scan

15 Build breaker criteria and governance Software quality governance in place. If software quality criteria are not met, build will fail and software developer needs to fix/improve the software before being able to publish a deployable artifact. Software quality criteria and roll out of build breakers are defined by a development community consisting of central quality teams, representatives in agile teams, our application development partners and security department. Initial build breakers in place for software quality, secure coding and dependency management, build breakers criteria will be strengthened in the future. Build breakers lead to improved software and less exception discussions in agile teams. Senior management commitment in place.

16 An IT4IT organisation has been set up to enable the CICD implementation JIRA dedication team Application Deployment support team Implement tooling upgrades Implement new tools Enhance and improve CICD pipelines Implement new CICD pipelines Change & configuration mgmt team Software Logistics team Test tooling team Handle user management Support Agile teams Conduct incident & problem management Portfolio mgmt team Application logging team Mainframe modernization Application monitoring team

17 ABN AMRO CICD Key Principles Automate all repetitive tasks Integrate quickly and often Everyone is equally responsible 4. Keep changes small 5. Get continuous feedback

18 Make It Happen Results so far CICD summer event held incl. CICD leadership program, demo s, best practice sharing, trainings Change management program set up with lots of focus upon Mindset & Behaviour CICD coaching framework defined and rollout in progress 100+ boot camps organised and teams coached Framework based upon certain set of deliverables and team needs CICD E-Learning module delivered and rolled out Various communities set up Internal meet ups and hackathons regularly held Platform set up in which teams can present their successes, failures and how they learn Internal meetups held with external speakers and tooling suppliers (eg. Jez Humble, Josh Long, Cloudbees, Sonatype, XebiaLabs, SonarSource)

19 Realised benefits within ABN AMRO Test environment uptime improved Improved code quality & secure coding From 4 Internet Banking releases to 18 releases per year We never thought it would be possible to develop, test and deploy something completely in one sprint Private Banking Interlnational team reduced build from 5 hours to 5 minutes Improved cooperation across stakeholders Improved time to market Core review times have been shortened and violations when merging are being prevented I-Markets doubled velocity after 1 sprint containing CICD improvements only First continuous deployment realised by identity access mgmt team Improved development processes Increased velocity Changes are being rolled out as soon as they are available Release times halved for teams using XL Release x3 deployments to UT +20% successful Builds -100% Package creation time -75% Testing time x2,5 deployments to ET Code push flow Build, QA and package flow Deployment flow Develop Source code mgt Build & Unit test Code quality review Package Component mgt Deploy Test (ST) Deploy Release tests (ET) Deploy Prod checks Continuous integration Zero touch platforms Continuous delivery Continuous deployment

20 Take aways Senior management commitment & involvement Invest in reducing technical debt Create a safe environment (failing is ok) Do not focus on tooling only Do not underestimate the journey and complexity Do not focus on long term but small improvements

21 Way forward Automate and improve tooling pipelines Further transform to DevOps Improve WoW and Mindset & Behaviour Facilitate increased team autonomy Hybrid cloud strategy Database automation CICD metrics

22 Questions 22