Adopting DevOps at Enterprise Scale Presented by: Sanjeev Sharma CTO, DevOps Technical Sales and Adoption IBM Distinguished Engineer @sd_architect 1 IBM Corporation
DevOps: Origins 2 IBM Corporation
DevOps in the Enterprise: Multiple Bottlenecks Line-of-business Customer Team Lead Team Member Users External System Team Auditors Domain Experts Product Owner Team Member Team Member Gold Owner Senior Executives Support Staff Operations Staff 3 IBM Corporation
DevOps approach: Apply Lean principles accelerate feedback and improve time to value Line-ofbusiness Continuously Improve: I. Application Delivered 1 Process 1. Get ideas into production fast 2. Get people to use it 3. Get feedback 2 People II. Environment Deployed 3 Customer III. Application and Environment Delivery Process 4 IBM Corporation
Lean & Agile are at the heart of IBM s DevOps approach Balance efficiency and effectiveness to deliver the right things right! LEAN Reduce work Eliminate bottlenecks and waste Efficiency Fast response times Small batch sizes Continuous feedback AGILE Effectiveness 5 IBM Corporation
Voice of the IT Senior Application Development Executive Steve Farley, VP Application Development Center, Nationwide Insurance Carmen DeArdo, Director, Build Technology Leader, Nationwide Insurance 6 IBM Corporation
More executive perspectives on DevOps Gareth Wharton, CTO, Hiscox Snehal Antani, CIO CDF & CIO EA, GE Capital Americas We built software and apps, such as our award-winning bank GECapitalBank.com from scratch in weeks to months instead of months to years. 7 IBM Corporation
Patterns of DevOps Adoption 8 IBM Corporation
Patterns of DevOps Adoption Three Key Business Drivers 1. Driving Business Agility 2. Scaling for the Enterprise 3. Driving Innovation 9 IBM Corporation
Adoption Pattern #1: Driving Business Agility 10 IBM Corporation
Adoption Pattern #1: Scenarios Driving Business Agility Continuous Delivery Shift Left Test Shift Left Ops Engagement Lean' Application Delivery 11 IBM Corporation
Where does DevOps Adoption Start? Agile Adoption Plan Develop Build Test Deploy Production Water-SCRUM-fall By the end of 2015, 75% of large organizations are expected to have adopted agile DevOps practices (IDC) 12 IBM Corporation
Going from Agile to DevOps End-to-End Delivery Pipeline 13 IBM Corporation
Collaborative Development Continuous Integration Single Source of Truth Requirements, Source Code, Tests, Deployable assets, Software Defined Environments Common Dashboards to show metrics 14 14 IBM Corporation
Deployment Automation Manage application components and versions Manage configurations across all environments 15 Offer secure self-service capabilities Increase transparency Ensure governance and compliancy 15 IBM Corporation
Shift Left Test: Deploy what is ready, Virtualize the rest IBM UrbanCode Deploy Test Environments virtual components integrated with Test Data Management Deploy private, secure right sized test data Databases Internal Messages Third-party Services Test Virtualization Test Automation Deploy what is ready, virtualize the rest Continuously test in production-like env. Dev QA Simultaneously test across multiple test stages Quality Management Rational Test Workbench Manage quality across the delivery lifecycle Dynamic Infrastructure 16 IBM Corporation
Shift Left Ops Engagement Deploying Infrastructure is the biggest bottleneck for the Delivery Pipeline Software Defined Environments enable you to capture infrastructure as a software artifact Extending the Agile Manifesto: Working software over comprehensive documentation -> Working software (in Production or production-like environment) over comprehensive documentation Application Changes Infrastructure Changes 17 IBM Corporation
Case Study: Large Bank Lean Delivery Pipeline Challenge: Developers were creating daily builds QA team had a 3 5 day cycle time Bottlenecks Identified: Large Sizes of Testing Deployments Lack of Deployment Automation Ticket based manual environment provisioning Lack of reliable source of Test Data Solution: Deployment Automation with IBM UrbanCode Deploy Cloud hosted on-demand environments with IBM UrbanCode Deploy with Patterns Test Automation and Service Virtualization with IBM Rational Test Workbench Test Data Management with IBM Optim Test Data Management 18 IBM Corporation
End-to-end testing with Continuous Delivery and Test Virtualization SCM Build Cloud Platform Deploy Deploy App and Configure Middleware UAT Optim plugin UrbanCode Deploy RTW plugin Test Data Management RTVS plugin Configure and start Virtual Services Test Rational Test Workbench Extract Data, Insert Data, Convert/Mask Data Update Data Databases Mainframe applications Third-party Services Virtualized Services Rational Test Virtualization Server Databases Mainframe applications Actual Services Optim Test Data Management 19 19 IBM Corporation
Adoption Pattern #2: Scaling for the Enterprise 20 IBM Corporation
Adoption Pattern #2: Scenarios Scaling for the Enterprise Release Automation for 2-speed IT Standardization across the enterprise 21 IBM Corporation
Applications, teams and environments are hybrid : 2-Speed IT Rapid iterations Plan Develop Build Test Deploy Production Cloud Alignment Continuous synchronization and planning Continuous testing Continuous deployment and monitoring Slower iterations Plan Develop Build Test Deploy Production Behind the firewall 22 IBM Corporation
Orchestrating Complex Releases Keep track of the inventory across the entire life cycle of the release Control the entire release lifecycle in an easy to use view 23 IBM Corporation
Application Deployment to Hybrid Environments IBM UrbanCode Deploy with Patterns Systems of Engagement IBM Bluemix Docker Cloud Foundry Containers Virtual Machines IBM PureApplication System IBM Cloud Orchestrator IBM Cloud Manager with OpenStack On-prem IaaS Systems of Record IBM UrbanCode Release On-prem Traditional IT 24 IBM Corporation 24
Adopting DevOps for 2-speed IT Case Study: Multinational Financial Services Provider Challenges: Delivering a Business Capability with multiple services and applications across Hybrid Environments Bottlenecks Identified: Solution: Lack of Release Planning Inability to test when needed, as services are not available/ready Coordinating complex Releases, made up of deployments across Hybrid Environments Release Planning with IBM UrbanCode Release Deployment Automation a cross Hybrid environments with IBM UrbanCode Deploy with Patterns Test Automation and Service Virtualization with IBM Rational Test Workbench Rapid Deployment across distributed datacenters with Aspera 25 IBM Corporation
Application Deployment for Hybrid environments IBM UrbanCode Release Rational Test Workbench IBM UrbanCode Deploy with Patterns Provision Application On-prem Customer Data Centers Provision Application + Infrastructure Public Data Centers Provision Application + Infrastructure CloudFoundry Community Buildpacks Platform as a Service (Public) Legacy Stack On-Prem Cloud Off-Prem Cloud IBM DevOps Services System z and/or Distributed Applications API Gateway APIs Pattern Engine Private Cloud IBM Cloud Orchestrator IBM Cloud Orchestrator as a Service APIs BlueMix Infrastructure Infrastructure Aspera 26 IBM Corporation
Adoption Pattern #3: Driving Innovation 27 IBM Corporation
Adoption Pattern #3: Scenarios Driving Innovation Rapid Delivery for Innovation/Experimentation Architecting for the Cloud 28 IBM Corporation
Innovation: Rapid Deployment and Feedback for Experimentation Line-ofbusiness 1 2 3 Customer 1. Get ideas into production fast 2. Get people to use it 3. Get feedback 4. Improve and Iterate Canary Testing, A/B Testing Containerization Microservices 29 IBM Corporation
Build your apps, your way Bluemix is powered by a full open cloud architecture Bluemix supports a wide diversity of workloads by offering developers a range of choices associated with how they build and run their apps. App-centric runtime environments Flexible and powerful VMs IBM Bluemix Portable and consistent containers Pure PaaS Support for node.js, Java, Go, Python, Ruby, Ruby on Rails,.php and includes the ability to bring your own buildpack Focus on your code and your data, the platform manages the rest Deploy and manage VMs consistently for dedicated and public clouds through a single dashboard using open source technologies. A user-specific repository of images which are trusted and secure High availability of the IBMhosted container images Integrated networking and storage 30 IBM Corporation
IBM Bluemix DevOps Services Source Control Hosted Git Jazz SCM Plan and code Agile Planning Web IDE Feedback Monitoring & Analytics Mobile Quality Build and Deploy Continuous Integration Continuous Delivery 31 IBM Corporation
Multiplicity of hardware environments Multiplicity of Stacks Docker A shipping container system for code Static website An engine that enables any payload to be encapsulated as a lightweight, portable, selfsufficient container User DB Web frontend Queue Analytics DB Do services and apps interact appropriately? Development VM that can be manipulated using standard operations and run consistently on virtually any hardware platform QA server Customer Data Public Cloud Production Contributor s Center Cluster laptop 32 IBM Corporation Can I migrate smoothly and quickly
Why it works DevOps Nirvana? Dan the Developer Worries about what s inside the container His code His Libraries His Package Manager His Apps His Data All Linux servers look the same Oscar the Ops Guy Worries about what s outside the container Logging Remote access Monitoring Network config All containers start, stop, copy, attach, migrate, etc. the same way 33 IBM Corporation
IBM s Delivery Pipeline for Containers IBM Container Image Build Service Image Registry IBM UrbanCode Deploy with Patterns 34 IBM Corporation
Creating a DevOps Culture And now for the hard part 35 IBM Corporation
36 IBM Corporation
It s all about the people! Building a DevOps Culture grounded in lean and agile principles Everyone is responsible for Delivery Common measures of Success Setup a DevOps Center of Excellence Team Lead Team Member Users External System Team Auditors Domain Experts Product Owner Team Member Team Member Gold Owner Senior Executives Support Staff Operations Staff 37 IBM Corporation
Start Here: Value Stream Mapping for Identifying and Addressing bottlenecks 38 IBM Corporation
Bottlenecks in your Delivery Pipeline Address issue of congestion in software delivery 1 per min 1 per min 4 per min 1 per min 4 per min 4 per min 39 IBM Corporation
Tasks Artifacts Mapping your Delivery Pipeline Line of Business PMO Requirements/ Solution: Agile Transformation with Analyst Risk-Value based Release Process Management Variants Feedback Idea/Feature/Bug Fix/ Enhancement Bottleneck: Rigid One-sizefits-all Development Customers process Deployment Engineer Deploy Bottleneck: Get Ticket Feedback Based Bottleneck: Weekend long Deployments that often fail Production Development Build QA SIT UAT Prod Customer or Customer Surrogate Solution: Frequent Deployment Environment Provisioningof Bottleneck: Small Batches Late of Discovery Change of Architectural Fragility Developer Code Repository Build Engineer QA Team Integration Tester User/Tester Operations Solution: Cloud Hosted Developer Artifact Self-Service Repository Metrics - Reporting/Dashboarding Solution: Agile Shift Left Integration Testing to early in LifeCycle Infrastructure as Code/ Cloud Patterns 40 IBM Corporation
Resources Continuing your DevOps Adoption journey For Dummies books: http://ibm.co/devopsfordummies http://ibm.co/agilefordummies http://ibm.co/servicevirtualizationfordummies http://ibm.co/ardfordummies DevOps Lean Assessment (Beta): http://bit.ly/ibmleanassess IBM DevOps Page: http://ibm.com/devops IBM DevOps YouTube Playlist: http://bit.ly/1fidotl 41 IBM Corporation
Questions? 42 42 IBM Corporation
Acknowledgements and disclaimers Availability: References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. Copyright IBM Corporation 2013. All rights reserved. U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. IBM, the IBM logo, ibm.com, Rational, the Rational logo, Telelogic, the Telelogic logo, Green Hat, the Green Hat logo, and other IBM products and services are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol ( or ), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at Copyright and trademark information at www.ibm.com/legal/copytrade.shtml If you have mentioned trademarks that are not from IBM, please update and add the following lines: [Insert any special third-party trademark names/attributions here] Other company, product, or service names may be trademarks or service marks of others. 43 IBM Corporation
Thank You Copyright IBM Corporation 2013. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others. 44 IBM Corporation
Backup Slides 45 IBM Corporation
End-to-end testing with Continuous Delivery and Test Virtualization SCM Build Cloud Platform Optim plugin Deploy UrbanCode Deploy RTW plugin Test Data Management RTVS plugin Deploy App and Configure Middleware Configure and start Virtual Services Test Rational Test Workbench Extract Data, Insert Data, Convert/Mask Data Update Data IBM UrbanCode Deploy: UAT Continuous Delivery to Dev Test environments Configure Middleware and Environments Configure Virtual Test Servers Inventory Management Security Management Databases Mainframe Third-party Challenge applications Addressed: Services Rational Test Virtualization Server Databases Mainframe applications Developer Virtualized Services Self Service Actual for end- Services to-endtesting Optim Test Data Management 46 46 IBM Corporation
End-to-end testing with Continuous Delivery and Test Virtualization IBM Rational Workbench: Virtualize Services for testing Manage Virtualized Test SCM Build environments Integration testing Performance testing Security Testing with AppScan Deploy UrbanCode Deploy Optim plugin RTW plugin RTVS plugin Deploy App and Configure Middleware Challenge Addressed: Early testing (Shift-left) Continuous end-to-end testing Configure and start Remove dependencies Virtual on Services backend service availability for testing Remove dependencies on test Test Data Management environment availability Virtualize expensive mainframe services Test Rational Test Workbench Extract Data, Insert Data, Convert/Mask Data Update Data Cloud Platform Databases Mainframe applications Third-party Services Virtualized Services Rational Test Virtualization Server UAT Databases Mainframe applications Actual Services Optim Test Data Management 47 Deploy what is ready, virtualize the rest 47 IBM Corporation
End-to-end testing with Continuous Delivery and Test Virtualization SCM Build Cloud Platform Optim plugin Deploy UrbanCode Deploy RTW plugin Test Data Management RTVS plugin Deploy App and Configure Middleware Configure and start Virtual Services Test Rational Test Workbench Extract Data, Insert Data, Convert/Mask Data Update Data Databases Mainframe Third-party Databases Mainframe applications Services Test Data Management applications Virtualized Services Rational Test Virtualization Server UAT IBM InfoSphere Optim: Test Data management Mask and provide sets of test data for Continuous Testing Challenge Addressed: Actual Services Optim Test Data Management 48 48 IBM Corporation
End-to-end testing with Continuous Delivery and Test Virtualization SCM Build Cloud Platform Optim plugin Deploy UrbanCode Deploy RTW plugin Test Data Management RTVS plugin Deploy App and Configure Middleware Configure and start Virtual Services Test Rational Test Workbench Extract Data, Insert Data, Convert/Mask Data Update Data Databases Virtualized Services Rational Test Virtualization Server UAT IBM UrbanCode Deploy with Patterns: Dev-Test-Prod Environment Provisioning Full Stack Deployment Environment Portability Challenge Mainframe Addressed: Third-party applications Full Stack Services Provisioning Databases Mainframe applications Actual Services Optim Test Data Management 49 49 IBM Corporation
Standardization Across the enterprise: Patterns and Templates What to deploy? Where to deploy (Physical or Cloud Resources) - Patterns How to deploy? Application Blueprints in IBM UrbanCode Deploy with Patterns 50 IBM Corporation