INFRASTRUCTURE AS CODE

Transcription

1 INFRASTRUCTURE AS CODE

2 Cloud Practice Lead (UK) DevOps, Continuous Delivery, Agile Ops Book: Site: June 2016

3 AGENDA CONTEXT Motivations Challenges INFRASTRUCTURE AS CODE Key Practices Simple Pipeline Scaling Pipelines

4 SPEED Get something to market quickly Iterate it Continuously improve it

5 TECHNOLOGY Cloud, automation, etc. lowers barriers for making changes

6 DANGER Security Performance Stability Compliance Maintainability

7 SECRET High quality services rely on the ability to make changes quickly

8 GOAL Be able to make changes Rapidly, Frequently, and Responsibly

9 CHALLENGES

10 SERVER SPRAWL Creating new servers is the easy part

11 CONFIGURATION DRIFT Servers start out identical But changes accumulate over time

12 AUTOMATION FEAR CYCLE

13 INFRASTRUCTURE AS CODE Applying software engineering tools and practices to infrastructure

14 UNATTENDED AUTOMATION Tools run on a schedule to apply, re-apply, and update configuration BENEFITS OF UNATTENDED: Discover problems quickly Force yourself to fix those problems Force yourself to improve your tools and processes Discourages out of band changes

15 AUTOMATE SERVER UPDATES Automation isn t just for new servers! Configuration synchronization Run Chef, Puppet, Ansible, etc. on a schedule Immutable servers Apply changes by rebuilding servers Containerized servers Apply changes by deploying new container instances

16 RE-USE & PROMOTE DEFINITIONS Re-use the same definition files across environments for a given application or service Playbooks, Cookbooks, Manifests, templates, etc. DEV STAGE PROD

17 TEST INFRASTRUCTURE CHANGES Preventing DevOops INFRA TEST DEV TEST PROD

18 PIPELINES Using Continuous Delivery pipelines to manage infrastructure

19 WHAT? Changes are made and committed to VCS Terraform, Puppet, etc. Tools are run on agents to apply changes to environments Changes are only promoted after passing tests & authorization

20 WHY? Validates changes to infrastructure before applying them to production Confidence for frequent, small improvements to infrastructure Limit direct changes to infrastructure

21 TESTING Correctness Performance Security policies Stability

22 GOVERNANCE The process for applying changes is auditable Changes can be traced back to commits Automation ensures processes are followed Authorization can be required as needed

23 SIMPLE An example with a fairly simple environment

24 DEFINING A SIMPLE ENVIRONMENT VPC TERRAFORM FILE Environment structure ANSIBLE PLAYBOOK Server configuration APPLICATION SOURCE Deployable application Subnet /16 Security Group /16 -> :443

25 SIMPLE PIPELINE DESIGN Deploy application, configuration, and infrastructure Application Ansible BUILD STAGE Terraform TEST STAGE QA STAGE PROD STAGE

26 SCALING Handling more complex infrastructure

27 ALIGN INFRASTRUCTURE DESIGN TO TEAMS Ensure teams can make the changes they need easily and safely

28 COMPLEX ENVIRONMENTS Infrastructure involving multiple teams

29 FAN-IN PIPELINE ServiceA BUILD SERVICE TEST ServiceB BUILD SERVICE TEST ServiceC BUILD SERVICE TEST SYSTEM TEST QA PROD

30 DECOUPLED PIPELINES ServiceA BUILD TEST QA PROD TEST QA PROD TEST QA PROD ServiceB BUILD ServiceC BUILD

31 DEPENDENCIES ServiceA BUILD Use mocks and stubs TEST QA Implement Consumer Driven Contract (CDC) Tests Create test instance of provider BUILD ServiceB PROD TEST QA PROD

32 ISSUE: DUPLICATION Multiple teams using similar systems, e.g. database clusters

33 RE-USE BY FORKING DEFINITIONS Disadvantages: - Divergence and Inconsistency Advantages: - Avoid tight coupling - Handles diverse requirements

34 RE-USE WITH DEFINITION LIBRARIES Challenges: - - Avoid tight coupling, so teams aren t blocked when making changes Ownership of code shared by multiple teams Guidance: - Use separate pipelines for each Use CDC & other dependency testing strategies

35 LIBRARY PIPELINE Test shared definitions before pulling them into dependent pipelines Server AMI BUILD TEST Service BUILD TEST

36 ISSUE: SHARED ELEMENTS ServiceA VIP Shared infrastructure definitions ServiceB VIP LOAD BALANCER Service-specific infrastructure definitions

37 SHARING ELEMENTS Avoid monoliths - optimize to simplify making changes

38 OUTCOMES Quickly provision and evolve infrastructure Effortlessly roll out fixes Keep systems consistent and up to date Spend time on high value work

39 Cloud Practice Lead (UK) DevOps, Continuous Delivery, Agile Ops Book: Site: