FOR THE LOVE OF DEVOPS

Transcription

1 FOR THE LOVE OF DEVOPS A Case for an Increased Focus on DevOps Administrator Training D. L. Campbell 2014 CBT Nuggets. All Rights Reserved.

2 DevOps and customer-centric product delivery processes are two of the hottest topics in IT today, and job growth in these areas is extremely dynamic. DevOps-focused administrators perform the crucial role of ensuring quality product delivery with high uptime. This high-stakes and fast-paced position is increasingly important as the industry struggles to deal with the migration to cloud computing, virtual servers and services, and the automation of development, deployment, and quality assurance tasks. However, there is also a great amount of confusion and misunderstanding about what DevOps is. Is it just a mindset or work process, or are there specific skills and abilities that make it work? The answer to this question is continually evolving. One thing is known: Organizations that adopt DevOps can realize significant benefits. From the 2014 State of DevOps Report: We can now assert with confidence that high IT performance correlates with strong business performance, helping to boost productivity, profitability and market share. Our analysis showed that the longer an IT organization continues to practice DevOps, the more its IT performance improves. Because business outcomes are linked to IT practices, greater DevOps maturity gives companies a clear lead, as their business outcomes continue to improve over time along with IT performance. Despite the growing recognition of the value of DevOps, many traditional system administrators are hesitant to change. They may feel intimidated by the thought of introducing DevOps to their environment, and don t know what direction to take or where to start trying to embrace this new philosophy. In many companies, the wall between development and operations is long and hard to surmount. With proper training, DevOps as both a work process and a skillset can be successfully integrated into an existing IT department. This white paper will present a number of DevOps-related technologies and the case for training on these processes and tools. 2

3 CONFIG MANAGEMENT Why: DevOps administrators create and maintain the infrastructure hosting the products/services (and sometimes internal products/ services). Infrastructure consists of hardware (or cloud machines) and configuration. DevOps administrators automate the configuration process to ensure the configuration can be maintained. When one bad line in a configuration can affect product availability to customers or the internal organization, it s important to have tools to manage change. Configuration management has the added benefit of helping facilitate auto-scaling or other re-configuration of infrastructure to respond to the needs of the customers (e.g. a flood of new users) without intervention. Although this is not the only piece in the DevOps puzzle, configuration management is an essential piece and the most common starting point. Config Management Chef Puppet Ansible Capistrano Salt SCM PowerShell Desired State Configuration BUILD PROCESS/DEVELOPMENT AUTOMATION/ CONTINUOUS INTEGRATION Why: DevOps administrators provide the tools and processes to improve developer efficiency. The goal in this area is to keep developers writing code and creating solutions rather than dealing with problems related to bad builds and merges. DevOps administrators isolate developers from production environments to prevent bugs from reaching production, but in a way that does not impede the deployment of good code and features to production as quickly as possible. DevOps administrators also provide feedback (in the form of performance metrics and test results) back to the developers. Build/Automation/CI Continuous Integration Jenkins Continuous integration build environment/server Automates application builds Provides web-based insights into build process Integrates with source version control, communications hubs, and more Bamboo Similar to Jenkins, but a commercial product from Atlassian Integrates with JIRA 3

4 Rundeck Ant Centralized web-based tool to execute jobs on other servers Can also schedule Automatic build tool Grunt Automatic build tool Docker Facilitates containerization of applications and enables better development environments that can be kept closer to production Drone Continuous integration for Docker QA/TESTING/MONITORING Why: DevOps administrators must create and maintain internal tools and processes to ensure quality application and product delivery. The goal in this area is to automate testing and evaluation of the infrastructure, code and user experience as much as possible to ensure coverage is complete as possible. DevOps is also creating new methods of monitoring for anomalies and minimizing false positives in alerts. Monitoring everything from product availability to the efficiency of internal processes means DevOps administrators interact with almost every process in an organization related to product delivery and customer experience. Quality Assurance Tool Categories: Performance/Load/Stress testing (Chaos Monkey, The Grinder, JMeter, etc.) User experience testing automation (Selenium, Huxley) Web/API unit testing Logging/Monitoring (Nagios, statsd, Graphite, Logstash/Kibana) CLOUD/UTILITY COMPUTING Why: DevOps administrators are often responsible for specifying, purchasing, and implementing cloud and utility computing resources. One of the biggest changes in the IT world in the past five years has been the explosion in growth of cloud and utility computing models. Infrastructureas-a-service was pioneered by virtualization providers like Xen and VMware and embraced and extended by utility computing companies like Amazon Web Services, Rackspace and Google, and is a natural extension of the Software-as-a-Service movement (SaaS) led by companies like Salesforce, Oracle and Google. Many Infrastructure-as-a-Service providers are also moving into the Platform-as-a-Service space, where many of the standard requirements for creating and running applications (web servers and frameworks, databases, load balancers, etc.) are easily provisioned with an easy-to-use interface. In addition to outsourcing hardware and platform 4

5 services, there is an increasing focus on virtualizing the remaining on-premise equipment in so-called private clouds with tools like Openstack. Amazon Web Services (AWS) Eucalyptus Openstack CDN providers Heroku Pivotal Microsoft Azure Google App Engine MESSAGE BUSSES AND WORKERS Why: DevOps administrators often build systems that interconnect many smaller services to ensure scalability, loose coupling behavior and efficient team development. This construction is done while working closely with the developers of the individual services. Because of the distributed nature of these systems, concepts and patterns previously reserved for the high-performance computing, financial computing and mainframe sectors are being brought into all sorts of new areas. A major piece of any distributed system is the messaging bus used by the individual components to communicate. These systems are communication platforms that enable other applications and services (workers) to talk to one another in a distributed fashion. The application architecture is then defined by streams of messages going through the bus and acted upon by various worker nodes. RabbitMQ ZeroMQ Celery ESB OPERATING SYSTEMS (AND CONTAINERS) VS. HYPERVISING Why: The rise of virtual machines (and, by extension, Platform-as-a- Service) occurred as a response to perceived inadequacies in operating systems. It was too difficult for system administrators to configure monolithic operating systems to efficiently support a lot of applications running on one system. Although UNIX itself evolved from the timesharing paradigm of the 60 s and 70 s, most application libraries and service delivery platforms we installed at a system-wide level and 5

6 different applications could often experience conflicts that were not able to be resolved without adding new systems. Virtual machines solve this problem in a most inefficient way, literally hosting an entire dedicated operating system as a virtual environment on another operating system running on a hardware machine. However, operating systems have not stood still in attempting to solve the problems of resource allocation. An idea originated with chroot jails in BSD has evolved into Linux kernel support for LXC or Linux containers. This technology is starting to be adopted quickly as a natural improvement and solution for many of the problems experienced in regular operating systems but without the inefficiencies of virtualization. Hundreds of projects are cropping up to manage creation and management of containers, deploying applications to containers, and even whole lightweight operating systems designed from the ground up to support running only containers. Tools to watch: Docker cgroups lmctfy CoreOS ZeroVM OSv LXC DATABASES, NOSQL AND CACHING Why: DevOps administrators are responsible for providing reliable storage and database services for applications to use for data storage. They also provide development environments and methods for deploying changes to database schemas and data transformations. Today s web applications have a wide variety of storage needs. Although the classic relational (SQL) database is still widely used (and needed), newer approaches have cropped up to deal with new storage needs such as document storage and fast, clustered reading loads. There are also a lot of places in today s application stack where caching can be inserted to vertically scale product applications. Also, the classic relational database vendors have been quick to pick up on and add features that support the newer, simpler ways of accessing and organizing certain data types. MongoDB memcached Redis Cassandra Postgres 6

7 SOFTWARE DEFINED NETWORKING AND STORAGE (NETWORK/STORAGE/ETC) Why: This bleeding-edge topic takes virtualization out of the computing realm and into the areas of storage and networking. Combined with configuration management and distributed architectures, softwaredefined networking is about removing the need to manage physical network devices individually. OpenFlow Frenetic/Pyretic NOX/POX SDS MICROSERVICES AND SERVICE DISCOVERY Why: As the growth of the utility computing model means large numbers of server hosts are available, numerous software architecture designs fall out of it. One of these is the microservice architecture. In a microservice architecture, applications are broken up into tiny service applications, each performing a singular type of task. Applications are built up by using the microservices as building blocks for larger projects. From a delivery perspective, any of the smaller blocks can be scaled as needed, which makes it extremely flexible for large-scale and fast-growing sites. Many companies are using Node.js to create tiny web services that perform very specific tasks. One of the big problems DevOps administrators need to solve with microservices is how to communicate to each microservice application the address(es) and names of all the other services. Two solutions are Service Registration and Service Discovery. Node.js ZooKeeper Service registry Doozer Etcd Airbnb SmartStack Netflix Eureka Serf DNS SkyDNS/SkyDock 7

8 CLUSTER MANAGEMENT/CLUSTER OPERATING SYSTEMS Why: DevOps administrators have to deal with a large number of machines and services. Many of these services are distributed into clusters for redundancy and scaling. This gives rise to new problems of efficiently allocating or scheduling resources to the services. This has given rise to a new type of cluster operating system to automate these solutions. Mesos Essentially a kernel/operating system for distributed clusters that supports.war files and now Docker containers Marathon Scheduler that runs jobs and creates a private PaaS on top of Mesos Fleet For Docker containers YARN DEVOPS PROJECT MANAGEMENT Why: Sound project management techniques help ensure that scopes, builds and deployments happen in the most efficient way possible. Applying Agile techniques to infrastructure and operations (and even other areas of the business, such as HR and Finance) really helps to facilitate a DevOps culture. Scrum Agile ITIL DEVOPS-RELATED INDUSTRY CERTIFICATIONS Amazon Web Services Certified Solutions Architect Certified Developer Certified SysOps Administrator Rackspace/OpenStack Certified Technician for OpenStack-Icehouse MongoDB DBA Developer 8

9 Linux Red Hat Certified Architect Puppet Labs Puppet Professional Puppet Developer SaltStack Certified Engineer (SSCE) Certified Architect (SSCA) Microsoft Microsoft Azure Developer Specialist CONCLUSION This list of tools and techniques is just a starting point for an organization interested in developing a DevOps culture and getting team members up to speed on the modern tools used to break down the barriers between Development and Operations. There are clear benefits to implementing DevOps in an organization, and by extension, training staff on DevOps tools and techniques. Training resources, including in-person and online training, are expanding as DevOps gains widespread acceptance. There is significant value in having staff who have the skills to implement and maintain DevOps practices, and that value is likely to increase. Sources 2014 State of DevOps Report DevOps Salary Indeed.com 9