USE CASES FOR CONNECTED SYSTEMS

Transcription

1 3 USE CASES FOR CONNECTED SYSTEMS How xmatters uses toolchains to move process forward

2 How does your development team deliver software to your customers? You can t just hand off to operations and walk away anymore. That s why development and operations teams automate software testing from development to staging to production. But you can t just assume nothing will ever go wrong. So what happens when you have to find the person who can compare software versions and fix code? As businesses have moved from quarterly deployments to multiple daily deployments, connecting insights from your systems to the right people is more important than ever. And it only works well enough if your systems and people are communicating. At xmatters we automate communication between connected systems and tools so people can move processes forward without toggling between systems. When connected via xmatters, our toolchains do more than just send alerts to people. They enable the right people to take immediate action without extra steps or manual work. In this paper we ll pull back the covers and explore how we use the xmatters communication platform to connect toolchains and resolve product defects in development, automate the release process, and manage major incidents. By showcasing best practices in the actions we take, you can determine some best practices with communications based on intelligence and automation. Connected systems and automation As software products move from development to testing and deployment, connections between these systems and tools for communication and storage can become a complex web. So if you don t automate the connections and align them with your business processes, they can produce some pretty messy mistakes that can do real harm to your customers. And if your customers lose, you lose too. You can make connected systems and toolchains more effective by automating the communication. Manually moving between systems to perform tasks slows down business processes and adds risk of human error. Standard, familiar situations quickly become unpredictable if manual processes drag them out. That s why the xmatters integration platform connects critical data and insights to the people who need them to take action. As business processes become more complex, enterprise organizations must integrate the systems and tools with the people who make crucial decisions to enable more efficient workflows. 2

3 GUIDE USE CASE Resolving Product Defects ROLE SOFTWARE ENGINEER When we optimize our bug fixes, we focus on the greatest source of delay: the time a ticket sits in the queue before an engineer touches it. Once a monitoring system or a person reports an issue, xmatters engages engineers and involves them in the process as quickly as possible. Sometimes the process starts when a customer discovers a bug and logs an issue in Zendesk, our customer support platform. Other times, someone in Support finds a bug and posts an issue in Zendesk. A Support representative then investigates the issue. If it is a legitimate problem, she includes Engineering in the issue. Zendesk, xmatters Transition to engineering If the bug is determined to be nothing, Support closes the ticket and sends a notification to the customer. Otherwise, the Support agent uses xmatters to find and notify an engineer who is on call and available. The notification includes options for taking action. An engineer can open a Slack channel, a Webex session, or another communication channel and use xmatters to pull in others for help. xmatters chains JIRA to communication applications so engineers collaborate without toggling between systems. Meanwhile, xmatters integrations enable JIRA to capture all communications so the chat conversation is available within JIRA. So a unique integration between JIRA and Zendesk allows Support and others to remain involved. Zendesk, Hydra, JIRA, Slack, Webex, xmatters An engineer can investigate the issue for seriousness and scope. Before creating a ticket, engineers run the same secure query against a number of servers to verify the issue through a tool called Hydra. The engineer can create a ticket in JIRA. xmatters pushes the information into the ticket so the engineer doesn t have to re-enter it. Engineers can follow and manage the ticket in JIRA. 3

4 Maintaining communication The Engineering team uses Git as source code repository, including Stash software. The team also uses Gradle for build automation. The most important element, though, is communication. The engineering team communicates through Slack and Webex. xmatters finds the on-call person with the right skills, and captures communication within the incident. Git, Gradle, JIRA, Slack, Webex, xmatters USE CASE Monitoring and Automating Releases ROLE OPERATIONS DIRECTOR Homegrown tools In a DevOps world, the engineer is expected to be the release team and to monitor and support software once it is in production. Engineering keeps a Swiss Army knife of software in a homegrown xmatters tool we call Planet Express to handle much of the product work. It provides a common interface for several external applications, including Puppet and Fabric. When client communication is warranted, people use xmatters, which sends communications to as many people as necessary. xmatters, Planet Express, JIRA The xmatters platform delivers targeted communications to specific people on their preferred devices. To ensure delivery, xmatters integrates with tools that perform telemetry to collect data at remote or inaccessible points and transmit them to receiving equipment where they can be monitored. xmatters also maintains constant monitoring in the production environment. xmatters uses Pingdom to monitor physical devices and other tools to intercept SMS messages in more than 40 countries to determine the reach of long SMS messages that use special characters and extreme complexity. 4

5 GUIDE Automated responses to long SMS messages indicate whether special characters and other unusual elements are preserved or get stripped out. Missing characters can cause unclear communications, potentially lengthening resolution times. So we run a diagnostic test on responses to determine whether special characters or other complexities like symbols, ellipses, abbreviations, or mixed capitalizations are being stripped out. COMMON TOOL Pingdom USE CASE Major Incident Management ROLE MAJOR INCIDENT MANAGER Continuous automated testing xmatters runs continuous automated testing with several tools to ensure the stability of its production environments before preparing for deployment. Pingdom tests uptime and performs synthetic user transactions to measure performance and page speed. JMeter performs server performance testing for static and dynamic resources. Splunk stores data for xmatters to share when a Splunk alert triggers. xmatters integrates with Pingdom and other monitoring tools, and has access to on-call schedules. So when Pingdom observes an issue, xmatters finds an available engineer or the last person to touch the code so he can triage the issue. xmatters also communicates to the development and automation toolchain so the right people can halt development and deployment. Pingdom, JMeter, Runscope, Splunk, xmatters We can catch a potential major incident one of two ways: a customer can log it in Zendesk, or one of our monitoring tools can find it. If the notification comes from customers, our client executives use external testing tools to see the product from a customer s perspective and reproduce the issue. Monitoring tools can also catch incidents because xmatters runs continuous automated testing with several tools: 5

6 Pingdom tests uptime and performs synthetic user transactions to measure performance and page speed. Runscope tests APIs and executes hundreds of synthetic notifications on synthetic instances every few minutes to an SMS simulator, so engineers always know how many failures there are (if any). Other tools perform global SMS testing through an SMS aggregation service across dozens of services to see delivery degradation. Splunk allows people to examine and catch errors in logs, and to detect patterns before they impact customers. Engineering can open a JIRA ticket from within a Zendesk JIRA form An integration with Slack automatically posts to a major incident Slack channel and opens a new ticket based on the reference number xmatters takes the guesswork out of major incident management events. All communication runs through xmatters. In the first seconds after a major incident is declared, employees use xmatters to assemble a resolution team. Zendesk, JIRA, Slack, xmatters The strength of xmatters is its ability to connect systems and tools to relevant people. When a client executive needs to investigate an initial report of an incident, she uses xmatters to find the right person or group so she can assess whether she has a major incident in minutes, not hours. Pingdom, Runscope, Splunk, xmatters Managing incident communication When does a situation become a major incident? When someone declares it a major incident, of course. Support acts as gatekeeper to determine whether the event is a major incident based on specific criteria. If it is, the support person opens a Zendesk ticket. In a more urgent situation, he might just open a Slack channel or call engineering. Zendesk acts as an umbrella ticket. xmatters enables Zendesk to integrate with other systems to make more information available: Proactive communications Engineers use xmatters to initiate proactive communications to other stakeholders. Engineers scope which customers might be affected, and automate regular updates to specific individuals who have subscribed to receive notifications. They also use xmatters to send regular notifications to executives, sales and advisors. JIRA automatically includes fields from the Zendesk ticket, and engineers scope which customers are affected so xmatters notifies the right customers. Using the open Webex session, everyone involved on the ticket receives an assignment and a designated time frame for checking back in on the Webex. All the information sits in the Confluence Incident Board, which can be updated via xmatters. From the Confluence web page, engineers can easily click to view discussions in other systems. Sending comments through an xmatters response option can update the company status page on a regular schedule as well. 6

7 GUIDE When the system has been tested and confirmed restored, xmatters informs affected customers, and everyone on the incident resolution team participates in an immediate post mortem and root cause analysis. xmatters, Zendesk, JIRA, Slack, Webex, Confluence Conclusion As our business processes become more efficient, they can become more complex because they involve more systems and people. Still, we can apply communication processes to resolve issues or complete projects quickly. Automation through integrations enable people to communicate and share information more efficiently. Sharing information across connected systems and tools is crucial for resolving issues and increasing the velicity of our business processes. Communication between systems and people who need to take action is arguably the most important factor in any business process. This is where the xmatters platform plays its most important role, delivering information with context and timeliness, and providing responses for taking action. Whether you re optimizing your development and deployment processes, automating software testing, releasing new products, or resolving critical incidents, a communication platform like xmatters vastly increases velocity or reduces resolution times. Success and failure is often determined in the details: choosing tools and systems that work with each other, determining the right culture, and automating connections between tools. Communicating insights to the right people in proper context leads to successful outcomes. 7

8 Contacts About Us CORPORATE HEADQUARTERS Alcosta Blvd., Suite 425 San Ramon, CA USA XMATTRS ( ) EMEA HEADQUARTERS 20 Little Britain London, EC1A 7DH UK +44 (0) APJ HEADQUARTERS Level 29 Chifley Tower, 2 Chifley Square, Sydney, NSW 2000 AU Copyright 2016 xmatters. All rights reserved. All other products and brand names are trademarks or registered trademarks of their respective holders. xmatters is an intelligent communications platform that connects insights from any system to the people that matter in order to accelerate essential business processes. The platform automates, operationalizes and contextualizes communications within key DevOps processes, fundamentally altering the way business units work together. xmatters also supports enterprises with major incident and change management, alerting the right people on the right channels to time-sensitive events and problems like network outages, supply-chain disruptions, natural disasters and medical emergencies. Founded in 2000 and named the #1 Best Mid-Sized Workplace in Technology, xmatters is headquartered in San Ramon, CA, with global offices. For more information, please visit