Change Management Standard Goal: The primary goal of the Change Management process is to reduce disruption and increase transparency of changes that occur in our IT environment. Managing change is one of the most difficult challenges facing IT organizations. To effectively support and facilitate enterprise business goals, IT must continually change. While planned, authorized changes have obvious benefits to systems and users, unexpected and even small changes can result in serious negative impact to IT systems and processes. Changes in any given layer can potentially affect every part of a business operation, posing various degrees of risk to the enterprise. Change management processes allow us to exert some level of control over this complexity and reduce risk. Change Manager: Greg Dressman Dressman@ksu.edu - 785-532-2583 Understanding Change Types Standard - Routine, low risk change that is operational in nature and happens frequently. A standard change follows an accepted and established procedure to provide a specific change requirement. Every Standard Change should follow an established procedure. Procedures should follow industry standards with deviations documented. The crucial elements of a Standard Change are as follows: The tasks are well known, documented and proven. Approval is not required by the change advisory board (CAB). The risk is usually low and always well understood. Examples: Create F5 pool, add system resources, grant user privileges, application deployment that happens on a regular basis and is known to be non-disruptive, adding permit firewall rules. Non-Standard - Any other change that does not fit the Standard Change or Emergency Change model The tasks are not well known and frequently conducted. The risk is usually high and not well understood (usually the defining difference between standard and non-standard changes). Approval is required by the CAB (Approval and Review guidelines below). The CAB encourages a 1 week window between approval and change execution. Examples: deny firewall rule (blocking printers from the border), major version upgrade, application deployment that is known to be disruptive, disruptive restarts (planned). Emergency - A change required to prevent an impending Critical Incident or resolve an active Critical Incident. Examples: replacing failed equipment, disruptive restarts (unplanned). Scope Includes: o Changes to any production environment that could impact multiple individuals. o Changes to Test/development environments that affect a large amount of people. Excludes: o Deployments (workflow already includes changes) 1
o o o o Non-disruptive restarts Manual vmotions End user changes Changes isolated to an individual (e.g., group membership changes, password changes for user accounts). Change Management Process Standard Change The typical time line for a standard change may utilize the following steps: 1. Technical staff receives a request for a well-known process, typically a request, incident, problem or request task in ServiceNow. 2. Technical staff identifies when the change will be made and this does not need to be in a maintenance window 3. Technical staff executes the change 4. Quick change request is created and closed in ServiceNow Non-Standard Change The typical time line for a non-standard change may utilize the following steps: 1. Technical staff in the supporting unit conceives and plans a change. If other units must make corresponding changes, they should be involved in the early planning. 2. Technical staff communicates with known affected groups directly affected. 3. Create a Change Request in ServiceNow at least by the end of business day prior to the Change Advisory Board (CAB) meeting (Wednesday). 4. CAB reviews the change and provides feedback and approval. Do not proceed until approval is granted. Affected groups, working with technical staff may request a change to be postponed even after CAB approval due to schedule conflicts or unforeseen risks. 5. Technical staff sends communication of the change to itproblems@ksu.edu and others as appropriate (sysadmins@ksu.edu, etc.). 6. NOC places the public announcement on the ITS status page, www.k-state.edu/its/status/. 7. Technical staff sends communication prior to starting the change to the #NOC channel in Rocket Chat. 8. Technical staff executes the change. 9. If any significant time lapse occurred, technical staff sends a communication to itproblems@ksu.edu and #NOC channel in Rocket Chat indicating issues or completion. 10. NOC marks public IT Status announcement as completed. 11. Technical staff mark the change request as completed in Service Now. Emergency Change The typical time line for an emergency change may utilize the following steps: 1. Technical staff in the supporting unit is notified of a critical incident or identifies an issue that could lead to a critical issue. 2. Impact, risk, resource availability and urgency are assessed. Technical staff plans the change. 3. Communications and making the change can be in parallel 2
4. Technical staff communicates with known affected groups directly affected. 5. Technical staff communicates the plan to the Change Manager who then communicates to the CIO. 6. If risk and/or impact is high, seek senior staff, team lead, change manager or CIO approval for the change to proceed as appropriate to the change. If the change is necessary and staff cannot be reached, Technical staff makes a judgement call on whether to proceed. 7. Technical staff executes the change. 8. Change manager follows up with a retrospective on the change. Typically, emergency changes are discussed in IT Status weekly meeting. If it merits detailed conversation, it is discussed in CAB meeting. Failed Changes If issues are encountered during a change follow the documented Blackout plan and communicate with the end users, #NOC and itproblems@ksu.edu. 1. Once the state of the change is set to Closed, Close code and Close notes will display on the form. 2. Set the change state to Closed and set the Close code to Unsuccessful and detail issues in the Close notes. 3. Save the change. The Change Management process may utilize the following stages to manage each change. Stage New Assess Authorize Scheduled Implement Review Closed Cancelled Description Change request has not been submitted yet for review and authorization. A change requester can save a change request as many times as necessary while building out the details of the change prior to submission. Peer review and technical approval of the change details are performed during this state. Change Management and IT Status will schedule the change and provide final authorization to proceed. The change is fully scheduled and authorized. It is now waiting for the planned start date. The planned start date has approached and the actual work to implement the change is being conducted. The work has been completed. The change requester now determines whether the change was successful. A post-implementation review can be conducted during this state. All review work is complete. The change is closed with no further action required. A change can be canceled at any point if it is no longer required. However, a change cannot be canceled from a Closed state. Creating Changes Option 1 Creating a Change Request from another Request/Incident/Problem/Request Task/Enhancement 3
Sometimes while working on an Incident, Requested Item, Enhancement or Problem you will need to create a Change Request and have it associated with the existing record. There are two ways this can be done from the existing record: Create Change or Create Quick Change Fig. 1 - Location of Create Change and Create Quick Change actions. Fig. 2 - Create Quick Change popup form. Comparison of Create Change and Create Quick Change Create Change Available in the context menu by clicking 3 bar icon in the form header. Immediately creates Change record and redirects user to it. Create Quick Change Directly available on the form as a button. Opens a popup window to create a new Change record with these actions: X button: Popup will close and record will not be created. Save Change: Change will be saved and popup will close. Save and Mark Closed: Change will be saved in the closed state and popup will close. 4
Shows full Change Request form with all fields. Auto-populates short description, description, configuration item, assignment group from current record. Default planned start and end dates are empty. Default type is Standard. Popup uses abbreviated form view with only the most relevant fields. Auto-populates short description, description, configuration item, assignment group, assigned to from current record. Defaults planned start date to now and planned end date to an hour later. Default type is Standard. In either case, the existing record becomes the parent record of the Change Request and for Incidents and Enhancements, the RFC field is populated with the new Change record. When the Change record is closed, it will automatically close the parent Enhancement (if applicable) as well as any other Enhancements that have their RFC value set to it. Fig. 3 - Change Request parent record breadcrumb. Using the RFC field allows you to attach multiple Enhancements, Incidents or Problems to a single Change Request. Records attached to the Change Request through the RFC field may be viewed in the related lists at the bottom of the Change Request form. Additionally, the attached records may be managed using the Edit button and selecting or removing records from the list. Fig. 4 - Attached records in Change Request related lists. Option 2 New Change Form Defaults to a standard change. Fully featured change form. Typically used for changes that did not originate from an end user and a do not have an existing ServiceNow record e.g. request, incident, problem, enhancement. State change is required to close the change. 1. Type Change in the type filter text 2. Select Create New 3. Select CI, will automatically populate Assignment Group. 4. Fill out the Short Description and Description 5
Fig. 5 Location of Create New action. Communication (optional) The Change management board had a goal of improving change communication so a template driven process was built into ServiceNow to assist with generating change communication. Fig. 6 Communication Plan Form 6
The Communication Plan tab contains fields used in preparing a change notification to inform key users and groups of the organization about the upcoming change. This notification can be created within the change form by checking the Change Notification field as shown below: Fig. 7 Location of Change notification action to bring up Communication Plan tab 7
Recipients may be added to the notification from the ITS groups to be notified or Users to be notified fields. Marking Notify IT Status or Notify Sysadmins will automatically populate the corresponding LISTSERV email address. The change manager can specify if any other groups or users should be notified: Fig. 8 Communication Plan Form 8
Clicking Begin a new notification message will open the popup email client window and populate a template message based on form field values. It will follow this formula: To: <ITS groups to be notified>, <Users to be notified> Subject: <Short description> Message: <service_name> will be undergoing a change on <planned_start_date>. It will affect <affected_users>. An outage of <duration> is anticipated. If outage required is NOT checked, it will say, "No outages are expected." If all the fields are populated, it will look like: Office 365 will be undergoing a change on 05/09/2017 06:00 PM CDT. It will affect all students, faculty and staff. An outage of <duration> is anticipated. Complete the following fields on the Communication Plan following these criteria: Field Name Description Notes ITS Groups to be Groups to be notified of the prechange List collector to select ITS Groups Notified notification Users to be Notified Users to be notified of the prechange notification List collector to select users or email addresses Notify IT Problems Option to include IT Status in notifications True/False will add/remove IT Problems listerv to/from Users to be Notified Notify Sysadmins Option to include Sysadmins in notifications True/False will add/remove Sysadmins listserv to/from Users to be Notified Affected Users Which of the users will be will be affected by the change? Text describing who will be affected 9
Approvals and Review IT Collaboration will serve as the change advisory board (CAB). The IT Collaboration meeting is a finite meeting that can only consider the review of a limited number of changes during each change cycle. The following guidelines identify criteria for changes that will receive CAB review: The requested change is High or Moderate priority (Note: Critical Priority Changes are handled as Emergency Changes). Priority Definitions Emergency changes that occurred should be reviewed retroactively. The requested non-standard change is proposed for deployment during a freeze period for that service or CI. The requested non-standard change is proposed for deployment outside the specified maintenance window for that service or CI. The Change Manager determines that for any reason there is a material benefit in the requested change being considered by the CAB attendees. This is accomplished in ServiceNow by the Change Manager raising the priority to high or moderate and indicating the reason for this change in Notes. Common reasons include, but are not limited to, the following: o Perception that the benefits do no warrant the risk of the change being performed o Inability to reach consensus with the change requester regarding deployment scheduling During the CAB meetings, a typical agenda will look like this: o o Review the following changes from the previous week Emergency changes Failed changes Backed-out changes Incidents resulting from implemented changes Review/assessment of proposed Requests for Non-Standard Changes (RFCs) Risk and impact in terms of: Service and Service Level impact Capacity and performance Security and compliance Financial Resources involved Maintenance & Blackout Windows ServiceNow provides for the management of maintenance windows that can be used to determine when changes should occur as well as help to determine the calculation of risk. Multiple maintenance windows can be setup and associated to various CIs. Maintenance Windows Scope Day Start Stop 10
Saturday Evening Saturdays 6PM CST 12PM CST Sunday Monday Mondays 6PM CST 6AM CST Tuesday Wednesday Wednesdays 6PM CST 6AM CST Thursday ServiceNow allows for the management of Blackout Windows that can be used to determine when changes should not occur. Blackout windows can be setup and associated to various CIs. Blackout windows are used in the Change Conflict link which will notify the requestor if the requested change window falls within a Blackout Window. Fig. 9 Conflict status Fig. 10 Conflict list The change manager is also responsible for determining and setting change freezes. A freeze is a point in time in the development process after which the rules for making changes to the source code or related resources become more strict, or the period during which those rules are applied. A typical change freeze period is finals week and grade posting. Change Freeze Windows Scope Notes 11
Fridays Finals Week Midterms Week Semester start Avoid Making Changes on Friday To be set for each term; all day To be set for each term; all day To be set for each term; all day. First week of the term 12