Wow! A configuration management program that worked the first time! Stuart Smith Transurban

Transcription

1 Wow! A configuration management program that worked the first time! Stuart Smith Transurban

2 Agenda 5 minutes 5 minutes 5 minutes 10 minutes 15 minutes 5 minutes 5 minutes Intro Why Bother What were the challenges how were these overcome Leveraging existing tools (Technical section) Benefits What does the CMDB look like today and what are the next steps Questions Configuration Management 2

3 Introduction Incumbent tool was due for upgrade Undertook a review of tools looking for best of breed, SAAS solution Tool must support the organisation s fast paced growth and align to corporate strategy Position for the future Get faster Deliver increasing value Incumbent system did not have an integrated CMDB One of the key deliverables was a fully functioning, largely auto populated CMDB Configuration Management 3

4 Why Bother? Applications and Infrastructure Projects asking how many servers, applications, databases or details about these Senior Management don t understand why you are so busy Complex and time consuming to understand life-cycle replacements Unable to explain everything you are managing and how it is interconnected Receive requests which do not have all the correct details Outages caused by expiration of digital certificates Infrastructure Not knowing who owns an application or who should approve a change Unable to get outage times and approvals Unsure of what groups of Servers to Patch and when Get flooded by alerts Service Management Have multiple regions you want to report by Inconsistent and poor details provided in service requests Security Find it complex to manage your Payment Card Industry (PCI) compliance Configuration Management 4

5 Create a High-Level Model Data Model: Determine level of depth you want to manage and what can reasonably be discovered automatically. 5

6 Leverage Existing Tools What tools are available and hence, what integrations are required What fields are required to meet the objectives How to automate the creation of most (if not all) CIs and relationships Technical considerations (eg. Integration method) Configuration Management 6

7 Tools and Integrations Slide subheading SCOM SCCM Solarwinds OEM Thawte Internal CA DISCOVERED CIs DISCOVERED ASSETS IN17 DISCOVERED CIs MID DISCOVERED CIs Integrations: Determine the integrations required to capture the source data Servers SCOM Network Solarwinds Database Oracle Enterprise Manager (OEM) Desk/Laptops SCCM IN2,4,16 IN2,4,16 Windows server SCCM CMDB Discovery DISCOVERED ASSETS Discovery DISCOVERED ASSETS ServiceNow Applications Manually created Mobile devices AirWatch (post go-live) VMs Vcentre (post go-live) Configuration Management 7

8 Integration Method Slide subheading Method Determine Target Tables, create integration specification and the integration method Webservice JDBC etc. Reconciliation Identifiers - are used to determine the order we try and match on keys try to match on SCOM ID, then FQDN, then Name? Copy Empty Fields - If set to YES and discovery source has a NULL field, the CMDB is over written with blanks. Configuration Management 8

9 Mapping Fields Choice Action: create enables the creation of new reference values (eg. create a new Make/Model in the CMDB, if one does not exist for this make/model) Mapped fields from the source system to the CMDB Determined if some values could be used to auto update reference data (eg. Make/Model) Determined what fields they required Met with each CI Class owner (eg. Server, Network Device etc.) Included any additional fields required to meet the overall objectives Configuration Management 9

10 Auto Create Relationships Based on data sources, determine what fields can be used to build relationships Using discovery precedence, load relationships between CIs (typically using the same source) Log exception report where correlation failed Tune engine to improve matches, or manually deal with exceptions Data from Solarwinds provides: The key/unique id from a network device u_network The key/unique id from a server device u_server We then create a parent to child relationship with the terms Connected By and Connects Don t create a new relationship, if one exists Don t create a new CI Source Target Coalesce Choice action source.u_network child TRUE REJECT Source.u_server parent TRUE REJECT "CONNECTED BY::CONNECTS" type TRUE REJECT Configuration Management 10

11 Technical considerations Slide subheading Alerting for failed discovery and transformation tasks Connection method (eg. Web Service, etc. ) Never perform creates on the same Class from multiple sources Discover from primary source and then update additional data from secondary source(s) Configuration Management 11

12 Challenges Challenge Overcome by Getting time and commitment of Subject Matter Experts from each CI owner group Strong sponsorship from senior management (CIO / GGM) Regular steering committee meeting where issues were flagged Ensure we built a CMDB that provided benefits to the team (explained in more detail upcoming slides) Loading data/mapping fields and mapping relationships Need a resource with in-depth understanding of configuration management Auto-populating data versus manual updates. Explaining cost benefit to SMEs Everyone wants everything auto-populated Escalating to the steering committee if the SMEs could not be convinced Ongoing population of the CMDB Ensured managers bought into the value All staff were put through training sessions Configuration Manager to drive/uplift compliance Reports compliance provided to GM and above Configuration Management 12

13 Tips Tips Explanation Configuration Management is a journey Ensure stakeholders expectations are met Expect 1 to 2 years to reach end state Don t under-estimate the effort Ensure adequate resourcing, such as: Ensure stakeholders understand the difference between Configuration and Asset Management A Configuration Manager Technical expert to create the CMDB Technical staff to provide the details and help with mapping Avoid confusion regarding the end state deliverable Understand the benefits To sell why you are implementing Configuration Management Configuration Management 13

14 End of Technical section Questions?

15 Benefits / Savings Want to ensure there are real savings / benefits?

16 Benefits Activity Saving Comment Change Manager effort 25% Easier to see impacts, add approvers Change creation 62.5% quicker Reducing alert SPAM ing 150 hours/month SCOM put in to maintenance mode Self Service Portal (SSP) Requests Days quicker Improvement based on empirical data. Previously there was no SSP. Average SSP provisioning is 4-hours Server patching 13 hours / month Based on 2 patching cycles (Dev then Prod) Easier Life-cycle management Several Months Previous project ran for over 12 months Providing Projects with accurate CI data Days/request Often requests took days, can now provide info in minutes Network firewall burns 30% Absorbed 30% growth with no FTE increase PCI Compliance N/A Improved compliance/governance Reduced SD call Volume 30% SSP task are sent directly to the resolver group Configuration Management 16

17 Location Data and Data Classifications Slide subheading User Locations: Locations that the services / application is provisioned. Staff understand which regions they need to take into consideration, different financial reporting, different public holidays etc. Info Classification: Is data highly confidential (eg. HR data) PCI Compliance: Is the data within scope of PCI compliance, what additional checks/approvals need to be under taken to ensure PCI rules are met Privacy Compliance: What privacy rules must be met or considered TISA compliance: TU has third party agreements with the roadside, this requires certain rules, such as outage lead times and communications Configuration Management 17

18 Business Criticality Slide subheading Business Criticality: Used to define the SLAs for the service/application. This is used for reporting and can be used for managing the overall end-to-end service (where the underlying Infrastructure is not built to the same level criticality). Transurban use Gold / Silver / Bronze Configuration Management 18

19 Life-Cycle Management Slide subheading Support End date: Dates can be used to pull reports for life-cycle replacement Potential Saving: Last time LCM was a project that ran for over 12 months (just to collect and validate data) Configuration Management 19

20 Assignment and Approvers Slide subheading Assignment: Incidents and requests are directly assigned from the self-service portal to teams based on populated assignment groups, thus skipping the IT Service Desk Approval: Approvers of a Change are driven from approval groups on the configuration item and a secondary approval group can be added, for instance to ensure that a Change is approved for both the Aus. and USA regions Effort Saved: Minor Change review is 50% quicker, saving 25% of total Change Manager effort Configuration Management 20

21 Dependency Mapping Slide subheading Dependency Maps: Can be used to understand the impact a server, database or Network component can have on applications and services Improvement: Helped to half the amount of time spent reviewing changes Configuration Management 21

22 Patching Cycles - Security Slide subheading Patching Group: Can be driven by adding CIs to a patching group and patching can be automated off this Patching Exemptions: Servers can be omitted from patching if there are risks associated with performing the patch (eg. Applications running on the server are not supported on the patch level) Patching Instructions: Special instructions can be added eg. Check alerting is re-enabled Effort Saved: 7 hours/patching cycle (Windows Fleet Production and 154 Dev/Test servers) Configuration Management 22

23 SCOM Alerts Put into Maintenance Mode Slide subheading SCOM Alerts: By integrating SNOW and SCOM and loading the CIs, when a Change is raised and outage records created, the SCOM servers are placed into maintenance mode, thus preventing spamming of alerts during a valid outage Effort saved: 150 hours/month Configuration Management 23

24 Upstream Applications Slide subheading Upstream Applications: Users can add upstream applications etc. to an Infrastructure change which will then show any impacted upstream components, reducing the likelihood of an application/service outage Configuration Management 24

25 PCI Compliance PCI Compliance: When Changes are approved by a peer and moved from Peer Review to Tech & Change Review the Dependency map is checked to see if the application is within PCI scope and an additional approver can be added. Weekly reports are also produced detailing if PCI configuration items have been changed. Configuration Management 25

26 Certificate Expiration Digital Certificates: Are managed across the enterprise to ensure certificates are upgraded, before they expire. Since go-live we have seen no incidents resulting from expired certificates. Value: Reduces the risk of an outage caused by expired certificates Configuration Management 26

27 Portal Home Page Configuration Management 27

28 Sample Service Requests Configuration Management 28

29 Self Service Request Driven from CMDB Improved self-service: As a number of requests can be driven off the CMDB. For example, a system to system firewall rule can have the source/destination firewalls populated, ensuring appropriate approval and that the right information is provided in the request. Effort saved: 30% increase in volume, with no increase in people Configuration Management 29

30 Improved Self Service Improved self-service: If the relevant Configuration Item is selected, then the request is immediately assigned to the correct team. Savings: Together with knowledge articles, and selfservice forms, this has reduced help desk call volume by ~50%. Configuration Management 30

31 Auto Create new CIs Improve CI Creation: New Application CIs are QA d and created through a new form. Savings: 5% Configuration Manager saving Configuration Management 31

32 End of Benefits section Questions?

33 Current CMDB and looking into the future Current Dashboard vs Improved Dashboard Loading vcentre data automatically Expected a saving: 4-weeks/annum Creating services and mapping to applications Loading mobile billing data Configuration Management 33

34 Current Dashboard Improved reporting: Ability to monitor exceptions in the CMDB, such as stale, duplicate and orphaned CIs Configuration Management 34

35 Improved Dashboard CMDB Completeness / Compliance / Correctness Configuration Management 35

36 Contact Details LinkedIn: Configuration Management 36

37 End Presentation Questions?