What does it take to Operate and Maintain a GIS system? Vishal Pahuja IT Manager GIS Operations and Maintenance (Electric and Gas)

Transcription

1 What does it take to Operate and Maintain a GIS system? Vishal Pahuja IT Manager GIS Operations and Maintenance (Electric and Gas)

2 Purpose/Goal The day you stop learning is the day you stop growing Someone intelligent and influential I think The intention is to share how things are being done; not to claim it s the best and only way to do things 2

3 History/Background of How GIS came Various GIS projects that started around 2012, for four major towers (ED, ET, GD and GT) ended in 2015 GISCOE took birth Jan 2016 to help operate & maintain GIS application; and to provide single point of contact for all GIS related projects RFP floated for GIS Operations managed services TCS started with GISCOE Ops managed service July 2016 Scope included Mission Critical Apps for all the four towers Integrity management applications (primarily for Gas) Mobile Applications Minor Enhancements, Break Fixes and emergency releases 3

4 History/Background of IT Windows/Unix I&O Network Citrix/Virtualization IT SAP BT Customer Care GISCOE 4

5 Current Org and functional structure Portal implementation Develop Enterprise GIS standards Design Enterprise GIS Architecture GIS Roadmap Executing Proof of Concepts Developing GIS Standards and best practices Share Deliver Foundational GIS systems operations and maintenance Minor Enhancements, breakfixes & emergency releases Continuous improvements Run Enterprise GIS Projects Providing GIS consulting to various projects IT GIS Book of Work 5

6 History/Background of GIS Technology Editors over Citrix, Web Viewer, nightly maintenance and integration jobs. ~500 servers in two data centers Prod, QA, Test, Dev, Data conversion, Performance testing environments for all the towers Combination of physical and virtual ~5% of compute in data center used by GIS GIS integration with SAP, DMS, MDMS, Customer Care and so on 6

7 Subscribers GIS Landscape Hydro Fiber EDGIS EDGIS ETGIS GDGIS GTGIS Landbase Crew Location Weather Videos Foundational Platform Incremental Data via GDB Replication Geomart/Portal/Sharing platform Foundational GIS Applications Hydro Leak Map Cathodic Protection Incremental Data Extracts Services Crossbore Emergency Operations Mobile Asset Inspectiona Locate & Mark Field GIS GIS for External users Linear Inspection FLISR WildFires Veg. Mgmt 7

8 History/Background of Technologies in use Tier Operating Systems Databases Web Server Application Server Client Virtualization Desktop Automation/Scheduled Jobs Technology Microsoft Windows, IBM AIX and Redhat Linux Oracle, SQL Server and PostGresSQL IIS, ArcGIS WebAdaptor ArcGIS Server, Portal Silverlight, JavaScript, Runtime, Engine Citrix, VMWare ArcGIS Desktop, ArcFM Automic UC4 Integration flat files, ftp, WebLogic, Layer 7 8

9 EDGIS and LBGIS Infrastructure Birds eye view Electric Distribution Geographic Information System (ED GIS- Production: 115 Servers) Landbase Geographic Information System (LB GIS Production: 38 Servers) Maintenance- EDER Publication- WEBR Data Mart Maintenance- EDER Publication 100 UDC External Mappers Citrix XenApp 6.5 ESRI ArcMap Windows Svr 2008 R2 Physical (PowerEdge) Remote Access (22 Servers) ESRI License Manager 10.4 Windows Svr 2008 R2 Virtual (Legacy IaaS) Oracle RAC DB 11.2 IBM AIX 6.1 IBM POWER 8 ESRI ArcGIS Windows Svr 2008 R2 Virtual (Legacy IaaS) 200 PG&E Mappers License (4 Servers) Database (6 Servers) Print (4 Servers) 1000 Estimators 4000 Others SilverLight 4 IIS 7.5 Windows Svr 2008 R2 Virtual (Legacy IaaS) Web (8 Servers) ESRI ArcGIS SE ArcFM Windows Svr 2008 R2 Virtual (Legacy IaaS) Oracle DB 11.2 IBM AIX 6.1 IBM POWER Engineers 50 Outage Coordinators Application (20 Servers) Database (4 Servers) Safe FME 2014 Windows Svr 2008 R2 Virtual (Legacy IaaS) Physical (PowerEdge) Batch (36 Servers) CC&B System Interfaces Oracle DB 11 IBM AIX 7.1 IBM POWER 8 CYM E M DMS DM S O ther Systems Management ESRI ArcMAP Windows Svr 2008 R2 Virtual (Legacy IaaS) ESRI System Monitor 1.x Windows Svr 2008 R2 Virtual (Legacy IaaS) Database (4 Servers) SA P Release (4 Servers) Monitor (3 Servers) 10 PG&E Mappers Citrix XenApp 6.5 ESRI ArcMap Windows Svr 2008 R2 Physical (PowerEdge) Remote Access (4 Servers) Oracle RAC DB 11.2 IBM AIX 6.1 IBM POWER 8 Database (2 Servers) System Interfaces IIS 7.5 Windows Svr 2008 R2 Virtual (Legacy IaaS) ESRI ArcGIS Windows Svr 2008 R2 Virtual (Legacy IaaS) Oracle DB 11.2 IBM AIX 6.1 IBM POWER 8 Web (6 Servers) Application (24 Servers) Database (2 Servers) 9

10 Support Structure GISCOE Manager MS Delivery Manager Tech Lead (Gas) Tech Lead (Electric) Tech Lead (Gas) Tech Lead (Electric) Infra. Lead Onshore Offshore Onshore Offshore Close collaboration with Managed Service provider Daily check-ins and huddles Periodic governance meetings 11

11 Support Model: Incidents Methodology Intake Triage Classification and assignment Evaluation Bug? Enhancement? Emergency? Prioritize and schedule Repeat SLA Driven 12

12 Support Model: Incidents Workflow (Sample) Intake Triage 1. New Device 2. Device Replacement 1. GPS Issue 2. Network Issue 3. Application Push 4. Font Issue 1. How To s 1. Application 2. Functional Front Door : Mobile Ops Field Support Front Door/ Mobile Ops Delivery Mobile Operation Field Support Provisioning(Device/OS/ User Profile) Provisione d Resolved Mobile Operations Delivery GPS, Network, Apps Push, Font Issue, Pre-Requisite of mobile device & testing Yes TSC Ticket Closed No Front Door : BSS Team Front Door : GD GIS O&M Front Door : Mobile Ops Field Support Front Door/ Mobile Ops Delivery GD GIS O&M Team Triage 13

13 Support Model: Work Orders (cont ) Methodology (repetitive tasks*) Intake (primarily through work order) Prioritize Execute Evaluate if need to be automated Repeat Not SLA driven State Zero Rebuild Geometric Network? * Eg. Access requests, one time data requests, periodic audits, 14

14 SLA/OLA (Sample) Service Levels Section Parameter Response/ Resolution Time Expected Service Level Minimum Service Level Measurement Period Priority Level 1 Service Levels 30 Mins/ 4 Hours 95.00% 90.00% Monthly Priority Level 2 Service Levels 60 Mins/9 Hours 95.00% 85.00% Monthly Priority Level 3 Service Levels 2 Hours/ 24 Hours 95.00% 85.00% Monthly Priority Level 4 Service Levels 4 Hours/ 72 Hours 95.00% 85.00% Monthly App Maintenance Application Availability - Mission Critical (Tier 1) 99.90% 99.70% Monthly Application Availability -Business Critical (Tier 2) 99.81% 99.54% Monthly Application Availability -Significant (Tier 3) 99.40% 98.50% Monthly Application Availability -Important (Tier 4) 97.90% 90.00% Monthly Enhancements Implemented On-Time 95.20% 91.00% Monthly 15

15 Support Model: Communication (cont ) Communication Plan Engagement with business upfront on communication expectations Various activities and tasks identified for applications and communication plan prepared and finalized Prepared by, Reviewed by, Sent by, Sent to, follow ups and so on. We can always do better 16

16 Support Model: Communication (Sample) (cont ) Sr. No. Activity/Task Application Communication Prepared By Reviewed By Sent By Sent to SP Links 2 Outage(Planned/Unplanned) SAP O&M Business O&M End- Users Documentum O&M Business O&M End- Users OSB O&M Business O&M End- Users GDGIS O&M Business O&M End- Users Link Remarks Editor Editor/Web Editor 3 Releases GD GIS Editor O&M Business Communication before and after GD GIS Mobile O&M Business GD GIS Viewer O&M Business Landbase O&M Business Link Only for O&M and Minor Enhancements 4 Releases Notes : IT GD GIS Editor O&M GD GIS Viewer O&M Link Not to be sent to the Users, For IT Deliverables Landbase O&M 17

17 Minor Enhancements Scope/Expectations Deliver 5000 hours of minor enhancements in one quarter. (1250 hours for each tower) The hours decrease as year go by Requirements gathering onsite; Offshore development Quarterly releases Full SDLC cycle 18

18 Keeping Data Upto date Ensure that the landbase data is latest and greatest Updated data from 2012 to 2017 Build a process around this Using AGOL Geocoder on as need basis 19

19 Performance Monitoring and Disaster Recovery Esri System Monitor now called ArcGIS Monitor Performance Monitoring and Tuning End to End monitoring Disaster Recovery Twice a year RTO (Recovery Time Objective) and RPO (Recovery Point Objective) 20

20 Case Study: Performance Monitoring Problem Statement EDGIS WebViewer does not load EDGIS WebViewer throws error 21

21 Case Study: Performance Monitoring (cont ) Analysis One of the foundational map service appears to be running out of bandwidth (instances) There are already 10*10 = 100 ArcSOC available and even they are running short Look into why so many ArcSOCs are being used? Add more ArcSOCs? 22

22 Case Study: Performance Monitoring (cont ) Solution and Monitoring Added more ArcSOCs in short term For long term, MXD tuning, working on migrating Silverlight viewer to JavaScript viewer (possibly also an offline viewer) 23

23 Case Study: Performance Monitoring (cont ) From To 24

24 GISOPS during disasters We have to operate as if disaster is around the corner. Always keep the lights on and the gas flowing in Trinity Ballroom 3 25

25 Future Managing Performance, Capacity and Utilization Agile Projects DevOps Lifecycle upgrades and improvements Capital v/s Expense Utility Network? 26

26 Thanks Questions? 27