TechArch Day Digital Decoupling. Oscar Renalias. Accenture

Transcription

1 TechArch Day 2018 Digital Decoupling Oscar Renalias Accenture

2

3 THE ERA OF THE BIG TRANSFORMATION PROJECT IS OVER. CxOs are no longer willing to pay for a multi-million, multi-year effort after which they might, possibly get business value. VALUE MUST BE DELIVERED QUICKLY, AND FREQUENTLY Value quickly delivered, continuously thereafter, with the freedom to pivot LEGACY ISN T MAGICALLY GOING AWAY We can t just jettison legacy systems. There s tons of business logic and data locked into those systems

4 DIGITAL DECOUPLING HELPS ENTERPRISES REACH IT AGILITY WHILE RETAINING THE VALUE IN LEGACY SYSTEMS

5 FROM TO Individual customers Corporate customers Partners Channels Individual customers Corporate customers Partners 3 rd party Seamless digital omni-channel experience as a default and personal when needed DIGITAL DECOUPLING Legacy system Legacy system Legacy system ETL Legacy system Legacy system Legacy system Reporting CRM ERP ETL HR INVOICING Reporting CRM ERP HR INVOICING

6 DIGITAL DECOUPLING PRINCIPLES

7 DATA-CENTRICITY

8 DATA-CENTRIC DECOUPLING CONSUMERS REGULATORY REPORTING AD-HOC/ EXPLORATORY BI/ VISUALIZATION BUSINESS APPLICATIONS STORAGE APIS MESSAGING & EVENTS DATA WAREHOUSE ANALYSIS & CONSUMPTION REAL-TIME ANALYTICS DATA SERVICES COMMON CAPABILITIES KPIs DATA LINEAGE ACCESS CONTROL AUDITING DEVOPS OPERATIONAL DATA INGESTION RAW DATA STAGING DATA REPORTING/ DW FILESYSTEM LONG-TERM ARCHIVAL PROCESSING STREAMING ETL BATCH/ MAP- REDUCE STREAM PROCESSING CHANGE DATA CAPTURE MACHINE LEARNING DATA SOURCES LEGACY DB CORE SYSTEM LEGACY DB LEGACY SYSTEMS

9 GETTING DATA OUT Database: Change Data Capture or batch export System: Data feed Database: Process and insert System: REST calls SOURCE SOURCE Event-by-event Processing (There is often some state here) TARGET CONTINOUSLY PROCESS UPDATES

10 DATA LAKE TECHNOLOGIES* STORAGE INGESTION PROCESSING ANALYSIS & CONSUMPTION COMMON CAPABILITIES *: not exhaustive

11 API-ENABLED

12 WRAP THE CORE Individual customers Corporate customers Partners 3 rd party API CONSUMERS API WRAPPER LEGACY SYSTEMS

13 WRAP THE CORE Individual customers Corporate customers Partners 3 rd party API CONSUMERS UPDATE API WRAPPER READ DATA SERVICES RAW DATA STAGING DATA REPORTING/ DW BI/ VISUALIZATION CORE SYSTEM LONG-TERM ARCHIVAL OPERATIONAL DATA BUSINESS APPLICATIONS AD-HOC/ EXPLORATORY LEGACY SYSTEMS LEGACY DB LEGACY DB LEGACY SYSTEMS ETL CHANGE DATA CAPTURE STREAMING BATCH/ MAP- REDUCE FILESYSTEM STORAGE STREAM PROCESSING MACHINE LEARNING DATA SERVICES REAL-TIME ANALYTICS DATA WAREHOUSE REGULATORY REPORTING APIS MESSAGING & EVENTS INGESTION PROCESSING ANALYSIS & CONSUMPTION DATA SOURCES COMMON CAPABILITIES KPIs DATA LINEAGE ACCESS CONTROL AUDITING DEVOPS CONSUMERS REPLICATION

14 REAL-TIME & EVENT-BASED

15 Service A Service B Consume Produce Consume Produce EVENT STREAM Consume Produce Service B

16 UI BUSINESS EVENT NEW CORE SYSTEM BUSINESS EVENT EVENT: CUSTOMER UPDATED EVENT STREAM EVENT: ROW UPDATED EVENT: DATA UPDATED EVENT: CUSTOMER UPDATED LEGACY DATABASE HYDRATOR TECHNICAL EVENT BUSINESS EVENT

17 SCALABLE, HIGH PERFORMANCE MESSAGING Producer Consumer Consumer Producer Consumer Kafka Cluster

18 CLOUD-NATIVE (CLOUD-READY)

19 Elastic MapReduce EKS HDInsight Event Hub Pubsub Dataproc Kinesis Streams AKS GKE

20 STATE OF THE ART USER EXPERIENCE

21 ACCOUNT TRANSACTIONS EVENT-DRIVEN FRONT-ENDS DISPLAYED LIVE SEE IF ROOM AVAILABILITY IS CHANGING ASYNCHRONOUS REAL- TIME INTEREACTIONS CHOOSE A FLIGHT SEAT THAT IS ACTUALLY AVAILABLE CHANNEL-INDEPENDENT

22 FROM BACKEND EVENTS TO FRONTEND EVENTS HTTP Amazon Kinesis WebSocket SSE HTTP

23 COMBINATORY EFFECTS

24 BUILD THE NEW ON TOP OF THE OLD PRESENTATION LAYER TRADITIONAL UI REACTIVE/REAL- TIME UI SERVICE LAYER ANALYTICS LEGACY APIS NEW SERVICES DECOUPLED DATA LAKE REPLICATION STREAMING DATA STORE STREAMING AI/ML LEGACY

25 MODERNIZE LEGACY VIA DECOUPLING PRESENTATION LAYER TRADITIONAL UI REACTIVE/REAL- TIME UI LEGACY APIS SERVICE LAYER NEW SERVICES ANALYTICS DECOUPLED DATA LAKE CORE SYSTEM LEGACY

26 DIGITAL DECOUPLING CASE STUDIES

27 DECOUPLED DATA LAKE FOR DATA OFFLOADING Data Management Online Channel Digitally Decoupled Business Processing C MBaaS Mobile ebanking Real-Time APIs APIs Core Banking Mainframe A Real-Time Data Replication Log File Replication B Real-Time HBase SQL Query Search Interactive Analytics Stream Processing Scripting YARN Map Reduce Workflow Machine Learning ZooKeeper Real-Time D Define Filter Native Mobile Push Hadoop Distributed File System (HDFS) Data replication Data Persistence A One-way replication Alnova B Store movements and C à Lake in operation for balances in Data Lake relevant DB2 tables Re-Route Reading Customer will get in realtime their movements and balances from the data lake via microservices D Everything is an event Real-time events can be captured and processed by any consumer, such as native mobile notifications

28 DECOUPLING FOR LEGACY TRANSFORMATION API & Presentation Layer Enterprise Wrapper Access / Service Platform Layers Enterprise Systems Legacy Frontends Apps 3 rd Parties Modern Web External / 3 rd Party Data Source 4 3 Service Calls Stream Processing 1 Replication Customer Claims Data capture Customer Policy Service Data capture Product Document Mgmt Payment Product Application Document Mgmt Digital Decoupling Service Platform Billing Re-insurance Social Monitor Operational Data Lake Re-insurance Lead Activity 5 Distribution Mgmt Application Agent Onboarding Distribution Mgmt Commission Agent Onboarding API Gateway Policy Service Activity KYC Claims Billing Payment Commission Lead BI Social Monitor KYC BI 2 2 Realtime Stream Processing Realtime BI BI / Reporting / Analytics DATA-CENTRYIC DATA REPLICATION Data is fed near real-time into data lake, allowing reliable operation on the replica REAL-TIME STREAM PROCESSING Stream processing enables realtime action on data WRAP THE MONOLITH SERVICE PLATFORM Services can be built on-top of the data lake in agile fashion ENTERPRISE WRAPPER Old services can be integrated and slowly modernized API CONSUMPTION exposing and consuming APIs

29 29 A REAL-TIME DIGITAL BANK BUILT ON TOP OF A CLASSIC ESTATE DATA-CENTRIC Data Sources Core Data Ingestion MIS Corporate Analytical Internal Banking Service Structured Data Databases Layer Hadoop Product Systems Customer Management Corporate Systems Channels External Social Networks Message Bus Standard Interface Structured Internal Data Structured External Data Online Queues Interactions Flume Business Events Kafka Non-structured Interfaces ETL Processing Hub Data Lake MIS IBM Banking DW Legacy MIS and reporting stack Batch Python Spark 1 (2) (1) 1 3 Streaming SAS Event Stream Processing 2 INTERACTIONSCUSTOMERS CONTRACTS PROSPECTS BUSINESS EVENTS REFERENCE DATA MASTER DATA External Analytical Databases DMP Engines 2 4 (3) Segmentation and Propensity (4) Online Interactions Engine SAS RT Decision manager Event-based Campaign Management SAS Marketing Automation Campaign Management SAS Marketing Automation Visualization and Analytics Visualization Data Exploration SAS Visual Analytics Descriptive and Predictive Analytics Advertising Engines (5) DATA REPLICATION Data is fed near real-time into data lake, allowing reliable operation on the replica DATA LAKE Stream processing enables realtime action on data REAL-TIME STREAM PROCESSING Stream processing enables realtime action on data INTELLIGENCE Data Providers Non-structured Data Low level service messages Raw Data Hadoop Ad Hoc Web / Mobile Analytics Security Quality Credit Approval Engine Custom Audiences 4 STREAM PROCESSING Stream processing enables realtime action on data Governance

30 Thank you