BIG DATA AT MAGYAR TELEKOM Márton Kelemen Head of BI Innovation Center

Transcription

1 BIG DATA AT MAGYAR TELEKOM Márton Kelemen Head of BI Innovation Center

2 Strategic program : Strategic Program IT & Business together, with Top Management Buy-in Outcome: Big Data Analytics Self Service/Data Discovery Improve Data Quality Network of Truth Agile BI Future-proof Architecture Established the foundations of Big Data 5/15/2018 2

3 BIG data WHY NOW? (2014) Big data fever (2013/14) Event based churn (teradata / marketing) Social media command center (hadoop / T-SYSTEMS) Location based ads (EMC / product house) DPI reporting (CLOUDERA hadoop / IT) DWH-BI-Bingo-Big Data 5/15/2018 3

4 DPI REPORTING PILOT (2014) 4bn records per day (1,5 TB raw data) Original report: once a year, 3 months delay Classic DWH trial: 1 day of data aggregated in 3 days Hadoop trial: 1 day of data aggregated in 15 secs 4

5 BIG DATA SUMMARY Network data Customer data Facebook data DATA SOURCES TV data Movement data Webshop data Competitive advantages, growth, transparency, Anonymization, cleansing, stitching, enrichment, event stream processing CONSUMERS OUTCOMES Network optimization Customer Experience Marketing E-Commerce Data Monetization Security, fraud management ENTERPRISE DATA LAKE 80 GB/day ingested 100+ TB data in total 200+ Big Data Users Business modeling, insight understanding 100+ Business Reports/Analytics DATA PROCESSING INSIGHTS Combine internal & external sources Search for patterns Visualize data Create models, analyses, predictions Machine Learning 5/15/2018 5

6 DWPedia BI LANDSCAPE BI Portal COMMON BI LAYER Data Mining Data Discovery Reporting Dashboards DWH DATA LAKE DN LAYER INTERFACE LAYER CONSUMPTION LAYER ANALYTICS LAYER ENGINEERING LAYER STAGING LAYER MIRROR LAYER Unified BI with seamless backend integration 5/15/2018 6

7 Data Flow Management Source systems Staging S M E A C Mirror M Data Lake Target systems, consumers Special Approval Sensitive data repository MIRROR ENGINEERING ANALYTICS CONSUMPTION Sensitive data processing with logically segregated layers (Privacy by Design) 5/15/2018 7

8 Big Data Development Process Overview Big Data DEVELOPMENT process Big Data DATA SCIENTIST activity Regular Prod. process Read only Upgrade Prod. Algorithm (release mgmt. process) Local development OR Big Data TST environment (developer user account) Testing newly developed applications Big Data TST environment (technical user account) Production deployment Only BI Inn. team! Big Data PROD environment Advantages: Dedicated space in PROD for Data Mining and other Data Science activity Dedicated resource queue (separated for Prod process and Data Sciences) Better performance, suitable for real-life tests Access to prod. tables, more data to work on ML algorithms

9 Big Data Architecture

10 Use Cases

11 CUSTOMER BEHAVIOR ANALYSIS 11

12 IT and Big Data OperationS Analytics 12

13 CUSTOMER BEHAVIOR ANALYSIS (POKÉMON) Best customer experience Benefits First real data-driven campaign launched in only 1 month after Pokémon launch that reached more than twice conversion rate. Fast launched campaign to mobilize Pokémon GO players LEVELUP DATA OPTIONS SOLD (15th August 15th September) I am really proud of our first big data driven campaign with excellent results (Marketing) 13

14 Anodot Anodot provides real time analytics and automated anomaly detection, discovering outliers in vast amounts of data and turning them into valuable business insights.

15 REAL-TIME ONLINE PERSONALIZATION Best customer experience Benefits First Machine Learning solution in MT with much higher conversion rate, lead generation than classic segment based solutions. This is the future (TU) We can build the next generation campaigning on BInGO solutions (E-commerce)

16 Improve THE user experience (Hard Drop calls or data sessions) 16

17 Competitivenes Analysis 17