EVENTUAL CONSISTENCY. Albrecht Stäbler, Shareholder of NovaTec Holding GmbH

Size: px

Start display at page:

Download "EVENTUAL CONSISTENCY. Albrecht Stäbler, Shareholder of NovaTec Holding GmbH"

Nigel Montgomery
6 years ago
Views:

EVENTUAL CONSISTENCY 06/25/2015, ALBRECHT

Eventual Consistency: New Paradigms are needed

and -Architectures to execute the digital

NovaTec Holding GmbH albrecht.staebler@gmx.

national and international Diskussionskreis

1 EVENTUAL CONSISTENCY 06/25/2015, ALBRECHT STÄBLER, SHAREHOLDER NOVATEC HOLDING GMBH Eventual Consistency: New Paradigms are needed for Software-Development, -Engineering -Processes and -Architectures to execute the digital Transformation Albrecht Stäbler, Shareholder of NovaTec Holding GmbH Best Practices and trends from projects, national and international Diskussionskreis Cloud, Forschung und Technologie together with the companies IBM, RedHat, DB- Systel, BG-Phoenics, University of Stuttgart and others Many thanks especially to the ICO- and BlueMix-teams from IBM

2 About the author / albrecht.staebler@gmx.net Albrecht Stäbler, born in 1965, until Mai 2015 chairman of the board of NovaTec Holding GmbH, is holding a university degree as Diplom-Ingenieur (University of Stuttgart) Albrecht Stäbler is teaching Cloud native Architectures EA/EAM, Software-Engineering & -Architectures, Operating System Software, Middleware and Workflow Management at universities domestically and abroad In strategic customer projects Albrecht Stäbler provided his experiences and his know-how for audits and crisis management As CEO Albrecht Stäbler was responsible for the corporate strategy Within the last years Albrecht Stäbler was strategically dealing with solution concepts for digital business transformations according to architectural and methodological aspects like cloud native architectures for new important business fields like IoT, BigData, Industry 4.0 and others 2

3 Disruption meets fields like technology, methods and user behaviour and habit Examples to the traditional thinking Pollution: calculation average for Baden Württemberg Traditional thinking (ACID based) Missing sensors are not allowed Completeness No fault tolerance New thinking Mass data statistical approach Missing sensor not relevant Fault tolerance, one missing sensor does not influence the overall statement Culture of the second chance To strand as an entrepreneur is not an issue, it s a feature to improve the second challenge 3

4 Disruption meets fields like technology, methods and user behaviour and habit Examples according to the new globalized approach Pillars of Israels startup success story Society: immigration enhances with multi cultural aspects Culture: Chutzpah The word is sometimes interpreted - particularly in business parlance - as meaning the amount of courage mettle or ardor that an individual has No hierarchy Political: army dominated new economy Famous Unit 8200 is an Israeli Intelligence Corps unit responsible for collecting signal intelligence (SIGINT) and code decryption 4

5 Different speed in digital business transformation in different countries Kingdom of Saudi Arabia: highest usage of twitter and facebook in average per habitant worldwide Political changes: Arabic spring was organized using messaging services Sometimes better infrastructure in so called developping countries Greenfield approach Investment protection Different culture and ages according to pay with own data Stability and obsolescenses Robustness inside systemic world / robustness / fault tolerances inside systems 5

Traditional Approach Business Architecture (Defined) IT / IS Strategy Traditional IT Strategy and Architecture Process Business Goals IT gets engaged late in Technology Implementation Projects after

6 Traditional Approach Business Architecture (Defined) IT / IS Strategy Traditional IT Strategy and Architecture Process Business Goals IT gets engaged late in Technology Implementation Projects after ability to collaborate has largely passed Enterprise IT Architecture Start Business Transformation Initiatives are started here, outside of the IT strategy process Business Exploration / Discovery Traditional Business Strategy and Transformation Process Business Transformation Design Business Change Programme IT Projects IT Projects Portfolio and Solution Architecture IT Projects 6

7 Consequences to the traditional approach Business and IT often remain poorly aligned/connected IT Departments are often still seen as a cost Business and IT find it hard to communicate effectively This lack of alignment often results in: Costs of IT being significantly larger than necessary Strategic investments being under-utilised Service Oriented Architecture makes this even more critical Technical and methodological evolution (lessons learned): micro services 7

8 Why is this approach unsustainable Too many external factors affecting technological competencies Government Prominent Role Disruptive Technology Changes Generation Y Need for Business Change Leadership Excellence Vision Regulation Executives need to stop looking at IT projects as technology installations and start looking at them as periods of organizational change that they have the responsibility to manage Harvard Business Review, November 2006 by Andrew McAfee, Associate Professor, Harvard Business School 8

9 Perception of IT is often falling behind expectations to find answers to new demands Source: gardeviance.org Smart Devices BigData GAP Real- Time BI / Mobile 9

10 We see an emergence of Systems of Engagement and Internet of Things co-existing with Systems of Record Systems of Record Systems of Engagement Data & Transactions Mobile CRM ERP App Infrastructure Virtualized Resources Next Generation Architectures Social Networking BigData and Analytics Insight Systems of Discovery Sensors Embedded intelligence Connected devices Signal from noise Internet of Things 10

IT also needs to become smarter about delivering services (continuous, automated delivery) OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) TC The growing complexity of

11 IT also needs to become smarter about delivering services (continuous, automated delivery) OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) TC The growing complexity of IT systems and soon a trillion connected things demand that sprawling processes become standardized services that are efficient, secure and easy to access. A Service Management System will provide visibility, control and automation across IT and business services to ensure consistent delivery. Cloud Computing describes a new consumption and delivery model for IT services Tosca describes a declarative and imperative topology and orchestration model for the cloud 11

12 Shifting Trends in the IT Environment CEOs in the meanwhile see IT as a business enabler, no longer as a cost factor only From monolithic applications To dynamic services From static infrastructure To cloud services From programmed systems To cognitive systems From structured data at rest To unstructured data in motion From stable well-defined workloads To unpredictable workloads From standard devices To a variety of devices From proprietary standards To open innovation From Corporate owned IT To Infrastructure as a Service 12

Platform as a Service Software as a Service

Data Data Data Data Runtime Runtime Runtime Runtime

O/S Virtualization Virtualization Virtualization

Storage Storage Storage Networking Networking

13 IaaS, PaaS, SaaS who manages what? Traditional On-Premises Infrastructure as a Service Platform as a Service Software as a Service Applications Applications Applications Applications Data Data Data Data Runtime Runtime Runtime Runtime Middleware Middleware Middleware Middleware O/S O/S O/S O/S Virtualization Virtualization Virtualization Virtualization Servers Servers Servers Servers Storage Storage Storage Storage Networking Networking Networking Networking Client Manages Vendor Manages in Cloud 13 Standardization; OPEX savings; faster time to value

14 Attributes of cloud-native software (1) Modular, non-monolithic architecture ( micro services ) System consists of many small, self-contained components Each component / service developed and operated by a selfcontained team Built-in Multi-tenancy The higher the level of multi-tenancy, the more efficient the app is Built from the beginning to be operated as a (very) large, shared distributed system avoiding dedicated SW installations Horizontal Scalability Elastic Clean Separation of stateful from stateless components Each component can be scaled horizontally Can flexibility grow and shrink depending on load or other needs Stateless components can be treated more efficiently than stateful ones Loosely coupled Non-blocking communication Components can come up at any point in time Since anything can fail anytime (and be restarted elsewhere), avoid startup dependencies 14

15 Attributes of cloud-native software (2) Design for failure The larger a distributed app is, the more likely it is that a failure occurs for one of its components Avoidance of single points of failure Code assumes any node can fail at any point in time All application components should be resilient to reboots Granular failure In case one component of an application fails, the remaining ones should continue to be available Accessible via idempotent APIs API-based accessibility for an app Both usage and lifecycle mgmt API Developed in a hosted-first model Development and operations covered by a single team, owning the application e2e from dev to ops Typically very small teams Updates are applied very frequently (vs. big shipments every 6-9 months) Management concerns need to be addressed by the application from the beginning Granular rollout features are continuously added to the code (no large feature branches), gradually opened up to select audiences Easy-of-use / self-service In today s cloud-native world it s critical to allow achieving early success very quickly without requiring the user to read lots of documentation 15

16 Multi-tenancy Options Degrees of Isolation and Sharing Shared Hardware Shared OS Shared Middleware Shared App Tenant Tenant Tenant Tenant Tenant Tenant Tenant Tenant Application Instance Application Instance Application Instance Application Instance Application Instance Application Instance Multi-Tenant Application AppServer & Database AppServer & Database AppServer & Database AppServer & Database AppServer & Database AppServer & Database OS OS OS OS OS Hardware Hardware Hardware Hardware shared hardware shared OS shared middleware shared application Isolation Sharing 16

17 Considerations for moving Workloads to the Cloud What workloads are we seeing move to Cloud delivery? Single virtual appliance workloads Test and Pre-production systems Mature packaged offerings like and collaboration Software development environments Batch processing jobs with limited security requirements Isolated workloads where latency between components is not an issue Storage solutions / storage as a service Backup solutions / backup and restore as a service Some data intensive workloads, if the provider has a cloud storage offering tied to the cloud compute offering What workloads may not be ready for Cloud delivery today? Workloads which depend on sensitive data normally restricted to the enterprise e.g. employee information, health care records Workloads composed of multiple, co-dependent services High throughput online transaction processing Workloads requiring a high level of auditability, accountability e.g. workloads subject to Sarbanes-Oxley Workloads based on 3 rd party software which does not have a virtualization or cloud aware licensing strategy Workloads requiring detailed chargeback or utilization measurement as required for capacity planning or departmental level billing Workloads requiring customization (e.g. customized SaaS 17

Cloud Services Spectrum Cloud Enabled Workloads Cloud Centric Workloads Scalable Elastic Virtualized Multi-tenant Automated LIfecycle Integrated Lifecycle Heterogeneous

18 Cloud Services Spectrum Cloud Enabled Workloads Cloud Centric Workloads Scalable Elastic Virtualized Multi-tenant Automated LIfecycle Integrated Lifecycle Heterogeneous Infrastructure Standardized Infrastructure Existing Middleware Workloads Compatibility with existing systems Emerging Platform Workloads Exploitation of new environments 18 18

19 A New Operating Model for Cloud Centric Applications Capabilities and User Experience Today Emerging Primary Workload Types Systems of Record (Transactional) Systems of Engagement (+ Record) (BigData, Analytics, Mobile/Social Channels) Delivery Model Planned Incremental (DevOps) Development and Operations Team Sizes 100s and Costly 10s with built-in DevOps automation Release Frequency Months to Years Days to Weeks, based on business opportunity Integration Frequency Weeks Continuous Infrastructure Deployment Days Minutes Time to Value Planned Opportunistic Operational Model Service Sourcing Systems Management Develop Built in to application, Recovery Oriented Computing, Continuous Availability Consume and Assemble (Public and Private) 19

20 Workloads and Applications Client Applications Driven by market needs vs. technology New workloads Cloud Centric Systems of Engagement + Mobile Access Social & Collaboration Analytics Hadoop PHP Cassandra Nginx Munin Optimized for agility and velocity for variable workloads, scale, dynamic composition, multiple programming models & services New scalable runtime focusing on progressive composition with loosely coupled delegated models integrating development, application services, operations, and infrastructure Existing Workloads Cloud Enabled Systems of Record + Custom Applications Packaged Apps (IBM, SAP, Oracle) Middleware (J2EE, Transactions) Database BPM WAS/J2EE CICS/IMS Portal Optimized for reducing OPEX for existing patterns and integrating with existing operational and service management processes Significant body of automation and integration content for existing workloads (SAP, Oracle etc.) Still a large opportunity as customers implement virtualization and evolve to standardization 20

21 Five Emerging Cloud Architectures / new DevOps- and SE processes according to 1. Virtualized Traditional - Extensions of Java Application Servers, Support for Traditional Transactional Workloads (Cloud enabled) Moving existing workloads to the cloud Requires best practices, patterns, tooling 2. Database Centric - data driven + small computation on small data With multi-tenancy attractive for enterprise and service providers 3. Content Centric - computation needs to be close to data + large computation on large data Data Mining, Analytics, Data Warehouse, 4. Loosely Coupled - computation and data are separate Can be addressed by existing middleware, but relaxed consistency models emerging 5. Storage Analytics - Data and Storage Integration Composable software development, integrated continuous delivery as a service, IDE as a service, i.e. BlueMix 21

Investment protection, non greenfield (standard situation): Manage cloud native architectures / Cloud enabling strategies Existing approaches for Enterprise Architecture Management are too

Architects) Hard to keep data current due to lacking engagement of the rest of the organization High initial invest Do-it-yourself Re-invent the wheel when defining own Architecture

22 Investment protection, non greenfield (standard situation): Manage cloud native architectures / Cloud enabling strategies Existing approaches for Enterprise Architecture Management are too limited Oversized High training effort due to strong), level of detail results in extensive data gathering upfront Tools, methods and vizualisations suitable for few experts only (IT Architects) Hard to keep data current due to lacking engagement of the rest of the organization High initial invest Do-it-yourself Re-invent the wheel when defining own Architecture Frameworks and data models Effort to customize generic tools such as SharePoint or Excel Manual creation of appealing vizual reports of the IT landscape Cluttered user interfaces which are hard to use 22

23 A different approach with more social interaction promises greater benefits for Architecture Management More than 50% of companies report increased speed to access internal experts, reduce communication cost and increased speed to access knowledge. 3) 15% of applications can be shut down if identified through transparency 4) 23 Source: 1) 3) McKinsey Global Survey, 2) IBM Global CIO Study, 4) Detecon, 5) Nucleus Research

The need for new approaches was born in daily

Application & Technology Lifecycle Management

Reduction Business & IT Transformation Legacy

24 The need for new approaches was born in daily projects and roles in a Logistics company Application & Technology Lifecycle Management IT Integration after M&A IT Complexity Reduction Business & IT Transformation Legacy IT landscape in > 100 business units IT Integration in > 80 countries ~ 1000 applications in 90 countries Cloud Roadmap for > 100 applications 24

collaboration focusses on information that can be turned into value for the company IT Inventory Interactive

training effort Fast and easy access to essential information for everybody (Smart Search) Harmonize the IT

the users needs (Filters & Views) Faster communication and engage your employees to collaborate across

25 collaboration focusses on information that can be turned into value for the company IT Inventory Interactive Reporting Collaboration Platform Capture the knowledge of distributed users in a single source of truth without training effort Fast and easy access to essential information for everybody (Smart Search) Harmonize the IT portfolio to optimize IT cost and reduce IT complexity Increase the use of information by reports tailored to the users needs (Filters & Views) Faster communication and engage your employees to collaborate across organizational boundaries Establish an efficient information exchange better than Ping-Pong and cluttered file-shares 25

smart data model covers all central building blocks of TOGAF architecture management TOGAF Attribute in Fact Sheets User Group Source: TOGAF Recommendation Should not be covered in a structured way,

Technology-Architecture can be reflected To ensure high data quality and usage of a broader audience not all details will be stored (avoid data silos) Business Capability Project Data Object

26 smart data model covers all central building blocks of TOGAF architecture management TOGAF Attribute in Fact Sheets User Group Source: TOGAF Recommendation Should not be covered in a structured way, because: Understand text format is easier Link to related objects difficult to achieve Good coverage of all essential information in repository All concepts in Business-, Information-Systems and Technology-Architecture can be reflected To ensure high data quality and usage of a broader audience not all details will be stored (avoid data silos) Business Capability Project Data Object Application IT Component Tech. Stack Planning & Collaboration supports transformation management Lifecycle, successor and visualization of roadmaps Management of the IT project poretfolio to achieve target architecture 26

27 Interaction and collaboration between different stakeholders make handling IT topics more easy CIO, CFO, CEO Business Unit IT Business Manager & Project Leaders Understandable Terms Self-explanatory fields Easy customization Inline-Forms Interactive Reporting Collaboration Platform Finance & Procurement IT Infrastruktur & Service Managers IT Architects 27

28 THANK YOU FOR YOUR ATTENTION 06/25/2015, ALBRECHT STÄBLER, HERMAN HOLLERITH ZENTRUM

Oracle Cloud Blueprint and Roadmap Service. 1 Copyright 2012, Oracle and/or its affiliates. All rights reserved.

Oracle Cloud Blueprint and Roadmap Service 1 Copyright 2012, Oracle and/or its affiliates. All rights reserved. Cloud Computing: Addressing Today s Business Challenges Business Flexibility & Agility Cost