Distributed Orchestration of Microservices

Transcription

1 Distributed Orchestration of Co-Founder & Developer Advocate Camunda With thoughts

2 Bernd Ruecker Co-founder and Developer Advocate of Camunda Berlin,

3

4 3 common hypotheses I check today: # Events decrease coupling # Orchestration needs to be avoided # Workflow engines are painful

5 Simplified example: dash button Photo by 0xF2, available under Creative Commons BY-ND 2.0 license.

6 Three steps

7 Who is involved? Checkout Inventory Shipment

8 Autonomous (micro-)services Checkout Dedicated Application Processes Dedicated infrastructure Inventory Shipment Dedicated Development Teams

9 Events decrease coupling

10 Request/response: temporal coupling Checkout The button blinks if we can ship within 24 hours Request Response Inventory Shipment

11 Temporal decoupling with events and read models Read Model Checkout The button blinks if we can ship within 24 hours Good Fetched Inventory Good Stored Shipment *Events are facts about what happened (in the past)

12 Events can decrease coupling* *e.g. decentral data-management, read models, extract cross-cutting aspects

13 Peer-to-peer event choreography Order placed Checkout received Inventory Goods fetched Shipment Goods shipped

14 Peer-to-peer event choreography Order placed Fetch the goods before the payment Checkout received Inventory Goods fetched Shipment Goods shipped

15 Peer-to-peer event choreography Order placed Fetch the goods before the payment Checkout received Inventory Goods fetched Shipment Goods shipped

16 Peer-to-peer event choreography Order placed Checkout Fetch the goods before the payment Customers can pay via invoice received Inventory Goods fetched Shipment Goods shipped

17 Photo by born1945, available under Creative Commons BY 2.0 license.

18 Extract the end-to-end responsibility Order placed Checkout Retrieve payment Order received Inventory Shipment *Commands have an intent about what needs to happen in the future

19 Commands help to avoid (complex) peer-to-peer event chains

20 Orchestration needs to be avoided

21 Smart ESB-like middleware Order Checkout Order placed received Inventory Shipment Good fetched Good shipped

22 Dumb pipes Checkout Order Martin Fowler Inventory Shipment Smart endpoints and dumb pipes

23 Danger of god services? Checkout Order Sam Newmann Inventory Shipment A few smart god services tell anemic CRUD services what to do

24 Danger of god services? Checkout Order Sam Newmann Inventory Shipment A few smart god services tell anemic CRUD services what to do

25 A god service is only created with bad API design!

26 Who is responsible to deal with problems? Order Retrieve

27 Who is responsible to deal with problems? Order Retrieve Credit Card

28 Who is responsible to deal with problems? Order Retrieve Credit Card Rejected

29 Who is responsible to deal with problems? Order Retrieve Rejected Rejected Credit Card If the credit card was rejected, the customer can provide new details Client of dumb endpoints easily become a god services.

30 Who is responsible to deal with problems? Order Retrieve Credit Card Rejected received canceled If the credit card was rejected, the customer can provide new details

31 Who is responsible to deal with problems? Order Retrieve Credit Card Smart endpoints are potentially long-running Rejected received canceled If the credit card was rejected, the customer can provide new details Clients of smart endpoints remains lean.

32 Photo by Tookapic, available under Creative Commons CC0 1.0 license.

33 There was an error while sending your boarding pass

34 Current situation Web-UI Me Check-in

35 Current situation Web-UI Me Check-in Output Mgmt Barcode Generator

36 Current situation Web-UI Me Check-in Output Mgmt Barcode Generator

37 Current situation the bad part Web-UI Me Check-in Output Mgmt Barcode Generator

38 Current situation the bad part Web-UI Me Check-in Stateful Retry Output Mgmt Barcode Generator

39

40 Possible situation much better! Web-UI Me Check-in Output Mgmt Barcode Generator

41 Possible situation much better! Me Web-UI Check-in Stateful Retry Output Mgmt Barcode Generator

42 Possible situation much better! Me Web-UI Check-in Stateful Retry The failure never leaves this scope! Output Mgmt Barcode Generator

43 Handling State Persist thing (Entity, Actor, ) State machine or workflow engine Typical concerns DIY = effort, accidental complexity Scheduling, Versioning, operating, visibility, scalability,

44 Workflow engines are painful Complex, proprietary, heavyweight, central, developer adverse,

45 Workflow engines, state machines It is relevant in modern architectures

46 Workflow engines, state machines CADENCE Silicon valley has recognized

47 Workflow engines, state machines CADENCE

48 Workflow engines, state machines CADENCE also at scale

49 Workflow engines, state machines CADENCE for todays demo

50 The workflow engine handles state REST Credit Card

51 Fallbacks increase resilience REST Credit Card

52 Distributed systems

53 It is impossible to differentiate certain failure scenarios. Client Service Provider Independant of communication style!

54 One HTTP call introduces complexity you have to tackle! REST Credit Card

55 One HTTP call introduces complexity you have to tackle! REST Credit Card

56 Timeouts

57 Distributed systems

58 Business transaction / compensation*

59 Living documentation for long-running behaviour

60 A visual HTML report for a specific test case

61 Do you prefer coded or graphical DSLs? Bpmn.createExecutableProcess("order") //....sendtask("retrieve").camundaclass(retrieveadapter.class).receivetask("received").message("receivedevent") //... * BPMN - ISO notation for modeling and executing long-running processes and flows

62 Before mapping processes explicitly with BPMN, the truth was buried in the code and nobody knew what was going on. Jimmy Floyd, 24 Hour Fitnesse

63 Workflows live inside service boundaries

64 Manigfold architecture options

65 Lightweight workflow engines are great don t DIY* *e.g. enabling potentially long-running services, solving hard developer problems, can run decentralized

66

67 Zalando Sales-Order and Order-Fulfillment via Camunda for every order worldwide Orders Q2-2017: 22,2 Mio. Sales 2016: 3,6 Mrd. EUR Growth 2016: 23%

68 Some code Checkout Order Inventory Shipping Monitor Human Tasks H2 H2

69 # Events decrease coupling: sometimes read-models, but no complex peer-to-peer event chains! # Orchestration needs to be avoided: sometimes no ESB, smart endpoints/dumb pipes, important capabilities need a home # Workflow engines are painful: some of them lightweight engines are easy to use and can run decentralized, they solve hard developer problems, don t DIY

70 Thank you!

71 Contact: Slides: Blog: application-development/ 3-common-pitfalls-of-microservicesintegrationand-how-to-avoid-them.html events-workflow-automation With thoughts