Äriprotsesside modelleerimine ja automatiseerimine Loeng 11 Business process measurement data mining. Enn Õunapuu

Transcription

1

2 Äriprotsesside modelleerimine ja automatiseerimine oeng 11 Business process measurement data mining Enn Õunapuu

3 Business process measurement Process mining ProM

4 Probleem

5 Chapter 17 Process Mining and Simulation Moe ynn nne ozinat il van der alst rthur ter Hofstede Colin Fidge a university 2009,

6 Overview Introduction Preliminaries Process mining (with ProM) Process simulation for operational decision support Tools:, ProM & CPN Tools Conclusions 6

7 Introduction Correctness, effectiveness and efficiency of business processes are vital to an organization Significant gap between what is prescribed and what actually happens Process owners have limited info about what is actually happening Model-based (static) analysis Validation Verification (correctness of a model) Performance analysis Process Mining post-execution analysis Process Simulation what-if analysis 7

8 Preliminaries 8

9 Preliminaries: Data ogging Keeping track of execution data ctivities that have been carried out Timestamps (Start and end times of activities) esources involved Data Purposes udit trails Disaster recovery Monitoring Data Mining Process Mining Process Simulation 9

10 Preliminaries: Process Mining Event logs (recorded actual behaviors) Covers a wide-range of techniques Provide insights into control flow dependencies data usage resource involvement performance related statistics etc. Identify problems that cannot be identified by inspecting a static model alone 10

11 Preliminaries: Process Simulation Develop a simulation model at design time Carry out experiments under different assumptions Used for process reengineering decisions Data input is time-consuming and error-prone equires careful interpretation bstraction of the actual behavior Different assumptions made Inaccurate or Incomplete data input Starts from an empty initial state 11

12 Process Mining Process discovery: "hat is really happening?" Conformance checking: "Do we do what was agreed upon?" Performance analysis: "here are the bottlenecks?" Process prediction: "ill this case be late?" Process improvement: "How to redesign this process?" Etc. 12

13 Example: mining student data Process discovery: "hat is the real curriculum?" Conformance checking: "Do students meet the prerequisites?" Performance analysis: "here are the bottlenecks?" Process prediction: "ill a student complete his studies (in time)?" Process improvement: "How to redesign the curriculum?" 13

14 Process mining: inking events to models world business processes people machines components organizations models analyzes supports/ controls specifies configures implements analyzes software system records events, e.g., messages, transactions, etc. process/ system model discovery conformance event logs 14

15 here to start? process control diagnosis process mining process enactm ent process design im plem entation/ configuration 15

16 Process Mining with ProM 16

17 ProM framework One of the leading approaches to Process Mining Covers a wide range of analysis approaches 250+ plug-ins Process Discovery Social Network Conformance Checking Conversion capabilities between different formalisms Petri nets, EPCs, BPMN, BPE, Mining XM (MXM) log format 17

18 Basic Performance nalysis 18

19 esource nalysis 19

20 T Checker 20

21 Performance analysis showing bottlenecks flow time from to B bottlenecks throughput time 21

22 Dotted chart analysis short cases time (relative) events case s long cases 22

23 ProM and logs workflow events and data attributes n extractor function available as a ProMImport plug-in ProM can analyze logs in MXM format Prom can transform models into Petri nets <Process id="payment_subprocess.ywl"> <ProcessInstance id="3f9dfc e7-b9f7-329b5c6f0ded"> <udittrailentry> <orkflowmodelelement>check_prepaid_shipments_10</orkflowmodelelement> <EventType>start</EventType> <Timestamp> T10:11: :00</Timestamp> <Originator>JohnsI</Originator> </udittrailentry> <udittrailentry> <Data><ttribute name="prepaidshipment">true</ttribute></data> <orkflowmodelelement>check_prepaid_shipments_10</orkflowmodelelement> <EventType>complete</EventType> <Timestamp> T10:11: :00</Timestamp> <Originator>JohnsI</Originator> </udittrailentry> </ProcessInstance> </Process> 23

24 Starting point: event logs logs or other event logs, audit trails, databases, message logs, etc. unified event log (MXM) 24

25 Process Simulation 25

26 Integrated Simulation pproach 26

27 inking process mining to simulation Gather process statistics using process mining techniques Calibrate simulation experiments with this data nalyze simulation logs in the same way as execution logs 27

28 Data sources for process characteristics Design (orkflow and Organizational Models) Control and data flow Organizational model Initial data values ole assignments Historical (Event logs) Data value range distributions Execution time distributions Case arrival rate esource availability patterns State (orkflow system) Progress state Data values for running cases Busy resources un time for cases 28

29 Tools:, ProM and CPN Tools 29

30 rchitecture II Create and execute process models Maintain organizational models Extractor functionalities for event logs, organizational models and current state of the workflow system ProM Translate and integrate all the components into a Petri nets model nalyze event logs and simulation logs CPN Tools un simulation experiments Incorporate current state of workflows Generate simulation logs 30

31 Tool: rchitecture 31

32 32

33 Tool: rchitecture Use existing models 33

34 Tool: rchitecture II Use existing models Derive parameters 34

35 Tool: rchitecture III Use existing models Derive parameters Consider current state 35

36 Tool: rchitecture IV Use existing models Derive parameters Consider current state Simulation logs in MXM 36

37 Simulation: Example Payment Start s: Supply dmin Officer payment shipment payment freight Issue Shipment Invoice s: Supply dmin Officer [else] Check Pre-paid shipments c: Finance Officer [pre-paid shipments] Check Invoice equirement s: Supply dmin Officer [Invoice required] Issue Shipment Payment Order c: Finance Officer Issue Shipment emittance dvice c: Finance Officer Produce Freight Invoice Update Shipment Payment Order c: Finance Officer pprove Shipment Payment Order c: Senior Finance Officer Complete Invoice equirement s: Supply dmin Officer s: Supply dmin Officer [s. order not approved] [s. order approved] customer notified of the payment, customer makes the payment Process Shipment Payment o: ccount Manager Process Freight Payment s: Supply dmin Officer [payment incorrect due to underpayment] [payment incorrect due to overcharge] [payment correct] issue Debit djustment o: ccount Manager Issue Credit djustment o: ccount Manager customer makes the payment account settled Finalise o: ccount Manager End s: Supply dmin Officer 37

38 Simulation: Example 13 staff members 5 `supply admin officers 3 `finance officers' 2 `senior finance officers' 3 `account managers Case arrival rate: 50 payments per week Throughput time: 5 working days on average 30% of shipments are pre-paid 50% of orders are approved first-time 20% of payments are underpaid 10% of payments are overpaid 70% of payments are correct 80% of orders require invoices 20% of orders do not require invoices ssumption: Payment process running in for some time. 38

39 Simulation: Scenario 4 weeks till the end of financial year backlog of 30 payments (some for more than a week) Goal: ll payments to be processed in 4 weeks time un simulation experiments to see if the backlog can be cleared using current resources evaluate the effect of avoiding underpayments Possible remedial action: llocate more resources 39

40 ProM screenshots 40

41 CPN Tools 41

42 Four Scenarios 1. n empty initial state ( empty ) 2. fter loading the current state file with the 30 applications currently in the system ( as is ) 3. fter loading the current state file but adding 13 extra resources ( to be ) 4. fter loading the current state file but changing the model so that underpayments are no longer possible ( to be B') 42

43 Evaluation 43

44 Simulation for operational decision support Combine the real process execution log (`up to now') and the simulation log (which simulates the future `from now on') ook at the process execution in a unified manner Track both the history and the future of current cases 44

45 lpha algorithm α

46 Process log Minimal information in log: case id s and task id s. dditional information: event type, time, resources, and data. In this log there are three possible sequences: BCD CBD EF case 1 : task case 2 : task case 3 : task case 3 : task B case 1 : task B case 1 : task C case 2 : task C case 4 : task case 2 : task B case 2 : task D case 5 : task E case 4 : task C case 1 : task D case 3 : task C case 3 : task D case 4 : task B case 5 : task F case 4 : task D

47 >,,,# relations Direct succession: x>y iff for some case x is directly followed by y Causality: x y iff x>y and not y>x Parallel: x y iff x>y and y>x Choice: x#y iff not x>y and not y>x case 1 : task case 2 : task case 3 : task case 3 : task B case 1 : task B case 1 : task C case 2 : task C case 4 : task case 2 : task B case 2 : task D case 5 : task E case 4 : task C case 1 : task D case 3 : task C case 3 : task D case 4 : task B case 5 : task F case 4 : task D >B >C B>C B>D C>B C> D E>F B C C B B C B D C D E F

48 Basic idea (1) x y x y

49 Basic idea (2) y x z x y, x z, and y z

50 Basic idea (3) y x z x y, x z, and y#z

51 Basic idea (4) x z y x z, y z, and x y

52 Basic idea (5) x z y x z, y z, and x#y

53 It is not that simple: Basic alpha algorithm et be a workflow log over T. a() is defined as follows. 1. T = { t T $ s t s}, 2. T I = { t T $ s t = first(s) }, 3. T O = { t T $ s t = last(s) }, 4. X = { (,B) T B T " a " b B a b " a1,a2 a 1 # a 2 " b1,b2 B b 1 # b 2 }, 5. = { (,B) X " (,B ) X B B (,B) = (,B ) }, 6. P = { p (,B) (,B) } {i,o }, 7. F = { (a,p (,B) ) (,B) a } { (p (,B),b) (,B) b B } { (i,t) t T I } { (t,o ) t T O }, and 8. a() = (P,T,F ). The alpha algorithm has been proven to be correct for a large class of free-choice nets.

54 Example case 1 : task case 2 : task case 3 : task case 3 : task B case 1 : task B case 1 : task C case 2 : task C case 4 : task case 2 : task B case 2 : task D case 5 : task E case 4 : task C case 1 : task D case 3 : task C case 3 : task D case 4 : task B case 5 : task F case 4 : task D E B C F D a()

55 DEMO lpha algorithm B get review 1 get review X C M time-out 1 D time-out X K invite additional reviewer get review 2 G H I invite reviewers E time-out 2 collect reviews decide accept J F reject get review 3 G time-out 3 48 cases 16 performers

56 ogging system Nlog Nog can process diagnostic messages emitted from any.net language (such as C# or Visual Basic), augment them with contextual information (such as date/time, severity, thread, process, environment enviroment), format them according to your preference and send them to one or more targets such as file or database.

57 Supported targets Files - single file or multiple, with automatic file naming and archival Event og - local or remote Database - store your logs in databases supported by.net Network - using TCP, UDP, SOP, MSMQ protocols Command-line console - including color coding of messages - you can receive s whenever application errors occur SP.NET trace... and many more

58 Conclusions Introduction Concise assessment of reality needed for processes Preliminaries Data logging, Process Mining, Process Simulation Process mining with ProM Understanding process characteristics Process simulation Operational decision support Utilizing log info for simulation experiments Tools:, ProM & CPN Tools Payment example Conclusion 58

59 ProM usage ProM usage example 59

60 og 60

61 Process model mined from log 61

62 Example The running example is about a process to repair telephones in a company. The company have 3 types of phones (\T1", \T2" and \T3"). The process starts by registering a telephone device sent by a customer. fter registration, the telephone is sent to the Problem Detection (PD) department. There it is analyzed and its defect is categorized. In total, there are 10 dierent categories of defects that the phones xed by this company can have. Once the problem is identifed, the telephone is sent to the epair department and a letter is sent to the customer to inform him/her about the problem. The epair () department has two teams. One of the teams can x simple defects and the other team can repair complex defects. However, some of the defect categories can be repaired by both teams. Once a repair employee nishes working on a phone, this device is sent to the Quality ssurance (Q) department. There it is analyzed by an employee to check if the defect was indeed fixed or not. If the defect is not repaired, the telephone is again sent to the epair department. If the telephone is indeed repaired, the case is archived and the telephone is sent to the customer. To save on throughput time, the company only tries to x a defect a limited number of times. If the defect is not xed, the case is archived anyway and a brand new device is sent to the customer. 62

63 hat we can do? Inspecting and Cleaning an Event og Mining the Control-Flow Perspective of a Process - lpha algorithm Social networks 63

64 Questions? 64 64