How to build an autonomous anything

Transcription

1 How to build an autonomous anything Jim Tung MathWorks Fellow MathWorks 2015 The MathWorks, Inc. 1

2 2

3 3

4 4

5 5

6 6

7 7

8 Autonomous Technology 8

9 Autonomous Acting independently 9

10 Autonomous Technology Provides the ability of a system to act independent of direct human control under unrehearsed conditions 10

11 11

12 Capabilities of an Autonomous System Sense 12

13 Capabilities of an Autonomous System Sense Perceive 13

14 Capabilities of an Autonomous System Sense Perceive Decide & Plan Learning Algorithms Optimization 14

15 Capabilities of an Autonomous System Sense Perceive Decide & Plan Act Control Algorithms 15

16 Responsibility Autonomous Technology Balancing Responsibility Computer Human Degree of Autonomy 16

17 Cost of rig: $1,000,000+ Repair cost: $100,000 Cost of valve: $200 17

18 18

19 19

20 20

21 Autonomous Service for Predictive Maintenance Sense Which sensor values should they use? Vibration Timing Perceive Pressure Temperature Decide & Plan Act Other variables 21

22 Autonomous Service for Predictive Maintenance Sense Perceive Decide & Plan Act Normal Operation Monitor Closely Maintenance Needed 22

23 Machine Learning or Deep Learning? Machine Learning Approach Sensor 1 Sensor 2 Sensor 25 Sensor a Sensor b Sensor c 1. Normal 2. Monitor 3. Maintain Correlation Analysis Feature Extraction Classification Output Deep Learning Approach Sensor 1 Sensor 2 Sensor Normal 2. Monitor 3. Maintain Feature Extraction & Classification Output 23

24 Mega Release of Deep Learning Capabilities Design Deep Learning & Vision Algorithm Accelerate and Scale Training High Performance Embedded Implementation Deep learning design is easy in MATLAB Apps for Ground Truth Labeling, Pixel Labeling Pre-trained model importer Training Visualization Parallel Computing Toolbox 7x faster than pycaffe 2x faster than TensorFlow GPU Coder 14x faster than pycaffe 4x faster than TensorFlow 1.6x faster than C++ Caffe 24

25 What are the best predictors? Data-driven Name of Presenter Time and Location 25

26 What are the best predictors? Data-driven Model-driven Name of Presenter Time and Location Jet Engine Monitoring 26

27 Autonomous Glucose Level Management 27

28 Autonomous Glucose Level Management Bigfoot Biomedical Sense + - Perceive Decide & Plan Target Glucose Level Insulin Pump Person Act Continuous Glucose Monitor 28

29 Autonomous Glucose Level Management Bigfoot Biomedical Sense + - Perceive Decide & Plan Target Glucose Level Insulin Pump Person + + Act Mobile App Continuous Glucose Monitor 29

30 Autonomous Glucose Level Management Bigfoot Biomedical Sense Simulink, Stateflow, Polyspace + - Perceive Decide & Plan Target Glucose Level Insulin Pump Person + + Act Mobile App Continuous Glucose Monitor 30

31 Autonomous Glucose Level Management Bigfoot Biomedical Sense + - Perceive Decide & Plan Target Glucose Level Insulin Pump Person + + Act Mobile App Continuous Glucose Monitor 31

32 Autonomous Glucose Level Management Bigfoot Biomedical 2017 Sense Perceive Decide & Plan Target Glucose Level Insulin Pump Person + + Act Mobile App Continuous Glucose Monitor 32

33 Virtual Clinic Generating data through simulation 33

34 Virtual Clinic Scaling computations to simulate 50 million patients a day 34

35 Where will you get your data? Simulation Public repositories In the lab In the field Internet of Things (IoT) 35

36 Working with Big Data Just Got Easier Tall arrays in MATLAB Machine Memory e.g. 4~8GB e.g. 100GB~ 1TB Stream large input signals from MAT-files Tall Data 36

37 CNH Develops Intelligent Filling System for Forage Harvesters 37

38 38

39 Autonomous Trailer Filling Sense Perceive Decide & Plan Act Control Algorithms 39

40 Autonomous Trailer Filling Sense Perceive Decide & Plan Act Control Algorithms 40

41 41

42 Autonomous Trailer Filling Sense 3D Cameras Perceive Actuators Decide & Plan CAN Act ECU Computer vision and controls algorithms 42

43 Autonomous Trailer Filling Sense 3D Cameras Vehicle Display Controller Driver Input Visualization Perceive Decide & Plan CAN Actuators Act Computer vision and controls algorithms ECU 43

44 Autonomous Trailer Filling Sense Perceive Decide & Plan 3D Cameras Embedded Coder Vehicle Display Controller CAN Driver Input Visualization Computer Vision Controls Actuators Act Computer vision and controls algorithms ECU 44

45 Autonomous Trailer Filling Sense Perceive Decide & Plan 3D Cameras Vehicle Display Controller CAN Driver Input Visualization Computer Vision Controls Actuators Act Monitoring ECU 45

46 How will you put it into production? Embedded Systems IT Systems Cloud Desktop Apps 46

47 Investments in Model-Based Design Efficient code generation Floating-point HDL code generation 47

48 Investments in Model-Based Design Code verification in support of CERT C standard Detect and fix standards compliance issues at design time 48

49 Connected Physical Assets in Operation Data Ingestion Long-Range Communications Integration Local Communications Edge Management Smart assets Edge systems OT Infrastructure IT Systems Value in services, not just assets SW in everything Lots of sensors Pervasively connected More computing and monitoring near the assets Data transport protocols Transition to Web technology Cloud & on-prem Cloud: Big data On-prem: IP, reliability, etc. Operational Technology Optimization end-to-end 49

50 Automation through Digital Twins Data Ingestion Long-Range Communications Integration Local Communications Edge Management Smart assets Edge systems OT Infrastructure IT Systems Sense Perceive Decide & Plan Digital Twin: Composite of artifacts that characterize and predict behavior of a specific real asset. Act 50

51 Digital Twin isn t a new concept Long-Range Communications Asset Management Smart assets OT Infrastructure Digital Twin concept has been used for a long time, especially when there is a small number of expensive assets and when reliability is critical (e.g., spacecraft, aircraft engines). The infrastructure has been one-off. 51

52 Re-imagining the Digital Twin Data Ingestion Long-Range Communications Integration Local Communications Edge Management Smart assets Edge systems OT Infrastructure IT Systems Sense Perceive Decide & Plan Act Digital Twin: models (dynamic, FEM, data-driven, etc.) and data for each asset (e.g., system, component, or system of systems) performance and conditions over the asset s history. continuously updated as the asset is operated. always represents a faithful representation of the current state of the asset. 52

53 MATLAB and Simulink for Digital Twins: Key Capabilities Data Ingestion Long-Range Communications Integration Local Communications Edge Management Smart assets Edge systems OT Infrastructure IT Systems V POC P POC Multi-domain system modeling P SIM Q POC Q SIM Parameter estimation Control design and analysis VHDL, CODE C, C++ Verilo MCGENERATION DS FP gasi U P GA Automatic code generation C Structu red Text PLC Variety and Volumes of Data Optimization Machine Learning and Deep Learning Enterprise system integration, with cluster/cloud execution 53

54 MATLAB and Simulink for Digital Twins throughout the lifecycle System Wind farm planning Ensuring power grid reserves Process Cloud-based HVAC optimization Engine prognostics & health monitoring Product Future car infrastructure Automatic braking systems Asteroid rendezvous Planning Development Operation Maintenance 54

55 55

56 56

57 57

58 Capabilities of an Autonomous System Sense Perceive Decide & Plan Act Control Algorithms 58

59 How to build an autonomous anything Focus on Perception Use the Best Predictors Look for autonomy in creative places Do more than manually possible Data-driven Model-driven Get the Right Data Go to Production 59

60 How to build an autonomous anything Focus on Perception Use the Best Predictors Get the Right Data Go to Production Look for autonomy in creative places Do more than manually possible Data-driven Model-driven Reduce to actionable data Take advantage of Big Data Use simulation to supplement available data Address the architecture Leverage Model-Based Design for embedded Automate integration with enterprise IT systems 60

61 What is your autonomous anything? 61