Deep Learning for Anomaly Detection in Manufacturing

Transcription

1 Deep Learning for Anomaly Detection in Manufacturing July, 2018 David Katz, Data Scientist, TIBCO Software Mike Alperin, Manufacturing Industry Consultant, TIBCO Software

2 Agenda Anomaly Detection in Manufacturing Deep Learning Autoencoder Neural Networks Autoencoders Software Tools Virtual Demo 2

3 Anomaly Detection in Manufacturing 3

4 Anomaly Detection in Manufacturing Detecting new problems Supervised vs. Unsupervised Learning Some Types of Anomalies Facilities Equipment - sensor & environmental data Process Equipment sensor FDC data Process Results Process history and measurements Physical Defects defect images & characteristics Device and Product PCM and Sort data A General Method 1. Detect anomalies 2. Cluster them 3. Classify with fingerprints or signatures 4. Determine causes of anomaly classes 5. Develop Action Plans to address causes 6. Predict cluster for new material and intervene to mitigate potential problems 4

5 Univariate Statistical Process Control Detect changes from baseline one variable at a time Shewhart Process Control Charts Statistically derived Control Limits Western Electric or Nelson rules Automated Alerting Individual Moving Range Control Charts 5

6 The Power of Multivariate Control Charts Suppose we measured 2 parameters y1 and y2 (e.g., person s height & 1/weight) Univariate charts would not detect some obvious outliers This happens in many real applications Bad Tester

7 Univariate & Multivariate Methods 7

8 Real-time equipment anomaly prediction & clustering High Tech Manufacturing Accelerator M 6 8

9 Deep Learning Autoencoder 9

10 From Neural Networks to Deep Learning Frank Rosenblatt, Cornell, inventor of the Perceptron. Brain mechanisms and models. Why the explosion? New algorithms and techniques Convolutional NN, Recursive NN, Generative Adversarial NNs New Hardware capabilities GPU Multicore Clusters More Data New Tools from the Open Source world. cs231n.github.io/neural-networks-1 10

11 Autoencoders Create an identity transformation with constraints Analogy to Principal Components but much more flexible/accurate. Anomalies the output is the reconstructed input, but it does not fully match the original input => Reconstruction Error Reconstruction Error: By component By sample. 11

12 Autoencoder Applications Fraud Credit Natural Language Speech Abstract Smart manufacturing refers to using advanced data analytics to complement physical science for improving system performance and decision making. With the widespread deployment of sensors and Internet of Things, there is an Video Manufacturing 12

13 Autoencoders Types and Variants Sparse Autoencoders Denoising Autoencoders Generative Adversarial Networks Variational Autoencoders 13

14 Deep Learning Software H2O DeepLearning Simple Structure of networks just specify number of fullyconnected layers (and optionally dropout) Settings for Sparse data can outperform GPU. H2O Deep Water Project uses GPU but no longer being developed; H2O recommends Keras for new projects. Keras Front end for Tensorflow, CNTK, Theano, MXNet Specify complex network topologies Use different types of layers CNN, RNN, Can leverage GPU 14

15 Virtual Demo 15

16 Industrial Plant: Raw Time series Data

17 Industrial Plant: Raw Time series Data

18 Industrial Plant: Raw Time series Data

19 Tag Training, Validation & Test Data Sets 19

20 Variable Selection

21 Model Configuration & Evaluation Validation Error has clear minimum Note Distribution of Reconstruction Error

22 Model Configuration & Evaluation Problems converging Problems Converging

23 Model Configuration & Evaluation

24 Anomalies & Component Signatures

25 Anomalies & Component Signatures

26 Anomalies & Component Signatures

27 Anomalies & Component Signatures Incident not detected on Univariate Chart

28 Identify Incidents Programmatically Reconstruction Error

29 Elbow Plot of Cluster Config

30 Cluster Similar Incidents, View Signatures

31 To Learn & Do More Visit the TIBCO Community Manufacturing Solutions page Download AI & Machine Learning Manufacturing Solutions from the TIBCO Exchange Visit the TIBCO Industry 4.0 page 31

32 Acknowledgments Thanks to Dr. Thomas Hill, Venkata Jagannath, Glenn Hoskins and Nico Rode for their contributions to this work Thanks to Michael O Connell, Steven Hillion and Heleen Snelting for their support and encouragement. 32

33 Thank you! Visit us in Booth 1021 See this and other Manufacturing Demos Learn more about the Technology Contacts: Mike Alperin Manufacturing Industry Consultant David Katz Data Scientist