An Intelligent Machine Monitoring System Using Gaussian...
Transcript of An Intelligent Machine Monitoring System Using Gaussian...
Raunak Bhinge, Nishant Biswas and David DornfeldMechanical EngineeringUC. Berkeley Berkeley, CA, USA
Jinkyoo Park and Kincho H. LawCivil and Environmental EngineeringStanford UniversityStanford, CA, USA
Moneer Helu and Sudarsan RachuriNational Institute of Standards and TechnologyGaithersburg, MD, USA
An Intelligent Machine Monitoring System Using Gaussian Process Regression for Energy Prediction
Industrial: 22% of U.S. use
Mileage for automobiles has greatly increasedWe also understand the energy consumption pattern well
2
Transportation: 28% of U.S. energy use
Industrial: 22% of U.S. energy use
What about manufacturing energy use?
3
Can we predict how much energy the manufacturing machine will consume when machining the part?
4
Designed part NC code Target machine Machined part
Part design NC code Target machine Machined part
Input parameters for the machine
• Feed rate• Spindle speed• Depth of cut
• Cutting direction• Cutting strategy• Dimensions
Output measurements from the machine
• Energy consumption• Part quality, e.g., surface roughness• Machine conditions, e.g., tool wear
XML-based structured data with automatic acquisition
From data to insight: Mapping control parameters to energy consumption
5
Training data set 𝑫
Learning Algorithm
Hypothesisℎ(𝒙)
Predict total energy consumption
𝑫 = 𝒙𝑖 , 𝑦𝑖 ; 𝑖 = 1,… ,𝑚
Using training data set 𝑫, learning algorithm finds the best function ℎ(𝒙) that is believed to accurately predict the output 𝑦 for a given input 𝒙
𝑖=1
𝑚
ℎ 𝒙𝑖
New NC Code
…
6
𝒙 = control parameter 𝑦 = Response
𝑛 control parameters
𝑚NC code
blocks
Procedure for constructing data-driven energy prediction model
• Fanuc Controller : Collect machine control parameters
• System Insights High Speed Power Meter (HSPM):Collect power time series
Hardware Data types
7
Data acquisition & simulator for this study
Step-NC file 1
Step-NC file 2
Step-NC file 18
Level Spindle Speed (RPM)
Chip Load (mm/tooth)
Depth of Cut (mm)
1 1500 0.0254 12 3000 0.0330 1.53 4500 0.0432 34 6000 0.0508
Condensed and contextualized data<Input features – output response>
8
First, one must acquire the training data set
Input:𝑥1 ∈ ℝ Feed rate𝑥2 ∈ ℝ Spindle speed𝑥3 ∈ ℝ Depth of cut𝑥4 ∈ 1, 2, 3, 4 Active tool axis ID (1 for 𝑥-axis, 2 for 𝑦-axis, 3 for 𝑧-axis and 4 for 𝑥-y direction)𝑥5 ∈ 1, 2, 3 Cutting strategy (1 for conventional, 2 for climbing and 3 for both)
Output:𝑦 = 𝐸/𝑙 ∈ ℝ Energy density (energy consumption per unit length of a tool path) in NC code block.
In total, 3,092 pairs of 𝒙 (machine operation feature vector) and 𝑦 (energy density) collected from the experiments.
That is, 𝒙𝑖 , 𝑦𝑖 |𝑖 = 1, … , 3,092 serve as the basis for this study.
𝑙 ∈ ℝ Length of tool a tool path in NC code block
9
Description of training data set
𝑥1 𝑥2 ⋯ 𝑥𝑛 𝑦
𝑥11 𝑥2
1 ⋯ 𝑥𝑛1 𝑦1
𝑥12 𝑥2
2 ⋯ 𝑥𝑛2 𝑦2
⋮ ⋮ ⋮ ⋮
𝑥1𝑚 𝑥2
𝑚 ⋯ 𝑥𝑛𝑚 𝑦𝑚
𝑛 machine parameters𝑫1 = 𝒙𝑖 , 𝑦𝑖 |𝑖 = 1, … ,𝑚1 𝑦 = 𝑓1 𝒙
Clustered input and output datafor each machine operation
Energy Prediction models for machine operations
…
Training data
Face milling
Contouring
𝑫2 = 𝒙𝑖 , 𝑦𝑖 |𝑖 = 1, … ,𝑚2
𝑫𝑞 = 𝒙𝑖 , 𝑦𝑖 |𝑖 = 1, … ,𝑚𝑞
𝑫𝑄 = 𝒙𝑖 , 𝑦𝑖 |𝑖 = 1, … ,𝑚𝑄
…
𝑦 = 𝑓2 𝒙
𝑦 = 𝑓𝑞 𝒙
𝑦 = 𝑓𝑄 𝒙
• Cutting operations : Face milling, Contouring, Slotting, Pocketing, Spiraling, Drilling • Non-cutting operations: Air-cut in 𝑥 − 𝑦 direction, Air-cut in 𝑧 direction, Rapid motion
10
Classifying training data set by machine operations
𝑫𝑞 = 𝒙𝑖 , 𝑦𝑖 |𝑖 = 1, … ,𝑚𝑞 𝑦 = 𝑓𝑞 𝒙
How to construct prediction function?
Gaussian Process • model complex input and output relationships without the basis functions• update model with new measurement data based on Bayesian framework• estimate uncertainty in prediction
12
Which learning algorithm to choose ? : Gaussian Process (GP) regression
𝒚1:𝑚𝑞
𝑦~𝑁 𝟎,
𝐊 𝒌𝒌𝑇 𝑘(𝒙, 𝒙)
𝑫𝑞 = 𝒙𝑖 , 𝑦𝑖 ; 𝑖 = 1,… ,𝑚𝑞
𝑦 = 𝑓𝑞 𝒙 + 𝜖, 𝜖~ 𝑁(0, 𝜎𝑒𝑟𝑟𝑜𝑟2 )
𝑦 |𝑫𝒒~𝑁 𝜇 𝒙|𝑫𝒒 , 𝜎2 𝒙|𝑫𝒒
𝜇 𝒙|𝑫𝒒 = 𝒌𝑇𝐊−1𝒚1:𝑚𝑞
𝜎2 𝒙|𝑫𝒒 = 𝑘 𝒙, 𝒙 − 𝒌𝑇𝐊−1𝒌𝑘 𝒙𝑖 , 𝒙𝑗 = 𝜏2exp −
1
2𝒙𝑖 − 𝒙𝑗 𝑇
diag 𝝀 −2 𝒙𝑖 − 𝒙𝑗 + 𝜎𝜖2𝛿𝑖𝑗
𝒌𝑇 = 𝑘 𝒙1, , 𝒙 , . . . , 𝑘 𝒙𝑚𝑞 , 𝒙
𝐊𝑖𝑗 = 𝑘 𝒙𝑖 , 𝒙𝑗
Conditionalizationon observed data
(Bayesian updating)
Given training data set for 𝑞th operation
Assumption : measurements are corrupted with noise
Prior on the measured outputs
13
Power of GP comes from Bayesian framework
𝑦1
𝑦2
𝑦
~𝑁 𝟎,𝒌
𝒌𝑇 𝑘(𝒙 , 𝒙 )𝑛 = 2
True 𝑓 𝑥 = 𝑥sin(𝑥)
Sampled without error
14
How GP constructs regression model from data?
𝑦1
𝑦2
𝑦3
𝑦
~𝑁 𝟎,𝒌
𝒌𝑇 𝑘(𝒙 , 𝒙 )
𝑛 = 3
True 𝑓 𝑥 = 𝑥sin(𝑥)
Sampled without error
15
How GP constructs regression model from data?
𝑦1
𝑦2
𝑦3
𝑦4
𝑦
~𝑁 𝟎,𝒌
𝒌𝑇 𝑘(𝒙 , 𝒙 )
𝑛 = 4
True 𝑓 𝑥 = 𝑥sin(𝑥)
Sampled without error
Gaussian Process (simple example)
16
𝑦1
𝑦2
𝑦3
𝑦4
𝑦5
𝑦
~𝑁 𝟎,𝒌
𝒌𝑇 𝑘(𝒙 , 𝒙 )
𝑛 = 5
True 𝑓 𝑥 = 𝑥sin(𝑥)
Sampled without error
17
How GP constructs regression model from data?
𝑦1
𝑦2
𝑦3
𝑦4
𝑦5
𝑦6
𝑦
~𝑁 𝟎,𝒌
𝒌𝑇 𝑘(𝒙 , 𝒙 )
𝑛 = 6
True 𝑓 𝑥 = 𝑥sin(𝑥)
Sampled without error
18
How GP constructs regression model from data?
True 𝑓 𝑥 = 𝑥sin(𝑥)
Sampled with error:
𝑓 𝑥 = 𝑥sin 𝑥 + 𝜖,𝜖 ~ 𝑁(0, 12)
𝑦1
𝑦2
𝑦
~𝑁 𝟎,𝒌
𝒌𝑇 𝑘(𝒙 , 𝒙 )𝑛 = 2
19
How GP constructs regression model from data?
True 𝑓 𝑥 = 𝑥sin(𝑥)
Sampled with error:
𝑓 𝑥 = 𝑥sin 𝑥 + 𝜖,𝜖 ~ 𝑁(0, 12)
𝑦1
𝑦2
𝑦3
𝑦
~𝑁 𝟎,𝒌
𝒌𝑇 𝑘(𝒙 , 𝒙 )
𝑛 = 3
20
How GP constructs regression model from data?
True 𝑓 𝑥 = 𝑥sin(𝑥)
Sampled with error:
𝑓 𝑥 = 𝑥sin 𝑥 + 𝜖,𝜖 ~ 𝑁(0, 12)
𝑦1
𝑦2
𝑦3
𝑦4
𝑦
~𝑁 𝟎,𝒌
𝒌𝑇 𝑘(𝒙 , 𝒙 )
𝑛 = 4
21
How GP constructs regression model from data?
True 𝑓 𝑥 = 𝑥sin(𝑥)
Sampled with error:
𝑓 𝑥 = 𝑥sin 𝑥 + 𝜖,𝜖 ~ 𝑁(0, 12)
𝑦1
𝑦2
𝑦3
𝑦4
𝑦5
𝑦
~𝑁 𝟎,𝒌
𝒌𝑇 𝑘(𝒙 , 𝒙 )
𝑛 = 5
22
How GP constructs regression model from data?
True 𝑓 𝑥 = 𝑥sin(𝑥)
Sampled with error:
𝑓 𝑥 = 𝑥sin 𝑥 + 𝜖,𝜖 ~ 𝑁(0, 12)
𝑦1
𝑦2
𝑦3
𝑦4
𝑦5
𝑦6
𝑦
~𝑁 𝟎,𝒌
𝒌𝑇 𝑘(𝒙 , 𝒙 )
𝑛 = 6
23
How GP constructs regression model from data?
𝑦 = 𝑓1 𝒙𝜇 𝒙|𝑫1mean function
𝜎 𝒙|𝑫1Standard deviation
Prediction can be represented with bound:
𝜇 𝒙|𝑫1 − 𝜎 𝒙|𝑫1 , 𝜇 𝒙|𝑫1 + 𝜎 𝒙|𝑫1
(5D function)
(5D function)
𝑥3
Depthof cut(mm)
𝑥2
SpindleSpeed(RPM)
Energy density prediction functionfor face milling
24
Constructed energy density prediction function
𝜇 𝒙|𝑫𝑞
𝑦 = 𝑓𝑞 𝒙
𝜎 𝒙|𝑫𝑞mean functionstandard deviation
function
𝐸𝑖 = 𝜇𝑞 𝒙𝑖|𝑫𝑞 × 𝑙𝑖 𝑆𝑖 = 𝜎𝑞(𝒙𝑖|𝑫𝑞) × 𝑙𝑖
• Energy consumption for 𝑖th NC code block performing machine operation type 𝑞 :
𝐸𝑞 = 𝒙𝑖,𝑦𝑖 ∈𝑫𝑞
𝜇𝑞 𝒙𝑖|𝑫𝑞 × 𝑙𝑖 𝑆𝑞 = 𝒙𝑖,𝑦𝑖 ∈𝑫𝑞
𝜎𝑞(𝒙𝑖|𝑫𝑞) × 𝑙𝑖
2
• Energy consumption for NC code blocks performing machine operation type 𝑞:
𝐸 = 𝑞=1
𝑄 𝐸𝑞 𝑆 =
𝑞=1
𝑄
𝑆𝑞2
• Energy consumption for the entire operations:
: Probabilistic prediction𝐸~𝑁( 𝐸, 𝑆)
• Energy density prediction model for machine operation type 𝑞
25
From density prediction to energy prediction
Used spindle speeds (RPM)Training parts 1~18 1,500, 3,000, 4,500, 6,000Test part 1 1,500, 3,000, 4,500Test part 2 1,700, 2,800, 4,300Test part 3 2,125, 2,400, 3,750
We test whether the prediction model prediction energy consumptions for machining parts• with different geometry • with different machine control parameters (in this case study, varying spindle speeds)
?
Energy consumption
𝐸~𝑁( 𝐸, 𝑆)
Level of generalization
26
Can we predict how much energy the manufacturing machine will consume when machining the part? YES
Energy density prediction for face milling 𝑦 = 𝑓1 𝒙
27
Prediction results for test parts
Energy consumptions for each NC code blockTotal Energy consumption
Error rates
No. of data
Averagedblock
duration (sec)
RAE(%)
Predictedtotal energy
(KJ)
Measured total energy
(KJ)
Standard deviation
(KJ)
RTE(%)
Test 1 188 10.27 13.004 22.492 21.689 0.434 3.702
Test 2 188 9.82 15.210 21.928 21.864 0.441 0.290
Test 3 188 9.70 23.143 21.747 21.074 0.477 3.192
𝐸~𝑁( 𝐸, 𝑆)
28
Prediction results for test parts
• RAE = {𝑖∈ NC blocks}
𝐸𝑖−𝐸𝑖
{𝑖∈ NC blocks } 𝐸𝑖
• RTE =𝐸− 𝐸
𝐸
29
Future work and Conclusion
TargetMachine Tool
External sensors
MTConnectAgent
DataProcessor
DataBuffer
KnowledgeExtraction Agent
KnowledgeArchive
PastExperience
Process Parameters
OptimizationProcess Parameters
Real time data acquisition and processor
AdaptiveMachine learning
Real time data acquisition
Adaptive machine learning
Continuous machine monitoring and control without saving big data
• The authors acknowledge the support in part of the Smart Manufacturing Systems Design and Analysis Program at the National Institute of Standards and Technology (NIST), Grant Numbers 70NANB12H225 and 70NANB12H273 awarded to University of California, Berkeley, and to Stanford University respectively. In addition, the authors appreciate the support of the Machine Tool Technologies Research Foundation (MTTRF) and System Insights for the equipment used in this research.
• Certain commercial systems are identified in this paper. Such identification does not imply recommendation or endorsement by the National Institute of Standards and Technology (NIST); nor does it imply that the products identified are necessarily the best available for the purpose. Further, any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of NIST or any other supporting U.S. government or corporate organizations.
30
Acknowledgements and disclaimer
Supplementary slides
31
Requirements:
• The amount of data storage should be minimized while maximizing the knowledge extraction
The extracted knowledge should be updated with new measurement data to account for the time varying characteristics of a target machine, i.e., tool wear or aging.
Time
Knowledge
Stored data• The amount of monitoring data increases with time.
• The extractable knowledge is not necessarily proportional to the amount of data.
• Constructing GP model is computationally expensiveO 𝑚3 , 𝑚 is number of data points.
Issues:
Time
Knowledge
Stored data
Limitations in GP
32
http://www.dailymail.co.uk/sciencetech/article-2020546
• Different types of knowledge are processed and retained by different region of brain.
• The knowledge is updated with new information, i.e., outdated knowledge is replaced with new one.
Online collective GP is composed of
• 𝑝𝑖(𝒙): input feature distribution constructed by Gaussian Mixture Model using local data set. (domain of knowledge).
• ℎ𝑖(𝒙): regression model mapping input to output constructed by GPusing local data set.(content of knowledge).
Collective Gaussian Process
33
Given 𝑁 pairs of energy prediction function and input features’ PDF, { ℎ𝑖 𝒙 , 𝑝𝑖(𝒙) |𝑖 =
𝑦𝑛𝑒𝑤 = ℎ 𝒙𝑛𝑒𝑤 =
𝑖
𝑁
𝑤𝑖 ℎ𝑖 𝒙𝑛𝑒𝑤
𝑤𝑖 =𝑝𝑖 𝒙𝑛𝑒𝑤
𝑗𝑁 𝑝𝑗 𝒙𝑛𝑒𝑤
where the weighting coefficient 𝑤𝑖 for the 𝑖th local energy prediction function is determined as
Prediction by local GPs
Global GP Collective GP
Collective Gaussian Process
34
𝑝 𝒙;𝝋, 𝝁, 𝚺 =
𝑘=1
𝐾
𝑔𝑘(𝒙; 𝝁𝑘 , 𝚺𝑘)𝜑𝑘
𝑔𝑘 𝒙; 𝝁𝑘 , 𝚺𝑘 = 𝑁 𝒙; 𝝁𝑘 , 𝚺𝑘 =1
(2𝜋)𝑛|𝚺𝑘|exp −
1
2𝒙 − 𝝁𝑘
𝑇𝚺𝑘−1 𝒙 − 𝝁𝑘
Modeling a probability density function as a combination of 𝐾 Gaussian components
(Ref: F. Wood, NIPS 1999)
For an feature vector 𝒙 = (𝑥1, 𝑥2, … , 𝑥𝑛), the 𝑘th component density is of a form of Gaussian
𝜑𝑘: (mixture) weight for 𝑘th Gaussian component ( 𝑘=1𝐾 𝜑𝑘 = 1)
𝝁𝑘: mean vector for the 𝑘th Gaussian component𝚺𝑘: covariance matrix for the 𝑘th PDF
𝐾: number of GPDFs𝝋 = 𝜑1, … , 𝜑𝐾 : set of weights𝝁 = 𝝁1, … , 𝝁𝐾 : set of mean vectors 𝚺 = {𝚺1, … , 𝚺𝐾}: set of covariance matrices
Gaussian Mixture Model (GMM)
35
Simple example
Weighted sum of Gaussian PDFs
𝑝 𝒙 =
𝑘=1
𝐾
𝑔𝑘(𝒙)𝜑𝑘
𝑔1(𝒙) 𝑔2(𝒙) 𝑔3(𝒙)
Gaussian Mixture Model (GMM)
36
collective GP
Error histogramEnergy prediction
Global GP
Global GP vs Collective GP
37
Accuracy Computational time RAE (%) RTE (%) Learning (sec) Testing (sec)
Collective GP 15.45 -0.41 102.88 0.20Global GP 17.81 0.72 8.87 0.21
Weighting methods RAE (%) RTE (%)Probability by GMM 17.81 0.72
Variance by GP 18.40 0.14Geometric distance to center 42.41 24.02
Average 48.47 39.50
The comparison of prediction accuracies between the collective GP regression and other local GP regression methods
Numerical results
38
Part design NC code Target machine Machined part
Prediction with blind test parts
𝑓𝑞 𝒙 |𝑞 = 1,… , 𝑄
Simulator
Energy consumption
𝐸~𝑁( 𝐸, 𝑆)
39
Energy prediction on test data set randomly selected from training data set
40
Data acquisition & simulator for this study
• Fanuc Controller : Collect machine control parameters
• System Insights High Speed Power Meter (HSPM):Collect power time series
• Direct data : Directly measured from hardware
• Derived data: Derived through simple calculations, e.g., mean and tool path computation.
• Simulated data : Obtained by simulation, accounting for workpiece geometry
Hardware Data types
41