Multi-Scale Microstructure-Property Modeling of Elastic...
Transcript of Multi-Scale Microstructure-Property Modeling of Elastic...
Multi-Scale Microstructure-Property Modeling of ElasticLocalization Relationships in High Contrast Composites
MURI Team Meeting III
Ruoqian (Rosanne) Liu1, Ankit Agrawal1, Alok Choudhary1
Yuksel Yabansu2, Surya Kalidindi2
1Electrical Engineering and Computer ScienceNorthwestern University
2Materials Science and EngineeringGeorgia Institute of Technology
August 18, 2015
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 1 / 37
A Major Struggling of the Title
A Data Science Modeling
a Multi-Scale Behavior a Typical Material
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 2 / 37
A Major Struggling of the Title
A Data Science Modeling
a Multi-Scale Behavior a Typical Material
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 3 / 37
A Major Struggling of the Title
A Data Science Modeling
a Multi-Scale Behavior a Typical Material
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 4 / 37
A Major Struggling of the Title
A Data Science Modeling
a Multi-Scale Behavior a Typical Material
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 5 / 37
Overview
1 What Happened in the PastThe Problem of Localization ModelingThe Data-Driven Modeling of It
2 The Same Problem with Better ResultsA Two-Scale Modeling SchemeThe Use of 2-Pt Statistics
3 The Expectation of Even Better OutcomesThe What’s Called “Deep Learning”Convolutional Learning of Images
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 6 / 37
What Happened in the Past
Outline
1 What Happened in the PastThe Problem of Localization ModelingThe Data-Driven Modeling of It
2 The Same Problem with Better ResultsA Two-Scale Modeling SchemeThe Use of 2-Pt Statistics
3 The Expectation of Even Better OutcomesThe What’s Called “Deep Learning”Convolutional Learning of Images
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 7 / 37
What Happened in the Past The Problem of Localization Modeling
The Big Picture
Keywords
material informatics; data science; predictive modeling.
Goal
combination of best known physics with data science; computationalefficiency; generalization.
Data
three dimensional (3-D); voxel based microstructure volume element(MVE); high contrast.
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 8 / 37
What Happened in the Past The Problem of Localization Modeling
Physics Based vs. Data Based
Physical Based Models
Generally accomplished by solving governing field equationsnumerically (e.g., finite element models), while satisfying theappropriate material constitutive laws and the imposed boundary andinitial conditions.
Computational resource requirements are usually very high.
No systematic learning from discarded simulations and solutions.
Data Models
Distill transferable knowledge from trials, even failed ones.
Calibration is a one-time computational cost.
Knowledge obtained can be generalized to future, unseen cases.
Dramatic savings in both time and effort.
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 9 / 37
What Happened in the Past The Problem of Localization Modeling
Localization
Localization: the spatial distribution of the response at the microscale foran imposed loading condition at the macroscale.
A microstructure in 3D
A process: impose a constant load of 5×10-‐4
A response of microstructure in 3D
Data Models?
Finite Element
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 10 / 37
What Happened in the Past The Problem of Localization Modeling
A Closer Look at the Data
A collection (2,500) of 3-D MVEs, each of a dimension21× 21× 21, of a digitally created high contrasttwo-phase composite.
Volume fractions vary from 1.0% to 99.4%, in total100 variations.
A corresponding collection of elastic deformation fields.The response field is captured as a continuous value ineach spatial voxel.
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 11 / 37
What Happened in the Past The Data-Driven Modeling of It
Past Attempts of Data Mining
Feature Extrac+on Regression
features Input: microstructure
Output: response
Data-driven predictive modeling: single-agent
Level 1 neighbors Level 2 neighbors
Level 3 neighbors Level 4 neighbors
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 12 / 37
What Happened in the Past The Data-Driven Modeling of It
Results Accomplished with Single Agent Modeling
0 20 40 60 80 100
Volume Fraction (%) of each MVE
0
5
10
15
20
25
30
Indiv
idual M
ASE (
%)
Ex 3a-1: RF + 57 features, e=13.02%
Ex 3a-2: RF + 93 features, e=14.25%
Ex 3b-1: SVR + 57 features, e=16.36%
Ex 3b-2: SVR + 93 features, e=17.01%
Mean absolute strain error (MASE) e = 1N
∑N
∣∣∣ p−ppimposed
∣∣∣× 100%,
N : number of samples; p, p, pimposed: actual, predicted, imposed strain on a sample.
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 13 / 37
What Happened in the Past The Data-Driven Modeling of It
Results Accomplished with Single Agent Modeling
FE Ex 3a-1 Ex 3b-1
Ex 3a-1 Ex 3b-1 Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 14 / 37
The Same Problem with Better Results
Outline
1 What Happened in the PastThe Problem of Localization ModelingThe Data-Driven Modeling of It
2 The Same Problem with Better ResultsA Two-Scale Modeling SchemeThe Use of 2-Pt Statistics
3 The Expectation of Even Better OutcomesThe What’s Called “Deep Learning”Convolutional Learning of Images
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 15 / 37
The Same Problem with Better Results A Two-Scale Modeling Scheme
Why Two-Scale?
The internal structure of a material system is hierarchical.
There are multiple length scales where information passes in between.
In our problem, the response needs to be addressed at micro-scale fora condition applied at macro-scale.
A microstructure
Macro-scale (cube-wise) Information: volume fraction, volume structure …
Micro-scale (voxel-wise) information: neighboring structure …
How to combine?
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 16 / 37
The Same Problem with Better Results A Two-Scale Modeling Scheme
Handling Multiple Scale Lengths in Machine Learning
In machine learning, this amounts to a problem of data representation.
How to represent a voxel in a cube?
Each voxel is represented by, say, its neighboring information.
Higher level information, like volume fraction, provides voxels withdistinct learning environments, which we call contexts.
How to realize, identify, and obtain distinct contexts from a data is achallenge.
How to include the contextual information relates to the field ofrepresentation learning.
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 17 / 37
The Same Problem with Better Results A Two-Scale Modeling Scheme
The Multi-Contextual Solution
Feature Extrac+on Regression
features Input: microstructure
Output: response
Data-driven predictive modeling: single-agent
Macro-‐Feature Extrac.on
Division/Resampling
Macro features Input: microstructure
Output: response
Data-driven predictive modeling
Local-‐feature Extrac.on
…
Regression
Regression Local-‐feature Extrac.on
Local-‐feature Extrac.on Regression
…
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 18 / 37
The Same Problem with Better Results A Two-Scale Modeling Scheme
The Multi-Contextual Solution
Feature Extrac+on Regression
features Input: microstructure
Output: response
Data-driven predictive modeling: single-agent
Macro-‐Feature Extrac.on
Division/Resampling
Macro features Input: microstructure
Output: response
Data-driven predictive modeling
Local-‐feature Extrac.on
…
Regression
Regression Local-‐feature Extrac.on
Local-‐feature Extrac.on Regression
…
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 18 / 37
The Same Problem with Better Results A Two-Scale Modeling Scheme
How to Partition Contexts
A multi-scale modeling solution:
1 Consider the “type of cube” as the context of learning.
Idea
“similar” cubes should be grouped together to create alearning context.
Question
What structural/compositional descriptors differentiatemicrostructure cubes from one another?
Answer
1 Developing lower-order geometric descriptorsNum. of clusters (connected components)Max, min, ave. size of clustersDispersion (ave. of cluster center distances)
2 Use 2-point statistics
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 19 / 37
The Same Problem with Better Results A Two-Scale Modeling Scheme
How to Partition Contexts
A multi-scale modeling solution:1 Consider the “type of cube” as the context of learning.2 Within each context, conduct predictive modeling.
Idea
Take inputs from local neighbors to establishstatistical/mathematical, rule-based relationships.
Question
How many neighbors to include? What neighboringinformation is important? What modeling techniques to use?
Answer
1 Separate neighboring voxels into levels.2 Extract information on each level.3 Rank and select the best set of features.
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 20 / 37
The Same Problem with Better Results The Use of 2-Pt Statistics
Quick Review of 2-Point Statistics
It describes the probability of two phases separated by this particulardistance.It can be thought of as a lumpiness factor – the higher the value forsome distance scale, the more lumpy the universe is at that distancescale.It enables a quantitative understanding of the microstructure-propertyrelationship.
r
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 21 / 37
The Same Problem with Better Results The Use of 2-Pt Statistics
Quick Review of 2-Point Statistics
It describes the probability of two phases separated by this particulardistance.It can be thought of as a lumpiness factor – the higher the value forsome distance scale, the more lumpy the universe is at that distancescale.It enables a quantitative understanding of the microstructure-propertyrelationship.
r = sqrt(10)
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 22 / 37
The Same Problem with Better Results The Use of 2-Pt Statistics
2-Point Statistics in 3D MVE
A reduction from 21× 21× 21 phasevalues to 179 correlation values.
E.g. 100 functions for MVE #5, 30, 55, ..., 2480, each for one unique volume fraction.
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 23 / 37
The Same Problem with Better Results The Use of 2-Pt Statistics
2-Point Statistics in 3D MVE
A reduction from 21× 21× 21 phasevalues to 179 correlation values.
E.g. 25 functions for MVE #1000, 1001, ..., 1024, all with the same volume fraction.
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 24 / 37
The Same Problem with Better Results The Use of 2-Pt Statistics
Results
Contextual partition is conducted with three methods.
P1:Make partition of MVEs based on the volume fraction.Results in 100 groups.
A sample slice shown with VF of 50.22% (one of the hardest cubes)
FEM No PartitionAll test MASE: 17.01
Cube MASE: 25.86
Slice MASE: 27.02
P1All test MASE: 8.89
Cube MASE: 12.32
Slice MASE: 12.13
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 25 / 37
The Same Problem with Better Results The Use of 2-Pt Statistics
Results
Contextual partition is conducted with three methods.
P2:Make partition of MVEs based on data clustering with a set of
selected macro-features.Results in 93 groups.
A sample slice shown with VF of 50.22% (one of the hardest cubes)
FEM No PartitionAll test MASE: 17.01
Cube MASE: 25.86
Slice MASE: 27.02
P2All test MASE: 8.43
Cube MASE: 12.66
Slice MASE: 12.69Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 26 / 37
The Same Problem with Better Results The Use of 2-Pt Statistics
Results
Contextual partition is conducted with three methods.
P3:Make partition of MVEs based on PCA of 2-point correlation
functions.Results in 90 groups.
A sample slice shown with VF of 50.22% (one of the hardest cubes)
FEM No PartitionAll test MASE: 17.01
Cube MASE: 25.86
Slice MASE: 27.02
P3All test MASE: 8.03
Cube MASE: 12.14
Slice MASE: 11.93Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 27 / 37
The Expectation of Even Better Outcomes
Outline
1 What Happened in the PastThe Problem of Localization ModelingThe Data-Driven Modeling of It
2 The Same Problem with Better ResultsA Two-Scale Modeling SchemeThe Use of 2-Pt Statistics
3 The Expectation of Even Better OutcomesThe What’s Called “Deep Learning”Convolutional Learning of Images
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 28 / 37
The Expectation of Even Better Outcomes The What’s Called “Deep Learning”
A Revolution in Machine Learning Society
Over the last few month, the what’s called “deep learning” has producedbreakthrough results in speech, image, and natural language.
MIT Tech Reviews list of top-10 breakthroughs of 2013
improved speech recognition technology by 30%, an earthquake inthis field
caused big companies, such as Microsoft, Facebook, Google, Apple,Baidu Yahoo! and IBM to heavily invest in this technology
the perfect method to exploit the information locked away in Big Data
now fully used GPU power for a huge performance boost
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 29 / 37
The Expectation of Even Better Outcomes The What’s Called “Deep Learning”
Deep Learning
Learning the representation
We know the way in which data are represented can make a hugedifference in the success of a learning algorithm.
Deep learning enables the learning of multiple levels of representation,discovering more abstract features in the higher levels.
Learning as human does
Because human brains appear deep, AI-tasks require deep circuits
Because it is natural for humans to represent concepts at multiplelevels of abstractions, deep architecture makes sense.
Because human learn mostly unsupervised, only partially supervised.
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 30 / 37
The Expectation of Even Better Outcomes The What’s Called “Deep Learning”
Deep Learning: the Basic Recipe
Greedy Layer-Wise Learning of Representations
1 Let h0(x) = x be the lowest-level representation of the data, given bythe observed raw input x.
2 For l = 1 to LTrain an unsupervised learning model taking hl−1(x) at levell − 1 as input, and after training, producing representationshl(x) = Rl(hl−1(x)) at the next level.
Several variants from this point on.
Supervised learning with fine-tuning: most common
Unsupervised: Deep autoencoders or a Deep Boltzmann Machine
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 31 / 37
The Expectation of Even Better Outcomes The What’s Called “Deep Learning”
The Achievements in Image Recognition
Computer vision is where deep learning in 2006 first showed itsbreakthrough. Then, voice, natural language, drug discovery ...
Automatically learning the representation from raw pixels
Using large amounts of data
Learning very complex problems
Mimic human brain representations
Wins all competitions:
IJCNN 2011 Traffic Sign Recognition Competition
ISBI 2012 Segmentation of neuronal structures in EM stacks challenge
ICDAR 2011 Chinese handwriting recognition
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 32 / 37
The Expectation of Even Better Outcomes The What’s Called “Deep Learning”
Deep Learning: the Power of Handling Big Data
Amount of data
Per
form
ance
Deep learning
Most learning algorithms
Reproduced from Andrew Ng’s Invited Talk at RSS2014
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 33 / 37
The Expectation of Even Better Outcomes Convolutional Learning of Images
Convolutional Neural Networks
What makes automatic image learning possible.
A regular 3-layer neural network A ConvNet arranges its neurons in three dimensions
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 34 / 37
The Expectation of Even Better Outcomes Convolutional Learning of Images
Convolutional Neural Networks
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 35 / 37
The Expectation of Even Better Outcomes Convolutional Learning of Images
Design of CNN for MVE Problem
Preliminary design of CNN applied to the MVE problem:
Take a local unit structure and loop over the whole cube.
At each position feed the input microstructure into a series of neurons.
Compute the forward function, remember the error between initialpredicted response and actual FE response.
After looping aggregate the error over every voxel.
Backpropagate the error to update neuron weights.
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 36 / 37
The End
Thank You!Questions and Discussion
Rosanne Liu (Northwestern) Mining of P-S-P August 18, 2015 37 / 37