department of mathematics and computer science 1212 1 6BV04 DOE: Optimization Response Surface...
-
Upload
antony-ferring -
Category
Documents
-
view
222 -
download
1
Transcript of department of mathematics and computer science 1212 1 6BV04 DOE: Optimization Response Surface...
department of mathematics and computer science
1
6BV04DOE: Optimization
Response Surface Methods
department of mathematics and computer science
2
Contents• Optimisation steps• Box method• Steepest ascent method• Practical example• Response surface designs• Multiple responses• EVOP• Software• Literature
department of mathematics and computer science
3
Optimisation stepsOptimisation is achieved by going through the following phases:
• screening (determine which factors really influence the outcome; tool: screening designs like fractional factorial)• improvement (approach optimum by repeated change of factor settings; tools: Box/simplex or steepest ascent approach)• determination of optimum (find optimal settings of factor settings; tool: response surface designs like CCD or Box-Behnken + analysis of response surface using eigenvalues)
department of mathematics and computer science
4
current settings
improvement
optimum
department of mathematics and computer science
5
Regression models used in optimisationStatistical techniques for optimisation assume the following (often reasonably satisfied in practice):“Far away” from the optimum a first order model often suffices. for example:
Y = ß0 + ß1x1 + ß2x2 +
“Near” the optimum often a quadratic (second order) model suffices. For example:
Y = ß0 + ß1x1 + ß2x2 + ß12x1x2 + ß11x12 + ß22x2
2 +
Lack-of-fit techniques must be applied in order to check whether these models are appropriate, since we cannot directly see whether we are near the optimum (cf. next slides).
department of mathematics and computer science
6
Models
Far away from optimum:first order
model
department of mathematics and computer science
7
Models
Near optimum:fitting a first order model
shows lack-of-fit
(curvature)
department of mathematics and computer science
8
Models
Near optimum:second order
model
department of mathematics and computer science
9
ImprovementIn order to efficiently move from current factor settings to factor setting that yield near-optimal values, 2 methods are available:• Box/Simplex method
– idea: form new full factorials in direction of largest increase in current full factorial
– simple; no statistics needed for implementation– not efficient
• Steepest ascent/descent method – idea: use 1st order regression model from fractional factorial
to obtain direction of largest increase (“steepest ascent”)– perform single runs in direction of largest increase until
increase stops– advanced– recommended since it is more efficient
department of mathematics and computer science
10
Box method
40.6 41.9
41.8
41.2
41.3
39.3 40.9
41.5 40.0
direction of largest increase
direction of largest increase
stop if one has to return to
previous settings
department of mathematics and computer science
11
Steepest ascent method
direction of steepest ascent
contour lines of first-order model
perpendicularto contour line
region where 1eorder-model
has been determined
department of mathematics and computer science
12
start
screening
RSM design(CCD, ...)
single observationin direction
steepest ascent
full factorial+ centre points
1st ordermodel OK?
better observation?
stationary point
optimum?
stationary point
nearby?
go to stationary point
yes
no
yes
no
yes
no
no
acceptstationary point
end
fit 2nd order model
yes
Optimization scheme
department of mathematics and computer science
13
Practical examplegoal: maximise yield of chemical reactor
significant factors obtained after screening experiment: • reaction time • reaction temperature
current factor setting: time = 35 min. temp = 155 °Ccurrent yield: 40 %
department of mathematics and computer science
14
Steepest ascent22-design with 5 centre points:time: 30 - 40 min; temp: 150 - 160 °C
results: montgomery14-1.sfx
• there is no significant interaction• there is no significant lack-of-fit• the regression model is significant
Hence, we are not near the optimum.
department of mathematics and computer science
15
Steepest ascent pathoutcome analysis of measurement:yield = 24.94 + 0.155*time + 0.065*temp
with coding: x1= (time-35)/5 x2 = (temp-155)/5
yield = 40.44 + 0.775*x1 + 0.325*x2
0.775
0.325
direction path: normal vector
step size: 5 min reaction time (choice of chemical engineer!)
5 0.3250.42
5 0.775 coded step size temp (= 2.1°C)
department of mathematics and computer science
16
Steepest ascent path experiments
40
50
60
70
80
90
0 2 4 6 8 10 12
yiel
d
Further experiments with factor settings of experiment nr. 10.
department of mathematics and computer science
17
Near the optimumSettings experiment 10:• time = 85 min• temperature = 175 °C
A 22 design with 5 centre points is executed.
results: montgomery14-4.sfx
Lack-of-fit indicates curvature. Hence, we now are probably near the optimum.
department of mathematics and computer science
18
Quadratic modelsIn order to fit a quadratic model (suitable when we are near the optimum), we must vary the factors at 3 levels. A 2p-design with centre points does not suffice, because then all quadratic factors are confounded.
A 3p-design is possible, but not to be recommended:• number of runs grows fast • uses more runs than necessary to fit quadratic model.
department of mathematics and computer science
19
Response surface designsThe following designs are widely used for fitting a quadratic model:• Central Composite Design (uniform precision of effect estimates)
• Box-Behnken Design (almost uniform precision of effect estimates, but usually fewer runs required than for CCD)
The choice between these models is usually decided by the availability of these designs for a given number of runs and number of factors.Note that there are other suitable designs (usually available in statistical software that supports DOE).
department of mathematics and computer science
20
Central Composite Design
A CCD consists of 3 parts:
• factorial points
• axial points
• centre points
A CCD is often executed by adding points to an already performed 2p-design (highly efficient, but beware of blocking!).
department of mathematics and computer science
21
RotatabilityIn a CCD there are 2 possible choices:• number of centre points• location axial points
By choosing the axial points at the locations (,0,…,0) etc.
with = (# factorial points)¼ , the design becomes
rotatable, i.e. the precision (variance) of the model
depends on the distance to the origin only. In other words,
one has the same precision for all factor estimates.
department of mathematics and computer science
22
Box-Behnken designsThese are designs that
consists of combinations from 2p-designs.
Properties:• efficient (few runs)• (almost) rotatable• no corner points of
hypercube (these are extreme conditions which are often hard to set)
department of mathematics and computer science
23
Stationary point
Near the optimum usually a quadratic model suffices:
jiji iji
k
iiii
k
ii xxxxY 2
110
How do find the optimum after we correctly estimated the parameters using a response surface design (CCD or Box-Behnken)?The next slides show the tools to derive optimal settings and the pitfalls that have to be avoided.
department of mathematics and computer science
24
Recap: optimisation in dimension1• necessary condition for extremum: 1st derivative = 0• not sufficient:
“point of inflection”
• extra sufficient condition: 2nd derivative 0
department of mathematics and computer science
25
Zero first derivatives: saddlepoint vs. maximum
saddle point (unfavourable) maximum (favourable)
2 2x y 2 2x y
department of mathematics and computer science
26
Determination of type of optimum • Graphically: make contourplot (if 2
factors)• Analytically:
jiji iji
k
iiii
k
ii xxxxY
2
110
matrix notation: xBxbxY TT
0
Note: B must be chosenas symmetric matrix, see example:
11 12 13
12 22 23
13 23 33
/ 2 / 2
/ 2 / 2
/ 2 / 2
department of mathematics and computer science
27
Stationarity and matrix analysisstationary point (zero first-order derivatives):
bBxBxbx
Y 10 2
102
characterisation through eigenvalues of matrix B:
xBx • all eigenvalues positive: min • all eigenvalues negative: max• eigenvalues different signs: saddle point
(the ’s are sometimes called “parameters of canonical form”)
department of mathematics and computer science
28
Stationarity and matrix analysis
In StatGraphics: • augment design• add star points
Please note that additional centre points are addedand a block variable.
We can remove the centre points from the data setand ignore the block variable in the analysis.
StatGraphics results: montgomery14-6.sfx
department of mathematics and computer science
29
Stationarity and matrix analysisUse Matlab to avoid manual computations: • create matrix B and vector b• compute eigenvalues and location of stationary point
>> B = [-2.75247 0.5/2 ; 0.5/2 -2.00253] / 2Analysis Summary----------------File name: D:\MyDocs\2DS01\collegesheets\montgomery14-6.sfxComment: Montgomery table 14-4
Estimated effects for opbrengst----------------------------------------------------------------------average = 79.94 +/- 0.11896A:tijd = 1.98994 +/- 0.188092B:temperatuur = 1.03033 +/- 0.188093AA = -2.75247 +/- 0.201705AB = 0.5 +/- 0.266003BB = -2.00253 +/- 0.20171----------------------------------------------------------------------Standard errors are based on total error with 7 d.f.
department of mathematics and computer science
30
Stationarity and matrix analysis
In Matlab: • create matrix B and vector b• compute eigenvalues and location of stationary point
>> B = [-2.75247 0.5/2 ; 0.5/2 -2.00253] / 2
B =
-1.3762 0.1250 0.1250 -1.0013
department of mathematics and computer science
31
Stationarity and matrix analysis
In Matlab: • create matrix B and vector b• compute eigenvalues and location of stationary point
>> eig(B)
department of mathematics and computer science
32
Stationarity and matrix analysis
In Matlab: • create matrix B and vector b• compute eigenvalues and location of stationary point
>> eig(B)
ans =
-1.4141 -0.9634
both negative → maximum
department of mathematics and computer science
33
Stationarity and matrix analysis
In Matlab: • create matrix B and vector b• compute eigenvalues and location of stationary point
>> b = [1.98994 ; 1.03033] /2Analysis Summary----------------File name: D:\MyDocs\2DS01\collegesheets\montgomery14-6.sfxComment: Montgomery table 14-4
Estimated effects for opbrengst----------------------------------------------------------------------average = 79.94 +/- 0.11896A:tijd = 1.98994 +/- 0.188092B:temperatuur = 1.03033 +/- 0.188093AA = -2.75247 +/- 0.201705AB = 0.5 +/- 0.266003BB = -2.00253 +/- 0.20171----------------------------------------------------------------------Standard errors are based on total error with 7 d.f.
department of mathematics and computer science
34
Stationarity and matrix analysis
In Matlab: • create matrix B and vector b• compute eigenvalues and location of stationary point
>> b = [1.98994 ; 1.03033] /2
b =
0.9950 0.5152
department of mathematics and computer science
35
Stationarity and matrix analysis
In Matlab: • create matrix B and vector b• compute eigenvalues and location of stationary point
>> spcoded = -0.5 * inv(B) * b
department of mathematics and computer science
36
Stationarity and matrix analysis
In Matlab: • create matrix B and vector b• compute eigenvalues and location of stationary point
>> spcoded = -0.5 * inv(B) * b
spcoded =
0.3893 0.3059
< 1.414 (distance star point) → inside experimental region
department of mathematics and computer science
37
In Matlab: • create matrix B and vector b• compute eigenvalues and location of stationary point
>> sporiginal = spcoded .* [5 ; 5] + [85 ; 175]
1 2
1 2
85 175,
5 55 85, 5 175
time tempx x
time x temp x
Stationarity and matrix analysis
department of mathematics and computer science
38
Stationarity and matrix analysis
In Matlab: • create matrix B and vector b• compute eigenvalues and location of stationary point
>> sporiginal = spcoded .* [5 ; 5] + [85 ; 175]
sporiginal =
86.9463 176.5293
department of mathematics and computer science
39
start
screening
RSM design(CCD, ...)
single observationin direction
steepest ascent
full factorial+ centre points
1st ordermodel OK?
better observation?
stationary point
optimum?
stationary point
nearby?
go to stationary point
yes
no
yes
no
yes
no
no
acceptstationary point
end
fit 2nd order model
yes
Optimization scheme
department of mathematics and computer science
40
Multiple responsesIf more than 1 response variable needs to be optimised, then a graphical way of optimising may be achieved by overlaying contour plots in case there are only 2 independent variables.
Overlay Plot
molecular_weightopbrengstviscositeit
tijd
tem
pe
ratu
ur
80 82 84 86 88 90170
172
174
176
178
180
department of mathematics and computer science
41
Evolutionary Operation (EVOP)
Optimisation of a running production process is not always possible or may not be allowed because of costs:• involves interruption• may (temporarily) yield low quality products
An alternative is Evolutionary Operation:• experimentation within running operation • frequent execution of 2k-designs, starting at current settings• high and low setting of factors are close to each other, thus no risk of low quality products
department of mathematics and computer science
42
Software• StatLab optimisation:
http://www.win.tue.nl/statlabInteractive software for teaching DOE through cases
• Box: http://www.win.tue.nl/~marko/box/box.html Game-like demonstration of Box method
• Matlab virtual reactor: Statistics toolbox -> Demos -> Empirical Modeling -> RSM demo
• Statgraphics: menu choice Special -> Experimental Design– design experiment with pre-defined catalogue– analysis of experiments with ANOVA
department of mathematics and computer science
43
Literature• J. Trygg and S. Wold. Introduction to Experimental
Design – What is it? Why and Where is it Useful?, Homepage of Chemometrics, editorial August 2002: http://www.acc.umu.se/~tnkjtg/Chemometrics/editorial/aug2002.html
• DOE booklet from Umetrics: http://www.umetrics.com/pdfs/books/DOEbook.pdf
• Introduction to DOE from moresteam.com http://www.moresteam.com/toolbox/t408.cfm
• StatSoft Electronic Statistics Textbook, chapter on experimental design
• NIST Engineering Statistics Handbook: http://www.itl.nist.gov/div898/handbook/