Myopic Policies for Budgeted Optimization with Constrained Experiments Javad Azimi, Xiaoli Fern,...
-
Upload
elvin-ryan -
Category
Documents
-
view
216 -
download
0
Transcript of Myopic Policies for Budgeted Optimization with Constrained Experiments Javad Azimi, Xiaoli Fern,...
1
Myopic Policies for Budgeted Optimization
with Constrained Experiments
Javad Azimi, Xiaoli Fern, Alan Fern
Oregon State University
AAAI, July 2010
2
Motivation: Electricity Production in a Microbial Fuel Cell (MFC)
AnodeCathode
bac
teri
aOxidation products
(CO2)
Fuel (organic matter)
e-
e-
O2
H2OH+
This is how an MFC works
SEM image of bacteria sp. on Ni nanoparticle enhanced carbon fibers.
Nano-structure of anode significantly impact the electricity production.
We should optimize anode nano-structure to maximize power.
3
Experiment Selection• Experiments are costly and there is a fixed budget.• How to select the best sequence of experiments.
Current Experiments Scientist selects Experiment
Run Experiment
Bayesian Optimization (BO)• Since Running experiment is very expensive we use BO.
• Select one experiment to run at a time based on results of previous experiments.Current Experiments Gaussian Process Surface Select Single Experiment
Run Experiment 4
5
Bayesian Optimization (BO)
• BO assumes that we can ask for specific experiment.• This is unreasonable assumption in many applications.
– In Fuel Cell it takes many trials to create a nano-structure with specific requested properties.
– Costly to fulfill.Space of Experiments
Average Circularity
Ave
rage
d A
rea
6
Constrained Experiments• It is less costly to fulfill a request that specifies ranges for the
nanostructure properties
• E.g. run an experiment with Averaged Area in range r1 and Average Circularity in range r2
• We will call such requests “constrained experiments”Space of Experiments
Average Circularity
Ave
rage
d A
rea
Constrained Experiment 1• large ranges • low cost• high uncertainty about which experiment will be run
Constrained Experiment 2• small ranges• high cost• low uncertainty about which experiment will be run
7
BO for Constrained Experiment
• Given a fixed budget, select the best constrained experiments.
Run Experiment
Current Experiments Gaussian Process Surface Select Single ExperimentSelect Constrained Experiment
8
Constrained Experiment• We generalized BO heuristics to constrained
experiments.
• Two challenges:– How to compute heuristics for constrained experiment?– How to take experimental cost into account?(which has
been ignored by most of the approaches in BO).
9
Standard BO Heuristics• Standard heuristics are statistics of the posterior p(y|x,D) where D is our
current observation.
• Maximum Upper bound Interval (MUI)– Select point with highest 95% upper confidence bound– Purely explorative approach.
• Maximum Probability of Improvement (MPI)– It computes the probability that the output is more than (1+m) times of the best
current observation , m>0. – Explorative and Exploitative.
• Maximum Expected of Improvement (MEI)– Similar to MPI but parameter free– It simply computes the expected amount of improvement after sampling at any
point.
10
Generalizing BO for Constrained Experiment
• Having the posterior distribution of p(y|x,D) and px(.|D) we can calculate the posterior of the output of each constrained experiment which has a closed form solution.
• Therefore we can compute standard BO heuristics for constrained experiments.– There are closed form solution for these heuristics.
Input spaceDiscretization
Level
11
Budgeted Constrained Experiments
• We are limited with Budget B.• Unfortunately heuristics will typically select the smallest and most
costly constrained experiments which is not a good use of budget.
• How can we consider the cost of each constrained experiment in making the decision?– Cost Normalized Policy (CN)– Constraint Minimum Cost Policy(CMC)
-Low uncertainty
-High uncertainty
-Better heuristic value
-Lower heuristic value
-Expensive -Cheap
12
Cost Normalized Policy
• It selects the constrained experiment achieving the highest expected improvement per unit cost.
• We report this approach for MEI policy only.
13
Constraint Minimum Cost Policy (CMC)
• Motivation:1. Approximately maximizes the heuristic value.2. Has expected improvement at least as great as spending
the same amount of budget on random experiments.• Example:
Very expensive: 10 random experiments likely to be better
Selected Constrained experiment
Poor heuristic value: not select due to 1st
condition
Cost=4 random Cost=10 random Cost=5 random
14
Experimental Results(Setup)
• Gaussian process is used as our model with squared exponential kernel.
• Cost function is defined as:
– There is a constant cost for running any constrained experiment plus an additional cost depending on the size of the experiment.
– The value of slope dictates how fast the cost increases as the size of a constrained experiment decreases.
Space of Experiments
r1
r2
15
Experimental Results(Real Applications)
• Two Real data sets:– Fuel Cell:• Fuel Cell electricity generation
– Hydrogene:• Biosolar hydrogen production
16
Experimental Results(Benchmarks Functions)
• 3 popular benchmarks used in BO literature.
Rosenbrock Discontinuous Cosines
17
Overall Performance• The normalized regret of each framework is shown which is
calculated as y*- ymax over Random performance for budget 15 where y* is the highest possible output.
Average regret of each approach over all frameworks.
Different Budget(1)
18
Random
Cosines
Fuel CellReal
Rosenbrock
CMC-MUI
Different Budget(2)
19
CN-MEI
Cosines
Fuel CellReal
Rosenbrock
Different Budget(3)
20
CMC-MPI(0.2)
Cosines
Fuel CellReal
Rosenbrock
21
Different Budget(4)
CMC-MEICosines
Fuel CellReal
Rosenbrock
22
Conclusion
• We introduced a new constrained experiment framework which asks for hyper-rectangle rather than exact point.
• We extended model-free BO heuristics to our frame work.
• We introduced two approaches to optimize our budgeted framework.
• CMC-MEI is working better than other approaches.
23
Thanks for attendance
Question?