Population Based Incremental Learning

Shumeet Baluja

Presented by KC Tsui

Background

• Populations based search, such as GA– Create a probability matrix by counting the number of 1s

and 0s in each gene position– Generate new population using the probability matrix– No information is carried from generation to generation!

• Supervised Competitive learning, e.g. LVQ– Winner-take-all reinforcement learning in ANN– Winner is a kind of prototype of the sample presented

• PBIL = GA + CL– Capture the trend from the best performer

Basic PBIL

P initialize probability vector (each position = 0.5)while (generations++ < limit)

for each vector i dofor each position j do

generate Vi(j) according to P(j)end-doevaluate f(Vi)

end-doVmax = max(f(Vi))update P according to Vmax

if random(0,1] < Pmutate

mutate Pend-if

end-while

Update and Mutation Rules

• Update Rule

• Mutation Rule

– Pmutate = 0.02

= 0.05

*)()0.1(*)()( max jVjPjP ii

*]0.10.0[)0.1(*)()( randjPjP ii

Extensions

• Learning from M top scorer: Adapt the probability matrix– equally according to the M vectors, or

– where there is consensus in, or most, of the M vectors

– according to the rank of the M top vectors

• Learning also from negative sample– move away the worst vector

– modify only those positions where the best and the worst disagree

Some applications

• Function optimization

• Job-shop scheduling

• TSP

• Bin-packing

• Knapsack Problem

• Neural Network weight training

Some ComparisonPBIL EDO EA

population Fixed size Variable size Fixed size

Probability matrix

One One per parent None

parameters fixed Adaptive Can be adaptive

learning Pos + neg Pos + neg pos

diversity high Very high Gradually decreasing

Search scope Local Local Global & local

References

• Shumeet Baluja, Population-Based Incremental Learning: a method for integrating genetic search based function optimization and competitive learning, Technical report CMU-CS-94-163. 1994.

• Shumeet Baluja & Rich Caruana, Removing the Genetics from the Standard Genetic Algorithm, ICML’95.

• Shumeet Baluja, An Empirical Comparison of Seven Iterative and Evolutionary Function Optimization Heuristics, Technical Report CMU-Cs-95-193, 1995.