Evolution, Brains and Multiple Objectives By Jacob Schrum [email protected].
46
-
Upload
solomon-rogers -
Category
Documents
-
view
214 -
download
0
Transcript of Evolution, Brains and Multiple Objectives By Jacob Schrum [email protected].
About Me
B.S. from S.U. in 2006Majors: Math, Computer Science and GermanHonors Thesis w/ Walt Potter:
Genetic Algorithms and Neural Networks Currently Ph.D. student at U.T. Austin
Received M.S.C.S. in 2009Neural Networks Research Group:
Genetic Algorithms and Neural Networks
Evolution Change in allele frequencies in population Alleles = variant gene forms Genes ⇨ traits Traits affect:
Survival Reproduction
Natural selection favors good traits
Genetic Algorithms Abstraction of evolution
Genes = bits, integers, reals Natural selection = fitness function Mutation = bit flip, integer swap, random perturbation, … Crossover = parents swap substrings
Other representations, mutation ops, crossover ops, …
)()()( ADCABA Boolean Satisfiability
Applications
K. A. De Jong and W. M. Spears, “Using Genetic Algorithms to Solve NP-Complete Problems” ICGA 1989
)()()( ADCABA
Applications
Magic SquaresT. Xie and L. Kang, "An evolutionary algorithm for magic squares" CEC 2003
)()()( ADCABA
Applications
Circuit DesignJ. D. Lohn and S. P. Colombano, "A circuit representation technique for automated circuit design" EC 3:3 Sep. 1999
)()()( ADCABA
Applications
Wing Design/Cost OptimizationJ. L. Rogers and J. A. Samareh, "Cost Optimization with a Genetic Algorithm" NASA Langley Research Center, RTA 705-03-11-03, October 2000
)()()( ADCABA
Applications
Traveling Salesman ProblemP. Jog, J. Y. Suh, and D. van Gucht. "The effects of population size, heuristic crossover and local improvement on a genetic algorithm for the traveling salesman problem" ICGA 1989.
)()()( ADCABA
Applications
Resource-Constrained SchedulingS. Hartmann, "A competitive genetic algorithm for resource-constrained project scheduling" NRL 45 1998
)()()( ADCABA
Applications
Lens Design
X. Chen and K. Yamamoto, "Genetic algorithm and its application in lens design", SPIE 1996
)()()( ADCABA
Applications
Weight Selection for Fixed Neural NetworksF.H.F. Leung, H.K. Lam, S.H. Ling and P.K.S. Tam, "Tuning of the structure and parameters of a neural network using an improved genetic algorithm" NN 14:1 Jan. 2003
)()()( ADCABA
Applications
What Are Neural Networks?
Artificial Neural Networks
Brain = network of neurons ANN = simple model of brain
Neurons organized into layers
What Can Neural Networks Do?
In theory, anything! Universal Approximation Theorem
NNs are function approximators
In practice, learning is hard Supervised: Backpropagation Unsupervised: Self-organizing maps Reinforcement Learning:
Temporal-difference learning and Evolutionary computation
M]1,0[[0,1]N
Neuro-Evolution Genetic Algorithms + Neural Networks Many different network representations
Fixed length string Subpopulations for each Evolve topology and weights hidden layer neuron [1] [2]
[1] F. Gomez and R. Miikkulainen, "Incremental Evolution Of Complex General Behavior" Adaptive Behavior 5, 1997.[2] K. O. Stanley and R. Miikkulainen, "Evolving Neural Networks Through Augmenting Topologies" EC 10:2, 2002.
Constructive Neuroevolution Population of networks w/ no hidden nodes Random weights and connections
Constructive Neuroevolution Evaluate, assign fitness Select the fittest to survive
Constructive Neuroevolution Fill out population Crossover and/or cloning
Crossover Clone
Constructive Neuroevolution Random mutations Perturb weight, add link, splice neuron
No mutation Perturb weight Add link Splice neuron
Constructive Neuroevolution Can add recurrent links as well Provide a form of memory
Neuroevolution Applications
F. Gomex and R. Miikkulainen, “2-D Pole Balancing With Recurrent Evolutionary Networks” ICANN 1998
Double Pole Balancing
Neuroevolution Applications
K. O. Stanley and R. Miikkulainen, "Competitive Coevolution through Evolutionary Complexification" JAIR 21, 2004
Robot Duel
Neuroevolution Applications
N. Kohl, K. Stanley, R. Miikkulainen, M. Samples, and R. Sherony, "Evolving a Real-World Vehicle Warning System" GECCO 2006
Vehicle Crash Warning System
Neuroevolution Applications
K. O. Stanley, B. D. Bryant, I. Karpov, R. Miikkulainen, "Real-Time Evolution of Neural Networks in the NERO Video Game" AAAI 2006
Training Video Game Agents
http://nerogame.org/
What I Do With Neuroevolution Discover complex behavior
Multiagent domainsSimulations, robotics, video games
Support for multiple modes of behavior Multiobjective optimization
Mutiobjective Optimization
Pareto dominance: iff
Assumes maximization Want nondominated points
NSGA-II [3] used Popular EMO method
uv
ii uvni :,,1
ii uvni :,,1 Nondominated
[3] K. Deb, S. Agrawal, A. Pratap, T. Meyarivan, "A Fast Elitist Non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II" PPSN VI, 2000
Non-dominated Sorting Genetic Algorithm II
Population P with size N; Evaluate P Use mutation to get P´ size N; Evaluate P´ Calculate non-dominated fronts of {P P´} size 2N New population size N from highest fronts of {P P´}
Evolve Game AI Game where opponents have multiple objectives
Inflict damage as a group Avoid damage individually Stay alive individually
Objectives are contradictory and distinct
Opponents take damage from batPlayer is knocked back by NPC
Intelligent Baiting Behavior
How to avoid stagnation
Some trade-offs are too easy to reach Focus on difficult objectives TUG: Targeting Unachieved Goals
Avoids need for incremental evolution
Evolution
Hard Objectives
Smaller Team w/ Expert Timing
Multitask Domains
Perform separate tasks Predator/Prey
Prey: run awayPred: prevent escape
Front/Back RammingAttack with ram on frontAttack with ram on back
Multitask Mode Mutation Two tasks, two modes Start with one mode, mutation adds another Appropriate mode used for task Preference neurons control mode choice
Multimodal Networks One network, multiple policies
Multitask [4] = one mode per taskMode mutation = network chooses mode to use
[4] R. A. Caruana, "Multitask learning: A knowledge-based source of inductive bias" ICML 1993
Multimodal Predator/Prey BehaviorLearned with Mode Mutation
Runs away in Prey task Corralling behavior in Predator task
Multimodal Front/Back Ramming Behavior
Learned with Multitask
Efficient front ramming Immediately turn around to attack with back ram
What about “real” domains?
Unreal Tournament 2004Commercial video gameBasis for BotPrize competition:
Bot Turing Test Placed 2nd with our bot: UT^2
UT^2 Behavior/Judging Game
Summary
Neural networks can represent complex behavior Neuroevolution = way to discover this behavior Multiobjective evolution needed in complex domains Success in challenging designed/commercial domains
Auxiliary Slides
Empirical results
Differences for Alternating and Chasing significant with p < .05
