Evolutionary algorithms vs. poker games
35
EVOLUTIONARY ALGORITHMS VS. POKER GAMES Yikan Chen ([email protected]) Weikeng Qin ([email protected]) 1
description
Evolutionary algorithms vs. poker games. Yikan Chen ([email protected]) Weikeng Qin ([email protected]). Outline. Evolutionary Algorithm. E-ANN. Poker!. Artificial Neural Network. Evolutionary algorithm. Evolution Process. Evolutionary algorithm. Crossover. Mutation. - PowerPoint PPT Presentation
Transcript of Evolutionary algorithms vs. poker games
EVOLUTIONARY ALGORITHMSVS.POKER GAMES
Yikan Chen ([email protected])Weikeng Qin ([email protected])
1
OUTLINE
2
Evolutionary Algorithm
Poker!
Artificial Neural
Network
E-ANN
EVOLUTIONARY ALGORITHM
3
EVOLUTIONARY ALGORITHM Evolution Process
4
Crossover
Mutation
Natural Selection
Evolutionary Algorithm
EVOLUTIONARY ALGORITHM Encoding and Crossover
5
1 1 1 0 0 1 1 0
0 1 0 0 1 0 1 1
0 0 1 1 0
0 1 0 1 1
0 1 0
1 1 1
EVOLUTIONARY ALGORITHM Mutation
6
1 1 1 0 0 1 1 0
1 1 0 0 0 1 1 1
EVOLUTIONARY ALGORITHM Natural Selection
7
Run the roulette-wheel selection based on the fitness value of candidates
EVOLUTIONARY ALGORITHM Important Parameters
Crossover rate Mutation rate Elite rate Fitness function
Demohttp://userweb.elec.gla.ac.uk/y/yunli/
ga_demo/
8
EVOLUTIONARY ALGORITHM & POKER AKQ 2-player game
$1 blinds for each player Player1 bet or fold Player2 call or fold
9
EVOLUTIONARY ALGORITHM & POKER Derive the optimal strategy using EA Chromosomal representations
Fij: fold threshold when Pi got Cardj
Fitness functions
10
Card1
Card2
Card3
P1 2/3 0 0P2 1 2/3 0
EVOLUTIONARY ALGORITHM & POKER Fitness functions
Fi: fitness function Wij: money won by candidate I against
candidate j
11
12
13
EVOLUTIONARY ALGORITHM & POKER
14
Decreased fluctuation
Further decreased fluctuation
400-500 generations
Var(f11) ; Var(f22)
Mean(f11);Mean(f22)
Count only wins
.065;
.067.67;.60
Penalize failure
.037;
.035.67;.70
Penalize Failure heavier
.028;
.024.67;.74
EVOLUTIONARY ALGORITHM & POKER Real Texas Hold’em Encoding Strategy (Turn and River)
Hand strength (player confidence) Fraction of opponent raise (opponent
confidence) Total raise (profit)
15
EVOLUTIONARY ALGORITHM & POKER Fitness Criterion
16
EVOLUTIONARY ALGORITHM & POKER Performance
17
ARTIFICIAL NEURAL NETWORK: REVIEW
18
ARTIFICIAL NEURAL NETWORK: REVIEW
19
∑
w1
w2
wn
b
……
a1
a2
an
1
f output
ARTIFICIAL NEURAL NETWORK: REVIEW
20
Inputoutput
Hidden Layer
EvolvingTopology
E-ANN (EVOLUTIONARY ANN) Simplest Encoding Method
21
a b c d d c b a
NEAT E-ANN http://www.cs.utexas.edu/users/nn/ Neuro Evolution of Augmenting
Topologies Encoding Strategy: Node-based
Neuron gene table Link gene table
Innovation number Global database of innovations Each innovation has unique ID number
22
NEAT E-ANN
23
NEAT E-ANN Mutation
Perturb weights Add a link gene Add a neuron gene
Crossover By innovation number
24
NEAT E-ANN Crossover
25
2 3
56
4
31 2
5
4
1
11->4
22->4
33->4
42->5
55->4
81->5
11->4
22->4
33->4
42->5
55->4
65->6
76->4
93>5
101->6
NEAT E-ANN Crossover
26
2 3
5
6
4
1
81->5
11->4
22->4
33->4
42->5
55->4
65->6
76->4
93>5
101->6
E-ANN & POKER Simplified Poker Model
1-10 Initial credit: 10 chips One chip ante at the beginning Call, raise (1 chip each time), fold Tournament
27
E-ANN & POKER
28
Two player game
E-ANN & POKER
29
E-ANN & POKER Four different types of
opponents
30
Tight Aggressive (TA) Tight Passive (TP)Loose Aggressive (LP) Loose Passive (LP)
E-ANN & POKER α: min win probability to call β: min win probability to raise
31
E-ANN & POKER
32
A: player typeB: player action
E-ANN & POKER
33
E-ANN & POKER Bluffing……
34
35
Thanks!
