Car Setup Optimization Competition @ EvoStar 2010
-
Upload
daniele-loiacono -
Category
Technology
-
view
5.809 -
download
1
description
Transcript of Car Setup Optimization Competition @ EvoStar 2010
![Page 1: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/1.jpg)
Car Setup Optimization @ EVOStar-2010
Car Setup Optimization Competition
@ EVOStar 2010
Luigi Cardamone, Daniele Loiacono,
Markus Kemmerling, Mike Preuss
& TU Dortmund
![Page 2: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/2.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
What is the competition about?
The goal is to submit an optimization algorithm for a challenging problem
The problem: find the best car setup that achieves the highest performance on a previously unknown track
The submitted algorithms will be compared on three different tracks
![Page 3: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/3.jpg)
Optimizing Car Setup in TORCS
![Page 4: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/4.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
Optimizing Car Setup
As in real racing competition, finding a good car racing setup is a very challenging problem.
The best set of parametersdepends on:
The track
The driver’s style (in this case AI controller)
The final measure
Challenges:
Limited amount of time available for the optimization
Evaluation times must be determined by the algorithm
Noisy fitness function
![Page 5: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/5.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
Which parameters?
A car presents many parameters that can be optimized:
Gear ratios
Rear/Front wing angle
Brakes
Rear differential
Rear/Front anti-roll bars
Wheels
• Ride
• Toe
• Camber
Suspensions
• Spring
• Bell crank…
![Page 6: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/6.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
Optimization Framework
The competition framework creates a physical separation between the optimizer and the TORCS engine:
Freedom to choose the programming language of the optimizer
Restrict the access only to a set of parameters defined by the designer
Optimization Algorithm
TORCS
Client Server
Parameter sequence
Fitness evaluation
UDP
![Page 7: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/7.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
The optimization process
The parameters involved in the optimization are normalized in the range [0,1]
The optimizer iteratively asks to the TORCS server to evaluate a given sequence of parameters
The optimizer must also specify the duration of the each evaluation (specified in game tics)
Each time an evaluation request arrives, the server:
1. Plug in the parameters in the car setup
2. A programmed policy (Berniw2) drives the car for the specified amount of time
3. Finally, the evaluation results are returned as: lap time, distance raced, top speed and damages
![Page 8: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/8.jpg)
Submissions
![Page 9: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/9.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
The competitors
Five new entries have been submitted to the competition:
Jorge Muñoz - Universidad Carlos III de Madrid, Spain -MOEA
Moisés Martínez Muñoz, Emilio Martín Gallardo, Yago SaezAchaerandio – Universidad Carlos III de Madrid, Spain –(12+10)-EA
Pablo José García Evans, Yago Saez Achaerandio –Universidad Carlos III de Madrid, Spain – PSO
Wolfgang Walz, TU Dortmund, Germany - PSO
Antonio J. Fernández, Carlos Cotta, Alberto Fuentes –University of Malaga, Spain
Entries (not ranked) of the organizers:
Basic Genetic Algorithm (Luigi Cardamone et al., Politecnico diMilano)
CMA-ES with well chosen parameters (Markus Kemmerling, TU Dortmund)
![Page 10: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/10.jpg)
The Results
![Page 11: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/11.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
Scoring process
Scoring process involved three tracks (unknown to the competitors):
Dirt 3 (a dirty track)
Poli-Track (unknown)
CG Track 2 (rather easy and fast)
For each track, 10 runs of each optimization technique have been performed
Each run is stopped after that 1.000.000 of game tics expires (approximately 2 minutes of computation)
The champion of each run was evaluated scoring the distance raced in the first 10.000 game tics of a race
![Page 12: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/12.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
2010 Tracks Pictures
Poli-trackDirt-3
CG track 2
![Page 13: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/13.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
CG track 2
0
2000
4000
6000
8000
10000
12000
avg
var
![Page 14: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/14.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
Poli-track
0100020003000400050006000700080009000
avg
var
![Page 15: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/15.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
Dirt-3
0
1000
2000
3000
4000
5000
6000
7000
avg
var
![Page 16: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/16.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
And now some Points (as in Formula-1):
Competitor CG track 2 Poli-track Dirt-3 Overall
Munoz-MOEA 10
Garcia/Saez-PSO 6
Walz-PSO 8
Fernandez/Cotta/Fuentes 4
Munoz/Martin/Saez-EA 5
![Page 17: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/17.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
And now some Points (as in Formula-1):
Competitor CG track 2 Poli-track Dirt-3 Overall
Munoz-MOEA 10 6
Garcia/Saez-PSO 6 10
Walz-PSO 8 5
Fernandez/Cotta/Fuentes 4 4
Munoz/Martin/Saez-EA 5 8
![Page 18: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/18.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
And now some Points (as in Formula-1):
Competitor CG track 2 Poli-track Dirt-3 Overall
Munoz-MOEA 10 6 8
Garcia/Saez-PSO 6 10 5
Walz-PSO 8 5 6
Fernandez/Cotta/Fuentes 4 4 10
Munoz/Martin/Saez-EA 5 8 4
![Page 19: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/19.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
And now some Points (as in Formula-1):
Competitor CG track 2 Poli-track Dirt-3 Overall
Munoz-MOEA 10 6 8 24
Garcia/Saez-PSO 6 10 5 21
Walz-PSO 8 5 6 19
Fernandez/Cotta/Fuentes 4 4 10 18
Munoz/Martin/Saez-EA 5 8 4 17
And the winner is:
Jorge Muñoz - MOEAUniversidad Carlos III de Madrid, Spain
![Page 20: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/20.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
……and what about the organizers?
Competitor CG track 2 Poli-track Dirt-3 Overall
Munoz-MOEA 9831.83 7654.01 6128.29 23614.13
Garcia/Saez-PSO 8386.77 7979.86 5021.41 21388.04
Walz-PSO 8408.35 7304.54 5336.88 21049.77
Fernandez/Cotta/Fuentes 7553.21 5931.47 6263.40 19748.08
Munoz/Martin/Saez-EA 8167.60 7718.36 4629.33 20515.29
Cardamone-SimpleGA 9563.08 7273.06 5932.09 22768.23
Kemmerling-CMA_ES 10410.13 8392.49 6415.87 25218.49
![Page 21: Car Setup Optimization Competition @ EvoStar 2010](https://reader034.fdocuments.in/reader034/viewer/2022051412/54c05b484a795911198b4631/html5/thumbnails/21.jpg)
Car Setup Optimization @ EVOStar-2010
TU Dortmund
Conclusions
Good news:
We had 5 new entries! Wow!
The center of car optimization in Europe is in Spain :-)
And:
Every track was won by a different algorithm
The winner algorithm is multi-objective and uses all 4 available return values: top speed, distanced raced, damage and lap time
3rd to 5th place very tight…
But:
It is obviously not that easy to beat the SimpleGA
Organizer entries show that there is still some potential