By Luigi Cardamone, Daniele Loiacono and Pier Luca Lanzi

Introduction
Related work
Torcs
Imitation learning
What sensors?
What actions?
What learning method?
What data?
!"erimental results
Discussion, conclusions and #uture work
What is imitation learning?
$u"er%ised learning
&euroe%olution
Two main methods
Direct methods
Indirect methods
Direct methods are well'known to (e %ery
ine##ecti%e)
*ur methods de%elo" dri%ers with only +,-
lower "er#ormance than (est (ot in T*RC$)
The trick is in .human'like/ high'le%el action
"rediction
Introduction
Related work
Torcs
Imitation learning
What sensors?
What actions?
What learning method?
What data?
!"erimental results
Discussion, conclusions and #uture work
Imitation learning in com"uter games
Rule'(ased &PC #or 0uake III %ia two'ste" "rocess
0uake II &PC %ia rein#orcement learning, #uzzy
clustering and a Bayesian motion'modeling)
&eural networks with (ack"ro"agation #or Legion
II and 1otocross The 2orce)
Dri%atar training #or 2orza 1otos"ort
Introduction
Related work
Torcs
Imitation learning
What sensors?
What actions?
What learning method?
What data?
!"erimental results
Discussion, conclusions and #uture work
The range#inder
sensor
The lookahead
sensor
3 low'le%el e##ectors in T*RC$
Wheel
4as "edal
Brake "edal
4ear change
5 high'le%el actions in this work
$"eed
Tra6ectory
7'nearest neigh(or
Training 8
Doesn9t need any training
:ow it was a""lied?
Directly during the T*RC$ race
;t each tic, the logged data is searched to #ind the k
most similar instances)
The k similar instances are selected and a%eraged
&eural networks
Training
&euroe%olution with ;ugmenting To"ology <&;T= to
e%ol%e (oth the weights and the to"ology o# a neural
network
:ow it was a""lied?
5 networks, #or s"eed and target "osition "rediction
Range#inder networks with +> angle in"uts ? + (ias in"ut
Lookahead networks with @ segments in"uts ? (ias in"ut
The #itness was de#ined as the "rediction error
In#erno (ot on A tracks #or A la"s each
$im"le #ast track
Di##icult track with many #ast turns
; di##icult track with many slow shar" turns
*nly the data o# second la" was recorded
A data sets with +>@5, A@>> and AB+> e!am"les
;dditional all'in set with >,CC e!am"les
Introduction
Related work
Torcs
Imitation learning
What sensors?
What actions?
What learning method?
What data?
Experimental results
Discussion, conclusions and #uture work
*%erall, we o(tained +B models
5 learning algorithms
A ? + datasets
5 ty"es o# sensors
7'nearest algorithm was a""lied with k D 5C
&;T was a""lied with +CC indi%iduals #or +CC
generations
;ll the e!"eriments were conducted using
T*RC$ +)A)+
ach model was e%aluated (y using it to dri%e a
car on each track #or +C)CCC game ticks)
The tracks
A tracks used #or training
5 unseen tracks
; sim"le #ast track
; track with many #ast and di##icult turns
The dri%er was also eEui""ed with standard
reco%ery "olicy)
In#erno was (etter than its imitations
Lookeaheads are (etter than range#inders
7'nearest neigh(or is (etter than &;T
*ne o# the models had only +,- lower
"er#ormance than In#erno (ot)
Direct methods result in low com"utational
cost
*ur a""roach needs AC times less CPF time
to o(tain reasona(le results
:ow much lookahead is use#ul?
$econd series o# tests with @ and +B
lookeahead %alues showed o%er#itting
Introduction
Related work
Torcs
Imitation learning
What sensors?
What actions?
What learning method?
What data?
!"erimental results
Discussion, conclusions and future work
4ood dri%ers
Close to the target (ot
Run out o# the track in di##icult turns as a result o#
"rediction error or a low reacti%ity in steering
Bad dri%ers
1any discontinues in the "rediction o# tra6ectories
Causes car to mo%e Euickly #rom one
side o# the track to the other one
Two di##erent "laces can (e "ercei%ed the
same
Fsually ha""ens on long straight "arts o# the
road
Can (e sol%ed %ia s"ecial treatment o#
straight "arts, #ull throttle or (igger
lookahead
$u"er%ised learning to imitate a dri%er
:igh'le%el as"ect o# dri%ing, s"eed and
tra6ectory rather than low'le%el e##ectors
&o%el lookahead sensor
4ood results with k'nearest neigh(or
In#erno (ot is still (etter due to "erce"tual
aliasing and slow steering during a(ru"t turns
!"loit structural symmetry on the track
Increase the ro(ustness to noise
Reduce com"utational cost
Im"ro%e steering reaction to a(ru"t turns