Proceedings of ICISIP - 2005

An Intelligent Neural network based Driving System using Artificial Net Extension #T Srinivasan 1, Arvind Chandrasekhar 2, Jayesh Seshadri 3, J B Siddharth Jonathan 4

Department of Computer Science and Engineering, Sri Venkateswara College of Eflgineering,

Sriperumbudur, India , I tsrin i@,\"Vce.at:.ill, 2 arvilldcac@hotmai!.com, 3 jayes!ls2000@yahoo.co.in, 4jollathansiddlla rtll@yallOo.co.in

Abstract

Over the past decade, the field of automated intelligent transport systems has been the focus of intensive research. This paper proposes an 1l1le/ligel1I Neural network based Driving System using Artificial Net Extension (INDSANEJ. an advanced auto'-;wted transport

' system with considerable

advantages over previoHs a!tell/pIS ill this field. 71w syS/e1/l uses a multi-layer feed-j'onl'a rd neum! network with back propagatiol! learning. /11 additioll . the design of INDSANE involves the convergence of a plethora of technologies like a Global Positioning System (GPS). a Geographic Information System (GIS), {llid laser wllgillg. INDSANE can guide a mobile agent tlrrmtgh a hosri/e and unfamiliar domaill afier beillg trailled by a human IIser with dOlliain expertise. aile of the many areas ill H'hich ]NDSANE scores against the calliper irian is tlnll ihe system is comp/erely domailJ ilJdependenr {md ilJcurs {/ lot less processor overhead. INDSANE thus provides more functionality even though it requires {/ 101 less inpul {IS compared 10 other attempts in th is field. 71tis reduction ill the si�e of the inpur vector tl'wlslales into //lore efficient and faster processillg, Another of INDSANE's hal/mark features is its ability'to negotiate WI'llS and implement lanc-changing 1II(1I1e1lVerS with a view to overtakillg obstacles. It does this bv qllploying a novel technique. Selective Net Masking. INDSANE also employs a lechnique called Artificial Nel F:x/ellsioll for negotiating

//"ajJ1c signals. A sil!lulation of lNDSANE's lIeural network ,,'(lS performed all {I V{lriCIY of nelll'ork topologies, and the best lIenvork selecTed.

1. I1'iTROnUCTION

Man has long dreamed of designing machines that are

capable of operating themselves. one of the basic milestones to develop truly intelligent systems that can intUitively learn how to perform specific operations and execute them to perfection. One field in which several forays have been made

to this end is that of automated transport systems, The primary tes t of any intelligent navigation system is its

anility 10 negotiate a standard road or highway environment successfully. Such a system must be capable of identifying curves in tl1e road and obstacles in its path and take appropriate action in each case. In addition. the machine

0·7803-8840·21051$20.00 11:l2005 IEEE

might be capable of changing lanes or overtaking slower vehicles. A large number of the systems proposed to date [1]-[4] use a video camera feed as input . In [1]-[3], this input is fed to a base neural network. The video feed incurs massive processor overhead, thereby increasing the complexity and size of the network and reducing the efficiency of the sy&tem as a whole. The design of a system that rivaled Il] but did not use a neural network was presented in [4]. However, all these

systems are constrained by the domain in which they operate.

Another issue plagu ing these systems lS that they are not light ing independent, as they operate based un a camera feed.

B. Freisleben et al presented . in {5], an interesting system that

relied on strategically placed sensors instead of a video feed . We use a similar concept in our approach.

This paper proposes INDSANE. a highly advanced system with considerable advantages over pre vious attempts in the field of intelligent transport systems, lNDSANE uses a lIlulri­

layer feed fonmrd neural lIetwork with hock propagation teaming. In addition, the design of INDSANE involves the fusion of a myriad of technologies like a Global Positioning

System (GPS), a Geographic Information System (GIS) and laser ranging. INDSANE can guide a mobile agent through a hostile and unfamiliar domain after being trained by a human user with domain expertise. This system incurs a lot less processor overhead compareci to the other approaches discussed, INDSANE thus provides more functionality even though the input vector is far smaller than in earlier systems. This reduction in the size of the input vector translates il1lO more efficient and faster processing. Another salient feature of INDSANE is its ability to change lal1l�s and pass a slower

vehicle or a stationary ohstacle ahead of it. INDSANE is also capablc of responding to traffic s ignals without being trained

to do so explicitly, The rest of the paper is organized as follows. Section 2 describes the previous work in this field. The proposed system and its design is presented in Section 3. All algorithms relevant to the operation of INDSANE arc dcalt with in Section 4. In 'Section 5. the performance analysis of INDSANE with a number of network topologies is addressed

and the best topology of those tested is subjected to more rigorous analysis, Finally , Section 6 sunm1arizes the paper and gives insight into possible further development and future directions of ongoing work,

'246

2. REUTED WORKS

There have been a number of all empts at developing inrelligcnr and automated clriving machines. Each of these attempts has had its share of success accompanied by specific drawbacks. Here. we consider four existing approaches:

A. ALVINN and MANIAC ALVINN (Autonomous Land Vehicle In a Neural

l\'etwork) and MANIAC (Mu ltiple AL VINN Networks In Autonomous Control). developed at Carnegie Mellon University [0 control the NavLab vehicles there. are outfitted with computer-controlled steering. acceleration and braking.

Sensors include color stereo video. scanning laser range finders. radar and inertial navigation. Both systems requ ire

training by a human driver for about five minutes (followed by application of the back propagation algorithm for ten minutes) before they are ready to drive. The Signal from the vehicle's video camera IS preprocessed to yield an array of pixel values that are connected to a 30x32 grid of input units in the neural network.

AL VIl\'N computes a function that maps from a single vldeo image of the road tr> a steering direction. The output is a layer of 50 units. each corresponding to a steering direction. ALVTNN's performance is impressive in the cases where it has been tramed. It is unable 10 drive on a road type for which J[ has not been trained, and is also not very robust with

. respect to changes in lighting conditions or the presence of other vehicles.

MANIAC (Multiple ALVINN Networks In Autonomous Control) is composed of s everal pre-trained ALVINN

networks. each trained ror a single road type that is expected

10 be encountered during driving, The superstructure combines data from each of the AL VIl'oIN networks and does not simply select the best one. The output from the AL VINN

networks can be taken from either their output or hidden units.

B. ELVIS ELVIS is a road-following system based on ALVINN. It is

an attempt 10 design a system that can rival AL VINN using the same input and output. but without using a neural network.

Like ALVINN. ELVIS observes the road through a video camera and observes human steering re sponse through encoders mounted on the steering colunm. ELVIS learns the eigenvectors of the image and steering training set vi'1 principal component analysis. These eigenvectors roughly correspond to the primary features of the image set and their correlations to steering. Road following is then perfomled by projecting new images onto the previously calculated �igenspace.

While ELVIS prinCipal component analysis minimizes the total error in the image reconstruction and steering vector. ALVINN directly minimizes the steering output alone. Since

Proceedings of ICISIP -Z005

there are bound to be some small areas of the image that are not highly correlated with the steering output. AL VINN is able to produce better steering results . In addition. although the speed of training of the two systems is roughly equal. ALVINN is faster at run-rime because it requires far fewer

calculations. The primary advan tage of ELVIS is its simplicity and lack of pre-defined structure.

C. Sensor networks One of the primary disadvantages of a system using a'video camera feed is the enormously hig.h processing power required. The performance can be improved by using a sensor network that depends on a number of non-vleleo sensors such as range finders attached to the vehicle to sense the environment around the vehicle and take appropriate action . INDSANE is essentially a sensor netwmk. such as that de scri bed in [5].

3. DESIGN OF. THE PROPOSED SYSTEM

INDSANE brings together a plethora' of technologies. namely a G lobal Positioning System (GPS), a GeographiC Information System (GIS). a laser transceiver bank and a multilayer feed forward neural network.

Fig. I. Gen�mt urganiz.1tion of INDSANE.

A. CPS The GPS in INDSANE is used to obtain the coordinates of

the location of the vehicle. This information is relayed to the GIS in order to obtain information necessary for the operation ofINDSANE.

B. CIS The characteristic of a GIS thaI is most relevant to our

discussion is the fact that one can 'point' at a location. object. or area on the screen and retrieve recorded information about it from off-screen files. With specific reference to INDSANE. the data from the GIS that are relevant are road locations/maps, road angles. road widths. and traffic signals.

In most cases, the GIS data is relatively statically updated. say, once a day or once a week as needed. Highly dynamic

247

Proceedings of ICISIP - 2005

information such as the color. of the signal at the intersection ahead can be obtained by maintaining a connection to the local traffic police records also.

AI! other data is readily available from standard GIS. Thus, based on the current location of the vehicle, all necessary details can be obtained.

In brief. we see that the GPS is instrumental in pinpointing the precise location' of the vehicle. while the GIS provides real time information based on this location.

C. Laser transceiver bunks The transceiver bank consists of arrays of laser beam

emitters and detectors. In a standard configuration, light is sent from the source by an emitter. When the beam reaches an object, it is reflected back to the source. A detector at the source receives the reflected beam and indicates that an object is present. Based on the time between emission and detection

T . and the speed of the 'laser beam S . the distance D of the object from the transceiver bank is determined.

D = SX(T 12) (1) The system possesses two' banks of transceivers. The

frontal transceiver bank provides a Frontal Impact Collision

Vector (FICV) that holds the information necessary to detect and avoid an obstacle in front of INDSANE. The transceiver banks at the sides of the vehicle provide a Side Impact Collisioll Vector (S/CV} that holds the information necessary to detect the presence of obstacles that are parallel to INDSANE in the adjacent lanes. The SICV is used during the process of overtaking by changing lanes. The neural network uses the collision vecror information to take appropriate

action (such as applying the brakes).

D. Mullilayer Feed Fonrard Neural Network The network under study here has two layers and one

hidden layer. Its input vector is comprised of

• FICV • Current sp'eed • Traffic signal information

The output vector is comprised or • Steering angle • Accelerator pressure • Brake pressure

4. ALGORITHMS

I) INDSANE algorithm

function INDSAt-I'E() define:

II(Mork. a multilayer network input, input vector output. Output vector RSL, a binary variable indicating that space is available on

the left for an overtaking maneuver when set.

RSR. a binary variable indicating that space is available on the right for an overtaking maneuver when set. begin DataAcquisitionO ArtificialNetExtensionO network f- NeuralNetworkLeamirig() repeat ,

input f- values from the input sensors output f- RunNetwork(lletlfork. inpur) if (RoadAlIgle = 0 and (output.SteeriligAngle *- 0) anel (RSL= l)or(RSR= I))

output f- Overtaking(output) else if (RoadAngle *- 0)

end if

onboard RTOS generates an Interrupt ISR executed is Tum(RoadAl1gle. OfIlPUt)

i mplement the output vector indefinitely end

This is the primary driver function of INDSANE. First. the training set sample data is acqUired (DataAcquisition). This is followed by the Artificial Net extension phase. The size of the current training set is doubled by adding an extra training example for each example already in the training set. to deal with traffic signal negotiation. Next is the Neural Network Learning phase where the link weights are updated based on the input-output sample sets. After this comes 'he real-Time application of INDSANE. The system Cl>ntinuously reads the inp'u[ from the sensors, derives the appropriate outputs by running the neural network and implements these outputs. If the system finds that the road angle is zero. the output sleering angle is not zero and the RSL or RSR flag is set. then· it enters the overtaking mode. If the road angle input is not zero, it enters the turning mode where it employs selective net masking to override the output steering direction alone.

2) DATA ACQUISITION algorithm

function DataAcquisitionO begin repeat

Read input from FIeV, Speedometer reading. Trame signal information

Read the human driver's response regard ing Steering angle, Accelerator pressure, Brake pressure

r�om1Ulate the training set entry which comprises or the input vector and the Driver response.

Add the training set entry to the training set table, until end of training run return

This is the initial period where a hUman driver trains the vehicle. In effect, the system just observes the human's

. actions under different input conditions. These input-output

248

mappings are stored in the form of a table.

3) ARTIFICIAL NET EXTENSION algorithm

function Artificial Net Extension(TraillingSet); inputs;

TraillillgSet, the set of train ing examples obtained after Data Acquisition output;

TraillillgSet. the training set after anificialnet extension

begin [or each example E in TraillillgSet do begin

Add an example E, to TraillillgSe! by setting, E,.il1put.FICV f- E.illpw.FlCV E,.inplIl.SpeedomcterRf!adillg f- Eillplll.SpeedometerReadillg

E,.illpllt.trajJicsigllal f-l E,.OUlpllt.SteerillgAligle f- EOHfpLft.SteeringAlIgle

ErOlliput.Accelel'morPI'(Ssure f- EoHtjlll1.AcceierarorPrcssul'c H,.amput.brake f- J

end

return Trail1illgSet

DUring the Artificial Net Extension phase, for every example in the training set . another example is added hy

setting the traffic signal inpur to high and the output brake pressure to high. This step adds the traffic Signal negotiation capability to INDSA.:'\1E without the overhead of expliCit training by the human driver.

The (input, output ) pairs obtained from the human driver timing the data acquisition phase and the pairs generated as a result of Artificial Net Ex.tension are now appliect to the network to set it up. In the NeuralNetworkLearningo function, the network link weights are updated by means of Back Propagation fl3].

4) TURNING algorithm

In ten'upt Service Routine Turn( e . output) returns output

inputs ; e. the turning angle in degrees. A value in the range of

o to 180 degrees implies a right tum and a value from 180 to 360 degrees implies a left turn. output, output vector

begin

nag f- I

if e is identified to be a right turn nag f- 0 if Ilag = 1 magnitude f- 360 - (-)

e lse magnitude f- (j limit f- ceiling(magnitude/lO) iterator f- 0 while iterator < limit begin

if (flag = 0) oll1put.SteeringAlIgle = 10° else output.SreeriligAligle = - 10° iterator � iterator + 1 Implement the OUlput vector

end limit f- magnitudeMODlO iterator f- 0 while iterator < (limit � I)

begin

if (flag = 0) output.Steer;lJgAngle = J ° else OIliput.SteeringAlIgle = - 10 i terator f-' iteratof+ I implement the output vector

end

if (flag=O) else return output

outplll.SrcerillgAligle = 10 outpm.Steer;lIgAngle = - 1"

iteralor � iterator+ I implement the output vector

end if (flag=O) o!ltpm.SteerillgAngle = 1"

Prooeedings of I CISIP - 2005

The GIS input for road angle is monitored external to the neural net at all times. If the road angle is not zero. the onboard Real time Opera t ing System generates an Interrupt. The Interrupt service routine that is executed handles the turning operation. The algorithm for this Interrupt service routine is presented above. The neural net output for steering angle is subjected to Selective Net Masking. This is an implementation technique by which only specific nodes of the neural network are overridden while the rest of the neural network operates as nornlal. Here, the steering angle output is

completely determined by the luming system and overrides the ex.isting value calculated by the neural network. The

system negotiates turns first in steps or 100 and then in steps

of 10 to achieve the desired turn angle accurately.

Proceedings of I CISIP - 2005

5) OVERTAKING algo[ithm

function Overtaking(outpUl) input;

output, output vector define:

network, a multilayer network input, input vector flag, a hinary variable'indicating an ongoing incomplete

overtaking maneuver SleV, inputs from the side in-ip3ct las�r bank

begin output f- RUlli'letwork(lIellFork, input) if(SICV"# Oor (output.Steeringdirection >0 and RSR=O)

or (Output.Stecring direction <0 and RSL = 0) return output

end if push output.Steering direction onto stack flag f- J repeat

implemem the output vector output f- RunNetwork (network, input)

if (illput,F/CV = 0) begin

it' (Stack is not empty) o[itput.Stei:rilig directiotl f- (- 1 Xtop or stack) Pop the topmos[ clement otT of the Stack

else flag f- 0

end if if (0 IIlp III , Steering directioll "# 0) and (input,FICV 1= 0)

push output. Steering directiol! onto stack until flag=O return output

When the system finds that the neural net output for steering angle is non-zero and the road angle is zero, it realizes that an opportunity to overtake exists, It first checks its SICV to see if there are any obstructions to its side. ]f not, the system turns the car in the required direction by 100• It sets a flag to indicate that it has n9t straightened yet and pushes the steering angle on to the stack. It does this so that once the car has turned by the appropriaLe amount to overtake the other car by changing lanes, it can straighten itself by popping the contents of the stack one by one and implementing the complement of the steering direction, The flag is reset once the stack is empty, and the system then resumes its usual modus operandi.

12 II [0 9

� a 8 t: 7 W

� 5 4 3 2 '" ,:p # .:;.'" #' " .. '" #' .... "' .. '" .... "'''''''

.... � .. '" .... 4"' .... .., .. '" '-,.,j>'" .... � .. '" Number of Epochs

(a)

12 11 10

9 8 -

� 1 W 6 5 4 3 2 '" ..... '" 4' � .. '" #' f.,�""" #' .... r:,.1� ,,�'O:J� ..... ��� .... ��� ,�,·r�) ,r:;.t;:. .... C)��

Number of Epochs (b)

12 11 10

9 h 8 2 W 1

6 5 4 3 2 '" ..... '" �ii' .... ", #' .... ", 4' .... "' .. '" ... ",,,,,,, ... � .. '" ,,�' .... � .. ", 0"'''' ... � .. ."

Number of Epochs (e)

Fig. 2. Back propagaled error as a function of the number of epochs of training in the initial stages of training in (al Scenario I. (b) Scenario 2. and

(cl Scenario 3,

S. PERFORMANCE ANAI.YSIS

In this section, we compare three sCl;!narios with varying neural network topologies as shown in Table I. These topologies differ only in the number or no des in the hidden layer.

250

TABLE! t\IN BER Of NODES PER LA YER IN EACH SCENARro

Scenario Inputlnyer Hidden layer Output layer

16 3 3 � 16 4 3

16 5 3 We first compare the three neural net topologies on the

basis of the number of epochs they take to stabilize, and the final error at which (hey stabilize. The back propagated error

as a function of the number of epochs of training in each of the scenarios is presented in Fig. 2 and 3.

! -- Scenario I . .• - . Scen6ri0 2 -- Scenario 31 2:;.0015.,-------------------,

'·�umber of Epochs �m thousands) Pig.. 3. The back prop3gated e.rror '" a flmetion oi the mlmber of epoch, of training in each of the scenmlos.

On analyzing the gniphs, we found that the network represented by Scenarin 2 (the 16-4-3 network) stabilized the fastest. We then plotted the percentage of standard test set entrie� this network responded correctly to a& a function of the size of the training se[ used to train [he network. The result, are shown in Fig. 4.

100 90 W ti aJ 70

15 u �o oJ en so ru " 'U oJ '-' 0> 3<) 0..

10

� r----

/ L

�/ L

/ /

10

10 30 iO lQ NumberOr entlrBS in training set

Fig. -I. The. percentage of the le�t set values for which the network responds COlTectly a� a function uf the number of entries tn tht" u"aining set.

6, CONCLUSION

A pos�ible direction of further work is to develop aids to support such intelligent systems, along the Jines of the methods discussed in [1 1].

Proceedings of ICISIP -1005

INDSANE marks a mile stone in the evolution of intelligent transport systems, Its numerous advanlages - domain independence, ease of training, low processor overhead,

reduction in size of the input vector and the ability to handle complex operations like overtaking and tuming make It truly revolutionary.

While there are, as always, certain avenues for

improvement or modification, INDSANE Jays the. groundwork on which several simple yet efricient intelligent

transportation systems can be based in the future. We intend to explore the use of Suppon Vector Machines

and Kohonen networks in place of BPNN, apart from fine­tuning the overtaking and turning modules. The technology is very exciting and promi sing. It is only a

matter of time before the automohiJe giants wake up to the neural net revolution and employ sy&tems similar to INDSANE to manufacture cars tha t are truly autonomous in every sense of the word,

REFERENCES [1J C. E. Thorpe, M. Hebert. T. Kanadr and S. Sltaf�r. "Toward

Autonomous Driving: The eMU Navlab." JEBE Expert. V (i #4. August 1991.

(2] n. A.· Pume.de"n. Nenral Ne(w"rk Perception fur Mobile Rohot Guidance, Kluwer Academic Publishers, 1993. ISBN O· 7923·9373· 2.

(31 T. M. lochem. D. A. Potned • .,,, and C. E. Thorpe, "MANIAC: A Next Generation Neurallv Based AutonOlllous R"ad Follower." Proceedings of the Internntiona} Conference on Intelligent Autonomous Systems, 1993.

[4] I. Hancock anJ C. Thorpe. "ELVIS: Eigen�eclors for LanJ Vehicle Image System." Proc. of IEEElRSJ I nte(mltiorlal Cmf. on Intelligent Robots and Systems, PtLtsburgh .. PA. 199:1, PI'. 35-4U,

[51 B. Freis leben and T. Ku nkelmann, "Comhining Fuzzy Logic and Neural Networks to Control an Aut"nomo", Vehicle." Proceedings of the 2nd IEEE International Conference On Fuzzy System,. 1993.

[6] R. Aufrere, 1. Gowdy, C. Mertz. C. Thorpe, C. Wang: and T. Yata. "Perception for collision avoidance anll autonomous driving." Mechatronics, Vol. 13, No. 10. December. 2003. pp. 1 1-19-1 16l.

[7] M. Berton, and A. Broggj, "GOLD: A I'amllel Real-Time Ste,eo Vision System for Generic Obstacle and Lane Detection." IEEE Transactions on [mage Pmce.ssing.. 1998. , [8) J. Hancock, M. Hebert. and C. Thorpe. "La,er Inlensity·Base<t Obstacle Detection", Proceedings 1998 IEEE/RSJ International Conference On Intelligent Robotic Sy,tcllls (lROS ' (8). Vol. 3. OctobeL 1998, pp. 1541 - 1546.

[9] C. Wang, C. TllOrpe and A. Suppe. "Lmlar·Based Detectiun and Tmclting of Moving Objects from " Gl"\lund Vehicle at High Speeds," IEEE Intelligent Vehicle, SYnl[X"itlm (IV"'.()()3}, June 2003.

[JOI C. Thorpe, R. Aufrere, J.D. Carlson, D. Duggins. T.W. Fong. J. GOWdy, J. Kozar. R. MacL,ehIan. C. McCabe. C. MeJ1Z. A. Suppe. C. Wang and T. Yata. '·Safe Robot Driving." Proceedings of the International Conference on Machine Automalion (lCMA 2U021-September, 2002.

[t 1] P. Griffiths. D, Langer. J. A. Misener. M. Seigel and C. Thorpe. "Sensor Friendly Roadway and Vehicle System,:· IEEE IMtmmentatiol\ and Mea�l\rel1\cnt Tedmoloo.v Conference, Budapest, May 200 1. "

(12] Amrutis, A.. Polychronopoulm. A . . Kar.,eitanidis, l., KatsouJis. G . . Beltims, E .. "Multiple . Sensor - Collision avoidance system fot alltomotiv. npplication< using an IMM approach for obstad� tracking", Proc. 5th International Conference on Information Fusion, July 2002, AnJlnpolis MD. USA.

[13 j Swrut Russell and Peter Notvig, '"Attiflcial Intelligence . A Modem Approach", 3,d Ed, Pearsons Ed"cmion, Singapore, 2000.

251