Source Seeking with a Unicycleusing Discrete Stochastic

Extremum Seeking Control

J.L. Holierhoek

DCT 2009.112

Master Traineeship Thesis

Coaches:Dr. Chris Manzie1(Coach)Prof. Dr. Ir. Maarten Steinbuch2 (Supervisor)

1 The University of MelbourneSchool of Engineering, Department of Mechanical Engineering

2 Eindhoven University of TechnologyDepartment of Mechanical Engineering, Control Systems Technology Group

Eindhoven, November 16, 2009

Abstract

The aim of this internship was to use extremum seeking control to mimic source seeking ofbacteria like E. Coli, with a unicycle. The path development of bacteria like E. Coli, calledbacterial chemotaxis, is often described as a random walk with subsequent random rotationsand forward movements. This allows them to search nutrient sources or avoid repellents de-spite their lack of memory and restricted freedom of movement.

In literature, extremum seeking control is used to perform source seeking with unicycles. Twoconfigurations are used. They both work in continuous time with sinusoidal disturbances andfrom simulations some interesting path developments appeared. However, to mimic bacte-rial chemotaxis, the control scheme needs to be adapted to discrete time and with randomdisturbances. These adjustments are made and simulations indeed show path developmentscomparable to bacterial chemotaxis.

A unicycle is designed and constructed to experiment with the extremum seeking controlscheme. A microcontroller is available to implement the control scheme. To create a gradientfield, a desk lamp is used in a dark room. On top of the unicycle, a sensor is installed toenable the unicycle to sense the gradient field. The discrete extremum seeking controller withstochastic disturbances is used to perform source seeking with the unicycle. Stateflow is usedto divide the movements of the unicycle in discrete steps, i.e. driving with constant velocityand rotating a “random” angle. First tests are performed with a infrared light sensor but thissensor resulted in a too narrow gradient field compared to the size of the unicycle. Therefore,visible light sensors were ordered, but they were not delivered in time to implement them onthe unicycle.

i

ii

Contents

Abstract i

1 Introduction 1

2 Extremum Seeking Control 3

2.1 Source seeking of bacteria like E. Coli . . . . . . . . . . . . . . . . . . . . . . 3

2.2 Basics of Extremum Seeking Control . . . . . . . . . . . . . . . . . . . . . . . 4

2.3 Prior work with unicycles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.3.1 Tuning of forward velocity . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.3.2 Tuning of angular velocity . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Discrete, stochastic extremum seeking control 11

3.1 Limitations of the previously discussed schemes . . . . . . . . . . . . . . . . . 11

3.2 Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3.3 Discrete, stochastic extremum seeking scheme . . . . . . . . . . . . . . . . . . 12

3.4 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4 Unicycle Design 15

4.1 Unicycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4.2 Motor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4.3 Microcontroller . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4.4 Motor Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

4.5 Sensor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.6 Control scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.7 Testing of components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4.8 Prototype and experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5 Conclusion and recommendations 25

5.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

5.2 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Bibliography 27

A Simulink file for tuning forward velocity 29

B Simulink file for tuning angular velocity stochastic and discrete 31

iii

CONTENTS

C Datasheets of used hardware 33C.1 Datasheet of the Banebots motor . . . . . . . . . . . . . . . . . . . . . . . . . 33C.2 Datasheet of the Solarbotics Motor Driver Kit . . . . . . . . . . . . . . . . . . 34

D Executing Simulink file on HC12 35

iv

Chapter 1

Introduction

The ability to locate the source of a signal while moving in a gradient field is a fundamentalproblem in nature. This problem is called ’source seeking’ and is visible from macro-scaleorganisms like sharks searching for food to at a microscopical level at bacteria like E. Coli,who are, despite their lack of memory and restricted freedom of movement, able to find nu-trient sources and avoid repellents. The path development of bacteria like E. Coli, calledbacterial chemotaxis, is often described as a random walk with subsequent random rotationsand forward movements.

A unicycle is a vehicle with two wheels who can be driven independently. Rotating the wheelsin opposite direction makes the unicycle spin, and rotating in the same direction makes theunicycle move in a straight line. If these two movements are performed separately, a unicycleshould be able to mimic the random walk of bacterial chemotaxis.

The challenge of this project is to find a control strategy that can steer a unicycle in a ’ran-dom walk’ towards the source of a gradient field. The technique Extremum Seeking Control(ESC) is explored to fulfill this task. It is an attractive method to describe chemotaxis, be-cause it only uses local information, so no memory is needed, and is non-model based, sono knowledge of the gradient field is required. ESC has its origins in the 1920’s, where itwas one of the first methods of adaptive control. The first documented use was in 1922 withLeblanc’s application to electric railway systems. In the 1950’s and 1960’s it has been widelystudied and used. After that there was a dip in the interest in ESC due to more interestin real-time optimisation and model reference adaptive control methods. But from the late1990’s, stability proofs have been obtained and this increased the interest in research andapplications of this method again.

Problem statement

The problem statement of this project can be splitted up in two parts:

• Explore the control technique Extremum Seeking Control to mimic the path developmentof bacteria with unicycles

• Design and construct a unicycle to validate the found control strategy

1

CHAPTER 1. INTRODUCTION

Outline of the report

The beginning of Chapter 2 explains in short the basics of bacterial chemotaxis. The followingof Chapter 2 gives an introduction on Extremum Seeking Control and how in literature thebasic scheme is adapted for use with unicycles. From simulations with these schemes someinteresting path developments are discovered. The limitations of these schemes are discussedand in Chapter 3 the basic scheme is adapted to make ESC suitable to describe bacterialchemotaxis. In the end of Chapter 3, simulations are performed with this scheme. Chapter 4describes the design of the unicycle and the components. The proposed control scheme isimplemented in Simulink Real-Time Workshop. In Chapter 5 conclusions are drawn about thestatus of the prototype so far and some recommendations are given to improve the prototypeand for further research on the Extremum Seeking Control scheme.

2

Chapter 2

Extremum Seeking Control

2.1 Source seeking of bacteria like E. Coli

The propulsion method of E. Coli is called bacterial chemotaxis and is obtained by a clockwiseor an anti-clockwise rotation of flagella. Counter-clockwise rotation of the flagella aligns theflagella into a single rotating bundle, causing the bacterium to swim in a straight line. Clock-wise rotation breaks the flagella bundle apart such that each flagellum points in a differentdirection, causing the bacterium to tumble in place. This behavior of swimming in a straightline, tumbling and then swim again is often described as a ‘random walk’ [1]. Figure 2.1shows a schematic overview of a single bacterium avoiding repellent and heading towards theattractant.

Figure 2.1: Bacterial chemotaxis [2]

Source seeking by bacteria like E. Coli is hampered by two difficulties. The first one is thatthe gradient field is unknown. E. Coli is only able to measure the field at a certain positionand is not able to reconstruct the field, since it doesn’t have any position knowledge and nomemory. The second difficulty is that they cannot move freely in space, since the movementis restricted to rotational and forward movement.To mimic bacterial chemotaxis in a control loop, these difficulties have severe implications.

3

CHAPTER 2. EXTREMUM SEEKING CONTROL

The first difficulty leads to non-model based optimization techniques. The second makes thetask of source seeking even more challenging. The control algorithm that will be used toachieve source seeking, is called Extremum Seeking Control (ESC).

2.2 Basics of Extremum Seeking Control

The idea behind ESC is that an estimate of the minimiser/maximiser of a map gets disturbedby a sinusoidal signal and by measuring the corresponding outputs, the estimate is adjusted.This estimate is then disturbed again and used as an input of the map. In this way, theestimate will approach the value of the minimiser/maximiser [3]. Figure 2.2 shows the basicESC-scheme. The nonlinear map is assumed to be quadratic and of the form shown in (2.1).If J” > 0, the function has a minimum J∗ with minimiser θ∗.

JθJ(θ)

ξ χθ

α sin ωt sin ωt

−β

s

ss+h

integrator high pass filter

nonlinear map

Figure 2.2: Basic extremum seeking scheme

J(θ) = J∗ +J”

2(θ(t) − θ∗)2 (2.1)

The variable θ is the estimate of the minimiser θ∗. This estimate is disturbed by a sinusoidalsignal and used as input to the nonlinear map:

θ(t) = θ(t) + α sin(ωt) (2.2)

The estimation error can than be defined as θ:

θ(t) = θ∗ − θ(t) (2.3)

Combining (2.2) and (2.3) leads to:

θ(t) − θ∗ = α sin(ωt) − θ(t) (2.4)

By substituting (2.4) in (2.1), this gives:

J = J∗ +J”

2(α sin(ωt) − θ(t))2 (2.5)

J = J∗ +J”

2θ2(t) − αJ”θ(t) sin(ωt) +

α2J”

2sin2(ωt) (2.6)

4

2.2. BASICS OF EXTREMUM SEEKING CONTROL

The term J”

2θ2(t) is supposed to be small since this is a local analysis and can therefore

be omitted. The resulting expression can be expanded by using the following trigonometricidentity: sin2(ωt) = 1

2−

1

2cos(2ωt).

J = J∗ +α2J”

4− αJ”θ(t) sin(ωt) −

α2J”

4cos(2ωt) (2.7)

The high pass filter filters out the DC component of the output signal J :

χ =s

s + hJ ≈ −αJ”θ(t) sin(ωt) −

α2J”

4cos(2ωt) (2.8)

This gives, after multiplying with sin(ωt) the following:

ξ ≈ −αJ”θ(t) sin2(ωt) −α2J”

4cos(2ωt) sin(ωt) (2.9)

This function can be expanded by using the following two trigonometric identities: cos(2ωt) sin(ωt) =1

2(sin(3ωt) − sin(ωt)) and sin2(ωt) = 1

2−

1

2cos(2ωt).

ξ ≈ −

αJ”

2θ(t) +

αJ”

2θ(t) cos(2ωt) +

α2J”

8(sin(ωt) − sin(3ωt)) (2.10)

The integrator will attenuate the higher frequency components with cos(2ωt), sin(ωt) andsin(3ωt).

θ(t) ≈β

s

αJ”

2θ(t) (2.11)

˙θ(t) ≈

αβJ”

2θ(t) (2.12)

When recalling (2.3) and with θ∗ is a constant, this gives

˙θ(t) = −

˙θ(t) (2.13)

Combining (2.12) and (2.13) gives the following:

˙θ(t) = −

αβJ”

2θ(t) (2.14)

and with α, β, J” > 0, this leads to a stable system

θ(t) → 0 (2.15)

θ(t) → θ∗(t) (2.16)

This scheme can also be used to find a maximum (J” < 0), the adaptation gain then has tobe chosen as β < 0.

This derivation gives an intuitively explanation of how ESC is able to track the source ofan unknown non-linear map. The solution will go to a neighborhood around the minimumbut will never stop on the minimum due to the constant disturbance α sin(ωt). Convergencespeed and accuracy can be adjusted by tuning the disturbance gain α and the adaptationgain β.

Although ESC has been used since the 1920’s, a complete proof for stability has been obtainedin 2003 [4]. This proof is based on an averaging analysis of this system and shows convergenceof J(θ) to J∗ + O(α3)

5

CHAPTER 2. EXTREMUM SEEKING CONTROL

2.3 Prior work with unicycles

In the previous section, the basics of ESC are discussed. In literature, adaptations are madeto this basic scheme to make it implementable on unicycles. A unicycle is modeled as depictedin Figure 2.3. As can be seen, the unicycle has a non-collocated sensor to measure the valueof the nonlinear map.

θ

θ

θ

r

r v

x

y

sensor

center of vehicle

Figure 2.3: Model of unicycle with non-collocated sensor

The equations of motion for the center of the vehicle are the following:

xc = v cos(θ), (2.17)

yc = v sin(θ), (2.18)

Because the sensor is located at a distance r and an angle θ from the center, the equationsof motion for the sensor are:

xs = v cos(θ) − rθ sin(θ), (2.19)

ys = v sin(θ) + rθ cos(θ), (2.20)

To handle the difficulty of the restricted motion of a unicycle, two different configurations arestudied. In [5], the rotational velocity is assumed to be constant, while the forward velocity istuned with ESC. This configuration is discussed in section 2.3.1. In the second configuration,discussed in [6],[7], the forward velocity is assumed to be constant and the angular velocityis tuned by ESC. This case is studied in section 2.3.2.

2.3.1 Tuning of forward velocity

Figure 2.4 shows the ESC-scheme used in [5]. This scheme differs from the scheme depictedin Figure 2.2 in the sense that the integrator in the feedback loop is removed and the additivedisturbance is replaced by it’s derivative. This can be explained by the working principle ofthe extremum seeking loop. The loop gives an estimate of the input of the non-linear map. Soin the normal case, it would give an estimate of the position of the sensor. Because the vehiclemodel needs a velocity as input, the integrator is removed and the additive disturbance isreplaced by it’s derivative. In this way, the extremum seeking algorithm gives an estimate ofthe velocity. The paper provides a convergence proof based on an averaging analysis. The

6

2.3. PRIOR WORK WITH UNICYCLES

extremum seeking performance (accuracy and speed) can be influenced by the parametersα, ω, ω0 and β.

xs = cos(θ) − rω0 sin(θ)

ys = sin(θ) + rω0 cos(θ)

β

J

ω0

v

v

vθ

αω cos ωt sin ωt

1

s

ss+h

Unicycle, non-collocated

nonlinear map

Figure 2.4: ESC-scheme used by Zhang for the unicycle with non-collocated sensor

This source seeking scheme is implemented in Simulink for simulations. The nonlinear mapis chosen as a quadratic function of the following form:

J(xs, ys) = f∗− qx(xs − x∗)2 − qy(ys − y∗)2 (2.21)

In Appendix A the Simulink scheme and the corresponding m-file to execute the simulationsare depicted. For the simulations the following values are used: f∗ = 0, x∗ = 0, y∗ = 0, qx =0.15, qy = 0.05. The frequency of the sinusoidal disturbance is chosen to be k times the rota-tional velocity of the unicycle: ω = kω0, with k a positive integer.

0 20 40 60 80 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.5 1 1.5 2 2.5

0

0.5

1

1.5

2

2.5

x-y plot

y-a

xis

x-axis

J=

f(x

,y)

Output

time (sec)

sensorcenter

Figure 2.5: Simulation results with k = 3 when tuning forward velocity

Figure 2.5 and 2.6 show the x-y plots and the output values versus time plots for simulationswith respectively k = 3 and k = 5. The other parameters are chosen at h = 1, ω = 25, β =−25, α = 0.1. The x-y plots are showing interesting shapes. For k = 3 (Figure 2.5) the x-y

7

CHAPTER 2. EXTREMUM SEEKING CONTROL

0 20 40 60 80 1000.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

−0.5 0 0.5 1 1.5 2 2.5−0.5

0

0.5

1

1.5

2

2.5

x-y plot

y-a

xis

x-axis

J=

f(x

,y)

Output

time (sec)

sensorcenter

Figure 2.6: Simulation results with k = 5 when tuning forward velocity

plot show a triangular shape and for k = 5 (Figure 2.6) a star shape. This can be explainedby the frequency of the sinusoidal disturbance on the estimate of the velocity which is k timeshigher than the angular velocity of the unicycle. In every rotation of the unicycle, the velocitygets k periods from the disturbance in the forward velocity. The sensor senses the gradientfield and the extremum seeking scheme pushes the angular velocity in the direction of theextremum.

When increasing the parameters α, the convergence speed will increase but this will decreasethe accuracy. Increasing the parameter β will speed up the convergence speed but increasingit to a too high value will lead to unstable behavior.

2.3.2 Tuning of angular velocity

In this case, the forward velocity is assumed to be constant, while ESC is used to find theheading angle that minimises or maximises the value of the nonlinear map. Essentially thesame scheme as Figure 2.4 is used with the difference that the equations of motion of theunicycle are slightly different (ω0 is replaced by θ), v is a fixed input and the extremum seek-ing loop gives an estimate for θ∗. This has been simulated with Simulink with the followingparameters: h = 50, w = 2, β = −20, a = 0.6, v = 0.1.

Figure 2.7 shows the results of simulations. On the left the phase plot of the center of theunicycle is shown and the right figure shows the value of the nonlinear map. It is clearthat the unicycle gets directed to the source of the map (x∗ = 0, y∗ = 0). But when theunicycle passes the extremum, it takes a while before the vehicle turns and heads back tothe extremum. This is because the optimal heading angle is searched, but when crossing theextremum, the optimal heading angle suddenly changes 180 degrees. It takes a while beforethe algorithm corrects for that.The observed motion, not moving in a straight line but in a kind of sinusoidal way in space,looks more natural than the triangular- and starformed motions shown in Figure 2.5 and 2.6and can be observed in nature. Two examples are sharks searching for food or bloodhoundsfollowing a trail.

8

2.3. PRIOR WORK WITH UNICYCLES

0 200 400 600 800 1000−200

−150

−100

−50

0

−5 0 5 10

−5

0

5

10

x-y plot

y-a

xis

x-axis

J=

f(x

,y)

Output

time (sec)

Figure 2.7: Simulation results for tuning the angular velocity

9

CHAPTER 2. EXTREMUM SEEKING CONTROL

10

Chapter 3

Discrete, stochastic extremum

seeking control

3.1 Limitations of the previously discussed schemes

The prior work discussed in Chapter 2 shows that a unicycle can be controlled by ESC. How-ever, these methods do not suit the purpose of this project for two reasons.Firstly, the heading angle is continuously adjusted in the previous control schemes. In thedesired path development is the heading angle adjusted discretely as illustrated in Figure 3.1.The unicycle starts at position xK , yK and rotates a ‘random’ angle of θK . After that, theunicycle drives with velocity v for a fixed time T . The traveled distance is vT = v · T . Whenthe unicycle arrives at position xK+1, yK+1, the vehicle rotates a ’random’ angle θK+1. Fromthis the vehicle drives again a distance vT and this pattern will repeat itself. For this reason,the original scheme is adjusted to discrete time. In each iteration, the heading angle of thevehicle can change and in the fixed time step, a distance can be traveled.

θK

vT

θK+1

vT

x

xK, yK

y

xK+1, yK+1

xK+2, yK+2

Figure 3.1: Path development

The second thing that is not suiting the purpose of this project is the sinusoidal disturbance.Sinusoidal disturbances will lead to a deterministic path and this does not cover the randombehavior as seen at bacterial chemotaxis. Besides advantages for this project, extremumseeking with stochastic signals also has other advantages. Firstly, the chance of finding theglobal optimum increases compared to ESC with sinusoidal disturbances. Secondly, when

11

CHAPTER 3. DISCRETE, STOCHASTIC EXTREMUM SEEKING CONTROL

applying standard ESC, the path development is predictable. This is unfavorable when it is,for instance, applied on unmanned military vehicles.Therefore the use of a stochastic signal instead of a periodic signal in a discrete ESC-schemehas been studied. In [8] the convergence to the extremum on a static map is proved with andiscrete, stochastic extremum seeking algorithm.

3.2 Modeling

To mimic the behavior of E. Coli, the unicycle is modeled with a collocated sensor. Thismeans that the sensor is right above the pivoting point of the vehicle. This is the same modelas shown in Figure 2.3 but with a difference that the distance r has length 0. This systemcan be modeled as shown in the following discrete time equations:

xK+1 = xK + vT cos(θK) (3.1)

yK+1 = yK + vT sin(θK) (3.2)

For stability analysis, the equations of motion have been rewritten to polar coordinates. Asshown in Figure 3.2, the distance from the center of the unicycle to the source is defined asrK . The desired heading angle of the unicycle is defined as θ∗K .

θ∗K

θK

˙θ∗K

vT

rK

xxK

y

yK

source

Figure 3.2: System in polar coordinates

The equations of motion defined for these polar coordinates are the following:

r2K+1 = r2

K + v2T − 2rKvT cos(θK − θ∗K) (3.3)

θ∗K+1 = θ∗K − arctan( vT sin(θK − θ∗K)

rK − vT cos(θK − θ∗K)

)

(3.4)

3.3 Discrete, stochastic extremum seeking scheme

In [9], extremum seeking with stochastic disturbances on a unicycle with a collocated sensoris investigated. First an analysis in continuous time is given where the same scheme asFigure 2.2 is used, but where the sinusoidal terms are replaced by stochastic perturbations(d(t) = N(0, σ2)). For ease of analysis, the nonlinear map is supposed to be J(x, y) = J(r) =J∗

− qrr2, with r2 = x2 + y2 and the high pass filter is assumed to have a sufficient high pole

12

3.4. SIMULATION RESULTS

such that ss+h

[J(r)] can be approximated as J(r).With an averaging analysis of the system and simulations it is shown that the system will

reach the extremum and stay within a circle with radius r =√

1

2qrβσ2 around the extremum.

This system is then transformed to discrete time by the following transitions:

• the high-pass filter, which is assumed to work as an differentiator, is replaced by 1 −1

z

• the integrator is replaced by a discrete time integrator

• the ’multiplying’ disturbance is replaced by the ’adding’ disturbance with a unit delay

Figure 3.3 shows the scheme with these changes.

xK+1 = xK + vT cos(θK)

yK+1 = yK + vT sin(θK)

vT

θK

nonlinear mapJK+1

Unicycle, collocated

-βT

z−1z−1

z

dK−1dK1

zWhite noise

integrator

Figure 3.3: Discrete, stochastic extremum seeking scheme

The update law now becomes:

θK+1 = θK + βdK−1(JK − JK1) (3.5)

And with JK = J∗− qrr

2K , equation 3.5 becomes:

θK+1 = θ + βdK−1qr(r2K−1 − r2

K) (3.6)

In [9] convergence to the extremum is proved by applying the found update law 3.6 on theequations of motion of the unicycle (3.3) and (3.4) and look at the expected value.

3.4 Simulation results

The scheme shown in Figure 3.3 has been implemented in Simulink and simulations havebeen executed. Appendix B shows the Simulink file together with the m-file to specify all theparameters.Figure 3.4 shows the results of one of the simulations. In this simulation, a forward veloc-ity v = 0.033 m/s is used with a discrete time step of T = 1 sec. The feedback parameter ischosen as β = −1. The source of the nonlinear map is chosen at (x, y) = (0, 0) with a valueof J∗ = 5. The figure shows that the unicycle gets directed to the extremum, while following

13

CHAPTER 3. DISCRETE, STOCHASTIC EXTREMUM SEEKING CONTROL

a semi-random walk. The path doesn’t show periodic behavior as in Figure 2.5, 2.6 and 2.7.From the simulations and the proofs in [9] it can be concluded that this method is applicableto use as a source seeking control scheme for the unicycle.

−10 −7.5 −5 −2.5 0 2.5−10

−7.5

−5

−2.5

0

2.5

0 1000 2000 3000−200

−150

−100

−50

0

x-y plot

y-a

xis

x-axis

J=

f(x

,y)

Output

time (sec)

Figure 3.4: Simulation results of discrete, stochastic scheme

14

Chapter 4

Unicycle Design

4.1 Unicycle

In this chapter the design of the unicycle for implementation of the ESC scheme from Chap-ter 3 is explained. Because the purpose of this project is proof-of-concept of the controlscheme, the design is kept simple. It consists of a circular ground plate where all parts aremounted on. For the implementation of the controller, the Motorola MC9S12DP256 develop-ment board is available. To fit the micro-controller, sized roughly 23x11.5 cm, the diameterof the ground plate is chosen as 26 cm. Figure 4.1 shows a schematic overview of the design.The rectangular holes of 9x2.5 cm for the wheels are located 2.5 cm from the edge. On thefront, a ball caster (www.robotshop.ca : RB-Tam-11) is mounted to support the unicycle.

26 cm9 cm

2.5 cm

2 cm

Figure 4.1: Schematic overview of the unicycle

4.2 Motor

For the propulsion of the unicycle, a stepper motor and a brushed DC motor are considered.The advantage of the stepper motor is that a rotation is divided in a certain number of steps.

15

CHAPTER 4. UNICYCLE DESIGN

This makes it possible to control the motor open-loop by describing the number of steps themotor must make. The disadvantage of a stepper motor is the higher price than a brushedDC motor.

A brushed DC motor generates torque as a voltage is applied to the motor. By switching thedirection of the current, the motor can drive in two directions. The advantage of this typeof motor is the low price but the disadvantage is that it cannot be controlled open-loop. Butsince the unicycle will always be used on the same surface and under the same conditionsit is supposed that by tuning the inputs of the brushed DC motor the desired rotation andforward displacement can be achieved accurately, so no feedback control is needed to steerthe wheels.

For these reasons, there has been chosen for a brushed DC motor. The following motoris selected: Banebots 6V 278RPM 50oz-in Spur Gearmotor w/FF-050 (www.robotshop.ca :RB-Ban-49). This motor has a maximum torque of 0.355 N/m and a maximum speed of 278RPM. The datasheet can be found in Appendix C.1. From calculations on a simple modelit followed that this motor in combination with 3” tires (www.robotshop.ca : RB-Lyn-71) ispowerful enough to drive the unicycle. The tyres are connected to the motor with a specialmounting hub (www.robotshop.ca : RB-Lyn-99).

4.3 Microcontroller

The micro-controller that was available at The University of Melbourne for the implemen-tation of the control scheme is the Motorola MC9S12DP256 development board. This de-velopment board consists of a 16 bits central processing unit (HC12). The most importantfunctions for use in this project are the 8-channel pulse-width-modulator (PWM) and the two8-channel, 8/10 bits analog-to-digital converters (ADC).

PWM uses a duty cycle to control the amount of power sent. By changing the duty cycle thespeed of the motor can be controlled. The ADC converts an analog signal to a digital value.The range of the input is [0-5V] and when using 8-bits, the resolution is 5V

28 = 19mV .

The development board can be used with Real-Time Workshop via The Embedded Targetfor Motorola HC12. In this way schemes can be designed in Simulink. A Simulink diagram isconverted to C-code using Realtime workshop and Embedded Coder. Metrowerks’ Codewar-rior then converts this C-code into a real-time executable file which can be downloaded intothe HC12.The micro-controller needs a power source of 9V. For that reason, a battery pack of 6xAAbatteries is used.

4.4 Motor Driver

A motor driver is needed to amplify the drive signal from the PWM ports. The SolarboticsCompact L298 Motor Driver Kit (www.robotshop.ca RB-Sbo-24) was advised by the technicalsupport center of www.robotshop.ca. The datasheet can be found in Appendix C.2. Themotor driver has 6 inputs, to control two motors. For each motor there is an enable input

16

4.5. SENSOR

(E1 − 2 or E3 − 4) and two control inputs (I1 and I2 or I3 and I4). With the enable lines,it is possible to enable a motor to turn and with the control lines the direction and speed ofrotation can be controlled. Table 4.1 shows the logic table of the motor driver for output M1and M2. This table is the same for outputs M3 and M4.The motor driver needs a power source to drive the motors, a power source of 6V (4xDbatteries) is used.

Table 4.1: Logic table motor driverInputs Outputs

E1-2 I1 I2 M1 M2 result

H H H H H motor brakeH L L L L motor brakeH H L H L rotate direction 1H L H L H rotate direction 2L X X L L motor brake

X = Doesn’t matterH = HighL = Low

In Figure 4.2 a schematic overview of the connections between the various hardware com-ponents is depicted. The wires of the connections to the left motor are crossed because inthis way, the same input to both motors will lead to a forward movement. The layout of thesensor with op-amp is explained in section 4.5.

4.5 Sensor

To perform extremum seeking, a gradient field is needed. Various gradients fields can be used,like acoustic or thermal, but the most most practical option appeared to be a light-based gra-dient field. First, the use of infrared (IR) was studied, because this can be installed in everylocation regardless of environmental light. Unfortunately, from tests it appeared that therange of an IR LED is too short to create a big enough field. The second option was to usea lamp. The disadvantage of this method is that it requires a dark room.

First, the infrared phototransistor that is used to test the usability of IR, is used as a sensor.This phototransistor looked useful because from tests it was able to detect the gradient fieldfrom a desk lamp.In [10], an electric scheme is depicted that gives an increasing output when more light isreceived on the phototransistor. The cathode of the phototransistor is connected to a resistorand the other end of the resistor to the ground. The cathode is connected to a 5V powersupply. The output voltage is measured on the cathode. The scheme is displayed in Fig-ure 4.3. When installing this scheme onto the unicycle, the 5V power supply is taken fromthe microcontroller which is equipped with a 5V regulated output.

This scheme is implemented on breadboard and tested under a regular desk lamp. The lampwas placed on a desk in a dark room 1.20 m from the ground and the output voltages are

17

CHAPTER 4. UNICYCLE DESIGN

I1 I2 E1-2 I3 I4 E3-4

PWM

4 3 2 1 0

MrMl

motordriver

micro-controller

M1M2

M3M4

6V (4xD)

9V(6xAA)

GND00

ADC

Vout op-amp

Figure 4.2: Schematic overview of the hardware

Vout

+5VR

Figure 4.3: Schematic overview of the sensor

measured. The second column of Table 4.2 shows the output voltages of the sensor for severaldistances from the center underneath the lamp.

As can be seen in the table, the voltage range is 204.20 mV till 8.20 mV. Because a 8-bit ADCis used with a reference voltage of 5 V is the resolution 19 mV. This is to little for the ADCto distinct the difference between points far from the center. For that reason an operationalamplifier (op-amp) is installed to amplify the sensor voltages.An op-amp is an electronic element with two inputs and one output that tries to minimisethe difference between it’s two inputs by adjusting the output. A non-inverting amplifier canbe build by connecting the output to one of the inputs with a combination of resistors [11].Figure 4.4 shows the scheme that is used.

18

4.5. SENSOR

Table 4.2: sensor measurementsPosition Vout [mV] Vout [V] with op-amp

-15 135.60 2.5744-10 160.80 3.0427-5 194.00 3.63810 204.20 3.79835 189.10 3.516710 153.80 2.858415 126.10 2.371820 94.50 1.772425 61.40 1.135130 28.00 0.522235 14.30 0.256740 10.80 0.191445 10.00 0.176550 8.20 0.1529

Vout

+9V

−9V

+5V

R2R1

R

Figure 4.4: Schematic overview of the sensor with the op-amp

On the non-inverting input of the op-amp the output voltage of the phototransistor is con-nected. The output of the op-amp is connected to the inverting input of the op-amp in sucha way that the output of the op-amp amplifies the output voltage of the phototransistorwith a factor R2

R1+ 1. When taking R1 = 560R and R2 = 10K, the amplification is 18.86.

The third column of Table 4.2 shows the values of Vout. The output voltages are now be-tween 0.15 V and 3.80 V. This field looks small (diameter of approximately 1 m) in comparisonto the size of the unicycle. To solve this a visible light sensor is ordered but is not deliveredin time. If another sensor or another light source is used, the amplifier can be tuned by usingdifferent resistors.

The op-amp needs a positive and a negative power source. The positive power source comesfrom the 6 AA batteries used for the power supply of the microcontroller and the negative 9Vcomes from an additional 9V battery, by connecting the +-side of the battery to the commonground and the –side to the op-amp.

19

CHAPTER 4. UNICYCLE DESIGN

4.6 Control scheme

Figure 4.5 shows the Simulink file that is designed to implement ESC on the microcontroller.Because in one discrete time step the UAV has 3 different motions, Stateflow is used to letthe system be able switch between different states. The 3 different states are:

• rotate

• drive

• read sensor

It is desired that each discrete time step has the same length. A complexity in this is that therotation is different in every iteration. The first idea to solve this was to set a fixed amount oftime for spinning and change the rotational speed for different angles. But after implementingthis, the forces for small angles appeared to be too low to exceed the static friction. To solvethis, a fixed rotational velocity is set and the time the unicycle spins is made variable. Anextra state is added that covers the variable time that the unicycle spins.

Zero−OrderHold

z

1

Unit Delay2

z

1

Unit Delay1

z

1

Unit Delay

Sign

In1

In2

In3

In4

Robot Inputs

Rate Transition

Product

trot

direc

FRP

FRM

FLP

FLM

Motor Driver

Master

beta

Gain

K Ts

z−1

Discrete−TimeIntegrator

Convert

Data Type Conversion2

Convert

Data Type Conversion1

int8

Data Type ConversionBand−LimitedWhite Noise

|u|

Abs ADC Input

ADC Input

Figure 4.5: Simulink file with Extremum Seeking implemented

A pulse generator sends a trigger every sample time to the Stateflow chart. This sample timeis not the same as the discrete time step used in the ESC-loop. The Stateflow chart checksthe guards of the transitions between the states every trigger. Figure 4.6 shows the differentstates with the transitions and guards. By specifying the time needed to spin, rotate etc. toa number of triggers in the guards, Stateflow will shift between the states when these timesare passed.

The Stateflow chart has as input the desired time to rotate trot (in number of triggers) andthe direction direc (+1 or -1). Stateflow will start in the SPIN-state and stay there fortrot triggers, after that it will go to the WAIT-state for ttot − trot triggers. In this way,the SPIN and WAIT state will always take a total of ttot triggers. After these two states,

20

4.7. TESTING OF COMPONENTS

Stateflow will go to the DRIVE-state for tdrive triggers. And finally Stateflow will go to theSENSOR-state for tens triggers. The parameters ttot, tdrive and tens are constant, so thetotal time in one cycle stays constant.

Printed 05−Jun−2008 21:34:33

SEen:robot(SENSOR)

WAen: robot(WAIT)

DRen:robot(DRIVE)

robot(mode) function

SPen:robot(SPIN)

[after(tens,sec)]

[after(trot,sec)]

[after(tdrive,sec)][after(ttot−trot,sec)]

Figure 4.6: Main file of Stateflow motor controller

In each state, Stateflow will call the function robot(mode). Figure 4.7 shows this functionfile. Dependent on the mode specified when the function is called, Stateflow will take one ofthe five paths. Each path gives the duty cycle of the PWM outputs for the plus and minussignal to the two motors specified for that state.

The rate transition and the zero-order hold are implemented to let the Extremum Seekingscheme work at a different sample time than the Stateflow chart. The Extremum Seekingloop works at a sample time of tzoh = ttot+ tdrive+ tens,this is one discrete time step, whilethe Stateflow chart works at the sample time specified in the trigger.Because the Stateflow chart needs the difference between de new en old heading angle asinput, the previous heading angle is subtracted from the new heading angle.Data converters are added to transform the signals into 8 bits signals. The PWM outputsand the ADC input both work with 8 bits, so it is preferable if all signals are of the samedata type.

In Appendix D a guide trough the converting and downloading the Simulink scheme onto themicrocontroller is given.

4.7 Testing of components

The Stateflow chart is uploaded on the microcontroller to perform open-loop testing. Testpaths are offered to the open loop scheme and from these tests it can be concluded thatthe switching of the Stateflow chart works properly. The PWM duty cycles specified in the

21

CHAPTER 4. UNICYCLE DESIGN

Printed 05−Jun−2008 21:31:28

robot(mode) function

[mode==SPIN && direc>0]{FRP=uint8(0);FRM=PWM_des_spin;FLP=PWM_des_spin;FLM=uint8(0);action=1}

1

[mode==SPIN && direc<=0]{FRP=PWM_des_spin;FRM=uint8(0);FLP=uint8(0);FLM=PWM_des_spin;action=1}

2

[mode==WAIT]{FRP=uint8(255);FRM=uint8(255);FLP=uint8(255);FLM=uint8(255);action=2}

3

[mode==SENSOR]{FRP=uint8(255);FRM=uint8(255);FLP=uint8(255);FLM=uint8(255);action=4}

5

[mode==DRIVE]{FRP=uint8(255);FRM=PWM_des_speed;FLP=uint8(255);FLM=PWM_des_speed;action=3}

4

Figure 4.7: Function file of Stateflow motor controller

function file of Stateflow have been tuned on several test paths to assure that the unicyclefollows the specified path.After the testing of the Stateflow chart, the connection between the sensor and the ADC istested. The IR sensor is connected to the op-amp on breadboard and connected to the ADCof the microcontroller. The ADC is tested by downloading a scheme on the microcontrollerwhere the ADC input is connected to a PWM output. The PWM output is then measuredwith an oscilloscope and in this way, it is concluded that the ADC is working properly.

These two tests show that the components are working properly and that the unicycle is ableto perform the discrete path development.

4.8 Prototype and experiments

An early prototype of the designed unicycle is depicted in Figure 4.8. This prototype is furtherimproved by installing proper battery holders for the motor driver and the microcontroller.The motor driver is mounted on the bottom of the unicycle. Unfortunately, there are nopictures of the final version of the unicycle.

Because the visible light sensor was not delivered in time, the IR sensor is installed on theunicycle to perform experiments. The unicycle is moving with a random walk under the lampbut it is not clear if the unicycle is really seeking the source or just moving around, becausethe measured distribution field with the IR sensor is not big enough.

22

4.8. PROTOTYPE AND EXPERIMENTS

Figure 4.8: Early prototype of the unicycle

23

CHAPTER 4. UNICYCLE DESIGN

24

Chapter 5

Conclusion and recommendations

5.1 Conclusion

Extremum seeking control is used in literature to perform source seeking with unicycles. Re-sults from these schemes show figures where vehicles are moving in triangular or star-shapedpatterns to the source or a pattern more similar to bloodhounds or sharks searching for a trail.The methods used in literature are limited to continuous time and sinusoidal disturbances.

To simulate path development of bacteria like E.Coli, the schemes need to be adapted to dis-crete time and working with stochastic disturbances instead of periodic disturbances. Fromsimulations we have concluded that with these adapted scheme it is possible to mimic bacte-rial chemotaxis with unicycles.

A unicycle is designed and build for proof-of-concept testing of the discrete stochastic ex-tremum seeking control. The control scheme is implemented in Simulink. The discrete move-ment of the unicycle is achieved by using Stateflow, which made it possible to divide themovement of the unicycle in several modes. This scheme is tested open loop by prescribinga path and this showed good results. The gradient field is made by using a lamp in a darkroom. First an IR sensor is used to measure the value of the field, but it appeared that themeasured distribution field with an IR sensor is relatively small in comparison to the size ofthe unicycle. A visible light sensor is ordered but was not delivered in time.

Experiments are performed with the IR sensor and the unicycle was moving with a randomwalk method under the lamp but it was not clear if the unicycle wass performing source seek-ing or just moving around in a random walk, since the distribution field is not well measured.

5.2 Recommendations

Firstly, to finalize the unicycle, the visible light sensor needs to be installed. Probably, thenon-inverting amplifier needs to be adjusted with different resistors to amplify the voltagesbetween 0 and 5 Volts. With this new sensor, the extremum seeking control parameters canbe tuned to let the unicycle seek the source under the desk lamp.

25

CHAPTER 5. CONCLUSION AND RECOMMENDATIONS

For further research, a couple of extensions can be made:

• If the unicycle is able to track a fixed source, it is interesting to see whether the unicycleis able to track a moving source.

• Further ut is interesting to investigate if the unicycle is able to track a certain level setof the non-linear map. This means that the unicycle will only follow a certain value ofthe non-linear map. In Cochran [6], this is achieved in continuous time and sinusoidaldisturbances by slightly adjusting the scheme.

• In the scheme used in this report, the forward movement is kept constant and therotational movement is tuned. An option that has not been explored, is to tune bothmovements. This might lead to a faster and more precise conversion towards the source.

26

Bibliography

[1] H. C. Berg, “Chemotaxis in bacteria,” Annual Review of Biophysics andBioengineering, vol. 4, no. 1, pp. 119–136, 1975. [Online]. Available:http://arjournals.annualreviews.org/doi/abs/10.1146/annurev.bb.04.060175.001003

[2] “What is chemotaxis?” May 2009. [Online]. Available:http://www.ccbi.cam.ac.uk/iGEM2005/index.php/Chemotactic Patterning

[3] N. J. Killingsworth and M. Krstic, “PID tuning using extremum seeking,” Control Sys-tems Magazine, vol. 26, pp. 70–79, Feb. 2006.

[4] K. B. Ariyur and M. Krstic, Real-Time Optimization by Extremum-Seeking Control.Wiley-Interscience, 2003.

[5] C. Zhang, D. Arnold, N. Ghods, A. Siranosian, and M. Krstic, “Source seeking with non-holonomic unicycle without position measurement and with tuning of forward velocity,”Systems & Control Letters, vol. 56, p. 245 252, 2007.

[6] J. Cochran and M. Krstic, “Source seeking with a nonholonomic unicycle without positionmeasurements and with tuning of angular velocity - part i: Stability analysis,” Decisionand Control, 2007 46th IEEE Conference on, pp. 6009–6016, Dec. 2007.

[7] ——, “Source seeking with a nonholonomic unicycle without position measurements andwith tuning of angular velocity - part ii: Applications,” Decision and Control, 2007 46thIEEE Conference on, pp. 1951–1956, Dec. 2007.

[8] C. Manzie and M. Krstic, “Discrete time extremum seeking using stochastic perturba-tions,” pp. 3096–3101, Dec. 2007.

[9] C. Manzie, J. Cochran, and M. Krstic, “Source seeking using discrete extremum seekingand a collocated sensor,” draft.

[10] Application Note Optoelectronics: Photodiode/Phototransistor Application Circuit,Sharp.

[11] Application Note 31: Op Amp Circuit Collection, National Semiconductor Corporation,2002.

27

BIBLIOGRAPHY

28

Appendix A

Simulink file for tuning forward

velocity

w0

thetadot

s

s+h

Transfer Fcn

t

To Workspace3

xs

To Workspace2

ys

To Workspace1

J

To Workspacev

theta

xs

ys

Subsystem

Sine Wave1 Sine Wave

Product

1s

Integrator1

beta

Gain

f(u)

Fcn2

Clock

Figure A.1: Simulink file for tuning the forward velocity

The m-file to execute the simulation is depicted below:

%%% parameters unicycle

r = 0.2; % distance sensor from center

%%% initial condition position center of unicycle

x_c0 = 2;

y_c0 = 2;

theta_0 = pi;

%%% initial condition position sensor

x_s0 = x_c0 + r* cos(theta_0);

y_s0 = y_c0 + r* sin(theta_0);

%%% parameters non-linear map

29

APPENDIX A. SIMULINK FILE FOR TUNING FORWARD VELOCITY

%%% J = Jmax-qx*(x-xmax)^2-qy*(y-ymax)^2

Jmax = 1;

xmax = 0;

ymax = 0;

qx = 0.15;

qy = 0.05;

%%% parameters ES scheme

h = 1; % pole of high pass filter

w = 25; % frequency ’disturbance’

a = .1; % amplitude ’disturbance’

beta = 20; % feedback parameter

k = 5; % ratio disturbance frequency and angular velocity

w0 = w/k; % angular velocity

%%% parameters solver

time = 100; % time simulated

tstep = 0.01; % step size solver

%%% execute simulation

sim unicycle_tune_forward_velocity

30

Appendix B

Simulink file for tuning angular

velocity stochastic and discrete

non−linear map

f(u)

XY Graph1

XY Graph

Unit Delay1

z

1

Unit Delay

z

1

To Workspace2

Jout

To Workspace1

xy

Product1Gain

beta

Fcn4

vT*sin(u[1])

Fcn3

vT*cos(u[1])

Discrete−TimeIntegrator3

K Ts

z−1

Discrete−TimeIntegrator2

K Ts

z−1

Discrete−TimeIntegrator1

K Ts

z−1Clock

Band−LimitedWhite Noise1

x x

z

y

y

theta

theta

theta

theta

Figure B.1: Simulink file for discrete and stochastic tuning of the angular velocity

The m-file to execute the simulation is depicted below:

%%% initial conditions

initg = 45/180*pi; % initial angle theta

initx = -10; % initial x position

inity = -10; % initial y position

%%% parameters non-linear map

%%% J = Jmax - qr*sqrt(x^2+y^2);

Jmax = 5; % value of nonlinear map at extremum

qr = 1; % steepness field

31

APPENDIX B. SIMULINK FILE FOR TUNING ANGULAR VELOCITY STOCHASTICAND DISCRETE

IC = qx*(initx-xmax)^2+qy*(inity-ymax)^2+zmax; % initial value non-linear map

%%% parameters band-limited white noise

wnd = pi/4; % noise power

seed = ceil(20000.*rand); % seed of white noise (each seed corresponds with a fixed output...

% so now every simulations has a different white noise)

%%% parameters ES scheme

beta = 1; % feedback parameter

vT = .1/3; % distance traveled in drive mode

%%% parameters solver

time = 3500; % time simulated

T = 1; % fixed stepsize

%%% execute simulation

sim unicycle_stochastic_discrete

32

Appendix C

Datasheets of used hardware

C.1 Datasheet of the Banebots motor

Figure C.1: Datasheet of the Banebots motor

33

APPENDIX C. DATASHEETS OF USED HARDWARE

C.2 Datasheet of the Solarbotics Motor Driver Kit

Figure C.2: Datasheet of the Solarbotics Motor Driver Kit

34

Appendix D

Executing Simulink file on HC12

The m-file to create all the necessary variables is depicted below:

hc12_base_st = hc12_closest_st(0.01,16000000); % sample time, use this...

% function to find the closest sample time...

% to the desired sample time that is implementable

% in the HC12

%%% Duty rate of PWM is (255 - value)/255

PWM_des_speed = uint8(100); % PWM value for FRM FLM in drive mode,...

% PWM value for FRP and FLP is 255

PWM_des_spin = uint8(208); % PWM value for FRM FLM in spin mode,...

% PWM value for FRP and FLP is 255

%%% variables used in Stateflow scheme

tdrive = (100*hc12_base_st); % time for drive mode [sec]

ttot = (250*hc12_base_st); % maximum time for spinmode [sec]

tens = (50*hc12_base_st); % time for reading out sensor [sec]

tzoh = tdrive+ttot+tens; % time for one stateflow cycle...

%(so this is the ’sampletime’ in the ES-loop) [sec]

tdrive = uint8(tdrive/hc12_base_st); % time for drive mode (in # samples)

ttot = uint8(ttot/hc12_base_st); % maximum time for spinmode (in # samples)

tens = uint8(tens/hc12_base_st); % time for reading out sensor (in # samples)

In the first line, the function hc12_closest_st.m is used to find the closest possible sample time toour desired sample time that is implementable on the HC12. This is necessary because it isn’t possibleto implement any sample time without looking at the specifications of the HC12.

The variables that are created further are all for the Stateflow-chart. In Stateflow, you need tospecify which data type the variables and parameters are and because the PWM-ports need 8-bitsignals, all data types are set at unsigned 8-bit. This means that all variables/parameters can haveinteger values from 0 till 255. The only variable that had a different data type is the input direc whichis a signed 8-bit. This is because this value is always -1 or 1 and so the unsigned 8-bit would not beable to represent this.

To convert the Simulink file with Real-Time Workshop (RTW) to C-code, the Simulink file needsto be build. This can be done by pressing Ctrl+B. Make sure that the m-file to create all the variablesis executed before the building process. RTW will produce some files and after a while the program

35

APPENDIX D. EXECUTING SIMULINK FILE ON HC12

Metrowerks CodeWarrior (CW) is started. This program can convert the C-code to a real-time exe-cutable file and download it on the HC12.

Make sure to connect the debug cable to the BDM IN port on the microcontroller and plug thepower in. By pressing the Debug button in CW as depicted in Figure D.1 the C-code gets convertedand downloaded on the HC12. By pressing the start-button in the screen that pops up (True-TimeSimulator & Real-Time Debugger), the code will be executed on the microcontroller.

Figure D.1: CodeWarrior screen

36