Following a line using the cameraof the e-puck robot

Course project

Alexis Baron, Cyril Wendl, Caroline Menard

30 mai 2016

Introduction Methodology Implementation Robustness and portability Conclusion

Table of contents

1. Introduction

2. Methodology

3. Implementation3.1 Line following (ST FOLLOWLINE)3.2 Obstacle avoidance (ST LOVER)3.3 Line finding (ST FIND)3.4 Interaction between the states

4. Robustness and portability

5. Conclusion

Introduction Methodology Implementation Robustness and portability Conclusion

Introduction : Goals

I Follow line

I Avoid obstacles

I Find line

I ...For different types of lines

I In real and simulated environment

I With different robots

Introduction Methodology Implementation Robustness and portability Conclusion

Introduction : Sensors

I CameraI Simulation: 52 × 39 RGBI Real: 640 × 480 panchromatic, downsampled to 40 × 40

I 8 IR sensors, used for proximity measurements

Introduction Methodology Implementation Robustness and portability Conclusion

State machine

”A (finite) state machine is a model of system behaviour whichcan consist of states, state transitions and actions.” [1]

1. Reading sensor values (proximity and camera), being updatedcontinuously in an infinite while-loop;

2. Determination of the e-puck’s state depending on the sensorvalues;

3. Execution of the part of code belonging to each state.

Introduction Methodology Implementation Robustness and portability Conclusion

Implementation strategy: state machine

Figure 1: State machine

Introduction Methodology Implementation Robustness and portability Conclusion

Line following (ST FOLLOWLINE)

Introduction Methodology Implementation Robustness and portability Conclusion

Line following: Camera values

I In the simulation: simply R=G=B=0

I In reality: fluctuation values, coded on 8 bits (28 = 256possible values, ranging from 0 to 255), perfect white or blackonly in case of dysfunctional pixel→ Threshold value for black / white pixels

Introduction Methodology Implementation Robustness and portability Conclusion

Line following: Camera values

5 10 15 20 25 30 35 40

5

10

15

20

25

30

35

40

(a) Original image

5 10 15 20 25 30 35 40

5

10

15

20

25

30

35

40

(b) Filtered image

Figure 2: E-puck camera output

Introduction Methodology Implementation Robustness and portability Conclusion

Line followingI Bottom 5 lines used (reduce fluctuations / noise)I Differential wheel speeds depending on the number of black

pixels on either side

Listing 1: Line following

1 case ST_FOLLOWLINE:

2 dp=((double)c_l_5)/((double)c_r_5+(double)c_l_5+1.0); //

check if more black pixels left or right

3 printf("DP: %f\n",dp);

4 if(dp>0.5){// more left pixels, turn left

5 left_speed = NORM_SPEED+NORM_SPEED*(1-dp);

6 right_speed = 2*NORM_SPEED-NORM_SPEED*(1-dp);

7 }

8 else{ // more right pixels, turn right

9 left_speed = 2*NORM_SPEED-NORM_SPEED*dp;

10 right_speed = NORM_SPEED+NORM_SPEED*dp;

11 }

12 follower++;

13 break;

Introduction Methodology Implementation Robustness and portability Conclusion

Obstacle avoidance (ST LOVER)

Introduction Methodology Implementation Robustness and portability Conclusion

Proximity sensors

Figure 3: E-puck IR sensors (moodle.epfl.ch)

Listing 2: Proximity sensor weights

1 prox_left = (3*prox[7] + 5*prox[6] + prox[5])/9;

2 prox_right = (3*prox[0] + 5*prox[1] + prox[2])/9;

Introduction Methodology Implementation Robustness and portability Conclusion

Obstacle avoidance: implementation

Listing 3: Obstacle avoidance

1 if(prox_right>prox_left){//obstacle on the right side

2 ds = NORM_SPEED * (prox_right / THRESH_PROX);

3 left_speed = 2*NORM_SPEED - ds;

4 right_speed = ds;

5 }

Obstacle

E-puck

Proximity treshold THRESH_PROX

State treshold THRESH_ST_LOVER 200

500

prox_rightdp > NORM_SPEED

dp < NORM_SPEED

Introduction Methodology Implementation Robustness and portability Conclusion

Line finding (ST FIND)

Introduction Methodology Implementation Robustness and portability Conclusion

Line finding: implementation

Listing 4: Line finding

1 case ST_FIND:

2 ...

3 if(prox_right>prox_left){//turn

left

4 ds = NORM_SPEED * (prox_right /

THRESH_PROX);

5 left_speed = NORM_SPEED - ds;

6 right_speed = NORM_SPEED + ds;

7 }

8 ...

9 }Figure 4: Find behaviour [2]

Introduction Methodology Implementation Robustness and portability Conclusion

Interaction between the states

Introduction Methodology Implementation Robustness and portability Conclusion

Listing 5: Interaction between the states

1 // Determine states

2 // if line is seen (5 last pixel lines) and no obstacle,

follow line

3 if((c_l_5>LINEPIX_MIN || c_r_5>LINEPIX_MIN) && (c_l_5<

LINEPIX_MAX && c_r_5<LINEPIX_MAX)){// line detected

4 state=ST_FOLLOWLINE;

5 count_lover = 0;

6 } // if obstacle detected, go around it for at least 70

iterations

7 else if((prox_right>TRESH_ST_LOVER || prox_left>

TRESH_ST_LOVER) && count_lover<70){// obstacle detected

8 state=ST_LOVER;

9 }

10 else if (count_lover>70){

11 state=ST_FIND;

12 }

Introduction Methodology Implementation Robustness and portability Conclusion

Robustness and portability

Introduction Methodology Implementation Robustness and portability Conclusion

Robustness and portability: camera

I CameraI Varying values, according to luminance of the surroundings and

e-puck used → Robust threshold value for black / white pixelsI Actual physical position of the camera in the e-puck

I Too high / straight: FOV too far awayI Too tilted: Looks inside its own casingI Looking to the left / right changes FOV accordingly, can

make it difficult to follow sharp curves in one direction

I Reading black pixels: robot also detects black pixels if in frontof shady object → LINEPIX MAX

→ Mainly related to the real e-puck

Introduction Methodology Implementation Robustness and portability Conclusion

Robustness and portability: camera

(a) Implemented shapes (b) Shape to improve

Figure 5: Tested line shapes

Introduction Methodology Implementation Robustness and portability Conclusion

Robustness and portability: proximity sensors

I Sometimes lag: due to low weight of front sensors (0 and 7)

I Slight differences between both sides left / right despite equalweighting → correction factor

I Possible interference with other emitters of IR, such asincandescent lamp, heat

Introduction Methodology Implementation Robustness and portability Conclusion

Further developing

I More rigorous / detailed reporting of success rates

I Test more shapes and more e-pucks

I Calibration of e-pucks via separate script

I Compare used method to Fourier processing of the image

I Use only two front proximity sensors for faster state change

I Initialize state (outside of while loop) to ST FOLLOWLINE orST FIND depending on line recognition

Introduction Methodology Implementation Robustness and portability Conclusion

Conclusion

I Good group work

I Important lessons about embedded systems, memoryconstraints and real-time computing learnt

I Debugging and code isolation as a good strategy for anefficient implementation of the code (→ state machine)

Introduction Methodology Implementation Robustness and portability Conclusion

References

Beat Hirsbrunner and Fulvio Frapolli. Endliche Automaten - FiniteState Machine (FSM). German. 2007. url:https://diuf.unifr.ch/pai/education/2007_2008/ca/

lectures/ln07_FSM.pdf (visited on 06/03/2016).

Valentino Braitenberg. Vehicles - Experiments in SyntheticPsychology. Cambridge, Massachusetts and London, England: TheMIT Press, 1984.

Introduction Methodology Implementation Robustness and portability Conclusion

Thank you for your attention!

Introduction Methodology Implementation Robustness and portability Conclusion