2D LADAR based localization - umu.se€¦ · Arsalan Siddiqui Master’s Thesis, 20 credits...

1

2D LADAR based localization

Searching for best applicable methodology

Arsalan Siddiqui

Master’s Thesis, 20 credits

Department of Computing Science

Umeå University SE-90187 Umeå, Sweden

Submitted to the Department of Computing Science at Umeå University in partial fulfillment of the requirements for the degree of Master of Science in Computing Science. Thesis Defense: June 10th, 2005, MIT building. Thesis Supervisor: Dr. Thomas Hellström, Department of Computer Science, Umeå University, Sweden. Examiner: Dr. Per Lindström.

3

Abstract

The use of Laser Range Finder or ladar in localization of mobile robots is increasing since last decade. One of the reasons of its popularity is that the ladar is faster and performs simpler modeling of surrounding environment than its counter part video camera. The aim of this thesis work is to test different 2D ladar based methodologies in order to find the best applicable methodology for 2D ladar based localization in unknown environments. In the beginning of thesis work, experimental evaluations of some ladar characteristics were performed. Later on, feature and other ladar based localization techniques were tested. We found that the pixel based localization methodology proposed by Jorma Selkäinaho in his PhD. desertion is simpler and faster. We further tested this methodology in different scenarios and discussed its drawbacks. A solution to minimize these drawbacks has been proposed. Experimental results show that the proposed algorithm is faster and seems to be more reliable than the original one. Keywords: localization and navigation in robotics, ladar based vision and computational geometry.

5

Acknowledgements This work has been performed under IFOR Research group, Umeå University. I would like to acknowledge my supervisor Dr. Thomas Hellström for giving me opportunity to work in my interested field and for his full support throughout the work. I am also grateful to other members of IFOR group for their support and corporation from time to time specially to Kalle Prorok and Olla Rindahl.

I have a special thanks to Dr. Per Lindström for his help in discussions and facilities. I would like to thank all the teachers and my friends who have given time to me for practical discussions and encourage me during the thesis work. I owe a debt of gratitude to my family members who have been a moral support to me.

7

Table of Contents Abstract......................................................................................................2 Acknowledgements ...................................................................................3 List of figures ............................................................................................6 Introduction ............................................................................................13 1.1 Latest research level in Intelligent Robotics .....................................................13 1.2 Thesis work description .....................................................................................14

1.2.1 Outline of thesis work ................................................................................................15 1.2.2 Why I have chosen this thesis? ……………………………………………………..10

A view on LADAR...................................................................................17 2.1 What is Laser Range Finder or LADAR ?.........................................................17 2.2 Operating principle and some basic terminologies regarding LADARs .........17

2.2.1 Operating principle.................................................................................................... 2.2.2 Scanning angle ......................................................................................................18 2.2.3 Angular resolution.................................................................................................18

2.3 Choosing the ladar LMS-221 for thesis work....................................................19 2.4 Features of LMS-221 [LMS Tech, 2004] ..........................................................19 2.5 Applications of LMS-221 ...................................................................................20

2.4.1 Avoiding collisions between industrial machines .................................................20 2.4.2 Uses in Autonomous vehicles ...............................................................................20 2.4.3 Some other applicable scenarios ...........................................................................21

2.6 Accuracy Problem due to ‘Specular Reflection’ ...............................................22 Experimental setup for thesis work .......................................................24 3.1 Introduction .......................................................................................................24 3.2 The driver software............................................................................................25 3.3 Software Description .........................................................................................25

3.3.1 Important functionalities .............................................................................................26 3.2.2 Process flow ................................................................................................................26

3.3 Summary ............................................................................................................28

8

Experimental evaluation of LADAR (LMS-221) characteristics.........29 4.1 Experiment type 1: To check illumination effects and sharp edged natural object effects on the readings of LMS-221 .........................................................................29

4.1.1 Objectives....................................................................................................................29 4.1.2 Working ......................................................................................................................29 4.1.3 Conclusions from the experiment..........................................................................31

4.2 Experiment type 2: To determine and/or verify some important parameters of the LADAR LMS-221 .....................................................................................................32

4.2.1 Objectives....................................................................................................................32 4.2.2 Working ......................................................................................................................32 4.2.3 Results.........................................................................................................................32 4.2.4 Conclusions.................................................................................................................34

4.3 Experiment type 3: To find the accuracy of measuring bushes.........................34 4.3.1 Objectives....................................................................................................................34 4.3.2 Working & Analysis ...................................................................................................34 4.3.3 Conclusions.................................................................................................................40

4.4 Experiment type 4: To find the accuracy of stand alone trees with different distances..................................................................................................................................40

4.4.1 Objectives....................................................................................................................40 4.4.2 Working and Analysis.................................................................................................40 4.4.3 Conclusion ..................................................................................................................42

4.5 Experiment type 5: To find the effects of snow on the ladar readings ..............42 4.5.1 Objectives....................................................................................................................42 4.5.2 Working & Analysis ...................................................................................................42 4.5.3 Conclusions.................................................................................................................45

4.6. Experiment type 6: To see the LADAR graph with different inclined and declined angles .......................................................................................................................45

4.6.1 Objectives....................................................................................................................45 4.6.2 Working & Analysis ...................................................................................................45 4.6.3 Conclusions.................................................................................................................51

4.7 Problems faced during the experiments ............................................................51 4.8 Summary and conclusion...................................................................................51 Pattern Matching and Localization: researching for best methodology52 5.1 What is Pattern Matching ?...............................................................................52 5.2 Localization and its relationship with Pattern Matching..................................52 5.3 Patterns from the LADAR LMS-221..................................................................53 5.4 Feature Based Pattern Matching ......................................................................54

5.4.1 In LADARs ...........................................................................................................54 5.4.2 In the case of LMS-221 .........................................................................................54 5.4.3 Conclusion of the Analysis ...................................................................................60

9

5.4.4 Moving towards more feasible approach ..............................................................61 5.5 Pixel Based Pattern Matching and Localization...............................................61

5.5.1 Selection of methodology ...........................................................................................61 5.5.2 Detailed Description of considered methodology.......................................................62

5.6 Testing and Comparisons ..................................................................................67 5.6.1 Test-bed and Working.................................................................................................67 5.6.2 Results and Analysis ...................................................................................................67

5.7 Example of localization using proposed methodology......................................73 5.8 Conclusion .........................................................................................................76 Conclusions & Future work...................................................................77 6.1 In Furtherance of pixel or non feature based proposed methodology.............77

References ...............................................................................................79

10

List of figures

Fig 2.1: Some laser scanners from SICK Company....................................................................................17 Fig 2.2: Measuring the time of reflecting ray .............................................................................................17 Fig 2.3: 100° and 180° scanning angles .....................................................................................................18 Fig 2.4: Visualizing concept of angular resolution.....................................................................................18 Fig 2.5: Preventing collision in container cranes.......................................................................................20 Fig 2.6: Testing LMS-221 for localization..................................................................................................21 Fig 2.7 a-d: Some important applicable uses of LMS-221 ....................................................................17-22 Fig 3.1: Experimental setup ........................................................................................................................24 Fig 3.2: Snapshot of the developed driver software....................................................................................25 Fig 3.3: Process flow of the software..........................................................................................................27 Fig 4.11: Close view of the object ...............................................................................................................29 Fig 4.12: Snapshot of the object in normal day light conditions ................................................................30 Fig 4.13: Apparatus snapshot .....................................................................................................................30 Fig 4.14: LADAR output in different illumination conditions ....................................................................31 Fig 4.21: Snapshots of working and some objects used in the experiment .................................................32 Fig 4.22: A graph between distances and their respected minimum objects ..............................................33 Fig 4.31: Different kinds of bushes which are considered during the experiment .....................................35 Fig 4.32: Bush related graphs from 1m till 30m distances .........................................................................38 Fig 4.33: Bush graphs in snow from 2m .....................................................................................................39 Fig 4.34: Bush graphs in snow from 5m .....................................................................................................40 Fig 4.41: A stand alone tree from two different views ................................................................................41 Fig 4.42: Displaying the apparatus and some working during the experiment..........................................41 Fig 4.51: Snow heap and its LADAR 2D sketch..........................................................................................43 Fig 4.52: Considered Snow shape 2 and its graph .....................................................................................43 Fig 4.53: Snow shapes 3 and 4 and their LADAR graphs ..........................................................................44 Fig 4.61: Actual scene and the scaled LADAR graph when laser rays are at 0° (straight) with ground...46 Fig 4.62: Scenery and the LADAR graph when laser rays are 10º inclined towards ground ....................47 Fig 4.63: Scenery and the LADAR graph when laser rays are 10 degrees inclined from the ground........48 Fig 4.64: Photographic image and the LADAR graph when laser rays are 20 degrees inclined from the ground .........................................................................................................................................49 Fig 4.65: Scenery and the LADAR graph when laser rays are 30 degrees inclined from the ground........50 Fig 5.1: Photographic image of an airplane and its extracted pattern ......................................................52 Fig 5.2: A photograph of a bush and its 2D sketch by the LADAR LMS-221.............................................53 Fig 5.3: The Snapshot of the tested environment of the graph sketched in figure 5.4 ................................54 Fig 5.4: 2D LADAR graphs of the environment with 180 degree scanning angle and 0.5 degree resolution......................................................................................................................................55 Fig 5.5: 2D LADAR graphs with 100 degree scanning angle and 0.5 degree angular resolution.............56 Fig 5.6: Displaying available points within 30m range in 100 degree scanning angle .............................56 Fig 5.7-11: Subpatterns comparisons from figure 5.6 ................................................................................56 Fig 5.12a: Coordinates of robot pose .........................................................................................................62 Fig 5.12b: Change in robot pose ................................................................................................................62 Fig 5.13: Searching the current position in a circle around the last position ............................................66 Fig 5.14: Path with equidistant points ........................................................................................................67

11

Fig 5.15: Method M1 and M2 output matching ratio comparison .............................................................69 Fig 5.16: Comparison of matching ratios with changing degrees in pose (-50º to 50º) of graphs at point P2.................................................................................................................................................71 Fig 5.17a: Locations of reference and current position for the results shown in figure 5.17b ..................71 Fig 5.17b: Bar chart of matching ratio with variation of Φ (-50º to 50º) in robot pose(x, y, Φ) of graphs at Euclidean distance of 448.24cm between the reference and current positions ....................72 Fig 5.18: Locations of reference and current pose .....................................................................................74 Fig 5.19: LADAR patterns at reference and current positions ...................................................................74 Fig 5.20: Patterns at reference position and at translated and rotated position using proposed method .75

13

Chapter 1

Introduction The thesis work is in the field of ‘Robotics’. The subfields can be localization and navigation of mobile robots, computer vision based on ladar and computational geometry. What is Intelligent Robotics (IR)? To understand the term ‘Intelligent Robotics’, one should firstly understand what is ‘Robotics’. In simple way, we can say that robotics is a field in which we play with robots and robots are the mechanical structures that are designed to do one or more specified set of tasks. The more the scenario of set of tasks is spread, the more the robot comes towards ‘generality’. Intelligent Robotics is the field in which we research and implement the ‘intelligent working’ into the robots. Often we observe nature and try to find how human beings and other natural artifacts do intelligent behavior. Through these observations, we derive principles and theories so that the robot is able to act, sense and plan in an intelligent way. 1.1 Regarding latest research in Intelligent Robotics The focus of one major paradigm in Intelligent Robotics is on physical embodiment. In this paradigm, research is centered mostly on lower forms of biological life that act as an inspiration for the design of robot control systems by exploring reactive and natural learning systems as well as evolutionary and reinforcement learning methodologies. We are also giving more attention to perceptual techniques such as active vision and task-oriented perception.{explain about them} Research on autonomous robotics is also concentrating on how to execute higher cognitive and organizational activities such as communication between learning, planning and reactive layers and division of roles and tasks between them. One good example of extracting behavior from natural artifacts is ‘The Innate Releasing Mechanisms’ (IRC). Some of the most well known areas of research in Intelligent Robotics are

i. Learning. ii. Localization / Navigation.

iii. Knowledge Representation. iv. Planning and Problem Solving. v. Introducing senses in Robots. (hear, see, smell etc.)

14

1.2 Thesis work description

We have tried to find the best way for localization of an autonomous vehicle in an unknown environment using 2D ladar. For detailed description regarding ladars and their uses, please refer to chapter 2 of this report. The reasons of preferring 2D ladar instead of 3D ladar are

� The solution will involve less computational resources so it will be simpler and less

computationally complex. � The solution would be faster and easier to implement. � 2D ladars are cheaper than 3D ladars therefore the solution would be less expensive

The thesis research work has been performed for one of the projects of IFOR research group, Umeå University. In this project, a researched has been performed to develop unmanned forest machines. We are planning to perform localization from the ladar as a backup localization strategy to differential GPS (Global Positioning System) based localization. By differential GPS, we mean that more than one GPS receivers will be used to increase the accuracy of GPS based localization. The use of differential GPS is increasing since last five years. Some systems have already been developed in which localization of heavy vehicles have been performed solely based on differential GPS. [Alexander et. al, 1999] During our search for finding best methodology for localization using 2D ladar, we have firstly tried feature based map matching. In feature based matching, features or characteristics of environment are extracted from raw data and the robot is localized by appropriate matching of features. We can also identify and classify surrounding objects in this approach. [Vandapel et. al, 2003], [Vandapel et. al, 2004]. Feature based pattern matching is more successful when we use 3D ladars because it is easier to extract features from raw 3D ladar based data than 2D ladar based data. Due to some reasons, later we shifted from feature based map matching to pixel based map matching. These reasons are explained in chapter 5. In pixel based pattern matching, the ladar output is considered as a set of points and we localizes robot by point to point matching of reference and current ladar based patterns. The important point is that we don’t extract any landmarks or features from the data. We have found that [Sälkainaho, 2002] has proposed an efficient pixel based localization methodology in his PhD. dissertation. It is simple, fast and flexible and it has produced good results (see tables 5.1-5.3). This method is based on zero order image distortion model (IDM). By zero order, we mean that the method is neglecting all dependencies of the pixels displacement [RWTH Keysers et al, 2004 ]. There are other pixel based methods but they are not neglecting all dependencies of pixels or points displacement. We have found that [Sälkainaho, 2002] method has big room for improvement. It can be made more general and applicable. We have changed some equations in this methodology and improved the results (see chapter 5 for details).

15

1.2.1 Outline of the thesis work The outline of the work completed in this thesis is as follows 1. Study the work completed by IFOR Research group: In order to work with the research group, I should know the work done by the group till now so I have studied the work done by them. 2. Write drivers for the laser scanner LMS221: We have planned to use LMS221 ladar for localization, pattern matching and map building so firstly I have to write customizable drivers for the Laser Scanner . I have chosen java for writing the drivers because

o It is platform independent. o It has very good performance on network. According to the test cases results of IFOR

group, it is more faster than C on the network applications. 3. Test the Laser Scanner indoors: With Kalle Prorok (a member of IFOR Research group member), test the Laser Scanner in indoor environment regarding

o What can be seen o What can not be seen o Other limitations

4. Test the Laser Scanner outdoors:

o What can be seen o What can not be seen o Other limitations o Collect data o Make chart about the reflectivity ratio with different objects

5. Localization using Laser Scanner:

o Research on algorithms for pattern matching and localization, finding the best possible applicable solution

o Testify and improve the algorithm.

16

Why I have chosen this thesis work? There are many reasons for me for selecting this thesis work. Some important reasons are given below.

� Laser Sensors for range finding and localization is a developing research area in Robotics these days. I wanted to work on the application of laser sensors in robotics field and testify different ideas.

� I have a chance to work with a research group. This thesis is tough and involve more work but I

have had been interested in doing research work in Robotics

� It is not only theoretical research, it includes core system programming and practical implementation of algorithms so this thesis could be much learning and helpful towards my goal which is to do something new and creative in the field of ‘Robotics’.

17

Chapter 2

A view on LADAR 2.1 What is Laser Range Finder or LADAR ? A device which uses laser to scan and find the ranges of the objects is Laser Range Finder or ladar. We have different kinds of Laser Scanners available in the market. As an example two different kinds of Laser Scanners are shown in fig 2.1

Fig 2.1: Some laser scanners from SICK Company [LMS Tech, 2004]

2.2 Operating Principle and some basic terminologies regarding LADARs 2.2.1 Operating Principle What is the Operating Principle of Laser Range Finder or LADAR? Laser Range Finders get the distance of the object by measuring the ‘time of flight’ which is the time taken by a laser ray to reach and reflect back from the object. The ladars send the laser rays and then measure the ‘time of flight’.

Fig 2.2: Measuring the time of reflecting ray

0t

∆t

18

In figure 2.4, the first pulse i.e. the blue one represents the original pulse and the time of flight taken by it is denoted by ∆t. The time of flight is directly proportional to the distance from scanner to the object.

T α D where T = Time of flight D = Distance from the laser range finder to the object which reflects back the laser ray. There are two basic terms which will be frequently used when one works with ladars. 2.2.2 Scanning angle

Angle of view at which a laser scanner can scan is known as Scanning Angle.

Fig 2.3: 100° and 180° scanning angles

In fig. 2.2, there are two different scanning angles, 100° and 180°. These two are most common scanning angles but it is possible to have other values of scanning angles.

2.2.3 Angular resolution

A difference in angles at which a laser scanner is shooting laser rays is called Angular Resolution. E.g. If a laser scanner is shooting laser rays with difference of 0.5 degrees then its angular resolution will be 0.5 degrees.

r1

r2r3 r4

r5

Angular resolution

Fig 2.4: Visualizing concept of angular resolution

19

In Fig 2.3, the concept of angular resolution is visualized. It is the shortest angle at which ladar can project the laser rays. Here, r1, r2, r3, r4 and r5 are the laser rays and angular resolution is the angle between r1 and r2 or the angle between any two other consecutive rays. 2.3 Choosing the ladar LMS-221 for thesis research work We have used LMS-221 due to following reasons.

� LMS-221 has good reputation in the market so it seems to be more reliable than other Laser Range Finders.

� As my thesis work is a part of IFOR Research group work and LMS-221 is recommended by IFOR for the autonomous vehicle project.

� LMS-221 can provide 180 degree scanning angle with angular resolution of 0.5 degree and the other features that we need for research work on autonomous vehicles.

LMS-221 [LMS Tech, 2004]

2.4 Features of LMS-221 [LMS Tech, 2004] It is a non-contact optical measurement system that can work even at over longer distances.

o Has wavelength of 902nm. o Can be operable over two different scanning angles. 180 and 100 degree scanning angle. o Has rapid scanning times therefore the objects can be detected at high speeds. o No special target-object reflective properties necessary. o Backgrounds and surroundings have negligible influence on the measurements.

The objects of measurement can be at any position and pose within the reliable range. The observations of measurement are available in polar coordinates and can be used for controlling tasks or for any other further processing.

o No illumination conditions are required for the measurements. o It has simple mounting and commissioning. o Simple interface provided with computers or micro controllers.

20

o Fog correction for outdoor use. o Rain drops and snow fall can be filtered at hardware level.

2.5 Applications of LMS-221

In this section, we have described some important practical uses of LMS-221. Some of the following uses are already implemented and some are still in research.

2.5.1 Avoiding collisions between industrial machines LMS-221 can help in preventing the collisions between industrial machines. Using highly accurate ladars like LMS-221, industrial cranes are now able to work pretty near to each other.

Fig 2.5: Preventing collision in container cranes [LMS Tech, 2004]

2.5.2 Uses in Autonomous vehicles Ladar is quite applicable in using for localization and map building in Autonomous vehicles. Ladar based localization is better than localization based on image from camera because we have enormous computational requirements in using image data. The use of two-dimensional laser range scans has also been proposed and demonstrated successfully [Cox et Wilfong, 1990]. Ladar has also been used in number of research projects for Terrain Classification in Autonomous Navigation [Hebert et al., 2003], [Kelly et al., 2004]. In this thesis which is a part of a research project of IFOR group, we have used LMS-221 in Localization and Navigation of Autonomous Forest Machine as shown in figure 2.6

21

Fig 2.6: Testing LMS-221 for localization

2.5.3 Some other applicable scenarios Similarly, LMS-221 can also be used in the following scenarios

Fig 2.7 a: Measuring the bulk materials

Fig 2.7b: Guarding building by monitoring. c: Classifying the vehicles from their shapes

22

d: Parking the cars accurately in automated multi storey car parks

Fig 2.7 a-d: Some important applicable uses of LMS-221 [LMS Tech, 2004] Some of the applicable scenarios have been shown in above fig.2.7. Among all research related applications, LMS-221 is most widely used in Autonomous vehicles because of the following reasons.

i) Less computational complexity: It gives out the distances to the objects directly, if we use other sensor like camera for getting these distances then it will involve much computational complexity.

ii) High accuracy in measuring distances: It provides good accuracy level for measurement by using laser technology.

iii) Less susceptible to environment: Environment has very less effects on LMS-221 readings. It has also fog correction option at hardware level.

2.6 Accuracy Problem due to ‘Specular Reflection’ There are no other important practical issues in measuring surrounding objects from the ladar except sometimes we get wrong distances due to specular reflection of ladar rays. In technical manual and other SICK company specifications of LMS-221, there is no description about any problem due to which laser scanner could not be able to read properly. We have founded this problem during experiments. The positive point is that we can computationally construct surfaces even if we get continuous specular reflection from objects. One of the well applicable algorithms in this regard, has been discussed by [Halstead et. al., 1996] where 3D surfaces have been constructed by using Spline Surface Fitting of Normals. In ladar, we do not get continuous specular reflection of rays only we get specular reflection at some part of some specific curved surfaces. Specular reflection phenomena occurs when the ladar ray hits some fine curved surface of an spherical shape as shown in figure 2.8

23

If one observe the pattern of measurements resulted from Laser Scanner then he can finds out the degrees on which rays are specularly reflected. One of the examples of the sequence having specular reflection is as follows

32(cm), 33, 32, 34, 37, 8152, 8152, 36, 35 (Readings taken from LMS-221)

In above readings, we can observe that two ranges from middle are 8152cm, these ranges are specularly reflected ranges because of very fine spherical curve of the object.

Specular reflected ray

Laser rays

Object

Fig 2.8: Visualizing problem of specular reflection of laser rays

24

Chapter 3

Experimental Setup for thesis research work

3.1 Introduction For experiments, ladar LMS-221 had been mounted on some trolley. For details of the ladar, please refer to chapter 2. Laptop, level checker instrument and other stationary stuff had also been fixed on the trolley as shown in figure below.

Fig 3.1: Experimental setup

In figure 3.1, the experimental setup is shown. Here, ladar and radar experiments are going on simultaneously. There is a rectangular box on the top of ladar. It is ground level checker. It has been used to verify the level of projection of laser rays.

25

3.2 The driver software During initial experiments, a driver software with customizable functions had been developed. Firstly, we had developed simple drivers for LMS-221 then we had developed GUI with customized functions. In next step we improved the software to get optimal performance. Customized functions were demanded on the basis of experimental work that had been planned to do with the ladar LMS-221. We had received a driver software from the SICK company(manufacturer of LMS-221) with the ladar then why we needed to develop our own customized software? We were needed to develop driver software because

� We were needed to have some customizable options which are not present in given driver software from the SICK company

� We need to have a java program that we can merge with other existing sensor softwares in java for autonomous forest machine.

Labor cost: This software development cost 245 man hours i.e. my one and half month full time work. 3.3 Software Description Our software contains two different menus in which one is mainly for connection, reset and setting baudrate. The other one is regarding functionalities like sketching output patterns, setting maximum and minimum for graphs to view patterns, etc as shown in figure. 3.2. Each menu has its own separate window.

Fig 3.2: Snapshot of the developed driver software

26

There are two different windows shown in fig 3.1. Window 1 is for connection related operations like reset, baudrate etc. and window 2 is for advanced operations like viewing runtime graph, saving data in log file etc. 3.3.1 Important functionalities Some important functionalities of the software are as follows

Setting Baud rate – User can select the baud rate for connection and he has also an option to change the baud rate while the program is connected.

Save data in batch file – User can save the runtime readings in batch file with counter, time and date

stamps. View runtime graph – User can view runtime graph with hardware supported resolution and with

different display options. 3.3.2 Process flow The software process of communication and operation with ladar is divided into steps accordingly. In figure below, the steps and their respected functions are shown orderly.

27

Fig 3.3: Process flow of the software

From figure 3.2, the software flow process is divided into two main steps. In GUI (Graphic User Interface), these two main steps are displayed in separate windows for the feasibility of user (as shown in fig 3.1). Each of two main steps consists of some sub-steps. Main step I: This is the first step, here we set baudrate and then try to get connection with ladar. Only after completing connection (main step I), user can go to main step II. Main step II: All the operation related functionalities are present in this step like setting resolutions for ladar graphs, range for x and y axis in the graph, name and location of batch file etc. From fig 2.2, we can check the main steps and their corresponding sub-steps. During runtime, default values are set in most of the sub-steps.

Start

Set baudrate

Open port

Select resolution

Select batch file storing / view graph options

Set filename, file location and other

display options

Start reading

Connection ( Main step I )

Operation ( Main step II )

28

3.4 Conclusion Due to the reasons described in the section # 3.2 of this chapter, we needed to develop customized software for our work. This customized software had helped us a lot in performing kinds of experiments. It can communicate with the ladar according to the requirements and can work with high refreshing rate in displaying runtime graphs. Although software development took some time but it helped a lot in performing experiments faster, and getting filtered data. Overall it is proved as a wise decision to develop customized driver software.

29

Chapter 4

Experimental evaluation of LADAR (LMS-221) characteristics

To evaluate the important practical characteristics of the ladar LMS-221, six different kinds of experiments have been performed as per needed. This evaluation is necessary in order to minimize the unreliability and errors from the sensor. These experiments are scenario based but their results can be applicable to any standard ladar which has wavelength around 902nm.

4.1 Experiment type 1: To check illumination effects and sharp edged natural object effects on the readings of LMS-221 4.1.1 Objectives The main objectives of this experiment are

I. What is the Illumination effect on ladar readings? II. As plant leafs are sharp so we want to check that whether these leafs can produce the problem of

specular reflection with the ladar readings. 4.1.2 Working As an object, a plant with sharp edged leafs is chosen. We have chosen this object because it is easily available and it is suitable according to the experiment’s objectives.

Fig 4.11: Close view of the object

30

The above displayed figure 1.1 shows a closer view of the object in which one can see the thin stem of the plant. In this experiment, we have also tried to measure the width of this thin stem by the ladar. To test the illumination conditions, two working environments are considered

Environment 1: In this scenario, we have normal lighting conditions

Fig 4.12: Snapshot of the object in normal day light conditions (environment 1).

Environment 2: It is an environment without any light i.e an completely dark environment. Simple apparatus is used for working in this experiment that consist of the ladar, Laptop, inch tape and some stationery as shown in below fig. 4.13

Fig 4.13: Apparatus snapshot

After completion of recording readings in Environment 1(normal day light scenario) and Environment 2(completely dark), the readings have been analyzed and different parameters have been calculated like measured distances, average S.D, maximum S.D, width etc. The results are shown in table 4.1. Description regarding table 4.11: In first column of the table, we have the actual distance to the object from the ladar. In the fifth column, actual width of the stem is listed. This width is shown by a circle in figure 4.11. We have a small range in column 2, this range shows the ladar readings for the object (fig 4.11). As the object is a curve surface so a range of points on the object are shown by the ladar.

31

Table 4.1: Readings from two different illumination conditions

It can be observed from the readings that the object is a bit far in dark environment but everytime the difference in readings is no more than 2 cm and 2cm is so small value that it could be an experimental error so this point is not so important. The output 2D graph of both environments is as follows.

(a) LADAR graph in normal light conditions (fig1.1a) (b) LADAR graph in complete dark (fig1.1b)

Fig 4.14: LADAR output in different illumination conditions By comparing the range values, it is observed that the ladar readings do not depend on the illumination conditions. This fact can also be observed from fig 4.14, the two graphs are more than 95% similar to each other. 4.1.3 Conclusions from the experiment

� The maximum difference between the readings from different illumination conditions is 2 cm . This difference could be experimental error hence we can conclude that illumination has no effect on the ladar readings.

� The plant has pretty sharp leafs and these leafs have not produce any specular reflection problem. In the light of the results from current and the other different experiments with natural landmarks, it is concluded that Laser Range Finder will almost not give any specular reflection on the naturally sharp edged objects like plant leafs etc.

� As expected, ladar is measuring the distance of the objects accurately but one can’t say anything regarding the width accuracy because the focused object has very small width and fifty percent of size of width is varying.

READINGS FROM ENVIRONMENT 1 (fig 4.11) Actual minimum distance to the object shown in

fig. 4.11 (m)

Measured Distance

by the ladar (m)

Average S.D (40+ readings)

(m)

Max. error in dist. (m)

Actual width of the stem (fig 4.11)

(cm)

Measured width of the stem

(cm)

Average error

in width (cm)

Max error in width (cm)

0.97 0.97-1.02 0.03 0.05 2.8-4.0 1.78-3.46 0.50(17%) 1.0 (36%)

READINGS FROM ENVIRONMENT 2

0.97 0.99 – 1.03 0.035 0.06 2.8-4.0 2.03-2.65 0.36 (17%) 0.77 (38%)

32

4.2 Experiment type 2: To determine and/or verify some important parameters

of the LADAR LMS-221

4.2.1 Objectives The objectives behind this experiment is to determine and/or verify

I. The height and width of the laser ray of ladar LMS221 till maximum available range (30 m) II. Minimum measurable objects till maximum available range.

4.2.2 Working The ruler has been used to notify the height of the laser ray at the point that is very near to laser ray emission from the LRF. We have firstly calculated the actual height at 4cm distance from the emission of the laser ray that is the nearest possible distance. We have moved the ruler up to down (downward direction) and detected the point (height) at which the ladar started detecting the ruler then repeated the same procedure is repeated from down to up (upward direction).

a) Finding the height of the laser ray at emission b) Objects : 1cm and 12cm size

Fig 4.21: Snapshots of working and some objects used in the experiment

To measure the minimum object with respect to distance, we had considered different objects during the experiment. In starting, an object of 1cm was used but after increasing some distance this object had become invisible then static objects of different width, had been used. 4.2.3 Results The results obtained from the experiment are shown in table 4.2. In the first column, the distance at which the minimum measurable object is noted is listed. In the second column, laser ray spacing with respect to distance is listed.

33

Angular resolution of the ladar: 0.5° ; Wavelength: 902nm Distance (d) (cm)

Laser ray spacing (cm)

From [LMS Tech] Minimum measurable

object (cm)

4 0.105 ± 0.01 1.0 100 1.25 ± 1.0 1.0 200 2.0 ± 1.0 1.2 ± 0.1 500 4.8 ± 1.0 2.5 ± 0.1 1000 8.0 ± 1.0 4.0 ± 0.5 2000 17.0 ± 1.0 6.5 ± 1.0 3000 20.5 ± 1.0 12 ± 1.0

Table 4.2: Table displaying the laser ray spacing and minimum measurable object with respect to distance

From the readings of table 4.2 , we can notify that the minimum object width is from 1cm till 12cm depending on how distant we are from the LRF. The plot between minimum object and the distances for relationship is as follows.

Fig 4.22: A graph between distances and their respected minimum objects

From fig 4.22, ladar LMS-221 can see the minimum object width of 12cm till 30 m range therefore the object of 12 cm can be considered as standard minimum object for SICK LMS-221. For the height of the laser ray of LMS-221, we have noticed that it is almost same with the change in distance so it is concluded that the laser of LMS-221 is not a beam. By the ladar, we can see concrete objects like trees till 35 meters but we can’t see the bush after 30 meters so the standard available range is 30m.

34

4.2.4 Conclusions The ladar parameters confirmed from this experiments are

� Laser beam of the ladar has a constant height of 1.5 – 2.5 cm regardless of the distance it travels. � Standard available range is 30m. � Minimum object that can be measured by the ladar is from 1cm to 12cm depending on the

distance. At 30m, the minimum object width is 12cm. 4.3 Experiment type 3: To find the accuracy of measuring bushes 4.3.1 Objectives

I. How accurately, the ladar measures the distance from bush. It is possible that it measure the bush more far away because, laser rays can reflect inside the bush

II. To note the change in shape of the object with change in the distance. This will determine that whether the bushes are reliable landmarks.

III. To find the effects of snow on the ladar readings when bush is partly covered with snow. 4.3.2 Working & Analysis In order to get general results of the ladar accuracy regarding bushes, we have evaluated three different kinds of bushes and estimated the error. These different kinds of bushes are shown in figure 4.31

a) Bush type 1 b) Bush type 2

35

c) Bush type 3

Fig 4.31: Different kinds of bushes which are considered during the experiment

The ladar has been placed at different distances(1m to 30m ) from the bush and accuracy of its distance measurement has been checked. Standard deviation and percentage of relative error in measuring the distance from the bush has been calculated. The evaluation of more than 34 readings has been performed in order to get generalized results. We have not found much difference in readings among the different kind of bushes. The maximum difference is 20cm that can be ignorable therefore we can present a general table for the bushes. In table 4.3, different measured distances by the ladar are listed. These distances are measured to different bushes.

Table 4.3: shows the distance measured by the ladar and relative error. From table 4.3, the error till 30 meters range is up to 3 percent of the actual distance so we can write a general equation for getting more accurate distance

D = dm - ∆x (4.1) where dm = distance measured by the ladar ∆x = k ×××× dm (4.2)

Actual nearest distance (meters)

Measured dist. by the ladar (m)

Error / S.D (more than 34 readings

are evaluated) (m)

Percentage of error (%)

1.2 ± 0.1 1.19 0.01 0.8 2.2 ± 0.1 2.23 0.03 1.4 5.2 ± 0.1 5.19 0.01 0.2 9.8 ± 0.1 10.08 0.28 2.9 19.7 ± 0.1 20.07 0.37 2.0 29.4 ± 0.1 30.19 0.79 2.7

36

k =

≅≅≅

<

30m d:027.020m d:020.010m d:029.0

10m d:0

m

m

m

m

(4.3)

There are errors in measuring distances less than 10m (table 4.1) but they are so small so we can ignore them. Due to this reason, the ‘k’ constant is valued 0 in equation 4.3 when distance is less than 10m. From table 4.3, it can be observed that the ladar is slightly overestimating the distance after 10 meters. This overestimating can be due to the fact that the probability that laser rays penetrates into the bush increases with increase in distance. Change in shape with distance To observe the change in shape with distance, we are displaying the 2D sketches of the object from the front view, drawn by the ladar from different distances. The distance range is from 1m to 30m.

Graph drawn by LADAR from 1 meter

LADAR Graph from 2 meters

37

LADAR Graph from 5 meters

LADAR Graph for the bush from 10m

The bush from 20m

38

Graph of the bush from 30m

Fig 4.32: Bush related graphs from 1m till 30m distances

By observing the graphs of fig. 4.32, one can notify the significance change of object shapes by changing of the distance. In landmark detection perspective, the distance from this kind of object is very important parameter and after the certain distance, object can be of completely different shape which has no matching with previous shape. The effect of snow To check the effect of snow on bush graphs by the ladar, we have taken two different bushes and checked their output ladar graphs and readings from different distances. Some figures in this regard are as follows. From 2m distance:

Bush 1 LADAR graph

39

Bush 2 LADAR graph

Fig 4.33: Bush graphs in snow from 2m

From 5m distance:

Bush 1 LADAR graph

40

Bush 2 LADAR graph

Fig 4.34: Bush graphs in snow from 5m

If we compare the graphs of fig 4.32, fig 4.33 and fig 4.34, one can observe that there is no difference in measurement of the ladar. As we were in doubt that the laser rays may be reflected by the snow .If this phenomenon happens then we can have very inaccurate readings but that hypothesis has proved to be wrong. It can be observed from the ladar graphs of fig 4.32 to fig 4.34 that the ladar readings are not affected by the snow. 4.3.3 Conclusions 1. The ladar is almost accurately measuring the bushes but if we analyze readings very carefully then it is measuring bushes a bit far from the actual distance. An equation has been derived in section # 4.3.2 which can help in calculating more accurate distances 2. The shape of the ‘bush‘ is quite changing with the distance so it is not a good landmark. There exist other more concrete objects natural landmarks like trees that have much less change in shape with respect to distance. 3. Snow has almost no effect on accuracy of the readings by the ladar LMS-221 when we are measuring distances to bushes.

4.4 Experiment type 4: Measuring stand alone trees at different distances

4.4.1 Objectives To find the measuring accuracy for trees up to 30m distance. In this experiment, we actually like to check the accuracy of this kind of natural landmark. 4.4.2 Working and Analysis In this experiment, we have chosen the category of different kind of stand alone trees. Following pictures are examples of considered objects from two different views.

41

(a) Object’s view from 7-9m (b) Object’s Closer view

Fig 4.41: A stand alone tree from two different views Firstly, we have measured the actual distance using measuring tape then we have taken the readings from the ladar. During the experiment, the distance variation is from 2 to 35 meters. For each distance, more than 30 readings have been taken in order to get general results. In analysis, we have calculated the error and standard deviation of the readings with respect to actual distance.

Fig 4.42: Displaying the apparatus and some working during the experiment

As shown in fig 4.42, we have used simple apparatus during the experiment. The person shown in the figure is measuring the actual distance from the ladar to the object with the help of measuring tape. As the stem of tree is circled by its branches and leaves therefore we have a range of ladar readings for the tree. We got the range because some laser rays hit and returned from the front part of the tree but some of them penetrated into inside and gave the distances of ending branches or stem of the tree therefore we expressed the distances in form of ranges in below table 4.4. These distances are measured by the ladar from the tree shown in figure 4.41.

42

Distance from stem(m)

Distance range of the tree

Measured distance range (by LMS-221 )

(m) 2 1.3– 2.0 m 1.51-1.78

4 3.0-5.0 m 3. 61-4.81

5 3.80-6.5 m 4.69 – 5.70

10 8.8-11.5 m 10.16-11.34

15 13.8-16.6 m 15.77 – 16.86

20 18.8 -21.6 m 20.22 – 21.66

25 23.8-26.6 m 24.92 – 26.81

30 28.8-31.6 m 30.39 – 31.28

35 33.8-36.6 m 36.09 – 36.96

Table 4.4: Reading from the ladar or LRF From table 4.4, it can be observed that the errors in measuring the tree are so small. Hence we can measure these kinds of objects from far away distances with high accuracy. 4.4.3 Conclusion For the trees, the ladar has good measuring accuracy level till 35m. Due to their high accurate measurements by the ladar, the shapes of these kinds of objects can be reliable landmarks for pattern matching and localization. 4.5 Experiment type 5: To find the effects of snow on the ladar readings

4.5.1 Objectives Although we have done some experiments to check the effect of snow on bushes but it is needed to check the effects of snow thoroughly on the performance of the ladar LMS-221 (wavelength = 902nm) 4.5.2 Working & Analysis We have taken four different perspectives and checked the effect of snow on the ladar readings. We have not found any other different perspective so we can say that these perspectives cover all possible snow surfaces and objects in the scenario of working in natural terrain. The snapshot of first perspective with the ladar graph is as follows.

43

a) Considered shape b) LADAR graph

Fig 4.51: Snow heap and its LADAR 2D sketch

In fig 4.51a, the ladar has sketched out zigzag snow heap. The ladar is around 4m away from snow heap and it is projecting laser rays on the middle of the heap. It is possible that the laser can just be reflected from the snow to some where else and do not give the accurate measurements of the curve but the results are very good and the ladar has measured correctly the snow curvature as shown in fig 4.51 b The second considered surface is as follows

a) Considered shape b) LADAR graph

Fig 4.52: Considered Snow shape 2 and its graph

In fig 4.52, we have considered a smooth snow ground and detected the ladar accuracy. The ladar is placed from approximately 3m away from snow. The ladar has measured this surface almost accurately with very small errors. These errors are so small that they can be considered as negligible.

44

Similarly, we have also checked following surfaces

a) Considered snow shape 3 and its LADAR graph

b) Considered snow shape 4 and its LADAR graph

Fig 4.53: Snow shapes 3 and 4 and their LADAR graphs

In figure 4.53a, there is a slope shape ground of snow is taken from 10 m distance. We can also observe the slope shape in the ladar graph. We have a cone shaped heap at approximately 4m away from the ladar in figure 4.53b, if we see the part of shape near its top then this part is a plane from front and triangular from sides and in the same way, the shape is measured by the ladar. Measuring accurately in any kind of snow fall/ rain: An option of object blanking is present at the hardware level of the ladar LMS-221. In this parameter, one can set the width of the tiny objects that are needed to be blanked in ladar scan. This parameter is especially built to block dense snowfall or raining in the ladar scan therefore it is possible that we can get the environment scan from the ladar LMS-221 regardless of the density of snowfall or rain. Further work in this regard will be done as per needed.

45

4.5.3 Conclusions We have considered as many different surfaces of snow as possible to check the accuracy of the ladar in this regard. The ladar has measured all surfaces in the same way as it measures the surfaces without snow therefore we can confidently say that the ladar (LMS221) can work accurately with covered snow surfaces and objects. By using the feature of object blanking, one can also use this ladar in heavy snow falls. From the results, we can also conclude that it is possible to measure the width of the different snow shapes by the ladar.

4.6. Experiment type 6: To see the LADAR graph with different inclination angles

4.6.1 Objectives As the ladar LMS-221 is a 2D ladar so we were interested to see its graphs with different inclination angles at the same position. Initially we have planned to consider angles between ±30 degrees but the ladar starts sketching just a plane ground from –10 degrees therefore it is decided to view the ladar graphs from +30 degree to –10 degree with 10 degree increment. 4.6.2 Working & Analysis: We have chosen one natural scene that consists of some trees and one pole. The snapshot of the scene and the ladar graph when the laser rays are at 0 degree with the ground, are as follows.

Fig 4.61a) Photographic image of the tested environment

46

b) LADAR graph of the scene shown in the part a

Fig 4.61: Actual scene and the scaled LADAR graph when laser rays are at 0° (straight) with ground We can see stems of the trees and the pole in the ladar graph in fig 4.61b. The nearest tree is at 7.25m on the y-axis. This tree can also be seen in the middle of the fig 4.61a. In fig 4.61b, there is sharp measurement of the pole because it is circular, smooth and has small diameter. In the next figure, we are going to display the graph and picture with minus ten degrees inclination

47

a) Scenery with minus 10 degree inclination

b) LADAR graph of the scene

Fig 4.62: Scenery and the LADAR graph when laser rays are 10º inclined towards ground

In the fig 4.62, one can see the ice ground graph. In the first quadrant of the graph, ground is closer as compare to the second quadrant because ice surface is higher on the right side of the ladar. Now, we are going to see the snapshot and the corresponding ladar graph which is pointing upwards with ten degrees inclination

a) Scenery with 10 degrees inclination

48

b) LADAR graph of the scene

Fig 4.63: Scenery and the LADAR graph when laser rays are 10 degrees inclined from the ground

We can see the branches of the trees in the ladar graph of fig 4.63b. As the trees are wider from up therefore the distance from the closest branches is less than the distance from the stem. Now, we are going to analyze the ladar graph with 20º inclination angle

a) Scenery with 20 degree inclination

49

b) LADAR graph of the scene(1small box in grid = 50cm) Fig 4.64: Photographic image and the LADAR graph when laser rays are 20 degrees inclined from the ground In fig 4.64b, we can see smoother curve in the ladar graph compared to the curve in fig 4.63b. As the trees (from fig 4.64a) are of long height so still at plus 20, we are seeing the upper middle branches of the trees. Now, the snapshots with 30º inclination angle are as follows.

50

Fig 4.65 a) Scenery with 30 degree inclination

b) LADAR graph from the scene

Fig 4.65: Scenery and the LADAR graph when laser rays are 30 degrees inclined from the ground In fig 4.65, we can see the curve of upper branches of the trees in fig 4.65a. It is also a smother curve as compare to the curve drawn by the ladar at 10º inclination angle in fig 4.63b. We can also notice that now the ladar is so inclined that it is not measuring the pole in figure 4.65a.

51

4.6.3 Conclusions The reason of this experiment is not to find any specific parameter or limitation of the ladar but to have a good and clear overview of the ladar LMS221 graphs at different inclination angles at the angular resolution of 0.5º and the scanning angle of 180º. 4.7 Problems faced during the experiments The major problems that we faced during experiments are as follows

– Since the experiments are done in natural environment, we faced a big problem regarding level of ground which is varying with the changing distance between object and the ladar therefore the same part of object is difficult to be measured with 2D ladar.

– In winter, the laptop batteries discharges more quickly so sometimes during the experiment laptop got out of power and we had to stop the experiment at the middle.

4.8 Conclusions The phase of evaluation of ladar characteristics took a long time but it was necessary for the future work of pattern matching and localization. Ladar LMS-221 is the main sensor and if the characteristics of the ladar are not confirmed then it would be difficult to do successful pattern matching. We have completed six different kinds of experiments in this phase and from analysis of theirs results we have learned many important characteristics and behaviors of the ladar which are not described in product technical manual. Conclusively, the important evaluations from the experiments of this phase are as follows.

I. The illumination conditions have no effect on the ladar readings. II. The ladar do not gets any specular reflection from sharp edged natural objects if they are more than

1 meter apart from the ladar III. Laser beam of the ladar has a constant height of 1.5 – 2.5 cm regardless of the distance it travels. IV. Reliable range for the ladar for measurements is 30m V. Minimum object that can be measured by the ladar is from 1cm to 12cm depending on the distance.

The minimum measurable object is 12cm at 30 m. VI. Bushes can not be considered as reliable landmarks because their shapes are widely changing with

the distances but we can use tree as a reliable landmark because their shapes are not changing much with the change in distances.

VII. Snow has almost no effect on the ladar readings We think that these conclusions will also be valid for the other ladars if their laser wavelength is around 902nm.

52

Chapter 5

Pattern Matching and Localization: researching for best methodology

5.1 What is Pattern Matching? The process of measuring similarity or difference between two or more patterns is called Pattern Matching. Pattern can be of many different kinds depending on the tool from which the pattern has been sketched. E.g.: If the pattern is taken from the camera then this kind of pattern will be a photographic image etc. One photographic image and the extracted pattern is shown in figure 5.1

Fig 5.1: Photographic image of an airplane and its extracted pattern

[Shape Matching U.U. Multimedia Research] 5.2 Localization and its relationship with Pattern Matching The process of localizing or getting position of a mobile robot with respect to the environment is known as ‘localization’. The mobile robot can be an autonomous vehicle or any kind of other robot. Pattern matching plays a key in localizing robot’s location. We need to match its current environment with some kind of reference map in order to identify robot’s current location. Localization is not possible without some kind of pattern matching of current environment with reference environment. It is possible to do pattern matching even if the environment consists of free form shapes [Vergeest et. al, 2002] therefore localization is possible even if the robot’s environment consist of free form shapes. We can also map cluttered indoor and outdoor environments and localize mobile robots [Minguez, 2002], [Christensen, 2004]

53

5.3 Patterns from the LADAR LMS-221 During this master thesis work, localization has been performed based on patterns from the ladar LMS-221 so we would like to describe the ladar patterns now. There is a photographic image in figure 5.1 of an aero plane and extracted pattern which is represented by a set of points. The pattern given by the ladar LMS-221 is also a set of points or angles and ranges. This set can be represented as a sketch of an environment in 2D frame of reference as shown in figure 5.2

Fig 5.2: A photograph of a bush and its 2D sketch by the LADAR LMS-221

In figure 5.2, we have a 2D pattern drawn by the ladar LMS-221. Here, one can checks that how the ladar draws patterns in 2D frame of reference. The output of the ladar is actually a set of points or a set of projection angles of laser rays with respective ranges from the ladar. By connecting those points, we can get a 2D graph or pattern of the environment. To match two or more patterns, one can apply many different procedures. All these procedures can be mainly divided into two main categories.

i) Feature based matching. ii) Pixel based matching.

We have tested both approaches and proposed a pixel based matching solution for 2D pattern matching in outdoor environments. This solution is proposed because of getting better results in the following parameters.

i) Complexity and Running time. ii) Reliability or Correctness iii) Ease of usage.

We have discussed in detail regarding our proposed solution and its test results in later sections. Now, we are starting the discussion about methodologies of pattern matching from Feature based matching

54

5.4 Feature Based Pattern Matching In this approach, we extract the features of the objects that are present in the environment around the mobile robot. We match two or more graphs on the basis of extracted features from objects. This method is more successful in 3D images where one can precisely extract the features of the objects. 5.4.1 In LADARs When ladar output is in 3D frame of references, we can extract more precise, more accurate and less distance dependent features of the objects. In [Forsman, 20001], a method is presented for 3D pattern matching which is based non feature based matching and can give an accuracy of 20cm. The downside of this method is that it takes fairly large running time. In [Hebert et. al, 2004], they have successfully classified 3D vegetarian terrain from ladar. They have extracted the objects from their features. These objects include trees, wire, ground etc. Feature based matching approach in not much successful when the ladar outputs are in 2D frame of references. This is due to the fact that the shape of an object varies much with the change of pose in most of the scenarios. The change in pose can be the change in distance or the change in pitch or roll angle from the previous pose. In 2D ladars, the pattern of an object is also specific with the parameter of height at which ladar is mounted. With the change in height, the pattern may be changed completely because some objects are different at bottom and different at the top. 5.4.2 In the case of LMS-221

We have tested feature based matching with the ladar LMS-221. Firstly, we have recorded a scan of the environment by using the ladar which is at specific position then we move 10cm backward and recorded another scan. When we have compared the two graphs, some major differences have been found among them.

Fig 5.3: The Snapshot of the tested environment of the graph sketched in figure 5.4

55

In above figure 5.3, the tested environment is shown. It is an outdoor environment where 95 % objects are natural that consist of different kinds of trees and bushes. The ladar graphs of this environment are as follows.

-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1

x 104

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

x coordinates of object positions

y co

ordi

nate

s of

obj

ect p

ositi

ons

At position1At position2

Fig 5.4: 2D LADAR graphs of the environment with 180 degree scanning angle and 0.5 degree resolution

Square marks represent the ladar output from the first or reference pose Cross marks represent the ladar output from the second pose (10cm backward)

In figure 5.4, we have two ladar based patterns. The square marked pattern represents the ladar output at first or reference pose. The cross marked pattern shows the ladar output at second pose which at 10cm distance from the first pose. 5.4.2.1 Analysis of Patterns

From figure 5.4, it can be observed that at least ten new points are coming at the left and right ends with the small difference of 10cm so we have decided to cut off the extreme right and left sides and go for further analysis on 100 degree scanning angle instead of whole 180 degree scanning angle. This step has been taken in order to reduce the errors. It is also decided that the ranges which are after 30m will be chopped because according to previous tests, the ladar output after 30 meters range is not reliable. Please refer to section # 4.2 for details.

cm

(cm)

56

-8000 -6000 -4000 -2000 0 2000 4000 6000 80000

1000

2000

3000

4000

5000

6000

7000

8000

9000


y co

ordi

nate

s of

obj

ect p

ositi

ons


Fig 5.5: 2D LADAR graphs with 100 degree scanning angle and 0.5 degree angular resolution

Chopping ranges beyond 30m

-3000 -2000 -1000 0 1000 2000 30000

500

1000

1500

2000

2500

3000


y co

ordi

nate

s of

obj

ect p

osito

ns


Fig 5.6: Displaying available points within 30m range in 100 degree scanning angle

cm

(cm)

(cm)

57

In figure 5.5, we have ladar output ranges at 100 degree scanning angle. In next step, the ranges above 30 meters are chopped (fig 5.6) and only the reliable ranges are acquired. Now, we can proceed to analyze the sets of reliable ranges in order to extract available features. From figure 5.6, it can be observed that the chopped graph can be further divided into four or five sub patterns and these patterns seems to be independent of each other so we have divided the main graph into subsets of sub patterns. After dividing, we have tried to find the similarity between the sub patterns of the two slightly different poses i.e. the pose marked by square (first pose) and the pose marked by cross(second pose) in figure 5.6. It can also be observed from figure 5.6 that the last pattern from right of the second pose main pattern does not exist in the patterns of first pose. As expected, with the change of pose we have got some new patterns which were not present in the patterns from reference pose. Since these patterns have no corresponding match so we can consider them as a ‘noise’ and exclude them. Hence the patterns that will be considered for comparisons will be derived by the intersection of the sub pattern sets at given poses.

P = {SP1} ∩ {SP2} ----------- (5.1) P : Patterns derived for comparison SP1 : Sub patterns at pose 1. In considered example, pose 1 is the reference pose SP2 : Sub patterns at pose 2. Here it is the new pose with 10cm backward distance.

58

Now we are moving to analyze the shapes of available sub patterns.

-3000 -2000 -1000 0 1000 2000 30000

500

1000

1500

2000

2500

3000


y co

ordi

nate

s of

obj

ect p

osito

ns


650 700 750 800 850 900 9501250

1300

1350

1400

1450

1500

1550

1600

1650

1700

550 600 650 700 750 8001400

1450

1500

1550

1600

1650

580 600 620 640 660 680 700 720 740 760 7801500

1550

1600

1650

1700

1750

1800

450 500 550 600 650 7001790

1800

1810

1820

1830

1840

1850

1860

1870

Fig 5.7a: Pattern 1 from reference position Fig 5.7b: Pattern 1 from new position


(cm)

Pattern 1 32 4 & 5

59

-135 -130 -125 -120 -115 -110 -105 -100 -95 -901310

1320

1330

1340

1350

1360

1370

1380

-220 -215 -210 -205 -200 -195 -190 -1851320

1330

1340

1350

1360

1370

1380

-760 -740 -720 -700 -680 -660 -6401685

1690

1695

1700

1705

1710

1715

1720

-830 -820 -810 -800 -790 -780 -770 -760 -750 -7401570

1580

1590

1600

1610

1620

1630

1640

1650

1660

1670

-1900 -1850 -1800 -1750 -1700 -1650 -1600 -1550 -15001520

1540

1560

1580

1600

1620

1640

1660

1680

1700

-1820 -1800 -1780 -1760 -1740 -1720 -17001500

1550

1600

1650

1700

1750

In above figures from fig 5.7 –11, we have displayed side by side comparisons of the extracted sub patterns from fig 5.6.

Fig 5.11b: Pattern 5 from new position



Fig 5.11a: Pattern 5 from reference position

60

5.4.2.2 Analysis of sub patterns (fig 5.7-11)

In fig 5.7, we can observe the pattern 1 at reference pose and at the new pose. With 10cm difference, the pattern has lost its shape from the lower side but the shape in middle and upper side is still in matching condition. If we translate the reference shape of fig 5.7a into a representation which is independent of distance then we could be able to match this shape with the shape in fig 5.7b therefore this pattern is matched by following these two rules

� The shape representation should be independent of the distance at which the shape is located. � We should not consider the shapes of lower and upper sides during the process of matching

similarities. Now we can come to the pattern of fig 5.8. Patterns of fig 5.8a and 5.8b are completely different and no similarities have been found therefore this pattern is excluded and we moved towards next available pattern i.e. pattern 3 (fig 5.9). The overall shape at reference point in fig 5.9a seems to be matching with the new point pattern (fig 5.9b) but it needs more curve approximation. According to this hypothesis, we can include one more rule in our list

� The reference shape or pattern should be approximated in such order the changes with less than 40 degree angle should be considered as a single line.

In pattern 4 from fig 5.10, the shapes seems to have some similarity after translating them in very general form. If we represent pattern 4 reference shape(5.10a) with three side box then it seems to be matching with its corresponding new point shape(fig 5.10b) but then it is also matching with pattern 3 shape therefore the generalization needed in order to do this matching could lead us to ambiguous results. In pattern 5(fig 5.11), the middle shape of the sub pattern in fig 5.11a seems to be matching with the sub pattern in fig 5.11b but the sides of these sub patterns are completely different from each other. By excluding the sides, these two sub patterns can be matched. 5.4.3 Conclusion of the Analysis In the example of the previous section (section # 5.3.2), the new pose is 10cm away from the reference pose. After this 10cm move, two of the five sub patterns are quite hard to match. In remaining three, the sides are not matching only the middles of patterns have similarities. The reliability of pattern’s shape depends on the kind of the object that is considered. If the object is bush then we have much variable patterns that are difficult to match but if the object is tree then the pattern has proved to be easily match able in 80% of the cases. In Natural terrain, objects like trees and bushes, are usually intermixed with each other so if we want to go for the feature based matching then we have to consider following guidelines during the pattern matching process.

� We have to extract patterns which are reliable and the reliability of the pattern depends on the kind of the object that generates the pattern.

61

� We have to approximate the patterns to some extent. This extent of approximation should neither

be so small that patterns will not be matched. Nor should it be too high that the pattern will be matched with wrong patterns.

5.4.4 Moving towards more feasible approach We have experienced that feature based matching needs different kinds of computation. Even after dividing main pattern into corresponding sub patterns, extraction of the reliable patterns is needed. Also, the approximation of the reliable patterns in order to start the pattern matching between reference and new positions therefore it is highly computationally expensive process. Considering this fact, we have decided to test the applicable methods of pixel based matching because these methods seem to be less computationally expensive.

5.5 Pixel Based Pattern Matching and Localization In this approach, we consider the ladar output just as a set of ranges and bearing angles to the object and we try to localize the mobile robot by pixel to pixel matching of reference and current poses. The important point is that we don’t extract any landmarks or any kind of features from the data. 5.5.1 Selection of the methodology [Brenneke, 2003] has proposed a pixel based method for localization based on level range scan but the method is based on readings from 3D ladars. One of the well known localization algorithm is Markov localization [Fox et al., 1999] that has been implemented and tested in many real world applications including deployment of two mobile robots as interactive museum tour-guides. It is an probabilistic algorithm using probability distribution techniques to perform localization. [Bailey et Nebot, 2001] developed a method for laser based odometry. This method is based on matching line or point features in successive range scans. [Lu et Millios, 1997] have proposed two methods based on non feature based matching of 2D ladar patterns. The accurate matching of ladar patterns for determine the pose and heading, is carried out by using modified ICP. The methods define successive laser scans by using odometry. It is possible to perform localization without using odometry. [Selkäinaho, 2002] has proposed an efficient pixel based matching method to do pattern matching and localization using 2D ladars without extracting features. This method is reliable and faster. It is also a zero order image distortion model (IDM). By zero order, we mean that it neglects all dependencies of the pixels displacement [RWTH Keysers et al, 2004]. Using the same approach, we have proposed a methodology with different equations. The extended/altered method has produced quite good results(see table 5.1 and 5.3) and seems to be better than the original method.

62

5.5.2 Detailed Description of the methodology Suppose that there lies an object O1 within the scan range of laser scanner. The Cartesian coordinates of this object in the global coordinate system are

xi = ri × cos (βi – Φk) + xk -----------------------------------------------(5.2)

yi = ri × sin (βi – Φk) + yk ---------------------------------------------- (5.3) where ri and βi are the range and bearing angle to the object given by the ladar mounted on a mobile robot with robot pose(xk, yk, Φk)

x

y

Fig 5.12a: Coordinates of robot pose

The coordinates of the robot pose(xk, yk, Φk) are visualized in fig 5.12. In next scan, the robot has changed its pose by (∆x , ∆y, ∆Φ). Suppose the new pose of the robot is (xn, yn, Φn) as shown in figure 5.12b.

x

y

Previous pose(xk,yk,Φk)

Current pose(xn,yn,Φn)P2(xj,yj)

P1(xi,yi)

object

Fig 5.12b: Change in robot pose

The new coordinates for the object O1 can be written as

xj = rj × cos(βj – Φk – ∆Φ ) + xk + ∆x ------------------------------------ (5.4)

yj = rj × sin(βj – Φk – ∆Φ) + yk + ∆y ------------------------------------- (5.5)

Φ

Robot pose P(x, y, Φ)

63

If we know the correct value of (∆x, ∆y, ∆Φ) then we can find the new coordinates of the object Oi by using equations 5.4 and 5.5. The optimal value for (∆x, ∆y, ∆Φ) will be the one that gives maximum number of N for the supposed translation (∆x, ∆y) and rotation (∆Φ). [Selkäinaho, 2002]

N(∆x , ∆y , ∆Φ) = Σ nk -------------------------------------------- (5.6) N is the summation of nk. A ladar based pattern consists of set of ranges or points. nk is one if a point (xj , yj) in new pattern is matching with a point (xi , yi) in the reference pattern. [Selkäinaho, 2002] has defined nk as

nk = 1, if Эj ((xj –xi )2 + (yj –yi ) 2) < dk2 -----------------------------(5.7)

where (xj , yj ) is the reference point (xj , yj) is the new or unknown point where dk is a constant value which is determined by:

dk = 0.6 × ∆s + 0.6 × ∆b × rj ---------------------------------------(5.8) where ∆s is the position step search resolution in cm ∆b is the bearing step search resolution in radians rj is the radius of the point in polar (r,Ө) coordinate system According to [Selkäinaho, 2002], he has proposed these parameters for the equation 5.8 because the values from these parameters yielded best results. [Selkäinaho, 2002] has considered the Euclidean distance formula in equation 5.7. The concept behind equation 5.7 is that we are trying to find a point near to the reference point. For every point, we take its Euclidean distance from the reference point and if this distance is less than dk then we consider this point as matching. dk is calculated from equation 5.8. We have specific shortest (Euclidean) distance value for every point and we compare this value with dk . The point is that why we use Euclidean distance formul? We can take any other distance formula and compare with its yielded distance value with some parameter like dk to check whether the point is matching or not matching. We think that it is better to use Manhattan distance formula (the distance formula with one norm). The equation of Manhattan distance (Dm) between two points P1(x1, y1) and P2(x2, y2) is

Dm = |x2 –x1 | + |y2 –y1 |

We think Manhattan distance is better because

� 2-norm Euclidian distance is more sensitive to errors than 1-norm Manhattan distance. e.g: An incorrect x-coordinate will dominate the y-coordinate if they are squared as we do in Euclidean distance formula.

� Computationally, Manhattan distance takes less time than Euclidean distance because of its more

simple equation. For each scan with 0.5 degree angular resolution and 180 degree scanning angle, ideally we can have 361×361 comparisons. We have lesser comparisons practically but still the simple of equation of Manhattan distance makes the method faster.

64

We have changed equation 5.7 in accordance with Manhattan distance formula and got better results. Please refer to section # 5.6.2 to observe the results. By putting Manhattan distance formula in equation 5.7, we get

nk = 1, if Эj (|xj –xi | + |yj –yi |) < Fk ------------------------------------------------(5.9) nk = 0, otherwise The dk proposed in equation 5.8 is the one on which [Selkäinaho, 2002] has found best results. According to our equation 5.9, we have got the best results by comparing the Manhattan distance with the value yielded from equation 5.10. The output of equation 5.10 also depends on the variation Γ given by the ladar

Fk = 0.6 * ∆s + 0.6 * ∆b * rj + 0.02 * rj + Γ ----------------------------------------- (5.10) where Γ is the variance (in cm) of the ladar readings at a stationary point. E.g. : If we are searching the maximum number of N(∆x , ∆y , ∆Φ) with the step difference of 10cm and with angular difference of 0.5 degree for the ladar LMS-221 then in this case ∆s = 10 ∆b = ½ * π/180 = π/360 Γ = 3 since LMS-221 gives 3cm variance Once optimal values for ∆x , ∆y and ∆Φ have been computed according to equation 5.9 and 5.10, the new position (xn , yn, Φn) of the robot can be computed as

xn = xk + ∆x ; yn = yk + ∆y and Φn = Φk + ∆Φ; ----------------------(5.11) 5.5.2.1 Further Possible improvements

� We have found a way to match the points using Manhattan distance formula. Further improvements may also be possible in equation 5.9. Distance formula of second norm (Euclidean distance) has been used in equation 5.7 and distance formula of first norm (Manhattan distance) has been used in equation 5.9. The other distance formulas can also be applied and further tested in different scenarios. The general equation of distance formula of any norm is given as equation 5.11

Dk(p,q) = (∑ =

n

ik

1 ii )q-p( )1/k -------------------------------- (5.12)

where k represents the norm and summation sign is there to sum the difference in all available dimensions or axis Due to limited time, we only tried distance of one norm and two norm in equation 5.7. One can also experiment with other distance norms and compare the results.

65

� The parameter Fk of equation 5.9 also plays very important role to increase or decrease the

matching ratio of points. Greater Fk values give higher number of matching points but the accuracy is reduced. When accuracy is reduced then probability of matching wrong graphs will increases. Therefore we have to find moderate a balance equation for Fk. We have researched and gave one possible equation (equation 5.10) for Fk but further research can be done in this regard. It would be interesting, if the graphs of all possible values of constant and variable are drawn then one may predict some better equation than equation 5.10

� It is also interesting, if one can try to do pixel based matching through the equations based on

polar coordinates instead of rectangular coordinates. This would make the computation simpler because the ladar LMS-221 gives output in form of polar coordinates so one don’t need to do conversion to rectangular coordinates for pixel based matching.

� Instead of matching every point with reference point of equation 5.9, we can match the reference

point with a small set of nearby points. This approach can make algorithm faster. Algorithm 5.1: For Pixel Matching and localization On the basis of equation 5.6, 5.9 and 5.10, a simple algorithm for localization is as follows 1. Determine the starting and ending boundries for ∆x , ∆y and ∆Φ 2 Set the position search resolution ∆s (distance step difference) and the bearing search resolution ∆b(angular step difference). 3 Start searching from lower boundry of ∆x in steps of ∆s, hold the value which gives maximum number of matching points according to equation 5.6 4 Using the best value for ∆x , start searching the value for ∆y that gives maximum number of points

according to equation 5.6 5 Using the new ∆x and ∆y, start searching the best value for ∆Φ in steps of angular search resolution ∆b.

6 Now, we can update the robot position according to equation 5.11. A better solution would be to estimate the location of new point with the help of more than one nearest points and the final location of the new point will be an average of the locations predicted by the nearest points. In algorithm 5.1, the new point will be searched within a circular approximation of last position as shown in figure 5.13

66

Fig 5.13: Searching the current position in a circle around the last position

In fig 5.12, we are searching the best ∆x, ∆y and ∆Φ with the help of three nearest points around last position of the robot. Here, the final location of the new point is the average of the locations predicted by these three nearest points. For using algorithm 5.1, we have to determine the starting and ending boundary values of ∆x , ∆y and ∆Φ (step 1 of the algorithm 5.1), we have followed a two step procedure

i. By knowing the last position and the velocity v of the mobile robot, we can predict the distance traveled from the last position to the new position using Newtion’s distance formula s = v×t Therefore we can guess maximum limit for its new position.

ii. From the estimated new position, we can take the nearest reference points as shown in fig. 5.12 and then we will have an starting and ending values of ∆x , ∆y and ∆Φ.

In ‘Step 2’ of the algorithm 5.1, we have linear searching technique i.e. we search firstly the best value in x-dimension then we search for best the value in y-dimension according to the searched best x-value. Similarly, we search for best value of Φ. In the linear searching technique, we have observed a relationship between the parameters of position and angular search resolution, the time taken ‘T’ by the algorithm and the accuracy ‘A’ of new position.

‘A’ and ‘T’ increases with decreasing position and angular search step sizes.

A α 1/ ∆s, A α 1/ ∆b (5.13)

T α 1/ ∆s , T α 1/ ∆b (5.14) where A is the accuracy of calculating rotation and translation and T is the time period taken by the algorithm The application of equation 5.9 and 5.10 are more specific to linear search. They will be bit different for other searching techniques. The choice of ∆s and ∆b is a trade off between position search resolution ∆s, angular search resolution ∆b, the time period ‘T’, and the accuracy ‘A’. This trade off will depend on

67

output accuracy requirements and environmental variables of the experiment like velocity of the robot, hardware response time etc.

5.6 Testing and Analysis 5.6.1 Test-bed and Working Different out door and indoor experiments have been conducted for testing of short distance pattern matching (1– 4m) as well as long distance pattern matching (5–20m). In short distance pattern matching, similar patterns of maximum 4m difference have been matched. In long distance pattern matching, the similar patterns of maximum 20m difference have been matched.

P3

P4

P2

P1

P5

Reference equidistant points

Current Positiond1

d3

d4

d2

Fig 5.14: Path with equidistant points In fig. 5.14, we have five equidistant points (P1- P5). The robot is at position P4 and d1, d2, d3 and d4 are the distances from P4 to the other points. As displayed in the figure, we have started the experiment by making a path which consists of five points. These points have almost 1m consecutive distance between them. Firstly, we have recorded the ladar pattern at one point then we moved to next consecutive point to get the difference of one meter. In this way we increased the distance till 4m which is the distance between the first point and the fifth point. After performing the trials for short distant matching, we moved to the trials for long distant matching i.e. pattern matching with distance difference up to 20m. Now, we have taken again five consecutive points as shown in figure 5.14 but with the 5 meters consecutive distance between them.

5.6.2 Results and Analysis The results of long distant trials (discussed in last paragraph of previous section) are shown in table 5.1. Here, the starting or reference pose P1(x , y, Φ) of the robot is (0,0,0). Method M1 refers to the method proposed by [Sälkainaho, 2002] and method M2 refers to our proposed methodology. These methods are described in detail in section # 5.5.2.

68

In first column of table 5.1, we have listed actual robot poses that we had noted using measuring-tape. In the third column, we have robot poses measured by the ladar. In sixth column, we have the number of matching points from method M1 and method M2. By number of matching points we mean that the points in reference and new ladar pattern that are matched. Reliable points are the number of points in a pattern that are allowed to go for pattern matching process after filtering the pattern from noises.

Table 5.1: Results of the experiments for localization of mobile robot with distance difference from 1m till 20m

We have matched corresponding patterns at different poses of mobile robot in table 5.1. These poses have distance from 1m till 20m between them. Method M2 is able to localize the robot till 20m with the maximum error of 12cm(x-dimension), 28cm(y-dimension) and 1º(bearing angle Φ). The matching ratio is listed in last columns of table 5.1. Method M1 can also localize the robot but sometimes it gives very less matching ratio. E.g. At actual pose of (30, 2000, 0), method M1 gives matching ratio of 6.7%. This matching ratio is so less that it is difficult to differentiate that whether the pattern is matching or this small value is coming from noise.

Number of points Number of matches ‘N’

Pattern 1 Pattern 2 From [Selkäinaho, 2002] method ‘M1’

from proposed method ‘M2’

126 87 0 4 126 79 0 3 87 165 3 11

Table 5.2: Matching wrong pattern to check the reliability of methods We have compared the wrong patterns in table 5.2. By wrong patterns, we mean that the ladar patterns which are taken from completely different places and which should not match with each other. We have compared these patterns to check the reliability of method M2 and M1. Both methods have worked well in this regard. Method M1 has given slightly less noise than method M2 but the difference is just few centimeters that small amount can be experimental error. Comparisons of matching ratios of methods M1 and M2 in the form of bar graph are given in fig 5.15. Here, the data has been taken from Table5.1.

Number of matching points / Total number of reliable points

Actual pose from P1 (cm, cm, degrees)

Estimated pose PM1 by the ladar

using method ‘M1’

(cm ,cm ,degrees)

|Error| in pose PM1 (cm, cm, degree )

Estimated pose PM2 by the ladar using proposed method ‘M2’

(cm ,cm ,degrees)

|Error| in pose PM2

(cm, cm, degree)

From method ‘M1’

From method ‘M2’

(0,120,0) (4,120,0.0) (4, 0, 0.0) (0,120, 0.0) (0, 0, 0.0) 28/79 = 35.4% 70/79 = 88.6% (10,225,0) (23,208,-2.0) (13, 17, 2.0) (17,230,-0.5) (7, 5, 0.0) 23/84 = 27.4% 76/84 = 90.5% (20,-490,0) (31,-473,1.0) (11, 17, 1.0) (32,-480,0.0) (12, 10, 0.0) 13/176 = 7.4% 100/176 = 56.8% (0,1020,0) (5,-1040,-2.0) (5, 20, 2.0) (-10,-1054,-1.0) (10, 24,1.0) 26/150 = 17.3% 133/150 = 88.7%

(30,1530,0) (38, -1540, 0.5) (8, 10, 0.5) (37,-1558,0.0) (7, 28,0.0) 14/148 = 9.5% 74/148 = 50.0% (30,2000,0) (32,-2020,0.0) (2, 20, 0.0) (32,-2020,0.0) (2, 20, 0.0) 11/165 = 6.7% 74/165 = 44.8%

Average ----- (7, 14, 0.9 ) ---- (6, 15, 0.2) 17.3% 69.9%

69

020406080

100

Percentage of matching

1m 2m 5m 10m 15m 20mDistance difference b/w matching patterns

Fig. 5.15 M1 and M2 output comparisons

Method M1Method M2

From table 5.1 and fig 5.15, we can observe that the method M2 has better matching ratio than its base method M1, especially at the distances of 15 and 20 meters. In these distances, method M1 almost seems to give very less matching ratio that can be neglected by considering it as a noise. Now we would like to discuss some other experimental results. In the following experiment, the change in bearing angle is also tested besides change in x and y values. In table 5.3, the readings at three different points (P2, P3 and P4) are taken. These points are at different horizontal and vertical displacements, and at different bearing angles from a fixed reference point P1. In first column, the 2nd points for taking ladar pattern are listed. Here, point number and bearing angle is described. E.g. p2-m50 means point P2 and bearing angle -50º. The other column headings are also present in table 5.1. They are explained in the description regarding table 5.1 on previous page.

70

Table 5.3: Analyzed results at three different points with bearing angle (Φ) variation at each point from -50º to 50º From table 5.3, both methods (M1 and M2) are giving out some matching points even at the bearing angle difference of 50 degrees between the poses. Method M1 is giving so less matching points (<15%) that it is not possible to differentiate that is it a noise or the method has really found a matching image. On the other hand method M2 is giving good matching ratio in matching images therefore through method M2;

2nd Point

Actual pos. from P1 (cm, cm,degrees)

Calculated pose from method M1

(cm, cm, degrees)

|Error| in pose

PM2 (cm, cm, degree )

Calculated pose from method M2

(cm, cm, degrees)

|Error| in pose PM2 (cm, cm, degree )

No. of matching

points at P M1’ pose

No. of matching points via

method ‘M2’

P2-m50 (184,206,-50.0) (188,217,-56.0) (4, 11, 6.0) (178,224,-53.5) (6, 18, -3.5) 30/218= 13.8% 99/218= 45.4%

P2-m40 (186,204,-40.0) (178, 187, -41.5) (8, 17, 1.5) (176,220,-41.0) (10, 16, -1.0) 38/218= 17.4% 105/218= 48.2%

P2-m30 (188,202, -30.0) (176, 216, -28.5) (12 , 14, 1.5) (176,216,-28.5) (12 , 14, -1.5) 39/218= 17.9% 116/218= 53.2%

P2-m20 (190,200, -20.0) (170, 180, -19.5 ) (20, 20, 0.5) (188, 214, -20) (2 , 14, 0) 37/218= 17.0% 132/218= 60.6%

P2-m10 (190,200, -10.0) (185, 197, -11.5) (5, 3, 1.5) (186, 198, -11.0) (4 , 2, -1.0) 85/218= 39.0% 143/218= 65.6%

P2-0 (200,200, 0.0) (190,192,-1.0) (10, 8, 1) (190, 192, -1.0) (10, 8, -1.0) 42/218 = 19. 3% 135/218= 61.9%

P2-p10 (184,204, 10.0) (173,223,9.5) (11, 19, 0.5) (186,205,8.5) (2, 1, 1.5) 41/215= 19.1% 129/215= 60.0%

P2-p20 (184,204,19.5) (188,216,18.5) (4, 12, 1.0) (188,216,18.5) (4, 12, 1.0) 44/221= 19.9% 91/221= 41.2%

P2-p30 (184, 204,29.5) (182, 221, 30.5) (2, 15, 1.0) (184,213,29.5) (0, 9, 0) 24/215= 11.2% 78/215= 36.3%

P2-p40 (184,204,39.5) (188, 219, 40) (4, 15, 0.5) (184,217,40) (5, 12, 0.5) 28/218= 12.8% 69/218= 31.7%

P2-p50 (184,210,49.5) (189, 222, 48.0) (5 , 12, 1.5) (189,222,48.0) (5, 12, 1.5) 19/218= 8.7% 78/218= 35.8%

P3-m50 (290,210,-48.5) (290, 189, -50.0) (0, 21, 1.5) (289,212,-50) (1, 2, 1.5) 51/217= 23.5% 105/217= 48.4%

P3-m40 (290,210,-38.5) (282,208,-37.0) (8, 2, 1.5) (282,208,-37.0) (8, 2, 1.5) 30/216= 13.9% 96/216= 44.4%

P3-m30 (295,210,-28.5) (300,208,-27.0) (5, 2, 1.5) (300,208,-27.0) (5, 2, 1.5) 44/218= 20.2% 124/218= 56.9%

P3-m20 (295,210,-20) (304,197, -20.0) (9, 13, 0) (295,210,-18.5) (0, 0, 1.5) 64/218= 29.4% 134/218= 61.5%

P3-m10 (295,211,-10.5) (296,200,-10.0) (1, 11, 0.5) (295,213,-9) (0, 2, 1.5) 90/218= 41.3% 139/218= 63.8%

P3-0 (300,210,0.0) (311,212,-4.0) (11, 2, 4,0) (310,220,-3.0) (10,10, 3.0 ) 38/218= 17.4% 125/218= 57.3%

P3-p10 (290,210,8.5) (298, 203, 6.0) (8, 7, 1.5) (292,212,7) (2, 2, 1.5) 76/218= 34.3% 135/218= 61.9%

P3-p20 (290,210,16.5) (302, 196, 14) (7, 14, 2.5) (297,200,14) (7, 10, 2.5) 59/218= 27.1% 113/218= 51.8%

P3-p30 (290,210,28.5) (292,210,26.5) (2, 0, 2.0) (292,210,26.5) (2, 0, 2.0) 34/218= 15.6% 94/218= 43.1%

P3-p40 (300,210,38.5) (300, 216, 37.5) (0, 6, 1.0) (309,218,37.5) (9, 8, 1.0) 45/214= 21.0% 78/218= 35.8%

P3-p50 (300,210,47.5) (308, 211, 46.0) (8, 1, 1.5) (305,212,46.0) (5, 2, 1.5) 28/201= 13.9% 71/201= 35.3%

P4-m50 (400,210,-51.0) (394,224,-51.5) (6, 14, 0.5) (394,224,-51.5) (6, 14, 0.5) 39/208= 18.8% 105/208= 50.5%

P4-m40 (400,210, -41.0) (394,224,-42.5) (6, 14, 1.5) (394,224,-42.5) (6, 14, 1.5) 62/217= 28.6% 110/217= 50.7%

P4-m30 (400,200, -31.0) (394,208,-33.0) (6, 8, 2.0) (394,208,-33.0) (6, 8, 2.0) 60/218= 27.5% 112/218= 51.4%

P4-m20 (400,200, -20.0) (392,208,-20.5) (8, 8, 0.5) (392,208,-20.5) (8, 8, 0.5) 63/216= 29.2% 131/216= 60.6%

P4-m10 (400,200,-11) (394,212,-12) (6, 12, 1.0) (394,212,-12) (6, 12, 1.0) 85/215= 39.5% 136/215= 63.3%

P4-0 (400,200, 0.0) (394, 200, -1.0) (6, 0, 1.0) (400,204,-1.5) (0, 4, 1.5) 63/218= 28.9% 128/218= 58.7%

P4-p10 (400,210,9.5) (392, 213, 11.0) (8 , 3, 1.0) (397,205,10.0) (3, 5, 0.5) 75/218= 34.4% 118/218= 54.1%

P4-p20 (400,210,19.0) (394,224,18.0) (6, 14, 1.0) (394,224,18.0) (6, 14, 1.0) 23/218= 10.6% 96/218= 44.0%

P4-p30 (400,215,29.0) (397, 215, 28.5) (3, 0, 0.5) (404, 212, 28.0) (4, 3, 1.0) 39/213= 18.3% 76/213= 35.7%

P4-p40 (400,215,39.0) (391, 228, 40.0) (9, 13, 1.0) (397,215,39.0) (3, 0, 0) 38/199= 19.1% 68/199= 34.2%

P4-p50 (400,215,49.0) (396,226,48.0) (4, 11, 1.0) (396,226,48.0) (4, 11, 1.0) 35/193= 18.1% 67/193= 34.7%

Average: (6, 10, 1.4) ---- (5, 8, 0.8) 22.0% 49.6%

71

matching and non matching images can easily be differentiated. Now we are going to check different graphs in this regard.

-50 -40 -30 -20 -10 0 10 20 30 40 500

10

20

30

40

50

60

70

Degrees of Variation in pose

Mat

chin

g R

atio

(%)

Original method (M1)Ext. method (M2)

Trade-off value for 'M2'

Fig 5.16: Comparison of matching ratios with changing degrees in pose (-50º to 50º) of graphs at point P2

[Data taken from table 5.3] In fig. 5.16, we can observe the difference in matching ratios of methods M1 and M2 with pose variation from -50 to 50 degrees. There is also one trade-off line (at 30%) in the figure, which is actually the point at which the image considered as matching in the method M2. It is difficult to draw any trade-off line for method M1 because of its less matching ratios.

Reference position

Current Position200cm

447.24cm

400cmdirect ion=-20º

direct ion=0º(straight)

Fig. 5.17a: Locations of reference and current position for the results shown in figure 5.17b

72

Fig 5.17b: Bar chart of matching ratio with variation of Φ (-50º to 50º) in robot pose(x, y, Φ) of graphs at

Euclidean distance of 448.24cm between the reference and current positions [Data from table 5.3] In figures 5.17, Bar chart of matching ratio at different poses is shown and in all bars of the chart, method M2 is giving better matching ratio than method M1. Also from 30º to 50º, method M1 is giving very less matching ratio. The problem of matching ratio in method M1 can be solved by increasing the dk factor in equation 5.7. When we have multiplied the dk

2 with 10 in equation 5.7, we have received almost the same number of matching points as we have received from method M2. Actual pose

(cm, cm, degrees)

Estimated pose PM1 from

method M1 (cm, cm, degree)

|Error| in pose PM1 (cm , cm, degree )







(10,225,0) (23,208,-2.0) (13, 17, 2.0) (17,230,-0.5) (7, 5, 0.5) (9,226,-1.0) (1, 1, 1.0 )

(184,204,39.5) (188, 219, 40) (4, 14, 0.5) (184,217,40) (0, 13, 0.5) (187, 219,40.0) (3, 15, 0.5)

(400,210,-51) (394,224,-51.5) (6, 14, 0.5) (394,224,-51.5) (6, 14, 0.5) (393, 222, -52.0) (7, 12, 1.0)

(400,210,-41) (394,224,-42.5) (6, 14, 1.5) (394,224,-42.5) (6, 14, 1.5) (391, 217, -42.0) (9, 7, 1.0)

(400,215,29) (397, 215, 28.5) (3, 0, 0.5) (404, 212, 28.0) (4, 3, 1.0) (409, 198, 27.0) (9, 17, 2.0)

Table 5.4: Comparison of robot’s localization from method M1, M2 and M3

73

In table 5.4, the method M3 is same as [Sälkainaho, 2002] method M1 except we have multiplied dk2 by

10 to equalize the matching ratio with the points matching ratio from method M2. The equations used in method M3 are as follows.

nk = 1, if Эj ((xj –xi )2 + (yj –yi ) 2) < ((dk2) ×10) (5.15)

where dk = 0.6 × ∆s + 0.6 × ∆b × rj

The equation 5.15 is same as equation 5.7 of method M1 except we have multiplied the right side with 10. We have compared the results from method M1 and its variant method M3, and the proposed method M2 in table 5.4 From table 5.4, there is no significant increase in error while estimating the poses from method M3. Therefore it can be concluded that the changing the equation of 5.7 to 5.15 does not affect the accuracy of pose. This change has been done to increase the points matching ratio of method M1. Conclusively, the points that shows that the proposed method M2 is better than the original method M1 are as follows

1. Conceptually, method M2 has used first degree norm distance and method M1 has used second degree norm distance. For matching in equation 5.9, we need a distance formula which gives unique value for every location. This value should change with the change in location and this value should increase with increase in distance. It is not important that the distance between reference and new point should be shortest but it is important that the distance should increase or decrease with change in location. We can observe the change more clearly if we use first norm distance formula(Manhattan distance) than second norm distance formula because the increase in value is always more in Manhattan distance with increase in distance. E.g. Let us suppose two points in x-y axis (1,1) and (8,9). The distance between them in accordance with Euclidean distance formula is ’10.6’ but distance in accordance with Manhattan distance formula is ‘15’. Manhattan distance is also less sensitive to errors as compare to Euclidean distance.

2. Computationally, method M2 takes less time than method M1 because it uses simple equation of Manhattan distance for comparison. The comparison equation of method M2 is given as equation 5.9 in section # 5.5.2.

5.7 Example of localization using proposed method We would like to show the process of rotation and translation using proposed method M2. Let us consider the two positions as shown in figure 5.20

74

Reference position

Current Position200cm

447.24cm

400cmdirect ion=-20º

direct ion=0º(straight)

Fig 5.18: Locations of reference and current pose

As shown in figure 5.18, the robot has straight heading at reference position and it is slightly turned on right side with -20º pose at new or current position. The Euclidean or shortest distance between the points is 447.24cm. The ladar patterns at reference and current position are sketched below.

-3000 -2000 -1000 0 1000 2000 30000

500

1000

1500

2000

2500

x-axis(cm)

y-ax

is(c

m)

Reference pos.Current(new) pos.

Fig 5.19: LADAR patterns at reference and current positions

We can observe in figure 5.19 that the graph at current position(square graph) is actually a distorted form of the graph at reference position(cross graph). Now we can translate and rotate the square graph (graph at new position) and try to find the best match. According to the proposed method M2 which is explained in section # 5.5.2, the best match has been found at translation of [x:392cm, y:208cm] and rotation of -20.5º. The actual value of translation is (400cm, 200cm) and the rotation is -20º so the error is 8cm in translation and 0.5º in rotation but this

75

difference is less than 10cm in translation and less than 1º in rotation therefore this error is within the limit of experimental error. It can be neglected. The output graph after rotation and translation is sketched below

-3000 -2000 -1000 0 1000 2000 3000-500

0

500

1000

1500

2000

2500

3000

x-axis(cm)

y-ax

is(c

m)

Reference pos.Translated and rotated pos.

Fig 5.20: Patterns at reference position and at translated and rotated position using proposed method M2

We can see the output pattern of translation and rotation in figure 5.20. There are some errors but most of the points are transformed correctly. For more localization examples using method M2 please refer to table 5.1 and 5.3. In these tables, more than 35 different transformations are calculated using method M2 and all of the results of translations and rotations are correct with some minor errors.

76

5.8 Conclusion We have started the localization process from feature based pattern matching approach but this approach proved to be highly computationally complex approach when one is performing 2D pattern matching. For details please refer to section # 5.4. We then shifted towards pixel based pattern matching approaches. The methodology proposed by [Sälkainaho, 2002] in his Phd dissertation has been founded worthy according to our test results (section # 5.6.2). It is a flexible method and it simpler and faster as compare to other pixel based localization methods because it just uses Euclidean distance formula to check the matching between corresponding points of ladar based pattern. The selection of this methodology is discussed in detail in section # 5.5.1. We have tested this methodology in different aspects and proposed a methodology based on it. We think this proposed method is better than its base method because it is faster and less susceptible to errors. For details, please refer to the reasons given in the end of section # 5.6.2 The proposed method has produced good pattern matching results for distances till 20m (see table 5.1) and bearing angle variation of ± 50 degrees (see table 5.3). Further improvement/alteration is possible. The future work will be discussed in next chapter (chapter 6).

77

Chapter 6

Conclusions & Future work

In finding the best methodology for 2D ladar based localization, firstly we have tested feature based pattern matching. We have found that feature based pattern matching in 2D ladars is difficult and includes lot of extra computation when we compare it with non feature pattern matching. We have to extract feature of an object after filtering it from noisy patterns. For some objects, features vary a lot with distances therefore we have to decide reliable landmarks in accordance with the environment. Feature based pattern matching is more successful when we work with 3D ladars because we can extract the feature based on whole object. In 2D ladars, a partial feature is extracted at a certain height of object and with a slight change in height the feature can be completely changed. E.g. when we see trees from bottom, we can just extract feature of their stem but from up when branches are included, the feature is completely changed. In non feature based localization, there are not many methods available for 2D ladars. We have found that the method proposed by [Selkäinaho, 2002] in his Phd. dissertation is simpler as compare to others because it is based on zero order image distortion model. In simple way, we can say that it does not consider any kind of dependencies of points in the ladar pattern and only based on point matching through distance formulas. We have tested this method in different aspects and proposed a method based on it. This new method is faster and less susceptible to errors than its base method. For details, please refer to the reasons given in the end of section # 5.6.2 This method has produced good matching results for distances till 20m (see table 5.1) and bearing angle variation till ± 50 degrees (see table 5.3). The algorithm of the proposed method is given in section # 5.5.2 (algorithm 5.1). There can be many directions regarding future work. Some of them are given below

6.1 In furtherance of pixel based proposed methodology � There is a big room for improvement/alteration in considered method of pixel based matching. We

can improve the accuracy, flexibility and other performance factors, the possible improvements under this methodology are discussed in section # 5.5.2.1

� The proposed algorithm 5.1 can be further enhanced to 3D localization using some 3D ladar or

using 2D ladar like LMS-221 at different roll and pitch angles. A further direction can be the determination of the minimum number of roll angles and pitch angles required in order to perform satisfactory 3D localization of robot’s position. I think this requirement will be quite dependent on the scenario. In some environments, may be 4 different pitch angles are enough to do 3D localization while in others, at least 7 or 8 different pitch angles are needed.

� A pixel or non feature based method has been established but more outdoor tests in different

terrains are needed to check its suitability and limitations.

78

� Although tests for localization at different distances are successful but these tests are performed

with supposition of static positions i.e. without including velocity factor so we need to test the localization on moving robot at different velocities. It would be interesting, if we build pattern database and robot will move and localize itself by selecting suitable patterns. This work can be done in any programming language but Java will be a better option because

– It can perform well on networks – The driver software with customizable options is written in Java. The software is

developed on 3-tier architecture and with standard styles so it is easily modifiable.

� The proposed algorithm is currently based on linear search. We need to implement some parallel parameters search technique like [Charalambous, 1992] has implemented gradient search. It can be exhaustive search or some other fast searching techniques to make the proposed algorithm 5.1 (section # 5.5.2) more efficient. I am pretty hopeful that we can make it to one of the fastest complete localization algorithm because this method does not consider any kind of extraction of features and take less time in localization.

� 3D localization using non feature based methodology will be a new idea. By 3D localization, I

mean that we will also consider the height, pitch and roll angles in the ladar pose using sensor fusion. I think it can work in some terrains and it will be faster than any feature based methodology because we skip the step of complex and computationally expensive work of extracting features from the environment.

---------------------------------------------------------X------------------------------------------------------------

79

References

[Hebert et. al, 2002]

Martial Hebert, Nicolas Vandapel, Stefan Keller, and Raghavendra Rao Donamukkala Evaluation and comparison of terrain classification techniques from ladar Data for Autonomous Navigation,. 23rd Army Science Conference 2002

[Selkäinaho, 2002]

Jorma Selkäinaho Doctorate Dissertation, “Adaptive Autonomous Navigation of Mobile Robots in Unknown Environments”, Helsinki University of Technology, Finland

[Charalambous, 1992]

C. Charalambous “Conjugate gradient algorithm for efficient training of artificialneural networks”, IEEE Proceedings, June, 1992

[Halstead et al., 1996]

Mark A. Halstead, Brian A. Barsky, Stanley A. Klein, Robert B. Mandell “Reconstructing Curved Surfaces From Specular Reflection Patterns Using Spline Surface Fitting of Normals”, Proceedings of ACM/SIGGRAPH'96, New Orleans, 4-9 August 1996

[Kelly et al., 2004]

Alonzo Kelly, Omead Amidi, Mike Happold, Herman Herman, Tom Pilarski, Pete Rander, Anthony Stentz, Nick Vallidis, Randy Warner, “Toward Reliable Off-Road Autonomous Vehicle Operating in Challenging Environments”, International Symposium on Experimental Robotics, June, 2004, Singapore.

[IRC, 2004]

Intelligent Robotics Course Literature, Dept. of Computer Science, Umeå University. (http://www.cs.umu.se/kurser/TDBD17/VT04/dl/)

[LMS Tech., 2004] SICK LMS product technical manual (www.sick.com). [Vergeest et. al., 2002]

JSM Vergeest, S. Spanjaard, Y. Song, "Direct 3D pattern matching in the domain of freeform shapes", Journal of WSCG, Vol. 10, Nr. 1, 1-1-2002

[Vandapel et al., 2004]

Nicolas Vandapel, Daniel F. Huber, Anuj Kapuria and Martial Hebert, , “Natural Terrain Classification using 3-D Ladar Data”, IEEE International Conference on Robotics and Automation, April, 2004

[Brenneke et al., 2003]

Christian Brenneke, Oliver Wulf, Bernardo Wagner, “Using 3D Laser Range Data for SLAM in Outdoor Environments”, International Conference on Intelligent Robots and Systems, 2003

[Alexander et al., 1999]

L Alexander and M Donath “Differential GPS based control of a heavy vehicle”, 1999 IEEE/IEEJ/JSAI International Conference on Intelligent Transportation Systems

80

[Vandapel et. al, 2003]

M. Hebert, N. Vandapel, S. Keller, R. Donamukkala, "Terrain classification techniques from ladar data for autonomous navigation", Collaborative Technology Alliances conference, May, 2003.

[Minguez et al., 2002]

J. Minguez, L. Montano, “Robot Navigation in very dense and cluttered indoor/outdoor environments”, International Federation of Automatic Control IFAC 15th World Congress, Barcelona, July, 2002.

[Christensen, 2004]

Henrik I. Christensen, Oliver Wulf, Kai O. Arras and Bernardo Wagner “2D Mapping of Cluttered Indoor Environments by Means of 3D Perception”, IEEE International Conference on Robotics and Automation. ICRA April 2004

[Bailey et Nebot, 2001]

Bailey, T. and Nebot, E. M. [2001]. Localisation in large-scale environments. Robotics and Autonomous Systems 37 pages 261-281, 2001

[Lu et Millios, 1997]

Feng Lu and Evangelos Milios, "Robot Pose Estimation in Unknown Environments by Matching 2D Range Scans", Journal of Intelligent Robotic Systems 18, pages 249-275, November 1997.

[Forsman, 2001]

P. Forsman, “Three-dimensional localization and mapping of static environments by means of mobile perception”, Helsinki University of Technology, Automation Technology Laboratory, Series A: Research reports no. 23, November 2001.

[Fox et al., 1999]

D. Fox, W. Burgard, S. Thrun “Markov localization for mobile robots in dynamic environments” Journal of Artificial Intelligence Research 11, pages 381-427, November 1999.

[Cox et Wilfong, 1990]

I. J. Cox and G. T. Wilfong “Blanche: Position estimation for an autonomous robot vehicle” Autonomous Robot Vehicles, (eds), Springer-Verlag, 1990.

2D LADAR based localization - umu.se€¦ · Arsalan Siddiqui Master’s Thesis, 20 credits...

Documents

Transcript of 2D LADAR based localization - umu.se€¦ · Arsalan Siddiqui Master’s Thesis, 20 credits...