Towards a realTowards a real--time time Towards a real ... · Towards a realTowards a real--time...

Towards a realTowards a real--time time software software architecture for architecture for autonomous mobile robotsautonomous mobile robots

Towards a realTowards a real--time time software software architecture for architecture for autonomous mobile robotsautonomous mobile robotsautonomous mobile robots autonomous mobile robots

Overview with a case studyOverview with a case studyautonomous mobile robots autonomous mobile robots

Overview with a case studyOverview with a case study

ErxaErxa S.r.l.S.r.l.

ERXA ERXA PRESENTATIONPRESENTATION

ERXA S.r.l. ERXA S.r.l. iis a private company that born and grew s a private company that born and grew f ff fdeveloping realdeveloping real--time software application for motion time software application for motion

& processo control.& processo control.

Main competencies of ERXA areMain competencies of ERXA areppDesign and development of robotics cells control Design and development of robotics cells control systems (from axis control to partsystems (from axis control to part--program program y ( py ( p p gp ginterpretation)interpretation)

CAM applications for robotics cell offCAM applications for robotics cell off--line line programmingprogrammingprogrammingprogramming

Erxa S.r.l., Towards a real-time software architecture for autonomous mobile robots

OUTLINE OF PRESENTATIONOUTLINE OF PRESENTATION

•• RTOS, RT Systems and Mobile RobotsRTOS, RT Systems and Mobile Robots

•• RT Software Architecture RT Software Architecture

•• MacMacππ4Log 4Log ArchitectureArchitecture

•• Case StudyCase StudyCase StudyCase Study


RTOS and RT SystemsRTOS and RT Systems

What is the goal of a real-time application?

Every real-time application has to do, at least, the following:

•Read input devices

•Process the data

•Write output devices•Write output devices

Most applications will have more than two devices, like operator keyboards and graphics displays.Moreover some devices could require to be serviced at the same time!time!



RT systemThere exist different possible definitions to describe a RT System. The most useful for our needs are

A RT System is a system that reacts in a time-predictable way to time-unpredictable external stimuli.

A RT System is a system where the correctness of the computed data and the time they are produced have the same, fundamental, importance.



There RT System definitions origin from the following key i trequirements

• Once an event has occurred an action has to be taken within aOnce an event has occurred an action has to be taken within a predetermined time limit; missing a deadline is considered a (severe) software fault (i.e. a RT system needs predictable/deterministic RTOS)RTOS)

• Even if more than one event happens simultaneously, all deadlinesEven if more than one event happens simultaneously, all deadlines for all of them should be met (i.e. a RT system needs pre-emption and multi-task capable RTOS)

Deadlines arise from the underlying phisical phenomena of the system under controlsystem under control



RTOS fundamental requirementsOther basic RTOS requirements are

• Thread priority with priority inheritance

• Predictable thread synchronization mechanism•Inter-task communication•Tasks synchronization•Tasks synchronization•Resources lock/unlock

• Predefined latencies•Task switch•Interrupt latency (task to interrupt handler)•Interrupt latency (task to interrupt handler)•Interrupt latency (interrupt handler to task)



RT systems can be further classified using its capability of respecting h i d d dlithe required deadlines.

HARD RT SystemHARD RT SystemMissing a deadline may reflect in catastrophic failures (ex: ABS –antilock braking system).

FIRM RT SystemTolerant to rare deadline missing The delayed results of processingTolerant to rare deadline missing. The delayed results of processing are unusable (ex: Food Production system).

SOFT RT SSOFT RT SystemTolerant to deadline missing. The delayed results of processing degrade the performances of the system (ex: video conference)degrade the performances of the system (ex: video conference).


Mobile Mobile RobotsRobots and RT Systemsand RT Systems

An Autonomous (Mobile) Robot is a machine that operates in a ti ll k d di t bl i tpartially unknown and unpredictable environment

Then, leaving a robot free to move in some environment requires to give it the ability to appropriately react to it’s surrounding.

The surrounding coincides with the internal representation of the world a robot built using 1) perception, 2) a-priori information.


Mobile Mobile RobotsRobots and RT Systemsand RT Systems

The concept of ‘appropriate reaction’ means in most conditions the b t t t ith t f h i t l t l itirobot must act with respect of happening external events exploiting

some real-time reaction.

This is why, finally, we claim a Mobile Autonomous Robot is a/should be modelled as a RT System.

The external stimuli (i.e. typologies of) have to be modelled as the perception inputs or, more in general, as measures of the surroundingperception inputs or, more in general, as measures of the surrounding inducing change of behaviours.


RTRT PLATFORMPLATFORMRTRT--PLATFORMPLATFORM

Long Long termterm objectivesobjectives

RT-Platform (aka RTP)i l ti ft l tf d l d ith th l f t ffis a real-time software platform developed with the goal of to offer a long-living foundation made of modelling patterns, architectural patterns and software libraries for service robotics applications.patterns and software libraries for service robotics applications.

N d l t f ifi b t t th ith th i ‘ ifi ’ ftNowadays a lot of specific robots together with their ‘specific’ software are made to solve/explore robotics problems.

Every specific robot has its sensors and actuators, its OS and swlibraries, its interfaces.

This is in contrast with parts changes/extension and software reuse/extension i.e. they miss both hardware and software modularityreuse/extension i.e. they miss both hardware and software modularity


ArchitecturalArchitectural DesignDesign

Hardware Abstraction (OS Independence)RT Pl tf t Wi d CE/XP Li RTAI V W kRT-Platform can execute on Windows CE/XP, Linux RTAI, VxWorksA hardware abstraction layer (HAL) has been built on the top of OS calls.calls.

R Ab t tiResources AbstractionA layer offering templates and patterns that defines the basic bricks of the RT architecture (tasks, inter-task communication resources,the RT architecture (tasks, inter task communication resources, diagnostic, timings, shared database).



Hierarchical layered organization split in four levels

FIELDWraps and hides the access to low-levelWraps and hides the access to low levelphysical devices (actuators, sensors)

C O ( )EXECUTOR (u-mission)Manages and orchestrates the physical devices (through the FIELD interface)devices (through the FIELD interface)

MANAGER (mission)Models and manages a real world entity or high level mission.

SUPERVISOR (task)SUPERVISOR (task)Manages and coordinates the managers at task/goal level



Elements of the architecture can be mainly grouped as

ACTIVE OBJECTS• event-driven fashion behaviourevent driven fashion behaviour• one task with an execution context• provide commanded services in an asynchronous way.

PASSIVE OBJECTSPASSIVE OBJECTSThey can be think of as libraries providing synchronous services by function calls. The calls are executed in the caller context



Finite State Machine ModellingTh f l h d l d Fi i S M hi d l (FSMThe formal methods related to Finite State Machine models (FSM or Automaton) is a helpful concept that can help designers and programmers to develop their software models by reducing theprogrammers to develop their software models by reducing the complexity both in specification and implementation tasks.

(Intuitively a robotics application can always be decomposed (i.e. represented) by a set of finite states and a state-transition function.

The same observation applies to every active component of the architecture

Thus we assumed the modelling using FSM’s would be the most correct way to approach the design and implementation ofcorrect way to approach the design and implementation of architecture components.


RTPRTP--KernelKernel

Prototype Task Library (PTL)It’ b i ll FSM d fi i th t i b h i f RT T kIt’s, basically, a FSM defining the systemic behaviour of a RT Taskthat• Represents an independentRepresents an independent execution flow

D fi it l t th t i•Definitely separates the systemicbehaviour from applicative behaviour

•Defines and implements the event-driven infrastructure

Systemic FSM

• Defines and implements the hooks to trace and log the system activities

Systemic FSM

to trace and log the system activities



Prototype Task Library (PTL) II

The ‘RUNNING’ state of a PTL task is the systemic state where the applicative control flow takes place.applicative control flow takes place.

This control flow can be modelled, it-self, by an applicative FSM ti t i i / t ( t d i FSM)reacting to incoming messages/events (event-driven FSM)


Applicative FSMSystemic FSM


Communication (COM)

Both the systemic (PTL) and applicative FSM use COM as support to message passing.message passing.



Virtual Clock Library (VCL)

Lets access to timing functionalities such as

Interrupt context• Periodic function call (ex: watchdogs)

O h t d l d f ti ll• One-shot delayed function call

PTL contextPTL context• Periodic event generation (ex: periodic tasks)• One-shot delayed event generation



Real-time Database (RDB)

The ‘net’ composed by single automata can exchange information through a real-time databasethrough a real time database.

This allows to increase the decoupling between components (even on same layer) and to collect and store information about the state of the whole system.



Booter (BTR)T tl t t th t it t h ld b i iti li dTo correctly start-up the system its components should be initialized and started in the right order eventually breaking the procedure because of some unexpected condition (configuration file).because of some unexpected condition (configuration file).

Diagnostic (DIA)It ff f ti liti t i l it d t li f thIt offers functionalities to signal, monitor and store anomalies of the system both for systemic and applicative layers.

Trace Library (TRL)Trace are used both at debug level and production level to have a f db k b t th k ifeedback about the works running on.


THE MACTHE MAC 4LOG ARCHITECTURE4LOG ARCHITECTURETHE MACTHE MACππ4LOG ARCHITECTURE4LOG ARCHITECTURE

MACMACππ4LOG ARCHITECTURE4LOG ARCHITECTURE

The main goals of MACπ4Log have been to develop a mobile robotic l f i h b d i i d i lplatform with on-board vision systems, sensors and wireless

communication in order to (mainly)

• build and update maps of both indoor and outdoor logistic spaces

f d d i ill• perform programmed and pro-active surveillance

• locate and, eventually follow, moving objectslocate and, eventually follow, moving objects

• locate on the map specific items

• be able to achieve coordination with other fix/mobile platforms



Information ModelTh i f i d l d i f h f iThe information model was created starting from the functionsrequired to the robotic system. It consists in the identification anddescription of entities, relationships and functionalities of the system.description of entities, relationships and functionalities of the system.

The analysis of requirements followed a top-down approach in order i lif h hi h l l f i i i l d hto simplify the highest level functions in simpler ones and then

identifying tasks, decomposing them in missions and the latter in u-missionsmissions.

This process let to 1) highlight the intrinsic hierarchical structure of the li i d 2) i l f i l b b dapplication and 2) isolate functional sub-components to be mapped

onto the software objects.



The capabilities (tasks) required to the robot can be split with respect f t f l l l ti l i i iof a set of lower level, sometimes overlapping, missions.

TASKS TO MISSIONS Build/Update Surveillance Pursuing Locate CoordinationTABLE Maps

Safe path movingX X X X

NavigationX X X X

Mapping X X

Inspection (items & motion detection)

X X Xmotion detection)Synchronization andcommunication X X


MACMACππ4LOG 4LOG -- SupervisorsSupervisors LayerLayer

Supervisors perform the tasks of

• Exploration for mappingThe goal of this task is to create the map of the whole logistic spaceThe goal of this task is to create the map of the whole logistic space.

• Surveillancef fThe goal of this task is to be aware of changes in surroundings trying

to identify and eventually pursuing the source of this change

• Identify and LocateThe goal of this task is to locate known items on the map

• Communicate The goal of this task is to manages the conditions for an exchange ofThe goal of this task is to manages the conditions for an exchange of information with other entities of the team


MACMACππ4LOG 4LOG -- SupervisorsSupervisors LayerLayer IIII

A higher level supervisor is the ‘Autonomy’. It d th b ti d i icommands the above mentioned supervisors in

order to

• Establish the current task to be performed

B id th d i d f t l C di t• Bridge the commands received from an external Coordinator


MACMACππ4LOG 4LOG -- ManagersManagers LayerLayer

Managers perform the missions of

• Organize and maintains a set of local maps (Atlas)Which goal is to decompose the whole logistic map in a set of local g p g pmaps.

S f l l th• Safely move along pathWhich goal is to let the robot moves following a known sequence of targets avoiding dynamic obstacles and optimizing the trajectory.g g y p g j y

• Inspect the surroundingWhi h l i t id tif it b t th i bj tWhich goal is to identify items, members team or other moving objects in its surrounding

• Share dataWhich goal is to accomplish with a data exchange with another robot


MACMACππ4LOG 4LOG -- ExecutorsExecutors LayerLayer

Executors perform the actions of

• Navigate (estimate/evaluate the current location of the robot)Navigate (estimate/evaluate the current location of the robot)The navigation makes possible to compensate for the odometry errors

Wh l l• Wheels control

• Process vision dataProcess vision data

• Special executors that act as ‘observer’ or ‘virtual sensors’h l b l d i i hil hThe laser scans can be also used as proximity sensors while the

scenes viewed can fused with laser/sonar scans etc.


MACMACππ4LOG 4LOG -- FieldField LayerLayer

Field masks the specific characteristics of sensorsd b i h i hand actuators by wrapping them with

• Laser RangeLaser Range

• Axis (wheels) controller

• Camera

• Digital or Analog I/O signalsNow used to access information about the robot state (i.e. robot b l l)battery level)


MACMACππ4LOG 4LOG –– Complete Complete viewview

The Applicative View of Macπ4Log Architecture


THE CASE STUDYTHE CASE STUDYTHE CASE STUDYTHE CASE STUDY

Case Case StudyStudy

The robotic hardware platform used by ERXA to perform the design, development and tests of software modules was composed by

• Pioneer P3DX wheeled mobile robotPioneer P3DX wheeled mobile robot

• SICK laser range

• Vision system (omni-vision camera)

• Netbook Acer Aspire One (running a Windows XP tailored OS)• EPIA motherboard (running WindowsCE 6)



The Functions

• Robot motion control

• SLAM

M B ildi• Map Building

• Safe motion planningSafe motion planning

• Synchronization with an external coordinator



The Software Components Architecture



The Software Components

ASC FIELD interface to loose actuators (wheels)LSR ‘’ interface (and driver) of SICK laser scanLSR interface (and driver) of SICK laser scanCMR ‘’ interface (and driver) of camera

VSN EXECUTOR i f d i f ifi i l i iVSN EXECUTOR interface and services of artificial visionINT ‘’ interface and driver of WMR kinematicsNAV ‘’ interface and services of navigationNAV interface and services of navigationMAP ‘’ interface and services of map buildingERT ‘’ interface and services for synchronization

PPO MANAGER manager of path planning functions


Case Case StudyStudy –– Robot Robot MotionMotion ControlControl

The control loop implemented in INT component and its FSM



The differential kinematics of a unicycle be giveny gin cartesian coordinates

it can be transformed in polar coordinates becoming of the formbecoming of the form

(ref. Aicardi, Casalino – Closed loop smooth steering of unicycle-like vehicle)



The Liapunov function γ0

δ

γ

together with the control laws

ργ

l t th t ll d t b G A t bl ilet the controlled system be G.A. stable i.e.

The choice of parameters Kj allows some further control tuningThe choice of parameters Kj allows some further control tuning


Case Case StudyStudy –– VisionVision

Artificial Vision (passive component / library)

It concerned with the identification in the (omni-)viewed scene of

• Vertical lines extractioncombined use of segmentation andSIFT lik f t t ti l t th bSIFT-like features extraction lets the base point of vertical lines becomes a robustvisual cuevisual cue

C l bl b t ti• Color blobs extraction


Case Case StudyStudy –– SLAMSLAM

Navigation

Th SLAM f iThe SLAM functionis founded on EKFwith corner/lineswith corner/lines laser features


Case Case StudyStudy –– SLAMSLAM

Navigation

Data association relies on theconstruction of aconstruction of acorrespondence graph and itsexploration with a MAXIMALCLIQUE l ithCLIQUE algorithm.(ref. “Data Association for Mobile Robot Navigation: A Graph Theoretic Approach”, Bailey et al )Bailey et al.)

A graph of all possible pairings landmark/feature is built by associating to each arc a weight depending on node types i.e.. corner to corner corners distancecorner to line point-line distance. corner to line point line distance

. line to line intersection angleErxa S.r.l., Towards a real-time software architecture for autonomous mobile robots

Case Case StudyStudy –– Features Features extractionextraction

Corner & Lines Features Extraction (passive component / library)

Based on following main steps

1. Laser scan clusterization

2. Cluster scaling and smoothing

3 Dominant lines identification3. Dominant lines identification

4. Lines and corner interpolation



Laser scan clusterization

C fCm : compensation for sensor noiseCr : range variation of cluster

Cluster scaling and smoothing

Smooth and preserve second derivative (curvature) independently ofthe scalethe scale.

(ref “Multi-scale interest regions from unorganized point clouds”- R. Unnikrishnan, M. Hebert)



Dominant lines identification

Gaussian weighted kernel ina 2-D Hough tablea 2 D Hough table.Selecting highest ‘peaks’ and theirassociated support points in order to identify dominant lines in a cluster.

f ( ) fThe quality of a line depends on size (cardinality) of support points set.

Corners and Lines DetectionCorners and Lines Detection

Support points of selected dominant lines, fin every cluster, are used to identify

lines equations; the lines intersections are the corners.


Case Case StudyStudy –– MapMap BuildingBuilding

Map Building

• Polygonal description to simplify the mapto simplify the map

representation.

B l i• Boolean operation let to take into account for slow changes of surroundingschanges of surroundings

• Triangular mesh generationlet to quickly generate a freespace graph for the path planning


Case Case StudyStudy –– PathPath PlanningPlanning

Roadmap Generation

RGRG:: GenerateGenerate aa roadmaproadmap representingrepresenting thethe undirectedundirected graphgraphGG == (V(V;; E)E)

•• VV == setset ofof allall centroidscentroids ofof thethe trianglestriangles formingforming thethe meshmesh ininVV setset ofof allall centroidscentroids ofof thethe trianglestriangles formingforming thethe meshmesh ininthethe freefree--spacespace•• EE == setset ofof allall edgesedges thatthat connectconnect eacheach nodenode VV withwith allalladjacentadjacent nodesnodes

MSTMST:: usingusing asas rootroot thethe centroidcentroid ofof triangletriangle closercloser toto thethe robotrobot position,position, aaMSTMST coveringcovering thethe roadmap,roadmap, isis builtbuilt withwith DijkstraDijkstra algorithmalgorithm

SPSP:: UsingUsing MSTMST wewe maymay foundfound thethe pathpath inin thethe roadmaproadmap graphgraph joiningjoining CCSS

CC((centroidcentroid closercloser toto thethe start)start) andand CCGG ((centroidcentroid closercloser toto thethe goal)goal) plusplus twotwoedgesedges..



Roadmap smoothingRoadmap smoothing

•• PPPP onlineonline usesuses asas mainmainattractionattraction pointpoint ananappropriateappropriate pointpoint ontoonto thetheroadroad mapmaproadroad mapmap

•• ReduceReduce redundantredundant pathspaths andand robotrobotReduceReduce redundantredundant pathspaths andand robotrobotacceleration/decelerationacceleration/deceleration phasesphases..



Dynamic Obstacles Avoidance

•• TheThe pathpath--planningplanning isis managedmanagedevaluatingevaluating anan artificialartificial potentialpotentialfi ldfi ld ThTh f tif ti th tth t d ibd ibfieldfield.. TheThe functionfunction thatthat describesdescribesthisthis fieldfield isis generallygenerally thethe sumsum ofofdistinctdistinct componentscomponents dependingdepending ononattractiveattractive andand repulsiverepulsive forcesforcespp((RimonRimon andand KoditschekKoditschek,, 19921992))..

11 TheThe currentcurrent attractionattraction pointpoint1.1. TheThe currentcurrent attractionattraction pointpointattractsattracts thethe robotrobot withwith forceforce

FFaa

2.2. RepulsiveRepulsive forceforce assignedassigned toto

allall thethe surroundsurround obstaclesobstacles FFrr


Case Case StudyStudy –– SynchronizationSynchronization

Synchronization Component

A team of exploring and surveilling robots that share information aboutthe surrounding state and receive coordination commands from athe surrounding state and receive coordination commands from a control station has the main requirement of to share the same wall-clock time; i.e. exchanged data need to be labelled with a common

f tireference time.

The Synchronization provides every robot with a high level protocolThe Synchronization provides every robot with a high level protocol based on UDP/IP protocol able to distribute a common clock over the team.


Case Case StudyStudy –– SynchronizationSynchronization

T2DMA protocol

The proposed protocol, referred as Tokenized Time Division Multiple Access needs the presence of a single master and a set of slavesAccess, needs the presence of a single master and a set of slaves.

The exchanged messages (UDP datagram) contain information that l h ‘ b l ’ i dlet every manager reconstructs the master ‘absolute’ time and even foreseen it in case of lacks of transmission.

A prediction-correction filter can beprovided in the manager to estimateh li h i i fthe linear characteristic of tslave versustmaster


DA TEAM …DA TEAM …

Erxa AdministrativesS f éMonica Bresciani, Stefania Anné

Erxa Designers and ManagersErxa Designers and ManagersMarco Ricci, Alessandro Dosio, Nicola Smaldone, Dario Getto, Lorenzo Bonardi

ResearchersGiuseppe Zuffanti Andrea Guadagnuolo Vito MacchiaGiuseppe Zuffanti, Andrea Guadagnuolo, Vito Macchia

ConsultantsMarco Pagliano

A special thanks to Marina Indri for her support in administrativeA special thanks to Marina Indri for her support in administrative issues (aka troubles).


THANK YOU FOR YOUR ATTENTIONTHANK YOU FOR YOUR ATTENTION!!

[:|][:|]

[email protected]@erxa.it


Towards a realTowards a real--time time Towards a real ... · Towards a realTowards a real--time...

Documents

Transcript of Towards a realTowards a real--time time Towards a real ... · Towards a realTowards a real--time...