Geometric Computing for Cybernetics

CINVESTAV

Center of Research and Advanced Studies, Guadalajara

Jalisco, Mexico

Department of Electrical Engineering and Computer Science

GEOVIS Laboratory

MOIMA June.23..2016, Hanover

Eduardo Bayro Corrochano edb@gdl.cinvestav.mx

http://www.gdl.cinvestav.mx/edb

Motivation

Mathematical Framework

Robot Manipulators and Mobile Robots

Humanoids Mechanisms

Perception

Medical Robotics

Low and Middle Level Perception

Conclusions

Content

Motivation: building perception action systems

What kind of

mathematical system?

Conformal geometric algebra

Modern unifying language for the

design of algorithms for PAC systems

Quaternions

Matrix algebra

Vector calculus Trigonometry

Development of

PAC systems

Tensor calculus

Symbolic computing

A sortiment of

mathematical tools

Motivation: shall we go for higher dimensions ?

3D

nD

mD

Real Time ?

Do we gain something by

using other kind of geometric

representations?

Are we sofisticating the

approch for sake for

mathematical elegancy ?

Geometric Algebra

Quaternions, o a new system

of imaginaries in algebra

Clifford algebra: fussion

of Grassman and

Hamilton ideas

William Kingdon Clifford William Rowan Hamilton

Die lineale Ausdehnungslehre

ein neuer Zweig der

Mathematik

Hermann Günther Grassmann

Evolution of Algebras

Conformal Geometric

Algebras

Lie algebras/groups

Directed distance

Projective Geometry

Versors

Homogeneous model

Null Cone

Horosphere (Wachter Gauss`

student 1792-1817)

Conformal Geometric Algebra

Incidence Algebra

Hongbo. Li, David Hestenes and Alyn Rockwood. "Generalized Homogeneous coordinates for

computational geometry".

G. Somer, editor,Geometric Computing with Clifford Algebras. Springer, Chapter I, pp. 1-45, 2001.

Math. Framework

Geometric Product of two vectors

The geometric product of two vectors a y b is written as ab and can be

expressed as the sum of a scalar and a wedge products:

babaab

Clifford Geometric Algebra

scalar bivector

trivector

n-vector

ncba ...

Math. Framework

Geometric Algebra Gp,q,r

In general a geometric algebra Gp,q,r is a linear space of dimension 2n, where

n=p+q+r.

Geometric Product

Let ei and ej be two orthonormal basis vectors of the vector space.

Then the geometric product for these vectors is defined as:

Clifford Geometric Algebra

jieeeee

ee

ijjiij

ji

for

n1,...,qpjifor0

qp1,...,pjifor1

p1,...,jifor1

bivector

Math. Framework

Example the geometric algebra G4,1,0 has 25=32 elements.

12

4

2

3

2

2

2

1 eeee

ee

ee

Note

EII

eeE

eeeI

ec

e

5

4

54

321

:

12

5 e

Conformal Geometric Algebra for 3D space

2)ba(2

1BA metric recovery !!

02

2

21

x

eexxx oee

1 5 10 10 5 1

Scalar Vectors Bivectors Trivectors Tetravectors Pentavectors or

Pseudoscalar

3D 5D

Math. Framework

Math. Framework

p

r X1

X2

X3

X4

0

2

21

sx

eps

c

c r 4321* xxxxS

0* Sxc

cISS *

d n

exxx 321

* den

cI *

Conformal Entities

Sphe

re

Pla

ne

S2 S1

z

321

* xxxz 21 ssz

exxl 21

*

1

2

21 l

cIZZ *

cIll *

Lin

e

Cir

cle

Conformal Entities Math. Framework

Sphere, circle, pair of points and point

Inversion and Dilation

r

eps 2

21 r

x

x’

The reflection of the point in an sphere.

1

21

'

sxsx

ees o

r

The dilation is the product of two

reflections

1

) ln(

2 1 2

2 1

2 2 1

2 1

2 1 2

2 1

'

) )( (

) )( ( ) )( ( ' '

2 1

1

r r

r r

r r

r r

r r

xD D x

e D

e e e e D

e e e e x e e e e x

E

o o

D

o o

D

o o

r

ees o

2

21

2 r

x

x’

ees o 21

1

Math. Framework

Rotation in an arbitrary axis :

The Motor (moment-vector):

MMOO

RIM

RTM

s

t

ss

s

~'

12

Rotor, Translator and Motor

L

s

ltelele

s

ls

eR

eeR

TTRRL

tt

2

2222)))((())1)()(1((

1

A motor for rigid motion of geometric objects O like points, lines, planes, sphere

Translator

I

tT

21

Math. Framework

Projection: sphere to affine plane

2)('

dcz

bazzMz 3R

3R

Mobius transformation result as the projection of

simple combination of motions on the sphere

Move one dimension higher and use

the Riemman sphere !!

By Douglas N. Arnold & Jonathan Rogness

Math. Framework

Conformal transformation in geometric algebra

)}~

)({~

' eTexeTeKxKx aaaa

52

3

2

1)(

oeec eexxxFx

x

In conformal geometry

oaaaac eexxeTexeTeFKxKFxFxax

xxx

2'2

1´)}

~)({()

~()'(

1

1'

Versor sequence!!

Math. Framework

Rotation in an arbitrary axis

Forward kinematics

L

L

ele

l

eM

eM

eM

TTRMtt

2

2

222

~

))1)()(1((

1

L

l

x

T-1

Rl

M

Mot

or

L1

L2 Ln

xp

Kin

em

atic

s ch

ain

s

Motor (dual quaternion) better operator for rigid transformation representation than a matrix representation (R, t)

MMXXtRxx~

´

Differential kinematics

Differential kinematics

Replacing both in previous equation

Differential kinematics

since

Robot Dynamics

Math. Framework

Math. Framework

22

Grad, div, curl and blade-Laplacian

Laplacian

Long term research goal

materials science

Neuroscience

Robotics

sensor stimuli

Brain coding

neocortex

World coordinates

Hyperspherical retina

fusion, storaging

learning

control

action

Ego coordinates

Geometric Cybernetics

Development of algorithms for PAC systems

Body-sensors

calibration

RGB-d cameras

3D reconstruction

navigation

Geometric

control

Model human vision

Learning

Perception

Image Processing

Computer Vision

Grasping, object

manipulation

Humanoid

Body-sensor calibration: relate the coordinate systems of

the sensors with the robot body

Hand-eye calibration in CGA

10101010

BBXXXXAA tRtRtRtRXBAX

A B

X

Using motors

However, according Chen´s theorem the only what matters is the relative

motion of the involved screw lines

Hand-eye calibration

The angle and pitch of the gripper (or endoscope) are equal to the angle and pitch of the camera, they remain

invariant under coordinate transformations, which is known as the Chen’s screw congruence theorem); therefore,

the problem is solved using only the screw axes lines defined by the motors

ld

ed

eM )22

sin()22

cos(

Lines can be represented using bivectors a and a`

Hand-eye calibration

D

The procecedure is repeated for each axis.

Calibrated robot: 3D reconstruction and controlled navigation

Grasping

Robot Manipulador

CINVESTAV

AdeptSix600

(6 DOF)

Barrett Hand

(4 DOF)

Grasping: three contact points

Three conctact points:

the forces met at the

center of mass

Three contact

points:

Forces are parallel

Multiple conctact

points

Visually guide object grasping

Segmentation

Pose estimation

Tracking

Grasping

Humanoid Mechanism and Perception

Long term research goal

materials science

Neuroscience

Robotics

sensor stimuli

Brain coding

neocortex

World coordinates

Cognitive Architecture

fusion, storaging

learning

control

action

Ego coordinates

Mathematical framework

Motivation

Humanoid walking

45

Human versus robot vision system

perception

Visual cortex

Signal flow

Use a stochastic geometric

framework for modelling biologic

concepts and develop algoriths for

intelligent mechatronic systems

Motivation

,

Concepts Lie Algebra of visual perception

Hoffman L. Model of Human Vision system using Lie algebras

1960’s

More complex combinations

the first ~14 years of life

Concepts Combinations of Lie operators for visual flow

Binocular vision Concepts

Human visual cortex has some intrinsic manner

(subgroup of the affine group) to convert this binocular

view of the world into a ciclopean view (without losing

the 3D scene)

E.L. Schwartz, V1-V2-V3

Quasi-conformal. Mapping

(log of Mobiüs transforamtion)

Cyclopean eye

• The Superior Colliculus contains a motor map; activity in a

given region correlates with a particular a saccade-vector.

• The saccade-vectors are independent of the absolute gaze

direction

Fixation, Stereopsis and Vergence

Moving an object slightly

off the horopter creates a small

retinal disparity that stimulates

stereoscopic depth perception

Relationship between the horopter,

Panum’s area and the zone of stereopsis.

Concepts

Source: Enright, 1983. Source: Bell, 1823.

Listings law using rotors and translators

Using rotors

Pure rotations

Motor to Include small displacements

Parameterization of plane deformation

Applications

Listings planes of a binocualr system Tracking of moving object

The computation of the motion axes is done with motors M in G4,1,0.

Body-Camera Calibration using screw lines Applications

Humanoid head

Conformal Model of Humanoid robot Vision

Stereo Rig, Azimuth Plane , 3D Reconstruction

Duals of poncelet points

Applications

Spherical representation of the 3D visual space

Algebra of spheres for selflocalization Applications

Modelbased Humamoid selflocalization Location hypotheses generation mechanism Rcognition

Inverse kinematics

Simplex in conformal GA for 3D reconstruction

simplex

ecba

Applications

Delanauy traingulation

This algorithm computes the axes of rotation based on measurements of lines taken from the real world.

The procecedure is repeated for each axis.

Tilt Vergence

Left and right

Pan

58

Head Control

;A A d A

Ae J A where J

L̂m

o

We obtain the orientation error dynamics as follows

So we need to use a robust control technique,

because of the unknown term .

Defining the orientation error

A de A A

Integral Sliding Modes (ISM) is a control strategy having the following

advantages.

Robustness against parameter uncertainty and external disturbances.

Robustness starting from the initial time instant.

Signal control low gain (Magnitude).

Chattering-free, pre-filtering the discontinuous control term.

dA

59

Designing ISM Controller

Assuming joint velocities as control signals, we can re-define

the error dynamics as:

1 2;A A de J U A where U U U

Using U1 to control “ideal” dynamics, without unknown and disturbance

terms, and U2 to reject these terms. We define the sliding surface as:

AS e z

where z is an auxiliar variable. Then its dynamic is given by:

1 2A dS J U U A z

Defining we obtain 1Az J U

2A dS J U A

then with

1 1

2 2 ( )

A A

A

U K J e

U K J sign S

a stable error system is obtained ( 0)Ae

and with we reject the disturbance terms,

where 1 20 ; .dK K A

60

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

Time (s)

X3

(ra

d)

e1 Red - e2 Blue - e3 Green

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5-0.9

-0.8

-0.7

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

Time (s)

X3

(ra

d)

e1 Red - e2 Blue - e3 Green

Figure A) P I D Figure B) I S M

Comparison of controlers

Estimation of Pose and 3D Structure

L R

Estimation of the orientation

and pose M=RT of the

cameras using Kalman filter

techniques

For

Oculomotorcontrol for:

vergence and tracking

Estimation von M_l=R_lT_l and M_r=R_rT_r

PC

PC

Bonus:

--independency of

errors in sensor-Body

calibration.

--3D reconstruction and

relocalization

1. Egomotion and 3D reconstrucion: object searching Applications

Uisng Kalman filter techniques the rotor R and the translator T are

estimated Cameras work in a master slave fashion by fixation

Cyclopean eye

Vieht-Müller circles

and fixarion points

Locus of ZOD

Estimated rotation angles

Of left and right cameras

Hand-eye calibratio is inaccurate due uncertanties. Free coordinates egomotion estimation

helps for a better selflocalization and 3D reconstruction.

Camera was moved by

hand in a 3D trayectory

1. Egomotion and 3D reconstrucion: object recognition and pose estimation Applications

Fixation point to get stereosis for better

correspondences and frature extracction

Focus of Attention

Object recognition

and pose estimation

Grasping !!

Egomotion and 3D reconstrucion Applications

view starcase for climbing

Egomotion and 3D reconstrucion

Applications

Views by changimg the vergence

Quaternion Spike Neurons

66

Quaternion Spike Neural Network

67

Quaternion Spike Neurons

68

Control of a 6 DOF robot manipulator

Neuronavigator

Segmentation

Neural net to adjust a topological map to the object we are interested in object contour

Gradient Vector Flow input selection (for the NN) and in the adaptation process.

Geometric Algebra each neural unit (neuron) has an associated transformation, which is expressed as a motor (rotation-translation) in the GA

All the motors of the net determine the object shape (when they are applied to a specific point)

Gradient Vector Flow (GVF)

Objective: spread the gradient vectors

GVF is a vector field

Minimizes

where is an image borders map

),(),(),( yxvyxuyx v

dydxffvvuu yxyx

222222 )( v

),( yxf

Big value f

0f

Generalized Gradient Vector Flow (GGVF)

Change the coefficient and by the weight functions

Thus

f

efg

)( )(1)( fgfh

dydxffhfg22 )()( vv

2f

Generalized Gradient Vector Flow (GGVF)

Input

im age

C om pute

the vector field

G G V F

C om pute

stream lines

D eterm ine

the inputs

to the net

Training of

the net

G N G

GGVF

Inputs

Set of

versorsA pply versors

to selected

point to define

the object shape

C onform al

G eom etric

A lgebra

Segm ented

im age or 3D

object

Segmentation

Algorithm (2D) to adjust the topologic map

Every certain number of iterations, insert new neurons Find the neuron ni with the highest value of rsf. If any of its

neighbors, say nj, is connected with an edge larger than cmax, then create a new neuron nnew between ni and nj with

Erase old connections between ni and nj , and create new

connections (edges) from nnew to ni and nj

Repeat previous steps

By this way, we are minimizing the quantification error

)))~

((( 2

0 XMPM

2

ji

new

TTT

2

ji

lnew

vvv

Input

image

Compute

GGVF

Compute

streamlines

Determine inputs

to the net

Neural net

Apply Ti and to

extract the

object shape

Output:

Segmented

image

Without GGVF With GGVF

Without GGVF With GGVF

Quantification error:

1.19

Topographic error:

0.142

Quantification

error: 1.10

Topographic

error: 0.153

Quantification

error: 2.04

Topographic

error: 0.133

Quantification

error: 0.98

Topographic

error: 0.05

Quantification

error: 0.23

Topographic

error: 0.0

Quantification

error: 1.002

Topographic

error: 0.0

Segmentation

Input image GGVF Streamlines

Result Inputs to the net and

initialization

Segmentation

Input image GGVF Streamlines

Result Inputs to the net and

initialization

Segmentation

Advantages against GGVF-Snake

Snake

Segmentation

Algorithm 3D to adjust the topologic map

Every certain number of iterations, insert new neurons Find the neuron ni with the highest value of rsf. If any of its neighbors,

say nj, is connected with an edge larger than cmax, then create a new neuron nnew between ni and nj with

Erase old connections between ni and nj , and create new

connections (edges) from nnew to ni and nj

Repeat previous steps

By this way, we are minimizing the quantification error

)))~

((( 2

0 XMPM

Surface modeling using Neural Nets

Result with

100 neurons

Result with

170 neurons

Results with volumetric data

Result with 100 neurons

Result with 200 neurons Result with 300 neurons

Inputs

3D Case Models using circles / spheres

Marching Spheres

Models based on spheres

0

2

0

2

0

2

)(2

1

)(2

1)(

2

1

eeccS

eeccSeeccS

iiii

iiiiiiii

ggg

j

g

mmm

j

mppp

j

p

r

rr

Tumor Model based on spheres

n / d Number of spheres with each

approach

UoS – DT Proposed

3370 / 1 13480 11866

3370 / 3 8642 2602

Motor Interpolation of points, lines, planes, spheres

89

Motor interpolation of the pose

90

Real time interpolation

91

Quaternionic Phase

Quaternion Wavelet Transform

QWT Is a natural extension of the real and complex wavelet transforms

Real

Complex

Quaternionic

Hypercomplex

Quaternion Gabor Filters

Quaternion Gabor Filter

)/()/()2/2/( 221122

221

2

)( ycjxciyx

eeexG

Low-Pass (G)

High-Pass (H)

Quaternion Gabor Wavelets with seletive orientations

15 45 75 –75 –45 –15 degrees

r

I

J

k

Quaternionic Wavelet Pyramid

Motion Detection

Horizontal and Vertical disparity is calculated with the formulas

With reference frequency

Confidence

A Confidence Matrix is calculated for each level as

With

Motion Estimation Algorithm

Motion Estimation Algorithm

QWT

Original Sequence

Ph

ase

Disparity

All the levels of the

pyramid are

combined

Results

Results

Original Images

Confidence in 4 levels

Resulting Vectors

Combination

Disparity levels are combinated

Horizontal Disparity Vertical Disparity

An Experiment

Original Images

Confidence in 4 levels

Resulting Vectors

Comparison

Bernard Algorithm, level 4 Using the QWT, level 4

Atomic function

106

The automic function up(x) is generated by infinite

convolutions of rectangular impulses. The fucntion up(x)

has the following representation in terms of the Fourier

transform.

Quaternion analytic signal

107

Monogenic signal

108

Hilbert and Riesz Transforms using Atomic Function

109

Riesz transform

Hilbert transfrom

Atomic Function Quaternion Hilbert transform

110

Atomic Function Riesz transform

111

Image Local phase orientation

Multi-scale Riesz transform for symmetry feature extraction

112 Atomic filters in spatial and frequency domains at two levels of the multiresolution pyramid.

113

Multi-scale Riesz transform for symmetry feature extraction

Feature extraction

114

Conformal Geometric Algebra Voting Scheme.

In computer vision, perceptual organization consists in obtaining meaningful

representations from row visual data.

We have reformulated the Tensor Voting algorithm [2] using representations

in terms of k-vectors of the Conformal Geometric Algebra, which allow us to

extract any kind of geometric entities or flags from images.

The essential components of this voting scheme can be summarized in two

methodologies:

◦ To represent information using CGA.

◦ To communicate information via a voting process and a clustering

technique.

115 [2] P. Mordohai and G. Medioni, Tensor voting. a perceptual organization approach to computer vision and machine learning.,

Synthesis Lectures on Image, Video, and Multimedia Processing., Morgan & Claypool Publishers, 2006.

Representation of salient circles and lines.

Let p0 be a pixel, then we want to compute the perceptual structure F, that

satisfy:

In order to compute F, all tokens in the neighborhood of p0 cast a vote. Each

vote is codify as a salient geometric entity.

Definition: A salient geometric entity is a set of points together with a

function, that assigns a scalar value to each element of the set.

Using CGA G3,1 a salient line is represented by:

where pc and lc are a point, and a line, respectively, in conformal

representation.

116

The geometric structure of

the vote from pi to p0 is:

The perceptual saliency is

defined by:

where:

Representation of salient circles and lines.

117

Voting process for extraction of circles and lines.

118

119

Voting process for extraction of circles and lines

Experimental results for extraction of circles and lines

120

Experimental results for extraction of circles and lines.

121

And the perceptual saliency of a symmetry axis is:

Where: ◦ pjc is the nearest pixel to the point LpiL.

◦ sigma is a parameter that fix the size of the neighbourhood in which we look for point pjc.

For each pair of points: For each pair of lines:

Representation of information for detection of bilateral symmetry

122

And the perceptual saliency of a symmetry axis is:

Where: ◦ pjc is the nearest pixel to the point LpiL.

◦ sigma is a parameter that fix the size of the neighbourhood in which we look for point pjc.

For each pair of intersection lines: For each pair of parallel lines

Representation of information for detection of bilateral symmetry

123

124

Voting process for extraction of symmetry axis

Experimental results for extraction of symmetry axis

125

128

edb@gdl.cinvestav.mx

http://www.gdl.cinvestav.mx

Geometric Computing for Cybernetics

Vol I

Geometric

Algebra for

Image

Processing,

Computer

Vision and

Machine

Learning

SpringerVerlag

Vol II

Geometric

Algebra

for Robotics

Springer Verlag

Vol IIiI

Integral

Transforms

for Science and

Engineering

using

Geometric

Algebra

Springer Verlag

(In preparation)