Motion AnalysisMotion Analysis

Slides are from RPI Registration Slides are from RPI Registration Class.Class.

22

RegistrationRegistration

Images are described as discrete sets of Images are described as discrete sets of point locations associated with a point locations associated with a geometric measurementgeometric measurement Locations may have additional properties Locations may have additional properties

such as intensities and orientationssuch as intensities and orientations

Registration problem involves two parts:Registration problem involves two parts: Finding correspondences between featuresFinding correspondences between features Estimating the transformation parameters Estimating the transformation parameters

based on these correspondencesbased on these correspondences

33

NotationNotation

Set of moving image featuresSet of moving image features

Set of fixed image featuresSet of fixed image features

Each feature must include a point location Each feature must include a point location in the coordinate system of its image. Set in the coordinate system of its image. Set of correspondences:of correspondences:

44

Error objective function depends on unknown Error objective function depends on unknown transformation parameters transformation parameters andand unknown unknown feature correspondencesfeature correspondences Each may depend on the other!Each may depend on the other!

Transformation may include mapping of more Transformation may include mapping of more than just locations than just locations Distance function, Distance function, D, D, could be as simple as the could be as simple as the Euclidean distance between location vectors.Euclidean distance between location vectors.

Mathematical FormulationMathematical Formulation

55

Correspondence ProblemCorrespondence ProblemDetermine correspondences Determine correspondences beforebefore estimating transformation parametersestimating transformation parameters Based on rich description of featuresBased on rich description of features Error proneError prone

Determine correspondences Determine correspondences at the same at the same timetime as estimation of parameters as estimation of parameters ““Chicken-and-egg” problemChicken-and-egg” problem

For the next few minutes we will assume a For the next few minutes we will assume a set of correspondences is given and set of correspondences is given and proceed to the estimation of parametersproceed to the estimation of parameters We will return to the correspondence problem We will return to the correspondence problem

laterlater

66

Example: Estimating Example: Estimating ParametersParameters

2d point locations:2d point locations:

Similarity transformation:Similarity transformation:

Euclidean distance:Euclidean distance:

77

Putting This TogetherPutting This Together

88

What Do We Have?What Do We Have?Least-squares objective functionLeast-squares objective functionQuadratic function of each parameterQuadratic function of each parameterWe can We can Take the derivative with respect to each Take the derivative with respect to each

parameterparameter Set the resulting gradient to Set the resulting gradient to 0 0 (vector)(vector) Solve for the parameters through matrix Solve for the parameters through matrix

inversioninversion

We’ll do this in two forms: component We’ll do this in two forms: component and matrix/vectorand matrix/vector

99

Component Derivative (a)Component Derivative (a)

1010

Component Derivative (b)Component Derivative (b)

At this point, we’ve dropped the leading factor of 2. It will be eliminated when this is set to 0.

1111

Component Derivatives tComponent Derivatives txx and t and tyy

1212

GatheringGathering

Setting each of these equal to 0 we obtain Setting each of these equal to 0 we obtain a set of 4 linear equations in 4 unknowns. a set of 4 linear equations in 4 unknowns. Gathering into a matrix we have:Gathering into a matrix we have:

1313

SolvingSolving

This is a simple equation of the formThis is a simple equation of the form

Provided the 4x4 matrix X is full-rank Provided the 4x4 matrix X is full-rank (evaluate SVD) we easily solve as(evaluate SVD) we easily solve as

1414

Matrix VersionMatrix Version

We can do this in a less painful way by We can do this in a less painful way by rewriting the following intermediate rewriting the following intermediate expression in terms of vectors and expression in terms of vectors and matrices:matrices:

1515

Matrix Version (continued)Matrix Version (continued)

Equal to zero. Take transpose on either side..

1616

Matrix Version (continued)Matrix Version (continued)

Taking the derivative of this wrt the Taking the derivative of this wrt the transformation parameters (we didn’t transformation parameters (we didn’t cover vector derivatives, but this is fairly cover vector derivatives, but this is fairly straightforward):straightforward):

Setting this equal to Setting this equal to 00 and solving yields: and solving yields:

1717

Comparing the Two VersionsComparing the Two Versions

Final equations are identical (if you Final equations are identical (if you expand the symbols)expand the symbols)

Matrix version is easier (once you have Matrix version is easier (once you have practice) and less error pronepractice) and less error prone

Sometimes efficiency requires hand-Sometimes efficiency requires hand-calculation and coding of individual terms calculation and coding of individual terms

1818

Registration (rigid+scale)Registration (rigid+scale)Given two point sets Given two point sets

Our goal is to find the similarity Our goal is to find the similarity transformation that best aligns the transformation that best aligns the moments of the two point sets.moments of the two point sets.

In particular we want to findIn particular we want to find The rotation matrix The rotation matrix RR, t, t The scale factor The scale factor ss, and , and The translation vector The translation vector tt

1919

The first moment is the center of massThe first moment is the center of mass

2020

Second MomentsSecond Moments

Scatter matrixScatter matrix

Eigenvalue / eigenvector decomposition (Eigenvalue / eigenvector decomposition (SSpp))

where where ii are the eigenvalues and the columns are the eigenvalues and the columns of of VVpp hold the corresponding eigenvectors: hold the corresponding eigenvectors:

2121

Rotation, Translation and ScaleRotation, Translation and Scale

Rotated, uniformly scaled points:Rotated, uniformly scaled points:

Center of the rotated data is the same as Center of the rotated data is the same as the rotated, scaled and translated center the rotated, scaled and translated center before rotation:before rotation:

2222

Scatter MatrixScatter Matrix

Rotated, scaled scatter matrix:Rotated, scaled scatter matrix:

2323

Eigenvectors and EigenvaluesEigenvectors and Eigenvalues

An eigenvector is a favorite direction for a An eigenvector is a favorite direction for a matrix.matrix.Any square matrix Any square matrix MM has at least one nonzero has at least one nonzero vector vector vv which is mapped in a particularly simple which is mapped in a particularly simple wayway

M v = M v = v v vv is eigenvector of is eigenvector of AA and and is is corresponding eigenvaluecorresponding eigenvalueIf If > 0, > 0, M vM v is parallel to is parallel to vv. If . If < 0, they are < 0, they are antiparallel. If zero, v is in the nullspace.antiparallel. If zero, v is in the nullspace.In all cases, In all cases, M vM v is just a multiple of is just a multiple of vv..

2424

Eigenvalues and EigenvectorsEigenvalues and Eigenvectors

Given an eigenvector Given an eigenvector vvjj of of SSp p and its rotation and its rotation vvjj’ = ’ = RvRvjj::

So, So, Eigenvectors are rotated eigenvectorsEigenvectors are rotated eigenvectors Eigenvalues are multiplied by the square of Eigenvalues are multiplied by the square of

the scalethe scale

So, So, Eigenvectors are rotated eigenvectorsEigenvectors are rotated eigenvectors Eigenvalues are multiplied by the square of Eigenvalues are multiplied by the square of

the scalethe scale

Multiply both sides by sMultiply both sides by s22RR, and manipulate:, and manipulate:Multiply both sides by sMultiply both sides by s22RR, and manipulate:, and manipulate:

R is rotation matrix

j

2525

The processThe process

compute similarity transformation that best aligns the first and compute similarity transformation that best aligns the first and second momentssecond momentsAssume the 2nd moments (eigenvalues) are distinctAssume the 2nd moments (eigenvalues) are distinctProcedure:Procedure:

Center the data in each coordinate systemCenter the data in each coordinate system Compute the scatter matrices and their eigenvalues and Compute the scatter matrices and their eigenvalues and

eigenvectorseigenvectors

Using this, computeUsing this, compute Scaling and rotationScaling and rotation Then, translationThen, translation

2626

RotationRotation

The rotation matrix should align the The rotation matrix should align the eigenvectors in order:eigenvectors in order:

Manipulating:Manipulating:Manipulating:Manipulating:

As a result:As a result:As a result:As a result:

2727

ScaleScale

Ratios of corresponding eigenvalues should determine the scale:Ratios of corresponding eigenvalues should determine the scale:

Taking the derivative with respect to sTaking the derivative with respect to s22, which we are treating at , which we are treating at Taking the derivative with respect to sTaking the derivative with respect to s22, which we are treating at , which we are treating at

Setting the result to 0 and solving yieldsSetting the result to 0 and solving yieldsSetting the result to 0 and solving yieldsSetting the result to 0 and solving yields

2828

TranslationTranslation

Once we have the rotation and scale we Once we have the rotation and scale we can compute the translation based on the can compute the translation based on the centers of mass:centers of mass:

2929

SummarySummary

Calculations are straightforward, non-iterative Calculations are straightforward, non-iterative and do not require correspondences:and do not require correspondences: Moments, firstMoments, first Rotation and scale separatelyRotation and scale separately TranslationTranslation

Assumes the viewpoints of the two data sets Assumes the viewpoints of the two data sets coincide approximatelycoincide approximately Can fail miserably for significantly differing viewpointsCan fail miserably for significantly differing viewpoints