A New Block Based Motion Estimation with True Region Motion Field

Jozef Huska & Peter KullaEUROCON 2007 The International Conference on “Computer as a Tool”

Outline Introduction Ambiguity in Motion Estimation Yielding from Optical

Flow Used Models and Estimation Criteria Region Based Multi-Stage Motion Estimation

Framework for the Algorithm Detection of the Regions, Corner, Edge and Flat Blocks Motion Vector Field Interpolation and Refinement Stage Simulation Results Conclusions

Introduction True motion vector fields are usually

smooth, both spatially and temporally.

To Finding the optimal trade off between minimizing the entropy of vectors and minimizing the displaced frame difference.

Ambiguity in Motion Estimation Yielding from Optical Flow Assumption : image point has the same

intensity along the trajectory of movement:

The consequences: 1) Underdetermined component 2) Aperture problem 3) Indeterminate flow vector

1) Underdetermined component: There is only one equation for two unknowns (vx and vy). Solution: apply motion measurement to a block of pixels.

2) Aperture problem: The aperture including only one spatial gradient. Solution: the estimated aperture contains

at least two different gradient directions. 3) Indeterminate flow vector:

In regions with constant brightness so that ∇I = 0. The estimation of motion is reliable only in regions with

edges or non-flat textures.

Ambiguity in Motion Estimation Yielding from Optical Flow (cont.)

Used Models and Estimation Criteria Considerations of a new motion estimator for

the video coding systems: The goal is to estimate the motion of image points. Motion is estimated based on the variations of

intensity and color in image points. The motion of neighboring image points within the

object’s projection area is very similar. Linear block trajectory model for temporal

motion :

Region Based Multi-Stage Motion Estimation Framework for the Algorithm

Strategies to get around the ambiguities while keeping the prediction error: Applying motion estimation to block of pixels. (1 Estimation of motion vector fields only for corner

blocks(two spatial gradients directions). reliable For edge blocks is the motion vector value first

interpolated from the corner blocks. (2 For flat blocks interpolation are token corner and

edge blocks. (3

The interpolated motion fields based on sparse reliable estimates are more accurate than unreliable dense estimates.

Interpolation concepts: Interpolation of the motion vectors is done with

vectors within the own region. Motion vector field

inside each region would be smooth.

Region Based Multi-Stage Motion Estimation Framework for the Algorithm

Detection of the Regions, Corner, Edge and Flat Blocks Corner detection algorithm:

Covariance matrix A :

The eigenvalues :

Two large eigenvalues of A represent corners.

An image is segmented into regions with the graph based segmentation [5].

The ability of segmentation is preserving detail in low variability image regions while ignoring detail in high variability regions.

Detection of the Regions, Corner, Edge and Flat Blocks (cont.)

[5] P. F. Felzenszwalb, D. P. Huttenlocher, "Efficient Graph-Based Image Segmentation," International Journal of Computer Vision, vol. 59, no. 2, Sept. 2004.

Segmentation parameters: Sigma(σ)= 0.5, K = 1000, Min = 100

Detection of the Regions, Corner, Edge and Flat Blocks (cont.2)

Motion Vector Field Interpolation and Refinement Stage

The MVs of corner blocks are estimated with the diamond search (DS).

Interpolation of the MVs for the edge blocks is done with the MVs of the corner blocks.

First, found four nearest corner blocks with MVs {dxi , dyi} from the same region.

The interpolated MV value is:

|RIi| : the distance between the reference block and currently interpolated block.

Interpolation of the MVs for the flat blocks is done with the same function, but consider corner and edge blocks as the reference.

Motion Vector Field Interpolation and Refinement Stage (cont.)

Predictive diamond search [19] was used for the refinement with small modification.

Like candidates in 3DRS algorithm, there is used interpolated motion vector .

Motion Vector Field Interpolation and Refinement Stage (cont.2)

Algorithm description

DS : diamond search,MV : motion vector,R : count of regions,C : count of corner locks, E : count of edge blocks, F : count of flat blocks.

Simulation Results Real sequence:

Simulation Results

Synthetic sequence:

Simulation Results

Vc : correct velocityVe : estimated velocity

Conclusions In this paper a new block based true motion

estimation algorithm named true region motion field (TRMF) search was presented.

In future work we would like slightly reduce computation demands of our TRMF search and increase output quality.