Video Processing Video Processing EN292EN292

Class ProjectClass Project

By Anat KaspiBy Anat Kaspi

The GoalThe GoalTracking vehicles by estimating the location of the Tracking vehicles by estimating the location of the vehicle’s front planevehicle’s front plane

Stereo tracking - using two pairs of cameraStereo tracking - using two pairs of camera

AssumptionsAssumptions

The front part of the car is plannerThe front part of the car is planner

Car is moving in straight line - only translating in the x Car is moving in straight line - only translating in the x directiondirection

The video is taking by two synchronized camerasThe video is taking by two synchronized cameras

The AlgorithmThe Algorithm

Refine the location by searching for the best location

in square error meaning

Determine the starting location point for frame n+1

Estimating the vehicle motion from frame n to frame n+1

using Kalman Filter

Starting location for the vehicle at frame n

Set upSet upCollecting videos with two cameras – stereoCollecting videos with two cameras – stereo

Mark the road for reference pointsMark the road for reference points

Calibration for the two camerasCalibration for the two cameras

CalibrationCalibration

Using calibration tool in VXL - …\brl\bmvl\bmvv\Using calibration tool in VXL - …\brl\bmvl\bmvv\mvbin\calmvbin\cal

Using know points on the roadUsing know points on the road

Estimating the plane locationEstimating the plane locationAssumptionsAssumptions

Projected world point of Lambertian surface into two images will Projected world point of Lambertian surface into two images will have the same intensity in both imageshave the same intensity in both images

Create Synthetic images from the stereo pair in order to have Create Synthetic images from the stereo pair in order to have Lambertian surface Lambertian surface

ProcessProcess Edge detection on the images Edge detection on the images

more stable more stable Create binary image from the edge mapCreate binary image from the edge map Smooth the image – relative distanceSmooth the image – relative distance

Estimating the plane locationEstimating the plane location

Looking to minimize the overall errorLooking to minimize the overall error

Sample points on the planeSample points on the plane

(x,y,z,1) – world point(x,y,z,1) – world pointP_L, P_R – projection matrixP_L, P_R – projection matrix

2)1,,,()1,,,( zyxPIzyxPIError rightleft

Motion Estimation Motion Estimation

Using Kalman filterUsing Kalman filter - prediction and correction loop - prediction and correction loopThe General modelThe General modelState dynamicsState dynamics

X(n+1) = A(n)*X(n)+W(n)X(n+1) = A(n)*X(n)+W(n)

Observation modelObservation model

Y(n) = X(n)+V(n)Y(n) = X(n)+V(n)Update equationUpdate equation

Weighted average of the present value and the present observationWeighted average of the present value and the present observation

)()()(ˆ)(1)()1(ˆ nynGnxnGnAnx

1222 )()()()(

nnnnG v

)()()(1)()()1( 222 nnAnGnnAn wT

Motion EstimationMotion EstimationX(n) – distance the vehicle is traveling between two framesX(n) – distance the vehicle is traveling between two frames

Only translating – x(n) scalarOnly translating – x(n) scalar

The motion modelThe motion model

X(n+1) = X(n)+W(n) X(n+1) = X(n)+W(n)

W(n)~N(0,sigma_w) white noiseW(n)~N(0,sigma_w) white noise

The observation modelThe observation model

Y(n) = X(n)+V(n) Y(n) = X(n)+V(n)

V(n)~N(0,sigma_v) white noiseV(n)~N(0,sigma_v) white noise

Update equationUpdate equation

)()()(ˆ)(1)1(ˆ nynGnxnGnx

)()(

)()( 22

2

nn

nnG

v

)()(1)()1( 222 nnGnn w

211 1

11

nnn ynn

n

Results…Results…

Running software in class…

Thank you !Thank you !