Effective Optical Flow Estimation Jan Kamenický21.10.2011.
-
Upload
imogene-tucker -
Category
Documents
-
view
215 -
download
2
Transcript of Effective Optical Flow Estimation Jan Kamenický21.10.2011.
![Page 1: Effective Optical Flow Estimation Jan Kamenický21.10.2011.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f285503460f94c40add/html5/thumbnails/1.jpg)
Effective Optical Flow Estimation
Jan Kamenický 21.10.2011
![Page 2: Effective Optical Flow Estimation Jan Kamenický21.10.2011.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f285503460f94c40add/html5/thumbnails/2.jpg)
Motivation
![Page 3: Effective Optical Flow Estimation Jan Kamenický21.10.2011.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f285503460f94c40add/html5/thumbnails/3.jpg)
Usage
• Motion detection• Object segmentation• Video encoding (compression)• Stereo disparity measurement
![Page 4: Effective Optical Flow Estimation Jan Kamenický21.10.2011.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f285503460f94c40add/html5/thumbnails/4.jpg)
Optical flow equation
• Color constancy
• Taylor series
• Optical flow equation
![Page 5: Effective Optical Flow Estimation Jan Kamenický21.10.2011.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f285503460f94c40add/html5/thumbnails/5.jpg)
Estimating optical flow
• Basic equation
• Local methods– Lucas & Kanade• flow field is locally constant (or affine)• least squares minimization• cannot handle interior parts of objects
![Page 6: Effective Optical Flow Estimation Jan Kamenický21.10.2011.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f285503460f94c40add/html5/thumbnails/6.jpg)
Estimating optical flow
• Basic equation
• Global methods– Horn & Schunk
• more sensitive to noisedata term smoothing term
![Page 7: Effective Optical Flow Estimation Jan Kamenický21.10.2011.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f285503460f94c40add/html5/thumbnails/7.jpg)
• Data term– non-homogenous surface (shading, reflections)– non-flat scene / non-uniform lighting– spatial discontinuities
• Smoothness term– discontinuities (moving objects boundaries)
Problems
![Page 8: Effective Optical Flow Estimation Jan Kamenický21.10.2011.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f285503460f94c40add/html5/thumbnails/8.jpg)
Estimating partial derivatives
• Discrete approximation by differences– forward, backward – not exact– use 2x2x2 cube in (x,y,t) space• compute the difference as an average of 4 adjacent first
order differences
– use larger support• e.g. [1, -8, 0, 8, -1]/12
i i+1
j
j+1
kk+1
![Page 9: Effective Optical Flow Estimation Jan Kamenický21.10.2011.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f285503460f94c40add/html5/thumbnails/9.jpg)
Data term
• L2 norm
• L1 norm
• Many modifications– generalized Charbonnier
![Page 10: Effective Optical Flow Estimation Jan Kamenický21.10.2011.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f285503460f94c40add/html5/thumbnails/10.jpg)
Regularization term
• Enforces smooth flow field• Similar norms can be used– L2, L1 (total variation), …
• Other possibilities– Laplacian instead of gradient
![Page 11: Effective Optical Flow Estimation Jan Kamenický21.10.2011.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f285503460f94c40add/html5/thumbnails/11.jpg)
Dealing with larger displacements
• Smoothing (blurring)– usually Gaussian kernel– decreases flow field accuracy
• Pyramidal approach– compute flow on down-sampled images– up-sample the flow to next level– compute the warping (using the optical flow)– repeat
![Page 12: Effective Optical Flow Estimation Jan Kamenický21.10.2011.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f285503460f94c40add/html5/thumbnails/12.jpg)
More optimizations
• Graduated non-convexity– iteratively move from convex energy function to
the more robust non-convex form
• Median filtering (5x5)– weighted modification
• More warping steps on one pyramid level
![Page 13: Effective Optical Flow Estimation Jan Kamenický21.10.2011.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f285503460f94c40add/html5/thumbnails/13.jpg)
OF methods comparison
• Optical flow estimation benchmark– http://vision.middlebury.edu/flow/
• Average end-point error
![Page 14: Effective Optical Flow Estimation Jan Kamenický21.10.2011.](https://reader035.fdocuments.in/reader035/viewer/2022062722/56649f285503460f94c40add/html5/thumbnails/14.jpg)
References
• Main described method– D. Sun, S. Roth, M. J. Black: Secrets of Optical Flow
Estimation and Their Principles, CVPR 2010– http://www.cs.brown.edu/~black/code.html