Some problems...
description
Transcript of Some problems...
![Page 1: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/1.jpg)
![Page 2: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/2.jpg)
![Page 3: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/3.jpg)
Some problems...
![Page 4: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/4.jpg)
Lens distortion
Uncalibrated structure and motion recovery assumes pinhole cameras
Real cameras have real lenses
How can we correct distortion, when original calibration is inaccessible?
![Page 5: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/5.jpg)
1. Even small amounts of lens distortion can upset uncalibrated structure from motion
2. A single distortion parameter is enough for mapping and SFX accuracy
3. Including the parameter in the multiview relations changes the 8-point algorithm from
4. You can solve such “Polynomial Eigenvalue Problems”
5. This is as stable as computation of the Fundamental matrix, so you can use it all the time.
![Page 6: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/6.jpg)
Even small amounts of lens
distortion can upset uncalibrated structure from motion—
![Page 7: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/7.jpg)
A map-building problem
(a) Input movie – relatively low distortion(b) Plan view: red is structure, blue is motion
(a) (b)
![Page 8: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/8.jpg)
Effects of Distortion
(a) Input movie – relatively low distortion(b) Recovered plan view, uncorrected distortion
(a) (c)
![Page 9: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/9.jpg)
Does distortion do that?
Distortion of image plane is conflated with focal lengthwhen the camera rotates
[From: Tordoff & Murray, ICPR 2000]
![Page 10: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/10.jpg)
Distortion correction in man-made scenes
![Page 11: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/11.jpg)
Distortion correction in natural scenes
In natural images, distortion introduces correlations in frequency domain
Choose distortion parameters to minimize correlations in bispectrum
Less effective on man-made scenes....
[Farid and Popescu, ICCV 2001]
![Page 12: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/12.jpg)
Distortion correction in multiple images
Multiple views, static scene• Use motion and scene rigidity [Zhang, Stein,
Sawhney, McLauchlan, ...]Advantages:• Applies to man-made or natural scenesDisadvantages:• Iterative solutions|require initial estimates
![Page 13: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/13.jpg)
A single distortion parameter
is accurate enough for map-building and cinema post production—
![Page 14: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/14.jpg)
Modelling lens distortion
x: xeroxednoxious
experimental artifax
p: perfect pinhole
perspective pure
xp p
x
Known Unknown
![Page 15: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/15.jpg)
Single-parameter models
![Page 16: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/16.jpg)
Single-parameter modelling power
Single-parameter model
Radial term onlyAssumes distortion
centre is at centre of image
A one-parameter model suffices
![Page 17: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/17.jpg)
A direct solution for
![Page 18: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/18.jpg)
Look at division model again
![Page 19: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/19.jpg)
![Page 20: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/20.jpg)
>> help polyeig
POLYEIG Polynomial eigenvalue problem.
[X,E] = POLYEIG(A0,A1,..,Ap) solves the polynomial eigenvalue problem
of degree p:
(A0 + lambda*A1 + ... + lambda^p*Ap)*x = 0.
The input is [etc etc...]
>>
A quick matlab session
![Page 21: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/21.jpg)
Algorithm
![Page 22: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/22.jpg)
T his is as stable as
computation of the fundamental matrix, so you can use it all the time—
![Page 23: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/23.jpg)
Performance: Synthetic data
0 0.2 0.4 0.6 0.8 1-0.4
-0.3
-0.2
-0.1
0
Noise (pixels)
Com
pu
ted
• Stable – small errorbars• Biased – not centred on true value
![Page 24: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/24.jpg)
Analogy: Linear ellipse fitting
True
Data
Fitted: 10 trials
Best-fit line
![Page 25: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/25.jpg)
Performance: Synthetic data
![Page 26: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/26.jpg)
Performance: Real sequences
![Page 27: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/27.jpg)
-0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.30
10
20
30
40
50
• 250 pairs• Low distortion• Linear estimate used to initialize nonlinear• Number of inliers changes by [-25..49]
![Page 28: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/28.jpg)
Conclusions
![Page 29: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/29.jpg)
![Page 30: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/30.jpg)
Environment matting
In: magnifying glass moving over background
Out: same magnifying glass, new background
![Page 31: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/31.jpg)
Environment matting: why?
• Learn– light-transport
properties of complex optical elements
• Previously– Ray tracing
geometric models– Calibrated
acquisition
• Here– Acquisition in situ
![Page 32: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/32.jpg)
Image formation model
• Purely 2D-2D– Optical element performs weighted sum of (image of)
background at each pixel
– suffices for many interesting objects
– separate receptive field for each output pixel
– Environment matte is collection of all receptive fields—yes, it’s huge.
![Page 33: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/33.jpg)
Image formation model
![Page 34: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/34.jpg)
Step 1: Computing backgroundInput:
Mosaic:
Clean plate:Point tracks:
![Page 35: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/35.jpg)
Step 2: Computing w...Input:
![Page 36: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/36.jpg)
Computing w(x,y,u,v) at a single (x,y)
![Page 37: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/37.jpg)
Assume wi independent
![Page 38: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/38.jpg)
Composite over new background
![Page 39: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/39.jpg)
A more subtle exampleInput: Two images
Moving cameraPlanar background
- Need priors
![Page 40: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/40.jpg)
Window example
![Page 41: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/41.jpg)
Discussion
• Works well for non-translucent elements– need to develop for diffuse
• Combination assumes independence– ok for large movements: “an edge crosses
the pixel”
• Need to develop for general backgrounds
![Page 42: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/42.jpg)
![Page 43: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/43.jpg)
A Clustering Problem
• Watch a movie, recover the cast list– Run face detector on every frame– Cluster faces
• Problems– Face detector unreliable– Large lighting changes– Changes in expression– Clustering is difficult
![Page 44: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/44.jpg)
A sample sequence
![Page 45: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/45.jpg)
Detected faces
![Page 46: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/46.jpg)
Face positions
![Page 47: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/47.jpg)
Lighting correction
![Page 48: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/48.jpg)
Clustering: pairwise distances
Raw distance
![Page 49: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/49.jpg)
Clustering: pairwise distances
Transform-invariant distance
![Page 50: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/50.jpg)
Clusters: “tangent distance”
![Page 51: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/51.jpg)
Clusters: Bayesian tangent distance
![Page 52: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/52.jpg)
Conclusions
• Extend to feature selection, texton clustering etc
• Remove face detector
![Page 53: Some problems...](https://reader036.fdocuments.in/reader036/viewer/2022062519/56815418550346895dc2142d/html5/thumbnails/53.jpg)