Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1
description
Transcript of Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1
![Page 1: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/1.jpg)
Combining Laser ScansCombining Laser Scans
Yong Joo KilYong Joo Kil11, Boris Mederos, Boris Mederos22, and Nina Amenta, and Nina Amenta11
1 1 Department of Computer Science, University of California at DavisDepartment of Computer Science, University of California at Davis22 Instituto Nacional de Matematica Pura e Aplicada - IMPA Instituto Nacional de Matematica Pura e Aplicada - IMPA
IDAV IDAV Institute for Data Analysis and VisualizationInstitute for Data Analysis and VisualizationVisualization and Graphics Research GroupVisualization and Graphics Research Group
![Page 2: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/2.jpg)
2D Super Resolution2D Super Resolution
A Fast Super-Resolution Reconstruction Algorithm, [Michael Elad, Yacov Hel-Or]
Low Resolution Images Super Resolution Image
![Page 3: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/3.jpg)
Surface Super ResolutionSurface Super Resolution
One Raw Scan Super resolved (100 scans) Photo
![Page 4: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/4.jpg)
Improve 3D Acquisition MethodsImprove 3D Acquisition Methods
• Better hardware– Costly
• Multiple scans + software– Refine output of current hardware – Cost effective– Smaller devices
![Page 5: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/5.jpg)
Physical SetupPhysical Setup
xy
z (viewing
direction)
Minolta Vivid 910
![Page 6: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/6.jpg)
3D Super Resolution Pipeline3D Super Resolution Pipeline
Input Scans Global Registration
Super Resolution
Super Registration
Convergence
No
Yes
Smoothing Super Resolution Mesh
![Page 7: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/7.jpg)
Viewing direction axisViewing direction axis
z
x
y
![Page 8: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/8.jpg)
Sample PointsLow Resolution Sample SpacingSample PointsLow Resolution Sample Spacing
WidthOf one Scan
![Page 9: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/9.jpg)
Super Resolution Sample SpacingSuper Resolution Sample Spacing
q
N(q)width/4
![Page 10: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/10.jpg)
2.5D Super Resolution2.5D Super Resolution
![Page 11: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/11.jpg)
First Super Resolution Mesh (S1)First Super Resolution Mesh (S1)
![Page 12: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/12.jpg)
Super Resolution MethodSuper Resolution Method
Input Scans Global Registration
Super Resolution
Super Registration
Convergence
No
Yes
Smoothing Super Resolution Mesh
![Page 13: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/13.jpg)
Bilateral FilterBilateral Filter
![Page 14: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/14.jpg)
Super Resolution MethodSuper Resolution Method
Input Scans Global Registration
Super Resolution
Super Registration
Convergence
No
Yes
Smoothing Super Resolution Mesh
![Page 15: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/15.jpg)
Super RegistrationSuper Registration
raw scan super resolution mesh
![Page 16: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/16.jpg)
Second Super Resolution Mesh S2Second Super Resolution Mesh S2
![Page 17: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/17.jpg)
Super Resolution MethodSuper Resolution Method
Input Scans Global Registration
Super Resolution
Super Registration
Convergence
No
Yes
Smoothing Super Resolution Mesh
![Page 18: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/18.jpg)
Point Samples (1st Model)Point Samples (1st Model)
Derived from Super-Resolution Reconstruction of Images - Static and Dynamic Paradigms [Michael Elad]
Nyquist Sampling Theorem:Sample signal finely enough, thenReconstruct original signal perfectly.
Band limited signal
![Page 19: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/19.jpg)
Sampling at lower resolutionSampling at lower resolution
Derived from Super-Resolution Reconstruction of Images - Static and Dynamic Paradigms [Michael Elad]
That’s it!
![Page 20: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/20.jpg)
Linear Model with Blur (2nd Model)Linear Model with Blur (2nd Model)
Nkkkkkk EXY 1 FCD
High-ResolutionImage X
Derived from Super-Resolution Reconstruction of Images - Static and Dynamic Paradigms [Michael Elad]
C
Blur
1 D1
Decimation
Low-Resolution
Images
Transformation
F1
Y1E1
Noise
+
CNFN DN
YNEN+
![Page 21: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/21.jpg)
Nkkkkkk EX 1Y FCD
The Model as One Equation
NNNNN E
E
E
X
Y
Y
Y
2
1
222
111
2
1
FCD
FCD
FCD
EX HY
Derived from Super-Resolution Reconstruction of Images - Static and Dynamic Paradigms [Michael Elad]
![Page 22: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/22.jpg)
Model for 3D laser scan? Model for 3D laser scan?
![Page 23: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/23.jpg)
Pipeline : Laser Scanner Pipeline : Laser Scanner
Derived from Better Optical Triangulation through Spacetime Analysis, Curless and Levoy, 1995
laser beam
SurfacePeak reconstructionCCD sensor
![Page 24: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/24.jpg)
Video sequenceVideo sequencex
y
time
![Page 25: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/25.jpg)
Non Linear functionsNon Linear functions
f ( ) =
g ( ) =
![Page 26: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/26.jpg)
SimplificationSimplification
• Assume– Points from Surface– Gaussian Noise
![Page 27: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/27.jpg)
Point Sampling ModelPoint Sampling Model
High-ResolutionImage X
C
Blur
k Dk
Decimation
Low-Resolution
ImagesTransformation
Fk x
[ ELAD M., HEL-OR Y.: A fast super-resolution reconstruction algorithm for pure translational motion and common space invariant blur. IEEE Transactions on Image Processing 10,8 (2001) ]
Solution Average
YkEk
Gaussian Noise
+
![Page 28: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/28.jpg)
SimplificationSimplification
• Solution– Register scans– Averaging
• Easy
• Inexpensive
• It works!
![Page 29: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/29.jpg)
Close-up Scan of ParrotClose-up Scan of Parrot• 146 Scans• 4 times the original resolution.
![Page 30: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/30.jpg)
Super resolve far & close objects?Super resolve far & close objects?
Derived from Better Optical Triangulation through Spacetime Analysis, Curless and Levoy, 1995
SurfaceCCD sensor
![Page 31: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/31.jpg)
Super resolve small & large objects?Super resolve small & large objects?
One raw Scan Super resolution (117 scans)
![Page 32: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/32.jpg)
Is it worth taking more than one scan? Is it worth taking more than one scan?
One raw scan Super resolution PhotographSubdivion of (a)
![Page 33: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/33.jpg)
Is it worth shifting?Is it worth shifting?
With Shifts (117scans) Without Shifts (117scans)
![Page 34: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/34.jpg)
How many scans are enough?How many scans are enough?
![Page 35: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/35.jpg)
Point DistributionPoint Distribution
![Page 36: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/36.jpg)
Tiling ArtifactTiling Artifact
![Page 37: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/37.jpg)
Sampling PatternSampling Pattern
Random xy shift + Rotation
![Page 38: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/38.jpg)
Mayan Tablet (One Scan)Mayan Tablet (One Scan)
![Page 39: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/39.jpg)
39
Mayan Tablet (90 scans)Mayan Tablet (90 scans)
![Page 40: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/40.jpg)
40
Before & AfterBefore & After
![Page 41: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/41.jpg)
41
Systematic ErrorsSystematic ErrorsSuper resolved Photo
![Page 42: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/42.jpg)
42
Parrot Model (6 views * 100 scans)Parrot Model (6 views * 100 scans)
![Page 43: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/43.jpg)
Future workFuture work
• 2.5D to 3D
• Resolving Systematic Errors
• Other Devices
![Page 44: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/44.jpg)
AcknowledgementsAcknowledgements
• Kelcey Chen
• Geomagic Studios
• NSF CCF-0331736
• Brazilian National Council of Technological and Scientific Development (CNPq)
![Page 45: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/45.jpg)
45
ExtrasExtras
![Page 46: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/46.jpg)
InterpolationsInterpolations
![Page 47: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/47.jpg)
Nyquist frequencyNyquist frequency
![Page 48: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/48.jpg)
48
DataData
![Page 49: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/49.jpg)
g-1( ) =
![Page 50: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/50.jpg)
50
N
k 1kk
tk
tk FDDFR
Solving this linear system is equivalent to an average. [ ELAD M., HEL-OR Y.: A fast super-resolution reconstruction algorithm for pure translational motion and common space invariant blur. IEEE Transactions on Image Processing 10,8 (2001) ]
Solving this linear system is equivalent to an average. [ ELAD M., HEL-OR Y.: A fast super-resolution reconstruction algorithm for pure translational motion and common space invariant blur. IEEE Transactions on Image Processing 10,8 (2001) ]
2
1k ||Y||)( XFDX k
N
kk
kF
PRX
N
k 1k
tk
tk YDFP
Mimize
Diagonal MatrixDiagonal Matrix
Can be a permutation or displacement matrixCan be a permutation or displacement matrix
Equivalent to Equivalent to
![Page 51: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/51.jpg)
51
Error between low res and super res.Error between low res and super res.
![Page 52: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/52.jpg)
52
Error between low res and super res.Error between low res and super res.
![Page 53: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/53.jpg)
53
Registeration resultRegisteration result
![Page 54: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/54.jpg)
54
Before and After RegistrationBefore and After Registration
![Page 55: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/55.jpg)
55
Error between low res and super res.Error between low res and super res.
![Page 56: Combining Laser Scans Yong Joo Kil 1 , Boris Mederos 2 , and Nina Amenta 1](https://reader035.fdocuments.in/reader035/viewer/2022062500/568152ba550346895dc0dd49/html5/thumbnails/56.jpg)
56
Least Squares Least Squares
)()(2
2XXXX T HYHYHY Minimize:
Solve by:
022
XX
TT HHYH
YHHH TT X , or
Steepest Descent Iteration:
N
kjkk
Tkjj XYXX
11 ]ˆ[ˆˆ HH
kkkk FCDH ,