RANSAC-Assisted Display Model Reconstruction for Projective

DisplayPatrick Quirk, Tyler Johnson, Rick Skarbez, Herman Towles, Florian Gyarfas, Henry Fuchs

Department of Computer Science

University of North Carolina at Chapel Hill

Emerging Display Technologies 2006 – March 26, 2006

2 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Multi-Projector Display

3 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Perspectively CorrectP

4 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Perspectively-correct Rendering

P matricesIntrinsics and extrinsics of each projector

Viewer’s locationDisplay Surface Model

New Method for Display Surface Estimation!

5 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Rendering Methods

Rendering Algorithm

1-pass (homography)

2-pass (projective tex.)

Display Shape

Planar surface Complex shaped surface

References

Raskar99,Yang01,Sukthankar01,Chen02, Steele02,Wallace04

Raskar98,Yang-Welch01,Brown02,Cotting04

6 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Display Surface

A Primer: 2-Pass Rendering

1st PassGeometry: application definedViewpoint: user’s positionResult: “ideal image”

2nd PassGeometry: screen surface•Use “ideal image” as projected

texture

Viewpoint: projector’s positionResult: projector’s image

Viewer

Projector

7 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Display Surface Estimation

3D Stereo

Reconstruction

Display Surface Model

Generation3D Point Cloud

Tessellated Mesh

8 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Artifacts with Tessellated Meshes

Inaccurate corner representations

Sampling issue

Texture mapping distortion on planar surfaces

Reconstruction errors

Holes in surface model

Meshing errors

9 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Visual ArtifactsP

10 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Rooms Are Piecewise Planar

New display model estimation method

Fit planes to point cloudConvert planes to polygons for rendering

AdvantageCorners can be accurately estimated (intersection of planes)Noiseless models eliminate texture distortion

11 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

RANSAC Plane Fitting

Random Sample ConsensusDesigned to work with many outliersFinds largest set of inliersCode from Peter Kovesi, Univ. Western Australia

Hypothesis plane from 3 random pointsFinds plane with most inliers

User specified fit tolerance

Fits least-squares plane to these inliers

12 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Method

FitPlane( C )• Fit one plane P to point cloud C using

RANSAC• Remove P inlier points from cloud C’ = C –

Pinliers

• Recursively loop using outliers from this pass as input for next plane

13 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

RANSAC Plane Fitting

14 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Plane To Polygons

Task: What planes intersect?

Reduce complexity to a 2D problemEstablish a “floor” plane orthogonal to the “wall” planesProject wall points onto this planeCasts problem as a line intersection problem

15 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Cast into a 2D Task

Intersecting Line (Plane)

Pairs

Red – Orange

Orange – Green

Green – Cyan

Cyan – Magenta

Magenta – Black

Black – Blue

16 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Results

Less than 2-3% dimensional errorRe-projection: Std. Dev. < 0.5 pixels

17 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Results

Improved corner matching (< 2 pixels)Lack of texture distortion in application scenery

PV

18 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

FlightGear DemoP P

19 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Installations

UNC Lab3 walls with columns in each corner

UNC lobby and I/ITSEC 2005

2-wall corner

Naval Research Lab

3 walls with a column in one corner

20 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

SUMMARY: New Surface Model Generation MethodRecursively extract planes from 3D point cloud with RANSAC-assisted algorithm

Removes outliers easilyFits noisy data well

Convert Planes to Polygons for 2-pass renderingResults in simple, accurate model

Much less distortion than with tessellated mesh

21 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Future Work

Use optimal triangulation algorithmRather than DLT; less noise in data

Generalized Planes to Polygon solutionExtract other surface functions

Applicability of cylinders, quadrics, etc.

Apply techniques to create larger virtual environmentsContinuous calibration/refinement during operation

22 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Thank You

FundingFundingOffice of Naval Research Award N00014-03-1-0589Office of Naval Research Award N00014-03-1-0589• DARWARS Training Superiority program DARWARS Training Superiority program • VIRTE – Virtual Technologies and Environments programVIRTE – Virtual Technologies and Environments program

23 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

24 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Planes to Polygons

1. Compute intersection line between two planes

Align this with Y axis

2. Project down all points to get floorplanLine segments, based on range of inliers

3. Find closest 2D line intersectionsDetermines which planes to intersect

4. Intersect the planes, make height uniform5. Truncate non-intersecting planes (edges)

25 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Room Coordinate System

Motivation Can easily estimate position of viewer

Auto-alignment performedIncorporated into Planes to Polygons algorithm

Chosen to be intersection of planesAlso have tool to change coordinate system

Extends to a tracked user

26 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

3D Stereo Reconstruction

1. Display feature set with projector(s)Checkerboard, Gray-coded stripes, Gaussian blobs, etc.

2. Image with stereo camera pair3. Extract, identify, match feature

points4. Triangulate corresponding points

Model represented by 3D point cloud!

27 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Plane 1

28 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Plane 2

29 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Plane 3

30 RANSAC-Assisted Display (Surface) Model Reconstruction for Projective Display

Plane 4