Meshless wavelets and their Meshless wavelets and their application to terrain modelingapplication to terrain modeling

A DARPA GEO* projectA DARPA GEO* project

Jack Snoeyink, Leonard McMillan, Jack Snoeyink, Leonard McMillan, Marc Pollefeys, Wei Wang (UNC-CH)Marc Pollefeys, Wei Wang (UNC-CH)

Charles Chui, Wenjie He (UMSL)Charles Chui, Wenjie He (UMSL)

OutlineOutline

• Project Team, Motivation, & Objectives• Meshless wavelets

– CK Chui: Compactly supported, refinable spline fcns– Y Liu: Order-k Voronoi diagrams & simplex splines

• Simplification/compression for applications– Mobility: elevation & slope mapping– Feature identification and matching

• Management, Risks & Rewards

• U of Missouri, St. Louis– Charles K Chui: wavelets & splines– Wenjie He: splines

• UNC Chapel Hill– Jack Snoeyink: computational geometry– Marc Pollyfeys: computer vision– Leonard McMillan: computer graphics– Wei Wang: spatial databases

– Yuanxin (Leo) Liu & Henry McEuen

Team IntroductionTeam Introduction

Self-evident truths …Self-evident truths …

• Terrain data volumes are increasing.– NIMA: “In only 9 days and 18 hours, SRTM collected elevation data for

80% of the world's landmass to enable the production of DTED Level 2.” • Old data formats were chosen for ease of computation more than

completeness of representation. – Consider USGS raster DEM’s use of integer identifiers.

• Terrain is irregular and multi-scale; its representation should be, too.– breaklines, multiple sources & sensors, viewer level of interest…

• Consistency is a virtue in multi-(use, resolution, sensor, spectral...)– Example of elevation and slope mentioned in BAA

• Image compression schemes are designed to look good.– TIFF, JPEG, JPEG2000, …

• The GIS industry cannot innovate on data reps.– Backward compatibility trumps even algorithmic improvements

• It is a good time to look at new options for terrain representation.

Key research questionKey research question

• What compact representations of terrain still support interesting queries?– Elevation + slope

for mobility + visibility– Feature identification across imaging modes

and viewing conditions for localization, change detection, and

terrain construction

Bivariate meshless waveletsBivariate meshless wavelets

We propose • a new compact representation for geospatial data that is

optimized for specific geometric and image queries. – ``meshless'' bivariate wavelets defined over scattered point sets

allow a flexible description since the point set can be specified without connectivity and each point's influence is local, while still supporting the multiscale analysis afforded by wavelets.

Objectives• complete the theory of bivariate meshless wavelets • point/knot selection algorithms optimized for specific

geometric tasks and data queries • demonstration implementation showing the advantages

of our modeling approach.

Meshless Wavelet Tight-Frames

Charles Chui

Wenjie He

University of Missouri-St. Louis

March 29, 2005 Savannah, Georgia

Stationary Wavelets

Stationary wavelet notation

2( ) 2 (2 )j jj k x x k j k Z

2( )L R

Definition of stationary wavelet tight-frames

2 2 2; , 2

1

|| || || || ( )n

i j ki j k Z

A f f B f f L R

; ,{ 1 , ,}i j k i n j k Z

2( )L R 0 A B

A family is a stationary wavelet

frame of , if there exist constants

such that

If , the frame is called a normalized tight frame.

1A B

Characterization of wavelet tight-frames

Theorem. Frazier-Garrigós-Wang-Weiss 1996, Ron-Shen 1997, Chui-Shi 1999.

Let . The family is a normalized tight frame of , if and only if

and

odd.

2

1

ˆ (2 ) 1 a en

ji

j Z i

0 1

ˆ (2 ) (2 ( 2 )) 0 a en

j jii

j i

k k

2( ),i L R 1 i n { }i j k

2( )L R

Wavelet tight-frames associated with

Multiresolution Analysis (MRA)

• Refinable function:• Frame generators:• Two-scale symbols:

• Vanishing moments of order K:

2

1

( ) (2 )N

kk N

x p x k

( )( ) (2 )k

k

x q x k

12( ) ,k

kkP z p z ( )1

2( ) ,kkk

Q z q z

iz e

( )Q z is divisible by (1 ) .Kz

Unitary matrix extension (UEP) for MRA tight frames

Let

Then is a normalized tight frame.

2 2

1

| ( ) ( ) 1,n

P z Q z

1

( ) ( ) ( ) ( ) 0,n

P z P z Q z Q z

| | 1.z

{ 1 }j k n j k Z

Equivalent matrix formulation

1 1

( ) ( )

( ) ( )The matrix has orthonormal columns

( ) ( )n n

P z P z

Q z Q z

Q z Q z

| | 1.z on

Limitations of UEP

Applicable only if

For , i.e., cardinal B-spline of order m,

at least one of the has only the factor of

but not a higher power, (i.e., only one vanishing moment

for the corresponding frame generator).

2 2( ) ( ) 1P z P z | | 1.z on

1( )

2

mz

P z

11

2 2

1 0

1 1( ) 1 ( ) 1,

2 2 2

kn m

ii k

x x z zQ z P z x z

( )iQ z (1 )z

Full characterization of MRA tight frames

2 2 2

1( ) ( ) ( ) ( )

n

iiS z P z Q z S z

2

1( ) ( ) ( ) ( ) ( ) 0 1

n

i iiS z P z P z Q z Q z z

Oblique Extension Principle (OEP)

Minimum-supported VMR functions for cardinal B-splines

For achieving vanishing moments for all tight-frame generators with symbols ( ) 1 ,iQ z i … n

11 12 20

1 110 04 1

( )

1 ( 1) 4j

m jm m x z zjj

jm m j mj

S z s x

ms s s

j

( )mS z

Orders of vanishing moments

Each has at least K vanishing moments, i.e.

has vanishing moments of order at least K, if and only if

2 2ˆ1 ( ) ( ) ( ) for 0i KS e O

Wavelet decomposition and reconstruction

Decomposition and perfect reconstruction scheme for computing DFWT

FIR schemes

New FIR filters for perfect reconstruction from DFWT with higher order of vanishing moments.

Existence of perfect reconstruction FIR filters

(Chui and He) Suppose that are Laurent polynomials, and that the matrix

has full rank for

Then there exist such that

1( ), ( ), , ( )nP z Q z Q z

1

1

( )( )( )

( )( )( )n

n

Q zQ zP z

Q zQ zP z

0.z

1( ), ( ), , ( ),nP z Q z Q z

1

1

( )( )( )

( )( )( )n

n

Q zQ zP z

Q zQ zP z

1 1

1 11 1

1 1

( ) ( )

( ) ( )

( ) ( )n n

P z P z

Q z Q z

Q z Q z

2I

Non-Stationary Wavelets

Non-stationary MRA (NMRA) wavelets

Let and be the two-scale matrices of the “refinable” functions and the wavelets

, respectively; that is,

where

1 1j j j j j jP Q

{ }jP { }jQ

{ }j

{ }j

[ ],j j k [ ].j j k

Vanishing moment condition

• is an approximate dual of order L.

• If I is a finite interval, the above condition is equivalent to

21|| || 0j LT f f f j

: the space of all polynomials of degree up to . 1L 1L

jSjS

NMRA wavelet tight-frames

VMR matrices are symmetric positive semi-definite banded matrices:

• If I is a finite interval,

• If I is an infinite interval,

jS

T

j j k j j kT f f S f

2 20 2

0

|| || ( )j

j kj k

T f f f f f L I

2 22|| || ( )

j

j kj Z k

f f f L I

NMRA tight-frame conditions

(1) For a finite interval I,

For an infinite interval I, each is bounded

on and

(2)

22lim || || ( )j

jT f f f L I

1T T

j j j j j jS P S P Q Q

2lim || || and lim 0.j jj j

T f f T f

jT

2( ),L I

Non-stationary filters

Non-stationary DFWT decomposition and perfect reconstruction

Matrix factorization for stationary tight frames

1 12 2 2 11 2 1 1

1 12 1 2 21 2 2 2

( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( ) ( )

Q z Q z Q z Q zS z S z P z S z P z P z

Q z Q z Q z Q zS z P z P z S z S z P z

Matrix factorization for non-stationary tight frames

1 1

1 1

Tee e e T ee eeeo e o T eo eoj j j j j jj j j j j j

oe o e T oo o o T oe oo oe ooj j j j j j j j j j j j

S P S P S P S P Q Q Q Q

S P S P S P S P Q Q Q Q

2 2 2 2 1[ ] [ ],ee eok kM m M m 2 1 2[ ],oe

kM m 2 1 2 1[ ].ookM m

where we use the notations

and

:ejP :o

jPthe even rows of ,jPjP the odd rows of .jP

FIR filters for non-stationary perfect reconstruction

1 1 1

1

# #e o

# #e e ne e o

No no o

# #e n o n

P P

P Q Q Q QI

P Q Q

Q Q

Two-scale matrix

• Consider two nested knot vectors

we have the refinement equation

where the matrix has non-negative entries, with each row summing to 1.

• can be derived by a sequence of knot insertions.

,t t

t m t m t t mP

t t mP

t t mP

Interior wavelets with simple knots

Boundary wavelets with simple interior knots

Interior wavelets with double knots

Boundary wavelets with double interior knots

Meshless Spline Wavelets

Simplex spline

{ }J jX j J T x

3J#X k 2

2

{ }volvol

IR( ) k

k

JM X

v v xx

T: a knot set in D

D: a bounded convex polygonal domain in 2IR

such that the projection of the set of vertices of simplex to is . 2IR JX

Neamtu’s work  on bivariate splines

• The space of bivariate polynomials of (total) degree k is locally generated by simplex splines defined on the Delaunay configuration of degree k

2k

{( )}k B IX X 1

2( ) area[ ] ( )

2B I B B I

kN X X X M X X

( )

( ) { ( ) 3}B K k K I

I B K k B I k BX X X X

B N X X I I I X X #X

A multi-level approximation by bivariate B-splines

(0) (1) ( )jT T T Let

be a nested sequence of knot sets.

Let denote the Delaunay configuration associated with the knot set .

( )jk

( )jT[ IN ]j j j

[ ]I kB I I represent bivariate B-splines

corresponding to ( )jk

Refinement matrices

can be derived by the “knot insertion" identity

1j j jP

0

( ) ( { })r

J Ji i

i

M X c M X

y

where { }J J i i JiX X \ X x x

0

r iii

c

y x0

1r

iic

and with

Tight-frame wavelets with maximum order of vanishing moments

• Wavelets• Define operators

that associate with some symmetric matrices ’s • Tight wavelet frames

1j j jQ T

j j j jT f f S f

jS

2 20 2

0

|| || ( )jj

T f f f f L D

Tight frame condition imposed on the nonstationary wavelets

22lim || || ( )j

jT f f f L D

and

1T T

j j j j j jS P S P Q Q

VMR matrices ’s construction

1[ ]j j

j

S Sj j r j jS

jS

is the row-vector of approximate duals for , j

that is,

( ) for1 and 0jSj I jp P X r j

where P is the polar form of 2kp

k-Voronoi diagrams k-Voronoi diagrams & &

simplex spline interpolationsimplex spline interpolation

k-Voronoi diagramsk-Voronoi diagrams A set of knots X in 2D

A family of (i+3) subsets of X ( features in (i+1)-Voronoi diagram )

A set degree-k of simplex spline basis

A set of terrain samples P in 2D

Simplex spline surface

k-Voronoi diagramsk-Voronoi diagrams• Definition: A k-Voronoi diagram in 2D partitions the

plane into cells such that points in each cell have the same closest k neighbors.

Order 1 Order 3

k-Voronoi diagramsk-Voronoi diagrams• Computation

- Theory: O(n log(n)) time O(n) space - Practice: O(n) time

• Engineering challenges: – speed – memory (streaming ) – robustness ( degeneracy, round-off errors )

Simplex spline interpolationSimplex spline interpolation• Problem: Given a set of terrain sample points,

reconstruct the terrain with simplex splines.

Simplex spline interpolationSimplex spline interpolation• What knot sets to use?

k-Voronoi diagramsk-Voronoi diagrams A set of knots X in 2D

A family of (i+3) subsets of X ( features in (i+1)-Voronoi diagram )

A set degree-k of simplex spline basis

A set of terrain samples P in 2D

Simplex spline surface

Simplify, preserving essentialsSimplify, preserving essentials

BAA says that GEO* emphasizes the development of math and algorithms that enable parsimonious representations coupled to end user applications: image to DEM, targeting, route planning, and motion mobility simulations.”

Key question: who defines end user application?General compression schemes are good. To be better, we need a user, even if the user is us.

Contour mapfor fishing…

(Imagine theboaters’ map)

What do you see in this map?What do you see in this map?

ManagementManagement

• POC: Jack Snoeyink

• UMSL - Mathematical development

• UNC - Algorithmic development

• Coupled by project wiki & visits

Four phasesFour phases

1. mathematics of meshless wavelets and finding key points for applications to include compression, registration, route planning, and visibility.

2. developing prototypes for these applications on top of the meshless wavelets and key points representations,

3. Option to develop one or more applications in detail,

4. Option for additional focused efforts by the PIs to transition technology to an industrial or military partner.

Perf period Primary focus Cost

Phase 1 Mathematical devel & feasibility

759,569

18 months

Phase 2 Application and prototype devel

843,787

18 mo

Phase 3 Intensive devel of key applications

389,793

12 months

Phase 4 Transition to industry

351,114

12 months

RisksRisks

• The mathematics is challenging– Goal is meshless wavelets, but

can begin with tensor-product constructions

• The implementation is complex– Order-k Voronoi + simplex splines + wavelets +

interpolation will initially be dominated by regular grids

• Need data and user contacts– Contact with Dr. Alexander Reid, terrain modeling

project leader, U.S. Army TACOM Lab (Warren, MI)

RewardsRewards

• Wavelet analysis of surfaces from irregular data samples.

• Compression that can be tuned to a particular application of the terrain

• Feature identification across imaging modalities, conditions, and scales

29 Mar 05 Snoeyink, McMillian, Polyfeys, Wang; Chui, He

Schedule•Phase I: mathematical development

•6 mo: tensor product representation order-k Voronoi for simplex splines point importance orders

•18mo: wavelet analysis for simplex splines initial feature identification

•Phase II: application development•Mobility, visibility, feature matching, localization

•Further work on applications & transition to military

UNC CH & UMSL GEO* BAA 04–12, Add 2 Meshless wavelets and their application to terrain modeling

Description / Objectives / Methods•Wavelet analysis for smooth terrain on irregularly sampled data

•Construct compactly supported, refineable spline functions•Tensor product splines & wavelets•Order-k Voronoi, simplex splines, VIP

•Compact level-of-detail representations with consistent analysis

•Feature identification in multimodal•Analysis for shortest paths, visibility

Military Impact / Sponsorship•Compact, yet accurate terrain reprsntns for mobility and multimodal feature analysis give better planning and positioning

•Seek DARPA help to obtain terrain data from Army TACOM Lab (contact: Dr. A. Reid)•Seek multimodal data – same area under various sensors & conditions