Post on 19-Dec-2015
Spectral bases for 3D shapes representation
Spectral bases – reminder
Last week we spoke of Fourier analysis 1D sine/cosine bases for signals:
0 πcos (2 1)
16x
πcos (2 1)
16x
2πcos (2 1)
16x
3πcos (2 1)
16x
4πcos (2 1)
16x
5πcos (2 1)
16x
6πcos (2 1)
16x
7πcos (2 1)
16x
Spectral bases – reminder
Last week we spoke of Fourier analysis 2D bases for 2D signals (images)
cos (2 1) cos (2 1)16 16
u vx y
How about 3D shapes?
Problem: general 3D shapes cannot be described as height functions
Height function, regularly sampled above a 2D domain
General 3D shapes
Irregular meshes
In graphics, shapes are mostly represented by triangle meshes
Irregular meshes
In graphics, shapes are mostly represented by triangle meshes
Geometry: Vertex coordinates
(x1, y1, z1)
(x2, y2, z2) . . .
(xn, yn, zn)
Connectivity: List of triangles
(i1, j1, k1)
(i2, j2, k2) . . .
(im, jm, km)
Irregular meshes
In graphics, shapes are mostly represented by triangle meshes The connectivity (the graph) can be very
efficiently encoded About 2 bits per vertex only
The geometry (x,y,z) is heavy! When stored naively, at least 12 bits per
coordinate are needed, i.e. 36 bits per vertex
How to define efficient bases?
Extension of the 2D DCT basis to a general (irregular) mesh
DCT ???
Basis shapes
We need a collection of basis functions First basis functions will be very smooth, slowly-varying Last basis functions will be high-frequency, oscillating
We will represent our shape (mesh geometry) as a linear combination of the basis functions
We will induce the basis functions from the mesh connectivity! We assume that the connectivity is known to the decoder
side (it is transferred first)
The Mesh Laplacian operator
( ) ( )
1( )i i i j i i j
j N i j N ii
L d dd
v v v v v
Measures the local smoothness at each mesh vertex
Laplacian operator in matrix form
1 1 1
2 2 2
3
1 1 1
1 0 1 0
0 1 1
0 1 1 1
1 1 1n n n
n n n
d
d
d
d
d
v
v
v
v
L matrix
Laplacian matrix
L is constructed based on the connectivity alone
So, if the decoder got the connectivity first, it can construct L
Spectral bases
L is a symmetric nn matrix Perform spectral analysis of L!
L =
T1
2
n
1 2
| | |
| | |nb b b 1 2
| | |
| | |nb b b
Basis vectors Frequencies,sorted in ascending order
How to represent our mesh geometry in the spectral basis?
X, Y, Z Rn. Decompose in the spectral basis: 1
21 1 2 2
1
21 1 2 2
1
21 1 2 2
n n
n
n n
n
n n
n
x
x
x
y
y
y
z
z
z
X b b b
Y b b b
Z b b b
How to represent our mesh geometry in the spectral basis?
Decompose the mesh geometry in the spectral basis:
T T T11 2
21 1 2 2
1 2
n
n n
nn
v
vb b b
v
The first components are low-frequency
The last components are high-frequency
The spectral basis First functions are smooth and slow, last oscillate a lot
horse connectivity
spectral basis of L =the DCT basis
chain connectivity
2nd basis function
10th basis function
100th basis function
Why this works?
Low-frequency basis vectors are smooth and slowly-varying: The first basis vector:
It is the smoothest possible function (Laplacian is constant zero on it)
1
T1
0
(1,1, 1)
L
b
b
Why this works?
Low-frequency basis vectors are smooth and slowly-varying: The first basis vector:
Lb1 = 0 because all the rows of L sum to zero
1
T1
0
(1,1, 1)
L
b
b
Why this works?
Low-frequency basis vectors are smooth and slowly-varying: The following basis vectors:
i i iL b b
The frequency is small the Laplacian operator gives small values the shape of the function is smooth
Why this works?
High-frequency basis vectors are non-smooth, oscillating:
i i iL b b
The frequency is large the Laplacian operator gives large values the shape of the function is less smooth!
Geometry compressionT T T1
1 22
1 1 2 2
1 2
n
n n
nn
v
vb b b
v
As in JPEG compression,drop the high-frequency components!
Geometry compression
Increasing number of participatingbasis vectors.The less coefficients are dropped, the less shape information is lost
Technical algorithm
The encoding side: Compute the spectral bases from mesh connectivity Represent the shape geometry in the spectral basis and
decide how many coeffs. to leave (K) Store the connectivity and the K non-zero coefficients
The decoding side: Compute the first K spectral bases from the connectivity Combine them using the K received coefficients and get
the shape
Challenges and trade-offs
Performing spectral decomposition of a large matrix (n>1000) is prohibitively expensive (O(n3)) Today’s meshes come with 50,000 and more vertices We don’t want the decompressor to work forever!
Solution: like in JPEG, work on small blocks:
Challenges and trade-offs
As in JPEG, blocking artifacts appear for low bit rates, because the compression is lossy (too much information is lost)
Further info
“Spectral Mesh Compression”, Zachi Karni and Craig Gotsman, ACM SIGGRAPH 2000http://www.mpi-inf.mpg.de/~karni/publications/spectral.pdf
“3D Mesh compression using fixed spectral bases”, Z. Karni and C. Gotsman, Graphics Interface, 2001http://www.mpi-inf.mpg.de/~karni/publications/fixed.pdf
See you next time