06 Warping

7/30/2019 06 Warping

1/34

Image Warping

15-463: Computational PhotographyAlexei Efros, CMU, Fall 2005

Some slides from Steve Seitz

http://www.jeffrey-martin.com
http://www.jeffrey-martin.com/http://www.jeffrey-martin.com/

7/30/2019 06 Warping

2/34

Image Warping

image filtering: change range of image

g(x) = T(f(x))

f

x

T

f

x

f

x

T

f

x

image warping: change domain of image

g(x) = f(T(x))

7/30/2019 06 Warping

3/34

Image Warping

T

T

f

f g

g

image filtering: change range of image

g(x) = h(T(x))

image warping: change domain of image

g(x) = f(T(x))

7/30/2019 06 Warping

4/34

Parametric (global) warping

Examples of parametric warps:

translation rotation aspect

affine perspective cylindrical

7/30/2019 06 Warping

5/34

Parametric (global) warping

Transformation T is a coordinate-changing machine:

p =T(p)

What does it mean that T is global? Is the same for any point p

can be described by just a few numbers (parameters)

Lets represent T as a matrix:p =M*p

T

p = (x,y) p = (x,y)

=

yx

yx M

''

7/30/2019 06 Warping

6/34

Scaling

Scaling a coordinate means multiplying each of its components bya scalar

Uniform scaling means this scalar is the same for all components:

2

7/30/2019 06 Warping

7/34

Non-uniform scaling: different scalars per component:

Scaling

X 2,

Y 0.5

7/30/2019 06 Warping

8/34

Scaling

Scaling operation:

Or, in matrix form:

byyaxx

==''

=

y

x

b

a

y

x

0

0

'

'

scaling matrix S

Whats inverse of S?

7/30/2019 06 Warping

9/34

2-D Rotation

(x, y)

(x, y)

x = x cos() - y sin()y = x sin() + y cos()

7/30/2019 06 Warping

10/34

2-D Rotation

x = r cos ()y = r sin ()x = r cos ( + )y = r sin ( + )

Trig Identityx = r cos() cos() r sin() sin()y = r sin() sin() + r cos() cos()

Substitute

x = x cos() - y sin()y = x sin() + y cos()

(x, y)

(x, y)

7/30/2019 06 Warping

11/34

2-D Rotation

This is easy to capture in matrix form:

Even though sin() and cos() are nonlinear functions of, x is a linear combination of x and y

y is a linear combination of x and y

What is the inverse transformation?

Rotation by For rotation matrices, det(R) = 1 so

( ) ( )

( ) ( )

=

y

x

y

x

cossin

sincos

'

'

TRR =1

R

7/30/2019 06 Warping

12/34

2x2 Matrices

What types of transformations can berepresented with a 2x2 matrix?

2D Identity?

yy xx =='' = yxyx 10 01''

2D Scale around (0,0)?

ysy

xsx

y

x

*'

*'

=

=

=

y

x

s

s

y

x

y

x

0

0

'

'

7/30/2019 06 Warping

13/34

2x2 Matrices


2D Rotate around (0,0)?

yxy yxx *cos*sin' *sin*cos' += =

=

yx

yx

cossinsincos

''

2D Shear?

yxshy

yshxx

y

x

+=

+=

*'

*'

=

y

x

sh

sh

y

x

y

x

1

1

'

'

7/30/2019 06 Warping

14/34

2x2 Matrices


2D Mirror about Y axis?

yy xx = ='' = yxyx 10 01''

2D Mirror over (0,0)?

yyxx

==

''

=

y

x

y

x

10

01

'

'

7/30/2019 06 Warping

15/34

2x2 Matrices


2D Translation?

y

x

tyytxx

+= +=''

Only linear 2D transformationscan be represented with a 2x2 matrix

NO!

7/30/2019 06 Warping

16/34

All 2D Linear Transformations

Linear transformations are combinations of Scale,

Rotation,

Shear, and Mirror

Properties of linear transformations: Origin maps to origin

Lines map to lines

Parallel lines remain parallel Ratios are preserved

Closed under composition

=

y

x

dc

ba

y

x

'

'

=

yx

lk

ji

hg

fedcba

yx

''

7/30/2019 06 Warping

17/34

Linear Transformations as Change of Basis

Any linear transformation is a basis!!! Whats the inverse transform? How can we change from any basis to any basis?

What if the basis are orthogonal?

j =(0,1)

i =(1,0)

p

p=4i+3j = (4,3) p=4u+3vpx=4ux+3vxpy=4uy+3vy

v =(vx,vy)

u=(ux,uy)

p

pp

=

=

yy

xx

yy

xx

vu

vu

vu

vu

3

4'

7/30/2019 06 Warping

18/34

Homogeneous Coordinates

Q: How can we represent translation as a 3x3matrix?

y

x

tyy

txx

+=+=

'

'

7/30/2019 06 Warping

19/34


Homogeneous coordinates represent coordinates in 2dimensions with a 3-vector

1

coordsshomogeneou

y

x

y

x

7/30/2019 06 Warping

20/34


Q: How can we represent translation as a 3x3matrix?

A: Using the rightmost column:

= 10010

01

y

x

t

t

ranslationT

y

x

tyy

txx

+=

+=

'

'

T l i

7/30/2019 06 Warping

21/34

Translation

Example of translation

++

=

=

11100

10

01

1

'

'

y

x

y

x

ty

tx

y

x

t

t

y

x

tx = 2

ty = 1


H C di t

7/30/2019 06 Warping

22/34


Add a 3rd coordinate to every 2D point (x, y, w) represents a point at location (x/w, y/w)

(x, y, 0) represents a point at infinity

(0, 0, 0) is not allowed

Convenientcoordinate system torepresent many

usefultransformations

1 2

1

2(2,1,1) or (4,2,2) or (6,3,3)

x

y

B i 2D T f ti

7/30/2019 06 Warping

23/34

Basic 2D Transformations

Basic 2D transformations as 3x3 matrices

=

1100

0cossin

0sincos

1

'

'

y

x

y

x

=

1100

10

01

1

'

'

y

x

t

t

y

x

y

x

=

1100

01

01

1

'

'

y

x

sh

sh

y

x

y

x

Translate

Rotate Shear

=

1100

00

00

1

'

'

y

x

s

s

y

x

y

x

Scale

Affi T f ti

7/30/2019 06 Warping

24/34

Affine Transformations

Affine transformations are combinations of

Linear transformations, and

Translations

Properties of affine transformations:

Origin does not necessarily map to origin

Lines map to lines

Parallel lines remain parallel

Ratios are preserved


Models change of basis

=

w

yx

fedcba

w

yx

100

''

P j ti T f ti

7/30/2019 06 Warping

25/34

Projective Transformations

Projective transformations

Affine transformations, and

Projective warpsProperties of projective transformations:

Origin does not necessarily map to origin

Lines map to lines

Parallel lines do not necessarily remain parallel

Ratios are not preserved


Models change of basis

=

w

yx

ihg

fedcba

w

yx

'

''

Matrix Composition

7/30/2019 06 Warping

26/34

Matrix Composition

Transformations can be combined bymatrix multiplication

=

w

yx

sysx

tytx

w

yx

1000000

1000cossin0sincos

1001001

'''

p = T(tx,ty) R() S(sx,sy) p

2D image transformations

7/30/2019 06 Warping

27/34

2D image transformations

These transformations are a nested set of groupsClosed under composition and inverse is a member

Image warping

7/30/2019 06 Warping

28/34

Image warping

Given a coordinate transform (x,y) =h(x,y) and asource image f(x,y), how do we compute atransformed image g(x,y) =f(T(x,y))?

x x

T(x,y)

g(x,y)

f(x,y)

y y

Forward warping

7/30/2019 06 Warping

29/34

f(x,y) g(x,y)

Forward warping

Send each pixel f(x,y) to its corresponding location

(x,y) =T(x,y) in the second image

x x

T(x,y)

Q: what if pixel lands betweentwo pixels?

y y

Forward warping

7/30/2019 06 Warping

30/34

f(x,y) g(x,y)

Forward warping

Send each pixel f(x,y) to its corresponding location

(x,y) =T(x,y) in the second image

x x

T(x,y)

Q: what if pixel lands betweentwo pixels?

y y

A: distribute color among neighboring pixels (x,y) Known as splatting

Inverse warping

7/30/2019 06 Warping

31/34

f(x,y) g(x,y)x

y

Inverse warping

Get each pixel g(x,y) from its corresponding location

(x,y) =T-1(x,y) in the first image

x x

yT-1(x,y)

Q: what if pixel comes from betweentwo pixels?

Inverse warping

7/30/2019 06 Warping

32/34

f(x,y) g(x,y)x

y

Inverse warping

Get each pixel g(x,y) from its corresponding location

(x,y) =T-1(x,y) in the first image

x x

T-1(x,y)

Q: what if pixel comes from betweentwo pixels?

y

A: Interpolate color value from neighbors nearest neighbor, bilinear, Gaussian, bicubic

Bilinear interpolation

7/30/2019 06 Warping

33/34

Bilinear interpolation

Sampling at f(x,y):

Forward vs inverse warping

7/30/2019 06 Warping

34/34

Forward vs. inverse warping

Q: which is better?

A: usually inverseeliminates holes

however, it requires an invertible warp functionnot always possible...

06 Warping

Documents

Transcript of 06 Warping