The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

Jigsaw Image Mosaics

Junhwan Kim, Fabio Pellacini(Cornell University)

Siggraph 2002

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

What is it?

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

What does it do?

The Jigsaw Image Mosaic (JIM) algorithm takes as input an arbitrarily-shaped container image and a set of image tiles of arbitrary shape and generates a mosaic.

It then packs the container as compactly as possible with tiles of similar color to the container taken from the input set while optionally deforming them slightly to achieve a more visually-pleasing effect.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

Related work (1)

Photomosaics: square tiles on a rectangular grid.[Silvers, Hawley, 1997; Finkelstein, Range 1998]

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

Related work (2)

Simulated Decorative Mosaics: square tiles with varying orientations.[Hausner 2001]

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

Related work (3)

• Regular tiling with slight distortions of tile shapes. [Kaplan, Salesin 2000]

• Random positions of tiles, gaps filled with samples from original image. [Haeberli 1990]

• Packing problem (NP-hard) heuristics. [Downsland 1995, Milenkovic 1999]

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

The problem

Given an arbitrarily-shaped container image and a set of arbitrarily-shaped tiles {Ti}, find a set of shapes {Sj} such that:

• the union over the Sj resembles the container image as closely as possible; and

• each Sj is a translated and rotated copy of one of the Ti, possibly incorporating a small deformation.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

Energy minimization framework

DDOOGGCC EwEwEwEwE **** (Color, Gap, Overlap, Deformation)

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

The algorithm-overview

• Prepare and segment images using active contours. [Kass 1987]

• Place tiles roughly, ignoring deformation (“packing”).

• Refine placement of tiles and deform if necessary.

• Assemble the final image.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

The algorithm

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

Packing (1)

Place tiles one by one, ignoring the deformation term, such that the energy is minimized, and the tile is aligned to the container boundary.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

Packing (2)

Place tiles until the container is full. If there is no suitable tile, backtrack to the last minimal energy configuration.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

Refining

Use active contours whose vertices are subject to forces that maintain the initial shape, repulse two penetrating contours and attract two contours separated by a gap.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

Optimizations (1)

Packing: use centroidal Voronoi diagrams with sites the size of the medium tile size, and always pick a site with the minimal number of neighbors.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

Optimizations (2)

• Branch-and-bound with look-ahead: to reduce backtracking overhead, penalize tiles that would make it harder to fill the container in the next iteration – favor tiles with a small area and short circumference.

• Container cleanup: separate fragments shallower than the shallowest tile.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

Optimizations (3)

Geometric hashing: create a grid of squares representing hash table entries, then for each possible position and orientation of a tile, record the tile in the hash table.

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

Results

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL

Adrian Ilie

Summary

• General energy-based framework.

• New kind of mosaic.• Effective algorithm.