Jigsaw Puzzles with Pieces of Unknown Orientation

Andrew C. GallagherEastman Kodak Research Laboratories

Rochester, New York

Outline

• Introduction• Solving Puzzles– Measuring Pairwise Compatibility– Tree-Based Reassembly for Types 1 and 2– An MRF for Solving Type 3 Puzzles

• Experiments• Conclusion

Introduction

• Solving puzzleassembling the pieces of a jigsaw puzzle into a complete picture

Introduction

Introduction

• Puzzle type– Type 1: known Rotation, unknown Location– Type 2: Unknown Rotation and Location– Type 3: Unknown Rotation, known Location

Solving Puzzles

• a measure of jigsaw piece compatibility• puzzle assembly

Solving Puzzles

• Measuring Pairwise Compatibility– describes the local gradients near the boundary of

a puzzle piece– Mahalanobis distance

Solving Puzzles

• Measuring Pairwise Compatibilitycompute the compatibility DLR(xi, xj) of a jigsaw piece xj on the right side of piece xi

mean distribution:

also compute the covariance matrix SiL

Solving Puzzles

• Measuring Pairwise Compatibilitycompute the gradient from the right side of piece xi to the left side of piece xj

then, Mahalanobis distance:

Solving Puzzles

• Measuring Pairwise Compatibilitymodified the above equations to compute DRL(xj, xi), then get the symmetric compatibility measure CLR(xi, xj)

store the confidence ratio in the 3D array S(xi, xj, r)

Solving Puzzles

• Evaluation in Puzzle Assembly

Similarity performance for types 1 and 2

Solving Puzzles

• Tree-Based Reassembly for Types 1 and 2– a greedy assembly algorithm inspired by Kruskal’s

algorithm for finding a minimal spanning tree– three stages:

• constrained tree stage• Trimming• Filling

Solving Puzzles

• Tree-Based Reassembly for Types 1 and 2– The constrained tree stage

• nothing prevents the MST from being a graph that results in an assembled puzzle that overlaps onto itself

• If a collision has occurred then the edge is discarded without merging the forests

Solving Puzzles

• Tree-Based Reassembly for Types 1 and 2– The constrained tree stage

Solving Puzzles

• Tree-Based Reassembly for Types 1 and 2– Trimming and Filling

Solving Puzzles

• An MRF for Solving Type 3 PuzzlesAn natural function to minimize is the total sum of the cost across the boundaries of any two pieces

Experiments

• Four measures– Direct comparison– Neighbor comparison– Largest Component– Perfect Reconstruction

Experiments

• Type 1 Puzzles

• Type 2 Puzzles

Experiments

• Type 3 Puzzles– orientation accuracy is 97.2% when considering puzzles

with 432 pieces each with 28 × 28 pixels

• Result

Experiments

• Mixed-Bag Puzzles

Conclusion

• a new class of square piece jigsaw puzzles that having pieces with unknown orientations

• a new measure (MGC) for the compatibility of a potential jigsaw piece matches

• a tree-based reassembly that greedily merges components

• a pair-wise MRF where each node represents a jigsaw piece’s orientation