Interactive Discovery and Semantic Labeling of Patterns in Spatial Data Thomas Funkhouser, Adam...
-
Upload
meredith-robertson -
Category
Documents
-
view
216 -
download
0
Transcript of Interactive Discovery and Semantic Labeling of Patterns in Spatial Data Thomas Funkhouser, Adam...
Interactive Discovery and Semantic Labeling of
Patterns in Spatial Data
Thomas Funkhouser, Adam Finkelstein,David Blei, and Christiane Fellbaum
Princeton University
FODAVA Review MeetingDecember 2009
Motivation
Lots of 3D data is available with spatial patterns that reveal semantic information
Archeology
Climate Simulation
Molecular Biology
Related Work
Specific Objects• [Chen and Chen 07]
Buildings, trees, etc.
Markov Random Fields• [Anguelov et al. 05]
Point labeling
Segment, label• [Secord and Zakhor 07]• [Carlberg et al. 09]• [Golovinskiy et al. 09]
Current Approach
Supervised learning:• Training data
Locate Segment Describe Build classifier
• New data Locate Segment Describe Apply classifier Training
Area
Current Approach
Supervised learning:• Problems
Training data only usefulif matches new data
Trainer prescribessemantic classes
Trainer must labelenough training datato cover all possiblenew data
TrainingArea
Problems II
Local spatial patterns may not be descriptive enough to assign semantic labels
What is this?
Problems III
Spatial patterns/features for objects of same typemay be different in different data sets
Problems IV
Semantic objects of interest may be different for different users
• What areas of city have too few street lights?• What is spacing between fire hydrants?• Where should trees be planted? • Where could a terrorist could hide a bomb?• Where do people park?
Another Possible Approach
Active learning:• Off-line
Locate Segment Describe
• On-line System
builds classifierby requesting user labels
Another Possible Approach
Active learning:• Problems
Computer drivestraining process – tries to learn user’ssemantic model
Usually classes are pre-specified
Discrete sequenceof visual recognition tasks – if jump from example to example
Our Approach
User-driven learning:• Off-line
Locate Segment Describe
• On-line User interactively
adjusts segments and labels on data
System builds clusters, classifiers, and provides visual feedback
Our Approach
User-driven learning:• Advantages:
System can guideuser towards most usefulinput with visualization
User drives process – can focus on whathe/she cares about
User can create/removeclasses during labelingprocess
Continuous visualrecognition, easiersince camera is controlled by user
Main Challenge
User-driven learning:• Integrate off-line analysis with unsupervised learning
with interactively updated probabilistic inference modelwhile providing interactive visual feedback
User
Class labels
Outline
Introduction
User-driven learning
Specific research issues• Segmentation• Shape description• Pattern discovery• Visualization
Wrap up
Outline
Introduction
User-driven learning
Specific research issuesSegmentation• Shape description• Pattern discovery• Visualization
Wrap up
Segmentation
Current approach: • Hierarchical clustering to find candidate clusters• Min-cut separation of foreground from clutter
SegmentationProximity graphInput data
Segmentation
Current approach: • Hierarchical clustering to find candidate clusters• Min-cut separation of foreground from clutter
Segmentation
Future challenges:1) Provide interactive, adaptive segmentation tools2) Integrate segmentation with recognition3) Integrate segmentation with inference
Outline
Introduction
User-driven learning
Specific research issues• SegmentationShape description• Pattern discovery• Visualization
Wrap up
Shape Description
Problem: • Describe cluster of points by a feature vector that
discriminates its semantic class
Shape Description
Current approach: • Shape features (volume, eccentricity, …)• Shape descriptors (spin images, shape contexts, …)• Contextual cues (distances to other objects)
Spin Image
Shape Description
Proposed approach: • Data-adaptive dictionaries -- adaptable filters designed to
discriminate user selected object types
Blue = positive, Red = negative Feature vector
. . .
-0.9 0.8 1.0 0.8
Adaptable filters
QueryShape
Shape Description
Future challenges: • Computational representation for adaptable descriptors• Efficient adaptation of descriptors as user labels examples• Interactive user guidance in refinement of descriptors
Outline
Introduction
User-driven learning
Specific research issues• Segmentation• Shape descriptionPattern discovery• Visualization
Wrap up
Pattern Discovery
Goal:• Recognize spatial patterns and use them to
segment and label clusters of points
Pattern Discovery
Current approach:• Learn probabilistic representation of symmetries
and use it to predict labels
Input data Marked lampposts Symmetry transform(probabilistic model of translational symmetry)
Pattern Discovery
Current results:• Adding probabilistic symmetry as a feature
helps recognition (by a little)
Symmetry
Pattern Discovery
Future challenges:1) Better representations for symmetries
and spatial relationships2) Integrate symmetries and spatial relationships
into probabilistic inference model3) Interactive specification and visualization
of symmetries and spatial patterns
Symmetry transform(probabilistic model of translational symmetry)
Outline
Introduction
User-driven learning
Specific research issues• Segmentation• Shape description• Pattern discoveryVisualization
Wrap up
Visualization
Goals:• Provide interactive displays to help user understand …
Input data Specified segments and labels Inferred segments and labels Value of further input Computational models
Visualization
Proposed approach:• Multiple views
3D space Feature space Symmetry space
Symmetry Space3D Space
Feature Space
Visualization
Future challenges:• Provide methods to …
Integrate multiple views Represent uncertainty Guide user input Reduce clutter
Wrap Up
Goal: • Segment and label patterns in 3D data
Approach: user-driven learning• User interactively guides segmentation and labeling• System learns model and provides visual feedback
Research challenges: user-driven …• Segmentation• Shape description• Pattern discovery• Inference• Visualization