Automated Patch Generation Adapted from Tevfik Bultan’s Lecture.
Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik...
-
Upload
jack-white -
Category
Documents
-
view
220 -
download
0
Transcript of Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik...
![Page 1: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/1.jpg)
Feature Point Detection and Curve Approximation for Early
Processing of Free-Hand Sketches
Tevfik Metin Sezgin and Randall Davis
MIT AI Laboratory
![Page 2: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/2.jpg)
Metin Sezgin
Outline
• Motivation and problem statement• Feature point detection
- Curvature and speed as information sources- Average based filtering; Scale space filtering- Hybrid fit generation- Handling strokes with curves
• Evaluation• Related work• Future work
![Page 3: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/3.jpg)
Metin Sezgin
Terminology
•Sketch: Informal,messy diagramsconsisting of several strokes
•Stroke: Array of timestampedpoints
•Feature Point: Cornersof a stroke (vertices)
![Page 4: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/4.jpg)
Metin Sezgin
Task• Given a freehand stroke, generate a
description of the stroke in terms of geometric primitives such as circles, lines, and curves
![Page 5: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/5.jpg)
Metin Sezgin
Lifecycle of a stroke
Data Collection
Data Collection
Vertex DetectionVertex
Detection
PolylineFit
PolylineFit
Oval FitOval Fit
Curve Detection
Curve Detection
Line Fit
Line Fit
Complex Fit(lines and
curves)
Complex Fit(lines and
curves)Best Fit
SelectionBest Fit
Selection OutputOutput
![Page 6: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/6.jpg)
Metin Sezgin
Lifecycle of a stroke
Data Collection
Data Collection
Vertex DetectionVertex
Detection
PolylineFit
PolylineFit
Oval FitOval Fit
Curve Detection
Curve Detection
Line Fit
Line Fit
Complex Fit(lines and
curves)
Complex Fit(lines and
curves)Best Fit
SelectionBest Fit
Selection OutputOutput
![Page 7: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/7.jpg)
Metin Sezgin
Collecting Strokes
![Page 8: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/8.jpg)
Metin Sezgin
Vertex detection
![Page 9: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/9.jpg)
Metin Sezgin
Vertex detection
Curvature
![Page 10: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/10.jpg)
Metin Sezgin
Vertex detection
Curvature Speed
![Page 11: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/11.jpg)
Metin Sezgin
Problems with local extrema
• Input is noisy- Digitization noise- Hand jittering (tremor)
• Picking all local extrema introduces many false positives
• Picking the global extremum gives us a single point
• Picking the n-best extrema doesn’t help either
![Page 12: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/12.jpg)
Metin Sezgin
Solutions
• Average-based filtering- Compute threshold based on data average- Partition using this threshold- Select global extremum in each partition
![Page 13: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/13.jpg)
Metin Sezgin
Average-based filtering
Curvature Speed
![Page 14: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/14.jpg)
Metin Sezgin
Average-based filtering
• Pros- Reliable (when data isn’t too noisy)- Fast: O(n)- False positives easy to filter out
• Cons- Still false positives- Need pre-set constant for scaling
![Page 15: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/15.jpg)
Metin Sezgin
Scale-space filtering
• Real world data has features at many scales
• Represent the data multiple different scales
• Selecting scale(s) at which data is meaningful
![Page 16: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/16.jpg)
Metin Sezgin
Deriving the scale space
Convolve the original signal with Gaussian signals of increasing width
Feature points in the curvature scale space
![Page 17: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/17.jpg)
Metin Sezgin
Selecting a scale
![Page 18: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/18.jpg)
Metin Sezgin
Selecting a scale
![Page 19: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/19.jpg)
Metin Sezgin
Scale selected by the algorithm
The original strokeOutput generated at the
scale chosen by our algorithm
![Page 20: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/20.jpg)
Metin Sezgin
Handling extremely noisy strokes (cont.)
Curvature fit using SSF, 9 fps Speed fit using SSF, 7 fps
![Page 21: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/21.jpg)
Metin Sezgin
Combining information sources
• Individual detection methods may miss points
Speed fit missing a point
Curvature fit missing a point
• Need a way to combine information provided by each method
• Generate a series of hybrid fits and pick the best
![Page 22: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/22.jpg)
Metin Sezgin
Hybrid fit generation
• Given N fits generated using N different methods- Within each fit, compute vertex certainties and sort by
certainty
• Initial fit contains points common to all methods• Generate N new fits by adding best candidate
from each method. • Select the one of these with the least error• Terminate when error smaller than preset error
bound
![Page 23: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/23.jpg)
Metin Sezgin
Data Collection
Data Collection
Vertex DetectionVertex
Detection
PolylineFit
PolylineFit
Oval FitOval Fit
Curve Detection
Curve Detection
Line Fit
Line Fit
Complex Fit(lines and
curves)
Complex Fit(lines and
curves) Best FitSelectionBest Fit
Selection OutputOutput
![Page 24: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/24.jpg)
Metin Sezgin
Curve approximation
c2
c1
uv
![Page 25: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/25.jpg)
Metin Sezgin
Results
![Page 26: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/26.jpg)
Metin Sezgin
Results
![Page 27: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/27.jpg)
Metin Sezgin
User study results
• All users except one perceived the system as more natural.
• Overall it took users much less time to specify geometries.
• All subjects except one indicated they would prefer an interface that feels like our system for specifying shapes in a sketchy fashion.
• Strokes classified correctly 96% of the time
![Page 28: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/28.jpg)
Metin Sezgin
Related work
• Existing systems lack- Support for arbitrary shape creation with a
single stroke- Automatic feature point detection- Modeless drawing- Natural feel
![Page 29: Feature Point Detection and Curve Approximation for Early Processing of Free-Hand Sketches Tevfik Metin Sezgin and Randall Davis MIT AI Laboratory.](https://reader035.fdocuments.in/reader035/viewer/2022070413/5697bfe31a28abf838cb4e4a/html5/thumbnails/29.jpg)
Metin Sezgin
Future work
• Integration with the new sketch recognition architecture
• Support for over-tracing
• Further investigation of sketching behavior to enhance/aid feature point detection and stroke classification