Odd Leaf Out Combining Human and Computer Vision
description
Transcript of Odd Leaf Out Combining Human and Computer Vision
![Page 1: Odd Leaf Out Combining Human and Computer Vision](https://reader036.fdocuments.in/reader036/viewer/2022062723/56813d6e550346895da74f6b/html5/thumbnails/1.jpg)
Odd Leaf OutCombining Human and Computer Vision
Arijit Biswas, Computer Science and Darcy Lewis, iSchool
Derek Hansen, Jenny Preece, Dana Rotman-University of Maryland’s iSchoolDavid Jacobs, Eric Stevens-University of Maryland Computer Science
Jen Hammock, Cynthia Parr-The Smithsonian Institution
![Page 2: Odd Leaf Out Combining Human and Computer Vision](https://reader036.fdocuments.in/reader036/viewer/2022062723/56813d6e550346895da74f6b/html5/thumbnails/2.jpg)
![Page 3: Odd Leaf Out Combining Human and Computer Vision](https://reader036.fdocuments.in/reader036/viewer/2022062723/56813d6e550346895da74f6b/html5/thumbnails/3.jpg)
Refining Metadata Associated with Images
![Page 4: Odd Leaf Out Combining Human and Computer Vision](https://reader036.fdocuments.in/reader036/viewer/2022062723/56813d6e550346895da74f6b/html5/thumbnails/4.jpg)
Existing Image Crowdsourcing Games
![Page 5: Odd Leaf Out Combining Human and Computer Vision](https://reader036.fdocuments.in/reader036/viewer/2022062723/56813d6e550346895da74f6b/html5/thumbnails/5.jpg)
How our game is different
• Anyone can play and can provide us with useful information.
• No expertise necessary
• Capitalizes on strengths of humans and algorithms– Humans are better than algorithms at identifying
similarity of images
![Page 6: Odd Leaf Out Combining Human and Computer Vision](https://reader036.fdocuments.in/reader036/viewer/2022062723/56813d6e550346895da74f6b/html5/thumbnails/6.jpg)
Game Mechanics
![Page 7: Odd Leaf Out Combining Human and Computer Vision](https://reader036.fdocuments.in/reader036/viewer/2022062723/56813d6e550346895da74f6b/html5/thumbnails/7.jpg)
Game Mechanics
![Page 8: Odd Leaf Out Combining Human and Computer Vision](https://reader036.fdocuments.in/reader036/viewer/2022062723/56813d6e550346895da74f6b/html5/thumbnails/8.jpg)
How Leaf Sets Are Constructed
• Designed to bring in useful data
• Not too easy or too hard
• Curvature based histograms used to get features from leaf shapes.– These features are used to find distance between
all possible pairs of leaves.
![Page 9: Odd Leaf Out Combining Human and Computer Vision](https://reader036.fdocuments.in/reader036/viewer/2022062723/56813d6e550346895da74f6b/html5/thumbnails/9.jpg)
What’s in it for us if people play this game?
• Identify errors in the dataset
• Discover if color helps humans identify leaves
• Feedback on how enjoyable or difficult the game is
![Page 10: Odd Leaf Out Combining Human and Computer Vision](https://reader036.fdocuments.in/reader036/viewer/2022062723/56813d6e550346895da74f6b/html5/thumbnails/10.jpg)
Game Variations
Before Leaf is Chosen
Multiple Guesses Skip
After Leaf is Chosen
Contest after Game is Finished Contest Previous Round
Feedback Mechanism
When Feedback Occurs
![Page 11: Odd Leaf Out Combining Human and Computer Vision](https://reader036.fdocuments.in/reader036/viewer/2022062723/56813d6e550346895da74f6b/html5/thumbnails/11.jpg)
Mechanical Turk Trial
1 2 3 4 50
5
10
15
20
25
30
Enjoyment
Num
ber C
orre
ct
![Page 12: Odd Leaf Out Combining Human and Computer Vision](https://reader036.fdocuments.in/reader036/viewer/2022062723/56813d6e550346895da74f6b/html5/thumbnails/12.jpg)
Mechanical Turk Trial
1 2 3 4 51
2
3
4
5
Difficulty
Enjo
ymen
t
![Page 13: Odd Leaf Out Combining Human and Computer Vision](https://reader036.fdocuments.in/reader036/viewer/2022062723/56813d6e550346895da74f6b/html5/thumbnails/13.jpg)
Summary
• Anyone can help in Computer Vision research work.
• Games can be fun for players and useful for researchers.
• Humans are better than machines in judging the similarity of two images.
![Page 14: Odd Leaf Out Combining Human and Computer Vision](https://reader036.fdocuments.in/reader036/viewer/2022062723/56813d6e550346895da74f6b/html5/thumbnails/14.jpg)
Funding
This work is made possible by National Science Foundation grant number 0968546