MSR-Bing Image Retrieval Challenge ,written by Win
-
Upload
win-yu -
Category
Technology
-
view
230 -
download
1
description
Transcript of MSR-Bing Image Retrieval Challenge ,written by Win
![Page 1: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/1.jpg)
Text
Image Retrieval Challenge -Enhance relationships between query and image
Instructor: MeiChen Yeh ChenLin Yu, ChiungWei Hsu
VIPLAB
![Page 2: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/2.jpg)
Outline
1. Proposed method
2. Evaluation Metric
3. Experiment Result
4. Finding and Difficulty
5. Demo
6. Conclusion
7. Future work
![Page 3: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/3.jpg)
Proposed Method
![Page 4: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/4.jpg)
Query
Natural Language Processing
Tokenization
POSt
QE by WordNet QE by WikipediaWordNet Wikipedia
Click_count ranking Top candidatesUser Clicklog from
MSR dataset
![Page 5: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/5.jpg)
Apple apple apples
an apple ….
![Page 6: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/6.jpg)
Query Processing
1. Stop word and removal
2. Tokenization
3. Stemming and Lemmatization
4. Part-of-speech Tagging
5. Wiki-suggestion (Misspelled words)
6. Expansion (wordnet and wikipeia)
![Page 7: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/7.jpg)
Apple apple apples
an apple ….
Ranking Tablelog candidate count image
apple 1890 QYQtQsx9lH1KwA
apple 503 QJ4gfSPJYhbw0A
… … …
apple mac 490 PvfGna70qGiBIA
![Page 8: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/8.jpg)
Click-count Ranking
MSR dataset provide real world data for user query log.
With this, generated homemade searching table by“Click-count”.
“Max click count rule”
Log data 1,000,000 (only 1/20)
We can make sure that candidate pictures are most popular.
![Page 9: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/9.jpg)
Apple apple apples
an apple ….
Ranking Tablelog candidate count image
apple 1890 QYQtQsx9lH1KwA
apple 503 QJ4gfSPJYhbw0A
… … …
apple mac 490 PvfGna70qGiBIA
![Page 10: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/10.jpg)
Evaluation Metric
![Page 11: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/11.jpg)
MSR vs DIY Method
!
!
[rel]={Excellent=3,Good=2,Bad=0}
X
✔
![Page 12: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/12.jpg)
Experiment Result
![Page 13: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/13.jpg)
Prepare and WorkOff-line:
NLTK to process user query log
Build Ranking table (1,000,000)
Include image(base64) to Database(800,000)
On-line:
NLTK to process query input
Query expansion by word net and wikipedia
Large-scale database query processing
![Page 14: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/14.jpg)
Single unit-query'president','frank','mars','chinese','taiwan','dargon','crash','bird','France','Eiffel','president','tony','frank','mars','chinese','taiwan','London','Mexican','ydney',
'google','yahoo','jessica','microsoft','amazon','windows','apple','line','linux','android',
'world','iphone','bacteria','cat','basketball','dog','micky','tom','jerry','christmas','table',
Test : 32 queries Acc:87.5 %
![Page 15: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/15.jpg)
Compound word-querybook store, picture frame, the lost and bewildered tourist, ice cream, cell phone, apple pie, a story as old as time, a cool wet afternoon, many cases of infectious disease
swimming pool, the senlie old man,pencil box , long and winding road, tiddy bear , hot dog, jennifer love hewitt, some cookie shaped like stars
hello kitty coloring page, kelly osbourne drinking, micky mouse, a wet amd stinky dog
Test : 20 queries Acc:42.28 %
![Page 16: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/16.jpg)
Finding and Difficulty
![Page 17: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/17.jpg)
Spelling correctly can improve retrieval accuracy.
Query expansion can find more related images
!
A ambiguous query can be difficult to used.
The gap exists between users and result images, because the word is polysemic.
The user query still has a semantic problem.
Finding
![Page 18: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/18.jpg)
In a compound word query, the relationship
between previous and next word is very
important.
Query semantic is still a challenge.
Large-scale data processing is a big problem.
How to speed up search performance?
Difficulty
![Page 19: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/19.jpg)
Demo
![Page 20: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/20.jpg)
Conclusion
![Page 21: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/21.jpg)
Enhance relationships between query and image
![Page 22: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/22.jpg)
Find relationships between query and image
![Page 23: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/23.jpg)
Future Work
![Page 24: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/24.jpg)
Query
Natural Language Processing
Tokenization
POSt
QE by WordNet QE by WikipediaWordNet Wikipedia
Click_count ranking Top candidates
Named Entity Recognition
User Clicklog from MSR dataset
Enhance
![Page 25: MSR-Bing Image Retrieval Challenge ,written by Win](https://reader034.fdocuments.in/reader034/viewer/2022052601/559454741a28aba6118b4580/html5/thumbnails/25.jpg)
–ChenLin Yu, ChiungWei Hsu
“Thank you”