Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... ·...
Transcript of Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... ·...
![Page 1: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/1.jpg)
Demo
Visualizing LSA
• Create a fake TFIDF matrix with a strong concept
• Plot the matrix on a two term axis
• Perform SVD decomposition
• Plot the new axes
• Reduce the dimensionality of SVD
• Plot the new axes
![Page 2: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/2.jpg)
Demo
Visualizing LSA
• Create a fake TFIDF matrix with a strong concept
• Plot the matrix on a two term axis
![Page 3: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/3.jpg)
Demo
Visualizing LSA
• Perform SVD decomposition
![Page 4: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/4.jpg)
Demo
Visualizing LSA
• Plot the new axes
![Page 5: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/5.jpg)
Demo
Using MATLAB For LSA
• Calculate the TFIDF score
• Plot the documents on a two term axis
• Perform SVD decomposition
• Validate decomposition
• Reduce rank of system
• Show “M”
• Demonstrate what SVD is capturing
• Execute a query
![Page 6: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/6.jpg)
Demo
Using MATLAB For LSA
• Calculate the TFIDF score
wik
i: b
aseballB
at
wik
i: b
at
wik
i: c
offee
djp
3 p
aper
starbucks
wiki:starbucks
ball
baseball
bat
California
coffee
company
encyclopedia
Irvine
run
species
starbucks
stores
university
users
![Page 7: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/7.jpg)
Demo
Using MATLAB For LSA
• Calculate the TFIDF score
![Page 8: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/8.jpg)
Demo
Using MATLAB For LSA
• Calculate the TFIDF score
ball
baseball
bat
California
coffee
company
encyclopedia
Irvine
run
species
starbucks
stores
university
users
wik
i: b
aseballB
at
wik
i: b
at
wik
i: c
offee
djp
3 p
aper
starbucks
wiki:starbucks
![Page 9: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/9.jpg)
Demo
Using MATLAB For LSA
• Perform SVD decomposition
• Validate decomposition
![Page 10: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/10.jpg)
Demo
Using MATLAB For LSA
• Reduce rank of system
![Page 11: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/11.jpg)
Demo
Using MATLAB For LSA
• Show “M”
![Page 12: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/12.jpg)
Demo
Using MATLAB For LSA
• Demonstrate what SVD is capturing
• 1st concept (1st row of M)
ball
baseball
bat
California
coffee
company
encyclopedia
Irvine
run
species
starbucks
stores
university
users
First concept is selecting for wikipedia?
First concept is selecting for baseball?
![Page 13: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/13.jpg)
Demo
Using MATLAB For LSA
• Demonstrate what SVD is capturing
• 2nd concept (2nd row of M)
ball
baseball
bat
California
coffee
company
encyclopedia
Irvine
run
species
starbucks
stores
university
users
First concept is selecting for wikipedia?
Second concept is selecting for UCI?
![Page 14: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/14.jpg)
Demo
Using MATLAB For LSA
• Demonstrate what SVD is capturing
• 3rd concept (3rd row of M)
ball
baseball
bat
California
coffee
company
encyclopedia
Irvine
run
species
starbucks
stores
university
users
First concept is selecting for wikipedia?
Third concept is selecting for coffee?
![Page 15: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/15.jpg)
Demo
Using MATLAB For LSA
• Execute a query “coffee stores”ball
baseball
bat
California
coffee
company
encyclopedia
Irvine
run
species
starbucks
stores
university
users
![Page 16: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/16.jpg)
Demo
Using MATLAB For LSA
• Execute a querysim(q, di) =
!V (q) · !V (di)|!V (q)||!V (di)|
wik
i: b
aseballB
at
wik
i: b
at
wik
i: c
offee
djp
3 p
aper
starbucks
wiki:starbucks
![Page 17: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/17.jpg)
Demo
Using MATLAB For LSA
• Execute a query “coffee stores”
• Answer:
• starbucks (0.1166)
• wiki:starbucks(0.1166)
• wiki:coffee (0.1166)
• wiki:bat (0.0091)
• djp3 paper (0.0)
• wiki:baseballBat (-0.0004)
![Page 18: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/18.jpg)
Demo
Using MATLAB For LSA
• Execute a query “baseball bat”ball
baseball
bat
California
coffee
company
encyclopedia
Irvine
run
species
starbucks
stores
university
users
![Page 19: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/19.jpg)
Demo
Using MATLAB For LSA
• Execute a querysim(q, di) =
!V (q) · !V (di)|!V (q)||!V (di)|
wik
i: b
aseballB
at
wik
i: b
at
wik
i: c
offee
djp
3 p
aper
starbucks
wiki:starbucks
![Page 20: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/20.jpg)
Demo
Using MATLAB For LSA
• Execute a query “baseball bat”
• Answer:
• wiki:baseballBat (0.1414)
• wiki:bat (0.1408)
• wiki:coffee (0.0035)
• djp3 paper (0.000)
• wiki:starbucks (-0.0009)
• starbucks (-0.0012)
![Page 21: Demo - Donald Bren School of Information and Computer Sciencesdjp3/classes/2008_09_26_CS221/... · Demo Using MATLAB For LSA • Demonstrate what SVD is capturing • 1st concept](https://reader033.fdocuments.in/reader033/viewer/2022060406/5f0f56347e708231d443a854/html5/thumbnails/21.jpg)