Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High...
Transcript of Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High...
![Page 1: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/1.jpg)
Integrated Movie Database
Group 9:
Muhammad Rizwan Saeed
Santhoshi Priyanka Gooty Agraharam
Ran Ao
![Page 2: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/2.jpg)
Outline
2
• Background
• Project Description
• Demo
• Conclusion
![Page 3: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/3.jpg)
Background
3
• Semantic Web
– Gives meaning to data
• Ontologies
– Concepts
– Relationships
– Attributes
• Benefits
![Page 4: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/4.jpg)
Background
4
• Semantic Web
– Gives meaning to data
• Ontologies
– Concepts
– Relationships
– Attributes
• Benefits
– Facilitates organization, integration and retrieval of data
![Page 5: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/5.jpg)
Project Description
5
• Integrated Movie Database
RDF Repository
Movies
Books
People
Querying
![Page 6: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/6.jpg)
Project Description
6
• Integrated Movie Database
– Project Phases
• Data Acquisition
• Data Modeling
• Data Linking
• Querying
![Page 7: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/7.jpg)
Data Acquisition
7
• Datasets
– IMDB.com
– BoxOfficeMojo.com
– RottenTomatoes.com
– Wikipedia.org
– GoodReads.com
• Java based crawlers using jsoup library
![Page 8: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/8.jpg)
Data Acquisition: Challenge
8
• Crawlers require a list of URLs to extract data from.
• How to generate a set of URLs for the crawler?
– User-created movie list on IMDB
• Can be exported via CSV
– DBpedia
1. Using foaf:isPrimaryTopicOf of dbo:Film or schema:Movie classes to get corresponding Wikipedia Page.
2. Crawl Wikipedia Page to get IMDB and RottenTomatoes link
![Page 9: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/9.jpg)
Data Acquisition: IMDB
9
• Crawled 4 types of pages
– Main Page• Title, Release Date, Genre, MPAA Rating, IMDB User Rating
– Casting• List of Cast Members (Actors/Actresses)
– Critics• Metacritic Score
– Awards• List of Academy Awards (won)
– Records generated for 36,549 movies
– Casting Records generated: 856,407
![Page 10: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/10.jpg)
Data Acquisition: BoxOfficeMojo
10
• BoxOfficeMojo provides an index of all the movies on their website
– Total movie links found: 16,945
• Main Movie Page
– Title, Release Date, Genre, Run time, Domestic Gross, Worldwide Gross, Budget
![Page 11: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/11.jpg)
Data Acquisition: Others
11
• RottenTomatoes
– Extracted another score based on % of positive reviews for the movie: (Scale: 0-100)
– Records generated: 10,000
• GoodReads.com
– Crawled user-generated lists of books which were adapted for movies
– Records generated: 3,000
![Page 12: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/12.jpg)
Data Modeling
12
• Integrated Movie Database Ontology
– Ontology creation using Protégé
– Automatic conversion of CSV data into RDF using Apache Jena API
![Page 13: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/13.jpg)
Data Modeling
13
![Page 14: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/14.jpg)
Data Linking
14
• Why do we need to link data?
![Page 15: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/15.jpg)
Data Linking
15
• Why do we need to link data?
– Movie: Prestige (2006)• http://www.imdb.com/title/tt0482571/
• http://www.boxofficemojo.com/movies/?id=prestige.htm
• http://www.rottentomatoes.com/m/prestige/
• http://www.goodreads.com/book/show/239239.The_Prestige
![Page 16: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/16.jpg)
Data Linking
16
• Connect independently modeled data sources
Box
Office
Mojo
$109m
2006
$40m
The Prestige
IMDB Movie
130 min
The Prestige
8.5
2006
http://www.imdb.com/title/tt0482571/ http://www.boxofficemojo.com/movies/?id=prestige.htm
![Page 17: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/17.jpg)
Data Linking
17
• Connect independently modeled data sources
IMDB Movie
130 min
The Prestige
8.5
2006
Box
Office
Mojo
$109m
2006
$40m
The Prestige
http://www.imdb.com/title/tt0482571/ http://www.boxofficemojo.com/movies/?id=prestige.htm
![Page 18: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/18.jpg)
Data Linking
18
• FRIL: Fine-grained Record Integration and Linkage Tool
– Allows record linkage based on combination of similarity metrics
– Linkage was fine-tuned based on multiple trials
![Page 19: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/19.jpg)
Data Linking
19
![Page 20: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/20.jpg)
Data Linking
20
![Page 21: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/21.jpg)
Data Linking
21
• Matching criteria:
– Movies vs Movies
• Match based on (similar) Title and same Release Year
• simScore = 50 * Edit Distance(Titles) + 50 * Equals(Release Years)
![Page 22: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/22.jpg)
Data Linking
22
• Edit Distance:
– Number of modifications (insertion, deletion, modification) that make string A equal to string B
• f(Max, May) = 1
• f(Ma, May) = 1
![Page 23: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/23.jpg)
Data Linking
23
• Matches found despite
– Punctuation Difference
– Textual Difference
– High Precision
– Recall?
Crank 2: High Voltage Crank: High Voltage
Love Wedding Marriage Love, Wedding, Marriage
The Hills have Eyes II The Hills have Eyes 2
![Page 24: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/24.jpg)
Data Linking
24
• Matching criteria:
– Movies vs Books
• Match based on (similar) Title and Book Publication Year <= Movie Release Year
Avatar Avatar
Amnesia Amnesiac
![Page 25: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/25.jpg)
Data Linking
25
• Matching criteria:
– Movies vs Books
• Match based on (similar) Title and Book Publication Year <= Movie Release Year
• Lower precision
• Required manual cleaning
Avatar (Movie) Avatar (The Last Air Bender)
Amnesia Amnesiac
![Page 26: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/26.jpg)
Querying
26
• Data hosted using Openlink Virtuoso– RDF Dataset created contains 2.3 million triples
• Demo: SPARQL Queries– Enrich user experience
– Link previously disconnected data
• e.g. Which author’s books have been most profitable for the Movie Industry?
– Path Query
• e.g. Finding collaboration
![Page 27: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/27.jpg)
Querying: Path Queries
27
• Finding Collaboration or Degree of Separation
– Show Business
• (Kevin) Bacon Number
– Research Community
• Erdős Number, Einstein Number
– Social Media
• LinkedIn Connections
Lee A. Rubel
Paper AErnst Gabor Straus
Paper B Einstein
2 1 0
![Page 28: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/28.jpg)
Demo
28
![Page 29: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/29.jpg)
Conclusion
29
• Integrated data allows to cross reference information that previously required accessing multiple web pages.
• The datasets can be augmented and used for applying ML and Social Media Analysis techniquese.g.
– Calculating Influence
– Similarity between Entities (e.g. Movies, Actors)
![Page 30: Integrated Movie Databasesaeedm/Integrated_Movie_DB_slides.pdf · –Textual Difference –High Precision –Recall? Crank 2: High Voltage Crank: High Voltage Love Wedding Marriage](https://reader036.fdocuments.in/reader036/viewer/2022071117/6000a1742999fb005c7244c2/html5/thumbnails/30.jpg)
30
Thank you!