Post on 25-Jul-2015
Seman&cally Linked Photos for building Visual Web
Ramesh Jain Dept. of Computer Science
University of California, Irvine jain@ics.uci.edu
Keynote at IEEE Int. Conf. on Seman0c Compu0ng 2015.
Experiences
Life =
Events
+
Literal
Visual
Oral
Progress: Knowledge Creation and Propagation
Oral
Literal
Visual
Vannevar Bush 1945
Memex:As we may think.
Scaling up Knowledge tools for humans
The human mind works by association.
The process of tying two items together is the important thing.
Berners-‐Lee: Suppose all the informa0on stored on computers everywhere were linked. Suppose I could program my computer to create a space in which anything could be linked to anything.
Web: Human experiences, knowledge, and understanding captured using
associative links.
And the World changed forever.
Volume determines organization.
Time
Num
ber
Increasing volume of documents brought us to the World Wide Web and Google
• Top 8 Apps in 2013 were Visual.
• Selfies are becoming a popular communication tool.
• Significantly more time spent on visual media.
• Photo Uploads continue exponential growth.
Source: h*p://digiday.com/pla5orms/5-‐charts-‐growth-‐visual-‐web/ ; h*ps://www.metabunk.org/threads/the-‐growth-‐of-‐photography-‐photos-‐per-‐year-‐and-‐the-‐illusion-‐of-‐frequency.2491/
21st Century Made WWW Visual.
Text based documents are not NATURAL. Language and Literacy come in the way.
Disruption: Capturing and sharing is easier for photos than text.
Imagine if every photo and video captured were connected to every
other.
Going beyond Gutenberg Legacy:
Photos and Videos are New Documents.
What is a camera?
Is a Smartphone camera s&ll a camera?
EXIF data is all metadata related to the Event. • Exposure Time • Aperture Diameter • Flash • Metering Mode • ISO Ratings • Focal Length • Time • Location • Face
Smartphone camera captures events.
Events are ‘Connectors’ Events create ‘Context’
Causal
Experiential
Structural
People Things Places Time Experiences Events
EMPT: Extractable Mobile Photo Tags
Exif
Content Analysis
EMPT
People Place Objects Events
…
EMPT Provides Implicit Seman&c Links for a Photo.
Next Generation Photos.
Mee&ng at Dagstuhl. Gardens by the Marina Bay.
Photo is an Experiential component of a Moment.
Photos and videos are experien&al data from an Event.
People
Places
Events
Objects
Each Photos has different types of implicit links.
How much is a Picture worth?
Mee&ng at Dagstuhl.
A SEMANTICALLY LINKED PICTURE IS WORTH A THOUSAND SEMANTICALLY LINKED PICTURES.
Time
1-‐ dimensional Space Events Happen
Time
1-‐ dimensional Space Events Are Connected
Event Base
Event Presentation/ Navigation
Control
Event Grouping, Linking, Assimilation
Atomic Event Entry
EXIF
Features
Tags/ Context
Photo stream Segment. Event
Ontology
User Annot- ations
Story Telling
Search
Control
Photos and Events
Time: 9AM on Feb 6, 2015
Location: Anaheim Marriott
EXIF: Indoor Scene; Face
Keynote Talk at IEEE-ICSC 2015 by Charles Elkan on Theory versus Practice in Data Science
Annotations
Calendars
Environment Conditions
Application Knowledge
Geographic Landmarks
Camera Parameters
Event: Keynote Talk at ICSC, Speaker: Dr. Charles Elkan
Snapviz Demo: Your Personal Web
www.snapviz.com Android App
iOS Android.
Krumbs: A*aches EMPT and creates a
message.
Too Many connec&ons for each photo – how to select the ‘RIGHT’ ones.
• Ranking Photos – Quality – Diversity – Coverage – Other Contextual Factors
• Addressing (URI) • Transfer Protocol (HTTP) • Authoring and Presentation (HTML) • Ranking • Contextual Processing • Content Analysis • Privacy and Security • Information Vs Experience
Challenges
Sky is the Limit. Applica&ons:
Lifestyle Health
Commerce Surveillance and Monitoring
Agriculture Research and development
Construc&on …