ICSC2015 KeyNote: Semantic links in visual web

Seman&cally Linked Photos for building Visual Web

Ramesh Jain Dept. of Computer Science

University of California, Irvine [email protected]

Keynote at IEEE Int. Conf. on Seman0c Compu0ng 2015.

Experiences

Life =

Events

+

Literal

Visual

Oral

Progress: Knowledge Creation and Propagation

Literal

Visual

Vannevar Bush 1945

Memex:As we may think.

Scaling up Knowledge tools for humans

The human mind works by association.

The process of tying two items together is the important thing.

Berners-‐Lee: Suppose all the informa0on stored on computers everywhere were linked. Suppose I could program my computer to create a space in which anything could be linked to anything.

Web: Human experiences, knowledge, and understanding captured using

associative links.

And the World changed forever.

Volume determines organization.

Time

Num

ber

Increasing volume of documents brought us to the World Wide Web and Google

•  Top 8 Apps in 2013 were Visual.

•  Selfies are becoming a popular communication tool.

•  Significantly more time spent on visual media.

•  Photo Uploads continue exponential growth.

Source: h*p://digiday.com/pla5orms/5-‐charts-‐growth-‐visual-‐web/ ; h*ps://www.metabunk.org/threads/the-‐growth-‐of-‐photography-‐photos-‐per-‐year-‐and-‐the-‐illusion-‐of-‐frequency.2491/

21st Century Made WWW Visual.

Text based documents are not NATURAL. Language and Literacy come in the way.

Disruption: Capturing and sharing is easier for photos than text.

Imagine if every photo and video captured were connected to every

other.

Going beyond Gutenberg Legacy:

Photos and Videos are New Documents.

What is a camera?

Is a Smartphone camera s&ll a camera?

EXIF data is all metadata related to the Event. •  Exposure Time •  Aperture Diameter •  Flash •  Metering Mode •  ISO Ratings •  Focal Length •  Time •  Location •  Face

Smartphone camera captures events.

Events are ‘Connectors’ Events create ‘Context’

Causal

Experiential

Structural

People Things Places Time Experiences Events

EMPT: Extractable Mobile Photo Tags

Exif

Content Analysis

EMPT

People Place Objects Events

…

EMPT Provides Implicit Seman&c Links for a Photo.

Next Generation Photos.

Mee&ng at Dagstuhl. Gardens by the Marina Bay.

Photo is an Experiential component of a Moment.

Photos and videos are experien&al data from an Event.

People

Places

Events

Objects

Each Photos has different types of implicit links.

How much is a Picture worth?

Mee&ng at Dagstuhl.

A SEMANTICALLY LINKED PICTURE IS WORTH A THOUSAND SEMANTICALLY LINKED PICTURES.

Time

1-‐ dimensional Space Events Happen

Time

1-‐ dimensional Space Events Are Connected

Event Base

Event Presentation/ Navigation

Control

Event Grouping, Linking, Assimilation

Atomic Event Entry

EXIF

Features

Tags/ Context

Photo stream Segment. Event

Ontology

User Annot- ations

Story Telling

Search

Control

Photos and Events

Time: 9AM on Feb 6, 2015

Location: Anaheim Marriott

EXIF: Indoor Scene; Face

Keynote Talk at IEEE-ICSC 2015 by Charles Elkan on Theory versus Practice in Data Science

Annotations

Calendars

Environment Conditions

Application Knowledge

Geographic Landmarks

Camera Parameters

Event: Keynote Talk at ICSC, Speaker: Dr. Charles Elkan

Snapviz Demo: Your Personal Web

www.snapviz.com Android App

iOS Android.

Krumbs: A*aches EMPT and creates a

message.

Too Many connec&ons for each photo – how to select the ‘RIGHT’ ones.

•  Ranking Photos – Quality – Diversity – Coverage – Other Contextual Factors

•  Addressing (URI) •  Transfer Protocol (HTTP) •  Authoring and Presentation (HTML) •  Ranking •  Contextual Processing •  Content Analysis •  Privacy and Security •  Information Vs Experience

Challenges

Sky is the Limit. Applica&ons:

Lifestyle Health

Commerce Surveillance and Monitoring

Agriculture Research and development

Construc&on …

ICSC2015 KeyNote: Semantic links in visual web

Internet

Transcript of ICSC2015 KeyNote: Semantic links in visual web