© Ramesh Jain

Ramesh Jain

CTO, PRAJA inc.and

Professor Emeritus, UCSDrjain@praja.com

Emergent Semanticsand

Experiential Computing

© Ramesh Jain

Semantics

Meaning Negotiation process: requires agreement

among all involved participants Every User has their ‘personal ontology’ Personal ontology can not be matched

with the system ontology in one step (one query) --- requires an emergent process.

© Ramesh Jain

Powerful Data Models: Relational

Information Sources

Video Audio SensorsData andStatistics Text

Index Index Index Index Index

XMLFeature IndexedEarly Developments

© Ramesh Jain

The Semantic Gulf Data is organized based on data

characteristics. Efficiency and Scalability are primary

User intentions and context are not considered. Query environment does not maintain the state

of the user.

© Ramesh Jain

Access: Impedance Mismatch

Computers are millions of times faster than humans in arithmetic and logic.

People are millions of times better than computers in perceptual and conceptual tasks. Any one year old can recognize objects!

Current computing environments were designed for people to serve computers.

Consider computers and humans part of a symbiotic system.

© Ramesh Jain

Semantic Indexing

Video Audio Sensors Data andStatistics Text

Index Index Index Index Index

Index and Link based on Event-graph

© Ramesh Jain

Information Assimilation

UserEventBase

Model-Based

UpdatingAnd

Linking

Navigationand

VisualizationEnvironment

.

Semantic procedural updating

Semantic links used to display unstructured data

© Ramesh Jain

Features of Experiential Environment

Natural Action Responses – No unusual metaphors

Query and Presentation spaces must be the same (What-You-See-Is-What-You-Get)

Continuity of User State and Context – Minimize Latency; Feedback

Multimedia Immersion and Exploration

Video games should be the model.

© Ramesh Jain

Example Applications

© Ramesh Jain

More than Five Century oldLegacy of GutenbergContinues…Despite all advances in technology…

DocumentWeb: Information Age

© Ramesh Jain

Strategic Inflection Points

Documents on Web(Information)

Events on Web(Experience)

1995 2000 2005 2010

Keyword Search

SemanticSearch

ContextualSearch

ImmersiveExperience

Updates and alerts

Ubiquitous Devices

© Ramesh Jain

EventWeb: Experience AgeFamily

Sports

Fun

Knowledge

Personal

Finance

Office

© Ramesh Jain

Top 5 Misconceptions

All users should have Ph.D. in Ontology. User queries are Context-Free. Users must be allowed only one query. Image and Video semantics is in features of

FULL images or video. All information is alpha-numeric.

© Ramesh Jain

Thanks.

© Ramesh Jain

Data ExperienceInformation

© Ramesh Jain

Information Integration: 1

QueryParsing

AndReport

Generation

User

© Ramesh Jain

© Ramesh Jain

Time Machine - Replay

Example Apps

© Ramesh Jain

Information System Evolution: Databases

QueryTranslation

And Response

Users

Queries

Declarative and Stateless

© Ramesh Jain

Information System Evolution : Personalization

QueryTranslation

And Response

Users

P- Queries

PersonalizationFilters

Queries

Declarative and Stateless

© Ramesh Jain

Information System Evolution : Contextual Systems

QueryTranslation

And Response

Users

P- Queries

PersonalizationFilters

QueriesContext

UserStates

Declarative and Stateless

© Ramesh Jain

Contextual Navigation

Context

Data PresentationSystem

Query

Query Transformation

What-You-See-Is-What-You-Get (WYSIWYG) Search.

© Ramesh Jain

Kalman Filtering

Processing

Model

Data

© Ramesh Jain

Basic Concept

Model: Mathematical State and other relations

Current State = F (Previous State, New Observations)

Models states and transitions based on each source of data

© Ramesh Jain

Symbolic Kalman Filtering

The Model is Hybrid – combination of symbolic and mathematical.

Each Data Source is an independent observation source.

Model is updated based on its current state and new observations.

Essential for live and sensory data assimilation.

© Ramesh Jain

Entities and Events

Events are dynamic.

Event Name

DurationLocation

AttributesData-streams

ProcessesAdjacent StatesRelated Links

Entity

Name

Attributes

Processes(Services)

Objects and Entitiesare static.

© Ramesh Jain

Modeling a Domain

Event Graphs: Capture relationships among events and entities.

Event Transitions: Conditions and probabilities of transitions.

© Ramesh Jain

Experience:Direct Observation of

or Participation in Events

as a basis of knowledge

© Ramesh Jain

Objectives

Experiential environments are the next major technology inflexion point.

Unified indexing of assimilated data is essential to implement experience-centric, rather than current information-centric, systems.

Semantics is ubiquitous and emerges with symbiotic interactions among a user and the system.

© Ramesh Jain

Information Integration: 2

UserIntegratedDatabase