THE SECOND LIFE OF A THE SECOND LIFE OF A SENSOR: INTEGRATING SENSOR: INTEGRATING

REAL-WORLD REAL-WORLD EXPERIENCE IN EXPERIENCE IN

VIRTUAL WORLDS VIRTUAL WORLDS USING MOBILE PHONESUSING MOBILE PHONES

Sherrin George Sherrin George &&

Reena RajanReena Rajan

TOPICSTOPICS

IntroductionIntroduction System Design ChallengesSystem Design Challenges System Prototype ImplementationSystem Prototype Implementation

INTRODUCTIONINTRODUCTION

Second Life represents the latest and most successful Second Life represents the latest and most successful online services for entertainment and business.online services for entertainment and business.

This paper focuses on providing a virtual This paper focuses on providing a virtual representation of humans, their surroundings as representation of humans, their surroundings as sensed by the humans themselves, and interactions sensed by the humans themselves, and interactions with the members of their social networks.with the members of their social networks.

Sensors embedded in mobile phones can be used to Sensors embedded in mobile phones can be used to infer real-world activities, that in turn can be infer real-world activities, that in turn can be reproduced in virtual settings. reproduced in virtual settings.

SYSTEM DESIGN CHALLENGESSYSTEM DESIGN CHALLENGES

Activity Recognition and VisualizationActivity Recognition and Visualization Intermittent ConnectivityIntermittent Connectivity ScalabilityScalability Data Sharing and Group ActivitiesData Sharing and Group Activities External sensing DevicesExternal sensing Devices Context Reflection Context Reflection Privacy and Social ImplicationsPrivacy and Social Implications

Activity Recognition and Activity Recognition and VisualizationVisualization

Activity Inference:- consists of the extraction Activity Inference:- consists of the extraction of high-level activities from time series of raw of high-level activities from time series of raw data sampled from sensors. data sampled from sensors.

Mapping:- mapping physical activities in the Mapping:- mapping physical activities in the real world into different/same virtual ones.real world into different/same virtual ones.

Visualization & Information Accuracy:- some Visualization & Information Accuracy:- some users may want to discard the information if users may want to discard the information if the inference is not sufficiently accurate. the inference is not sufficiently accurate.

Intermittent ConnectivityIntermittent Connectivity

Cell phones may experience intermittent Cell phones may experience intermittent connectivity due to limits in network coverage.connectivity due to limits in network coverage.

Avatars should be set to a meaningful state.Avatars should be set to a meaningful state. Delay tolerant mechanisms can be adopted to Delay tolerant mechanisms can be adopted to

temporarily store user status locally on the temporarily store user status locally on the phone in case of disconnection.phone in case of disconnection.

ScalabilityScalability

To reduce the number of status updates To reduce the number of status updates flowing into the Second Life Visualization, the flowing into the Second Life Visualization, the status/activity of the virtual avatar is updated status/activity of the virtual avatar is updated only when the inferred status/activity of the only when the inferred status/activity of the human user changes.human user changes.

Running activity recognition algorithms on the Running activity recognition algorithms on the mobile devices reduces communication costs mobile devices reduces communication costs and computational burden on the server.and computational burden on the server.

Data Sharing and Group ActivitiesData Sharing and Group Activities

The system should leverage the sharing of raw The system should leverage the sharing of raw or processed data among different people in or processed data among different people in physical proximity by means of short range physical proximity by means of short range radio communication.radio communication.

Individuals will be able to update not only Individuals will be able to update not only their status, but also those of their neighbors.their status, but also those of their neighbors.

External sensing DevicesExternal sensing Devices

A small size sensing module external to the A small size sensing module external to the phone can be used to allow for more accurate phone can be used to allow for more accurate activity inference compared with using activity inference compared with using embedded phone sensors which may be ill embedded phone sensors which may be ill positioned.positioned.

Privacy and Social ImplicationsPrivacy and Social Implications

The users should be able to disconnect from The users should be able to disconnect from the virtual worlds at any time or be able to the virtual worlds at any time or be able to interrupt the sampling of their activities.interrupt the sampling of their activities.

SYSTEM PROTOTYPE SYSTEM PROTOTYPE IMPLEMENTATIONIMPLEMENTATION

CenceMeCenceMe Bridging CenceMe and Virtual WorldsBridging CenceMe and Virtual Worlds Design and Preliminary Evaluation of the Design and Preliminary Evaluation of the

PrototypePrototype

CenceMeCenceMe

CenceMe is a general personal sensing system CenceMe is a general personal sensing system that enables members of social networks to that enables members of social networks to share their sensing presence extracted by share their sensing presence extracted by means of mobile phones with their buddies. means of mobile phones with their buddies.

CenceMe can be used to interconnect virtual CenceMe can be used to interconnect virtual worlds and social networking systems such as worlds and social networking systems such as Facebook.Facebook.

Bridging CenceMe and Virtual Bridging CenceMe and Virtual WorldsWorlds

Design and Preliminary Evaluation Design and Preliminary Evaluation of the Prototypeof the Prototype

The integration is based on the exploitation of virtual The integration is based on the exploitation of virtual CenceMe objects that the Second Life avatar can wear or CenceMe objects that the Second Life avatar can wear or carry. This object alters the avatar animation to reflect the carry. This object alters the avatar animation to reflect the actual activity state of a human user carrying a cell phone that actual activity state of a human user carrying a cell phone that is running the CenceMe sensing application.is running the CenceMe sensing application.

The CenceMe object is implemented as a standard second Life The CenceMe object is implemented as a standard second Life object that we program to communicate with the CenceMe object that we program to communicate with the CenceMe infrastructure via XML encoded requests and responses infrastructure via XML encoded requests and responses exchanged via HTTP.exchanged via HTTP.

Second Life CenceMe object behavior is written in LSL.Second Life CenceMe object behavior is written in LSL. The avatar can disconnect from CenceMe system by taking off The avatar can disconnect from CenceMe system by taking off

the object.the object.

Design and Preliminary Evaluation Design and Preliminary Evaluation of the Prototype contd..of the Prototype contd..

The communication between Second Life and The communication between Second Life and CenceMe platforms is pull based.CenceMe platforms is pull based.

A library of potential Second Life activities is A library of potential Second Life activities is maintained from which the user chooses to maintained from which the user chooses to map specific real-world activities. map specific real-world activities.

Each virtual object within Second Life Each virtual object within Second Life retrieves data independently from the retrieves data independently from the CenceMe server. CenceMe server.

Delay MeasurementsDelay Measurements

An average delay of 30 sec An average delay of 30 sec is observed in reporting the is observed in reporting the activity changes. activity changes.

This delay is due to the This delay is due to the activity recognition process activity recognition process on the phones, the on the phones, the transmission of the inferred transmission of the inferred activity to the back-end, its activity to the back-end, its retrieval from CenceMe retrieval from CenceMe database, and its display in database, and its display in Second Life.Second Life.

Thank YouThank You