CSG541-SSG531-Lecture-2-2012-2013

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

1/41

Dr. Rahul Banerjee, BITS Pilani, India

BITS PilaniPilani Campus

Pervasive Computing

Rahul Banerjee, PhDDepartment of Computer Science & Information Systems

Lecture-2, 2012-2013

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

2/41


Typical Problems and Issues related to technologyintervention in the Pervasive Computing

environments HCI issues, HCI modes, User-acceptance, Usage Patterns: Needs versus

Viability Privacy and Security Issues Economic feasibility and Deployment issues Computing capabilities and scalability

Case Study of the BITS Life-guard Project Case Study of the Touch-Lives Initiative: Project

Connect and Project Communicate

Summary of Concepts learnt

Interaction Points

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

3/41


Lets have a brief look at the select examples and then identifythe problems related to the technology intervention in form of

the Pervasive Computing solutions!

Examples taken so far: InSTARTM Augmented Reality-based Car Navigation System Steering Display-based Systems Digital Graffiti Systems

Examples to be taken for brief exposure during the second and thirdlectures:

The GeorgiaTech experiment on child behaviour during game-playing inpresence of a caregiver and child-to-child interactions in absence of adults

The Project-Connect at BITS Pilani for the elderly-assistance The Project Communicate at BITS Pilani for the Children suffering from

Autism

The BITS Life-Guard Wearable Computing project for alerting drivers incase of reflex-level deterioration towards levels considered safe for driving

Typical Problems and Issues

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

4/41


The Georgia Tech Experiment

Sensor-Compute Nodes

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

5/41








Interaction Points

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

6/41


The Touch Lives Initiative is an outreach initiative aimed atassisting the underserved segments of the society with

technology-driven simple solutions.

http://www.touch-lives.org/

How does it operate? Idea > need-analysis > research > solution The Ideas-Crucible:

Research being carried out by the consenting researchers at BITSPilani works as the seed to the outreach projects

Ideas get generated through multiple routes: Classroom discussions based on real-life engineering applications Idea-fests / Project Competitions / Literature-survey Keen observations of faculty/student researchers about peoples

hardships

Project-related brainstorming sessions

Introduction to the TouchLives Initiative at BITS Pilani

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

7/41 Dr. Rahul Banerjee, BITS Pilani, India

Project Connect for the elderly

Project Communicate for the children suffering from autism

Project Smile for orphans Project Light

for the illiterate adultsA few more candidate projects are in various stages of conceptualization and initial feasibility analysis

Focus of the current presentation: Project ConnectHow did it all begin?

Observed the difficulties faced by my own parents once their fitness level gotaffected partly due to normal age-related factors and partly due to accidents and

prolonged illness

This led to realization of many difficulties that the elderly may have to face

Projects Under the TouchLives Initiative

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013


Select Problems of the Elderly that warrant non-intrusivetechnology intervention Unavailability of someone nearby to support them if they feel too weak to walk,

use bathroom or operate a gadget etc.

Unavailability of any support in the times like the dead of the night, noisyfunctions / parties / events etc. when they may need assistance but some to taketheir care may not even get to know

Social issues like moving from the place of their years of association to a newsetting with children and / or grandchildren and finding themselves almost out of

place due to:

Not knowing enough people with who they could relate, share and connectsocially and mentally

In-job children not being able to spend the expected time either due togenuine reasons or personal preferences

Not having the kind of say, they may have enjoyed when they werefinancially, physically and socially on their own

Some of these problems at times might bring the elderly to the fringes of societywhere they might begin feeling uncomfortable, unwanted and helpless

A brief look at related issues

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013




It is this set of problems and issues that the Project Connect

seeks to address so as to be able to provide a healing touch to

the lives of the elderly and bring them a better quality of life, make

them feel useful and thus feel happier.

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013




Our approach: Multi-pronged, but primarily technology-assisted Keeps people of different capabilities, backgrounds and

needs in mind

Is partly, pro-active and partly reactive, by design

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013


This project has several elements including: Context-sensitive Voice-based Search:

This project aims at providing voice-driven and voice-search-enabled multi-functionalsupport system to the elderly residents in the Old Age Homes and on different floors

or non-contiguous areas on the same floor of a building with their family members

and / or caretakers. The idea of the project came from two young undergraduate students: Arpit Agrawal

and Shrainik Jain; who were interested in voice-search algorithms.

Having the backdrop of my observation on the needs of the elderly, I thought to linktheir desire to work in this area with a project of wider (social) significance

Purpose: building a solution allowing them to seamlessly interface with the external world for

multiple relevant services

Examples may include: right from assistance for bodily support from the volunteers /attendants to their communication with their physicians and even connecting them

automatically with their children and grand children anywhere around the globe --- all

through a simple voice request and without any textual entry.

About Project Connect

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013


Other elements of this project include: Non-intrusive Sensor Network involving sensors of several types like

those for sensing

Temperature Ambient Light-level Humidity level Oxygen-level Pressure

Touch based gesture recognition units Thermal Imaging sensor units in areas like bathrooms Video-sensor units for gaze-detection, expression-recognition and gaze-

directed activation of services / gadgets

Medicine-reminder and supply-level threshold alerts Social event reminders and associated assistive messaging (Audio/Video/Textual)


7/29/2019 CSG541-SSG531-Lecture-2-2012-2013


Other involved elements include: Computer Networks and internetworks involving:

Fixed and mobile computational devices including Regular computers Smartphones Low-end Mobile phones Sensor-Compute Nodes

Wired and wireless modes of connectivity support forallowing networking of various kinds between theinvolved devices


7/29/2019 CSG541-SSG531-Lecture-2-2012-2013


Typical Problems and Issues related to technologyintervention in the environments of the elderly people HCI issues, HCI modes (eHCI and iHCI etc.), User-acceptance, Usage Patterns: Needs versus Viability Privacy and Security Issues Economic feasibility and Deployment issues

Solution approaches Pre-programmed in part, if historical data is available Sensor-driven in part, for collecting data in real-time and using it by

relating it to the context of the person-in-question

Designed to be preferably transparent to the end-user

Typical Problems and Issuesrelated to technology intervention

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013


Currently, context-sensitive voice search module is nearlyready but needs about another years work for correcting

certain shortcomings and improving results. A research communication about this part of the work has been published

in the Proceedings of the HCII international conference A related piece of work was presented in another international conference

CiC

Gesture-recognition has been well-known in literature andis being adapted for providing more precise recognition of

what the elderly might actually wish to communicatethrough a gesture that might not exactly fit in the standard

framework

Current Status of ProjectConnect

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013


Use of Body Area Networks, Wireless and Wired SensorNetworks along with Personal Area Networks like Bluetooth

and Wireless Local Area Networks like Wi-Fi has been

utilized in synchronism with 2G/2.5G/3G/4G networks for

identifying mechanisms and service elements that the finalsolution would be able to provide.

There are a few areas of concern involving seamless hand-off, battery-power depletion, safety of the wearer / user in

case of wearable computing devices and mobilecomputing/telephony devices etc. which are being

addressed now.

Expression-recognition and associated event/deviceactuation part of the work is in its very early stages.

Ongoing Work

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013








Interaction Points

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013


7/29/2019 CSG541-SSG531-Lecture-2-2012-2013


Began in 1999, this research aims to protect human livesfrom those road accidents that result from the reducedlevels of the physical fitness or mental alertness of thedriver.

Initially, it is focusing on light vehicles and their drivers /occupants. However, the concept is easily extensible tolarge vehicles and their drivers / occupants as well.

This research also draws on the works done by lifescientists on human sensory system, brain and select

externally measurable parameters (that can bemeasured, calibrated or accurately estimated withoutpiercing human body).

TheBITS-LifeguardWearableComputing Initiative

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013


The overall life-saving environment in which the BITS-Lifeguard is envisioned to work shall have two corecomponents: The wearable computing component: The BITS-Lifeguard The vehicular computing component

The scenario of action would include: Part-I: sensing of select critical parameters that help estimate the current

level of alertness and physical ability to drive safely,

comparing these with the pre-fed threshold levels and generate analert to the driver;

in case, driver fails to respond quickly enough, send and SoS signalto the vehicular computer wirelessly These responsibilities are handled by the wearable computer

The Vision behindthe BITS-Lifeguard System (1 of 2)

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013


The scenario of action would include: Part-II

Taking over control from the driver, Safely attempting to move the vehicle as per the pre-fed GIS

map and GPS data

Stopping the vehicle on a side Sending information wirelessly to the rescue / recovery

agencies providing the location details, vehicles details anddrivers details

Intimating to the pre-registered relative / friend about theevent and location

These steps are taken by the vehicles computer which formspart of the ITS and Vehicular Computing aspects of thispervasive computing initiative

The Vision behindthe BITS-Lifeguard System (2of 2)

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013


A wearable computing system of this categoryneeds at least five basic elements: Non-Intrusive Sensory elements to sense the

wearers environment,

Computing elements to take care of computationalneeds;

Communication elements to interconnect thesecomputing elements (with mobility)

Body safe Power Supply / Generation elements toprovide the necessary power to the wearablecomputing system

Fabric or placeholder elements to allowinterconnected elements in place

Elements of the BITS-Lifeguard Non-

Intrusive Wearable Computing System

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

23/41


Howdoescontextawareness/contextpredic1oncombinedwithotherelementslikeactua1onmechanismsetc.allow

SmartorIntelligentPervasiveCompu1ngEnvironments?

Intelligent/SmartSpaces/Environments

DriverWith

body mountedwearable sensors

as part of theBITS Lifeguard

VehicularComputer

Outsideworld,

preferablyITS-enabled

(or at least2.5G / 3G /

4G / Wi-Fi

supported

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

24/41


Pervasive computing systems know about the context ofthe environment of the user(s) and involve multiple

compute-sense-communicate elements embedded in

users environment.

Wearable computing systems know more about thecontext of an individual wearer and involve compute-

sense-communicate elements mounted / embedded on /

in the body of the user in various ways.

A typical wearable computing system may either workindependently or may form part of an inclusive largerpervasive computing solution.

So, how do the Pervasive and Wearable

Computing systems go together?

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

25/41








Interaction Points

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

26/41


Summary of the ConceptsLearnt

Pervasive computing systems know more:about the context of the environment of the user(s) andinvolve multiple compute-sense-communicate elementsembedded in users environment.

Wearable computing systems know more:about the context of an individual wearer andinvolve compute-sense-communicate elementsmounted / embedded on / in the body of the user invarious ways.

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

27/41


Rahul Banerjee:From Research to Classroom: A Course in Pervasive Computing, IEEE Pervasive Computing, July-

Sept. 2005, Vol. 4, No. 3, pp. 83-86.

Sailesh Conjeti, Rajiv Ranjan Singh, & Rahul Banerjee: Bio-Inspired Wearable Computing Architecture

and Physiological Signal Processing for On-Road Stress Monitoring, Proceedings of the IEEE-

EMBS International Conference on Bio-medical and Health Informatics, Shenzhen, Hong Kong, Jan

5-7, 2012, pp. 479-482. Available on IEEE Xplore: pp. 479 - 482, doi: 10.1109/BHI.2012.6211621.

Rajiv Ranjan Singh, Sailesh Conjeti & Rahul Banerjee: An Approach for Real-Time Stress-Trend

Detection in Physiological Signals in Wearable Computing Systems for Automotive Drivers,Proceedings of 14th International IEEE Conference on Intelligent Transportation Systems - ITSC

2011, October 5-7, 2011, George Washington University, Washington, DC, USA, pp. 1477-1482.

Mahesh M. Bundele & Rahul Banerjee: ROC Analysis of a Fatigue Classifier for Vehicular

Drivers, Proceedings of the 2010 IEEE International Conference on Intelligent Systems (IS 2010),

London, UK; < also available through the IEEE Explore>

.

Mahesh M. Bundele & Rahul Banerjee: Design of Early Fatigue Detection Elements of a WearableComputing System for the Prevention of Road Accidents, Proceedings of the 2nd International

Workshop on Intelligent System and Applications (ISA2010), Wuhan, China; . .

References to the Research atBITS Pilani (1 of 2)

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

28/41


Rajiv Ranjan Singh & Rahul Banerjee:Multi-parametric Analysis of Sensory Data collected from Automotive Drivers for

Building a Safety-Critical Wearable Computing System, Proceedings of the 2ndInternational Conference on Computer Engineering and Technology (ICCET

2010), April 16-18, 2010, Chengdu, China; .

Mahesh M. Bundele & Rahul Banerjee:Detection of Fatigue of Vehicular Driver using Skin Conductance and Oximetry

Pulse: A Neural Network Approach, ACM-sponsored International Conference:iiWAS2009, December 1416, 2009, Kuala Lumpur, Malaysia. Proceedings of

the iiWAS2009, pp. 725-730

Mahesh M. Bundele & Rahul Banerjee:

An SVM Classifier for Fatigue-Detection using Skin Conductance for Use in the

BITS-Lifeguard Wearable Computing System, Proceedings of the IEEE-

sponsored ICETET-2009, Nagpur, December 16-18, 2009, pp. 936-941.

References to the Research atBITS Pilani (1 of 2)

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

29/41


E-Fabric of a WearableGarment

Source: ETH, Zurich

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

30/41


Wireless Communication in e-Fabric

Source: ETH, Zurich

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

31/41


User Interface & Sensors

Elek Tex belongs to Eleksen. Peratech Ltd.

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

32/41


Several highly computationally intensive applications ofpervasive computing benefit from the Cloud-assisted

Pervasive Computing paradigm.

Cloud-assistance in such cases often comes at the back-end for handling various compute and data-intensivetasks related to a set of pervasive services.

However, there is a distinct possibility of multi-layer /multi-level cloud support at even intermediate levels of

support particularly in terms of collaboration betweenheterogeneous cloud units or between private clouds via

the public cloud.

About the Cloud-AssistedPervasive Computing

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

33/41


About the Course Pervasive Computing: What is it? Examples of Pervasive Computing Environments Situations that warrant Pervasive Computing Solutions Typical Problems and Issues related to technology intervention in

the Pervasive Computing environments HCI issues, HCI modes, User-acceptance, Usage Patterns: Needs versus Viability Privacy and Security Issues Economic feasibility and Deployment issues Computing capabilities and scalability

Case Study of the BITS Life-guard Project Case Study of the Touch-Lives Initiative: Project Connect and

Project Communicate

Summary of Concepts learnt and the Exercise for the day

Interaction Points

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

34/41


Pervasive Computing with AR

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

35/41


AScent-emi6ngDisplaySystem

Aninnova1vedisplaysystemdevelopedrecentlybytheNTT

Communica1onCorpora1onhad

beeninstalledatanunderground

MallsituatedintheTokyostrain

sta1ononanexperimentalbasisinJuly2008.

ITemitsappe1zingaromasalongwiththeadver1singvideosbeing

displayedona42-inchLCDDisplaypanel.

Thiscommercialtex1ngwasdonebyRecruitCo.Ltdforadver1sing

cafesandrestaurants.

NEC, Japan, 2008

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

36/41


DIGITAL GRAFFITI ON SEE-THROUGH

DISPLAYS IN AR SYSTEMS

Source:PERVASIVE2006-7.May2006(c)AloisFerscha

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

37/41


DISPLAYS WHICH YOU CANSTEER

Steerable Displays

Source:

PERVAS

IVE200

6-7.

May200

6

(c)AloisFersch

a

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

38/41


Pervasive Computing is all about designing and deploying solutions

that would use an inherently distributed infrastructure Such an infrastructure comprises of compute-sense-communicate elements and is

often used to support various types of customizable services via appropriate software

and services. A pervasive computing system has an ability to recognize context of the

target environment for the purpose of offering desired set of pre-defined

as well as adaptive services, preferably without the users being required

to be aware of its presence.

The design principles of Pervasive Computing are often driven by thedesirable characteristics like:

Simplicity Versatility Pleasurability

Summary of the ConceptsLearnt

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

39/41


Study Project Aura (http://www.cs.cmu.edu/~aura/) atCMU and identify the involved elements.

Consider using the basic framework of the ProjectOxygen for applications other than those conceived by

the CMU researchers. Conceptualize your own model of a simple pervasive

computing system that could allow a given room to be

converted into a simple pervasive computing unit by

using commonly available off-the-shelf devices and otherelements.

Your exercise for the day!

f

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

40/41


References

7/29/2019 CSG541-SSG531-Lecture-2-2012-2013

41/41

Thank you very much for you time and attention!

My Home page may have some additional information ofpossible interest to you. You may want to access it at the

URL: http://www.bits-pilani.ac.in/~rahul

Just in case, you may wish to contact me by Email, my E-

mail address is: [email protected]

Please take a look at the reading advisory at

Nalanda LMS Portal for additional information!

CSG541-SSG531-Lecture-2-2012-2013

Documents

Transcript of CSG541-SSG531-Lecture-2-2012-2013