Post on 03-Apr-2018
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
1/41
Dr. Rahul Banerjee, BITS Pilani, India
BITS PilaniPilani Campus
Pervasive Computing
Rahul Banerjee, PhDDepartment of Computer Science & Information Systems
Lecture-2, 2012-2013
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
2/41
Dr. Rahul Banerjee, BITS Pilani, India
Typical Problems and Issues related to technologyintervention in the Pervasive Computing
environments HCI issues, HCI modes, User-acceptance, Usage Patterns: Needs versus
Viability Privacy and Security Issues Economic feasibility and Deployment issues Computing capabilities and scalability
Case Study of the BITS Life-guard Project Case Study of the Touch-Lives Initiative: Project
Connect and Project Communicate
Summary of Concepts learnt
Interaction Points
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
3/41
Dr. Rahul Banerjee, BITS Pilani, India
Lets have a brief look at the select examples and then identifythe problems related to the technology intervention in form of
the Pervasive Computing solutions!
Examples taken so far: InSTARTM Augmented Reality-based Car Navigation System Steering Display-based Systems Digital Graffiti Systems
Examples to be taken for brief exposure during the second and thirdlectures:
The GeorgiaTech experiment on child behaviour during game-playing inpresence of a caregiver and child-to-child interactions in absence of adults
The Project-Connect at BITS Pilani for the elderly-assistance The Project Communicate at BITS Pilani for the Children suffering from
Autism
The BITS Life-Guard Wearable Computing project for alerting drivers incase of reflex-level deterioration towards levels considered safe for driving
Typical Problems and Issues
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
4/41
Dr. Rahul Banerjee, BITS Pilani, India
The Georgia Tech Experiment
Sensor-Compute Nodes
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
5/41
Dr. Rahul Banerjee, BITS Pilani, India
Typical Problems and Issues related to technologyintervention in the Pervasive Computing
environments HCI issues, HCI modes, User-acceptance, Usage Patterns: Needs versus
Viability Privacy and Security Issues Economic feasibility and Deployment issues Computing capabilities and scalability
Case Study of the BITS Life-guard Project Case Study of the Touch-Lives Initiative: Project
Connect and Project Communicate
Summary of Concepts learnt
Interaction Points
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
6/41
Dr. Rahul Banerjee, BITS Pilani, India
The Touch Lives Initiative is an outreach initiative aimed atassisting the underserved segments of the society with
technology-driven simple solutions.
http://www.touch-lives.org/
How does it operate? Idea > need-analysis > research > solution The Ideas-Crucible:
Research being carried out by the consenting researchers at BITSPilani works as the seed to the outreach projects
Ideas get generated through multiple routes: Classroom discussions based on real-life engineering applications Idea-fests / Project Competitions / Literature-survey Keen observations of faculty/student researchers about peoples
hardships
Project-related brainstorming sessions
Introduction to the TouchLives Initiative at BITS Pilani
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
7/41 Dr. Rahul Banerjee, BITS Pilani, India
Project Connect for the elderly
Project Communicate for the children suffering from autism
Project Smile for orphans Project Light
for the illiterate adultsA few more candidate projects are in various stages of conceptualization and initial feasibility analysis
Focus of the current presentation: Project ConnectHow did it all begin?
Observed the difficulties faced by my own parents once their fitness level gotaffected partly due to normal age-related factors and partly due to accidents and
prolonged illness
This led to realization of many difficulties that the elderly may have to face
Projects Under the TouchLives Initiative
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
8/41 Dr. Rahul Banerjee, BITS Pilani, India
Select Problems of the Elderly that warrant non-intrusivetechnology intervention Unavailability of someone nearby to support them if they feel too weak to walk,
use bathroom or operate a gadget etc.
Unavailability of any support in the times like the dead of the night, noisyfunctions / parties / events etc. when they may need assistance but some to taketheir care may not even get to know
Social issues like moving from the place of their years of association to a newsetting with children and / or grandchildren and finding themselves almost out of
place due to:
Not knowing enough people with who they could relate, share and connectsocially and mentally
In-job children not being able to spend the expected time either due togenuine reasons or personal preferences
Not having the kind of say, they may have enjoyed when they werefinancially, physically and socially on their own
Some of these problems at times might bring the elderly to the fringes of societywhere they might begin feeling uncomfortable, unwanted and helpless
A brief look at related issues
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
9/41 Dr. Rahul Banerjee, BITS Pilani, India
BITS PilaniPilani Campus
Dr. Rahul Banerjee, BITS Pilani, India
It is this set of problems and issues that the Project Connect
seeks to address so as to be able to provide a healing touch to
the lives of the elderly and bring them a better quality of life, make
them feel useful and thus feel happier.
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
10/41 Dr. Rahul Banerjee, BITS Pilani, India
BITS PilaniPilani Campus
Dr. Rahul Banerjee, BITS Pilani, India
Our approach: Multi-pronged, but primarily technology-assisted Keeps people of different capabilities, backgrounds and
needs in mind
Is partly, pro-active and partly reactive, by design
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
11/41 Dr. Rahul Banerjee, BITS Pilani, India
This project has several elements including: Context-sensitive Voice-based Search:
This project aims at providing voice-driven and voice-search-enabled multi-functionalsupport system to the elderly residents in the Old Age Homes and on different floors
or non-contiguous areas on the same floor of a building with their family members
and / or caretakers. The idea of the project came from two young undergraduate students: Arpit Agrawal
and Shrainik Jain; who were interested in voice-search algorithms.
Having the backdrop of my observation on the needs of the elderly, I thought to linktheir desire to work in this area with a project of wider (social) significance
Purpose: building a solution allowing them to seamlessly interface with the external world for
multiple relevant services
Examples may include: right from assistance for bodily support from the volunteers /attendants to their communication with their physicians and even connecting them
automatically with their children and grand children anywhere around the globe --- all
through a simple voice request and without any textual entry.
About Project Connect
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
12/41 Dr. Rahul Banerjee, BITS Pilani, India
Other elements of this project include: Non-intrusive Sensor Network involving sensors of several types like
those for sensing
Temperature Ambient Light-level Humidity level Oxygen-level Pressure
Touch based gesture recognition units Thermal Imaging sensor units in areas like bathrooms Video-sensor units for gaze-detection, expression-recognition and gaze-
directed activation of services / gadgets
Medicine-reminder and supply-level threshold alerts Social event reminders and associated assistive messaging (Audio/Video/Textual)
About Project Connect
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
13/41 Dr. Rahul Banerjee, BITS Pilani, India
Other involved elements include: Computer Networks and internetworks involving:
Fixed and mobile computational devices including Regular computers Smartphones Low-end Mobile phones Sensor-Compute Nodes
Wired and wireless modes of connectivity support forallowing networking of various kinds between theinvolved devices
About Project Connect
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
14/41 Dr. Rahul Banerjee, BITS Pilani, India
Typical Problems and Issues related to technologyintervention in the environments of the elderly people HCI issues, HCI modes (eHCI and iHCI etc.), User-acceptance, Usage Patterns: Needs versus Viability Privacy and Security Issues Economic feasibility and Deployment issues
Solution approaches Pre-programmed in part, if historical data is available Sensor-driven in part, for collecting data in real-time and using it by
relating it to the context of the person-in-question
Designed to be preferably transparent to the end-user
Typical Problems and Issuesrelated to technology intervention
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
15/41 Dr. Rahul Banerjee, BITS Pilani, India
Currently, context-sensitive voice search module is nearlyready but needs about another years work for correcting
certain shortcomings and improving results. A research communication about this part of the work has been published
in the Proceedings of the HCII international conference A related piece of work was presented in another international conference
CiC
Gesture-recognition has been well-known in literature andis being adapted for providing more precise recognition of
what the elderly might actually wish to communicatethrough a gesture that might not exactly fit in the standard
framework
Current Status of ProjectConnect
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
16/41 Dr. Rahul Banerjee, BITS Pilani, India
Use of Body Area Networks, Wireless and Wired SensorNetworks along with Personal Area Networks like Bluetooth
and Wireless Local Area Networks like Wi-Fi has been
utilized in synchronism with 2G/2.5G/3G/4G networks for
identifying mechanisms and service elements that the finalsolution would be able to provide.
There are a few areas of concern involving seamless hand-off, battery-power depletion, safety of the wearer / user in
case of wearable computing devices and mobilecomputing/telephony devices etc. which are being
addressed now.
Expression-recognition and associated event/deviceactuation part of the work is in its very early stages.
Ongoing Work
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
17/41 Dr. Rahul Banerjee, BITS Pilani, India
Typical Problems and Issues related to technologyintervention in the Pervasive Computing
environments HCI issues, HCI modes, User-acceptance, Usage Patterns: Needs versus
Viability Privacy and Security Issues Economic feasibility and Deployment issues Computing capabilities and scalability
Case Study of the BITS Life-guard Project Case Study of the Touch-Lives Initiative: Project
Connect and Project Communicate
Summary of Concepts learnt
Interaction Points
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
18/41 Dr. Rahul Banerjee, BITS Pilani, India
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
19/41 Dr. Rahul Banerjee, BITS Pilani, India
Began in 1999, this research aims to protect human livesfrom those road accidents that result from the reducedlevels of the physical fitness or mental alertness of thedriver.
Initially, it is focusing on light vehicles and their drivers /occupants. However, the concept is easily extensible tolarge vehicles and their drivers / occupants as well.
This research also draws on the works done by lifescientists on human sensory system, brain and select
externally measurable parameters (that can bemeasured, calibrated or accurately estimated withoutpiercing human body).
TheBITS-LifeguardWearableComputing Initiative
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
20/41 Dr. Rahul Banerjee, BITS Pilani, India
The overall life-saving environment in which the BITS-Lifeguard is envisioned to work shall have two corecomponents: The wearable computing component: The BITS-Lifeguard The vehicular computing component
The scenario of action would include: Part-I: sensing of select critical parameters that help estimate the current
level of alertness and physical ability to drive safely,
comparing these with the pre-fed threshold levels and generate analert to the driver;
in case, driver fails to respond quickly enough, send and SoS signalto the vehicular computer wirelessly These responsibilities are handled by the wearable computer
The Vision behindthe BITS-Lifeguard System (1 of 2)
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
21/41 Dr. Rahul Banerjee, BITS Pilani, India
The scenario of action would include: Part-II
Taking over control from the driver, Safely attempting to move the vehicle as per the pre-fed GIS
map and GPS data
Stopping the vehicle on a side Sending information wirelessly to the rescue / recovery
agencies providing the location details, vehicles details anddrivers details
Intimating to the pre-registered relative / friend about theevent and location
These steps are taken by the vehicles computer which formspart of the ITS and Vehicular Computing aspects of thispervasive computing initiative
The Vision behindthe BITS-Lifeguard System (2of 2)
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
22/41 Dr. Rahul Banerjee, BITS Pilani, India
A wearable computing system of this categoryneeds at least five basic elements: Non-Intrusive Sensory elements to sense the
wearers environment,
Computing elements to take care of computationalneeds;
Communication elements to interconnect thesecomputing elements (with mobility)
Body safe Power Supply / Generation elements toprovide the necessary power to the wearablecomputing system
Fabric or placeholder elements to allowinterconnected elements in place
Elements of the BITS-Lifeguard Non-
Intrusive Wearable Computing System
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
23/41
Dr. Rahul Banerjee, BITS Pilani, India
Howdoescontextawareness/contextpredic1oncombinedwithotherelementslikeactua1onmechanismsetc.allow
SmartorIntelligentPervasiveCompu1ngEnvironments?
Intelligent/SmartSpaces/Environments
DriverWith
body mountedwearable sensors
as part of theBITS Lifeguard
VehicularComputer
Outsideworld,
preferablyITS-enabled
(or at least2.5G / 3G /
4G / Wi-Fi
supported
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
24/41
Dr. Rahul Banerjee, BITS Pilani, India
Pervasive computing systems know about the context ofthe environment of the user(s) and involve multiple
compute-sense-communicate elements embedded in
users environment.
Wearable computing systems know more about thecontext of an individual wearer and involve compute-
sense-communicate elements mounted / embedded on /
in the body of the user in various ways.
A typical wearable computing system may either workindependently or may form part of an inclusive largerpervasive computing solution.
So, how do the Pervasive and Wearable
Computing systems go together?
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
25/41
Dr. Rahul Banerjee, BITS Pilani, India
Typical Problems and Issues related to technologyintervention in the Pervasive Computing
environments HCI issues, HCI modes, User-acceptance, Usage Patterns: Needs versus
Viability Privacy and Security Issues Economic feasibility and Deployment issues Computing capabilities and scalability
Case Study of the BITS Life-guard Project Case Study of the Touch-Lives Initiative: Project
Connect and Project Communicate
Summary of Concepts learnt
Interaction Points
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
26/41
Dr. Rahul Banerjee, BITS Pilani, India
Summary of the ConceptsLearnt
Pervasive computing systems know more:about the context of the environment of the user(s) andinvolve multiple compute-sense-communicate elementsembedded in users environment.
Wearable computing systems know more:about the context of an individual wearer andinvolve compute-sense-communicate elementsmounted / embedded on / in the body of the user invarious ways.
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
27/41
Dr. Rahul Banerjee, BITS Pilani, India
Rahul Banerjee:From Research to Classroom: A Course in Pervasive Computing, IEEE Pervasive Computing, July-
Sept. 2005, Vol. 4, No. 3, pp. 83-86.
Sailesh Conjeti, Rajiv Ranjan Singh, & Rahul Banerjee: Bio-Inspired Wearable Computing Architecture
and Physiological Signal Processing for On-Road Stress Monitoring, Proceedings of the IEEE-
EMBS International Conference on Bio-medical and Health Informatics, Shenzhen, Hong Kong, Jan
5-7, 2012, pp. 479-482. Available on IEEE Xplore: pp. 479 - 482, doi: 10.1109/BHI.2012.6211621.
Rajiv Ranjan Singh, Sailesh Conjeti & Rahul Banerjee: An Approach for Real-Time Stress-Trend
Detection in Physiological Signals in Wearable Computing Systems for Automotive Drivers,Proceedings of 14th International IEEE Conference on Intelligent Transportation Systems - ITSC
2011, October 5-7, 2011, George Washington University, Washington, DC, USA, pp. 1477-1482.
Mahesh M. Bundele & Rahul Banerjee: ROC Analysis of a Fatigue Classifier for Vehicular
Drivers, Proceedings of the 2010 IEEE International Conference on Intelligent Systems (IS 2010),
London, UK; < also available through the IEEE Explore>
.
Mahesh M. Bundele & Rahul Banerjee: Design of Early Fatigue Detection Elements of a WearableComputing System for the Prevention of Road Accidents, Proceedings of the 2nd International
Workshop on Intelligent System and Applications (ISA2010), Wuhan, China; . .
References to the Research atBITS Pilani (1 of 2)
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
28/41
Dr. Rahul Banerjee, BITS Pilani, India
Rajiv Ranjan Singh & Rahul Banerjee:Multi-parametric Analysis of Sensory Data collected from Automotive Drivers for
Building a Safety-Critical Wearable Computing System, Proceedings of the 2ndInternational Conference on Computer Engineering and Technology (ICCET
2010), April 16-18, 2010, Chengdu, China; .
Mahesh M. Bundele & Rahul Banerjee:Detection of Fatigue of Vehicular Driver using Skin Conductance and Oximetry
Pulse: A Neural Network Approach, ACM-sponsored International Conference:iiWAS2009, December 1416, 2009, Kuala Lumpur, Malaysia. Proceedings of
the iiWAS2009, pp. 725-730
Mahesh M. Bundele & Rahul Banerjee:
An SVM Classifier for Fatigue-Detection using Skin Conductance for Use in the
BITS-Lifeguard Wearable Computing System, Proceedings of the IEEE-
sponsored ICETET-2009, Nagpur, December 16-18, 2009, pp. 936-941.
References to the Research atBITS Pilani (1 of 2)
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
29/41
Dr. Rahul Banerjee, BITS Pilani, India
E-Fabric of a WearableGarment
Source: ETH, Zurich
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
30/41
Dr. Rahul Banerjee, BITS Pilani, India
Wireless Communication in e-Fabric
Source: ETH, Zurich
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
31/41
Dr. Rahul Banerjee, BITS Pilani, India
User Interface & Sensors
Elek Tex belongs to Eleksen. Peratech Ltd.
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
32/41
Dr. Rahul Banerjee, BITS Pilani, India
Several highly computationally intensive applications ofpervasive computing benefit from the Cloud-assisted
Pervasive Computing paradigm.
Cloud-assistance in such cases often comes at the back-end for handling various compute and data-intensivetasks related to a set of pervasive services.
However, there is a distinct possibility of multi-layer /multi-level cloud support at even intermediate levels of
support particularly in terms of collaboration betweenheterogeneous cloud units or between private clouds via
the public cloud.
About the Cloud-AssistedPervasive Computing
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
33/41
Dr. Rahul Banerjee, BITS Pilani, India
About the Course Pervasive Computing: What is it? Examples of Pervasive Computing Environments Situations that warrant Pervasive Computing Solutions Typical Problems and Issues related to technology intervention in
the Pervasive Computing environments HCI issues, HCI modes, User-acceptance, Usage Patterns: Needs versus Viability Privacy and Security Issues Economic feasibility and Deployment issues Computing capabilities and scalability
Case Study of the BITS Life-guard Project Case Study of the Touch-Lives Initiative: Project Connect and
Project Communicate
Summary of Concepts learnt and the Exercise for the day
Interaction Points
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
34/41
Dr. Rahul Banerjee, BITS Pilani, India
Pervasive Computing with AR
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
35/41
Dr. Rahul Banerjee, BITS Pilani, India
AScent-emi6ngDisplaySystem
Aninnova1vedisplaysystemdevelopedrecentlybytheNTT
Communica1onCorpora1onhad
beeninstalledatanunderground
MallsituatedintheTokyostrain
sta1ononanexperimentalbasisinJuly2008.
ITemitsappe1zingaromasalongwiththeadver1singvideosbeing
displayedona42-inchLCDDisplaypanel.
Thiscommercialtex1ngwasdonebyRecruitCo.Ltdforadver1sing
cafesandrestaurants.
NEC, Japan, 2008
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
36/41
Dr. Rahul Banerjee, BITS Pilani, India
DIGITAL GRAFFITI ON SEE-THROUGH
DISPLAYS IN AR SYSTEMS
Source:PERVASIVE2006-7.May2006(c)AloisFerscha
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
37/41
Dr. Rahul Banerjee, BITS Pilani, India
DISPLAYS WHICH YOU CANSTEER
Steerable Displays
Source:
PERVAS
IVE200
6-7.
May200
6
(c)AloisFersch
a
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
38/41
Dr. Rahul Banerjee, BITS Pilani, India
Pervasive Computing is all about designing and deploying solutions
that would use an inherently distributed infrastructure Such an infrastructure comprises of compute-sense-communicate elements and is
often used to support various types of customizable services via appropriate software
and services. A pervasive computing system has an ability to recognize context of the
target environment for the purpose of offering desired set of pre-defined
as well as adaptive services, preferably without the users being required
to be aware of its presence.
The design principles of Pervasive Computing are often driven by thedesirable characteristics like:
Simplicity Versatility Pleasurability
Summary of the ConceptsLearnt
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
39/41
Dr. Rahul Banerjee, BITS Pilani, India
Study Project Aura (http://www.cs.cmu.edu/~aura/) atCMU and identify the involved elements.
Consider using the basic framework of the ProjectOxygen for applications other than those conceived by
the CMU researchers. Conceptualize your own model of a simple pervasive
computing system that could allow a given room to be
converted into a simple pervasive computing unit by
using commonly available off-the-shelf devices and otherelements.
Your exercise for the day!
f
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
40/41
Dr. Rahul Banerjee, BITS Pilani, India
References
7/29/2019 CSG541-SSG531-Lecture-2-2012-2013
41/41
Thank you very much for you time and attention!
My Home page may have some additional information ofpossible interest to you. You may want to access it at the
URL: http://www.bits-pilani.ac.in/~rahul
Just in case, you may wish to contact me by Email, my E-
mail address is: rahul@bits-pilani.ac.in
Please take a look at the reading advisory at
Nalanda LMS Portal for additional information!