Hou Hsiao-hsien (Hou Xiaoxian) is an internationally acclaimed ...
Presenter: Jennifer C. Hou ‡
description
Transcript of Presenter: Jennifer C. Hou ‡
Personal Assistant System: Use of Computer Technologies to Facilitate Elderly People with Assisted Living
Presenter: Jennifer C. Hou‡ Collaboration with Linda Ball*, Stanley Birge*, Marco Caccamo‡, Carl Gunter‡, Karrie G. Karahalios‡, Narasimhan Nitya #, Lui Sha‡,
Yang Yu#
‡ Dept of Computer Science University of Illinois at Urbana Champaign*College of Medicine, Washington University in Saint Louis#Pervasive Platforms and Architecture Lab, Motorola [email protected], 217-265-6329
Population Aging
Table compiled by the U.S. Administration on Aging based on data from the U.S. Census Bureau.
Aging of the baby boomer has become a social and economical issue.
In the United States alone, the number of people over age 65 is expected to hit 70 million by 2030, almost doubling from 35 million in 2000.
Percentage of People of 65+ and 85+
People over age 65 are expected to constitute 20% of the population in 2030.
Similar increases are expected worldwide.
Table compiled by the U.S. Administration on Aging based on data from the U.S. Census Bureau.
Similar Expenses Worldwide
SOURCE: United Nations ▪ “Population Aging ▪ 2002”
2002
Similar Expenses Worldwide
SOURCE: United Nations ▪ “Population Aging ▪ 2002”
2030
Consequences Are …..
Along with the increase of elderly people population, the expenditures of the United States for health care will project to rise to 15.9% of the GDP ($2.6 trillion) by 2010.
-- Health care industry study, Digital Foresight
Many elderly people will stay at home, rather than being consigned to expensive retirement homes.
Even today, only 10% of elderly people of age 65-85 and 25% of those of age >= 85 are institutionalized.
Many elderly people choose to stay at home also for privacy/dignity issues.
Overview of the Talk
Why Do We Care About Senior Care? Overview of Personal Assistant System (PAS) Underlying Research that Makes Things Work
Systems Architecture Wireless communications Tracking and localization Security and privacy Software safety, reliability, and availability Human computer interface
Do Elderly People Accept This Technology? Pilot studies
Where Do We Go From Here?
Why Do We Care About Senior Care
Because your parents are the next in line to be qualified as seniors, and you are next to the next in line.
Can advances in sensing, object localization, wireless communications technologies
enable elderly people to regain their capability of independent living?
make possible unobtrusive supervision of basic needs of frail elderly and thereby replicate services of on-site health care providers?
We believe the answers are Yes!
We Believe Technology Can Help
Time Driven Reminders of Daily Activities:
Home PC Serves as the
intelligence. Sends reminder
messages to wireless-enabled appliances.
Closes the loop with HCI/ localization techniques.
Takes action in the lack of response: A reminder can be sent more times, after which a designated on-site personnel or a healthcare provider is notified.
Jennifer, It is 8:30am.Time to take your Insulin injection before breakfast.
Overview of PAS
ALSP
Clinician Service
Gateway
Assisted Living Hub (ALH)
MedicalDevice
A drop-box architecture in which A security-enhanced, assisted living device called Authentication
Manager for You (AMY) co-exists with a home PC, called the Assisted-Living Hub (ALH).
The ALH is equipped with multiple wireless interfaces and serves as the local intelligence. It also communicates with the Assisted Living Service Provider (ALSP).
Through web interfaces, ALSP allows healthcare providers and clinicians to retrieve/analyze data and give instructions.
A More Technical View of PAS
Home Environment
. AMY Monitoring
ServiceInternet
ALH
IEEE 802.11WLAN
Clinicians
ALSP
• Easy to deploy.• Few entities that are subject to security attacks.• Numerous applications can be built on top of it.
What Applications Are Most Critical
% of Residents % of Residents asNeed With Need Primary Cause________________________________________________________________________Needs prompting to take medications 95 42Risk of injury due to falls 42 17Unable to get up after a fall 20 17Monitoring of vital signs too labor intensive 12 20Needs physical assistance with Activities of Daily Living 90 67Needs prompting to toilet on a schedule 67 17Needs prompting to go to meals 33 10Needs prompting to bathe 75 0Gets lost in apartment 17 0May wander out of facility 12 10Needs monitoring of blood sugar frequently 20 8Needs monitoring of weight daily/weekly 25 0
Factors Contributing to Loss of Independence and Institutionalization*
* Respondents include 8 geriatricians, 10 nurse administrators of assisted living facilities, and 6 home health clinicians at Washington University in Saint Louis
Application I: Time-based Reminder Services
PAS can help reminding residents of daily activities. ALH obtains from the ALSP updated prescription and
appointment records of a resident. When it is time for the resident to carry out their time
driven routines, the ALH locates active wireless-enable devices and sends reminder messages to one or more devices that are in the proximity of the resident.
Whether or not these routines are followed as advised is detected in a non-intrusive manner by exploiting sensor localization techniques.
ALSP
Clinician Service
Gateway
Assisted Living Hub (ALH)
MedicalDevice
Application II: Monitoring of Physiological Functions A number of physiological functions critical to maintaining homeostasis for different
medical conditions can be measured by Bluetooth-enabled medical devices, transmitted to the ALH and then to the ALSP to be evalauted by healthcare providers.
Measures will have a prescribed desired range and deviations from that range will generate an alert from the ALSP to the health care provider.
This enables prompt intervention before the situation deterioates to a point requiring hospitalization and may simply be in the form of additional instructions to the resident.
Home Environment
. . AMY Monitoring ServiceInternet
IEEE 802.11WLAN
ALSP
Application III: Non-intrusive Monitoring of Daily
Activities and locations Detection of early warning signs for depression and/or other chronic diseases:
The location of a resident and the duration in which he/she stays at that location are profiled by Motes-based (or Ubisense-based) sensing and tracking in a privacy-preserving manner.
Movement profiles are transmitted to the ALSP and analyzed (if necessary) for Early warning signs for severe depression (e.g., not taking medicine, not eating, staying
in bed for very long time) Preventive measures (behavior changes) for chronic diseases commonly seen in elderly
people (such as Parkinson’s disease and/or Alzheimer’s disease).
Application IV: Fall Detection
Residents wear sensors equipped with accelerometers (with fall detection algorithms that detect falls with the combination of speed and orientation changes).
In the case of fall detection, the sensor device beeps and an alert message is sent to the ALSP and to a designated healthcare provider.
In the case of false alarm, the resident can press a button and disable the message sending.
Overview of the Talk
Why Do We Care About Senior Care? Overview of Personal Assistant System (PAS) Underlying Research that Makes Things Work
Systems Architecture Wireless communications Tracking and localization Security and privacy Software safety, reliability, and availability Human computer interface
Do Elderly People Accept This Technology? Pilot studies
Where Do We Go From Here?
Software Infrastructure Needed
A software infrastructure that integrates sensing, communication, and event/information management.
Understand, analyze, structure and control the complex interactions across the layers of computing, communication and sensing along the dimensions of robustness, reliability, QoS, security and privacy.
Interface mgmt
Events/data management
Communication
Sensing
Dependability Real-timeSecurity
& Privacy
Evolvability
Reminders/social interaction
Monitoring Emergency event processingTelemedicine
J2ME API for AccessingBT Nodes and Internet
Under Normal Situation
When the Gateway is Available
Active Application A
BT StackFixedALHOS
TCP/IPComm.
API
ALSP Server
TCP/IP Stack
ALSPServer
OS
Cellphone Network
+ Internet
Bluetooth
TCP/IP Stack
802.11Wireline
MAC
IP Routing Service
802.11MAC
Gateway
ServerSocket ss = new ServerSocket(90);
ss.accept();
Socket s = new Socket(myaslp.com:90)
Vital Sign Meters
Networked Software Architecture
System Architecture of the ALH
OS Layer in the ALH Architecture
The OS layer Is equipped with
various communication stacks and corresponding platform-dependent APIs
Depending on the family of peripherals to support, other stacks and APIs may include Bluetooth, Zigbee, and Infrared.
Middleware Layer in the ALH Architecture
Middleware Device monitoring
daemons: monitor the join/leave of peripheral devices in the environment and registers/de-registers the devices in the Device Registry Services.
The register-de-register process includes creating/destroying proxy or singular proxy stub objects for the device.
The proxy/proxy stub allows applications to be built upon well-known device APIs instead of vendor specific APIs/semantics. It also allows off-the-shelf devices to be integrated into the PAS system, as long as the vendor provides the semantics specifications on how to communicate with the device.
Middleware Layer in the ALH Architecture
Device registry service: maintains a database of peripherals available in the environment. Each entry of the database is a proxy (or proxy stub) object created for a specific peripheral device.
Applications will query this service for appropriate proxies. When an appropriate proxy is found, a clone of the proxy migrates to the application.
Middleware Layer in the ALH Architecture
The Unified Peripheral Communication APIs
Abstracts different network stack programming interfaces into a consistent paradigm, which basically follows java.net.* APIs.
Allows users to specify (in XML) their QoS demands when establishing communication links, and the middleware returns approved QoS guarantees.
This allows proxies to be developed independently of specific network stack programming interfaces.
Middleware Layer in the ALH Architecture
Internet Heartbeat Daemon:
periodically checks the availability of Internet access through the gateway router.
When the gateway router fails/recovers, this daemon activates/deactivates the Bluetooth cell phone to access to the ALSP.
ALH Main Daemon: is in charge of managing (start, suspend, stop, restart etc.) all the application daemons and middleware daemons on the ALH.
Overview of the Talk
Why Do We Care About Senior Care? Overview of Personal Assistant System (PAS) Underlying Research that Makes Things Work
Systems Architecture Wireless communications and networking Tracking and localization Security and privacy Software safety, reliability, and availability Human computer interface
Do Elderly People Accept This Technology? Pilot studies
Where Do We Go From Here?
ALSP
Gateway
ALH
TV
earplug Internet
WLAN(e.g. IEEE 802.11)
Networked Software Architecture Peripheral Bluetooth Network
Internet-capableMedicalMeters
J2ME API for AccessingBT Nodes and Internet
Under Normal Situation
When the Gateway is Available
Active Application A
BT StackFixedALHOS
TCP/IPComm.
API
ALSP Server
TCP/IP Stack
ALSPServer
OS
Internet
Bluetooth
TCP/IP Stack
802.11Wireline
MAC
IP Routing Service
802.11MAC
Gateway
ServerSocket ss = new ServerSocket(90);
ss.accept();
Socket s = new Socket(myaslp.com:90)
Vital Sign Meters
Networked Software Architecture
J2ME API for AccessingBT Nodes and Internet
Passive Application A
BT StackFixedALHOS
TCP/IPComm.
API
ALSP Server
TCP/IP Stack
ALSPServer
OS
Cellphone Network
+ Internet
Bluetooth
TCP/IP Stack
802.11Wireline
MAC
IP Routing Service
802.11MAC
Gateway
Socket s = new Socket(128.174.11.11:90)
ServerSocket ss = new ServerSocket(90);
ss.accept()
Dumb Dev
PC IP: 128.174.11.11
Under Normal Situation
When the Gateway is Available
Networked Software Architecture
What If the Internet Gateway Is Down?
One major deficiency of PAS is that all the traffic is transported through the gateway AMY, which becomes unavailable when it fails or when the resident is away from home.
To enhance robustness and ubiquity of PAS, we have used cell phones (transparent to users) as both a backup AMY and the local intelligence for data aggregation and acquisition.
we have leveraged the programming capability of Motorola EZX platforms (mainly the A780, E680 and E680i phones) to incorporate the following novel features that are not currently present in any cell phones:
Enabling cell phones to serve as local data storage/fusion intelligence.
Enabling cell phones as a delivery endpoint for reminder messages
ALSP
ALH
TV
earplug Internet
WLAN(e.g. IEEE 802.11)
Networked Software Architecture
Peripheral Bluetooth Network
When the Gateway is Not Available
Internet-capableMedicalMeters
Base Station
cellphone
GPRS network
or WiFi
mobile medicaldevices
APIs (TAPI/NAPI, File system, etc.) J2ME + Bluetooth Dial-Up Networking Profile
(BT-DUN) for Internet AccessService Layer (Network, Connectivity, Telephony, etc)
GPRSBT StackCellphone
OS
Active Application A
BT StackFixedALHOS
TCP/IPComm.
API
ALSP Server
TCP/IP Stack
ALSPServer
OS
Cellphone Network
+ Internet
Bluetooth
802.11Wireline
MAC
IP Routing Service
802.11MAC
A. L. Device
Bluetooth MedicalMeter
ServerSocket ss = new ServerSocket(90);
ss.accept();
Socket s = new Socket(myaslp.com:90)
Cellphone DUN Modem(service provided by most BT enabled cellphones)
The cellphone modem is activated
J2ME + Bluetooth Dial-Up Networking Profile
(BT-DUN) for Internet Access
Active Application A
BT StackFixedButler
OS
TCP/IPComm.
API
ALSP Server
TCP/IP Stack
ALSPServer
OS
Cellphone Network
+ Internet
Bluetooth
802.11Wireline
MAC
IP Routing Service
802.11MAC
Gateway
Bluetooth MedicalMeter
The cellphone modem is activated Socket s = new Socket(128.174.11.12:90)
ServerSocket ss = new ServerSocket(90);
ss.accept()
APIs (TAPI/NAPI, File system, etc.) Service Layer (Network, Connectivity, Telephony, etc)
GPRSBT StackCellphone
OS
Cellphone DUN Modem(service provided by most BT enabled cellphones)
ALSP
mobile medicaldevices
earplug Internet
WLAN(e.g. IEEE 802.11)
Networked Software Architecture
Peripheral Bluetooth Network
When cell phone takes the roll of ALH
Internet-capableMedicalMeters
Base Station
GPRSor WiFi
Cellphone
APIs (TAPI/NAPI, File system, etc.) J2ME
Service Layer (Network, Connectivity, Telephony, etc)
GPRSOr WiFi
BT StackCellphone
OS
Active Application A
BT StackFixedButler
OS
TCP/IPComm.
API
ALSP Server
TCP/IP Stack
ALSPServer
OS
Cellphone Network
+ Internet
Bluetooth
802.11Wireline
MAC
IP Routing Service
802.11MAC
A. L. Device
Bluetooth MedicalMeter
ServerSocket ss = new ServerSocket(90);
ss.accept();
Socket s = new Socket(myaslp.com:90)
Application 1(Listening to PC)
The cellphone is used as a smart device
Application 2(Sending data to ALSP
Server)
QoSbased
J2ME API for AccessingBT Nodes and Internet
When cell phone takes the roll of ALH
Active Application A
BT StackFixedALHOS
TCP/IPComm.
API
ALSP Server
TCP/IP Stack
ALSPServer
OS
Internet
Bluetooth
GPRSOr WiFi
GPRS network
or 802.11
IP Routing Service
Base Station
ServerSocket ss = new ServerSocket(90);
ss.accept();
Socket s = new Socket(myaslp.com:90)
Vital Sign Meters
Networked Software Architecture
Overview of the Talk
Why Do We Care About Senior Care? Overview of Personal Assistant System (PAS) Underlying Research that Makes Things Work
Systems Architecture Wireless communications and networking Tracking and localization Security and privacy Software safety, reliability, and availability Human computer interface
Do Elderly People Accept This Technology? Pilot studies
Where Do We Go From Here?
Real-Time Tracking and Localization
We have built a system that uses a combination of ultrasonic and RFID technologies as the underlying sensing mechanism for real-time tracking of both human and objects.
Human Tracking: Ultrasonic transmitters,
called beacons and denoted by Bi, occasionally sends out an ultrasonic pulse and a short RF signal at the same time.
Due to the speed difference between an RF signal and an ultrasonic signal, the listener denoted by Lj on the resident can infer its distance from the beacon using Time Difference Of Arrivals of the two signals.
This distance measurement can be reported to the ALH for real-time tracking of the use (marked as (2) and (3)).
Real-Time Tracking and Localization
Object tracking The RFID (radio
frequency identification) reader attached on the wristband can read RFID tags of objects touched by the user.
The objects can be tracked, whenever the user touches and/or carries them.
The whereabouts of the objects are logged.
Overview of the Talk
Why Do We Care About Senior Care? Overview of Personal Assistant System (PAS) Underlying Research that Makes Things Work
Systems Architecture Wireless communications and networking Tracking and localization Security and privacy Software safety, reliability, and availability Human computer interface
Do Elderly People Accept This Technology? Pilot studies
Where Do We Go From Here?
How Medical Data Can Be Securely Transmitted ?
SOAP Envelope
Header
MedicalInformation
SOAP Envelope
Header
MedicalInformation
Sign. on Msg.
Sign. on Key
PubC KF
PubC/PrivC
KFPWAP
PWAP
PubAS/PrivAS
SOAP Envelope
Header
MedicalInformation
Sign. on Msg.
PubAS
PWAP
[Michael et al. 06]
PubC/PrivC
Sign. on Msg.
KFPWAP
PubAS/PrivAS
SOAP Envelope
Header
MedicalInformation
Sign. on Msg.
PubAS
Sign. on KeyPWAP
SOAP Envelope
Header
MedicalInformation
PubAS/PrivAS
PubC KF
PWAP
PWAPPWAP
How Medical Data Can Be Securely Transmitted ?
PubC/PrivC
KFPWAP
PubAS/PrivAS
SOAP Envelope
Header
MedicalInformation
PubAS/PrivAS
PubC KF
PWAPPWAP
Sign. on Msg.PrivAS
SOAP Envelope
Header
MedicalInformation
Sign. on Msg.
PubC
Sign. on KeyPrivAS
How Medical Data Can Be Securely Transmitted ?
PubC/PrivC
KFPWAP
PubAS/PrivAS
SOAP Envelope
Header
MedicalInformation
PubAS/PrivAS
PubC KF
PWAPPWAP
PubAS
SOAP Envelope
Header
MedicalInformation
Sign. on Msg.
PubC
Sign. on KeyPrivAS
PubC/PrivC
SOAP Envelope
Header
MedicalInformation
PubC
Sign. on Msg.PrivAS
SOAP Envelope
Header
MedicalInformation
PubC KF
PubAS PubC/PrivC
MedicalInformation
PubAS
How Medical Data Can Be Securely Transmitted ?
Implementation
Apache Axis(WSDD / SOAP, XML-RPC / Java RPC, EJB, HTTP / WSDL Tools)
AssistedPerson
Clinicians / Family or Friends / Assisted Person
ALSPCompany
MedicalDevice
AL Hub
MessageGenerator
EncryptData
ALSPServer
SAAJ
Computer
AL SecurityEngines
Add Signatures/
EncryptEnvelope
DB Access
XMLSec
DecryptEnvelope/
VerifySignatures
MessageGenerator
Add Signatures/
EncryptEnvelope
Add Signatures/
EncryptEnvelope
MessageGenerator
DecryptEnvelope/
VerifySignatures
DecryptData
DecryptEnvelope/
VerifySignatures
UserInterface
JCE
Drop-box Prototype
AL Messages
Implementation
H/W Device: Digital Pulse Oximeter AL Hub: A labtop, Windows ALSP: A desktop, Linux Clinician’s computer: A labtop, Windows
S/W Language: Java Database: MySQL Web service: Apache web server, tomcat, axis 1.4 Libraries: XML-SEC, SAAJ, avetanaBluetooth, etc.
Security tokens
The following security tokens are set up Doctor’s public key certificate The ALSP’s public key certificate A secret key shared among AP and AP’s family members or friends AP’s and family/friends’ password to ALSP Access control matrix (ALSP), DataID table (AL Hub)
• Doc: Doctors office’s public key certificate.
• AS: The ALSP's public key certificate. • U*; V*: URL pointers to the ALSP's public key certificate. U*; V* are the ones which AP and clinician have, respectively.• KAP: A secret key shared the AP and her family members.
• PPA; PFA: AP's and Family's secret passwords (respectively) shared with the ALSP.• ACLAP: An access control list for the AP's records at the ALSP.• ΔAP: A mapping table from a serial number of a medical device to DataID
Doc, AS, KAP, U*; V*, PPA , PFA , ACLAP , ΔAP
doc, AS, KAP, U*, V*, PPA, PFA, ACLAP, ΔAP
Bootstrapping
1) Clinician sets up relationship with ALSP / Exchange URL pointers2) Assisted person (AP) subscribes to an ISP / Set up a gateway for WPA network3) AP visits clinician. Clinician recommends a list of devices, hubs, and ALSPs. AP
buys devices and hub and adds them to home network4) AP subscribes to ALSP. ALSP sends a URL pointer and a password token. AP loads
them into the hub.5) AP creates a shared “family key” for family members’ and friends’ access to data
on the ALSP. Family members register passwords to ALSP.6) Hub generates ΔAP , a set of DataID. A dataID is mapped to a device serial number
and recipients’ URL pointers. Each entry is a tuple (serial number, dataID, URL pointer)
7) With the recipient information (clinicians and family members), ALSP creates access control list ACLAP.
AP
Clinician
FamilyALSP
Overview of the Talk
Why Do We Care About Senior Care? Overview of Personal Assistant System (PAS) Underlying Research that Makes Things Work
Systems Architecture Wireless communications and networking Tracking and localization Security and privacy Software safety, reliability, and availability Human computer interface
Do Elderly People Accept This Technology? Pilot studies
Where Do We Go From Here?
Pilot Study at Nazareth Living Center*
With the help of geriatricians at Washington University in Saint Louis, we have carried out a 3-week pilot study at the Nazareth Living Center for Assisted Living in June-July 2006.
This facility houses 110 well-educated, predominantly female residents, whose average age was 88. Of 30 residents who attended a presentation, 14 agreed to participate.
After administering a standard cognitive assessment, two residents were consented to test the PAS prototypes (application 1 and applicatoin2) and ten residents to carry/wear a placeholder device for a period of two weeks.
* Nazareth Living Center is a Catholic, nonprofit skilled nursing and assisted living community, sponsored by the Sisters of St. Joseph of Carondelet.
Pilot Study at Nazareth Living Center
Residents lack in confidence in PAS when it did not work properly:
Wireless connectivity is made possible by a low-end Linksys WRT54G wireless router. This, coupled with concrete walls between residents' rooms and the nurse station, led to intermittent connectivity.
Residents, as a result, were not confident in relying solely on PAS for medical monitoring.
To address this problem for the time being, we will add a wireless repeater (i.e., an additional Linksys WRT54G router configured in the client-mode and forwarding all packets to another, master router) between residents' rooms and the nurse station.
Residents found PAS useful and were willing to wear the sensing apparatus:
The two residents using the PAS prototype found it to be quite useful.
Residents not chosen to use the working prototype expressed their desire to use the working version as well.
Pilot Study at Nazareth Living Center
Nurses/caretakers desire interfaces that provide security/privacy.
With a high resident-to-nurse ratio, nurses were usually very busy and could not ensure that the information being displayed would not be viewed by unauthorized personnel. The need for privacy should be addressed by designing adequate access control to PAS.
Nurses inquired whether or not medical data could be securely transmitted via wireless technology. This implies they also had concerns about PAS security.
Overview of the Talk
Why Do We Care About Senior Care? Overview of Personal Assistant System (PAS) Underlying Research that Makes Things Work
Systems Architecture Wireless communications and networking Tracking and localization Security and privacy Software safety, reliability, and availability Human computer interface
Do Elderly People Accept This Technology? Pilot studies
Where Do We Go From Here?
Where Do We Go From Here?
Technology enhancement with respect to software robustness/reliability, security/privacy, and HCI.
Extensive, hypothesis-driven clinical trials with Washington University in Saint Louis and Buehler Center on Aging, Health & Society, Northwest University
The average cost of nursing home stay is about $8,300/month*. Can PAS delay transfers of elderly people to skilled nursing facilities (SNFs) and improve the quality of their lives (by preserving independence)?
Delaying institutionalization by simply one month would cover the cost of deploying PAS.
*Phibbs CS, Holtz J-EC, Goldstein MK et al. The effect of geriatrics evaluation and management on nursing home use and health care costs: Results from a randomized trial. Med Care 2006;44:91-5.
Concluding Remark
With most of the PIs being baby boomers, it is our sincere hope that we will be the first generation to live independently and with dignity with the help of PAS when we age, and the project will benefit the healthcare system and the society as a whole.