Cognitive

Assistance in

Smart

Homes

Sylvain Giroux Sylvain Giroux and Hélène and Hélène

PigotPigot

Plan

Cognitive Assistance

Context, objectives, and approach

From homes…

Hardware level: networks, sensors, effectors

… to smart homes …

Middleware level: pervasive infrastructure

… to smart care !

Application level: cognitive assistance & tele-monitoring

Validation

Usability and clinical studies

Conclusion

ContextP

eople suffering of cognitive impairments in Quebec Alzheimer disease : 5.1% of people over 65 years old Head trauma : 3000 new cases each yearSchizophrenia : 1% of the population

In many cases, they would be able to stay at home

if light assistance was provided.

But healthcare resources are scarce So relatives have to take responsibility for care

It then turns to an exhausting burdenHence relatives and caregivers urge for help.

Objectives

Provide adapted and personalized environmental cues to

Foster the autonomy of cognitively impaired people

Reduce risks and hazards

Pervasive computing & Tangible user interfaces

Keep ensuring continuous cognitive assistance outside people’s home

Mobile computing & Location-based services

Help relatives and caregivers to stay in touch at distance with cognitively impaired people

Our intention is NOT to replace relatives and caregivers

From homes…S

mart homes are augmented environments

Heterogeneous networks

Sensors networks

Embedded processors in devices, clothes, jewels…

Information devices

Networked communicating objects

DOMUS: an augmented apartment

… smart homes …S

mart homes are augmented environments

Heterogeneous networks

Wireless : WiFi, Bluetooth, RFID, UWB...Wired : Ethernet, Electrical wires, X10, power line…

ServersFull control over audio and video streams

… smart homes …S

mart homes are augmented environments

Sensors networks

Identification and localization of objects and people

Ubisense tags, UWBSmart tags (RFID)

… smart homes …S

mart homes are augmented environments

Embedded processors in devices and clothesUnder process

Perkowitz et al., Mining Models of Human Activities from the Web, WWW 2004, May 17-22, 2004, New York, NY USA.

… smart homes …S

mart homes are augmented environments

Information devicesFixed: smart boards, Icebox… Mobile: laptop, wireless screen, PDAs…

… smart homes …S

mart homes are augmented environments

Networked communicating objectsSightHearing

Smell (not yet investigated)Touch (not yet investigated)

Taste (is it possible ?)

From homes to smart homes…

… to smart care

Application level

Cognitive assistance

Tele-monitoring

Cognitive deficits

Initiation

inactive periods whereas the person is supposed to perform actions

Memory

difficulties to remember the activity to perform, the steps, the locations of the objects involved

Planning

difficulties to perform an appropriate sequence of actions in the right order to achieve a goal

Attention

shifts of attention from the activity under progress to a stimulus causing interference

Cognitive

Assistance: a

Demo

Cognitive deficits

Initiation, memory, planning, attention

User interfaces

Traditional GUI vs Tangible user interfaces : Transforming the whole house into a cognitive orthotic

Who initiate the interactions?

User vs Assistant

Telemonitoring

Asynchronous vs Synchronous

Usability

& clinical studies

Where : Domus laboratory

Task : Meal preparation

Whom : Intellectually handicapped people

When : January 2007

                                

                

ConclusionGoing beyond the usual view of computing as “PC-based”

Pervasive computing enables a seamless integration of assistance in residents’ everyday life

Going beyond traditional human computer interfaces

Tangible user interfaces help to turn the whole house into a cognitive orthotic

Such smart homes can Provide cognitive assistanceFoster people’s autonomyLead to smarter care

                                

                

What’s next ?What’s next ?Current prototype

Monitoring of one activity by one personThe assistance system reacts after the user “error”

In a real environment, Must monitor many concurrent activities, many persons

Next issues Manage concurrent activities recognitionPredict the user behaviour (and ideally anticipating his “errors”)Disambiguate activities from observed eventsIdentify opportunities for assistance

Cognitive assistance

What is the available information ?

Identification and location of people and objectsObjects involved in an activityPrimitive actions

Who the user is ?

personalization

What is the user doing ?

Activity recognitionHierarchical modelsLattice-theory

How to assist the user?

Highligth objectsTangible user interfaces

Initiation deficitsInitiation deficits leads to inactive periods

whereas the person is supposed to perform actions

Wandering for a long time could be attributed to an initiation deficit

>> Prompt the resident

Memory deficits

Difficulties to remember

the activity to perform the steps of the activity the locations of the tools and materials involved in that activity.

>> Show-me objects + « Follow-me » applied to objects

The lamp turns off when the object is too far away.

The lamp turns on to highlight the searched red book

Planning deficitsD

ifficulties to perform an appropriate sequence of actions in the rightorder to achieve a goal.

>> Prepare_coffee = {take_milk, take_cup…}

>> Show to the user where to perform the next action

Usability

& clinical studies

Clinical studies using prototypes are in preparation and will soon be on-going at

Fernand-Séguin research center, L-H Lafontaine Hospital, Montreal

Dr Emmanuel Stip, psychiatristSchizophrenia

Centre de Réadaptation Estrie, SherbrookeHead trauma

Domus laboratoryIntellectually handicapped people