SISARL Appliance Reference Model

Jane W. S. LiuInstitute of Information Science

Academia SinicaNankang, Taipei, Taiwanjaneliu@iis.sinica.edu.tw

September 2004

1. Introduction

This paper proposes a reference model of appliances in Sensor Information Systems for Active Retirees and Assisted Living (SISARL). Here, the term SISARL refers broadly to intelligent sensors, smart devices, autonomous appliances, and sensor information systems and services that are designed to enhance the safety and quality of life of active retirees1, assist them in health maintenance, enable them to live independently for the most part of old age, and when assisted living becomes necessary, easy the burden of their caretakers and reduce the costs of home care. Examples include object locators that can be used to find misplaced household items; smart pantries that can inventory grocery items and notify designated suppliers or a caretaker for just-in-time replenishments or appropriate actions; smart medication dispensers that can be programmed, monitored and validated remotely; heart beat monitors that can record and process heart beat samples, detect abnormal patterns, and send notifications or alarms; and robotic helpers and walkers that enhance accessibility, mobility and safety of the weak and disabled.

Recent advances in sensors, signal processing, autonomous systems and robots, wireless communication, multimedia, sensor information management and e-commerce have made a broad spectrum of SISARL appliances and services feasible. In the coming decades, the need for SISARL will become increasingly more critical as an increasingly larger percentage of peoples in developed and developing countries enter retirement age, their life expectancy and old-age time span increase, and work force in health and home care decline2. Low-cost and dependable SISARL products and services will not only help to improve the well-being of the elderly, but also present a new opportunity in the consumer electronics market place worldwide.

1 Active retirees are elderly individuals who are still in relative good health, live an active lifestyle and are typically financially sound. They constitute the majority of people beyond retirement age and offer an increasingly larger target for consumer electronics and services designed to enrich their quality of life. 2 According to statistics compiled by Japan Associative Products Association in April 2003, the average percentage population over 65 in Japan, USA, England, Germany and France is 15.4, 17.2, and 20.3 in years 2000, 2010, and 2020, respectively. In Taiwan, the percentage population over 65 in these years is 8.6, 10.3 and 15.1, respectively. In contrast, the average percentage population under 15 will decline to less than 14 during these decades.

DRAFT

This paper is concerned with componentization as a way to reduce the cost and ensure the quality of SISARL. While some SISARL appliances are home and personal electronics, which are luxuries in life, others are point-of-care devices that are necessities of life for their users and must be affordable by users of all income levels. For this reason, how to keep the cost of high quality SISARL appliances and services low is among the most important problems in the development of SISARL. Many products and services that are indispensable today have demonstrated the merits of componentized design undisputedly: The best examples are networks, distributed systems and Internet. Other examples are automobiles. The costs of SISARL can be kept low in a similar way by constructing diverse appliances and services from common reusable and configurable components that can be produced independently and customized to suit the needs and preferences of individual users and social and technical support infrastructures available to them.

Specifically, this paper describes a SISARL appliance reference model that can serve as a basis for componentization of diverse SISARL appliances. Similar to the classical ISO-OSI reference model of networked and distributed systems [1], the SISARL appliance reference model is not a proposed architecture of SISARL appliances. Rather, the model intends to provide a framework within which similarities and differences of SISARL appliance and system architectures can be compared and contrasted in concrete and specific terms. Such a model is essential for the identification, definition and standardization of internal and external interfaces of SISARL appliances and system. It will also provide a much needed foundation for the exploitation of system-on-chip (SOC) and software/hardware co-design technologies.

The remainder of the paper is organized as follows. Section 2 discusses related work. As a way to motivate the proposed reference model, Section 3 describes possible configurations of several sample SISARL appliances. Section 4 describes the proposed reference model. Section 5 summaries the paper. The appendix discusses first-generation SISARL appliances and services that can be built with current technologies, together with common underlying assumptions.

2. Related Work

Again, the proposed SISARL appliance model intends to serve a similar purpose as the ISO-OSI reference model of distributed systems [1]. Despite criticisms against it, no one can dispute the fact that the ISO-OSI reference model has contributed significantly to the standardization of network and communication interfaces and protocols and, thus, stimulated and enabled the componentization of networks, distributed platforms and applications. Subsequent generic and reference models and standards of terminals, files and applications (e.g., [2-5]) have made it possible for network, platform and application components to be developed independently and evolve smoothly. Some SISARL appliance may serve as point-of-care medical devices; they must compliant to standards such as IEEE/ISO 1073 [6] which govern interconnections and communications among such devices.

Many academic and industrial research efforts focus specifically on technologies for aging-in-place, assisted living and home care. Examples ([6-15]) are technologies for motion and activity tracking, memory assistance and enhancement, and customizable personal health

2

DRAFT

management. The results of these efforts should enable future SISARL to deliver a wider range of services and evolve from helpful consumer electronics and personal care devices for active retirees to critical assisted living and home care appliances. Very few proof-of-concept prototypes in this area can be readily transitioned into affordable and effective products and services, however. A reason is that the problems on how to design and produce them amass and tailor them for users who have different habits, needs, income levels, and supporting infrastructures have been overlooked. Componentization is one of these problems.

Recent research efforts on sensor networks and sensor information systems have lead to low power and wearable sensors and sensor platforms, location tracking schemes, sensor network programming models, sensor data management, etc. (e.g., [7, 16, 17]). These efforts typically focus on challenging problems in applications such as seismological and habitat monitoring, inventory control, weapons and military personal tracking, and smart offices. The problems include how to maintain connectivity among a large number of similar sensors in an ad hoc network and how to aggregate low-level data collected by individual sensors into high-level information. SISARL differs from these applications in many ways. For example, SISARL sensors are diverse in both functionality and complexity; they are embedded in appliances that are relatively loosely coupled; the underlying interconnection need not be ad hoc; and so on. One expects that effective and affordable SISARL appliances and services can be built without the breakthroughs sought by sensor network research.

SISARL can benefit from technology advances in embedded and real-time systems [18-20]. Typical assumptions made by efforts in these areas include that the system is closed or semi-closed and that the user can afford a relatively high initial cost and expenses in maintenance and upgrade. These assumptions are not valid for SISARL. Consequently, many existing solutions for embedded and real-time systems cannot be applied directly to SISARL.

Many challenging problems in building high quality, easy-to-use and maintained SISARL at affordable price have not been adequately addressed by research communities. As an example, a majority of SISARL appliances are mission critical; some are safety critical. Similar to other mission critical systems, SISARL devices and appliances should test themselves. While low-level techniques (e.g., hardware built-in-self-tests and driver and application verifiers) are available, system-level diagnostic and repair methods are not. To be effective, many SISARL appliances and devices need to be integrated seamlessly into a ubiquitous system. Life cycles of some components may be as long as ten to twenty years, but the supporting infrastructures may evolve much more rapidly. There must be no glitch in services to a user as SISARL components and infrastructure evolve independently. Maintenance and upgrade have not been a popular research topic, even in system and software engineering. Yet, without effective upgrade and maintenance strategies, SISARL will not “just work”. These issues are beyond the scope of this paper and will be in discussed in an accompanied white paper [21].

3. Motivating Examples

To motivate the proposed references model and provide supporting rationale, we consider first possible designs of a medicine dispenser. We then discuss three other SISARL devices (heart

3

DRAFT

beat monitor, robotic helper and smart pantry) and highlight the similarities and differences of their architectures.

3.1 Medicine Dispensers

There are already medicine dispensers on the market. For example, http://epill.com offers many choices of dispensers at a price from less than 50 to a few hundred US dollars. They range from simple 7-day organizer pillboxes with timer that remind the user at times when medications should be taken, to automatic dispensers that not only generate reminders for the user but also can check for user compliance to medication schedule and notify a caretaker when some medication is not taken according to the schedule.

In addition to their relative high price, these dispensers can be improved in many ways. For example, the existing devices are not designed with usability, flexibility and upgrade in mind. A user of a 7-day organizer pillbox must get more boxes and set them independently when the number of medications exceeds the capacity of a box. This increases handling error, while such a device should be designed to minimize the chance for error. Indeed, prevention of human errors in setting of medication schedule and handling of the medications is the most important aspect of usability. Dispensers should be easy to upgrade, but the simple pillboxes available today are not: As a user of a pillbox grows older, an automated dispenser may become necessary. At that time, existing pillboxes become useless. Forcing the user to switch to a completely new device is not desirable for reasons of cost and emotional comfort.

Modular Design

Figure 1 shows a possible block diagram of a dispenser designed to improve upon existing ones in these aspects. In this and subsequent figures, filled block arrows represent relatively wideband data and control flow between components. The dotted lines and arrows represent inputs and outputs to and from the appliance or interaction among components. The inputs to a dispenser may include program to control the timed actions of the dispenser and queries for medication history, while outputs may include notifications on compliance.

Depending on the functions it supports, a dispenser may contain only a subset of the components shown in the figure. To enhance flexibility, medicine containers together with id tags, sensors, timers and dispenser control may be separate from a programmable base. These parts can be used as a simple pillbox with timer and alarms. The pillbox component can be connected to a base unit or a computer and become a part of an automated dispenser.

Ideally, each medication container is identified by the name (or id) of the medication it holds, and the name is input by the pharmacist when the medication is filled. Similarly, rather than relying on the user or a caretaker to program the dispenser according to pharmacist’s instructions, one envisions that a pharmacist can input the medication calendar from his/her computer, verify the correctness of the instructions, check for drug interactions, etc. while filling a prescription, using an easy-to-use authoring tool. The tool translates medication calendar and associated instructions into a program for the dispenser (i.e., set the timer actions, reminders and other instructions). The program is downloaded to the dispenser either through a network, as

4

DRAFT

shown in the figure, or via a removable storage that goes with the containers.3 Indeed, such a dispenser can serve as a point-of-care device in an integrated system that interconnects physicians, pharmacists, caretakers and users.

Medication containers

Programmable base

Sensor bank, timers

and alarms

Figure 1 Block Diagram of a Medicine Dispenser

Sensor dataprocessing

unit

Programmable medication

calendar and timed actions

Compliance Checker

Medication record

Comm.unit

A high-end dispenser can monitor the time and quantity when each medication is taken each time and produces and records a medication history. It may have a compliance checker that compares the history with the programmed medication schedule. When it detects instances of non-compliance (i.e., the user did not take some medication according to the programmed schedule), the checker may send a notification or alarm to a designated caretaker automatically for each instance, or merely records the information on each such instance, or sends notifications to a caretaker after a specified number of instances, and so on. Clearly, the appropriate actions depend on the medication as well as the user. The dispenser should be designed so that programming it to act accordingly is easy.

Alternative Designs and Configurations

Conceptually, we can view the pillbox part (i.e., the medicine containers, together with timer, alarms and dispensing control) of a dispenser as a plant. The plant is programmed to carry out (or allow the user to carry out) timed sequences of actions. The compliance checker and the programmable medication calendar are in essence parts of a monitor-controller. Thus far, the assumption is that the command sequences (i.e., the timed action sequences of the dispenser) are computed a priori, at the time when a prescription is filled. In other words, the controller is open loop because the commands do not take into account of the compliances record.

An alternative is to structure the calendar and compliance checker as a feedback controller. The controller computes the medication calendar on-line, based on usage protocols of the

3 An option is to have drug store own the programmable base, which a user can take home by paying a small safe deposit. For liability reason, drug stores may prefer to own and maintain the dispensers.

5

DRAFT

medications, as well as compliance record and other data on the user’s condition and past medication history. Indeed, the proposed reference model views the dispenser this way and allows both the open and closed loop options. We will return to this discussion in Section 4.

Like existing automated dispensers, the dispenser in Figure 1 has an integrated base unit. A user with a computer should be able to use the computer instead. In that case, functions supported by the base unit are provided by a monitor and control application on a commodity platform. Even a naïve user should be able to easily switch from one configuration to another.

A question is then how the pillbox component should interface with the monitor and controller: Should the sensor data processing unit be part of the monitor and control or part of the pillbox? Should the medication calendar and actuators be parts of the pillbox? Choices of these interfaces in turn impact the design choices in how the pillbox is to be connected to the computer (e.g., as a USB device or a wireless device). More importantly, flexibility and usability, as well as cost, of the dispenser depend on these architectural choices. The proposed reference model intends to stimulate discussions on the choices.

Appendix A will discuss advantages of using common house devices, such as cell, Personal Handy-phone System (PHS) and portable phones, as user interface to multiple SISARL devices. For example, a medicine dispenser can use phones to deliver timer alarm to the user, rather than relying on an alarm built in the dispenser unit. Using a phone, a base unit can serve multiple dispensers, e.g., one at home, one in office and one during travel. Then it is a matter of providing extensible interfaces between the base unit and dispensers, alarms, and compliance monitors.

3.2 Heart Beat Monitor

Heart beat monitors are familiar to athletes as well as heart disease patients. The ones used by athletes simply monitor and display heart beat rate. The ones used to detect irregular and missed heartbeats may record the user’s heartbeats over a long period. (For example, Holter monitor is worn externally for 24 hours.) Figure 2 shows a block diagram of a smart heart beat monitor, which not only can record heart beat history, but also can process heart beat signals and patterns in real-time and notify a caretaker when anomalies are detected.

Comparing the block diagrams in Figures 1 and 2, we note that while the type and number of sensors in the two devices differ and specific functions provided by them differ, the devices have many similarities architecturally. Similar to the automatic medicine dispenser shown in Figure 1, a monitor represented by the diagram in Figure 2 is also self-contained: It does not rely on a computer or other device to provide storage and computation capabilities. A self-contained medicine dispenser is a reasonable design, because it need not be mobile. In contrast, a heart beat monitor is worn and may be mobile while in use. A self-contained configuration may not be the best design: It is likely to be large, heavy and bulky, has limited storage and processing capacity or consumes large power. An alternative architecture is shown in Figure 3. The components to the left of the communication unit are worn and mobile; the components to the right are provided either by an application on a computer or by a stationary base unit.

6

DRAFT

Figure 2 A Configuration of a Heart Beat Monitor

Heart-beat

sensorSensor data processor

record

Pattern recognizer

Comm. Unit

Figure 3 An Alternative Configuration of Heart Beat Monitor

Heart-beat

sensorSensor data processor

Record

Pattern recognizerComm.

UnitComm.

Unit

\

3.3 Other Sample SISARL Appliances

While medication dispenser and heart beat monitor has many similarities, automated pantry and robotic helper appear to have no commonality with them. Nevertheless, as it will become evident, they also fit, to a great extent, the same architectural framework.

Automated Pantry

A related line of products and services for active retirees and assisted living is automated smart pantry, for storage and inventory monitoring of bulky and heavy items such detergent, shampoo, etc. Even people who enjoy grocery shopping are not likely to find pleasure in shopping for these items. Like an ordinary pantry, a smart pantry provides compartments, each sufficiently large to hold one or two items of some kind. The pantry may have a front door in the house for user access to the compartments, and a back door to the outside for semi-automatic supply delivery.

For ease of use, compartments are initialized individually. An un-initialized compartment is considered to be full and therefore is ignored by the system. To initialize a compartment, the user enters the name of the item and the number of units to be placed there, e.g., a bottle of Johnson’s baby shampoo. (Ideally, the identifier of the item is scanned or read by an RFID reader.) In addition, one also enters the urgency of replenishment: the length of wait tolerable or as soon as possible. This information is stored along with contact and order information on the suppliers.

7

DRAFT

Figure 4 shows a possible configuration of such a pantry. A sensor in each compartment can detect whether the compartment is empty. The appliance has a degenerate sensor data processor: It merely decodes the locations of empty compartments and looks up their contents. When the sensor detects that a (initialized) compartment used to hold an item is empty, a message is send via a home computer or messaging device to a designated supplier in order to place an order for the item. Based on the urgency of the item, each order has a deadline, which is the date by which the ordered item is to be delivered. The supplier can decide whether to batch the deliveries of individual items based on this information.

Figure 4 Block Diagram of a Smart Pantry

Comm.unit

Initialized Sensors

Sensor location decoder

Pantry content and supplier

DB

Replenishment order

generator

An alternative is to use RFID technology, instead of an on/off sensor per compartment: A smart pantry is just an inventory-controlled storage area. Items are tagged with their ids and an RFID reader is use to keep track the items as they are moved in and out of the area. This alternative is likely to be more advantageous in terms of cost in the future, since tags and reader are likely to cost less and less in the future.

While an automated pantry is a luxury for healthy and active retirees, it can also be used as an assisted living and home-care device. It allows a caretaker to monitor the activities of a user, e.g., nutrition intake, etc. This capability, however, introduces considerable complexity, as well as the required dependability of pantries. Similarly, a low-cost pantry capable of monitoring expiration dates of individual items remains infeasible until low-cost RFID tags for supply chain applications become available and widely deployed to tag individual items. Such tags are not yet available [22].

Robotic Helpers

Robotic helpers for simple household chores (e.g., vacuum cleaning) are already available on the market. One expects that their cost will decline as their utility and popularity increase. In contrast, robotic helpers for enhancing mobility of the weak and disabled individuals are by far the most complex and technologically challenging SISARL appliances. Completely autonomous robotic helpers for assisted living are likely to be challenging in many aspects. Not only robotic and automation technologies have not yet reached the maturity needed to produce autonomous

8

DRAFT

helpers with sufficiently high dependability and low cost. But also, many problems in user interfaces, ergonomics, etc. require satisfactory understanding and solutions.

A practical approach is to offer semi-automatic, robotic helpers that can operate remotely, either over long distances (1 mile or over) or within the same household. An example would be a helper that a family member (who may also be elderly) or caretaker can operate to move objects, remove clothing, aid a person to stand up and navigate about in the room, etc. Such a device makes it possible for two or more elderly people to take care of each other.

Figure 5 shows a possible architecture of robotic helper. The left half the figure shows components local to the helper, including legs and arms of the helper and their controller. The right half shows components of a remote control: An operator input the desired movement of the helper using the remote control.

Figure 5 Block Diagram of a Robotic Helper

Robotic control

Plant models & rule base

Local sensors

Robotplant

Local sensor data processor

Localcomm.

unit

Remote sensors

Remotesensor data

processorRemotecomm.

unit

ActuatorCommandgenerator

The assumption of this configuration is that the robotic helper itself is not power and size limited. The justification is that it must have sufficient power so that it can levitate objects and enhance user mobility. By comparison, the weight, power consumption, and size of the storage and computation units are small.

4. Reference Model

From Figures 1-5, one can see that despite significant differences in their functionalities and complexity, SISARL appliances and devices are similar architecturally in many ways. We want to exploit their similarities in the design of building blocks that can be configured and reused to build different types of appliances. The appliance reference model described here intends to support this exploitation.

4.1 Generic Base and Remote Units

According to the proposed reference model, an appliance is divided into two parts: a base unit and a remote unit. Every appliance has a base unit. It may or may not have a remote unit. The

9

DRAFT

parts are depicted in Figures 6 and 7, respectively. Their components are labeled in generic terms in these diagrams. The parts are connected to each other and to the external world via a network of some kind. – In both figures, the network and network interfaces are depicted as a cloud and boxes labeled NI (network interface). They are called the communication unit, collectively, in earlier figures.

Figure 6 A Generic Configuration of Appliance Base

Model base, rules & policies

Sensors

Plant Command generator

Sensor data

Processor

Monitor &controller

N

I

Figure 7 A Generic Configuration of Remote Unit

Sensors

RemotePlant

Command generator

Sensor data

Processor

NN

I

Specifically, each part is modeled as a plant together with a monitor and controller. Some inputs to the controller are from the local sensor data processor and database while some inputs are from the remote part.

Except for the names of the components, the based unit shown in Figure 6 is essentially the same as the portion of the robotic helper to the left of the communication units in Figure 5: A robotic helper is indeed a plant together with a feedback controller. In general, the base unit of a specific appliance may not have all the components in Figure 6. An example is the base part of

10

DRAFT

the heart beat monitor in Figure 3. It is the part to the right of the communication unit, and it has no plant, sensor and sensor data processor. Some components may take a degenerate form. An example is the sensor data processor in a smart pantry; it does little or nothing.

Table1 lists specific names and functions of the components in the base units of the appliances described in Section 3. Every appliance contains a communication unit, a set of sensors and a sensor data processor of some form. (The only exception is heart beat monitor with configuration depicted by Figure 3: The sensor data processor is in the remote unit; this point will be discussed shortly.) These components are omitted in the table. The column labeled “Command Generator” lists what the command generator does, and the column labeled “Monitoring” lists what are monitored. The table also indicates whether the appliance has a controller and, if it does, whether the control is open or closed loop.

Table 1 Appliance Specific Names and Functions of Base Unit Components

Plant Commands Generator

Monitoring Controller Type

Local DB Contents

MedicationDispenser (Figure 1)

Medicationcontainers

Sets alarms & enables dispensing

Medication schedule

compliance

Open loop Medication calendar and

compliance record

Heart beat monitor (Figure 3)

None Nothing Heart beat patterns

NA Heart beat record, abnormal patterns

Smart pantry Pantry compartments

Generates purchase

orders

States of compartments

Closed loop

Item ids, supplier ids, prices, etc.

Robotic helper

Robot arms and legs

Generates actuator

commands

Location and moments of

arms and legs

Closed loop

Plant model and control laws

Table 2 Appliance Specific Names and Functions of Remote Unit Components

Plant Sensor Data Processor

Command Generator

Controller Type

Heart beat monitor (Figure 3)

User Extracts heart beat patterns

NA NA

Robotic helper Remote Controller

Processes controller position

Generates feedback to operator

Closed loop

11

DRAFT

According to the configurations depicted in Figures 1 and 4, medicine dispensers and smart pantries do not have remote unit. Table 2 lists the components in the remote units of a heart beat monitor and robotic helper. The remote unit of the heart beat monitor in Figure 3 consists of components to the left of the communication unit. Heart beat signal is picked up by a sensor, digitized and processed here. This configuration assumes that the total energy required to process bits in the heart beat signal, generate transforms of signal segments and transmit the transforms is less than the energy required to transmit raw bits used to encode the time-domain signal. The component should be moved to the base unit if this assumption is not true for the network used to link the base and remote units.

4.2 Building Blocks and Their Reusability

Sensors and plants of different appliances share little or no commonality in most cases: This fact is evident from the sample appliances described in Section 3 and Appendix. Indeed, these components must be designed and fabricated for each kind of appliances. In contrast, it should be possible to implement most of the other major components (including sensor data processor, monitor and controller, database and communication unit) from configurable building blocks.

Sensor Data Processor

Sensor data processors of different appliances perform similar operations. They offer great potential of reuse and can be implemented from a few (e.g., two) configurable building blocks. Specifically, sensor data processors can be divided roughly into two kinds: encoder and digital signal processor (DSP).

Encoder An encoder digitizes and encodes one or more sensor data stream. It may also perform some kind coordinate transformation or correlation. Sensor data processors in medicine dispenser, smart pantry and robotic helper are encoders. As a specific example, let us consider the sensor data processor in the remote control unit of a robotic helper: Regardless the input device, the function of the sensor data processor is to encode the trajectory of the input device based on data picked up by sensors. It may also need to convert the input device trajectory to a reference (i.e., desired) trajectory that is to be tracked by the controller in the base unit. (An alternative is to do the conversion in the base unit.)

The complexity of the encoder in a robotic helper may differ significantly depending on the input device and hence the amount of sensor data. For example, suppose that the operator controls the robot legs and arms by joysticks. Then, the sensor data give the joystick positions and movement, and the processor is only required to digitize these sample data for transmission. On the other hand, a future robotic appliance may provide a wearable remote controller that allows an operator to perform movements (e.g., to put on a coat) to be imitated by the robot arms. In this case, the sensor data processor will need to perform additional processing on the data to generate the reference trajectory. The sensor data processor in the base unit of a complex robotic helper may need to filter sensor noise, estimate plant state, etc. In these cases, a full fledged DSP may be required.

12

DRAFT

Digital Signal Processor A DSP not only encodes each data stream but also computes some kind(s) of transform on the signal. Sensor signals are typically pseudo-periodic. An example is the signal picked up by a heart beat sensor. Many schemes for extracting patterns of interest from such signals work with transform-domain representation of the signal, rather than time-domain representation. For example, to extract abnormal patterns in the signal picked up by a heart beat sensor, a monitor with real-time analysis capability may first compute some kind of transform on signal segments and compare the transforms with training templates (i.e., transforms of training signals that are known to exhibit abnormal heart beat patterns). The operations performed by the sensor data processor in a heart beat monitor are the same regardless whether the processor and the pattern recognizer are in the same unit (as shown in Figure 2) or in separate units (as shown in Figure 3).

Similarly, the sensor data processor in an acoustic fall detector, which will be discussed in the appendix, may work with frequency spectral densities of acoustic signal segments. The processor can also be implemented with a configurable DSP. Indeed, the work done by the DSP of these appliances is analogous to data compression but has complexity significantly lower than that of a MPEG3 processor.

Monitor and Controller

Except for the heart beat monitor, the key component of other appliances described in Section 3 is a controller. Controllers in different appliances may be divided into types along three dimensions: control structure, complexity and dynamics. They differ in number of the data paths, control laws and sampling rates. Different control laws generate different processing loads. Nevertheless, they can be built from a small number of building blocks.

Control Structure Table 1 divides the controllers in different appliances into two types: open loop and closed loop. The inputs to a controller include data from the local sensor(s) and data from the communication unit. (Figure 6 shows that the command generator also has input from the local database. This input is temporarily ignored here and will be discussed shortly.) Sensor data give an estimate of the current state of the plant. Data from the communication unit gives the desired state of the plant. (This input is typically called the reference input.)

The command generator of an open-loop controller uses only the reference input in its computation of the commands to be carried by the plant. The command generator in a feedback (closed-loop) controller computes commands based on both the reference input and state estimate. Table 1 shows that the medicine dispenser in Figure 1 has an open loop controller while both the robotic helper and smart pantry have closed loop controllers. A more intelligent dispenser may use a closed loop controller that updates the medication calendar on-line base on both the desired medication schedule (i.e., the reference input) and user compliance record (i.e., the current state of the plant).

Complexity The controllers in some appliances are multivariate: A multivariate controller has multiple inputs and outputs and multiple control loops at different rates. For example, the controller of a robotic helper may have three control loops for each leg or arm joint, one for each direction of joint movements. Similarly, the controller of a smart pantry may have a control loop

13

DRAFT

for each compartment or a few compartments, allowing the states of the compartments be polled at different frequencies. Alternatively, a pantry may use a simple controller that has only one loop: The controller simply polls the states of all compartments each time.

In addition to having multiple inputs and outputs, controllers of complex appliances (again, such as some robots) may need one or more modules that update the plant model, filter sensor data, estimate current plant state, and so on. Except for robotic helpers, the other appliances discussed here do not need these components.

Dynamics The required response times (i.e., their open or closed loop bandwidths) of controllers in different appliances are likely to differ significantly. Consequently, the frequencies at which they sample input data and compute actuator commands differ widely. As an example, the required closed-loop bandwidth of the feedback loops in the controller of a robot helper is likely to be in the order of kilohertz. Consequently, sensor data and reference input data are read and control laws are computed a few hundred or thousand times a second. At the other extreme, the controllers of smart pantries and medicine dispensers are required to read their inputs and compute the actuator commands at most once a few seconds, minutes or hours.

Local Database

The term database is used loosely to mean a component to hold data, codes, rules, policies, etc. It should be possible to use a single modular design for many types of appliances (e.g., for all appliances discussed in Section 3 and Appendix.)

As the previous section pointed out, the database in the base unit is used for many purposes. Firstly, it is a repository of data on the user: heart beat record, medication and compliance record, pantry item preferences and replenishment times, and so on.

Secondly, the database stores rules and policies governing the behavior and usage of the appliance: Examples are rules governing issuance of abnormal heart beat notifications, dispatching pantry supply orders, and tolerable false fall alarm rate. By choosing appropriate rules and policies, the user can customize the appliances to a great extent.

Lastly, by storing configuration files, alternate code, and plant models in the database, the designer of an appliance makes it configurable. Indeed, a fast and physically small memory with a sufficiently large capacity makes it possible to extend the appliance in many ways.

Communication Unit (Network Protocol Stack and Interface)

The bandwidth and latency demands of all the SISARL appliances described here are modest and can be met using existing standards, including Bluetooth, Zigbee, and WiFi. The design of the communication unit is, to a great extent, dedicated by the chosen network and protocol standards. It follows straightforwardly that the communication units of all SISARL appliances can be built from a common set of basic building blocks, just like network stacks in modern computers and handheld devices.

Issues to be addressed include configurability and extensibility, especially regarding how an appliance accesses the external world. To explain, take the choices between dial-up and Internet

14

DRAFT

links between a smart pantry and grocery suppliers, for example. A pantry in a household without Internet access should be able to deal up a supplier and place an order by fax or voice message. Later, the user should be able to configure the pantry to use the Internet when the household has internet access. This requirement presents no challenge: the pantry comes with all the modem and network cards and protocol stacks, like modern computers and laptops. In the case of smart pantry, the arguments against multiple interconnect choices is cost: the cost of modem and network cards and software. For appliances such as heart beat monitor, the additional size, weight and power due to duplicate, unused hardware and software make this approach even more unattractive.

The above discussion assumes that each appliance communicates with the outside world independently other appliances. A better alternative is to have a SISARL appliance serve as a gateway for multiple appliances within a household. The gateway can be a smart cell or wired phone, a messaging device or a computer. Each appliance communicates with the gateway via a home network.

5. Summary

Acknowledgements

The author thanks Daniel Burkhardt, Tei-Wei Kuo, Mark Liao, Kwei-Jay Lin, Adrian Oney and Daniel Shih for their comments and suggestions on contents and presentation of the paper.

References

[1] Cerf and Kaln, ISO-OSI reference model.

[2] J. Nabielsky and A. P. Skelton, “Virtual terminal management model,” RFC 782, http://www.faqs.org/ftp/rfc/, January 1981.

[3] J. Postal and J. Reynolds, “Telnet protocol specification,” RFC 864, http://www.faqs.org/ftp/rfc/pdf, May 1983.

[4] J. Postal and J. Reynolds, “File transfer protocol, RFC 959,” Network Working Group, http://www.w3.org/protocols/rfc959/, October 1985.

[5] D. Voorhies, D. Kirk, and O. Lathrop, “Virtual Graphics,” ACM Computer Graphics, Volume 22, No. 4, August 1988.

[6] IEEE/ISO Standards 11073, Point of Care – Medical Device Communication.

[7] E. Dishman, “Inventing Wellness Systems for Aging in Place,” IEEE Computer, May 2004 and Age-in-place, http://www.intel.com/research/prohealth/, Intel Corporation

[8] A. Pentland, “Healthwear: Medical Technology Become Wearable,” IEEE Computer, May 2004

[9 Changing Places Consortium, http://architecture.mit.edu/house_n/, MIT

[10] Aware home, http://www.cc.gatech.edu.fce/ahri/, Georgia Tech

[11] Center for Future Health, http://www.futurehealth.rochester.edu/, University of Rochester

15

DRAFT

[12] Marc smart home, http://marc.med.virginia.edu/projects_smarthomemonitor.html, University of Virginia.

[13] Assisted Cognition Projects, http://www.cs.washington.edu/assistcog/, University of Washington.

[14] FCE Smart House Research Survey, http://www.cc.gatech.edu/fce/seminar/fa98-info/smart_homes.html

[15] D. A. Ross, “Cyber Crumbs for Successful Aging with Vision Loss,”, IEEE Pervasive Computing, April 2004.

[16] I. F. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci, “A Survey on Sensor Networks,” IEEE Communication Magazine, August 2002.

[17] D. Culler, D. Estrin, and M. Srivastava, “Overview of Sensor Networks” and other articles on sensor networks and sensor network applicaitions, IEEE Computer, August 2004; Also, http://www.cs.virginia.edu/~th7c/links.htm, links to websites on sensor networks and related research,

[18] L. Abeni and G. Buttazzo, “Resource Reservation in Dynamic Real-Time Systems”, Journal of Real-Time Systems, July 2004.

[19] Proceedings of IEEE RTSS and RTAS in years 1987 - 2004

[20] J. W. S. Liu, Real-Time Systems, Prentice Hall, 2000.

[21] J. W. S. Liu, “Challenging problems in SISARL,” white paper in preparation.

[22] R. Glidden, et al, “Design of Ultra-Low-Cost UHF RFID Tags for Supply Chain Applications,” IEEE Communications Magazine, August 2004.

Appendix First-Generation SISARL Appliances and Services

This appendix, together with Section 3, provides a partial list of first generation SISARL appliances and devices. Some are already available in specialty stores and over the web. Others can be productized today; many can be built from off-the-shelf building blocks and platforms. None rely on beyond the state-of-art technologies and infrastructures. Many important issues (including usability, modularity, extensibility and cost) have not been adequately addressed in the process of bringing them to all the people who may enjoy or need them. The list intends to present sample straw men for the purpose of stimulating and focusing discussions on these issues. Before doing so, the appendix first discusses briefly common assumptions and rationales.

A.1 Rationales and Assumptions

Two kinds of assumptions and rationales underlie all the appliances presented here. One is concerned with users, while the other is concerned with the required technologies and infrastructures.

An observation is that the time for first generation SISARL is now as an increasingly larger number of baby boomers begin to retire or will retire within this decade. Most of them are still in good health and live an active life style; some still work. SISARL is not essential yet for them.

16

DRAFT

They may use some appliances for reasons of convenience, safety and health maintenance; they may acquire and enjoy health and safety related products, much like one acquires and enjoys fancy portable and cell phones. Many first-generation SISARL products discussed here are targeted for this segment of the population. As an active retiree ages and relies on some appliances more or more heavily for assistance in daily activities, the appliances will become more and more critical and evolve into assisted living and home care devices. With improving health and increasing life expectancy of retirees, this transition may take ten to twenty years.

Whenever possible, the first generation SISARL appliances should make use of existing end-user devices. There are many advantages. Take cell or portable phone for example. A phone already provides keyboard, display, and audio capabilities. Although revolutionary changes in these devices are not likely in the near future, one can count on continued improvements in their processing and storage capabilities and audio and display qualities. Adding several SISARL functions to the devices is feasible and is a way to reduce the cost of SISARL. More importantly, the user will not have to carry multiple gadgets and learn to use them. From usability standpoint, the fewer distinct user interfaces, the better.

It is reasonable to suppose that the effective lifetime of an appliance ranges from three to ten years. Bandwidth requirements of first-generation appliances are relatively small. Take heart beat (and vital sign) monitor for example. The bit rate for telephone quality voice is sufficient for encoding of the heart beat sensor data, as well as data collected by other vital sign sensors. None of the appliances described here make use of video. Many competing wireless standards (e.g., Bluetooth and Zigbee) will work for interconnection within a household. There will be an increasing number of acceptable choices with different tradeoffs in the future. It is important for the current designs to be open for adoption of new choices.

In contrast, information technological infrastructures, social services, personnel care culture, and health care and delivery practices will not change significantly in the next 3-5 years. In particular, a significant percentage of households will not have computers and Internet access during this period of time. Only relatively affluent households will have broadband Internet connections. For this reason, first-generation appliances should require only dial-up connections to the outside world. At the same time, they must be extensible so that users with computers, Internet and broadband linkage can benefit from these facilities now. Extensibility is also essential to ensure smooth, glitch-free transition to the next generation systems.

A.2 Sample First-Generation Appliances

Section 3 has already discussed some of the first generation appliances. They are medicine dispenser, heart beat monitor and smart pantry. Additional examples are object location device, multi-function phones, fall detector and accident alarm, time and smart memory enhancer, etc.

Object Location

A typical object location device, such as those offered by specialty stores, contains a few (e.g., 4) locators together with a key-box size interrogator. Each locator is a beeper. It can be attached to an object, such as a key or a reading glass. The interrogator has a button for each beeper. By

17

DRAFT

pressing a button and causing the corresponding beeper to sound out, a user can locate the attached object within 20-30 feet. Almost everyone has the experience of misplacing items big and small, from time to time, and can surely appreciate how useful such a simple device can be. For an individual who has become increasingly more forgetful, a good object location device may become increasingly more essential.

Both cost and usability of current location devices need significant improvement, however. 40-50 US dollars for a package of 4 is far too expensive. From usability point of view, the most serious shortcoming is too few locators for too few items. An interrogator should be able to handle a larger number of locators. (Some number between 64 to128 should be sufficient for most users.) A user should be able to query for an object like one looks up and dials an entry in the phone book of a phone. Bulky interrogators with one key per locator are not acceptable.

Active Locator Design Existing location devices use active (i.e., powered) locators. Any low power transponders can be used for this purpose. When triggered by a query from the interrogator, a transponder emits a signal. By tracking the signal, the interrogator can lead the user to the object attached to the transponder.

One can get an acceptably large capacity by extending the existing active-locator design in straightforward ways. However, because locators are active, extensibility of the device is limited by locator complexity and battery life. For example, the 4-object location device offered by Sharper Image can be extended to handle 2n locators by giving each locator an n-bit binary id encoded by an n-bit sequence of pulses of a low-power signal. The energy consumption of each locator increases (and consequently, battery life decreases) exponentially with the number of locators. This is obviously a serious disadvantage.

RFID transponders and active RFID tags (such as what ActiveWave, Inc. offers) are frequently mentioned alternatives. Unfortunately, the cost of active RFID tags and readers are still high, and they offer no advantage in terms of battery life. Indeed, all choices of active locators are far from ideal because of the need for periodic battery or locator replacements4.

Passive Locator Design RFID tags can be used as passive locators (i.e., locators that are not powered). A location device with passive locators uses multiple very short-range RFID readers/transponders to cover the monitor area. The user initializes each object that will be queried and located by attaching an RFID tag to the object and having the id read and recorded by a reader in the vicinity of the object at the time. Subsequently, whenever the object is moved, reader(s) in the vicinity tracks and updates its location. Queries for the object are responded by the reader(s)/transponder(s) in its vicinity: The signals they emit in response to a query allow the interrogator (and the user) to pinpoint the neighborhood where the object is.

Because there are fewer readers/transponders than id tags (locators) and readers are at fixed locations, this design is more advantageous from maintenance standpoint. The cost of RFID readers/transponders determines the viability of this design. Several questions, regarding optimal range and placements of the readers, etc., need to be answered to complete this design, but these questions can be answered readily if the cost of reader/transponder is not an issue.

4 Solar cell battery is not an option since an object may not be exposed to light at the time when it is queried.

18

DRAFT

Interrogator Design Ideally, some existing household device (e.g., a cell or portable phone or a computer) should be used as an interrogator. Combining object location interrogator with cell or portable phone, or making it phone-like, has many advantages: The user does not need to learn a new interface. This is especially important for elderly since learning how to use new gadgets becomes increasingly more difficult with age. Many portable and cell phones have a small display. The display can be used to provide visual output (e.g., a blinking curser) for users who have deteriorated hearing or live in a house too large for beepers to be effective5.

Multi-Function Cell and Portable Phones

As stated above, cell and portable phones can be made into ideal multi-function end-user devices. Current multi-function phones include cell phones that contain digital cameras. NTT (Japan) has prototype cell phones that provide controls to window shades, pet feeders, and so on. Phones that support both cellular and Handy-phone System (PHS) communications are also available. It will be straightforward to extend them further to support SISARL functions and low power, short range communication within appliances and within a household.

Phones that support multiple SISARL functions will differ from today’s multi-function phones in an important way, however. It is true that a cell phone can perform many tasks: dial a number, lookup the phone book, take a picture, turn down a window shade, etc. However, it can perform only one task at a time. (The user cannot look up the phone book and turn down a window shade while talking, for example.) Moreover, the user has complete control over which task the phone performs at any time. A phone that supports some of the SISARL functions must support multi-tasking. As an example, suppose that a medicine dispenser uses a cell phone for the transmission of an urgent compliance alarm to a caretaker. The phone must be able to interrupt a long conversation of the user, much like a fire alarm system can grab the line to dial the fire department when fire or smoke is detected. This capability may not be sufficient if the phone is also used to collect the user’s heart beat and vital sign patterns. Depending on the functions it is used to support, a phone will need a multi-task platform.

Fall Detector (Accident Alarm)

Anyone, young or old, who lives alone, can appreciate a device that can phone a relative or friend in case of an accidental fall. Both commercial versions and research prototypes of such device exist. Most commercial fall detectors are some form of wearable alarm; the alarm provides a push button for hailing a monitoring service or caretaker, who in turn calls for emergency service as needed. The majority of research prototypes rely on video and machine vision: The device processed live or recorded video to detect abnormal and sudden movements such as a fall. The former is cumbersome. It requires active participation of the wearer, who may not be able to push the button after a serious fall. Video and vision based systems are not only costly; they are also overly intrusive. Both kinds may be tolerable for individuals in assisted

5 At the first glance, a usability problem with visual display is the generation of a digital floor plan of the household. With a floor plan, the user can tell where the object locator or reader/transponder is blinking; without it, a blinking curser is not helpful. A second thought will point out there are very simple ways to initialize the interrogator and generate a simple floor plan in the process.

19

DRAFT

living or home care environments, but not for healthy and active retirees, for whom such alarm, while useful for peace of mind, is not yet essential.

An alternative is to use low-cost motion (acoustic and vibration) sensors, such as those proposed for smart homes and offices or those used for burglar alarms. By processing inputs from multiple sensors, a detector can determine whether an occupant has fallen.

As an example, sub-sonic detector used is burglar alarms is a possibility. A problem with such device is the reliability of detection: how to keep false alarm rate acceptably low. (Add use of smart templates and temporal variation information and can use the same components as vital sign monitors.) Smart Memory Enhancer

20