SAB ReviewFebruary 2004 Pervasive 2004April 2004
Using an Extended Episodic Memory Within a Mobile Companion
Using an Extended Episodic Memory Within a Mobile Companion
Alexander Kröner, Stephan Baldes, Anthony Jameson, and Mathias Bauer
German Research Center for Artificial Intelligence (DFKI) GmbHStuhlsatzenhausweg 3, 66123 Saarbrücken, Germany
BMB+F 524-40001-01 IW C03 (SPECTER)
SAB ReviewFebruary 2004 Pervasive 2004April 2004
About this TalkAbout this Talk
• About Specter– Goals, example, challenges
• About the personal journal– User interface
• Adding value to the journal– Binding services to journal entries
SAB ReviewFebruary 2004 Pervasive 2004April 2004
What is Specter?What is Specter?
• Specter is about…– … context- and affect-aware personal assistance– … in instrumented environments– … using a long-term memory
• Major issues– Extension of perception– Learning about behavior and affect– Augmentation of decision making and effecting– Reflection and introspection
SAB ReviewFebruary 2004 Pervasive 2004April 2004
Example Scenario: Shopping AssistantExample Scenario: Shopping Assistant
• Technology– RFID enabled shelf and
basket– RFID-labeled Products– Integrates software of
projects Specter and REAL (Saarland University)
• Logging low-level actions– Product in shelf, removed
from shelf, etc.
• Applying logged information– Proposing alternative
products
SAB ReviewFebruary 2004 Pervasive 2004April 2004
Obtaining Data from the EnvironmentObtaining Data from the Environment
Stream of raw, low-level sensor data
UM
PersonalJournal
decisio
n supp
orte
pisode
retrieval
situationclassification
motion dataclassification
phys. dataclassification
planrecognition
objectclassification
locationclassification
actionrecognition
affectrecognition
motion profiling
abst
ract
ion
of d
ata
...
... ...
... ...
reflection/introspection
SAB ReviewFebruary 2004 Pervasive 2004April 2004
Obstacles in Obtaining DataObstacles in Obtaining Data
• Automatically retrieved input– Sensor input may be incorrect – Sensor input may be completely missing– Inference mechanisms are not perfect
• user input required
• Manual (user) input– Lack of motivation
• manual input has to be minimized• input value has to be maximized
– Recorded information may be sensible• user's trust has to be strengthened
SAB ReviewFebruary 2004 Pervasive 2004April 2004
Design PrinciplesDesign Principles
• Input has to be acquired collaboratively[addresses: manual input, lack of motivation, trust]
– filling the journal automatically– asking for additional information if required– providing control of information and bound services
• Provision of diverse, multiple benefits[addresses: maximizing value]
– providing the option of binding various services to journal content
• Support for diverse forms of collaboration[addresses: lack of motivation]
– providing various ways of entering data– enabling input/feedback in varying scenarios
SAB ReviewFebruary 2004 Pervasive 2004April 2004
Accessing the Personal JournalAccessing the Personal Journal
The personal journal user interface
SAB ReviewFebruary 2004 Pervasive 2004April 2004
Browser/Viewer ApproachBrowser/Viewer Approach
Browser• Goal: browsing the journal in a similar fashion like the Web
• Adopted concept of Web browser
• Browsed documents are generated on-the-fly by data viewers
• Browser and/or viewers can be exchanged
Viewers
SAB ReviewFebruary 2004 Pervasive 2004April 2004
Navigating the Personal JournalNavigating the Personal Journal
• Content requests– History (forward,
backward,…) – Bookmarks – Hyperlinks
• Reminder Points– Created by the user
during interaction with the instrumented environment
– Indicate the need to inspect points within the timeline during introspection
SAB ReviewFebruary 2004 Pervasive 2004April 2004
Adding ContentAdding Content
• Annotating entries– Free text comment,
references, categories, ratings
• Grouping entries– A group indicates that
some entries are related together
– For manual organization, and input for Specter
SAB ReviewFebruary 2004 Pervasive 2004April 2004
Collaborative Learning of the User ModelCollaborative Learning of the User Model
Binding services to the personal journal
SAB ReviewFebruary 2004 Pervasive 2004April 2004
Example: Assisting an EC Card PurchaseExample: Assisting an EC Card Purchase
• Goal: checking bank account automatically before payment is required
• Binding a service– Identifying situation where check is required
• Identifying relevant conditions, e.g., shop size, total prize of wares contained in basket
– Binding a service “account check” to that situation• To be stored as trigger in the user model
• Evoking a service– Comparing given situations with triggers from the
user model
SAB ReviewFebruary 2004 Pervasive 2004April 2004
Binding Services: WorkflowBinding Services: Workflow
User or Specter chooses a servicethat should be triggered automatically
Specter collects attributes from (potentially) relevant journal entries
User discards irrelevant attributesand/or adds relevant ones
Specter computes a rule and shows it to the user
User accepts or changes the rule
Specter adds the new rule as service trigger to the user model
SAB ReviewFebruary 2004 Pervasive 2004April 2004
Defining Decision TreesDefining Decision Trees
EC CardPurchase
Store PriceEdeka Hela
117,5€ > 117,5€
Price
– +
DayOfWeek
– +
TimeOfDay
– +
TimeOfDay
– +
So Sa...
112€ > 112€
5:25pm > 5:25pm 1:07pm > 1:07pm
SAB ReviewFebruary 2004 Pervasive 2004April 2004
Customizing Decision TreesCustomizing Decision Trees
EC CardPurchase
Store
Price DayOfWeek
Edeka Hela
117,5€ > 117,5€ So Sa...
userdef #1
TimeOfDay
otherwise[12am,2pm]
+ –
TimeOfDay
otherwise[12am,2pm]
+ –
TimeOfDay
otherwise[12am,2pm]
+ –
TimeOfDay
otherwise[12am,2pm]
+ –
SAB ReviewFebruary 2004 Pervasive 2004April 2004
Creating a Service TriggerCreating a Service Trigger
SAB ReviewFebruary 2004 Pervasive 2004April 2004
Conclusion and OutlookConclusion and Outlook
• Specter– Overview– Personal journal user interface– Binding services to journal content
• Next steps– Journal user interface
• Queries• Setting up plans• Additional viewers
– Identifying additional services
Top Related