ATOMate It! - People | MIT CSAILpeople.csail.mit.edu/emax/talks/atomate-talk-slides... · 2010. 10....

ATOMate It! End-user Context-Sensitive Automation using Heterogeneous Information Sources on the Web

Max Van Kleek, Brennan Moore, David KargerMIT CSAIL { emax, zamiang, karger } @ csail.mit.edu

Max Van Kleek, Paul André, mc schraefelenAKTing ECS, University of Southampton{ emax, pa2, mc+www } @ ecs.soton.ac.uk

Friday, May 7, 2010

ATOMate It! End-user Context-Sensitive Automation using Heterogeneous Information Sources on the WebHow to regain some of the time that Web 2.0 took away

Max Van Kleek, Brennan Moore, David KargerMIT CSAIL { emax, zamiang, karger } @ csail.mit.edu

Max Van Kleek, Paul André, mc schraefelenAKTing ECS, University of Southampton{ emax, pa2, mc+www } @ ecs.soton.ac.uk

Friday, May 7, 2010

1. motivation - can we beat overload by delegating to machines?

2. atomate - reactive automation driven by web feeds

3. study

4. related work

5. next steps

Friday, May 7, 2010

1. motivation

Friday, May 7, 2010

"...in an information-rich world, the wealth of information

means a dearth of something else: a scarcity of [ ... ] the

attention of its recipients. Hence a wealth of information

creates a poverty of attention and a need to allocate that

attention efficiently among the overabundance of

information sources that might consume it"

(Simon 1971, p. 40-41)

Friday, May 7, 2010

In 2010:

Friday, May 7, 2010

In 2010:

peer/friend/citizen-produced content

Friday, May 7, 2010

In 2010:

peer/friend/citizen-produced contentsocially and algorithmically recommended

Friday, May 7, 2010

In 2010:

more interesting content produced every day than we could possibly consume

Friday, May 7, 2010

In 2010:

‘social maintenance’ - updating friends

Friday, May 7, 2010

In 2010:

keeping on top of the world

Friday, May 7, 2010

In 2010:

keeping on top of the world

responding to others’ needs

Friday, May 7, 2010

In 2010:

keeping on top of the worldmore responsibilities --professional, personal, socialthan we could possibly perform

responding to others’ needs

Friday, May 7, 2010

P. Maes,“Agents the Reduce Work and Information Overload”Communications of the ACM Volume 37 , Issue 7 (July 1994)

Friday, May 7, 2010

music listened torunning

desktop activity physical locations

events

documents

messages

travels friends/enemies

Friday, May 7, 2010

wouldn’t it be great if computers coulduse all this information to do stuff for us?

Friday, May 7, 2010

for example:

• remind me to take out the trash when I get home on Tuesdays...

• bug my friend who hasn’t replied to me in 2 days...

• send me my grocery shopping list when I arrive at the grocery store

• remind friends about an event I am going to attend

• text me important emails when I am traveling

...how do we get there?

Friday, May 7, 2010

what we need

Friday, May 7, 2010

1. a way for users to express: what they want to happen, and when, in terms of predicates relating

the states and properties of people, places + things in their world.

what we need

Friday, May 7, 2010

what we need

actions

Friday, May 7, 2010

what we need

actionsconditions

Friday, May 7, 2010

what we need

actionsconditionspredicates

Friday, May 7, 2010

what we need

actionsconditionspredicatesproperties

Friday, May 7, 2010

what we need

actionsconditionspredicatespropertiesentities

Friday, May 7, 2010

what we need

Controlled Natural Language Interface (CNLI) for Rules

Friday, May 7, 2010

what we need

2. a way to retrieve and interpret data from our many heterogeneous web sources as descriptions of these familiar people, places and things.

Friday, May 7, 2010

what we need

2. a way to retrieve and interpret data from our many heterogeneous web sources as descriptions of these familiar people, places and things.

ATOM/RSS/REST APIs, End-user mashups + RDF

Friday, May 7, 2010

Controlled/Constrained Natural Language Interface

Abraham Bernstein and Esther Kaufmann and Christian Kaiser and Christoph Kiefer, Ginseng: A Guided Input Natural Language Search Engine for Querying Ontologies, Jena User Conference, 2008.

previous work: for the construction of RDF KBs and queries:

Friday, May 7, 2010

express behaviors as rules

when <something happens> do <action>

Friday, May 7, 2010

query statement

Friday, May 7, 2010

Data streams: ATOM/RSS/REST APIs, End-user mashup for alignment + RDF

Friday, May 7, 2010

RSS0.95

RSS0.95RSS0.95

Friday, May 7, 2010

people

messagesemails

events

www2010 ballroom

www2010 302

starbucks

office

places

david karger

max van kleek

brennan moore

paul andré

photos

web pages

RSS0.95

RSS0.95RSS0.95

Friday, May 7, 2010

people

messagesemails

events

www2010 ballroom

www2010 302

starbucks

office

places

david karger

max van kleek

brennan moore

paul andré

photos

web pages

RSS0.95

RSS0.95RSS0.95

Friday, May 7, 2010

people

messagesemails

events

www2010 ballroom

www2010 302

starbucks

office

places

david karger

max van kleek

brennan moore

paul andré

photos

web pages

RSS0.95

RSS0.95RSS0.95

currently located at

Friday, May 7, 2010

people

messagesemails

events

www2010 ballroom

www2010 302

starbucks

office

places

david karger

max van kleek

brennan moore

paul andré

photos

web pages

RSS0.95

RSS0.95RSS0.95

Friday, May 7, 2010

people

messagesemails

events

www2010 ballroom

www2010 302

starbucks

office

places

david karger

max van kleek

brennan moore

paul andré

photos

web pages

RSS0.95

RSS0.95RSS0.95

end-user mashup

Friday, May 7, 2010

people

messagesemails

events

www2010 ballroom

www2010 302

starbucks

office

places

david karger

max van kleek

brennan moore

paul andré

photos

web pages

RSS0.95

RSS0.95RSS0.95

RDF Activity Streams

RSS1.0

end-user mashup

Friday, May 7, 2010

sharing

production system

RDF world model

Friday, May 7, 2010

sharing

Friday, May 7, 2010

2 atomatea few examples

architecture + world model

extending the system

sharing updates to other atomates

Friday, May 7, 2010

Example 1: Simple context-sensitive reminding

Remind me to take the trash outwhen I get home on Tuesday evenings

Friday, May 7, 2010

Example 2: Travel management

When I’m traveling, warn people who are e-mailing me that I might not get back to them for a while.

Friday, May 7, 2010

How do we tell when we’re traveling?

Friday, May 7, 2010

going on now?

Friday, May 7, 2010

currently traveling?going on now?

Friday, May 7, 2010

currently traveling?

Google Latitude

going on now?

Friday, May 7, 2010

Google Latitude

going on now?

Friday, May 7, 2010

Google Latitude

current location

going on now?

Friday, May 7, 2010

Google Latitude

atomate-wifi

current location

going on now?

Friday, May 7, 2010

Google Latitude

atomate-wifi

current location

> 100 km from home?

going on now?

Friday, May 7, 2010

architecture, rule representation

Friday, May 7, 2010

atom-smasher server for connecting atomates

RDF KB(mozStorage)

rule engine(core atomate XPCOM)

location logger(nsIWiFi)

feeder

web activity tracker

(xulrunner mozilla seamonkey, firefox, fennec)

web sources

rule creator + inspector UI

customized RSSand RDF-JSON feeds for atomates offriends, family, colleagues

RSS feeds/REST APIs

actionsREST APIs() atomate world model updates

RDF-JSON

Friday, May 7, 2010

inside a rule:

when (one-shot) / whenever (repeating)

ANTECEDENT (conditions for execution) AND predicate( subj-pathquery, obj-pathquery-or-val )

AND predicate2(...) AND ...

CONSEQUENT (what to do)action(arg-path-or-val, arg-path-or-val..)

Friday, May 7, 2010

rules in constrained natural language

possessives for path queries(e.g., my current location’s address)

infix english verbs for predicateseq(number,number) => “is”near(Location,Location) => “near”

entities represented by their label (e.g., “David Karger”, “home”, and special pronoun “me”)

“500 Fayetteville St..”addresscurrentlocation

Friday, May 7, 2010

rules in constrained natural language

variables represented with “any/new <type>”

?x rdf:type :Person => “any Person”newly created Person entity: “new Person”

bound variables with “that <type>”

“any Person’s birthday is today email that person ‘happy birthday’”

Friday, May 7, 2010

actions in constrained natural language

actions represented as fill-in-the-blank sentences with typed blanks

[“reply to”, {name:”email” type:”schemas.Email”}, “with”, { name: “message”, type:”schemas.String”} ]

reply to email with message

Friday, May 7, 2010

architecture + world model

Friday, May 7, 2010

modifying/extending atomate’s schema

adding new data sources (sensors or entities)

Friday, May 7, 2010

predicates and actionsnew JV3.schemas.AtomatePredicate( { id:"occurs_date_dows", name:"occurs on a", arg0type:"schemas.SpecificDate", arg1type:"schemas.SpecificDOW", impl:"function(JV3,x,y) { return x.getDate().getDay() == y.getDate().getDay();}" });

“occurs on a”

new JV3.schemas.AtomateAction( { id:"tweet-action", args:[“tweet”, {type:["schemas.String"], name:'message'}], impl:"function(x) { this.TwitterHelper.tweet(x); }" });

“tweet”

Friday, May 7, 2010

connecting atomateswhen my currently viewed page is the same asany person’s currently viewed page

Friday, May 7, 2010

rss pub-subserver

update feeds

to different groups

of friends

Friday, May 7, 2010

3 studyCan users create rules?

Perceived difficulty of use

Pitfalls

Ideas for fixing these problems

Friday, May 7, 2010

Rule creation study (method)Recruited over the web

Basic demographics, sign up, 2 minute tutorial video

9 Rule creation exercises

(2 time, 3 easy, 3 medium, 1 difficult)

Short exit survey:

On average, how difficult was it to create the rules?Was there anything that was confusing/difficult?How useful would such a system be to you?What would you use this system for?What else do you wish this system could do?

Friday, May 7, 2010

Rule creation study

november 2009

33 participants recruited (26 completed)

(ages 25-45)

14 had some programming experience

All experienced with the Web

Friday, May 7, 2010

Rule creation study

correct - rule expressed perfectly

half-correct - rule insufficiently specific; will trigger more often than intended

wrong - 1 or more incorrectly expressed clause will not fire at all, or at all as intended

missing - rule not completed

Friday, May 7, 2010

average time to complete each rule

Friday, May 7, 2010

What caused problems?

Friday, May 7, 2010

perceived difficulty of creating rules

Friday, May 7, 2010

perceived usefulness

Friday, May 7, 2010

survey

(P4) Identifying when two locations converge (i.e. mine and a friends are close). This is like social networking, but moving it towards actual life. People could grant access to their friends to view their locations, and thus know if people are close at a given time.

(P7) Reminding my friends and I that we have a shared event when we're both near each other. For example, I'm often meeting with someone and both of us want to go to the same event in an hour, but we get into a coding session and we forget about the event.

what would you use atomate for?

Friday, May 7, 2010

(P15) When I send email to someone and I want a response, I can tell atomate to send them a reminder email in 3 days if they haven't gotten back to me or something like that.

follow-ups and coordination

(P24) Emailing or responding to people when I am in transit or unavailable (no network connectivity or in an event where my phone's silenced)

Friday, May 7, 2010

mashups and mashup-makersYahoo Pipes!, IBM Mashup Center, Intel Mashmaker [R. Ennals]

end-user web customization toolsChickenfoot [M. Bolin], Coscripter [G. Leshed]

end-user reactive behavioral systemsiCAP [T. Sohn], CAMP [K. Truong], ReBA [A. Kulkharni]

constrained NL interfaces (CLNIs)GINO/Ginseng [A. Bernstein], Sloppy [G. Little]

4 related work

Friday, May 7, 2010

5 ongoing work

reducing errors in rule specificationrule authoring by “picking” from your lifelog

predictability/understandabilitywhy’d it do that? what’ll it do next time?

taking into account variable feed quality (latency)making actions end-user authorable

Friday, May 7, 2010

6 conclusionsBy treating web feeds as sensor streams for creating a simple, unified world model, we can start derive simple but useful reactive automation thathelps regular users manage the deluge of information.

Plenty of challenges remain!Reducing/eliminating errors in rule specification, predictability, improving efficiency of execution, etc.

Think beyond newsreaders and visual mashupswhen designing your Web APIs and data feeds

Friday, May 7, 2010

try it!For adventurers who use Firefox (3.6+):http://atomate.me/pre-alpha

help us improve it ~http://code.google.com/p/list-it [List-it/Atomate Framework]

acknowledgements ~Web Science Trust, ECS, University of Southampton

contact me ~emax@alum.mit.edu / emax@ecs.soton.ac.uk

thanks!

Friday, May 7, 2010

ATOMate It! - People | MIT CSAILpeople.csail.mit.edu/emax/talks/atomate-talk-slides... · 2010. 10....

Documents

Transcript of ATOMate It! - People | MIT CSAILpeople.csail.mit.edu/emax/talks/atomate-talk-slides... · 2010. 10....

eMAX AERIALS - E-One · 2018. 9. 10. · eMAX AERIALS TAKE EMERGENCY RESPONSE TO THE MAX, THE eMAX. Our eMAX pump location and apparatus configuration is designed to offer maximum

URBAN INTERFACE eMAX PUMPER - E-One · URBAN INTERFACE eMAX PUMPER TAKE EMERGENCY RESPONSE TO THE MAX, THE eMAX. One of the industry’s most innovative apparatus designs, the eMAX,

Emax ChairsideIFU

Library Mashups

eMax 2 and eMax 2 Plus High Speed Electric Systems · eMax 2 or eMax 2 Plus System may result in increased emissions or decreased immunity. • The eMax 2 or eMax 2 Plus should not

ODF Mashups

Agile Mashups

Ig1 Mashups

Mashups Presentation

ABB Emax Series Power Circuit Breakers - · PDF file18 Power breakers Emax / X1 by Emax Emax ABB’s Emax series of low voltage power circuit breakers embodies over half a century’s

emax volume 2

Emax to Emax 2 Cross Reference - library.e.abb.com

ABB Emax DC Power Circuit Breakers - clrwtr.com€¦ · Emax eakers 18 Non-automatic switches Emax DC UL The Emax DC UL MS range of switch-disconnectors make it possible to cover

ABB Emax DC Power Circuit Breakers - clrwtr.com · Emax eakers 18 Power breakers Emax DC Emax DC The Emax range of low voltage circuit breakers is completed by the new Emax DC series

Mashups Dr. Mohammad Ahmad. 2 Mashups Agenda Introduction What are Mashups Why Mashups Types of Mashups Architecture Mashup Tools & Editors Mashups Vs.

Map Mashups

eMAX SERIES eMAX 20S eMAX 22L HST HST | eMAX 25L HST … · 2020. 10. 30. · MAHINDRA SUB-COMPACT TRACTORS eMAX SERIES ATTACHMENTS TRACTOR eMAX 20S HST eMAX 20S HST CAB eMAX 22L

Emax Press Data

Point Mashups

Emax (Ch5) Accessories