Analyzing and Predicting Task Reminders

Post on 07-Jan-2017

98 views 0 download

Transcript of Analyzing and Predicting Task Reminders

Analyzing and Predicting Task RemindersDavid Graus, Paul N. Bennett, Ryen W. White, Eric Horvitz

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 2

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 3

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

When would you like to be reminded?

Alright, remind you to do the laundryat 12:00pm on Sunday, is that right?

Great, I’ll remind you!

Sunday at noon.

Remind me to do the laundry.

Yes.

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 4

Research questions

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 4

Research questions

1. Is there a body of common tasks that underlie the reminder creation process?

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 4

Research questions

1. Is there a body of common tasks that underlie the reminder creation process?

2. Can we identify patterns in the times at which people create reminders, and, via notification times, when the associated tasks are to be executed?

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 4

Research questions

1. Is there a body of common tasks that underlie the reminder creation process?

2. Can we identify patterns in the times at which people create reminders, and, via notification times, when the associated tasks are to be executed?

3. Can we predict when certain tasks are most likely to happen?

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 5

Is there a body of common tasks that underlie the reminder creation process?

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 5

Is there a body of common tasks that underlie the reminder creation process?

• Common reminder structure: • “Remind me to <verb> <object>” • take out laundry • call mom • pay bills

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 5

Is there a body of common tasks that underlie the reminder creation process?

• Common reminder structure: • “Remind me to <verb> <object>” • take out laundry • call mom • pay bills

• Identify frequent verb/object-pairs in 2-month sample (~950k reminders)

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 5

Is there a body of common tasks that underlie the reminder creation process?

• Common reminder structure: • “Remind me to <verb> <object>” • take out laundry • call mom • pay bills

• Identify frequent verb/object-pairs in 2-month sample (~950k reminders)

• Manually group them into tasks

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 6

~60% of reminders start with one of these (52) verbs

take eathave send email textcall tell setmake scheduleturn on turn off

checkleave come be go tostop by take out feed clean wash charge do write

change cancel order renew book mail submit fill out print pay wake set

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 7

Manual labeling

Ò Dimensions; Ò Interruption vs. continuation of user activity Ò Context of task (home, work, on the go) Ò (Expected) duration of task Ò Impact on user “availability” Ò …

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8

common tasks that underlie the reminder creation process

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8

Go somewhere Switch context Run errand

common tasks that underlie the reminder creation process

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8

Go somewhere Switch context Run errand

common tasks that underlie the reminder creation process

Go somewhere

Chores

Switch context Run errand

Recurring Standalone

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8

Go somewhere Switch context Run errand

common tasks that underlie the reminder creation process

Go somewhere

Chores

Switch context Run errand

Recurring Standalone

Go somewhere

Chores

Communicate

Switch context Run errand

Recurring Standalone

General Coordinate

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8

Go somewhere Switch context Run errand

common tasks that underlie the reminder creation process

Go somewhere

Chores

Switch context Run errand

Recurring Standalone

Go somewhere

Chores

Communicate

Switch context Run errand

Recurring Standalone

General Coordinate

Go somewhere

Chores

Communicate

Manage Ongoing

External Process

Manage Ongoing

User Activity

Eat/Consume

Switch context Run errand

Recurring Standalone

General Coordinate

StartPrepare Stop

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 8

Go somewhere Switch context Run errand

common tasks that underlie the reminder creation process

Go somewhere

Chores

Switch context Run errand

Recurring Standalone

Go somewhere

Chores

Communicate

Switch context Run errand

Recurring Standalone

General Coordinate

Go somewhere

Chores

Communicate

Manage Ongoing

External Process

Manage Ongoing

User Activity

Eat/Consume

Switch context Run errand

Recurring Standalone

General Coordinate

StartPrepare Stop

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 9

RQ2: Can we identify patterns in the reminder creation process?

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 9

RQ2: Can we identify patterns in the reminder creation process?

Ò Creation time (CT) Ò When user creates the reminder (remembers task)

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 9

RQ2: Can we identify patterns in the reminder creation process?

Ò Creation time (CT) Ò When user creates the reminder (remembers task)

Ò Notification time (NT) Ò When reminder is set to trigger (executes task)

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 9

RQ2: Can we identify patterns in the reminder creation process?

Ò Creation time (CT) Ò When user creates the reminder (remembers task)

Ò Notification time (NT) Ò When reminder is set to trigger (executes task)

Ò Time delta (NT – CT) Ò How far task is planned in advance

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 9

RQ2: Can we identify patterns in the reminder creation process?

Ò Creation time (CT) Ò When user creates the reminder (remembers task)

Ò Notification time (NT) Ò When reminder is set to trigger (executes task)

Ò Time delta (NT – CT) Ò How far task is planned in advance

Ò Text/task description

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 10

00:00am–04:00am

04:00am–08:00am

08:00am–12:00pm

12:00pm–4:00pm

4:00pm–8:00pm

8:00pm–0:00am

SunMonTueWedThuFriSat

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 11

RQ2: Can we identify patterns in the reminder creation process?

1. Aggregate 2. Per task type 3. By Creation Time and Terms (in task description)

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 12

Creation times Notification times

1: Aggregate patterns (across all tasks)

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 12

Creation times Notification times

1: Aggregate patterns (across all tasks)

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 12

Creation times Notification times

1: Aggregate patterns (across all tasks)

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 12

Creation times Notification times

1: Aggregate patterns (across all tasks)

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 13

Findings: Aggregate patterns

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 13

Findings: Aggregate patterns

Ò Creation/Notification Ò People create reminders in evenings (end of day), w/

notifications set in mornings (start of day).

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 13

Findings: Aggregate patterns

Ò Creation/Notification Ò People create reminders in evenings (end of day), w/

notifications set in mornings (start of day).

Ò (not shown) Time deltas Ò Most reminders are for “short-term tasks”

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 14

2: Per task type patterns

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 15

All rem

inders

Chore (re

curri

ng)

Chore (st

andalone)

Comm. (coordinate)

Comm. (gen

eral)

Eat/Consu

me

Go (switc

h contex

t)

Go (erra

nd)

Manage acti

vity

Manage proce

ss

Del

ay (h

ours

)

72

60

48

36

24

12

0

Time deltas per task typeTime deltas per task type

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 15

All rem

inders

Chore (re

curri

ng)

Chore (st

andalone)

Comm. (coordinate)

Comm. (gen

eral)

Eat/Consu

me

Go (switc

h contex

t)

Go (erra

nd)

Manage acti

vity

Manage proce

ss

Del

ay (h

ours

)

72

60

48

36

24

12

0

Time deltas per task typeTime deltas per task type

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 15

All rem

inders

Chore (re

curri

ng)

Chore (st

andalone)

Comm. (coordinate)

Comm. (gen

eral)

Eat/Consu

me

Go (switc

h contex

t)

Go (erra

nd)

Manage acti

vity

Manage proce

ss

Del

ay (h

ours

)

72

60

48

36

24

12

0

Time deltas per task typeTime deltas per task type

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 15

All rem

inders

Chore (re

curri

ng)

Chore (st

andalone)

Comm. (coordinate)

Comm. (gen

eral)

Eat/Consu

me

Go (switc

h contex

t)

Go (erra

nd)

Manage acti

vity

Manage proce

ss

Del

ay (h

ours

)

72

60

48

36

24

12

0

Time deltas per task typeTime deltas per task type

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 16

Findings: per task type patterns

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 16

Findings: per task type patterns

Ò Time deltas Ò Differ between task type.

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 16

Findings: per task type patterns

Ò Time deltas Ò Differ between task type.

Ò (not shown) Creation/Notification Ò broadly two types of tasks:

Ò Created/set to notify during office hours Ò e.g., communicate, go somewhere

Ò Created/set to notify outside of office hours Ò e.g., manage ongoing process, chores

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17

Patterns 3: By creation time

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17

What is the relation between creation and notification time?

What is the relation between creation and notification time?

Patterns 3: By creation time

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17

What is the relation between creation and notification time?

What is the relation between creation and notification time?

Patterns 3: By creation time

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17

What is the relation between creation and notification time?

What is the relation between creation and notification time?

What is the relation between creation and notification time?

What is the relation between creation and notification time?

Patterns 3: By creation time

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17

What is the relation between creation and notification time?

What is the relation between creation and notification time?

What is the relation between creation and notification time?

What is the relation between creation and notification time?

Patterns 3: By creation time

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17

What is the relation between creation and notification time?

What is the relation between creation and notification time?

What is the relation between creation and notification time?

What is the relation between creation and notification time?

What is the relation between creation and notification time?

Patterns 3: By creation time

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17

What is the relation between creation and notification time?

What is the relation between creation and notification time?

What is the relation between creation and notification time?

What is the relation between creation and notification time?

What is the relation between creation and notification time?

Patterns 3: By creation time

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 17

What is the relation between creation and notification time?

What is the relation between creation and notification time?

What is the relation between creation and notification time?

What is the relation between creation and notification time?

What is the relation between creation and notification time?

Patterns 3: By creation time

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 18

“call”

Patterns 3: By terms

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 18

“call”

Patterns 3: By terms

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 18

“call”

Patterns 3: By terms

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 18

“call”

Patterns 3: By terms

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 19

“kids”

What is the relation between creation and notification time?

What is the relation between creation and notification time?

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 19

“kids”

What is the relation between creation and notification time?

What is the relation between creation and notification time?

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 20

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 20

“church”

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 21

RQ2: Can we identify patterns in the reminder creation process?

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 21

RQ2: Can we identify patterns in the reminder creation process?

Ò On average people tend to set plans in the evening.

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 21

RQ2: Can we identify patterns in the reminder creation process?

Ò On average people tend to set plans in the evening. Ò Most tasks that drive reminder setting are for short-term

tasks to be executed in the next 24 hours.

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 21

RQ2: Can we identify patterns in the reminder creation process?

Ò On average people tend to set plans in the evening. Ò Most tasks that drive reminder setting are for short-term

tasks to be executed in the next 24 hours.Ò Patterns differ between task types (suggesting the

distinctions are meaningful).

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 21

RQ2: Can we identify patterns in the reminder creation process?

Ò On average people tend to set plans in the evening. Ò Most tasks that drive reminder setting are for short-term

tasks to be executed in the next 24 hours.Ò Patterns differ between task types (suggesting the

distinctions are meaningful).Ò Terms and creation times are informative w.r.t.

notification time.

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 22

RQ3: Can we predict when certain tasks are most likely to happen?

Ò Motivation: Ò See if patterns are informative.

Ò Applications: Ò improve resource scheduling, Ò detect collisions Ò tailored advertising, Ò developing systems to automatically terminate

ongoing tasks/allocate time for task completion

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 23

Findings

Ò Predict notification day with ~70% accuracy. Ò Creation time provides the most information. Ò Terms provides significant additional information.

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 24

In summary

1.Studied frequent reminders, developed task type taxonomy.

2.Shown reminders’ temporal patterns

3.Demonstrated a direction in harnessing patterns; predicting notification times.

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 25

Future work

Ò User studies? Ò assumptions w.r.t. “wrapping up day, planning ahead” Ò validate/check task type taxonomy

Ò Classify task types for improved predictions? Ò More sophisticated predictive models Ò Look at additional reminder types (e.g., location-based)

Analyzing and Predicting Task RemindersWed 13 July 2016, UMAP 2016 26

Thanks!

Ò Questions?

d.p.graus@uva.nl @dvdgrs

www.graus.co