Carat: Collaborative Detection of Energy Bugs - The UC Berkeley Big Data...

Carat: Collaborative Detection of Energy Bugs

Adam Oliner, Anand Iyer, and Ion StoicaAMP Lab, UC Berkeley

Eemil Lagerspetz, Sasu TarkomaU of Helsinki

AMP Camp 22/08/12

Mobile is Hot• ... sometimes, literally:

2

AMP Camp 22/08/12

A Day in the Life

3

AMP Camp 22/08/12

Users’ Questions• Why is my battery draining?

• Is that normal?

• What can I do about it?

4

AMP Camp 22/08/12

Prior Approaches• Ad hoc

• e.g., no-sleep bug

• Intrusive

• Generic

• “Kill all background apps”

• “Dim the screen”

5

Our Approach

instrumentationdata

actions andreports

Spark onEC2

thecrowd

thecloud

bigdata

raw andderived data

statistical analysis

DynamoDBand S3

Our Approach

instrumentationdata

actions andreports

Spark onEC2

thecrowd

thecloud

bigdata

raw andderived data

statistical analysis

First collaborative approach for diagnosing energy problems.

DynamoDBand S3

AMP Camp 22/08/12

Carat• Mobile app for iOS and Android

• Personalized energy debugging

• What is misbehaving

• Whether that is normal

• What you can do about it

• How much it will help

7

AMP Camp 22/08/12

Today• Collecting data

• Generating results

• Quantifying uncertainty

• Real-world deployment

9

Carat Sampling

t1 WiFi? iOS5.1? ...

Carat Sampling





Computing Rates



Computing Rates

∆t

∆t



Computing Rates

∆t∆% = discharge rate (%/s)

∆%



Computing Rates

∆t∆% = discharge rate (%/s)

F

| F

Energy Hog

Energy Rate (% / s)

Prob

abili

tyF

Energy Hog

Energy Rate (% / s)

Prob

abili

tyF

energy hog

Energy Bug

Energy Rate (% / s)

Prob

abili

ty

Energy Bug

Energy Rate (% / s)

Prob

abili

ty

Without the crowd, there is no way to know whether this use is normal.

Energy Bug

Energy Rate (% / s)

Prob

abili

ty

(Given is not a Hog.)

Energy Bug

Energy Rate (% / s)

Prob

abili

tyenergy bug

(Given is not a Hog.)

AMP Camp 22/08/12

Original Distribution

14

Energy Rate (% / s)

Prob

abili

ty

AMP Camp 22/08/12


14

Energy Rate (% / s)

Prob

abili

ty

E[reference]E[subject]

AMP Camp 22/08/12


14

Energy Rate (% / s)

Prob

abili

ty

E[reference]E[subject]

D

AMP Camp 22/08/12

Mean Distribution

15

E[Energy Rate] (% / s)

Prob

abili

ty

E[subject]E[reference]

AMP Camp 22/08/12

Error and Confidence

16


Prob

abili

ty

EV

AMP Camp 22/08/12


16


Prob

abili

ty

EV ± e %/s50% confidence

AMP Camp 22/08/12

± E %/s95% confidence


16


Prob

abili

ty

EV

AMP Camp 22/08/12

Confidence Factors

17

EV

Energy Rate (% / s)

Prob

abili

ty

AMP Camp 22/08/12

Classification

18

Energy Rate (% / s)Pr

obab

ility

Energy Rate (% / s)

Prob

abili

ty

Energy Rate (% / s)

Prob

abili

ty

Fnot-F

AMP Camp 22/08/12

Diagnosis

19

a

not-v

x

AMP Camp 22/08/12

Diagnosis

19

a

not-v

x

“Killing app a will give x±e of battery life

(95% confidence), as would upgrading the

OS to version v.”

AMP Camp 22/08/12

Initial Deployment• TestFlight over-the-air beta

• 100 sign-ups (Apple’s max)

• 75 installs

• Collected 10,000+ samples

20

AMP Camp 22/08/12

Initial Results• Found 35 apps exhibiting energy bugs

• e.g., Facebook, Kindle, Flipboard

• Corroborated with forum posts, news articles, and data correlations

• Injected three bugs in Wikipedia

• Detected all of them

21

AMP Camp 22/08/12

Flipboard Bug• Digital magazine app

• Bug on 6 clients

• Support thread: iOS 5.0.1

• Confirmed by our data

• OS upgrade → 44m 17s improvement

22

AMP Camp 22/08/12

Kindle Bug• E-book reader

• Bug on 5 clients

• Forum: WhisperSync

• Confirmed by our data

• Turn on WiFi → 1h 33m improvement

23

Suddenly...

Suddenly...

Carat went viral. In 24 hours, there were dozens of articles and we had

more than 100,000 users.

AMP Camp 22/08/12

Carat Today• 320,000+ devices

• 190,000+ on iOS

• 130,000+ on Android

• 20M+ samples

• 20,000+ hogs and 2M+ buggy instances

25

AMP Camp 22/08/12

Some Lessons Learned• Cloud ≠ free scaling

• “Research product” cuts both ways

• Contract with the crowd

26

AMP Camp 22/08/12

Next Steps• Quantifying and improving confidence

• API for developers

• Deploy to the crowd; debug in the cloud

• Platform for collaborative debugging

• Statistics as a service

27

carat.cs.berkeley.edu

AMP Camp 22/08/12

FinYou have reached the end of the presentation. Please turn back.

28

Carat: Collaborative Detection of Energy Bugs - The UC Berkeley Big Data...

Documents

Transcript of Carat: Collaborative Detection of Energy Bugs - The UC Berkeley Big Data...