Tackling the Battery Problemfor Continuous Mobile Vision

June 11, 2013

Victor Bahl

Robert LeKamWa (MSR/Rice), Bodhi Priyantha, Mathai Philipose, Lin Zhong (MSR/Rice)

MIT Technology Review Mobile Summit 2013

resource poverty hurts

Adam & Eve 2000 AD

Hu

ma

n A

tte

ntio

n

courtesy. M. Satya, CMU

technology should reduce the demand on human attention

clever exploitation of {context awareness, computer vision, machine learning, augmented reality} needed to deliver vastly superior mobile user experience

• no “Moore’s Law” for human attention

• being mobile consumes greater human attention

• already scarce resource is further taxed by resource poverty

continuous mobile visionreality vs. movies

Steve Mann (early 90s)

Mission Impossible 4 (2011)

COBOT, CMU (2013)

iRobot (2004)

C-3PO (1977)

Victor Bahl, MSR

perennial challenges

computation

connectivity & bandwidth

battery

Resource constraints prevent today’s mobile apps from reaching their full potential

white space networks, small cell networks, mm-wave networks

Augmented Reality

cloudlets

MSR’s SenseCam for memory assistance

Victor Bahl, MSR

battery trends

0

50

100

150

200

250

91 92 93 94 95 96 97 98 99 00 01 02 03 04 05

Wh

/Kg

Year

Li-Ion Energy Density

CPU performance improvement during same period: 246x

A silver bullet seems unlikely

• Lagged behindoHigher voltage batteries (4.35

V vs. 4.2V) – 8% improvemento Silicon anode adoption (vs.

graphite) – 30% improvement

• Trade-offs o Fast charging = lower capacityo Slow charging = higher

capacity

Victor Bahl, MSR

Single Core

Processor CPU + GPU

~150 mW

Sensors +

Memory + Disk

~ 15 mW

Display

~500 mW

Network Stack(5 min. of usage / hour)

~100 mW

so where is the energy going?

assuming a typical SmartPhone battery of 1500 mAh (~5.5 W)

battery lifetime ~7.25 hours

power consumption of a typical image sensor

5 MP, 5 fps

345 mW

1 MP, 5 fps

250 mW

0.3 MP, 15 fps

245 mW

1 MP, 15 fps

295 mW

0.3 MP, 5 fps

232 mW

0.3 MP, 30 fps

268 mW

0.3 MP, 5 fps

232 mW

low resolution, low frame rate image sensing for vision related tasks can reduce battery life by > 25%

Reduceresolution

Reduce frame rate

state of artEnergy / pixel is inversely proportional to the frame rate & image resolution

0 10 20 300

50

100

150

200

FPSP

ow

er

(mW

)

Video at 0.1 MP power vs. resolution

0 5 10

x 106

0

100

200

300

Pow

er

(mW

)

Npixels

Video at 30 fps

power vs. frame rate

Regardless of image resolution & frame rate, image sensors consume about the same power

Profiled 5 image sensors from 2 manufacturers

Victor Bahl, MSR

digging deeper (1 MP, 5 fps)

Active Period

Idle Period

function of pixel count

& clock speed

function of frame rate

reduce power by reducing pixel readout time

Number of Pixels

divided by

Clock Frequency

reduce this

one pixel is read out per clock period

Victor Bahl, MSR

reducing pixel count (N)

ActiveFrame

Readout

Active

Readout

Po

we

r

Active

Readout Po

we

r

Time Time

Scaled Resolution

(Pixel Skipping)

Region-of-Interest

(Windowing)

reduce power by aggressive use of standby

Turn off sensor during idle period

Idle mode necessary to allow exposure before readout

Active

Readout

Active

Readout

Active

Readout

Active

Readout

Active

Readout

Active

Readout

Standby mode

Idle mode

Best when frame rate and resolution are sufficiently low

reduce power by adjusting clock frequency

Adjust clock frequency to minimize power

Adjust this

frequency

1 fps

5 fps

Power

frequency

At low frame rates, run the clock as slow as possible

Tradeoff

summarizing power reduction techniques

reduce Tactive & increase Tidle

decrease frame rate

reduce total pixel readout time (by reducing N)

adapt clock frequency

Instead of idle-ing put sensor in standby state

reduce Pactive (not covered in this talk, see paper)

applying these techniquesF

ram

e r

ate

(F

PS

)

Resolution (MP)

30

25

20

15

10

5

021 3 4 5

Fra

me

ra

te (

FP

S)

Resolution (MP)

30

25

20

15

10

5

021 3 4 5

Po

we

r (m

W)

350

300

250

200

150

100

50

0

Aggressive Standby &

Clock OptimizationUnoptimized

impact on vision algorithms

480 x 270

Person DetectionImage registration

Image Registration

Success

Person Detection

Success

Actual Power

Reduction with

software assist

Estimated Power

Reduction with

hardware assist

Full Resolution

(129600 pixels)99.9% 94.4% 51% 84%

Frame Rate- 3 FPS 95.7% 83.3% 95% 98%

30% Window

(63504 pixels)96.5% 77.8% 63% 91%

Subsampled by 2

(32400 pixels)91.8% 72.2% 71% 94%

MSR’s Glimpse project

collaborators & references

R. LeKamWa, B. Priyantha, M. Philipose, L. Zhong, P. Bahl, Energy

Characterization and Optimization of Image Sensing Towards

Continuous Mobile Vision, Proceedings of ACM MobiSys 2013, Taipei,

Taiwan, June 26-29, 2013

P. Bahl, M. Philipose, L. Zhong, Cloud-Powered Sight for All: Showing the

Cloud What You See, ACM Mobile Cloud Computing & Services

Workshop, Lake District, U.K. June 25, 2012

Robert Bodhi Matthai Lin

Thanks!

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.

The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it

should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation.

MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.