Quality of Service-Aware, Scalable Cache Tuning Algorithm in Consumer-based Embedded Devices

M. Hammam Alsafrjalani and Ann Gordon-Ross Department of Electrical and Computer Engineering, University of Florida, Gainesville, FL, USA

q  Consumer-basedEmbeddedDevices(CEDs)areubiquitous

q  Highconsumer-de:inedqualityofservice(QoS)expectations(consumergoals)andstringentenergyconstraints,oftencontending

q  Goal:innovateaCEDsdesignapproachthatenablesCEDsto

adheretoconsumergoalsandenergyrequirements,independentlyofapplications/deployment

q  Challenges

q  Consumergoalsmustbeknownduringdesigntimeq  Implausiblegivenubiquitynatureofsystems(diverse

consumers)q  Softwareapplicationsmustbeknownduringdesigntime[1]

q  Implausible,givenrapidgrowthofunknownthirdpartyapplications(~1.5million+appsforAndroid)

Usingcon0igurablehardwareandassociatedtuningalgorithmsq  Con0igurablehardware

q  Containsmodi:iable/tunableparametersq  Clock,voltage,memory,etc.

q  Parametervalueschangetoapplication’shardwarerequirements

q  Parametervaluesdictatedbyatuningalgorithmq  Tuningalgorithm

q  Monitorsapplicationexecution,evaluateenergyconsumptionandqualityofservice

q  Exploresthedesignspace(availableparametervalues)q  AdjustsparametervaluessuchthatCED

q  MeetsQoSexpectationandconsumelowestenergyq  Targeteffectivecomponent

q  Cachememoryq  Hashighimpactonenergyandperformance

Application-scalablehardwareandruntimetuningalgorithmq  Scalablehardware

q  Compressedtuninginformationintoauxiliarytablesq  EmployedLRUpolicytoenablescalabilityofapplication-

tuningq  Only4%area(hardware)overhead

q  Dynamic,generalpurposeCEDcache-tuningalgorithmq  Requiresnoaprioriknowledgeofapplicationsq  Flexible:ConservativeandModeratemodes

q  FordisparateQoSexpectationsq  TradesoffenergysavingsforhigherQoS

DesigningapproachforCED’sSavingenergydoesnotrequireaprioriknowledgeofapplicationsü  Enablesscalabilitytoanarbitrarynumberofunknown(future),third-partyapplications

ü  MeetsconsumergoalsTuningmodesprovide:lexibleadherencetoQoSexpectationsü  Canincorporatetuningmodesfordisparateconditions/situations,suchas

ComparisontopriorworkPriorwork[1][2]savedmoreenergy,however

q  (-)Incurredasmuchas109.2%and132.5%moretuningoverheadwhiletuning

q  (-)Requireslongapplicationsexecutiontoamortizethetuningoverhead

PriorworkdegradedQoSupto7Xmore,comparedtoourapproachq  (-)Canbecompletelyavoidedonlywithapriori

knowledgeofapplications[1]

q  NotpossibleforCEDs

InterdisciplinarycollaborationØ  Psychologybehindconsumerexpectationsanditsimpactondesignconstraints

Ø  QuantiNicationofconsumerfeelings,moods,andthoughts

Ø  Otherfactorssuchastimeofuse(day,night,etc.),placeofoperation(ofNice,constructionsite,on-route,etc.),...

ArchitectureinnovationsØ  Performance&energyexpectationsvs.privacy&security

Ø  AdaptabilitytoInternetofThings(IoT)devices

Ø  TuningthroughIoT

[1]Wang,W.,Mishra,P.,Gordon-Ross,A.“SACR:Scheduling-AwareCacheRecon:igurationforReal-TimeEmbeddedSystems,”Int.Con.onVLSIDesign,2009.[2]Zhang,C.,Vahid,F.“Cachecon:igurationexplorationonprototypingplatforms,”IEEEInternationalWorkshoponRapidSystemsPrototyping,2003

V.S.

v  Quad-coresystemv  Con0igurable,privatelevel1cachememory

Ø  Possiblecon:igurations:2KB1-way;4KB1-or2-way;and8KB1-,2-,or4-way

v  HardwarecachetunerØ  Global,monitorsallcoresØ  TunesthecoresbasedonthetuningalgorithmØ  Storestuninginformationinlookupandauxiliary

tables

v  Information Input: Reads tuning information from lookup andauxiliarytables;tunestobestcon:iguration,orresumesexploration

v  Exploration:Determinesthecon:igurationtotunethehardwareto,basedonthetuninginformationandtuningmode;updatesthetables

v  Evaluation: Evaluates if the con:iguration saves energy and/ordegradesQoS;decidesiftuningisdone

39.76%

20.68% 19.00%

00.050.10.150.20.250.30.350.40.45

TuningMode

Percen

tofE

nergysavings

Datacache

Aggressive Moderate Conserva>ve

34.98%

25.14% 23.49%

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

TuningMode

Percen

tofE

nergysavings

Instruc5oncache

Aggressive Moderate Conserva>ve

ü  Aggressive(priorwork)achievedhighestsavingsØ  Howeverrequiredprioriknowledgeof

applicationü  Moderateandconservativemodesresultsincomparable

energysavings

E(total)=E(sta)+E(dyn)E(dyn)=cache_hits*E(hit)+cache_misses*E(miss)E(miss)=E(off_chip_access)+miss_cycles*E(CPU_stall)+E(cache_fill)MissCycles=cache_misses*miss_latency+(cache_misses*(line_size/16))

*memory_band_width)E(sta)=total_cycles*E(staCc_per_cycle)E(sta3c_per_cycle)=E(per_Kbyte)*cache_size_in_KbytesE(per_Kbyte)=(E(dyn_of_base_cache)*10%)/(base_cache_size_in_Kbytes)

QoSExpecta3on=performance>thresholdThreshold=minimumacceptableperformanceIfperformance<threshold,QoS_degradaCon=trueElse QoS_degradaCon=false

EnergysavingsandQoS:ourapproachvs.basesystemandpriorworkv  Energy savings: Measured application’s energy consumption of

best con:iguration determined by the tuning algorithm, calculatedenergypercentdecreasewithrespecttothebasesystem,andaveragedtheenergysavingsforallapplications(34total)

v  Quality of Service: Calculated the number of QoS degradationoccurrences while tuning each application, compared the result to aperfect (no QoS degradation) system, and averaged the QoSdegradationforallapplications(34total)

v  Comparison to priorwork: Incorporated a tuningmodewhichrepresents prior work approach; an aggressive mode whichdeterminedthelowest-energycon:igurationwithoutconsideringQoS

7.70

1.000.41

0123456789

TuningMode

AverageQoSdegrada

5on

Datacache

Aggressive Moderate Conserva>ve7.40

1.00 0.70

0

1

2

3

4

5

6

7

8

TuningMode

AverageQoSdegrada

5on

Instruc5oncache

Aggressive Moderate Conserva>ve

Aggressive(priorwork)imposedQoSdegradationashighas7X,onaverage

ü  ModeratemodeimposedQoSdegradation,atmost1ü  ConservativemodeaverageQoSdegradation<1.00

Otherwise

Environmental Experience/Personal