Security Protocols CS 136 Computer Security Peter Reiher February 14, 2008
PARAID: A Gear-Shifting Power-Aware RAID Charles Weddle, Mathew Oldham, Jin Qian, An-I Andy Wang –...
-
Upload
jonathan-basil-foster -
Category
Documents
-
view
214 -
download
0
Transcript of PARAID: A Gear-Shifting Power-Aware RAID Charles Weddle, Mathew Oldham, Jin Qian, An-I Andy Wang –...
PARAID:PARAID:A A Gear-ShiftingGear-Shifting
Power-Aware RAIDPower-Aware RAID
Charles Weddle, Mathew Oldham, Jin Qian, An-I Andy Wang – Florida St. UniversityCharles Weddle, Mathew Oldham, Jin Qian, An-I Andy Wang – Florida St. University
Peter Reiher – University of California, Los AngelesPeter Reiher – University of California, Los Angeles
Geoff Kuenning – Harvey Mudd CollegeGeoff Kuenning – Harvey Mudd College
MotivationMotivation
Energy costs are risingEnergy costs are rising An increasing concern for serversAn increasing concern for servers
No longer limited to laptopsNo longer limited to laptops
Energy consumption of disk drivesEnergy consumption of disk drives 24% of the power usage in web servers24% of the power usage in web servers 27% of electricity cost for data centers27% of electricity cost for data centers Root to other issues, e.g. server room coolingRoot to other issues, e.g. server room cooling
Is it possible to reduce energy consumption in Is it possible to reduce energy consumption in RAID devices without degrading performance RAID devices without degrading performance while maintaining reliability?while maintaining reliability?
PARAID: A Gear-Shifting Power-Aware RAID
ChallengesChallenges
EnergyEnergy Not enough opportunities to spin down RAIDsNot enough opportunities to spin down RAIDs
PerformancePerformance Essential for peak loadsEssential for peak loads
ReliabilityReliability Server-class drives are not designed for Server-class drives are not designed for
frequent power switchingfrequent power switching
PARAID: A Gear-Shifting Power-Aware RAID
Existing WorkExisting Work
Most trade performance for energy Most trade performance for energy savings directly. savings directly. For example, vary the speed of disksFor example, vary the speed of disks
Most are simulated resultsMost are simulated results
PARAID: A Gear-Shifting Power-Aware RAID
ObservationsObservations
Over provisioning of resourcesOver provisioning of resources RAID is configured for peak performanceRAID is configured for peak performance RAID keeps all drives spinning for light loadsRAID keeps all drives spinning for light loads
Unused storage capacityUnused storage capacity Over-provision of storage capacityOver-provision of storage capacity Unused storage can be traded for energy savingsUnused storage can be traded for energy savings
Fluctuating loadFluctuating load Cyclic fluctuation of loadsCyclic fluctuation of loads Infrequent on-off power transitions can be effectiveInfrequent on-off power transitions can be effective
PARAID: A Gear-Shifting Power-Aware RAID
Power-Aware RAIDPower-Aware RAID
Skewed striping for energy savingsSkewed striping for energy savings
Preserving peak performancePreserving peak performance
Maintaining reliabilityMaintaining reliability
EvaluationEvaluation
ConclusionConclusion
QuestionsQuestions
PARAID: A Gear-Shifting Power-Aware RAID
Skewed StripingSkewed Striping for Energy Saving for Energy Saving
Use over-provisioned spare storageUse over-provisioned spare storage Can use fewer disks for light loadsCan use fewer disks for light loads
PARAID: A Gear-Shifting Power-Aware RAID
RAID
SoftState
Gears123
1 2 3 4 5
Skewed StripingSkewed Striping for Energy Saving for Energy Saving
Operate in gear 1Operate in gear 1 Disks 4 and 5 are powered offDisks 4 and 5 are powered off
PARAID: A Gear-Shifting Power-Aware RAID
RAID
SoftState
Gears123
1 2 3 4 5
Skewed StripingSkewed Striping for Energy Saving for Energy Saving
Approximate the workloadApproximate the workload Gear shift into most appropriate gearGear shift into most appropriate gear
Minimize the opportunity lost to save powerMinimize the opportunity lost to save power
Energy( PoweredOn Disks )
Workload( Disk Parallelism )
Conventional RAID PARAID
workload
PARAID: A Gear-Shifting Power-Aware RAID
Skewed StripingSkewed Striping for Energy Saving for Energy Saving
Adapt to cyclic fluctuating workloadAdapt to cyclic fluctuating workload Gear shift when gear utilization threshold is metGear shift when gear utilization threshold is met
time
load
utilization threshold
gear shift
PARAID: A Gear-Shifting Power-Aware RAID
Preserving Peak PerformancePreserving Peak Performance
Operate in the highest gear Operate in the highest gear When the system demands peak performanceWhen the system demands peak performance Uses the same disk layoutUses the same disk layout
Maximize parallelism within each gearMaximize parallelism within each gear Load is balanced on each gearLoad is balanced on each gear Uniform striping pattern within each gearUniform striping pattern within each gear
Delay block replication until gear shiftsDelay block replication until gear shifts Track block writes as an optimizationTrack block writes as an optimization
PARAID: A Gear-Shifting Power-Aware RAID
Maintaining ReliabilityMaintaining Reliability
Reuse existing RAID levels (RAID-5)Reuse existing RAID levels (RAID-5) Also used in soft stateAlso used in soft state
Drives have a limited number of power Drives have a limited number of power cyclescycles Ration number of power cyclesRation number of power cycles
PARAID: A Gear-Shifting Power-Aware RAID
File System
RAID
PARAID Block HandlerPARAID Block Handler
Disk Device Driver
User Space
Linux kernel
Soft RAIDSoft RAID
Reliability ManagerReliability Manager
MonitorMonitor
Disk ManagerDisk Manager
Admin ToolAdmin Tool
Logical Component DesignLogical Component Design
PARAID: A Gear-Shifting Power-Aware RAID
Data LayoutData Layout
PARAID: A Gear-Shifting Power-Aware RAID
Disk 1 Disk 2 Disk 3 Disk 4 Disk 5
Soft State(RAID-5)
(1-4) 8 12 ((1-4),8,12)
16 20 (16,20,_) _
RAID-5
1 2 3 4 (1-4)
5 6 7 (5-8) 8
9 10 (9-12) 11 12
13 (13-16) 14 15 16
(17-20) 17 18 19 20
Parity for 5 disks does not work for 4 disksParity for 5 disks does not work for 4 disks For example, replicated block 12 on disk 3For example, replicated block 12 on disk 3
Data LayoutData Layout
PARAID: A Gear-Shifting Power-Aware RAID
Disk 1 Disk 2 Disk 3 Disk 4 Disk 5
Soft State(RAID-5)
(1-4) 8 12 ((1-4),8,12)
16 20 (16,20,_) _
RAID-5
1 2 3 4 (1-4)
5 6 7 (5-8) 8
9 10 (9-12) 11 12
13 (13-16) 14 15 16
(17-20) 17 18 19 20
Cascading parity updatesCascading parity updates Must also update parity in soft stateMust also update parity in soft state
Update PropagationUpdate Propagation
Up-shift propagationUp-shift propagation Full synchronizationFull synchronization On-demand synchronizationOn-demand synchronization For example, shifting from 3 to 5 disksFor example, shifting from 3 to 5 disks
Downshift propagationDownshift propagation Full synchronizationFull synchronization
PARAID: A Gear-Shifting Power-Aware RAID
Asymmetric Gear-Shifting PoliciesAsymmetric Gear-Shifting Policies
Up-shift (aggressive)Up-shift (aggressive) Moving average + moving standard deviation > Moving average + moving standard deviation >
thresholdthreshold
Downshift (conservative)Downshift (conservative) Modified moving average + moving standard Modified moving average + moving standard
deviation < thresholddeviation < threshold Moving average modified to account for extra parity Moving average modified to account for extra parity
updatesupdates
PARAID: A Gear-Shifting Power-Aware RAID
ImplementationImplementation
Prototyped in Linux 2.6.5Prototyped in Linux 2.6.5 Open source, software RAIDOpen source, software RAID
Implemented block I/O handler, monitor, Implemented block I/O handler, monitor, disk managerdisk manager
Implemented user admin tool to configure Implemented user admin tool to configure devicedevice
Updated Raid Tools to recognize PARAID Updated Raid Tools to recognize PARAID levellevel
PARAID: A Gear-Shifting Power-Aware RAID
EvaluationEvaluation
ChallengesChallenges Prototyping PARAIDPrototyping PARAID Commercial machinesCommercial machines Benchmarks are designed to measure peak Benchmarks are designed to measure peak
performanceperformance Trace replayTrace replay Time consumingTime consuming
PARAID: A Gear-Shifting Power-Aware RAID
EvaluationEvaluation
multimeter
USB cableclient
server
powersupply
12v & 5vpower lines
powermeasurement
probes
SCSIcable
crossover cable
Xeon 2.8 Ghz, 512 MB RAM36.7 GB 10k RPM SCSI
P4 2.8 Ghz, 1 GB RAM160 GB 7200 RPM SATA
RAID
RAID
RAID
RAID
RAID
BOOT
PARAID: A Gear-Shifting Power-Aware RAID
Measurement frameworkMeasurement framework
EvaluationEvaluation
Measurement frameworkMeasurement framework
PARAID: A Gear-Shifting Power-Aware RAID
Server Client
Processor Intel Xeon 2.8 Ghz Intel Pentium 4 2.8 Ghz
Memory 512 Mbytes 1 Gbytes
Network Gigabit Ethernet Gigabit Ethernet
Disks36.7Gbytes 15K RPM
SCSI Ultra 320160 Gbytes 7200 RPM
SATA
EvaluationEvaluation
Three different workloads using two different Three different workloads using two different RAID settingsRAID settings Web trace - RAID level 0 – 2 gears (2,5)Web trace - RAID level 0 – 2 gears (2,5)
Mostly read activityMostly read activity Cello99 - RAID level 5 – 2 gears (3,5)Cello99 - RAID level 5 – 2 gears (3,5)
I/O intensive workload with writesI/O intensive workload with writes Postmark - RAID level 5 – 2 gears (3,5)Postmark - RAID level 5 – 2 gears (3,5)
Measure peak performance and gear shifting overheadMeasure peak performance and gear shifting overhead
Speed up trace playbackSpeed up trace playback To match hardwareTo match hardware Explore range of speed up factors and power savingsExplore range of speed up factors and power savings
PARAID: A Gear-Shifting Power-Aware RAID
Web TraceWeb Trace
UCLA CS Dept Web Servers (8/11/2006 – 8/14/2006)UCLA CS Dept Web Servers (8/11/2006 – 8/14/2006) File system: ~32 GB (~500k files)File system: ~32 GB (~500k files) Trace replay: ~95k requests with ~4 GB data (~260 MB unique)Trace replay: ~95k requests with ~4 GB data (~260 MB unique)
PARAID: A Gear-Shifting Power-Aware RAID
0
0.1
0.2
0.3
0.4
0.5
0.6
1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96
hours
GB/hour
Web Trace Energy SavingsWeb Trace Energy Savings
0
10
20
30
40
50
60
0 5 10 15 20 25 30
hours
wattsRAID-0
PARAID-0
0
10
20
30
40
50
60
0 5 10 15 20 25 30
hours
wattsRAID-0
PARAID-0
PARAID: A Gear-Shifting Power-Aware RAID
64x – 60 requests/sec
128x – 120 requests/sec 256x – 240 requests/sec
0
10
20
30
40
50
60
0 5 10 15 20 25 30
hours
wattsRAID-0
PARAID-0
64x - 34%
128x - 28%
256x - 10%
Energy Savings
Web Trace LatencyWeb Trace Latency
0
0.2
0.4
0.6
0.8
1
1 10 100 1000 10000 100000
msec
RAID-0
PARAID-0
0
0.2
0.4
0.6
0.8
1
1 10 100 1000 10000 100000
msec
RAID-0
PARAID-0
PARAID: A Gear-Shifting Power-Aware RAID
256x
128x 64x
0
0.2
0.4
0.6
0.8
1
1 10 100 1000 10000 100000
msec
RAID-0
PARAID-0256x - within 2.7%
64x - 240% 80ms vs. 33ms
Overhead
Web Trace BandwidthWeb Trace Bandwidth
-20
30
80
130
180
0 5 10 15 20 25 30
hours
MB/secRAID-0
PARAID-0
-20
30
80
130
180
0 5 10 15 20 25 30
hours
MB/secRAID-0
PARAID-0
PARAID: A Gear-Shifting Power-Aware RAID
256x
128x 64x
0
20
40
60
80
100
120
140
160
180
0 5 10 15 20 25 30
hours
MB/secRAID-0
PARAID-0
256x - within 1.3% in high gear
Overhead
Cello99 TraceCello99 Trace
Cello99 WorkloadCello99 Workload HP Storage Research LabsHP Storage Research Labs 50 hours beginning on 9/12/199950 hours beginning on 9/12/1999 I/O intensive with 42% writesI/O intensive with 42% writes
PARAID: A Gear-Shifting Power-Aware RAID
Cello99 Energy SavingsCello99 Energy Savings
0
10
20
30
40
50
0 10 20 30 40 50
hours
wattsRAID-5
PARAID-5
05
101520253035404550
0 10 20 30 40 50
hours
wattsRAID-5
PARAID-5
PARAID: A Gear-Shifting Power-Aware RAID
128x – 1000 requests/sec
32x – 270 requests/sec
64x – 550 requests/sec
0
10
20
30
40
50
0 10 20 30 40 50
hours
wattsRAID-5
PARAID-532x - 13%
64x - 8.2%
128x - 3.5%
Energy Savings
Cello99 Completion TimeCello99 Completion Time
0.9
0.92
0.94
0.96
0.98
1
1 10 100 1000 10000 100000
msec
RAID-5
PARAID-5
0.9
0.92
0.94
0.96
0.98
1
1 10 100 1000 10000 100000
msec
RAID-5
PARAID-5
PARAID: A Gear-Shifting Power-Aware RAID
128x
64x 32x
0.9
0.92
0.94
0.96
0.98
1
1 10 100 1000 10000 100000
msec
RAID-5
PARAID-5
32x - 1.8ms,26% slowerdue to time spent in lowgear
Overhead
Cello99 BandwidthCello99 Bandwidth
1
10
100
1000
0 500000 1000000 1500000
request number
MB/secRAID-5
PARAID-5
1
10
100
1000
0 500000 1000000 1500000
requests
MB/secRAID-5
PARAID-5
PARAID: A Gear-Shifting Power-Aware RAID
1
10
100
1000
0 500000 1000000 1500000
request number
MB/secRAID-5
PARAID
64x 32x
128x
Overhead
< 1% degra-dation duringpeak hours
Postmark BenchmarkPostmark Benchmark
Popular synthetic benchmarkPopular synthetic benchmark Generates ISP-style workloadsGenerates ISP-style workloads Stresses peak read/write performance of storage Stresses peak read/write performance of storage
devicedevice
PARAID: A Gear-Shifting Power-Aware RAID
Postmark PerformancePostmark Performance
PARAID: A Gear-Shifting Power-Aware RAID
0
50
100
150
200
1K files, 50K trans 20K files, 50K trans 20K files, 100Ktrans
seconds
RAID-5 PARAID-5 high gear PARAID-5 low-gear
Postmark Power MeasurementsPostmark Power Measurements
PARAID: A Gear-Shifting Power-Aware RAID
0
10
20
30
40
50
60
70
80
1 11 21 31 41 51 61 71 81 91 10 111 12 13 14 151 16 171
seconds
wat
ts RAID5
PARAID
Ongoing WorkOngoing Work
Try more workloadsTry more workloads Optimize PARAID gear configurationOptimize PARAID gear configuration Explore asynchronous update propagationExplore asynchronous update propagation Speed up recoverySpeed up recovery Live testingLive testing
PARAID: A Gear-Shifting Power-Aware RAID
Lessons LearnedLessons Learned
Third version of design, early design not portableThird version of design, early design not portable Data alignment problemsData alignment problems Difficult to measure system under normal loadDifficult to measure system under normal load Hardware and operating system optimizationsHardware and operating system optimizations Matching trace environmentMatching trace environment
PARAID: A Gear-Shifting Power-Aware RAID
ConclusionConclusion
PARAID reuses standard RAID-levels without PARAID reuses standard RAID-levels without special hardware while decreasing their energy special hardware while decreasing their energy use by 34%.use by 34%. Optimized version can save even more energyOptimized version can save even more energy
Empirical evaluation importantEmpirical evaluation important
PARAID: A Gear-Shifting Power-Aware RAID
PARAID Gear-ShiftingPARAID Gear-Shifting
256x 128x 64x
Number of gear switches 15.2 8.0 2.0
% time spent in low gear 52% 88% 98%
% extra I/Os for updatepropagations
0.63% 0.37% 0.21%
128x 64x 32x
Number of gear switches 6.0 5.6 5.4
% time spent in low gear 47% 74% 88%
% extra I/Os for updatepropagations
8.0% 15% 21%
Web Trace Gear-Shifting Stats
Cello99 Gear-Shifting Stats
Storage consumption Si for the total RAID for the ith gear;
MiSGGGS
S
iiii
M
ijj
M
ii
..2,1
1
111
1
• PARAID uses around (D – G1)/(D – 1) of the total RAID-5 storage to store soft states.
• For RAID-5, D > 3 disks, M gears with Gi disks within the ith gear (1 < i < M, 3 < Gi < Gi+1 < GM = D)
Storage Requirement for PARAID5Storage Requirement for PARAID5
Storage Requirement for PARAID5Storage Requirement for PARAID5
Disk 1 Disk 2 Disk 3 Disk 4 Disk 5
Gear 1RAID-5
(1-4) 8 12 ((1-4),8,12)
16 20 (16,20,_) _
Gear 2RAID-5
1 2 3 4 (1-4)
5 6 7 (5-8) 8
9 10 (9-12) 11 12
13 (13-16) 14 15 16
(17-20) 17 18 19 20
S1
S2
G1 = 4 G2 = 5
S2(G2 – G1)
S1(G1 – 1)
Space needed for extra parity blocks
S1 + S2 = 1
S1(G1 – 1) = S2(G2 – G1)
PARAIDPARAID
Target percentage energy savings;
idleactive
idleactivedbys
DP
PGPGD
/
/1tan11
• Energy savings increase with more disks and fewer disks in the lowest gear.
• A higher active/standby ratio.
• D = the number of disks in the array• G1 = the disks in gear 1.• P = power in standby/active/idle
PARAIDPARAID
Modified moving utilization;
• Accounts for cascading parity writes in lower gear.
• Gi = the disks in gear i.• Aread/write = the read or write activity• W = the weight to account for additional parity writes (RAID5 = 1.5)
WG
GG
G
GAA
G
GUU
i
ii
i
iwriteread
i
i 11
1
'
PARAIDPARAID
max min configuration empty storagemax energy
savings formula
4 3 3-4 33.00% 25.00% 0.333333
5 3 3-5 50.00% 40.00% 0.5
5 4 4-5 25.00% 20.00% 0.25
5 3 3-4-5 50.00% 40.00% 0.5
6 3 3-6 60.00% 50.00% 0.6
6 4 4-6 40.00% 33.00% 0.4
6 5 5-6 20.00% 17.00% 0.2
6 3 3-4-6 60.00% 50.00% 0.6
6 3 3-5-6 60.00% 50.00% 0.6
6 4 4-5-6 40.00% 33.00% 0.4
6 3 3-4-5-6 60.00% 50.00% 0.6
7 3 3-7 66.67% 57.14% 0.666667
7 4 4-7 50.00% 42.86% 0.5
7 5 5-7 33.00% 28.57% 0.333333
7 6 6-7 16.67% 14.29% 0.166667