Post on 26-Jul-2020
Pros and Cons of Various DC Distribution Architectures
Randy Malik
Power Technology And Qualification
IBM RTP Raleigh NC
Rick Fishbune
Power Technology And Qualification
IBM Rochester, MN
Source: Solar Powered Datacenter by Google
High Voltage Distribution Bus
© 2004 IBM Corporation2 9/26/2008
Large Energy ConsumersFor 100 units of Coal (BTU) only 33 units available for use !!!
Data Centers Consume 1.5% of total Electricity in USA
High Voltage Distribution Bus
© 2004 IBM Corporation3 9/26/2008
Sources: US EIA, US Census Bureau, (Patzek, 2007).
Energy Crisis for the Planet Earth
High Voltage Distribution Bus
© 2004 IBM Corporation4 9/26/2008
Critical Power &
Cooling
Hardware & Software
Desired Business Objectives
Investment is 2 – 4% of the total
Capital Expenditure in the Datacenter
Power and Cooling - the foundation to achieve business Objectives
Annual Energy Cost in a datacenter is approaching Capital Expenditure in a Datacenter …… Source IDC
High Voltage Distribution Bus
© 2004 IBM Corporation5 9/26/2008
A Typical Tier 4 Datacenter Infrastructure
High Voltage Distribution Bus
© 2004 IBM Corporation6 9/26/2008
Raised Floor
ROW 1
ROW 2
ROW 3
ROW 4
STS
PDU
PDU
Air
Han
dler
Air
Han
dler
Battery Plant
3 Phase
AC UPS 2
3 Phase
AC UPS 1
ATS for Chillers
Row of IT Racks
480V AC 3 Phase
480V AC 3 Phase
Free Air
Free Air
Simplified Diagram of a Tier 4 Datacenter
480 V
208 V
208 V
High Voltage Distribution Bus
© 2004 IBM Corporation7 9/26/2008
Power Distribution in a Tier 4 Datacenter
13.2KV/480
Transformer
13.2KV/480
Transformer
AC Generators
AC Generators
Double Conversion
AC UPS
Double Conversion
AC UPS
Battery Bank
3 Phase
480Vac
3 Phase
480Vac
Electronic
Transfer
Switch
PDU
480/208 V AC
PDU
480/208 V AC
AC
AC 2
Raised Floor Data Processing Room
A row of Data Processing Racks
208 V AC
3 Phase
208 V AC
3 Phase
96.3%
96.3%
99% 98.9%
98.9%
90%Best efficiency for 12V AC – DC Rectifier
Efficiency From 480 Vac – 12V DC : ( .99x .989 x .90 ) = 88%
13.5 KV
13.5kV
High Voltage Distribution Bus
© 2004 IBM Corporation8 9/26/2008
48V or 400V DC Distribution Concept for Tier 4 Datacenter
13.2KV/488 Vac
Transformer
13.2KV/488 Vac
Transformer
AC Generators
AC Generators
AC – DC Rectifiers in
parallel
Battery Bank
Raised Floor
AC – DC Rectifiers in
parallel
DC Bus 1 ( 400V DC or 48V DC)
480V AC 3 Phase
480V AC 3 Phase
DC Bus 2 ( 400V DC or 48V DC)
98%
98%
97%
13.5KV
13.5KV
Neutral
Neutral
400V - 12V Unregulated
Best Possible Efficiency (Non-isolated) = .98 x .97 = 95%
Best Possible Efficiency (Isolated) = .96 x .97) = 93%
400V DC Distribution Issues:
1. Safety Concern
2. Customer Perception
3. Certified connectors are not available
Rack
High Voltage Distribution Bus
© 2004 IBM Corporation9 9/26/2008
13.2KV
13.2KV
13.2KV/488 Vac
Transformer
13.2KV/488 Vac
Transformer
AC Generators
AC Generators
AC – DC Rectifiers in
parallel
Battery Bank
Raised Floor
AC – DC Rectifiers in
parallel
DC Bus 1 ( 400V DC or 48V DC)
480V AC 3 Phase
480V AC 3 Phase
DC Bus 2 ( 400V DC or 48V DC)
98%
98%
97%
Neutral
Neutral
400V - 12V Unregulated Rack
Solar Powered Datacenter with DC Distribution
DC – DC
Converter
X
X
Estimated Savings in Power = 5 MW
Estimated Savings in Electric Bill = 5 M Dollars/Year !
High Voltage Distribution Bus
© 2004 IBM Corporation10 9/26/2008
DC Distribution in a Datacenter
480V AC
UPS
PDU
PDU
208 /120V
208 /120V
IT Racks
UPS
6.5%
2.5%
Rectifiers
400V DC or 48V DC
Battery Plant
High Voltage Distribution Bus
© 2004 IBM Corporation11 9/26/2008
70% 75% 80% 85% 90% 95%
0
500
1000
1500
2000
2500
3000
3500
Max
imum
Out
put P
ower
(W)
X Series Power Supply Efficiency Spread @ 100% Load
Multiple Output Power Supplies
High Voltage Distribution Bus
© 2004 IBM Corporation12 9/26/2008
480Vac Plug to Processor Efficiency for a typical DatacenterOverall
Efficiency and Normalized Power
Raised Floor Transformer
Distribution AC – DC Power Supply
DC – DC Regulator
Processors, Memory, ICs)
AC Distribution
63.236 %
1.58
77.16 %
1.295
85.10%
1.175
85.25%
1.14
89.605%
1.116
480V AC– 400 VDC – 480V AC
400V AC – 208V AC : 3 Phase
Data Center Distribution
208Vac – 12V DC 12 VDC – 1V DC 1V to Heat
Typical Efficiency 91.6 97.5 98 85 85
Normalized Power 1.58 1.448 1.412 1.384 1.176 1
Best in Class Efficiency 96.3 98.9 99 93 88
Normalized Power 1.295 1.247 1.234 1.221 1.136 1
Best Efficiency Technology limit
96.3 98.9 99 95 95
Normalized Power 1.175 1.13 1.12 1.11 1.05 1
400V DC Or 48V NA NA Distribution Efficiency
AC – DC Rectifiers
12 VDC – 1VDC
Or 5Vdc – 1V1V to Heat
Efficiency :Isolated 99 93 95
Normalized Power 1.14 1.129 1.05 1
Efficiency No Isolated Rectifiers
99 95 95
Normalized Power 1.116 1.105 1.05 1
UPS System
High Voltage Distribution Bus
© 2004 IBM Corporation13 9/26/2008
Compute Capacity Increase due to DC Distribution and Power System Efficiency Increase
51%
10%
38%
1%
IT Power System Cooling Lighting
Increase in Compute Capacity in a Datacenter
Power Consumption in a Typical Datacenter
38%
23%
38%
1%
IT Power System Cooling Lighting
High Voltage Distribution Bus
© 2004 IBM Corporation14 9/26/2008
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Typical Best in Class Best Possible Isolated Nonisolated
Pow
er D
issi
pate
d (W
att)
DC - DC AC - DC Distribution PDU UPS
Power Saved by using DC Distribution and Efficient Power Supplies
DC DistributionAC Distribution
- 0.464W
Power Saved = 0.464W to power 1 Watt LoadTotal Electrical Utility Bill savings = 464000 Dollars/year for 1 MW IT load
High Voltage Distribution Bus
© 2004 IBM Corporation15 9/26/2008
N+1 vs. N+N Downtime (seconds) in a year
0
10
20
30
40
50
60
1+1 2+1 3+1 4+1 5+1 6+1 7+1 8+1 9+1Number of rectifiers
Dow
ntim
e in
Sec
onds
)
Downtime
( MTTR = 48 Hours, MTBF = 250K Hours
Availability (A) = 1 – MTTR/MTBF for a single Rectifier
Availability with N +1 Rectifiers in parallel = An+1 + (n+1) x An x (1-A)
High Voltage Distribution Bus
© 2004 IBM Corporation16 9/26/2008
N+1 vs. N+N Service Requirement Frequency
0.00%
0.05%
0.10%
0.15%
0.20%
0.25%
1+1 2+1 3+1 4+1 5+1 6+1 7+1 8+1 9+1Number of rectifiers
Ser
vice
Fre
quen
cy
Service Frequency
Service Frequency = 1 – An+1 MTTR = 48 Hours, MTBF = 250K Hours
High Voltage Distribution Bus
© 2004 IBM Corporation17 9/26/2008
Comparison of Different Power Architectures
Typical AC AC – Best in Class
AC – Best Possible
Isolated 48V Isolated / Non-isolated 400V DC
DC Redundancy At Server or Rack Level
At Server or Rack Level
At Server or Rack level
Central Rectifier Central Rectifier
Industry Standard Components
Yes Yes Yes Yes No
Customer Acceptance
High High High Acceptable Very Low
Battery Issue NA NA NA Low Failure High Failure due to cells in series
AC UPS required Yes Yes Yes No No
Neutral availability on Raised Floor
NA NA NA No Concern Not Required /Required
Repair Frequency High High High Low Low
Overall Cost Low at Power Supply level but high at Datacenter level
Low at Power Supply level but high at Datacenter level
Low at Power Supply level but high at Datacenter level
Low at Power Supply level but high at Datacenter level
High at Power Supply level but low at Datacenter level
Safety Concern No Concern No Concern No Concern No Concern Great Concern
High Voltage Distribution Bus
© 2004 IBM Corporation18 9/26/2008
Minimum Power Point Tracking Tool
Air Handler
UPS
Rectifiers
CRAC
Humidifier
Server Racks
MPP
Tracking Tool
High Voltage Distribution Bus
© 2004 IBM Corporation19 9/26/2008
Additional Energy Saving Techniques
% Energy Savings
16%
18%
23%11%
29%
3%
Low power Processors
Power Management
Server Virtualization
VFD Cooling
Supplemental Cooling
Cooling units work as ateam
Source: Emerson Network Power White Paper
High Voltage Distribution Bus
© 2004 IBM Corporation20 9/26/2008
Conclusion
Lower Overall Cost
No Step Down Transformers
No AC UPS
Less Feeds from Wall
Higher Reliability, Availability, and Scalability
Higher overall Efficiency
DC Distribution best suited for Solar Array Power
Less Repair frequency