Data Center Simulation Modeling must do
-
Upload
robert-schmidt -
Category
Technology
-
view
192 -
download
0
Transcript of Data Center Simulation Modeling must do
Modeling Must Do’s – Part I
• Proper Cooling Path Modeling for Centralized Distribution Systems• What aspects must be modeled when analyzing central distribution systems
modeled?• Rooftop delivery cooling units• Vertical plenum• Ducted delivery
• How is this different from a underfloor only cooling path?• What type of simulation results must be looked at for proper analysis?
• Proper Use and Analysis of Containment systems• Does containment always work?
• Does it need to be modeled?• Does containment affect the airflow delivery under the raised floor?
• What simulation results matter?• How can we improve a room using containment?
Modeling Must Do’s – Part IIMinimum CFD Inputs
• Accurate Representation of IT Devices• How does representation of air jets from high density servers, such as blades,
affect results?• How to model the affects of blanking plates and leakage through rails
• Buoyancy modeling• How much does buoyancy affect the flow field?
Modeling of Centralized Air Handling Schemes
What is a Centralized Cooling Distribution System?
Why are these being Used?
• Centralized cooling provides better redundancy – the air is mixed prior and after entering cooling infrastructure
• Cost • Less floor space required• Maintenance is away from computing space
• These scheme is different from raised floor perimeter cooling design• Plenum orientation is different• Exhaust path is different• Mechanical cooling is different
• For all these reasons – this must be modeled and analyzed properly!
What must be considered?
• The entire cooling path must be considered – from cooling air supply to heat load to exhaust path back to the cooling device• Penthouse or gallery design
• How are the units characterized?• How is the space characterized?
• Mixing chamber or plenum• Does the cooling air get mixed? This will distribute the cooling load
amongst the cooling units• How does the air enter the data center space? This can be with the use of
a raised floor or direct from the vertical plenum• How is this different from raised floor design?
• All these details must be taken into account! How?
Sample Central Air Distribution System
Penthouse Air Handlers
Vertical Plenum
Internal Wall Air Distribution
Equipment Racks on a Non-Raised
Floor
Complete Flow Path
Airflow Distribution Concerns
• How is the pressure/airflow distributed in the vertical plenum?
• How is the air distributed coming out of the vertical plenum?
• What consequences (symptoms) are a result of certain design choices
Design Variations
24 in
32 in
• Vertical plenum acts similarly to a raised floor plenum
• Delivers air from cooling units above to equipment on the ground
• 2 Variations are being modeled
• 24 inches versus 32 inches
• Typically, air distributes better in a higher raised floor – does this apply to vertical plenums?
Pressure Distribution – PLENUM WIDTH CALLOUT
• Different pressure profiles exist going from the plenum into the computing space
• The wider plenum distributed pressure more evenly
• Similar to a higher raised floor distributes perforated tile air more evenly
• Simulation must be used to determine an adequate width of the vertical plenum
• How wide is wide enough?
Vent Velocity Plot
• Different discharge profiles can be seen between the two different designs
• Discharge profiles are dependent on the plenum design and NOT uniform
• The wider plenum has a more even distribution from top to bottom of the vent
• Simulation must be used to determine an adequate width of the vertical plenum and also provides a platform to design the distribution vent
• Similar to raised floor depth and perforated tile / grate type deign
Vertical Velocity Plot – MAKE A MORE EASY TO DIFFERENTIATE IMAGE
• Vertical slice through the center of the cold aisle shows the velocity profile
• The two different plenums create a different velocity profile in the cold aisle
• Simulation shows the affect of the gallery width to the airflow in the cold aisle
Velocity – SCALE
• High Velocity air (in excess of 1200 fpm) penetrates into the cold aisle more in the case with the more narrow vertical plenum
• Cross flow in excess of 800 fpm is NOT RECOMMENDED for computer room design
• Simulation is necessary to evaluate the affect of the vertical plenum on cross flow velocity
Temperature
• Hot spots develop in both cases
• The case with the wider vertical plenum develops a smaller hot spot
• These are symptoms of the overall issues with the delivery system – the plenum design must be examined more
Centralized Distribution System Must Do
• The delivery path must be analyzed from the air handler down to the equipment and back
• The entire cooling path must be considered and modeled properly• Proper penthouse layout• Proper vertical plenum dimensions• Entry of air into underfloor/datacenter space• Air passing by server inlets• Return of exhaust air
• Critical variables are all important• Pressure distribution in the vertical plenum• Velocity of cross flow in front of the computing equipment• Temperature of computing space and server inlet air
Modeling of Containment Solutions
Various Applications of Containment
• Containment is being implemented for various reasons• Energy efficiency• Mitigating risk in critical facilities• Creating high density computing areas• Reducing server temperatures
• How has containment been implemented?• Retrofits for hot rows or high density rows• Overall upfront data center design
Uncontained Hot Aisle / Cold Aisle Distribution
ASHRAE Temperature Compliance
Two Areas of Concern
Containment Modeling
ADD PRESSURE SLIDE PRIOR TO THIS - Containment Modeling
• Containment worked in the local area it was intended for
• Containment took racks out of ASHRAE compliance at uncontained locations
• Modeling must be done and the entire room must be looked at for proper localized containment installations
Full Containment Design
• What happens if a design utilizes full containment of cold aisles?• Will containment always work?• What must be looked at to ensure a good design?
Case Specifications
• The following is a case of the same room with more challenging design envelopes• High flow servers
• 180 cfm/kW• High efficiency room – not a lot of excess cooling
Full Cold Aisle Containment
Uncontained Contained
Temperature Differences
Uncontained Contained
Where is the Hot Air Coming From?
• Can we just contain everything and seal the leakages?• Fundamental issues in the room WON’T go away with sealing and
containment
Full Containment
• Why is this room having problems?
Pressure Analysis
• Full cold aisle containment is similar to extending the raised floor plenum to the level of the floor• Pressure MUST be looked at!• Post-processing/Plotting MUST be looked at above the raised floor
Lower Pressure in Cold Aisle
versus Hot Aisle
Local Fix to Increase Pressure in Problem Aisle
• Since the fundamental issue is lack of pressurization, fixed can be done to increase pressurization in problem aisle• Floor tiles can be swapped out to grates in low pressure aisle
• In the following analysis, only this first remedy was tried. Other fixes can be used as well:
• Increasing flow in local area• Sealing leakages• Moving CRAC’s to more evenly distribute flow and pressure
Pressure Differences
• Changing out some of the perforated tiles creates a much better pressure situation in the contained cold aisle
Contained Contained with Local Fixes
ASHRAE Compliance
• Fixing the fundamental pressure issue also fixes the temperature issues
Contained Contained with Local Fixes
Proper Containment Modeling
• Not all containment is the same
• Freezer curtains, rigid partitions, various assembly schemes affect the effectiveness of containment
Proper Containment Modeling
Idealized (no leakage)
Containment
Containment Must Do’s
• Effect of containment on the entire datacenter MUST be analyzed• Local containment can negatively affect other areas of data center• Both the contained and uncontained areas must be looked at
• When using containment, pressure MUST be analyzed• Room level pressure is an indication of how effective a containment system can
be• Identifying low room level pressure allows the designer to look at local design
solutions and fixes
• Containment type MUST be characterized and modeled like they would exist in a real installation
• Appropriate leakage must be looked characterized and applied• Realistic gaps must be modeled
Minimum CFD Inputs for ANY Data Center Model
Capturing Exhaust Jets
“Simple CFD” – • Little or no capability to model multiple
outflows – lumped outflow is best way of modeling…
• The outflow for this model is same height and width of the device
RoomLite - Outflows are modeled as per specification i.e. width and height
Thermal Footprint – ADD TABLE TO SHOW DIFFERENCE IN AREA AND VELOCITY
Both servers are the same power density and same exhaust flow volume but have a different thermal footprint
Sample Room
• 7 Racks with IBM Blade Center Chassis
• 2’ Raised Floor• Single ACU• 30% Perforated Grills
Result of Modeling Details
With Accurate Thermal Footprint of Blade Center Fans
With a Lumped Exhaust from Blade Center
Demonstration of Ducted Delivery and Buoyancy
• Overhead ducted Air delivery system
• Return Vent in hot aisle
• 3.5 kW per cabinet
Demonstration of Ducted Delivery and Buoyancy
Gravity Modeled Gravity Ignored
Buoyancy driven flow not captured when Gravity Ignored. Hot air seems to sink!
Demonstration of Ducted Delivery and Buoyancy
Gravity Modeled Gravity Ignored
Major Variation in Airflow Paths
Demonstration of Ducted Delivery and Buoyancy
Gravity Modeled Gravity Ignored
Results in Major Variation in Cabinet Inlet Temperature