Post on 27-Jun-2020
!1
100%Warm-WaterCoolingwithCoolMUC-3Nochillers,nofans,noproblems…
AxelAuweter–MEGWARECTO WorkshoponEnergyEfficiencyinHPC
May30,2018,Ljubljana
• Providerofturn-keysolutionsforcompute,
storage,andacceleratorclusters
• Morethan1,200HPCdeploymentssince1990
• ̴50employees
• StrongfocusonR&D
• Morethan8yearsexperienceinliquidcooling
technologiesforenergyefficiency!
MEGWAREComputerVertriebundServiceGmbHCompanyOverview
!2
AJointVision...…forGreenDataCentersoftheFuture
!3
TechnologyPartners
EarlyAdopters
•
•
•
Chiller-FreeCooling…forimprovedenergyefficiencyratio(EER)
!4
Vapor-CompressionRefrigeration Chiller-FreeCooling
14ºC
10ºC
45ºC
40ºC
Compressor
ExpansionValve
CondenserEvaporator
HeatExchanger
DataCenter OutsideAir DataCenter OutsideAir
• ASHRAEspecifies
designpointsfor
coolingequipment
• Useofpercentiles:
specifiedvalueis
reachedor
exceededforatotal
of3hours(0.4%),7
hours(1%),or14
hours(2%)
TemperatureDesignPointChartsLjubljana,Slovenia
!5
0
10
20
30
40
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
0.4%Dry-Bulb 1%Dry-Bulb 2%Dry-Bulb
• ASHRAEspecifies
designpointsfor
coolingequipment
• Useofpercentiles:
specifiedvalueis
reachedor
exceededforatotal
of3hours(0.4%),7
hours(1%),or14
hours(2%)
TemperatureDesignPointChartsLjubljana,Slovenia
!6
0
10
20
30
40
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
0.4%Wet-Bulb 1%Wet-Bulb 2%Wet-Bulb
ThermalConductivitydependson:• ΔTbetweenheatsourceandsink
• Geometry(transfersurfaceA&distanced)
• Material
ThermalCapacitydependson:• Mass
• Material
TransferringHeat…intheoryandpractice!
!7
HeatSource
HeatSink
TransferMaterial
d
A
max. 7 K in Ljubljana (for air-cooling)
Air (0.0026 W/mK)
HeatSource
HeatStorage
Air (1.005 J/kgK)
Water (0.598 W/mK)
Water (4.128 J/kgK)
Copper (401 W/mK)
=1.2 kJ/m3K=4178 kJ/m3K
Aluminum (237 W/mK)
• Goingtowater-coolingsignificantly
facilitateswaste-heatreuse
• Highertemperaturesenablemoreoptions
WasteHeatReuse…you’vealreadypaidforthepower-useit!
!8
DistrictHeating >70ºC
HouseholdRadiator 60ºC
UnderfloorHeating 35ºC
AbsorptionChiller >70ºC
AdsorptionChiller >50ºC
MEGWARECooLMUCSupercomputer(2012)withSorTechACS08adsorptionchiller.
PhotocourtesyofLeibnizSupercomputingCentreoftheBavarianAcademyofSciencesandHumanities.
PowerBreakDownofaTypicalComputeNode…accordingtocomponentmaxTDPs
!9
Others
PSU
Interconnect
Mainboard
MemoryCPU&VRs
BasedonMegwareDualSocket“Skylake”6140,192GBRAM,Omni-PathHFI
EvolutionofMEGWAREColdCon®DirectLiquidCoolingsince2011
!10
ColdCon®2011
Liquidcoolingfor • CPUs • VRs
Maxinlettemp:40°C
ColdCon®2013
Liquidcoolingfor • CPUs • VRs • DIMMs
Maxinlettemp:40°C
ColdCon®2015
Liquidcoolingfor • CPUs • VRs • DIMMs • I/O
Maxinlettemp:50°C
ColdCon®2017
Liquidcoolingfor • CPUs, VRs • DIMMs • I/O • Powersupplies
• NetworkSwitches Maxinlettemp:55°C
• Upto10nodeswithyourchoiceof- Intel®Xeon®Scalable
- Intel®Xeon®Phi™x200
- Intel®Xeon®E5v4
• Highperformancenetwork- Intel®Omni-PathFabric
- IBFDR,EDR
• Integratedchassismanagement
• Hotpluggable,easymaintenance
MEGWARESlideSX-LCFullyintegratedchassissolutionfordirectliquidcooling!
!11
MEGWAREColdCon®…forComputeNodes
!12
• HighPerformanceHeatSink- e.g.CPUs
• DirectAttachCopper- e.g.DIMMCooling
• ColdPlate- e.g.NICs
• PassiveCooling- Yourmainboardconductsheataswell!
• 1620WperPSU
• Upto5PSUsperSlideSXChassis
• 80PlusPlatinumCertified
• LiquidCoolingwithhot-plugsupport
MEGWAREColdCon®…forPowerSupplyUnits
!13
• RefitofIntel®OPAswitch48-portfor
ColdCon®directliquidcooling
• 12VDCpowersupplyviaMEGWARE
liquidcooledpowerbank
MEGWAREColdCon®…forOmni-PathHighPerformanceNetworkSwitches
!14
• 15MEGWARESlideSX-LC®chassis
• 148computenodeswithIntel®Xeon®Phi™Processor7210F
• Dual100GBpslinkstoIntel®Omni-Pathfabric
• MEGWAREColdCon®forpowersupplies
• MEGWAREColdCon®forIntel®OPAswitch
• MEGWAREEnergyMeterforhigh-frequencypowerand
energymeasurements&Intel®NodeManagerintegration
• Thermalinsulationforracks
• Lessthan3%heatlosstocomputerroomair
TheNewColdCon®inActionCooLMUC-3Supercomputer@LeibnizSupercomputingCentre
!15
• Year-roundchiller-lesscoolingfor
theentiresystem
• Highoutlettemperaturesfor
optimalheat-reuse
• Insulatedracksforclosetozero
heatemissiontoroomair
• Additionalpowersavingsdueto
removaloffans
100%Warm-WaterCooling…whyyouneedit,too…!
!16
!17
100%Warm-WaterCoolingwithCoolMUC-3Nochillers,nofans,noproblems…
Thankyou!