.ppt
-
Upload
networkingcentral -
Category
Documents
-
view
503 -
download
0
description
Transcript of .ppt
![Page 1: .ppt](https://reader036.fdocuments.in/reader036/viewer/2022081413/546e11dcaf7959f83e8b5eed/html5/thumbnails/1.jpg)
Architecture for Modular Data Centers
James Hamilton2007/01/17
[email protected]://research.microsoft.com/~jamesrh/
![Page 2: .ppt](https://reader036.fdocuments.in/reader036/viewer/2022081413/546e11dcaf7959f83e8b5eed/html5/thumbnails/2.jpg)
• 15 years in database engine development– Lead architect on IBM DB2– Architect on SQL Server
• Have lead most core engine teams: Optimizer, compiler, XML, client APIs, full text search, execution engine, protocols, etc.
• Lead the Exchange Hosted Services Team– Email anti-spam, anti-virus, and archiving for 2.2m seats with $27m
revenue– ~700 servers in 10 world-wide data centers
• Currently Architect on the Windows Live Core team• Automation & redundancy is only way to:
– Reduce costs– Improve rate of innovation– Reduce operational failures and downtime
Background and Biases
1/17/2007 2
![Page 3: .ppt](https://reader036.fdocuments.in/reader036/viewer/2022081413/546e11dcaf7959f83e8b5eed/html5/thumbnails/3.jpg)
Commodity Data Center Growth• Software as a Service
– Services w/o value-add going off premise• Payroll, security, etc. all went years ago
– Substantial economies of scale• Services at 10^5+ systems under mgmt rather than ~10^2
– IT outsourcing also centralizing compute centers• Commercial High Performance Computing
– Leverage falling costs of H/W in deep data analysis– Better understand customers, optimize supply chain, …
• Consumer Services– Google estimated at over 450 thousand systems in more than 25 data centers (NY Times)
• Basic observation:– No single system can reliably reach 5 9’s (need redundant H/W with resultant S/W complexity)– With S/W redundancy, most economic H/W solution is large numbers of commodity systems
1/17/2007 3
![Page 4: .ppt](https://reader036.fdocuments.in/reader036/viewer/2022081413/546e11dcaf7959f83e8b5eed/html5/thumbnails/4.jpg)
An Idea Whose Time Has Come
1/17/2007 4
Nortel Steel EnclosureContainerized telecom equipment
Sun Project Black Box242 systems in 20’
Rackable Systems1,152 Systems in 40’
Rackable Systems Container Cooling Model
CaterpillarPortable Power
![Page 5: .ppt](https://reader036.fdocuments.in/reader036/viewer/2022081413/546e11dcaf7959f83e8b5eed/html5/thumbnails/5.jpg)
Shipping Container as Data Center Module• Data Center Module
– Contains network gear, compute, storage, & cooling– Just plug in power, network, & chilled water
• Increased cooling efficiency– Variable water & air flow– Better air flow management (higher delta-T)– 80% air handling power reductions (Rackable Systems)
• Bring your own data center shell– Just central networking, power, cooling, & admin center– Grow beyond existing facilities– Can be stacked 3 to 5 high– Less regulatory issues (e.g. no building permit)– Avoids (for now) building floor space taxes
• Meet seasonal load requirements• Single customs clearance on import• Single FCC compliance certification1/17/2007 5
![Page 6: .ppt](https://reader036.fdocuments.in/reader036/viewer/2022081413/546e11dcaf7959f83e8b5eed/html5/thumbnails/6.jpg)
Unit of Data Center Growth• One at a time:
– 1 system– Racking & networking: 14 hrs ($1,330)
• Rack at a time:– ~40 systems– Install & networking: .75 hrs ($60)
• Container at a time:– ~1,000 systems– No packaging to remove– No floor space required– Power, network, & cooling only
• Weatherproof & easy to transport• Data center construction takes 24+ months
– Both new build & DC expansion require regulatory approval
1/17/2007 6
![Page 7: .ppt](https://reader036.fdocuments.in/reader036/viewer/2022081413/546e11dcaf7959f83e8b5eed/html5/thumbnails/7.jpg)
Manufacturing & H/W Admin. Savings• Factory racking, stacking & packing much more efficient
– Robotics and/or inexpensive labor• Avoid layers of packaging
– Systems->packing box->pallet->container– Materials cost and wastage and labor at customer site
• Data Center power & cooling expensive consulting contracts– Data centers are still custom crafted rather than prefab units– Move skill set to module manufacturer who designs power & cooling once– Installation design to meet module power, network, & cooling specs
• More space efficient– Power densities in excess of 1250 W/sq ft– Rooftop or parking lot installation acceptable
• Service-Free– H/W admin contracts can exceed 25% of systems cost– Sufficient redundancy that it just degrades over time
• At end of service, return for remanufacture & recycling– 20% to 50% of systems outages caused by Admin error (A. Brown & D. Patterson)
1/17/2007 7
![Page 8: .ppt](https://reader036.fdocuments.in/reader036/viewer/2022081413/546e11dcaf7959f83e8b5eed/html5/thumbnails/8.jpg)
DC Location Flexibility & Portability• Dynamic data center– Inexpensive intermodal transit anywhere in world– Move data center to cheap power & networking– Install capacity where needed– Conventional Data centers cost upwards of $150M &
take 24+ months to design & build– Political/Social issues
• USA PATRIOT act concerns and other national interests can require local data centers
• Build out a massively distributed data center fabric– Install satellite data centers near consumers
1/17/2007 8
![Page 9: .ppt](https://reader036.fdocuments.in/reader036/viewer/2022081413/546e11dcaf7959f83e8b5eed/html5/thumbnails/9.jpg)
Systems & Power Density• Estimating DC power density hard
– Power is 40% of DC costs• Power + Mechanical: 55% of cost
– Shell is roughly 15% of DC cost– Cheaper to waste floor than power
• Typically 100 to 200 W/sq ft• Rarely as high as 350 to 600 W/sq ft
– Modular DC eliminates the shell/power trade-off• Add modules until power is absorbed• Data Center shell is roughly 10% of total DC cost
• Over 20% of entire DC costs is in power redundancy– Batteries able to supply 13 megawatt for 12 min– N+2 generation (11 x 2.5 megawatt)– More smaller, cheaper data centers
• Eliminate redundant power & bulk of shell costs
1/17/2007 9
![Page 10: .ppt](https://reader036.fdocuments.in/reader036/viewer/2022081413/546e11dcaf7959f83e8b5eed/html5/thumbnails/10.jpg)
Where do you Want to Compute Today?
1/17/2007 10Slides posted to: http://research.microsoft.com/~jamesrh/