Microsoft’s Cloud Server Specification Design
Kushagra Vaid General Manager, Cloud Infrastructure Server Engineering Microsoft
2
2.4+ million emails per day
5.8+ billion worldwide queries each month
250+ million
active users
8.6+ trillion objects in Windows Azure
storage
1 in 4 enterprise customers
50+ billion minutes of connections
handled each day
48+ million users in 41
markets
50+ million
active users
200+ Cloud Services 1+ billion customers · 20+ million businesses · 90+ markets worldwide
400+ million
active accounts
Chicago
Cheyenne
Dublin Amsterdam
Hong Kong
Singapore
Japan
San Antonio
Boydton Shanghai
Quincy
Des Moines
Brazil
Australia
Huge infrastructure: scale is the enabler Microsoft has datacenter capacity around the world…and we’re growing
35+ factors in site selection: § Proximity to customers § Energy, Fiber Infrastructure § Skilled workforce
Server scale implications: Design
<10K SMB/Enterprise
100K Hosters
1M Cloud-Scale
# SKUs Several Limited Extremely limited
Redundancy model Hardware based
(Hot-*) Software based
(Local datacenter) Software based
(Geo-distributed)
HW availability 99.999% or higher 99.9% - 99.999% 99% - 99.9%
HW type Enterprise SKU Off-the-shelf design, custom integration
Custom designs, custom integration
Infrastructure co-design
None Limited integration
with Datacenter and Network
OS, Datacenter, Server and Network tightly
integrated
Server scale implications: Supply Chain
<10K SMB/Enterprise
100K Hosters
1M Cloud-Scale
Deployment unit Individual servers 100-1000 servers >10,000 servers
Procurement process
EA / RFP RFP Direct component
sourcing
Materials management
N/A Supplier managed Consignment
Manufacturing N/A Supplier managed Contract
manufacturing
HW quality N/A Supplier managed Self-warranty
Server scale implications: Operations
<10K SMB/Enterprise
100K Hosters
1M Cloud-Scale
Break/fix support 24 hours x 7 days 8 hours x 5 days Up to 1-2 weeks
Issue triage model IT admin Some automation,
admin support Fully automated, machine learning
OOB HW management
Full command set, BMC required
Basic feature set, BMC required
Power On/Off only, No BMC
Management domain scale
100’s of servers 1000’s of servers 10’s of 1000’s
of servers
FRU granularity Hot-swappable
components Component
replacement Entire server replacement
Microsoft cloud server spec: Goals
§ Component re-use
§ Targeted technology refresh
§ Environmental sustainability
Decommission Architecture & Design
§ Commoditized
§ Modular with feature simplicity
§ Power efficient
§ Technology innovation
§ Minimum materials use
§ Low cost mass production
§ Streamlined assembly
§ Industry standard components
Manufacturing & Supply Chain
§ Scalable systems management
§ HW and SW security
§ Reduced operator errors
§ Diagnostics and self-healing
Management & Operations
Microsoft cloud server spec: Infrastructure
▪ Standardization & Modularization
▪ Design Simplicity
▪ Operations Excellence
Key features
Shared infrastructure for efficiency and TCO optimization ▪ Power delivery, mechanicals,
thermals/cooling, management
▪ Optimized for mass contract manufacturing and assembly
▪ Up to 40% cost savings and 15% power efficiency benefits
▪ Saves 10,000 tons of metal per one million installed servers
Compute blade
Signal backplane
Shared power
Shared management
JBOD expansion
Shared fans
12U Shared Chassis EIA Rack Mountable
Blind-mated connectors (12V Power, Ethernet, SAS,
Management)
Signal Backplane
Key features
Blind-mated signal connectivity ▪ Decoupled architecture for server
node and chassis enabling simplified installation and repair
▪ Cable free design for significantly fewer operator errors during servicing
▪ Up to 50% improvement in deployment and servicing time
Key features
Network and storage cabling via backplane architecture ▪ Architectural flexibility for multiple
network types ▪ One-time cable install at factory
▪ No cable touch needed during production operations
▪ Saves 1,100 miles of cable for a deployment of one million servers
2x4 SAS 6G 2x10Gbe SPF+
Key features
Secure & scalable systems management
REST API for chassis infrastructure ▪ TPM boot, SSL transport and role-based
authentication built-in ▪ Simplified server management via extremely
small subset of IPMI commands ▪ Support for server diagnostics and
self-health checks ▪ Up to 75% improvement in operational agility
Management card
Microsoft cloud server spec: OCP contribution Specifications
Chassis, Blade, Chassis Manager, Mezzanines, Management APIs
Mechanical CAD Models Chassis, Blade, Chassis Manager,
Mezzanines
Source Code Chassis management source code through Open Source
/// <summary> /// Gets Fan speed in RPM /// </summary> /// <param name=“fanId">target fan Id</param> /// <returns>Fan speed in RPM</returns> internal FanSpeedResponse GetFanSpeed(byte fanId) {
Board Files & Gerbers Chassis Manager, Tray Backplane,
Power Distribution Backplane
More information: Technical breakouts
Technical Workshop Presenter
Hardware Overview Mark Shaw, Director
Management Software Overview Badriddine Khessib, Director
Blade Overview – Compute & Storage Martin Goldstein, Principal Systems Architect
Chassis Manager Hardware Overview Bryan Kelly, Senior Platform Software Engineer
Visit the Microsoft booth for live demos
Microsoft datacenter resources
Microsoft Datacenters Web Site & Team Blogs www.microsoft.com/datacenters
Windows Azure http://www.windowsazure.com
Office 365 http://www.office365.com
Q & A
Thank you!
© 2014 Microsoft Corporation. All rights reserved. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
© 2014 Microsoft Corporation. All rights reserved. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
© 2014 Microsoft Corporation. All rights reserved. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
Top Related