CC-4153, Verizon Cloud Compute and the SM15000, by Paul Curtis
-
Upload
amd-developer-central -
Category
Technology
-
view
336 -
download
1
description
Transcript of CC-4153, Verizon Cloud Compute and the SM15000, by Paul Curtis
VERIZON CLOUD COMPUTE AND THE AMD SEAMICRO 15000
Paul Curtis Chief Architect Verizon Cloud Compute APU13
2
Collaboration
• Unbelievable amount of support from Seamicro
3
Verizon cloud development goals
• Very few different hardware components
• Consistent predictable performance
• Secure
• High performance
• Highly available
• No modification to customer applications
• No special purpose hardware
4
Verizon Cloud Differentiation
• Value for Performance – User defined availability and performance
– User defined resources
• Reserved Performance – Network, Storage and Compute
• Workload Simplicity – Seamless integration with other deployments
– Single point of control
• Security ‒ Market leading security capabilities
‒ Embedded into every aspect of platform
• Continuum of Services ‒ Bridging private, public and hybrid clouds
‒ Allow the blending with colocation, managed services, networking
5
SM15000 SYSTEM 10 Rack Units, draws 3-3.5 KW
! Compute – Up to 512 Opteron, Xeon or Atom cores in 10 RU – 2,048 cores in a rack – Up to 64GB DRAM/socket = 4 terabytes/system
! Networking – 10 Gbps half duplex bandwidth to each CPU socket – 16 x 10GbE Line Rate uplinks to the network
! Storage – Up to 1,408 disks: HDD or SSD – Up to 128 Terabytes of internal SSD storage – Up to 5.3 Petabytes of storage
! Fabric – 1.28 Tbps Freedom Supercompute Fabric
! Software – Off the shelf OS, Hypervisors
6
Hardware architecture
• There are only three hardware component types. This simplifies maintenance
– Arista 7508 a 384 port x 10Ge non-blocking L2 switch
– AMD Seamicro SM15000
– SSDs
• Network connections
7
Hardware diagram
Juniper MX960 for external connectivity
Arista 7508
Up to 90 AMD Seamicro SM15000s
4x10Gb links from arista to each chassis
8
Verizon’s use of seamicro chassis
• 160 GB of external bandwidth (network and storage)
• 54 Server cards for customer loads
• 2 Server cards for Verizon orchestration
• 8 Server cards for storage services
• ~1000000 IOPs
• 96 T usable SSD storage
9
Combine Hardware and Verizon software to get
• A flat layer 2 ethernet switch – ~12000 port 1 Gb/sec – ~1500000 vlans – 8.5M mac address table entries – 11.5M traffic flows. – Software configurable
• A storage array – 90M IOPs – 8.6 PB of SSD storage
• Scalable router firewall 1Gb- 400Gb/sec • Scalable load balancers 1Gb-400Gb/sec • Configurable IO performance
10
Network Packet flow VM
Queue
Nic
text
Queue
Nic
Hypervisor
NPU
10 G NIC
Queue
Hypervisor presents nic of specified speed to VM. Back pressure applied by hypervisor
Hypervisor fairly mixes flows from
different VMs
Prioritized queue.
Arista Switch
Queue
Queue
10 G NIC
Shaped to max speed
of receiving
nic
Shaped to max speed of receiving
Nic
Policer
VM
Queue
Nic
text
Queue
Nic
Hypervisor
Random packet
drop back pressure form
destination queue
Limited to max Nic
speed
NPU
Layer 2 Switching
Layer 2 Switching
11
Networking Layer 2
• Hypervisor – Shapes egress traffic
• NPU – Provides true layer 2 ethernet switching – Polices ingress flows – Shapes egress flows
• Arista 7508 – Lots of bandwidth
• Remote congestion control – Switch learns speeds of remote flows – Switch performs remote drop if destination is congested
• Hardware based security – Each customer network is on its own vlan
• Software configurable
12
o
o
VM
Paravirt (xvdb) Storage VLAN AoE Initiator
HV
Ethernet NPU
NPU
Ethernet AoE Target
AoE Initiator Block ZFS AIO
Ethernet
Ethernet
Block Device(s) AoE Target
SCARD
SS N
Arista
Data I/O Stack
13
Storage
• Hypervisor – Shapes disk traffic (IOPs and Bandwidth) – Participates in disk replication
• AoE – Storage over layer 2 ethernet – Allows storage targets to be any where in world – Shared volumes
• Replication • NPU
– Shapes read and write bandwidth • Storage Service
– Snap shots – Raid
• Storage card – AoE target
14
Networking Layer 3+
Layer 3 and above network services just work since they are all based on layer 2 networking.
• Soft routers • Load balancers • Public IP (No Nat) • Tunnels • Wan optimizers
15
Inter-data center features
• Single user interface
• Networks can span multiple data centers
• Replicated disks can span multiple data centers
• Taking advantage of being part of a network company
16
Availability
• No single point of failure for network traffic – “Bonded nics” – “Bonded NPUs” – Fabric reroutes itself – Multiple paths through arista switches
• No single point of failure for replicated storage – Raid 1 on SSDs – Multiple storage servers – Option to have replicated volumes span data centers
17
Security
• Physical security
• DDOS
• Network security – Customer traffic on independent VLANs – Untrusted entities (Hypervisors) firewalled from rest of system
• Storage security – Each volume on a separate vlan – Storage vlans firewalled (only AoE traffic, no target to target traffic)
• Management software – Audit logs – Security alerts
18
Possible Applications
• Move a current three tier app with your choice of soft router/firewall/load balancer into the cloud
• Bridge a network from your data center to one in the cloud • Move XEN and VMWare VMs into the cloud without modification • Write a clustered app using shared storage • Configure an applications performance so that you know it won’t fall over
when it is 3:00 in the afternoon and the cloud gets busy • Write and test a new L3 protocol • Voice • Storage arrays • Network devices