1© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
Server Networking eVirtual Data Center
Roma, 8 Febbraio 2006
Luciano Pomelli
Consulting Systems [email protected]
2© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
Typical Compute Profile ata Fortune 500 Enterprise
Compute Infrastructure Challenges:• Proliferation of disparate server platforms. 1200 servers
across 12 data centers and 200 branches- 35% Intel/Windows, 15% Intel/Linux 20% SUN Solaris, 10 HPUX, 20% IBM AIX
• Low average CPU utilization—Windows and Linux (15%)UNIX (30%)
• Continued use of Mainframes. High hardware and softwaremaintenance costs
• Need for lower cost data analysis compute and storageresources
• Need faster server deployments for research projects andapplication development teams
• Piloting Blade Servers—challenges with cooling, server I/O,network connectivity
3© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
Server Optimization“Scale Up” or “Scale Out”
Consolidate several mid-sized systems with fewer large servers
Evolve mid-sized systems to a system of standardized servers (blades, 1RU, small form-factor)
Scale Up
Scale Out
4© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
Industry Trend Towards Server “Scale-out”
Server Revenue Mix
0%
2%
4%
6%
8%
10%
12%
14%
16%
18%
$0-2.9K $3-5.9K $6-9.9K $10-24.9K
$25-49.9K
$50-99.9K
$100-249.9K
$250-499.9K
$500-999.9K
$1M-3M $3M+
Price Band
Shar
e of
Rev
enue
s
Scale Out Scale Up
Server Virtualization
GridComputing
HPCClusters
BladeServers
Scale out servers are low-cost, 1 rack unit (RU) orblade servers purchased as needed.
5© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
Cisco Computing Networking strategy
ServerClustering
• High PerformanceComputing (HPC)
• “Enterprise-class” HPC• Database scalability
Utility or GridComputing
• Applicationprovisioning
• Server Virtualization• Data Center
Virtualization
VFrame™
Performance
Economics
6© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
High Performance Computing e ClusteringNetworking
7© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
Cluster types
• High Availability Clusters
• Application Clusters (Load-balancing)
• Parallel Processing Clusters
• High Performance Clusters
Bio-molecular
Computational Fluid Dynamics
Data Mining
Financial Analysis
• Enterprise Clusters
On Line Transaction Processing (OLTP)
Decision Support System (DSS)
Oracle, IBM DB2, MySQL, Microsoft SQL, Exchange
8© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
High Performance Clusters
Cluster Topology Cluster Components
9© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
Performance impact
Latenza end-to-end dei messaggi
Utilizzo della CPU
10© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
What is InfiniBand?
• InfiniBand is a high speed – low latency technologyused to interconnect servers, storage and networkswithin the datacenter
• Standards Based – InfiniBand Trade Associationhttp://www.infinibandta.org
• Scalable Interconnect based on Link Speed and LinkAggregation:
120603012X
4020104X
10 Gb/s5 Gb/s2.5 Gb/s1X
QDRDDRSDRLinks
11© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
InfiniBand architecture
CPU
CPU
Hos
t Int
erco
nnec
tMemCntlr
SystemMemory
IB L
ink
HCA
SM
Switch
IB Link TCA
IB Link TCA
Ethernet linkIB Link
HCA – Host Channel AdaptorSM - Subnet managerTCA – Target Channel Adaptor
(gateway)
End-node
Network
LAN
Infiniband Storage
12© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
IB Technology: Kernel Bypass
Hardware
Application
Traditional Model Kernel BypassModel
KernelUser
TCP/IPTransport
Driver
Hardware
Application
Kernel
User
TCP/IPTransport
Driver
RDMA ULP
SocketsLayer
SocketsLayer
13© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
RDMA and OS bypass advantage
Data traverses bus 3 timesCPU involved in Data Movement
Direct Access from HCA to ApplicationbufferData traverses bus 1 timeCPU NOT involved in Data Movement
IPC using TCP over Ethernet IPC using uDAPL or SDP over IB
14© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
InfiniBand PerformanceMeasured Results
BSD Sockets Async I/O extension
Application
1GE
DirectAccess
IPoIB
TCPIP
SDP
10G IB
uDAPLSRP MPI
5 usec8 usec18 usec18 usec30 usec40-60 usecLatency
8 Gb/s8 Gb/s7.9 Gb/s4.5 Gb/s4.1Gb/s1-3 Gb/sThroughput
Transparent Custom / Enhanced Performance
15© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
Unmatched Price / Performance
$100
$2K-$6K
$2K-$5K
50us
900MB/s
10GigE
$100-$300$400$400$250Switch Port
$175
$880
5.7us
495MB/s
Myrinet E
$25$175$100Cable Cost(3m Street Price)
Free$535$550HCA Cost(Street Price)
50us6.5us5usMPI Latency(Small Messages)
100MB/s245MB/s950MB/sData Bandwidth(Large Messages)
GigEMyrinet DInfiniBandPCI-Express
InfiniBand Offers the Best Price / Performance for HPC
•Myrinet pricing data from Myricom Web Site (Dec 2004) utilizing Myrinet’s new switch** InfiniBand pricing data based on Topspin avg. sales price (Dec 2004)*** Myrinet, GigE, and IB performance data from June 2004 OSU study****10GigE and GigE Cost and Performance data from Cisco Internal document • Note: MPI Processor to Processor latency – switch latency is less
16© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
Oracle RAC and Cisco Infiniband example
InfinibandServer Fabric
Switching
Cluster Storage
DataVisualization
Catalyst 6500NAS
ORACLE Application Servers
OracleNet:over SDP
EthernetGateways
ORACLE RAC
IPC over uDAPL over IB
17© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
Oracle 10G Infiniband linear scalabilityHorizontal Database Clusters
• Advantages of InfiniBand over Gigabit Ethernet interconnect2 to 4 times cluster performance improvementMuch higher application scalability
Source: Oracle whitepaper: Achieving Mainframe-ClassPerformance on Intel Servers Using InfiniBand Building Blocks
0
5000
10000
15000
20000
25000
30000
2-node cluster 4-node cluster
Gigbit Ethernet
InfiniBand
ORACLE CLUSTER SCALABILITY
2X performanceimprovement in2-node cluster
3 to 4X performanceimprovement in4-node cluster
BlockTransfersPer Second
18© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
The Cisco SFS Product LineSe
rver
Fab
ricSw
itch
Bla
deSe
rver IBM BladeCenter
• HCA (2) 1XIB PCI-X• Embedded switch (14) 1XIB (Internal) +
(1) 4XIB and (1) 12XIB (External)
HC
A
SFS 7000 (TS120)
(24) 4XIB
SFS 7008 (TS270)
SFS 3012 (TS360)
(24) 4XIB + 12 Gws
SFS 3001 (TS90)
(12) 4XIB + 1 Gw
(6) GE Gateways
(96) 4XIB
• (2) 4XIB PCI-X• (2) 4XIB PCI-ex
• Remote Boot• Linux Host Driver• Windows Host Driver
Infin
iBan
dM
ultif
abric
Dell 1855• HCA (2) 4XIB PCI-ex• Passthru Module (10) 4XIB
19© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
Blade 2
Blade 1
Blade 3
Blade 14
BladeCenter #1BladeCenter #2
Blade Server Infiniband connectivity
40 Gbps
40 Gbps
80 Gbps
2.5 Gbps
2.5 Gbps
2.5 Gbps
2.5 Gbps
2.5 Gbps
2.5 Gbps
2.5 Gbps
2.5 Gbps
Cisco (IB)
Cisco (IB)
Infiniband StorageEg: IBM DS4000IP Network
Cisco SFS
20© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
Case Study: Large Wall Street BankEnterprise Grid Computing
• Application:Replace proprietary platforms withstandards-based componentsBuild scalable “on-demand” compute gridfor financial applications
• Environment:500+ Intel Servers per sliceTopspin Server Switch with Ethernet andFibre Channel GatewaysHitachi RAID StorageSAN SwitchesEthernet Switches
• Benefits:20X Price/Performance Improvement overfour years30-50% Application PerformanceImprovementStandards-based solution for on-demandcomputingEnvironment that scales using 500-nodebuilding blocks
LAN
HDSStorage
CatalystSwitch
ExistingN/W
SFS 3012(TS360)96 ports
SFS 7008(TS270)
44x 24-portSFS 7000(TS120)
GridI/O
CoreFabric
Edge
12 hosts 12 hosts12 hosts 12 hosts512 Nodes
21© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
Utility Computing and Data CenterVirtualization
22© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
The Promise of Utility Computing
• Racks of compute resources virtualized in the data centerAllocated to applications dynamically
Network, storage and compute services applied dynamically topools of resource
Application or Policy
• Virtual I/O• Cluster computing w/ RDMA• Compute Virtualization
• LAN, WAN, Internet Access• Networked application services• Integrated security
• Storage Virtualization• Fabric-assisted Applications• Data Replication Services
23© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
What is VFrame?Cisco’s end-to-end virtualization, provisioning manager
What is VFrame™
• Cisco’s data center-wide virtualization software suite
• Enterprise software package that runs in the data center
• Delivers the end-to-end manageability, control, and virtualizationbenefits of the mainframe on top oftoday’s commodity components and the Cisco IIN
• Provides virtualization, orchestration, and provisioning for the datacenter resources that sit between the “OS” and the “wire”
Virtualization
24© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
CSMLoadBalancer
Servers
VFrame identifies rightApp / OS ImageFrom storage
VFrame translatespolicies to actions
and passes toinfrastructure
Catalyst 6500
SAN
ServerGRID
SFS 3012
FWSMFirewall
Administrator
MDS 9500
IB
GigE
FC
Campus/WAN/VPN
Data Center
Policy
Application: SAP
Performance
Security
Availability
Image
Define applicationservices and passpolicy to VFrame
VFrame™ VFrame picks serverwith right criteria torun application and
boots serverVFrame gives newserver right VLAN,
VSAN and LUN info so itcan find/be found by
right clients andstorage
VFrame provisionssecurity policies to
FWSM
VFrame provisionsCSM to add new serverto load balancing poolApplication Service Provisioned!
VFrame VisionCisco Virtual Data Center
25© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
CSMLoadBalancer
Servers
VFrame identifies rightApp / OS ImageFrom storage
VFrame translatespolicies to actions
and passes toinfrastructure
Catalyst 6500
SAN
FWSMFirewall
Administrator
MDS 9500
FC
Campus/WAN/VPN
Data Center
Policy
Application: SAP
Performance
Security
Availability
Image
Define applicationservices and passpolicy to VFrame
VFrame™ VFrame picks serverwith right criteria torun application and
boots serverVFrame gives newserver right VLAN,
VSAN and LUN info so itcan find/be found by
right clients andstorage
VFrame provisionssecurity policies to
FWSM
VFrame provisionsCSM to add new serverto load balancing poolApplication Service Provisioned!
VFrame VisionCisco Virtual Data Center
26© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
StorageNetwork
Campus/WAN/VPN
1. Physical server fails
2. VFrame detectsfault
3. VFrame programsMDS to map standbyphysical server to LUN
4. New serverrestarts in newgroup
xGroup 1 Group 2
StandbyPool
Utility Computing exampleServer Failover
27© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
Utility Computing exampleAdd Capacity on Demand
2. VFrame programsMDS to add newservers from standbypool
3. Standby servers restart innew group with new mappings
Group 1 Group 2 Standby Pool
1. Applicationmonitors issuetriggers to VFrame
Campus/WAN/VPN
StorageNetwork
28© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
VFrame Benefits
• Manage the data center from a service-oriented,application-centric perspective
• Eliminate number of layers/devices required to betouched to provision or modify
• Treat the entire data center infrastructure (from the“OS” to the “wire”) as one manageable entity ofshared virtualized resources (Virtual Mainframe)
• Expose a single orchestration and provisioninginterface for all data center infrastructure
• Dramatically reduce TCO
29© 2005 Cisco Systems, Inc. All rights reserved.Session NumberPresentation_ID
Subtitle: Size 26, Left Aligned
17.15-17.45
16.45-17.15
16.15-16.45
15.45-16.15
15.15-15.45
14.45-15.15
14.00-14.45
12.30-13.30
Agenda:
Q&A e conclusioni
Special GuestCase study
Luciano PomelliServer Networking e Virtual Data Center
Coffee break
Andrea VERRIOttimizzazione delle prestazioni applicative
Davide CATTONIConsolidamento e virtualizzazione dello storage
Luciano POMELLIIntroduzione all'architettura Cisco Enterprise DataCenter
Registrazione e buffet lunch
Top Related