VoltaireVoltaireThe Grid Backbone™The Grid Backbone™
VoltaireVoltaireThe Grid Backbone™The Grid Backbone™
InfiniBand CERN Seminar
Asaf Somekh
VP Strategic Alliances
June 2006
© 2006 Voltaire, All Rights Reserved. 2
AgendaAgenda
About Voltaire
Products Update
InfiniBand – Ethernet connectivity
© 2006 Voltaire, All Rights Reserved. 3
Voltaire: The CompanyVoltaire: The Company
2005 Sales: 4x over 2004, 15x over 2003- Over 50,000 Ports in 2005
Solid Financing- Recent Round: $17M, Summer 2005
- Sufficient funds to breakeven
Global Locations- Business HQ: Boston
- R&D Center: Hertzliya, Israel
- Sales Offices: San Francisco, Philadelphia, New York, Frankfurt, London, Tokyo
- Headcount : 120
© 2006 Voltaire, All Rights Reserved. 4
Deploying Grids Globally Across IndustriesDeploying Grids Globally Across Industries
Government Education & ResearchManufacturing
Financial Srv.EntertainmentOil & Gas
© 2006 Voltaire, All Rights Reserved. 5
Expanding ISV And Solution PartnershipsExpanding ISV And Solution Partnerships
Infrastructure
Application and Middleware Storage Software
Management and Scheduling ServerOEMs
© 2006 Voltaire, All Rights Reserved. 6
Voltaire Products SummaryVoltaire Products Summary
Grid Directors
ISR 9096 - 96 4X (DDR or SDR) or 32 12X ports
ISR 9288 – 288 4X (DDR or SDR) or 96 12X ports
Copper and optical support
Integrated GbE and/or FC ports (up to 144)
Full redundancy (PS, FAN, Fabric, Management, ..)
Blade Solutions
High performance blade solutions for IBM BladeCenterH
Blade Server Connectivity Solutions
Voltaire ISR 9096 Voltaire ISR 9288
© 2006 Voltaire, All Rights Reserved. 7
Voltaire Products SummaryVoltaire Products Summary
Grid switches- ISR 9024 - 24 port IB (SDR and DDR)
- ISR 6000 - Multi-Protocol (IP / FC / IB)
Management & Virtualization- Embedded in switches
- Highly scalable Grid management
Host software
- Unique High-Performance offeringsfor HPC, Financials, and Storage
ISR 9024
ISR 6000
GridVision™ Management Suite
© 2006 Voltaire, All Rights Reserved. 8
Voltaire Top500 LeadershipVoltaire Top500 Leadership
Largest number of Top500 InfiniBand sites
Broadest and Strongest Channels- Joint Installations with 7 OEMs and Resellers
Top InfiniBand Site: NASA Ames #4
Most network-efficient InfiniBand site
Number of InfiniBand sites doubled from previous list, tripled in a year
Proprietary interconnects keep going down
GbE pushed lower on the list
Interconnect Change from Nov 2004
InfiniBand +200%
GbE +38%
Quadrics -19%
Myrinet -23%
© 2006 Voltaire, All Rights Reserved. 9
Voltaire InfiniBand = Network Efficiency = $$$Voltaire InfiniBand = Network Efficiency = $$$
Assumptions:
- Cluster Size: 256 Nodes
- 2P Opteron Server Price: $3,000
- InfiniBand Price/Node: $900 80% Network Efficiency
- Myrinet Price/Node $800 65% Network Efficiency
- GbE Price/Node: $100 55% Network Efficiency
Actual Performance
Myr GbE
TF
lop
s
2
1.5
1
2
1.61.4
Cost Per TFlop
Myr GbE
$500K
$600K
$400K
507
609587
Example:
© 2006 Voltaire, All Rights Reserved. 10
Multi-Protocol SwitchingMulti-Protocol Switching
Same Switch, Multiple Protocol and I/F Combinations
© 2006 Voltaire, All Rights Reserved. 11
Voltaire IP Router Unique CapabilitiesHardware Configuration and Chassis ManagementVoltaire IP Router Unique CapabilitiesHardware Configuration and Chassis Management
ISR 9096/9288, as well as ISR 6000 chassis are supported
- sRBD is required for holding up to 3 routers (any mix)
- sRBD-D is required for DDR configurations, although the IPR is an SDR device
Copper as well as optic-fiber SFPs are supported
- IPRs are provided with 4 copper SFPs by default (included in price)
Cluster mode is supported (multiple devices per chassis, any mix)
Voltaire unique InfiniBand-Ethernet Silicon
Link aggregation
© 2006 Voltaire, All Rights Reserved. 12
Voltaire IP Router Unique CapabilitiesNetworking ManagementVoltaire IP Router Unique CapabilitiesNetworking Management
Centralized management – single point representation
- Feedback from the field leads to an improved mechanism - in development
CLI and GUI based management support
- Including full TFM (Traffic Flow Management) and partitioning support
- Click on Routers launches their VDM
Automated configuration and application transparency by cluster learning and ARP Proxy
Monitoring - management access to programmable hardware classification counters (CLI)
© 2006 Voltaire, All Rights Reserved. 13
Healthcare GridHealthcare Grid
Voltaire Grid Director ISR 9096
Native InfiniBand Storage with FalconStor IPStor
Local Users
InfiniBand VTL
ApplicationServers
Oracle 10g RAC Cluster
Data from Hospitals
WAN Router
IPoIBRDS
IPoIBSDP
iSERTCP/IP
© 2006 Voltaire, All Rights Reserved. 14
32 Gb Ethernet Connectivity with InfiniBand 32 Gb Ethernet Connectivity with InfiniBand
ISR 9288
ISR 9288
…
ISR 9288
…
ISR 9288
…
ISR 9288
…
ISR 9288
…
ISR 9288
…
240 4X links to 120 compute
nodes
240 4X links to 120 compute
nodes
240 4X links to 120 compute
nodes
240 4X links to 120 compute
nodes
240 4X links to 120 compute
nodes
194 4X links to 55 compute nodes
+ 42 4X links to 90 storage nodes
48 4X links to each lower tier
ISR 9288
Upper tier blocking (over-subscription) factor = 5:1Node to node hops (worst case) = 9
ISR 9288
IP Router
IP Router
IP Router
IP Router
IP Router
IP Router
IP Router
IP Router
Multiple trunks of 4 x 1 GbE 16 GbE Links 16 GbE Links
© 2006 Voltaire, All Rights Reserved. 15
TITECH Grid - ProtocolsTITECH Grid - Protocols
650 16-Core NodesEach with 2 HCAs
Stretching over 3 floors
1PB InfiniBand Storage
with Lustre
Legacy NAS Storage
650 16-Core NodesEach with 2 HCAs
Stretching over 3 floors
1PB InfiniBand Storage
with Lustre
Legacy NAS Storage
MPI(MVAPICH)Lustre
NFS
300 Opticallinks
300 Opticallinks
iSER
© 2006 Voltaire, All Rights Reserved. 16
Enabling High Performance Commodity Storage with iSCSI RDMA (iSER) and InfiniBand Enabling High Performance Commodity Storage with iSCSI RDMA (iSER) and InfiniBand
iSER Target BOM Qty Cost Sum
Target platform (PC) 1 $1-2K $1-2K
SATA drives 16 $0.7 * GB 16*300GB*.7 = $3.4K
SATA RAID Adapter 1-2 $300 $300-600
InfiniBand Adapter 1 $500 $500
FCFC
PerformancePerformance
400 MB/s
400 MB/s
900 MB/s
900 MB/s
iSERiSER
Delivering unprecedented cost/performance, leveraging on industry standard components
Commercial grade target
1 $2K -5K $2K-5K (config dependent)
Total (4.8 TB) ~$2-3/GB
Thank YouThank YouThank YouThank You
© 2006 Voltaire, All Rights Reserved. 18
1,024 Compute Nodes on A Single Global File Systemwith Voltaire Grid Backbone™ and HP SFS ™1,024 Compute Nodes on A Single Global File Systemwith Voltaire Grid Backbone™ and HP SFS ™
87 Voltaire ISR 9024Grid Switches
87 Voltaire ISR 9024Grid Switches
4 Voltaire ISR 9288 Grid Directors
4 Voltaire ISR 9288 Grid Directors
1,024 Compute Nodes4 Login Nodes2 Admin Nodes
1,024 Compute Nodes4 Login Nodes2 Admin Nodes
12 HP SFS OSS Nodes (Lustre Servers) with 132 TB
OSS Throughput 900 MB/s1 SFS Admin Node
1 MDS Node
12 HP SFS OSS Nodes (Lustre Servers) with 132 TB
OSS Throughput 900 MB/s1 SFS Admin Node
1 MDS Node
Client Throughput:
350 MB/s
Client Throughput:
350 MB/s
Customer: D.O.DUS Air ForceProject TI-05
Customer: D.O.DUS Air ForceProject TI-05
Fully Non-blocking10 Gbps InfiniBand Fabric
Fully Non-blocking10 Gbps InfiniBand Fabric
Top Related