First P2P Measurements on Infiniband Luciano Berti INFN Laboratori Nazionali di Legnaro

Click here to load reader

download First P2P Measurements on Infiniband Luciano Berti INFN Laboratori Nazionali di Legnaro

of 7

  • date post

    16-Jan-2016
  • Category

    Documents

  • view

    55
  • download

    0

Embed Size (px)

description

First P2P Measurements on Infiniband Luciano Berti INFN Laboratori Nazionali di Legnaro. Infiniband in brief. Storage Target. Channel based message passing. TCA. Internet Intranet. Mem Cntlr. CPU. 1. HCA. Link. Link. Link. Link. Switch. Mem Cntlr. CPU. HCA. 2. Link. Link. - PowerPoint PPT Presentation

Transcript of First P2P Measurements on Infiniband Luciano Berti INFN Laboratori Nazionali di Legnaro

  • First P2P Measurements on Infiniband

    Luciano BertiINFN Laboratori Nazionali di Legnaro

  • Infiniband in briefSame networkto transport lowlatency ipc, storage I/O and network I/OInternet IntranetLink speed1x 2.5 Gbps4x 10.0 Gbps.12x 30 Gbps

    RouterxCALinkLinkCPUMem CntlrHCACPUMem CntlrHCACPUMem CntlrHCALinkCPUMem CntlrHCASwitchLinkLinkLinkLinkLinkTCAN/WTargetTCAStorageTargetLinkLinkLink123nChannel basedmessage passing1000s nodeper subnet1 x Link4 x Link12 x Link

  • Infiniband Transport ProtocolsIBA has been developed with Virtual Interface in mind. VIPL 2.0 includes IBA extensions and RDMA operations. SCSI RDMA Protocol (SRP). It is a T10 standard.SRP defines mapping to IBA architectureit is the transport protocol over IBASRP is based on VIDirect Access Files System (DAFS)Direct Access Socket (DAS)TCP/IP functionality over VI/IBIBA Host Channel AdapterVirtual Interface over IBDASDAFSSRPfast, low latencyTCP/IP socketsFile AccessBlock Accessfast, low latency network storage

  • LNL Infiniband Test BedLeaf Switch32 1x (2.5 Gbps) ports in 1 U chassisPCI-X (max ~ 380 Mbyte/s)Supermicro P4DL6IBA Host Channel Adapter

  • Status of Infiniband test bedAll the hardware has been provided by Infiniswitch (1 switch + 4 HCA)

    All the hardware is up and running

    First p2p measurements have been performed

    SoftwareVirtual Interface Library (VIPL) as provided by InfiniswitchSend/Receive over reliable connectionsRDMA over reliable connections

    Sourceforge has a infiniband project over LinuxVIPL source is available. Compiled and works!Performance as the infiniswitch VIPL (probably they are the same code)ResultsRound trip time small buffers ~ 40 msec (latency 20 msec)P2p ~ 80% link saturation

  • First IB p2p tests220Link SaturationMbyte/sBuffer Size (Bytes)~ 40 Mbyte/s

  • Work Plan2 months IB hardware evaluation (started end of May)

    Mainly focus on functionality tests both of the hardware and software available:SRP SCSI remote protocol already got, to be testedDAS and DAFS if early version will be available

    Contacts started with major companies (IBM, Dell, Fujitsu/Siemens) to understand their IB products road map