Pedge m1000e

download Pedge m1000e

of 42

description

Pedge m1000e

Transcript of Pedge m1000e

  • 5/22/2018 Pedge m1000e

    1/42

    One Dell WayRound Rock, Texas 78682

    www.dell.com

    Dell PowerEdge M1000e Modular EnclosureArchitecture

    January 2008

    By John Loffink, Engineer/Strategist

    This white paper is for informational purposes only, and may contain typographical errors and technical inaccuracies. Thecontent is provided as is, without express or implied warranties of any kind.

    Dell Enterprise White Paper

  • 5/22/2018 Pedge m1000e

    2/42

    Contents

    Key Benefits 2Summary 2Index of Figures 3Index of Tables 3

    Acronyms and Defini tions 4Background 7Server Modules 7PowerEdge M1000e Midplane and

    I/O 17Power 28Cooling 34System Management 37Conclusion 42

    Acknowledgments 42

    Key Benefits

    16servermodules

    3x2(Redundant)I/Ofabrics

    RedundantPower

    RedundantCooling

    RedundantChassisManagement

    Controllers

    AvocentiKVMinterface

    LCDControlPanel

    Summary

    TheDellPowerEdgeM1000eModularServerEnclosure

    isa

    breakthrough

    in

    enterprise

    server

    architecture.

    The

    enclosureanditscomponentsspringfroma

    revolutionary,groundupdesignincorporatingthelatest

    advancesinpower,cooling,I/O,andmanagement

    technologies.Thesetechnologiesarepackedintoa

    highlyavailablerackdensepackagethatintegratesinto

    standardDelland3rdparty1000mmdepthracks.

    Theenclosureis10Uhighandsupports:

    Upto16servermodules.

    Upto6network&storageI/Ointerconnect

    modules.

    Ahighspeedpassivemidplaneconnectstheserver

    modulesinthefrontandpower,I/O,and

    managementinfrastructureintherearofthe

    enclosure.

    ComprehensiveI/Ooptionssupportduallinksof20

    Gigabitspersecondtoday(with4xDDRInfiniBand)

    withfuturesupportofevenhigherbandwidthI/O

    deviceswhenthosetechnologiesbecomeavailable.

    Thisprovideshighspeedservermodule

    connectivitytothenetworkandstoragenowand

    wellinto

    the

    future.

    Thoroughpowermanagementcapabilitiesincluding

    deliveringsharedpowertoensurefullcapacityof

    thepowersuppliesavailabletoallservermodules.

    Broadmanagementabilityincludingprivate

    Ethernet,serial,USB,andlowlevelmanagement

    connectivitybetweentheChassisManagement

    Controller(CMC),Keyboard/Video/Mouseswitch,

    andservermodules.

    UptotwoChassisManagementControllers(CMC

    1isstandard,2ndprovidesoptionalredundancy)and

    1optional

    integrated

    Keyboard/Video/Mouse

    (iKVM)switch.

    Upto6hotpluggable,redundantPowerSupplies

    and9hotpluggable,N+1redundantFanModules.

    Page2

    SystemFrontControlpanelw/LCDpanelandtwo

    USBKeyboard/MouseandoneVideocrashcart

    connections.

  • 5/22/2018 Pedge m1000e

    3/42

    Index of Figures

    Figure1PossibleServerModuleSizes,FrontPanelView..................................................................... 8Figure

    2

    Example

    Server

    Module

    Configurations .................................................................................. 8

    Figure3DualSocketModularServerArchitecture................................................................................ 9Figure4ModularServerFabricI/OMezzanineCard,FibreChannel4Gbps .................................. 10Figure5M600DualSocketModularServerwithIntelProcessors .................................................... 13Figure6M605DualSocketModularServerwithAMDProcessors .................................................. 14Figure7HalfHeightModularServerFrontPanelView ..................................................................... 14Figure8 ModularServerRemoval......................................................................................................... 15Figure9PowerEdgeM1000eFrontView............................................................................................... 17Figure10PowerEdgeM1000eMidplaneFrontView........................................................................... 19Figure11PowerEdgeM1000eMidplaneRearView ............................................................................ 20Figure12HighSpeedI/OArchitecture.................................................................................................. 21Figure1310GEPotentialGrowthPath................................................................................................... 23Figure

    14

    PowerEdge

    M1000e

    Modular

    Server

    System

    Bandwidth................................................... 23

    Figure15PowerEdgeM1000eRearView .............................................................................................. 25Figure16DifferencebetweenPassthroughandSwitchModules ...................................................... 26Figure17I/OModule,DellPowerEdgeGigabitEthernetPassthrough ............................................ 26Figure18I/OModule,DellPowerEdgeGigabitEthernetSwitch ...................................................... 26Figure19PowerEdgeM1000eSinglePhasePDUPowerSystem....................................................... 30Figure20PowerEdgeM1000eThreePhasePDUPowerSystem........................................................ 30Figure21PowerEdgeM1000e2360WattPowerSupply ..................................................................... 31Figure22PowerRedundancyModes..................................................................................................... 32Figure23PowerArchitecture.................................................................................................................. 33Figure24PowerEdgeM1000eFan .......................................................................................................... 34Figure

    25

    Server

    Module

    Cooling

    Air

    Profile........................................................................................ 35

    Figure26I/OModuleCoolingAirProfile ............................................................................................. 35Figure27PowerSupplyCoolingAirProfile......................................................................................... 36Figure28SystemManagementArchitectureSimplifiedBlockDiagram .......................................... 37Figure29PowerEdgeM1000eChassisManagementController........................................................ 38Figure30ChassisManagementControllerFrontPanel....................................................................... 38Figure31PowerEdgeM1000eiKVMModule ....................................................................................... 40Figure32iKVMFrontPanel .................................................................................................................... 41Figure33PowerEdgeM1000eLCDPanelRecessedPosition ............................................................. 41Figure34PowerEdgeM1000eLCDPanelDuringUsage.................................................................... 42Figure35LCDGraphicExamples........................................................................................................... 42Index of Tables

    Table1ModularServerFabricI/OMezzanineFabricBandCOptions............................................ 11Table2TypicalModularServerFabricConfigurations....................................................................... 12Table3HalfHeightModularServerDimensions................................................................................. 13Table4HalfHeightServerModuleOptions ......................................................................................... 16Table5PowerEdgeM1000eModularSystemFeaturesandParameters........................................... 18

    Page3

    Table6RelativeComparisonofIndustryStandardHighSpeedI/O ................................................. 22

  • 5/22/2018 Pedge m1000e

    4/42

    Page4

    Table7EthernetI/OModuleOptions..................................................................................................... 27Table8FibreChannelI/OModuleOptions........................................................................................... 27Table9InfiniBandI/OModuleOption................................................................................................... 28Table

    10

    Typical

    Modular

    Server

    System

    Rack

    Height

    and

    Cable

    Reduction .................................. 28Table11PowerEdgeM1000eSinglePhaseandThreePhasePDUOptions...................................... 29

    Acronyms and Defini tions

    1000BASEKXIEEEstandardfortransmissionof1GbpsEthernetoverbackplanes

    1000M1000MegabitpersecondEthernet

    100BaseT100MegabitpersecondEthernetoverTwistedPair

    100M100MegabitpersecondEthernet

    10GBASEKRIEEEstandardfortransmissionof10GbpsEthernetoverbackplanesusing1

    differentiallanepair

    10GBASEKX4

    IEEE

    standard

    for

    transmission

    of

    10

    Gbps

    Ethernet

    over

    backplanes

    using

    4

    differentiallanepairs

    10GE10GigabitEthernet

    10M10MegabitpersecondEthernet

    3G3Gbps

    4G4Gbps

    6G6Gbps

    8G8Gbps

    ACAlternatingCurrent

    ACIAnalogConsoleInterface

    ASICApplicationSpecificIntegratedCircuit

    Bel10Decibels

    BERBitErrorRate

    BMCBaseboardManagementController

    CDCompactDisk

    CERC6CostEffectiveRAIDControllerversion6

    CFMCubicFeetperMinute

    CIMCommonInformationModel

    CMCChassisManagementController

    CLICommandLineInterface

    CPUCentralProcessingUnit

    DDRDoubleDataRate

    DDR2DoubleDataRate2DynamicRandomAccessMemory

    DHCP

    Dynamic

    Host

    Configuration

    Protocol

    DIMMDualInlineMemoryModule

    DVDDigitalVideoDisk

    ECCErrorCorrectingCode

    EIAElectronicIndustriesAlliance

    FBDIMMFullyBufferedDIMM

    FCFibreChannel

    FPFullPower

  • 5/22/2018 Pedge m1000e

    5/42

    GB Gigabyte

    GbpsGigabitspersecond

    GEGigabitEthernet

    Gen1

    Generation

    1

    Gen2Generation2

    GPIOGeneralPurposeI/O

    GUIGraphicalUserInterface

    HAHighAvailability

    HBAHostBusAdapter

    HCAHostControllerAdapter

    HDDHardDiskDrive

    HPCCHighPerformanceComputingCluster

    IB InfiniBand

    ICHInput/OutputControllerHub

    I/O

    Input/Output

    iDRACIntegratedDellRemoteAccessController

    iKVMIntegratedKVM

    I/OInput/Output

    IOHInput/OutputHub

    IOMInput/OutputModule

    IPCInterprocessCommunications

    IPMIIntelligentPlatformManagementInterface

    IPv6InternetProtocolversion6

    IRIntegratedRAID

    iSCSIInternetSCSI

    ISOInternationalOrganizationforStandardization

    KVM

    Keyboard

    Video

    Mouse

    LANLocalAreaNetwork

    LCDLiquidCrystalDisplay

    LEDLightEmittingDiode

    LOMLANonMotherboard

    LRLongReach

    LVLowVoltage

    MbpsMegabitspersecond

    NICNetworkInterfaceCard

    MB Megabyte

    MCHMemoryControllerHub

    MHz

    MegaHertz

    MVMidVoltage

    N+0N(1,2,3)pluszero,nofaulttolerance

    N+1N(1,2,3)plusone,faulttolerancewithonebackupunit

    N+NN(1,2,3)plusN(1,2,3)withNbackupunits

    NPIVN_PortIDVirtualization

    OPSFOpenShortestPathFirst

    Page5

    OSOperatingSystem

  • 5/22/2018 Pedge m1000e

    6/42

    PCIePCIExpress,orPeripheralComponentInterconnectExpress

    PDUPowerDistributionUnit

    PMBusPowerManagementBus

    QDR

    Quad

    Data

    Rate

    QOSQualityofService

    R2Release2

    RACADMRemoteAccessControllerandAdministrator

    RAIDRedundantArrayofInexpensiveDisks

    RIPRoutingInformationProtocol

    SANStorageAreaNetwork

    SASSerialAttachedSmallComputerSystemInterface,orSerialSCSI

    SAS6/IRSASI/OCard6withintegratedRAID

    SATASerialAdvancedTechnologyAttachment,orSerialATA

    SDDCSingleDeviceDataCorrection

    SDR

    Single

    Data

    Rate

    SFPSmallFormfactorPluggableTransceiver

    SMASHCLPSystemsManagementArchitectureforServerHardwareCommandLineProtocol

    SNMPSimpleNetworkManagementProtocol

    SOLSerialoverLAN

    SRShortReach

    SSHSecureShell

    SSLSecureSocketsLayer

    TCP/IPTransmissionControlProtocol/InternetProtocol

    TFTPTrivialFileTransferProtocol

    TOETCP/IPOffloadEngine

    USBUniversalSerialBus

    VGA

    Video

    Graphics

    Array

    vKVMVirtualKVM

    VLANVirtualLAN

    vMediaVirtualMedia

    VRRPVirtualRouterRedundancyProtocol

    W2K3WindowsServer2003OperatingSystem

    WOLWakeOnLAN

    WSMANWebServicesforManagement

    XMLExtensibleMarkupLanguage

    XFP10GigabitSmallFormFactorPluggableTransceiver

    Page6

  • 5/22/2018 Pedge m1000e

    7/42

    Background

    ITISALLABOUTEFFICIENCY.ThenewPowerEdgeM1000eisdesignedtohelpyoubemore

    efficientwith

    your

    time,

    your

    power

    and

    cooling,

    your

    investment,

    and

    your

    systems

    performance. ItisabreakthroughDellengineeredandpatentpendingdesignthatmaximizes

    flexibility,powerandthermalefficiency,systemwideavailability,performance,and

    manageability.Thechassisintegratesthelatestinmanagement,I/O,powerandcooling

    technologiesinamodular,easytousepackage. Designedfromthegrounduptosupportcurrent

    andfuturegenerationsofserver,storage,networking,andmanagementtechnologies,the

    PowerEdgeM1000eincludestheheadroomnecessarytoscaleyourenvironmentforthefuture.

    ThePowerEdgeM1000eModularServerEnclosuresolutionsupportsservermodules,network,

    storage,andclusterinterconnectmodules(switchesandpassthroughmodules),ahigh

    performanceandhighlyavailablepassivemidplanethatconnectsservermodulestothe

    infrastructurecomponents,powersupplies,fans,integratedKVMandChassisManagement

    Controllers(CMC).

    The

    PowerEdge

    M1000e

    uses

    redundant

    and

    hot

    pluggable

    components

    throughouttoprovidemaximumuptime.

    Introducingnewlevelsofmodularitytoserverbasedcomputing,thePowerEdgeM1000e

    providesidenticalandsymmetricfabricoptionsBandCforeachmodularserver. EthernetI/O

    switchessupportI/OsubmodulesthatprovideexternalI/Oflexibilityofstackingports,10GE

    copperports,or10GEopticalports. Truemodularityatthesystemandsubsystemlevelprovides

    simplicityofextensionandenhancement,nowandinthefuture.

    Server Modules

    Virtuallyunlimitedinscalability,thePowerEdgeM1000eEnclosureprovidesultimateflexibility

    in

    server

    processor

    and

    chipset

    architectures.

    Both

    Intel

    and

    AMD

    server

    architectures

    are

    plannedforintroductionintotheinfrastructure,whilecuttingedgemechanical,electricaland

    softwareinterfacedefinitionsenablemultigenerationalserversupportandexpansion.

    ThePowerEdgeM1000eenclosuresupportsupto16HalfHeightservermodules,each

    occupyingaslotaccessibleinthefrontoftheenclosure. ServerModulesotherthanstandard

    HalfHeightformfactorarealsosupportedinthefuture,occupyingfullslotheights,dualslot

    widths,orboth. Themechanicalslotsintheenclosurecansupportserversthataretwicethe

    heightand/orwidthoftheHalfHeightmodule,asshowninFigure1.

    ServerModulescanbefreelylocatedwithineach2x2HalfHeightquadrant.Themechanical

    designofthePowerEdgeM1000ehassupportstructuresforHalfHeightServerModulesabove

    orbelowDoubleWidthServerModules,andforHalfHeightServerModulessidebysidewith

    FullHeight

    Server

    Modules,

    as

    shown

    in

    Figure

    2.

    Page7

    ThefirsttwomodularserversavailableforthePowerEdgeM1000eModularEnclosureareHalf

    Height2CPUsocket,8DIMMcapablesystems,withtwo2.5inchhotpluggableHardDiskslots,

    andtwoflexiblefabricmezzaninecardsinadditiontothetwoonboard1GigabitEthernetLAN

    onMotherboard(LOMs).AsimplifiedblockdiagramofthePowerEdgeM600modularservers

    architecture,basedontheIntelprocessor/chipsetarchitecture,isshowninFigure3.

  • 5/22/2018 Pedge m1000e

    8/42

    Figure1PossibleServerModuleSizes,FrontPanelView

    Figure2ExampleServerModuleConfigurations

    Server Processors

    ThefirstgenerationofHalfHeightservermodulessupportafullrangeofdualandquadcore

    processorsineitherLowVoltage(4068W),MediumVoltage(8095W)orFullPower(120W)

    profiles.

    Somerestrictions

    on

    memory

    support

    may

    apply

    to

    the

    120W

    quad

    core

    full

    power

    processors,baseduponthesystempowerenvelopeandcoolingcapabilities.BothAMDandIntel

    processorarchitecturesaresupported.

    Server Memory

    MemorysupportinthefirstgenerationHalfHeightservermodulesiscomprisedof8DIMM

    slots. IntelbasedarchitecturesuseFullyBufferedDIMMtechnology,whileAMDbased

    Page8

    architecturesuseDDR2DIMMtechnology. Interleaving(2waylockstep),ECC,memorysparing

  • 5/22/2018 Pedge m1000e

    9/42

    Figure3DualSocketModularServerArchitecture

    andChipKill/SDDCaresupportedwithbothAMDandIntelbaseddesigns. Memorymirroring

    issupportedonthePowerEdgeM600Intelbaseddesign. SupportingarangeofDIMMcapacities

    from512MBto8GB,eachservermodulemaycontainamaximumof64GBofmemory.

    Server I/O

    Eachservermoduleconnectstotraditionalnetworktopologies,whileprovidingsufficient

    bandwidthformultigenerationalproductlifecycleupgrades. I/Ofabricintegrationencompasses

    networking,storage,

    and

    interprocess

    communications

    (IPC).

    To

    understand

    the

    PowerEdge

    M1000earchitecture,itisnecessarytofirstdefinefourkeyterms:fabric,lane,linkandport.

    Page9

    Afabricisdefinedasamethodofencoding,transporting,andsynchronizingdatabetween

    devices. ExamplesoffabricsareGigabitEthernet(GE),FibreChannel(FC)orInfiniBand(IB).

    FabricsarecarriedinsidethePowerEdgeM1000esystem,betweenservermoduleandI/O

    Modulesthroughthemidplane. Theyarealsocarriedtotheoutsideworldthroughthephysical

    copperoropticalinterfacesontheI/Omodules.

  • 5/22/2018 Pedge m1000e

    10/42

    AlaneisdefinedasasinglefabricdatatransportpathbetweenI/Oenddevices. Inmodernhigh

    speedserialinterfaceseachlaneiscomprisedofonetransmitandonereceivedifferentialpair. In

    reality,asinglelaneisfourwiresinacableortracesofcopperonaprintedcircuitboard,a

    transmitpositive

    signal,

    a

    transmit

    negative

    signal,

    a

    receive

    positive

    signal,

    and

    a

    receive

    negativesignal.Differentialpairsignalingprovidesimprovednoiseimmunityforthesehigh

    speedlanes.Variousterminologyisusedbyfabricstandardswhenreferringtolanes. PCIecalls

    thisalane,InfiniBandcallsitaphysicallane,andFibreChannelandEthernetcallitalink.

    Alinkisdefinedhereasacollectionofmultiplefabriclanesusedtoformasinglecommunication

    transportpathbetweenI/Oenddevices. Examplesaretwo,fourandeightlanePCIe,orfourlane

    10GBASEKX4. PCIe,InfiniBandandEthernetcallthisalink. Thedifferentiationhasbeenmade

    herebetweenlaneandlinktopreventconfusionoverEthernetsuseofthetermlinkforboth

    singleandmultiplelanefabrictransports.SomefabricssuchasFibreChanneldonotdefinelinks

    astheysimplyrunmultiplelanesasindividualtransportsforincreasedbandwidth.Alinkas

    definedhereprovidessynchronizationacrossthemultiplelanes,sotheyeffectivelyacttogether

    asa

    single

    transport.

    AportisdefinedasthephysicalI/Oendinterfaceofadevicetoalink. Aportcanhavesingleor

    multiplelanesoffabricI/Oconnectedtoit.

    TherearethreesupportedhighspeedfabricsperPowerEdgeM1000eHalfHeightservermodule,

    withtwoflexiblefabricsusingoptionalpluginmezzaninecardsontheserver. Theportsonthe

    servermoduleconnectviathemidplanetotheassociatedI/OModules(IOM)intherearofthe

    enclosure,whichthenconnecttothecustomersLAN/SAN/IPCnetworks.

    Figure4Modular

    Server

    Fabric

    I/O

    Mezzanine

    Card,

    Fibre

    Channel

    4Gbps

    Page10

    Thefirstembeddedhighspeedfabric(FabricA)iscomprisedofdualGigabitEthernetLOMsand

    theirassociatedIOMsinthechassis. TheLOMsarebasedontheBroadcom5708NetXtremeII

    Ethernetcontroller,supportingTCP/IPOffloadEngine(TOE)andiSCSIbootcapability.In

    addition,fulliSCSIHBAcapabilitywithfullprotocoloffloadandabroaderarrayofOSboot

    supportisplannedtobeavailableinearly2008. Whileinitiallyserversreleasedinthisseries

  • 5/22/2018 Pedge m1000e

    11/42

    haveadualGELOMconfiguration,thesystemmidplaneisdesignedtoallowfuturesupportfor

    uptoquadGELOMsintheHalfHeightformfactor.

    OptionalcustomerconfiguredfabricsaresupportedbyaddinguptotwodualportI/O

    Mezzaninecards.

    Each

    Half

    Height

    server

    module

    supports

    two

    identical

    I/O

    Mezzanine

    card

    connectorswhichenableconnectivitytoI/OfabricsBandC.TheseoptionalMezzaninecards

    offerawidearrayofEthernet(includingiSCSI),FibreChannel,andInfiniBandtechnologies,with

    otherspossibleinthefuture. Figure4showsanexampleofadualportFibreChannel4Gigabits

    persecond(Gbps)mezzaninecard.Liketheconnector,I/OMezzaninecardformfactorsare

    commontobothfabricBandC,forahighlevelofflexibilityinfabricconfiguration.

    Theoptionalmezzaninecardsaredesignedtoconnectvia8lanePCIetotheservermodules

    chipset.ForPCIeGen1thisprovidesupto16Gbpsofdatabandwidthpermezzaninecard.

    MezzaninecardsmayhaveeitheronedualportASICwith4or8lanePCIeinterfacesordual

    ASICs,eachwith4lanePCIeinterfaces.BothPCIefabricsandexternalgoingfabricsarerouted

    throughhighspeed,10Gigabitpersecondcapableairdielectricconnectorpinsthroughthe

    planarandmidplane. Forbestsignalintegritythesignalsisolatetransmitandreceivesignalsfor

    minimumcrosstalk. Differentialpairsareisolatedwithgroundpinsandsignalconnector

    columnsarestaggeredtominimizesignalcoupling.

    WakeonLAN(WOL)issupportedbytheservermodulesLOMs. EthernetWOLforFabricsB

    andCisalsosupported. BootfromSANissupportedbyFibreChannelandiSCSIenabledcards.

    ThePowerEdgeM1000esystemmanagementhardwareandsoftwareincludesFabric

    ConsistencyChecking,preventingtheaccidentalactivationofanymisconfiguredfabricdeviceon

    aservermodule. SincemezzaninetoI/OModuleconnectivityishardwiredyetfullyflexible,a

    usercouldinadvertentlyhotplugaservermodulewiththewrongmezzanineintothesystem.

    Forinstance,ifFibreChannelI/OModulesarelocatedinFabricCI/OSlots,thenallserver

    modulesmust

    have

    either

    no

    mezzanine

    in

    fabric

    C

    or

    only

    Fibre

    Channel

    cards

    in

    fabric

    C.

    If

    a

    GEmezzaninecardisinaMezzanineCslot,thesystemautomaticallydetectsthis

    misconfigurationandalertstheuseroftheerror.Nodamageoccurstothesystem,andtheuser

    hastheabilitytoreconfigurethefaultedmodule.

    Table1showsoptionsfortheinitialreleaseoffabricmezzaninesinthePowerEdgeM1000e

    productline. ThesamemezzaninecardcanbeusedinFabricBorC. Table2showssome

    typicalservermodule

    FabricType Ports Speedperport Features

    GigabitEthernet 2 1.25Gbps TOE,iSCSIboot,JumboFrames,

    WOL

    FibreChannel

    4G

    2

    4.25

    Gbps

    Boot

    from

    SAN,

    NPIV

    FibreChannel8G* 2 8.50Gbps BootfromSAN,NPIV

    4xDoubleDataRate

    (DDR)InfiniBand*

    2 20.00Gbps

    10GigabitEthernet* 2 10.3125Gbps TOE,iSCSIoffload,JumboFrames

    *planned

    Page11

    Table1ModularServerFabricI/OMezzanineFabricBandCOptions

  • 5/22/2018 Pedge m1000e

    12/42

    TypicalFabric

    Configurations

    FabricA FabricB FabricC

    LowCostCompute

    ClusterNode

    2x1GELOM Empty Empty

    Ethernet/SANserver

    orHAClusterNode

    2x1GELOM 2x1GEMezzanine 2xFC4Mezzanine

    HighPerformance

    ClusterNode

    2x1GELOM 2x4DDRInfiniBand Empty

    UnifiedFabric 2x1GELOM 2x1GEMezzanine 2x10GEMezzanine

    HighPerformance

    UnifiedFabric

    2x1GELOM 2x10GEMezzanine 2x10GEMezzanine

    Table2TypicalModularServerFabricConfigurations

    fabricconfigurations,baseduponcurrentandfuturearchitecturalmodels.Speedandlanecount

    increasesare

    anticipated

    for

    future

    mezzanine

    releases.

    Local Storage

    Servermodulelocalstorageiscomprisedofuptotwo2.5inchHardDrives. Bothhotpluggable

    SASandnonhotpluggableSATAharddrivesaresupported. Localstoragecontrolleroptions

    includelowcostchipsetSATAI/O,SASDaughtercardwithintegratedRAID(SAS6/IR)orSAS

    RAIDDaughtercardwithcacheandhighperformanceRAIDprocessor(CERC6/i).

    Integrated Server Management

    ThePowerEdgeM1000esupportstheintroductionofDellsnewIntegratedDellRemoteAccess

    Controller(iDRAC)whichisintegratedoneachM600/605servermodule.iDRACcontains

    featuresthat

    are

    typically

    add

    on

    options

    to

    standard

    monolithic

    rack

    based

    servers.

    The

    iDRAC

    addsvirtualMedia(vMedia)andvirtualKVM(vKVM)functionsaswellasoutofbandGUI

    status/inventory.TraditionalIPMIbasedBaseboardManagementController(BMC)featureslike

    hardwaremonitoringandpowercontrolaresupported. Additionally,onboardgraphicsand

    keyboard/mouseUSBconnectstoanoptionalsystemlevelIntegratedKVM(iKVM)modulefor

    localKVMaccess. FullUSBaccessisavailablethroughtheservermodulefrontpanel.

    ThenewiDRACisconnectedtotheCMCviadedicated,fullyredundant100MbpsEthernet

    connectionswiredthroughthemidplanetoadedicated24portethernetswitchontheCMC,and

    exposedtotheoutsideworldthroughtheCMCsexternalManagementEthernetinterface

    (10/100/1000M).ThisconnectionisdistinctfromthethreeredundantdataFabricsA,BandC.

    UnlikepreviousgenerationsofDellservermodules,theiDRACsconnectivityisindependentof,

    andinadditionto,theonboardGELOMsontheservermodule.EachservermodulesiDRAC

    hasitsownIPaddressandcanbeaccessed,ifsecuritysettingsallow,directlythrougha

    supportedbrowser,telnet,SSH,orIPMIclientonthemanagementstation.

    Server Module and Packaging

    Page12

    Theservermoduleisasheetmetalmodulethathousestheserverplanarandhotplughard

    drivesandisdesignedtobetoollessforanyassemblyordisassemblyoperation. Anypart

    removalfromtheserverisintuitiveandrequiresnotools. Thisincludesthetopcover,planar

  • 5/22/2018 Pedge m1000e

    13/42

    Figure5M600DualSocketModularServerwithIntelProcessors

    installation,harddrives,fabriccardsandremovableHardDrivecage.PhotosoftheM600and

    M605bladesareshowninFigure5andFigure6respectively,whileoverallbladedimensionsare

    showninTable3.

    BladeMeasurement Dimension

    Length 520mm

    Height 193mm

    Width 50mm

    Table3HalfHeightModularServerDimensions

    Server Module Front Panel

    TheHalfHeightservermodulesfrontpanelprovidesaccesstotwohotpluggableharddrives,

    twobootableUSBconnectors,oneserverpowerbuttonandonestatusindicator. Allserver

    modulessupporthotpluggable2.5inchharddiskdrives. Thefrontpanelisvisibleinthephoto

    inFigure7.

    Server Module Removal and Translating Handle

    Theservermoduleismanuallyinstalledbyaligningitwithoneoftheopenfrontslotlocations

    andslidingituntiltheconnectorsbegintoengagewiththemidplane. Atranslatinghandle,

    whichtravelsinthesamedirectionastheservermodule,isthenusedtofullyengageandlockit

    inplace. Toremovetheservermodule,abuttonreleasesthetranslatinghandlelatch. The

    handleisthenusedforpullingtheserveroutofthesystem,disengagingtheinternallatchasthe

    handleispulledoutwardsandunmatingtheconnectors.

    Page13

    Dual Hard Disk DriveSlots, RAID underneath

    8 DIMM Slots Two FabricMezzanine Cards

    ChipsetTwo Processors

  • 5/22/2018 Pedge m1000e

    14/42

    Figure6M605DualSocketModularServerwithAMDProcessors

    Figure7HalfHeightModularServerFrontPanelView

    Thereareguidepinsonbothsidesoftheservermodulesignalconnectorforprecisealignmentto

    thechassis. Keyingfeaturespreventtheservermodulefrominstallingupsidedown. Adefined

    maximumslidingforceinsuresergonomicfriendliness,whileprecisionpointofcontact

    positioningandcammingfeaturesensureproperconnectorwipeformaximumsignalquality.

    Modularserverpowerisreceivedthroughadedicated2x3powerblock,withwidepowerpins

    designatedinsuchawaytopreventpowerrailshortingincaseapinbendsforanyreason.

    Page14

    Dual Hard Disk DriveSlots, RAID underneath

    4 DIMM Slots Two FabricMezzanine Cards

    Chipset4 DIMM Slots

    2 Processors(1 socket empty)

  • 5/22/2018 Pedge m1000e

    15/42

    Figure8ModularServerRemoval

    Server Module Summary

    Table4summarizesthefeaturesforthefirsttwoservermodulesshippingforthePowerEdge

    M1000e. AdditionalservermodulesareexpectedtoshipfollowingthelaunchofthePowerEdge

    M1000e,andthroughoutthelifetimeofthemodularsystem.

    Page15

    ServerModule M600 M605

    Processor IntelXeonDualandQuadCore

    40W,65W,80W,and120Woptions

    AMDOpteron2000seriesDualand

    QuadCore

    68Wand95Woptions

    Chipset Intel5000P(Blackford) NVIDIAMCP55

    MemorySlots 8FullyBufferedDIMMs(667MHz) 8DDR2(667/800MHz)

    MemoryCapacity 32GB(4GBx8)atlaunch

    64GB

    (8GB

    x

    8)

    planned

    Q108

    32GB(4GBx8)atlaunch

    64GB

    (8GB

    x

    8)

    planned

    Q108

    LOM 2xGEwithhardwareTCP/IP

    OffloadEngineandiSCSIFirmware

    Boot

    UpgradetofulliSCSIoffloadvia

    licensekey

    2xGEwithhardwareTCP/IPOffload

    EngineandiSCSIFirmwareBoot

    UpgradetofulliSCSIoffloadvia

    licensekey

  • 5/22/2018 Pedge m1000e

    16/42

    ServerModule M600 M605

    FabricExpansion 2x8lanePCIemezzanine

    daughtercards

    1. DualportGEw/TOE

    2. DualPortFC4(Emulex&Qlogic)

    3. DualPort4xDDRInfiniBand

    2x8lanePCIemezzanine

    daughtercards

    1. DualportGEw/TOE(*duetoPCIecontrollerlimitationsthedual5708S

    TOEbasedcardcannotbeusedin

    FabricC. Dellplanstoreleasean

    integrateddualTOEsolutionto

    resolvethislimitation.)

    2. DualPortFC4(Emulex&Qlogic)

    3. DualPort4xDDRInfiniBand

    BaseboardManagement iDRACw/IPMI2.0+vMedia+

    vKVM

    iDRACw/IPMI2.0+vMedia+vKVM

    LocalStorage

    Controller

    OptionsSATA

    (chipset

    based

    no

    RAID

    or

    hotplug)

    SAS6/IR(R0/1)

    CERC6/i(R0/1w/Cache)

    SATA(chipset

    based

    no

    RAID

    or

    hotplug)

    SAS6/IR(R0/1)

    CERC6/i(R0/1w/Cache)

    LocalStorageHDD 2x2.5inchhotpluggableSASor

    SATA

    2x2.5inchhotpluggableSASorSATA

    Video ATIRN50 ATIRN50

    USB 2xUSB2.0bootableportsonfront

    panelforfloppy/CD/DVD/Memory

    2xUSB2.0bootableportsonfront

    panelforUSB

    floppy/CD/DVD/Memory

    Console

    VirtualKVM

    via

    iDRAC

    IPMISerialoverLAN(SoL)via

    iDRAC

    RearmountediKVMswitch

    ports(tierable)

    FrontKVMportsonmodular

    enclosurecontrolpanel

    VirtualKVM

    via

    iDRAC

    IPMISerialoverLAN(SoL)via

    iDRAC

    RearmountediKVMswitchports

    (tierable)

    FrontKVMportsonmodular

    enclosurecontrolpanel

    HAClustering FibreChannelandiSCSIbased

    clusteringoptions

    FibreChannelandiSCSIbased

    clusteringoptions

    OperatingSystems MicrosoftW2K3andW2K3R2,Red

    HatEnterpriseLinux4/5,SuSELinux

    EnterpriseServer9/10

    MicrosoftW2K3andW2K3R2,Red

    HatEnterpriseLinux4/5,SuSELinux

    EnterpriseServer9/10

    Page16

    Table4HalfHeightServerModuleOptions

  • 5/22/2018 Pedge m1000e

    17/42

    PowerEdge M1000e Midplane and I/O

    PowerEdge M1000e Front View

    Server

    modules

    are

    accessible

    from

    the

    front

    of

    the

    PowerEdge

    M1000e

    enclosure.

    At

    the

    bottom

    oftheenclosureisaflipoutmultipleangleLCDscreenforlocalsystemsmanagement

    configuration,systeminformation,andstatus. ThefrontoftheenclosurealsocontainstwoUSB

    connectionsforUSBkeyboardandmouse,avideoconnectionandthesystempowerbutton. The

    frontcontrolpanelsUSBandvideoportsworkonlywhentheiKVMmoduleisinstalled,asthe

    iKVMprovidesthecapabilitytoswitchtheKVMbetweentheblades.

    Figure9PowerEdgeM1000eFrontView

    TheDellPowerEdgeM1000esupportsuptosixteenHalfHeightservermodules. Thechassis

    guideandretentionfeaturesaredesignedsuchthatalternativemoduleformfactorsarepossible.

    See

    the

    Server

    Module

    section

    for

    more

    details.

    The

    chassis

    architecture

    is

    flexible

    enough

    that

    server,storageorothertypesoffrontloadingmodulesarepossible.

    Notvisiblyobvious,butimportantnonetheless,arefreshairplenumsatbothtopandbottomof

    thechassis. Thebottomfreshairplenumprovidesnonpreheatedairtothepowersupplies,

    enablinggroundbreakinglevelsofpowerdensity. Thetopfreshairplenumprovidesnon

    preheatedairtotheCMC,iKVMandI/OModules. SeetheCoolingsectionformoredetails.

    Page17

  • 5/22/2018 Pedge m1000e

    18/42

    Feature Parameter

    ChassisSize 10Uhighrackmount

    Bladesper

    Chassis

    16

    Half

    Height,

    8

    Full

    Height

    TotalBladesina42URack 64HalfHeight,32FullHeight

    TotalI/OModuleBays 6(3redundantordualfabrics)

    TotalPowerSupplies 6(3+3redundant)

    TotalFanModules 9(8+1redundant)

    ManagementModulesandInterfaces 2CMCs(1+1redundant),1iKVM,Front

    ControlPanel,GraphicalLCDControlPanel

    Width,notincludingrackears 447.5mm

    Height 440.5mm

    Depth,RearofEIAFlangetoRearofChassis 753.6mm

    TotalSystemDepth(FrontBezeltoPSLatch): 835.99mm

    Table5PowerEdgeM1000eModularSystemFeaturesandParameters

    PowerEdge M1000e Midplane

    Thoughhiddenfromviewinanactivelyrunningsystem,themidplaneisthefocalpointforall

    connectivitywithinthePowerEdgeM1000eModularSystem. Themidplaneisalargeprinted

    circuitboardprovidingpowerdistribution,fabricconnectivityandsystemmanagement

    infrastructure. Additionallyitallowsairflowpathsforthefronttobackcoolingsystemthrough

    ventilationholes.

    Asisrequisiteforfaulttolerantsystems,thePowerEdgeM1000emidplaneiscompletelypassive,

    withnohiddenstackingmidplanesorinterposerswithactivecomponents. I/OFabricsand

    system

    management

    are

    fully

    redundant

    from

    each

    hot

    pluggable

    item.

    The

    system

    managementEthernetfabricisfullyredundantwhentwoCMCsareinstalled,withtwopointto

    pointconnectionsfromeachservermodule.

    I/Ofabricsareroutedthrough10Gbpscapablehighspeedconnectorsanddielectricmaterial. All

    tracesaredifferential100ohmcharacteristicimpedance. Placementofconnectorsandpluggable

    componentswasdeterminedbaseduponminimumroutingdistanceforhighspeedcriticalnets.

    HighspeedI/Oconnectorholesarebackdrilledinthemidplanetominimizeelectricalstublength

    andimprovehighspeedsignalintegrity. MidplanematerialisanimprovedFR408dielectric

    material,providingbetterhighfrequencytransmissioncharacteristicsandlowerattenuation.The

    I/OChannelshavebeensimulatedto10GBASEKRchannelmodels. Perindustrystandard

    requirements,fabricsinternallysupportaBitErrorRateof1012orbetter. Atthesystemlevelthis

    meansthat

    a

    high

    level

    of

    investment

    has

    been

    made

    to

    insure

    scalable

    bandwidth,

    for

    current

    andfuturegenerationsofserverandinfrastructure.

    Page18

    Themidplaneservesastransportforapatentpendingtimedivisionmultiplexedserialbusfor

    GeneralPurposeI/Oreduction. ThisserialbuscontributesgreatlytothemidplanesI/Olane

    countreduction,whichistypicallyburdenedwithasignificantI/Opinandroutingchannelcount

    oflargelystaticorlowspeedfunctions. Forinstance,allFibreChannelI/OPassthroughmodule

    LEDandSFPstatusinformationiscarriedoverthisbus,whichaloneeliminatesoverone

    hundredpointtopointconnectionsthatwouldotherwiseberequired.Thetimedivision

  • 5/22/2018 Pedge m1000e

    19/42

    multiplexedserialbusisfullyredundant,withhealthmonitoring,separatelinksperCMCand

    errorcheckingacrossalldata.

    Figure10PowerEdgeM1000eMidplaneFrontView

    Thesystemisdesignedforreceptaclesonallmidplaneconnectorsandpinsonallpluggable

    components,soanypotentialforbentpinsislimitedtothepluggablefieldreplaceableunit,not

    tothesystem. ThiscontributestothehighreliabilityanduptimeofthePowerEdgeM1000e

    modularsystem.

    Themidplaneisphysicallyattachedtotheenclosurefrontstructuralelement. Itisalignedby

    guidepinsandedgesinall3axes. Thisprovidesclosetolerancealignmentbetweentheserver

    modulesandtheirmidplaneconnections.

    Themidplanehasbeencarefullydesignedtominimizetheimpacttotheoverallsystemairflow.

    Themidplanedesignwasevaluatedatvariouspointsthroughoutthedesigncycletomake

    tradeoffsbetweenthermalperformanceandmidplanecosts. Earlyanalysisofthemidplane

    showedlargeincreasesinairflowduetoamovefrom15%to20%midplaneopening. The

    ventilationholesresultingfromthisexpansionareseeninFigure10andFigure11. These

    changesaddedtothedemandingrequirementsofthemidplanedesign,butprovidedthebest

    overallsystemsolution.

    Page19

    BladeSignalConnectors

    Blade

    PowerConnectors

  • 5/22/2018 Pedge m1000e

    20/42

    Figure11PowerEdgeM1000eMidplaneRearView

    Whileyoumightthinkthatmultiplemidplanesarebetterfortheproduct,therereallyisno

    systemadvantageifthemidplaneisproperlydesigned. Allmidplaneroutingisfullyisolated,

    supportingallchassispower,fabric,systemmanagementandfaulttolerancerequirements.The

    singleintegratedmidplanealsosimplifiesmanufacturingassembly,reducingnetacquisitioncost

    tothecustomer.

    PowerEdge M1000e I/O

    I/Ois

    fully

    scalable

    to

    current

    and

    future

    generations

    of

    server

    modules

    and

    I/O

    Modules.

    There

    arethreeredundantmultilanefabricsinthesystem,asillustratedinFigure12.

    FabricAisdedicatedtoGigabitEthernet. Thoughinitialservermodulereleasesaredesignedas

    dualGigabitEthernetLOMcontrollersontheservermoduleplanar,themidplaneisenabledto

    supportuptofourGigabitEthernetlinksperservermoduleonFabricA. Potentialdata

    bandwidthforFabricAis4GbpsperHalfHeightservermodule.

    Page20

    PowerSupplyConnectors

    FanConnectors

    I/O ModuleConnectors

    LCD andControlPanelConnector

    CMCConnectors iKVM

    Connector

  • 5/22/2018 Pedge m1000e

    21/42

    OneimportantnewcapabilityinthePowerEdgeM1000eisfull10/100/1000MEthernetsupport

    whenusingEthernetpassthroughmodules. Inthepast,customerswerelimitedtoconnecting

    onlytoexternalswitcheswith1000Mports.Nowthesystemcanconnecttolegacyinfrastructure

    whetherusing

    Ethernet

    passthrough

    or

    switch

    technology.

    This

    technical

    advance

    uses

    in

    band

    signalingon1000BASEKXtransport,andrequiresnouserinteractionforenablement.

    External

    Fabric

    Connec-

    tions

    M1000e MidplaneHalf Height Modular Server (1 of 16)

    Fabric B

    Mezzanine

    Fabric C

    Mezzanine

    Fabric A

    LOM

    Fabric A1

    Ethernet

    I/O Module

    Fabric A2

    EthernetI/O Module

    Fabric B1

    I/O Module

    Fabric B2

    I/O Module

    Fabric C1

    I/O Module

    Fabric C2

    I/O Module

    1-2 lane

    1-4 lane

    1-4 lane

    1-2 lane

    MCH/

    IOH

    4-8 lane

    PCIe

    8 lane

    PCIe

    8 lane

    PCIe

    Half Height Modular Server (16 of 16)

    Fabric B

    Mezzanine

    Fabric C

    Mezzanine

    Fabric A

    LOM

    MCH/

    IOH

    4-8 lane

    PCIe

    8 lane

    PCIe

    8 lane

    PCIe

    1-4 lane

    1-4 lane

    1-2 lane

    1-2 lane

    1-4 lane

    1-4 lane

    1-4 lane

    1-4 lane

    CPU

    CPU

    CPU

    CPU

    Figure12HighSpeedI/OArchitecture

    Page21

    FabricBandCareidentical,fullycustomizablefabrics,routedastwosetsoffourlanesfrom

    mezzaninecardsontheservermodulestotheI/OModulesintherearofthechassis. Supported

    bandwidthrangesfrom1to10Gbpsperlanedependingonthefabrictypeused. Table6shows

  • 5/22/2018 Pedge m1000e

    22/42

    comparativebandwidthbetweencurrentandforthcomingfabrics. SymbolRateisdefinedasthe

    maximumfabricspeedwithalldataencodedasperitsencodingmechanism. DataRateisthe

    maximumamountofactualdatabitsthatcanbetransportedonthefabric. Somefabricsare

    definedas

    single

    lane

    only,

    while

    others

    are

    defined

    in

    various

    lane

    increments

    for

    link

    to

    lane

    aggregation,inordertoprovideincreasedtotalbandwidth.

    Aseachmezzaninecardisconnectedbyan8lanePCIelinktotheserverchipset,thereareno

    throttlepointsinthesystemwhereI/Obandwidthisconstricted. Whenmultilane10GBASEKR

    becomesareality,DellanticipatesservermoduleswithPCIeGen2orbettersupport,providing

    fullendtoendI/ObandwidthfromtheservermodulestotheI/OModules.

    Fabric Encoding

    SymbolRate

    PerLane

    (Gbps)

    DataRate

    PerLane

    (Gbps)

    DataRate

    PerLink

    (Gbps)

    LanesPerLinkPer

    IndustrySpecification

    PCIeGen1 8B/10B 2.5 2 8

    (4

    lane) 1,

    2,

    4,

    8,

    12,

    16,

    32PCIeGen2 8B/10B 5 4 16(4lane) 1,2,4,8,12,16,32

    PCIeGen3 scrambling 8 8 32(4lane) 1,2,4,8,12,16,32

    SATA3G 8B/10B 3 2.4 2.4 1

    SATA6G 8B/10B 6 4.8 4.8 1

    SAS3G 8B/10B 3 2.4 2.4 1Any

    SAS6G 8B/10B 6 4.8 4.8 1Any

    FC4G 8B/10B 4.25 3.4 3.4 1

    FC8G

    8B/10B+

    scrambling 8.5 6.8 6.8 1

    IBSDR 8B/10B 2.5 2 8(4lane) 4,12

    IBDDR 8B/10B 5 4 16(4lane) 4,12

    IBQDR 8B/10B 10 8 32(4lane) 4,12

    GE:1000BASE

    KX 8B/10B 1.25 1 1 1

    10GE:10GBASEKX4 8B/10B 3.125 2.5 10(4lane) 4

    10GE:10GBASEKR 64B/66B 10.3125 10 10 1

    Table6RelativeComparisonofIndustryStandardHighSpeedI/O

    Ethernet Bandwidth Migration Path

    Page22

    ThebandwidthgrowthpathforEthernetisfromGEto10GE,supportedthroughthreebackplane

    standardsdefinedbytheIEEE802.3apCommittee. 1000BASEKXsupportsGEthrough1.25

    Gbpsdifferentialbackplanesignaling. 10GBASEKX4supports10GEthroughfourlanesof3.125

    Gbpsdifferentialbackplanesignaling.Thefourlanesareaggregatedtoprovidethesumtotal10

    Gbpsbandwidthrequirement. 10GBASEKRisanemergingIEEE802.3apstandardthatsupports

    10GEthroughasinglelaneof10.3125Gbpsdifferentialbackplanesignaling. Intermsof

    customerimpact,thismeansthattodaysmodularsystemsusing10GBASEKX4arelimitedto

    one10GEchannelper4lanesofinternaldifferentialsignaling,whileinthefuturethosesame

    lanescancarrymultiplelanesof10GBASEKRandprovidearesultantincreaseintotalsystem

    throughput. SeeFigure13foranillustrationofthe10GEgrowthpath.Thisgrowthpathapplies

    tobothFabricsBandC.

  • 5/22/2018 Pedge m1000e

    23/42

    GEfabricsusesinglelanesperlinkover1000BASEKX. Initially10GEistobesupportedthrough

    10GBASEKX4,whichlikeInifiniBandusesfourlanesperlink.Astechnologymaturesand

    enableshighlevelintegration,10GBASEKRisexpectedtoreplace10GBASEKX4asthe10GE

    backplanetransport

    of

    choice,

    enabling

    10

    Gbps

    over

    single

    lanes

    within

    the

    system

    midplane.

    Finally,multiplelanesof10GBASERpermezzaninecanprovideexpandedEthernetbandwidth

    upto80Gbpsperfabricmezzanine.

    2x10GE

    Mezzanine using

    4 lane

    10GBASE-KX4

    10GBASE-KX4

    takes 4 lanes to

    achieve 10 Gbps

    data bandwidth,

    2 links per

    mezzanine for 20

    Gbps total

    2x10GE

    Mezzanine using

    1 lane

    10GBASE-KR

    10GBASE-KR

    takes 1 lane to

    achieve 10 Gbps

    data bandwidth,

    2 links per

    mezzanine for 20

    Gpbs total

    2x4x10GE

    Mezzanine using

    multilane

    10GBASE-KR

    Up to 4 lanes of

    10GBASE-KR

    can achieve 40

    Gbps data

    bandwidth,

    8 links per

    mezzanine for 80

    Gpbs total

    Figure1310GEPotentialGrowthPath

    Total System Bandwidth

    Inmostcases,onlyasmallpercentageofthisbandwidthisusedinasystem,asillustratedin

    Figure

    14.

    A

    typical

    configuration

    is

    defined

    as

    4

    lanes

    of

    GE

    and

    2

    lanes

    of

    Fibre

    Channel

    per

    servermodule,anduseslessthan10%ofthesystemspotentialbandwidth. Anexampleofa

    unifiedEthernetconfiguration,aggregatingallnetwork,storageandInterprocess

    Communications onEthernetlinks,isshownas2lanesofGEand4lanesof10GEperserver

    module,using25.7%ofsystempotentialbandwidth. AnexampleofaHighPerformance

    0.0% 20.0% 40.0% 60.0% 80.0% 100.0%

    MaximumPotentialBandwidth

    HighPerformanceSAN/IPC

    UnifiedEthernet

    Typical

    System

    Configuration

    TotalSystemBandwidth

    Page23

    Figure14PowerEdgeM1000eModularServerSystemBandwidth

  • 5/22/2018 Pedge m1000e

    24/42

    SAN/IPCconfigurationisshownas2lanesofGE,2lanesofFC8,and2lanesofQDRInfiniBand,

    andstillusesonly35%ofPowerEdgeM1000epotentialbandwidth. Theresultsofthegraph

    showthatthePowerEdgeM1000eisfullyscalableforI/Oinfrastructureneedsformany

    generationsof

    servers

    and

    switches.

    AssumingafullpopulationofGEand10GbElanesinFabricsA,BandC,thePowerEdgeM1000e

    candelivertheoreticaltotalbandwidthof5.44Terabitspersecond. However,thisextreme

    bandwidthcomparisonmaynoteverbefullyrealizedinrealworldapplicationusage. Italso

    doesnotaddressfactorssuchasflexibilityoffabricmigrationpaths.

    ThePowerEdgeM1000eisdesignedforfullsupportofallnear,mediumandlongtermI/O

    infrastructureneeds. WhilethePowerEdgeM1000esystemsbandwidthcapabilitiesleadthe

    industry,thePowerEdgeM1000eisalsointelligentlydesignedformaximumcost,flexibilityand

    performancebenefit. Anexampleofthisistheroutingofdual4lanepathsforFabricsBandC,

    anddual2lanepathsinFabricA. Inthenearterm,10GBASEKX4routingsupports10GE

    connectivity. 10GBASEKX4routingusesall4lanes,eachrunningata2.5Gbpsdatarateto

    achievetotallinkdatabandwidthof10Gbps. Todayandintheneartermfuture,10GBASEKX4

    isthemostcosteffective,ubiquitoussolutionfor10GEfabricsovermidplanes. Afully

    configuredsystemsupports2x2GEand2x210GEimplementationswith10GBASEKX4routing,

    supportingtheleadingedgeofunifiednetworktopologies. Suchaconfigurationcould,for

    instance,provideredundant10GElinksperbladefortraditionalnetworktraffic,andanotherset

    ofredundant10GElinksforiSCSIorFibreChannelOverEthernet(FCoE)networkedstorage.

    I/OModulesarefullycompatibleacrossslots. WhileFabricAisdedicatedtotheservermodule

    LOMs,requiringEthernetswitchorpassthroughmodulesforI/OslotsA1andA2,FabricsBand

    CcanbefreelypopulatedwithEthernet,FibreChannelorInfiniBandsolutions. AspartofDells

    efforttosimplifyIT,thereisnoconfusingsupportmatrixtofollowformezzaninesorI/O

    modules

    and

    no

    multiplication

    of

    mezzanine

    and

    I/O

    Module

    form

    factors

    and

    design

    standards,

    withresultantcompatibilityorconfigurationconcerns.Dellsupportsonemezzaninedesign

    standardandoneI/OModuledesignstandardfortruemodularcomputing.

    PowerEdge M1000e Rear View

    TherearofthePowerEdgeM1000eEnclosurecontainssystemmanagement,cooling,powerand

    I/Ocomponents.AtthetopoftheenclosureareslotsfortwoChassisManagementCardsandone

    integratedKeyboard/Video/Mouseswitch. TheenclosureshipsbydefaultwithasingleCMC,

    withthecustomerhavingtheoptionofaddingasecondCMCtoprovideafullyredundant,

    activestandbyfaulttolerantsolutionformanagementaccessandcontrol.

    InterleavedinthecenterofthechassisarefansandI/OModules. Thisarrangementoptimizesthe

    balanceof

    airflow

    through

    the

    system,

    allowing

    lower

    pressure

    buildup

    in

    the

    system

    and

    resultinginlowerairflowrequirementsforthefans.

    Page24

    I/OModulesareusedaspairs,withtwomodulesservicingeachservermodulefabricandfully

    redundant.I/OModulesmaybepassthroughsorswitches.Passthroughmodulesprovidedirect

    1:1connectivityfromeachLOM/mezzaninecardportoneachservermoduletotheexternal

    network. SwitchesprovideanefficientwaytoconsolidatelinksfromtheLOMorMezzanine

    cardsontheservermodulestouplinksintothecustomersnetwork. TheEnclosureaboveis

    shownwith10/100/1000MbEthernetandFC4PassthroughModules.Figure16showsthe

  • 5/22/2018 Pedge m1000e

    25/42

    differencebetweenthesetwotypesofIOMs.Figure17showstheDellGigabitEthernet

    PassthroughModule.

    Figure15PowerEdgeM1000eRearView

    TheDellandCiscoEthernetswitchesforthePowerEdgeM1000eenclosureuniquelydemonstrate

    thelevel

    of

    forethought

    that

    went

    into

    the

    design

    of

    the

    I/O

    modules.

    These

    Ethernet

    switch

    modulesintroducesubmoduleI/Oexpansionformaximumflexibilityandextension,acapability

    neverbeforeofferedinmodularsystems.Theyprovideasinglehardwaredesignthatallows

    scalingfromalowcostallGEsolution,toasolutionthatutilizesswitchstackingtechnologiesto

    interconnectmultipleswitcheswithinorbetweenchassis,and/ortoasolutionthatsupportsallof

    theseplus10GEuplinkstothecorenetworkwithflexibleinterfacetypes,bothcopperand

    optical. Figure18showstheDellPowerConnectGigabitEthernetSwitchModulewithitstwo

    submodulebaysavailableforswitchI/Oexpansion.

    Page25

    iKVMCMCs1 blank

    Fans

    I/O Modules(blank inmiddle)

    Power Supplies(3 blanks to left)

  • 5/22/2018 Pedge m1000e

    26/42

    External Fabric ConnectionsM1000e Midplane

    Passthrough Module

    Switch Module

    Switch

    PHY

    PHY

    PHY

    PHY

    PHY

    .server2data.

    .server1data.

    .server4data.

    .server3data.

    .server1data.

    .server2data.

    .server3data.

    .server4data.

    .server5data.

    .server6data.

    .server7data.

    .server8data.

    .server5data.server6data.server7data.server8data.

    Representation of

    data passing over the

    midplane from the

    server module

    mezzanine cards.

    Only four ports are

    shown per IOM.

    A Passthrough Module electrically buffers the

    signal and may perform some low level link layer

    conversion. The data is not touched and all

    data streams maintain a one to one

    correspondence to the data from the mezzanine.

    A Switch Module logically combines multiple

    streams of data onto one or more streams

    coming out from the system. The data is then

    electrically buffered and formatted for the

    proper external interface.

    Figure16DifferencebetweenPassthroughandSwitchModules

    Figure17I/OModule,DellPowerEdgeGigabitEthernetPassthrough

    Page26

    Figure18I/OModule,DellPowerEdgeGigabitEthernetSwitch

  • 5/22/2018 Pedge m1000e

    27/42

    TheinitialoptionsforI/OModulesareoutlinedinTable7,Table8andTable9.Dellplansfuture

    I/OModulereleasessupportingalltechnologyupgradepaths,including8GFC,QDRIBand

    10GE.

    I/OModule/Features

    10/100/1000MEthernet

    Passthrough

    DellPowerConnectM6220EthernetSwitch

    CiscoCatalystBladeSwitch3032*,3130SG*,

    3130SX*

    Internal

    ports

    16 16 16

    External

    ports

    16x

    10/100/1000M

    4xfixedcopper10/100/1000M

    plus2ofthefollowingoption

    modules:

    1)2x24Gstackingports

    2)2x10GOptical(XFPSR/LR)

    uplinks

    3)2x10GcopperCX4uplinks

    4)2

    x

    10GBASE

    T

    copper

    uplinks*

    3032:4fixedcopper10/100/1000M+2

    optionalbayseachsupporting2x1G

    copperoropticalSFPs

    3130SG:Above+2x32GStackwise

    PlusStackingports

    3130SX:

    Above+upto2xX2modulesfor10G

    CX4or

    SR/LR

    uplinks

    Speed 10/100/1000M Internalports 1G

    Externalports

    FixedRJ45 10/100/1000M

    CX4uplink 10GE

    OpticalXFP 10GE

    10GBASET10GE

    Internalports 1G

    ExternalportsFixedRJ45

    10/100/1000M

    X2CX4uplink 10GE

    X2SR/LRMOptical 10GE

    Manage

    ability

    N/A CLI/Web CLI/Web,Ciscomgmttools

    Features Supportfor

    10/100/1000M

    speeds(hardset

    andauto

    negotiation)

    Layer3routing(OSPF,RIP,

    VRRP),10GEconnections,

    Stacking,IPv6,Layer2/3QOS,

    AccessControlLists

    L2switching,BaseL3routing(static

    routes,RIP),AccessControllists,L2/3

    QoS

    OptionalupgradestoIPServices(Adv.

    L3protocolsupport)andAdvancedIP

    Services(IPv6)

    *planned

    Table7EthernetI/OModuleOptions

    I/OModule/

    Features

    EmulexFC4Passthrough Brocade4424AccessGatewayandFC4Switch

    Internalports 16 8/16

    Externalports 16x4GSFP 4/8x4GSFP

    Speed

    1/2/4G

    1/2/4G

    Manageability N/A CLI/Web,

    BrocadeandEMCFabricManagementtools

    Features N/A AccessGatewayModeenablesNPIVfunctionalityon

    externalports(enhancedinteroperability,simplified

    setup,doesntconsumeanFCdomain)

    12or24portenablementoptions

    Page27

    Table8FibreChannelI/OModuleOptions

  • 5/22/2018 Pedge m1000e

    28/42

    I/OModule/

    Features

    CiscoSFSM7000InfiniBandSwitch*

    Internalports 16

    External

    ports

    8

    Copper

    or

    Optical

    ports

    Speed 4xDDR

    Manageability Fabricbasedmanagement

    *planned

    Table9InfiniBandI/OModuleOption

    Atthebottomofthechassisarethesystempowersupplies. Seethepowersectionformore

    detailsonpowersupplyoperationandthepowersysteminfrastructure.

    Cable Management

    Oneoftheadvantagesofamodularserversystemisthereductionincablemanagementneeds

    withina

    rack

    system.

    The

    inclusion

    of

    fabric

    switches,

    integrated

    KVM

    and

    system

    management

    aggregationattheCMCsprovidessixfoldorbettercablereduction.Table10showsa

    comparisonofatypicalreductionavailablewhenusingthePowerEdgeM1000eModularsystem

    withintegratedswitches,comparedtotraditionalrackandstackcomponents.The

    configurationinthetableassumesaserverwithfourEthernetportsandtwoFibreChannelports.

    InsupportofthePowerEdgeM1000eDellisreleasinganimprovedmodularsystemcable

    managementsystemtoeasesysteminstallationinDellorotherindustrystandardracks.

    Component RackHeight ACpower

    cables

    Ethernet

    Cables

    FCCables KVM

    Cables

    2socketserver 1Ux16 2x16 4x16 2x16 USBx16+

    VGAx16

    KVM 1U 1 USBx1+

    VGAx1

    Ethernet

    Switches

    1Ux4 1x4 4x4

    FCSwitches 1Ux2 1x2 2x2

    TotalRack 23Uheight 39ACcables 72Ethernet

    Cables

    36FC

    Cables

    USBx17+

    VGAx17

    PowerEdge

    M1000e

    Equivalent

    10Uheight 6ACcables 16Ethernet

    Cables

    4FCCables USBx1+

    VGAx1

    Table10TypicalModularServerSystemRackHeightandCableReduction

    Power

    Page28

    Dellleadstheindustryinpower/performanceefficiencywithitsEnergySmartproducts,andthe

    PowerEdgeM1000econtinuesthistradition. Amodularsystemhasmanyadvantagesover

    standardrackmountserversintermsofpoweroptimization,andthisaspectwasafocalpoint

    throughoutthePowerEdgeM1000esconceptualizationanddevelopment. Thekeyareasof

    interestarePowerDeliveryandPowerManagement.

  • 5/22/2018 Pedge m1000e

    29/42

    Power Delivery

    InsupportofthePowerEdgeM1000e,DellintroducesaSinglePhase60AmpPDUandaThree

    Phase30AmpPDU. Becausethreephase30Amppoweristhemostcommondatacenterhigh

    powerfeed,

    the

    system

    has

    been

    sized

    to

    fit

    perfectly

    within

    that

    envelope,

    taking

    one

    three

    phase30Ampfeedpersetofthreepowersuppliesunderanyloadingcondition.Thethreephase

    powergridiscomplementedbythe3+3redundantpowersystemofthePowerEdgeM1000e.

    FeaturesforthePDUsareshowninTable11.PDUssupportworldwidestandardhighpower

    connectionsforeaseofinstallation,andstandardzeroUverticalmountingwithinDellracks.

    TypicalconfigurationsforthePowerEdgeM1000ewithsinglephaseandthreephasePDUsare

    showninFigure19andFigure20.

    Feature SinglePhasePDU ThreePhasePDU

    Mounting 1UHorizontalorzeroUvertical

    Outlets 3xC19

    InputVoltage Singlephase200to240VAC

    nominal

    3phase200to240VAC

    nominal

    LineFrequency 47to63Hertz

    RecommendedACservice 60Amps 30Amps(NA/Japan),32

    Amps(International)

    FixedInputPlug/CordRating IEC30960APin&Sleeve

    Plug

    NEMAL1530P(NA/Japan),

    IEC3094pole,4wire,380415

    VAC,32A(International)

    OutputRating

    Voltage

    200

    240

    VAC

    60/50

    Hz,

    1

    phase

    OutputRatingCurrent

    (IEC320C19)

    16Amps

    CircuitBreaker,OverCurrent

    Protection

    20A,peroutletreceptacle

    Table11PowerEdgeM1000eSinglePhaseandThreePhasePDUOptions

    ThepowerdistributioninsidethePowerEdgeM1000eModularServerSystemconsistsofa3+3

    redundantpowersupplysystem,locatedintherearbottomofthechassis. Eachpowersupplyis

    ratedat2360W. Withcurrentsharingbetweenpowersupplies,totalsystemredundantpoweris

    approximately6700W

    in

    a

    3+3

    power

    supply

    configuration.

    Page29

    ThePowerEdgeM1000eprovidesindustryleadingpowerefficiencyanddensity,accomplished

    throughhighlyefficientcomponents,improveddesigntechniquesandafreshairplenumthat

    reducestheairtemperaturetothepowersupplycomponents. Loweroperatingtemperature

    equatestohigherpowerdensityforthepowersupply,exceeding21Wattspercubicinch,and

    higherpowerefficiency,betterthan86%at20%load,andhigheratheavierloads,approaching

    91%efficiencyundernormaloperatingconditions.

  • 5/22/2018 Pedge m1000e

    30/42

    Figure19PowerEdgeM1000eSinglePhasePDUPowerSystem

    PSU-1

    2360W

    PSU-6

    2360W

    PSU-52360W

    PSU-4

    2360W

    PSU-3

    2360W

    PSU-2

    2360W

    Three

    Phase

    30 Amp

    PDU-1

    12

    Three Phase

    Power Cord

    (30 Amp

    Service)

    Three

    Phase30 Amp

    PDU-2

    Three Phase

    Power Cord(30 Amp

    Service)

    3

    123

    13.8 Amps

    Single Phase

    24 AmpsThree Phase

    Page30

    Figure20PowerEdgeM1000eThreePhasePDUPowerSystem

  • 5/22/2018 Pedge m1000e

    31/42

    Figure21PowerEdgeM1000e2360WattPowerSupply

    PowerefficiencyinthePowerEdgeM1000edoesnotstopwiththepowersupply. Everyaspectof

    efficiencyhasbeentweakedandimprovedfrompreviousdesigns,addingmorecoppertoPC

    boardpowerplanestoreduceI2Rlosses,improvedinductorsandothercomponents,increasing

    efficienciesofDCDCconverters,andreplacingsomelinearvoltageregulatorswithmore

    efficientswitchingregulators.

    PowerredundancyinthePowerEdgeM1000esupportsanynecessaryusagemodel. The

    PowerEdgeM1000erequiresthree2360Wattpowersuppliestopowerafullypopulatedsystem

    orsixpowersuppliesinafullyredundantsystem. Figure22showstypicalpowerredundancy

    models. IntheN+Npowersupplyconfiguration,thesystemprovidesprotectionagainstACgrid

    lossorpowersupplyfailures. Ifonepowergridfails,threepowersupplieslosetheirACsource,

    andthethreepowersuppliesontheothergridremainpowered,providingsufficientpowerfor

    thesystemtocontinuerunning.IntheN+1configurationonlypowersupplyfailuresare

    protected,notgridfailures. Thelikelihoodofmultiplepowersuppliesfailingatthesametimeis

    remote. IntheN+0configurationthereisnopowerprotectionandanyprotectionmustbe

    providedatthenodeorchassislevel. TypicallythiscaseisanHPCCorotherclustered

    environmentwhereredundantpowerisnotaconcern,sincetheparallelismoftheprocessing

    nodesacrossmultiplesystemchassisprovidesalltheredundancythatisnecessary.

    Themidplanecarriesall12VoltDCpowerforthesystem,bothmainpowerandstandbypower.

    TheCMCs,LCDandControlPanelarepoweredsolelyby12VoltStandbypower,insuringthat

    chassislevelmanagementisoperationalinthechassisstandbystate,wheneverACpoweris

    present.Theservermodules,I/OModules,Fans,andiKVMarepoweredsolelyby12VoltMain

    power.

    Page31

  • 5/22/2018 Pedge m1000e

    32/42

    Page32

    Figure22PowerRedundancyModes

  • 5/22/2018 Pedge m1000e

    33/42

    Figure23PowerArchitecture

    Power Management

    Powerisnolongerjustaboutpowerdelivery,itisalsoaboutpowermanagement. The

    PowerEdgeM1000eSystemaddsanumberofadvancedpowermanagementfeatures. Mostof

    thesefeaturesoperatetransparentlytotheuser,whileothersrequireonlyaonetimeselectionof

    desiredoperating

    modes.

    Sharedpowertakesadvantageofthelargenumberofresourcesinthemodularserver,

    distributingpoweracrossthesystemwithouttheexcessmarginrequiredindedicatedrack

    mountserversandswitches. ThePowerEdgeM1000eintroducesanadvancedpowerbudgeting

    feature,controlledbytheCMCandnegotiatedinconjunctionwiththeiDRAConeveryserver

    module. Priortoanyservermodulepoweringup,throughanyofitspowerupmechanismssuch

    asACrecovery,WOLorasimplepowerbuttonpress,theservermoduleiDRACperformsa

    sophisticatedpowerbudgetinventoryfortheservermodule,baseduponitsconfigurationof

    CPUs,memory,I/Oandlocalstorage. Oncethisnumberisgenerated,theiDRACcommunicates

    thepowerbudgetinventorytotheCMC,whichconfirmstheavailabilityofpowerfromthe

    systemlevel,baseduponatotalchassispowerinventory,includingpowersupplies,iKVM,I/O

    Modules,fans

    and

    server

    modules.

    Since

    the

    CMC

    controls

    when

    every

    modular

    system

    element

    powerson,itcannowsetpowerpoliciesonasystemlevel.

    Page33

    IncoordinationwiththeCMC,iDRAChardwareconstantlymonitorsactualpowerconsumption

    ateachservermodule. Thispowermeasurementisusedlocallybytheservermoduletoinsure

    thatitsinstantaneouspowerconsumptionneverexceedsthebudgetedamount. Whilethe

    systemadministratormaynevernoticethesefeaturesinaction,whattheyenableisamore

    aggressiveutilizationofthesharedsystempowerresources. Nolongeristhesystemflying

    blindinregardstopowerconsumption,andthereisnodangerofexceedingpowercapacity

  • 5/22/2018 Pedge m1000e

    34/42

    availability,whichcouldresultinaspontaneousactivationofpowersupplyovercurrent

    protectionwithoutthesefeatures.

    Thesystemadministratorcanalsosetprioritiesforeachservermodule. Thepriorityworksin

    conjunctionwith

    the

    CMC

    power

    budgeting

    and

    iDRAC

    power

    monitoring

    to

    insure

    that

    the

    lowestprioritybladesarethefirsttoenteranypoweroptimizationmode,shouldconditions

    warranttheactivationofthisfeature.

    ThePowerEdgeM1000eintroducescompliancetoPMBusSpecification1.1,usingthisnewpower

    managementstandardforstatus,measurementandcontrol. PowerEdgeM1000epowersupplies

    continuouslymonitorACinputcurrent,voltageandpower,enablingexposureofdatatoDell

    OpenManageITAssistantortootherenterpriselevelmanagementtools. Realtimepower

    consumptionisnowviewablepersystem.

    Cooling

    The

    cooling

    strategy

    for

    the

    PowerEdge

    M1000e

    supports

    a

    low

    impedance,

    high

    efficiency

    designphilosophy. Drivinglowerairflowimpedanceallowsthesystemtodrawairthroughthe

    systemataloweroperatingpressurethancompetitivesystems. Insomecases,thiscanbeupto

    40%lessbackpressurethansimilarlyconfiguredcompetitiveproducts. Thelowerbackpressure

    reducesthesystemfanpowerconsumedtomeettheairflowrequirementsofthesystem.

    Thelowimpedancedesigniscoupledwithahighefficiencyairmovingdevicedesigned

    explicitlyforthePowerEdgeM1000e. Theefficiencyofanairmovingdeviceisdefinedasthe

    workoutputofthefanascomparedtotheelectricalpowerrequiredtorunthefan. Thesystem

    fanoperatesatefficienciesupto40%greaterthantypicalaxialfansonthemarket,which

    correlatesdirectlyintosavingsinthecustomersrequiredpowertocool.

    Figure24PowerEdgeM1000eFan

    Page34

    Thehighefficiencydesignphilosophyalsoextendsintothelayoutofthesubsystemswithinthe

    PowerEdgeM1000e. TheServerModules,I/OModules,andPowerSuppliesareincorporated

  • 5/22/2018 Pedge m1000e

    35/42

    intothesystemwithindependentairflowpaths. Thisisolatesthesecomponentsfrompreheated

    air,reducingtherequiredairflowconsumptionsofeachmodule.

    Figure25ServerModuleCoolingAirProfile

    Figure26I/OModuleCoolingAirProfile

    Page35

    Server Module Inlet(M1000e front view)

    I/O Module Inlet(M1000e front view)

    IOM Locations and Airflow Direction(M1000e back view)

    I/O Cooling Air Profile(M1000e side view)

    Server Module Cooling Air Profile(M1000e side view)

  • 5/22/2018 Pedge m1000e

    36/42

    TheServerModulesarecooledwithtraditionalfronttobackcooling. AsshowninFigure25,the

    frontofthesystemisdominatedbyinletareafortheindividualservermodules. Theairpasses

    throughtheservermodules,throughventingholesinthemidplane,andisthendrawnintothe

    fanswhich

    exhaust

    the

    air

    from

    the

    chassis.

    There

    are

    plenums

    both

    upstream

    of

    the

    midplane,

    betweenthemidplaneandtheblades,anddownstreamofthemidplane,betweenthemidplane

    andthefans,tomoreevenlydistributethecoolingpotentialfromthethreecolumnsoffans

    acrosstheservermodules.

    TheI/OModulesuseabypassducttodrawambientairfromthefrontofthesystemtotheI/O

    Moduleinlet,asseeninFigure26. Thisductislocatedabovetheservermodules. Thiscoolairis

    thendrawndownthroughtheI/OModulesinatoptobottomflowpathandintotheplenum

    betweenthemidplaneandfans,fromwhereitisexhaustedfromthesystem.

    ThePowerSupplies,locatedintherearofthesystem,usebasicfronttobackcooling,butdraw

    theirinletairfromaductlocatedbeneaththeservermodules,asseeninFigure27. Thisinsures

    thatthepowersuppliesreceiveambienttemperatureair.

    Figure27PowerSupplyCoolingAirProfile

    Thishardwaredesigniscoupledwithathermalcoolingalgorithmthatincorporatesthe

    following:

    ServerModulelevelthermalmonitoringbytheiDRAC

    I/OModulethermalhealthmonitors

    FancontrolandmonitoringbytheCMC

    TheiDRAConeachservermodulescalculatestheamountofairflowrequiredonanindividual

    servermodulelevelandsendsarequesttotheCMC. Thisrequestisbasedontemperature

    conditionsontheservermodule,aswellaspassiverequirementsduetohardwareconfiguration.

    Concurrently,eachIOMcansendarequesttotheCMCtoincreaseordecreasecoolingtotheI/O

    subsystem. TheCMCinterpretstheserequests,andcancontrolthefansasrequiredtomaintain

    ServerandI/OModuleairflowatoptimallevels.

    Page36

    Power Supply Inlet(M1000e front view)

    Power Supply Cooling Air Profile(M1000e side view)

  • 5/22/2018 Pedge m1000e

    37/42

    FansareN+1redundant,meaningthatanysinglefancanfailwithoutimpactingsystemuptime

    orreliability. Intheeventofafanfailure,systembehaviorisdependentontheresultant

    temperaturesofthesystem,asmonitoredbytheServerModuleiDRACandI/OModules.The

    CMCcontinues

    to

    interpret

    the

    airflow

    needs

    of

    each

    server

    and

    I/O

    module

    to

    control

    the

    fan

    speedsappropriately. Thesystemdoesnotrampthefanstofullspeedintheeventofafan

    failureunlessdeemednecessarybyonboardmonitoring.

    ThePowerEdgeM1000eisengineeredforgoodsoundqualityinaccordancewiththeDell

    Enterpriseacousticalspecification. Comparedtopreviousgenerationsofproductsthefanshave

    morelevelsofcontrol,allowingmuchfinertuningofthefanbehavior.Firmwareisoptimizedto

    choosethelowestfanspeedsandthereforethelowestacousticaloutputforanyconditionand

    configuration.

    System Management

    DellsPowerEdgeM1000emodularserverdeliversmajorenhancementsinmanagementfeatures.

    Thesefeatures

    are

    based

    on

    direct

    customer

    feedback

    and

    assist

    customers

    deploying

    modular

    serversintheirenvironment. Eachsubsystemhasbeenreviewedandadjustedtooptimize

    efficiencies,minimizingtheimpactstoexistingmanagementtoolsandprocesses,andproviding

    futuregrowthopportunitiestostandardsbasedmanagement.

    Blade-16

    CMC 1

    L2+ Switch

    CMC MPUChassis

    Mngmt

    CMC 2

    L2+ Switch

    CMC MPUChassis

    Mngmt

    Blade-1

    iDRAC iDRAC

    IOM-1

    Mgmt

    IOM-6

    Mgmt

    Local KVM

    Management Network 1

    Management Network 2

    Analog

    video, USB

    Keyboard

    and mouse

    I2C

    Power

    Supply 1

    Power

    Supply 2

    Power

    Supply 6

    Chassis

    buttons, FRU,

    LEDs

    PWM, TACH

    Intranet Intranet

    OSCAR

    FAN 1 FAN 2 FAN 9

    I2C

    LCD Panel

    Figure28SystemManagementArchitectureSimplifiedBlockDiagram

    Page37

    Figure28illustratesthemanagementconnectionswhichtransferhealthandcontroltraffic

    throughoutchassis. Thesystemmanagementfabricisarchitectedfor100BaseTEthernetover

    differentialpairsroutedtoeachmodule.Therearetwo100BaseTinterfacesbetweenCMCs,one

  • 5/22/2018 Pedge m1000e

    38/42

    switchedandoneunswitched.AllsystemmanagementEthernetisroutedfor100Mbps

    signaling.EverymodulehasamanagementnetworklinktoeachCMC,withredundancy

    providedatthemodulelevel. Failureofanyindividuallinkcausesfailovertotheredundant

    CMC.

    Chassis Management Controller (CMC)

    ChassismanagementisaccessedandmanagedbytheChassisManagementControllerorCMC.

    ThePowerEdgeM1000epossessesatleastoneCMCandsupportsanoptionalredundant

    module,eachoccupyingaslotaccessiblethroughtherearofthechassis. Redundancyis

    providedinanActiveStandbypairingofthemodulesandfailoveroccurswhentheactive

    modulehasfailedordegraded. TheCMCinterfacesthroughdualstacking10/100/1000Ethernet

    portsandoneserialport. TheCMCserialportinterfaceprovidescommonmanagementofupto

    sixI/Omodulesthroughasingleconnection.

    Figure29PowerEdgeM1000eChassisManagementController

    Page38

    Figure30ChassisManagementControllerFrontPanel

  • 5/22/2018 Pedge m1000e

    39/42

    TheCMCprovidessecureremotemanagementaccesstothechassisandinstalledmodules. It

    managesorfacilitatesthemanagementofthefollowing:

    Status, Inventory and Alerting for server modules, chassis infrastructure and I/O

    Modules

    CentralizedconfigurationforiDRAC,I/OModulesandCMC

    SSL/SSH

    Powersequencingofmodulesinconjunctionwiththedefinedchassispowerstates

    Powerbudgetmanagementandallocation

    Configurationoftheembeddedmanagementswitch,whichfacilitatesexternalaccessto

    manageablemodules.

    Remoteusermanagement

    Userinterfaceentrypoint(web,telnet,SSH,serial)

    Monitoring and alerting for chassis environmental conditions or component health

    thresholds. Thisincludesbutisnotlimitedtothefollowing:

    o

    Real

    time

    power

    consumption

    o Powersupplies

    o Fans

    o Powerallocation

    o Temperature

    o CMCredundancy

    o I/Ofabricconsistency

    SMASHCLPfeaturesprovidingchassisservermodulepowercontrolfunctions

    WSMAN

    CIMXML

    SNMP

    RACADM

    Ethernettraffic

    management

    (firewall)

    Ethernetswitchmanagement

    PrivateVLAN

    PrivateDHCPservice

    PrivateTFTPservice

    VirtualMedia

    VirtualKVMinterfacebetweentheIPnetworkandtheservermoduleiDRACvKVM

    FirmwaremanagementofCMC,IOM,iDRAC,iKVM

    Integrated Dell Remote Access Controller (iDRAC)

    Theservermodulebasemanagementsolutionincludesadditionalfeaturesforefficient

    deploymentand

    management

    of

    servers

    in

    a

    modular

    server

    form

    factor.

    The

    base

    circuit,

    which

    integratestheBMCfunctionwithhardwaresupportforVirtualKVM(vKVM)andVirtualMedia

    (vMedia),iscalledtheintegratedDellRemoteAccessControlleroriDRAC.iDRAChastwo

    Ethernetconnections,oneforeachCMC,providingsystemmanagementinterfaceredundancy.

    Page39

    DellmodularserversnowincludevKVMasastandardfeature,routingtheoperatorskeyboard

    output,mouseoutputandvideobetweenthetargetservermoduleandaconsolelocatedonthe

    systemmanagementIPnetwork.WithuptotwosimultaneousvKVMsessionsperblade,remote

    managementnowsatisfiesvirtuallyanyusagemodel. vMediaisalsonowstandard,providing

  • 5/22/2018 Pedge m1000e

    40/42

    emulationofUSBDVDR/W,USBCDR/W,USBFlashDrive,USBISOimageandUSBFloppy

    overanIPinterface. ConnectiontovKVMandvMediaisthroughtheCMC,withencryption

    availableonaperstreambasis.

    Highlightsof

    the

    iDRAC

    solution

    include

    the

    following:

    Dedicatedmanagementinterfaceforhighperformancemanagementfunctions

    VirtualMedia

    VirtualKVM

    IPMI2.0OutOfBandmanagement

    SerialoverLANredirection

    SMASHCLP

    Bladestatusandinventory

    Activepowermanagement

    IntegrationwithActiveDirectory

    Security,LocalandActiveDirectory

    Integrated Keyboard and Mouse Controller

    ThemodularenclosuresupportsoneoptionalIntegratedKVM(iKVM)module. Thismodule

    occupiesasingleslotaccessiblethroughtherearofthechassis. TheiKVMredirectslocalserver

    modulevideo,keyboard,andmouseelectricalinterfacestoeithertheiKVMlocalportsorthe

    PowerEdgeM1000efrontpanelports. TheiKVMallowsasingledonglelessVGAmonitor,USB

    keyboard,andUSBmousetobeattachedandthenmovedbetweenservermodulesasneeded

    throughanonscreendisplayselectionmenu. TheiKVMalsohasanAnalogConsoleInterface

    (ACI)compatibleRJ45portthatallowstheiKVMtotietheinterfacetoaKVMappliance

    upstreamoftheiKVMviaCAT5cabling. DesignedwithAvocenttechnology,theACIport

    reducescostandcomplexitybygivingaccessforsixteenserversusingonlyoneportonan

    external

    KVM

    Switch.

    Figure31PowerEdgeM1000eiKVMModule

    Page40

    TheiKVMcontainsanewseventeenthbladefeature,connectingtheCMCCommandLine

    InterfaceviatheKVMswitchandallowingtextbaseddeploymentwizardsonVGAmonitors.

    iKVMfirmwareisupdatedthroughtheCMC.

  • 5/22/2018 Pedge m1000e

    41/42

    Figure32iKVMFrontPanel

    Control Panel and LCD

    The

    control

    panel

    contains

    the

    local

    user

    interface.

    Functions

    include

    chassis

    level

    diagnostic

    LEDs,LCDdisplayandpowerbutton. Thisdeviceisnothotpluggableandisalwayspowered,

    eveninchassisstandbymode.

    Figure33PowerEdgeM1000eLCDPanelRecessedPosition

    Page41

  • 5/22/2018 Pedge m1000e

    42/42

    Page42

    Figure34PowerEdgeM1000eLCDPanelDuringUsage

    TheChassisLCDshowschassisinformation,chassisstatusandfaults,majorservermodulefaults

    andprovides

    an

    interface

    for

    server

    module

    and

    CMC

    network

    configuration.

    Included

    is

    an

    errormessagewhennoCMCispresentinthechassis.ServiceTag,AssetTagandIPAddressare

    noweasilyvisibleoneveryPowerEdgeM1000emodularsystemthroughtheLCDdisplay. The

    LCDpanelcanberetractedintothechassisbody,orextendedandangledoncedeployedforfull

    visibilitynomatterwherethePowerEdgeM1000eismountedintherack.

    *Graphicimagesmaydifferfromfinalproduct

    Figure35LCDGraphicExamples

    Conclusion

    ThePowerEdgeM1000eModularServerEnclosureisabreakthroughinenterpriseclassserver

    design.DelloptimizedthenewPowerEdgeM1000eModularServerEnclosureandM600/605

    ServerModulesto:

    Maximizeflexibility modularI/O,power,cooling,andmanagementarchitecture.

    Maximizelongevity optimizedpowerandcoolingdesignsupportscurrentandfuture

    generationsofservermodulesandI/O.I/Obandwidthtosupportnotonlytodaysgeneration

    of10GbEthernet,20GbpsInfiniBandand4GbpsFibreChannel,butupto40GbpsQDR

    InfiniBand,10GbpsSerialEthernet,and8GbpsFibreChannel.

    Lower

    TCO

    lower

    cost

    than

    monolithic

    servers

    with

    equivalent

    features.

    Best

    in

    class

    power

    andcoolingefficiency.

    Acknowledgments

    TheauthorwouldliketothankRichardCrisp,MikeRobertsandK.C.Coxefortheir

    contributionstothiswhitepaper.