PCI Express ExpressModule Electromechanical Specification...

PCI Express® ExpressModule™

Electromechanical Specification Revision 1.0

February 14, 2005

PCI EXPRESS EXPRESSMODULE ELECTROMECHANICAL SPECIFICATION, REV. 1.0

2

Revision Revision History Date 1.0 Initial release. 02/14/05

PCI-SIG® disclaims all warranties and liability for the use of this document and the information contained herein and assumes no responsibility for any errors that may appear in this document, nor does PCI-SIG make a commitment to update the information contained herein.

Contact the PCI-SIG office to obtain the latest revision of this specification.

Questions regarding the PCI Express ExpressModule Electromechanical Specification or membership in PCI-SIG may be forwarded to:

Membership Services www.pcisig.com E-mail: [email protected] Phone: 503-619-0569 Fax: 503-644-6708

Technical Support [email protected]

DISCLAIMER This PCI Express ExpressModule Electromechancial Specification is provided "as is" with no warranties whatsoever, including any warranty of merchantability, noninfringement, fitness for any particular purpose, or any warranty otherwise arising out of any proposal, specification, or sample. PCI-SIG disclaims all liability for infringement of proprietary rights, relating to use of information in this specification. No license, express or implied, by estoppel or otherwise, to any intellectual property rights is granted herein.

ExpressModule, PCI, PCI Express, PCIe, and PCI-SIG are trademarks or registered trademarks of PCI-SIG.

All other product names are trademarks, registered trademarks, or service marks of their respective owners.

Copyright © 2005 PCI-SIG


3

Contents

1. INTRODUCTION .................................................................................................................11

1.1. TERMS AND DEFINITIONS............................................................................................... 11 1.2. REFERENCE DOCUMENTS............................................................................................... 12 1.3. SPECIFICATION CONTENTS............................................................................................. 12 1.4. OBJECTIVES ................................................................................................................... 13 1.5. OVERVIEW ..................................................................................................................... 13

2. EXPRESSMODULE AUXILIARY INTERFACES.............................................................15

2.1. REFERENCE CLOCK........................................................................................................ 15 2.1.1. Low Voltage Swing, Differential Clocks................................................................. 15 2.1.2. Spread Spectrum Clocking (SSC) ........................................................................... 17 2.1.3. REFCLK Specifications .......................................................................................... 17 2.1.4. REFCLK Phase Jitter Specification........................................................................ 21

2.2. MRST# SIGNAL............................................................................................................. 22 2.3. WAKE SIGNAL ............................................................................................................. 22 2.4. INTERNAL STORAGE INTERFACE .................................................................................... 23

2.4.1. Storage Interface Signal Definitions....................................................................... 25 2.5. POWER MANAGEMENT................................................................................................... 26

2.5.1. Initial Power-Up (G3 to L0) ................................................................................... 26 2.5.2. Power Management States (S0 to S3/S4 to S0)....................................................... 27 2.5.3. Power Down............................................................................................................ 29

2.6. MANAGEMENT............................................................................................................... 29 2.6.1. Capacitive Load of High-power SMBus Lines........................................................ 30 2.6.2. Minimum Current Sinking Requirements for SMBus Devices................................ 30 2.6.3. SMBus “Back Powering” Considerations.............................................................. 31 2.6.4. Power-on Reset ....................................................................................................... 31 2.6.5. SMBus Termination and Power .............................................................................. 31 2.6.6. SMBAlert................................................................................................................. 31 2.6.7. Management Bus Topology..................................................................................... 31 2.6.8. Data Integrity.......................................................................................................... 32 2.6.9. Basic Management Status Register......................................................................... 33 2.6.10. Satellite Management Controller ........................................................................ 34 2.6.11. Remote Management Card Access...................................................................... 34 2.6.12. VPD (FRU) Information Format......................................................................... 35 2.6.13. Using an SMBus Multiplexer with a Host Controller ......................................... 48 2.6.14. Implementing ExpressModule Management in an IPMI Environment............... 48 2.6.15. Sending Data to a Management Controller on a Module................................... 49 2.6.16. Receiving Data From a Management Controller on a Module .......................... 51 2.6.17. SMBAlert Control and Status.............................................................................. 54 2.6.18. Message Class Values for Management Controller Messaging ......................... 55 2.6.19. Retention of Output Data .................................................................................... 55 2.6.20. SMBAlert Signal Handling for Message Transfers............................................. 56 2.6.21. Polling for Output Data ...................................................................................... 56


4

2.6.22. SMBus NACKs and Error Recovery.................................................................... 56 2.6.23. PEC Handling ..................................................................................................... 56 2.6.24. Summary of SMBus Commands Values for Management Controller Messaging57 2.6.25. SMBus Timeout and Hang Handling .................................................................. 58 2.6.26. Management Controller Messaging Timing ....................................................... 58 2.6.27. IPMI Management Controller Message Formats ............................................... 59 2.6.28. Pre-assigned/Reserved Slave Addresses ............................................................. 61

2.7. AUXILIARY SIGNAL PARAMETRIC SPECIFICATIONS ....................................................... 63 2.7.1. DC Specifications.................................................................................................... 63 2.7.2. AC Specifications .................................................................................................... 64

3. HOT INSERTION/REMOVAL ............................................................................................67

3.1. SCOPE ............................................................................................................................ 67 3.2. HOT-PLUG SUB-SYSTEM ARCHITECTURE ...................................................................... 67

3.2.1. Power Enable.......................................................................................................... 72 3.2.2. Power Fault ............................................................................................................ 72 3.2.3. Wake........................................................................................................................ 72 3.2.4. Module Power Good ............................................................................................... 73 3.2.5. Module Reset........................................................................................................... 73 3.2.6. Present Detection.................................................................................................... 73 3.2.7. System Management Bus......................................................................................... 73 3.2.8. System Management Bus Alert................................................................................ 74 3.2.9. Power LED.............................................................................................................. 74 3.2.10. Attention LED...................................................................................................... 74 3.2.11. Manual Retention Latch ...................................................................................... 74 3.2.12. Electromechanical Interlock ............................................................................... 74 3.2.13. Electromechanical Interlock Status .................................................................... 75 3.2.14. Attention Switch................................................................................................... 75

4. MODULE POWER INTERFACE ........................................................................................77

4.1. POWER........................................................................................................................... 77 4.1.1. Module Primary Power Supply............................................................................... 77 4.1.2. Module Auxiliary Power Supply ............................................................................. 77

4.2. EXPRESSMODULE POWER SUPPLY REQUIREMENTS ....................................................... 78 4.3. POWER CONSUMPTION................................................................................................... 79 4.4. POWER SUPPLY SEQUENCING ........................................................................................ 79 4.5. POWER SUPPLY DECOUPLING ........................................................................................ 79

5. MODULE PCI EXPRESS INTERFACE ..............................................................................81

5.1. PCI EXPRESS LINK SIGNALS.......................................................................................... 81 5.2. PCI EXPRESS ELECTRICAL TOPOLOGIES AND LINK DEFINITIONS .................................. 81

5.2.1. Topologies............................................................................................................... 81 5.2.2. Link Definitions....................................................................................................... 82

5.3. PCI EXPRESS ELECTRICAL BUDGETS............................................................................. 83 5.3.1. AC Coupling Capacitors......................................................................................... 83 5.3.2. Insertion Loss Values (Voltage Transfer Function)................................................ 83 5.3.3. Jitter Values ............................................................................................................ 86


5

5.3.4. Crosstalk ................................................................................................................. 88 5.3.5. Lane-to-Lane Skew.................................................................................................. 88 5.3.6. Equalization ............................................................................................................ 89 5.3.7. Skew Within the Differential Pair ........................................................................... 89

5.4. EYE DIAGRAMS AT THE EXPRESSMODULE INTERFACE .................................................. 89 5.4.1. ExpressModule Transmitter Path Compliance Eye Diagram ................................ 90 5.4.2. ExpressModule Minimum Receiver Path Sensitivity Requirements ....................... 91 5.4.3. System Board Transmitter Path Compliance Eye Diagram ................................... 92 5.4.4. System Board Minimum Receiver Path Sensitivity Requirements .......................... 94

6. EXPRESSMODULE CONNECTOR....................................................................................97

6.1. CONNECTOR PIN COUNTS .............................................................................................. 97 6.2. CONNECTOR PIN ASSIGNMENTS..................................................................................... 99 6.3. CONNECTOR INTERFACE DEFINITIONS ......................................................................... 102 6.4. CONNECTOR SIGNAL INTEGRITY REQUIREMENTS AND TEST PROCEDURES.................. 108 6.5. CONNECTOR ENVIRONMENTAL AND OTHER REQUIREMENTS ...................................... 111

6.5.1. Environmental Requirements................................................................................ 111 6.5.2. Mechanical Requirements..................................................................................... 113 6.5.3. Current Rating Requirement................................................................................. 114 6.5.4. Additional Considerations .................................................................................... 114

7. MODULE MECHANICAL SPECIFICATION ..................................................................117

7.1. MECHANICAL OVERVIEW ............................................................................................ 117 7.2. DIMENSIONS AND TOLERANCES................................................................................... 118 7.3. SYSTEM DATUM PLANE DEFINITION............................................................................ 119 7.4. MODULE DESCRIPTION ................................................................................................ 119

7.4.1. Module Materials.................................................................................................. 120 7.4.2. Singlewide and Doublewide Module Form Factor............................................... 122 7.4.3. Module Raw Card ................................................................................................. 124 7.4.4. Module Ejector and Latch Details........................................................................ 125 7.4.5. I/O Plate Details ................................................................................................... 126 7.4.6. Module Air Vent Design for EMI.......................................................................... 127

7.5. FILLER COMPONENT OR MODULE ................................................................................ 128 7.6. CHASSIS SLOT DESCRIPTION........................................................................................ 128

7.6.1. Backplane and Chassis Slot Details ..................................................................... 129

8. DESIGN CONSIDERATIONS ...........................................................................................131

8.1. COOLING/THERMAL ENVIRONMENT ............................................................................ 131 8.1.1. Longitudinal Cooling – Default Mode.................................................................. 132 8.1.2. Lateral Cooling – Alternate Mode........................................................................ 134 8.1.3. Cooling Consideration for Storage Extension Slot............................................... 137 8.1.4. Module EMI Design.............................................................................................. 138

8.2. MODULE ESD DESIGN................................................................................................. 140 8.3. MODULE INTEROPERABILITY ....................................................................................... 140 8.4. SLOT/MODULE COLOR CODING AND LABELING .......................................................... 140

8.4.1. Module Hot Remove and Add Capability ............................................................. 142 8.4.2. Modules That May Require a System Power Down.............................................. 142


6

8.4.3. Internal Storage Modules ..................................................................................... 142 8.4.4. Optional Module Labeling.................................................................................... 142 8.4.5. Slot Labeling on the System .................................................................................. 143 8.4.6. Optional x16 Doublewide Slot Labeling............................................................... 143 8.4.7. Slot Numbering and Labeling ............................................................................... 143


7

Figures FIGURE 2-1: DIFFERENTIAL REFCLK WAVEFORM ...................................................................... 16 FIGURE 2-2: EXAMPLE REFERENCE CLOCK SOURCE TERMINATION ............................................. 16 FIGURE 2-3: SINGLE-ENDED MEASUREMENT POINTS FOR ABSOLUTE CROSS POINT AND SWING . 19 FIGURE 2-4: SINGLE-ENDED MEASUREMENT POINTS FOR DELTA CROSS POINT .......................... 19 FIGURE 2-5: SINGLE-ENDED MEASUREMENT POINTS FOR RISE AND FALL TIME MATCHING........ 20 FIGURE 2-6: DIFFERENTIAL MEASUREMENT POINTS FOR DUTY CYCLE AND PERIOD ................... 20 FIGURE 2-7: DIFFERENTIAL MEASUREMENT POINTS FOR RISE AND FALL TIME ........................... 20 FIGURE 2-8: DIFFERENTIAL MEASUREMENT POINTS FOR RINGBACK ........................................... 21 FIGURE 2-9: REFERENCE CLOCK SYSTEM MEASUREMENT POINT AND LOADING ......................... 21 FIGURE 2-10: POWER UP .............................................................................................................. 27 FIGURE 2-11: POWER MANAGEMENT STATES............................................................................... 28 FIGURE 2-12: POWER DOWN......................................................................................................... 29 FIGURE 2-13: EXAMPLE SMBUS TOPOLOGY ................................................................................ 32 FIGURE 2-14: BASIC MANAGEMENT STATUS REGISTER ACCESS .................................................. 33 FIGURE 2-15: BMC SINGLE-PART WRITE..................................................................................... 50 FIGURE 2-16: MULTI-PART WRITE START .................................................................................... 51 FIGURE 2-17: MULTI-PART WRITE MIDDLE.................................................................................. 51 FIGURE 2-18: MULTI-PART WRITE END........................................................................................ 51 FIGURE 2-19: SINGLE-PART READ ................................................................................................ 52 FIGURE 2-20: MULTI-PART READ START ...................................................................................... 52 FIGURE 2-21: MULTI-PART READ MIDDLE ................................................................................... 53 FIGURE 2-22: BMC MULTI-PART READ END................................................................................ 53 FIGURE 2-23: MULTI-PART READ RETRY ..................................................................................... 53 FIGURE 2-24: GET MESSAGE INTERFACE STATUS......................................................................... 54 FIGURE 2-25: SET MESSAGE INTERFACE CONTROL ...................................................................... 55 FIGURE 2-26: BMC TO MODULE IPMI STANDARD REQUEST....................................................... 59 FIGURE 2-27: BMC TO MODULE IPMI STANDARD RESPONSE ..................................................... 60 FIGURE 2-28: MODULE TO BMC IPMI STANDARD REQUEST....................................................... 61 FIGURE 2-29: MODULE TO BMC IPMI STANDARD RESPONSE ..................................................... 61 FIGURE 2-30: WAKE RISE AND FALL TIME MEASUREMENT POINTS ........................................... 65 FIGURE 3-1: TYPICAL HOT-PLUG INTERFACE IMPLEMENTATION.................................................. 69 FIGURE 5-1: EXAMPLE INTERCONNECT TERMINATED AT THE CONNECTOR INTERFACE................ 84 FIGURE 5-2: INSERTION LOSS BUDGETS ....................................................................................... 85 FIGURE 5-3: JITTER BUDGET......................................................................................................... 86 FIGURE 5-4: EXPRESSMODULE TRANSMITTER PATH COMPLIANCE EYE DIAGRAM ...................... 90 FIGURE 5-5: REPRESENTATIVE COMPOSITE EYE DIAGRAM FOR EXPRESSMODULE RECEIVER PATH

COMPLIANCE ......................................................................................................................... 91 FIGURE 5-6: SYSTEM BOARD TRANSMITTER PATH COMPOSITE COMPLIANCE EYE DIAGRAM...... 92 FIGURE 5-7: TWO PORT MEASUREMENT MODEL.......................................................................... 94 FIGURE 5-8: REPRESENTATIVE COMPOSITE EYE DIAGRAM FOR SYSTEM BOARD RECEIVER PATH

COMPLIANCE ......................................................................................................................... 95 FIGURE 6-1: EXPRESSMODULE CONNECTOR FORM FACTOR ...................................................... 103


8

FIGURE 6-2: EXPRESSMODULE CONNECTOR FORM FACTOR WITH STORAGE EXTENSION .......... 104 FIGURE 6-3: RECOMMENDED FOOTPRINTS.................................................................................. 105 FIGURE 6-4: ADD-IN MODULE EDGE-FINGER DIMENSIONS ........................................................ 106 FIGURE 6-5: ADD-IN MODULE WITH STORAGE EDGE-FINGER DIMENSIONS ............................... 107 FIGURE 6-6: ILLUSTRATION OF ADJACENT PAIRS ....................................................................... 111 FIGURE 6-7: CONTACT RESISTANCE MEASUREMENT POINTS ..................................................... 112 FIGURE 7-1: CHASSIS ASSEMBLY................................................................................................ 118 FIGURE 7-2: EXPLODED VIEW OF SINGLEWIDE MODULE............................................................ 120 FIGURE 7-3: SIMPLE CIRCUIT REPRESENTATION OF AFP STEEL USING FOUR-POINT RESISTANCE

METHOD .............................................................................................................................. 121 FIGURE 7-4: CRITICAL DIMENSIONS ........................................................................................... 123 FIGURE 7-5: SINGLEWIDE CROSS SECTION ................................................................................. 124 FIGURE 7-6: DOUBLEWIDE CROSS SECTION................................................................................ 124 FIGURE 7-7: MODULE RAW CARD REFERENCE DRAWING .......................................................... 125 FIGURE 7-8: EJECTOR ASSEMBLY ............................................................................................... 126 FIGURE 7-9: MODULE I/O PLATE................................................................................................ 127 FIGURE 7-10: MODULE VENT DESIGN ........................................................................................ 128 FIGURE 7-11: CHASSIS REQUIREMENTS ...................................................................................... 129 FIGURE 8-1: LONGITUDINAL SYSTEM AIRFLOW ......................................................................... 132 FIGURE 8-2: SINGLEWIDE MODULE: REQUIRED CFM VS. MODULE INLET TEMPERATURE,

DEFAULT MODE................................................................................................................... 133 FIGURE 8-3: SINGLEWIDE MODULE: REQUIRED PRESSURE DROP RANGE VS. FLOW RATE,

DEFAULT MODE................................................................................................................... 134 FIGURE 8-4: UNINTENDED RECIRCULATION ............................................................................... 135 FIGURE 8-5: SINGLEWIDE MODULE: REQUIRED CFM VS. MODULE INLET TEMPERATURE,

ALTERNATE MODE .............................................................................................................. 136 FIGURE 8-6: SINGLEWIDE MODULE: REQUIRED PRESSURE DROP RANGE VS. FLOW RATE,

ALTERNATE MODE .............................................................................................................. 137 FIGURE 8-7: STORAGE SLOT VENTING........................................................................................ 138 FIGURE 8-8: EMI GASKET PROFILE............................................................................................ 138 FIGURE 8-9: REQUIRED MODULE I/O PLATE ATTENUATION ...................................................... 139 FIGURE 8-10: MODULE LATCH LABELING LOCATIONS............................................................... 141 FIGURE 8-11: REQUIRED MODULE LABEL WITH EXAMPLES ....................................................... 141 FIGURE 8-12: REQUIRED SLOT LABELING .................................................................................. 143 FIGURE 8-13: OPEN VENT SYMBOL ............................................................................................ 144 FIGURE 8-14: CLOSED VENT SYMBOL ........................................................................................ 144


9

Tables TABLE 2-1: REFCLCK DC SPECIFICATION AND AC TIMING REQUIREMENTS ............................ 18 TABLE 2-2: MAXIMUM ALLOWED PHASE JITTER WHEN APPLIED TO FIXED FILTER

CHARACTERISTIC................................................................................................................... 22 TABLE 2-3: STORAGE SIDEBAND SIGNALS: I2C/SMBUS.............................................................. 25 TABLE 2-4: STORAGE SIDEBAND SIGNALS: SGPIO...................................................................... 26 TABLE 2-5: BASIC MANAGEMENT STATUS REGISTER................................................................... 33 TABLE 2-6: VPD (FRU) COMMON HEADER................................................................................. 35 TABLE 2-7: OEM RESERVED SPACE HEADER .............................................................................. 36 TABLE 2-8: VPD (FRU) RECORD FORMAT .................................................................................. 36 TABLE 2-9: VPD (FRU) RECORD TYPES...................................................................................... 37 TABLE 2-10: BOARD INFO DATA FORMAT.................................................................................... 38 TABLE 2-11: PRODUCT INFO DATA FORMAT ................................................................................ 39 TABLE 2-12: PRODUCT GUID DATA FORMAT.............................................................................. 40 TABLE 2-13: EXPRESSMODULE MODULE INFO DATA FORMAT .................................................... 41 TABLE 2-14: PCI EXPRESS MODULE INFO DATA FORMAT ........................................................... 42 TABLE 2-15: EXPRESSMODULE MULTIPLEXER RECORD DATA FORMAT...................................... 43 TABLE 2-16: OEM RECORD DATA FORMAT................................................................................. 44 TABLE 2-17: LANGUAGE CODES................................................................................................... 45 TABLE 2-18: INTERFACE STATUS FIELD DEFINITION .................................................................... 54 TABLE 2-19: INTERFACE CONTROL FIELD DEFINITION ................................................................. 55 TABLE 2-20: MESSAGE CLASS VALUES ........................................................................................ 55 TABLE 2-21: SUMMARY OF SMBUS COMMANDS FOR MESSAGE PROTOCOLS............................... 57 TABLE 2-22: SMBUS MESSAGING TIMING SPECIFICATIONS ......................................................... 58 TABLE 2-23: MANAGEMENT I/O ADDRESSES ............................................................................... 62 TABLE 2-24: AUXILIARY SIGNAL DC SPECIFICATIONS ................................................................ 63 TABLE 2-25: POWER SEQUENCING AND RESET SIGNAL TIMINGS.................................................. 64 TABLE 3-1: POWER SIGNALS ........................................................................................................ 70 TABLE 3-2: MANAGEMENT INTERFACE ........................................................................................ 70 TABLE 3-3: USER INTERFACE ....................................................................................................... 71 TABLE 3-4: EMLS SIGNAL........................................................................................................... 75 TABLE 4-1: POWER SUPPLY RAIL REQUIREMENTS ....................................................................... 78 TABLE 5-1: ALLOCATION OF INTERCONNECT PATH INSERTION LOSS BUDGET............................. 85 TABLE 5-2: TOTAL SYSTEM JITTER BUDGET ................................................................................ 87 TABLE 5-3: ALLOCATION OF INTERCONNECT JITTER BUDGET...................................................... 87 TABLE 5-4: ALLOWABLE INTERCONNECT LANE-TO-LANE SKEW ................................................. 89 TABLE 5-5: EXPRESSMODULE TRANSMITTER PATH COMPLIANCE EYE REQUIREMENTS .............. 90 TABLE 5-6: EXPRESSMODULE MINIMUM RECEIVER PATH SENSITIVITY REQUIREMENTS............. 91 TABLE 5-7: SYSTEM BOARD TRANSMITTER PATH COMPLIANCE EYE REQUIREMENTS ................. 92 TABLE 5-8: SYSTEM BOARD MINIMUM RECEIVER PATH SENSITIVITY REQUIREMENTS................ 94 TABLE 6-1: EXPRESSMODULE CONNECTOR PIN TYPE/COUNT ..................................................... 97 TABLE 6-2: OPTIONAL STORAGE PIN COUNT/TYPE ...................................................................... 98 TABLE 6-3: EXPRESSMODULE CONNECTOR PIN ASSIGNMENTS.................................................... 99 TABLE 6-4: SIGNAL INTEGRITY REQUIREMENTS AND TEST PROCEDURES .................................. 109


10

TABLE 6-5: TEST DURATIONS..................................................................................................... 112 TABLE 6-6: MECHANICAL TEST PROCEDURES AND REQUIREMENTS........................................... 113 TABLE 6-7: END-OF-LIFE CURRENT RATING TEST SEQUENCE.................................................... 114 TABLE 6-8: ADDITIONAL REQUIREMENTS .................................................................................. 114 TABLE 7-1: DATUM TABLE......................................................................................................... 119 TABLE 7-2: LED COLOR ............................................................................................................ 127 TABLE 8-1: X8 SLOT CONNECTOR INTEROPERABILITY ............................................................... 140


11

1. Introduction This document is a companion specification to the PCI Express Base Specification. Its primary focus is the implementation of a modular I/O form factor that is focused on the needs of workstations and servers from mechanicals and electrical requirements. The discussions are confined to the modules and their chassis slots requirements. Other form factors are covered in other separate specifications.

1.1. Terms and Definitions x1, x2, x4, x8, x16 x1 refers to one PCI Express™ lane of basic bandwidth; x4 to a collection of four

PCI Express lanes; etc.

Auxiliary signals Signals not required by the PCI Express architecture but necessary for certain desired functions or system implementation; for example, the SMBus signals

Basic bandwidth Contains one PCI Express lane

BMC Base Management Controller

Down-plugging Plugging a module with larger number of Links into a connector with a smaller number of Links wired; for example, plugging a module with x8 lanes wired into a slot with x4 lanes wired on the connector

Down-shifting Plugging a PCI Express card into a connector that is not fully routed for all of the PCI Express lanes; for example, plugging a x4 card into a x8 capable connector with only four lanes being routed

ExpressModule™ A PCI Express I/O module with the two form factors defined in this specification

FRU Field Replaceable Unit

Hot plug Insertion and/or removal of an ExpressModule into an active backplane or system board as defined in Revision 1.0 of the PCI Standard Hot Plug Controller and Subsystem Specification. This specification defines the special module requirements to support hot-plug.

Hot Swap Insertion and/or removal of an ExpressModule into a passive backplane. The card must satisfy specific requirements to support Hot Swap.

Interoperability Ability to plug a PCI Express module into different Link connectors and the system works; for example, plugging a x8 PCI Express I/O module into a x16 module slot

Link A collection of one or more PCI Express lanes

Pro-class graphics Graphics hardware, such as AGP PRO, that requires more than 40 watts of power

Up-plugging Plug a module with a smaller number of Links wired into a slot with a larger number of Links wired in the connector; for example, plugging a module wired for x1 into a slot connector wired for x4

1


12

PCI Express lane One PCI Express lane contains two differential lines for transmitter and two differential lines for receiver. A by-N Link is composed of N lanes.

VPD Vital Product Data. Information about the product.

1.2. Reference Documents This specification references the following documents:

PCI Express Base Specification, Revision 1.1

PCI Express Card Electromechanical Specification, Revision 1.1

PCI Express Jitter Modeling

PCI Express Jitter and BER

System Management Bus (SMBus) Specification, Version 2.0

EIA-364-1000.01: Environmental Test Methodology for Assessing the Performance of Electrical Connectors and Sockets Used in Business Office Applications

EIA-364: Electrical Connector/Socket Test Procedures Including Environmental Classifications

1.3. Specification Contents This specification contains the following information:

Adapter Module system interface signals

Adapter Module hot insertion and removal

Management interface

Power delivery

Adapter Module/backplane electrical budget

Adapter Module form factor and implementation

System slot requirements

Bandwidth and interoperability matrix


13

1.4. Objectives The objectives of this specification are:

Support PCI Express bit rate of 2.5 Gb/s with consideration for future bit rates up to 6.25 Gb/s

Enable Hot-plug and hot-swap as a native function

Leverage server/workstation commonality (I/O connectors and other components)

Maximize adapter module interoperability for user flexibility

Provide a low cost solution

1.5. Overview PCI Express is the third generation of multi-purpose I/O interface that can be used across the computing industry from mobile through high-end servers and even communication equipment. The broad usage of this technology yields the need to package the I/O adapters in form factors that meet the needs for the environment that they will be used in.

The focus of this specification is on a form factor that fulfills the needs of servers and workstations. Servers have always been required to be highly reliable. The demands on server reliability are steadily increasing while the I/O structure has not effectively changed in 20+ years. While the industry has been able to work around many issues with this, its shows up as increased cost to the server. A good example of this was adding hot plug or hot swap to PCI cards. This was done at a large expense to the system and required opening the chassis while operating (risking interruptions of service caused by moving the systems or by moving internal or external cables).

Servers need an I/O adapter form factor that addresses items like closed chassis adapter removal/insertion, integrated hot-plug, adapter protection from ESD and handling damage, standardized management interfaces and features, adequate cooling, single bulk power supply, and reduced I/O footprint.

Workstations have many of the same needs as servers with one exception, the graphics adapter. Workstation high-end graphics adapters require high wattages and require more adapter board space than the typical server I/O application. Workstation high-end graphics adapter requirements are not within the scope of this specification.

This specification defines two PCI Express modular form factors for I/O adapters, otherwise defined as ExpressModules. These modular I/O adapters are designed to provide closed chassis installation and removal. They are also designed for native hot-plug, meaning that no damage will occur to the module or chassis from installing or removing the modules while the chassis is powered.

The two form factors are both the same height and depth and vary only by the width or slot space they consume in the chassis. The first module is considered a singlewide and will fit in any slot that supports ExpressModules. The second module is the doublewide module and requires two adjacent slots in a chassis supporting ExpressModule slots.


14

A singlewide module supports up to a x8 PCI Express connection, and doublewide modules can support up to x16 connection. Chassis support for x16 doublewide slots is optional, and these slots shall be marked as defined in this specification. Chassis that support hot-plug x16 doublewide slots shall also support dynamic reconfiguration so that two modules may be exchanged for a single x16 module or vice versa during a hot-removal/hot-install events.

A minimum management interface is standard on all modules. This consists of an EEPROM on each module that provides management data about the adapter to the host system.

An optional internal storage interface is supported for up to four 1x ports of SAS/SATA and a sideband interface for controlling drive LEDs.

ExpressModules and chassis that support them share requirements for cooling. Chassis that support ExpressModules are required to provide a minimum airflow to each slot as defined by this specification. ExpressModules are required to provide a minimum and maximum airflow resistance. This results in value to both ExpressModules and supporting chassis. ExpressModules will have a defined airflow for I/O cooling, and the I/O thermal load is not added to the chassis.


15

2. ExpressModule Auxiliary Interfaces The auxiliary signals are provided on the connector to assist with certain system level functionality or implementation. These signals are not required by the PCI Express architecture. The high-speed signal voltage levels are compatible with advanced silicon processes. The low speed signals are defined to be compatible with +3.3V signaling, as this is the lowest common signaling available and are referenced to common logic ground. ExpressModule slots only provide +12V and a +3.3Vaux supplies to the module. A DC to DC converter on the module is required to provide a supply for the 3.3 V compatible signals that are not aux powered. The use of 3.3 V signaling allows the non-PCI Express signals to be used with existing control bus structures avoiding a buffered set of signals and bridges between the buses.

The PCI Express connector and ExpressModule interfaces support the following auxiliary signals:

REFCLK-/REFCLK+ (Must be supplied by system): low voltage differential signals.

MRST# (required): indicates when the applied main power is within the specified tolerance and stable. MRST# goes inactive after a delay of TPVPERL time from the power rails achieving specified tolerance on power up.

WAKE (optional): an active high signal that is driven high by a PCI Express function to re-activate the PCI Express Link hierarchy’s main power rails and reference clocks. WAKE is required on any ExpressModule or system board that supports wakeup functionality compliant with this specification.

Internal Storage interface (optional): Four ports of SAS/SATA plus sideband interface.

SMBCLK (required): the SMBus interface clock signal. It is an open-drain signal.

SMBDAT (required): the SMBus interface address/data signal. It is an open-drain signal.

SMBAlert (optional Module only): the SMBus alert is an open-drain signal.

Hot-plug signals (required): ATNSW, ATNLED, PWRLED, PWREN#, MPWRGD#, PWRFLT# PRSNT#, EMIL, and EMILS. See Chapter 3 for details.

2.1. Reference Clock

2.1.1. Low Voltage Swing, Differential Clocks

To reduce jitter and allow for future silicon fabrication process changes, low voltage swing, differential clocks are being used, as illustrated in Figure 2-1. The nominal single-ended swing for each clock is 0 to 0.7 V and a nominal frequency of 100 MHz ±300 PPM. The clock is defined in Chapter 4 of the PCI Express Base Specification, Revision 1.1.

2


16

Tperiod

REFCLK–

REFCLK+

A-0444

Figure 2-1: Differential REFCLK Waveform

The reference clock pair is routed point-to-point to each connector from the system board according to best-known clock routing rules. The reference clock distribution to all devices must be matched to within 15 inches on the system board. The phase delay between the transmitter and receiver clock is assumed to be less than 10 ns. The combination of the maximum reference clock mismatch and the maximum channel length will contribute approximately 7-8 ns and the remaining time is allocated to the difference in the insertion delays of the Tx and Rx devices. The routing of each signal in any given clock pair between the clock source and the connector must be well matched in length (< 0.005 inch) and appropriately spaced away from other non-clock signals to avoid excessive crosstalk.

The PCI Express silicon in an ExpressModule is not required to use the reference clock on the connector but is required to maintain the 600-ppm data rate matching specified in Section 4.3.1.1 of the PCI Express Base Specification, Revision 1.1.

Any terminations required by the clock are to be on the system board. An example termination topology for a current-mode clock generator is shown in Figure 2-2. Electromagnetic Interference (EMI) emissions will be reduced if clocks to open sockets are shut down at the clock source. The method for detecting the presence of a module in a slot and controlling the clock gating is platform specific.

A-0445

Rs

Rs

ZC-DCZC-DC

ReferenceClock

PCIExpress

Add-In CardREFCLK+

REFCLK-

Figure 2-2: Example Reference Clock Source Termination

Termination on the add-in card is allowed, but is not covered by the specifications in Section 2.7. While the same measurement techniques can be used as specified in that section, receiver termination will reduce the nominal swing and rise and fall times by half. The low input swing and low slew rates need to be validated against the clock receiver requirements as they can cause excessive jitter in some clock input buffer designs.


17

The reference clock timings are based on nominal 100 Ω, differential pair routing with approximately 5-mil trace widths. This timing budget allows for a maximum add-in card trace length of 4.0 inches. No specific trace geometry, however, is explicitly defined in this specification.

2.1.2. Spread Spectrum Clocking (SSC)

The reference clocks may support spread spectrum clocking. Any given system design may or may not use this feature due to platform-level timing issues. The minimum clock period cannot be violated. The preferred method is to adjust the spread technique to not allow for modulation above the nominal frequency. This technique is often called “down-spreading.” The requirements for spread spectrum modulation rate and magnitude are given in the PCI Express Base Specification, Revision 1.1.

2.1.3. REFCLK Specifications

All specifications in Table 2-1 are to be measured using a test configuration as described in Note 11 with a circuit as shown in Figure 2-9.


18

Table 2-1: REFCLCK DC Specification and AC Timing Requirements

100 MHz Input Unit Note Symbol Parameter

Min Max

Rise Edge Rate Rising Edge Rate 0.6 4.0 V/ns 2, 3

Fall Edge Rate Fall Edge Rate 0.6 4.0 V/ns 2, 3 VIH Differential Input High Voltage +150 mV 2

VIL Differential Input Low Voltage -150 mV 2

VCROSS Absolute crossing point voltage +250 +550 mV 1, 4, 5

VCROSS DELTA Variation of Vcross over all rising clock edges

+140 mV 1, 4, 9

VRB Ring-Back Voltage Margin -100 +100 mV 2, 12

TSTABLE Time before VRB is allowed 500 ps 2, 12

TPERIOD AVG Average Clock Period Accuracy -300 +2800 Ppm 2, 10, 13

TPERIOD ABS Absolute Period (including jitter and Spread Spectrum)

9.847 10.203

ns 2, 6

TCCJITTER Cycle to Cycle jitter 150 ps 2

VMAX Absolute Max input voltage +1.15 V 1, 7

VMIN Absolute Min input voltage -0.3 V 1, 8

Duty Cycle Duty Cycle 40 60 % 2

Rise-Fall Matching Rising edge rate (REFCLK+) to falling edge rate (REFCLK-) matching

20 % 1, 14

ZC-DC Clock source DC impedance 40 60 Ω 1, 11

Notes:

1. Measurement taken from single ended waveform.

2. Measurement taken from differential waveform.

3. Measured from -150 mV to +150 mV on the differential waveform (derived from REFCLK+ minus REFCLK-). The signal must be monotonic through the measurement region for rise and fall time. The 300 mV measurement window is centered on the differential zero crossing. See Figure 2-7.

4. Measured at crossing point where the instantaneous voltage value of the rising edge of REFCLK+ equals the falling edge of REFCLK-. See Figure 2-3.

5. Refers to the total variation from the lowest crossing point to the highest, regardless of which edge is crossing. Refers to all crossing points for this measurement. See Figure 2-5.

6. Defined as the absolute minimum or maximum instantaneous period. This includes cycle to cycle jitter, relative PPM tolerance, and spread spectrum modulation. See Figure 2-6.

7. Defined as the maximum instantaneous voltage including overshoot. See Figure 2-3.

8. Defined as the minimum instantaneous voltage including undershoot. See Figure 2-3.

9. Defined as the total variation of all crossing voltages of Rising REFCLK+ and Falling REFCLK-. This is the maximum allowed variance in VCROSS for any particular system. See Figure 2-4.


19

10. Refer to Section 4.3.2.1 of the PCI Express Base Specification, Revision 1.1 for information regarding PPM considerations.

11. System board compliance measured at the connector using the circuit of Figure 2-9. REFCLK+ and REFCLK- are to be measured at the load capacitors CL. Single ended probes must be used for measurements requiring single ended measurements. Either single ended probes with math or differential probe can be used for differential measurements. Test load CL = 2 pF.

12. TSTABLE is the time the differential clock must maintain a minimum ±150 mV differential voltage after rising/falling edges before it is allowed to droop back into the VRB ±100 mV differential range. See Figure 2-8.

13. PPM refers to parts per million and is a DC absolute period accuracy specification. 1 PPM is 1/1,000,000th of 100.000000 MHz exactly or 100 Hz. For 300 PPM, then, we have an error budget of 100 Hz/PPM * 300 PPM = 30 kHz. The period is to be measured with a frequency counter with measurement window set to 100 ms or greater. The ±300 PPM applies to systems that do not employ Spread Spectrum or that use common clock source. For systems employing Spread Spectrum, there is an additional 2500 PPM nominal shift in maximum period resulting from the 0.5% down spread resulting in a maximum average period specification of +2800 PPM.

14. Matching applies to rising edge rate for REFCLK+ and falling edge rate for REFCLK-. It is measured using a ±75 mV window centered on the median cross point where REFCLK+ rising meets REFCLK- falling. The median cross point is used to calculate the voltage thresholds the oscilloscope is to use for the edge rate calculations. The Rise Edge Rate of REFCLK+ should be compared to the Fall Edge Rate of REFCLK-. The maximum allowed difference should not exceed 20% of the slowest edge rate. See Figure 2-5.

REFCLK–

A-0446

REFCLK+

VMAX = 1.15 V

VMIN = -0.30 V

VCROSS MAX = 550 mV

VCROSS MIN = 250 mV

Figure 2-3: Single-Ended Measurement Points for Absolute Cross Point and Swing

REFCLK–

A-0447

REFCLK+

VCROSS DELTA = 140 mV

Figure 2-4: Single-Ended Measurement Points for Delta Cross Point


20

A-0448

REFCLK-

REFCLK+

REFCLK-

REFCLK+

VCROSS MEDIAN

VCROSS MEDIAN +75 mV

VCROSS MEDIAN -75 mV

VCROSS MEDIAN

TRISETFALL

Figure 2-5: Single-Ended Measurement Points for Rise and Fall Time Matching

A-0449

REFCLK+minus

0.0 V

Clock Period (Differential)

Positive DutyCycle (Differential)

Negative DutyCycle (Differential)

Figure 2-6: Differential Measurement Points for Duty Cycle and Period

A-0450

Rise Edge Rate Fall Edge Rate

REFCLK+minus

VIH = +150 mV

VIL = -150 mV

0.0 V

Figure 2-7: Differential Measurement Points for Rise and Fall Time


21

A-0451

TSTABLE TSTABLE

VRB

VRB

REFCLK+minus

VIH = +150 mV

VIL = -150 mV

0.0 VVRB = +100 mV

VRB = -100 mV

Figure 2-8: Differential Measurement Points for Ringback

A-0452

ReferenceClock

Rs

Rs MotherboardTraceZC-DCZC-DC

PCI ExpressConnector

PCI Express Add-InModule Test

REFCLK+

REFCLK-

CL = 2 pF CL = 2 pF

Figure 2-9: Reference Clock System Measurement Point and Loading

2.1.4. REFCLK Phase Jitter Specification

The phase jitter of the reference clock is to be measured using the following clock recovery function

[ ] )()(*)()( 32_*

1 sHsHesHsH delayts ⋅−= −

where:

211

2

211

1 22

)(ωζω

ωζω++

+=

sss

sH ,

222

2

222

2 22

)(ωζω

ωζω++

+=

sss

sH ,

33 )(

ω+=

sssH ,


22

( )

( )

sdelayt

sRad

sRad

sRad

9

63

222

6

2

222

6

1

1010_

/105.1**2

/12121

105.1**2

/12121

1022**254.0

−⋅=

⋅=

++++

⋅=

++++

⋅=

=

πω

ζζ

πω

ζζ

πω

ζ

The maximum allowed magnitude of the peak-peak reference clock jitter is given in Table 2-2. For information about the maximum peak-peak phase jitter value, see PCI Express Jitter Modeling. Multiple methods can be used to measure the maximum allowed peak-peak phase jitter value. Real time sampling scopes must use a sampling rate of 20 giga-samples per second or better and take enough data to guarantee the proper bit error rate (BER). Reference clock measurements for cards should be taken with a differential, high-impedance probe using the circuit of Figure 2-9 at the load capacitors CL. Measurements for devices on the same board should be made using a differential, high-impedance probe as close to the REFCLK+ and REFCLK- input pins as possible.

Table 2-2: Maximum Allowed Phase Jitter When Applied to Fixed Filter Characteristic

BER Maximum Peak – Peak Phase Jitter Value (ps)

10-6 86

10-12 108

2.2. MRST# Signal The MRST# signal is used to indicate when the power supply is within its specified voltage tolerance and is stable. It also initializes a component’s state machines and other logic once power supplies stabilize. On power up, the de-assertion of MRST# is delayed (TPVPERL) from the power rails achieving specified operating limits. Also, within this time, the reference clocks (REFCLK+, REFCLK-) also become stable, at least TMRST-CLK before MRST# is de-asserted. MRST# is asserted in advance of the power being switched off in a power-managed state like S3.

2.3. WAKE Signal The WAKE signal is an active high signal that is driven high by a PCI Express component to reactivate the PCI Express slot’s main power rails and reference clocks. Only ExpressModules that support the wake process connect to this pin. If the ExpressModule has wakeup capabilities, it must support the WAKE function. Likewise, only systems that support the wakeup function need to connect to this pin; but if they do, they must fully support the wake functionality. Such systems are not required to support Beacon as a wakeup mechanism, but are encouraged to support it. If the wakeup process is used, the +3.3Vaux supply is used for this function. The assertion and de-


23

assertion of WAKE are asynchronous to any system clock. (See Chapter 5 of the PCI Express Base Specification, Revision 1.1, for more details on PCI-compatible power management.)

The WAKE signal directly from the ExpressModule may not be bused to other PCI Express ExpressModule connectors. Platforms that choose to wire “OR” each slot’s wake function to a single input to the power management (PM) controller can connect the WAKE signal through an open collector or open drain inverting buffer to a common WAKE# input to the PM.

Hot-Plug requires that WAKE be controlled and driven inactive during the Hot-Plug/Hot Removal events. Refer to Section 6.2 for the connector pin assignment for the WAKE signal.

Auxiliary power (+3.3Vaux) must be used by the asserting and receiving ends of WAKE in order to revive the hierarchy. The system vendor must also provide a pull-down on WAKE to prevent activation when the slot is empty. Note that all potential drivers of the WAKE signal must be 3.3 V tolerant.

Note: The inversion of WAKE (WAKE#) is not PME# and should not be attached to the PCI-PME# interrupt signals. WAKE causes power to be restored but must not directly cause an interrupt.

Additionally, the ExpressModule must ensure that it does not pull WAKE high unless WAKE is being intentionally asserted in all cases, including when the related function is in D3cold.

Other requirements on the system board/ExpressModule designer include:

If WAKE is supported on one ExpressModule connector in a chassis, it must be supported on all ExpressModule connectors in that chassis.

ExpressModules are permitted to generate the Beacon wakeup mechanism in addition to using the wake mechanism; although, the system is not required to provide support for Beacon.

Note: If the ExpressModule uses the Beacon mechanism in addition to the wake mechanism, the Beacon may be ignored by the system.

PCI Express ExpressModule designers must be aware of the special requirements that constrain WAKE and ensure that their ExpressModules do not interfere with the proper operation of WAKE. The WAKE input into the system may deassert as late as 100 ns after the WAKE output from the ExpressModule deasserts (i.e., the WAKE pin must be considered indeterminate for a number of cycles after it has been deasserted).

The maximum of the pull-down resistor for WAKE on the system board shall be sufficient to hold the WAKE input inactive when a module is not present and the minimum value shall be 2.2 kΩ ± 5%. The module WAKE driver shall ensure that WAKE charges up to a logic high voltage level in no more than 100 ns.

2.4. Internal Storage Interface This is an optional interface that enables an ExpressModule storage initiator to control a modest number of internal (to server chassis) SAS and/or SATA drives. Prior to ExpressModule, there was no need to define a storage interface as part of the adapter system interface specification, since standard adapters cable to internal storage via a connector(s) on the (undefined) top edge of the adapter PCB. ExpressModule has no such “free” PCB edge. The only way back into the server


24

chassis is through the ExpressModule backplane connector, hence the need to include the storage interface as part of this specification.

Once the storage signals are brought through the connector onto the ExpressModule backplane, they can be cabled to internal storage via a connector(s) on the edge of the ExpressModule backplane PCB. The intent is that the same internal storage infrastructure (connectors, cables, drive backplanes) be usable regardless of whether the signals come from the top edge of a standard form-factor adapter, or the edge of an ExpressModule backplane.

Issue: The electrical path from the initiator IC on board an adapter or ExpressModule to the connector that cables to internal storage is more complex (extra connector) and potentially longer in the ExpressModule case than in the standard adapter case.

Key features of the internal storage interface include:

Up to four SAS/SATA channels. This enables direct support for up to four drives. Larger drive counts are supported via SAS Expanders or SATA II Port Multipliers, which typically reside on the internal drive backplane.

In SAS mode, the four channels can act as independent singlewide pipes, or be aggregated into two doublewide or a single fourwide.

Initial target is 3G SAS and 3G SATA II. The goal is to support 6G SAS in tandem with gen2 PCI Express.

The storage interface is implemented via an optional connector extension.

Dual-initiator support: A system can optionally provide two storage-enabled slots, which can be cabled to a dual-port drive backplane that supports dual-port SAS drives (or “dual-port” SATA drives by virtue of SATA II Port Multiplexors). Note: Any required inter-ExpressModule communications path is outside the scope of this specification. (The needed path is typically provided on the drive backplane.)

A storage-enabled slot can accept a nonstorage enabled ExpressModule (assuming of course something else is driving internal storage). A storage enabled ExpressModule can be plugged into nonstorage enabled slot.

The storage interface supports an optional sideband interface for miscellaneous storage management functions like driving LED status indicators or reporting drive presence. Currently two different 4-wire sideband standards exist in the market for this function, referred in this specification as “I2C/SMBus” and “Serial GPIO (SGPIO).” Since only one or the other will be in use in any given system, this specification maps these two standards on the same four ExpressModule connector pins. If a storage-enabled ExpressModule is to work in either type system, it must implement both interfaces and multiplex them onto the same physical signals. Before it drives one or the other, it must first determine which type system it is plugged into and configure the multiplexer (and select appropriate management firmware) accordingly.

The general philosophy of the ExpressModule is to enable mistake-proof closed chassis addition or replacement of I/O modules without having to take a server off-line. It is important to realize and account for the fact that when an ExpressModule is used to support internal storage, some of these objectives are unavoidably compromised. For example, you have to make sure the storage-enabled module gets into the storage-enabled slot. If a storage-enabled module is offered as an after-market upgrade (the system may initially ship with internal drives managed by a system board chip), then the


25

chassis may have to be opened to rearrange internal cables. Finally, it is unlikely a system can be built that will tolerate a hot-swap of the adapter that provides access to its boot volume.

To mitigate these issues, an OEM (on select products) may prefer a non-ExpressModule strategy like RAID-on-Motherboard (ROMB) to manage internal drives.

2.4.1. Storage Interface Signal Definitions

The storage interface signals identified below and are listed in Table 6-2 with the connector pins locations identified in Table 6-3.

2.4.1.1. High Speed SAS/SATA Channels

SAS/SATA_TX+[3:0], SAS/SATA_TX-[3:0]: SAS/SATA transmit pairs (eight signals total) driven by ExpressModule.

SAS/SATA_RX+[3:0], SAS/SATA_RX-[3:0]: SAS/SATA receive pairs (eight signals total) sensed by ExpressModule.

2.4.1.2. Storage Sideband Signals – I2C/SMBus or SGPIO

As stated before, two orthogonal sideband interfaces – I2C/SMBus and SPGIO – are supported over the same physical signal pins. The “STOR” prefix in front of the I2C/SMBus signal names distinguishes them from the ExpressModule management interface, which also uses the I2C/SMBus standard. Note in the storage case, the module rather than system is the master. (The drive backplane is the slave.)

Table 2-3: Storage Sideband Signals: I2C/SMBus

Connector Pin Name Signal Name Function Module I/O STOR_SB-1 STOR_SMBCLK I2C/SMB CLOCK O

STOR_SB-2 STOR_SMBDAT I2C/SMB DATA I/O

STOR_SB-3 STOR_SMBINT I2C/SMB INTERRUPT I

STOR_SB-4 STOR_SMBRST I2C/SMB RESET O


26

Table 2-4: Storage Sideband Signals: SGPIO

Connector Pin Name Signal Name Function Module I/O STOR_SB-1 SCLOCK Clock O

STOR_SB-2 SDATAOUT Output data O

STOR_SB-3 SDATAIN Input data I

STOR_SB-4 SLOAD Load data signal O

2.5. Power Management Power management signals are the signals that control module power, indicate power good, and wake the chassis.

If the platform with the PCI Express interface supports the wake function, then the WAKE signal must be accepted from all PCI Express ExpressModule connectors. Table 4-1 lists the required specifications for the power supply rails available at the PCI Express connectors. The system designer is responsible for ensuring that the power delivered to the PCI Express connectors meets the specifications called out in Table 2-24.

2.5.1. Initial Power-Up (G3 to L0)

As long as PWREN# is inactive, the primary power converter on the adapter is off and all PCI Express functions are powered off. When the system is powered on, the adapter’s bulk power ramps up to its specified levels.

The power enable (PWREN#) signal can be active during or after this system power stabilization time. In ExpressModule slots that do not support hot plug, the power enable pin can be connected to ground. This allows the module to power up at the same time as the rest of the system. In ExpressModule slots that do support hot-plug, the power enable pin is controlled by the hot-plug logic in the PCI Express port under software control and happens after the system power has stabilized in most cases.

Once the ExpressModule primary power converter has been enabled (PWREN# active) and producing stable adapter primary power, the ExpressModule power good signal (MPWRGD#) goes active.

The chassis combines ExpressModule present detection (PRSNT#) and module power good active (MPWRGD#) on the chassis to enable the REFCLK signals to the module slot.

Once the adapter MPWRGD# signal is active for TPVPERL and the system reset input signal is inactive, the module slot reset (MRST#) signal goes inactive allowing the module to exit the reset state and the PCI Express Link to become active.

The module initial power up order requires that 3.3V aux power be applied first or at the same time as the bulk 12V.


27

A-0453

+12V

PWREN#

MPWRGD#

System Reset input

MRST#

REFCLK

PCI Express link

SMBus

+3.3Vaux

1. Aux stable to SMBus driven.2. Minimum time from +12V stable to MRST# inactive TPVPERL.3. Minimum time from reference clock (REFCLK) valid to MRST# inactive TMRST-clk.4. Minimum time from reference clock to PCI Express link active.

2

1

3

4

Active

Active

Figure 2-10: Power Up

2.5.2. Power Management States (S0 to S3/S4 to S0)

If the system wants to enter S3/S4, devices are placed into D3hot states with Links in L2 prior to any power transitions at the slot. The main power and reference clock supplied to the PCI Express slot will go inactive and stay inactive until a wakeup event. As a result of the removal of main power, devices enter the D3cold state. During the D3cold state, +3.3Vaux remains at 3.3 V. On the wakeup event (5 in Figure 2-11), the power manager restores the main power and reference clocks. As in the last section, MRST# de-asserts TPVPERL after the clocks and power are stable.


28

A-0454

1. PCI Express link goes to idle prior to MRST# going active.2. REFCLK goes inactive after MRST# goes active.3. PWREN# goes inactive after REFCLK goes inactive.4. Wakeup event or WAKE goes active causing power to return then PWREN# goes active, restarts REFCLK and MRST# goes inactive as in power up sequences.5. Minimum active time for MRST# is TMRST.6. Maximum hold time for WAKE from MRST# active is TWAKE.

+3.3Vaux

+12V

MRST#

MPWRGD#

WAKE

PWREN#

REFCLK

PCI Express link

SMBus Active

ActiveActive

4

5

12

3

6

Figure 2-11: Power Management States


29

2.5.3. Power Down

Figure 2-12 shows the power down sequences for an orderly power down. That is, the PCI Express bus goes idle then module reset (MRST#) goes active to the module. This is followed by the reference clock (REFCLK) going inactive and, finally, by power enable (PWREN#) going inactive which, in turn, deactivates the on-module power converter.

During a surprise removal, present detection and power enable will both go inactive at the same time (short pins on the module connector) forcing the power converter and reference clock off.

A-0455

1. PCI Express link goes inactive prior to MRST# (Device in D3hot).2. REFCLK goes inactive after MRST# goes active.3. MRST# goes active before +12V goes unstable.4. In the case of a surprise power down, MRST# goes active TFAIL after +12V goes unstable.

+3.3Vaux

+12V

PWREN#

MPWRGD#

MRST#

REFCLK

PCI Express link

SMBus Active

Active

3

2

1

Inactive

Figure 2-12: Power Down

2.6. Management The System Management Bus (SMBus) is a two-wire interface and a management alert through which various system component chips can communicate with each other and with the rest of the system. It is based on the principles of operation of I2C.

This out-of-band management (not through the PCI Express interface) interface is used to provide information to the host chassis and for out-of-band control of some features of the adapter.

SMBus provides a control bus for system and power management related tasks. A system may use SMBus to pass messages to and from devices instead of tripping individual control lines. Removing the individual control lines reduces pin count. Accepting messages ensures future extensibility.


30

The SMBus is the host’s interface to the adapter’s Vital Product Data (VPD) EEPROM. The VPD devices are required of all ExpressModules and provide manufacturer and service information (see Table 2-13 for required records). The VPD EEPROM shall be powered from the Vaux power. The VPD EEPROM can also provide additional information through optional record types; these records can enhance the serviceability of the module or provide information about special functions it supports.

SMBus is described in the System Management Bus (SMBus) Specification, Version 2.0. Refer to this specification for DC characteristics and all AC timings. The system board and ExpressModule shall support SMBus topology as defined in Section 2.6.7.

Each ExpressModule slot in a chassis shall be provided with its own SMBus segment, and each slot shall have its own address space (the SMBuses of ExpressModule slots are not bused together). This allows all modules to have VPD and other components at the same bus addresses without address conflicts. Host chassis connect to each slot’s management interface individually for module management; empty slots management interface will be weakly pulled down on the host side of the interface to minimize hot-plugging issues. The host choice of management multiplexer/switch needs to consider this because it could be locked to an empty slot.

2.6.1. Capacitive Load of High-power SMBus Lines

The maximum capacitive load for a physical SMBus segment is 400 pF. Each ExpressModule SMBus and its host interface is a physical segment and thus can support a maximum capacitive load of 400 pF. The ExpressModule portion of the segment is limited to 200 pF per signal pin to provide system board layout flexibility. There is no limitation on the number of SMBus devices on the ExpressModule provided that meet the requirements specified within this specification.

Capacitive load for each bus line includes all pin, wire, and connector capacitances. The maximum capacitive load affects the selection of the pull-up resistor or the current source in order to meet the rise time specifications of SMBus.

Normally, pin capacitance is defined as the total capacitive load of one SMBus device as seen in a typical manufacturer’s data sheet. The value in the DC specifications (COUT in Table 2-24) is a recommended guideline so that two SMBus devices may, for example, be populated on an add-in card.

2.6.2. Minimum Current Sinking Requirements for SMBus Devices

The ExpressModule uses the high-power DC electrical specification as defined in the System Management Bus (SMBus) Specification, Version 2.0. ExpressModule SMBus devices are required to sink a minimum current of 4 mA while maintaining the VOL(max) of 0.4 V. The requirement for 4 mA sink current determines the minimum value of the pull-up resistor RP that can be used in SMBus systems.


31

2.6.3. SMBus “Back Powering” Considerations

Unpowered devices connected to either a low-power or high-power SMBus segment must provide, either within the device or through the interface circuitry, protection against “back powering” the SMBus. Unpowered devices connected to high-power segments must meet leakage specifications in Section 3.1.2.1 of the System Management Bus (SMBus) Specification, Version 2.0.

2.6.4. Power-on Reset

SMBus devices detect a power-on event in one of three ways:

By detecting that power is being applied to the device

By MRST# being asserted

For self-powered or always powered devices, by detecting that the SMBus is active (clock and data lines have gone high after being low for more than 2.5 s)

2.6.5. SMBus Termination and Power

The ExpressModule SMBus is terminated on the module by resistors to the Vaux input. Any SMBus devices on the module that are not powered by Vaux power must be isolated from the SMBus during low power states. This allows the SMBus to operate in the absence of primary power providing VPD data and management access before powering the card or while in a reduced power state. Components attached to these signals need to have a 3.3V signaling tolerance.

2.6.6. SMBAlert

The ExpressModule management bus alert is used to make the host aware of a need for attention from the management system. An example of this would be an optional temperature-monitoring device on a ExpressModule that is set to alert the management system when a set temperature is reached on the adapter. This is an optional output from ExpressModule but a required input for the system. If not supported by the ExpressModule, it shall be grounded. The host side of this signal is pulled low with a weak pull-down.

2.6.7. Management Bus Topology

Figure 2-13 illustrates the general connections between the modules and a host system that incorporates a management microcontroller. In this example, the individual SMBAlert signals are routed to an intermediate I/O port device in order to speed the ability for the management controller to detect which given module generated the alert. The I/O port device provides both a mechanism for “OR-ing” the SMBAlert signals as well as status bits that the management microcontroller can use to rapidly determine which bus was the source of the alert. The approach also has the advantage of providing a way for the management controller to disable or isolate different segments in case there was a problem with one of the segments. Otherwise, if something


32

like a simple “OR” of the SMBAlert signals was done, a module that erroneously forced the SMBAlert signal into the asserted state could lock out alerts from the other modules.

Note that an alternative implementation could also use GPIO pins on the management controller, external parallel latches, or other approaches to accomplish the same goals. However, this approach shown also works equally well if the device is a generic chipset SMBus controller.

A-0456

SMBus

SMB Alert

Host ManagementController

(e.g. IPMI BMC)or SMBus Host

Controller

SMBusMultiplexer

SMBus I/OPort withInterrupt

Module 1

Module 2

Module 3

SMBus 1

SMBus 2

SMBus 3

SMB Alert 1

SMB Alert 2

SMB Alert 3

Figure 2-13: Example SMBus Topology

2.6.8. Data Integrity

Data integrity for VPD devices is handled by checksums that are included in the data format. Data integrity for management controller messaging is provided using the SMBus Packet Error Code (PEC) byte (an 8-bit CRC defined in the System Management Bus (SMBus) Specification, Version 2.0). The Basic Management Status register is not protected by data integrity. However, because the values are “read only,” software can issue multiple reads as a means to verify that a change in data has occurred.


33

2.6.9. Basic Management Status Register

The optional Basic Management Status register is a 16-bit register that is accessed using an SMBus “Read Word” transaction. Figure 2-14 illustrates the basic format of this transaction. Refer to the System Management Bus (SMBus) Specification, Version 2.0 for details of operation.

S 0100_000x 1 A Data 0 A Data1 A=1 P 7 1 8 8 Slave Addr R/W LSB MSB

Where "S" = SMBus START condition, "P" = SMBus STOP condition, A = SMBus ACK bit. Shaded portions indicate data from module to host. Unshaded portions indicate data from host to module.

Figure 2-14: Basic Management Status Register Access

The host can detect the presence of the register by noting whether the device ACKs the device address. If the device is not present, the slave address will be NACK’d (a “1” received for the first ACK bit).

The Basic Management Status bits are “self clearing.” That is, the bits are not latched and will clear themselves when the condition clears.

Table 2-5 specifies the individual bits of this status.

Table 2-5: Basic Management Status Register

Bit Function/Description Bit Function/Description 15 reserved 7 Fatal error

14 reserved 6 Predictive failure

13 reserved 5 Power subsystem initiated shutdown

12 reserved 4 Monitored power out-of-range

11 reserved 3 Monitored power warning

10 reserved 2 Over temperature shutdown

9 reserved 1 Critical over temperature

8 reserved 0 Over temperature warning


34

2.6.10. Satellite Management Controller

If more sophisticated monitoring is desired, such as returning various voltage and temperature readings, delivering asynchronous events to a BMC, a module can optionally implement an IPMI Satellite Controller (an autonomous microcontroller that provides IPMI-based, discoverable management functions). If implemented, the controller must meet the following requirements:

Has its interface at address D4h (1101_000x) on SMBus and is accessed using the “PCI SMBus” protocol per [IPMI].

Meets the mandatory requirements for a Satellite controller per “General Mgmt. Controller Requirements” in [IPMI].

Provides “Device SDRs (Sensor Data Records)” for the sensors it provides per [IPMI].

Remains operational on standby power (i.e., can be communicated with via IPMI commands on the SMBus). Some sensors or functions may become unavailable on standby power but must meet IPMI requirements; for example, implementing the “reading/state unavailable bit” for sensors that become unavailable due to power state.

Is not reset with PCI resets. Only “resets” and initializes when standby power is first applied or by IPMI command. (There is no requirement to implement command-based resets.)

Has its primary FRU data provided by a VPD (FRU) device directly accessed on the SMBus of the module. This FRU device shall follow the VPD (FRU) Information Format given in this specification. The module may also provide a primary FRU device that is accessible via the satellite controller via IPMI “Read/Write FRU” commands. This FRU information, if present, must be formatted per [IPMI].

2.6.11. Remote Management Card Access

A module may provide a “Remote Management Card” function for the system. Remote Management Cards typically operate on standby power, provide their own communications interfaces (e.g., LAN), and have on-board microcontrollers that can be used to provide system access, recovery operations, and health status under conditions where the system is inaccessible via software running on the host processor(s).

If the motherboard contains an IPMI BMC (baseboard management controller) that is accessible via the SMBus connection, this connection can be used by the Remote Management Card to access motherboard sensor, event log, control (power on/off, reset, etc.), and other functions offered via IPMI. Because the BMC is required to remain available on standby power, functions such as event log and power/reset control are available even if the system is powered down. The presence of an IPMI BMC can be detected by sending an IPMI “Get Device ID” command to SMBus address 20h (0010_000x) using the PCI SMBus protocol specified in [IPMI].


35

2.6.12. VPD (FRU) Information Format

The VPD information is formatted as a series of variable length records following a “Common Header.” The Common Header is located at offset 00h in the VPD device and specifies which version of the specification defined the VPD Format and the offset to the first record, as shown in Table 2-6.

Table 2-6: VPD (FRU) Common Header

Byte Field 1 [7] - 1b = OEM Reserved Space present

[6:4] - reserved [3:0] VPD Version - 01h for this specification

2 VPD Record Offset LSB - Offset to first VPD record, least significant byte, 0-based

3 VPD Record Offset MSB - Offset to first VPD record, most significant byte, 0-based

4 53h “S” The four “ExpressModule” bytes are to improve the robustness of discovery of an ExpressModule VPD device.

5 49h “I”

6 4Fh “O”

7 4Dh “M”

8 Common Header Checksum Note that typically the first record data will immediately follow the common header. However, a manufacturer can reserve a section of the VPD device for private use by setting an offset and leaving space between the end of the Common Header and the first VPD Record and indicating that this space is being reserved for OEM use by setting the “OEM Reserved Space” bit in the Common Header.

If OEM Reserved Space is used, the first bytes of data following the Common Header shall be an OEM Reserved Space Header, as shown in Table 2-7.


36

Table 2-7: OEM Reserved Space Header

Field Length Field

3 Manufacturer ID. IANA Enterprise ID number for the manufacturer or organization that has defined the format and content of the space between the Common Header and first VPD Record.

1 Header Checksum (zero checksum) Utilities that are specific to the module manufacturer may change the Common Header as part of field updates of the module. Generic utilities are not allowed to alter the values of the Common Header.

2.6.12.1. VPD (FRU) Record Format

Each VPD Record starts with a Record Header. The Record Header for each record defines the type of information contained in the record. Any data bytes associated with the record immediately follow the last field of the header. Unless otherwise specified, multi-byte numeric data is stored least-significant byte first. The Record Header also contains offset information needed to access any additional following records, if any, in a sequential list manner. The Record Header format is shown in Table 2-8.

Table 2-8: VPD (FRU) Record Format

Field Length Field

1 Record Type ID

1 7:7 – End of list 6:4 – Reserved, write as 000b 3:0 – Record Format version (=1h unless otherwise specified)

1 Record Length

1 Header Checksum (zero checksum)

N Record Data (according to Record Type)

1 Record Checksum (zero checksum). The record checksum covers all bytes in the Record Data field.


37

2.6.12.2. Record Header Fields

The following provides additional information about the individual fields in the Record Header:

Record Type ID This unsigned byte is used to identify the information contained in the record. Table 2-9 shows the currently defined record types.

Table 2-9: VPD (FRU) Record Types

ID Record Type Required* 0x00 reserved -

0x01 Board Info Record O

0x02 Product Info Record M

0x03 Product GUID O

0x04 Board GUID O

0x05 ExpressModule Module Info M

0x06 PCI Express Device Info M

0x07 ExpressModule Multiplexer O**

0x08-0xBF reserved -

0xC0-0xFF OEM Record Types O

* Indicates whether record is required for the VPD device on a module ** The ExpressModule Multiplexer record is mandatory to be provided by a VPD device on the base board when the SMBuses are accessed via an SMBus host controller. “O” = Optional “M” = Mandatory

End of List. This bit indicates if this record is the last record in the Multi-Record area. If this bit is zero, it indicates that one or more records follow.

Record Format Version. The area version format is stored in the lower nibble of the second byte. This field is used to identify the revision level of information stored in this area. This number will start at zero for each new area. If changes need to be made to the record, e.g., fields added/removed, the version number will be increased to reflect the change.

Unless otherwise specified, the record format version for all record types shall be 01h for this specification (including OEM Record Types). The latter provision is to provide for standardized data fields that precede the OEM specific data within the OEM Record.

Record Length. This unsigned byte indicates the number of bytes of data in the record. This byte can also be used to find the next area in the list. If the “End of List” bit is zero, the length can be added to the starting offset of the current Record Data to get the offset of the next Record Header. This field allows for 0 to 255 bytes of data for each record.

Header Checksum. This unsigned byte is used to give the generic header a zero checksum; i.e., the modulo 256 sum of the preceding bytes (starting with the first byte of the header) plus the checksum byte equals zero.


38

Record Checksum. This unsigned byte can be used to calculate a zero checksum for the Record Data following the Record Header; i.e., the modulo 256 sum of the record data bytes plus the checksum byte equals zero.

2.6.12.3. Board Info Record

This optional record provides Serial Number, Part Number, and other information about a board or other FRU that is a sub-component of the module. Thus, the name “Board Info Area” is somewhat a misnomer, because the usage is not restricted to just circuit boards. This area can also be used to provide FRU information for any replaceable entities, boards, and or sub-assemblies within the module.

Table 2-10: Board Info Data Format

Field Length Field

1 Board Info Record Format Version 7:4 - reserved, write as 0000b 3:0 - format version number = 1h for this specification

1 Language Code (see Section 2.6.12.10)

1 Board Number A number assigned by the module manufacturer to identify different types of sub-component within the module FFh = unspecified

3 Mfg. Date / Time Number of minutes from 0:00 hrs 1/1/96 LS byte first (little endian) 00_00_00h = unspecified

1 Board Manufacturer type/length byte

P Board Manufacturer bytes

1 Board Product Name type/length byte

Q Board Product Name bytes

1 Board Serial Number type/length byte*

N Board Serial Number bytes*

1 Board Part Number type/length byte

M Board Part Number bytes

1 FRU File ID type/length byte*

R FRU File ID bytes* The FRU File version field is a predefined field provided as a manufacturing aid for verifying the file that was used during manufacture or field update to load the FRU information. The content is manufacturer-specific. This field is also provided in the Product Info area. Either or both fields may be “null.”


39

Field Length Field

1 Board EC Number type/length byte. Engineering change level (revision) of this board.

X Board EC Number bytes

xx Additional custom Mfg. Info fields. Defined by manufacturer. Each field must be preceded by a type/length byte.

1 C1h (type/length byte encoded to indicate no more info fields)

Y 00h - any remaining unused space

* These areas are always encoded as if the Language Code were English.

2.6.12.4. Product Info Record

This record data is present if the FRU itself is obtainable as a separate product. This is typically seen when the FRU is an add-in card, sub-assembly, or a power supply from a separate vendor. This record will be present in all PCI Express modules.

Table 2-11: Product Info Data Format

Field Length Field

1 Product Info Record Format Version 7:4 - reserved, write as 0000b 3:0 - format version number = 1h for this specification

1 Language Code (See Section 2.6.12.10)

1 Manufacturer Name type/length byte

N Manufacturer Name bytes

1 Product Name type/length byte

M Product Name bytes

1 Product Part/Model Number type/length byte

O Product Part/Model Number bytes

1 Product Version type/length byte R Product Version bytes 1 Product Serial Number type/length byte*

P Product Serial Number bytes*

1 Asset Tag type/length byte

Q Asset Tag

1 FRU File ID type/length byte


40

Field Length Field

R FRU File ID bytes The FRU File version field is a predefined field provided as a manufacturing aid for verifying the file that was used during manufacture or field update to load the FRU information. The content is manufacturer-specific. This field is also provided in the Board Info area. Either or both fields may be “null”.

xx Custom product info area fields, if any (must be preceded with type/length byte)

1 C1h - (type/length byte encoded to indicate no more info fields)

Y 00h - any remaining unused space

* These fields are always encoded as if the Language Code were English; i.e., if the type/length code bits 7:6 = 11b the serial number will always be interpreted as ASCII+Latin 1, not UNICODE.

2.6.12.5. Product GUID Record

This record provides a readable copy of the product (module) GUID.

Table 2-12: Product GUID Data Format

Field Length Field

1 Product Area Format Version 7:4 - reserved, write as 0000b 3:0 - format version number = 1h for this specification

16 GUID - least significant byte first


41

2.6.12.6. ExpressModule Module Info Record

This record data provides information specific to ExpressModules.

Table 2-13: ExpressModule Module Info Data Format

Field Length Field

1 Product Info Record Format Version 7:4 - reserved, write as 0000b 3:0 - format version number = 1h for this specification

1 [7:5] - reserved [4] - 1b = management controller present [3] - 1b = basic status register present [2:0] - Module Size: 000b = Singlewide module 001b = Doublewide module all others = reserved

1 7:3 - reserved 2:0 - Link Width: 000b = x1 001b = x2 010b = x4 011b = x8 100b = x16 all others = reserved

2 Basic Status Support [15:0] - 1b = corresponding bit in Basic Status register is supported.

1 IPMI Satellite Controller Slave Address 00h = unspecified

1 Device Max Power Consumption in watts (1-based) 2 Module Peak 12 V Power consumption in mA

2 Module Typical (average) 12 V Power consumption in mA


42

2.6.12.7. PCI Express Device Info Record

This record data provides a readable copy of the PCI Express device’s class and function information. For multi-function devices, the record provides information about which PCI Device Functions are potentially available on a module. It does not, however, provide information on whether a given function is presently enabled for use or not.

The record contains a Device Info Sub-record for each possible, separately accessible, function in the device.

Table 2-14: PCI Express Module Info Data Format

Field Length Field

1 PCI Express Device Info Record Format Version 7:4 - reserved, write as 0000b 3:0 - format version number = 1h for this specification

1 Number of device info sub-records that follow (1-based)

Device Info Sub-record

1 7:5 - reserved, write as 000b 4 - 0b = Real Function Number. The following

function number corresponds to the PCI Function number used to select the function on the device.

1b = Pseudo Function Number. In some cases, the function number assignments may vary according to how the device is configured, and the VPD data may not be able to be kept in synch with the real function number. In this case, the functions in the device info sub-record can be given pseudo function numbers for the purpose of indicating a particular instance of a function.

3:0 - Function Number

2 Vendor ID*. LS byte first.

2 Device ID*. LS byte first.

1 Base Class Code*

1 Sub-class Code*

1 Interface Code*

9

1 Revision ID*

* Refer to the PCI Express Base Specification, Revision 1.1 for the definition.


43

2.6.12.8. ExpressModule Multiplexer Record

This record provides a signature that can be used to discover and confirm the presence of an ExpressModule-formatted “baseboard” VPD device on the motherboard.

Table 2-15: ExpressModule Multiplexer Record Data Format

Field Length Field

1 ExpressModule Multiplexer Record Format Version 7:4 - reserved, write as 0000b 3:0 - format version number = 1h for this specification

1 Multiplexer type: 00h = reserved 01h = Philips Semiconductors 9541 or compatible 02h = Philips Semiconductors 9542 or compatible 03h = Philips Semiconductors 9543 or compatible 04h = Philips Semiconductors 9544 or compatible 05h = Philips Semiconductors 9545 or compatible

1 Multiplexer level. 7:2 - reserved 1:0 - 00b = level 0. Connects directly to SMBus host controller. 01b = level 1. Multiplexer is directly connected behind a level 0 multiplexer. 10b = level 2. Multiplexer is directly connected behind a level 1 multiplexer. 11b = reserved

1 Multiplexer slave address. 7-bits, left justified. Least significant bit always is 0b. For example, a slave address of 1010_000x results in a value of A0h in this byte.

1 Slave address of multiplexer of “upstream” multiplexer (closer to SMBus host) that this multiplexer connects to. The address is 00h if this multiplexer is a “level 0” multiplexer (connects directly to host).


44

Field Length Field

1 Starting bus/connector ID. The ID is 0-based. This is the lowest ID for the ExpressModule SMBus segments connected to the multiplexer. This does not include the segment to the host controller.

1 Ending bus/connector ID. 0-based. This is the highest ID for the ExpressModule SMBus segments connected to the multiplexer. This does not include the segment to the host controller. Segments must be connected to multiplexers sequentially and with no gaps in the sequence. In addition, the lowest connector ID must be connected to the lowest numbered (or lettered) bus identified on the multiplexer.

2.6.12.9. OEM Records

OEM Records have one predefined data field: a 3-byte Manufacturer ID field, as specified in the Get Device ID command in the IPMI v1.0 specification. This field occupies the first 3 bytes of the data portion of the record. Subsequent fields within the record are defined by the given OEM.

Note: OEM Records are also followed by a Record Checksum Byte, per the VPD (FRU) Record Format.

Table 2-16: OEM Record Data Format

Offset Length Description 0 3 Manufacturer ID. LS Byte first.

3:N N OEM data, as specified by the party identified by the Manufacturer ID field


45

2.6.12.10. Language Codes

The following table should be used when interpreting the Language Code fields. The index number (1-136) is stored in the Language Code field. Any language code other than English indicates that the string data is encoded as Unicode when bits 7:6 of the Type/Length code = 11b.

Table 2-17: Language Codes

1. aa Afar 51. it Italian 101. si Singhalese 2. ab Abkhazian 52. iw Hebrew 102. sk Slovak 3. af Afrikaans 53. ja Japanese 103. sl Slovenian 4. am Amharic 54. ji Yiddish 104. sm Samoan 5. ar Arabic 55. jw Javanese 105. sn Shona 6. as Assamese 56. ka Georgian 106. so Somali 7. ay Aymara 57. kk Kazakh 107. sq Albanian 8. az Azerbaijani 58. kl Greenlandic 108. sr Serbian 9. ba Bashkir 59. km Cambodian 109. ss Siswati 10. be Byelorussian 60. kn Kannada 110. st Sesotho 11. bg Bulgarian 61. ko Korean 111. su Sudanese 12. bh Bihari 62. ks Kashmiri 112. sv Swedish 13. bi Bislama 63. ku Kurdish 113. sw Swahili 14. bn Bengali; Bangla 64. ky Kirghiz 114. ta Tamil 15. bo Tibetan 65. la Latin 115. te Tegulu 16. br Breton 66. ln Lingala 116. tg Tajik 17. ca Catalan 67. lo Laothian 117. th Thai 18. co Corsican 68. lt Lithuanian 118. ti Tigrinya 19. cs Czech 69. lv Latvian, Lettish 119. tk Turkmen 20. cy Welsh 70. mg Malagasy 120. tl Tagalog 21. da danish 71. mi Maori 121. tn Setswana 22. de german 72. mk Macedonian 122. to Tonga 23. dz Bhutani 73. ml Malayalam 123. tr Turkish 24. el Greek 74. mn Mongolian 124. ts Tsonga 25. en English 75. mo Moldavian 125. tt Tatar 26. eo Esperanto 76. mr Marathi 126. tw Twi 27. es Spanish 77. ms Malay 127. uk Ukrainian 28. et Estonian 78. mt Maltese 128. ur Urdu 29. eu Basque 79. my Burmese 129. uz Uzbek 30. fa Persian 80. na Nauru 130. vi Vietnamese 31. fi Finnish 81. ne Nepali 131. vo Volapuk 32. fj Fiji 82. nl Dutch 132. wo Wolof 33. fo Faeroese 83. no Norwegian 133. xh Xhosa 34. fr French 84. oc Occitan 134. yo Yoruba 35. fy Frisian 85. om (Afan) Oromo 135. zh Chinese 36. ga Irish 86. or Oriya 136. zu Zulu 37. gd Scots Gaelic 87. pa Punjabi 38. gl Galician 88. pl Polish 39. gn Guarani 89. ps Pashto, Pushto 40. gu Gujarati 90. pt Portuguese 41. ha Hausa 91. qu Quechua 42. hi Hindi 92. rm Rhaeto-Romance 43. hr Croatian 93. rn Kirundi 44. hu Hungarian 94. ro Romanian 45. hy Armenian 95. ru Russian 46. ia Interlingua 96. rw Kinyarwanda 47. ie Interlingue 97. sa Sanskrit 48. ik Inupiak 98. sd Sindhi 49. in Indonesian 99. sg Sangro 50. is Icelandic 100. sh Serbo-Croatian


46

2.6.12.11. Type/Length Byte Format

The following presents the specification of the type/length byte. Type/Length encoded field data is always stored most-significant digit/character first.

7:6 - type code

00 - binary or unspecified

01 - BCD plus (see below)

10 - 6-bit ASCII, packed (overrides Language Codes)

11 - Interpretation depends on Language Codes. 11b indicates 8-bit ASCII + Latin 1 if the Language Code is English for the area or record containing the field or 2-byte UNICODE (least significant byte first) if the Language Code is not English. At least two bytes of data must be present when this type is used. Therefore, the length (number of data bytes) will always be >1 if data is present; 0 if data is not present.

5:0 - number of data bytes.

000000 indicates that the field is empty. When the type code is 11b, a length of 000001 indicates “end of fields;” i.e., Type/Length = C1h indicates “end of fields.”

“ASCII+LATIN1”is derived from the first 256 characters of Unicode 2.0. The first 256 codes of Unicode follow ISO 646 (ASCII) and ISO 8859/1 (Latin 1). The Unicode “C0 Controls and Basic Latin” set defines the first 128 8-bit characters (00h-7Fh), and the “C1 Controls and Latin-1 Supplement” defines the second 128 (80h-FFh).

“6-bit ASCII” is the 64 characters starting from character 20h (space) from the ASCII+LATIN1 set. So 6-bit ASCII value 000000b maps to 20h (space), 000001b maps to 21h (!), etc. Packed 6-bit ASCII takes the 6-bit characters and packs them four characters to every 3 bytes, with the first character in the least significant 6 bits of the first byte. A table of 6-bit ASCII codes and an example of packed 6-bit ASCII characters are included later in this section.


47

2.6.12.11.1. BCD Plus Definition

0h - 9h = digits 0 through 9

Ah = space

Bh = dash “-”

Ch = period “.”

Dh = reserved

Eh = reserved

Fh = reserved

2.6.12.11.2. 6-bit ASCII Definition

0 sp 10 0 20 @ 30 P

1 ! 11 1 21 A 31 Q

2 " 12 2 22 B 32 R

3 # 13 3 23 C 33 S

4 $ 14 4 24 D 34 T

5 % 15 5 25 E 35 U

6 & 16 6 26 F 36 V

7 ' 17 7 27 G 37 W

8 ( 18 8 28 H 38 X

9 ) 19 9 29 I 39 Y

A * 1A : 2A J 3A Z

B + 1B ; 2B K 3B [

C , 1C < 2C L 3C \

D - 1D = 2D M 3D ]

E . 1E > 2E N 3E ^

F / 1F ? 2F O 3F _

2.6.12.11.3. 6-bit ASCII Packing Example

"IPMI" encoded in 6-bit ASCII is:

I = 29h (101001b)

P = 30h (110000b)

M = 2Dh (101101b)

I = 29h (101001b)

Which gets packed into bytes as follows: bit 7 6 5 4 3 2 1 0 hex

byte 1 0 0 1 0 1 0 0 1 29h byte 2 1 1 0 1 1 1 0 0 DCh byte 3 1 0 1 0 0 1 1 0 A6h


48

2.6.13. Using an SMBus Multiplexer with a Host Controller

If the modules are connected to the host via an SMBus host controller rather than a BMC or other management controller, the motherboard must provide an ExpressModule baseboard VPD Device located at address A6h (1010_011x) on the non-multiplexed SMBus segment straight out of the host controller. This device must be a validly formatted ExpressModule VPD (FRU) Device containing an ExpressModule Multiplexer Record. It can optionally contain other VPD records for the baseboard.

Software can scan the addresses A6h to look for the VPD device and confirm it is an ExpressModule VPD Device by verifying the common header and checksums and by locating and verifying the presence of one or more ExpressModule Multiplexer records. If present, this record tells the software what type of multiplexer device is being used, the corresponding slave address, and what ExpressModule module buses are connected to the multiplexer. More than one multiplexer device can be used; in which case, there will be more than one ExpressModule Multiplexer record. Note that multiplexers can be cascaded up to two levels deep behind a multiplexer that is directly connected to SMBus segment to the host controller.

2.6.14. Implementing ExpressModule Management in an IPMI Environment

The use of the SMBus for management access facilitates the use of IPMI Management Controllers. This can be accomplished by providing a mechanism whereby the SMBus can be used for sending and receiving IPMI management messages to the modules.

IPMI presently defines communication protocols for an SMBus-based management bus used for PCI, and an I2C-based bus called the IPMB. These protocols define how to use those bussed for sending and receiving IPMI management messages between management microcontrollers on the bus.

There are significant differences between those busses and the ExpressModule management bus. The PCI SMBus and the IPMB are implemented as single-segment busses where all management controllers share the same bus segment. This enables communication between any two devices as peers using the multi-master arbitration capabilities of SMBus and I2C.

The protocols use a “request/response” approach where one party arbitrates for and masters the bus, and then transmits a “request” to another party on the bus. The receiver of the message later arbitrates for and masters the bus and sends a “response” back to the originator of the request. This request/response protocol allows peer-to-peer communication and also provides a way to asynchronously generate traffic to the host management controller (system board management controller, or BMC, in IPMI terminology) or any other controller on the bus. Thus, the PCI SMBus and the IPMB do not utilize an SMBAlert lines.

The ExpressModule Management Bus, however, is implemented as a segment per connector, where there is electrical isolation between the segments. This eliminates issues with address assignment and contention and the complexities of multi-master arbitration. But because the segments are isolated, direct peer-to-peer communication is not possible. Furthermore, in order to facilitate using multiplexers that are controlled by the BMC, the devices on the modules are always accessed as slaves.


49

Therefore, a modified approach is required to enable an equivalent level of management communications.

2.6.14.1. Slave Attention-based Protocol

Instead of using a request/response protocol where the BMC masters the bus to send a request to the module, and the module in turn masters the bus to send a response back to the BMC, the request/response protocol is modified so that the BMC can “pull” messages from management controllers on modules by accessing them as slaves.

It is straightforward for the BMC to select the connection to a given bus segment and transmit a message to a management controller on any module. In order for a management controller on the module to generate a message for the BMC however, it first places its message in a buffer and then raises an SMBAlert that signals the BMC to retrieve it. The BMC then retrieves the message from the module by reading the module’s management controller as a slave.

To facilitate this messaging, the slave must support a new Retrieve Message command that enables the BMC to retrieve message data from the slave. Standard IPMI messages can then be received from modules using this slave transaction mechanism.

2.6.14.2. Peer-to-Peer Proxy Messaging

The PCI SMBus and IPMB also have the characteristic that different management controllers can directly talk to each other as peers. In order to allow the ExpressModule management bus segments to be isolated by simple multiplexers, the BMC must provide a “forwarding” service where the BMC routes requests and responses between peers. The BMC therefore acts as a proxy or router for messaging between management controllers on ExpressModules.

In order to support this, there must be additional addressing information that enables the BMC to forward messages to the appropriate management bus segment. The management controller slave addresses are not sufficient for this purpose, because different management controllers on different segments can have the same slave address. Thus, the messaging format must include a bus ID that can be used for routing purposes.

2.6.15. Sending Data to a Management Controller on a Module

Writing to a management controller on a module is accomplished using protocols that are based on top of the SMBus “Write-Block” protocols. The SMBus Write-Block protocol is limited to transferring 32 bytes of data payload plus a “CMD” (command) byte.

This specification defines two transfer protocols that use the SMBus Write-Block protocol to deliver IPMI message data to a management controller on a module. The first protocol is called the “Single-part Write” protocol. This protocol is used when the message can fit within the 32-byte data payload of the SMBus Write-Block. The second protocol is called the “Multi-part Write” protocol and is used when more than 32 bytes are required to transfer the IPMI message.

Both IPMI Requests and Response messages can be carried over these protocols.


50

2.6.15.1. Single-part Write Protocol

The Single-part Write requires a single SMBus Write-Block transaction. The following table shows the format of this transaction. The values in parentheses indicate the number of bits for the particular field when the given field is not 8 bits. Only the address and data portions of the SMBus transactions are shown. SMBus START and STOP conditions and ACK/NACK bits are left out for simplification. The length field provides the count of data bytes of IPMI message content, up to 32 bytes. [PEC] indicates the presence of SMBus PEC (packet error code) byte. Note that this byte is not included in the byte count provided in the length field.

Slave Address (7)

R/W=0 (1)

SMBus CMD = 02h

Length Message Class

Message Data (1 to 31 bytes)

PEC

Figure 2-15: BMC Single-part Write

2.6.15.2. Multi-part Write Protocol

A multi-part write is used when more than 32-bytes of message data need to be written to the BMC. This requires two or more SMBus Write-Block transactions. Three possible types of Write-Block transactions can be used in a multi-part write: A “Start” transaction, a “Middle” transaction, and an “End” transaction. A given message transfer consists of a Start transaction followed by zero or more Middle transactions, and then an End transaction.

The first part of the message is written using the Start transaction. Since Multi-part writes are for the purpose of transferring message data that exceeds 32 bytes, the Start transaction is required to always move 32 bytes of data and the value of the length byte for the Start transaction is always 20h.

The combination of a Start transaction followed by an End transaction can transfer up to 63 bytes of IPMI message. The Middle transaction is available when there is a need to transfer an IPMI message of greater than 63 bytes. As of this writing, there are no standard IPMI messages to the BMC that are longer than 63 bytes. Therefore, the Middle transaction is defined solely as needed by any OEM/group network functions (network function codes 2Ch:3Fh) in the particular BMC implementation.

It is required that all multi-part write transfers end with an End transaction. Middle transactions must always move 31 bytes of data, plus a 1-byte block number, therefore the value of the length byte for Middle transactions is always 20h.

The End transaction is used for the last portion of message data that is written to the BMC. It indicates to the BMC that the message data transfer has completed and the BMC can process the message. The number of message data bytes in the End transaction can range from 1 to 31 bytes.

The following figures show the format of the SMBus transactions for multi-part writes. The “block number” is a value that starts from 0 and is incremented for each Middle transaction;. i.e., the first Middle transaction has a block number of “1,” the second “2,” and so on.


51

The “last block number” is either “0” (if no middle transactions are used) or matches the block number that was used in the last “middle” transaction.

Slave Address (7)

R/W=0 (1)

SMBus CMD = 06h

Length =20h

message class

Message Data (31 bytes)

PEC

Figure 2-16: Multi-part Write Start

Slave Address

(7) R/W=0

(1) SMBus CMD

= 07h Length =20h

block number

Message Data (31 bytes)

PEC

Figure 2-17: Multi-part Write Middle

Slave Address

(7) R/W=0

(1) SMBus CMD

= 08h Length=

01h to 20h last block number

Message Data (1 to 31bytes)

PEC

Figure 2-18: Multi-part Write End

2.6.15.3. Error Conditions for Multi-part Writes

It is possible that out-of-order operations may occur in the course of restarting systems or loading and unloading software. For example, the controller could have just received a Middle transaction when a system restart causes the next operation to be a Start transaction, or data could get corrupted or hot-plug could cause a transaction to be lost.

The management controller shall discard any multi-part write data it has received if a Start transaction is received prior to receiving a complete End transaction.

If the management controller receives an incorrect length (not = 20h) in a Start or Middle transaction, it shall discard any message data that has been received for the Multi-part Write until the next Start transaction is received.

If the management controller detects missing block numbers, it shall discard any message data that has been received until the next Start transaction is received.

2.6.16. Receiving Data From a Management Controller on a Module

Data is read from a management controller on a module using protocols built on top of the SMBus Read-Block protocol. Like the SMBus Write-Block protocol, the Read-Block is limited to 32 data bytes of payload. The Read-Block protocol is driven from the host (typically the BMC) accessing the management controller on the module as a slave. An SMBAlert from the module indicates when the management controller on the module has data to be retrieved.

Similar to the write operations, there are two types of protocol for reads: a Single-part Read protocol that can get up to 31 bytes of data from a management controller on a module and a Multi-part Read protocol for retrieving more than 31 bytes.


52

2.6.16.1. Single-part Read Protocol

The following figure illustrates the format of a SMBus Read Block protocol for a Single-part Read Protocol. The shading indicates the direction of data on the bus. Unshaded fields represent data written from the master (typically a BMC) to the slave (management controller on the module). Shaded fields represent data being read from the slave by the master. Tag is a fixed value field that is used to differentiate between a Single-part Read and a Multi-part Read Start transaction. The Tag is included in the length count; e.g., if there is one byte of message data, the value of the Length field would be 2.

Slave Address

(7)

R/W = 0 (1)

SMBus CMD = 03h

Slave Address

(7)

R/W=1 (1)

Length (1 to 32)

Tag = 0x00

message class

Msg. Data (1 to 31 bytes)

PEC

Figure 2-19: Single-part Read

2.6.16.2. Multi-part Read Protocol

The following table illustrates the format of a SMBus Read Block transaction for the Multi-part Read protocol. There are four different transactions that can be used: Multi-part Read Start, Multi-part Read Middle, Multi-part Read Retry, and Multi-part Read End.

The Read Start and Read End transactions are used together when more than 30 bytes of message data must be read from the management controller. The Read Start transfers the first 30 bytes of message data and the Read End transfers the next 31 bytes. If more than 61 bytes must be read, one or more Read Middle transactions are also used.

The Read Middle includes a 1-byte Block Number field as the first byte of SMBus data in the Read Block. Thus, each Read Middle transaction can carry up to 31 bytes of message data.

The Multi-part Read Retry is used to retry Middle blocks of data in a multi-part read transaction that uses Read Middle transactions. This is described in more detail below.

The following figures illustrate the SMBus protocol formats for the multi-part read transactions. All multi-part read transactions follow the SMBus Read Block protocol with the exception of the Multi-part Read Retry transaction which uses the Write Block protocol.

Slave Address (7)

R/W = 0 (1)

SMBus CMD = 03h

Slave Address (7)

R/W = 1 (1)

Length =20h

Tag = 0x01

messageClass

Message Data (30 bytes) PEC

Figure 2-20: Multi-part Read Start


53

Slave Address

(7) R/W = 0

(1) SMBus CMD

= 09h Slave Address

(7) R/W = 1

(1) Length =20h

Block number

Message Data (31 bytes) PEC

Figure 2-21: Multi-part Read Middle

Where: Tag is a fixed value field that is used to differentiate between a Single-part Read and a Multi-part Read Start transaction. Block number is a number starting with 1 for the first block of message data returned using the Middle transaction and then incremented by 1 for each following Middle Transaction. Block number values 00h and FFh are reserved for identifying the Multi-part Read End transaction.

The multi-part read End transaction is a Read-Block transaction that concludes a multi-part read operation.

Slave Address (7)

R/W = 0 (1)

SMBus CMD = 0Ah

Slave Address (7)

R/W=1 (1)

Length (1 to 32)

[Block Number]

Message Data (1 to 31 bytes) [PEC]

Figure 2-22: BMC Multi-part Read End

Where: The Block Number field contains the block number from the last middle transaction or 00h if no middle transactions were used.

The Multi-part Read Retry transaction is an SMBus Write-Block transaction that tells the management controller to return the given middle Block Number on the next multi-part read Middle transaction. This transaction is only required to be supported when multi-part read Middle transactions are implemented by the controller.

Slave Address (7)

R/W = 0 (1)

SMBus CMD = 0Bh

Length =1

Block number

IPMI Data (1 to 32 bytes) [PEC]

Figure 2-23: Multi-part Read Retry


54

2.6.17. SMBAlert Control and Status

The following transactions retrieve status information for the messaging interface of a management controller on a module.

2.6.17.1. Get Message Interface Status

This operation uses a single SMBus Read Block transaction.

Slave Address (7)

R/W = 0 (1)

SMBus CMD = 04h

Slave Address (7)

R/W = 1 (1)

Length =01h

Interface Status PEC

Figure 2-24: Get Message Interface Status

Where:

Table 2-18: Interface Status Field Definition

Field Length Field

1 Interface Status 7 - 1b = OEM non-message SMBAlerts enabled.

Indicates that SMBAlert is enabled to be generated for sources other than messages that can be retrieved with the Single- and Multi-part read protocols. Note that such sources may require explicit OEM commands or operations to clear the SMBAlert.

6 - 1b = IPMI non-message SMBAlerts enabled. Indicates that SMBAlert is enabled to be generated for IPMI “non-communication” interrupts.

5 - 1b = Message SMBAlerts enabled. Indicates that SMBAlert is enabled to be generated for messages that are retrieved via Single- and Multi-part read protocols.

4:3 - reserved 2 - 1b = OEM non-message notification asserted.

(Indicates a non-message source that would assert SMBAlert is active, and would be asserting SMBAlert if enabled to do so.)

1 - 1b = IPMI non-message notification asserted. This is a source that reflects IPMI “non-communication interrupt” status that is read and cleared using the IPMI Get and Set Interrupt Flags commands, respectively.

0 - 1b = Message available for retrieval. A message is available for retrieval via the Single- or Multi-Part Read commands.


55

2.6.17.2. Set Message Interface Control

This operation uses a single SMBus Write Block transaction.

Slave Address (7)

R/W = 0 (1)

SMBus CMD = 05h

Length =01h

Interface Control PEC

Figure 2-25: Set Message Interface Control

Where:

Table 2-19: Interface Control Field Definition

Field Length Field

1 Interface Control Note that disabling a given SMBAlert source removes it from generating SMBAlert even if the root source of the notification is still asserted. 7 - 1b = Enable OEM non-message SMBAlerts 6 - 1b = Enable IPMI non-message SMBAlerts 5 - 1b = Enable message SMBAlerts 4:0 - reserved

2.6.18. Message Class Values for Management Controller Messaging

Table 2-20: Message Class Values

Message Class Definition 0x00 IPMI Messages

2.6.19. Retention of Output Data

A BMC that implements PEC must retain previous output message data until the occurrence of a valid Write Start transaction, at which time the output message data is cleared. This behavior is needed to better support PEC. If the BMC automatically discarded data as it was read out, there might be no way to recover the message data if the PEC indicated the data was corrupted. However, with this provision, system software can retry the read transaction that had the error.

The BMC will return the retained data if the Single- and Multi-part Read Start transactions are retried prior to the next valid write Start transaction. To re-read a given block of Middle data in a multi-part read, the block number must first be written to the BMC using the Multi-part Read Retry transaction. The next Multi-part Read Middle transaction will then return that block number, and


56

any subsequent Multi-part Read Middle transactions will increment the block number and return following blocks.

2.6.20. SMBAlert Signal Handling for Message Transfers

The management controller on the module asserts SMBAlert to signal that it has data to be retrieved or a status change to be queried. The SMBAlert signal is automatically cleared by the management controller the first time a Single-part Read Start or Multi-part Read Start is used to read a given set of data. SMBAlert will not be asserted by the management controller again until a new instance of data or status is available.

A management controller is allowed to implement OEM functions that can assert SMBAlert. However, since such functions could interfere with the generic operation of the bus, special knowledge may be required to clear such sources. Therefore, such sources of SMBAlert must be disabled by default and require explicit transactions with the management module to be enabled.

2.6.21. Polling for Output Data

If SMBAlert is disabled, a management controller can be polled for output by Read Start transactions until data is returned. If there is no data available, the management controller will NACK the read portion of the SMBus transfer.

2.6.22. SMBus NACKs and Error Recovery

The management controller can NACK the SMBus host controller if it is not ready to accept a new transaction. For example, this could occur if write transactions follow too closely together. (See Table 2-22.) Typically, this will be exhibited by the management controller NACK’ing its slave address. In some cases, the management controller may NACK a SMBus data byte that is being written to it. This can occur if software attempts to write more data bytes to the management controller than it can handle (for example, in a multi-part write), or if some internal state change caused the management controller to need to reset its internal operation.

If the management controller NACKs a single part transaction, software can simply retry it. If a Middle or End transaction is NACK’d, software should not retry the particular but should restart the multi-part read or write from the beginning Start transaction for the transfer.

2.6.23. PEC Handling

The System Management Bus (SMBus) Specification, Version 2.0 allows a slave that receives a PEC to check the PEC and NACK the byte that carried the PEC value if the PEC is incorrect. Accomplishing this may require special hardware in order to generate the NACK without significant SMBus clock stretching. In order to avoid this requirement, a BMC implementation is allowed to always ACK the PEC. A BMC that receives an invalid PEC shall drop the data for the write transaction and any further transactions (read or write) until the next valid read or write Start transaction is received.


57

A management controller that implements PEC will automatically start using PEC in read transactions after it receives any SSIF single-part write or multi-part write Start transaction that includes a valid PEC. The management controller shall cease using PEC in read transactions after it receives any single-part write or multi-part write Start transaction that does not include a PEC byte. (A management controller detects PEC by noting that it has received one more byte in the SMBus Write-Block transaction than was indicated by the length byte. If this occurs, it assumes that the additional byte was the PEC byte and then checks it for validity.)

2.6.24. Summary of SMBus Commands Values for Management Controller Messaging

The following table summarizes the allocation of SMBus commands for module management controller messaging. Note that there are command values that are reserved for future definition by the specification.

Table 2-21: Summary of SMBus Commands for Message Protocols

Operation SMBus CMD SMBus Protocol

Single-part Write 02h Write Block

Multi-part Write Start - first part 06h Write Block Middle part(s) if any 07h Write Block, first data byte after length = block number End - last part 08h Write Block, first data byte after length = block number

Single-part Read 03h Read Block, first data byte after length = Tag = 0x00

Multi-part Read Start - first part 03h Read Block, first data byte after length = Tag = 0x01 Middle part(s) if any 09h Read Block, first data byte after length = block number End - last part 0Ah Read Block, first data byte after length = block number Retry 0Bh Write Block, first data byte after length = block number

Get Message Interface Status

04h Read Block

Set Message Interface Control

05h Write Block

Reserved 0Ch-17h Reserved for future definition for module management functions

00h, 01h OEM

All other

Available for OEM use


58

2.6.25. SMBus Timeout and Hang Handling

The System Management Bus (SMBus) Specification, Version 2.0 provides an option for devices to drop off the bus and go back to waiting for the SMBus START condition if the SMBus clock is held low for greater than 25 ms. There is no requirement for management controllers to implement this option.

Per the System Management Bus (SMBus) Specification, Version 2.0, the management controller must synch up to a SMBus START or STOP condition, regardless of which SMBus clock the condition occurs on. In order for a master to place a START or STOP condition on the bus, there must be no other party driving the SMBus data line low during the SMBus clock. It is possible on SMBus that a missed clock or incorrectly terminated transfer could leave a slave device that is being read in the state where it is outputting a “0” data bit on the bus waiting for the master to continue to clock the bus for the next bit(s). In this condition, the bus is in a state where START and STOP conditions cannot be generated by the master because the slave is holding the data line low.

2.6.26. Management Controller Messaging Timing

The management controller must allow the bus master to resynchronize the bus by allowing the master to clock the management controller until the master can issue a START or STOP condition. This means that a management controller should “drop off the bus” and let its data and clock lines go high (not driven) if it gets clocked for returning more data than it has available.

Table 2-22 lists the recommended timing specifications on SMBus for a management controller implementing protocols for SMBus messaging. Note that this timing can be dependent on the clock rate at which the master drives the bus. Specifications are given for a SMBus driven at 100 kbps.

Table 2-22: SMBus Messaging Timing Specifications

Internal Timing Specifications Min Max Comments Overall Message Duration T1 - 20 ms

Time-out waiting for bus free T2 60 ms -

Time-out waiting for a response, internal

T3 60 ms[1] T6max[1]

Time between Event Message Requests

T4 5 ms -

Request-to-Response time T5 - T6max-T1max-3 ms[1]

This interval is measured from the end of the request transmission through the end of response transmission (SMBus STOP to SMBus STOP).

Number of Request retries C1 5[2] - Recommended

Time between Request retries T6 60 ms 250 ms

Number of Event Message Request retries

C2 3 10


59

Internal Timing Specifications Min Max Comments Overall Message Byte Duration T7 per the

SMBus spec.

3 ms Recommended

SM Bus Clock Low hold T8 per the SMBus spec.

3 ms Recommended The BMC should avoid stretching the clock more than 3 ms at a time. The BMC must tolerate the clock being stretched up to the maximum value specified by the System Management Bus (SMBus) Specification, Version 2.0.

Notes: 1. Unless otherwise specified, this timing applies to the mandatory and optional commands specified in the

Intelligent Platform Management Interface Specification. For controller-specific Application and Firmware commands, the Responder should attempt to meet this specification. In cases that it cannot, the interface specification for the Responder must clearly specify the “Request to Response” time that was implemented. Because timing can vary according to command and controller, communication routines should be designed to support response timeouts and retry counts accordingly.

2. This is a recommended value only. The protocol does not require that non-Event Message requests be retried. The implementation of retries and the number used is based on the application’s requirements for message delivery.

2.6.27. IPMI Management Controller Message Formats

The following tables show examples of IPMI messages carried as the message data using the Single-part Write and Single-part Read protocols. IPMI messages use message class 0x00. If multi-part transfers are required, the “IPMI CMD data” will span into the message data of the transactions following the Start transaction.

2.6.27.1. BMC to Module IPMI Message Format Examples

Figure 2-26 shows the example of a standard IPMI Request message delivered to a module using the Single-part Write protocol. This is typical of a transaction that a BMC would use to send a request to a module. The format for sending a response is similar, with the main difference of the inclusion of a Completion Code in the IPMI message data and the swapping of addressing fields between the request and response.1

Responder’s Slave

Address (7)

R/W=0 (1)

SMBus CMD = 02h

Length Message Class

(0x00 = IPMI)

NetFn [even] (6),

Responder’s LUN (2)

Requester’s Slave

Address

Requester’s Seq (6),

Requester’s LUN (2)

IPMI CMD

IPMI CMD data

[PEC]

Figure 2-26: BMC to Module IPMI Standard Request

1 Note that some OEM IPMI NetFn values specify different fields following the CMD byte, but the presence of a CMD field and all fields preceding the CMD byte are common for all IPMI messaging, regardless of the NetFn value.


60

Requester’s Slave

Address

R/W=0 (1)

SMBus CMD = 02h

Length

Message Class

(0x00 = IPMI)

NetFn [odd] (6),


Responder’s Slave

Address (7)



IPMI CMD

completion code

IPMI CMD data

[PEC]

Figure 2-27: BMC to Module IPMI Standard Response

Where:

Responder’s Slave Address = Slave address of the target management controller on the module.

SMBus CMD = Identifies the particular Write transaction operation to perform on the bus. This field is used to differentiate between operations at the transaction level. For a request, this field is 02h indicating a Single-part Write transaction.

Length = SMBus length for an SMBus Write Block transaction. This value represents the number of following data bytes less PEC. Per the System Management Bus (SMBus) Specification, Version 2.0, the LEN must be the third byte of an SMBus Block Write protocol transaction. The Bus Op 1 field occupies the position of the SMBus v2.0 CMD byte for a Block Write protocol transaction.

NetFn = Network Function Code. This 6-bit field identifies the type of message data that follows. Besides identifying IPMI message classes, it also provides values that support namespaces for messages from other defining bodies and OEMs. The IPMI convention is that even NetFn values are used for Requests and odd values for Reponses.

LUN = Logical Unit Number. This 2-bit field is auxiliary addressing information for IPMI. A management controller can accept messages up to four different logical units. Typically, a management controller implements just one LUN, 00b. For non-IPMI namespaces, this value can be defined by the respective OEM or defining body.

Seq = Sequence Number. This 6-bit field is used to match responses up with requests. This enables the responder the option of not immediately responding to a request and allows management controller messaging to be multi-threaded. The originator of the request (the Requester) picks the sequence number for the request; the responder returns that same sequence number in the response.

IPMI CMD = The command value for IPMI messages as defined in the IPMI specification.

IPMI data = The accompanying request data bytes for the given command, if any.

PEC = Packet Error Code. The PEC is an 8-bit CRC calculated per the System Management Bus (SMBus) Specification, Version 2.0. PEC is MANDATORY for IPMI messages to/from PCI Express Modules.


61

2.6.27.2. Module to BMC IPMI Message Format Examples

The following figures show the format of Single-part Read protocol accesses to retrieve IPMI requests and responses from a management controller on a module. In these examples, the Request and Response are being addressed to the BMC (address 0x20, 0010_000x). See Section 2.6.27.1 for field definitions.

Slave Address (7)

R/W = 0 (1)

SMBus CMD = 03h

Slave Address (7)

R/W=1 (1) Length

message class = 0x00 (IPMI)

Tag = 0x00

Responder’s Slave

Address (e.g. 0010_000)

0 NetFn [even] (6),


Requester’s Slave

Address

Requester’s Seq (6), Requester’s LUN (2)

IPMI CMD

IPMI CMD data

[PEC]

Figure 2-28: Module to BMC IPMI Standard Request

Slave Address (7)

R/W = 0 (1)

SMBus CMD = 03h

Slave Address (7)

R/W=1 (1) Length

message class = 0x00 (IPMI)

Tag = 0x00

Requester’s Slave

Address (e.g. 0010_000)

0 NetFn [odd] (6),


Responder’s Slave Address

(7)



IPMI CMD

completion code

IPMI CMD data

[PEC]

Figure 2-29: Module to BMC IPMI Standard Response

2.6.28. Pre-assigned/Reserved Slave Addresses

The following table lists the SMBus address assignments and ranges available for on-module devices. The address ranges are split into four ranges:

“b” for “board-set.” These are defined or reserved for definition by the board-set manufacturer.

“M” for “Module.” These are defined or reserved for use by the module manufacturer.

“c” for “chassis.” These are reserved for use by a chassis that a board-set goes into. A vendor that is a “system integrator” that provides both the board and chassis as a packaged product can combine the board and chassis ranges and use them for either purpose.

“-” for “reserved.” These are addresses that are reserved under this specification.

Unshaded rows indicate pre-assigned and reserved addresses. Shaded rows indicate addresses that are available for use by the designated party. This specification provides the addresses for an individual SMBus segment connected to a module slot. Note that the incorporation of multiplexing can provide isolation that enables overlapping address assignments on SMBus segments that are separate from the module SMBus segments. This specification does not cover the assignment of addresses to non-ExpressModule segments, except for the location of a baseboard VPD device that is required if a multiplexer is used for accessing modules via an SMBus host controller.


62

Table 2-23: Management I/O Addresses

Address Range Function/Device Number of Addresses

0000_000x 00h - Reserved (I2C/IPMI Broadcast) 1

0000_0001:0000_111x 01h:0Eh - Reserved (SMBus/I2C) 7

0001_000x 10h b SMBus Host 1

0001_001x 12h b Board OEM device 1

0001_010x 14h M Module OEM device 1

0001_011x 16h c Chassis OEM device 1

0001_100x 18h b SMBus ARA 1

0010_000x 20h b BMC 1

0010_001x:0010_011x 22h:26h b Board OEM devices 3

0010_100x:0010_101x 28h:2Ah M Module OEM devices 2

0010_110x:0010_111x 2Ch:2Eh c Chassis OEM devices 2





0100_011x 46h M Module OEM device 1

0100_100x 48 M Basic Management Status Register 1

0100_100x:0100_111x 4Ah - Reserved 1



0101_100x:0101:101x 58h:5Ah M Module OEM devices 2

0101_100x:0101:101x 5Ch:5Eh c Chassis OEM devices 2






1000_110x:1000_111x 8Ch:8Eh b Chassis OEM devices 2

1001_000x:1001:011x 90h:96h b Board OEM devices 4

1001_100x:1001:101x 98h:9Ah M Module OEM devices 2

1001_110x:1001:111x 9Ch:9Eh c Chassis OEM devices 2

1010_000x:1010_011x A0h:A6h M Module FRU/VPD Devices 4

1010_100x:1010_111x A8h:AEh b Board OEM devices 4

1100_001x C2h b,M,c SMBus 2.0 Device Default Address 1

1101_010x:1101_011x D0h:D2h b Board OEM devices 2


63

Address Range Function/Device Number of Addresses

1101_010x:1101_011x D4h:D6h M Module Satellite Management Controller 2

1101_100x:1101_101x D8h:DAh M Module OEM devices 2

1101_100x:1101_111x DCh:DEh c Chassis OEM devices 2

1110_000x:1110_011x E0h:E6h B Board OEM devices 4

1110_100x:1110_101x E8h:EAh M Module OEM devices 2

1110_110x:1110_111x ECh:EEh c Chassis OEM devices 2

1111_000x:1111:111x F0h:FEh - Reserved (SMBus/I2C) 8

2.7. Auxiliary Signal Parametric Specifications

2.7.1. DC Specifications

Table 2-24: Auxiliary Signal DC Specifications

Symbol Parameter Conditions Min Max Unit Notes

VIL1 Input Low Voltage -0.5 0.8 V 2, 7, 9

VIH1 Input High Voltage 2.0 3.6 V 2, 7

VIL2 Input Low Voltage -0.5 0.8 V 4

VIH2 Input High Voltage 2.1 Vaux + 0.5 V 4

VIH3 Input High Voltage Min voltage at 20 ma sink 2.7 3.4 V 9

VOL Output Low Voltage 3.0 mA 0.4 V 1, 3, 4, 6, 8

VHMAX Max High Voltage 3.6 V 3, 8

IIH Input High Current -20 0 mA 9

Iin Input Leakage Current 0 to 3.3 V -10 +10 µA 2, 4

Ilkg Output Leakage Current 0 to 3.3 V -50 +50 µA 3, 5, 6, 8

Cin Input Pin Capacitance 7 pF 2

Cout Output (I/O) Pin Capacitance 30 pF 3, 4, 6, 8

Notes: 1. These are open-drain outputs that require pull-ups on the system board. There is no VOH specification

for these signals. The voltage listed is the maximum voltage that can be applied to the pin. 2. Applies to MRST#.


64

3. Applies to WAKE. 4. Applies to SMBus signals SMBDATA and SMBCLK. 5. Leakage at the pin when the output is not active (high impedance). 6. Applies to SMBAlert. 7. Applies to PWREN#. 8. Applies to MPWRGD# and PWRFLT#. 9. Applies to ATTLED and PWRLED module inputs. These are input drive LEDs with series current limiting

resistors on the module.

2.7.2. AC Specifications

Table 2-25: Power Sequencing and Reset Signal Timings

Symbol Parameter Min Max Units Notes Figure

TPVPERL Power stable to MRST# inactive 100 ms 1 Figure 2-10

TMRST-CLK REFCLK stable before MRST# inactive 100 µs 2 Figure 2-10

TMRST MRST# active time 100 µs Figure 2-12

Tfail Power level invalid to MRST# active 500 ns 3 Figure 2-12

Twkrf WAKE rise – fall time 100 ns 4 Figure 2-30

TWAKE WAKE hold time from MRST# active 1 ms Figure 2-11

Notes: 1. Any supplied power is stable when it meets the requirements specified for that power supply.

2. A supplied reference clock is stable when it meets the requirements specified for the reference clock. The MRST# signal is asserted and deasserted asynchronously with respect to the supplied reference clock.

3. The MRST# signal must be asserted within Tfail of any supplied power going out of specification.

4. Measured from WAKE assertion/deassertion to valid input level at the system PM controller. Since WAKE is an open-source signal, the rise time is dependent on the total capacitance on the platform and the system board pull-up resistor. It is the responsibility of the system designer to meet the rise time specification.


65

A-0457

WAKE

VIH_PMC1

VIL_PMC1

WAKE

TWRKF

Note 1: Power Management Controller input switching levels are platform dependent and are not set by this specification.

Figure 2-30: WAKE Rise and Fall Time Measurement Points


66


67

3. Hot Insertion/Removal The following text references to mechanical elements should be interpreted in the context of the PCI Express ExpressModule form factor definitions.

3.1. Scope This ExpressModule form factor intrinsically supports Hot Insertion and Removal (hot-plug) of add-in modules. This modular form factor is physically different than that of a PCI card. This changes some aspects of the hot-plug usage model but, in general, it is based on the standard model defined in the PCI Standard Hot Plug Controller and Subsystem Specification, Rev. 1.0. The following sections describe the add-in module native hot-plug features. For a detailed explanation of the register requirements and standard usage model, see Section 7.7 of the PCI Express Base Specification, Revision. 1.1.

3.2. Hot-Plug Sub-System Architecture The module hot-plug model is very similar to that of the PCI card hot-plug model with most of the same steps required. The primary differences are mechanical along with a few electrical changes; that is, the module is an enclosed adapter with on-board power regulation. This changes the host adapter interface from the standard PCI and PCI Express cards model. That model requires a number of electrical switches to control power and interface signals on the host side of the interface to prevent damage to the card or host system during hot-plug operations.

This interface reduces the number of signals that must be controlled to that of the reference clock. This is accomplished by two basic changes to the adapter/host interface. The first is to provide bulk power to the module, thus eliminating the need for power control devices on the system side of the interface. This allows the module to better meet the voltage needs of the logic on the adapter over time. New signals are also added to the adapter interface: power enable (PWREN#) and module power good (MPWRGD#). The power enable signal (PWREN#) controls the bulk power DC to DC converter on the module. This signal is connected to the standard hot-plug controller’s power enable output signal on slots that fully support hot-plug and to ground on slots that do not fully support hot-plug. The adapter contact for PWREN# is shorter than the power and ground contacts thus ensuring that the module’s DC to DC converters cannot be activated until the module has a good connection in the slot connector. The shorter contact also ensures that power is disabled before the module is fully removed from the slot connector. This is to prevent arcing during plugging or unplugging. The module power good signal (MPWRGD#) indicates to the system that the modules onboard power converter is stable.

3


68

The second basic change to the adapter/host interface is that the rest of the active interfaces are driven by open collector/open drain or tri-state drivers. Some of these signals are terminated on the adapter from Aux power. This protects the adapter during the hot-plug event and eliminates the need for switching these signals on the host side of the interface.

This leaves only the reference clock (REFCLK) signals that must be disabled during a hot-plug event, which is enabled by MPWRGD# being active (logic 0) and the PRSNT# signal being active. This forces the reference clock signals inactive during a hot-plug or removal event and eliminates any EMI that could be produced by driving these signals into an empty slot connector.

Note: Modules and slots designed to this specification will not be physically damaged by a hot insertion or removal event. However, slots that do not implement hot-plug controllers and drivers may experience unpredictable behavior during hot insertion or removal of module.

All modules are required to support the hot-plug architecture. System slots, however, are encouraged to support the full hot-plug architecture but are only required to support the features that prevent damage to the adapter or slot, i.e., gating of the reference clock.

Doublewide modules have the possibility to connect to two backplane connectors. The primary connector for a doublewide module is the one next to the module carrier or frame and the one closest to the cover of the module is the secondary interface connector. Doublewide modules with only a single interface connection are allowed to use either the primary or secondary connector in the module. There are no differences in the interface from a singlewide module. Doublewide modules that connect to both slot connectors have special interface requirements related to the secondary connector. The secondary connector in a doublewide module may only be used to support additional power (maximum of 50 watts) and or the additional x8 PCI Express lanes to form a x16 interface for the module. If power is used from the secondary connector, it must be controlled by the primary connectors PWREN# signal. The following interface signals are no connects for the secondary connector: ATNLED, PWRLED, ATNSW, WAKE, PRSNT#, PWREN#, PWRFLT#, MPWRGD#, MRST#, REFCLK, SMBCLK, SMBDAT, and SMBAlert.

Figure 3-1 shows the elements of the PCI Express ExpressModule hot-plug interface.


69

A-0458

PCI ExpressHot-PlugController

HostInterface

PCIExpressBridge

SystemManagementBus Controller

SystemManagement

Interface

Module Adapter InterfaceSystem Slot Interface

Aux Power

PCI ExpressI/O Applications

SMBus

SMBAlert

Delay

Bulk PowerConverter

Electro-mechanicalLock

AND

Short Pin

Short Pin

Short Pin

Long Pin

PCI Express Link

PCI Express Ref Clock

WAKE#

PERST#

EMIL

S

EMIL

ATN

SW#

PWR

LED

ATN

LED

PWRLED

ATNLED

AUX PWR

Bulk PWR

PWREN#

PRSNT#

PWRFLT#

MPWRGD#

MRST#

WAKE

Mod

ule

Bac

kpla

ne C

onne

ctor

Short Pin

Figure 3-1: Typical Hot-Plug Interface Implementation

As the module engages with the slot connector, most of the interface pins make contact at the same time. The exceptions are the presence detection (PRSNT#), power enable (PWREN#), power fault (PWRFLT#), module power good (MPWRGD#), Vaux (Vaux), and wake (WAKE) pins. These pins are shorter than the other pin on the module connector such that they make contact last during insertion and break contact first during removal.

Some of the signals shown in Figure 3-1 are described in the PCI Express Base Specification, Revision 1.1 and the PCI Express Card Electromechanical Specification, Revision 1.1, but are listed in the following tables for clarity. This is important because some have different polarity than what is identified in the other specifications.


70

Table 3-1: Power Signals

Signal Name I/O Definition as Viewed From the Host Side PWREN# O Power Enable: Required

A slot Control Logic output that controls the power state of the slot. If this signal is low, power is enabled to the slot.

PWRFLT# I Power Fault: Required This pin function is Module Power fault, indicating that the module’s power converter has a failure. The Module driver is open drain. See the PCI Standard Hot Plug Controller and Subsystem Specification for more details.

WAKE

I Adapter Power Request: Required Requests the system to return to a full power state. Adapters that do not support this function must connect to ground. See the PCI Express Base Specification, Revision 1.1, for more details.

MPWRGD# I Module Power Good: Required Indicates to the system that all module power supplies are good. The output of the on module DC to DC converter is stable. Module driver is open drain.

MRST# O Module Reset: Required This system output provides two functions to the adapter module. First, that power and REFCLK are stable (within specification) during power up. Second, as an early warning that a power down is imminent. Note when the system’s primary power is off and Aux power on, this output must be driven to a logic 0 (less than 0.4 V DC).

Table 3-2: Management Interface

Signal Name I/O Definition as Viewed From the Host Side PRSNT#

I Present Detect Input: Required Indicates that a module is in the slot. Used to detect insertion or removal of a module as well as part of the reference clock enabling function.

SMBus I/O Systems Management Bus: Required A two line interface compatible with most I2C components. See the System Management Bus (SMBus) Specification, Version 2.0, for a detailed description. Note: These inputs must have a weak pull-down on the host side to condition signals during hot-plug event.


71

Signal Name I/O Definition as Viewed From the Host Side SMBAlert I Systems Management Bus Alert: Optional for module only

Input to the system management subsystem that an adapter needs attention. If this function is not implemented on the module, it must be connected to ground. Note: This input must have a weak pull-down on the host side to condition signals during hot-plug event.

Table 3-3: User Interface

Signal Name I/O Definition as Viewed From the Host Side PWRLED O Power LED: Required

A slot control logic output that is used to drive the Power Indicator. This output is asserted to illuminate the Indicator. See the PCI Express Base Specification, Revision 1.1, for more details. Note: This interface signal sources the LED current for the ExpressModule; this is to minimize any dependencies on the module. See Table 2-24 for signal voltage and current.

ATNLED O Attention LED: Required A slot control logic output that is used to drive the Attention Indicator. This output is asserted to illuminate the Indicator. See the PCI Express Base Specification, Revision 1.1, for more details. Note: This interface signal sources the LED current for the ExpressModule; this is to minimize any dependences on the module. See Table 2-24 for signal voltage and current.

MRL# I Manual Retention Latch Sensor: Not supported The MRL is not supported for ExpressModule slots.

EMIL O Electromechanical Interlock: Optional The ExpressModule is designed to support an optional locking mechanism that is controlled by a bit in the hot-plug controller. See the PCI Express Base Specification, Revision 1.1, for more details on programming. This output is pulsed when a logic “1” is written to the control bit. The function of this pulse is to set a latching interlock mechanism. Each output pulse is 100 ms long.

EMILS I Electromechanical Interlock State: Optional This input is required if an electromechanical Lock is implemented. This input is used to determine the current state of electromechanical lock that holds a module in the chassis. See the PCI Express Base Specification, Revision 1.1, for more details.


72

Signal Name I/O Definition as Viewed From the Host Side ATNSW# I Attention Switch: Required

A low true (low logic level when the switch is pressed) input that is connected directly to the Attention switch. If the platform does not support Attention switch, this input must be wired to a high logic level. See the PCI Express Base Specification, Revision 1.1, for more details.

3.2.1. Power Enable

The power enable (PWREN#) input is used to control the adapter module’s primary power and associated supplies. In slots that do not support hot-plug, this interface signal is connected to logic ground. In slots that support hot-plug, the hot-plug controller power enable output is connected to this signal to manage the adapter’s primary power. The PWREN# signal pad on the add-in module connector is shorter than most of the pads on the connector. This is to ensure that it is one of the last signals to make contact in the connector and the first to break contact.

Note: Slots that do not support hot-plug controllers must enable power by grounding the PWREN# signal to the card slot and gate REFCLK to the module with the MPWRGD# and PRSNT# signals. See the PCI Express Base Specification, Revision 1.1, for more details on the REFCLK function.

3.2.2. Power Fault

The power fault signal (PWRFLT#) is an open drain output from the module and provides an indicator that the module has a power fault; i.e., module power supplies cannot maintain power regulation. This signal is intended to warn the Hot-Plug Controller of power problems on the module. The power fault signal is a negative active signal in the ExpressModule interface to minimize the requirements of the module power supplies and hot-plug support. The slot input has a strong pull-up to prevent generating a false power fault during a hot-plug event or when a module is not present.

Once the power fault condition has occurred, the PWRFLT# signal is latched by the adapter. It will remain latched in the fault state until the PWREN# signal is toggled, after which it is reset.

The PWRFLT# signal pad on the add-in module connector is shorter than most of the pads on the connector. This is to ensure that it is one of the last signals to make contact in the connector and the first to break contact. See the PCI Standard Hot Plug Controller and Subsystems Specification, Revision 1.0, for more details on the power fault function.

3.2.3. Wake

The wake (WAKE) signal is an output from the module and provides an indicator that the module request that the system power on. The wake connector pad on the module is short to give the module more time to ensure that WAKE event is not falsely triggered during a hot-plug event. The Wake signal is a positive signal in the ExpressModule interface to minimize the requirements of hot-plug support. The slot input has a weak pull-down to prevent generating a wake event when a


73

module is not present. See the PCI Express Base Specification, Revision 1.1, or Section 3.2 for more details on the function of the Wake signal.

3.2.4. Module Power Good

The module power good (MPWRGD#) signal is an open drain output from the module that indicates the on module power supplies are within operational limits for the application. This signal is used to generate the MRST# signal to reset the module application when supplies are not within operational limits. The MPWRGD# signal pad on the add-in module connector is shorter than most of the pads on the connector. This is to ensure that it is one of the last signals to make contact in the connector and the first to break contact. The slot input requires a strong pull to prevent a false indication during hot-plug events.

3.2.5. Module Reset

The Module Reset (MRST#) signal is an input to the module and indicates that the system power supply is stable and the on-module power converters are stable. This signal is made from system reset and a delayed MPWRGD#. The system reset goes active as soon as system power is unstable driving MRST# active. The delayed MPWRGD# provides a delay after module power is stable to allow the reference clock to be enabled and stable before MRST# goes inactive. See Chapter 2 for more details on the function of the Module Reset signal.

3.2.6. Present Detection

The PCI Express hot-plug controller detects the presence of an add-in module using the presence detection signal (PRSNT# ). It is the responsibility of the Root Complex or the Switch to determine the presence of the add-in module and set the present bits in the appropriate register as described in PCI Express Base Specification, Revision 1.1. The presence detection signal is required on all PCI Express ExpressModule connectors and add-in modules.

Figure 6-4 shows the short pins on the card edge connector. These pins are used as the presence detection mechanism. The add-in module must be fully inserted in to the connector to ensure that the module is detected by the system. The PRSNT# signal pad on the add-in module is shorter than the rest of the pads on the connector. This is to ensure that it is one of the last signals to make contact in the connector and the first to break contact. This provides the system with a valid indicator that the module is installed in the slot. See the PCI Express Base Specification, Revision 1.1, for more details on the function of the presence detection signal.

3.2.7. System Management Bus

See Section 2.5 and the PCI Express Base Specification, Revision 1.1, for details on the function of the System Management Bus.


74

3.2.8. System Management Bus Alert

See Section 2.6.6 for details on the System Management Bus Alert.

3.2.9. Power LED

The Power LED (PWRLED) signal is an output from the PCI Express port’s hot-plug controller. This signal controls the Power LED. Each module is required to support the Power LED. The slot Power LED signal provides the voltage and current to light the Power LED (see Table 2-24). The module provides the LED and current limiting resistor. The Power LED is defined in this fashion to increase its reliability by reducing its dependence on the module. See Figure 3-1 for an example of how this function is wired. See the PCI Express Base Specification, Revision 1.1, for more details on the Power LED’s function.

3.2.10. Attention LED

The Attention LED (ATNLED) signal is an output from the PCI Express port’s hot-plug controller. This signal controls the Attention LED. Each module is required to support the Attention LED. The slot Attention LED signal provides the voltage and current to light the Attention LED (see Table 2-24). The module provides the LED and current limiting resistor. The Attention LED is defined in this fashion to increase its reliability by reducing its dependence on the module. The chassis can optionally have an Attention LED per slot to identify an empty slot. When a slot has a module installed (PRSNT# active), the chassis Attention LED must be disabled. See Figure 3-1 for an example of how this function is wired. See the PCI Express Base Specification, Revision 1.1, for more details on the Attention LED’s function.

3.2.11. Manual Retention Latch

The Manual Retention Latch (MRL) is not supported for ExpressModule slots. See the PCI Express Base Specification, Revision 1.1, for more details on the MRL’s function.

3.2.12. Electromechanical Interlock

The electromechanical interlock (EMIL) signal is an optional output from the PCI Express port’s hot-plug controller. All slots supporting the EMIL function shall implement the EMIL as defined by this specification and the PCI Express Base Specification, Revision 1.1. This signal controls an electromechanical interlock for a single slot in the chassis. Silicon supporting this function shall drive this signal when activated for a pulse length greater than 100 ms and less than 150 ms duration. This signal causes the locking mechanism to change position (locked/unlocked). The EMIL mechanism must change states (locked/unlocked) in less than 500 ms from EMIL signal active. Software that controls this function shall provide a minimum of 1 second between pulses of the EMIL.

The optional electromechanical interlock feature provides physical security for the module while also protecting the service personnel from accidentally removing the wrong module. The use of the


75

electromechanical interlock on doublewide module requires that the secondary slot control the electromechanical interlock. Note that the SMBus VPD may be the only means to determine that the module is doublewide. Systems that implement the electromechanical interlock must also support SMBus management. See the PCI Express Base Specification, Revision 1.1, for more details on the EMIL signal’s function.

3.2.13. Electromechanical Interlock Status

The electromechanical interlock status (EMLS) signal is an input to the PCI Express port’s hot-plug controller. It is required only if the electromechanical interlock is implemented. The function of this signal is to provide the current state of the electromechanical interlock (locked/unlocked). The electromechanical interlock sensor status is read from the slot control register and has the definitions listed in Table 3-4.

Table 3-4: EMLS Signal

Value Status 0b Disengaged

1b Engaged See the PCI Express Base Specification, Revision 1.1, for more details on the EMLS signal’s function.

3.2.14. Attention Switch

The Attention switch is a required feature of all modules. The system is responsible for de-bouncing of the Attention switch signal if it is desired. See the PCI Express Base Specification, Revision 1.1, for more details on the Attention switch’s function.


76


77

4. Module Power Interface

4.1. Power The I/O adapter module is supplied two power sources, one for Bulk or Primary power and one for Aux power.

All of the adapter module’s primary power needs are supplied from the Bulk power supply.

The Aux supply provides limited power to a slot for management interface and wake features.

The power delivery requirements defined in this section define the environment that the ExpressModule will be required to operate within.

4.1.1. Module Primary Power Supply

The PCI Express modules defined in this specification require a primary supply of +12V. Systems that provide slots for ExpressModules are required to provide +12V supply to each slot connector with sufficient power and regulation to meet minimum requirements. See Table 4-1 for power level and tolerances.

4.1.2. Module Auxiliary Power Supply

The PCI Express modules defined in this specification require an auxiliary supply of +3.3V. Systems that provide slots for ExpressModules are required to provide +3.3Vaux supply to each slot connector. See Table 4-1 for power level and tolerances.

4


78

4.2. ExpressModule Power Supply Requirements Table 4-1 identifies the power supply requirements for each module type.

Table 4-1: Power Supply Rail Requirements

Power Rail Singlewide Doublewide +12V Bulk Voltage Tolerance Continuous Current Initial Hot-plug Capacitance Input Capacitance

±15% (max) 2.08 A (max) 5000 pF (max) 500 µF (max)

±15% (max) 4.17 A (max) 5000 pF (max) 500 µF (max)

+3.3V Aux Voltage Tolerance Continuous Current Peak Pre-charge Current Input Capacitance Pre-charge Pin Timing

±10% (max) 475 mA (max) 475 mA (peak) 150 µF (max) 3 ms (max)

±10% (max) 950 mA (max) 950 mA (peak) 300 µF (max) 3 ms (max)

Notes: 1. The +12V power hot swap circuits shall be located on the module.

2. Currents during hot insertion shall not exceed the module maximum continuous current.

3. The module and connector shall not be damaged during hot removal or insertion.

4. The +3.3Vaux power shall pre-charge the module’s 3.3Vaux input capacitors during hot insertion via mate first pre-charge pin.

5. Peak pre-charge current during hot insertion is determined by the value of the pre-charge resistor. Single wide example: Pre-charge resistor = +3.3V/475 mA = 7 Ω.

6. +3.3Vaux Pre-charge Pin Timing is the maximum time guaranteed during hot insertion from the +3.3Vaux pre-charge pin mating to the main power pins mating. The time constant with the maximum input capacitance and pre-charge resistor shall not exceed 1/3 of the Pre-charge Pin Timing. Example: 150 µF x 7 Ω = 1 ms (which is 1/3 of the maximum pre-charge pin timing of 3 ms).

7. The maximum current slew rate for each add-in module shall be no more than 0.1 A/µs.

8. Each add-in module shall limit its capacitance on each power rail at the backplane connector to that listed in the above table.

9. Continuous current = the highest averaged current value over any 1-second period.


79

4.3. Power Consumption There are two power configurations defined for the modules:

Singlewide Module = Up to 25 watts

Doublewide Module = Up to 50 watts

This breaks down from a connector perspective to the following:

The singlewide module connector supports the doublewide maximum power (50 watts) to allow a doublewide module to draw all its power from a single connector.

The doublewide module can use both connectors available for power, but the maximum power that can be drawn is 50 watts. This allows a doublewide module to draw its power from either connector or both to the maximum of 50 watts.

The power drawn from the connector has no relation to the width of the PCI Express Link supported in the connector; that is, a doublewide module that only uses a x4 Link can draw up to 50 watts from a single connector.

Systems that do not support the maximum power on all slots must support active power management.

4.4. Power Supply Sequencing The Aux power supply must be applied to the module before or at the same time as the primary power. The module slot connector has both Aux and Primary power applied when the system is power on. During hot-plug events, the system does not remove power from the module slot connector. The system must drive the module reset (MRST#) signal active (logic 0) any time the Primary or Aux power goes out of tolerance.

4.5. Power Supply Decoupling Due to the low level signaling of the PCI Express interface, it is strongly recommended that sufficient decoupling of all power supplies be provided. This is recommended to ensure that power supply noise does not interfere with the recovery of data from a remote upstream PCI Express device. Some basic guidelines to help ensure a quiet power supply are provided below.

Note: The following are guidelines only. It is the responsibility of the ExpressModule designer to properly test the design to ensure that ExpressModule circuitry does not create excessive noise on power supply or ground signals at the ExpressModule edge fingers.

The add-in adapter module device decouple value should average 0.01 µF per device Vcc pin (for all devices on the add-in card).

The trace length between a decoupling capacitor and the power supply or ground via should be less then 0.2 inches (5.08 mm) and be a minimum of 0.02 inches (0.508 mm) in width.


80


81

5. Module PCI Express Interface

5.1. PCI Express Link Signals See the PCI Express Base Specification, Revision 1.1, for details about Link signals.

The singlewide ExpressModule supports Link widths of x1, x2, x4, and x8.

The doublewide ExpressModule supports Link widths of x1, x2, x4, x8, and x16.

5.2. PCI Express Electrical Topologies and Link Definitions

This section describes the electrical characteristics of PCI Express add-in ExpressModule. The electrical characteristic at the adapter module interface is defined in terms of electrical budgets. This budget allocation decouples the electrical specification for system designer and the adapter module vendor and ensures successful communication between the PCI Express signal inputs and outputs Links on the system and adapter module interface. Unless otherwise noted, the specifications contained herein apply to all high-speed signals of each interface width definitions. The signaling rate for encoded data is 2.5 G transfers/s and the signaling is point-to-point.

5.2.1. Topologies

There are four different electrical topologies for PCI Express:

PCI Express devices on the same system board

PCI Express devices across one connector, on systems with a system board and adapter card

PCI Express devices across two connectors, on systems with a system board, riser card, and adapter card

PCI Express devices across three connectors, on systems with a cable connecting to a secandary backplane with an adapter card plugged into it

The first topology is defined for devices on the same board and not related to add-in adapter modules. Refer to the PCI Express Base Specification, Revision 1.1, for implementation of this topology.

The second topology is defined for devices across one connector. This allows a plug-in adapter module to be plugged into a backplane that has a PCI Express bridge on it or directly onto a system board.

5


82

The third topology is defined for devices across two connectors. This allows plug-in adapter modules to be plugged into a backplane (sometimes referred to as a riser card) and the backplane to be plugged into the system board. The system board to backplane connector is not defined by this specification but is required to meet the topology budget defined in this specification.

The fourth topology is defined for devices across three connectors. This allows plug-in adapter modules to be plugged into a backplane and the backplane to be connected to the system board via a cable. The system board to backplane cable is not defined by this specification but is required to meet the topology budget defined in this specification.

5.2.2. Link Definitions

Typical ExpressModule PCI Express Links consist of the following:

Transmitters/receivers on an ASIC on a system board

Package fan-in-out trace topologies

PCB coupled microstrip and/or striplines

Vias for layer changes

Optional system board to riser card connector and interface

Optional riser card with microstrip and/or stripline trace

Standard baseline riser card connector and card interface

Coupled microstrip line and/or stripline traces on the ExpressModule

Decoupling capacitors

Transmitter/receivers on an ASIC on the ExpressModule

The electrical parameters for the Link are subdivided into three components:

Adapter plug-in card (ExpressModule)

Riser card/backplane connector (ExpressModule connector)

System board (and riser card with associated connector if exists)

The electrical impact of discontinuities on the Link, such as via, bend, and test points, should be included in the respective components.


83

5.3. PCI Express Electrical Budgets A budget is defined for each of the following electrical parameters associated with the Link:

AC coupling capacitors

Insertion Loss (Voltage Transfer Function)

Jitter

Lane-to-lane skew

Crosstalk

Equalization

Skew within a differential pair

The electrical budgets are different for each of the two Link components:

ExpressModule budget

System board and PCI Express connector budgets

The interconnect Link budget allocations associated with the Transmitters and Receivers differ. This is to account for any electrical characteristics the AC coupling capacitors may contribute to the Link.

5.3.1. AC Coupling Capacitors

The PCI Express ExpressModule and system board shall incorporate AC coupling capacitors on the Transmitter differential pair. This is to ensure blocking of the DC path between the PCI Express ExpressModule and the system board. The suggested maximum size is a 603-type capacitor with a value as specified in the PCI Express Base Specification, Revision 1.1. Any additional attenuation or jitter caused by the coupling capacitors (larger than 603-type) must be accounted for as part of the budget allocation for the physical interconnect component’s path on which the capacitors are mounted. The electrical budgets allocated for the AC coupling capacitors are defined in the following subsections. The allocated budget includes the electrical parasitic effects associated with the component’s placement as mounted on the printed circuit board.

5.3.2. Insertion Loss Values (Voltage Transfer Function)

The maximum loss values in decibels (dB) are specified for the system board and the add-in card. The insertion loss values are defined as the ratio of the voltage at the ASIC package pin (Transmitter/Receiver) and the voltage at the PCI Express connector interface, terminated by 100 Ω differential termination, realized as two 50 Ω resistances. These resistances are referenced to ground at the interface (see Figure 5-1).


84

A-0459

Add-in Card

System Board Measurement reference pointat the top of the add-in card edgefingers which have been matedwith the connector

Transmitterand Package

Transmitterand Package

Add-in CardInterconnect

System BoardInterconnectAC Coupling

Capacitor

R = 50 Ω R = 50 Ω

R = 50 Ω R = 50 Ω

Measurement reference pointat the top of the add-in card edge

finger pads. (The edge finger padsare considered part of the connector/

system board interconnect.)P

CI E

xpress Connector

Figure 5-1: Example Interconnect Terminated at the Connector Interface

All PCI Express differential trace pairs are required to be referenced to the ground plane. The loss values associated with any riser card interface and adjoining connector implementation must collectively meet the system board loss budget allocations and associated eye diagrams.


85

Add-in CardSystem Board

PC

I Express C

onnectorA-0460

LST LAR

LSR LAT

Figure 5-2: Insertion Loss Budgets

Table 5-1: Allocation of Interconnect Path Insertion Loss Budget

Loss Parameter Loss Budget Value at 1.25 GHz (dB)

Loss Budget Value at 625 MHz (dB)

Comments

PCI Express ExpressModule

LAR < 2.65 LAT < 3.84 LAR < 1.95 LAT < 2.94 Note 1, 2

System Board and Connector

LST < 9.30 LSR < 8.11 LST < 6.00 LSR < 5.01 Notes 1, 3

Guard Band 1.25 1.25 Note 1 Total Loss LT < 13.2 LT < 9.2

Notes: 1. All values are referenced to 100 Ω, realized as two 50 Ω resistance to ground. The loss

budget values include all possible crosstalk impacts (near-end and far-end) and potential mismatch of the actual interconnect with respect to the 100 Ω reference load. The PCI Express Base Specification. Revision 1.1, allows an interconnect loss of 13.2 dB for 1.25 GHz (non de-emphasized) signals and 9.2 dB for 625 MHz (de-emphasized) signals. From this, a total of 1.25 dB is held in reserve as guard band to allow for any additional attenuation that might occur when the ExpressModule and system board are actually mated. The allocated loss budget values in the table directly correlate to the eye diagram voltages in Section 5.4. Tradeoffs in terms of attenuation, crosstalk, and mismatch can be made within the budget allocations specified.

2. As a guide for design and simulation, the following derivation of the budgets may be assumed for 1.25 GHz signals: 5.2 dB is subtracted from 13.2 dB to account for near-end crosstalk and impedance mismatches. Out of this, the 1.25 dB is reserved as guard band. The following loss allocations are then assumed per differential pair: LAR = 1.4 dB; LAT = 1.8 dB; LSR = 6.2 dB; LST = 6.6 dB. These allocation assumptions must also include any effects of far-end crosstalk. 625 MHz values may be derived in a similar manner. The ExpressModule budget does not include the ExpressModule edge finger or connector. However, it does include potential AC coupling capacitor attenuation on the Transmitter (TX) interconnect on add-in card. Note that the budget allocations generally allow for a maximum of 4-inch trace lengths for differential pairs having an approximate 5-mil trace width. No specific trace geometry, however, is explicitly defined in this specification. The subscripts of the Symbol designators, T and R, represent the Transmitter and Receiver respectively.


86

3. The system board budget includes the PCI Express connector and assumes it is mated with the card edge finger. Refer to Section 5.3 for specifics on the stand alone connector budget. The system board budget includes potential AC coupling capacitor attenuation on the Transmitter (TX) interconnect on the system board. The subscripts of the symbol designators, T and R, represent the Transmitter and Receiver respectively.

Note: The insertion loss budget distributions above are used to derive the eye diagram heights as described in Section 5.4. However, they are provided here only as a design guideline. Compliance measurements must actually be verified against the eye diagrams themselves as defined in Section 5.4.1.

5.3.3. Jitter Values

The maximum jitter values in terms of percentage of Unit Interval (UI = 400 ps for 2.5 G transfers/s) are specified for the system board and the add-in card. The jitter associated with the riser card and associated proprietary connector will be part of the system board jitter budget. The jitter values are defined with respect to 100 Ω differential termination, realized as two 50 Ω resistances. These resistances are referenced to ground at the interface (see Figure 5-1).

Add-in CardSystem Board

PC

I Express C

onnector

JST JAR

JSR JAT

A-0461 Figure 5-3: Jitter Budget

The total system jitter budget is derived with the assumption of a minimum Rj for each of the 4 budget items. This minimum Rj component is used to determine the overall system budget. The probability distribution of the Rj component is at the Bit Error Rate (BER) indicated and is Gaussian.

For any jitter distribution, the total Tj must always be met at the BER. Tradeoffs of Rj and Dj are allowed, provided the total Tj is always met. More information on the calculation of the system budget can be found in PCI Express Jitter and BER.


87

Table 5-2: Total System Jitter Budget

Jitter Contribution Min Rj (ps) One Sigma Max Dj (ps) P-P Tj at BER 10-12 (ps) Tj at BER 10-6 (ps)

Tx 2.8 60.6 100 87

Ref Clock 4.7 41.9 108 86

Media 0 90 90 90

Rx 2.8 120.6 160 147

Linear Total Tj: 458 410

Root Sum Square (RSS) Total Tj: 399.13 371.52

Notes:

1. RSS equation for BER 10-12 Tj = n

Dj∑ + 14.069 * 2

nRj∑

2. RSS equation for BER 10-6 Tj = nDj∑ + 9.507 * 2∑ nRj

Table 5-3: Allocation of Interconnect Jitter Budget

Jitter Parameter Peak–Peak Jitter Budget Value (UI) Comments PCI Express ExpressModule

JAR < 0.0575 JAT < 0.0650 Notes 1, 2

System Board and Connector

JST < 0.1675 JSR < 0.1600 Notes 1, 3

Total Jitter JT < 0.225 Note 1

Notes: 1. All values are referenced to 100 Ω, realized as two 50 Ω resistances to ground.

The jitter budget values include all possible crosstalk impacts (near-end and far-end) and potential mismatch of the actual interconnect with respect to the 100 Ω reference load. The PCI Express Base Specification, Revision 1.1 allows an interconnect jitter budget of 0.225 UI (equivalent to 90ps for a 400 ps Unit-Interval). The allocated jitter budget values in the Table 5-2 and Table 5-3 directly correlate to the eye diagram widths in Section 5.4. Tradeoffs in terms of attenuation, crosstalk, and mismatch can be made within the budget allocations specified. No additional guard band is specifically allocated. The jitter allocations are then assumed per differential pair according to the table. These allocation assumptions must also include any effects of far-end crosstalk.

2. All values are referenced to 100 Ω. The add-in card budget does not include the add-in card edge finger or connector. However, it does include potential jitter from the AC coupling capacitors on the Transmitter (TX) interconnect of the add-in card. The budget allocations generally allow for a maximum of 4-inch trace lengths for differential pairs having an approximate 5-mil trace width. No specific trace geometry, however, is explicitly defined in this specification. The subscripts of the symbol designators, T and R, represent the Transmitter and Receiver respectively.


88

3. All values are referenced to 100 Ω. The system board budget includes the PCI Express connector and assumes it is mated with the card edge finger. Refer to Section 6.4 for specifics on the stand alone connector budget. The system board budget includes potential jitter from the AC coupling capacitors on the Transmitter (TX) interconnect on the system board. The subscripts of the symbol designators, T and R, represent the Transmitter and Receiver respectively.

Note: The jitter budget distributions above are used to derive the eye diagram widths as described in Chapter 4. However, they are provided here only as a design guideline. Compliance measurements must actually be verified against the eye diagrams themselves as defined in Section 5.4.

5.3.4. Crosstalk

All ExpressModule designs must properly account for any crosstalk that may exist among the various pairs of differential signals. Crosstalk may be either near-end (NEXT) or far-end (FEXT). Each component can have potential impact on a design and must be planned for accordingly.

Note that the total maximum crosstalk that a Receiver component in Electrical Idle is required to tolerate is less than the minimum value dictated by the Electrical Idle Detect Threshold (VRX-IDLE-DET-DIFFp-p) in Table 4-6 of the PCI Express Base Specification, Revision 1.1. Additionally, crosstalk between differential pairs on the ExpressModule will influence and impact the data signals and any subsequent loss and jitter budgets as noted in Sections 5.3.2 and 5.3.3. Note that all eye diagrams in Section 5.4 must account for any and all crosstalk present. In order to limit crosstalk impacts and implications, it is recommended that the ExpressModule limit the total amount of NEXT to a maximum of 50 mV.

All system boards interfacing with an ExpressModule must also properly account for crosstalk. The system board must also account for potential crosstalk that can occur on the printed circuit board as well as within the connector itself (see Section 6.4).

5.3.5. Lane-to-Lane Skew

The skew at any point is measured using zero crossings of differential voltage of the compliance pattern, while simultaneously transmitting on all physical lanes. The compliance pattern is defined in the PCI Express Base Specification, Revision 1.1.


89

Table 5-4: Allowable Interconnect Lane-to-Lane Skew

Skew Parameter Symbol Skew Values Comments Total Interconnect Skew

TS 1.6 ns This does not include Transmitter output skew, LTX-SKEW (specified in the PCI Express Base Specification, Revision 1.1). The total skew at the Receiver (ST + LTX-SKEW ) smaller than LRX-SKEW (specified in the PCI Express Base Specification, Revision 1.1) to minimize latency for this ExpressModule topology.

PCI Express Add-in Card

AS 0.35 ns Estimates about a 2-inch trace length delta on FR4 boards

System Board sS 1.25 ns Estimates about a 7-inch trace

length delta on FR4 boards

5.3.6. Equalization

To reduce ISI, 3.5 dB (+/-0.5 dB) below the first bit de-emphasis in the Transmitter is required for the ExpressModule and the system board. For implementation details, refer to Chapter 4 in the PCI Express Base Specification, Revision 1.1.

5.3.7. Skew Within the Differential Pair

The skew within the differential pair gives rise to a common-mode signal component, which can, in turn, increase EMI. The differential pair should be routed such that the skew within differential pairs is less than 5 mils for the ExpressModule and 10 mils for the system board.

5.4. Eye Diagrams at the ExpressModule Interface The eye diagrams defined in this section represent the compliance eye diagrams that must be met for both the ExpressModule and a system board interfacing with such an add-in card. The specific measurement requirements (probe test points, calibrated system board specifics, etc.) for compliance of physical components are to be specified in a separate PHY Electrical Test Considerations for PCI Express Architecture document. A BER of 10-6 is assumed for the eye diagram measurements. These measurements are to be made using the clock recovery function as defined in the PCI Express Base Specification, Revision 1.1. These compliance eye diagram with BER of 10-12 can also be used for simulation by following the guidelines explained in Section 5.3.


90

5.4.1. ExpressModule Transmitter Path Compliance Eye Diagram

The eye diagram for the add-in card’s Transmitter path compliance is defined in Table 5-5 and Figure 5-4.

Table 5-5: ExpressModule Transmitter Path Compliance Eye Requirements

Parameter Value Comments VTXA

VTXA_d

>= 514 mV

>= 360 mV

Notes 1, 2, 5 Notes 1, 2, 5

TTXA JTXA-MEDIAN-to-MAX-JITTER

>= 287 ps

<= 56.5 ps

Notes 1, 3, 5 Notes 1, 4, 5

Notes: 1. An ideal reference clock without jitter is assumed for this specification. All Links are

assumed active while generating this eye diagram.

2. Transition and non-transition bits must be distinguished in order to measure compliance against the de-emphasized voltage level (VTXA_d). VTXA and VTXA_d are minimum differential peak-peak output voltages.

3. TTXA is the minimum eye width. The sample size for this measurement is 106 UI. This value can be reduced to 274 ps for simulation purpose at BER 10-12.

4. JTXA-MEDIAN-to-MAX-JITTER is the maximum median-to-max jitter outlier as defined in the PCI Express Base Specification, Revision 1.1. The sample size for this measurement is 106 UI. This value can be increased to 63 ps for simulation purpose at BER 10-12.

5. The values in Table 5-5 are referenced to an ideal 100 Ω differential load at the end of the interconnect path at the edge-finger boundary on the add-in card (see Figure 5-1). The eye diagram is defined and centered with respect to the jitter median. Exact conditions required for verifying compliance while generating this eye diagram are given in the PHY Electrical Test Considerations for PCI Express Architecture document.

A-0462

TtxA

VtxA

VtxA_d

Figure 5-4: ExpressModule Transmitter Path Compliance Eye Diagram


91

5.4.2. ExpressModule Minimum Receiver Path Sensitivity Requirements

The minimum sensitivity values for the add-in card’s Receiver path compliance are defined in Table 5-6, and a representative eye diagram is shown in Figure 5-5.

Table 5-6: ExpressModule Minimum Receiver Path Sensitivity Requirements

Parameter Value Comments VRXA

VRXA_d

238 mV

219 mV

Notes 1, 2, 5 Notes 1, 2, 5

TRXA JRXA-MEDIAN-to-MAX-JITTER

246 ps

77 ps

Notes 1, 3, 5 Notes 1, 4, 5



2. Transition and non-transition bits must be distinguished in order to measure compliance against the de-emphasized voltage level (VRXA_d). VRXA and VRXA_d are differential peak-peak output voltages.

3. TRXA is the eye width. The sample size for this measurement is 106 UI. This value can be reduced to 233 ps for simulation purpose at BER 10-12.

4. JRXA-MEDIAN-to-MAX-JITTER is the maximum median-to-peak jitter outlier as defined in the PCI Express Base Specification, Revision 1.1. The sample size for this measurement is 106 UI. This value can be increased to 83.5 ps for simulation purpose at BER 10-12.

5. The values in Table 5-6 are initially referenced to an ideal 100 Ω differential load. The resultant values, when provided to the Receiver interconnect path of the add-in card, allow for a demonstration of compliance of the overall add-in card Receiver path. The sensitivity requirements are defined and centered with respect to the jitter median. Exact conditions required for verifying compliance against these values are given in the PHY Electrical Test Considerations for PCI Express Architecture document.

A-0463

TrxA

VrxA

VrxA_d

Figure 5-5: Representative Composite Eye Diagram for ExpressModule Receiver Path

Compliance


92

5.4.3. System Board Transmitter Path Compliance Eye Diagram

The eye-diagram for the system board’s Transmitter compliance is defined in Table 5-7 and Figure 5-6.

Table 5-7: System Board Transmitter Path Compliance Eye Requirements

Parameter Value Comments VTXS

VTXS_d

>= 274 mV

>= 253 mV

Notes 1, 2, 5 Notes 1, 2, 5

TTXS JTXS-MEDIAN-to-MAX-JITTER

>= 246 ps

<= 77 ps

Notes 1, 3, 5 Notes 1, 4, 5



2. Transition and non-transition bits must be distinguished in order to measure compliance against the de-emphasized voltage level (VTXS_d). VTXS and VTXS_d are minimum differential peak-peak output voltages.

3. TTXS is the minimum eye width. The sample size for this measurement is 106 UI. This value can be reduced to 233 ps for simulation purpose at BER 10-12.

4. JTXS-MEDIAN-to-MAX-JITTER is the maximum median-to-max jitter outlier as defined in the PCI Express Base Specification, Revision 1.1. The sample size for this measurement is 106 UI. This value can be increased to 83.5 ps for simulation purpose at BER 10-12.

5. The values in Table 5-7 are referenced to an ideal 100 Ω differential load at the end of the interconnect path at the edge-finger boundary on the add-in card when mated with a connector (see Figure 5-1). The eye diagram is defined and centered with respect to the jitter median. Exact conditions required for verifying compliance while generating this eye diagram are given in the PHY Electrical Test Considerations for PCI Express Architecture document.

A-0464

TtxS

VtxS

VtxS_d

Figure 5-6: System Board Transmitter Path Composite Compliance Eye Diagram


93

It is not always possible to measure the System Board Transmitter Path Eye Diagrams with an ideal reference clock. In this case a two port measurement can adjust the measurement for the non-ideal reference clock. The notation in this section assumes the discrete time domain form of xn. Equivalent notation sometimes used in other literature is x(n) and x[n]. The convolution operator is denoted as ⊗.

Referring to Figure 5-7,

xn is the sampled phase jitter on the reference clock at the connector. A band-pass function is used to measure the reference clock as described in the PCI Express Card Electromechanical Specification. hn is the impulse response of the transmitter PLL on the system board. nn is the intrinsic transmitter jitter. Using an ideal reference clock with 0 phase jitter, this would be the measured transmitter jitter. The peak-peak value of the total jitter must meet the eye requirements as specified in the PCI Express Base Specification. yn = [xn ⊗ hn] + nn is the total jitter of the reference clock, transmitter and system board interconnect at the connector, where ⊗ is the discrete convolution operator.

In the case of non-zero reference clock noise, the intrinsic jitter of the transmitter can be calculated in the discrete time domain by the following:

nn = yn – [xn ⊗ hn]

where xn is convolved with hn. The peak-peak value of nn is then calculated.

The equation above has two unknowns, hn and nn. If hn is known, it can be used directly. Otherwise, hn must be assumed to be the lowest limit of the allowed PLL bandwidth with no peaking (see the PCI Express Base Specification for the specification limits).

A summary of this procedure is:

1. Take the simultaneous measurements of yn and xn for the system board at the connector. 2. Find xn ⊗ hn. 3. Calculate nn = yn – [xn ⊗ hn] 4. Calculate the peak-peak value of nn.


94

A-0465

FromReference

Clock

Total Jitterat Connector

ReferenceClock Jitterat Connector

hn +xn ⊗ hn yn = xn ⊗ hn + nnxn

nn

xn

Figure 5-7: Two Port Measurement Model

5.4.4. System Board Minimum Receiver Path Sensitivity Requirements

The minimum sensitivity values for the system board’s Receiver path compliance are defined in Table 5-8, and a representative eye diagram is shown in Figure 5-8.

Table 5-8: System Board Minimum Receiver Path Sensitivity Requirements

Parameter Value Comments VRXS

VRXS_d

445 mV

312 mV

Notes 1, 2, 5 Notes 1, 2, 5

TRXS JRXS-MEDIAN-to-MAX-JITTER

287 ps

56.5 ps

Notes 1, 3, 5 Notes 1, 4, 5

Notes:

1. An ideal reference clock without jitter is assumed for this specification. All Links are assumed active while generating this eye diagram.

2. Transition and non-transition bits must be distinguished in order to measure compliance against the de-emphasized voltage level (VRXS_d). VRXS and VRXS_d are differential peak-peak output voltages.

3. TRXS is the eye width. The sample size for this measurement is 106 UI. This value can be reduced to 274 ps for simulation purpose at BER 10-12.

4. JRXS-MEDIAN-to-MAX-JITTER is the maximum median-to-peak jitter outlier as defined in the PCI Express Base Specification, Revision 1.1. The sample size for this measurement is 106 UI. This value can be increased to 63 ps for simulation purpose at BER 10-12.

5. The values in Table 5-8 are referenced to an ideal 100 Ω differential load. The resultant values, when provided to the Receiver interconnect path of the system board, allow for a demonstration of compliance of the overall system board Receiver path. The sensitivity requirements are defined and centered with respect to the jitter median. Exact conditions required for verifying compliance while generating this eye diagram are given in the PHY Electrical Test Considerations for PCI Express Architecture document.


95

A-0466

TrxS

VrxS

VrxS_d

Figure 5-8: Representative Composite Eye Diagram for System Board Receiver Path

Compliance


96


97

6. ExpressModule Connector The PCI Express ExpressModule card edge connectors support x1, x4, x8, and x16 Link widths to suit the different bandwidth requirements. These connectors support the PCI Express signal and power requirements, as well as auxiliary signals used to facilitate the interface between system board and add-in ExpressModule. This chapter defines the connector mating interfaces and footprints, as well as the electrical, mechanical, and environmental requirements. The primary ExpressModule interface connector is physically the same for all Link widths. The ExpressModule connector is limited to a single PCI Express Link. Multiple x1 or x4 Links are not allowed and unused PCI Express Link signal pins may not be used for any other purpose. This supports the maximum interchangeability between ExpressModule and system slots. The primary ExpressModule interface connector supports up to x8 lanes. x16 lanes are only supported in a doublewide module utilizing two connectors of x8 lanes each.

6.1. Connector Pin Counts Table 6-1 identifies the type and quantity of pins for a x8 backplane connector.

Table 6-1: ExpressModule Connector Pin Type/Count

Pin Qty Type PERp(0-7) 8 Differential Plus Input

PERn(0-7) 8 Differential Minus Input

PETp(0-7) 8 Differential Plus output

PETn(0-7) 8 Differential Minus output

Refclk (-/+) 2 Differential Inputs

Gnd 40 Ground

Bulk Power 6 12VDC power input

VauxP 1 Resistive path to Aux input

Vaux 1 3.3VDC power input

SMB-Clk 1 System management clock (I/O)

SMB-data 1 System management data (I/O)

SMBAlert 1 SMbus Alert output

MRST# 1 Module Reset input

MPWRGD# 1 Module Power Good

PWRFLT# 1 Power Fault output

6


98

Pin Qty Type PWREN# 1 Power Enable input

PRSNT# 1 Present Detection output

WAKE 1 Wake system output

ATNSW 1 Attention switch output

ATNLED 1 Attention LED input

PWRLED 1 Power LED input

RSVD 4 Reserved pins

Total pins 98

Table 6-2: Optional Storage Pin Count/Type

Pin Qty Type SRp(0-3) 4 Storage Differential Plus Input

SRn(0-3) 4 Storage Differential Minus Input

STp(0-3) 4 Storage Differential Plus output

STn(0-3) 4 Storage Differential Minus output

Gnd 18 Ground

STOR_SB-1 1 See Table 2-3 or Table 2-4 for details




RSVD 2 Reserved

Total Pins 40


99

6.2. Connector Pin Assignments

Table 6-3: ExpressModule Connector Pin Assignments

Side B Side A Pin #

Name Description Name Description 1 +12V Bulk Supply GND Ground

2 +12V Bulk Supply GND Ground


4 RSVD Reserved RSVD Reserved

5 ATNSW Attention Switch PWRLED Power LED

6 MRST# Module Reset ATNLED Attention LED

7 SMDAT SM Data SMBAlert SMbus alert

8 SMCLK SM Clock PWRFLT# Power Fault *

9 +VauxP Pre AUX Power MPWRGD# Module Power Good *

10 GND Ground WAKE Signal to Wakeup *

11 +Vaux AUX Power * PRSNT# Present Detection *

Key

12 PWR_EN# Power Enable * GND Ground

13 RSVD Reserved REFCLK+ Reference Clk +

14 GND Ground REFCLK- Reference Clk -

15 PETp0 GND Ground

16 PETn0

Transmitter Differential pair lane 0 GND Ground

17 GND Ground PERp0

18 GND Ground PERn0

Receive Differential pair lane 0

19 PETp1 GND Ground

20 PETn1


21 GND Ground PERp1

22 GND Ground PERn1


23 PETp2 GND Ground

24 PETn2


25 GND Ground PERp2

26 GND Ground PERn2



100

Side B Side A Pin #

Name Description Name Description 27 PETp3 GND Ground

28 PETn3


29 GND Ground PERp3

30 GND Ground PERn3


31 PETp4 GND Ground

32 PETn4


33 GND Ground PERp4

34 GND Ground PERn4


35 PETp5 GND Ground

36 PETn5


37 GND Ground PERp5

38 GND Ground PERn5


39 PETp6 GND Ground

40 PETn6


41 GND Ground PERp6

42 GND Ground PERn6


43 PETp7 GND Ground

44 PETn7


45 GND Ground PERp7

46 RSVD Reserved PERn7





End of the x8 connector and Key for x8 with storage

50 GND Ground RSVD Reserved

51 SRp0 GND Ground

52 SRn0 Receive Differential pair, storage lane 0 GND Ground

53 GND Ground STp0

54 GND Ground STn0

Transmitter Differential pair storage lane 0

55 SRp1 GND Ground

56 SRn1 Receive Differential pair, storage lane 1 GND Ground


101

Side B Side A Pin #

Name Description Name Description 57 GND Ground STp1

58 STOR_SB-1 SideBand 1 STn1


59 STOR_SB-2 SideBand 2 GND Ground

60 GND Ground STOR_SB-3 SideBand 3

61 SRp2 STOR_SB-4 SideBand 4

62 SRn2

Receive Differential pair storage lane 2 GND Ground

63 GND Ground STp1

64 GND Ground STn1


65 SRp3 GND Ground

66 SRn3

Receive Differential pair storage lane 3 GND Ground

67 GND Ground STp3

68 GND Ground STn3


69 RSVD Reserved GND Ground The following points should be noted:

The short pins on the card connector are WAKE, PWRFLT#, VAUX, PRSNT#, MPWRGD# and PWR_EN#. These pins are denoted by an “*” after the name in Table 6-3.

The pins are numbered as shown in Figure 6-3 in ascending order from the left to the right with side A on the top of the centerline and side B on the bottom of the centerline. The PCI Express interface pins PETp(x), PETn(x), PERp(x), and PERn(x) are named with the following convention: “PE” stands for PCI Express high speed, “T” for transmitter, “R” for receiver, “p” for positive (+), and “n” for negative.

By default, PETp(x) and PETn(x) pins (the transmitter differential pair of the connector) shall be connected to the PCI Express transmitter differential pair on the system board and to the PCI Express receiver differential pair on the add-in card. By default, PERp(x) and PERn(x) pins (the receiver differential pair of the connector) shall be connected to the PCI Express receiver differential pair on the system board and to the PCI Express transmitter differential pair on the add-in card.

However, the “p” and “n” connections may be reversed to simplify PCB trace routing and minimize vias if needed. All PCI Express receivers incorporate automatic Lane Polarity Inversion as part of the Link Initialization and Training and will correct the polarity independently on each lane. Refer to Section 4.2.4 of the PCI Express Base Specification, Revision 1.1.

If the component on the system board or ExpressModule does not support the optional PCI Express Lane Reversal functions, they must connect each transmitter and receiver lane to the ExpressModule connector lanes as shown in Table 6-3. For example, a x4 component must connect lane 0 to 0, lane 1 to 1, lane 2 to 2, and lane 3 to 3.


102

If the component on the system board or ExpressModule supports the optional PCI Express Lane Reversal function, it may connect each transmitter and receiver lane to the ExpressModule connector lanes as shown in Table 6-3 or it may connect the transmitter and receiver lanes using a reversed lane ordering. Either lane ordering may be used to simplify PCB trace routing and minimize vias. However, the transmitting and receiving lanes must be connected with the same lane ordering. For example, a x4 component may connect lane 0 to 0, lane 1 to 1, lane 2 to 2, and lane 3 to 3, or it may connect lane 0 to 3, lane 1 to 2, lane 2 to 1, and lane 3 to 0.

The system connectors and the ExpressModule are the same for all configurations. This allows full up-plugging and down-plugging between connectors and ExpressModule for all lane widths.

Adjacent differential pairs are separated by two ground pins to manage the connector crosstalk.

See Chapter 2 for auxiliary signals description and implementation, except the +3.3Vaux and PRSNT# pins. The requirements for +3.3Vaux are discussed in Chapter 4 and presence detect is discussed in Chapter 3.

Detailed requirements on hot plug are covered in Chapter 3.

Power pins (+3.3Vaux, and +12V) are defined based on the PCI Express power delivery requirements specified in Chapter 4 with the connector contact carrying capability being 1.1 A per pin. The power that goes through the connector shall not exceed the maximum power specified for a given ExpressModule size, as defined in Table 4-1.

ExpressModule add-in cards shall utilize 30 microinches of gold plating over a 50-microinch Nickel underplate on all edge card contact positions. This is to protect the backplane connector from excessive wear while supporting full up- or down-plugging.

ExpressModule connectors should provide a minmum 30 microinches of gold or equivalent (e.g., 30-microinch PdNi with gold flash) plating over a minimum of 50-microinch Nickel underplate in the mating area of all contacts.

6.3. Connector Interface Definitions The PCI Express ExpressModule connector is defined as a card edge connector similar to that defined in the PCI Express Card Electromechanical Specification, Revision 1.1, but with enhancements for lead-in and reliability. The PCI Express ExpressModule connector outline footprint and corresponding add-in edge-finger dimensions are shown in the following figures.


103

CENTERLINE OF KEY

CENTERLINE OF PEG

MAXSEE NOTE 5

NOTES:1. BY DESIGN2. SIZE AND LOCATION FOR DEFINED FOOTPRINT

AND MEET THE FOLLOWING REQUIREMENT:

THE DIFFERENCE BETWEEN THE NOMINAL HOLE DIAMETER 0.70 [.028] AND THE NOMINAL DIAGONAL DIMENSION "B" OF

THE SOLDER TAIL CROSS SECTION. 0.70-B .25.

3. POST DIAMETER SHOULD BE SIZED SO THAT THE MAJOR DIAMETEROF THE POST IS ENGAGED WITH THE PCB HOLE PRIOR TO THE LEADS ENTERING THE PLATED THRU HOLES.

4. PIN #A1 IDENTIFIED.5. FREQUENCY AND LOCATION AT SUPPLIER DISCRETION. RIDGE MAY BE CONTINUOUS WITH NO BREAKS.

B B

SECTION B-B

SECTION D-D

ORIENTATION POSTSSIZE & LOCATION FORDEFINED FOOTPRINT.FULL DIAMETER NOT REQUIREDEX:

SEE NOTE 3

LEAD-IN REQUIRED TOASSIST IN PLACEMENTOF CONNECTOR TO PCB

SEE NOTE 4

D

D

PCB THK. (REF) DIM "A"

1.57[.062]

2.30 +0.25/-0.13[.091 +.010/-.005]

[.028-B .010]

56.00±0.23[2.205±.009]

39.54[1.55]

40.15[1.581]REF

3.40±.15[.134±.006]

4.40±.20[.173±.008]

SEE NOTE 1

REF

REF

[.122 +.010/-.005]3.10 +0.25/-0.13

[.093]2.36

REF

REF

REFREF

B B

SECTION D-DD

D

SECTION B-B

0.70[.028]

DIM "B"

DEFINED ADD-IN CARDSIZE AND LOCATION FORCONTACT INTERFACE

(10 SPACESAT 1.00[.039])

REF

10.00 [.394]

REF

(37 SPACESAT 1.00[.039])

37.00 [1.457]

6. SLOT MUST ACCEPT STANDARD 1.57mm (.062)±10% THICK ADD-IN CARD.

45° TYP

7. 0.15mm OFFSET BETWEEN THE CENTERLINE OF THE PEG AND THE CENTERLINE OF THE CONNECTOR KEY.

B

D

C

A

X8 CONNECTOR

D

A

0.785 [0.031]SEE NOTE 9

8. DATUM STRUCTURE SHOWN IS IN REFERENCE TO THESYSTEM ASSEMBLY DEFINED IN THIS SPECIFICATION. SEE SECTION 7.3

9. DIMENSION SHOWN IS FROM CENTERLINE OF THE SLOT TO THE EDGE OF NOMINAL MODULE BOARD WHEN INSERTED INTO CONNECTOR.

SEE NOTE 7

REF

REF

SEE FIGURE 6-4

Figure 6-1: ExpressModule Connector Form Factor


104

MAXSEE NOTE 5

ORIENTATION POSTSSIZE & LOCATION FORDEFINED FOOTPRINT.FULL DIAMETER NOT REQUIREDEX:

SEE NOTE 3

LEAD-IN REQUIRED TOASSIST IN PLACEMENTOF CONNECTOR TO PCB

SEE NOTE 4

PCB THK. (REF) DIM "A"

1.57[.062]

2.30 +0.25/-0.13[.091 +.010/-.005]

81.00[3.19]

39.54[1.55]

40.15[1.581]

3.40±.15[.134±.006]

4.40±.20[.173±.008]

SEE NOTE 1DIM "A"

CENTERLINEOF

KEY

PEG

CENTERLINEOF

REF

REF REF

REF

REF REF

REF

RIDGE MAY BE CONTINUOUS WITH NO BREAKS.5. FREQUENCY AND LOCATION AT SUPPLIER DISCRETION.4. PIN #A1 IDENTIFIED.

ENTERING THE PLATED THRU HOLES.OF THE POST IS ENGAGED WITH THE PCB HOLE PRIOR TO THE LEADS

3. POST DIAMETER SHOULD BE SIZED SO THAT THE MAJOR DIAMETER

THE SOLDER TAIL CROSS SECTION. 0.70-B .25. 0.70 [.028] AND THE NOMINAL DIAGONAL DIMENSION "B" OF THE DIFFERENCE BETWEEN THE NOMINAL HOLE DIAMETER

AND MEET THE FOLLOWING REQUIREMENT:2. SIZE AND LOCATION FOR DEFINED FOOTPRINT1. BY DESIGNNOTES:

[.028-B .010]

DIM "B"[.028]0.70

SECTION D-DSECTION D-D

SECTION B-BSECTION B-B

DD

DD

[.093]2.36

[.122 +.010/-.005]3.10 +0.25/-0.13

REF

REF

REF

MAX

REF

DEFINED ADD-IN CARDSIZE AND LOCATION FORCONTACT INTERFACE

REF

10.00 [.394] REF

(37 SPACESAT 1.00[.039])

37.00 [1.457]

REF

(20 SPACESAT 1.00[.039])

20.00 [.787]

±0.23[±.009]

B BB B

6. SLOT MUST ACCEPT STANDARD 1.57mm (.062)±10% THICK ADD-IN CARD.

45° TYP

X8E CONNECTOR

A

D

B

D

OF THE PEG AND THE CENTERLINE OF THE CONNECTOR KEY.7. 0.15mm OFFSET BETWEEN THE CENTERLINE

7.50 [.295]MAX

C

A

8. DATUM STRUCTURE SHOWN IS IN REFERENCE TO THESYSTEM ASSEMBLY DEFINED IN THIS SPECIFICATION. SEE SECTION 7.3

9. DIMENSION SHOWN IS FROM CENTERLINE OF SLOT TO EDGE OF NOMINALMODULE BOARD WHEN INSERTED INTO CONNECTOR.

0.785 [0.031]

SEENOTE 7

REF

SEENOTE 9

REF

REF

REF

SEE FIGURE 6-4 & 6-5

(10 SPACESAT 1.00[.039])

Figure 6-2: ExpressModule Connector Form Factor with Storage Extension


105

REF REF

37X

10XA4A2

PIN #A1 IDENTIFIED BYSQUARE ANNULAR RING

A3B3

B1

B2B4

Ø2.35±.05[Ø.093±.002]

40.15±0.10 [1.581±.004]]

-D-PCB SURFACE

Ø2.35±0.05[.093±.002]

2X

2X

1.25 [0.049]

Z

C

Ø0.70±0.08[Ø.028±.003]

Ø0.10[.004 M D Z M ZZ M

A

RECOMMENDED FOOTPRINT FOR X898 POSITIONS NOTES:

1. DATUM A REFERENCES THE CONNECTOR KEY CENTERLINE LOCATION SEE SECTION 7.3.

SEE NOTE 1

2. DATUM C REFERENCES THE COMPONENT SURFACE OF THE

SEE NOTE 2

REF

NOMINAL MODULE CARD SEE SECTION 7.3.

ZZ

2.35 REF

[Ø.093±.002]

B4

B3

B1

B2

PIN #A1 IDENTIFIED BYSQUARE ANNULAR RING

A3

A4A2

140 POSITIONSRECOMMENDED FOOTPRINT FOR X8E

Z

Ø2.35±.05

10X

REF

43.35 [1.70]

PCB SURFACE-D-

REF

1. DATUM A REFERENCES THE CONNECTOR KEY CENTERLINE LOCATION SEE SECTION 7.3.

[.093±.002]

NOTES:

Ø2.35±0.05

20X

REF

1.25 [0.049] C

2X

Ø0.10[.004] M D Z M ZZ M[Ø.028±.003]Ø0.70±0.08

ASEE NOTE 1

REF

SEE NOTE 2

2. DATUM C REFERENCES THE COMPONENT SURFACE OF THE

37X

2X

NOMINAL MODULE CARD SEE SECTION 7.3.

ZZ

2.35 REF

Figure 6-3: Recommended Footprints


106

1.90±.05[.075±.002]

PIN B1PIN "C"

SEE TABLESEE NOTE 1

.10[.004] L C B A L

.05[.002] L

PRIMARY (COMPONENT SIDE)CONNECTORLINK WIDTH

# POSREF

"C"NOTE 1

X8 98 A8, A9, A10, A11B11, B12

C

B

TYP

37 SPACES AT1.00[.039]

10 SPACES AT1.00[.039]

20°±5°

NOTE 2SEE

B11

1.57 ± 0.157 [.062 ± 0.006]

PIN B12

ACROSS PADS

.50[.020]x45°TYP

REF

LEADING EDGE OF PAD FOR PINS "C".

2. CHAMFER EDGES MUST BE FREE OF CUTTING BURRS.

1. NO TIE BAR PERMITTED FROM CARD EDGE TO

NOTES:

SECONDARY (SOLDER SIDE)

SEE TABLE

A1

PIN A11PIN A8

3. DATUMS A,B,C REFERENCE THIS PART AS WELL ASTHE SYSTEM ASSEMBLY DEFINED BY THIS

REF

4. TOLERANCE: .XX ±0.13 [0.005] UNLESS OTHERWISE NOTED

A

0.20 [ .008]

5. FLATNESS REQUIREMENTS APPLIES FROM EDGE OF CARD TO

SEE NOTE 5

TOP OF KEY

SEE NOTE 1SPECIFICATION IN SECTION 7.3

Figure 6-4: Add-in Module Edge-Finger Dimensions


107

PIN B1

CONNECTORLINK WIDTH

# POSREF

"C"NOTE 1

X8E 140 A8, A9, A10, A11B11, B12

LEADING EDGE OF PAD FOR PINS "C".

2. CHAMFER EDGES MUST BE FREE OF CUTTING

1. NO TIE BAR PERMITTED FROM CARD EDGE TO

BURRS.

NOTES:

PRIMARY (COMPONENT) SIDE

C ACROSS PADS

B

1.90±.05[.075±.002]

TYP

10 SPACES AT1.00[.039] 1.00[.039]

37 SPACES AT 20 SPACES AT1.00[.039]

.50[.020]X45°TYP.

SECONDARY (SOLDER SIDE)

2.40[0.94]

SEE NOTE 1SEE TABLE

.10[.004] L C B A L

.05[.002] L

DETAIL "A" PIN "C"

3.20[.126]REF

DETAIL A

B11

B12B11

A8

A11

A1

ASSEMBLY DEFINED BY THIS SPECIFICATION IN SECTION 7.3.3. DATUMS A,B,C, REFERENCE THIS PART AS WELL AS THE SYSTEM

1.57 ± 0.157[.062 ± 0.006]

REF

SEE NOTE 1SEE TABLE

4. TOLERANCE: .XX ±0.13 [0.005] UNLESS OTHERWISE NOTED

A

0.20 [ .008] SEE NOTE 5

5. FLATNESS REQUIREMENTS APPLIES FROM EDGE OF CARD TO TOP OF KEY

Figure 6-5: Add-in Module with Storage Edge-Finger Dimensions

The following points should be noted:

The connector has a 1.00-mm contact pitch.

The contact shall be pre-loaded, similar to the PCI connector.

The connector shall have greater than 1.0 mm of wipe length.


108

The connector footprint (Figure 6-1 and Figure 6-2) requires two 2.35-mm diameter location holes, working with either plastic pegs/posts or metal board locks. Metal board locks are allowed, although Figure 6-1 and Figure 6-2 shows only the plastic pegs on the connector housing.

Figure 6-5 defines only the mating interface related dimensions. Other ExpressModule dimensions are defined in Chapter 7.

The short pins defined in the pin list and shown in Figure 6-4 are 1 mm shorter than the other fingers. No plating tie bar is allowed between the pad and leading edge of card for the short pins because these pins are meant to be last-mate and first-break.

Detailed connector contact and housing designs are up to each connector vendor, as long as the requirements of form, fit, and function are met.

6.4. Connector Signal Integrity Requirements and Test Procedures

A common electrical test fixture is specified and used for evaluating connector signal integrity. The test fixture will have 6-mil wide 50 Ω single ended traces that must be uncoupled. The impedance variation of those traces shall be controlled within ±5%. Refer to the PCI Express Connector High Speed Electrical Test Procedure for detailed discussions on the test fixture.

Detailed testing procedures, such as the vector network analyzer settings, operation, and calibration, are specified in the PCI Express Connector High Speed Electrical Test Procedure. This document should be used in conjunction with the standard test fixture.

For the insertion loss and return loss tests, the measurement shall include 1.2-inch long PCB traces (0.6 inches on the system board and 0.6 inches on the add-in card). Note that the edge finger pad is not counted as the ExpressModule PCB trace. It is considered to be part of the connector interface. The 1.2-inch PCB trace included in the connector measurement is a part of the trace length allowed on the system board. See Section 5.3 for a discussion of the electrical budget.

Either single ended measurements that are processed to extract the differential characteristics or true differential measurements are allowed. The detailed definition and description of the test fixture and the measurement procedures are provided in the PCI Express Connector High Speed Electrical Test Procedure. An additional consideration for the connector electrical performance is the connector-to-system board and the connector-to-add-in-card launches. The connector through hole pad and anti-pad sizes shall follow good electrical design practices to minimize impedance discontinuity. On the add-in card, the ground and power planes underneath the PCI Express high-speed signals (edge fingers) shall be removed. Otherwise, the edge fingers will have too much capacitance and greatly degrade connector performance. A more detailed discussion on the ExpressModule electrical design can be found in the PCI Express Connector High Speed Electrical Test Procedure.


109

Table 6-4 lists the electrical signal integrity parameters, requirements, and test procedures.

Table 6-4: Signal Integrity Requirements and Test Procedures

Parameter Procedure Requirements Insertion Loss (IL) EIA 364-101

The EIA standard must be used with the following considerations:

1. The step-by-step measurement procedure is outlined in the PCI Express Connector High Speed Electrical Test Procedure document (Note 1).

2. A common test fixture for connector characterization shall be used.

3. This is a differential insertion loss requirement. Either true differential measurements must be made or post processing of the single ended measurements must be done to extract the differential characteristics of the connector. The methodology of doing so is covered in the PCI Express Connector High Speed Electrical Test Procedure document (Note 1).

1 dB max up to 1.25 GHz; ≤ [1.6* (F - 1.25)+1] dB for 1.25 GHz < F ≤ 3.75 GHz (for example, ≤ 5 dB at F = 3.75 GHz)

Return Loss (RL) EIA 364-108 The EIA standard must be used with the following considerations:

1. The step-by-step measurement procedure is outlined in the PCI Express Connector High Speed Electrical Test Procedure document (Note 1).


3. This is a differential return loss requirement. Either true differential measurements must be made or post processing of the single ended measurements must be done to extract the differential characteristics of the connector. The methodology of doing so is covered in the PCI Express Connector High Speed Electrical Test Procedure document (Note 1).

≤ -12 dB up to 1.3 GHz; ≤ -7 dB up to 2 GHz; ≤ -4 dB up to 3.75 GHz

Intra-pair Skew Intra-pair skew must be achieved by design; measurement not required.

5 ps maximum


110

Parameter Procedure Requirements Crosstalk: NEXT

EIA 364-90 The EIA standard must be used with the following considerations:

1. The crosstalk requirement is with respect to all the adjacent differential pairs including the crosstalk from opposite sides of the connector, as illustrated in Figure 6-6. This is reflected in the measurement procedure.

2. The step-by-step measurement procedure is outlined in the PCI Express Connector High Speed Electrical Test Procedure document.


4. This is a differential crosstalk requirement between a victim differential signal pair and all of its adjacent differential signal pairs. Either true differential measurements must be made or post processing of the single ended measurements must be done to extract the differential crosstalk of the connector. The methodology of doing so is covered in the PCI Express Connector High Speed Electrical Test Procedure document (Note 1).

-32 dB max up to 1.25 GHz; ≤ -[32 - 2.4*(F-1.25)] dB for 1.25 GHz < F ≤ 3.75 GHz (for example, ≤ -26 dB at F = 3.75 GHz)

Jitter By design; measurement not required. 10 ps maximum

Notes: 1. The PCI Express Connector High Speed Electrical Test Procedure is available separately. 2. A network analyzer is required. Differential measurements require the use of a two port (or a four port)

network analyzer to measure the connector. The differential parameters may be measured directly if the equipment supports “True” differential excitation. (“True” differential excitation is the simultaneous application of a signal to one line of the pair and a 180 degree phase shifted version of the signal to the second line of the pair). If single ended measurements are made, the differential connector parameters must be derived from the single ended measurements as defined in the PCI Express Connector High Speed Electrical Test Procedure.

3. The connector is recommended to be 100 Ω differential impedance.


111

In Figure 6-6, pairs marked as 11-9, 5-7, 15-13, and 17-19 are the adjacent pairs with respect to the victim pair 1-3.

A-0467

15 13 17 19

11 9 1 3

Victim PairNeed to Terminate All Ports

5 7

Figure 6-6: Illustration of Adjacent Pairs

6.5. Connector Environmental and Other Requirements

6.5.1. Environmental Requirements

Connector environmental tests shall follow EIA-364-1000.01, Environmental Test Methodology for Assessing the Performance of Electrical Connectors and Sockets Used in Business Office Applications. The test groups/sequences and durations shall be derived from the following application requirements:

Durability (mating/unmating) rating of 50 cycles

Field temperature: 65 °C

Field life: seven years

Since the connector defined in Section 6.3 has far more than 0.127 mm wipe length, Test Group 6 in EIA-364-1000.01 is not required. Test Group 7 in EIA-364-1000.01 is optional since the durability cycles is ≤ 50. The temperature life test duration and the mixed flowing gas test duration values are derived from EIA 364-1000.01 based on the field temperature using simple linear interpolation. Table 6-5 lists these values.


112

Table 6-5: Test Durations

Test Duration/Temperature Durability (preconditioning) 20 cycles

Durability 50 cycles

Temperature Life 168 hours at 105 °C

Temperature Life (preconditioning) 92 hours at 105 °C

Mixed Flowing Gas 10 days, unmated for 2/3 of the test duration, then mated for the remaining 1/3 of the test

The low level contact resistance (LLCR) is required to be 30 mΩ or less, initially. Note that the contact resistance measurement points shall include the solder joint of the solder tail and the top of the contact-mating pad on the add-in card as illustrated in Figure 6-7. The resistance change, which is defined as the change in LLCR between the reading after stress and the initial reading, shall not exceed 25 mΩ, and the resistance change of no more than 1 percent of the contacts shall exceed 10 mΩ.

A-0468

PCI ExpressConnector

PCB

Add-inCard

ResistanceMeasurementPoints

Figure 6-7: Contact Resistance Measurement Points

To be sure that the environmental tests measure the stability of the connector, it is highly desirable that the add-in cards used for testing be at the lower and upper limit of the card thickness requirement to give an indication of the stability of the connector. In any case, both the edge tab plating thickness and the card thickness shall be recorded in the environmental test report.


113

6.5.2. Mechanical Requirements

Table 6-6 lists the mechanical parameters and requirements. Note that the sample size shall follow Section 2.2.1 of EIA-364-1000.01.

Table 6-6: Mechanical Test Procedures and Requirements

Test Description Procedure Requirement Visual and dimensional inspections

EIA 364-18 Visual, dimensional, and functional per applicable quality inspection plan

Meets product drawing requirements

Insertion force EIA 364-13 Measure the force necessary to mate the connector assemblies at a maximum rate of 12.5 mm (0.492 inches) per minute, using a steel gauge 1.70 mm thick with a tolerance + 0.00, - .01 mm

1.15 N maximum per contact pair

Removal force EIA 364-13 Measure the force necessary to un-mate the connector assemblies at maximum rate of 12.5 mm (0.492 inches) per minute, using a steel gauge 1.44 mm thick with a tolerance + .01, - 0.00 mm

0.15 N minimum per contact pair


114

6.5.3. Current Rating Requirement

Table 6-7 lists the contact current rating requirement and test procedure.

Table 6-7: End-of-Life Current Rating Test Sequence

Test Order

Test Procedure Condition Requirement

1 Contact current rating

EIA 364-70 method 2 The sample size is a minimum of three mated connectors. The sample shall be soldered on a PC board with the appropriate footprint. Wire the six power pins (B1, B2, B3, B47, B48, and B49) and the six nearest ground pins (A1, A2, A3, A47, A48, A49, and B16) in a series circuit. The mated ExpressModule is included in this circuit. The ExpressModule shall have 1 oz. copper traces and its mating geometry shall conform to the applicable PCI Express drawings. A thermocouple of 30 AWG or less shall be placed on the card edge finger pad (pins B2 and B48) as close to the mating contact as possible. Conduct a temperature rise vs. current test.

Mated 1.1 A per pin minimum. The temperature rise above ambient shall not exceed 30 °C. The ambient condition is still air at 25 °C.

6.5.4. Additional Considerations

Table 6-8 lists the additional requirements.

Table 6-8: Additional Requirements

Parameter Procedure Requirement Flammability UL94V-1

minimum Material certification or certificate of compliance is required with each lot to satisfy the Underwriters Laboratories follow-up service requirements.

Lead-free soldering Connector must be compatible with lead-free soldering process.

Connector Color Color of the connector should be black. Exceptions will be made for color coding schemes

This specification does not attempt to define the connector requirements that are considered application-specific. It is up to the users and their connector suppliers to determine if additional requirements shall be added to satisfy the application needs. The system level shock and vibration


115

tests are considered application-specific because results will depend on card weight and size, chassis stiffness, and retention mechanisms, as well as the connector. Therefore, those tests are not specified in the connector specification. It will be up to each system OEM to decide how the shock and vibration tests shall be done.


116


117

7. Module Mechanical Specification

7.1. Mechanical Overview The module mechanical specification defines the module and chassis dimensional parameters that are required for module and systems manufacturers to be able to design their components such that mechanical interoperability is achieved (see Figure 7-1). Only critical design parameters are described. This allows for the manufacturer to optimize their design for their functional needs and manufacturing processes.

The module form factors supported are singlewide and doublewide. The singlewide module provides 26.4 square inches of primary surface for component placement. The doublewide module card area is dependent on how the design is implemented. Daughter cards are permissible within the limits of module volume.

The modules are installed into the chassis’ system slots without requiring covers to be removed. The system slots are designed to allow the insertion of single or doublewide modules. Note that doublewide modules are not supported in 1U systems.

Critical system slot requirements for thermal design, EMI, and module retention are included in this chapter. Internal system storage is supported by a dedicated slot.

7


118

Figure 7-1: Chassis Assembly

7.2. Dimensions and Tolerances All dimensions included in this chapter are in millimeters and shall have a general tolerance of ±0.25 mm unless otherwise specified by the drawings.


119

7.3. System Datum Plane Definition The sections of the mechanical specification are based on detailed dimensioned drawings which all use a common set of references (datum planes). Refer to Figure 7-4 and Figure 7-11. These datum planes are defined in the following table in reference to a vertical module orientation.

Table 7-1: Datum Table

Datum Name

Definition

A Horizontal plane, perpendicular to both the board and backplane, which passes through the center of the key on the backplane connector

B Vertical plane, coincident with the end of the card edge connector and parallel with the system backplane, that represents the mated surface with the backplane connector

C Vertical plane, coincident with the primary component surface of the module circuit board

D Vertical plane, coincident with the primary component surface of the system backplane in which the module plugs

7.4. Module Description The module consists of a base housing to which the electronics card is mounted. Refer to Figure 7-2. At one extreme is the card edge that plugs into the system’s connector. At the opposite end are the connectors that allow the attachment of cables from peripherals outside of the system. The area where the I/O connectors escape is called the I/O plate. Attached to the side of the I/O plate is a lever that is used for ejection, extraction, and module retention. Adjacent to the lever is an EMI gasket. See Section 8.1.4 for gasket details. The material adjacent to the I/O plate provides the surfaces for EMI containment. A cover protects the internal components of the module. If there are replaceable components within the module, the cover must be removable without the use of tools. Examples are batteries, memory upgrades, and daughter cards.

Described in the following sections are the critical parameters that define the singlewide and the doublewide modules.


120

A-0469

Cover

Card-EdgeConnector

Module Board

Module Carrier

Auxiliary Venting

Primary Venting

Ejection Lever

Button

EMI Gasket

Figure 7-2: Exploded View of Singlewide Module

7.4.1. Module Materials

The module materials are not defined by this specification with the exception of the surfaces that contact the chassis and EMI and ESD contact surfaces (guide rails, ejector, EMI gasket). These surfaces shall be corrosion free and galvanically compatible to zinc finishes including any EMI gasket material chosen. The conductivity of the contact surfaces shall not exceed 1 mΩ using the test equipment and methodology defined in Section 7.4.1.1. All module parts shall be free of burrs and sharp edges.

It is strongly recommended that the materials selected for construction of the module comply with applicable hazardous materials restrictions as defined by the countries in which the product is marketed.

There are a number of materials that modules could be formed from that will meet this specification’s requirements. One common material in the industry today is precoated steel. Precoated steel can provide EMI containment, grounding, and structural support for the electronics card within the module. Systems are typically produced from precoated zinc steels or are post plated with nickel plate. The module can be formed out of sheet steel that is either preplated with electro galvanic zinc or hot dipped zinc. If zinc coating is chosen, it should be 20 to 25 g/m2 on each side and should be finished with an antifinger print treatment, AFP, for corrosion resistance. Due to the industry’s switch away from hex-chrome based AFP treatments, the corrosion resistance and the


121

surface conductivity of some of these sheet metals may not have sufficient performance to effectively meet this specification’s requirements. Therefore, a minimum corrosion and electrical conductance are defined. The tests for corrosion and electrical conductivity are detailed below. The same test panel may be used for both tests provided that the conductivity test is performed before the corrosion test. If the sample fails the conductivity test, there is no point in running the corrosion test.

7.4.1.1. Conductivity Measurement Procedure

The surface conductivity of the base sheet metal needs to be measured using a four point Loresta-EP CP-T360 meter equipped with type BSP probes. The maximum allowable impedance of the surface shall not exceed 1 mΩ.

The Loresta BSP probes are gold plated for low resistance and spring loaded to give approximately 210 grams of force when fully depressed. The tips are rounded with a 0.37 mm radius. A mechanical fixture can be designed to hold the two probes at a constant separation with full deflection of the probe tips to achieve maximum force. A separation between probes of 2.5 cm minimum was found to be satisfactory for the purpose of evaluating the AFP coating.

A-0470

Current Source

Voltage

Resistance

Surface Resistance

Internal Resistance

Figure 7-3: Simple Circuit Representation of AFP Steel Using Four-point Resistance Method

The following procedure is used to measure AFP material conductivity:

1. Draw a 5 x 5 cm grid on the metal surfaces, front and back.

2. Gently wipe the surfaces using a soft dry towel or cloth to remove surface contaminants. DO NOT USE ABRASIVES, CHEMICAL CLEANERS, or SOLVENTS.

3. Place the probe tips in the approximate center of each grid cell with the probe tips separated by a minimum of 2.5 cm and fully depressed.

4. Allow the meter to stabilize.

5. Record the resistance in each cell.


122

Measurements may be repeated up to three times if there are inconsistencies or unexplained variations observed. The sample passes when all of the cells have attained a reading of no more than 1 mΩ.

7.4.1.2. Corrosion Measurement Procedure

The following defines an acceptable test for zinc precoated steel; other surface coatings must be tested to an equivalent corrosion level.

If zinc precoated sheet steel is used for the module to chassis contact surfaces, it shall be able to withstand a minimum of 72 hours of salt spray testing per specification ASTM B117. After a rinse of the sample, “white rust” should not cover more than 5% of the total area. White rust is the corrosion of the zinc coating and will precede the onset of “red rust” (corrosion of base steel). The ASTM specification details the specific control parameters for chamber operation and sample orientation so that the corrosion rate is controlled. When testing precoated steel samples, the edges are sealed (either with a plastic tape or some other type of barrier coating such as epoxy or silicone) so that there is no exposed base material which would rust prematurely.

7.4.2. Singlewide and Doublewide Module Form Factor

Figure 7-4 provides the critical dimensions for both single and doublewide modules. Figure 7-5 and Figure 7-6 show section views with dimensional relationships and component heights for the singlewide and doublewide modules. There are instances where a double wide module exists that contains two cards that are expected to mate with adjacent ExpressModule back plane connectors. In order to engage both of the connector’s contacts at very nearly the same moment, the individual card’s datum B positions, card edges, must be located within a window of 0.4 mm from each other. Refer to Figure 7-6. Doublewide modules that engage two adjacent backplane connectors from datum C to CC, must implement a board mounting scheme that accommodates the associated tolerances of mounting, module boards, and backplane connectors. While tolerances are dependent on the individual design, card mounting must be flexible enough to accommodate a window of 0.5 mm in order to provide adequate mating.


123

Figure 7-4: Critical Dimensions


124

B

C

BOARD LENGTH168.2

15.9

5.1

TOP SIDECLEARANCE

14.1

BOTTOM SIDECLEARANCE

2.7

( )21

( )1.6( )GAP

1LEAD-IN RECOMMENDED

Figure 7-5: Singlewide Cross Section

B

C

CC

BOARD LENGTH168.2

37.9

5.1

TOP SIDECLEARANCE

36.1

BOTTOM SIDECLEARANCE

2.7

122

( )43

( )1.6 ( )GAP

1

MATING TWO ADJACENT BACKPLANE CONNECTORS REQUIRES THAT THEBOARD-TO-BOARD MOUNTING ACCOMODATE ASSOCIATED TOLERANCES

1

LEAD-IN RECOMMENDED

Figure 7-6: Doublewide Cross Section

7.4.3. Module Raw Card

The raw card drawing in Figure 7-7 shows the maximum dimensions for the ExpressModule adapter card. The dotted lines indicates that portion of the card that extends into the slot guide feature of the module enclosure. Refer to the section AA view of Figure 7-4 for an example of a card that extends into the slot guide feature. Note that cards that do extend into the slot guide feature must observe a component height restriction in those areas inside the feature. Cards that do not extend into the slot guide feature do not have this restriction and are subject only to the enclosure component height limit.


125

Figure 7-7: Module Raw Card Reference Drawing

7.4.4. Module Ejector and Latch Details

For the purpose of describing the module ejector function, this description assumes that the module is vertical in the chassis. The ejector functions the same for singlewide and doublewide modules. The ejection lever is attached to the module base housing by a pressed in pivot pin. Refer to Figure 7-4 and Figure 7-8. The lever rotates from an open position where it is stopped by a protrusion from the module base housing. In the open position, the lever is aligned such that the module can be slid into the system. A tab (1) on the lever stops the module at the proper displacement to start rotating the lever and start the engagement of the card edge into the system’s connector. A second tab (2) on the lever is aligned with a slot in the bottom of the system. When the lever is at 45 degrees, the second tab on the lever engages the edge of the slot and starts to apply sufficient advantage to overcome the insertion forces of the system connector. When the lever is in the vertical orientation, the module is fully inserted into the system. A catch is positioned on the top of the module to retain the module by holding the lever at the vertical orientation. A spring loaded blue plastic button engages the catch. Recesses are provided on the button for labeling purposes. These are detailed in Section 8.4. When the module is not in the system, the lever assembly may be held in place by the button and the catch. To release the module from the system, one presses down on the blue button and rotates the lever down.


126

Figure 7-8: Ejector Assembly

7.4.5. I/O Plate Details

Refer to Figure 7-9 for details of the I/O plate. The I/O plate provides openings for ventilation, a connector area, a Power LED, an Attention LED, and an Attention switch. The Attention switch shall not extend through the I/O plate; this is to prevent accidental activation. The LEDs and the switch must be positioned at the locations shown. Round, square, or rectangular LEDs are acceptable. The Attention LED is yellow and the Power LED is green. Refer to Table 7-2 for details. Sufficient ventilation opening must be supplied so that the module pressure drop and flow requirements are met as detailed in the thermal section of this specification. The remainder of the area of the I/O plate is allowed for connector population.


127

Table 7-2: LED Color

LED Color Wavelength Power Green (Green/Blue) 555-565 nm

Attention Yellow (Amber) 582-592 nm The LEDs shall have a minimum luminescence of 80 candelas per square meter over an area of at least 1.5 mm and no more than 3.0 mm.

Figure 7-9: Module I/O Plate

7.4.6. Module Air Vent Design for EMI

The module’s airflow venting through the I/O plate causes special attention be required of the design of the vents so that the minimum EMI performance is supported by the I/O plate. Figure 7-10 shows an example of ventilation perforation pattern. Larger openings are possible but not recommended because waveguide technology will be required to meet minimum EMI shielding. All modules and fillers are required to meet minimum EMI shielding defined in Section 8.1.4.


128

Figure 7-10: Module Vent Design

7.5. Filler Component or Module It is recognized that it may be necessary to fill an empty slot to maintain a system’s EMI or thermal management intent. A specific design for a filler entity is not contained in this specification. This responsibility is left to the system manufacturer.

7.6. Chassis Slot Description The chassis slot for the PCI Express ExpressModule is defined by the mechanical interfaces that are required in order for a module to function in a system. Dimensions and details not defined here are left to be defined by the system developer. Figure 7-11 is based on the reference design. The slot consists of the following key features:

Chassis guides that contain lead-ins that ensure smooth module insertion.

A slot which functions with the ejection lever described in the module section. This slot is outside of the module EMI gasket contact area; proper EMI sealing techniques must be used.

The ExpressModule system connector.

EMI/ESD gasketing at the top, bottom, and leftmost side of the chassis slot to complete the four-sided EMI seal. An EMI gasket on the adjacent module completes the seal on a chassis with multiple slots.

Chassis module guides may be formed out of plastic but must provide for a direct or resistive connection between the chassis and the module to complete the EMI management solution module grounding.

An optional opening of the module guide to provide for airflow in a transverse direction. See the thermal section for greater detail.


129

7.6.1. Backplane and Chassis Slot Details

Refer to Figure 7-11 for critical details for the design of the ExpressModule system slots. The slots may be oriented either horizontally or vertically at the system developer’s discretion. For reference, all design details are shown in the vertical orientation. Slot connectors are on 22-mm centers (see datum CC to C in Figure 7-11).

C

A

CC

C BD CC

AA

REF150.4

REF169.4

REF139.5

MIN10

REF13.4

REF18.4

3.4

PIN A1REF

11.5

PIN A1REF

2.035

MODULE PITCH22

REF114

REF4.2

REF9

EMI GASKET

CHASSISEMI SEALS

SECTION A-A

Figure 7-11: Chassis Requirements


130

7.6.1.1. Chassis Slot EMI Design

The same EMI gasket that is used on the module will be applied to the chassis in the locations shown. The gaskets must be attached to the system in a manner that would create a continuous seal around a module when the slot is filled. The gasket should be recessed in a manner that would prevent it from being sheared off when a module is being installed. See Figure 8-8 for details.


131

8. Design Considerations This specification defines a consistent and repeatable method for ensuring a minimum level of integrity and environmental performance. This section defines these details and standard test processes that all chassis, modules, and backplanes specified in this standard can use to be compliant.

It is the system designer’s responsibility to evaluate the relationship between the modules, chassis, and the system environment.

The module’s external surface plays a role in the enclosure’s Cooling and EMI Containment strategy. In addition to meeting the pressure drop requirements of this section, the vent holes must also be sized to meet the EMI containment requirements of Section 8.1.4.

8.1. Cooling/Thermal Environment The design objective is to supply an adequate combination of airflow and temperature to ensure that module component temperature specifications are met. The module is designed to allow two basic types of cooling: (1) longitudinal flow, considered to be the default mode, which consists of air flowing through the length of the module passing through the external vent surface and (2) the alternate mode, lateral cooling, which consists of air flowing across the module and may or may not allow air to pass through the external vent surface. The system manufacturer will decide whether or not it is appropriate to allow air to pass through the external vent surface. If it is not acceptable, the system manufacturer shall specify the slot, slots, or entire chassis as “closed vent” operation and label according to Section 8.4.7. To maintain the most flexibility between the cooling modes, module designs should incorporate multidirectional heat-sinks (like pin fin) rather than unidirectional (plate fin). Filler modules for empty slots will be left up to system manufactures and will not be detailed in this specification.

In order to provide detail for the specification, a thermal test vehicle was fabricated. This test vehicle incorporated a discrete chip with a heatsink that was not fully ducted. The external module surface contained the maximum suggested connector blockage area of 975 mm. This subsequently results in the minimum suggested external venting surface area. All temperature and airflow requirements were generated from the test results from this test vehicle.

8


132

8.1.1. Longitudinal Cooling – Default Mode

When the modules are located in an enclosure in series with the enclosure’s air movers such that air is pushed or pulled through the module, it is being cooled in the default mode and will use “open vent” operation. The module’s external surface is an exhaust or intake vent.

A-0471A

ExpressModule

ExpressModule

Figure 8-1: Longitudinal System Airflow


133

8.1.1.1. Airflow Supplied to Module – Default Mode

The requirement for airflow supplied to each module is dependent upon the air temperature supplied. Each module shall receive a minimum flow rate as specified in Figure 8-2.

A-0472

35 40 45 50 55 60 65 700

2

4

6

8

10

12

Mod

ule

CF

M

˚C, Module Inlet

Figure 8-2: Singlewide Module: Required CFM vs. Module Inlet Temperature, Default Mode

8.1.1.2. Module Pressure Drop – Default Mode

The pressure drop through the module must not be so low that it allows bypass nor so high that it does not allow the minimal flow rate. The module pressure drop shall fall in the shaded area within the curves specified in Figure 8-3. The module shall meet this pressure drop requirement with all side vent holes blocked so that all the airflow passes through the external vent surface.

Doublewide modules require twice the flow rate as singlewide modules and would be represented in Figure 8-2 and Figure 8-3 by doubling the flow rates.


134

Figure 8-3: Singlewide Module: Required Pressure Drop Range vs. Flow Rate, Default Mode

8.1.2. Lateral Cooling – Alternate Mode

Lateral cooling exists when the air is drawn through one side of the module and passes predominantly or entirely through the other side of the module. This can be accomplished in several ways and has an impact on the intended configuration of the module. When the modules are located in an enclosure in parallel with the enclosure’s air movers such that drawing air through the module would cause an unintended short circuit due to the enclosure being under negative pressure (see Figure 8-4), the slot may need closed vent operation. It is the system manufacturer’s responsibility to label the slot, slots, or entire chassis as in need of closed vent operation per Section 8.4.7. No labeling is required for default longitudinal cooling or any configuration where open vent operation is designed by the system manufacturer. With a module operating in closed vent mode, external module venting must be blocked to avoid a short circuit of cooling air and introduction of re-circulated exhaust. The uses of a module in operating in closed vent mode must modify the module by adding an air re-circulation barrier (refer to Section 8.1.2.1).


135

A-0474A

ExpressModule

ExpressModule

Figure 8-4: Unintended Recirculation

8.1.2.1. Air Re-circulation Barrier – Closed Vent Operation

In order to use a module in closed vent mode, the module should be configured to block external module vents with an air re-circulation barrier. The air re-circulation barrier may be as simple as an adhesive backed label that prohibits air flow. An air re-circulation barrier shall block a minimum of 90% of the external module vent holes. The ExpressModule supplier shall provide a re-circulator barrier with each module.

8.1.2.2. Airflow Supplied to Module – Alternate Mode

The requirement for airflow supplied to each module is dependent upon the supplied air temperature. Whether it is operating in open or closed vent mode, if the slot has a lateral cooling direction, each module shall receive a minimum flow rate as specified in Figure 8-5.


136

A-0475

35 40 45 50 55 60 65 700

2

4

6

8

10

12

Mod

ule

CF

M

˚C, Module Inlet

Figure 8-5: Singlewide Module: Required CFM vs. Module Inlet Temperature, Alternate Mode

8.1.2.3. Series Modules

In a lateral cooling orientation, it is possible that modules may be installed such that they are cooled serially. In this case, the same airflow passes through the first, is preheated, and then passes through one or more subsequent modules. The flow rate through these modules shall meet the requirements for the worst case module inlet temperature, presumably the last module in series.

8.1.2.4. Module Pressure Drop – Alternate Mode

The pressure drop through the module must not be so low that it allows bypass nor so high that it does not allow the minimal flow rate. The module pressure drop shall fall in the shaded area within the curves specified in Figure 8-6. The module shall meet this pressure drop requirement with the external vent surface and the connector end of the module blocked so that all the airflow passes through the side module vent surfaces.


137

A-0476

0 2 4 6 8 10 12

CFM

0.15

0.1

0.05

0.2

0.25

0.3

0

Inch

es H

2O

Figure 8-6: Singlewide Module: Required Pressure Drop Range vs. Flow rate, Alternate Mode

Doublewide modules require twice the flow rate as singlewide modules and would be represented in Figure 8-5 and Figure 8-6 by doubling the flow rates.

8.1.3. Cooling Consideration for Storage Extension Slot

Slots that support the optional storage extension are a special case for cooling. The storage extension blocks most of the area reserved for airflow in the default mode.

To make the description easer to understand, the airflow will be defined as entering the module from the backplane and exiting from the modules I/O plate with the module in the vertical orientation.

The storage extension slot is limited to one per backplane segment in this configuration. The storage extension slot must be placed at the end of the backplane segment such that backplane does not extend beyond the backplane connector with the storage extension; see Figure 8-7. This creates an air-inlet to the module of about 10 mm wide from bottom to the top of the module. The long air-inlet provides the necessary cooling for the storage extension slot.


138

STOR

AGE

EXTE

NSIO

N CO

NNEC

TOR

Figure 8-7: Storage Slot Venting

8.1.4. Module EMI Design

The module design must consider three areas of EMI containment. The first is the module to the system chassis, the second is the thermal air vents, and the third is any connector in the I/O plate. The EMI management of modules and fillers shall meet the EMI curve below independent of any agency requirements for the product’s market.

Figure 8-8: EMI Gasket Profile


139

Due to the nature of how the module and chassis slot fit together, the module is an integral portion of the chassis EMI containment strategy. The five-sided box that forms the modules I/O plate is also part of the chassis EMI containment and, therefore, must be defined such that it can contain system levels of EMI (not just card generated EMI). The module’s EMI contacts mate with the chassis on at least two sides and to other modules or the chassis on the other two sides forming a 360-degree EMI contact around each module to contain the chassis and module’s EMI.

The chassis shall provide an EMI gasket on the top and bottom surfaces of the slot that mate with the module per Figure 7-11.

The module shall make a near continuous (no gap greater than 4 mm) EMI seal with the chassis and surrounding modules around its perimeter.

The module shall provide an EMI gasket located per Figure 7-4.

The gasket material shall be used with a nominal compression of up to 30% assuming it is recessed 1 mm into the chassis and compressed in a nominal gap of 1 mm. The gasket material shall be such that sliding contact causes no degradation. The gasket material shall have a finish that is galvanically compatible with zinc.

Because the module also contains system EMI, the five-sided box that composes the module’s I/O plate and EMI interface must provide a known level of EMI containment. The attenuation levels of Figure 8-9 shall be met by all modules regardless of the relative level of internal module emissions.

A-0477

45

35

25

15

100 1,0000 10,000

Frequency (MHz)

Atte

nuat

ion

(dB

)

100

1,000

12,500

2,000

Figure 8-9: Required Module I/O Plate Attenuation

The implementation of the connection between signal ground and chassis ground is the responsibility of the system integrator. That connection should be made at a point that minimizes EMI.


140

Although not recommended, it is permissible to connect signal ground and chassis ground together on an ExpressModule. If such connection(s) must be made, and because a logic to frame ground connection has a greater potential for causing EMI problems if done improperly, modules making this connection shall be EMI tested with all supported I/O cables attached while exercising all I/O interfaces using typical customer data and the module shall continue to meet the ESD specifications.

8.2. Module ESD Design The ESD performance of the module shall meet the requirements of EN55024 following the procedures specified in IEC 61000-4-2. This standard requires that the module provide sufficient shielding and grounding to meet ESD discharges of the appropriate category.

For general purpose computing equipment, a level 2 test of 4 kV contact and 8 kV indirect of both polarities is required.

For network equipment building systems (NEBS), a level 4 test of 8 kV contact and 15 kV indirect of both polarities is required.

The module guide surface shall be conductive to provide an ESD discharge path to the slot during insertion.

8.3. Module Interoperability PCI Express module and slots can exist with a variety of Link widths. The interoperability of module and slots is summarized in the following table:

Table 8-1: x8 Slot Connector Interoperability

Slot → Module ↓

x8 Connector with x1 Wired




x1 Yes Yes Yes Yes

x2 At x1 only Yes Optional Optional

x4 At x1 only x1 Yes Yes

x8 At x1 only x1 x4 and x1 Yes Note: x16 slots are composed of two adjacent singlewide x8 slots that have the ability to be combined into a single x16 slot. This is an optional configuration for both modules and chassis. x16 slots will be identified by markings defined in Section 8.4.6. A x16 module must support operation at x8 width and be clearly labeled as x16 width.

8.4. Slot/Module Color Coding and Labeling Since the ExpressModule is designed to be installed from the outside of a system, the usual indicators that tell the user the system attributes are not visible. The user is literally blind to what’s inside the system. For example in a PCI system you can tell a 32-bit slot from a 64-bit slot by the size of the system’s connector and match the appropriate bandwidth of the card by the size of its


141

card edge connector. The goal of this labeling section is to ensure that enough information is contained on the module and at the system’s slot to allow a user to easily match the module to an appropriate slot. A sans serif font is recommended for use on the labels because its characters are most easily recognizable.

A-0478

Optional ModuleLabel

Required ModuleLabel

Figure 8-10: Module Latch Labeling Locations

Figure 8-11: Required Module Label with Examples

Required: A label on the top surface of the latch which has four fields: A, B, C, and D. Note that the total label space available is 16.5 mm x 17.5 mm. The image is purely illustrative and not to scale. The size of the fields may be determined by the needs of the manufacturer. Refer to Figure 8-11.

Field A should show the highest bandwidth supported by the PCI Express module; for example, x8. It is recommended that the text be 14 point and have a 10:1 contrast ratio with the background. It is recommended that the background be black with white text.

Field B should list the interoperable bandwidths that are supported in descending order; for example, x4, x2, x1. It is recommended that text be 9 point and have a 10:1 contrast ratio with the background. It is recognized that, on occasion, a smaller font may be needed to fit in the required information. The “/” are shown at 10 point. It is recommended that the background be black with white text.

A B C

D

16.5mm

17.5mm


142

Field C is used to indicate whether the PCI Express module has the additional connectors on it for internal storage. This should be indicated by the bold capital letter S. It is recommended that text be a minimum of 10 point and that the background be black with white text.

Field D is where the type of module is indicated: for example, Ethernet. The method of designation may be by word(s), abbreviations, or icons, according to the conventions used by the system supplier. It is recommended that text be a minimum of 8 point. The color of the background is determined by the type of module: this is explained in Section 8.4.1 and Section 8.4.2.

8.4.1. Module Hot Remove and Add Capability

PCI Express modules that may be removed, replaced, or added without powering down the system should be clearly indicated as such. Because different companies use different colors to indicate hot swap capability, it is recommended that field D on the label have the hot swap color of each company as its background color or the appropriate hot swap icon or designator in the field. Clearly, depending on the background color, a 10:1 contrast ratio will not always be achievable in field D. Refer to Figure 8-11 where an example of burnt orange surrounds the Ethernet icon.

8.4.2. Modules That May Require a System Power Down

Modules that should not be removed, replaced, or added without first powering down the system, and modules that may only be removed in special circumstances, should also be clearly indicated. It is recommended that for these modules, field D is black and a small white triangle is added in the field. An example of special circumstances would be where two SAS modules are set up so that one is redundant to the other. If there is only one SAS module, then removing it may cause the system to go down. An example of the proceed-with-caution triangle location is provided in Figure 8-11.

8.4.3. Internal Storage Modules

Because internal storage modules have to go into dedicated slots, it is recommended that, for these modules, field D have a black background with white text. The text should indicate the storage type (for example, SAS or SATA) and may be accompanied by a supplier specific icon. Refer to Figure 8-11.

8.4.4. Optional Module Labeling

A second label space is provided on the right side of the plastic latch; its use is optional. It may be used for the manufacturer’s logo, it may be for the customer to write on (e.g., cable destination addresses), or it may be used to provide further information about the module. The size of optional module labels shall be determined by the supplier. Refer to Figure 8-10 for label location.

A third label space is provided on the top cover of the module.


143

8.4.5. Slot Labeling on the System

Figure 8-12: Required Slot Labeling

The layout of these labels will depend upon the space available on any particular chassis but should, as far as possible, mimic the layout on the module label described above. To this end, it is recommended that fields A, B, and C described above be repeated on the chassis for each slot, thereby allowing the user to match the bandwidth of the module to that of the slot and recognize slots that are designed for internal storage modules. It is recognized that these labels will probably be printed as one continuous strip for ease of manufacture. Note that the D field is not on this label.

8.4.6. Optional x16 Doublewide Slot Labeling

When the chassis design is such that two adjacent slots can accommodate either a doublewide x16 module or two singlewide x8 modules, this will be represented visually by a line connecting the respective “8’s” with a “16” in the middle. See Figure 8-12.

8.4.7. Slot Numbering and Labeling

8.4.7.1. Slot Numbering

Required: Slots shall be numbered on the system.

Slot numbering will be according to the convention used by each system supplier (either 0 to N or 1 to N).

Numbering will be on the chassis, and so located as to be visible to the user, and of sufficient point size and contrast as to be readable at the rear of a rack. The number may be on the slot label itself or as a separate label.

8.4.7.2. Vent Designation

In Section 8.1, potential cooling configurations are given where it is required that the vents on the I/O plate of the module be blocked. When I/O ventilation is needed, the module is open vented. When I/O ventilation cannot exist, then the module is referred to as close vented. The vast majority of ExpressModule applications will be open vented; air will either flow out of or into the system through the I/O bulkhead of the module. Systems that support only open vented modules are not required to notify the user. Open vented modules are considered the default configuration. In those cases where the system requires either closed vented modules or both closed and open

A B C

22mm Max System Specific


144

vented modules, the system manufacturer must notify the user. As these cooling scenarios can confuse the user, the graphics shown in Figure 8-13 and Figure 8-14 have been developed to promote consistency in the industry. These icons must be located in the proximity of the system’s ExpressModule slots. The minimum viewable size of the icon is 10 mm by 10 mm. The expectation is that most of the alternate cooling modes will require only the use of the closed vented module icon shown in Figure 8-14. The open module icon, Figure 8-13, is only required in those cases where a system supports both open and closed vented modules. An example of both types of modules being used is where there are modules located on the front and rear of a cabinet and the front inlet has a filter. Open modules in front would bypass the filter. Providing filters for each module would not be practical as different modules have unique I/O cable connections. The rear modules would be open vented and used for the exhaust air.

Figure 8-13: Open Vent Symbol

Figure 8-14: Closed Vent Symbol

PCI Express ExpressModule Electromechanical Specification...

Documents

Transcript of PCI Express ExpressModule Electromechanical Specification...