Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

An introduction to PCI Express

Proma Goswami

August 19, 2014

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

Table of contents

1 Introduction to PCI ExpressCharecteristics of PCIE

2 Comparison with different version of PCIEComparisonOperation

3 PCIE Protocol DetailsProtocolPCIE Link

4 Pci DMA Controller

5 References

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

Charecteristics of PCIE

PCI Express

PCI Express (Peripheral Component Interconnect Express),officially abbreviated as PCIe, is a high-speed serial computerexpansion bus standard designed to replace the older PCI, PCI-X,and AGP bus standards.

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

Charecteristics of PCIE

PCI Express is a point to point connection ,i.e, it connectsonly two devices no other device can share the connection.

PCIE is based on high speed serial communication.

PCIE is based on individual lanes which can be grouped tocreate higher bandwidth connections.

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

Comparison

Comparison with different version of PCIE

Slot Clock(GHz) No of Bits Data/ Cycle Bandwidth

PCIe 1.0 x1 2.5 1 1 250 MB/s

PCIe 1.0 x4 2.5 4 1 1000 MB/s

PCIe 1.0 x8 2.5 8 1 2000 MB/s

PCIe 1.0 x16 2.5 16 1 4000 MB/s

PCIe 2.0 x1 2.5 1 1 500 MB/s

PCIe 2.0 x4 2.5 4 1 2000 MB/s

PCIe 2.0 x8 2.5 8 1 4000 MB/s

PCIe 2.0 x16 2.5 16 1 8000 MB/s

PCIe 3.0 x1 2.5 1 1 1000 MB/s

PCIe 3.0 x4 2.5 4 1 4000 MB/s

PCIe 3.0 x8 2.5 8 1 8000 MB/s

PCIe 3.0 x16 2.5 16 1 16000 MB/s

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

Comparison

Operation Modes

A connection between two a PCIe device and a PCIe switch iscalled a link. Each link is composed of one or more lanes, and eachlane is capable of transmitting one byte at a time in both directionsat once. This full-duplex communication is possible because eachlane is itself composed of one pair of signals: send and receive.Sothis lane is single bit ,full duplex,high speed serial communication.

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

Comparison

In order to transmit PCIe packets, which are composed of multiplebytes, a one-lane link must break down each packet into a series ofbytes, and then transmit the bytes in rapid succession. The deviceon the receiving end must collect all of the bytes and thenreassemble them into a complete packet. This disassembly andreassembly happens must happen rapidly enough to where it’stransparent to the next layer up in the stack. This means that itrequires some processing power on each end of the link. This isbecause each lane is only one byte wide, very few pins are neededto transmit the data.Actually serial transmission scheme is a wayof turning processing power into bandwidth. One of PCIe’sfeatures is the ability to aggregate multiple individual lanestogether to form a single link. In other words, two lanes could becoupled together to form a single link capable of transmitting twobytes at a time, thus doubling the link bandwidth.

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

Comparison

Switch

Lane (x1)

Switch

Lane(x3)

Figure : PCIE connectionProma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

ProtocolPCIE Link

Protocol

Till now we were concerned with the system level impact of PCIe.We did not look at the protocol itself. The following material willmake an attempt to explain the details of PCIe protocol, its layersand the functions of each of the layers in a brief way.PCI Express is a high performance, general purpose I/Ointerconnect defined for a wide variety of future computing andcommunication platforms.

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

ProtocolPCIE Link

PCIE Link

A Link represents a dual-simplex communications channel betweentwo components. The fundamental PCI Express Link consists oftwo, low-voltage, differentially driven signal pairs: a Transmit pairand a Receive pair.

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

ProtocolPCIE Link

PCIE Fabric Topology

CPU

RootPCI Express Endpoint

Memory

PCI Express

Switch

PCI Express to PCI/PCI-x Bridge

PCI ExpressPCI Express

Legacy Endpoint Legacy Endpoint Legacy Endpoint Legacy Endpoint

PCI ExpressPCI Express

PCI Express PCI Express

PCI /PCI x

Figure : PCIE Fabric Topology

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

ProtocolPCIE Link

PCIE Fabric Topology

Root ComplexA Root Complex (RC) denotes the root of an I/O hierarchythat connects the CPU/memory subsystem to the I/O.EndpointsEndpoint refers to a type of Function that can be theRequester or Completer of a PCI Express transaction either onits own behalf or on behalf of a distinct non-PCI Expressdevice (other than a PCI device or Host CPU), e.g., a PCIExpress attached graphics controller or a PCI Express-USBhost controller. Endpoints are classified as either legacy, PCIExpress, or Root Complex Integrated Endpoints.PCI Express to PCI/PCI-X BridgeA PCI Express to PCI/PCI-X Bridge provides a connectionbetween a PCI Express fabric and a PCI/PCI-X hierarchy.

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

ProtocolPCIE Link

PCI Express Layering Overview

PCI Express can be divided into three discrete logical layers: theTransaction Layer, the Data Link Layer, and the Physical Layer.Each of these layers is divided into two sections: one thatprocesses outbound (to be transmitted) information and one thatprocesses inbound (received) information.

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

ProtocolPCIE Link

PCI Express Layering

PCI Express uses packets to communicate information betweencomponents. Packets are formed in the Transaction and Data LinkLayers to carry the information from the transmitting componentto the receiving component. As the transmitted packets flowthrough the other layers, they are extended with additionalinformation necessary to handle packets at those layers. At thereceiving side the reverse process occurs and packets gettransformed from their Physical Layer representation to the DataLink Layer representation and finally (for Transaction LayerPackets) to the form that can be processed by the TransactionLayer of the receiving device. Figure below shows the conceptualflow of transaction level packet information through the layers.

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

ProtocolPCIE Link

PCI Express Layering

Transaction

Data Link

Logical Sub Block

Electric Sub Block

Physical

RX Tx

Data Link

Transaction

Logical Sub Block

Electric Sub Block

Physical

Rx Tx

Figure : PCIE LayeringProma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

ProtocolPCIE Link

Layers of the protocol

Transaction LayerThis is the top layer that interacts with the software above.Functions of Transaction Layer:

Mechanisms for differentiating the ordering and processingrequirements of Transaction Layer Packets (TLPs)

Credit-based flow control

TLP construction and processing

Association of transaction-level mechanisms with deviceresources including Flow Control and Virtual Channelmanagement

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

ProtocolPCIE Link

Layers of the protocol

Data Link LayerThe Data Link Layer acts as an intermediate stage between theTransaction Layer and the Physical Layer. Its primary responsibilityis to provide a reliable mechanism for exchanging TransactionLayer Packets (TLPs) between the two components on a Link.Functions of Data Link Layer:

Data Exchange:

Error Detection and Retry:

Initialization and power management:

5.4.3 Physical Layer The Physical Layer isolates the Transactionand Data Link Layers from the signaling technology used for Linkdata interchange. The Physical Layer is divided into the logicaland electrical subblocks.

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

ProtocolPCIE Link

Layers of the protocol

Logical Sub-block Takes care of Symbol Encoding, framing, datascrambling, Link initialization and training, Lane to lane de-skew

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

ProtocolPCIE Link

Layers of the protocol

Electrical Sub-block The electrical sub-block section defines thephysical layer of PCI Express 5.0 GT/s that consists of a referenceclock source, Transmitter, channel, and Receiver. This sectiondefines the electrical-layer parameters required to guaranteeinteroperability among the above-listed PCI Express components.This section comprehends both 2.5 GT/s and 5.0 GT/s electricals.In many cases the parameter definitions between 2.5 and 5.0 GT/sare identical, even though their respective values may differ.However, the need at 5.0 GT/s to minimize guardbanding, whilesimultaneously comprehending all phenomena affecting signalintegrity, requires that all the PCI Express system components -Transmitter, Receiver, channel, and Refclk, be explicitly defined inthe specification. For this reason, each of these four componentshas a separate specification section for 5.0 GT/s.

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

Impact of PCIE DMA performance

PCIE architecture has no central direct memory access controller.

PCI-Express Protocol Overhead

Maximum Payload Size, i.e. the maximum allowed size of asingle packet

Signal Integrity on the link, causing retransmission of packets(packet replays)

Poor flow control update from the host computer system,causing long stall times (i.e. times where the endpoint is notallowed to send packets

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

PCI Express DMA performance MPS

MPS defines how many bytes of user data (= payload) iscontained in a PCI-Express data packet.

The higher this value is, the less is the protocol overhead,since Packet header and Packet Footer remain the same.

The actual MPS value is negotiated during link training.Since FPGA endpoints can implement big MPS values, thehost chipset determines the MPS value

Proma Goswami An introduction to PCI Express

Introduction to PCI ExpressComparison with different version of PCIE

PCIE Protocol DetailsPci DMA Controller

References

References

References

www.arstechnica.com

www.pcisig.com

www.pcstats.com

www.digitalhomedesignline.com

Proma Goswami An introduction to PCI Express