Powering Next-Generation IP Broadcasting using QFX Series ... · of sending the sync message. Delay...

Powering Next-Generation IP Broadcasting using QFX Series Switches

Tech Note

March 2018

Powering Next Generation IP Broadcasting using QFX Series Switches

2 © 2018 Juniper Networks, Inc.

Juniper Networks, Inc. 1133 Innovation Way Sunnyvale, California 94089 USA 408-745-2000 www.juniper.net Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper Networks reserves the right to change, modify, transfer, or otherwise revise this publication without notice. The information in this document is current as of the date on the title page. Copyright © 2018, Juniper Networks, Inc. All rights reserved.


© 2018 Juniper Networks, Inc. 3

Contents Introduction .................................................................................................................................................. 5

What is PTP? ................................................................................................................................................. 5

PTP Clock Types ......................................................................................................................................... 6

PTP Message Types ................................................................................................................................... 7

PTP Message Exchange ............................................................................................................................. 7

PTP Synchronization Process .................................................................................................................... 8

IP in Live Production Broadcast TV Environments ........................................................................................ 9

Typical Network Deployment ................................................................................................................. 11

Simple Star .......................................................................................................................................... 11

Hub and Spoke .................................................................................................................................... 12

Scale-Out using a Spine and Leaf Architecture ................................................................................... 13

Juniper QFX Series Switches – Scaling PTP as a Data Center Service ......................................................... 13

Supported Switches – Spine.................................................................................................................... 13

Supported Switches – Access/Leaf ......................................................................................................... 14

Typical Deployment ................................................................................................................................ 14

Configuration Commands ....................................................................................................................... 15

Operational Commands .......................................................................................................................... 15

Configuration Example ................................................................................................................................ 16

PTP Parameters ....................................................................................................................................... 17

CLI Configuration ..................................................................................................................................... 17

Management Interface ....................................................................................................................... 17

Spine-Leaf Interface ............................................................................................................................ 17

End Device-Facing Interface ................................................................................................................ 17

PTP Parameters ................................................................................................................................... 17

PTP Master-Slave Interfaces ............................................................................................................... 18

OSPF .................................................................................................................................................... 18

Multicast (PIM).................................................................................................................................... 18

Verification .............................................................................................................................................. 18

PTP Status ........................................................................................................................................... 18

PTP Synchronization ............................................................................................................................ 19



PTP Hierarchy ...................................................................................................................................... 19

Clock Status on QFX5110 .................................................................................................................... 20

Conclusion ................................................................................................................................................... 20



Introduction The IEEE 1588v2 standard defines the Precision Time Protocol (PTP), which is used to synchronize clocks throughout a packet-switched network, including Ethernet switches and IP routers. Precision timing is critical in environments where there is a desire to correlate or synchronize events within microseconds, or measure utilization or latency with the highest accuracy. Popular applications for PTP are often found in the financial sector and in the media broadcasting space.

Recent trends indicate that the media and broadcast industry, in particular, is moving towards creating IP-based studios where 1588 can be used to synchronize multiple cameras with a common time code, as well as to lock video and audio devices to a uniform time base. Traditionally, to achieve the required reliability and low latency necessary for production-quality broadcasts, broadcasters have relied on serial digital interface (SDI) and coaxial cabling technology to transport live content from outside broadcasts to production studios. This landscape now requires letting go of static, hardwired networks, manually configured workflows, and SDI-centric infrastructures that are unable to scale to meet the advances in technology in video and audio. There is also a shift from using dedicated, specialized media processing hardware to software running on servers in a data center.

This document reviews the PTP protocol, which is at the backbone of this transition from SDI to IP, and ties it back to a media and broadcast use case deployment using Juniper’s data center portfolio of switches – namely the QFX Series.

What is PTP? PTP is used to synchronize the clock of a network client with a server. This synchronization is achieved through packets that are transmitted and received in a session between different PTP clocks. As defined in IEEE 1588v2, PTP is used to distribute system time of day (TOD) and clock frequency from a grandmaster clock to slave clocks within the same network and clock domain using multicast communications.

These clocks are organized into a master-slave synchronization hierarchy, with the grandmaster clock-- the clock at the top of the hierarchy--determining the reference time for the entire system.



Synchronization is achieved by exchanging PTP timing messages, with the members using the timing information to adjust their clocks to match the time of their master in the hierarchy.

PTP Clock Types

The type of PTP clock used depends on the function performed by the PTP node in the network:

• Boundary clock: A node with this clock type has multiple PTP ports in a domain and maintains the time scale used in the domain. A PTP node can be configured to act as a master clock on one PTP port while simultaneously acting as a slave on another port. In such a case, the boundary clock receives the time of day on the slave clock port and relays it as a reference to the master clock ports. It is also possible to operate different parts of the network in different PTP domains.

• Transparent clock: This type of clock measures the time taken for a PTP event message to transit the device and provides this information to clocks receiving this event message. In transparent E2E (End to End) mode, each clock in the path to the slave measures the residence time (RT) of a PTP packet in its queue and modifies the RT field in the PTP message to indicate packet delay. The slave receives the PTP message and uses the RT data to calculate and remove the jitter the network introduced, thereby maintaining lock with the master based on a consistent delay.

• Ordinary clock: A node with this clock type has a single PTP port in a domain and maintains the time scale used in the domain. It can be a master clock or a slave clock. For example, if the grandmaster clock is an ordinary clock, a PTP slave on the server will be an ordinary clock, too.



PTP Message Types

Each PTP message has a specific role to play in the handshake between the master and slave.

Message Value

Sync 0

Delay_Req 1

Follow_Up 8

Delay_Resp 9

Announce B

Signaling C

Management D

Reserved E-F

NOTE: Each of these message types has a specific format; more detail can be found in the IEEE1588v2 specification. Also, this table concentrates only on the end-to-end messages between the master and slave.

Sync messages are sent from the master to the slave. Sync messages convey the master time at the time of sending the sync message.

Delay request messages are identical to sync messages in their format. A delay request is sent by the slave clock and contains the slave time at the time of sending the message.

Follow-up messages are optional messages that are received by the slave after the first synchronous message. We will discuss their use in two-step clock mode in the next section.

Delay response messages are sent by the master in response to the delay requests sent by slaves. These contain the time at which the master received the delay request message.

Announce messages are the first message type that all PTP clocks listen for. Announce messages carry Best Master Clock Algorithm (BMCA) information – the clock characteristics of the source. The workings of BMCA are beyond the scope of this paper, but just know that it is an algorithm used to select the best clock quality in the network.

Signaling messages convey TLV information (Type, Length and Value) to the target slave ports.

Management messages are used to exchange messages between multiple clocks and the network nodes. It is sufficient to understand that these come into play in multiple-clock scenarios.

PTP Message Exchange

PTP master and slave devices exchange sync and delay request messages, timestamping the arrival and departure times of each message.



In the figure above:

• T1 - Master time at point of sending Sync message • T2 – Slave time at point of receiving Sync message • T3 – Slave time at point of sending Delay Request message • T4 – Master time at point of receiving Delay Request message

The message exchange conveys precise timing—T1, T2, T3, T4—from the master to the slave and vice versa. The slave uses T1, T2, T3, and T4 to calculate its offset and one-way delay (mean propagation delay) relative to the master, and then uses that information to synchronize the slave clock to the master.

The formula to calculate one-way delay is as follows:

Mean Propagation Delay = (T2-T1) + (T4-T3) / 2

Once the propagation delay is known, periodic clock offset corrections may be done using follow-up messages.

PTP Synchronization Process

Clock synchronization can be achieved across a network within both networking equipment and the attached devices with extremely precise clock accuracy. PTP relies on instrumenting the delay between the master and slave to correctly model the time offset across the network. The less variability in the delay, the more accurate the synchronization will be maintained.

The PTP synchronization process consists of two phases: establishing the master-slave hierarchy, and synchronizing the clocks.

The following are the steps to attain a master-slave hierarchy within a network deployment:



1. An interface on which clock information is distributed downstream is configured as a master interface; an interface on which the master clock is to be received is configured as a slave interface.

2. Configure the clock step that determines whether the timing information is sent only with the synchronous message (one-step), or a subsequent follow-up message (two-step) is received for the sent synchronous message.

• One-step—One clock step to send timing information along with the synchronous message.

• Two-step—Two clock steps to send timing information and receive a subsequent follow-up message. I.e., the timestamp in the sync message is set to zero and the actual timestamp is sent afterwards in a follow-up message.

3. Configure a domain of communication. PTP operates within a logical scope called a PTP domain. 4. Configure the attributes of master-slave communication, such as frequency of sync, announce

messages, etc.

The following are the steps to attain a PTP lock synchronization within a network deployment:

1. Each port of a network switch or router examines the contents of all received announce messages issued by a GM clock or ports of other switches that are in the master state.

2. The priority, clock class, accuracy, and strength of the local clock is compared to that of the foreign clock master received through its slave ports.

3. After steps 1-2, the master-slave hierarchy is established. The master then sends a synchronization message to the slave. This is followed by the slave reception of the same.

4. The last step in the process is the exchange of a delay request message from slave to the master. The master then responds in kind with a delay response.

IP in Live Production Broadcast TV Environments The transition from SDI (Serial digital interface) to IP infrastructures will be much like other technology advances – slow and steady. Broadcasters and production facilities that have significant investments in large-scale SDI plants are going to face technical and budgetary challenges as they transition to newer IP-based topologies.

However, for many years now, broadcast and video production professionals have been seeking ways to create effective and efficient production environments using the latest technology, while providing the flexibility to support new technologies in the future. The advantage with IP is that even for a complicated studio environment like the one shown below, IP networking provides the necessary performance at equal or lower cost than SDI.



IP also provides flexibility to accommodate upcoming Broadcast TV formats of various bit-rates (e.g. 4K/Ultra High Definition Video (UHDV), 8K with/without High Dynamic Range (HDR), High Frame Rate (HFR), and other technologies). In the next two or three years, an increasing number of higher-resolution content will be produced for broadcast, forcing broadcasters to embrace this change to accommodate 4K and HDR with an upgraded infrastructure — network, storage, and media workflow applications. Hence, IP provides investment protection and the necessary adaptability for studios and broadcast.

The figure below shows the typical workflow in a studio with an upgraded network infrastructure.



Typical Network Deployment

Before getting into the specifics of how to configure a particular example in a broadcast environment, let’s look at some typical network deployment options for a broadcast TV environment.

Simple Star A star is the simplest of network deployments. With this network topology the network node is at the center, acting as a single information source and communicating with the media devices connected to it. The main disadvantage of a star topology is that the central network device exposes a single point of failure; if it fails, every attached end device is out of service. Also, this topology will have scalability issues as the studio expands.



Hub and Spoke This topology allows a network to expand dynamically with only one active data path between any two network endpoints. Let us consider the central node to be the broadcaster and the spanned-out nodes to be streaming nodes. One potential use case could be that a video stream is pushed along defined routes from the broadcaster to the streaming nodes.



Scale-Out using a Spine and Leaf Architecture This topology is one of the most popular ones in modern IP studios. Single- and multi-spine deployments give broadcasters the necessary flexibility and resiliency for medium to large scale deployments. Media sources, controllers and receivers connect to the leaf switches, which in turn connect to spine switches.

Juniper QFX Series Switches – Scaling PTP as a Data Center Service Juniper QFX Series switches meet the needs of today’s most demanding timing environments—media and broadcast environments. QFX10000 switches can act as both boundary clocks and transparent clocks, supporting multiple PTP connections while acting as source and client using BMCA. These data center switches also provide flexibility to deploy clocking in different layers of the network.

Supported Switches – Spine

The following switches are ideal as spine devices:

• QFX10002-60C o 2U fixed configuration o 60x100G o Throughput = 6 Tbps

• QFX10002-72Q o 2U fixed configuration o 288x10G or 72x40G or 24x100G o Throughput = 5.76 Tbps



• QFX10002-36Q o 2U fixed configuration o 144x10G or 36x40G or 12x100G o Throughput = 2.88 Tbps

• QFX5200-32C o 1U fixed configuration o 32x100G or 32x40G or 128x10G o Throughput = 3.2 Tbps

All supported spine switches can act as boundary clocks in all deployment topologies specified in the previous section.

Supported Switches – Access/Leaf

QFX5110/QFX5200 devices can act as leaf layer top-of-rack switches for the transparent clock, and if needed, for the boundary clock as well.

• QFX 5110-48S o 1U fixed configuration or line card on Virtual Chassis o 48x10G or 48x1G ports with 40G/100G uplink ports o Throughput = 1.76 Tbps

• QFX5200-48Y o 1U fixed configuration o 48x25G 1U switch with 6x100G uplink ports o Throughput = 3.6 Tbps

• QFX5200-32C o 1U fixed configuration o 32x100G or 32x40G or 128x10G o Throughput = 3.2 Tbps

Typical Deployment

The figure below shows a typical real-world network deployment. The spine devices act as the L2/L3 aggregation switches, operating in boundary clock mode. These switches are generally connected to the PTP clock network. The clock network consists of the grandmaster clocks, which are synchronized through GPS.

The grandmaster clocks connect to slave ports on the boundary clock switches. The boundary clock switches in turn act as the master driving a network of access/leaf switches, forming the master-slave hierarchy discussed in earlier sections. The access switches connect to the PTP clients (media devices).



We will use a much simpler deployment in our configuration example (below). The takeaway, however, is to note that the implementation can easily be scaled up as needed.

Configuration Commands

This section identifies the configuration commands that you will need to know in order to configure PTP on a QFX Series switch.

Two important attributes must be configured to operate PTP over Ethernet in a hybrid mode of operation:

set interfaces - This is the configuration hierarchy for all PTP-related interface configurations.

set protocols ptp - This is the configuration hierarchy for PTP and hybrid-related configurations.

Operational Commands

This section identifies the operational commands that you will need to know in order to monitor PTP on a QFX Series switch.

show ptp lock-status detail - Used to check the PTP clock recovery status from the upstream master. States include “Free-run”, “Holdover”, “Acquiring”, “Phase-aligned”. The output also provides information on how long the system/node has been in this state, locked to which master, using which interface, and what is the offset from the master.

show ptp statistics detail - Provides PTP packet statistics per clock stream (client and master) with explicit counters for sync, delay-request, delay-response and announce packets.



show ptp clock - Provides detail about parent node (upstream master) and grandmaster information. This includes, clock-class (Receive and Transmit), clock accuracy, P1 and P2 priorities, clock identity etc.

show ptp slave detail - Provides the status of all configured slave (upstream master) ports on this router/device.

show ptp port - Provides per stream (master and slave ports) status including the rate of packet exchange, port state, associated interface information and other end connection.

show ptp master detail - Provides the status of all configured master (clients) ports on this router/device.

Configuration Example This example uses the spine and leaf scale-out design as the network topology. The end devices could be any chosen media device, such as audio devices, controllers, cameras, synchronizers, video servers, multimedia viewers, video production switches, etc.

For this example, we are going to configure the QFX10002 and QFX5200 in boundary clock mode, and the QFX5110 in transparent mode. Both spine and leaf devices will run simple OSPF and PIM. Note that no ACLs or special filters are configured, although these may be necessary in some deployments.



PTP Parameters

• Domain 80 • Sync -3 (8Hz) • Announce 0 (1Hz) • Announce timeout 3 • Communication mode multicast

CLI Configuration

This section shows how to configure the scenario shown in the figure above. We will concentrate primarily on the interface setup and PTP aspects of the configuration. Note that the configurations for the slave devices are largely identical, hence it is implied that a similar configuration is employed on both the QFX5200 and QFX5110 devices

Management Interface Configure the management interface for the device and specify a default route. admin@QFX5200# set interfaces vme unit 0 family inet address 192.168.131.40/23 admin@QFX5200# set routing-options static route 0.0.0.0/0 next-hop 192.168.130.1

Spine-Leaf Interface Configure point-to-point links from the leaf devices to the spine device. admin@QFX5200# set chassis fpc 0 pic 0 port 31 speed 100g

admin@QFX5200# set interfaces et-0/0/31 description To_QFX10kSpine55_100G

admin@QFX5200# set interfaces et-0/0/31 unit 0 family inet address 192.168.99.42/31

End Device-Facing Interface Configure an interface to connect one of the leaf devices to the video server. admin@QFX5200# set interfaces xe-0/0/0:0 unit 0 family inet address 192.168.150.13/30

admin@QFX5200# set interfaces xe-0/0/0:0 description VideoServer_1

PTP Parameters Configure the PTP parameters on the spine device. The commands below configure the mode of operation of the switch, the PTP domain, and the sync/announce message exchange parameters.

A popular profile in media and broadcast is SMPTE, an industry standard which defines a set of cooperating standards used to label packet frames in media with a timestamp. QFX Series products are SMPTE-ready for the 2022-6, 2022-7, and 2110 set of standards. admin@QFX10002# set protocols ptp clock-mode boundary

admin@QFX10002# set protocols ptp profile-type smpte

admin@QFX10002# set protocols ptp domain 80

admin@QFX10002# set protocols ptp master announce-interval 0

admin@QFX10002# set protocols ptp master sync-interval -3

Note: Other profile types include 1588, G8275, etc.

Configure the QFX5110 in transparent clock mode. admin@QFX5110# set protocols ptp e2e-transparent



PTP Master-Slave Interfaces Configure the PTP master and slave interfaces on the leaf devices. The port facing the spine is configured as a slave and the port facing the end device is configured as a master. admin@QFX5200# set protocols ptp slave interface et-0/0/31 multicast-mode transport ipv4

admin@QFX5200# set protocols ptp slave interface et-0/0/31 multicast-mode local-ip-address 192.168.99.42

admin@QFX5200# set protocols ptp master interface xe-0/0/0:0.0 multicast-mode transport ipv4

admin@QFX5200# set protocols ptp master interface xe-0/0/0:0.0 multicast-mode local-ip-address 192.168.150.13

A similar configuration is performed on the QFX10002 spine, with the port connecting to the Meinberg GM configured as a slave and the port facing the leaf configured as a master.

OSPF Configure the leaf and spine devices to share routing information using OSPF.

QFX 5200: admin@QFX5200# set protocols ospf area 0.0.0.0 interface et-0/0/31.0



admin@QFX10002# set protocols ospf area 0.0.0.0 interface et-0/0/59.0

Multicast (PIM) Configure PIM on the spine device, including any media streams. A similar configuration should also be added on the leaf devices. admin@QFX10002# set protocols pim interface et-0/0/55.0

admin@QFX10002# set protocols pim interface et-0/0/59.0

admin@QFX10002# set protocols pim rp local group-ranges 239.120.0.0/16






Verification

With the configuration complete, it is time to verify that PTP is working. To reiterate, PTP status should be similar on all three devices, other than the QFX5110 having a slightly different status report due to using transparent clock mode.

PTP Status Review the PTP statistics output, which shows the sync, announce, and delay message exchanges between the grandmaster and the QFX10002 (which is in the role of slave). Note also the similar message exchange on the master port connected to the QFX 5200-32C.



admin@QFX10002> show ptp statistics detail Local Clock Remote Clock Role Stream Received Transmitted et-0/0/0.0 224.0.1.129 Slave 2 4889 2342

Signalling Announce Sync Delay Error Rx: 0 295 2341 2253 0 Tx: 0 0 0 2342 0

Local Clock Remote Clock Role Stream Received Transmitted et-0/0/55.0 224.0.1.129 Master 4 0 1114

Signalling Announce Sync Delay Error Rx: 0 0 0 0 0 Tx: 0 12392 99080 0 0 Note: the remote clock address is a reserved default multicast address for PTP messaging.

PTP Synchronization Next, we observe the iterative process of PTP synchronization in the QFX10002.

The switch goes from a free running clock to ACQUIRING, where it starts sending and receiving messages with the grandmaster. admin@QFX10002> show ptp lock-status Lock Status:

Lock State : 3 (ACQUIRING)

Phase offset : -0.000001490 sec

Moments later, we observe that the QFX10002 is lock-step with the grandmaster. At this point, the device is ready to relay the same timing information to downstream network nodes. admin@QFX10002> show ptp lock-status detail

Lock Status:

Lock State : 5 (PHASE ALIGNED)

Phase offset : -0.000000202 sec

Selected Master Details:

Upstream Master address : 224.0.1.129

Slave interface : et-0/0/0.0

Parent Id : 74:83:ef:ff:ff:19:5a:37

GMC Id : 74:83:ef:ff:ff:19:5a:37

PTP Hierarchy Now, verify the PTP hierarchy on the leaf switch.

The downstream interface should appear as a master. admin@QFX5200> show ptp master detail

PTP Master Interface Details:

Interface : et-0/0/0:31.0



Status : Master, Active

Clock Info :

Local Address: 192.168.99.42 Status: Configured, Master, Active

Remote Address: 224.0.1.12 Status: Configured, Slave, Active

Total Remote Slaves: 1

The upstream interface should appear as a slave. admin@QFX5200> show ptp slave detail

PTP Slave Interface Details:

Interface : xe-0/0/0.0:0

Status : Slave, Active

Clock Info

Local Address : 192.168.150.13 Status: Configured, Slave, Active

Remote Master: 224.0.1.129 Status: Configured, Master, Active

Total Remote Masters: 1

Clock Status on QFX5110 Finally, verify the transparent clock status on the QFX5110. admin@QFX5110> show ptp global-information

PTP Global Configuration:

Transparent-clock-config : ENABLED

Transparent-clock-status : ACTIVE

Conclusion In this document, we have established some basic PTP concepts, assessed the direction of the media and broadcast industries, and configured a small deployment using Juniper’s QFX Series data center switches. QFX switches are the ideal partner for the industry’s upcoming migration from SDI to IP. QFX switches replace specialized hardware (SDI routers) with an extensive generic switch portfolio that includes port speed combinations from 1G to 10G, 25G, 40G, 100G, and beyond. QFX switches provide investment protection for future upgrades and the essential scalability aspect that was missing with SDI: simply add an additional leaf switch to support more end devices. The spine and leaf scale-out architecture provides resiliency and bandwidth, and QFX switches’ superior timing capabilities will help create a lock-step network with deterministic latency and flexible design. All QFX Series devices are SMPTE-ready, to provide live broadcast services for the next generation of studio IP networks.

Powering Next-Generation IP Broadcasting using QFX Series ... · of sending the sync message. Delay...

Documents

Transcript of Powering Next-Generation IP Broadcasting using QFX Series ... · of sending the sync message. Delay...