Server Time Protocol Implementation Considerations

102
© 2010 IBM Corporation Server Time Protocol Implementation Considerations

Transcript of Server Time Protocol Implementation Considerations

Page 1: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation

Server Time Protocol Implementation Considerations

Page 2: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation2

Agenda

General HMC Panel Overview

Setting up a Coordinated Timing Network (CTN)– Configuring an External Time Source

• Configure Dial Out on HMC• Configure an NTP Server

– Migration from ETR to STP-only CTN– Configuring an STP-only CTN

Restoring STP configuration information – Power outage affecting both servers of two server CTN– Across a Power on Reset for single server CTN

Additional Information

Page 3: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation3

System (Sysplex) Time taskTiming Network tab

Page 4: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation4

General HMC Panel Overview

ETR Feature Installed ETR Feature NOT Installed

STP Feature Installed

STP Feature NOT Installed

STP Feature Installed

STP Feature NOT Installed

ETR Configuration

System (Sysplex) Time

task is not available

ETR Status

Timing Network

Network Configuration

STP Configuration

STP Status

ETS Configuration

Page 5: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation5

Agenda

General HMC Panel Overview

Setting up a Coordinated Timing Network (CTN)– Configuring an External Time Source

• Configure Dial Out on HMC• Configure an NTP Server

– Migration from ETR to STP-only CTN– Configuring an STP-only CTN

Restoring STP configuration information – Power outage affecting both servers of two server CTN– Across a Power on Reset for single server CTN

Additional Information

Page 6: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation6

Configuring an External Time Source (ETS)

The ETS can be configured as:

– Dial out to a telephone time service• NIST Automated Computer Time Service (ACTS)• NRC Canadian Time Service (CTS)• IEN Telephone Date Code (CTD)

– NTP server

– NTP server with pulse per second (PPS) output

Page 7: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation7

Agenda

General HMC Panel Overview

Setting up a Coordinated Timing Network (CTN)– Configuring an External Time Source

• Configure Dial Out on HMC• Configure an NTP Server

– Migration from ETR to STP-only CTN– Configuring an STP-only CTN

Restoring STP configuration information – Power outage affecting both servers of two server CTN– Across a Power on Reset for single server CTN

Additional Information

Page 8: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation8

Configuring Dial out as ETS

Steps include:

– Configure an HMC to dial out to a telephone time service. At the Customize Outbound Connectivity task on the HMC, check both:• Enable local system as a call-home server• Allow external time source dialing using the local

modem

– Add scheduled operations to the Preferred Time Server (PTS) and the Backup Time Server (BTS), if applicable, to access the ETS.

Page 9: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation9

HMC Setup for ETS Dial out configuration

Customize Outbound Connectivity:

External Time Source

Page 10: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation10

HMC Setup for ETS Dial out configuration

Test External Time Source Phone number

Page 11: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation11

Configure ETS Dial Out Schedule

Need to set up for CTS

Recommended to set up for BTS also

These panels are specific to the Support Element (SE) that you are setting up for

Page 12: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation12

Setup Scheduled Operation/Repeat

Page 13: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation13

Agenda

General HMC Panel Overview

Setting up a Coordinated Timing Network (CTN)– Configuring an External Time Source

• Configure Dial Out on HMC• Configure an NTP Server

– Migration from ETR to STP-only CTN– Configuring an STP-only CTN

Restoring STP configuration information – Power outage affecting both servers of two server CTN– Across a Power on Reset for single server CTN

Additional Information

Page 14: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation14

NTP Server on HMC as ETSnon-S ystem ztim e synchron izedservers

P TS / C TS

S ystem z H M C N TP server enab ledS tratum 2 o r above

In te rne t

July 14 14:21 :00 2007 U TC

N TP serverS tra tum 1

In tranet

S N TPc lient

E therne tS w itch

eth0

eth 1

10.10.10.53

us .poo l.n tp .o rg

10.1 .6 .48

N TP serverS tra tum 1

F1

F2

Page 15: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation15

Configuring an NTP Server on HMCCustomize Console Date and Time – initial view

Page 16: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation16

Configuring an NTP Server on HMCAdd a corporate NTP server

The server host name or the IP address of a known NTP server must be entered in this window.

This is the IP address of a NTP server on the corporate network

Page 17: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation17

Configuring an NTP Server on HMCEnable NTP service on this console

To turn on the NTP service, check the “Enable NTP service on this console” checkbox and click OK.

When you click OK, you should be able to see the console clock at the bottom right corner get updated to the server's time.

Even though the time change is instantaneous, it still takes time for the console's NTP code to ensure that the HMC’s NTP server has started properly.

Page 18: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation18

Configuring an NTP Server on HMCEnable this Hardware Management Console to act as a time server

To allow another HMC console, TKE console, or an STP-only CTN to use the HMC as a NTP server, check the “Enable this Hardware Management Console to act as a time server” checkbox and click the OK button.

Page 19: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation19

Configuring an NTP server on the PTS/CTS or BTS

This is the IP address of the HMC NTP Server residing on the private network.

It is different from the IP address of the NTP server on the corporate network.

Page 20: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation20

Agenda

General HMC Panel Overview

Setting up a Coordinated Timing Network (CTN)– Configuring an External Time Source

• Configure Dial Out on HMC• Configure an NTP Server

– Migration from ETR to STP-only CTN– Configuring an STP-only CTN

Restoring STP configuration information – Power outage affecting both servers of two server CTN– Across a Power on Reset for single server CTN

Additional Information

Page 21: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation21

Forward Migration

The following example of a forward migration is comprised of excerpts from

– System z Expo customer presentation, with the customer’s permission

– STP Implementation Guide (SG24-7281)

Some displays will not match the configuration diagrams. The examples shown were not meant to have continuity, they are used to show the logical migration flow for learning purposes.

DISCLAIMER

Page 22: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation22

Starting Environment

9037-002

ETR ID = 01 ETR1

9037-002

ETR ID = 01 ETR2

* DWDM not shownSite 1 and Site 2 Data Centers arelocated approx X km apart.

ETR • Blue Lines

STP (Coupling Links)• Solid Green Lines = ICB4+3• Dash Green Lines = ISC3* Link redundancy not shown

Site 2 Data Center *

Site 1 Data Center *

Z9BE - 2094

z/OS 1.8 SYSB

z/OS 1.8 SYSE

ETR Timing Mode

Z9G - 2096

z/OS 1.8 SYSG

ETR Timing Mode

Z9CD - 2094

z/OS 1.8 SYSC

z/OS 1.8 SYSD

CFCC CF14

ETR Timing Mode

Z9LPC4 - 2094

CFCC CF14

ETR Timing Mode

Z9OC - 2094

z/OS 1.8 SYSA

GDPS/CBU SYSK

CFCC CF14

ETR Timing Mode

Page 23: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation23

Preparatory Work

General Pre-Implementation Tasks– Hardware

• Install FC 1021 to STP-enable our processors• Check hardware and HMC driver levels• Installed 9037 LIC upgrade• Check STP website to ensure that DWDMs are qualified

– Software• All systems at a STP-supported z/OS release with latest RSUs applied.• Enhanced PSP Bucket (FUNCTION_STP).

– www14.software.ibm.com/webapp/set2/psp/srchBroker• CLOCKxx was updated (Note: These are the default values)

– STPMODE=YES– STPZONE=YES

• IPL all systems*.

* Images must be IPL’ed after installing new STP maintenance and STP FC 1021.

Page 24: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation24

Selection of Qualified DWDM Products

Selecting qualified WDM vendor does not mean that WDM model selected is qualified

– Vendors do not qualify all models

Selecting qualified WDM model does not mean that specific Release level is qualified

– Vendors do not qualify all release levels

Ensure that the vendor qualification letter is inspected carefully by a DWDM technical expert.

– Letter specifies model number, release level, interface modules qualified, protocols, application limitations, etc

Vendors to date having qualified DWDM models:– Adva Optical Networking– Ciena Corporation– Cisco Systems– Ericsson AB– Huawei Technology– Lucent Technologies (qualified 2002 – not current technologies)– Nortel

The latest list of qualified DWDM vendor products can be found on the Resource Link™ Web site at:

https://www.ibm.com/servers/resourcelink/

They are listed under the Hardware Products for Servers on the Library page

Page 25: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation25

Implementation

Migrate ETR Network to Mixed CTN

– Set CTN ID on all servers

– Migrate first server to STP timing mode

– Migrate next two servers to STP timing mode

Migrate from Mixed to STP-Only CTN

Page 26: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation26

A T T E N T I O N !

Selecting the System (Sysplex) Time icon results in this panel

Do not be alarmed! This is normal

The user should read this panel – To be aware of what is

disruptive

– Answer Yes to get to the STP panels

Page 27: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation27

Set CTN ID

ETR MODE

Note: In ETR mode, the ETR Network ID cannot be changed.

Note: Enter STP Network ID here, then hit Apply.

Page 28: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation28

Timing Network - after assigning CTN ID to FIRST server

Mixed CTN

Page 29: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation29

STP Status - after assigning CTN ID to FIRST server

Page 30: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation30

STP Status - after assigning CTN ID to SECOND server

Page 31: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation31

SYSLOG Message

IXC438I COORDINATED TIMING INFORMATION HAS BEEN UPDATEDFOR SYSTEM: SYSAPREVIOUS ETR NETID: 01CURRENT CTNID: DDS_STP1-01

Mixed CTN

CTN ID! We are now in a Mixed CTN

Page 32: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation32

“D ETR” output, Before and AfterD ETRIEA282I 17.45.06 TIMING STATUSSYNCHRONIZATION MODE = ETR

CPC PORT 0 ACTIVE ==> CPC PORT 1OPERATIONAL OPERATIONALENABLED ENABLEDETR NET ID=01 ETR NET ID=01ETR PORT=05 ETR PORT=05ETR ID=01 ETR ID=02

D ETRIEA282I 17.46.55 TIMING STATUSSYNCHRONIZATION MODE = ETR

CPC PORT 0 ACTIVE ==> CPC PORT 1OPERATIONAL OPERATIONALENABLED ENABLEDETR NET ID=01 ETR NET ID=01ETR PORT=05 ETR PORT=05ETR ID=01 ETR ID=02THIS SERVER IS PART OF TIMING NETWORK DDS_STP1-01

CTN ID Displayed

Mixed CTN

BEFORE

AFTER

Page 33: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation33

“D XCF,S,ALL” output, Before and After

D XCF,S,ALLIXC335I 17.44.24 DISPLAY XCFSYSTEM TYPE SERIAL LPAR STATUS TIME SYSTEM STATUSSYSA 2094 XXXX 02 07/07/2008 17:44:21 ACTIVE TM=ETRSYSH 2094 XXXX 03 07/07/2008 17:44:23 ACTIVE TM=ETR

D XCF,S,ALLIXC335I 17.48.34 DISPLAY XCFSYSTEM TYPE SERIAL LPAR STATUS TIME SYSTEM STATUSSYSA 2094 XXXX 02 07/07/2008 17:48:34 ACTIVE TM=ETRSYSH 2094 XXXX 03 07/07/2008 17:48:33 ACTIVE TM=ETR

As Expected, Nothing Changed!

BEFORE

AFTER

Page 34: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation34

Environment – After CTN ID Set on ALL servers

9037-002

ETR ID = 01 ETR1

9037-002

ETR ID = 01 ETR2

* DWDM not shownSite 1 and Site 2 Data Centers arelocated approx X km apart.

ETR • Blue Lines

STP (Coupling Links)• Solid Green Lines = ICB4+3• Dash Green Lines = ISC3* Link redundancy not shown

Site 2 Data Center *

Site 1 Data Center *

Z9BE - 2094

z/OS 1.8 SYSB

z/OS 1.8 SYSE

ETR Timing Mode

Stratum 1, DDS_STP1-01

Z9G - 2096

z/OS 1.8 SYSG

ETR Timing Mode

Stratum 1, DDS_STP1- 01

Z9CD - 2094

z/OS 1.8 SYSC

z/OS 1.8 SYSD

CFCC CF15

ETR Timing Mode

Stratum 1, DDS_STP1- 01

Z9LPC4 - 2094

CFCC CF15

ETR Timing Mode

Stratum 1, DDS_STP1-01

Z9OC - 2094

z/OS 1.8 SYSA

GDPS/CBU SYSK

CFCC CF15

ETR Timing Mode

Stratum 1, DDS_STP1-01

Page 35: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation35

Implementation

Migrate ETR Network to Mixed CTN

– Set CTN ID on all servers

– Migrate first server to STP timing mode– Migrate next two servers to STP timing mode

Migrate from Mixed to STP-Only CTN

Page 36: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation36

Migrate to STP timing mode – STP Status tab VerificationBefore disabling ETR ports, attached systems connectivity must be verified from STP Status tab

Page 37: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation37

Migrate to STP timing mode – Display ETR Ports

Page 38: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation38

Migrate to STP timing mode – Disable ETR Ports

Click Port 1 State -> Disabled radio button, then Apply.

Repeat for Port 0.

Note: Ports can be disabled at the same time

Page 39: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation39

Migrate to STP timing mode – SYSLOG Messages

*IEA393I ETR PORT 1 IS NOT OPERATIONAL. THIS MAY BE A CTNCONFIGURATION CHANGE.

IEA390I TOD CLOCKS DYNAMICALLY ADJUSTED TO MAINTAIN STPSYNCHRONISM.

*IEA393I ETR PORT 0 IS NOT OPERATIONAL. THIS MAY BE A CTNCONFIGURATION CHANGE.

IEA380I THIS SYSTEM IS NOW OPERATING IN STP TIMING MODE.

All systems still alive, CPR (or resume) NOT Required! Mixed CTN

JOHN

Page 40: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation40

Migrate to STP timing mode – STP Status tab verificationAfter the ports have been disabled, the timing mode can be checked using the STP Status tab.

Stratum levelTiming mode

Page 41: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation41

Z9OC Hardware Messages

Do not be alarmed!These messages are normal when ETR ports are disabled.

Page 42: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation42

Z9OC Hardware Messages

Page 43: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation43

Z9OC Hardware Messages

Page 44: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation44

Migrate to STP timing mode – ETR Status After

Page 45: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation45

Z9OC D ETR Before and AfterD ETRIEA282I 07.05.37 TIMING STATUS 514SYNCHRONIZATION MODE = ETRCPC PORT 0 ACTIVE ==> CPC PORT 1OPERATIONAL OPERATIONALENABLED ENABLEDETR NET ID=01 ETR NET ID=01ETR PORT=10 ETR PORT=08ETR ID=01 ETR ID=02THIS SERVER IS PART OF TIMING NETWORK DDS_STP1-01

D ETRIEA386I 07.07.41 TIMING STATUSSYNCHRONIZATION MODE = STPTHIS SERVER IS A STRATUM 2CTN ID = DDS_STP1-1NUMBER OF USABLE TIMING LINKS = 9

Page 46: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation46

Z9OC D XCF Before and After

D XCF,S,ALLIXC335I 07.05.44 DISPLAY XCFSYSTEM TYPE SERIAL LPAR STATUS TIME SYSTEM STATUSSYSA 2094 XXXX 02 07/19/2008 07:05:43 ACTIVE TM=ETRSYSH 2094 XXXX 03 07/19/2008 07:05:41 ACTIVE TM=ETR

D XCF,S,ALLIXC335I 07.09.48 DISPLAY XCFSYSTEM TYPE SERIAL LPAR STATUS TIME SYSTEM STATUSSYSA 2094 XXXX 02 07/19/2008 07:09:47 ACTIVE TM=STPSYSH 2094 XXXX 03 07/19/2008 07:09:41 ACTIVE TM=STP

Page 47: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation47

Z9CD - 2094

z/OS 1.8 SYSC

z/OS 1.8 SYSD

CFCC CF15

ETR Timing Mode

Stratum 1, DDS_STP1- 01

Z9BE - 2094

z/OS 1.8 SYSB

z/OS 1.8 SYSE

ETR Timing Mode

Stratum 1, DDS_STP1-01

Z9OC - 2094

z/OS 1.8 SYSA

GDPS/CBU SYSK

CFCC CF15

STP Timing Mode

Stratum 2, DDS_STP1-01

9037-002

ETR ID = 01 ETR1

9037-002

ETR ID = 02 ETR2

Site 1 / Site 2 – Mixed CTNDiagram 1 (Z9OC to STP timing mode)

Z9LPC4 - 2094

CFCC CF15

ETR Timing Mode

Stratum 1, DDS_STP1-01

X X

ETR • Blue Lines

STP (Coupling Links)• Solid Green Lines = ICB4+3• Dash Green Lines = ISC3* Link redundancy not shown

Z9G - 2096

z/OS 1.8 SYSG

ETR Timing Mode

Stratum 1, DDS_STP1- 01

Site 2 Data Center *

Site 1 Data Center *

* DWDM not shownSite 1 and Site 2 Data Centers arelocated approx X km apart.

Page 48: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation48

Implementation

Migrate ETR Network to Mixed CTN

– Set CTN ID on all servers

– Migrate first server to STP timing mode

– Migrate next two servers to STP timing mode

Migrate from Mixed to STP-Only CTN

Page 49: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation49

Site 2 Data Center *

Site 1 Data Center * 9037-002

ETR ID = 01 ETR1

9037-002

ETR ID = 02 ETR2

Site 1 / Site 2 – Mixed CTNDiagram 2 (Z9BE and Z9G to STP timing mode)

X X

ETR • Blue Lines

STP (Coupling Links)• Solid Green Lines = ICB4+3• Dash Green Lines = ISC3* Link redundancy not shown * DWDM not shown

Site 1 and Site 2 Data Centers arelocated approx X km apart.

Z9BE - 2094

z/OS 1.8 SYSB

z/OS 1.8 SYSE

ETR Timing Mode

Stratum 2, DDS_STP1-01

Z9G - 2096

z/OS 1.8 SYSG

STP Timing Mode

Stratum 2, DDS_STP1- 01

Z9CD - 2094

z/OS 1.8 SYSC

z/OS 1.8 SYSD

CFCC CF15

ETR Timing Mode

Stratum 1,DDS_STP1-01

Z9LPC4 - 2094

CFCC CF15

ETR Timing Mode

Stratum 1,DDS_STP1- 01

Z9OC - 2094

z/OS 1.8 SYSA

GDPS/CBU SYSK

CFCC CF15

STP Timing Mode

Stratum 2, DDS_STP1-01

X

X

Page 50: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation50

Implementation

Migrate ETR Network to Mixed CTN

– Set CTN ID on all servers

– Migrate first server to STP timing mode

– Migrate next two servers to STP timing mode

Migrate from Mixed to STP-Only CTN

Page 51: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation51

Network Configuration (Where to go to Assign)

Important: Assignment of server roles must be done on the server that will become the Current Time Server for the STP-only CTN.

Page 52: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation52

Assign STP Roles

Page 53: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation53

Z9CD SYSLOG Messages – STP-Only CTN

IEA390I TOD CLOCKS DYNAMICALLY ADJUSTED TO MAINTAIN STP SYNCHRONISM. *IEA393I ETR PORT 0 IS NOT OPERATIONAL. THIS MAY BE A CTN CONFIGURATION CHANGE. *IEA393I ETR PORT 1 IS NOT OPERATIONAL. THIS MAY BE A CTN CONFIGURATION CHANGE.

IEA380I THIS SYSTEM IS NOW OPERATING IN STP TIMING MODE. IXC438I COORDINATED TIMING INFORMATION HAS BEEN UPDATED

FOR SYSTEM: SYSC PREVIOUS CTNID: DDS_STP1-01 CURRENT CTNID: DDS_STP1

Page 54: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation54

Z9CD SYSLOG Messages – STP-Only CTNAt 00:10:30.89*IXC439E ALL SYSTEMS IN SYSPLEX HQPLEX1 ARE NOT SYNCHRONIZED

TO THE SAME TIME REFERENCE. SYSTEM: SYSK IS USING CTNID: DDS_STP1 SYSTEM: SYSB IS USING CTNID: DDS_STP1 SYSTEM: SYSG IS USING CTNID: DDS_STP1 SYSTEM: SYSC IS USING CTNID: DDS_STP1-01 SYSTEM: SYSD IS USING CTNID: DDS_STP1-01 SYSTEM: SYSE IS USING CTNID: DDS_STP1

At 00:10:31.90IXC435I ALL SYSTEMS IN SYSPLEX HQPLEX1 ARE NOW SYNCHRONIZED

TO THE SAME TIME REFERENCE. SYSTEM: SYSK IS USING CTNID: DDS_STP1 SYSTEM: SYSB IS USING CTNID: DDS_STP1 SYSTEM: SYSG IS USING CTNID: DDS_STP1 SYSTEM: SYSC IS USING CTNID: DDS_STP1 SYSTEM: SYSD IS USING CTNID: DDS_STP1 SYSTEM: SYSE IS USING CTNID: DDS_STP1

AFTER

BEFORE

Page 55: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation55

Site 2 Data Center *

Site 1 Data Center * 9037-002

ETR ID = 01 ETR1

9037-002

ETR ID = 02 ETR2

Site 1 / Site 2 – Migrate to STP CTNDiagram 3 (STP LIC disables ETR ports on z9CD & Z9LPC4 when roles assigned)

X

X

ETR • Blue Lines

STP (Coupling Links)• Solid Green Lines = ICB4+3• Dash Green Lines = ISC3* Link redundancy not shown * DWDM not shown

Site 1 and Site 2 Data Centers arelocated approx X km apart.

Z9BE - 2094

z/OS 1.8 SYSB

z/OS 1.8 SYSE

STP Timing Mode

Stratum 2, DDS_STP1

Z9G - 2096

z/OS 1.8 SYSG

STP Timing Mode

Stratum 2, DDS_STP1

Z9CD - 2094

z/OS 1.8 SYSC

z/OS 1.8 SYSD

CFCC CF15

STP Timing Mode

Stratum 1,DDS_STP1

Z9LPC4 - 2094

CFCC CF15

STP Timing Mode

Stratum 2,DDS_STP1

Z9OC - 2094

z/OS 1.8 SYSA

GDPS/CBU SYSK

CFCC CF15

STP Timing Mode

Stratum 2, DDS_STP1

XX

PTS/CTS

BTS

Arbiter

Page 56: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation56

Z9CD “D ETR” output, Before and After

D ETR IEA386I 00.11.54 TIMING STATUS SYNCHRONIZATION MODE = STP THIS SERVER IS A STRATUM 1 CTN ID = DDS_STP1 THE STRATUM 1 NODE ID = 002094.S18.IBM.02.0000000XXXXX THIS IS THE PREFERRED TIME SERVER

D ETR IEA282I 00.03.46 TIMING STATUS SYNCHRONIZATION MODE = ETR CPC PORT 0 ACTIVE ==> CPC PORT 1 OPERATIONAL OPERATIONAL ENABLED ENABLED ETR NET ID=01 ETR NET ID=01 ETR PORT=08 ETR PORT=00 ETR ID=01 ETR ID=02 THIS SERVER IS PART OF TIMING NETWORK DDS_STP1-01

BEFORE

AFTER

Page 57: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation57

Z9CD “D XCF,S,ALL” Before and AfterD XCF,S,ALL IXC335I 00.04.18 DISPLAY XCF 976 SYSTEM TYPE SERIAL LPAR STATUS TIME SYSTEM STATUS SYSK 2094 XXXX 04 08/03/2008 00:04:14 ACTIVE TM=STPSYSB 2094 XXXX 01 08/03/2008 00:04:17 ACTIVE TM=STPSYSG 2096 XXXX 01 08/03/2008 00:04:17 ACTIVE TM=STPSYSC 2094 XXXX 01 08/03/2008 00:04:15 ACTIVE TM=ETRSYSD 2094 XXXX 02 08/03/2008 00:04:16 ACTIVE TM=ETRSYSE 2094 XXXX 02 08/03/2008 00:04:17 ACTIVE TM=STP

D XCF,S,ALL IXC335I 00.10.51 DISPLAY XCF 134 SYSTEM TYPE SERIAL LPAR STATUS TIME SYSTEM STATUS SYSK 2094 XXXX 04 08/03/2008 00:10:50 ACTIVE TM=STPSYSB 2094 XXXX 01 08/03/2008 00:10:50 ACTIVE TM=STPSYSG 2096 XXXX 01 08/03/2008 00:10:50 ACTIVE TM=STPSYSC 2094 XXXX 01 08/03/2008 00:10:50 ACTIVE TM=STPSYSD 2094 XXXX 02 08/03/2008 00:10:49 ACTIVE TM=STPSYSE 2094 XXXX 02 08/03/2008 00:10:51 ACTIVE TM=STP

BEFORE

AFTER

Page 58: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation58

Selected “RO *ALL,D ETR” outputZ9BE Systems RESPONSES -------------------------------SYNCHRONIZATION MODE = STP THIS SERVER IS A STRATUM 2 THIS IS THE ARBITER SERVER THE STRATUM 1 NODE ID = 002094.S18.IBM.02.0000000ZZZZZ

Z9CD Systems RESPONSES -------------------------------SYNCHRONIZATION MODE = STP THIS SERVER IS A STRATUM 1 THIS IS THE PREFERRED TIME SERVER THE STRATUM 1 NODE ID = 002094.S18.IBM.02.0000000ZZZZZ

Z9G Systems RESPONSES -------------------------------SYNCHRONIZATION MODE = STP THIS SERVER IS A STRATUM 2 THE STRATUM 1 NODE ID = 002094.S18.IBM.02.0000000ZZZZZ

Z9OC Systems RESPONSES -------------------------------SYNCHRONIZATION MODE = STP THIS SERVER IS A STRATUM 2 THIS IS THE BACKUP TIME SERVER THE STRATUM 1 NODE ID = 002094.S18.IBM.02.0000000ZZZZZ

Page 59: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation59

STP Status – after migrating to STP-Only CTN

Page 60: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation60

Site 2 Data Center *

Site 1 Data Center *

Site 1 / Site 2 – STP only CTN

ETR • Blue Lines

STP (Coupling Links)• Solid Green Lines = ICB4+3• Dash Green Lines = ISC3* Link redundancy not shown * DWDM not shown

Site 1 and Site 2 Data Centers arelocated approx X km apart.

Z9BE - 2094

z/OS 1.8 SYSB

z/OS 1.8 SYSE

STP Timing Mode

Stratum 2, DDS_STP1

Z9G - 2096

z/OS 1.8 SYSG

STP Timing Mode

Stratum 2, DDS_STP1

Z9CD - 2094

z/OS 1.8 SYSC

z/OS 1.8 SYSD

CFCC CF15

STP Timing Mode

Stratum 1,DDS_STP1

Z9LPC4 - 2094

CFCC CF15

STP Timing Mode

Stratum 2,DDS_STP1

Z9OC - 2094

z/OS 1.8 SYSA

GDPS/CBU SYSK

CFCC CF15

STP Timing Mode

Stratum 2, DDS_STP1

ARB

To be used as a Backup server for outages on the PTS, BTS or ARB servers. Manually reassign roles…….

PTS/CTS

BTS

Arbiter

Page 61: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation61

Setting the Time Zone Offset - Initial

Page 62: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation62

Set Time Zone

Automatically adjust is selected by default when the time zone selected supports automatic adjustment of daylight saving time. Otherwise, this button is disabled.

Even if automatic adjustment is supported, the user still has the option of selecting Set standard time or Set daylight saving time.

Page 63: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation63

Adjust Time Zone Offset

Page 64: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation64

Setting the Time Zone Offset

Page 65: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation65

Implementation

Migrate ETR Network to Mixed CTN

– Set CTN ID on all servers

– Migrate servers to STP timing mode

Migrate from Mixed to STP-Only CTN

Page 66: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation66

Agenda

General HMC Panel Overview

Setting up a Coordinated Timing Network (CTN)– Configuring an External Time Source

• Configure Dial Out on HMC• Configure an NTP Server

– Migration from ETR to STP-only CTN– Configuring an STP-only CTN

Restoring STP configuration information – Power outage affecting both servers of two server CTN– Across a Power on Reset for single server CTN

Additional Information

Page 67: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation67

Ending Environment for STP-Only CTN Example

Page 68: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation68

Implementation – Configuring a STP-Only CTN

Set CTN ID

Initialize the time

Assign CTN Roles

Page 69: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation69

Set CTN ID

The same CTN ID must be entered on every server that will participate in the STP-only CTN.

Page 70: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation70

Implementation – Configuring a STP-Only CTN

Set CTN ID

Initialize the time

Assign CTN Roles

Page 71: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation71

Initialize the timeInitialize Time button is accessed from the Network Configuration tab

CTS and PTS roles have not been assigned AND

After CTN ID has been defined for server that will become CTS/PTS.

Important: Initializing the time must be done on the server that will become the Current Time Server for the STP-only CTN.

Initialize Time button enabled ONLY IF

Page 72: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation72

Set Leap Seconds

Clicking Initialize Time on the Network Configuration tab displays the Initialize Time window

There are three radio buttons on the window, each representing a task that needs to be completed before a Network Configuration can be applied to an STP-only CTN.

Page 73: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation73

Set Leap Seconds

Although the installation may not be sensitive to leap seconds, a valid Leap Seconds Offset value needs to be entered to complete the Initialize Time task.

If Leap Seconds are not used, enter a zero value.

Page 74: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation74

Set Time Zone

Automatically adjust is selected by default when the time zone selected supports automatic adjustment of daylight saving time. Otherwise, this button is disabled.

Even if automatic adjustment is supported, the user still has the option of selecting Set standard time or Set daylight saving time.

Page 75: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation75

Adjust Time Zone Offset

Page 76: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation76

Set date and time

If the local date and time are to be set to specific values, click the Set date and time button.

If an ETS is configured, the date and time should be initialized by selecting the option Use External Time Source to set date and time

Regardless of the method chosen, STP uses the information to calculate the Coordinated Server Time and set the servers’ TOD clock when the OK button is clicked.

Page 77: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation77

Set date and time

At this point, the three tasks on the Initialize Time window have a check mark in the Complete column

The user needs to click Cancel to exit the Initialize Time task and return to the Network Configuration tab.

The Coordinated Server Time will be passed to other participating servers in the CTN when the server roles and the Current Time Server are assigned

Page 78: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation78

Implementation – Configuring a STP-Only CTN

Set CTN ID

Initialize the time

Assign CTN Roles

Page 79: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation79

Assign Server Roles

Initialize Time task is complete

Apply button is enabled

Server roles can now be assigned

Important: Assignment of server roles must be done on the server that will become the Current Time Server for the STP-only CTN.

Page 80: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation80

Assign PTS/CTS

Force configuration box must be specified when configuring a new STP-only CTN for the first time in order to bypass connectivity verification, as a Current Time Server does not yet exist.

Because the PTS will be the only server assigned in this first step, it has to be the CTS

Page 81: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation81

Assign BTS

Because the CTS already exists, the Force configuration option is not required or recommended – allows STP to perform connectivity check between PTS and BTS.

Page 82: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation82

Verify on HMC – Timing Network tab

Time field of the Coordinated Server Time section identifies the current time on the server.

Page 83: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation83

Verify on HMC- STP Status tabSTP Timing Mode – Stratum 1

Page 84: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation84

Verify on HMC – STP Status tabSTP Timing Mode – Stratum 2

Page 85: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation85

z/OS Verification – DISPLAY ETR

Synchronization modeStratum level

Node ID of Stratum 1 Server

Number of usable links

Server role (if applicable)

NUMBER OF USABLE TIMING LINKS message will not appear here because it is not applicable for the CTS

This is the PTS/CTS

Page 86: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation86

z/OS Verification – DISPLAY XCF,SYSPLEX,ALL

DISPLAY XCF,SYSPLEX,ALL identifies the timing mode of all sysplex members that are STP

Page 87: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation87

Implementation – Configuring a STP-Only CTN

Set CTN ID

Initialize the time

Assign CTN Roles

Page 88: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation88

Agenda

General HMC Panel Overview

Setting up a Coordinated Timing Network (CTN)– Configuring an External Time Source

• Configure Dial Out on HMC• Configure an NTP Server

– Migration from ETR to STP-only CTN– Configuring an STP-only CTN

Restoring STP configuration information – Power outage affecting both servers of two server CTN– Across a Power on Reset for single server CTN

Additional Information

Page 89: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation89

STP configuration and time information saved across Power on Resets or power outages – two server CTN

Previously– If both servers experienced an unplanned or planned power

outage, user had to:• Initialize the Time; • Redefine the PTS, BTS, CTS roles

System Management Enhancement• STP CTN configuration (PTS/CTS, BTS) can be configured so that

– PTS/CTS, BTS roles possibly restored after IML/POR– Initialize Time not required

Page 90: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation90

Prerequisites / required set up

MCLs required:– z9 EC and BC Bundle 44– z10 EC and BC Bundle 26– Function not available on D55 (z990, z890)

If migrating from ETR network– PTS/CTS and BTS must first be assigned using procedures outlined in

Server Time Protocol Implementation Guide, SG24-7281, section 2.5.1

If setting up a new STP-only CTN– PTS/CTS and BTS must first be assigned using procedures outlined in

Server Time Protocol Implementation Guide, SG24-7281 section 3.4

STP-only CTN can only have two servers to enable this function– If more than two servers in operational CTN,

• Remove additional servers in the CTN by assigning them to a different to a different CTN, if possible

• Cannot proceed with this enhancement with more than two servers in CTN

Page 91: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation91

Enable “Save configuration” enhancement

Launch the System (Sysplex) Time task of the server that is the CTS– Go to the Network Configuration

tab– Check the box “Only allow the

server(s) specified above to be in the CTN”

– Click Apply – “Only allow the server(s) specified

above to be in the CTN” can be enabled only if two servers can communicate over coupling links• Cannot select “Force

Configuration” and “Only allow the server(s) specified above to be in the CTN” at the same time – error message generated

Note: Recovery and re-takeover rules unchanged for a CTN with PTS and BTS assigned

Page 92: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation92

Agenda

General HMC Panel Overview

Setting up a Coordinated Timing Network (CTN)– Configuring an External Time Source

• Configure Dial Out on HMC• Configure an NTP Server

– Migration from ETR to STP-only CTN– Configuring an STP-only CTN

Restoring STP configuration information – Power outage affecting both servers of two server CTN– Across a Power on Reset for single server CTN

Additional Information

Page 93: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation93

STP configuration and time information saved across Power on Reset or power outage – Single server CTN

Previously– For an already operational single server CTN, when IML/POR done,

user was required to:• Initialize the Time• Redefine that server as the stratum 1

System Management enhancement– Customer selects the option to not allow any other servers to join the

single server CTN when configuring the CTN

– STP CTN configuration (PTS/CTS) will be reinstated after IML/POR

• Initialize Time not required

Also available on z990, z890

Page 94: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation94

STP configuration and time information saved across Power on Resets or power outages (continued)

Launch the System (Sysplex) Time task of the server that is the CTS

–Go to the Network Configuration tab–Check the box “Only allow the server(s) specified above to be in the CTN”–Click Apply

Page 95: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation95

Agenda

General HMC Panel Overview

Setting up a Coordinated Timing Network (CTN)– Configuring an External Time Source

• Configure Dial Out on HMC• Configure an NTP Server

– Migration from ETR to STP-only CTN– Configuring an STP-only CTN

Restoring STP configuration information– Across a Power on Reset for single server CTN– Power outage affecting both servers of two server CTN

Additional Information

Page 96: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation96

Redbooks®

– Server Time Protocol Planning Guide SG24-7280

– Server Time Protocol Implementation Guide SG24-7281

Education– Introduction to Server Time Protocol (STP)

• Available on Resource Link™

• www.ibm.com/servers/resourcelink/hom03010.nsf?OpenDatabase

STP Web site– www.ibm.com/systems/z/pso/stp.html

Systems Assurance– The IBM team is required to complete a Systems Assurance Review (SAPR Guide

SA06-012) and to complete the Systems Assurance Confirmation Form via Resource Link

Techdocs and WSC Flashes– http://www-03.ibm.com/support/techdocs/atsmastr.nsf/Web/Techdocs

• Search on “STP”

Additional Information

Page 97: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation97

IBM Implementation Services for System z – Server Time Protocol (6948-J56)

Offering Description

• This offering is designed to assist clients to quickly and safely implement Server Time Protocol within their existing environments. STP provides clients with the capability to efficiently manage time synchronization within their multi-server infrastructure. Following best practices and using detailed planning services, IBM helps clients identify various implementation models and engage in the appropriate configuration required to effectively support STP for driving a more responsive business and IT infrastructure.

Program, Play, Industry Alignment

• Infrastructure Improvement; Energy Efficiency; Better performance and lower operational cost

Client Value (enables customers to...)

• Swift and secure implementation of STP for improved availability, integrity and performance

• Improves multi-server time synchronization without interrupting operations• Enables integration with next generation of System z infrastructure

Target Audience • Primarily core, Large Enterprise customers. • Existing z midrange clients

Key Competitors • In house staff

Competitive Differentiation

• Leverages best practices with secure implementation • Short implementation time – lower risk• Provides support and facilitates knowledge sharing through IBM’s mainframe

expertise

Proof Points & Claims for Client Value / Differentiation

• Need to safely implement a reliable replacement for Sysplex Timer® while maintaining continuous operations

• Cost of providing and maintaining hardware, floor space and solution support for additional Sysplex Timer intermediate site

• Lack of in-house expertise, skills and resources for implementing Server Time Protocol

Engagement Portfolio • http://spimweb1.boulder.ibm.com/services/sosf/dyno.wss?oid=50423&loc=All&langc d=en-US#1

Offering Manager • Anna Lee/Southbury/IBM, 512-590-8914, T/L: 268-9318

Page 98: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation98

IBM Announces –IBM Implementation Services for System z – Server Time Protocol

Offering Assist clients to quickly and safely implement Server Time Protocol within their existing environments. IBM helps clients identify various implementation models and engage in the appropriate configuration required to effectively support STP for driving a more responsive business and IT infrastructure

Customer Value: - Improves multi-server time synchronization without interrupting

operations

- Enables integration with next generation of System z infrastructure

- Swift and secure implementation of STP for improved availability, integrity, and performance

- Reduces hardware maintenance and power costs while eliminating intermediate site requirements for Sysplex Timer

Leverages IBM’s knowledge and best

practices to help implementation of

Server Time Protocol

Implementation of STP for improved availability and performance

Page 99: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation99

Reference Material - Terminology

APAR Authorized Program Analysis Report

ARB Arbiter

BTS Backup Time Server

CF Coupling Facility

CTS Current Time Server

CTN Coordinated Timing Network

DWDM Dense Wave Division

Multiplexer

ETR External Time Reference

ETS External Time Source

FC Feature Code

HMC Hardware Management

Console

HCA Host Channel Adapter

ICB Integrated Cluster Bus

IPL Initial Program Load

ISC InterSystem Coupling Channel

LAN Local Area Network

LIC Licensed Internal Code

LPAR Logically Partition

NTP Network Time Protocol

PR/SM Processor Resource / Systems Manager

PSIFB Parallel Sysplex Infiniband

PTF Temporary Program Fix

PTS Preferred Time Server

SW Software (programs and operating systems)

SE Support Element

TPF Operating System

UTC Coordinated Universal Time

zVM Operating System

zVSE Operating System

z/OS Operating System

z/VM Operating System

Page 100: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation100

Questions?

Page 101: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation101

The End

Thank You!

Page 102: Server Time Protocol Implementation Considerations

© 2010 IBM Corporation102102

TrademarksThe following are trademarks of the International Business Machines Corporation in the United States, other countries, or both.

The following are trademarks or registered trademarks of other companies.

* All other products may be trademarks or registered trademarks of their respective companies.

Notes: Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here. IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.UNIX is a registered trademark of The Open Group in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office.IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency, which is now part of the Office of Government Commerce.

For a complete list of IBM Trademarks, see www.ibm.com/legal/copytrade.shtml:

*, AS/400®, e business(logo)®, DBE, ESCO, eServer, FICON, IBM®, IBM (logo)®, iSeries®, MVS, OS/390®, pSeries®, RS/6000®, S/30, VM/ESA®, VSE/ESA,

WebSphere®, xSeries®, z/OS®, zSeries®, z/VM®, System i, System i5, System p, System p5, System x, System z, System z9®, System z10®, BladeCenter®

Not all common law marks used by IBM are listed on this page. Failure of a mark to appear does not mean that IBM does not use the mark nor does it mean that the product is not actively marketed or is not significant within its relevant market.

Those trademarks followed by ® are registered trademarks of IBM in the United States; all others are trademarks or common law marks of IBM in the United States.