z9 EC System Overview - IBM

System z9 Enterprise Class

System OverviewSA22-6833-06

Level 06a, April 2009

��

NoteBefore using this information and the product it supports, read the information in “Safety” on pagexiii, Appendix D, “Notices,” on page D-1, and IBM Systems Environmental Notices and User Guide,Z125-5823.

Seventh Edition (January 2009)

This edition, SA22-6833-06, applies to the IBM® System z9® Enterprise Class (z9® EC) server. This replacesSA22-6833-05. Technical changes to the text are indicated by a vertical bar (|) to the left of the change.

There may be a newer version of this document in a PDF file available on Resource Link™. Go tohttp://www.ibm.com/servers/resourcelink and click on Library on the navigation bar. A newer version is indicated by alowercase, alphabetic letter following the form number suffix (for example: 00a, 00b, 01a, 01b).

© Copyright International Business Machines Corporation 2005, 2009.US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contractwith IBM Corp.


|

||

http://www.ibm.com/servers/resourcelink

Contents

Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Safety . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiiiSafety Notices . . . . . . . . . . . . . . . . . . . . . . . . . xiii

World Trade Safety Information . . . . . . . . . . . . . . . . . . xiiiLaser Safety Information . . . . . . . . . . . . . . . . . . . . . xiii

Laser Compliance. . . . . . . . . . . . . . . . . . . . . . . xiii

About this Publication . . . . . . . . . . . . . . . . . . . . . xvWhat is Included in this Publication . . . . . . . . . . . . . . . . . xvPrerequisite Publications . . . . . . . . . . . . . . . . . . . . . xvRelated Publications . . . . . . . . . . . . . . . . . . . . . . . xv

Parallel Sysplex Publications. . . . . . . . . . . . . . . . . . . xviOSA Publications . . . . . . . . . . . . . . . . . . . . . . . xviiCryptographic Publications . . . . . . . . . . . . . . . . . . . xviiMiscellaneous Publications . . . . . . . . . . . . . . . . . . . xvii

Related Web Sites . . . . . . . . . . . . . . . . . . . . . . . xviiAdditional Online Information . . . . . . . . . . . . . . . . . . xviii

Engineering Change (EC) Level Considerations . . . . . . . . . . . . xviiiHow to Send Your Comments . . . . . . . . . . . . . . . . . . . xviii

Summary of changes . . . . . . . . . . . . . . . . . . . . . . xix

Chapter 1. Introduction . . . . . . . . . . . . . . . . . . . . . 1-1z9 EC Highlights . . . . . . . . . . . . . . . . . . . . . . . . 1-2

Scalability . . . . . . . . . . . . . . . . . . . . . . . . . 1-3z9 EC Models . . . . . . . . . . . . . . . . . . . . . . . . 1-6Performance . . . . . . . . . . . . . . . . . . . . . . . . 1-6Resource Link . . . . . . . . . . . . . . . . . . . . . . . . 1-7Fiber Optic Cabling . . . . . . . . . . . . . . . . . . . . . . 1-7

z/Architecture . . . . . . . . . . . . . . . . . . . . . . . . . 1-7Upgrade Progression . . . . . . . . . . . . . . . . . . . . . . 1-8Unsupported Features/Functions . . . . . . . . . . . . . . . . . . 1-8

Chapter 2. Hardware Characteristics . . . . . . . . . . . . . . . . 2-1System Frame Configuration . . . . . . . . . . . . . . . . . . . 2-1Central Processor Complex (CPC) . . . . . . . . . . . . . . . . . 2-2

Books . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2Multichip Module (MCM) . . . . . . . . . . . . . . . . . . . . 2-4Memory . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5MBA and HCA1-O fanout cards . . . . . . . . . . . . . . . . . 2-8Distributed Converter Assembly-CXT (DCA-CXT) Cards . . . . . . . . 2-9Oscillator (OSC) Card . . . . . . . . . . . . . . . . . . . . . 2-9External Time Reference (ETR) Card . . . . . . . . . . . . . . . 2-9

I/O Cages . . . . . . . . . . . . . . . . . . . . . . . . . . 2-10I/O Features . . . . . . . . . . . . . . . . . . . . . . . . 2-10Distributed Converter Assembly-CX (DCA-CX) Cards . . . . . . . . . 2-12PSC24V Card . . . . . . . . . . . . . . . . . . . . . . . 2-12

Internet Protocol Version 6 for HiperSockets . . . . . . . . . . . . . 2-12Cooling Equipment. . . . . . . . . . . . . . . . . . . . . . . 2-12System Power Supply . . . . . . . . . . . . . . . . . . . . . 2-12


© Copyright IBM Corp. 2005, 2009 iii

Multiple Subchannel Sets (MSS). . . . . . . . . . . . . . . . . . 2-13Logically Partitioned (LPAR) Mode . . . . . . . . . . . . . . . . . 2-13

Processor Units . . . . . . . . . . . . . . . . . . . . . . . 2-13Storage . . . . . . . . . . . . . . . . . . . . . . . . . . 2-14Channels . . . . . . . . . . . . . . . . . . . . . . . . . 2-14Logical Partition Time Offset Support . . . . . . . . . . . . . . . 2-15

Server Time Protocol (STP) . . . . . . . . . . . . . . . . . . . 2-16Support Element . . . . . . . . . . . . . . . . . . . . . . . 2-16

Support Element Attachment to HMC . . . . . . . . . . . . . . . 2-17Ethernet Switch/Hub . . . . . . . . . . . . . . . . . . . . . 2-17

Hardware Management Console. . . . . . . . . . . . . . . . . . 2-17Internal Battery Feature-360 . . . . . . . . . . . . . . . . . . . 2-18Bolt-Down Kit. . . . . . . . . . . . . . . . . . . . . . . . . 2-19Power Sequence Control . . . . . . . . . . . . . . . . . . . . 2-19Additional Features/Functions Supported . . . . . . . . . . . . . . 2-19

z/Architecture. . . . . . . . . . . . . . . . . . . . . . . . 2-20ESA/390 Architecture . . . . . . . . . . . . . . . . . . . . . 2-21Floating-Point Operations . . . . . . . . . . . . . . . . . . . 2-22Data Representation . . . . . . . . . . . . . . . . . . . . . 2-23Data Compression . . . . . . . . . . . . . . . . . . . . . . 2-23Dynamic I/O Configuration . . . . . . . . . . . . . . . . . . . 2-24Subspace Group Facility Support . . . . . . . . . . . . . . . . 2-24Dedicated Move Page Engine . . . . . . . . . . . . . . . . . 2-25Fast Sync/Data Mover Facility . . . . . . . . . . . . . . . . . 2-25Immediate and Relative Instruction Facility . . . . . . . . . . . . . 2-25Perform Locked Operation (PLO) Facility . . . . . . . . . . . . . 2-25Logical String Assist . . . . . . . . . . . . . . . . . . . . . 2-25Multipath Initial Program Load (IPL) . . . . . . . . . . . . . . . 2-26Monitoring and Estimating CPC Power Consumption and Temperature 2-26

Chapter 3. Software Support . . . . . . . . . . . . . . . . . . . 3-1z/OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2

z/OS V1R8 . . . . . . . . . . . . . . . . . . . . . . . . . 3-2z/OS V1R7 . . . . . . . . . . . . . . . . . . . . . . . . . 3-2z/OS V1R6 . . . . . . . . . . . . . . . . . . . . . . . . . 3-2z/OS V1R5 . . . . . . . . . . . . . . . . . . . . . . . . . 3-2z/OS V1R4 . . . . . . . . . . . . . . . . . . . . . . . . . 3-2

z/VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3z/VM V5.4 and z/VM V5.3 . . . . . . . . . . . . . . . . . . . 3-3

z/VSE . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3VSE/ESA V2.7. . . . . . . . . . . . . . . . . . . . . . . . . 3-3z/VSE V3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3z/TPF V1.1 . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4Linux Support . . . . . . . . . . . . . . . . . . . . . . . . . 3-4

Integrated Facility for Linux (IFL) . . . . . . . . . . . . . . . . . 3-4

Chapter 4. Channel Subsystem Structure . . . . . . . . . . . . . . 4-1IOCP Channel Definitions. . . . . . . . . . . . . . . . . . . . . 4-2

Coupling Link Peer Channels . . . . . . . . . . . . . . . . . . 4-3Subchannel Connectivity . . . . . . . . . . . . . . . . . . . . . 4-3Availability Guidelines for z9 EC Models . . . . . . . . . . . . . . . 4-4

STI and InfiniBand Configurations. . . . . . . . . . . . . . . . . 4-6Planning for Channel Subsystem . . . . . . . . . . . . . . . . . . 4-7

CHPID and PCHID Assignments . . . . . . . . . . . . . . . . . 4-8AID Assignments . . . . . . . . . . . . . . . . . . . . . . . 4-9PCHID Report . . . . . . . . . . . . . . . . . . . . . . . 4-10


iv z9 EC System Overview

CHPID Mapping Tool . . . . . . . . . . . . . . . . . . . . . 4-11Multiple Image Facility (MIF) . . . . . . . . . . . . . . . . . . . 4-12Spanned Channels. . . . . . . . . . . . . . . . . . . . . . . 4-13Internal Coupling and HiperSockets Channels . . . . . . . . . . . . . 4-13IOCP Considerations . . . . . . . . . . . . . . . . . . . . . . 4-14

Logical Partition Definition . . . . . . . . . . . . . . . . . . . 4-14Channel Path Definition . . . . . . . . . . . . . . . . . . . . 4-14I/O Device Definition . . . . . . . . . . . . . . . . . . . . . 4-15

Hardware Configuration Definition (HCD) Considerations. . . . . . . . . 4-15

Chapter 5. I/O Connectivity . . . . . . . . . . . . . . . . . . . 5-1FICON and FCP Channels . . . . . . . . . . . . . . . . . . . . 5-1

The MIDAW Facility . . . . . . . . . . . . . . . . . . . . . . 5-1Multipath Initial Program Load (IPL) . . . . . . . . . . . . . . . . 5-2

FICON Express4 . . . . . . . . . . . . . . . . . . . . . . . . 5-2FICON Express2 . . . . . . . . . . . . . . . . . . . . . . . . 5-2

Purge Path Extended . . . . . . . . . . . . . . . . . . . . . 5-4FICON Express Features . . . . . . . . . . . . . . . . . . . . . 5-4

FICON Channel Considerations . . . . . . . . . . . . . . . . . 5-5FICON Express Features Support 2 Gbps Link Data Rate. . . . . . . . 5-5

Fibre Channel Protocol for SCSI (FCP) Devices . . . . . . . . . . . . 5-7SCSI Initial Program Load (IPL) . . . . . . . . . . . . . . . . . 5-8FCP Channel . . . . . . . . . . . . . . . . . . . . . . . . 5-9

ESCON Channels . . . . . . . . . . . . . . . . . . . . . . . 5-12Multipath Initial Program Load (IPL) . . . . . . . . . . . . . . . 5-13ESCON Converter Operation . . . . . . . . . . . . . . . . . . 5-13I/O Operations Control . . . . . . . . . . . . . . . . . . . . 5-15I/O Interface Protocol . . . . . . . . . . . . . . . . . . . . . 5-17ESCON Channel Performance . . . . . . . . . . . . . . . . . 5-19

OSA Channels . . . . . . . . . . . . . . . . . . . . . . . . 5-25Supported CHPID Types . . . . . . . . . . . . . . . . . . . 5-25OSA/SF. . . . . . . . . . . . . . . . . . . . . . . . . . 5-26

OSA-Express Channels . . . . . . . . . . . . . . . . . . . . . 5-26OSA-Express2 . . . . . . . . . . . . . . . . . . . . . . . . 5-27

OSA-Express2 Gigabit Ethernet LX and SX . . . . . . . . . . . . 5-28OSA-Express2 10 Gigabit Ethernet LR . . . . . . . . . . . . . . 5-28OSA-Express2 1000Base-T Ethernet . . . . . . . . . . . . . . . 5-28Layer 2 (Link Layer) Support . . . . . . . . . . . . . . . . . . 5-28640 TCP/IP Stacks. . . . . . . . . . . . . . . . . . . . . . 5-28Large Send . . . . . . . . . . . . . . . . . . . . . . . . 5-28Concurrent LIC Update . . . . . . . . . . . . . . . . . . . . 5-28OSA LAN Idle Timer . . . . . . . . . . . . . . . . . . . . . 5-29OSA-Express Network Traffic Analyzer Trace Facility . . . . . . . . . 5-29Queued Direct I/O Diagnostic Synchronization (QDIOSYNC) . . . . . . 5-29Dynamic Link Aggregation Support for the z/VM Environment . . . . . . 5-29Layer 3 Virtual MAC . . . . . . . . . . . . . . . . . . . . . 5-29

HiperSockets . . . . . . . . . . . . . . . . . . . . . . . . . 5-30VLAN Support for Linux . . . . . . . . . . . . . . . . . . . . 5-30Broadcast Support . . . . . . . . . . . . . . . . . . . . . . 5-30Network Concentrator . . . . . . . . . . . . . . . . . . . . 5-31

Chapter 6. Sysplex Functions . . . . . . . . . . . . . . . . . . 6-1Parallel Sysplex . . . . . . . . . . . . . . . . . . . . . . . . 6-1Parallel Sysplex Coupling Link Connectivity . . . . . . . . . . . . . . 6-2

Peer Mode . . . . . . . . . . . . . . . . . . . . . . . . . 6-3ISC-3 Links . . . . . . . . . . . . . . . . . . . . . . . . . 6-3


Contents v

ICB Links . . . . . . . . . . . . . . . . . . . . . . . . . 6-4IC Links . . . . . . . . . . . . . . . . . . . . . . . . . . 6-5InfiniBand (IFB) Links . . . . . . . . . . . . . . . . . . . . . 6-5Timing-only Links . . . . . . . . . . . . . . . . . . . . . . . 6-6

Coupling Facility . . . . . . . . . . . . . . . . . . . . . . . . 6-6Coupling Facility Control Code (CFCC) Considerations . . . . . . . . . 6-6Coupling Connection Considerations . . . . . . . . . . . . . . . 6-8I/O Configuration Considerations . . . . . . . . . . . . . . . . . 6-8

Server Time Protocol (STP) . . . . . . . . . . . . . . . . . . . . 6-9Message Time Ordering . . . . . . . . . . . . . . . . . . . . . 6-11System-Managed CF Structure Duplexing . . . . . . . . . . . . . . 6-11Geographically Dispersed Parallel Sysplex (GDPS) . . . . . . . . . . . 6-12

GDPS/PPRC . . . . . . . . . . . . . . . . . . . . . . . . 6-13GDPS/XRC . . . . . . . . . . . . . . . . . . . . . . . . 6-13GDPS/Global Mirror . . . . . . . . . . . . . . . . . . . . . 6-14Enhancements in GDPS V3.3. . . . . . . . . . . . . . . . . . 6-14

Intelligent Resource Director (IRD) . . . . . . . . . . . . . . . . . 6-14LPAR CPU Management (Clustering) . . . . . . . . . . . . . . . 6-15I/O Priority Queuing (IOPQ) . . . . . . . . . . . . . . . . . . 6-15Dynamic Channel Path Management (DCM) . . . . . . . . . . . . 6-16Workload Manager (WLM) . . . . . . . . . . . . . . . . . . . 6-16

EAL5 Certification . . . . . . . . . . . . . . . . . . . . . . . 6-17

Chapter 7. Cryptography . . . . . . . . . . . . . . . . . . . . 7-1CP Assist for Cryptographic Function (CPACF) . . . . . . . . . . . . . 7-1Crypto Express2 . . . . . . . . . . . . . . . . . . . . . . . . 7-2User-Defined Extensions . . . . . . . . . . . . . . . . . . . . . 7-2Trusted Key Entry (TKE) Workstation . . . . . . . . . . . . . . . . 7-3

Trusted Key Entry (TKE) with Smart Card Reader. . . . . . . . . . . 7-3RMF Monitoring . . . . . . . . . . . . . . . . . . . . . . . . 7-3Certification . . . . . . . . . . . . . . . . . . . . . . . . . . 7-4

FIPS Certification. . . . . . . . . . . . . . . . . . . . . . . 7-4EAL5 Certification . . . . . . . . . . . . . . . . . . . . . . 7-4

Remote Loading of ATM and POS Keys . . . . . . . . . . . . . . . 7-4

Chapter 8. Cabling . . . . . . . . . . . . . . . . . . . . . . . 8-1Fiber Quick Connect (FQC) ESCON Cabling . . . . . . . . . . . . . 8-2Cable Ordering . . . . . . . . . . . . . . . . . . . . . . . . 8-2Cabling Responsibilities . . . . . . . . . . . . . . . . . . . . . 8-3Cabling Report. . . . . . . . . . . . . . . . . . . . . . . . . 8-3

Chapter 9. Hardware Management Console and Support Element . . . . 9-1Closed Platform . . . . . . . . . . . . . . . . . . . . . . . . 9-1Hardware Management Console Enhancements for System z9 EC . . . . . 9-2

z/VM Integrated Systems Management. . . . . . . . . . . . . . . 9-2User Authentication . . . . . . . . . . . . . . . . . . . . . . 9-2New and Revised Tasks . . . . . . . . . . . . . . . . . . . . 9-3

Hardware Management Console Application (HWMCA) . . . . . . . . . . 9-3Customization . . . . . . . . . . . . . . . . . . . . . . . . 9-3Status Reporting . . . . . . . . . . . . . . . . . . . . . . . 9-3Hardware Messages . . . . . . . . . . . . . . . . . . . . . 9-4Service Required State . . . . . . . . . . . . . . . . . . . . 9-4Degrade Indicator . . . . . . . . . . . . . . . . . . . . . . 9-5Operating System Messages . . . . . . . . . . . . . . . . . . 9-5Problem Analysis and Reporting . . . . . . . . . . . . . . . . . 9-5Virtual RETAIN . . . . . . . . . . . . . . . . . . . . . . . 9-5


vi z9 EC System Overview

Licensed Internal Code (LIC) . . . . . . . . . . . . . . . . . . 9-6Remote I/O Configuration and IOCDS Management . . . . . . . . . . 9-6Scheduled Operations . . . . . . . . . . . . . . . . . . . . . 9-6Remote Support Facility (RSF) . . . . . . . . . . . . . . . . . . 9-7Automation and API Support . . . . . . . . . . . . . . . . . . 9-7Activation . . . . . . . . . . . . . . . . . . . . . . . . . 9-8

Hardware Management Console and Support Element Network Connection 9-8Network Protocols . . . . . . . . . . . . . . . . . . . . . . . 9-8Customizable Console Date and Time . . . . . . . . . . . . . . . . 9-8User Interface . . . . . . . . . . . . . . . . . . . . . . . . . 9-9User Authority . . . . . . . . . . . . . . . . . . . . . . . . . 9-9Security Considerations . . . . . . . . . . . . . . . . . . . . . 9-10

Change Management Considerations . . . . . . . . . . . . . . . 9-11I/O Configuration . . . . . . . . . . . . . . . . . . . . . . . 9-11

Recommended Approach . . . . . . . . . . . . . . . . . . . 9-11Coupling Facility Continuous Availability . . . . . . . . . . . . . . 9-12

Remote Operations and Remote Access. . . . . . . . . . . . . . . 9-13Remote Manual Operations . . . . . . . . . . . . . . . . . . 9-14Remote Automated Operations . . . . . . . . . . . . . . . . . 9-15

Chapter 10. Reliability, Availability, and Serviceability (RAS) . . . . . . 10-1Reliability . . . . . . . . . . . . . . . . . . . . . . . . . . 10-1Availability . . . . . . . . . . . . . . . . . . . . . . . . . . 10-1

Enhanced Book Availability. . . . . . . . . . . . . . . . . . . 10-1Redundant I/O Interconnect . . . . . . . . . . . . . . . . . . 10-1Enhanced Driver Maintenance . . . . . . . . . . . . . . . . . 10-1Dynamic Oscillator Switchover . . . . . . . . . . . . . . . . . 10-1Program Directed Re-IPL . . . . . . . . . . . . . . . . . . . 10-1Concurrent MBA Fanout Card Hot-Plug . . . . . . . . . . . . . . 10-2Processor Unit (PU) Sparing . . . . . . . . . . . . . . . . . . 10-2Processor Design . . . . . . . . . . . . . . . . . . . . . . 10-2Support Element (SE) . . . . . . . . . . . . . . . . . . . . 10-2Hardware Management Console. . . . . . . . . . . . . . . . . 10-2Two SAP Standard. . . . . . . . . . . . . . . . . . . . . . 10-3Application Preservation . . . . . . . . . . . . . . . . . . . . 10-3Dynamic ICF Expansion . . . . . . . . . . . . . . . . . . . . 10-3Dynamic Coupling Facility Dispatching . . . . . . . . . . . . . . 10-4Error Correction Code (ECC) . . . . . . . . . . . . . . . . . . 10-4Memory Sparing . . . . . . . . . . . . . . . . . . . . . . 10-4Memory Scrubbing . . . . . . . . . . . . . . . . . . . . . . 10-4LPAR Dynamic Storage Reconfiguration . . . . . . . . . . . . . . 10-4Subsystem Storage Protect . . . . . . . . . . . . . . . . . . 10-4Partial Memory Restart . . . . . . . . . . . . . . . . . . . . 10-4Dynamic I/O Configuration . . . . . . . . . . . . . . . . . . . 10-4ESCON Port Sparing . . . . . . . . . . . . . . . . . . . . . 10-5FICON Cascaded Directors . . . . . . . . . . . . . . . . . . 10-5FCP Full-fabric Connectivity . . . . . . . . . . . . . . . . . . 10-5Maintenance/Upgrade for Coupling . . . . . . . . . . . . . . . . 10-5Concurrent Channel Upgrade . . . . . . . . . . . . . . . . . . 10-5Dual Power Feeds . . . . . . . . . . . . . . . . . . . . . . 10-5Redundant Power and Thermal Subsystems . . . . . . . . . . . . 10-5Redundant Flexible Service Processors . . . . . . . . . . . . . . 10-5Oscillator (OSC) and External Timer Reference (ETR) cards . . . . . . 10-5Preferred Time Server and Backup Time Server . . . . . . . . . . . 10-6Concurrent Hardware Maintenance. . . . . . . . . . . . . . . . 10-6Concurrent Licensed Internal Code (LIC) Patch . . . . . . . . . . . 10-6


Contents vii

Electronic Service Agent (Service Director) . . . . . . . . . . . . . 10-7Internal Battery Feature (IBF) . . . . . . . . . . . . . . . . . . 10-7Redundant Coupling Links . . . . . . . . . . . . . . . . . . . 10-7System-Managed CF Structure Duplexing (CF Duplexing) . . . . . . . 10-7On/Off Capacity on Demand (On/Off CoD) . . . . . . . . . . . . . 10-8Capacity Upgrade on Demand (CUoD) . . . . . . . . . . . . . . 10-9Capacity Backup (CBU) . . . . . . . . . . . . . . . . . . . . 10-9Geographically Dispersed Parallel Sysplex (GDPS) . . . . . . . . . . 10-9CBU Smart Reactivation. . . . . . . . . . . . . . . . . . . . 10-9Concurrent Undo CBU . . . . . . . . . . . . . . . . . . . . 10-9Fiber Optic Cabling . . . . . . . . . . . . . . . . . . . . . 10-9CHPID Mapping Tool . . . . . . . . . . . . . . . . . . . . 10-10Multipath Initial Program Load . . . . . . . . . . . . . . . . . 10-10System-initiated CHPID Reconfiguration . . . . . . . . . . . . . 10-10

Serviceability . . . . . . . . . . . . . . . . . . . . . . . . 10-10

Appendix A. System z9 EC Version 2.9.2 Purpose and Description . . . . A-1New Function, Features, and Differences . . . . . . . . . . . . . . . A-1Operating Systems . . . . . . . . . . . . . . . . . . . . . . . A-2Preventative Service Planning (PSP) Bucket Considerations . . . . . . . . A-2

Software Corequisites . . . . . . . . . . . . . . . . . . . . . A-2Engineering Change (EC) Considerations. . . . . . . . . . . . . . . A-2

CPC EC G40965 + MCLs . . . . . . . . . . . . . . . . . . . A-2HMC EC G40969 + MCLs . . . . . . . . . . . . . . . . . . . A-3Miscellaneous Lower Level ECs Included in Version 2.9.2. . . . . . . . A-3

Related Publications . . . . . . . . . . . . . . . . . . . . . . A-3

Appendix B. Resource Link . . . . . . . . . . . . . . . . . . . B-1Resource Link Functions . . . . . . . . . . . . . . . . . . . . . B-1

Appendix C. Capacity Upgrades . . . . . . . . . . . . . . . . . C-1Customer Initiated Upgrade (CIU) . . . . . . . . . . . . . . . . . C-1

Permanent Upgrades . . . . . . . . . . . . . . . . . . . . . C-2CIU Process . . . . . . . . . . . . . . . . . . . . . . . . C-2

On/Off Capacity on Demand (On/Off CoD) . . . . . . . . . . . . . . C-3On/Off Capacity on Demand (On/Off CoD) Enhancements . . . . . . . C-4Administrative On/Off CoD Test . . . . . . . . . . . . . . . . . C-4

Capacity Backup (CBU) . . . . . . . . . . . . . . . . . . . . . C-4Concurrent PU Conversions. . . . . . . . . . . . . . . . . . . . C-6Reserved CP Support in LPAR Mode . . . . . . . . . . . . . . . . C-6Non-disruptive Upgrades . . . . . . . . . . . . . . . . . . . . . C-6Processor Capacity Downgrades . . . . . . . . . . . . . . . . . . C-7Capacity Upgrade on Demand (CUoD). . . . . . . . . . . . . . . . C-7

Appendix D. Notices . . . . . . . . . . . . . . . . . . . . . . D-1Trademarks. . . . . . . . . . . . . . . . . . . . . . . . . . D-2Electronic Emission Notices. . . . . . . . . . . . . . . . . . . . D-3

Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . E-1

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . X-1


viii z9 EC System Overview

Figures

1-1. IBM System z9 EC . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-12-1. z9 EC Frame Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 2-12-2. z9 EC Book Positions . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-34-1. I/O Cage Layout for the z9 EC Models . . . . . . . . . . . . . . . . . . . . . 4-54-2. ICB and InfiniBand connections . . . . . . . . . . . . . . . . . . . . . . . . 4-75-1. Control Unit (CU) Priority on ESCON Channels Attached to a 9034 ES Connection Converter 5-198-1. Cabling Section of the PCHID Report Sample. . . . . . . . . . . . . . . . . . . 8-49-1. Hardware Management Console Configuration . . . . . . . . . . . . . . . . . . 9-29-2. Remote Operation Example Configuration. . . . . . . . . . . . . . . . . . . . 9-14


© Copyright IBM Corp. 2005, 2009 ix


x z9 EC System Overview

Tables

1. Summary of Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix1-1. System z9 Model Structure . . . . . . . . . . . . . . . . . . . . . . . . . 1-52-1. z9 EC Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-32-2. z9 EC Memory Card Characteristics . . . . . . . . . . . . . . . . . . . . . . 2-72-3. I/O Channels Summary per System . . . . . . . . . . . . . . . . . . . . . . 2-112-4. New Build Hardware Management Console FC 0075 Hardware Minimum Requirements 2-172-5. New Build Hardware Management Console FC 0077 Hardware Minimum Requirements 2-172-6. New Build Hardware Management Console FC 0079 Hardware Minimum Requirements 2-182-7. System IBF Hold Up Times . . . . . . . . . . . . . . . . . . . . . . . . . 2-193-1. Supported Operating Systems for z9 EC. . . . . . . . . . . . . . . . . . . . . 3-13-2. Running z/OS as a Guest of z/VM . . . . . . . . . . . . . . . . . . . . . . . 3-24-1. Channel Maximums (z9 EC compared to z990) . . . . . . . . . . . . . . . . . . 4-14-2. I/O Channels with CHPID Type . . . . . . . . . . . . . . . . . . . . . . . . 4-24-3. PCHID assignments for I/O cage . . . . . . . . . . . . . . . . . . . . . . . 4-84-4. AID assignments for HCA fanout cards . . . . . . . . . . . . . . . . . . . . . 4-95-1. Contrasting Unrepeated Distances using Single Mode and Multimode . . . . . . . . . . 5-66-1. Coupling Link Options . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-26-2. IOPQ in a Single-System Environment . . . . . . . . . . . . . . . . . . . . . 6-168-1. Cabling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8-29-1. Comparing z9 EC Task Roles to User Modes . . . . . . . . . . . . . . . . . . . 9-99-2. Comparing z9 EC Managed Resource Roles to User Modes . . . . . . . . . . . . . 9-9A-1. Software Corequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . A-2A-2. ECs Included in Version 2.9.2 . . . . . . . . . . . . . . . . . . . . . . . . A-3


© Copyright IBM Corp. 2005, 2009 xi


xii z9 EC System Overview

Safety

Safety NoticesSafety notices may be printed throughout this guide. DANGER notices warn you ofconditions or procedures that can result in death or severe personal injury.CAUTION notices warn you of conditions or procedures that can cause personalinjury that is neither lethal nor extremely hazardous. Attention notices warn you ofconditions or procedures that can cause damage to machines, equipment, orprograms.

There are no DANGER notices in this guide.

World Trade Safety InformationSeveral countries require the safety information contained in product publications tobe presented in their national languages. If this requirement applies to your country,a safety information booklet is included in the publications package shipped with theproduct. The booklet contains the safety information in your national language withreferences to the US English source. Before using a US English publication toinstall, operate, or service this IBM product, you must first become familiar with therelated safety information in the booklet. You should also refer to the booklet anytime you do not clearly understand any safety information in the US Englishpublications.

Laser Safety InformationAll System z models can use I/O cards such as PCI adapters, ESCON, FICON,Open Systems Adapter (OSA), InterSystem Coupling-3 (ISC-3), or other I/Ofeatures which are fiber optic based and utilize lasers or LEDs.

Laser ComplianceAll lasers are certified in the U.S. to conform to the requirements of DHHS 21 CFRSubchapter J for class 1 laser products. Outside the U.S., they are certified to be incompliance with IEC 60825 as a class 1 laser product. Consult the label on eachpart for laser certification numbers and approval information.

CAUTION:Data processing environments can contain equipment transmitting on systemlinks with laser modules that operate at greater than Class 1 power levels. Forthis reason, never look into the end of an optical fiber cable or openreceptacle. (C027)

CAUTION:This product contains a Class 1M laser. Do not view directly with opticalinstruments. (C028)


© Copyright IBM Corp. 2005, 2009 xiii


xiv z9 EC System Overview

About this Publication

This publication describes the design, components, functions, features, andcapabilities of the IBM® System z9® Enterprise Class (z9® EC) models, formerlyknown as the System z9 109. It is intended for executives, data processingmanagers, data processing technical staff, consultants, and vendors who wish toexploit z9 EC advantages.

You should be familiar with the various publications listed in “PrerequisitePublications” and “Related Publications.” A “Glossary” on page E-1 and an Indexare provided at the back of this publication.

What is Included in this PublicationThis publication contains the following chapters and appendices:v Chapter 1, “Introduction,” on page 1-1v Chapter 2, “Hardware Characteristics,” on page 2-1v Chapter 3, “Software Support,” on page 3-1v Chapter 4, “Channel Subsystem Structure,” on page 4-1v Chapter 5, “I/O Connectivity,” on page 5-1v Chapter 6, “Sysplex Functions,” on page 6-1v Chapter 7, “Cryptography,” on page 7-1v Chapter 8, “Cabling,” on page 8-1v Chapter 9, “Hardware Management Console and Support Element,” on page 9-1v Chapter 10, “Reliability, Availability, and Serviceability (RAS),” on page 10-1v Appendix A, “System z9 EC Version 2.9.2 Purpose and Description,” on page A-1v Appendix B, “Resource Link,” on page B-1v Appendix C, “Capacity Upgrades,” on page C-1.v Appendix D, “Notices,” on page D-1

Prerequisite PublicationsBefore reading this publication you should be familiar with IBM z/Architecture®, IBMS/390®, and IBM Enterprise Systems Architecture/390 (ESA/390) as described inthe following publications:v z/Architecture Principles of Operation, SA22-7832v Enterprise System Architecture/390 Principles of Operation, SA22-7201.

Related Publications

ImportantPlease ensure that you are using the most recent version of all relateddocumentation.

Other IBM publications that you will find helpful and that you should use along withthis publication are in the following list. You can access these books from ResourceLink™ under the Library section.

v System z Application Programming Interfaces, SB10-7030

v System z Capacity on Demand User’s Guide, SC28-6846

v System z CHPID Mapping Tool User’s Guide, GC28-6825

v System z ESCON and FICON Channel-to-Channel Reference, SB10-7034


© Copyright IBM Corp. 2005, 2009 xv



v System z Hardware Management Console Operations Guide, SC28-6857

v System z9 Stand-Alone Input/Output Configuration Program User’s Guide,SB10-7152

v System z Input/Output Configuration Program User’s Guide for ICP IOCP,SB10-7037

v System z9 Enterprise Class Installation Manual, GC28-6840

v System z9 Enterprise Class Installation Manual for Physical Planning,GC28-6844

v System z9 Processor Resource/Systems Manager Planning Guide, SB10-7041

v System z9 Enterprise Class Service Guide, GC28-6841

v System z9 Support Element Operations Guide, SC28-6845

v System z Small Computer Systems (SCSI) IPL - Machine Loader Messages,SC28-6839

v Enterprise Systems Architecture/390 Common I/O-Device Commands,SA22-7204

v Enterprise Systems Architecture/390 Data Compression, SA22-7208

v Enterprise Systems Architecture/390 ESCON Channel-to-Channel Adapter,SA22-7203

v Enterprise Systems Architecture/390 ESCON I/O Interface, SA22-7202

v Enterprise System Architecture/390 System 360 and System 370 I/O InterfaceChannel to Control Unit OEMI, GA22-6974

v IBM Cabling System Optical Fiber Planning and Installation Guide, GA27-3943

v Introducing Enterprise Systems Connection, GA23-0383

v Planning for: Fiber Optic Links (ESCON, FICON, Coupling Links, and OpenSystem Adapters), GA23-0367

v S/390 Parallel Enterprise Server™ and OS/390 Reference Guide, G326-3070

v Storage Subsystem Library Introduction to Nonsynchronous Direct AccessStorage Subsystems, GC26-4519.

Parallel Sysplex PublicationsA Parallel Sysplex® system consists of two or more z/OS® images coupled bycoupling links to a common Coupling Facility and synchronized by a common timesource, such as Server Time Protocol (STP) or a Sysplex Timer®. A Parallel Sysplexcan be used to present a single image to the user. A Parallel Sysplex can use thecoupling facility to provide data sharing among the systems participating in theParallel Sysplex.

The following publications provide additional information to help you understand andprepare for a Parallel Sysplex that uses coupling facility for data sharing purposes.

v z/OS Parallel Sysplex Application Migration, SA22-7662, provides planningconsiderations for defining CICS/ESA® and IMS/ESA® applications to run in aParallel Sysplex environment.

v z/OS Parallel Sysplex Overview: Introducing Data Sharing and Parallelism in aSysplex, SA22-7661, provides an overview of the sysplex and coupling facilitydata sharing and describes basic concepts and values of the sysplex to yourbusiness.

v z/OS MVS Setting Up a Sysplex, SA22-7625, provides information you need toset a z/OS system to run in a sysplex.


xvi z9 EC System Overview

||

OSA PublicationsThe following publication provides additional information for planning and using theOSA-Express features:

v System z10, System z9 and eServer zSeries Open Systems Adapter-ExpressCustomer’s Guide and Reference, SA22-7935

v System z Open Systems Adapter-Express Integrated Console Controller User’sGuide, SA22-7990.

Cryptographic PublicationsThe following publications provide additional information about the Cryptographicfunction:

v z/OS ICSF Trusted Key Workstation User’s Guide 2000, SA22-7524

v z/OS Integrated Cryptographic Service Facility Administrator’s Guide, SA22-7521

v z/OS Integrated Cryptographic Service Facility Application Programmer’s Guide,SA22-7522

v z/OS Integrated Cryptographic Service Facility Messages, SA22-7523

v z/OS Integrated Cryptographic Service Facility Overview, SA22-7519

v z/OS Integrated Cryptographic Service Facility System Programmer’s Guide,SA22-7520

v OS/390 ICSF Trusted Key Entry Workstation User’s Guide 2000, GA22-7430

v OS/390 Integrated Cryptographic Service Facility Administrator’s Guide,SC23-3975

v OS/390 Integrated Cryptographic Service Facility Application Programmer’sGuide, SC23-3976

v OS/390 Integrated Cryptographic Service Facility Messages, SC23-3977

v OS/390 Integrated Cryptographic Service Facility Overview, GC23-3972

v OS/390 Integrated Cryptographic Service Facility System Programmer’s Guide,SC23-3974

v Cryptographic Subsystem Concepts and Facilities, GC22-9063

v Data Security Through Cryptography, GC22-9062

v Transaction Security System General Information and Planning Guide,GA34-2137.

Miscellaneous PublicationsThe following publications provide additional miscellaneous information:

v IBM 3390 Direct Access Storage Introduction, GC26-4573

v IBM TotalStorage Enterprise Storage Server Host Systems Attachment Guide2105 Models E10, E20, F10, and F20, SC26-7296

v IBM TotalStorage Enterprise Storage Server Introduction and Planning Guide,GC26-7294

v Technical Introduction IBM System z9 EC, SG24-6863

v Server Time Protocol Planning Guide, SG24-7280

v Server Time Protocol Implementation Guide, SG24-7281

v Getting Started with InfiniBand on System z10 and System z9, SG24-7539

Related Web SitesThe following web sites provide additional System z9 information:


About this Publication xvii

v http://www.ibm.com/servers/resourcelink - Resource Link is a key element insupporting the System z9 product life cycle. Some of the main areas include:– Education– Planning– Library– CHPID Mapping Tool– Fiber Cabling– Customer Initiated Upgrade (CIU).

v http://www.ibm.com/systems/z/pso - Parallel Sysplex and Coupling Facilityinformation.

v http://www.ibm.com/developerworks/linux/linux390 - Linux® on System z®

information. When searching, specify “Linux” instead of “All of dW.”

v http://www.ibm.com/systems/z/hardware/connectivity - FICON® Information

v http://www.ibm.com/servers/eserver/zseries/lspr - IBM’s Large SystemsPerformance Reference (LSPR).

Additional Online InformationOnline information is available about the z9 EC system to define tasks and to aid incompleting tasks. The following information is available under the Books icon on theHardware Management Console:v Application Programming Interfaces (API)v Programming Interfaces for Java™

v Coupling Facility Control Code Commandsv Hardware Management Console Operations Guidev Support Element Operations Guide.

Help is available for panels, panel options, and fields on panels.

Engineering Change (EC) Level ConsiderationsFuture enhancements available for z9 EC models may be dependent on the EClevel of the Central Processor Complex (CPC) and/or Hardware ManagementConsole. Additionally, some enhancements may further be dependent on theMicrocode Load (MCL) level of the EC on the CPC and/or Hardware ManagementConsole. The required MCL level will be available to the IBM field representative.

EC levels can be tracked by accessing Resource Link, http://www.ibm.com/servers/resourcelink, go to Tools -> Machine Information.

How to Send Your CommentsYour feedback is important in helping to provide the most accurate and high-qualityinformation. Send your comments by using Resource Link at http://www.ibm.com/servers/resourcelink. Select Feedback on the navigation bar on the left. Be sure toinclude the name of the book, the form number of the book, the version of the book,if applicable, and the specific location of the text you are commenting on (forexample, a page number or table number).


xviii z9 EC System Overview


http://www.ibm.com/systems/z/pso/


http://www.ibm.com/developerworks/linux/linux390


http://www.ibm.com/systems/z/hardware/connectivity

http://www.ibm.com/servers/eserver/zseries/lspr






Summary of changes

Summary of changes for the System z9 EC System Overview, SA22-6833.

Table 1. Summary of Changes

Release Level Date Changes in Level

06a 04/09 This revision contains editorial changes and the followingtechnical changes:

v STP enhancements

v Crypto Express2 support for 13-19 personal accountnumbers (PANs)

v Support for 12x InfiniBand single data rate (12x IB-SDR)connection between any System z9 EC server and Systemz10™ server. There is no longer the restriction for theSystem z9 EC to be a standalone coupling facility.

06 01/09 This revision contains editorial changes and technical changesdescribing the Server Time Protocol (STP) enhancements.

05 05/08 This revision contains editorial changes and the followingtechnical changes:

v Support for InfiniBand coupling links (IFB) available on aSystem z9 EC dedicated Coupling Facility (CF).

04a 09/07 This revision contains editorial changes and the followingtechnical changes:

v Addition of Network Time Protocol (NTP) support to theServer Time Protocol (STP) feature.


© Copyright IBM Corp. 2005, 2009 xix

||||

|

||

||||

||||

Table 1. Summary of Changes (continued)

Release Level Date Changes in Level


v On/Off Capacity on Demand enhancements

v LPAR group capacity limit

v Multiple IPL path retry

v New Hardware Decimal Floating Point instructions

v CFCC Level 15 support

v RMF™ measurement enhancements

v OSA LAN idle value enhancement

v Dynamic link aggregation support for the z/VM®

environment

v OSA-Express2 network traffic analyzer

v Simultaneous capture of software and OSA hardwaretraces (QDIO diagnostic synchronization)

v Shared channels across multiple operating systems

v LDAP support for user authentication

v Power Estimation tool

v Tree-style Web user interface for the z9 Support Element

v Bolt-down kit for raised floor

v Change in HSA size for large system I/O configuration(S38)

v Updated and new HMC and/or Support Element windowsallow you to:

– Display power consumption data and internaltemperature of a specific CPC

– Display a list of logical channels and/or cryptosassociated with an assigned physical channel identifier(PCHID) that requires configure Off/On actions

– Create groups of LPARs and defines a group capacity

– Create, modify, or delete enterprise server (LDAP)definitions

– Display the node ID of each attached FICON control unit

– Display the actual formatted purge path extended (PPE)data from the channel record

– Display the fabric security attributes for an FCP channel.

v Integrated systems management support from the HMCprovides basic management of z/VM guests.


v Addition of Server Time Protocol (STP) support.


xx z9 EC System Overview

Chapter 1. Introduction

The IBM System z9 Enterprise Class (z9 EC), formerly known as System z9-109,represents the latest state of the art scalable server designed and optimized for OnDemand Business. The z9 EC provides an advanced combination of scalability,availability, reliability, security, and virtualization.

Built with modular book design, the z9 EC delivers higher total system capacity,memory and I/O bandwidth. The z9 EC is focused on providing a strongcombination of mainframe characteristics plus new functions designed aroundscalability, availability and security. The intent of the z9 EC is to help increase theflexibility to configure a server to meet current needs, provide a powerful andadvanced data serving environment, and help reduce risks and business costswithout sacrificing features or functions.

With a range of five models, the System z9 EC delivers enriched functions over theprevious System z servers:v Increased processing powerv Increased memoryv Improved availabilityv Increased I/O capacity.v Technology on demand.

The z9 EC allows virtualization of resources such as:

v Sharing without boundaries.

v Empowerment of business by applying intelligence to adapt and optimize tochanging requirements.

IBM

server

Figure 1-1. IBM System z9 EC


© Copyright IBM Corp. 2005, 2009 1-1

v Smart and secure management of global transactions.

v Positioning the mainframe at the center of a heterogeneous on demandinfrastructure.

To address the growing complexity of fiber optic connectivity in the InformationTechnology (IT) infrastructure, IBM Networking Services is introducing scalable fiberoptic cabling services to satisfy e-business infrastructure requirements at both theproduct-level and the enterprise-level. Refer to Chapter 8, “Cabling,” on page 8-1 formore information. You can also access Resource Link at http://www.ibm.com/servers/resourcelink and click Services on the navigation bar for the networkcabling services.

z9 EC HighlightsThe z9 EC is designed and optimized for On Demand Business. The z9 ECprovides:

v IBM System z9 Integrated Information Processor (zIIP)

v 24 new capacity settings provide granularity and the ability to offer a portion ofthe full capacity which allows greater flexibility to configure a server to meet yourneeds.

v FICON Express4 supporting 1, 2, or 4 Gbps with 4km and 10km

v Conversions of CPs, IFLs, zIIPs or ICFs to and from zAAPs

v Reduced impact of planned and unplanned server outages through:

– Enhanced book availability

– Redundant I/O interconnect

– Enhanced driver maintenance

– Dynamic oscillator switchover

– Concurrent MBA fanout card hot-plug

– System-initiated CHPID reconfiguration.

v Up to 54 Processor Units (PUs)

v Up to 128 GB of memory per book

v Increased bandwidth between memory and I/O; up to 16 STIs per book, 2.7GBps per I/O

v Improved device connectivity for Parallel Access Volumes through MultipleSubchannel Sets (MSS)

v Improved FICON throughput for Channel Command Word (CCW) and ModifiedData Indirect Address Word (MIDAW) applications

v Increased connectivity for Crypto, FICON, and OSA in one I/O cage

v OSA-Express2 1000BASE-T Ethernet

v Up to 336 FICON Express2 channels

v OSA Express2 OSN CHPID type for the OSA Network Control Program (OSANCP) support. OSA-Express2 OSN supports the Channel Data Link Control(CDLC) protocol to provide channel connectivity from the operating system to theIBM Communication Controller for Linux (CCL)

v Coupling using InfiniBand. A System z9 to System z10 connection is provided byan InfiniBand Single Data Rate (12x IB-SDR) fiber optical link with a link datarate of 3 GigaBytes per second (GBps) and a maximum link distance of 150meters (492 feet).

v Up to 64 concurrent simultaneous FICON Express2 I/O operations per channel

v Extended Capacity Backup and On/Off CoD test


1-2 z9 EC System Overview

|



v Server Time Protocol feature provides:

– Improved time synchronization for z9 EC, z9 BC, z990, and z890 servers

– Increased multisite sysplex distance to 100 km

– Coexistence of servers and coupling facilities (CFs) synchronized in an ETRnetwork with servers and CFs that are synchronized with Coordinated ServerTime (CST)

– Concurrent migration from an ETR network

– Messaging over ICB-4 links, ICS-3 links, and InfiniBand (IFB) links

– NTP client support. The NTP client attaches to an NTP server that will providetime accuracy across heterogeneous platforms in an enterprise

– Enhanced accuracy to an external time source utilizing pulse per second(PPS) output from an NTP server

– Continuous availability of NTP servers used as an external time source

– Enhanced STP recovery when the Internal Battery Feature is in use

– Ability to save the STP configuration and time information across Power onResets (POR) or power outages for a single or dual server STP-only CTN

– Automation of STP Coordinated Timing Network (CTN) reconfiguration usingthe System z Application Programming Interface (API).

v HiperSockets™ IPv6 support.

v Supports FCP Channel Protocol with N_Port ID Virtualization (NPIV)

v Integration of new Crypto functions such as Remote Keyload for ATMs andimplementation of ISO 16609 CBC Mode TDES for MAC

v OSA/SF enhancements to facilitate management of IP and MAC addresses

v OSA Layer2/Layer 3 transport enhancements to enable sharing

v On/Off Capacity on Demand enhancements

v CFCC Level 15 support

v Integrated systems management from the HMC

v GDPS® Enhancements to improve performance and availability.

The z9 EC facilitates server consolidation to allow you to continue to reduce thenumber of servers while hosting additional applications.

ScalabilityThe z9 EC delivers increased capacity through the following:

v Up to 128 GigaBytes of memory per book

v Up to 2.7 GigaBytes per second of bandwidth for data communication byusing up to 48 Self-Timed Interconnect (STI) host buses.

Connection of the Input/Output (I/O) subsystem to the Central ProcessorComplex (CPC) subsystem is done by using 48 Self-Timed Interconnect (STI)links each having a capacity of 2 Gigabytes per second (GBps).

v Double the number of Logical Partitions (LPARs) up to 60 LPARs, 15 perLogical Channel Subsystem.

v Enhanced book availability allows a single book, in a multibook server, to beconcurrently removed from the server and reinstalled during an upgrade or repairaction. Enhanced book availability is an extension of the support for ConcurrentBook Add (CBA) and is exclusive on the z9 EC.

v Redundant I/O Interconnect helps maintain critical connections to devices. Thez9 EC allows a single book, in a multibook server, to be concurrently removed


Chapter 1. Introduction 1-3

||

||

|

|

||

||

and reinstalled during an upgrade or repair, continuing to provide connectivity tothe server I/O resources using a second path from a different book.

v Dynamic Oscillator SwitchoverThe z9 EC has two oscillator cards, a primary and a backup. In the event of aprimary card failure, the backup card is designed to detect the failure, switchover, and provide the clock signal to the server transparently.

v MBA Fanout Card Hot-plugA Memory Bus Adapter (MBA) fanout card is designed to provide the path fordata between memory and I/O using Self-Timed Interconnect (STI) cables. Thez9 EC has a hot-pluggable and concurrently upgradeable MBA fanout cardavailable. Up to 8 MBA fanout cards are available per book for a total of up to 32with four books are installed. In the event of an outage, an MBA fanout card,used for I/O, can be concurrently repaired using redundant I/O interconnect.

v HCA1-O (optical) fanout cardA Host Channel Adapter (HCA) optical fanout card is designed to provide point topoint coupling over an Infiniband link. An IFB link is used by coupled servers topass information back and forth over high speed links in a Parallel Sysplexenvironment. Using this card, a System z9 server can connect to a System z10server over the high speed IFB link.

v Number of HiperSockets Expanded

Internet Protocol Version 6 (IPv6) is now available for HiperSockets, CHPID typeIQD. IPv6 is the protocol designed by the Internet Engineering Task Force (IETF)to replace Internet Protocol Version 4. IPv6 expands the IP address space from32 bits to 128 bits enabling a far greater number of unique IP addresses.

v Integrated Cluster Bus-4 (ICB-4)

The ICB-4 link is a member of the family of Coupling Link options. ICB-4operates at STI speeds, and like ICB-3, it is used by coupled servers to passinformation back and forth over high speed links in a Parallel Sysplexenvironment when the distance between servers is no greater than 7 meters or23 feet.

v Cryptographic options:– CP Assist for Cryptographic Function (CPACF), which delivers cryptographic

support on every PU with data encryption/decryption.– Crypto Express2, which supports the Federal Information Processing Standard

(FIPS). (Crypto Express2 replaces PCICA and PCIXCC.)– User Defined Extension (UDX), which is supported by Crypto Express2.

v Remote Keyload for ATM/POS refers to the process of loading Data EncryptionStandard (DES) keys to Automated Teller Machines (ATMs) and Point of Sale(POS) devices from a central administrative site without the need for personnel tovisit each machine to manually load DES keys.

v IBM System z9 Application Assist Processor (zAAP) is a specializedprocessor unit which provides an attractively priced Java execution environmentif you desire the traditional qualities of service and the integration advantages ofthe System z9 platform.

Note: The zAAP is a specific example of an assist processor that is knowngenerically as an Integrated Facility for Applications (IFA). The genericterm IFA often appears in panels, messages, and other online informationrelating to the zAAP.

v IBM System z9 Integrated Information Processor (zIIP) is a specialty enginedesigned to help improve resource optimization and lower the cost of eligibleworkloads enhancing the role of the mainframe as the data hub of the enterprise.



|

v FICON Express4 is designed to deliver increased performance compared toFICON Express2. It offers two uprepeated distance options 4KM and 10KM whenusing single mode fiber optic cabling.

v Up to 54 Processor Units (PUs)

There are five models, each model indicates the maximum number of PUsavailable for purchase (Models S08, S18, S28, S38, S54).

A PU is the generic term for the z/Architecture processor on the Multichip Module(MCM) that can be characterized as a:– Central Processor (CP) to be used by the operating system– Internal Coupling Facility (ICF) to be used by the CFCC– Integrated Facility for Linux (IFL)– Additional System Assist Processors (SAPs) to be used by the CSS– IBM System z9 Integrated Information Processor (zIIP)– IBM System z9 Application Assist Processor (zAAP).

With the introduction of z9 EC and the five-model structure, only one CP, ICF, orIFL must be purchased and activated for any model (refer to Table 1-1). PUs canbe purchased in single PU increments and are orderable by feature code. Thetotal number of PUs purchased may not exceed the total number available forthat model. One CP must be installed with or prior to any zAAPs being installed.The total number of zAAPs installed must be less than or equal to the sum of theunassigned CPs and active CPs installed on a single machine. The number ofspares and SAPs provided to you as standard PUs is as follows:– Two SAPs and two spares for Model S08– Four SAPs and two spares for Model S18– Six SAPs and two spares for Model S28– Eight SAPs and two spares for Model S38.– Eight SAPs and two spares for Model S54.

Table 1-1. System z9 Model Structure

ModelsProcessorUnits (PUs) Memory SAPs Spares STIs I/O Cages

MaximumChannels

S08 1 to 8 16 to 128GB

2 2 16 1 to 3 960

S18 1 to 18 16 to 256GB

4 2 32 1 to 3 1024

S28 1 to 28 16 to 384GB

6 2 48 1 to 3 1024

S38 1 to 38 16 to 512GB

8 2 64 1 to 3 1024

S54 1 to 54 16 to 512GB

8 2 64 1 to 3 1024

Server ConsolidationThe expanded capacity and enhancements to the I/O infrastructure facilitates theconsolidation of multiple servers into one z9 EC with increased memory, LogicalPartitions (LPARs), STIs, and an increase in the speed of an STI. The new z9 ECcan reduce the number of servers while hosting additional applications.

LCSSs allow you to install more operating system images and up to a total of 1024channels (up to 256 CHPIDs maximum per operating system) allowing each z9 ECto access more I/O and networking channels:



v ESCON® (up to 1024 channels)v FICON Express™ (up to 120 channels)v FICON Express2 and FICON Express4 (up to 336 channels)v OSA (up to 48 ports)v ISC-3 (up to 48 links)v ICB-4 (up to 16 links)v IFB (up to 16 links).

z9 EC ModelsThe z9 EC is offered in five models. The model is directly related to the number ofbooks included in the system (refer to “Books” on page 2-2). The number ofprocessor units purchased for each book is selected by feature codes and is notreflected in the system model number. The models are:

v IBM 2094-S08

This model supports a maximum of 8 PUs, a maximum of 128 GB of memory,and 16 STIs.

v IBM 2094-S18


v IBM 2094-S28


v IBM 2094-S38


v IBM 2094-S54


The system model number does not reflect the number of processors that areenabled for use. It reflects only the maximum number of customer-used processorsthat can be enabled when purchased. The model number also reflects the numberof books installed. However, as a convenience in discussing various modelconfigurations as well as a value that will be returned by the STSI instruction,processor capacity is identified using a 3xx notation for what is called a modelcapacity indicator, where xx is the number of activated Central Processor Units.The 700 model would indicate no active CPs and the 701 to 754 models wouldindicate 1 to 54 active CPs.

PerformanceWith the expanded capacity of the z9 EC and enhancements to the I/Oinfrastructure, IBM continues to facilitate the consolidation of multiple servers intoone z9 EC with a substantial increase in:

v Memory

v Logical Partitions (LPARs)

v STIs and STI speed.

IBM’s Large Systems Performance Reference (LSPR) method providescomprehensive z/Architecture processor capacity data for different configurations ofCentral Processor Units across a wide variety of system control program andworkload environments. The LSPR ratios defining the capacity for z9 EC will be



associated with the 701 to 754 model names as discussed in the previous section.For more information on LSPR, refer to http://www.ibm.com/servers/eserver/zseries/lspr.

Resource LinkResource Link is a key component in getting your z9 EC server up and running andmaintained. Key areas of improvement include the Customized planning aids,CHPID mapping tool, Customer Initiated Upgrades (CIU), Fiber Cabling Service,and additional Education courses. Refer to Appendix B, “Resource Link,” on pageB-1 for detailed information about Resource Link and all the functions that it canassist you with your z9 EC processor.

Fiber Optic CablingIBM Site and Facilities Services has a comprehensive set of scalable solutions toaddress IBM cabling requirements from product-level to enterprise level. IBM Siteand Facilities Services is designed to deliver convenient, packaged services to helpreduce the complexity of planning, ordering, and installing fiber optic cables. Referto Chapter 8, “Cabling,” on page 8-1 for additional information.

z/ArchitectureThe zSeries® is based on z/Architecture which is 64-bit addressing that eliminatesbottlenecks associated with lack of addressable memory by making the addressingcapability virtually unlimited (up to 16 Exabytes compared to the prior capability of 2Gbytes in S/390 systems.) As an application’s single systems image increases insize and demands a linear increase in server capacity, the requirement for greateraddressability actually increases geometrically. As addressability begins to beconstrained, the lack of real storage increases overhead and ultimately constrainssystem growth. This situation exacerbates critical situations in which explosivesystems growth has been a result of successful eBusiness implementation

IBM’s z/Architecture has added the following enhancements that are supported onz9 EC:v List-Directed Initial Program Load (IPL)v Hexadecimal-Floating-Point (HFP) Multiply-and-Add/Subtract facility (ESA/390

and z/Architecture modes)v Message-Security Assist (ESA/390 and z/Architecture modes)v Long-Displacement facility (z/Architecture mode)v Extended-I/O-Measurement-Block facility (ESA/390 and z/Architecture modes)v Extended-I/O-Measurement-Word facility (ESA/390 and z/Architecture modes).

For more detailed information on these enhancements, refer to “z/Architecture” onpage 2-20 and the z/Architecture Principles of Operation.

IBM’s z/Architecture also supports the following functions:

v Enables optimal use of memory resources.

v Provides for migration flexibility.

v Allows non-disruptive test and migration and provides for ease of transition forsystems support staff.

v Ensures maximum transparency to the operating system and completetransparency to system applications.

v Offers a doubleword Indirect Data Address Word (IDAW) which allows allprograms to designate 64-bit data addresses for I/O. Use of 64-bit operands andgeneral registers for all I/O instructions is added.






v Gigabit Ethernet architecture is expanded to allow 64-bit Queued I/O queuingstructures, in addition to the doubleword Indirect Data Address Word for dataaddresses for I/O.

v IEEE Floating-Point architecture has twelve instructions for 64-bit integerconversion.

v New Hardware Decimal Floating Point instructions in support of the IEEE (Draft)Standard for Floating-Point Arithmetic P754.

Upgrade ProgressionzSeries models z900, z990, and System z z9 BC are the only models that can beupgraded to a z9 EC. An upgrade includes all frames, cages, support cards, andnew I/O features.

Unsupported Features/FunctionsThis section lists the features/functions that are not supported on z9 EC and arecommended alternative, if applicable.

OSA-Express Token RingThe OSA-Express Token Ring, feature code 2367, is not supported on thez9 EC and cannot be carried forward when upgrading from z900 or z990.

ICB-2 LinksICB-2 links are not supported on the z9 EC

ISC-3 Links in Compatibility ModeISC-3 links in compatibility mode are not supported on the z9 EC (CHPIDtypes CFS and CFR).



Chapter 2. Hardware Characteristics

This chapter describes the hardware features and functions for the five z9 ECmodels, S08, S18, S28, S38, and S54 (machine type 2094).

Note: You can also refer to the System z9 Enterprise Class Installation Manual forPhysical Planning, available on Resource Link at http://www.ibm.com/servers/resourcelink, for initial system physical planning requirements.

System Frame ConfigurationThe z9 EC frames are enclosures built to Electronic Industry Association (EIA)standards. The ’A’ or CEC frame and ’Z’ or I/O feature frame make up the z9 ECconfiguration as shown in the following figure.

The basic ’A’ frame consists of:

v Processor Cage (or Central Processor Complex (CPC)) which consists of:

– One to four processor books (up to five models available: S08, S18, S28,S38, and S54), each book provides:- Twelve PUs (CP, IFL, ICF, SAPs, or zAAP)- Two spare PUs.- Four or eight memory cards (physical card sizes are 4GB, 8GB, and

16GB),

v Input/Output (I/O) cage for channel attachment capability

v Cooling Equipment.

Front ViewZ Frame A Frame

IntegratedBatteries

SystemPowerSupply

I/O Cage 3

SupportElements

I/O Cage 2

IntegratedBattery

CoolingEquipment

CentralProcessorComplex

(CPC)Cage

I/O Cage 1

COOLINGACTIVE

COOLINGACTIVE

COOLINGACTIVE

COOLINGACTIVE

BATTERY ENBLD

J1

IBMIntegrated Battery

BATTERY ENBLD

J1


BATTERY ENBLD

J1


JT0

J00

D1 D2

D3 D4 D5

D6 D7 D8

J00

J00

J00

J00

J00

J00

J01

J01

J01

J01

J01

J01

J01

J00

J01

JT0

J00

D1 D2

D3 D4 D5

D6 D7 D8

J00

J00

J00

J00

J00

J00

J01

J01

J01

J01

J01

J01

J01

J00

J01

JT0

J00

D1 D2

D3 D4 D5

D6 D7 D8

J00

J00

J00

J00

J00

J00

J01

J01

J01

J01

J01

J01

J01

J00

J01

JT0

J00

D1 D2

D3 D4 D5

D6 D7 D8

J00

J00

J00

J00

J00

J00

J01

J01

J01

J01

J01

J01

J01

J00

J01

Figure 2-1. z9 EC Frame Configuration





The basic ’Z’ frame consists of:v Two optional I/O cagesv Two internal Support Elements (SEs)v System Power Supply.

Both ’A’ and ’Z’ frames can include the optional Internal Battery Features (IBFs).They are installed in pairs in the top of the ’A’ and ’Z’ frames. There can be up tosix IBFs installed. Refer to “Internal Battery Feature-360” on page 2-18 for moreinformation.

Note: If all book positions are not used in the CPC, the following are required:

v Air block book for book positions not containing a physical book

v Distributed Converter Assembly (DCA) air block book for unpopulatedDCA positions

v External Timer Reference (ETR) cards air flow book for unpopulatedpositions.

Central Processor Complex (CPC)The Central Processor Complex (CPC) is comprised of one to four booksconnected by a ring topology of two concentric loops. This design provides growthpaths up to a 54 engine system where each of the 54 PUs have full access to allsystem resources, specifically memory and I/O.

Each CPC cage basically consists of:

v One to four books, each book containing:– One Multichip Module (MCM)– Up to eight memory cards– Up to a combination of eight Memory Bus Adapters (MBAs – two STIs per

MBA) and Host Channel Adapters (HCAs).

v Distributed Converter Assembly (DCA) cards (two per book, maximum eight persystem)

v Two Oscillator (OSC) cards.

BooksA book plugs into one of four slots in the CPC cage of the z9 EC. A system has atleast one book and may have up to four. A book is a physical package containingmemory, an MCM, and the MBAs and HCAs. Naming of the books begins withBook 0, followed by Book 1, Book 2, and Book 3. The controlling book is the lowestnumber book functioning in the configuration. Figure 2-2 on page 2-3 displays thebook positions in the CPC cage of the z9 EC.



The number of processor books in your system determines which z9 EC model youhave. Table 2-1 identifies the z9 EC models.

Table 2-1. z9 EC Models

Model CPs*ICFsOpt*

IFLsOpt*

zAAPsOpt**

zIIPsOpt**

SAPsStd

SAPsOpt

SparePUs PUs

Memory(GB)

CBU, CIU,On/Off CoD,CUoD PUsAvailable

S08 0 - 8 0 - 8 0 - 8 0 - 4 0 - 4 2 0 - 5 2 12 16 to128

0 - 8

S18 0 - 18 0 - 16 0 - 18 0 - 9 0 - 9 4 0 - 13 2 24 16 to256

0 - 18

Figure 2-2. z9 EC Book Positions


Chapter 2. Hardware Characteristics 2-3

Table 2-1. z9 EC Models (continued)

Model CPs*ICFsOpt*

IFLsOpt*

zAAPsOpt**

zIIPsOpt**

SAPsStd

SAPsOpt

SparePUs PUs

Memory(GB)

CBU, CIU,On/Off CoD,CUoD PUsAvailable

S28 0 - 28 0 - 16 0 - 28 0 - 14 0 - 14 6 0 - 21 2 36 16 to384

0 - 28

S38 0 - 38 0 - 16 0 - 38 0 - 19 0 - 19 8 0 - 24 2 48 16 to512

0 - 38

S54 0 - 54 0 - 16 0 - 54 0 - 27 0 - 27 8 0 - 24 2 54 16 to512

0 - 54

Notes:1. * - Only one CP, ICF, or IFL is required for any model. The total number of PUs purchased may not exceed the

total number available for that model.2. **- One CP must be installed with or prior to any zIIPs or zAAPs that are installed. The total number of zIIPs and

zAAPs installed may be less than or equal to the number of unassigned and installed CPs installed on a singlemachine.

3. PU selection is completed by identifying the number of features when ordering4. Minimum storage is 16GB for all models.

Multichip Module (MCM)The z9 EC models S08 through S38 use 12 PU MCM and Model S54 uses 16 PUMCM.

Processor Unit (PU)The z9 EC models contain the number of physical Processor Units (PUs) listed inTable 2-1 on page 2-3. For each z9 EC model you must select at least one CP, IFL,or ICF as shown in Table 2-1 on page 2-3. Any remaining PUs may be assigned asadditional SAPs or additional spares, or may be assigned to optional functions suchas ICFs, IFLs, zIIPs, zAAPs, CPs, On/Off CoD, or CBU engines, as indicated inTable 2-1 on page 2-3. The total number of CPs, SAPs, spares, ICFs, IFLs, andCBU engines activated may not exceed the number of PUs listed for that model.

Central Processor (CP): A Central Processor (CP) is a PU that has thez/Architecture and ESA/390 instruction sets. It can run z/VM, z/OS, Linux, and TPFoperating systems and the Coupling Facility Control Code (CFCC). z9 ECprocessors operate only in LPAR mode; consequently all CPs are dedicated to apartition or shared between partitions. Reserved CPs can also be defined to alogical partition, to allow for nondisruptive image upgrades.

All CPs within a configuration are grouped into a CP pool. Any z/VM, z/OS, andTPF operating systems can run on CPs that were assigned from the CP pool.Within the capacity of the z9 EC books, CPs can be concurrently added to anexisting configuration permanently by using CIU, CUoD, or temporarily by usingOn/Off CoD or CBU.

Internal Coupling Facility (ICF): An ICF provides additional processing capabilityexclusively for the execution of the Coupling Facility Control Code (CFCC) in acoupling facility LPAR. Depending on the model, optional ICF may be ordered. ICFscan only be used in coupling facility logical partitions, but can be shared ordedicated, because only CFCC runs on these PUs. Software Licensing charges arenot affected by the addition of ICFs. For more information, refer to “CouplingFacility” on page 6-6.



Integrated Facility for Linux (IFL): An IFL provides additional processingcapacity exclusively for Linux workloads. Optional IFL features may be ordered.These IFLs can only be used in Linux or z/VM logical partitions but those IFLs canbe shared or dedicated, since only Linux software runs on these CPs. Softwarelicensing charges are not affected by the addition of IFLs. For more information onSoftware licensing, contact your IBM Representative.

System z9 Applications Assist Processor (zAAP): The System z9 ApplicationAssist Processor is a specialized processor unit that provides a Java executionenvironment for a z/OS environment. zAAPs are designed to operateasynchronously with the CPs to execute Java programming under control of theIBM Java Virtual Machine (JVM).

When configured with CPs within logical partitions (LPARs) running z/OS, zAAPsmay help increase CP productivity and may contribute to lowering the overall costof computing for z/OS and z/OS.e Java technology-based applications. zAAPs aredesigned to operate asynchronously with the CPs to execute Java programmingunder control of the IBM Java Virtual Machine (JVM). This can help reduce thedemands and capacity requirements on CPs which may then be available forreallocation to other System z9 workloads.

The IBM JVM processing cycles can be executed on the configured zAAPs with noanticipated modifications to the Java application. Execution of the JVM processingcycles on a zAAP is a function of the Software Developer’s Kit (SDK) 1.4.1 forSystem z9, zSeries, z/OS V1R6, and Processor Resource/Systems Manager™

(PR/SM™).

Note: The zAAP is a specific example of an assist processor that is knowngenerically as an Integrated Facility for Applications (IFA). The generic termIFA often appears in panels, messages, and other online information relatingto the zAAP.

IBM System z9 Integrated Information Processor (IBM zIIP): The IBM zIIP is aspecialty engine designed to help improve resource optimization, enhancing the roleof the server as the data hub of the enterprise. The z/OS operating system, actingon the direction of the program running in SRB mode, controls the distribution ofwork between the general purpose processor (CP) and the zIIP. Using a zIIP canhelp free capacity on the general purpose processor.

System Assist Processor (SAP): A SAP is a PU that runs the channelsubsystem Licensed Internal Code (LIC) to control I/O operations. One of the SAPsin a configuration is assigned as a Master SAP, and is used for communicationbetween the z9 EC books and the Support Element. All SAPs perform I/Ooperations for all logical partitions.

A standard SAP configuration provides a very well balanced system for mostenvironments. However, there are application environments with very high I/O rates(typically some TPF environments), and in this case additional SAPs can increasethe capability of the channel subsystem to perform I/O operations. Additional SAPscan be added to a configuration by either ordering optional SAPs or assigning somePUs as SAPs. Orderable SAPs may be preferred since they do not incur softwarecharges, as might happen if PUs are assigned as SAPs.

MemoryEach z9 EC CPC has its own processor memory resources. CPC processormemory can consist of both central and expanded storage.



Central Storage (CS)Central storage consists of main storage, addressable by programs, and storagenot directly addressable by programs. Nonaddressable storage includes theHardware System Area (HSA). Central storage provides:v Data storage and retrieval for the Processor Units (PUs) and I/Ov Communication with PUs and I/Ov Communication with and control of optional expanded storagev Error checking and correction.

Part of central storage is allocated as a variable-sized Hardware System Area(HSA) which is not addressable by application programs. Factors affecting size aredescribed in “Hardware System Area (HSA)” on page 2-7.

In z/Architecture, storage addressing is 64 bits, allowing for an addressing range upto 16 Exabytes. Consequently, all central storage in a z9 EC (up to 512GB) can beused for central storage.

Key-controlled storage protection provides both store and fetch protection. Itprevents the unauthorized reading or changing of information in central storage.

Each 4KB block of storage is protected by a 7-bit storage key. Forprocessor-initiated store operations, access key bits 0 – 3 from the active programstatus word (PSW) are compared with bits 0 – 3 from the storage key associatedwith the pertinent 4KB of storage to be accessed. If the keys do not match, thecentral processor is notified of a protection violation, the data is not stored, and aprogram interruption occurs. The same protection is active for fetch operations if bit4 of the storage key (the fetch protection bit) is on. Refer to System z9 ProcessorResource/Systems Manager Planning Guide for more information on centralstorage.

Expanded Storage (ES)Expanded storage can optionally be defined on System z9 servers. It is controlledby the control program, which can transfer 4KB pages between expanded storageand central storage. The control program can use expanded storage to reduce thepaging and swapping load to channel-attached paging devices in astorage-constrained environment and a heavy-paging environment.

The z9 EC models offer a flexible storage configuration which streamlines theplanning effort by providing a single storage pool layout at IML time. The storage isplaced into a single pool which can be dynamically converted to ES and back to CSas needed. Logical partitions are still specified to have CS and optional ES asbefore. Activation of logical partitions as well as dynamic storage reconfigurationswill cause LPAR to convert the storage to the type needed.

No new software support is required to take advantage of this function.

The control program initiates the movement of data between main storage (theaddressable part of central storage) and expanded storage. No data can betransferred to expanded storage without passing through main storage. With z9 ECmodels, a dedicated move page engine assists in efficiently transferring databetween main and expanded storage. Refer to System z9 ProcessorResource/Systems Manager Planning Guide for more information on expandedstorage.



Memory CardsUp to 8 memory cards reside on the processor board within the book. The physicalcard capacity of each card is 4GB, 8GB, and 16GB. The following table displays thecharacteristics of the memory cards for z9 EC.

Note: The sum of enabled memory on each card is the amount available for use inthe system.

Table 2-2. z9 EC Memory Card Characteristics

Memory Card Feature Code GB Supported

4GB 1504 16GB, 32GB

8GB 1508 32GB, 48GB, 64GB

16GB 1516 64GB, 80GB, 96GB, 112GB,128GB

The following list contains some general rules for memory.

v Each book may contain different physical capacities from other books.

v The physical memory card capacity is not required to be identical on each bookinstalled in the system.

v Larger capacity cards may be used for repair actions and manufacturingsubstitution. LICCC will dial down to ordered size.

v Memory downgrades are not supported.

v Minimum memory orderable on any model is 16GB.

v Memory is only upgradeable in 16GB increments between the defined minimumand maximum.

v Changing an installed physical memory card is not a disruptive action usingEnhanced Book Availability.

v LICCC dialing is utilized to offer concurrent memory upgrades within the physicalmemory card installed.

v The Memory LICCC record for each book is combined with the PU LICCC recordfor each book. Both memory and PU LICCC are shipped on a single CD.

For stocking and service flexibility, larger capacity memory cards may be used toreplace smaller capacity cards. With the proper code they are made into logicalequivalents. z9 EC models will logically make larger capacity cards the equivalentof smaller capacity cards.

Hardware System Area (HSA)The HSA contains the CPC Licensed Internal Code (LIC) and configurationdependent control blocks. HSA is not available for program use.

The HSA size varies according to:v Size and complexity of the system I/O configurationv Whether or not dynamic I/O configuration is enabledv Whether or not concurrent code changes are authorized.

Note: The z9 EC models support 63K per channel subsystem. The subchannelcapabilities affect HSA storage requirements to accommodate the associatedconfiguration tables. The machine no longer has a unique HSA subchannellimit.



The HSA Estimation Tool, available on Resource Link at http://www.ibm.com/servers/resourcelink, estimates the HSA size for the specified configuration andproduct.

The HSA allocation can vary according to the I/O configuration. The followingsystem I/O configurations require the given HSA size on a z9 EC.

v Small system I/O configuration (S08): 1344 MB– 1 LCSS– 2 logical partitions– 4096 devices– 96 physical control units– Concurrent code change not authorized– 64 channels– Dynamic I/O not enabled.

v Large system I/O configuration (S38): 4288 MB– 4 LCSSs– 60 logical partitions– 65280 devices per LCSS for subchannel set 0– 65535 devices per LCSS for subchannel set 1– Concurrent code change authorized– Dynamic I/O enabled.

Intermediate HSA sizes exist for configuration sizes in between the given small andlarge I/O configuration examples.

Your HSA size includes 5MB to 25MB for coupling facility depending on the numberof logical partitions defined and on the installed storage. The HSA size required forconcurrent code patch is approximately 100MB. The HSA granularity is fixed at 64MB.

If future additions of I/O are planned (concurrent conditioning), be sure to provideenough HSA for additional devices by specifying a large enough number of totaldevices in HCD or IOCP (dynamic I/O configurations only), or by overdefining yourIOCDS to define anticipated additions (both dynamic I/O and non-dynamic I/Oconfigurations).

Error Checking and Correction (ECC)Data paths between central storage and expanded storage (if configured), andbetween central storage and the central processors and channels are checkedusing either parity or Error Checking and Correction (ECC). Parity bits are includedin each command or data word. ECC bits are stored with data in central storage.ECC codes apply to data stored in and fetched from central storage. Memory ECCdetects and corrects single bit errors. Also, because of the memory structuredesign, errors due to a single memory chip failure are corrected. Unrecoverableerrors are flagged for follow-on action. ECC on z9 EC models is performed on thememory data bus as well as memory cards.

MBA and HCA1-O fanout cardsz9 EC machines can have one to four books installed in the CPC. Each bookincludes a combined count of eight fanout slots and each fanout slot canaccommodate one fanout card with a maximum combined count of 32 fanout cards.There are two main fanout cards - an MBA fanout card and an HCA1-O fanoutcard.



Each MBA supports two 2.0 GigaBytes per second (GBps) Self-Timed Interconnect(STI) connections for ICB-4 connections and 2.7 GBps for I/O connections. EachHCA1-O supports a two-port 12x IB-SDR optical link operating at a link rate of 3GBps. The z9 EC is designed to contain up to a maximum of 64 total connections.

For more information on STI connections, refer to “Availability Guidelines for z9 ECModels” on page 4-4.

Distributed Converter Assembly-CXT (DCA-CXT) CardsThe Distributed Converter Assembly-CXT (DCA-CXT) cards are DC-to-DC convertercards in the CPC cage that convert −350 volts DC to logic voltages. There are aminimum of two DCA cards per book in the CPC with a maximum of eight per fourbook system.

Oscillator (OSC) CardTwo quarter high Oscillator (OSC) cards (FC H310) are standard on z9 EC. Thesecards provide the main oscillator for z9 EC, including the STI, reference, and MBAclocks.

External Time Reference (ETR) CardThere are two optional (quarter high) External Time Reference (ETR) cards (FC6155) on a z9 EC. Each ETR card connects to a different Sysplex Timer in anexpanded availability configuration. Each card has a single port supporting anMT-RJ fiber optic connector to provide the capability to attach to a Sysplex Timerand a PPS input port supporting a coaxial connector to provide the capability toattach to the PPS output of an NTP server. The Sysplex Timer has an opticaltransceiver that supports an ESCON Duplex connector.

Note: The System z9 server ETR feature does not support a multimode fiber opticcable terminated with an ESCON Duplex connector. However, 62.5 micronmultimode ESCON Duplex jumper cables can be reused to connect to theETR feature. This is done by installing an MT-RJ/ESCON Conversion kitbetween the ETR feature MT-RJ port and the ESCON Duplex jumper cable.

The Sysplex Timer is a centralized time source that sets the Time-of-Day (TOD)clocks in all attached servers to maintain synchronization. The Sysplex Timer canprovide the stepping signal that ensures that all TOD clocks in a multisystemenvironment increment in unison to permit full read/write data sharing with integrity.

If you are not using Server Time Protocol (STP), the Sysplex Timer is a mandatoryhardware requirement for a sysplex consisting of more than one CPC. Additionally,the CPCs must be connected to the same External Time Reference (ETR) network.

If you are using STP, the Sysplex Timer is not required for an STP-only CoordinatedTiming Network (CTN). STP is designed for servers that have been configured to bein a Parallel Sysplex or a basic sysplex (without a Coupling Facility) as well asservers that are not in a sysplex, but need to be time synchronized.

For more information on the parallel sysplex environment, refer to Chapter 6,“Sysplex Functions,” on page 6-1.

The System z9 server, implemented in the server’s Support Element LicensedInternal Code (LIC), requires the ETR Network ID of the attached Sysplex Timernetwork to be manually set in the Support Element at installation time. This function



|||||||

|||

||||

||

checks that the ETR Network ID being received in the timing signals via each of theserver’s two ETR ports matches the ETR Network ID manually set in the server’sSupport Element. This function provides greater checking; therefore, it helpseliminate cabling errors where either server ETR port may be incorrectly connectedto Sysplex Timer units of an incorrect Sysplex Timer ETR network. If the ETRNetwork ID received on one or both server ETR ports does not match the value setin the server’s Support Element, that ETR port state is made semi-operational bythe server. Timing signals are still received from the semi-operational ETR port, butare not used for stepping the server TOD clock. This has some importantoperational considerations at sysplex system IPL time as well as for running sysplexsystems. Another important value of this function is that it allows verification ofcabling connectivity from the Sysplex Timer to the z9 EC server prior to IPLingz/OS.

If the z9 EC server is using STP, configured in a Mixed CTN, and directlyconnected to the Sysplex Timer, connections to the ETR ports are required. If thez9 EC server is using STP and configured in an STP-only CTN using NTP withpulse per second as the external time source, cables are required from the PPSoutput of the NTP server to the PPS input port on the External Time Reference(ETR) card.

I/O CagesThere is always an I/O cage in the bottom of the ’A’ frame. An I/O cage is includedeven if no I/O cards are in the configuration. The z9 EC supports up to three totalI/O cages. Additional I/O cages can be driven by either the total number of cards ina configuration, plan-ahead for additional I/O, or because of the cage power budget.The placement of the second I/O cage is at the bottom of the ’Z’ frame. Theplacement of the third I/O cage is at the top of the ’Z’ frame.

The I/O cage(s) support ESCON, FICON, ISC-3, OSA, and Crypto features. An I/Ocage allows you to add channels up to the amount supported by a particular I/Ocage and the CPC. There is a system maximum of 1024 channels.

Note: Installation of I/O cages is disruptive.

Figure 4-1 on page 4-5 is an example of the I/O cage layout for z9 EC.

I/O FeaturesThe I/O cards that are supported in the I/O cages are shown in Table 2-3 on page2-11. There are a total of 84 slots in three I/O cages (28 I/O slots per cage). Youcan also refer to Chapter 5, “I/O Connectivity,” on page 5-1 for more detailedinformation on the I/O channels and adapters.

Notes:

1. Crypto features utilize I/O slots. They have PCI cryptographic cards (eitheraccelerator cards or coprocessors). They do not have ports and do not use fiberoptic cables. They are not defined in the IOCDS and therefore do not receiveCHPID numbers, however, they are assigned PCI cryptographic numbers.

2. ICB, specifically ICB-3. (connects z9 EC to z990, z900 or z800), has a specialSTI-3 distribution card occupying I/O slots.



||||||

Table 2-3. I/O Channels Summary per System

Feature NameFeatureCode

MinimumFeatures

Maximum I/Oslots used by

FeaturesMaximum

Connections

Ports/Channels/Increments per

FeaturePurchase

Increments

ESCON, 16-port 2323 01 692 1024 channels2 16 channels 3 4 channels 4

FICON Express 23192320

01 60 120 channels 2 channels 1 feature

FICON Express2 33193320


FICON Express4 332133223324


OSA-Express 136413651366236423652366

0 24 48 ports 2 ports 1 feature

OSA-Express2 3364336533663368

0000

24242424

48 ports48 ports48 ports24 ports

2 ports2 ports2 ports1 port

1 feature1 feature1 feature1 feature

STI-35

ICB-3 link0993 0

018

N/AN/A

16 links62 outputs

N/AN/A

1 link

ICB-4 3993 01 N/A7 16 links6 N/A 1 link

ISC-38 0219 01 12 48 links6 4 links 1 link9

HCA1-O (IFB) 0167 01 N/A7 16 links6 2 links 2 links / 1feature

Crypto Express2 0863 0 or 2 81o 16 coprocessors 2 coprocessors 1 feature

Notes:

1. A minimum of one I/O feature (ESCON, FICON) or one coupling link (ICB, IFB, ISC-3) is required.

2. A maximum of 48 ESCON features/720 active channels on Model S08. A maximum of 64 FICON features/up to192 channels on S08.

3. Each ESCON feature has 16 channels (15 channels may be activated, one channel is reserved as a spare).

4. ESCON channels are purchased in increments of four and are activated via Licensed Internal CodeConfiguration Control (LIC CC). Channels are activated equally across all installed 16-port ESCON features forhigh availability.

5. The STI distribution cards, which support ICB-3, reside in the I/O cage. Each STI distribution card occupies oneI/O slot.

6. The maximum number of coupling links combined (ICs, ICB-3s, ICB-4s, IFBs, and active ISC-3 links) cannotexceed 64 per server.

7. ICB-4s and IFBs do not require connectivity to a card in the I/O cage. ICB-4s and IFBs are not included in themaximum feature count for I/O slots.

8. A maximum of 48 ISC-3s can be defined in peer mode (operating at a link data rate of 2 Gbps).

9. It is recommended that an initial order for ISC-3 include two links. ISC-3 is offered in one link increments. Whentwo links are purchased, two ISC-3 features are shipped and activated links are balanced across ISC-3 featuresfor high availability.

10. These features can only be carried forward on an upgrade. The maximum increments/ports of each of these willnever be more than 12 features, 24 ports.



Distributed Converter Assembly-CX (DCA-CX) CardsThe Distributed Converter Assembly-CX (DCA-CX) cards are DC-to-DC convertercards in the I/O cages that convert −350 volts DC to logic voltages. There are twoDCA-CX cards in each I/O cage.

PSC24V CardThe PSC24V card is a Power Sequence Control (PSC) card used to turn on/offspecific control units from the CPC. The PSC24V card in the I/O cage provides thephysical interface between the cage controller and the PSC boxes, located outsidethe I/O cage in the system frame. Up to four PSC boxes can be attached with twintail cables to the two D15 connectors on the PSC24V card. Each PSC box thenprovides 16 power control interfaces to external devices. You can only have onePSC per I/O cage.

Note: The PSC24V card is not hot pluggable.

For more information on PSC, refer to “Power Sequence Control” on page 2-19.

Internet Protocol Version 6 for HiperSocketsInternet Protocol Version 6 (IPv6) support is now being offered for HiperSockets,CHPID type IQD. IPv6 is the protocol designed by the Internet Engineering TaskForce (IETF) to replace Internet Protocol Version 4 (IPv4) to satisfy the demand foradditional IP addresses. Refer to “HiperSockets” on page 5-30 for more information.

Cooling EquipmentThe Modular Cooling Units (MCUs) are the main components of the z9 EC models’hybrid refrigeration subsystem located in the ’A’ frame. Each MCU cools the MCMin one or two books. The MCUs are not redundant and backup fans provide thecooling in case of an MCU failure. MCUs are made up of three subsystems:v Modular Refrigeration Unit (MRU)v Motor Scroll Assembly (MSA)v Motor Drive Assembly (MDA).

System Power SupplyThe System Power Supply located in the top of the ’Z’ frame provides the controlstructure to support the z9 EC power requirements for the CPC cage and three I/Ocages.

The z9 EC power subsystem basic components include:

v Bulk Power Assembly (BPA) - provides the prime power conversion and highvoltage DC distribution

v Base Power Interface (BPI) - main power controller and cage controller for theBPA

v Base Power Distribution (BPD) - distributes -350 VDC, Ethernet, and RS422communications to logic cage power Field Replaceable Units (FRUs)

v Bulk Power Fan (BPF) - cooling device

v Bulk Power Regulator (BPR) - main front end power supply which converts linevoltage to regulated -350 VDC

v Bulk Power Enclosure (BPE) - metal enclosure which contains the back plane

v Distributed Converter Assemblies (DCAs)



v Supporting software.

The BPI is the principal control node for the z9 EC diagnostic/service andpower/cooling system. It is the cage controller for the BPA cage and connects toboth Ethernet service networks. It provides the Ethernet connection to the SupportElement and also the DCAs.

Multiple Subchannel Sets (MSS)A new Multiple Subchannel Sets (MSS) structure for z9 EC allows increased deviceconnectivity for Parallel Access Volumes (PAVs). Two subchannel sets per LogicalChannel Subsystem (LCSS) are designed to enable a total of 63.75K subchannelsin set-0 and the addition of 64K-1 subchannels in set-1. MSS is supported byESCON (CHPID type CNC), FICON when configured as CHPID type FC or FCV,and z/OS.

Logically Partitioned (LPAR) ModeLPAR mode is the mode of operation for the z9 EC. It allows you to:v Define ESA/390, ESA/390 TPF, coupling facility, and Linux-only logical partitionsv Define and use up to 256 GB (minus HSA) in a single LPAR.v Dynamically reconfigure storage between logical partitions.

You can define and activate up to 60 logical partitions for each CPC.

After you define and activate an ESA/390 or ESA/390 TPF logical partition, you canload a supporting operating system into that logical partition.

Processor Resource/Systems Manager (PR/SM) enables logical partitioning of theCPC.

Resources for each logical partition include:v Processor Units (CPs, ICFs, IFLs,zIIPs, or zAAPs)v Storage (central storage and expanded storage)v Channels.

Processor UnitsOn z9 EC, PUs can be used within a logical partition as Central Processors (CPs),Internal Coupling Facilities (ICFs), Integrated Facilities for Linux (IFLs), System z9Integrated Information Processor (zIIPs), or System z9 Application AssistProcessors (zAAPs). The initial allocation of CPs, ICFs, IFLs, zIIPs, and zAAPs to alogical partition is made when the logical partition is activated.

Within a logical partition on z9 EC models, they may be used as follows:

v CPs can be dedicated to a single logical partition or shared among multiplelogical partitions. The use of CP resources shared between logical partitions canbe limited and modified by operator commands while the logical partitions areactive. CPs that are dedicated to a logical partition are available only to thatlogical partition. z/OS and z/VM support up to 24 CPs in a single LPAR on the z9EC. The maximum number of combined zAAPs and CPs supported in a singleLPAR is 24.

v ICFs, IFLs, and zAAPs are available as orderable features on z9 EC models foruse in a logical partition. ICFs are available as a feature for use in a CouplingFacility (CF) logical partition (refer to “Internal Coupling Facility (ICF)” on page2-4 for additional information). IFLs are available as a feature for running Linux



(refer to “Integrated Facility for Linux (IFL)” on page 3-4 for additionalinformation). zAAPs are available as a feature for providing special purposeassists that execute JAVA programming under control of the IBM Java VirtualMachine (JVM) (refer to “System z9 Applications Assist Processor (zAAP)” onpage 2-5 for additional information).

Dynamic ICF Expansion is a function that allows an ICF logical partition definedwith dedicated ICFs to acquire additional processing power from the LPAR pool ofshared CPs or shared ICFs being used to execute production and/or test work onthe system. Dynamic ICF Expansion is standard on all z9 EC models that have atleast one ICF. Refer to “Dynamic ICF Expansion” on page 10-3 and the System z9Processor Resource/Systems Manager Planning Guide for additional details of thisfunction.

StorageBefore you can activate logical partitions, you must define central storage andoptional expanded storage to the logical partitions. Refer to “Central Storage (CS)”on page 2-6 and “Expanded Storage (ES)” on page 2-6 for more information.

All installed storage is initially configured as central storage. Each individual logicalpartition is limited to a maximum of 2 GB (in ESA/390 mode) or 256 GB (inz/Architecture mode) of central storage. When a logical partition is activated, thestorage resources are allocated in contiguous blocks.

For z9 EC models, logical partition central storage granularity is a minimum of 64MB and increases as the amount of storage defined for the logical partitionincreases. You can dynamically reallocate storage resources for z/Architecture andESA/390 architecture logical partitions using Dynamic Storage Reconfiguration.Dynamic storage reconfiguration allows both central and expanded storageallocated to a logical partition to be changed while the logical partition is active. Itprovides the capability to reassign storage from one logical partition to anotherwithout the need to POR the CPC or IPL the recipient logical partition. For moreinformation, refer to System z9 Processor Resource/Systems Manager PlanningGuide.

Note: You cannot share allocated central storage or expanded storage amongmultiple logical partitions.

Expanded storage granularity for logical partitions is fixed at 64 MB.

ChannelsYou can allocate channels to logical partitions as follows:

v Dedicated channels

Dedicated channels are unshared channels and can only be used by one logicalpartition. All channel types supported by the model can be allocated as dedicatedchannels.

v Reconfigurable channels

Reconfigurable channels are unshared channels that can be moved amonglogical partitions within an LCSS but can only belong to one logical partition at agiven time. All channel types supported by the model can be allocated asreconfigurable channels.

v Shared channels



The Multiple Image Facility (MIF) allows channels to be shared among multiplelogical partitions in a Logical Channel Subsystem (LCSS). Shared channels areconfigured to a logical partition giving the logical partition a channel image of theshared channel that it can use. Each channel image allows a logical partition toindependently access and control the shared channel as if it were a physicalchannel assigned to the logical partition. For more information, refer to “MultipleImage Facility (MIF)” on page 4-12.

You can define the channels, shown in Table 4-2 on page 4-2, as shared amongmultiple logical partitions within an LCSS so that the shared channels can beaccessed by more than one logical partition in an LCSS at the same time.

On z9 EC models with coupling facility logical partitions, CFP, CBP, and ICPchannels may be shared by many ESA logical partitions and one CouplingFacility logical partition.

v Spanned channels

Spanned channels are channels that are configured to multiple Logical ChannelSubsystems (LCSSs) and are transparently shared by any or all of the configuredLPARs without regard to the LCSS to which the LPAR is configured.

v Device Sharing

You can share a device among logical partitions by:– Using a separate channel for each logical partition– Using a shared channel– Using a spanned channel.

Logical Partition Time Offset SupportLogical Partition Time Offset support provides for the optional specification of afixed time offset (specified in days, hours, and quarter hours) for each logicalpartition activation profile. The offset, if specified, will be applied to the time that alogical partition will receive from connected Sysplex Timers or from the CurrentTime Server (CTS) in a Coordinated Timing Network (CTN).

This support can be used to address the customer environment that includesmultiple local time zones with a single Sysplex Timer, two Sysplex Timers in anExpanded Availability configuration, or a Current Time Server (CTS) in a CTN.

It is sometimes necessary to run multiple Parallel Sysplexes with different localtimes and run with the time set to GMT=LOCAL. This causes the results returned inthe store clock (STCK) instruction to reflect local time. With logical partition timeoffset support, logical partitions on each z9 EC CPC in a Parallel Sysplex that needto do this can specify an identical time offset that will shift time in the logicalpartition sysplex members to the desired local time. Remaining logical partitions onthe z9 EC CPCs can continue to participate in current date production ParallelSysplexes utilizing the same Sysplex Timer(s) or CTS with the time provided by theSysplex Timer(s) or CTS.

This function is supported by all in service releases of z/OS.

For more information on logical partitions, refer to the System z9 ProcessorResource/Systems Manager Planning Guide and to the System z Input/OutputConfiguration Program User’s Guide for ICP IOCP.



Server Time Protocol (STP)Server Time Protocol (STP) (FC 1021) provides the means for multiple System z9,z890, and z990 servers to maintain time synchronization with each other withoutusing a Sysplex Timer. STP is designed to synchronize servers configured in aParallel Sysplex or a sysplex without a coupling facility, as well as servers that arenot in a sysplex.

STP uses a message-based protocol to transmit timekeeping information overexternally defined Coupling Links between servers. Unlike the Sysplex Timer, whichdistributes time to multiple servers in a star pattern, STP distributes time messagesin layers (called stratums). The timekeeping information is needed to determine theCoordinated Server Time (CST) at each server. The Coupling Links used totransport STP messages include ISC-3 links configured in peer mode, ICB-3 links,ICB-4 links, and IFB links. These links can be the same links already being used ina Parallel Sysplex for Coupling Facility communications.

For more details about Server Time Protocol, refer to “Server Time Protocol (STP)”on page 6-9.

For hardware and software requirements, refer to the STP Web site located athttp://www.ibm.com/systems/z/pso/stp.html.

Support ElementThe z9 EC is supplied with two integrated laptop computers that function as aprimary and alternate Support Elements (SEs). Mounted inside the ’Z’ frame, theSEs communicate with the CPC and with each other through the service network.The SE sends hardware operations and management controls to the HardwareManagement Console for the CPC and allows for independent and paralleloperational control of a CPC from the Hardware Management Console. Thesecond, or alternate SE is designed to function as a backup and to preload SELicensed Internal Code. Power for the SE is supplied by the server frame. Refer toFigure 2-1 on page 2-1 to see the location of the SEs on a z9 EC.

The SE contains the following:

v Licensed Internal Code for the CPC.

v Hardware system definitions for the CPC (contained in the reset, image, and loadprofiles for the CPC and IOCDs).

v Battery-powered clock used to set the CPC time-of-day (TOD) clock at power-onreset. In ETR timing mode, CPC attachment to a Sysplex Timer causes the CPCTOD clock to take its time setting from the Sysplex Timer. In STP timing mode,the CPC TOD clock is initialized to Coordinated Server Time (CST).

v Two 10/100 Mbps Ethernet LAN adapters to connect the SE to the HardwareManagement Console through an external Ethernet switch.

v An Ethernet mini-PCI LAN adapter or LAN on Board to connect the SE to theCPC through the power service network.

For more detailed information on the Support Element, refer to the Chapter 9,“Hardware Management Console and Support Element,” on page 9-1 or to theSystem z9 Support Element Operations Guide.



http://www.ibm.com/systems/z/pso/stp.html

Support Element Attachment to HMCThe Support Elements have dual Ethernet LAN adapters used to communicate withHardware Management Consoles. This is offered as part of the initial order or as aManufacturing Engineering Specification (MES). An Ethernet switch (FC 0089) isshipped automatically on every order unless you deselected FC 0089.

Ethernet Switch/HubAn Ethernet switch/hub is required to provide an Ethernet attachment between theHardware Management Console and the internal SEs. The IBM supplied Ethernetswitch is capable of 10Mbps or 100Mbps link data rates.

Hardware Management ConsoleOn z9 EC, the Hardware Management Console is a Desktop PC that controls andmonitors status for the System z9, zSeries z800, z890, z900, and z990, and S/3909672 G5 and G6 Central Processor Complexes (CPCs). The HMC provides a singlepoint of control and single system image for those CPCs defined to it. OneHardware Management Console can control up to 100 CPCs. One CPC can becontrolled by up to 32 Hardware Management Consoles.

The Hardware Management Console with Dual Ethernet, is used to control andmonitor multiple systems, local or remote, on an Ethernet local area network (LAN)using the Hardware Management Console Application (HWMCA). The HMC issupplied with two Ethernet ports.

The Hardware Management Console hardware configuration also includes newbuild hardware requirement minimums for FC 0075, FC 0076, FC 0077, FC 0078,FC0079, and FC0080 shown in the following tables.

Table 2-4. New Build Hardware Management Console FC 0075 Hardware MinimumRequirements

Hardware Requirement

General Optional display monitorKeyboardMouseParallel printer port

Processor 2.4 GHz Pentium® 4

Hard drive 40 GB

Memory 1 GB (field upgrade from 512 MB)

Video Integrated Intel® 845

LAN Ethernet 10/100/1000 Mbps via motherboradEthernet 10/100 via adapter

Removable Storage DVD-RAM






Table 2-5. New Build Hardware Management Console FC 0077 Hardware MinimumRequirements (continued)


Processor 3.0 GHz Pentium 4

Hard drive 40 GB

Memory 1 GB (field upgrade from 512 MB)

Video Integrated Intel 865G

LAN Ethernet 10/100/1000 Mbps via motherboardEthernet 10/100 Mbps via Adapter





Processor 3.2 GHz Pentium 4

Hard drive 40 GB

Memory 1 GB

Video Integrated Intel 900

LAN Ethernet 10/100/1000 Mbps via motherboardEthernet 10/100 via adapter


The physical location of the Hardware Management Console hardware features(standard and/or optional) are dictated by the specific PC. Some features may bemutually exclusive with other features depending on the PC model. Each CPC mustbe connected to at least one Hardware Management Console on the same networkas the SEs of the CPC.

The following Hardware Management Console feature may be present on fieldupgraded Hardware Management Consoles, but it is not available and must not betransferred to the new FC 0075 Hardware Management Console:

v 3270 Hardware Management Console PCI card (FC 0036; for emulationfunction).

For more detailed information on the Hardware Management Console, refer toChapter 9, “Hardware Management Console and Support Element,” on page 9-1 orto the System z Hardware Management Console Operations Guide.

Internal Battery Feature-360The optional Internal Battery Feature-360 (IBF-360) (FC 3210) provides the functionof a local Uninterruptible Power source. It has continuous self-testing capability forbattery backup which has been fully integrated into the diagnostics, includingRemote Service Facility (RSF) support.



The IBF provides battery power to preserve processor data in case of a loss ofpower on both of the AC supplies from the utility company. The IBF can hold powerbriefly over a ’brownout’ or for an orderly shutdown in case of a longer outage.

In the event of input power interruption to the system, the IBF provides sustainedsystem operation for the times listed in the following table.

Table 2-7. System IBF Hold Up Times

I/O Configuration

Model with 1 I/O Cage with 2 I/O Cages with 3 I/O Cages

S08 - 1 book 7 minutes 11 minutes 7.3 minutes

S18 - 2 books 12 minutes 7.5 minutes 10 minutes

S28 - 3 books 8.5 minutes 11 minutes 8 minutes

S38 - 4 books 13 minutes 9.5 minutes 7.3 minutes

S54 - 4 books 13 minutes 9.5 minutes 7.3 minutes

Battery hold up time are estimates based on:v Batteries three years old or lessv Normal service Life (two or less complete discharges per year)v System in N+1 mode of operation.

If the IBF is ordered, they must be installed in pairs. Two to six battery units areused depending on the number of Bulk Power Regulators (BPRs) are required.

The IBF is fully integrated into the server power control/diagnostic system whichprovides full battery charge, and test and repair diagnostics. For more informationon the IBF, refer to System z9 Enterprise Class Installation Manual for PhysicalPlanning.

Bolt-Down KitA bolt-down kit is available for a low raised floor installation (9 to 13 inches) and ahigh raised floor installation (12 to 22 inches). You will need to order two bolt-downkits.

Power Sequence ControlThe optional Power Sequence Controller (PSC) is available on the z9 EC models.The PSC feature provides the ability to turn on/off specific control units from theCPC. The PSC feature consists of two PSC24V cards with PSC cables that aremounted near the I/O cages within the server. You can only order one per I/O cage.For more information on the PSC24V card, refer to “PSC24V Card” on page 2-12.

Additional Features/Functions SupportedThe z9 EC models provide high performance and flexibility due to an improveddesign and use of technology advances. In addition to the standard and optionalfeatures previously listed, the design of the z9 EC also supports thefeatures/functions discussed in this section.



z/ArchitectureThis section discusses the enhancements to z/Architecture that z9 EC supports inaddition to the functions already discussed in “z/Architecture” on page 1-7. Refer toz/Architecture Principles of Operation for more detailed information onz/Architecture.

List-Directed IPL

Note: The List-directed IPL implementation is the z9 EC SCSI IPL featurediscussed in this publication. Refer to “SCSI Initial Program Load (IPL)” onpage 5-8 for more information.

List-directed IPL supports the use of I/O devices which are not accessed by CCWs.List-directed IPL may be provided for a logical partition or virtual machine,depending on the model and the amount of memory available to the logical partitionor virtual machine.

Hardware Decimal Floating Point FacilityNew Hardware Decimal Floating Point instructions provide 4, 8, and 16 byte dataformats, provide an encoded decimal (base 10) representation for data, performdecimal floating point computations, and perform data conversions to and from thedecimal floating point representation. The new instructions support the IEEE (Draft)Standard for Floating-Point Arithmetic P754.

Hexadecimal-Floating-Point (HFP) Multiply-and-Add/SubtractFacilityThe HFP-multiply-add/subtract facility provides instructions for improved processingof hexadecimal floating-point numbers. The MULTIPLY and ADD (or SUBTRACT)instruction is intended to be used in place of MULTIPLY followed by ADD (orSUBTRACT) NORMALIZED.

Message-Security Assist (MSA)Message Security Assist (MSA) provides access to CPACF for IBM and customerwritten programs when ICSF is not available. MSA instructions for invoking CPACFfunction for DES, TDES, AES, SHA-1, and SHA-256 are found in the z/ArchitecturePrinciples of Operation, SA22-7832.v CIPHER MESSAGEv CIPHER MESSAGE WITH CHAININGv COMPUTE INTERMEDIATE MESSAGE DIGESTv COMPUTE LAST MESSAGE DIGESTv COMPUTE MESSAGE AUTHENTICATION CODE.

Long-Displacement FacilityThe long-displacement facility provides a 20-bit signed-displacement field in 69previously existing instructions (by using a previously unused byte in theinstructions) and 44 new instructions. A 20-bit signed displacement allows relativeaddressing of up to 524,287 bytes beyond the location designated by a baseregister or base-and-index-register pair and up to 524,288 bytes before thatlocation. The enhanced previously existing instructions generally are ones thathandle 64-bit binary integers. The new instructions generally are new versions ofinstructions for 32-bit binary integers. The new instructions also include a LOADBYTE instruction that sign-extends a byte from storage to form a 32-bit or 64-bitresult in a general register and new floating-point LOAD and STORE instructions.The long-displacement facility provides register-constraint relief by reducing the



need for base registers, code size reduction by allowing fewer instructions to beused, and additional improved performance through removal of possibleaddress-generation interlocks.

Extended-I/O-Measurement-Block FacilityThe extended-I/O-measurement-block facility may be available on modelsimplementing z/Architecture. The facility includes the following features:

v A new format of the channel-measurement block. The new measurement block,termed a format-1 channel-measurement block, is expanded to 64 bytes and isaddressed using a separate measurement-block address for each subchannel.The new measurement-block format provides additional measurement informationand the flexibility to store the measurement blocks in non-contiguous, realstorage.

v The previously existing channel-measurement block is termed a format-0channel-measurement block. A device-busy-time-field is added to the format-0channel-measurement block.

Extended-I/O-Measurement-Word FacilityThe extended-I/O-measurement-word facility may be available on modelsimplementing z/Architecture. The extended-measurement-word (EMW) is anextension to the interruption-response block (IRB) and allows channel-measurementdata to be provided on an I/O operation basis. This reduces program overhead byalleviating the previous requirement that the program fetch the measurement blockbefore and after an operation and calculate the difference between the respectivemeasurement data values.

N_Port ID Virtualization (NPIV)NPIV for Fibre Channel Protocol (FCP) channels (CHPID type FCP) is designed toallow the sharing of a single physical FCP channel among operating systemimages, whether in logical partitions or as z/VM guests in virtual machines. This isachieved by assigning a unique World Wide Port Name (WWPN) to each operatingsystem connected to the FCP channel. In turn, each operating system appears tohave its own distinct WWPN in the SAN environment, enabling separation of theassociated FCP traffic on the channel. Access controls based on the assignedWWPN can be applied in the SAN environment, using standard mechanisms suchas zoning in FC switches and Logical Unit Number (LUN) masking in the storagecontrollers.

NPIV is exclusive to the System z9 and is the preferred method for sharing SCSIdevices among multiple operating system images when using a single physical FCPchannel.

Note: NPIV exploitation requires a Fibre Channel director or switch that supportsthe NPIV standard. It is not supported in a point-to-point topology.

ESA/390 ArchitectureESA/390 architecture provides:

v 31-bit addressing with a virtual address range of 2GB.

v ESCON, FICON, ISC-3, ICB (ICB-4 and ICB-3), IC channel, OSA channels, andHiperSockets.

v Channel path selection and I/O-busy-condition management as hardwarefunctions (rather than control program functions that provide:

– As many as eight channel paths available to each I/O device.



– Increased I/O device accessibility by allowing each central processor to initiateoperations with any of the I/O devices and to handle any I/O interruptionconditions.

v A significantly extended addressability through access to multiple address spacesand data spaces while maintaining compatibility with existing 24-bit and 31-bitsubsystems and user applications. Each address space can contain as many as2GB of programs and data. Each data space can contain as many as 2GB ofdata.

Floating-Point Operationsz/Architecture and ESA/390 architecture floating-point supports sixteen 64-bitfloating-point registers, a 32-bit floating-point-control register, and 175 instructionsoperating on floating-point data in either the hexadecimal or binary floating-pointformat. Extensions are grouped into four areas:v Basic floating-point extensionsv Binary floating-point (BFP) facilityv Hexadecimal floating-point (HFP) extensionsv Floating-point-support (FPS) extensions.

The HFP and BFP instructions operate on data in the HFP and BFP formats,respectively; while the FPS instructions operate on floating-point data independentof the format or convert it from one format to the other.

Basic Floating-Point ExtensionsThe basic floating-point extensions provides the following:

v 16 floating-point registers.

Floating-point registers 0, 2, 4, and 6, are available on all z9 EC machines. Theremaining 12 floating-point registers (1, 3, 5, 7, 8, 9, 10, 11, 12, 13, 14, and 15)are called the additional-floating-point (AFP) registers. The AFP registers can beused only if the AFP-register-control bit is one. All floating-point instructions (FPS,HFP, and BFP) use the same floating-point registers. The floating-point registersare identified by the numbers 0-15 and are designated by a four-bit R field infloating-point instructions. Each floating-point register is 64 bits long and cancontain either a short (32-bit) or a long (64-bit) floating-point operand. Pairs offloating-point registers can be used for extended (128-bit) operands.

v A 32-bit floating-point-control (FPC) register.

The floating-point-control (FPC) register is a 32-bit register, which contains maskbits, flag bits, data exception code, and rounding mode bits.

v An AFP-register-control bit, (bit 13 of control register 0) which controls whetherthe new registers and the BFP instructions can be used.

Bit 13 of control register 0 is the AFP-register-control bit. The AFP registers andthe BFP instructions can be used only when the AFP-register-control bit is one.The AFP-register-control bit permits optimum dispatching of old programs whichdo not use the AFP registers and also provides for compatible operation for thecase of an old operating system running on new hardware and the old operatingsystem does not save and restore the new registers.

v Extension to the program interruption for data exception to store a data-exceptioncode (DXC).

v In ESA/390, a signal processor (SIGP) order, store-extended-status-at-address isprovided to allow programs to logout the new floating point registers. Inz/Architecture, this function is provided by the store-status-at-address order.



Binary Floating-Point (BFP) ExtensionsThe BFP facility defines 32-bit, 64-bit, and 128-bit binary-floating-point (BFP) dataformats, and provides 87 additional instructions in ESA/390 and 93 additionalinstructions in z/Architecture, to operate on data in these formats.

Except for binary-decimal conversion, which must be provided in software, the BFPformats and operations provide everything necessary to conform to ANSI/IEEE Std754-1985, IEEE Standard for Binary Floating-Point Arithmetic, dated August 12,1985.

Hexadecimal Floating-Point (HFP) ExtensionsHFP extensions includes 26 additional instructions in ESA/390, and 32 additionalinstructions in z/Architecture, to operate on data in the HFP format.

All of these are counterparts to additional instructions provided by the BFP facility,including conversion between floating-point and fixed-point formats, and a morecomplete set of operations on the extended format.

Floating-Point-Support (FPS) ExtensionsFPS extensions provides eight additional instructions, including four to convert databetween the HFP and BFP formats.

Data RepresentationThe basic addressable data unit is an 8-bit byte that can be used as one character,2 hexadecimal digits, or 8 binary bits. z9 EC model CPCs provide the following datarepresentation features:

v Efficient use of storage and effective I/O rates for decimal data.

v Variable-length fields.

v Broad and flexible code conversion.

v Decimal arithmetic.

v Fixed-point and floating-point arithmetic.

v 32-bit words, 64-bit doublewords, and 128-bit extended words (for floating-pointarithmetic) in ESA/390 architecture.

v 64-bit words, 128-bit doublewords, and 256-bit extended words (for floating-pointarithmetic) in z/Architecture.

v Instructions for functions such as translate, edit, convert, move, and compare.

Data Compressionz9 EC model CPCs support hardware-assisted data compression for DB2®, IMS™,DFSMS/MVS™, and z/OS.

Hardware facilities allow the compression and expansion of data which can providesignificant reductions in the total cost of computing by significantly reducing:

v DASD and tape storage requirements.

v The number of communication lines or required line speeds for network datatransmission.

v Central processor (CP) cycles needed to perform data compression andexpansion (when compared to software-only data compression).

Data compression, along with the increased data rate of ESCON and FICONchannels and sequential data striping, may provide further enhancements to theeffective bandwidth of general purpose models. When combined, these attributes



may improve channel utilization and enhance system level performance in situationswhere I/O bandwidth has been a limiting factor.

This feature requires a software corequisite.

Dynamic I/O ConfigurationDynamic I/O configuration allows you to modify the I/O configuration withoutperforming:v Power-on Reset (POR) of the CPC, orv Initial Program Load (IPL) of the System Control Program (SCP).

Dynamic I/O configuration allows you to add, delete, or modify the I/O definitions ofchannel paths, control units, and I/O devices in the CPC. You can also namepreviously reserved logical partitions and you can save the changes you made tothe I/O configuration definitions and apply them to the active I/O Configuration DataSet (IOCDS).

Dynamic I/O configuration requires z/OS or z/VM. Linux, VSE/ESA™, z/VSE™, andTPF do not provide Dynamic I/O configuration support.

When z/OS with exploitation support is not installed, z/OS can only run in a singleLCSS. Therefore, z/OS may only execute in LCSS 0. z/VM may execute in allLCSSs.

When z/VM is controlling the I/O configuration, z/VM’s dynamic I/O support isdesigned to handle all of the elements of the multiple Channel Subsystem facility fordynamic I/O configuration changes. To dynamically change the I/O configuration,one of two techniques can be employed:v z/VM Control Program (CP) suite of interactive dynamic I/O commandsv HCM/HCD - configuration management tools.

Dynamic I/O support for adding and deleting logical partition names is provided inz/OS V1R6.

Subspace Group Facility Supportz9 EC model CPCs provide support for the subspace group facility which canenhance the data integrity and reliability of application server subsystems likeCustomer Information Control System Transaction Server (CICS® TS), reducingapplication failures, service outages, and incorrect modification of critical businessdata.

Application server subsystems like CICS TS use a single application serverprogram to provide common function to multiple application programs all runningwithin a single address space. The subspace group facility allows the applicationserver program to assign address space private storage to each applicationprogram running in the address space.

This prevents one application program from accidentally overwriting the code anddata of another application program running in the same address space. Preventingaccidental overwrites reduces application failures, service outages, and incorrectmodification of critical business data. (This storage isolation feature was firstimplemented for CICS/ESA, but it is exploited by CICS TS.)



Additionally, there should be little or no modification required of existing applicationprograms because you implement storage isolation and protection through thesubsystem application server program.

CICS TS Use of the Subspace Group FacilityCICS TS uses the subspace group facility to implement transaction isolation toprevent one application program from accidentally overwriting the storage ofanother application program running in the same address space. Transactionisolation extends the subsystem storage protection capability which protects CICSTS system storage from being accidentally overwritten by user-written applicationprograms.

Additionally, transaction isolation use of the subspace group facility enhancesapplication development and maintenance for the subsystem. When an applicationattempts to access the address space private storage of another application, CICSabends the program making it easy to identify the failing application that couldcause a data integrity problem.

This feature requires a software corequisite.

Dedicated Move Page EngineA dedicated move page engine assists in efficiently transferring data betweencentral and expanded storage, or between two main storage locations.

Fast Sync/Data Mover FacilityAsynchronous Data Mover Facility (ADMF) was invented to improvestorage-to-storage movement of large blocks of data. The continued evolution ofCMOS processor and memory technology in z9 EC has improved synchronous datamovement using the Move Page instruction to the point where its performance is ona par with ADMF.

The Fast Sync/Data Mover facility is implemented on z9 EC models and futureprocessors as an indicator to DB2 that Move Page should be used in place ofADMF. A PTF released for DB2 checks this indicator and uses Move Page insteadof ADMF. DB2 will continue to use ADMF on pre-G5 machines. ADMF is notavailable on z9 EC models.

Immediate and Relative Instruction FacilityThe Immediate and Relative Instruction facility reduces the need to use generalregisters to address branch targets. As a result, the general registers and accessregisters can be allocated more efficiently in programs that require many registers.

Perform Locked Operation (PLO) FacilityThis facility consists of the unprivileged Perform Locked Operation instruction,which provides concurrent interlocked-update reference to multiple storagelocations.

Logical String AssistThe logical string assist provides instructions that more efficiently process stringdata types. These instructions are of particular value to large, complexC-programming language applications that rely heavily on null-character-terminatedstring manipulation functions. This feature requires software corequisites.

The logical string assist provides the following architected instructions:



COMPARE LOGICAL STRINGCompare Logical String compares bytes between two strings until it eitherfinds an inequality or a specified ending character.

MOVE STRINGMove String moves a string until it finds a specified ending character.

SEARCH STRINGSearch String searches a string of a specified length for a specifiedcharacter.

The first two instructions, COMPARE LOGICAL STRING and MOVE STRING, areparticularly useful to the C-programming language that normally delimits strings bya null ending character. For more information, refer to the Enterprise SystemArchitecture/390 Principles of Operation.

Multipath Initial Program Load (IPL)z/OS on System z9 EC allows the system to attempt an IPL on alternate paths, ifavailable, if I/O errors occur during the IPL. The system will attempt the IPL on analternate path until all paths have been attempted or until the IPL is successful. Thissupport increases the availability by allowing IPL to complete if possible usingalternate paths and eliminates the need for manual problem determination whenattempting an IPL.

This function is applicable for all FICON features with CHPID type FC and allESCON features with CHPID type CNC.

Monitoring and Estimating CPC Power Consumption and TemperatureYou can monitor the power consumption and the internal temperature of a specificCPC using the Hardware Management Console (HMC). This information is locatedin the Activity task under the Daily task category. If there is a change intemperature, it is reflected in the System Activity window either every 5 minutes orsooner if the temperature change is at least 0.5 degrees Celsius. The powerconsumption value (provided in kilowatts) is accurate to a tolerance ofapproximately 1%.

You can also estimate the power consumption of a specific System z9 machinemodel and its associated configuration using the Power Estimation tool. This tool isavailable on Resource Link.



Chapter 3. Software Support

This chapter describes the software support for the System z9 EC. The z9 ECsupports LPAR mode only. The following table displays a summary of the minimumsupported operating systems levels for the z9 EC models.

Table 3-1. Supported Operating Systems for z9 EC

Operating System ESA/390 (31-bit) z/Arch (64-bit)

z/OS Version 1 Releases 4, 5, 6, 7 and 84 No1, 2 Yes

Linux for System z: SUSE SLES 9, RedHat RHEL4.0 4

Yes Yes

Linux for S/390: SUSE SLES 9, RedHat RHEL 4.04 Yes Yes

z/VM Version 5 Release 3, 4 No Yes

VSE/ESA Version 2 Release 7 4 Yes No

z/VSE Version 3 Release 14 Yes No

TPF Version 4 Release 1 (ESA mode only)4 Yes No

z/TPF Version 1 Release 1 No Yes

Notes:1. ESA/390 mode permitted for migration purposes only.2. IBM Bimodal Accommodation offering is available for z/OS and z/OS V1R4 (not

available for z/OS V1R5).3. The z/VSE operating system can execute in 31-bit mode only. It does not implement

z/Architecture, and specifically does not implement 64-bit mode capabilities. The z/VSEoperating system is designed to exploit select features of IBM eServer™ zSerieshardware.

4. With applicable PTFs.

Note: The z9 EC CPCs do NOT support S/370™ architecture in operation in alogical partition or as a guest of VM.

This information applies to z9 EC CPCs running in an LPAR mode.

Any program written for z/Architecture or ESA/390 architecture mode can operateon CPCs operating in the architecture mode for which the program was written,provided that the program:

v Is not time-dependent.

v Does not depend on the presence of system facilities (such as storage capacity,I/O equipment, or optional features) when the facilities are not included in theconfiguration.

v Does not depend on the absence of system facilities when the facilities areincluded in the configuration.

v Does not depend on results or functions that are defined as unpredictable ormodel dependent in the z/Architecture Principles of Operation, or in theEnterprise System Architecture/390 Principles of Operation.

v Does not depend on results or functions that are defined in this publication (or,for logically partitioned operation, in the System z9 Processor Resource/SystemsManager Planning Guide) as being differences or deviations from the appropriatePrinciples of Operation publication.

v Does not depend on the contents of instruction parameter fields B and C oninterception of the SIE instruction.



Any problem-state program written for ESA/390 architecture mode can operate inz/Architecture mode provided that the program complies with the limitations foroperating in ESA/390 mode and is not dependent on privileged facilities which areunavailable on the system.

Previously, z/OS had to run in 64-bit z/Architecture mode on a z/Architecture server.The z/OS Bimodal Accommodation offering is your alternative to running in 64-bitmode when you first migrate to z/OS V1R3, or z/OS V1R4 on a z/Architectureserver. It gives you the security of knowing you can return to 31-bit mode if thereare any 64-bit issues during your migration. For more information, you can refer tohttp://www.ibm.com/servers/eserver/zseries/zos/installation/bimodal.html

For z/OS to operate as a guest of z/VM on a z9 EC, z/OS and z/VM must beoperating in 64-bit mode. z/VM V5 is only supported running in 64-bit mode., referto Table 3-2.

Table 3-2. Running z/OS as a Guest of z/VM

Option 1 Option 2

Processor System z9 System z9

z/VM z/VM z/VM

z/VM mode 31-bit 64-bit

z/OS z/OS z/OS

z/OS mode 31-bit 64-bit

Supported No Yes

z/OSThis section describes the z/OS software support available for z9 EC. Refer to thez/OS subset of the 2094DEVICE Preventive Service Planning (PSP) bucket prior toinstalling a z9 EC server. For the minimum cryptographic software requirements,refer to Chapter 7, “Cryptography,” on page 7-1.

z/OS V1R8z/OS V1R8 is the current version associated with the z9 EC platform.

z/OS V1R7z/OS V1R7 (with maintenance) is required for STP support and HCA1-O fanoutcards supporting InfiniBand coupling links.

z/OS V1R6z/OS V1R6 is required for support of System Application Assist Processors (zAAPs)and supports the dynamic naming of logical partitions.

z/OS V1R5z/OS V1R5 includes the exploitation function from the prior z/OS release.

z/OS V1R4The following support for z/OS V1R4 includes these features:

v z/OS V1R4 z990 Exploitation Support Feature



http://www.ibm.com/servers/eserver/zseries/zos/installation/bimodal.html

This unpriced optional feature allows z/OS V1R4 to take advantage of newfunctions provided by the z9 EC server. This feature also provides thecompatibility support that was in the z/OS V1R4 System z9 Compatibility Supportfeature, which this feature replaces. Any z/OS V1R4 customer who does notalready have the compatibility support but requires it, must now order the z/OSV1R4 System z9 Exploitation Support feature. This feature can be required oncoexisting non-z9 systems.

z/VMThis section describes the z/VM software support available for z9 EC.

Refer to the z/VM subset of the 2094DEVICE Preventive Service Planning (PSP)bucket prior to installing a z9 EC.

z/VM V5.4 and z/VM V5.3z/VM V5.4 and z/VM V5.3 will provide the following z9 EC support:

v Virtual Switch (VSWITCH) support for IEEE Standard 802.3ad Link Aggregation

v Integrated System Management from the HMC

v Guest directed re-IPL

v Ability to define, modify, and delete a coupling connection using an InfiniBandlink, CHPID type CIB, when z/VM is the controlling LPAR for dynamic I/O.

z/VSECompatibility and exploitation support is provided by:v z/VSE V3.1 with PTFsv VSE/ESA V2R7 with PTFs.

For the latest information on compatibility support requirements, refer to the VSEsubset of the 2094DEVICE Preventive Service Planning (PSP) bucket prior toinstalling the z9 EC server.

VSE/ESA V2.7VSE/ESA V2.7 provides exploitation support for the following functions:v 60 LPARsv Two logical channel subsystemsv HiperSockets (IQD) spanned channels (VSE/ESA V2R7 and later releases)v OSA-Express Integrated Console Controller (OSA-ICC)v Adapter interruptions for OSA (CHPID type OSD)v OSA-Express2 OSN (OSA for NCP).

z/VSE V3.1z/VSE is the latest step in the ongoing evolution of VSE. It is designed to helpaddress needs of VSE clients with growing core VSE workloads, those who want toexploit Linux on IBM System z for on demand business solutions or infrastructuresimplification.

z/VSE V3.1 provides the following z9 EC support in addition to those supported byVSE/ESA V2.7:

v Adapter interruptions for FICON (CHPID type FCP)

v N_Port ID Virtualization


Chapter 3. Software Support 3-3

v CPACF

v Configurable Crypto Express2 (both accelerator and coprocessor mode)

v CP Assist for Cryptographic Functions (CPACF).

Note: z/VSE V3 can execute in 31-bit mode only. It does not implementz/Architecture, and specifically does not implement 64-bit mode capabilities.z/VSE V3 is designed to exploit selected features of IBM System zhardware.

z/TPF V1.1z/Transaction Processing Facility Enterprise Edition (z/TPF) V1.1 (5748-T15) is ahigh-performance operating system specifically designed to provide high availabilityfor demanding high volume, real-time transaction processing for mission criticale-business applications. z/TPF runs on and exploits System z servers, aninfrastructure offering for transaction processing with high quality service demands.

Linux SupportYou can run Linux natively in a logical partition or as a guest under z/VM. You canalso isolate your Linux on a z9 EC in its own workspace, using the IntegratedFacility for Linux (IFL). IFL is a hardware option for capacity dedicated to Linuxworkload.

Integrated Facility for Linux (IFL)This optional feature, available on z9 EC, enables you to purchase additionalprocessing capacity exclusively for Linux workload, with no effect on the z9 ECmodel designation. For example, a z9 EC model Sxx, with additional workspaceadded by the IFL, will report its capacity as a model Sxx. Consequently, executingLinux workload on the IFL will not result in any increased IBM software charges fortraditional z/OS, z/VM, z/VSE, or TPF operating systems/applications.

Only Linux applications or Linux operating in conjunction with z/VM are supportedby the IFL. Up to 54 IFL features may be ordered for z9 EC models, dependingupon the server model and its number of maximum unused PUs.

The IFL is managed by Processor Resource/Systems Manager (PR/SM). A logicalpartition which is defined as a Linux logical partition, can run Linux or z/VM Version4. These Linux-mode logical partitions can be defined to use the IFLs in either adedicated or shared processor configuration. IFLs cannot be defined for use innon-Linux-mode logical partitions. Implementation of this feature will require alogical partition definition, followed by a normal logical partition activation procedure.

As with any change in the logical partition configuration of a processor, theintroduction of additional resources to manage may have an impact on the capacityof the existing partitions and workloads running on the server. The size of theimpact is dependent on the quantity of added resources and the type of applicationsbeing introduced. Also, one should carefully evaluate the value of sharing resources(like CHPIDs and devices) across partitions to assure the desired balance ofperformance, security, and isolation has been achieved.

BenefitsThe IFL enables you to:

v Add processing capacity dedicated to running Linux on the z9 EC server.



v Run multiple Linux images independently of the traditional z/Architecture orESA/390 workload, with associated savings of IBM z/Architecture.

v Define many virtual Linux images on fewer real z9 EC resources.


Chapter 3. Software Support 3-5

Chapter 4. Channel Subsystem Structure

A Channel Subsystem structure for z9 EC is designed for 256 channels. With theintroduction of a new system structure and all of its scalability benefits, it isessential that the Channel Subsystem Structure (CSS) also be scalable and permit″horizontal″ growth. This is facilitated by allowing more than one Logical ChannelSubsystem (LCSS) on a single z9 EC. Consistent with this premise, IBM isintroducing increased connectivity by offering the following channel/port maximums.z9 EC maximums for ESCON, FICON and OSA and all are contrasted with z990.

Table 4-1. Channel Maximums (z9 EC compared to z990)

Channel Type z9 EC Maximum z990 Maximum

ESCON 1024 1024

FICON Express2 336 240

FICON Express4 336 240

OSA-Express 48 48

OSA-Express2 48 48

IC link* 32 32

ISC-3 link* 48 48

ICB-4 link*ICB-3 link*

1616

1616

Crypto Express2 8 8

IFB link* 16 N/A

Note: The maximum number of coupling links (any combination of IC, IFB, ICB-4, ICB-3,and active ISC-3 links) is 64.

The Channel Subsystem Structure offers the following:

v Four Logical Channel Subsystems (LCSSs)– Each LCSS can have up to 256 channels defined– Each LCSS can be configured with one to 15 logical partitions (cannot exceed

60 LPARs per system).

v Spanned channels are shared among logical partitions across LCSSs. For moreinformation on spanned channels, refer to Table 4-2 on page 4-2 and to“Spanned Channels” on page 4-13.

Note: There is no change to the operating system maximums. One operatingsystem image continues to support up to a maximum of 256 Channel PathIdentifiers (CHPIDs).

Support for dynamic I/O configuration allows channel paths, control units, anddevices to be dynamically added, changed, and deleted in multiple LCSSs.

When z/VM is controlling the I/O configuration, z/VM’s dynamic I/O support isdesigned to handle all of the new elements of the multiple Channel Subsystemfacility for dynamic I/O configuration changes. To dynamically change the I/Oconfiguration, one of two techniques can be employed:v z/VM Control Program (CP) suite of interactive dynamic I/O commandsv HCM/HCD - configuration management tools.



The I/O Subsystem (IOSS) continues to be viewed as a single Input/OutputConfiguration Data Set (IOCDS) across the entire system with up to four LCSSs.Only one Hardware System Area (HSA) is used for the multiple LCSSs.

A CHPID is a two-digit hexadecimal number that identifies a channel path in theCPC. A Physical Channel Identifier (PCHID) is a three-digit number that identifiesthe physical location (cage, slot, card port) for a channel path in the CPC. Anadapter ID (AID) is a two-digit hexadecimal number that identifies HCA1-O fanoutcard location. CHPIDs are associated with ports on an adapter and the AID is usedin that definition.

The CHPID Mapping Tool can help you map your PCHIDs to the CHPID definitionsin your IOCP source statements. The tool will provide you with a new report withyour CHPID assignment in addition to the PCHID values. CMT is available fromResource Link, http://www.ibm.com/servers/resourcelink, as a standalone PC-basedprogram. For more information on the CHPID Mapping Tool, CHPIDs, PCHIDs, orAIDs, refer to System z CHPID Mapping Tool User’s Guide.

IOCP Channel DefinitionsThe following table lists the channel types as defined in an IOCDS that are usedwith z9 EC systems.

Table 4-2. I/O Channels with CHPID Type

ChannelsCHPIDType

May bedefined as

Shared

May bedefined asSpanned

ESCON channels:

Connection Channel (ESCON architecture)Channel-to-Channel (connects to CNC)

CNCCTC

yesyes

nono

ESCON channels connected to converter:

Conversion Channel (ESCON to Parallel Block Multiplexer (BL))Conversion Channel (ESCON to Parallel Byte Multiplexer (BY))

CVCCBY

nono

nono

Fibre Channel converted. The channel path is an FICON channel thatattaches to an ESCON Director Model 5.

FCV yes no

FICON native channels that attach to FICON directors or directly toFICON control units

FC yes yes

FICON channels that attach to Fibre channel devices, switches,directors, or Fibre-Channel-to-SCSI bridges

FCP yes yes

ISC-3 peer mode links (connects to another ISC-3) CFP yes yes

ICB-4 peer links (connects to another ICB-4) CBP yes yes

ICB-3 peer links (connects to another ICB-3) CBP yes yes

IC peer links (connects to another IC) ICP yes yes

IFB peer links (connects to another IFB) CIB yes yes

HiperSocket (iQDIO) channels IQD yes yes

OSA channels using QDIO architecture OSD yes yes

OSA channels using non-QDIO architecture OSE yes yes

OSA 1000Base-T channels for OSA-ICC OSC yes yes

OSA-Express2 channels for NCP OSN yes yes




Each of these channel types requires that a CHPID be defined, even if it is aninternal channel and no physical hardware (channel card) exists. Each channel,whether a ’real’ channel or a virtual (such as HiperSocket) must be assigned aunique CHPID within the LCSS. You can arbitrarily assign a number within the X'00'to X'FF' range. Real channels require a PCHID value to be defined. Most of thesechannel types can be shared and used concurrently among multiple LPARs withinthe same LCSS. Refer to “Multiple Image Facility (MIF)” on page 4-12 for moreinformation on shared channels.

Coupling Link Peer ChannelsYou may define an ISC-3 feature as CFP, an ICB-4 and ICB-3 feature as CBP, andan IFB link as CIB. Any available/unused CHPID may be defined as ICP.

You can configure a CFP, CBP, ICP, or CIB channel path as:

v An unshared dedicated channel path to a single logical partition.

v An unshared reconfigurable channel path that can be configured to only onelogical partition at a time but which can be dynamically moved to another logicalpartition by channel path reconfiguration commands. Reconfigurable support forCFP, CBP, ICP, and CIB is limited to two coupling facility logical partitions total.One coupling facility logical partition in the initial access list and one othercoupling facility partition in the candidate list.

v A shared or spanned channel path that can be concurrently used by the logicalpartitions to which it is configured. A peer channel cannot be configured to morethan one coupling facility logical partition at a time, although it can be configuredto multiple z/Architecture or ESA/390 logical partitions in addition to the singlecoupling facility logical partition.

v Timing-only links. These are coupling links that allow two servers to besynchronized using Server Time Protocol (STP) messages when a CouplingFacility does not exist at either end of the coupling link.

Each ICP channel path must specify which ICP channel path it is logicallyconnected to.

The System z9 models support Dynamic I/O configuration for all peer channel pathtypes.

Subchannel ConnectivityWith four Logical Channel Subsystems comes more subchannels. There is amaximum of 65280 subchannels per LCSS in subchannel set 0 and 65535subchannels per LCSS in subchannel set 1. With four Logical Channel Subsystems,each LCSS can have its own set of subchannels in both subchannel sets and eachlogical partition in each LCSS can have access to the subchannels in both sets ofits LCSS.

With four Logical Channel Subsystems you can have:v Up to a maximum of 65280 devices/subchannels per LCSS for subchannel set 0v Up to a maximum of 65535 devices/subchannels per LCSS in subchannel set 1v Up to a maximum of 523260 devices for four LCSSs (four LCSSs times 65280

plus 65535 subchannels for each LCSS).

Each LPAR can access all the devices in its assigned LCSS.


Chapter 4. Channel Subsystem Structure 4-3

This capability relieves the I/O device configuration constraints experienced by largesystem configurations.

Availability Guidelines for z9 EC ModelsWhen configuring devices with multiple paths to the same CPC, select any of thechannel paths from any I/O card shown in Figure 4-2 on page 4-7 that:

v Are available on the CPC you are defining

v Are the correct type (FICON, ESCON, etc.) to meet the control unit, couplingfacility, or network attachment requirements

v Satisfy the rules regarding the mixing of channel types to a control unit.



Legend:

STI Self-timed interface

H1 Half-high card in top half of slot

H2 Half-high card in bottom half of slot

S Slot

For maximum availability of the device, OSA network, or coupling facility on z9 ECmodels, you should consider the following guidelines:

S01

S03

S06

S08

S02

S04

S07

S09

S10

S12

S15

S17

S11

S13

S16

S18

H105

H205

STI

STI

STI

STI

STI

STI

STI

STI

H114

H214

H123

H223

H128

H228

S32

S31

S30

S29

S26

S24

S21

S19

S27

S25

S22

S20

CU1

CU2

Figure 4-1. I/O Cage Layout for the z9 EC Models



v Choose channels plugged in different I/O domains (see CU 1). An I/O domaincontains four channel cards controlled by a single STI-MP card. For example, thedomain for the STI-MP card in H105 contains slots 01, 03, 06, and 08.

Note: We also recommend this for optimum performance of your mostheavily-used I/O devices.

When choosing the I/O domains to use, whether from different cages or thesame cage, consider using a combination of I/O domains from different books.When you must use STIs from the same book, try to use STIs from differentMBA fanout cards on that book. Each fanout card plugged into a book providesan MBA. Refer to your PCHID Report to determine which STI links belong towhich MBA fanout cards on a book. If you have multiple paths to the device andmultiple domains available that have the correct channel type, spreading thepaths across as many books and MBAs as possible is also advisable.

Redundant I/O Interconnect is a function that allows one STI to back up anotherSTI in case of a failure. The STIs in slot 05 (same is true for those in slots 14,23, and 28) back up each other. Therefore, in the event of an STI cable, fanoutcard, or book failure, the surviving STI will control both domains. There arefailures (for example, the STI-MP card) that may prevent the redundant takeover,which is why it is advisable to spread your paths over multiple domains.

When configuring ICB channel paths for the same target CPC or coupling facility,use a combination of STIs from different books and different MBA fanout cardson those books. This allows for continued connectivity if you have to rundegraded after a book or MBA fanout card failure.

When configuring ICB-3 channel paths for the same target CPC or couplingfacility, use links that originate from different STI-3 extender cards in the I/Ocage. This eliminates the STI-3 card as a single point of failure.

v If you define multiple paths from the same STI, distribute paths across differentchannel cards (see CU 2). Also, if you define multiple coupling facility channels tothe same coupling facility or to the same ESA image, distribute paths acrossdifferent coupling facility channel adapter cards or different coupling facilitydaughter cards.

Note: Each example (CU1 and CU2) in Figure 4-1 on page 4-5 is only an attemptto illustrate an availability guideline and does not represent an overall optimalconfiguration.

z9 EC servers automatically balance installed channel cards across all availableSAPs. The CPC attempts to assign an equal number of each channel card type(ESCON, FICON, etc.) in each available SAP. While all channels on a given I/Ocard will always be in the same SAP, it is not predictable which I/O cards will beassigned to which SAP. There are two exceptions to the previous statements. First,ICBs are always given affinity to a SAP on the local book. Second, if an OSAchannel is defined as OSD or a FICON channel is defined as FCP, these channelsuse QDIO architecture and, therefore, really do not use any SAP resource duringnormal operations. IBM recommends you configure for normal Reliability,Availability, Serviceability (RAS) as described in the preceding paragraphs, and themachine will balance the channels across your available SAPs.

STI and InfiniBand ConfigurationsA z9 EC can have one to four books installed. Each book can be configured with acombined total of eight MBA fanout cards (for STI connections) and HCA1-O fanoutcards (for IFB connections.)



Each MBA fanout card is designed to support two Self-Timed Interconnect (STIs)either operating at 2.7 GBps for I/O or 2.0 GBps for ICBs. The STI cable for ICBs is10 meters (33 feet) in length, of which 3 meters are reserved for interserverconnection.

The HCA1-O fanout card is used to connect a z9 EC server to a z10 EC serverusing an InfiniBand connection. The HCA1-O supports a two-port 12x IB-SDRoptical link operating at a link rate of 3 GBps. The InfiniBand optical cable can beup to 150 meters in length.

Each book can contain eight fanout cards and up to combination of 64 STI and IFBconnections when four books are installed.

Figure 4-2 depicts the z9 EC ICB (ICB-4 and ICB-3) links, IFB links, and CPC to I/Oconnections.

Planning for Channel SubsystemThis section contains information to aid in the planning for maximum channelsubsystem availability on z9 EC models. It addresses ESCON, FICON, andd OSAchannels; ISC-3, ICB (ICB-4 and ICB-3), IFB, and IC links; and HiperSockets. Theinformation shows the major components of the channel subsystem and suggestsways to configure the CSS for maximum availability.

The overall process of assigning CHPIDs, PCHIDs, and AIDs begins when youorder the z9 EC or an MES to an existing machine. After placing the order, the

Figure 4-2. ICB and InfiniBand connections



|

configurator prepares a report (PCHID report) detailing the physical location foreach channel in the machine. For the z9 EC, this report shows PCHID and AIDassignments.

This information is in the PCHID report or CFReport provided by the IBM accountteam for your machine.

The following references, available on Resource Link, can assist you with yourchannel subsystem planning:v z9 EC Technical Information education modulev CHPID Mapping Toolv System z CHPID Mapping Tool User’s Guide.

CHPID and PCHID AssignmentsThere are no default CHPIDs assigned. You are responsible for assigning thelogical CHPID number to a physical location, identified by a PCHID number. Youcan complete this task in either HCD or IOCP using the PCHID report provided byyour representative. The CHPID Mapping Tool may be used to help with theseassignments. (Refer to “CHPID Mapping Tool” on page 4-11 for more information.)You will use the data in the CFReport, which you can either obtain from yourrepresentative or retrieve from Resource Link, and the IOCP input for assigningPCHID values using the CHPID Mapping Tool.

A PCHID relates directly to a jack location in a specific slot in a specific cage.Table 4-3 lists the PCHID assignments for slots in the I/O cages. Only the activeports on an installed card are actually assigned a PCHID, the remainder areunused.

Table 4-3. PCHID assignments for I/O cage

Cargo Slot

PCHID Range

Cage 1/Bottom ’A’ Cage 2/Bottom ’Z’ Cage 3/Top ’Z’

1 100 - 10F 300 - 30F 500 - 50F

2 110 - 11F 310 - 31F 510 - 51F

3 120 -12F 320 - 32F 520 - 52F

4 130 - 13F 330 - 33F 530 - 53F

6 140 - 14F 340 - 34F 540 - 54F

7 150 - 15F 350 - 35F 550 - 55F

8 160 - 16F 360 - 36F 560 - 56F

9 170 - 17F 370 - 37F 570 - 57F

10 180 - 18F 380 - 38F 580 - 58F

11 190 - 19F 390 - 39F 590 - 59F

12 1A0 - 1AF 3A0 - 3AF 5A0 - 5AF

13 1B0 - 1BF 3B0 - 3BF 5B0 - 5BF

15 1C0 - 1CF 3C0 - 3CF 5C0 - 5CF

16 1D0 - 1DF 3D0 - 3DF 5D0 - 5DF

17 1E0 - 1EF 3E0 - 3EF 5E0 - 5EF

18 1F0 - 1FF 3F0 - 3FF 5F0 - 5FF

19 200 - 20F 400 - 40F 600 - 60F

20 210 - 21F 410 - 41F 610 - 61F




Table 4-3. PCHID assignments for I/O cage (continued)

Cargo Slot

PCHID Range

Cage 1/Bottom ’A’ Cage 2/Bottom ’Z’ Cage 3/Top ’Z’

21 220 - 22F 420 - 42F 620 - 62F

22 230 - 23F 430 - 43F 630 - 63F

24 240 - 24F 440 - 44F 640 - 64F

25 250 - 25F 450 - 45F 650 - 65F

26 260 - 26F 460 - 46F 660 - 66F

27 270 - 27F 470 - 47F 670 - 67F

29 280 - 28F 480 - 48F 680 - 68F

30 290 - 29F 490 - 49F 690 - 69F

31 2A0 - 2AF 4A0 - 4AF 6A0 - 6AF

32 2B0 - 2BF 4B0 - 4BF 6B0 - 6BF

Direct book connections (z9 EC to z990, or z9 EC to z10 EC) use ICB-4 links.(These links are used for connections to Coupling Facilities.) There can be up to 16STI ports on a book, and a PCHID number is assigned to each one.

AID AssignmentsHCA1-O fanout cards used for coupling are identified by adapter IDs (AIDs) ratherthan PCHIDs.

CHPID numbers need to be associated with ports on an adapter, and the AID isused for this purpose. You are responsible for assigning CHPID numbers. You canuse either IOCP or HCD. The CHPID assignment is done by associating the CHPIDnumber to an AID and port. You cannot use the CHPID Mapping Tool to assign AIDvalues.

You cannot change an AID assignment. AID values are preassigned and change ifthe HCA1-O fanout card is moved.

There are two ports for each HCA. The CHPID to PCHID ratio was one-to-one, butan AID can have up to 16 associated CHPIDs.

Table 4-4 shows the AID assignments for the ports on the HCA fanout cardsplugged into the books.

Table 4-4. AID assignments for HCA fanout cards

Book number Cage slot number Fanout card slots AIDs

0 06 D1-D8 08-0F

1 10 D1-D8 10-17

2 15 D1-D8 18-1F

3 01 D1-D8 00-07

Note: Book number refers to the number in the configurator output.

Each fanout slot is allocated an AID number.



PCHID ReportThe PCHID report, from the Order Process Configurator (eConfig), provides detailson the placement of all the I/O features in your order. Your representative willprovide you with this report. Using this report and the guidelines listed in“Availability Guidelines for z9 EC Models” on page 4-4, you can plan theconfiguration of your I/O. Refer to 4-10 for a partial sample of a PCHID report.

Other resources available are the System z Input/Output Configuration ProgramUser’s Guide for ICP IOCP and the CHPID Mapping Tool. These resources areavailable on Resource Link.

The PCHID report includes the following information:

Book/Fanout/JackDisplays the book number, fanout slot number, and port number where theMBA or HCA1-O fanout card is plugged. Jack is the port number on theMBA fanout card. Jack does not apply to the HCA1-O fanout card.

Note: If Cage identifies a CPC cage, this is a CPC cage to CPC cageconnection. If Cage identifies an I/O cage, this is a CPC cage to I/Ocage connection.

If Cage identifies an I/O cage, Book/Fanout/Jack identifies the booknumber, fanout slot number, and port number on the CPC that is connectedto the STI-MP on the I/O cage.

Cage Identifies the type of cage (I/O or CPC) and location of the cages. (CPCcages are shown only if ICB-4s or IFBs are part of the configuration.) In thisexample, there are four cages: a CPC cage in the top of the “A” frame(A19B), an I/O cage in the bottom of the “A” frame (A01B), an I/O cage inthe bottom of the “Z” frame (Z01B), and an I/O cage in the top of the “Z”frame (Z15B).

Slot Identifies the fanout slot number in the specified book and the cage slotnumber in the CPC when Cage is a CPC cage.

Identifies the slot number in the I/O cage when Cage is an I/O cage.

F/C Displays the feature code for the I/O type (ESCON, FICON, etc.) installed inthat location.for the link or channel

PCHID/Ports or AIDDisplays either the PCHID and the physical port (jack number) assigned tothe MBA card or the adapter ID (AID) assigned to the HCA1-O card.

Note: If you use the CHPID Mapping Tool, available from Resource Link, to aid youin assigning PCHIDs to CHPIDs, the tool will provide you with a new reportwith your CHPID assignment in addition to the PCHID values.

CHPIDSTART07449655 PCHID REPORT Mar 14,2008Machine: 2094-S38 NEW1- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -Book/Fanout/Jack Cage Slot F/C PCHID/Ports or AID Comment0/D5 A19B D506 0167 AID=0C

0/D7 A19B D706 0167 AID=0E

1/D5 A19B D510 0167 AID=14

1/D7 A19B D710 0167 AID=16



2/D5 A19B D515 0167 AID=1C

2/D7 A19B D715 0167 AID=1E

3/D5 A19B D501 0167 AID=04

3/D7 A19B D701 0167 AID=06

3/D8/J00 A19B D801 3393 00E/J00

3/D8/J01 A19B D801 3393 00F/J01

...

3/D4/J00 A01B 01 3368 100/J00

2/D4/J00 A01B 02 3322 110/D1 111/D2 112/D3 113/D4

...

3/D4/J00 A01B 06 2323 140/J00 141/J01 142/J02 143/J03144/J04 145/J05 146/J06 147/J07148/J08 149/J09 14A/J10 14B/J1114C/J12 14D/J13 14E/J14

...

1/D6/J00 Z01B 01 3368 300/J00

0/D6/J00 Z01B 02 3322 310/D1 311/D2 312/D3 313/D4

...

1/D6/J00 Z01B 06 2323 340/J00 341/J01 342/J02 343/J03344/J04 345/J05 346/J06 347/J07348/J08 349/J09 34A/J10 34B/J1134C/J12 34D/J13 34E/J14

...

3/D6/J01 Z15B 01 3322 500/D1 501/D2 502/D3 503/D4

...

2/D6/J01 Z15B 04 3368 530/J00

3/D6/J01 Z15B 06 2323 540/J00 541/J01 542/J02 543/J03544/J04 545/J05 546/J06 547/J07548/J08 549/J09 54A/J10 54B/J1154C/J12 54D/J13 54E/J14

...

Legend:A19B Top of A frameA01B Bottom of A frameZ01B Bottom of Z frameZ15B Top of Z frame3322 FICON Express4 SX3368 OSA Express2 10 GbE LR2323 ESCON Channel 16 Ports3393 ICB 4 Link0167 HCA1

CHPID Mapping ToolThe CHPID Mapping Tool is a Java-based standalone application available fromIBM Resource Link, and it must be downloaded to your personal computer for use.In addition to the CHPID Mapping Tool file, you also need the CFReport file. You




can obtain this file from your representative or you can download the file fromResource Link. Once you have the CHPID Mapping Tool and the CFReport file onyour personal computer, you can make CHPID assignments without further internetconnections.

Note: The CHPID Mapping Tool does not assign AID values.

The intent of the CHPID Mapping Tool is to ease installation of new z9 ECprocessors and for making changes to an already installed z9 EC processor eitherto make slight changes to the mapping or as part of an MES action to add orremove channel features on the processor.

The System z9 server does not have default CHPIDs assigned to ports as part ofthe initial configuration process. It is your responsibility to perform theseassignments by using the HCD/IOCP definitions and optionally the CHPID MappingTool. The result of using the tool is an IOCP deck that will map the defined CHPIDsto the corresponding PCHIDs for your processor. However, there is no requirementto use the mapping tool. You can assign PCHIDs to CHPIDs directly in IOCP decksor through HCD but it is much easier to use the tool to do the channel mapping andthe tool can help make PCHID to CHPID assignments for availability.

For more information on the CHPID Mapping tool refer to any of the following:v System z CHPID Mapping Tool User’s Guidev CHPID Mapping Tool on Resource Link.

Multiple Image Facility (MIF)The Multiple Image Facility (MIF) allows channel sharing among multiple logicalpartitions and optionally shares any associated I/O devices configured to theseshared channels. MIF also provides a way to limit the logical partitions that canaccess a reconfigurable channel, spanned channel, or a shared channel to enhancesecurity.

With multiple LCSSs, the CSS provides an independent set of I/O controls for eachlogical channel subsystem called a CSS image. Each logical partition is configuredto a separate CSS image in order to allow the I/O activity associated with eachlogical partition to be processed independently as if each logical partition had aseparate CSS. For example, each CSS image provides a separate channel imageand associated channel path controls for each shared channel and separatesubchannel images for each shared device that is configured to a shared channel.

With MIF, you can configure channels as follows:

v ESCON (TYPE=CNC or TYPE=CTC), FICON (TYPE=FCV, TYPE=FC, orTYPE=FCP), ISC-3 peer (TYPE=CFP), ICB-4/ICB-3 peer (TYPE=CBP), IC peer(TYPE=ICP), IFB peer (TYPE=CIB), HiperSockets (TYPE=IQD), and OSA(TYPE=OSC, TYPE=OSD, TYPE=OSE or TYPE=OSN)

You can configure a channel path as:

– An unshared dedicated channel path to a single logical partition.

– An unshared reconfigurable channel path that can be configured to only onelogical partition at a time it can be moved to another logical partition within thesame LCSS.

– A shared channel path that can be concurrently used by the ESA/390 imagesor CF logical partitions within the same LCSS to which it is configured.



v FICON (TYPE=FC or TYPE=FCP), ISC-3 peer (TYPE=CFP), ICB-4/ICB-3 peer(TYPE=CBP), IC peer (TYPE=ICP), IFB peer (TYPE=CIB), HiperSockets(TYPE=IQD), and OSA (TYPE=OSC, TYPE=OSD, TYPE=OSE or TYPE=OSN)

You can configure these CHPID types for spanning. They can be shared bylogical partitions in different logical channel subsystems.

v ESCON (TYPE=CVC and TYPE=CBY)

You can configure a channel path as:

– An unshared dedicated channel path to a single logical partition.

– An unshared reconfigurable channel path that can be configured to only onelogical partition at a time but which can be moved to another logical partitionin the same LCSS by channel path reconfiguration commands.

ESCON (TYPE=CVC or TYPE=CBY) cannot be configured as shared.

With MIF and multiple channel subsystems, shared and spanned channel paths canprovide extensive control unit and I/O device sharing. MIF allows all, some, or noneof the control units attached to channels to be shared by multiple logical partitionsand multiple CSSs. Sharing can be limited by the access and candidate list controlsat the CHPID level and then can be further limited by controls at the I/O devicelevel.

For example, if a control unit allows attachment to multiple channels (as is possiblewith a 3990 control unit), then it can be shared by multiple logical partitions usingone or more common shared channels or unique unshared channel paths.

Spanned ChannelsWith multiple LCSSs, transparent sharing of internal (ICs and HiperSockets) andexternal (FICON, ICB-3, ICB-4, ISC-3, IFB, OSA) channels across LCSSs isintroduced, extending Multiple Image Facility (MIF). MIF allows sharing of channelresources across logical partitions. ICs, HiperSockets, FICON (except when definedas FCV channels), ICB-3s, ICB-4s, ISC-3s, IFBs, and OSA features can all beconfigured as MIF spanning channels.

Spanning channels is the ability for the channels to be configured to multipleLCSSs, and be transparently shared by any or all of the configured logical partitionswithout regard to the Logical Channel Subsystem to which the partition isconfigured. For more information on MIF, refer to “Multiple Image Facility (MIF)” onpage 4-12. For information on the channel CHPID types and spanning capabilities,refer to Table 4-2 on page 4-2.

Internal Coupling and HiperSockets ChannelsInternal Coupling (IC) channels and HiperSockets are virtual attachments and, assuch, require no real hardware. However, they do require CHPID numbers and theydo need to be defined in the IOCDS. The CHPID type for IC channels is ICP; theCHPID type for HiperSockets is IQD.

1. It is suggested that you define a minimum number of ICP CHPIDs for InternalCoupling. For most customers, IBM suggests defining just one ICP for eachCoupling Facility (CF) logical partition in your configuration. For instance, if yourz9 EC configuration has several ESA LPARs and one CF LP, you would defineone pair of connected ICP CHPIDs shared by all the logical partitions in yourconfiguration. If your configuration has several ESA LPARs and two CF logicalpartitions, you still would define one connected pair of ICP CHPIDs, but one ICPshould be defined as shared by the ESA images and one of the CF LPARs,



while the other ICP is defined as shared by the ESA LPARs and the other CFLPAR. Both of these examples best exploit the peer capabilities of thesecoupling channels by using the ″sending″ and ″receiving″ buffers of bothchannels. If your ESA images and CF images are in different CSSs and youwant to exploit the optimal use of ICP then your ICP CHPIDs must be definedas spanned.

2. Each IQD CHPID represents one internal LAN. If you have no requirement toseparate LAN traffic between your applications, only one IQD CHPID needs tobe defined in the configuration. If the partitions sharing the LAN are in differentLCSSs your IQD CHPID must be defined as spanned.

IOCP ConsiderationsICP IOCP supports the z9 EC model CPCs and multiple LCSSs. Refer to System zInput/Output Configuration Program User’s Guide for ICP IOCP for moreinformation.

Enhancements to IOCP allow you to define controls for multiple channelsubsystems. These changes include changes to the way you define logicalpartitions, channel paths, and I/O devices.

Logical Partition DefinitionYou must specify the required RESOURCE statement. Use the RESOURCEstatement to define LCSSs and the logical partitions in each LCSS. You can alsoassign a MIF image ID to each logical partition. If you do not specify a MIF imageID using the RESOURCE statement, ICP IOCP assigns them. You can also definereserved logical partitions which can be configured later with dynamic I/O.

Channel Path DefinitionYou can define shared channel paths in addition to dedicated and reconfigurablechannel paths. The CHPID statement has an additional SHARED keyword toaccomplish this. You can also define spanned channel paths using the PATHkeyword. You can define:

v All channel paths as dedicated or reconfigurable.

v Only CNC, CTC, FCV, FC, FCP, CFP, CBP, ICP, IQD, CIB, OSC, OSD, OSE andOSN channel paths as shared.

v Only FC, FCP, CFP, CBP, ICP, IQD, CIB, OSC, OSD, OSE, and OSN channelpaths as spanned.

ICP IOCP provides access controls for spanned, shared or reconfigurable channelpaths. Parameters on the PART | PARTITION or NOTPART keyword on the CHPIDstatement allow you to specify an access list and a candidate list for spanned,shared and reconfigurable channel paths.

The access list parameter specifies the logical partition or logical partitions that willhave the channel path configured online at logical partition activation following theinitial power-on reset of an LPAR IOCDS. For exceptions, refer to System z9Processor Resource/Systems Manager Planning Guide.

The candidate list parameter specifies the logical partitions that can configure thechannel path on-line. It also provides security control by limiting the logical partitionsthat can access shared or reconfigurable channel paths.



Note: PR/SM LPAR manages the channel path configuration across POR. Refer toSystem z9 Processor Resource/Systems Manager Planning Guide.

I/O Device DefinitionYou can specify either the optional PART | PARTITION keyword or the optionalNOTPART keyword on the IODEVICE statement to limit device access by logicalpartitions for devices assigned to shared ESCON, FICON, or OSA channels, orHiperSockets. (The IODEVICE candidate list is not supported for shared CFP, CBP,CIB, or ICP CHPIDs.)

By limiting access to a subset of logical partitions, you can:v Provide partitioning at the device level.v Provide security at the device level.v Better manage the establishment of logical paths.

Hardware Configuration Definition (HCD) ConsiderationsHCD provides the capability to make both dynamic hardware and software I/Oconfiguration changes.

Note: Support for MVS™ for multiple logical channel subsystems in HCD is onlyavailable on OS/390®® V2R10 and z/OS V1R2 and later.

HCD also provides:

v An online, interactive way to more usably manage the I/O configuration thanIOCP.

v The capability to define the I/O configuration for dynamic or non-dynamic I/Oconfiguration purposes.

HCD allows you to define LPAR controls for defining logical partitions, channelpaths, and I/O devices. The following HCD panels (or corresponding HCM dialogs)support these controls.

Add PartitionAllows explicit definition of logical partitions and associated logical partitionnumbers.

Define Access ListAllows definition of initial access list for channel path access control ofshared and reconfigurable channel paths.

Define Candidate List (for channel paths)Allows definition of candidate list for channel path access control of sharedand reconfigurable channel paths.

Define Candidate List (for devices)Allows definition of candidate list for device access control for devicesassigned to shared channels.

Add ProcessorAllows you to determine the capabilities of a CPC.

Add Channel PathOperation mode field allows definition of a channel path as dedicated,reconfigurable, or shared.

Define Device / ProcessorAdditional field to specify candidate list.



Chapter 5. I/O Connectivity

This chapter discusses the channels associated with the z9 EC I/O connectivity.You can also refer to Table 2-3 on page 2-11 for a summary of the I/O channelcharacteristics.

FICON and FCP ChannelsThe FICON Express channel uses the industry standard Fibre Channel Standard asa base. It is an upper layer protocol that maps the channel architecture on thegeneral transport vehicle used throughout the industry for such other upper layerprotocols as SCSI, IPI, and IP, among others. This transport vehicle includes thephysical definition, the transmission protocol, and signalling protocol that is thesame for all of the other upper layer protocols.

There are three FICON CHPID types specified using IOCP or HCD:

v FICON native (FC).

Native FICON devices can be attached to servers using the FC CHPID typeeither directly or through a Fibre Channel Director. FC also supportsChannel-to-Channel (CTC) communication (Server-to-Server or LPAR-to-LPAR).

v FICON bridge (FCV)

A FICON channel is defined as CHPID type FCV to communicate with ESCONcontrol units when the FICON channel is attached to an ESCON Director Model5 with a FICON Bridge card.

v Fibre Channel Protocol (FCP).

Storage controllers and devices with an FCP interface can be attached to theSystem z9 server using Fibre Channel switches or directors in z/VM, z/VSE, andLinux on System z9 environments.

FICON builds upon the strengths of ESCON. The maximum link data rate isincreased from 20 MBps to 200 MBps. The FICON implementation enables fullduplex data transfer. So data travels both directions simultaneously, rather than theESCON half duplex data transfer. Furthermore, concurrent I/Os can occur on asingle FICON channel, a fundamental difference between FICON and ESCON. Thedata rate droop is minimal with FICON even at distances up to 100 km.

Native FICON supports up to 32 concurrent I/O operations per second. ESCONsupports one I/O operation at a time. FICON Bridge supports up to eight I/Ooperations at a time to different control units.

In conjunction with the Fibre Channel protocol (FCP), N_Port ID virtualization(NPIV) is supported, which allows the sharing of a single physical FCP channelamong operating system images.

The MIDAW FacilityThe Modified Indirect Data Address Word (MIDAW) facility is a new systemarchitecture and software designed to improve FICON performance. This facility,available on System z9 servers, is exploited by z/OS access methods andsupported by z/VM 5.3 for guests exploitation.. The MIDAW facility can:

v Significantly improve FICON performance for extended format data sets.Non-extended data sets can also benefit from MIDAW.



v Improve channel utilization and can significantly improve I/O response time byreducing FICON channel and control unit overhead.

Multipath Initial Program Load (IPL)z/OS on System z9 EC allows the system to attempt an IPL on alternate paths, ifavailable, if I/O errors occur during the IPL. The system will attempt the IPL on analternate path until all paths have been attempted or until the IPL is successful.


FICON Express4The FICON Express4 feature conforms to the Fibre Connection (FICON)architecture and the Fibre Channel (FC) architecture, providing connectivitybetween any combination of servers, directors, switches, and devices in a StorageArea Network (SAN).

There are three FICON Express4 features, exclusive to IBM System z9, to choosefrom:

1. FICON Express4 10KM LX, with four channels per feature, is designed tosupport unrepeated distances up to 10 kilometers (6.2 miles) over single modefiber optic cabling.

2. FICON Express4 SX, with four channels per feature, is designed to carry trafficover multimode fiber optic cabling.

3. FICON Express4 4KM LX, with four channels per feature, is designed to supportunrepeated distances up to 4 kilometers (2.5 miles) over single mode fiber opticcabling.

All channels on a single FICON Express4 feature are of the same type; 4KM LX,10KM LX, or SX. You can carry forward from z890 or z800 your current FICONExpress2 and FICON Express features.

FICON Express4 features, for z9 EC, increase channel capacity and increaseperformance. FICON Express4 offers two unrepeated distance options, 4 km and10km, when using single mode fiber optic cabling. FICON Express4 supports a 4Gbps link data rate with auto-negotiation to 1 or 2 Gbps for synergy with existingswitches, directors, and storage devices. An entry level 4 km LX feature supportingtwo channels per feature for data centers with limited requirements for single modefiber optic cabling connectivity is offered.

Note: FICON Express4 Data Link Rate

Care should be taken when migrating from a 2 Gbps infrastructure to a 4Gbps infrastructure. As the link data rate increases, the distance decreaseswhen using multimode fiber optic cabling.

FICON Express2The FICON Express2 feature conforms to the Fibre Connection (FICON)architecture and the Fibre Channel (FC) architecture, providing connectivitybetween any combination of servers, directors, switches, and devices in a StorageArea Network (SAN). FICON Express2 supports cascading to concentrate and



minimize the number of cross-site connections and reduce implementation costs fordisaster recovery applications, Graphically Dispersed Parallel Sysplex (GDPS), andremote copy.

FICON Express2 features, for z9 EC, double the channel capacity and increaseperformance. FICON Express2 SX and LX, with four channels per feature, providesup to 336 channels in the same amount of physical space. The maximum numberof any combination of FICON Express2 and FICON Express features per server isnow 84 for z9 EC. FICON Express2 SX and LX features can be added to anexisting z9 EC concurrently.

FICON Express2 has two CHPID types, FC and FCP.

The connectivity options for each channel are defined as:

v Auto-negotiated as 1 or 2 Gbps

v Shared among LPARs

v Defined as a spanned channel

v Intermixed, FC and FCP, in the same director

v Cascaded.

FICON Express2 features include:

1. FICON Express2 LX - long wavelength (FC 3319)The FICON Express2 long wavelength feature provides connectivity betweenany combination of servers, directors, switches, and devices in a Storage AreaNetwork (SAN). Each of the four independent ports/channels is capable of 1gigabit per second (Gbps) or 2 Gbps depending upon the capability of theattached switch or device. The link speed is auto-negotiated, point-to-point, andis transparent to users and applications.

2. FICON Express2 SX - short wavelength (FC 3320)The FICON Express2 short wavelength feature occupies a single I/O slot, usingone CHPID per channel, four CHPIDs per feature, while continuing to support 1Gbps and 2 Gbps link data rates. The link speed is auto-negotiated,point-to-point, and is transparent to users and applications.

The two modes of operation supported by FICON Express2 SX and LX features areconfigured on a channel-by-channel basis with each of the four channels configuredin either of the two supported modes. The two modes are:

1. Fibre Channel (CHPID type FC) which is native FICON in addition to FICONChannel-to-Channel (server-to-server).

2. Fibre Channel Protocol (CHPID type FCP) which supports attachment to SCSIdevices through Fibre Channel switches and directors in z/VM and Linux onSystem z9 environments.

FICON Express2 replaces the FICON Express card for support of FC, includingCTC, and FCP channel types. Channel type FCV and FICON Bridge are notsupported by the FICON Express2 feature. For existing infrastructures using FCV,FICON Express will continue to be supported on the z9 EC.

Note: IBM ESCON Director Model 5 (9032-005), including FICON to ESCONBridge are no longer orderable.


Chapter 5. I/O Connectivity 5-3

Purge Path ExtendedFor additional information about purge path extended, refer to “Purge PathExtended” on page 5-6.

FICON Express FeaturesThere are two FICON Express features.

v FICON Express LX (FC 2319) has two independent ports and is an optionalfeature which supports the FIbre CONnection (FICON) architecture and the FibreChannel architecture providing connectivity between servers, directors, switches,and devices (control units, disk, tape, printers) that conform to the FICON orFibre Channel architecture. Each of the two independent ports/channels iscapable of 1 Gigabit per second (Gbps) or 2 Gbps depending upon the capabilityof the attached device. The link speed is designed to be auto-negotiated,point-to-point, between server and device and is transparent to users andapplications. FICON Express (CHPID types FC or FCP only) can be defined as aspanned channel and can be shared among LPARs within and across LCSSs.

The FICON Express LX feature also supports cascading (the connection of twoFICON Directors in succession) to concentrate and minimize the number ofcross-site connections and reduce implementation costs for disaster recoveryapplications, Geographically Dispersed Parallel Sysplex (GDPS), and remotecopy.

There are three modes of operation supported by each port, independently.– FCV– FC– FCP.

The FICON Express LX feature supports an LC Duplex connector versus the SCDuplex connector used with the prior generations of FICON features.

Each FICON Express LX port supports attachment to an ESCON Director Model5 with a FICON LX Bridge feature, to a Fibre Channel Director with a FibreChannel/FICON LX feature, or to a control unit with a Fibre Channel/FICON LXfeature. FICON Express LX communicates with any LX feature supportingFICON. The FICON Express LX feature utilizes a long wavelength (LX) laser asthe optical transceiver and supports use of a 9/125-micrometer single mode fiberoptic cable terminated with an LC Duplex connector.

v FICON Express SX (FC 2320) feature has two independent ports and is anoptional feature which supports the FIbre CONnection (FICON) architecture orthe Fibre Channel architecture and is used to provide connectivity between anycombination of servers, directors, switches, and devices (control units, disk, tape,printers) that conform to the FICON or Fibre Channel architecture. Each of thetwo independent ports/channels is capable of 1 gigabit per second (Gbps) or 2Gbps depending upon the capability of the attached device. The link speed isdesigned to be auto-negotiated, point-to-point, between server and device and istransparent to users and applications. FICON Express SX (CHPID type FC orFCP) can be defined as a spanned channel and can be shared among LPARswithin and across LCSSs.

The FICON Express SX feature also supports cascading (the connection of twoFICON Directors in succession) to concentrate and minimize the number ofcross-site connections and reduce implementation costs for disaster recoveryapplications, Geographically Dispersed Parallel Sysplex (GDPS), and remotecopy.

There are two modes of operation supported by each port, independently.– FC



– FCP.

The FICON Express SX feature supports an LC Duplex connector versus the SCDuplex connector used with the prior generations of FICON features.

Each FICON Express SX port supports attachment to a Fibre Channel Directorwith a Fibre Channel/FICON SX feature, or to a control unit with a FibreChannel/FICON SX feature. FICON Express SX communicates with any SXfeature supporting FICON. The FICON Express SX feature utilizes a shortwavelength (SX) laser as the optical transceiver, and supports use of a50/125-micrometer multimode fiber optic cable or a 62.5/125-micrometermultimode fiber optic cable terminated with an LC Duplex connector.

The two channels residing on a single FICON Express card can be configuredindividually, and can be of a different channel type.

Note: Conversion kits may be required if the connecting devices do not supportuse of LC Duplex connectors.

FICON Channel ConsiderationsYou can define a maximum of 120 FICON channels on your z9 EC, up to amaximum of 60 features per system. The FICON Express features support 1 Gbpsor 2 Gbps link data rate, auto-negotiating the speed, point-to-point, withoutapplication, or end-user intervention.

Each FICON Express Bridge channel can support up to 3000 start I/Os per secondwith a 50% channel utilization. Each native FICON Express 2 Gbps channel cansupport up to 4150 start I/Os per second with a 50% channel utilization. Thesefigures represent an average and a 4K block size.

FICON is supported by MIF, so that multiple logical partitions on a server can sharea FICON channel.

The FICON Express channel cards can be plugged nondisruptively into the z9 EC.

FICON Express Features Support 2 Gbps Link Data RateThe System z9 FICON Express features configured as native FICON or FibreChannel Protocol channel support 2 Gbps link data rates. These features arecapable of auto-negotiation with the attached device to operate at 1 or 2 Gbps.These faster link speeds result in faster data transfer times and increased datathroughput across links for those applications that transfer large amounts of datasequentially. For example, it takes 20 microseconds to transfer 2 KB of data acrossa 1 Gbps link. With a 2 Gbps link, this transfer takes place in 10 microseconds -half the transfer time of a 1 Gbps link. This is noticeable for applications with highlysequential large data transfers of all reads or all writes.

In order to benefit from 2 Gbps link data rates, the infrastructure needs to bepositioned to take advantage of this higher data rate capability. For full 2 Gbps linkcapability, each FICON/Fibre Channel port and each FICON/Fibre Channel devicemust be capable of 2 Gbps, for a 2 Gbps end-to-end data path.

In the event where System z9 and the director negotiate a 2 Gbps link data rateand that same director and a storage device negotiate a 1 Gbps link data rate, careshould be taken that the overall system throughput is balanced. The totalthroughput for that channel connection both in and out of the director should beexamined and the number of channels adjusted. For example, a customer might



have to double the number of links and host adapters to the storage device to avoidloading the capability of a 1 Gbps control unit port due to the higher bandwidthcapability of the 2 Gbps FICON Express channel port.

We recommend you engage IBM Networking Services to assist with the planningand installation of the fiber infrastructure and to assure that possible bandwidthconstraints are identified, particularly where there is a mix of 1 Gbps and 2 Gbpsdevices.

If only part of your I/O infrastructure is 2 Gbps capable then the link speed will beauto-negotiated to 1 Gbps (100 MBps). When the System z9 and ESS are 2 Gbpscapable, but the rest of the I/O infrastructure is not, then the System z9 and ESSwill communicate at 1 Gbps (100 MBps) link speeds. Also, when the switches andfiber infrastructure are 2 Gbps capable but the System z9 and ESS are not, then inthe case of FICON cascading and/or FCP fabrics, the switches will auto-negotiatebetween each other 2 Gbps link speeds and the System z9 and ESS willauto-negotiate with the switches 1 Gbps (100 MBps) link speeds.

Table 5-1 contrasts the unrepeated distances when using single mode or multimodefiber at 1 Gbps and 2 Gbps.

Table 5-1. Contrasting Unrepeated Distances using Single Mode and Multimode

1 Gbps link 2 Gbps link

Fiber Type inMicrons (µ)

LightSource

UnrepeatedDistance

LinkBudget

UnrepeatedDistance

LinkBudget

9 µ single mode LX laser 10 km6.2 miles

7.8 db 10 km6.2 miles

7.8 db

50 µ multi-mode SX laser 500 meters1640 feet

3.9 db 300 meters984 feet

2.8 db

62.5 µ multi-mode SX laser 250 meters820 feet

2.8 db 120 meters394 feet

2.2 db

Note: These numbers reflect the Fibre Channel Physical Interface specification. The linkbudget above is derived from combining the channel insertion loss budget with theunallocated link margin budget. The link budget numbers have been rounded to the nearesttenth.

You need to ensure that the tactical as well as the strategic requirements for yourdata center, Storage Area Network (SAN), and Network Attached Storage (NAS)infrastructures are taken into consideration as you employ 2 Gbps and beyond linkdata rates.

Note: Mode Conditioning Patch (MCP) cables are not supported at the 2 Gbps linkdata rate.

Purge Path ExtendedThe purge path extended function provides enhanced capability for FICON problemdetermination. The FICON purge path error-recovery function is extended totransfer error-related data and statistics between the channel and entry switch, andfrom the control unit and its entry switch to the host operating system.

FICON purge path extended is available for z9 EC, z890, and z990 and issupported by z/OS and z/OS.e. FICON purge path extended applies to the FICONfeatures when configured as a native FICON channel.



Fibre Channel Protocol for SCSI (FCP) DevicesFibre Channel (FC) is a computer communications protocol that attempts tocombine the benefits of both channel and network technologies. Fibre Channelmade the biggest impact in the storage arena, specifically, using Small ComputerSystem Interface (SCSI) as an upper layer protocol.

Fibre Channel is broken down into five layers: FC-0, FC-1, FC-2, FC-3, and FC-4.The layers define the following functions:v FC-0 defines the physical characteristicsv FC-1 defines the character encoding and link maintenancev FC-2 defines the frame format, flow control, classes of servicev FC-3 defines the common services

FICON and FCP implement those layers, unchanged.

v FC-4 defines the upper layer protocol mapping which includes SCSI as well asFibre Channel - Single Byte-2 (FC-SB-2), which is FICON.

The Fibre Channel Protocol (FCP) capability, supporting attachment to SmallComputer Systems Interface (SCSI) is based on the Fibre Channel (FC) standardsdefined by the INCITS, and published as ANSI standards. SCSI devices in Linux onSystem z9 environments, as well as SCSI devices defined to z/VM and z/VSE, arebased on the Fibre Channel standards. FC is an upper layer fibre channel mappingof SCSI on a common stack of Fibre Channel physical and logical communicationlayers. HIPPI, IPI, IP, and FICON (FC-SB-2) are other examples of upper layerprotocols.

SCSI is an industry-standard protocol which is supported by a wide range ofcontrollers and devices which complement the classical System z9 storageattachment capability through FICON and ESCON channels. FCP channels onSystem z9 systems are provided to enable operating systems on System z9 toaccess industry-standard SCSI storage controllers and devices.

The System z9 FCP function can facilitate the consolidation of UNIX® server farmsonto System z9, protecting investments in SCSI-based storage.

FC is the base for open industry-standard Fibre Channel networks or Storage AreaNetworks (SANs).

Fibre Channel networks consist of processors and storage controllers and devicesas end nodes, interconnected by Fibre Channel switches, directors, and hubs.While switches and directors are used to build Fibre Channel networks or fabrics,Fibre Channel loops can be constructed using Fibre Channel hubs. In addition,different types of bridges and routers may be used to connect devices with differentinterfaces (like parallel SCSI). All of these interconnects may be combined in thesame network.

For information about the configurations supported by the System z9 FCP channel,refer to “Configurations” on page 5-9.

An FCP channel is defined in the IOCP as channel type FCP and is available onFICON features.

FCP channels also support 2Gbps and 4Gbps link data rates (refer to “FICONExpress Features Support 2 Gbps Link Data Rate” on page 5-5) and full-fabricsupport. The FCP full-fabric support means that multiple numbers of



directors/switches can be placed between the server and FCP/SCSI device, therebyallowing many hops through a storage network for I/O connectivity.

In addition, for FCP channels, a high integrity fabric solution is not required but isrecommended. If an FCP Interswitch Link (ISL) is moved, data could potentially besent to the wrong destination without notification.

SCSI Initial Program Load (IPL)This function allows you to IPL an operating system from an FCP-channel-attacheddisk, to execute either in a logical partition or as a guest operating system underz/VM. In particular, SCSI IPL can directly IPL a System z9 operating system whichhas previously been installed on a SCSI disk. Thus, there is no need for a classicalchannel (ESCON or FICON) attached device, such as an ECKD™ disk control unit,in order to install and IPL a System z9 operating system. The IPL device isidentified by its Storage Area Network (SAN) address, consisting of the worldwideport number (WWPN) of the disk controller, and the Logical Unit Number (LUN) ofthe IPL device.

You can also IPL a standalone-dump program from an FCP channel attached SCSIdisk. The standalone-dump program can also store the generated dump data onsuch a disk.

To use this function with an FCP channel, a no-charge orderable platform feature(FC 9904) is required.

Support of SCSI IPL in z/VM V5.3 and later, allows Linux and other guest operatingsystems that support SCSI IPL to be IPLed from FCP-attached SCSI disk, whenz/VM is running on a z9 EC or equivalent server equipped with the SCSI IPLfunction. Therefore, Linux guests may be started and run completely from FCPchannel attached disk in your hardware configuration. z/VM V5.3 and earlier,continue to require ESCON-attached or FICON-attached disk or tape for their ownIPL and other functions.

z/VM V5.3 and later provides the capability to install z/VM from a DVD to anEnterprise Storage Server® (ESS) SCSI disk emulated as a Fixed BlockArchitecture (FBA) disk as well as Enterprise Storage Server from a DVD to a 3390disk. Thus, z/VM and its Linux guests may be started and run completely from FCPdisks on your hardware configuration. Refer to z/VM subset of the 2094DEVICEPreventive Service Planning (PSP) bucket for any service required for z/VM supportfor SCSI IPL.

z/VM V5.3 and later supports SCSI-attached disks to be used for installation, IPL,and operations such as storing dumps, and other functions, while continuing toprovide support for ESCON-attached of FICON-attached disk or tape.

z/VM V5.3 and later SCSI support allows a Linux server farm to be deployed onz/VM in a configuration that includes only SCSI disks.

For Linux on System z9 support for SCSI IPL refer to this web site:http://www.ibm.com/developerworks/linux/linux390.

For additional information on:

v How to use the SCSI IPL feature for a logical partition, refer to the System z9Support Element Operations Guide or to the System z Hardware ManagementConsole Operations Guide




v Messages that can show up on the operating systems console on the SE orHardware Management Console, refer to zSeries Small Computer Systems(SCSI) IPL - Machine Loader Messages

v How to use the SCSI IPL feature for a z/VM guest, refer to http://www.vm.ibm.com/pubs for appropriate z/VM publications

v How to prepare a Linux IPL disk or a Linux dump disk, refer tohttp://www.ibm.com/developerworks/linux/linux390 for appropriate Linux onSystem z9 publications.

FCP ChannelThe FICON Express features, when defined as FCP CHPID type in the IOCP,support storage controllers and devices with an FCP interface in z/VM and Linux onSystem z9 environments. The total number of FICON channels, whether FCV, FC,or FCP cannot exceed 336 channels per system.

Each port on a single FICON card can be configured individually, and can be of adifferent CHPID type.

Software SupportSCSI controllers and devices can be accessed for Linux on System z9 with theappropriate I/O support. Linux may run either natively in a logical partition, or as aguest operating system under z/VM.

z/VM V5.3 or later provides support for the installation, IPL, and system operation ofz/VM on IBM ESS SCSI disks. SCSI disks are emulated under z/VM as 9336 model20 FBA DASD to allow VM guests, applications, and system functions that supportFBA operations to run under z/VM without charge. z/VM V5.3 continues to supportdedicated FCP subchannels for use by VM guests that support SCSI, such asz/Linux. This support allows z/VM and guests running under z/VM that support FBADASD or dedicated FCP subchannels to run in a SCSI only disk environment.

For the availability of the FCP channel support in Linux on System z9 and forappropriate support in z/VM in order to let a guest operating system access an FCPchannel, refer to http://www.ibm.com/developerworks/linux/linux390.

ConfigurationsStorage controllers and devices with an FCP interface can be directly attached tothe System z9 server (point-to-point connection), or by using Fibre Channelswitches or directors. A storage controller or device with an appropriate FCPinterface may be attached to each port of a FICON feature, or of a Fibre Channelswitch or director.

In addition, the following devices and controllers can be attached to each port on aFibre Channel switch or director:

v FC-AL controllers or devices, and FC-AL hubs.

If the switch or director supports the Fibre Channel Arbitrated Loop (FC-AL)protocol, devices implementing this protocol may be attached to that port andaccessed from the System z9 server. Devices typically implementing the FC-ALprotocol are tape units and libraries, and low-end disk controllers.

If the switch or director does not support the FC-AL protocol, you can also installa FC-AL bridge between the switch or director and the FC-AL controller ordevice.



http://www.vm.ibm.com/pubs/

http://www.vm.ibm.com/pubs/



If more than one FC-AL controller or device should be attached to a FC-ALswitch port, it is convenient to use a Fibre Channel hub, where multiple deviceswith a FC-AL interface can be directly attached to that hub.

v Fibre-Channel-to-SCSI bridges.

Fibre-Channel-to-SCSI bridges can be used to attach storage controllers anddevices implementing the electrical, parallel SCSI interface. Different types ofFibre-Channel-to-SCSI bridges may support different variants of the parallel SCSIinterface, such as Low Voltage Differential (LVD), High Voltage Differential (HVD),Single Ended, wide (16-bit) versus narrow (8-bit) interfaces, and different linkspeeds.

Each System z9 FCP channel (CHPID) can support up to 480 subchannels, whereeach subchannel represents a communication path between software and the FCPchannel. Refer to “Channel and Device Sharing” on page 5-11 for more information.

Host operating systems sharing access to a System z9 FCP channel can establishin total up to 2048 concurrent connections to up to 510 different remote FibreChannel ports associated with Fibre Channel controllers.

The total number of concurrent connections to end devices, identified by logical unitnumbers (LUNs), must not exceed 4096.

I/O DevicesThe FCP channel implements the FCP standard as defined by the INCITS FibreChannel Protocol for SCSI (FCP), and Fibre Channel Protocol for SCSI, SecondVersion (FCP-2), as well as the relevant protocols for the SCSI-2 and SCSI-3protocol suites. Theoretically, each device conforming to these interfaces shouldwork when attached to a System z9 FCP channel as previously defined. However,experience tells us that there are small deviations in the implementations of theseprotocols. Therefore, it is advisable to do appropriate conformance andinteroperability testing to verify that a particular storage controller or device can beattached to an FCP channel in a particular configuration (i.e. attached via aparticular type of Fibre Channel switch, director, hub, or Fibre-Channel-to-SCSIbridge).

Also, for certain types of FCP and SCSI controllers and devices, specific drivers inthe operating system may be required in order to exploit all the capabilities of thecontroller or device, or to cope with unique characteristics or deficiencies of thedevice.

For a list of storage controllers and devices that have been verified to work in aFibre Channel network attached to a System z9 FCP channel, and for specificsoftware requirements to support FCP and SCSI controllers or devices, refer tohttp://www.ibm.com/systems/z/hardware/connectivity.

AddressingFCP channels use the Queued Direct Input/Output (QDIO) architecture forcommunication with the operating system. IOCP is only used to define the QDIOdata devices. The QDIO architecture for FCP channels, derived from the QDIOarchitecture that had been defined for communications via an OSA card, definesdata devices that represent QDIO queue pairs, consisting of a request queue and aresponse queue. Each queue pair represents a communication path between anoperating system and the FCP channel. It allows an operating system to send FCPrequests to the FCP channel via the request queue. The FCP channel uses theresponse queue to pass completion indications and unsolicited status indications tothe operating system.



http://www.ibm.com/systems/z/hardware/connectivity

IOCP is not used to define the actual Fibre Channel storage controllers anddevices, nor the Fibre Channel interconnect units such as switches, directors, orbridges. IOCP is only used to define the QDIO data devices. The Fibre Channeldevices (end nodes) in a Fibre Channel network are addressed using World WideNames (WWNs), Fibre Channel Identifiers (IDs), and Logical Unit Numbers (LUNs).These addresses are configured on an operating system level, and passed to theFCP channel together with the corresponding Fibre Channel I/O or service requestvia a logical QDIO device (queue).

Channel and Device SharingAn FCP channel can be shared between multiple operating systems, running in alogical partition or as a guest operating system under z/VM. Under z/VM V5.3,multiple z/VM, CMS, Linux, and VSE guests are able to share SCSI channels anddevices using z/VM Fixed Block Architecture (FBA) emulation. To access the FCPchannel directly, each operating system needs its own QDIO queue pair, defined asa data device on an FCP channel directly in the IOCP.

Each FCP channel can support up to 480 QDIO queue pairs. This allows each FCPchannel to be shared among 480 operating system instances.

Channel and Device Sharing Using NPIV: N-Port ID Virtualization (NPIV) allowsthe sharing of a single physical FCP channel and attached devices, logical units,among operating system images, whether in logical partitions or as z/VM guests invirtual machines. This is achieved by assigning a unique World Wide Port Name(WWPN) to each subchannel that is defined for an FCP Channel using IOCP.

Each operating system instance using such a subchannel and its associated QDIOqueues therefore also uses its own WWPN. When the operating system imagestarts using its subchannel, the FCP channel performs a login to the Fibre Channelfabric and acquires a unique Fibre Channel ID, also called N_Port ID. This ID isused in all further Fibre Channel communication that is done on behalf of thisoperating system image. System z firmware takes care for the persistent andunique assignment of WWPNs to FCP subchannel.

Access controls based on the assigned WWPN can be applied in the SANenvironment, using standard mechanisms such as zoning in FC switches andLogical Unit Number (LUN) making in the storage controllers.

NPIV is exclusive to the System z9. NPIV exploitation requires a Fibre Channeldirector or switch that supports the NPIV standard. If such a director or switch isinstalled, NPIV mode can be enabled for the FCP Channel that attaches to thisFibre Channel switch or director through the Support Element. This enablement canbe done on logical partition base, ie.e., epr FCP channel image.

NPIV is not supported in a point-to-point topology.

Channel and Device Sharing without NPIV: Without NPIV suppport, multipleoperating system images can still concurrently access the same remote FibreChannel port through a single FCP channel. However, Fibre Channel devices orlogical units, identified by their LUNs, cannot be shared among multiple operatingsystem images through the same FCP Cahnnel.

PositioningFCP and SCSI are industry-standard protocols, which have been implemented bymany vendors in a large number of different types of storage controllers anddevices. These controllers and devices have been widely accepted in the market



place, and proven to be adequate to meet the requirements regarding Reliability,Availability, and Serviceability (RAS) in many environments.

However, it must be noted that there are some advanced, unique RAScharacteristics of System z9 storage attachments based on ESCON and FICONattachments, using System z9 channel programs (and the Extended Count KeyData (ECKD) protocol in the case of disk control units) which may not be readilyavailable in such an FCP or SCSI based world. Therefore, whenever there are verystringent requirements regarding isolation, reliability, availability, and serviceability, aconscious decision must be made whether FCP attached storage controllers anddevices or classical System z9 FICON or ESCON attached control units should beused. Customers requiring the more robust RAS characteristics should chooseFICON or ESCON channels.

ESCON ChannelsThe ESCON Channel provides a high-speed connection between host and controlunits for I/O devices. It is a channel that uses fiber optic cabling, which is thinnerand weighs less than copper cabling. Fiber optic cabling also reduceselectromagnetic emissions, which provides for improved security.

The ESCON Channel provides a light-emitting diode (LED) light source for fiberoptic cables. It can extend up to 3 kilometers (1.86 U.S. miles), a range that can befurther extended to 6 or 9 kilometers (km) by retransmission through one or twoESCON directors.

With the availability of four LCSSs, you can define a maximum of 1024 ESCONchannels on your z9 EC, up to a maximum of 69 features per system. Themaximum number of configurable channels is 256 per LCSS and per operatingsystem image. The high density ESCON feature has 16 ports, 15 of which can beactivated for your use. One port is always reserved as a spare, in the event of afailure of one of the other ports. When four ports are ordered, two 16-port ESCONfeatures are installed and two ports are activated on each feature. After the firstpair, ESCON features are installed in increments of one. ESCON channels continueto be ordered in increments of four channels.

Note: Model S08 supports a maximum of 720 ESCON channels, up to a maximumof 48 features per system. The maximum number of configurable CHPIDs is256 per LCSS and per operating system image.

ESCON supports these operating system environments: z/OS, z/VM, z/VSE,VSE/ESA, TPF and Linux on System z9.

With the ESCON Channel, ’channel attached’ achieves a new meaning. Devicesdirectly attached to the processor no longer need to be in close range. With fiberlinks, the channel-to-channel and channel-to-control-unit distances can be up to 3kilometers. With the signal repeated through ESCON directors, thechannel-to-channel distances can be up to 26 kilometers. Display users within a 26kilometer range can be connected through a channel link instead of atelecommunications link. The IBM 3490/3490E Tape range can extend up to 23kilometers, and IBM 3990 or 9343 DASD Storage Controllers can be located up to9 kilometers away. In some cases, longer distances can be achieved withrepeaters.

The use of the ESCON Channel in combination with ESCON directors saves spaceand relocation costs - less space is required to house cabling for any distance; the



same cabling can be shared by multiple channels and control units interconnectedwith an ESCON director; the total length of cabling will be considerably reduced;lighter and more easily installed fiber optic cabling is easier to relocate if necessary.

You can also use ESCON Channel hardware and cabling, in combination with anESCON converter. This capability allows you to mix your current data processingenvironment with the ESCON environment. You can continue to use existing controlunits and copper cables while you evolve to fiber optic cables and ESCON controlunits as your requirements for enterprise growth increase.

ESCON channels affect the performance of the channel subsystem. Maximizingchannel subsystem performance is an important consideration in configuring I/Odevices to a z9 EC general purpose model CPC. Channel subsystem performancedepends on the factors described in this chapter.

For an explanation of basic ESCON channel concepts, refer to IntroducingEnterprise Systems Connection. For detailed information about synchronous andnonsynchronous I/O operation, refer to Storage Subsystem Library Introduction toNonsynchronous Direct Access Storage Subsystems.

Multipath Initial Program Load (IPL)z/OS on System z9 EC allows the system to attempt an IPL on alternate paths, ifavailable., if I/O errors occur during the IPL, The system will attempt the IPL on analternate path until all paths have been attempted or until the IPL is successful.


ESCON Converter OperationYou can configure ESCON converter channels (attached to a parallel converter - theIBM 9034 or the Optica 34600 FXBT) for block and byte multiplexer mode ofoperation.

As many as eight channel paths are available to attach to any I/O device. Duringany I/O operation, one of the available channel paths is selected. Channel pathselection is a hardware function rather than a function of the system controlprogram.

At the start of an I/O operation, a central processor signals the channel subsystemthat an I/O operation is needed. An I/O request is posted to a queue; meanwhile,instruction execution in the central processor continues.

Channel Multiplexing ModesThe data mode of operation is determined by the multiplexer mode (byte or block).This is selected for specific channels when either the CPC or a logical partition isinitialized.

Block Multiplexer Mode of Operation: In block multiplexer mode of operation, adevice stays connected to a channel continuously during the transfer of a full blockof data.

Block multiplexer mode of operation allows a control unit to present ’channel end’and to disconnect from a channel at the completion of a specified operation. ’DeviceEnd’ is presented at a later point. During the interval between ’channel end’ and’device end’ another device attached to the same channel can be started or can



complete an operation that is ready. However, if the second device does connect tothe same channel during this interval, the first device may find the channel busywhen it tries to reconnect, and then the first device must wait for service.

ESCON can be configured for block multiplexer mode of operation. In blockmultiplexer mode of operation, ESCON channels configured as CVC channel pathscan operate in either interlock (high-speed transfer) mode or in data-streamingmode. They can also be attached to control units that operate in high-speedtransfer or in data-streaming mode. Data rates can be as high 4.5 MBps forESCON CVC channel paths.

Byte Multiplexer Mode of Operation: Byte interleave mode of operation allowsthe execution of multiple I/O operations concurrently. Byte multiplexer mode permitsseveral relatively slow-speed I/O devices to operate at the same time. Eachaddressed device requesting service is selected for transfer of a byte or a group ofbytes to or from main storage. Bytes from multiple devices are interleaved on thechannel and routed to or from the desired locations in main storage.

The load that a byte multiplexer channel can sustain is variable. It is governed byI/O device performance factors such as the data transfer rate, device buffers,number of bytes per data burst on the channel, channel program requirements,synchronized mechanical motion, and priority sequence position on the I/Ointerface.

ESCON converter channels (defined as CBY) can be configured for byte multiplexermode of operation. In byte multiplexer mode of operation, ESCON channelsconfigured as CBY channel paths can operate in either byte multiplexer mode or inburst mode. CBY channels require a 9034 ESCON converter. Byte multiplexermode permits several relatively slow-speed I/O devices to operate at the same time.

Refer to the 2094IO subset id in the 2094DEVICE upgrade ID of the preventiveservice planning (PSP) bucket for prerequisite 9034 EC level information.

Byte Multiplexer Mode and Burst Mode: A byte multiplexer channel can bemonopolized by one I/O device (burst mode) or shared by many I/O devices (bytemultiplexer mode). The number of bytes transferred at a time in byte multiplexermode can be one (single byte transfers) or more than one (multibyte transfers).Most control units that operate in byte multiplexer mode can also operate in burstmode. A manually set switch at the control unit determines whether the control unitoperates in burst mode or byte multiplexer mode.

Some devices offer a choice of how many bytes are transferred during a single datatransfer sequence in byte multiplexer mode.

Because most of the time spent in a data-transfer control sequence is for control,increasing the burst size (the number of bytes transferred per sequence) results ina relatively small increase in the total channel busy time for the sequence. Also,increasing the burst size reduces the number of data transfer sequences required.The net effect is a significant improvement in channel efficiency and a higherallowable data rate.

Burst mode, although most effective in the use of channel resources, can causeanother device on the byte multiplexer channel to exceed its critical time. From theperspective of the control unit, burst mode occurs when the time contributed by the



control unit in a transfer sequence is more than 32 microseconds. (Refer to theEnterprise System Architecture/390 System 360 and System 370 I/O InterfaceChannel to Control Unit OEMI.)

If the device configuration guidelines are followed for byte multiplexer channels on ageneral purpose model CPC, deferred accesses are minimized and data transfersequences exceeding 32 microseconds are acceptable when large burst sizes arespecified.

Most class-2 and class-3 devices that can operate in burst mode should beattached to block multiplexer channels for better performance.

I/O Operations ControlESA/390 I/O operations are performed by executing a channel program thatconsists of one or more chained Channel Command Words (CCWs). Each CCWcontains a command and other information that is used by both the channel andcontrol unit in executing the I/O operation.

Channel commands are segmented into six basic categories with many variationsbased on control unit type. A channel program is initiated and controlled byexecuting one or more of the ESA/390 I/O instructions described below. I/Ointerruptions may result during the execution of a channel program to notify the CPof progress or completion.

Channel CommandsThe six basic channel commands are:

Write Initiates the transfer of data from main storage to an I/O device.

Read Initiates the transfer of data from an I/O device to main storage.

Read BackwardInitiates the transfer of data from an I/O device to main storage, storingdata bytes in reverse order.

ControlSpecifies operations such as set tape density, rewind tape, advance paperin a printer, or sound an audible alarm.

Sense Requests information from a control unit. The information contains unusualconditions detected during the last I/O operation and detailed device status.

Transfer in Channel (TIC)Specifies the location in main storage where the next CCW in the channelprogram is to be fetched. The TIC command provides branching betweenCCWs in noncontiguous storage areas. A TIC command cannot specify aCCW containing another TIC command.

ESA/390 Mode I/O InstructionsIn ESA/390 mode, any CP can initiate I/O operations with any I/O device and canhandle I/O interruptions from any I/O device. Each I/O device is assigned a uniquedevice number, and is associated with one subchannel.

The CPs communicate with devices by specifying the appropriate subchannel. Thesubchannel uses the assigned device address to communicate with the device overone or more channel paths. The device number provides a path-independentmeans to refer to a device for use in operator messages or at the time of IPL.



For descriptions of these instructions, refer to Enterprise System Architecture/390Principles of Operation.

The I/O instructions for operation in ESA/390 mode are:v Start Subchannel (SSCH).v Test Subchannel (TSCH).v Clear Subchannel (CSCH).v Halt Subchannel (HSCH).v Resume Subchannel (RSCH).v Store Subchannel (STSCH).v Modify Subchannel (MSCH).v Test Pending Interruption (TPI).v Reset Channel Path (RCHP).v Set Channel Monitor (SCHM).v Store Channel Report Word (STCRW).v Cancel I/O (XSCH).

The SSCH instruction specifies an operation request block, which designates thechannel program.

Chaining OperationsFollowing the transfer of information over a channel designated by a ChannelCommand Word (CCW), an operation initiated by the Start Subchannel (SSCH)instruction can be continued by fetching a new CCW. Fetching a new CCWimmediately following the completion of the previous CCW is called chaining.Chaining is described in more detail in the Enterprise System Architecture/390Principles of Operation.)

CCWs located in contiguous areas of central storage (successive doublewordlocations) can be chained. Chains of CCWs located in noncontiguous storage areascan be coupled for chaining purposes by using a Transfer in Channel command. AllCCWs in a chain refer to the I/O device specified in the original instruction.

The type of chaining (data or command) is specified by chain-data andchain-command flag bits in the CCW.

Data ChainingWhen the data transfer specified by the current CCW is finished, datachaining causes the operation to continue by fetching a new CCW andusing the storage area defined by the new CCW. Execution of the operationat the I/O device is not affected.

Command ChainingEach time a new CCW is fetched during command chaining, a new I/Ooperation is specified. The new operation is initiated when the device endsignal for the current operation is received, unless suspension is specifiedin the new CCW. When command chaining takes place, the completion ofthe current operation does not cause an I/O interruption.

I/O InterruptionsI/O interruptions report the completion of I/O operations to the CPs, error andtime-out conditions, and progress.

Ending status information about the operation is available to the control program atthe end of the I/O operation. When an I/O operation is completed, an I/Ointerruption request is sent to a central processor. When the request is honored, an



I/O interruption occurs and places the central processor under control of the I/Onew program status word (PSW). Until an I/O interruption condition is honored, it iscalled a pending I/O interruption.

Errors detected by the channel subsystem are reported to the CPs as I/Ointerruptions or machine-check interruptions. I/O interruptions report the followinghardware-related conditions:

v Interface Control Check (IFCC); for example, interface tag errors and time-outs.

v Channel Control Check (CCC); for example, parity, decode, or control errors.

v Channel Data Check (CDC); for example, a parity error detected in centralstorage.

Machine-check interruptions include the following:

v Unrecoverable errors (retry is unsuccessful).

v Persistent errors (retry can be attempted, but the error threshold is exceeded).

v Serious channel element errors that require immediate reporting or cannot bereported as an IFCC or CCC with an I/O interruption.

ResetsAn I/O system reset is issued to all channels, and the channels signal a systemreset to all attached I/O devices. An I/O system reset:v Stops all subchannel operations.v Resets interruptions and status in all subchannels.

An I/O system reset occurs as part of:v Channel subsystem power-on reset.v Initial program load.v System reset.

A channel issues a selective reset to a specific I/O device in response to an IFCC,CCC, or as part of execution of the clear subchannel instruction. The status of thespecific device is reset.

I/O Interface ProtocolThe I/O interface protocol is determined by the interface sequencing operationsselected for specific control units and their associated devices that are attached tothe channel.

Channel-to-Channel ConnectionThe Channel-to-Channel (CTC) function simulates an I/O device that can be usedby one system control program to communicate with another system controlprogram. It provides the data path and synchronization for data transfer betweentwo channels. When the CTC option is used to connect two channels that areassociated with different system, a loosely coupled multiprocessing system isestablished. The CTC connection, as viewed by either of the channels it connects,has the appearance of an unshared I/O device.

The CTC is selected and responds in the same manner as any I/O device. It differsfrom other I/O devices in that it uses commands to open a path between the twochannels it connects, and then synchronizes the operations performed between thetwo channels.



ESCON CTC Support: The parallel I/O CTC architecture defines two operatingmodes for CTC communication: basic mode and extended mode. ESCON CTCsupport for both of these modes is available.

ESCON Channels (Using Link-level and Device-level Protocols): You canachieve ESCON channel-to-channel connections between CPCs with ESCON orFICON Express channels if one of the ESCON channels is defined to operate inchannel-to-channel (CTC) mode.

ESCON CTC type channels can talk with FICON Express FCV type channelsthrough a FICON bridge only, not with FICON FC type channels.

ESCON channels that operate in CTC mode (extended mode or basic mode) canbe defined as shared ESCON channels. For more information, refer to “MultipleImage Facility (MIF)” on page 4-12.

For detailed information about the ESCON channel-to-channel adapter, refer toEnterprise Systems Architecture/390 ESCON Channel-to-Channel Adapter.

Channel Time-out FunctionsThe optional time-out function described here applies only to ESCON channels thatattach to a 9034 ESCON converter channel.

Each channel path has I/O interface time-out functions that time the control unitdelays in completing the following I/O interface sequences:

v A 6-second time-out for all selection and status presentation sequences. Atime-out occurs if the sequence is not complete within 6 seconds.

v A 30-second time-out for data transfer. A time-out occurs if a byte of data is nottransferred within 30 seconds.

If a time-out occurs, the channel terminates the I/O request to the control unit andgenerates an IFCC interruption.

The time-out function detects malfunctions in control units and I/O devices that cancause the channel path to be unusable to other control units and I/O devices. Thetime-out function is specified as active or inactive for a device by IOCP when theIOCDS is created.

Control Unit (CU) Priority on an I/O InterfaceCU priority on an I/O interface applies only to ESCON channels attached to a 9034ES Connection Converter channel.

CU priority on the I/O interface of a channel depends on the order in which theywere attached. If the CUs are connected to the ’select out’ line, the first CU has thehighest priority. If the CUs are attached to the ’select in’ line, the priority sequenceis reversed. CUs attached to the ’select out’ line have priority over CUs attached tothe ’select in’ line.



Dynamic ReconnectionIn ESA/390 mode, the channel subsystem permits dynamic reconnection of I/Odevices that have the dynamic-reconnection feature installed and that are set up tooperate in a multipath mode, such as the IBM 3390 Direct Access Storage ModelA14 or A22. Dynamic reconnection allows the device to reconnect and continue achain of I/O operations using the first available channel path (one of as many aseight possible channel paths defined in an IOCP parameter). The selected path isnot necessarily the one used initially in the I/O operation.

ESCON Channel PerformanceChannel subsystem performance can be examined by observing twomeasurements:

v Response time (the amount of time taken to complete an I/O operation).

v Throughput (the number of I/O operations an I/O subsystem can complete in agiven amount of time).

Figure 5-1. Control Unit (CU) Priority on ESCON Channels Attached to a 9034 ESConnection Converter



Channel subsystem response time and throughput can be divided into four majorcomponents:

v Queuing and setup time

– The time taken for a channel path, control unit, and device to becomeavailable.

– The time taken for a channel to send the I/O operation commands to thecontrol unit.

v Control unit and device time

The time required by the control unit and device to prepare for the transfer ofdata for the I/O operation. For example, a non-cached DASD control unit mayhave to wait for the DASD’s seek and latency times before being ready to acceptor send data.

v Data transfer time

The time it takes to transfer the data for the I/O operation.

v Completion time

The time it takes for the channel and control unit to post the status of and endthe I/O operation.

These major components of channel performance are affected by:v Type of operation (nonsynchronous versus synchronous)v Data transfer rate

– Distance.v Attached device characteristicsv Workload characteristics.

Factors Affecting the Components of ESCON ChannelPerformanceFactors that affect the various components of performance include:v Synchronous or nonsynchronous type of operationv Data transfer rate

– Distance.v Attached device characteristicsv Channel subsystem workload characteristics.

Synchronous and Nonsynchronous I/O Operation: For detailed informationabout concepts described in this section, refer to Storage Subsystem LibraryIntroduction to Nonsynchronous Direct Access Storage Subsystems.

Synchronous operation

Most DASD devices in a parallel environment transfer data synchronously.Synchronous operation requires that the channel, control unit, and devicebe active at the same time.

All work involved in ending an operation and advancing to the nextoperation must be completed before the DASD head reaches the nextrecord (commonly referred to as the inter-record gap). If this does notoccur, a rotational positional sensing/sensor (RPS) miss or an overrun isgenerated and the operation must wait for one DASD revolution beforecontinuing.

Nonsynchronous operation



Nonsynchronous operation removes the requirements of synchronousoperation. During nonsynchronous operation, the channel, control unit, anddevice do not have to be active at the same time to perform an I/Ooperation; thereby:

v Increasing DASD storage potential (by reducing inter-record gap).

v Allowing the channel and control units to be separated by longerdistances.

v Eliminating command overruns.

v Reducing response time (by reducing RPS misses).

v Permitting the channel to perform other operations during the time itwould normally wait for the device (this increases the throughput of thesystem).

Extended count key data (ECKD) channel programs are required to gainthe benefits of nonsynchronous I/O operations. Count key data (CKD)channel programs are supported, but without the benefit of nonsynchronousoperation. CKD channel-program performance could be degraded relative toECKD channel programs in a nonsynchronous environment.

Data Transfer Rate: One of the factors that affects channel performance is thedata transfer rate. The I/O subsystem data rate is the data transfer rate betweenprocessor storage and the device during an I/O operation.

The I/O subsystem data rate is made up of three components:

v Channel data rate

The rate that the channel transfers data between the transmission link andprocessor storage during an I/O operation. For ESCON channels, the link speedis 20 MBps and the channel data rate is 17 MBps at 0 distance. The data rateincreases with distance.

v Control unit data rate

The rate that the control unit transfers data between the control unit and thetransmission link during an I/O operation.

v Device data rate

The rate of data transfer between the control unit and the device. This ratedepends on the control unit and device you use.

The I/O subsystem data rate is the lowest of the channel data rate, the control unitdata rate, and the device data rate. In cases where the data comes from the controlunit or is stored on the control unit and not directly to the device (for example, acache read), the I/O subsystem data rate is the lower of the two: channel data rateor the control unit data rate.

The I/O subsystem data rate affects only the data transfer portion of the responsetime for an I/O operation. Response time and throughput both improve (responsetime decreases and throughput increases).

I/O Device Characteristics: The characteristics of devices attached to a channelsubsystem can have a substantial effect on performance. Device characteristicssuch as caches, buffers, and data transfer rates all affect response time andthroughput.

Channel Subsystem Workload Characteristics: The performance of a specificI/O configuration varies based on the workload characteristics of that configuration.



Two significant factors that determine workload characteristics and affect responsetime and throughput are channel program characteristics and cache-hit rates.

Channel program characteristics

Channel program characteristics affect channel subsystem performance.ESCON channel subsystems using link-level and device-level protocolsperform nonsynchronous data transfers, and should use extended countkey data (ECKD) channel programs.

Count key data (CKD) channel programs run in an ESCON environment,but may increase response times and reduce throughput due to lost DASDrotations.

Channel programs that contain indirect data address words (IDAWs),Transfer in Channel commands (TICs), and chained data commands, orthat have poorly-aligned data boundaries, cause longer storage-responseand increase channel subsystem response times.

Chained data commands increase response time due to an additionalinterlocked exchange between the channel and control unit. Refer to“ESCON Performance Characteristics” for more information.

The amount of data to be transferred per I/O operation affects throughput.As the amount of data transferred per I/O operation increases (the ratio ofdata transferred to overhead improves), throughput improves.

Cache-hit rates

For control units which implement caches, cache-hit rates affect the channelsubsystem performance. As the cache-hit rate increases, response time andthroughput improve. The cache-hit rate is the percentage of times whendata needed for a read operation is in the control unit’s cache. For example,a cache-hit rate of 70% means that the required data is in the cache for 7out of 10 read operations.

The cache-hit rate is significant because data is transferred out of thecache at the control unit’s maximum data transfer rate, while data from thedevice is transferred at lower device speeds. This means that the higher thecache-hit rate, the better the response time and the better the throughput.

ESCON Performance CharacteristicsThis section describes the performance characteristics of ESCON channels andESCON channels attached to a 9034.

All ESCON Channels: With ESCON channels you need to consider the distancebetween the channel and the control unit since this affects the setup andcompletion times of an I/O operation. As the distance between the channel and thecontrol unit increases, the response time increases and the throughput decreases.Channel and control unit utilization also increases as distance between the channeland control unit increases.

The speed of data transfer through fiber optic cable is subject to the Propagationdelay time is determined by two factors: the speed of light through the optical fiber(which is fixed), and the length of the fiber optic link. Propagation delay timeincreases as the distance between elements in a fiber optic environment increase.

Interlocked exchange affects response time. Interlocked exchange requires that thechannel (or control unit) wait for a response from the control unit (or channel)



before proceeding with the next step of an I/O operation. As distance increases, theinterlocked-exchange response time increases because of longer propagation delaytimes.

The throughput and response time for a shared ESCON channel are comparable tothat of an unshared ESCON channel with comparable workload.

ESCON Channels Attached to 9034: This section describes channel performanceconcepts for ESCON channels (attached to a 9034) and parallel channels.

Performance-related factors that should be considered in configuring ESCONchannels (attached to a 9034) and parallel channels include:

Elapsed timeThe elapsed time to complete service for an I/O device connected to achannel includes the device tag time, the cable propagation time, thechannel busy time to service the requested control sequence, and the waittime for channel resources.

Critical time

The time that an I/O device can wait for channel service without a negativeimpact on the performance of the device is called critical time. Each I/Odevice has limits on the elapsed times of various sequences between thedevice and the channel. When these time limits are not satisfied, deviceperformance may be reduced.

For devices operating on block multiplexer channels, the control sequencesthat have the most significant critical-time constraints are those related tochaining operations. For devices operating on byte multiplexer channels,the time for connecting and disconnecting of the device to the channel foreach byte or burst of data sent is the most critical sequence.

In some cases, the time limit is related to synchronized motion (forexample, the time between columns on a card moving through an IBM 2501Card Reader, or the time between the end of a Search ID Equal and thebeginning of a Read or Write at an IBM 3380 Direct Access Storage). Ineach case, a control sequence must be completed within a time limit thatrelates directly to the physical motion of the device to sustain maximum I/Operformance.

The critical time of a device can be exceeded because of other traffic onthe channel or other traffic in the channel subsystem. Central storageloading can also contribute to the elapsed time, but not significantly. If theelapsed time is greater than the critical time, some performancedegradation occurs. The result of exceeding critical time is described underDeferred access on page 5-23 and “Device Class” on page 5-24.

Deferred access

A data deferred access is caused by the inability of the channel to transmitor accept data at the rate requested or transmitted by an I/O device.

A data deferred access is much less likely to occur on buffered devicesthan on unbuffered devices because buffered devices can wait for channelservice. Unbuffered devices (such as start-stop terminals) may have datadeferred accesses when the time required for an error-recovery systemlogout exceeds the critical time.



Data-chaining operations involving the transfer of one or more blocks ofdata increase the probability of data deferred accesses with devices that donot respond to ’suppress out’.

A chaining check is an error detected in a channel when a channel acceptsmore data (in an input operation) than was specified by the count in theCCW. The check occurs when an I/O data rate is too high to be handled bythe channel and storage.

A command deferred access is the inability of the channel to present thenext command within the critical command-chaining time of a control unit.

Degradation (a loss in performance) can result from a deferred access. Adeferred access that requires operator intervention can create significantdegradation. In most cases, a deferred access that is handled automaticallyby retry does not significantly affect throughput.

Depending on the device and the type of deferred access, the operationmay be halted when the need for a deferred access occurs, or it maycontinue transferring data until the end of the block is reached. A deferredaccess may cause a unit check to be presented to the channel. Anychaining is suppressed and an I/O interruption request is generated at theend of the operation. Certain control units, however, may initiate acommand retry sequence without generating an I/O interruption request.Refer to “I/O Interruptions” on page 5-16 for additional information.

Operation in Byte Multiplexer ModeESCON converter (CBY channel path) can operate in byte multiplexer mode.

Note: z9 EC model CPCs are able to define and use CBY channel paths toconnect to 3720, 3725, and 3745 control units requiring byte multiplexerchannel path attachment. Connection is through a 9034 ESCON ConverterModel 1.

Refer to the 2094IO subset id in the 2094DEVICE upgrade ID of the PreventiveService Planning (PSP) bucket for prerequisite 9034 EC level information.

Device Class: Devices that can operate in byte multiplexer mode of operation areclassified by what happens when the device is not serviced within the critical timefor the requested control sequence for that device. Depending on how overallchannel performance is impacted by the critical time for the device being exceeded,a device falls into one of three classes:

Device Class 1

When the critical time is exceeded, a deferred access occurs and the datais not transferred successfully. The consequent error indication causes anI/O interruption request, and program recovery action is required.

Device Class 2

When the critical time is exceeded, the device must be resynchronized. Theadditional delay results in performance degradation. The deviceperformance is degraded by the combined delay of waiting for the channeland resynchronization.

Device Class 3

When the critical time is exceeded, the device waits for channel service andcauses performance degradation (the delay of waiting for the channelservice).



Operation in Block Multiplexer ModeThe ESCON Architecture provides two protocols for block multiplexer mode ofoperation on the I/O interface for the serial transmission of data:v Link-level protocolsv Device-level protocols.

Block multiplexer mode of operation using link-level and device-level protocols cansustain a maximum data rate of 17 MBps.

I/O operations for the serial transmission and reception of data have link-level anddevice-level protocols present in both the channel and the control unit.

The channel subsystem provides two modes for block multiplexer mode ofoperation on the I/O interface in a parallel environment:

v Interlocked.

Data transfer performance using the interlocked mode depends on overall tagtimings (including channel subsystem service), cable length, and control unitservice. Block multiplexer mode of operation using the interlocked protocol cansustain a maximum data of 1.5 MBps.

v Data streaming.

The data-streaming protocol does not require interlocking of data transfer signalsbetween the channel and the control unit; once data transfer is established overthe interface, it continues at a rate governed by the control unit. Block multiplexermode of operation using the data-streaming protocol can sustain a maximumdata rate of 4.5 MBps.

OSA ChannelsOSA channels include all OSA-Express and OSA-Express2 features.

Supported CHPID TypesOSA channels supports two modes of operation: QDIO, non-QDIO. If anOSA-Express feature is running in QDIO mode the channel type is OSD and if it isrunning in non-QDIO mode the channel type is OSE. OSA-Express GigabitEthernet, OSA-Express2 GbE, and OSA-Express2 10 Gigabit Ethernet only supportchannel type OSD.

The 1000Base-T Ethernet feature supports an additional operating mode:OSA-Express Integrated Console Controller (OSA-ICC), with a channel type ofOSC, supporting TN3270E (RFC 2355) emulation. 3270 emulation for consolesession connections is integrated in the z9 EC. This is supported at any of thefeature settings (10/100/1000 Mbps - half or full duplex). For more information onthe OSA-ICC, refer to the System z Open Systems Adapter-Express IntegratedConsole Controller User’s Guide.

OSA NCP is an additional operating mode exclusive to System z9 on theOSA-Express2 Gigabit Ethernet SX, Gigabit Ethernet LX, and 1000Base-T Ethernetfeatures. OSA NCP requires the port to be configured as channel type OSN withconfiguring done on a port-by-port basis. OSA-Express2 OSN is used exclusivelyfor internal communication, LPAR-to-LPAR. It supports concurrent LIC updates andis supported by z/OS, z/VM, z/VSE, TPF, and Linux on System z9.



For more detailed information on these operating modes, refer to System z10,System z9 and eServer zSeries Open Systems Adapter-Express Customer’s Guideand Reference, SA22-7935.

OSA/SFThe Open Systems Adapter Support Facility (OSA/SF) is a host-based tool tosupport and manage the OSA features operating in QDIO, non-QDIO mode, or forNCP. The OSA/SF is used primarily to manage all OSA ports, configure all OSAnon-QDIO ports, and configure local MACs.

One OSA/SF application can communicate with all OSA features in a hardwarecomplex. OSA/SF communicates with an OSA feature through a device predefinedon the OSA feature. The device type is OSAD.

OSA/SF is not required to set up the OSA features in QDIO mode (CHPID typeOSD). However, it can be used to set up MAC addresses and set adapter speed.For OSA NCP channels, OSA/SF does not provide any configuration managementcapabilities but provides capabilities only for operations management.

OSA/SF includes a Java-based Graphical User Interface (GUI) in support of theclient application. The Java GUI is independent of any operating system/server(transparent to operating system), and is expected to operate wherever the Java1.4 runtimes are available.

Interoperability testing has been performed for Windows®™ 2000, Windows XP, andLinux on System z9.

Use of the GUI is optional; a REXX™ command interface is also included withOSA/SF. OSA/SF has been, and continues to be, integrated in z/OS, z/VM,VSE/ESA, and z/VSE and runs as a host application. For OSA/SF, Java GUIcommunication is supported via TCP/IP only.

OSA/SF EnhancementsThe Layer 3 OSA Address Table (OAT) is now designed to display all IP addressesregistered to an OSA port.

OSA/SF now has the capability of supporting virtual Medium Access Control (MAC)and Virtual Local Area Network (VLAN) identifications (IDs) associated withOSA-Express2 and OSA-Express features configured as a Layer 2 interface.

These OSA/SF enhancements are applicable to CHPID type OSD (QDIO traffic)and to all of the OSA features supported on System z9.

For more detailed information on OSA/SF, refer to System z10, System z9 andeServer zSeries Open Systems Adapter-Express Customer’s Guide and Reference.

OSA-Express Channels

Note: Refer to System z10, System z9 and eServer zSeries Open SystemsAdapter-Express Customer’s Guide and Reference to assist you indetermining the minimum software requirements to support OSA-Express.

The Open Systems Adapter-Express (OSA-Express) is an integrated hardwarefeature that provides direct connection to clients on Local Area Networks (LANs).



The OSA-Express feature plugs into an I/O slot just like a channel card. Up to 24OSA-Express features can be installed in a z9 EC, allowing you 48 ports of LANconnectivity.

You can choose any combination of z9 EC OSA-Express features:v Gigabit Ethernet (GbE) LX (FC 1364) - Cannot be ordered after OSA-Express2

GbE features are available.v Gigabit Ethernet (GbE) SX (FC 1365) - Cannot be ordered after OSA-Express2

GbE features are available.v 1000Base-T Ethernet (FC 1366)

You can also carry forward on an upgrade from z900:v Gigabit Ethernet (GbE) LX (FC 2364)v Gigabit Ethernet (GbE) SX (FC 2365)v Fast Ethernet (FC 2366)

All of these OSA-Express features can use IBM’s Queued Direct Input/Output(QDIO) architecture to eliminate the need for Channel Control Words (CCWs) andhost interrupts, resulting in accelerated TCP/IP data packet transmission. While inQDIO mode all the OSA-Express features support z/OS Communications ServerIntrusion Detection Services (IDS). IDS helps detect attacks on the TCP/IP stackthat can potentially harm its ability to function normally and cause a misuse ofsystem resources.

Each OSA-Express feature has two separate channels for direct LAN attachment. Ifyou install the maximum of 24 OSA-Express features, you will use 48 CHPIDs.OSA-Express channels (CHPID types OSC, OSD, OSE) can be defined as spannedchannels and can be shared among LPARS within and across LCSSs. This isreferred to as the Multiple Image Facility (MIF), sharing of channels across logicalpartitions.

For more detailed information on OSA-Express, refer to System z10, System z9and eServer zSeries Open Systems Adapter-Express Customer’s Guide andReference.

OSA-Express2OSA-Express2 LAN adapters help to ensure you have a balanced system to satisfythe bandwidth demands of your applications. The OSA-Express2 features arehot-pluggable, support the Multiple Image Facility (MIF) which shares channelsacross logical partitions (LPARs), and can be defined as a spanned channel to beshared among LPARs within and across LCSSs. The maximum combined numberof OSA features supported on the z9 EC server is 24.

OSA-Express2 features include:

v Gigabit Ethernet (GbE) LX (FC 3364)

v Gigabit Ethernet (GbE) SX (FC 3365)

v 10 Gigabit Ethernet LR (FC 3368).

v 1000Base-T Ethernet (FC 3366)

All OSA-Express2 features support the Queued Direct Input/Out (QDIO) mode,carrying TCP/IP traffic only. QDIO continues to be the preferred architecture onSystem z9 for high-speed communication, helping to reduce host interruptions and



improve response time. OSA-Express2 features operating in QDIO mode includesupport for 640 TCP/IP stacks per CHPID, TCP Segmentation Offload or largesend, and concurrent LIC update.

The OSA-Express2 1000Base-T Ethernet feature supports the non-QDIO mode,OSA-ICC, and OSA for NCP. The OSA-Express2 Gigabit Ethernet feature supportsOSA for NCP.

OSA-Express2 Gigabit Ethernet LX and SXOSA-Express2 Gigabit Ethernet (GbE) supports full duplex operation, and jumbo8992 byte-frame size. GbE features continue to be dual-port, occupying a single I/Oslot and using one CHPID per port, two CHPIDs per feature.

OSA-Express2 10 Gigabit Ethernet LROSA-Express2 10 Gigabit Ethernet Long Reach (FC 3368) LAN adapter is used asthe enterprise backbone, between campuses, to connect server farms to, andconsolidate file servers on, your z9 EC server. OSA-Express2 10 Gigabit Ethernet(GbE) is a one port feature that operates in QDIO mode only providing 24 ports ofLAN connectivity.

OSA-Express2 1000Base-T Ethernet1000Base-T Ethernet (FC 3366) supports the non-QDIO mode, OSA-ICC, and OSAfor NCP. The OSA-Express2 1000Base-T Ethernet feature which supports a linkdata rate of 10, 100, or 1000 Mbps over a copper infrastructure is available only onthe z9 EC.

Layer 2 (Link Layer) SupportOSA features can support two transport modes: Layer 2 (Link Layer) as well asLayer 3, the Network or IP Layer. Layer 2 support can help facilitate serverconsolidation and will allow applications that do not use IP protocols to run on thez9 EC server. Layer 2 is also supported on OSA-Express LIC levels October 2004and later.

640 TCP/IP StacksIncreasing the TCP/IP stacks allows you to host more Linux images. OSA-Express2supports 640 TCP/IP stacks or connections per dedicated CHPID, or 640 totalstacks across multiple LPARs using a shared or spanned CHPID when priorityspecification is disabled. When priority specification is enabled, 160 TCP/IP stacksare supported just as they are in OSA-Express.

Large SendLarge send improves performance by offloading TCP packet processing from thehost to the OSA-Express2 features. Offloading allows the host to send IPdatagrams up to 60K in size. The IP datagram is controlled by the host TCP/IPstack. Sending larger data blocks reduces host processor utilization while increasingnetwork efficiencies.

Concurrent LIC UpdateThe increased memory of OSA-Express2 features facilitates concurrent applicationof LIC updates without requiring a configuration off/on, thereby minimizing thedisruption of network traffic during the update.



OSA LAN Idle TimerThe OSA LAN idle timer value defines how long OSA will hold packets beforepresenting the packets to the host. The LAN idle function now allows the host OS todynamically update the existing LAN idle timer values (defined within the QIB) whilethe specific QDIO data device is in the QDIO active state.

Dynamic LAN idle timer support is supported by OSA-Express2 features in QDIOmode (CHPID type OSD) and is exploited by z/OS V1.8 with PTFs and z/VM 5.3 orlater for guest exploitation.

OSA-Express Network Traffic Analyzer Trace FacilityThe OSA-Express Network Traffic Analyzer Trace facility is a diagnostic tool used tocopy frames as they enter or leave an OSA adapter for an attached host. Thisfacility is controlled and formatted by the z/OS Communications Server, but the datais collected in the OSA at the network port. Because the data is collected at theEthernet frame level, you can trace the MAC headers for packets. You can alsotrace ARP packets, SNA packets, and packets being sent to and from other userssharing the OSA, including other TCP/IP stacks, z/Linux users, and z/VM users.

To enable the OSA-Express Network Traffic Analyzer Trace facility, you must berunning OSA-Express2 in QDIO mode and z/OS V1.8 with PTFs.

Queued Direct I/O Diagnostic Synchronization (QDIOSYNC)Queued Direct I/O Diagnostic Synchronization provides the ability to coordinate andsimultaneously capture software (z/OS) and hardware (OSA-Express2) traces. Thisfunction allows the host operating system to signal the OSA-Express2 feature tostop traces and allows the operator to capture both the hardware and softwaretraces at the same time. You can specify an optional filter that alters what type ofdiagnostic data is collected by the OSA-Express2 adapter. This filtering reduces theoverall amount of diagnostic data collected and therefore decreases the likelihoodthat pertinent data is lost.

To use the Queued Direct I/O Diagnostic Synchronization facility, you must berunning OSA-Express2 in QDIO mode and z/OS V1.8 with PTFs.

Dynamic Link Aggregation Support for the z/VM EnvironmentThis function dedicates an OSA port to the z/VM 5.3 operating system for linkaggregation under z/VM Virtual Switch-controlled link aggregation. Link aggregation(trunking) is designed to allow you to combine multiple physical OSA-Express2ports of the same type into a single logical link. You can have up to eightOSA-Express2 ports in one virtual switch. This increases bandwidth and permitsnondisruptive failover in the event that a port becomes unavailable. This functionalso supports dynamic add/remove of OSA ports and full-duplex mode (send andreceive).

This function is supported by OSA-Express2 features in Layer 2 mode (GigabitEthernet LX and SX and 1000BASE-T Ethernet) in QDIO mode (CHPID type OSD).

Layer 3 Virtual MACThe z/OS Layer 3 Virtual MAC (VMAC) function simplifies the network infrastructureand facilitates IP load balancing when multiple TCP/IP instances are sharing thesame OSA port or Media Access Control (MAC) address. With Layer 3 VMACsupport, each TCP/IP instance has its own unique "virtual" MAC address instead of



sharing the same universal or "burned in" OSA MAC address. Defining a Layer 3VMAC provides a way for the device to determine which stack, if any, shouldreceive a packet, including those received for IP addresses that are not registeredby any TCP/IP stack. With Layer 3 VMAC in a routed network, OSA appears as adedicated device to the particular TCP/IP stack, which helps solve manyport-sharing issues.

Layer 3 Virtual MAC function is supported by OSA-Express and OSA-Express2features in QDIO mode (CHPID type OSD) and is exploited by z/OS V1.8 withPTFs.

HiperSocketsHiperSockets ″network within the box″ functionality allows high speed any-to-anyconnectivity among OS images within the z9 EC server without requiring anyphysical cabling. This ″Network within the Box″ concept minimizes network latencyand maximizes bandwidth capabilities between z/VM, Linux on System z9, z/VSE,and z/OS images (or combinations of these) to enable optimized e-business andERP solutions within a single server. These images can be first level (i.e. directlyunder LPAR), or second level images (i.e. under z/VM). Up to 16 separate internalLANs can be configured within a server thereby allowing OS images to be groupedaccording to the function they provide. These groupings are independent of sysplexaffiliation. In addition the number of communication queues is 4096 and each queuecan have three subchannels. If you want the internal LANs shared betweenpartitions in different LCSSs then the channel must be spanned. For moreinformation on spanned channels, refer to “Spanned Channels” on page 4-13.

Separate HiperSocket LANs are mainly required if some logical partitions need tobe isolated from other logical partitions. Each LAN is configured as an IQD CHPID.

Internet Protocol Version 6 (IPv6) is now supported for HiperSockets (CHPID typeIQD). IPv6 support is available on the OSA-Express2 and OSA-Express features inthe z/OS, z/VM, and Linux on System z9 environments. IPv6 was introducd toexpand the IP address space from 32 bits to 128 bits to enable a greater number ofunique IP addresses in support of the proliferation of devices, such as cell phonesand PDAs, now connecting to the Internet. IPv4 and IPv6 broadcast support isavailable for HiperSockets on z/OS and z/VM. Only IPv4 broadcast support onHiperSockets is available on Linux.

VLAN Support for LinuxVirtual Local Area Networks (VLANs), IEEE standard 802.1q, is being offered forHiperSockets in a Linux on System z9 environment. VLANs increase bandwidth andreduce overhead by allowing networks to be organized for more optimum trafficflow. The network is organized by traffic patterns rather than physical location. Thisallows traffic to flow on a VLAN connection over HiperSockets and betweenHiperSockets and an OSA GbE, 10 GbE, 1000Base-T Ethernet, or Fast Ethernetfeature.

Broadcast SupportInternet Protocol Version 6 (IPv6) broadcast packets are now supported overHiperSockets internal LANs. TCP/IP applications that support IPv6 broadcast, suchas OMPROUTE when running Routing Information Protocol Version 1 (RIPv1), cansend and receive broadcast packets over HiperSockets interfaces. z/OS V1R5 andlater, z/VM V5.3 and later, and Linux on System z9 provide broadcast support for



HiperSockets. Refer to http://www.ibm.com/developerworks/linux/linux390 for moreinformation on Linux on System z9 support.

Network ConcentratorHiperSockets Network Concentrator simplifies network addressing betweenHiperSockets and OSA allowing seamless integration of HiperSockets-connectedoperating systems into external networks, without requiring intervening networkrouting overhead, thus helping to increase performance and simplify configuration.

HiperSockets Network Concentrator is implemented between HiperSockets, OSA,and Linux on System z9. The Network Concentrator provides support for unicast,broadcast, and multicast. For more information, refer to http://www.ibm.com/developerworks/linux/linux390.






Chapter 6. Sysplex Functions

This chapter describes the following z9 EC sysplex functions:v “Parallel Sysplex”v “Coupling Facility” on page 6-6v “System-Managed CF Structure Duplexing” on page 6-11v “Geographically Dispersed Parallel Sysplex (GDPS)” on page 6-12v “Intelligent Resource Director (IRD)” on page 6-14.

Parallel SysplexThe Parallel Sysplex makes use of a broad range of hardware and softwareproducts to process, in parallel, a transaction processing workload across multiplez/OS images running in a Parallel Sysplex and sharing data in a coupling facility.

The Parallel Sysplex allows you to manage a transaction processing workload,balanced across multiple z/OS images running on multiple Central ProcessorComplexes (CPCs), as a single data management system. It also offers workloadavailability and workload growth advantages.

The Parallel Sysplex enhances the capability to continue workload processingacross scheduled and unscheduled outages of individual CPCs participating in aParallel Sysplex using a coupling facility by making it possible to dynamicallyreapportion the workload across the remaining active Parallel Sysplex participants.Additionally, you can dynamically add processing capacity (CPCs or LPs) duringpeak processing without disrupting ongoing workload processing.

z9 EC CPC support for the Parallel Sysplex consists of having the capability to doany or all of the following:

v Install ISC-3 links and define them as CHPID type CFP.

v Install ICB-4 links (connects z9 EC to z990 and z890) or ICB-3 (connects z9 ECto z900) and define them as CHPID type CBP.

v Install IFB links (connects z9 EC to z10 EC) and define them as CHPID typeCIB.

v Define CHPID type ICP for internal connections within the CPC.

v Define, as a logical partition, a portion or all of the CPC hardware resources(CPs, ICFs, storage, and coupling connections) for use as a coupling facility thatconnects to z/OS or other operating system images for data sharing purposes.

v Connect to a coupling facility to share data.

The z9 EC models provide the following support for the Parallel Sysplex:

v The z9 EC’s Parallel Sysplex support consists of supporting coupling facilities onz9 EC, supporting attachment to remote coupling facilities via various type ofcoupling links, supporting Server Time Protocol (STP) or attachment to a SysplexTimer for purposes of sysplex-consistent time, and supporting various ancillaryCPC functions used by Parallel Sysplex support.

Note: If you want to implement a Parallel Sysplex between two sites, the fiberdistance has been increased to 100 km. Refer to page 6-13 for moreinformation.

v Internal Coupling links can be used to connect either z/OS images to CouplingFacilities (CFs) or CF images to other CF images within a z9 EC CPC to provideCF duplexing connections between LPARs in the same CPC.



|

These various interconnect formats provide the connectivity for data sharingbetween a coupling facility and the CPCs or logical partitions (LPs) directly attachedto it.

Parallel Sysplex Coupling Link Connectivityz9 EC supports IC, ICB, ISC-3, and IFB for passing information back and forth overhigh speed links in a Parallel Sysplex environment. These technologies are allmembers of the family of Coupling connectivity options available on z9 EC. WithServer Time Protocol (STP), coupling links can also be used to exchange timinginformation. Refer to “Server Time Protocol (STP)” on page 6-9 for more informationabout Server Time Protocol. Refer to Table 6-1 for a summary of the coupling linkoptions.

Table 6-1. Coupling Link Options

Link Type Description Communication Use Maximum Links1

IC Internal coupling channel Internal between CFsand z/OS LPARs

32

ICB-3 Copper connection betweenoperating system and couplingfacility

z9 EC, z9 BC,z800, z900

16

ICB-4 Copper connection betweenoperating system and couplingfacility

z9 EC, z9 BC,z890, z990

16

ISC-3 Fiber connection betweenoperating system and couplingfacility

z9 EC, z9 BC,z800, z900,z890, z990

48

IFB1, 2 InfiniBand fiber connectionbetween coupling facility andoperating system

z9 EC (CF), z9 BC(CF)z10 EC

16

Note:

1. The maximum number of coupling links combined (ICs, ICB-3s, ICB-4s, IFBs, and activeISC-3 links) cannot exceed 64 per server.

2. Each link supports definition of multiple CIB CHPIDs, up to 16 CHPID type CIB perfanout.

ICs are for internal communication between Coupling Facilities defined in LPARsand z/OS images on the same server.

ICBs are used for server-to-server communication over short distances. ICBs carrytraffic over 10 meter (33 feet) copper cables, of which three meters are reserved forintraserver connection. ICB-4s connect z9 BC to z990 and z890 and support a linkdata rate of 2 GigaBytes per second (GBps). ICB-3s connect z9 EC to z900 andz800 and support a link data rate of 1 GigaBytes per second (GBps).

The ISC-3 is another member of the family of Coupling Link options available on z9EC. ISC-3s support a link data rate of 2 Gbps and carry traffic over 9 micron singlemode fiber optic cables.

IFBs connect z9 EC coupling facility to z10 EC and support a link data rate of 3GBps over 12 lanes of fiber in each direction.



ISC-3 links only support peer mode. For ICB features, peer mode is supported forICB-4 and ICB-3. For IFB links, peer mode is supported.

Notes:

1. ISC-3, ICB-4, ICB-3, and IFB links require a point-to-point connection (directchannel attach between a CPC and a coupling facility).

2. ISC-3, ICB-4, ICB-3, and IFB links can be redundantly configured (two or moreISC-3, ICB-4, ICB-3, or IFB links from each CPC involved in coupling facilitydata sharing) to enhance availability and avoid extended recovery time.

Peer ModePeer mode applies to ICBs (ICB-4 to ICB-4 and ICB-3 to ICB-3), (ISC-3 to ISC-3),and IFBs (IFB to IFB). When in peer mode, ICB-4 operates at 2 GBps, ICB-3operates at 1 GBps, ISC-3 operates at 2 Gbps, and IFB operates at 3 GBps. Itprovides both sender and receiver capability on the same link. Peer links provide upto seven expanded buffer sets (compared to two buffers with send or receiver links).With these design improvements, the peer mode coupling facility links transmitfaster than older links as follows:v IFB vs ICB-4: 1.5x fasterv ICB-4 vs ICB-3: 2x fasterv ICB-3 vs ICB-2: 3x fasterv ISC-3 vs ISC: 2x fasterv IC-3 vs IC: 1.75x faster.

ISC-3 links operating in peer mode, perform at up to 200 Mbps for distances lessthan 10 km. ICB-3 links operating in peer mode can perform at a peek capacity of1000 Mbps and ICB-4 links can perform at a peek capacity of 2000 Mbps. IC linksoperating in peer mode can perform at up to 1250 Mbps.

When coupling within a z9 EC server, the IC channel can be shared among severalESA logical partitions and one coupling facility logical partition.

ISC-3 LinksThe ISC-3 feature, with a link data rate of 2 Gbps, is a member of the family ofCoupling Link options available on z9 EC. It has four ports and represents the thirdgeneration of coupling technology. The ISC-3 feature is used by coupled systems topass information back and forth over high speed links in a Parallel Sysplexenvironment. When STP is enabled, ISC-3 links can be used to transmit STPtimekeeping information and can also be defined as Timing-only links to other z9EC, z9 BC, z990, and z890 servers. The z9 EC ISC-3 feature is compatible withISC-3 features on z800, z900, z890, and z990 systems. ISC-3 (CHPID type CFP)can be defined as a spanned channel and can be shared among LPARS within andacross LCSSs. z9 EC supports 48 ISC-3 links in peer mode, 12 features (four linksper feature).

The ISC-3 feature is composed of:v One Mother card (ISC-M), FC 0217v Two Daughter cards (ISC-D), FC 0218.

Each daughter card has two ports or links, for a total of four links per feature. Eachlink is activated by using the Licensed Internal Code Configuration Control (LICCC)and can only be ordered in increments of one. The ISC-D is not orderable. ExtraISC-M cards can be ordered, up to a maximum of 12 or the number of ISC-D cards,


Chapter 6. Sysplex Functions 6-3

which ever is less. When the quantity of ISC links (FC 0219) is selected, theappropriate number of ISC-Ms and ISC-Ds is selected by the configuration tool.Each port operates at 2 Gbps.

Each port utilizes a Long Wavelength (LX) laser as the optical transceiver, andsupports use of a 9/125-micrometer single mode fiber optic cable terminated withan industry standard small form factor LC Duplex connector. The ISC-3 featureaccommodates reuse (at reduced distances) of 50/125-micrometer multimode fiberoptic cables when the link data rate does not exceed 1 Gbps. A pair of ModeConditioning Patch cables are required, one for each end of the link.

ICB LinksThe Integrated Cluster Bus (ICB-4, ICB-3, and ICB-2) uses the Self-TimedInterconnect (STI) cables to perform the coupling communication. When STP isenabled, ICB-3 and ICB-4 links can be used to transmit STP timekeepinginformation and can also be defined as Timing-only links to other z9 EC, z9 BC,z990, and z890 servers. Refer to Figure 4-2 on page 4-7 for an overview of the ICBlink connections.

ICB cables are ordered to match the quantity of ICBs on order. The quantity of ICBcables can be reduced, but cannot exceed the quantity of ICB functions on order.

ICB-4 Links (FC 3393)The ICB-4 link, with a link data rate of 2 GBps is a member of the family ofCoupling Link options available on z9 EC. ICB-4 is a ″native″ connection usedbetween z9 ECs. This is the recommended connection between z9 ECs and z990sand z890s. ICB-4s (CHPID type CBP) can be defined as a spanned channel andcan be shared among LPARs within and across LCSSs. An ICB-4 connectionconsists of one link that attaches directly to an STI port in the system, does notrequire connectivity to a card in the I/O cage, and provides one output port tosupport ICB-4 to ICB-4 connectivity. One ICB-4 connection is required for each endof the link.

The ICB-4 cable (FC 0228) is 10 meters, of which three meters are reserved forintraserver connection. Only one cable is required for each pair of ICB features.

ICB-4 channel paths are defined as CHPID type CBP. ICB-4 channel paths allow asingle channel to perform both send and receive functions on the same channel.

ICB-3 Links (FC 0993)The Integrated Cluster Bus-3 (ICB-3) link, with a link data rate of 1 GBps, is amember of the family of Coupling Link options available on z9 EC. It is used bycoupled systems to pass information back and forth over high speed links in aParallel Sysplex environment. ICB-3 links are used to connect z9 EC servers toz900 and z800 servers and can only be ordered in increments of one. Even thoughICB-3 is supported for connection to z990 and z890 servers, the recommended ICBconnection to these servers is ICB-4. ICB-3s (CHPID type CBP) can be defined asa spanned channel and can be shared among LPARs within and across LCSSs.There is an STI-3 card which resides in the I/O cage and provides two output portsto support the ICB-3 connections. The STI-3 card converts the 2 GBps input intotwo 1 GBps links. One ICB-3 feature is required for each end of the link. If theICB-3 feature already exists on the z9 EC server, it does not need to be reordered.

The ICB-3 cable (FC 0227) is 10 meters of which three meters are reserved forintraserver connection. Existing ICB-3 cables can be reused. Only one cable isrequired for each pair of ICB features.



ICB-3 channel paths are defined as CHPID type CBP (Cluster Bus Peer). ICB-3channel paths allow a single channel to perform both send and receive functions onthe same channel.

IC LinksThe Internal Coupling (IC) channel emulates the coupling facility link functions inLIC between images within a single system. IC channel implementation is totallylogical requiring no channel or cable hardware. However, a pair of CHPID numbersmust be defined in the IOCDS for each IC connection. IC channels cannot be usedfor coupling connections to images in external systems.

IC channels will have channel path CHPID type of ICP (Internal Coupling Peer).The rules that apply to the ICP CHPID type are the same as those which apply toCFP (InterSystem Coupling Peer) CHPID type, with the exception that the followingfunctions are not supported:v Service On/Offv Reset I/O Interfacev Reset Error Thresholdsv Swap Channel Pathv Channel Diagnostic Monitorv R/Vv Configuration Manager Vital Product Data (VPD).

IC channels have improved coupling performance over ICB-4 and ISC-3 channels.IC channels also improve the reliability while reducing coupling cost. Up to 32 ICchannels can be defined on z9 EC models but it is unusual to require more thanone link (two IC CHPIDs).

Refer to “Internal Coupling and HiperSockets Channels” on page 4-13 forrecommendations on CHPID usage.

InfiniBand (IFB) LinksAn InfiniBand (IFB) link is primarily used to connect a System z9 server to aSystem z10 server. IFB links use fiber optic cabling containing 12 “lanes” of fiber.Each “lane” consists of one fiber for receiving traffic and one fiber for sending traffic.Each “lane” supports a link data rate of .25 GBps for a total interface link data rateof 3 GBps.

An HCA1-O fanout card resides on the front of each processor book on the Systemz9 server. Each HCA1-O fanout card contains 2 ports, and a fiber optic cable isconnected to a port on an HCA1-O fanout card. The two ports exiting the fanoutcard use duplex MPO connectors.

All links initialize at the single data rate (3.0 GBps per port (.25 GBps per lane)) andauto-negotiate to a higher speed if both ends of the link support the higher speed.Therefore, when a System z10 EC is connected to a System z9 using point-to-pointIFB cabling, the link auto-negotiates to the highest common data rate - 3 GBps.

The maximum link distance over the fiber optic cabling is 150 meters (492 feet) ofwhich three meters are reserved for intraserver connection. The CHPID typeassigned to InfiniBand is CIB. A z9 EC coupling facility can support up to 16 IFBlinks.



|

|

Timing-only LinksIf you are configuring a timing network using STP, time synchronization may berequired in configurations other than a Parallel Sysplex. For a server that is not partof a Parallel Sysplex, but required to be in the same Coordinated Timing Network(CTN), additional Coupling links must be configured in order for the server to beconfigured in the CTN. These Coupling links are called Timing-only links. Use HCDto define Timing-only links and generate an STP control unit.

Coupling FacilityThe coupling facility provides shared storage and shared storage managementfunctions for the sysplex (for example, high speed caching, list processing, andlocking functions). Applications running on z/OS images in the sysplex define theshared structures used in the coupling facility.

PR/SM LPAR allows you to define the coupling facility, which is a special logicalpartition that runs Coupling Facility Control Code (CFCC). Coupling Facility ControlCode is Licensed Internal Code (LIC). There are two versions of CFCC:v ESA architecture (31-bit)v z/Architecture (64-bit).

When the CFCC is loaded by using the LPAR coupling facility logical partitionactivation, the z/Architecture CFCC is always loaded. However, when CFCC isloaded into a Coupling Facility guest of z/VM, the ESA architecture or z/ArchitectureCFCC version is loaded based on how that guest is running.

At LPAR activation, Coupling Facility Control Code automatically loads into thecoupling facility LPAR from the Support Element hard disk. No Initial Program Load(IPL) of an operating system is necessary or supported in the coupling facilityLPAR.

Coupling Facility Control Code runs in the coupling facility logical partition withminimal operator intervention. Operator activity is confined to the Operating SystemMessages task. PR/SM LPAR limits the hardware operator controls usuallyavailable for logical partitions to avoid unnecessary operator activity. For moreinformation, refer to System z9 Processor Resource/Systems Manager PlanningGuide.

Coupling facility channel hardware provides the connectivity required for datasharing between the coupling facility and the CPCs directly attached to it. Couplingfacility channels are point-to-point connections that require a unique channeldefinition at each end of the channel.

Note: CFCC compatibility code must be installed on other coupling facility imagesin the sysplex in order for them to be able to connect to coupling facilities onSystem z9 processors with large numbers of LPARs (i.e. more than 15)defined on them.

Coupling Facility Control Code (CFCC) ConsiderationsTo support migration from one coupling level to the next, you can run differentlevels of the Coupling Facility (CF) concurrently in different CF LPARs on the sameor different CPCs. Refer to “CFCC Enhanced Patch Apply” on page 6-7 for adescription of how a CF patch or a new CF code level can be applied to one CFLPAR in a CPC while not affecting the code level of other CF LPARs in the sameCPC.



When migrating CF levels, lock, list, and cache structure sizes increase to supportnew functions. This adjustment can have an impact when the system allocatesstructures or copies structures from one Coupling Facility to another at differentCFCC levels.

For any CFCC level upgrade, you should always run the CFSIZER tool which takesinto account the amount of space needed for the current CFCC levels. TheCFSIZER tool is available at the Parallel Sysplex web site, http://www.ibm.com/systems/z/pso.

CFCC Enhanced Patch ApplyThe CFCC patch apply process is enhanced to eliminate the need for a Power OnReset (POR) of the z9 EC to apply a disruptive CFCC patch. This enhancementprovides you the ability to:

v Selectively apply the new patch to one of possibly several CFs running on a z9EC. For example, if you have a CF that supports a test Parallel Sysplex and aCF that supports a production Parallel Sysplex on the same z9 EC, you nowhave the ability to apply a disruptive patch to only the test CF without affectingthe production CF. After you have completed testing of the patch, it can beapplied to the production CF as identified in the example.

v Allow all other LPARs on the z9 EC where a disruptive CFCC patch will beapplied to continue to run without being impacted by the application of thedisruptive CFCC patch.

This enhancement does not change the characteristics of a concurrent CFCCpatch, but does significantly enhance the availability characteristics of a disruptiveCFCC patch by making it much less disruptive.

CFCC Level 15CFCC Level 15 increases the allowable tasks in the Coupling Facility from 48 to112.

CFCC Level 15 includes the support introduced in previous CFCC levels.

CFCC Level 14CFCC Level 14 with dispatcher enhancements supports CF Duplexing. The CFCCdispatcher and internal serialization mechanisms have been modified to improve themanagement of coupled workloads from all environments under certaincircumstances.

CFCC level 14 support is available on z9 EC, z890, and z990. CFCC Level 14includes the support introduced with CFCC Level 13.

CFCC Level 13CFCC level 13 provides Parallel Sysplex availability and performanceenhancements. It provides changes that affect different software environments thatrun within a Parallel Sysplex. For example, DB2 data sharing is expected to see aperformance improvement, especially for cast-out processing against very largeDB2 group buffer pool structures.

CFCC Level 13 also includes features and functions from previous CFLEVELs.

CFCC Level 12CFCC level 12 provides support for the following functions:

v 64-bit addressing



http://www.ibm.com/systems/z/pso

http://www.ibm.com/systems/z/pso

The 64-bit addressing supports larger structure sizes and eliminates the 2 GB’control store’ line in the Coupling Facility. With this support, the distinctionbetween ’control store’ and ’non-control store’ (data storage) in the CouplingFacility is eliminated, and large central storage can be used for all CouplingFacility control and data objects.

v 48 internal tasks

Up to 48 internal tasks for improved multiprocessing of coupling facility requests.

v Message Time Ordering

Refer to “Message Time Ordering” on page 6-11 for more information.

v System-Managed CF Structured Duplexing (CF Duplexing)

CF Duplexing is designed to provide a System z9 model, hardware assisted,easy-to-exploit mechanism for duplexing CF structure data. This provides arobust recovery mechanism for failures such as loss of single structure or CF, orloss of connectivity to a single CF, through rapid failover to the other structureinstance of the duplex pair. Refer to “System-Managed CF Structure Duplexing”on page 6-11 for more information.

Note: There is a substantial structure size growth (due to 64-bit support) whengoing from CFLEVEL 10 or 11 up to CFLEVEL 12. Structure re-sizing isrequired when migrating to CFLEVEL 12.

CFCC Level 11CFCC Level 11 provides support for the following function:

v System-Managed CF Structured Duplexing (CF Duplexing)

CF Duplexing is designed to provide an S/390 G5/G6 model, hardware assisted,easy-to-exploit mechanism for duplexing CF structure data. This provides arobust recovery mechanism for failures such as loss of single structure or CF, orloss of connectivity to a single CF, through rapid failover to the other structureinstance of the duplex pair. Refer to “System-Managed CF Structure Duplexing”on page 6-11 for more information.

Coupling Connection ConsiderationsThere are several limits regarding coupling connections to be aware of whenordering and configuring these resources:v There is a combined maximum of 64 ISC, ICB, IC, and IFB links on a system.v Refer to Table 6-1 on page 6-2 for information on the individual link limits.

IOCP issues caution messages and HCD issues errors for the following:v ICP greater than 32.

I/O Configuration ConsiderationsICP IOCP supports ISC-3, ICB-4, ICB-3, IC, and IFB link definitions on z9 ECmodels CPCs.

For z9 EC, with z/OS V1.5 or later, HCD provides the following enhancements tosupport coupling facility definition:

v Controls for defining coupling facility channels. HCD also automatically generatesthe control unit and device definitions associated with CFP, CBP, CIB, and ICPchannel paths, when they are connected to their respective peer or receiverchannel paths. All IC channels paths must be connected.

v Controls for defining a logical partition as either a coupling facility or an operatingsystem logical partition. HCD also allows the definition of the logical partition asboth so its usage does not have to be pre-specified. This allows the flexibility of



usage to be determined at logical partition activation. This way, if a partition isused one day as a Coupling Facility and the next day as a z/OS image logicalpartition, the I/O definitions do not need to change. Additionally, you must usethese controls when defining a new logical partition in HCD.

IBM recommends that if you know a logical partition is used exclusively for ESA orexclusively for a coupling facility that you define it that way. This supplies the bestHCD rule checking. They also recommend that you use the HCD when possible, todefine the coupling facility channel configuration to the channel subsystem.

Server Time Protocol (STP)Server Time Protocol (STP) (FC 1021) provides the means for multiple System z9,z890, and z990 servers to maintain time synchronization with each other withoutusing a Sysplex Timer. STP is designed to synchronize servers configured in aParallel Sysplex or a sysplex without a coupling facility, as well as servers that arenot in a sysplex.

Server Time Protocol is a server-wide facility that is implemented in the LicensedInternal Code (LIC) of z9 EC server and CFs and presents a single view of time toProcessor Resource/Systems Manager (PR/SM). STP uses a message-basedprotocol to transmit timekeeping information over externally defined Coupling Linksbetween servers. The Coupling Links used to transport STP messages includeISC-3 links configured in peer mode, ICB-3 links, ICB-4 links, and IFB links. Theselinks can be the same links already being used in a Parallel Sysplex for CouplingFacility communications.

By using the same links to exchange timekeeping information and Coupling Facilitymessages in a Parallel Sysplex, STP can scale with distance. Servers exchangingmessages over short distance links, such as ICB-3, ICB-4, and IFB links, aredesigned to meet more stringent synchronization requirements than serversexchanging messages over long distance links, such as ISC-3 (distances up to 100km), where the synchronization requirements are less stringent.

The STP design introduces a concept called Coordinated Timing Network (CTN). ACoordinated Timing Network (CTN) is a collection of servers and Coupling Facilitiesthat are time synchronized to a time value called Coordinated Server Time. Theconcept of a Coordinated Timing Network fulfils two key goals:

v Concurrent migration from an existing External Time Reference (ETR) network toa time network using STP

v Capability of servers that cannot support STP to be synchronized in the samenetwork as servers that support STP (z9 EC, z9 BC, z990, and z890).

A CTN can be configured in two ways:

v Mixed CTN - Allows the coexistence of non-STP capable servers (z800 and z900servers) in an External Time Reference (ETR) timing network with STP capableservers. In a Mixed CTN, the Sysplex Timer provides the timekeeping for thenetwork. z800 and z900 servers are the only non-capable servers that cancoexist in a Mixed CTN.

v STP-only CTN - Each server must be configured with same CTN ID. The HMCor Support Element provides the user interface for all time related functions, suchas time initialization, time adjustment, and offset adjustment. The HMC orSupport Element must also be used to initialize or modify the CTN ID andnetwork configuration.



With STP you can:

v Initialize the time manually or by using an External Time Source (ETS). The ETScan be a dial-out time server or a connection to a Network Time Protocol (NTP)server so the Coordinated Server Time can be set to within 100 milliseconds ofthe ETS.

v Schedule periodic dial outs to a time service so that Coordinated Server Timemay be gradually steered to an international time standard

Note: If scheduled operations is set up to access the external time source, and“Use NTP” or “Use NTP with pulse per second (PPS)” is selected on theETS Configuration page, the requests generated by scheduled operationswill be ignored.

v Initialize Time Zone offset, Daylight Saving Time (DST) offset, and Leap secondsoffset

v Schedule changes to Time Zone offset, Daylight Saving Time (DST) offset, andLeap Seconds offset. STP can automatically schedule Daylight Saving Time,based on the selected Time Zone.

v Enhance the accuracy to an external time source by utilizing pulse per second(PPS) output from NTP servers. This allows you to set your time to an externaltime source to within 10 microseconds.

v Improve availability when power has failed for a single server (a server havingthe role of Preferred Time Server (PTS) and Current Time Server (CTS)) or whenthere is a site power outage in a multisite configuration where the server havingthe role of PTS/CTS is installed.

If an Internal Battery Feature (IBF) is installed on your z9 EC server, STP canreceive notification that power has failed and that the IBF is engaged. When STPreceives this notification from a server that has the role of PTS/CTS, STP canautomatically reassign the role of the Current Time Server (CTS) to the BackupTime Server (BTS), thus automating the recovery action and improvingavailability.

v Use an Application Programming Interface (API) to automate an STP CTNreconfiguration. If the Preferred Time Server (PTS) fails and the Backup TimeServer (BTS) takes over as the Current Time Server (CTS), an API is availableon the HMC so you can automate the reassignment of the PTS, BTS, and Arbiterroles. This improves availability by avoiding a single point of failure after the BTShas taken over as the Current Time Server (CTS).

v Save the STP configuration and time information across Power on Resets (POR)or power outages for a single or dual server STP-only CTN. This means you donot need to reinitialize the time or reassign the PTS/CTS role for a single serverSTP-only CTN or the PTS, BTS, or CTS roles for a dual server STP-only CTNacross POR or power outage events.

The benefits of STP include:

v Allowing clock synchronization without requiring the Sysplex Timer and dedicatedtimer links. This reduces costs by eliminating Sysplex Timer maintenance costs,power costs, space requirements, and fiber optic infrastructure requirements.

v Supporting a multisite timing network of up to 100 km over fiber optic cabling,thus allowing a sysplex to span these distances. This overcomes the limitation oftimer to timer links being supported only up to 40 km.

v Potentially reducing the cross-site connectivity required for a multisite ParallelSysplex. Dedicated links are no longer required to transport timing informationbecause STP and Coupling Facility messages may be transmitted over the samelinks.



||||

||||

|||

||||

||||||

||||||

|||||

v Supporting the configuration of different NTP servers for the Preferred TimeServer (PTS) and the Backup Time Server (BTS), which improves the availabilityof NTP servers used as an external time source. Only the PTS or the BTS canbe the Current Time Server in an STP-only CTN. If the PTS/CTS cannot accessthe NTP server or the pulse per second (PPS) signal from the NTP server, theBTS, if configured to a different NTP server, may be able to calculate theadjustment required and propagate it to the PTS/CTS. Then, the PTS/CTS willperform the necessary time adjustment steering.

Message Time OrderingAs server and Coupling Facility link technologies have improved over the years, thesynchronization tolerance between servers in a Parallel Sysplex has become morerigorous. In order to ensure that any exchanges of time-stamped informationbetween servers in a Parallel Sysplex involving the Coupling Facility observe thecorrect time ordering, time-stamps are now included in the message-transferprotocol between the servers and the Coupling Facility. This is known as MessageTime Ordering.

Therefore, when a Sysplex Timer exists and a Coupling Facility is configured as anICF on any z9 EC model, the Coupling Facility requires connectivity to the sameSysplex Timer that the other servers in its Parallel Sysplex are using for the timesynchronization. If the ICF is on the same server as a member of its ParallelSysplex, no additional Sysplex Timer connectivity is required, since the serveralready has connectivity to the Sysplex Timer. However, when an ICF is configuredon any z9 EC model which does not host any systems in the same ParallelSysplex, it is necessary to attach the server to the Sysplex Timer.

If you have an STP Mixed CTN, the only servers and Coupling Facilities that do notsupport STP but can coexist in the same timing network are servers that supportMTOF. The z900 and z800 servers and CFs satisfy this requirement. They mustalso be connected to the Sysplex Timer.

System-Managed CF Structure DuplexingA set of architectural extensions to the Parallel Sysplex is provided for the supportof System-Managed Coupling Facility Structure Duplexing (CF duplexing) ofcoupling facility structures for high availability. All three structure types (cachestructures, list structures, and locking structures) can be duplexed using thisarchitecture.

Support for these extensions on z9 EC is concurrent with the entire zSeries familyof servers in CFCC Level 12, and in z/OS V1R2, V1R3, and V1R4. It also requiresthe appropriate level for the exploiter support of CF Duplexing. Installing thesereleases and enabling this new function is designed to:

v Provide the necessary base for highly available coupling facility structure datathrough the redundancy of duplexing.

v Enhance Parallel Sysplex ease of use by reducing the complexity of CF structurerecovery.

v Enable some installations to eliminate the requirement for standalone CFs in theirParallel Sysplex configuration.

For those CF structures that support use of CF duplexing, customers have theability to dynamically enable (selectively by structure) or disable the use of CFduplexing.



||||||||

The most visible change for CF duplexing is the requirement to connect couplingfacilities to each other with coupling links. The required connectivity is bi-directionalwith a peer channel attached to each coupling facility for each remote CFconnection. A single peer channel provides both the sender and receivercapabilities; therefore, only one physical link is required between each pair ofcoupling facilities. If redundancy is included for availability, then two peer mode linksare required. However, this connectivity requirement does not necessarily imply anyrequirement for additional physical links. Peer mode channels can be sharedbetween ICF partitions and local z/OS partitions, so existing links between Systemz9 servers can provide the connectivity between both:v z/OS partitions and coupling facility images,v Coupling facility images.

One of the benefits of CF duplexing is to hide coupling facility failures and structurefailures and make total loss of coupling facility connectivity incidents transparent tothe exploiters of the coupling facility. This is handled by:

v Shielding the active connectors to the structure from the observed failurecondition so that they do not perform unnecessary recovery actions.

v Switching over to the structure instance that did not experience the failure.

v Reestablishing a new duplex copy of the structure at a specified time. This couldbe as quickly as when the coupling facility becomes available again, on a thirdcoupling facility in the Parallel Sysplex, or when it is convenient for the customer.

System messages are generated as the structure falls back to simplex mode formonitoring and automation purposes. Until a new duplexed structure is established,the structure will operate in a simplex mode and may be recovered throughwhatever mechanism provided for structure recovery prior to the advent of CFduplexing.

As the two instances of a System-Managed Duplex structure get update requests,they must coordinate execution of the two commands to ensure that the updatesare made consistently to both structures. Most read operations do not need to beduplexed.

z/OS operator commands display the status of the links for problem determination.In addition, the Resource Management Facility (RMF), as of z/OS V1R2, providesthe performance management aspects about the CF-CF connectivity and theduplexed structures. Together, these enable the installation to manage and monitorthe coupling facility configuration and new structure instances resulting from CFduplexing.

For more information on CF duplexing you can refer to the technical white paper,System-Managed CF Structure Duplexing, GM13-01303, at the Parallel Sysplexweb site, http://www.ibm.com/systems/z/pso.

Geographically Dispersed Parallel Sysplex (GDPS)In e-business, two important objectives for survival are systems designed to providecontinuous availability and near transparent disaster recovery (DR). Systems thatare designed to deliver continuous availability combine the characteristics of highavailability and near continuous operations to deliver high levels of service -targeted at 24 x 7.

To attain high levels of continuous availability and near-transparent disasterrecovery, the solution should be based on geographical clusters and data mirroring.




These technologies are the backbone of the GDPS solution. GDPS offers threedifferent solutions based on the underlying mirroring technology:

v The GDPS solution, based on Peer-to-Peer Remote Copy (PPRC, recentlyrenamed to IBM TotalStorage® Metro Mirror), is referred to as GDPS/PPRC.

v The GDPS solution based on Extended Remote Copy (XRC, recently renamed toIBM TotalStorage z/OS Global Mirror), is referred to as GDPS/XRC.

v The GDPS solution based on IBM TotalStorage Global Mirror is referred to asGDPS/Global Mirror.

GDPS is an integrated, automated application and data availability solutiondesigned to provide the capability to manage the remote copy configuration andstorage subsystem(s), automate Parallel Sysplex operational tasks, and performfailure recovery from a single point of control, thereby helping to improve applicationavailability. GDPS is independent of the transaction manager (e.g., CICS TS, IMS,WebSphere®) or database manager (e.g., DB2, IMS, and VSAM) being used, and isenabled by means of key IBM technologies and architectures.

GDPS/PPRCGDPS/PPRC is designed to manage and protect IT services by handling plannedand unplanned exception conditions, and maintain data integrity across multiplevolumes and storage subsystems. By managing both planned and unplannedexception conditions, GDPS/PPRC can help to maximize application availability andprovide business continuity.

GDPS/PPRC is capable of the following attributes:

v Near continuous Availability solution

v Near transparent D/R solution

v Recovery Time Objective (RTO) less than an hour

v Recovery Point Objective (RPO) of zero (optional)

v Protects against localized area disasters (distance between sites limited to 100km fiber)

Server Time Protocol (STP) allows you to have a GDPS/PPRC configuration acrosstwo sites up to 100 km apart. STP overcomes the limitations of the Sysplex Timerto Sysplex Timer links being supported only up to 40 km. STP is designed toeliminate the need for a third site to locate the second Sysplex Timer forinstallations where the multisite sysplex spans a distance greater than 40 km butless than 100 km.

GDPS/XRCExtended Remote Copy (XRC, recently renamed to IBM TotalStorage z/OS GlobalMirror) is a combined hardware and z/OS software asynchronous remote copysolution. GDPS/XRC includes automation to manage remote copy pairs andautomates the process of recovering the production environment with limitedmanual intervention, including invocation of CBU. This provides the ability toperform a controlled site switch for a planned or unplanned site outage and givessignificant value in reducing the duration of the recovery window and requiring lessoperator interaction.

GDPS/XRC is capable of the following attributes:

v Disaster recovery solution

v RTO between an hour to two hours



v RPO less than one minuteProtects against localized as well as regional disasters(distance between sites is unlimited)

v Minimal remote copy performance impact.

GDPS/Global MirrorGlobal Mirror enables a two-site disaster recovery and backup solution for z/OS andopen systems environments. Using asynchronous technology, Global Mirror isdesigned to maintain a consistent and restartable copy of data at a remote site thatcan be located at virtually unlimited distances from the local site.

GDPS/Global Mirror is capable of the following attributes:

v Disaster recovery solution

v RTO between an hour to two hours

v RPO less than one minute

v Protects against regional disasters (distance between sites is unlimited)

v Minimal remote copy performance impact

v Support for z/OS and open data.

Enhancements in GDPS V3.3IBM Global Services continues to enhance GDPS with:

v Extended HyperSwap™ functionality with IOS timing trigger. This is designed toallow HyperSwap to be invoked automatically when user-defined I/O timingthresholds are exceeded.

v Improved availability with enhanced recovery support in a CF structure duplexingenvironment. This is designed to ensure that the secondary PPRC volumes andthe CF structures are time consistent, thereby helping to provide consistentpplication restart times without any special recovery procedures.

v Performance improvements for System Logger in a z/OS Global Mirror(previously known as XRC) environment. XRC+ provides the option forasynchronous writes to staging data sets for logstreams. Previously, all writeshad to be synchronous. This limited the throughput for high-volume loggingapplications such as WebSphere, CICS, and IMS. The ability to do asynchronouswrites can allow the use of z/OS Global Mirror (XRC) for some applications forwhich it was not previously practical.

v Scalability improvements for XRC. This includes support for > 14 SDMs,improved parallelism of XRC commands, and support for Write Pacing to simplifyXRC tuning.

For a detailed description of the functions and capabilities of each of the aboveGDPS solutions, visit the GDPS web site at: http://www-03.ibm.com/systems/z/gdps/

Intelligent Resource Director (IRD)Intelligent Resource Director (IRD) is a function which optimizes your workload’sresource utilization of the z9 EC across multiple logical partitions.

This extension strengthens key z9 EC and S/390 platform technologies, includingz/OS Workload Manager, Processor Resource/Systems Manager (PR/SM) (logicalpartitioning hardware technology) and Parallel Sysplex Clustering technology. Thispowerful combination provides the ability to dynamically manage workloads withinmultiple logical operating system images executing on a single z9 EC server, as a



single large-scale computer resource, with dynamic workload management andphysical resource balancing built into the native operating system and underlyinghardware.

With IRD, z/OS WLM will exploit Parallel Sysplex technologies to monitorperformance of workloads on multiple images against those workload goals. z/OSWLM will then interact with the PR/SM hipervisor, directing PR/SM to dynamicallyadjust the physical CPU and I/O resource allocation of the hardware across themultiple operating system instances, without requiring Parallel Sysplex data-sharingto achieve these benefits, and totally transparent to customer workload applications.

IRD not only combines PR/SM, z/OS WLM, and Parallel Sysplex for LPAR CPUmanagement, but it also includes two additional z9 EC exclusives: DynamicChannel Path Management (DCM), and I/O (Channel) Subsystem Priority toincrease business productivity.

Through IRD technology extensions, the Parallel Sysplex will be able to dynamicallychange system image weights, reconfigure channels on the fly, and vary logicalprocessors on and off-line dynamically to maximize overall throughput across all ofthe system images to enable the most critical business application of highest priorityto get the resources (CPU and I/O) it needs.

LPAR CPU Management (Clustering)An LPAR cluster is the subset of the systems in a Parallel Sysplex that are runningas logical partitions on the same server.

LPAR CPU management allows dynamic adjustment of processor resources acrosspartitions in the same LPAR cluster. Through the z/OS WLM policy, installationsspecify the business importance and goals for their workloads. WLM will thenmanage these sets of logical partitions to provide the processor resources neededfor the work to meet its goals based on business importance.

LPAR CPU management requires z/OS WLM goal mode and a coupling facilitystructure which contains critical status information enabling cross-partitionmanagement of CP and I/O resources. These functions are delivered as a part ofz/OS and CFCC Level 12 on the z9 EC machines.

Note: The structure requires CFCC Level 9, although Level 12 comes in the z9 ECmachines. For example, a 9672 R06 with CFCC Level 9 can hold theIRD/WLM structures.

LPAR cluster support is also provided for Linux in the CP management area.

I/O Priority Queuing (IOPQ)I/O Subsystem Priority Queuing extends the classic strengths of I/O priority queuingby addressing other challenges that are not currently handled by existing I/O priorityschemes.

For example, prior to I/O Subsystem Priority Queuing, discretionary work in onepartition could dominate channels shared with business critical work in anotherpartition. With this new function, z/OS WLM and the Hardware ManagementConsole set priorities that will be used to give the business-critical work higherpriority access to the channels. This in turn may allow customers that do not exploit



MIF, in order to prevent such problems, to be able to do so now and may lead toreduced overall channel requirements. These new capabilities will help provideoptimal workload management.

The range of I/O weights for each logical partition is set within the HardwareManagement Console. WLM adjusts the I/O weights within this range. It can be afixed range, in which WLM would play no part.

Dynamic Channel Path Management (DCM)This portion of IRD is a combination of hardware strengths and software flexibility.Paths can now be managed between the processor and the control units in thesystem. Dynamic Channel Path Management (DCM) enables the system to respondto ever changing channel requirements by moving channels from lesser usedcontrol units to more heavily used control units as needed. DCM can managecontrol units connected to ESCON channels.

When used with z/OS Workload Manager (z/OS WLM) in Goal Mode, z/OS WLM isable to direct Dynamic Channel Path Management to move channels to helpbusiness critical work achieve its goals. This also helps reduce the requirement forgreater than 256 channels.

I/O Priority Queuing and Dynamic Channel Path Management (DCM) benefit theParallel Sysplex environment, with increased benefit in a multi-image environment(Parallel Sysplex). Although Parallel Sysplex data sharing is not required for IRD,the benefits of combining the two are unsurpassed.

I/O Priority Queueing has no value in a single-system environment.

Table 6-2. IOPQ in a Single-System Environment

IRD Function Require CF?Require Goal

Mode?Value in Single-System

Cluster

LPAR CPU Mgmt Yes Yes Little (Vary Logical CP)

DCM Yes No Yes

IOPQ No No No

Note: Both DCM and IOPQ do have more value with Goal Mode.

Workload Manager (WLM)With the z9 EC, Workload Manager (WLM) provides industry leading partitioningand workload management. Maximum utilization of all system resources is enabledthrough dynamic, automatic allocation of processor, memory, and I/O resourcesacross partitions based on real time workload demand and customer policy.

Workload Manager on the z9 EC provides end-to-end management of transactions,from the web-browser to data storage then back to the web-browser. WorkloadManager can exploit Cisco routers and facilitate dynamic and automaticself-management of data based on business priorities.

Using IBM’s Discrete Server Technology with the z9 EC and z/OS, installations maytake advantage of Workload Based Pricing to further reduce the cost of computingas applications continue to grow by using:

v Software pricing based on what you define, not what capacity has been installed.

v Common pricing for many cross-platform products.



v License Manager, which simplifies and centralizes via a standard licensingcertificate to control software usage billing.

Workload Based Pricing is adopted by many tools vendors, and provides for ’rigid’management within a flexible system.

EAL5 CertificationThe z9 EC server has received the Common Criteria EAL5 certification level(Evaluation Assurance Level 5) for the security of its logical partitions which rununder the control of the Processor Resource/Systems Manager (PR/SM). The z9EC's logical partitioning was evaluated against the ISO 15408 Common Criteriastandard, recognized in the global IT market.



Chapter 7. Cryptography

The Cryptographic support for z9 EC provides cryptographic function on everyProcessor Unit (PU). Integrated hardware is important to achieve the requiredthroughput and implement functions while maintaining balanced usage of systemresources. z9 EC provides the Message Security Assist Architecture (MSA) alongwith the expanded CP Assist for Cryptographic Function (CPACF) deliveringcryptographic support on every PU with DES and TDES data encryption/decryptionand Secure Hash Algorithm-1 (SHA-1) and Secure Hash Algorithm-256 (SHA-256)hashing.

On the z9 EC, the Crypto Express2 feature with two PCI-X adapters can be definedas a Coprocessor or as an Accelerator.

Support for CPACF is also available through the Integrated Cryptographic ServiceFacility (ICSF). ICSF is a component of z/OS that is designed to transparently useeither the CPACF or Crypto Express2 functions to balance the workload and satisfythe bandwidth requirements of the applications.

Products that include any of the cryptographic feature codes contain cryptographicfunctions which are subject to special export licensing requirements by the U.S.Department of Commerce. It is the your responsibility to understand and adhere tothese regulations whenever moving, selling, or transferring these products.

The cryptographic features are eligible for export under License Exception ENC asretail items to all end users in all countries except the embargoed, subject to theusual customer screening. The dormant cards themselves, without the enablingsoftware, are also eligible for export an NLR (No License Required) to all customersin all countries except the embargoed, subject to the usual screening.

CP Assist for Cryptographic Function (CPACF)The CP Assist for Cryptographic Function (CPACF) is available on the z9 ECserver. CPACF has been enhanced to include support of the Advanced EncryptionStandard (AES) for 128-bit keys; Secure Hash Algorithm-256 (SHA-2); and PseudoRandom Number Generation (PRNG). PRNG is a standard function supported inCrypto Express2. The CPACF provides a set of symmetric cryptographic functionsthat focus on the encryption/decryption function of SSL, Virtual Private Network(VPN), and data storing applications not requiring FIPS 140-2 level 4 security.

The CPACF for Data Encryption Standard (DES) and Triple Data EncryptionStandard (TDES) functions require enablement of the CPACF function (FC 3863)for export control. The DES, TDES, and AES functions use clear key values. TheCPACF for SHA-1 and SHA-256 hash function is shipped enabled.

If ICSF is not available, Message Security Assist (MSA) provides access to CPACFfor IBM and customer written programs. MSA instructions for invoking CPACFfunction for DES, TDES, AES, SHA-1, and SHA-256 are found in the z/ArchitecturePrinciples of Operation.



Crypto Express2The Crypto Express2 (CEX2C) feature (FC 0863), is designed to satisfy high-endserver security requirements. Crypto Express2 executes functions that werepreviously offered by PCICA and PCIXCC features, performing hardwareacceleration for Secure Sockets Layer (SSL) transactions and clear key RSAacceleration. The Crypto Express2 feature, with two PCI-X adapters, is configurableand can be defined for secure key transactions or Secure Sockets Layer (SSL)acceleration.

The Crypto Express2 feature conforms to the Federal Information ProcessingStandard (FIPS) 140-2 Level 4 Certification.

The Crypto Express2 feature supports:

v Consolidation and simplification using a single cryptographic coprocessor

v Up to a maximum of eight features per server

v Public key cryptographic functions

v Hardware acceleration for Secure Sockets Layer (SSL) and Transport LayerSecurity (TLS) protocols

v User Defined Extension (UDX)

v 13-19 Personal Account Numbers (PANs)

v Secure (encrypted) keys for AES-128

v 2048-bit key RSA management capability.

Feature

Max number offeatures supported

per server

Number ofcoprocessors

per feature

Max number ofcoprocessors

supported per server

Number of domainssupported percoprocessor

Number ofactive LPARs

per server

CryptoExpress2

8 2 16 16 60

The total quantity of FICON, OSA, and crypto features cannot exceed 60 featuresper server.

All logical partitions (LPARs) in all Logical Channel Subsystems (LCSSs) haveaccess to the Crypto Express2 feature, up to 30 LPARs per feature. Although thefeature occupies an I/O slot and is assigned two PCHID values, it does not useChannel Path Identifiers (CHPIDs).

Notes:

1. Enhancements to Cryptographic Support for z/OS and z/OS.e Web deliverableis found at: http://www.ibm.com/eserver/zseries/zos/downloads

2. The end of service for a web deliverable occurs at the end of service for therelease on which it runs.

User-Defined ExtensionsUser-Defined Extensions to the Common Cryptographic Architecture (CCA) supportprogram that executes within Crypto Express2 will be supported by an IBM ServiceOffering. For unique customer applications, Crypto Express2 will support the loadingof customized cryptographic functions on z9 EC. Support is available through ICSFand the Cryptographic Support for z/OS V1R6/R7 and z/OS.e V1R6/R7 webdeliverable. Under a special contract with IBM, as a Crypto Express2 customer, youwill gain the flexibility to define and load custom cryptographic functions yourself.



|

|

http://www.ibm.com/eserver/zseries/zos/downloads

This service offering can be requested by referring to the IBM ’Cryptocards’ website, then selecting the Custom Programming option.

The following is required for UDX support:

v One or more Crypto Express2 features

v A Hardware Management Console

v A TKE workstation at 5.0 code level, if the UDX requires access control point

v Z/OS V1R6 or later

v Cryptographic Support for z/OS.e V1.6/V1.7 web deliverable (no longerdownloadable), or Enhancements to Cryptographic Support for z/OS and z/OS.eV1 R6/R7 web deliverable, and PTFs.

Trusted Key Entry (TKE) WorkstationThe Trusted Key Entry (TKE) workstation (FC 0839) and the TKE 5.0 LicensedInternal Code (FC 0855) or TKE 5.1 Licensed Internal Code (FC 0856) are optionalfeatures that provide a basic key management system: master key and operationalkey entry support. Both TKE 5.0 and TKE 5.1 continue to support the ability to storekey parts on diskettes, paper, or, optionally, smart cards and to store authority keyson a diskette or, optionally, a smart card. You can also use TKE to log on to theCryptographic Coprocessors using a passphrase or, optionally, a logon key pair.

The Trusted Key Entry (TKE) workstation supports two user categories –auto-logged user and service user. With TKE 5.1, the service user category wasintroduced to help separate the tasks for servicing the console and the auto-loggeduser category has access only to the basic operations tasks. TKE 5.1 also providesa new task layout display to help improve the look and usability of the TKE console.

The optional TKE features are:v TKE 5.0 code (FC 0855) or TKE 5.1 code (FC 0856) and TKE Workstation (FC

0839)v TKE Smart Card Reader (FC 0887)v TKE Additional Smart Cards (FC 0888)

To use the TKE function on a z9 EC, CPACF must be installed. The TKEworkstations require the TKE 5.0 code or TKE 5.1 code and the TKE unit thatcontains Ethernet capability and PCIX Card.

Trusted Key Entry (TKE) with Smart Card ReaderSupport for an optional Smart Card Reader attached to the TKE workstation allowsthe use of smart cards that contain an embedded microprocessor and associatedmemory for data storage. Access to and use of confidential data on the smart cardis protected by a user-defined Personal Identification Number (PIN).

RMF MonitoringThe Cryptographic Hardware Activity report provides information about the activitiesin Crypto Express2 feature when it is configured as a coprocessor (CEX2C).

For CEX2C, the request rate is reported per daughter card which is the number ofrequests per second. In addition, the utilization (how much of the interval thefeature is busy) and the average execution time of all operations is reported.


Chapter 7. Cryptography 7-3

Certification

FIPS CertificationThe Crypto Express2 feature configured as a coprocessor is designed for FederalInformation Processing Standard (FIPS) 140-2 Level 4 certification.

EAL5 CertificationThe z9 EC server has received the Common Criteria EAL5 certification level(Evaluation Assurance Level 5) for the security of its logical partitions which rununder the control of the Processor Resource/Systems Manager (PR/SM). The z9EC's logical partitioning was evaluated against the ISO 15408 Common Criteriastandard, recognized in the global IT market.

Remote Loading of ATM and POS KeysRemote key loading refers to the process of loading Data Encryption Standard(DES) keys to Automated Teller Machines (ATMs) and Point of Sale (POS) devicesfrom a central administrative site. These two new features provide:

v The ability to load initial keys to an ATM or a POS device from a remote location

v Enhanced capabilities for exchanging keys with non-CCA cryptographic systems.



Chapter 8. Cabling

z9 EC utilizes Small Form Factor (SFF) connectors for ESCON (MT-RJ), FICON(LC Duplex), ISC-3 (LC Duplex), and Gigabit Ethernet (LC Duplex). The speed ofthe link is determined by the architecture and ranges from 17MBps (ESCON), to 1,2, or 4 Gbps (FICON), and 10 Gbps (10 Gigabit Ethernet). Up to three generationsof features can co-exist on z9 EC. Each feature has its own unique requirements;transceiver, connector, unrepeated distance, and link loss budget.

Fiber optic cables for the z9 EC are available via IBM Site and Facilities Services.

IBM Site and Facilities Services has a comprehensive set of scalable solutions toaddress IBM cabling requirements, from product-level to enterprise-level for small,medium, and large enterprises. IBM Site and Facilities Services is designed todeliver convenient, packaged services to help reduce the complexity of planning,ordering, and installing fiber optic cables. The appropriate fiber cabling is selectedbased upon the product requirements and the installed fiber plant.

The services include:v IBM Facilities Cabling Services — fiber transport systemv IBM IT Facilities Assessment, Design, and Construction Services —

optimized airflow assessment for cabling

These services take into consideration the requirements for all of the protocols andmedia types supported on the System z10 EC, System z9, and zSeries (forexample, ESCON, FICON, Coupling Links, OSA) whether the focus is the datacenter, the Storage Area Network (SAN), the Local Area Network (LAN), or theend-to-end enterprise.

Under IBM Facilities Cabling Services, there is the option to provide IBM FiberTransport System (FTS) trunking commodities (fiber optic trunk cables, fiberharnesses, panel-mount boxes) for connecting to the z10 EC, z9 EC, z9 BC, z990,and z890. IBM can reduce the cable clutter and cable bulk under the floor. Ananalysis of the channel configuration and any existing fiber optic cabling isperformed to determine the required FTS trunking commodities. IBM can also helporganize the entire enterprise. This option includes enterprise planning, new cables,fiber optic trunking commodities, installation, and documentation.

Under IBM IT Facilities Assessment, Design, and Construction Services, theOptimized Airflow Assessment for Cabling option provides you with acomprehensive review of your existing data center cabling infrastructure. Thisservice provides an expert analysis of the overall cabling design required to helpimprove data center airflow for optimized cooling, and to facilitate operationalefficiency through simplified change management.

For additional information on cabling, you can refer to any of the following:v System z9 Enterprise Class Installation Manual for Physical Planningv Resource Link (http://www.ibm.com/servers/resourcelink), under Services from

the navigation bar.




Fiber Quick Connect (FQC) ESCON CablingFiber Quick Connect (FQC) ESCON harness cabling, harness brackets, andmounting hardware are ordered with the z9 EC server as feature codes and areordered in conjunction with IBM Facility Cabling Services solution. This featureenables trunk cables to connect to ESCON channels using under-the-coverattachment harnesses. These harnesses are installed when your system is built,and your z9 EC arrives ready to connect the trunk cables at your site.

Cable OrderingFiber optic cables for the z9 EC are available from IBM Site and Facilities Services.

The following table lists the connectors and cabling supported for each of thefeatures available for ordering on the z9 EC. The features brought forward on anupgrade from z990 are also listed.

Table 8-1. Cabling

FeatureCode Feature Name

ConnectorType Cable Type

0167 HCA1-O 12x MPO 50 micron, OM3 12x IB-SDR

0219 ISC-3 link LC Duplex 9 micron SM

6155 ETR MTRJ 62.5 micron MM

2323 16-port ESCON channel MTRJ 62.5 micron MM1

2319 FICON Express LX LC Duplex 9 micron SM2

2320 FICON Express SX LC Duplex 50, 62.5 micron MM

3319 FICON Express2 LX LC Duplex 9 micron SM2

3320 FICON Express2 SX LC Duplex 50, 62.5 micron MM

3321 FICON Express4 LX LC Duplex 9 micron SM2

3322 FICON Express4 SX LC Duplex 50, 62.5 micron MM

3368 OSA-E2 10 GbE LR (2)8 SC Duplex 9 micron SM

3364 OSA-E2 GbE LX LC Duplex 9 micron SM

3366 OSA-E2 1000Base-T RJ-45 Category 5 UTP

1364 OSA-E GbE LX3 LC Duplex 9 micron SM

23644 OSA-E GbE LX SC Duplex5 9 micron SM

3365 OSA-E2 GbE SX LC Duplex 50, 62.5 micron MM

1365 OSA-E GbE SX LC Duplex 50, 62.5 micron MM

23654 OSA-E GbE SX SC Duplex5 50, 62.5 micron MM

1366 OSA-E 1000Base-T6 RJ-45 Category 5 UTP7

23664 OSA-E Fast Ethernet6 RJ-45 Category 5 UTP



Table 8-1. Cabling (continued)

FeatureCode Feature Name

ConnectorType Cable Type

Notes:1. MM - Multimode fiber2. SM - Single mode fiber3. OSA-E - OSA-Express4. Brought forward to z9 EC on an upgrade from z9905. The OSA-Express GbE features brought forward from z990 have a different connector

(SC Duplex) than the new OSA-E GbE features (LC Duplex)6. 1000Base-T - Ethernet feature7. UTP - Unshielded Twisted Pair8. OSA-E2 - OSA-Express2

Refer to the Services section of Resource Link for additional information.

Cabling ResponsibilitiesFiber optic cables ordering, cable planning, labeling, and placement are thecustomer responsibilities for new installations and upgrades. Fiber optic conversionkits and Mode Conditioning Patch (MCP) cables are not orderable as features on az9 EC. Representatives will not perform the fiber optic cabling tasks without aservice contract.

The following tasks are required to be performed by the customer prior to machineinstallation:

v All fiber optic cable planning.

v All purchasing of correct, qualified, fiber cables.

v All installation of any required Mode Conditioning Patch (MCP) cables.

v All installation of any required Conversion Kits.

v All routing of fiber optic cables to correct floor cutouts for proper installation toserver.

– Use the Physical Channel Identifier (PCHID) report or the report from theChannel Path Identifier (CHPID) Mapping Tool to accurately route all cables.

v All labeling of fiber optic cables with PCHID numbers for proper installation toserver.

– Use the PCHID report or the report from the CHPID Mapping Tool toaccurately label all cables.

Additional service charges may be incurred during the machine installation if thepreceding cabling tasks are not accomplished as required.

Cabling ReportWhen the Fiber Quick Connect feature is ordered, a second part of the PCHIDReport is provided to document the connections between the ESCON ports and theMTP couplers. Figure 8-1 on page 8-4 shows an example of the cabling portion ofthe report.


Chapter 8. Cabling 8-3

The columns in this part of the report represent the following data:

Cage Displays the number of cages installed and the location of the I/O and CPCcages containing I/O. (CPC cages are shown only if ICBs are part of theconfiguration.) In this example there is one cage: a CPC cage in the top ofthe ’A’ frame (A19B).

Slot Displays the I/O slot where the harness is plugged.

F/C Displays the feature code of the channel card where the harness isplugged.

Brkt Displays the MTP bracket that the harness plugs into (F is the bracket inthe front of the frame, B is the bracket in the back of the frame, R is thebracket to the right of the frame, L is the bracket to the left of the frame).

PCHID/Harn-LegDisplays the PCHID number port harness is plugged into, the harnessnumber based on the MTP coupler the harness is plugged to, the harnessleg that is plugged into the port.

-------------------------- Fiber Trunking Service ---------------------------Cage Slot F/C Brkt PCHID/Harn-LegA19B 09 2323 1F(6.10)R 17/3-4 18/3-5 19/3-6 1A/4-1

1B/4-2 1C/4-3 1D/4-4 1E/4-5A19B 17 2323 2F(6.10)R 1F/6-1 20/6-2 21/6-3 22/6-4

23/6-5 24/6-6 25/7-1 26/7-2Legend:

A19B - Top of 'A' frameA01B - Bottom of 'A' frameZ01B - Bottom of 'Z' frameZ15B - Top of 'Z' frame2323 - ESCON Channel 16 Ports2F(1.5)R - Bracket number/Frame side/Group/Frame side

Figure 8-1. Cabling Section of the PCHID Report Sample



Chapter 9. Hardware Management Console and SupportElement

The z9 EC models include a Hardware Management Console and two internalSupport Elements (SEs) located on the ’Z’ frame. The second Support Element, thealternate SE, is standard on z9 EC models and is configured the same as, andserves as an alternate to, the primary SE.

Closed PlatformUnlike previous Hardware Management Consoles, the new HMC operates as aclosed platform which makes it easier to ensure the security and integrity. Thepurpose of the HMC is to provide a platform for the execution of the HMCapplication.

To enhance security, the new HMC is configured with a firewall to limit networkaccess in and out. By default, no external connections are allowed through thefirewall. As objects are defined to the HMC application, the necessary firewall rulesare added to allow for communications to and from these objects. Firewall rules arealso added to allow external user connections, access by Product Engineering, andthe customization of network settings to allow specific applications.

A new task, Format Media, is used to allow the customer or service representativeto format removable media such as diskettes and DVDs.

The Console Action task, Shutdown or Restart, has been added to the HMC toallow the user to perform an orderly shutdown.

The Hardware Management Console communicates with each CPC through theCPC’s Support Element. When tasks are performed at the Hardware ManagementConsole, the commands are sent to one or more SEs which then issue commandsto their CPCs. Commands can be sent to as many as all of the CPCs defined to theHardware Management Console. One Hardware Management Console can controlup to 100 CPCs and one CPC can be controlled by 32 Hardware ManagementConsoles. Refer to Figure 9-1 on page 9-2 for an example of a typical HardwareManagement Console configuration.



On z9 EC models, CPCs configured to a Hardware Management Console are thoseCPCs whose internal SEs are:v Attached by local area network (LAN) to the Hardware Management Consolev Defined to have the same domain name and domain password as the Hardware

Management Consolev Defined in the Defined CPCs group at the Hardware Management Console.

The internal SEs for each CPC allows the Hardware Management Console tomonitor the CPC by providing status information. Each internal SE provides theHardware Management Console with operator controls for its associated CPC sothat you can target operations:v In parallel to multiple or all CPCsv To a single CPC.

Hardware Management Console Enhancements for System z9 ECThis section highlights the significant changes and enhancements for the z9 ECVersion Code 2.9.2. For more detailed information on these enhancements, refer tothe System z Hardware Management Console Operations Guide. You can also referto the education module on Resource Link, How to Use the Hardware ManagementConsole.

z/VM Integrated Systems Managementz/VM integrated systems management from the HMC provides out-of-the-boxintegrated GUI-based basic management of z/VM guests. The HMC will detectsz/VM images. The z/VM integrated systems management function includes disk,network adaptor and memory management, guest activation and deactivation, andthe display of guest status.

User AuthenticationYou can configure the HMC to use a LDAP server to perform user ID and passwordauthentication at logon time. The HMC still defines the user ID and the roles given

HardwareManagement

Console

The frame contains the CPC

The frame contains the SEs

A

Z

Local AreaNetwork(LAN)

IBM

server

Figure 9-1. Hardware Management Console Configuration



to that user ID, but an enterprise directory (LDAP) server is used to authenticate theuser. This eliminate the need to store the user ID’s password locally.

This function allows the use of the current user ID/password policy for HMC userID/passwords, and provides one centralized user ID/password control mechanism tohelp meet the user’s corporate security guidelines.

New and Revised TasksDetails about the HMC changes can be found in the System z HardwareManagement Console Operations Guide. Details about the SE changes can befound in the System z9 Support Element Operations Guide.

Hardware Management Console Application (HWMCA)The Hardware Management Console Application (HWMCA) is a licensed softwareapplication installed on the Hardware Management Console. The applicationprovides an easy-to-use object-oriented Graphical User Interface (GUI) you use tomonitor and operate your CPCs. Starting the application makes the user interfaceavailable. You can directly manipulate objects displayed in the HardwareManagement Console or Support Element Workplace™ using a mouse or keycombinations. The application begins whenever the console is powered-on orrebooted. For more detail about the Hardware Management Console and SEworkplace, refer to the System z Hardware Management Console Operations Guideand to the System z9 Support Element Operations Guide. You can also take aneducation course on Resource Link on How to use the Hardware ManagementConsole.

CustomizationYou can use the Hardware Management Console Workplace or Support ElementWorkplace Settings task to customize the presentation characteristics of theHardware Management Console or Support Element. These customized settingscan be saved to a diskette and used on other Hardware Management Consoles orSupport Elements if desired. The Settings task allows you to:

v Modify the default colors or use grey patterns instead of colors.

v Associate a color or pattern with any unacceptable status value you define todistinguish between types of exceptions.

v Change the background color of the Views Area used to indicate exception andnon-exception situations.

v Modify the default color associated with pending hardware or operating systemmessages.

v Enter the Hardware Management Console or Support Element TCP/IP addressand domain name.

Status ReportingEach internal SE monitors the operation of its associated CPC and any CPCimages running on the CPC and sends status to the Hardware ManagementConsole for consolidation and exception processing.

Exception processing surfaces only those hardware status items you define asunacceptable to the running of your systems. You can define the conditions forexception processing at the Hardware Management Console or Support Elementusing the CPC Details panel associated with each CPC or CPC image. The


Chapter 9. Hardware Management Console and Support Element 9-3

Hardware Management Console and Support Element use customizable panelcolors to indicate hardware status requiring attention.

The Hardware Management Console Workplace and Support Element Workplacedisplay hardware status by using color (or grey patterns) to indicate acceptable orunacceptable status for objects. Objects include CPCs, CPC Images, CPs, andCHPIDs. Unacceptable status results in an exception situation that causes the colorto change for the:

v Entire Views Area background.

v Object background in the Work Area for the object with the unacceptable status.

v Group object background in the Work Area for any group that the object with theunacceptable status is part of.

The default color change is from green (acceptable status) to red (unacceptablestatus). You can customize these colors (or patterns) using the HardwareManagement Console Workplace or Support Element Workplace Settings task.

Hardware MessagesThe Hardware Management Console allows you to monitor the hardware messagesfrom any CPC, CPC Images, or any group of CPCs or CPC Images configured to it.The Support Element allows you to monitor the hardware messages from its CPCor any CPC Images configured to it.

The Hardware Management Console Workplace and Support Element Workplaceuse color (or grey patterns) and flashing icons to indicate when hardware messagesare pending for CPCs or CPC Images.

Hardware messages present information about problems that are detected, suggestactions where possible, and aid in requesting service when appropriate. When amessage has been handled, it is deleted from all the Hardware ManagementConsole(s) and SE(s).

When hardware messages are pending for a hardware object or group of hardwareobjects:v The background of the object and its associated group turns blue (the default)v The Hardware Messages icon turns blue and flashes.

Service Required StateThe ’Service Required’ state indicates that the spare hardware shipped with theCPC has been depleted. When a part fails causing the use of the last redundantparts of that type, you now have just the required number of parts to keep the CPCrunning. This message is a reminder to you and the service representative thatrepairs should be made at the earliest possible time before addition.

The following conditions can cause a Service Required state:v Whenever the system is running with N Mode Powerv Whenever the system is running in N Mode on the Service Networkv Primary SE loss of communications with the Alternate SEv A PU is needed for sparing and no designated spare PU is availablev Insufficient spare PUs to support Capacity BackUp (CBU) or Disaster Recovery

Assurance (if either feature is installed)v Memory Sparing Threshold is reachedv IML will cause CPC to go down or to an unacceptable conditionv Alternate SE is fenced due to automatic switchover.



Degrade IndicatorThe text ’Degraded’ indicates that, although the CPC is still operating, somehardware is not working. It displays on an object in the CPC group on the remoteconsole, the Hardware Management Console, and on the SEs when:v Loss of channels due to CPC hardware failurev Loss of memoryv One or more books are no longer functioningv The ring connecting the books is openv Capacity BackUp (CBU) resources have expiredv Processor cycle time reduced due to temperature problemv CPC was IMLed during cycle time reduction.

Operating System MessagesLocal operating systems and coupling facility control code running in a couplingfacility logical partition can use the console integration facility of the hardware tosend operator messages to be displayed by the Hardware Management Console orSupport Element. The Hardware Management Console and Support Element allowyou to monitor and respond to the operating system messages from any CPCImage, coupling facility, or any group of CPC Images configured to it.

The Hardware Management Console Workplace and Support Element Workplaceuse color (or patterns) and flashing icons to indicate when important operatingsystem messages are pending for CPC(s) or CPC Images.

When important operating system messages are pending for a hardware object orgroup of hardware objects:v The background of the object and its associated group turns cyan (the default)v The Operating System Messages icon turns cyan and flashes.

For a coupling facility logical partition, coupling facility control code uses theconsole integration facility to display coupling facility messages and to acceptcoupling facility control code commands. The console integration facility, through theOperating System Messages task, provides the only interface for enteringcommands to an operating coupling facility logical partition.

Problem Analysis and ReportingEach primary SE monitors and analyzes problems detected on the associated CPC.For problems that are isolated to a single CPC, the results are reported to theHardware Management Console as a hardware message. For those problems thatpotentially involve multiple CPCs, that problem data is sent to the HardwareManagement Console, where data from multiple CPCs is analyzed and reported.The Hardware Management Console configured as a problem analysis focal pointcan perform:

v Problem analysis for ESCON, coupling facility, and Sysplex Timer link faultsencountered by the CPCs configured to it.

v Service calls for all CPCs configured to it. Enabling the Hardware ManagementConsole as a call home server identifies the Hardware Management Console ashaving a modem or LAN/Internet connection that all CPCs configured to it canuse for placing service calls.

Virtual RETAINThe Virtual RETAIN® function provides a way to capture problem data when aproblem is to be called into IBM service and RETAIN is not available. If RETAIN is



not available, the problem data is put into Virtual RETAIN on the Support Elementhard disk, and a Hardware Message is displayed for the Hardware ManagementConsole, SE, and/or remote console user to instruct the customer to contact IBMService to gather the problem data.

Licensed Internal Code (LIC)Each Hardware Management Console and each SE has Licensed Internal Code(LIC) and is subject to periodic updates from IBM.

On systems with multiple Hardware Management Consoles, one of the HardwareManagement Consoles should be configured as a LIC change management focalpoint. The Hardware Management Console configured can:

v Retrieve and distribute Licensed Internal Code updates for the HardwareManagement Consoles remotely from IBM.

v Retrieve and distribute SE LIC updates to all the SEs of all the CPCs configuredto the Hardware Management Console.

Remote I/O Configuration and IOCDS ManagementEach CPC requires a definition of the I/O attached to it. The HardwareConfiguration Definition (HCD) is a z/OS application that aids in the definition of allthe I/O and aids in the distribution of the appropriate I/O definitions to theappropriate CPCs.

The Hardware Management Console configured as a change management focalpoint assists HCD in finding the names of all defined CPCs. A single HCD thendistributes the appropriate IOCDS and IPL parameters to the various SEs of theCPCs defined to the same Hardware Management Console with changemanagement capability.

Scheduled OperationsThe Hardware Management Console and Support Element provide support forscheduling the times and dates for automatic Licensed Internal Code (LIC) updatesand backup of critical hard disk data for the Hardware Management Console, theCPCs configured to the Hardware Management Console, or the Support Element.You can accomplish this by using the Customize Scheduled Operations task.

For the Hardware Management Console the Customize Scheduled Operations task,available from the Console Actions Work Area, allows you to schedule the followingLIC-related operations:v Accept internal code changesv Backup critical hard disk informationv Install internal code changes / Activatev Remove internal code changes / Activatev Retrieve internal code changesv Retrieve internal code changes for defined CPCsv Single Step code changes retrieve and applyv Transmit electronic service agentv Transmit system availability data.

For the CPCs configured to the Hardware Management Console the CustomizeScheduled Operations task, available from the Operational Customization Tasks list,allows you to schedule the following LIC-related operations:v Accept internal code changesv Activate



v Backup critical hard disk informationv Deactivatev Install internal code changes / Activatev Remove internal code changes / Activatev Retrieve internal code changesv Single step code changes retrieve and applyv Transmit system availability datav Access external time source.

For the Support Element the Customize Scheduled Operations task, available fromthe CPC Operational Customization Tasks list, allows you to schedule the followingLIC-related operations:v Accept internal code changesv Activatev Backup critical hard disk informationv Deactivatev Install internal code changes / Activatev Remove internal code changes / Activatev Retrieve internal code changesv Single step code changes retrieve and applyv Transmit electronic service agentv Transmit system availability datav Access external time source.

Remote Support Facility (RSF)The Hardware Management Console provides Remote Support Facility (RSF) to aidin the service and maintenance of your system. RSF provides:v Automatic or customer initiated call for servicev Automatic or customer downloading of the latest LIC change levelsv Automatic downloading of the latest phone list.

Remote Support Facility communicates with the IBM Service Support System usingsecure TCP/IP protocols. The communication may be through the enterprise LAN tothe Internet (either directly or via a customer supplied SSL Proxy) or through theHMC-provided modem and customer supplied phone line. In both cases, theconnection uses high grade SSL encryption, and is an outbound only connection.

Automation and API SupportApplication Programming Interfaces (APIs) on the Hardware Management Consoleand Support Element provide an end-user with the ability to view and manipulatemanaged objects, and support of the SNMP protocol for development and use byremote management applications.

These APIs contain the ability to get/set a Hardware Management Console orSupport Elements managed object’s attributes, issue commands to be performed ona managed object from a local or remote application, and receive asynchronousevent notifications. These APIs provide a mechanism to IBM, independent systemmanagement vendors, and an enterprise, to integrate with the HardwareManagement Console Application (HWMCA). For more detailed information on theAPIs, refer to System z Application Programming Interfaces.



ActivationActivating a CPC is an intelligent (LIC controlled) process that takes the CPC fromits current state to a fully operational state. The activation may involve a power-on,power-on reset, and IPL, as necessary.

To activate a CPC, you must specify system activation information for each CPCconfigured to the Hardware Management Console.

You specify the CPC system activation information using activation profiles.Activation profiles allow you to define multiple power-on reset (POR) configurationsfor the same CPC. These profiles allow you to tailor the CPC resources (centralprocessors, storage, and channels) to meet varying business and workload needs.

You use activation profiles to define PR/SM LPAR mode configurations. Activationprofiles are stored on the SE hard disk so that they are available for future activaterequests.

You can modify default activation profiles that are provided with the CPC and usethem as templates to produce customized profiles to fit your processing needs.

There are three types of activation profiles you can produce:v Reset - Used during power-on reset processingv Load - Used to load an operating systemv Image - Used to define an logical partition.

For PR/SM LPAR mode, you must define a Reset profile and one Image profile foreach logical partition.

Hardware Management Console and Support Element NetworkConnection

A local Hardware Management Console must be connected to its Support Elementsthrough a Local Area Network (LAN). z9 EC models use a Dual Ethernet (FC 0089)for the LAN wiring between the Hardware Management Console and the SupportElements. The necessary LAN adapters for the Support Elements and theHardware Management Console may be specified as features on the system order.

Network ProtocolsThe new HMC for z9 EC uses a single network protocol, TCP/IP, whencommunicating with the Support Elements (SEs). This network protocol is used forboth discovery and normal communications purposes.

Customizable Console Date and TimeThe Customize Console Date and Time task has been updated to use moretraditional time zone definitions rather than specific offsets from GMT. This allowsfor the automatic handling of special time zone characteristics such as daylightsavings time



User InterfaceThe default Hardware Management Console user interface style is Drag and Drop.This interface involves using the mouse to pick up and drag one or more objectsand dropping them on a task. An alternate tree-style user interface provides drilldown and Launch in context capabilities. For additional information about changesto the functional features, see the System z Hardware Management ConsoleOperations Guide, SC28-6840.

User AuthorityUser authority for the new Hardware Management Console is determined by a userrole that is associated with tasks. Each user role is actually a collection ofauthorizations. The new HMC allows additional user roles to be defined to meetyour needs.

There are two types of user roles, task roles and managed resource roles. Thefollowing table compares the default task roles on a z9 EC HMC to the former usermodes on a zSeries HMC:

Table 9-1. Comparing z9 EC Task Roles to User Modes

z9 EC Task Roles User Modes

Operator tasks Operator

Advanced Operator tasks Advanced operator

System Programmer tasks System Programmer

Access Administrator tasks Access Administrator

Service Representative tasks Service Representative

Universal Director/Timer tasks Operator, Advanced Operator, SystemProgrammer, Access Administrator, andService Representative

Access Administrator Director/Timer tasks Access Administrator and ServiceRepresentative

Universal Fiber Saver tasks Operator, Advanced Operator, SystemProgrammer, Access Administrator, andService Representative

Access Administrator Fiber Saver tasks Access Administrator

Service Fiber Saver tasks Service Representative

The following table shows a comparison of the z9 EC HMC default ManagedResource roles to the former user modes on the zSeries HMC:

Table 9-2. Comparing z9 EC Managed Resource Roles to User Modes

z9 EC Managed Resource Roles User Modes

Limited managed objects Operator, Advanced Operator, SystemProgrammer, Access Administrator, andService Representative

Defined Director/Timer managed objects Operator, Advanced Operator, SystemProgrammer, Access Administrator, andService Representative



Table 9-2. Comparing z9 EC Managed Resource Roles to User Modes (continued)

z9 EC Managed Resource Roles User Modes

Defined Fiber Saver managed objects Operator, Advanced Operator, SystemProgrammer, Access Administrator, andService Representative

All managed objects Access administrator and ServiceRepresentative

All Director/Timer managed objects Access administrator and ServiceRepresentative

All Fiber Saver managed objects Access administrator and ServiceRepresentative

These tables show how a user can now be associated with multiple user roles. Themanagement of user roles is performed using the Customize User Control ConsoleActions task.

Security ConsiderationsBecause multiple Hardware Management Consoles and internal SEs requireconnection through a LAN, it is important to understand the use and capabilitiesenabled for each Hardware Management Console.

Hardware Management Consoles operate as peers with equal access to the CPCsconfigured to them. The SE for each CPC serializes command requests fromHardware Management Console Applications on a first come, first served basis.There is no guarantee of exclusive control across a sequence of commands sentfrom a single Hardware Management Console.

You should consider these security recommendations:

v Following installation of the CPC(s), Hardware Management Console(s), andSE(s) in your configuration, the access administrator should change the defaultlogon passwords at the Hardware Management Console(s) and SE(s).

v Create a private LAN to interconnect the HMCs with the controlled SEs.

Using a private LAN for your configuration offers several security, availability, andperformance advantages as follows:

– Direct access to the LAN is limited to the Hardware Management Console(s),SE(s), CPC(s), and control unit(s) attached to it. Outsiders cannot connect toit.

– Traffic disruption due to temporary outages on the LAN is reduced, includingdisruptions caused by plugging in and powering on new devices on the LAN(minor) to LAN adapters being run at the wrong speed (catastrophic).

– LAN traffic is minimized reducing the possibility of delays at the HardwareManagement Console/SE user interface.

v Connect the HMCs to the enterprise LAN using the second LAN adapter in theHMC.

v Assign a unique domain name that includes all the CPCs controlled from one ormore Hardware Management Consoles.

v Install one or more Hardware Management Consoles that have all of the CPCsyou want to control defined to it.

Place at least one of these Hardware Management Consoles in the machineroom near the CPCs that form its domain.



Use the following enable/disable controls to help you control access and providefocal point capabilities:– Licensed Internal Code (LIC) update (change management focal point)– Remote service support– Remote customer access– Remote service access– Auto-answer of the modem.

v Physically secure the Hardware Management Console (keep it in a locked room).

v If a remote console is used for remote operations access, assign a secure logonpassword.

v Logoff each Hardware Management Console when it is not in use. The HardwareManagement Console provides a status bar capable of displaying status colors(or grey patterns) to alert you when operator activity is needed, even when noone is logged on.

v Establish a limited list of objects and actions available to the operator.

Change Management ConsiderationsAll Hardware Management Consoles are shipped with change managementenabled. If you want to limit the number of Hardware Management Consoles thathave change management capability such as LIC update control, I/O definition andremote IOCDS management capability using HCD, enable only those HMCs to beused as change management consoles. A least one Hardware ManagementConsole in the domain must be enabled for change management.

I/O ConfigurationWhen installing z9 EC model CPCs, you must define the I/O configuration to thechannel subsystem (CSS) of the CPC. The CSS for a CPC controls thecommunication between the CPC (or the logical partitions on the CPC), itschannels, and any control units and devices attached to the channels.

To do this, you must create the source necessary to build an input/outputconfiguration data set (IOCDS) that contains the appropriate I/O configurationinformation for the CPC.

Recommended ApproachIf you are upgrading an existing CPC, you may be able to write an IOCDS to yourcurrent CPC in preparation for the upgrade using the batch version of IOCP or byusing HCD. The batch version of IOCP is available on the following operatingsystems: Linux, z/OS, z/VM, z/VSE, and VSE/ESA. The Hardware ConfigurationDefinition (HCD) tool is available on z/OS, and z/VM. Refer to the System zInput/Output Configuration Program User’s Guide for ICP IOCP to determine whichCPCs can write an IOCDS in preparation for an upgrade and which CPCs can havean IOCDS written to them by IOCP. If you can write an IOCDS to your current CPCin preparation for upgrading it to a new CPC, do so and inform the servicerepresentative which IOCDS should be used.

If you are not upgrading or cannot write an IOCDS in preparation for the upgrade,stand-alone IOCP must be used to define the initial IOCDS for your CPC. It is yourresponsibility to supply the service representative who installs the CPC with an I/OConfiguration Source (IOCS) file, consisting of IOCP card-image input statements.HCD users should use HCD to define the I/O configuration and then build the IOCPinput data set (i.e. the IOCS). The IOCS contains specific data about the I/Oconfiguration. An IOCS file must contain ASCII data. It must either be in DOS file



format or have been compressed (e.g. using a zip compatible program). If the file iscompressed, it must contain a single IOCS file. The IOCS file name must have amaximum of eight characters and the file extension must have a maximum of threecharacters. Either give the service representative a diskette containing the IOCS fileor, if the Hardware Management Console is not being upgraded, copy the IOCS fileto the Hardware Management Console hard drive and inform the representative ofits location. Refer to the System z9 Stand-Alone Input/Output ConfigurationProgram User’s Guide, for more information.

Coupling Facility Continuous AvailabilityYou can use z/OS HCD running on z9 EC model CPCs to remotely write an IOCDSto z9 EC, coupling facility model, to a 9674 coupling facility, to another z990 model,to a z900 or z800 server, to an S/390 G6, G5, G4, or G3 Enterprise Server or to anIBM Enterprise Server CPC that is running a coupling facility logical partition aslong as both CPCs are configured to the same Hardware Management Consolewith change management capability.

Note: The following information applies to a Coupling Facility-only model or a CPCwith only ICF PUs.

The IOCDS can then be used during activation of the CPC serving as a couplingfacility. However, you cannot use HCD to dynamically add or delete channel pathsdefined in the coupling facility channel configuration.

This can be a problem if you want to add CPCs or logical partitions to a ParallelSysplex configuration to increase its processing capacity. In this case, you need todefine and, possibly install, additional coupling links (ISC-3 or ICB) to connect fromthe coupling facility to the newly added CPCs or LPARs.

Defining additional coupling links to the coupling facility I/O configuration requiresyou to update or create a new IOCDS to include the additional coupling linkdefinitions, deactivate the coupling facility, then re-POR and reactivate the couplingfacility using the updated or new IOCDS.

In either case, you could cause a partial or complete disruption of data sharingcapacity characteristics of the coupling facilities in the Parallel Sysplexconfiguration. To avoid this situation and to enhance the continuous availability ofthe Parallel Sysplex configuration, you should consider doing the following:

v Pre-define a maximum number of coupling links in the coupling facility IOCDSwhether they are installed or not. The extra links that you configured will show upas non-operational. (This is not a practical choice unless you know previouslywhere the new hardware will be plugged into your machine.)

This will eliminate the need to create or modify an IOCDS and re-POR thecoupling facility to add the coupling links to the I/O configuration when additionalCPCs are added to the Parallel Sysplex configuration. However, you must knowif the new links will be for peer (e.g. CFP) or for compatibility mode (e.g. CFR).

Non-Dedicated Coupling Facility Configurations: If the coupling facility is notthe only logical partition running on the CPC you can use dynamic I/O from yournon-CF (z/OS or z/VM) LPARs to add the new coupling hardware to theconfiguration definition and CFCC configuration commands to vary the newhardware into the CF logical partition.



Remote Operations and Remote AccessRemote Operations provides the ability to monitor or control a system, or group ofsystems, from a central or remote location. Remote capability creates a powerfultool for problem determination and diagnosis and operations assistance. Remoteoperations can save time and money while increasing the productivity of supportstaff. Technical expertise can be centralized, reducing the need for highly skilledpersonnel at remote locations.

Remote operations become increasingly important as:

v Data center operations and staff consolidate, with operations centers separatefrom those data centers.

v Companies and their DP staffs merge.

v World-wide operations become more common.

When considering remote operation of your z9 EC processor, there are two optionsavailable. You can choose one or both, based on your needs and configuration.

The first set of options deal with manual interaction and provide various methods ofallowing a person to interact with the user interface. Manual control allows anoperator to monitor and control the hardware components of the system using ahardware management console or a web browser.

A second set of options deal with machine interaction and provide methods ofallowing a computer to interact with the consoles through an Application ProgramInterface (API). These Automated interfaces allow a program to monitor and controlthe hardware components of the system. The automated interfaces are used byvarious automated products, including those from IBM and other vendors of otherSystem Management products.



Remote Manual OperationsRemote manual operations use the same Graphical User Interface (GUI) used by alocal HMC operator. There are two ways to perform remote manual operations, by:

v Using a remote HMC

v Using a web browser to connect to a local HMC.

The choice between a remote HMC and a web browser connected to a local HMCis determined by the scope of control needed. A remote HMC defines a specific setof managed objects that will be directly controlled by the remote HMC, while a webbrowser to a local HMC controls the same set of managed objects as the localHMC. An additional consideration is communications connectivity and speed. LANconnectivity provides acceptable communications for either a remote HMC or webbrowser control of a local HMC but dialup connectivity is only acceptable foroccasional web browser control.

Using a Hardware Management ConsoleA remote HMC gives the most complete set of functions because it is a completeHardware Management Console; only the connection configuration is different froma local Hardware Management Console. As a complete HMC, it requires the samesetup and maintenance as other HMCs. A remote HMC needs LAN TCP/IPconnectivity to each Support Element to be managed. Therefore, any existing

CorporateNetwork

Public SwitchedNetwork

BackupOperations

PrimaryOperations Monitor

Remote ControlProgram ProductBrowser

HardwareManagementConsole Browser

to IBMto IBMto IBM

Hardware Management Console

DataCenter

1

Br

Local

SubnetLoca

l

SubnetLoca

l

Subnet

idge Bridge Bridge

DataCenter

2

DataCenter

nIBM

server

IBM

server

IBM

server

Figure 9-2. Remote Operation Example Configuration



customer installed firewall between the remote HMC and its managed objects mustpermit communications between the HMC and SE. The remote HMC also requiresconnectivity to IBM or another HMC with connectivity to IBM for service andsupport.

Using a Web BrowserEach HMC contains a web server that can be configured to allow remote access fora specified set of users. When properly configured, an HMC can provide a remoteuser with access to all the functions of a local HMC except those that requirephysical access to the diskette or DVD media. The user interface on the remoteHMC is the same as the local HMC and has the same constraints as the localHMC.

The web browser can be connected to the local HMC using either a LAN TCP/IPconnection or a switched, dial, network PPP TCP/IP connection. Both typesconnections can only use encrypted (HTTPS) protocols, as configured in the localHMC. If a PPP connection is used, the PPP password must be configured in thelocal HMC and in the remote browser system. Logon security for a web browser isprovided by the local HMC user logon procedures. Certificates for securecommunications are provided, and can be changed by the user.

Browser level is the responsibility of the customer and browser service or supportand maintenance does not require connectivity to IBM.

Refer to the System z Hardware Management Console Operations Guide , SC24-6840 for Web Browser requirements and information on getting ready to configureand use the Web Server and things to consider during your Web session.

Remote Automated OperationsAs an alternative to manual operations, z9 EC models allow a computer to interactwith the consoles through a programmable interface, or API. The automatedinterface allows a program to monitor and control the hardware components of thesystem in the same way a human can monitor and control the system. The HMCAPIs provide monitoring and control functions through TCP/IP SNMP to an HMC.These APIs provide the ability to get and set a managed object’s attributes, issuecommands, receive asynchronous notifications, and generate SNMP traps. Foradditional information about APIs, see the System z Application ProgrammingInterfaces.

The automated interfaces are used by various automation products, including Tivoli®

System Automation for z/OS - Processor Operations.



Chapter 10. Reliability, Availability, and Serviceability (RAS)

z9 EC models reduce downtime by using standard features that provide high levelsof Reliability, Availability, and Serviceability (RAS).

ReliabilityThe features that provide a high level of reliability include the following:

v High-reliability technology components

v Parts integration to reduce the number of parts in the machine

v MCM System Run-In Test in manufacturing to reduce very early life fails.

AvailabilityThe features that provide a high level of availability include those that are discussedin this section.

Enhanced Book AvailabilityThe z9 EC is designed to allow a single book, in a multi-book server, to beconcurrently removed from the server and reinstalled during an upgrade or repairaction. Enhanced book availability or Concurrent Book Replacement, exclusive tothe z9 EC, is an extension of the support for Concurrent Book Add (CBA).

Redundant I/O InterconnectWhen a single book in a multi-book server is removed from the server, RedundantI/O Interconnect, exclusive to z9 EC, provides connectivity to the server I/Oresources using a second path from a different book.

Enhanced Driver MaintenanceLicensed Internal Code (LIC) updates contribute to downtime during plannedoutages. The z9 EC can be configured to permit planned LIC updates to the serverat specified driver sync points; that is, points in the maintenance process when LICmay be applied concurrently. A limited number of sync points exist throughout thelife of the LIC level and once a sync point has passed, the LIC can no longer beapplied concurrently. Availability is improved by taking advantage of these syncpoints.

Dynamic Oscillator SwitchoverThe new z9 EC is designed with two oscillator cards, a primary and a backup. Inthe event the primary card fails, the backup is designed to detect the failure, switchover, and provide the clock signal to the server transparently. Dynamic oscillatorswitchover is exclusive to the z9 EC.

Program Directed Re-IPLProgram Directed Re-IPL is designed to allow Linux on System z9 to re-IPL withoutoperator intervention. Linux can identify how it was IPL’d from the load device.Program directed re-IPL uses LIC to request a reload, using the same loadparameters, from the same load device. Program directed re-IPL allows a Linuxsystem running natively in an LPAR to execute a re-IPL. This function is exclusiveto the z9 EC and Linux on System z9.



Concurrent MBA Fanout Card Hot-PlugThe MBA fanout cards reside on the front of a book in the processor cage. In theevent of an outage, an MBA fanout card, used for I/O, can be concurrently repairedusing redundant I/O interconnect. The MBA fanout card is exclusive to the z9 EC.

Processor Unit (PU) SparingTwo spare PUs are provided per server to maintain performance levels should anactive CP, ICF feature, IFL feature, zAAP, or System Assist Processor (SAP) fail ona z9 EC model.

v Cross-book PU sparing.

Transparent sparing for failed processors is supported for z9 EC. There are twospare PUs per system and sparing is supported across the books in the unlikelyevent that the book with the failure has no spares available.

v Transparent CP/ICF/IFL/zAAP sparing.

CP/ICF/IFL/zAAP sparing is transparent in all modes of operation and requiresno operator or operating system intervention to invoke a spare PU. It is effectiveon all models including uniprocessor models.

With transparent sparing, the application that was running on the failed PU willbe preserved and will continue processing on a new PU with no customerintervention required. Refer to “Application Preservation” on page 10-3 forsituations where no spare processors are available.

v Dynamic SAP Sparing / Reassignment.

Dynamic recovery is provided for failure of the System Assist Processor (SAP). Inthe event of a SAP failure, if a spare Processor Unit (PU) is available, in mostcases the spare PU will be dynamically activated as a new SAP. If there is nospare PU available, and the CPC has more than one Central Processor (CP), anactive CP will be reassigned as a SAP. In either case, there is no customerintervention required. This capability eliminates an unplanned outage and permitsa service action, if necessary, to be deferred to a more convenient time.

Processor DesignEach Processor Unit (PU) contains dual instruction and execution units that areused to verify internal operation and which operate simultaneously. Results arecompared, and in the event of a miscompare, Instruction Retry is invoked. Thisdesign simplifies checking, and virtually eliminates PU failures due to soft errors.

Support Element (SE)The z9 EC has two Support Elements (SEs). In the event the primary SE fails,switchover to the alternate is usually handled automatically.

Hardware Management ConsoleOne Hardware Management Console is required for system monitoring andoperation of the CPC(s) configured to it. For high availability applications, it isrecommended that you have at least two Hardware Management Consoles for yourconfiguration to guarantee that the Hardware Management Console functions areavailable when needed.

The Hardware Management Console is concurrently maintainable with the operationof the CPCs configured to it. Having more than one Hardware ManagementConsole provides continuous availability of Hardware Management Consolefunctions, including the following:



v Hardware operator controls, hardware status monitoring, and hardware andoperating system messages for all configured CPCs

v Capability to call for service

v Remote operations control

v Problem Analysis.

Attaching to IBM Service through the InternetThe z9 EC provides the ability to connect to IBM Service using the Internet. TheSSL connection is made from the HMC through the corporate network and firewallto IBM Service using the Internet. This is an outbound connection only.

Hardware Management Console Monitor System EventsThe Hardware Management Console Monitor System Events is available on z9 ECmodels. The Message and State Monitor facilities of the HMC can be enabled tosend e-mail notification to a specified recipient whenever a specific message isreceived from either the hardware subsystem or an operating system, or when aCPC (hardware object) or a CPC Image (Operating system object) changes fromone ’state’ to another ’state’. The state of an object represents its current conditionand functional capability, so a state change represents a change in functionalcapability that may require attention. Hardware and operating system messages areintended to keep the operator informed of conditions that may require attention.However, not all messages and not all state changes are important; only specificones require attention and notification of a responsible person.

Two SAP StandardAll z9 EC models are provided with two SAPs per book for I/O processing. ModelS08 has two, model S18 has four, model S28 has six, and models S38 and S54have eight.

Application PreservationApplication Preservation is used in the case where a CP fails and there are nospares. The state of the failing CP is passed to another active CP where theoperating system uses it to successfully resume the task in most cases withoutcustomer intervention.

Dynamic ICF ExpansionDynamic ICF Expansion is a function that allows an ICF logical partition withdedicated ICF processors to acquire additional processing power from the logicalpartition pool of shared CPs or shared ICFs being used to execute productionand/or test work on the system. This function is very useful when the CF logicalpartition in some general purpose models backs up another CF logical partition. Inthe event of a failure and backup to this CF logical partition, the backup CF logicalpartition, using Dynamic ICF Expansion, can acquire additional processing capacityto handle the full CF logical partition workload. Also, Dynamic ICF Expansion canbe used to handle a peak workload situation when the CF logical partition is beingused as the primary coupling facility. This feature adds considerable flexibility to theconfigurability of the CF logical partitions in general purpose models and optimizesthe use of the processing power in the system.

The Dynamic ICF Expansion feature is available to every CF logical partition withdedicated ICF processors using coupling facility links. Each ICF coupling facilitypartition has its own ability to specify the number of ICF features that are dedicatedto that partition and the amount of additional capability it can acquire. The trade offbetween using ICF features and the CPs in the logical partition shared pool is the


Chapter 10. Reliability, Availability, and Serviceability (RAS) 10-3

exemption from software license fees. Dynamic ICF Expansion is available ongeneral purpose models that have at least one ICF.

Dynamic Coupling Facility DispatchingThe dynamic Coupling Facility (CF) dispatching function helps enable continuouscomputing in the event of a coupling facility failure without requiring a standalonebackup coupling facility. Enhanced dispatching algorithms enable you to define abackup coupling facility in a logical partition on your system. While this logicalpartition is in backup mode, although it is sharing resources with other logicalpartitions running other active workload, it uses very little processor resource. Whenthe backup CF becomes active, only the resource necessary to provide coupling isallocated.

Error Correction Code (ECC)Memory error checking and correction code detects and corrects single bit errors.Also, because of the memory structure design, errors due to a single memory chipfailure are corrected.

Memory SparingThe z9 EC does not contain spare memory DIMMs. Instead it has redundantmemory distributed throughout its operational memory and these are used tobypass failing memory. Replacing memory cards may require power-off of the bookwhich is disruptive. The extensive use of redundant elements in the operationalmemory greatly minimizes the possibility of a failure that requires memory cardreplacement.

Memory ScrubbingStorage background scrubbing provides continuous monitoring of storage for thecorrection of detected faults before the storage is used.

LPAR Dynamic Storage ReconfigurationPR/SM LPAR storage reconfigurations can occur allowing nondisruptive add orremoval to any partition with a cooperating guest. This capability removes therestriction of storage reconfigurations only being possible from an adjacent andabove logical partition.

Subsystem Storage ProtectSubsystem storage protection and subspace group facility support, for use withCICS/ESA, prevents application software from overwriting CICS system software,control blocks, and address spaces.

Partial Memory RestartIn the event of a memory card failure, the system can be restarted with reducedmemory capacity. Processing can be resumed until replacement memory isinstalled.

Dynamic I/O ConfigurationDynamic I/O configuration enhances system availability by supporting the dynamicaddition, removal, or modification of channel paths, control units, I/O devices, andI/O configuration definitions to both hardware and software without requiring aplanned outage.



Note: Dynamic I/O Configuration is not available on a model with only ICFs orIFLs.

ESCON Port SparingESCON 16-port I/O card includes one unused port dedicated for sparing in theevent of a port failure on that card. Other unused ports are available for growth ofESCON channels without requiring new hardware.

FICON Cascaded DirectorsFICON cascaded directors allow a native FICON (FC) channel or a FICONChannel-to-Channel (CTC) to connect a server to a device or other server with twonative FICON directors between them. This is only for a two-switch configuration.

FCP Full-fabric ConnectivityThe FCP full-fabric connectivity supports multiple numbers of directors/switches thatcan be placed between the server and the FCP/SCSI device, thereby allowingmany hops through a storage network for I/O connectivity.

Maintenance/Upgrade for CouplingThe z9 EC provides concurrent maintenance for the ISC-3 adapter card. Also,ISC-3 coupling links may now be added concurrently. This eliminates the need forscheduled downtime in the demanding sysplex environment.

Concurrent Channel UpgradeIt is possible to concurrently add ESCON, FICON, ISC-3, and OSA channels to anI/O cage provided there are unused channel positions in the I/O cage. In addition,ICBs and their associated STI cables can be added provided there are unusedcable jack positions. This capability may help eliminate an outage to upgrade thechannel configuration.

Dual Power FeedsThe power system offers dual primary (AC) power feeds. Each feed is electricallyisolated and enables redundant power paths to each server. To take full advantageof the redundant power paths built into the z9 EC, it is necessary to provide dualelectrical service to the server to minimize any outage due to a single-path powerinterruption.

Redundant Power and Thermal SubsystemsThe DC power and thermal subsystems are designed with N +1 redundancy. Failureof a power or thermal component does not cause a system outage.

Redundant Flexible Service ProcessorsThe z9 EC Power and Service Control Network features redundant cage controllersfor logic and power control. This design enables non-disruptive service to thecontrollers and eliminates customer scheduled outage.

Oscillator (OSC) and External Timer Reference (ETR) cardsThere are two standard OSC cards and two optional ETR cards on z9 EC. An ETRcard failure will automatically switch to the other ETR card; the effectiveness of thisaction depends on the proper planning of the external ETR connections. Dynamicoscillator switchover is designed to detect the failure of the primary oscillator card



and switch over to the backup, providing the clock signal to the server transparently.Dynamic oscillator switchover is exclusive to the z9 EC.

Preferred Time Server and Backup Time ServerIn an STP-only CTN configuration, it is required that at least one server is definedas the Preferred Time Server. It is also required that a Current Time Server isassigned. The Current Time Server (the Stratum 1 server) has the highest level ofhierarchy in the STP-only CTN and has connectivity to the servers designated asStratum 2 servers. If only a Preferred Time Server is defined in an STP-only CTN, itis assigned as the Current Time Server.

If there is a failure in the Preferred Time Server, synchronization fails if a backupconfiguration is not established. Therefore, it is highly recommended that a BackupTime Server is defined in an STP-only CTN. The Backup Time Server is normally aStratum 2 server that has connectivity to the Preferred Time Server, as well as to allother Stratum 2 servers that are connected to the Preferred Time Server. Byproviding this connectivity, the Backup Server can easily take over as the CurrentTime Server if there is a failure with the Preferred Time Server or if areconfiguration is planned. Therefore, the servers in the STP-only CTN canmaintain synchronization.

Additionally, when the external time source for the STP-only CTN is configured touse NTP (with or without PPS), having the ETS configured on the Backup TimeServer using different NTP server(s) provides continuous availability of NTP servers.In the event that the Preferred Time Server cannot access its configured NTPserver(s), adjustments can be made using information from the Backup TimeServer. This is achieved without reconfiguring the Backup Time Server as theCurrent Time Server.

Concurrent Hardware MaintenanceConcurrent maintenance enables the replacement of failed units concurrently withsystem operation. This enhances the processor availability by eliminating the needfor system outage to effect the repair. Concurrent maintenance capability exists forthe following elements:v Powerv Thermalv ESCON cardsv FICON Express cardsv OSA feature cardsv Crypto Express2 featurev ISC-3 feature cardv ICB (ICB-4, and ICB-3) STI cablev MBA fanout cardv HCA1-O fanout cardv OSC cardv ETR cardv Hardware Management Consolev Support Element.

Concurrent Licensed Internal Code (LIC) PatchConcurrent Code Patch allows the activation of a patch concurrent with systemoperation thereby increasing the availability of the processor by reducing scheduledoutage for LIC maintenance. This capability exists for code for the followingelements:



|||||||

v CPv SAPv Cage Controllersv LPv CFCCv Powerv Thermalv ESCON channelsv FICON channelsv FCP channelsv OSA channelsv ISC-3 linksv ICB linksv IFB linksv IC linksv HiperSocketsv Hardware Management Consolev Support Element.

Notes:

1. OSA-Express channels always require CHPID vary off/vary on cycle to activateLIC patches.

2. Concurrent patch support is available for OSA-Express2 features configured forOSD.

Electronic Service Agent (Service Director)Electronic Service Agent™ (Service Director™) will have I/O error data collected by acomponent integral to the operating system, forwarded from the operating systemthrough a z9 EC Hardware Management Console, and then to an eService serverin IBM. Electronic Service Agent provides the analysis of the data and providesvarious users access to the data through a web browser interface.

Internal Battery Feature (IBF)The Internal Battery Feature (IBF) provides backup input power. The feature ispackaged internal to the machine. It can be used with a UPS to provide additionalprotection.

Redundant Coupling LinksRedundant Coupling Links (ISC-3s from different ISC-M cards, ICBs from differentMBA fanout or STI-3 extender cards, and IFBs from different HCA fanout cards) canbe configured between a processor and the Coupling Facility. This potentiallyremoves a single point of failure for the processor’s data sharing capability in theParallel Sysplex environment.

System-Managed CF Structure Duplexing (CF Duplexing)A set of architectural extensions to the Parallel Sysplex is provided for the supportof System Managed CF Structure Duplexing (CF duplexing) of coupling facilitystructures for high availability.

Installing this software and microcode, and enabling this function is designed to:

v Provide the necessary base for highly available coupling facility structure datathrough the redundancy of duplexing.



v Enhance Parallel Sysplex ease of use by reducing the complexity of CF structurerecover.

v Enable some installations to eliminate the requirement for standalone CFs in theirParallel Sysplex configuration.

On/Off Capacity on Demand (On/Off CoD)When your business needs short term additional capacity, On/Off Capacity onDemand (On/Off CoD) is designed to deliver it. On/Off CoD is designed totemporarily turn on Central Processors (CPs), Integrated Facilities for Linux (IFLs),Internal Coupling Facilities (ICFs), and/or System z9 Application Assist Processors(zAAPs) that are available within the current model. This capability can coexist withCapacity Backup Upgrade (CBU) enablement.

On/Off CoD is delivered through the function of Customer Initiated Upgrade (CIU).To participate in this offering, you must have installed CIU Enablement (FC 9898),and On/Off CoD Enablement (FC 9896). Subsequently, you may concurrently installtemporary capacity by ordering On/Off CoD Active CP (FC 9897) up to the numberof current CPs (FC 0716), On/Off CoD Active IFL (FC 9888) up to the number ofcurrent IFLs (FC 0516), and On/Off CoD Active ICF (FC 9889) up to the number ofcurrent ICFs (FC 0518).

You may similarly concurrently install temporary capacity by ordering On/Off CoDActive zAAP (FC 9893) up to the number of current zAAPs (FC 0520), with therestriction that the total number of On/Off CoD Active zAAPs (FC 9893) plus zAAPs(FC 0520) may not exceed the number of On/Off CoD Active CPs (FC 9897) plusthe number of CPs (FC 0716) plus the number of unassigned CPs (FC 1716). Inaddition, the number of On/Off CoD Active zAAPs (9893) may not exceed thecurrent number of zAAPs (FC 0520) that are permanently purchased.

On/Off Capacity on Demand (On/Off CoD) EnhancementsThe On/Off CoD enhancements for z9 EC are:

v Removal of Temporary Capacity - Currently designed to remove temporarycapacity when ordering a different capacity setting, the z9 EC now supportsordering a different capacity setting and switching to the different setting bydownloading the second configuration while the previous setting is active.

v API Used to Activate - The z9 EC is now designed to permit activation of On/OffCoD with the input parameter of an order number using an HMC SNMP API foractivation.

v No-Charge Test - The On/Off Capacity on Demand (On/Off CoD) test can beused to validate the processes to download, activate, and deactivate On/Off CoDcapacity non disruptively. Each On/Off CoD-enabled server is entitled to oneno-charge test. The test may run for a maximum duration of 24 hours beginningwith the download and activation of an On/Off CoD order. In addition to validatingthe On/Off CoD function, you can use this test as a training session for personnelwho are authorized to activate On/Off CoD.

v Storing records in an unactivated state- It is possible to store up to 100On/Off CoD LICCC records in an unactivated state on the Support Element atany given time. This provides greater flexibility to quickly enable neededtemporary capacity. The billing begins upon activation.

v One hour grace period to the 24 hour billing period - You have up to onehour before the next billing period starts to increase the On/Off CoD configurationfor the next 24-hour billing period without affecting the billing of the current24-hour period. You also have up to one hour after the current 24-hour billing



period to either decrease the On/Off CoD configuration for the next 24-hourbilling period or deactivate the current On/Off CoD configuration without affectingthe billing of the next 24-hour period.

Capacity Upgrade on Demand (CUoD)Capacity Upgrade on Demand provides the capability to add CPs, ICFs, IFLs,zAAPs, memory, and channels non-disruptively, eliminating the need for ascheduled outage. Installations who take advantage of the CUoD option may invokethe additional capacity non-disruptively. Refer to “Capacity Upgrade on Demand(CUoD)” on page C-7 for more information.

Capacity Backup (CBU)The z9 EC Capacity BackUp capability (temporary upgrade) enables enterprises toprovide flexible, cost-effective Disaster Recovery on z9 EC models. Refer to“Capacity Backup (CBU)” on page C-4 for more information.

Geographically Dispersed Parallel Sysplex (GDPS)GDPS is a multisite solution that is designed to provide the capability to managethe remote copy configuration and storage subsystems, automate Parallel Sysplexoperational tasks, and perform failure recovery from a single point of control,thereby helping to improve application availability.

GDPS/PPRC Hyperswap FunctionThis function is designed to broaden the continuous availability attributes ofGDPS/PPRC by extending the Parallel Sysplex redundancy to disk subsystems.

CBU Smart ReactivationCBU Smart Reactivation reduces outage duration, during a disaster event, if aprocessor book (containing the MCM) needs to be changed (while CBU isactivated).

Concurrent Undo CBUA prerequisite to executing this feature is that the customer or z/OS applicationmust configure offline the processors that are being removed. So the best rule tofollow is, ″Configure offline the same logical processors that were configured onlinefollowing the CBU activation″. The Concurrent Undo CBU will require the followingactions to configure the processors offline based on how it will be invoked:v GDPS invoked Hardware Management Console/SE API.v Customer program invoked Hardware Management Console/SE API.

Notes:

1. As the user (or z/OS automation) deconfigures logical CPs, there is noguarantee that the logical CPs will remain in sequential numbering.

2. The SE panel will give no directions as to which CPs, ICFs, or IFLs to configureoffline.

Fiber Optic CablingTo serve the cabling needs of System z customers, IBM Site and Facilities Serviceshas fiber optic cabling services available whether the requirements are product-levelor enterprise-level. These services take into consideration the requirements for all ofthe protocols and media types supported on the System z10 EC, System z9, and



zSeries (for example, ESCON, FICON, Coupling Links, OSA) whether the focus isthe data center, the Storage Area Network (SAN), the Local Area Network (LAN), orthe end-to-end enterprise.

CHPID Mapping ToolThis tool provides a convenient interface to map hardware I/O ports on order toyour CHPID definitions. An availability option automatically assigns PCHIDs to yourCHPID definitions to minimize connecting critical paths to a single points of failure.This is recommended for all new z9 EC hardware builds or for upgrades from az990 processor to a z9 EC, as well as for making changes to an already installedmachine after hardware upgrades that change or increase the number of channels.

Multipath Initial Program Loadz/OS on System z9 EC allows the system to attempt an IPL on alternate paths, ifavailable, if I/O errors occur during the IPL. This function increases the availabilityby allowing IPL to complete if possible using alternate paths and eliminates theneed for manual problem determination when attempting an IPL.


System-initiated CHPID ReconfigurationThis function allows you to submit one request to all operating systems to configureoffline or configure online all the CSS.CHPIDs associated with a partciular chid. Itreduces the duration of a repair action when an ESCON or FICON channel, anOSA port, or and ISC-3 link is shared across logical partitions (LPARs).

ServiceabilityThe features that provide a high level of serviceability include the following:

v Automatic error detection and fault isolation concurrent with system operation.

v Automatic remote support capability.

v High degree of concurrent maintenance capability in hardware and code.

v Multiple Channel Swap - an enhancement for channel problem determinationallowing up to 16 channels to be swapped.

v Status Panel showing status of N+1 power system.



Appendix A. System z9 EC Version 2.9.2 Purpose andDescription

This appendix contains detailed information about Version 2.9.2 licensed internalcode.

New Function, Features, and DifferencesFor a list of new function and features available on the z9 EC, refer to “z9 ECHighlights” on page 1-2

This summary contains the operational differences between the HMC Version 2.9.2and previous zSeries and System z9 releases:

v There is a new logon procedure that does not permit password changes duringlogon.

v The Hardware Management Console (HMC) is a closed platform that ensuressystem security and integrity. It does not allow the installation of non-HMCapplications.

v Previously the HMC provided five different levels of customer level userauthorization or user modes. User modes have been replaced with user rolesand users can now exist in multiple user roles.

v The HMC now uses a single network protocol, TCP/IP, to communicate with otherservers.

v The HMC is now configured with a firewall to limit network access in and out ofthe system. By default, no external connections are allowed through the firewall.This default can be modified.

v Full function remote access is the single access that provides the fullcomplement of tasks and functions to all users, both local and remote.

v All remote browser access to an HMC must use Secure Sockets Layer (SSL)encryption. Only the local user interface can use non-encrypted access.

v Although the HMC panels are similar to prior levels, they now resemble a webinterface.

v While the main HMC user interface retains the previous version’s look, feel, andoperation, an alternate ″tree-style″ interface is provided. Access to the alternateinterface requires permission from the access administrator.

v A new Disconnect option retains the user state information for that sessionallowing the user to reconnect to that session later.

v A new workstation based tool for performing HSA estimations is available fromResource Link function which replaces the function previously available on theHMC/SE.

v Some panels require you to click ″Cancel″ when a function is completed, thenreturn to the prior panel. Using Cancel in this manner does not cancel thefunction.

v The “tree-style” interface is available on the Support Element (SE). Access to thealternate interface requires permission from the access administrator.

v Some existing tasks have been moved within, removed from, or renamed in theHMC and the Support Element.

v New tasks have been added to the HMC and the Support Element.

v z/VM integrated systems management from the HMC provides out-of-the-boxintegrated GUI-based basic management of z/VM guests.


© Copyright IBM Corp. 2005, 2009 A-1

v The following legacy systems are not supported: 9221, 967x G1-G4, Multiprise®

2003, and Multiprise 3000 MT 7060.

v The following legacy systems, upgraded to a new AROM level, are supported atthe driver level indicated: z890/z990 (Driver 55), z800/z900 (Driver 3G), and9672 G5 & G6 (Driver 26).

For a complete list of unsupported features and functions, refer to “UnsupportedFeatures/Functions” on page 1-8

Operating SystemsFor a complete list of operating systems supported by the z9 EC, refer toChapter 3, “Software Support,” on page 3-1

Preventative Service Planning (PSP) Bucket ConsiderationsUse IBM Service Link or contact your IBM representative to obtain a current copy ofthe 2094DEVICE bucket applicable to your environment. The PSP bucket containscorequisite software and hardware planning information that applies to variousoperating system environments. This includes Authorized Program Analysis Reports(APARS), Program Temporary Fixes (PTFs), and Licensed Internal Code (LIC)product patches.

Software CorequisitesSee the appropriate 2094DEVICE Preventative Service Planning (PSP) bucketssubset ID for APAR and PTF information for the System z9 EC models

Table A-1. Software Corequisites

Software PSP Bucket Subset ID

z/OS 2094ZOS

z/VM 2094ZVM

z/VSE and VSE/ESA 2094ZVSE

Engineering Change (EC) ConsiderationsVersion Code 2.9.2 for z9 EC includes the following Central Processor Complex(CPC) and Hardware Management Console (HMC) Licensed Internal Code (LIC),engineering change (EC) and Microcode Load (MCL) levels:

v CPC level: EC G40965 + MCLs

v HMC Level: EC G40969 + MCLs

To verify that the enhancements described in this document apply to your system,display the LIC EC levels running on the CPC and the HMC.

CPC EC G40965 + MCLsYou can display the LIC EC and MCL level of the system’s CPC as follows:

1. Open the Task List from the Views area

2. Open Change Management from the Task List Work Area

3. Open Groups from the Views area.

For individual CPC support elements:


A-2 z9 EC System Overview

a. Open the group that contains the CPC support elements with the systeminformation you want to display.

b. Select one or more objects.

For a group of CPC support elements:

a. Select the group of CPC support elements with the system information thatyou want to display.

4. Open System Information in the Change Management task area. TheSystem Information window displays.

5. Verify that the EC level is in this list.

HMC EC G40969 + MCLsYou can display the LIC EC and MCL level of the system’s HMC as follows:

1. Open Console Actions from the Views area

2. Open View Console Information from the Console Action Work Area. TheView Console Information window displays.

3. Verify that the EC level is in this list.

Miscellaneous Lower Level ECs Included in Version 2.9.2The following table provides miscellaneous changes included in CPC system codeEC G40965 with Hardware Management Console system code EC G40969.

Table A-2. ECs Included in Version 2.9.2

EC Number Name

H25080 Backup-DVD New Build & MES Upgrade

G40980 SUL-DVD Driver 67

H25078 Security-Log DVD

G40983 HMC/SE DIAGS CDR

J14623 SE Upgrade ″Frame Roll″ Diskette MES/FBM Only

J14624 HMC Upgrade Diskette OS/2® to Linux MES

J14654 TKE Backup-DVD New Build & MES Upgrade

J14860 SE Upgrade Data DVD MES Only

G40991 STP Enablement CDR (FC 1021)

G40676 9037 STP Enablement Code

J99696 HDD Configuration (Clip) Diskette

J99633G35054

Diskettes for Use with HMC Version 2.9.0 or higher

G40090 HWMCA Save/Restore Customizable Console Data Diskette

G42645 HWMCA PC BIOS Update CDR Version 2.9.X

Related PublicationsA complete list of related publications for the z9 EC is found in “RelatedPublications” on page xv of this book.


Appendix A. System z9 EC Version 2.9.2 Purpose and Description A-3


A-4 z9 EC System Overview

Appendix B. Resource Link

Resource Link is a customized Web-based solution that provides everything youneed to plan for, install, and maintain IBM System z9, IBM Eserver zSeries andIBM S/390 servers and associated software.

You can access Resource Link at http://www.ibm.com/servers/resourcelink.

Resource Link content areas include:

v Personal FoldersOrganize site content according to your personal interests by creating folders forshortcuts to subscriptions. Subscribing to pages or individual documents meansyou will be notified of updates and changes.

v PlanningInteractive planning provides a streamlined plan for the installation of a systemusing online planning information tailored for your system.

v EducationWeb-based multimedia education provides product courses that can be used fortraining or refreshing skills.

v LibraryProduct documentation that can be viewed, printed, or downloaded.

v ForumsForums and the additional function called work rooms are used for groupdiscussions and to collaborate with fellow product owners.

v FixesInteractive tools allow you to request, receive, and install system upgrades.

v Problem SolvingResources to help you investigate and solve specific hardware and softwareproblems.

v ServicesSupport for services such as installation, migration, networking, planning andrelocating servers, Fiber Cabling, and links to IBM software support.

v ToolsInformation about tools such as machine information, CHPID mapping, CouplingFacility structure sizer, and links to software tools.

v Customer Initiated Upgrade (CIU)A web-based application that allows you to download licensed internal code (LIC)to upgrade processors and memory. On/Off Capacity on Demand (On/Off CoD)for short-term additional capacity is also available through CIU.

Resource Link FunctionsResource Link contains the following additional functions:

v Customized Planning Aids - Prepares you for the arrival and installation of yournew System z9 hardware. To use Customized Planning Aids you need a validorder number and a Configuration Control Number (CCN), both available fromyour IBM Sales Representative.

v Customer Initiated Upgrade (CIU) - A web-based solution that enables you toorder and download licensed internal code (LIC) to upgrade processors andmemory. On/Off Capacity on Demand (On/Off CoD) for short-term additionalcapacity is also available through CIU.


© Copyright IBM Corp. 2005, 2009 B-1


v CHPID Mapping Tool - Downloadable from Resource Link, this tool allows youto map hardware I/O ports on order to your IOCP CHPID definitions. Anavailability option automatically maps hardware I/O ports to CHPIDs minimizingsingle points of failure.

v Fiber Cabling Service - Provides the cabling connectivity necessary for installingyour z9 EC processor.

v Machine Information - Provides detailed information about your System z9 orzSeries machine including information about he last time your machine calledhome, service states, and hardware status.

v HSA Estimator - The HSA Estimator is a web based tool that allows you toestimate the HSA size for specific configurations on your IBM System z9machine.

v Power Estimation Tool - Allows you to estimate the power consumption of aspecific System z9 machine model and its associated configuration.


B-2 z9 EC System Overview

Appendix C. Capacity Upgrades

The z9 EC is designed to support concurrent upgrades that provide additionalcapacity with no server outage. In most cases, with prior planning and operatingsystem support, a concurrent upgrade can also be non-disruptive, meaning with nosystem outage.

The Licensed Internal Code Configuration Control (LICCC) provides for processorupgrade with no hardware changes by enabling the activation of additional installedcapacity. Concurrent upgrades using LICCC can be done for:

v CPs, SAPs, ICFs, IFLs, zIIPs,and zAAPs - requires available unused PUs ininstalled books

v Memory - requires available capacity on installed memory cards

v Channel cards (ESCON channels and ISC-3 links) - requires available ports onchannel cards.

I/O configuration upgrades can also be concurrent by installing, nondisruptively,additional channel cards.

Planned upgrades can be done using the Customer Initiated Upgrade (CIU) functionor Capacity Upgrade on Demand (CUoD). Unplanned upgrades for emergency ordisaster/recovery situations are accomplished using Capacity BackUp (CBU) andshort-term additional capacity to handle peak processing loads is provided throughOn/Off Capacity on Demand.

Customer Initiated Upgrade (CIU) is a used to obtain additional processor ormemory capacity quickly. For more information, refer to “Customer Initiated Upgrade(CIU)”

Capacity Backup (CBU) is a non-disruptive temporary addition of CPs, IFLs, ICFs,zIIPs, and zAAPs due to an emergency situation. For more information, refer to“Capacity Backup (CBU)” on page C-4.

On/Off Capacity on Demand (On/Off CoD) is used to temporarily upgrade capacityfor CPs, IFLs, ICFs, zIIPs, and zAAPs. For more information, refer to “On/OffCapacity on Demand (On/Off CoD)” on page C-3.

Important NoteIf On/Off CoD is active, concurrent activation of CBU is not allowed and theGeographically Dispersed Parallel Sysplex (GDPS) use of CBU will fail.

Customer Initiated Upgrade (CIU)Customer Initiated Upgrade (CIU) is an application that enables you to respond tosudden increased capacity requirements by requesting a System z9 Processor Unit(PU) or memory upgrade through the web using IBM Resource Link. The upgradeis downloaded and applied it to your z9 EC server through your system’s RemoteSupport Facility (RSF).

Using the CIU Express option, an upgrade may be installed within a few hours afterorder submission.


© Copyright IBM Corp. 2005, 2009 C-1

Permanent UpgradesOrders (MESs) of PUs, CP, IFLs, ICFs, the System z9 Application Assist Processors(zAAPs), the System z9 Integrated Information Processor (zIIPs), and memory forIBM System z9 EC servers can be delivered by Licensed Internal Code ControlCode (LIC CC) using CIU.

The CIU feature, available through an IBM CIU contract, also allows you tounassign previously purchased CPs and IFLs.

CIU ProcessThe major components in the CIU process are enabling, ordering, and downloadingand activating an upgrade.

Enabling CIUBefore you can process an order for temporary capacity, you must have a signedCustomer Initiated Upgrade (CIU) agreement (FC 9898) in place. The followingsteps explain the enablement of CIU:

v Begin by ordering the CIU enablement feature (FC 9898) and signing the IBMCustomer Agreement Supplement for Customer Initiated Upgrade and On/OffCapacity on Demand. To use CIU, you need to have the following:– An IBM Registration ID– A secondary approver with an IBM Registration ID, if applicable– A Customer Number– The Machine Type and Serial Number of the purchased machine– An RSF link to IBM.

v A CIU Administrator creates a Machine Profile from the information supplied inthe Supplement.

v Working with the Sales Representative, you will establish upgrade pricing for themachine.

v CIU notifies you that the enablement feature is available.

After all the prerequisites are in place, the whole process from ordering to activationof the upgrade is performed by the customer. As a CIU customer, you must makesure that your machine is set to call home a minimum of once every fourteen (14)days.

Placing an OrderAdditional capacity is ordered through the IBM Resource Link web site. Using thisinterface, you can create, cancel, and view your CIU upgrade order. You can alsoview the history of orders placed through this interface.

v Using your IBM Registration ID, sign in to Resource Link.

v Create a CIU order for processors and/or memory.

v If a secondary approval is required, a notice is sent to the secondary approver.

v The order is staged for download.

A CIU order can be cancelled at any time during the order process. However, bydownloading the code you are accepting the charge and the option to cancel is nolonger available.

Downloading and Activating an OrderNotification is sent to you when the upgrade is available for downloading. Youdownload the code to the Support Element (SE) through the Hardware


C-2 z9 EC System Overview

Management Console using the Single Object Operations task. Billing isautomatically initiated when the memory or processor upgrade is downloaded.

The download process is described in detail in the education course UsingCustomer Initiated Upgrade and On/Off Capacity on Demand which is availablefrom the education area of Resource Link.

The CIU panel option offers you the ability to retrieve and apply upgrade dataimmediately or to retrieve it with the option to apply it later. All upgrades will beapplied concurrently unless system conditions prevent it. If the CIU upgrade cannotbe applied concurrently an informational panel will be displayed to give you theoption to continue with the disruptive upgrade or to cancel and apply the disruptiveupgrade at a later time.

Extended Staging of OrdersAll CIU-Express and On/Off CoD orders may be staged for an extended period oftime, more than 30 days, unless one of the following conditions occurs:

v Order is canceled by the customer

v Machine is no longer under warranty or IBM Maintenance Service Agreement

v Permanent PU or memory configurations are changed.

Note: Extended staging is available in most countries. You should check with yourIBM Sales Representative to determine whether any additional restrictionsapply.

On/Off Capacity on Demand (On/Off CoD)When your business needs short term additional capacity, On/Off Capacity onDemand (On/Off CoD) is designed to deliver it. On/Off CoD is designed totemporarily turn on Central Processors (CPs), Integrated Facilities for Linux (IFLs),Internal Coupling Facilities (ICFs), System z9 Integrated Information Processor(zIIPs), and System z9 Application Assist Processors (zAAPs) that are availablewithin the current model. This capability can coexist with Capacity Backup Upgrade(CBU) enablement. On/Off CoD is delivered through the function of CustomerInitiated Upgrade (CIU) and is automatically activated when downloaded. Toparticipate in this offering, you must have installed CIU Enablement (FC 9898) andOn/Off CoD Enablement (FC 9896). All engines are limited to the number ofcurrently purchased engines of the same type: CPs, ICFs, IFLs, zIIPs, and zAAPs.In addition, zAAPs have the restriction that you cannot purchase more zAAPs thanCPs.

All temporary hardware capacity will be billed on a 24-hour basis until the temporarycapacity is removed. The On/Off CoD 24 hour billing period does includes a onehour grace period. This means you have up to one hour before the next billingperiod starts to increase the On/Off CoD configuration for the next 24-hour billingperiod without affecting the billing of the current 24-hour period. You also have up toone hour after the current 24-hour billing period to either decrease the On/Off CoDconfiguration for the next 24-hour billing period or deactivate the current On/OffCoD configuration without affecting the billing of the next 24-hour period.

Each month your bill will be calculated for the sum of all orders installed within theprior month. Monitoring will occur through the server call home facility and a bill willbe generated if the hardware capacity has been enabled for any portion of acalendar month. You will continue to be billed for use of temporary capacity untilyou return the server to the original state. After concurrently returning to the original


Appendix C. Capacity Upgrades C-3

state, you may choose to activate a new On/Off CoD upgrade which can bedifferent from the previous upgrade. When you dispose of the server, or decide thatyou want to disable future temporary upgrades, you are required to remove theOn/Off CoD Enablement (FC 9896) feature.

For System z9 server, you can also store up to 100 On/Off CoD LICCC records inan unactivated state on the Support Element at any given time. This providesgreater flexibility to quickly enable needed temporary capacity. Billing would beginupon activation of a record.


On/Off Capacity on Demand (On/Off CoD) EnhancementsThe On/Off CoD enhancements for z9 EC are:

v Removal of Temporary Capacity - Currently designed to remove temporarycapacity when ordering a different capacity setting, the z9 EC now supportsordering a different capacity setting and switching to the different setting bydownloading the second configuration while the previous setting is active.

v API Used to Activate - The z9 EC is designed to permit activation of On/OffCoD wit hthe input parameter of an order number using an HMC SNMP API foractivation.

v No-Charge Test - The On/Off CoD test can be used to validate the processes todownload, activate, and deactivate On/Off CoD capacity non-disruptively. EachOn/Off CoD-enabled server is entitled to one no-charge test. The test may run fora maximum duration of 24 hours beginning with the download and activation ofan On/Off CoD order. In addition to validating the On/Off CoD function, you canuse this test as a training session for personnel who are authorized to activateOn/Off CoD.

Administrative On/Off CoD TestThis is a no-charge test tha can be used to train personnel and validate theprocesses used for ordering, downloading, activating, and deactivating On/Off CoDupgrades. The test does not actually activate capacity. Tests can be downloaded atany time, and there is no limit to the amount of time the test can remain active. Inaddition, there are no restrictions on the number of On/Off CoD administrative testsused.

Capacity Backup (CBU)The z9 EC Capacity Backup (CBU) capability is the non-disruptive temporaryaddition of Central Processors (CPs), IFLs, ICFs, and zAAPs on servers with theCBU feature installed. It enables enterprises to provide flexible, cost-effectiveDisaster Recovery on z9 EC models.

A Special Bid Contract for CBU must be approved before the CBU features can beordered. The feature identifies how many CPs and types of PUs are in the CapacityBackup to IBM’s vital product data base for the customer system.



This contract is required and the model must be configured for the system whenadditional capacity is invoked. I/O cage channel requirements must be planned inadvance and should already be in the configuration prior to the test or CBU beinginvoked.

Capacity Backup is done concurrently with system operations and performs thefollowing:

v Activating Capacity Back-Up using Password Panel

There are five, 10-day tests allowed per CBU feature record and a REAL (90day) CBU upgrade.

If not already done, the CBU feature must be added by using the Add Capacity(CBU) feature task. During activation, an access panel displays for authorizationto activate the CBU feature. You must contact IBM for the correct passwordbased on the Processor Unit (PU) data displayed. There are five 10 ten day testsallowed per CBU feature prior to executing a REAL CBU upgrade. After 3unsuccessful attempts to enter the password, the CBU feature will no longer beavailable to the system. You must contact IBM for a new CBU record and itspassword. After a CBU record has been installed, the same record cannot bereinstalled.

v CBU Automatic Activation using the IBM Service Support System

z9 EC will call home by using RSF, verify authorization, and automatically unlockthe increased server capacity.

v Viewing CBU Feature Information Panel

A View CBU Feature Information panel is available which displays the followinginformation/status:– If CBU is installed– If CBU is Active– The date CBU was activated– The date CBU will expire– How many CBU Tests are remaining– Whether the activated CBU is a Test CBU or Temporary Upgrade CBU– If the system is enabled for CBU and the IBM Service Support option– The additional resources for installing the CBU feature.

v Automatic Enablement of CBU for the GDPS

GDPS, upon detection of a site failure or planned disaster test, will ’call home’,by using RSF, verify authorization, and automatically unlock the increased servercapacity. GDPS cannot be activated if On/Off COD is active.

v CBU Smart Reactivation

CBU Smart Reactivation reduces outage duration, during a disaster event, if anMCM needs to be changed (while CBU is activated).

v Concurrent Undo CBU

A prerequisite to executing this feature is that the customer or operating systemmust configure offline the processors that are being removed. The rule to followis, ″Configure offline the same logical processors that were configured onlinefollowing the CBU activation″. The Concurrent Undo CBU requires the followingactions to configure the processors offline based on how it will be invoked:– GDPS invoked Hardware Management Console/SE API.– Customer program invoked Hardware Management Console/SE API.

Notes:

1. As the user or operating system deconfigures logical CPs, there is noguarantee that the logical CPs will remain in sequential numbering.



2. The SE panel will give no directions as to which CPs, ICFs, or IFLs toconfigure offline.

For more information on CBU, refer to System z Capacity on Demand User’sGuide.


Concurrent PU ConversionsThe z9 EC supports concurrent conversion of different Processor Unit (PU) types.This capability is extended to Central Processors (CPs), Integrated Facilities forLinux (IFLs), Internal Coupling Facilities (ICFs), System z9 Integrated InformationProcessor (zIIPs), and System z9 Application Assist Processors (zAAPs). Thiscapability provides flexibility in configuring a z9 EC to meet the changing businessenvironments. For example, a z9 EC Model S08 configured with eight CPs could bereconfigured to seven CPs and one IFL by ordering the appropriate PU conversion.This order will generate a new LIC CC which can be installed concurrently in twosteps:

1. The one CP is removed from the z9 EC configuration.

2. The newly available PU is activated as an IFL. This assumes that at least oneCP remains active during the conversion of other processor units.

Note: Concurrent PU conversion is not supported by CIU.

Reserved CP Support in LPAR ModeWith Reserved CP Support in LPAR mode, a logical partition may be defined withthe number of logical CP’s greater than the number of physical CP’s. Additional CPscan be specified for the logical partition definition beyond the number of physicalCPs currently installed on the model. Therefore, an enterprise planning to do anon-disruptive upgrade (with an logical partition defined of logical CPs equal to thenumber of physical CPs available on the installed hardware) does not need todeactivate, redefine, then reactivate in order to take advantage of the new CPs thathave been activated. The enterprise simply needs to have defined additional CPsfor the logical partition in advance. This insures that any planned logical partitioncan be as large as the possible physical machine configuration, non-disruptively.For more information, refer to System z9 Processor Resource/Systems ManagerPlanning Guide.

Non-disruptive UpgradesThe z9 EC’s Plan-Ahead process links the use of Capacity Upgrade on Demandwith planning performed between IBM's Account team and IBM's customer.Planning ahead enables customers to determine a future server configuration. IBMwill also support its customers planning effort via capacity planning tools, IBM’sorder processing configurative and team sessions, with the objective ofnon-disruptive growth to satisfy essential capacity demand.



Processor Capacity DowngradesYou are allowed to downgrade your machine using CIU, CUoD, or MES. Theprimary benefit to downgrading is a reduction in software charges based on a lowerreported machine capacity.

Some additional considerations should be noted when downgrading:

v Downgrades are done by ″unassigning″ either CPs or IFLs.

v There may be a charge to unassign and then reactivate engines.

v Unassigned engines are still owned by the customer.

v Unassigning unused engines can reduce software charges since many softwareproducts are priced based on the number of active engines.

v Unassigned engines can be reactivated by CIU, CUoD, or MES.

v Unassigned engines may be temporarily activated using On/Off CoD. When usedas a temporary engine, unassigned engines can be used as any of the supportedengine types (thus an unassigned IFL can be activated as a CP). Reducedhardware usage charges are available when using unassigned engines as thesame type.

v Unassigning of engines and later reactivation is concurrent.

Capacity Upgrade on Demand (CUoD)Capacity Upgrade on Demand (CUoD) allows for the non-disruptive addition ofadditional Central Processors (CPs) capacity, Internal Coupling Facilities (ICFs),Integrated Facilities for Linux (IFLs), System z9 Integrated Information Processor(zIIPs), and System z9 Application Assist Processors (zAAPs). CUoD can quicklyadd processor capacity up to the maximum number of available inactive engines.This provides you with the capacity for much needed dynamic growth in anunpredictable e-business world. The CUoD function, combined with Parallel Sysplextechnology, enables virtually unlimited capacity upgrade capability.

The CUoD functions are:v Non-disruptive CP, ICF, IFL, and zAAP upgrades within minutesv Dynamic upgrade of all I/O cards in the I/O cagev Dynamic upgrade of spare installed memory (24 - 32 GB only)v Plan Ahead and Concurrent Conditioning.

Concurrent Conditioning configures a system for hot plugging of I/O based on afuture specified target configuration. Concurrent Conditioning of the System z9 I/Ois minimized by the fact that all I/O cards plugging into the System z9 I/O cage arehot pluggable. This means that the only I/O to be conditioned is the I/O cage itself.The question of whether or not to concurrently condition a cage is an importantconsideration, especially with the rapid change in the IT environment (e-business)as well as the technology. Migration to FICON or additional OSA networking isexceptionally easy and non-disruptive with the appropriate LIC and if the cagespace is available.

The z9 EC supports concurrent memory upgrade. This capability allows aprocessor’s memory to be increased without disrupting the processor operation. Totake advantage of this capability, a customer should not plan processor storage onthe 16 or 32 GB increments. If you have a Model S08 with 8 GB of storage you willbe able to concurrently upgrade to 16 GB but you will not be able to get to the nextincrement of 24 GB without a disruption.

8 GB-16 GB (Concurrent LIC CC upgrade)



24 GB-32 GB (Concurrent LICC CC upgrade)40 GB-48 GB and 56 GB-64 GB (Concurrent LIC CC upgrade)



Appendix D. Notices

This information was developed for products and services offered in the U.S.A.

IBM may not offer the products, services, or features discussed in this document inother countries. Consult your local IBM representative for information on theproducts and services currently available in your area. Any reference to an IBMproduct, program, or service is not intended to state or imply that only that IBMproduct, program, or service may be used. Any functionally equivalent product,program, or service that does not infringe any IBM intellectual property right may beused instead. However, it is the user’s responsibility to evaluate and verify theoperation of any non-IBM product, program, or service.

IBM may have patents or pending patent applications covering subject matterdescribed in this document. The furnishing of this document does not grant you anylicense to these patents. You can send license inquiries, in writing, to:

IBM Director of LicensingIBM CorporationNorth Castle DriveArmonk, NY 10504-1785 U.S.A.

The following paragraph does not apply to the United Kingdom or any othercountry where such provisions are inconsistent with local law:

INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THISPUBLICATION “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSOR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIESOF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR APARTICULAR PURPOSE. Some states do not allow disclaimer of express orimplied warranties in certain transactions, therefore, this statement may not apply toyou.

This information could include technical inaccuracies or typographical errors.Changes are periodically made to the information herein; these changes will beincorporated in new editions of the publication. IBM may make improvements and/orchanges in the product(s) and/or the program(s) described in this publication at anytime without notice.

Any references in this information to non-IBM Web sites are provided forconvenience only and do not in any manner serve as an endorsement of thoseWeb sites. The materials at those Web sites are not part of the materials for thisIBM product and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believesappropriate without incurring any obligation to you.

Any performance data contained herein was determined in a controlledenvironment. Therefore, the results obtained in other operating environments mayvary significantly. Some measurements may have been made on development-levelsystems and there is no guarantee that these measurements will be the same ongenerally available systems. Furthermore, some measurements may have beenestimated through extrapolation. Actual results may vary. Users of this documentshould verify the applicable data for their specific environment.


© Copyright IBM Corp. 2005, 2009 D-1

Information concerning non-IBM products was obtained from the suppliers of thoseproducts, their published announcements or other publicly available sources. IBMhas not tested those products and cannot confirm the accuracy of performance,compatibility or any other claims related to non-IBM products. Questions on thecapabilities of non-IBM products should be addressed to the suppliers of thoseproducts.

All statements regarding IBM’s future direction or intent are subject to change orwithdrawal without notice, and represent goals and objectives only.

All IBM prices shown are IBM’s suggested retail prices, are current and are subjectto change without notice. Dealer prices may vary.

This information is for planning purposes only. The information herein is subject tochange before the products described become available.

This information contains examples of data and reports used in daily businessoperations. To illustrate them as completely as possible, the examples include thenames of individuals, companies, brands, and products. All of these names arefictitious and any similarity to the names and addresses used by an actual businessenterprise is entirely coincidental.

If you are viewing this information softcopy, the photographs and color illustrationsmay not appear.

TrademarksIBM, the IBM logo, and ibm.com are trademarks or registered trademarks ofInternational Business Machines Corporation in the United States, other countries,or both. If these and other IBM trademarked terms are marked on their firstoccurrence in this information with a trademark symbol ( ® or ™ ), these symbolsindicate U.S. registered or common law trademarks owned by IBM at the time thisinformation was published. Such trademarks may also be registered or common lawtrademarks in other countries. A current list of IBM trademarks is available on theWeb at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml.

InfiniBand is a trademark of the InfiniBand Trade Association.

Intel and Pentium are trademarks or registered trademarks of Intel Corporation orits subsidiaries in the United States and other countries.

Linux is a registered trademark of Linus Torvalds in the United States, othercountries, or both.

Windows is a trademark of Microsoft Corporation in the United States, othercountries, or both.

Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. inthe United States, other countries, or both.

UNIX is a registered trademark of The Open Group in the United States and othercountries.

Other company, product and service names may be trademarks or service marks ofothers.


D-2 z9 EC System Overview

http://www.ibm.com/legal/copytrade.html

http://www.ibm.com/legal/copytrade.html

Electronic Emission NoticesThe following statement applies to this IBM product. The statement for other IBMproducts intended for use with this product will appear in their accompanyingmanuals.

Federal Communications Commission (FCC) Statement

Note: This equipment has been tested and found to comply with the limits for aClass A digital device, pursuant to Part 15 of the FCC Rules. These limitsare designed to provide reasonable protection against harmful interferencewhen the equipment is operated in a commercial environment. Thisequipment generates, uses, and can radiate radio frequency energy and, ifnot installed and used in accordance with the instructions contained in theinstallation manual, may cause harmful interference to radio communications.Operation of this equipment in a residential area is likely to cause harmfulinterference, in which case the user will be required to correct theinterference at his own expense.

Properly shielded and grounded cables and connectors must be used inorder to meet FCC emission limits. IBM is not responsible for any radio ortelevision interference caused by using other than recommended cables andconnectors, by installation or use of this equipment other than as specified inthe installation manual, or by any other unauthorized changes ormodifications to this equipment. Unauthorized changes or modifications couldvoid the user’s authority to operate the equipment.

This device complies with Part 15 of the FCC Rules. Operation is subject tothe following two conditions: (1) this device may not cause harmfulinterference, and (2) this device must accept any interference received,including interference that may cause undesired operation.

Canadian Department of Communications Compliance Statement

This Class A digital apparatus complies with Canadian ICES-003.

Avis de conformlté aux normes du ministère des Communications du Canada

Cet appareil numérique de la classe A est conform à la norme NMB-003 duCanada.

European Union (EU) Electromagnetic Compatibility Directive

This product is in conformity with the protection requirements of EU CouncilDirective 2004/108/EC on the approximation of the laws of the Member Statesrelating to electromagnetic compatibility. IBM cannot accept responsibility for anyfailure to satisfy the protection requirements resulting from a non-recommendedmodification of the product, including the fitting of non-IBM option cards.

This product has been tested and found to comply with the limits for Class AInformation Technology Equipment according to European Standard EN 55022. Thelimits for Class equipment were derived for commercial and industrial environmentsto provide reasonable protection against interference with licensed communicationequipment.


Appendix D. Notices D-3

Warning: This is a Class A product. In a domestic environment, this product maycause radio interference in which case the user may be required to take adequatemeasures.

European Community contact:IBM Technical RegulationsPascalstr. 100, Stuttgart, Germany 70569Telephone: 0049 (0) 711 785 1176Fax: 0049 (0) 711 785 1283email: [email protected]

EC Declaration of Conformity (In German)

Deutschsprachiger EU Hinweis: Hinweis für Geräte der Klasse A EU-Richtliniezur Elektromagnetischen Verträglichkeit

Dieses Produkt entspricht den Schutzanforderungen der EU-Richtlinie 89/336/EWGzur Angleichung der Rechtsvorschriften über die elektromagnetische Verträglichkeitin den EU-Mitgliedsstaaten und hält die Grenzwerte der EN 55022 Klasse A ein.

Um dieses sicherzustellen, sind die Geräte wie in den Handbüchern beschrieben zuinstallieren und zu betreiben. Des Weiteren dürfen auch nur von der IBMempfohlene Kabel angeschlossen werden. IBM übernimmt keine Verantwortung fürdie Einhaltung der Schutzanforderungen, wenn das Produkt ohne Zustimmung derIBM verändert bzw. wenn Erweiterungskomponenten von Fremdherstellern ohneEmpfehlung der IBM gesteckt/eingebaut werden.

EN 55022 Klasse A Geräte müssen mit folgendem Warnhinweis versehen werden:″Warnung: Dieses ist eine Einrichtung der Klasse A. Diese Einrichtung kann imWohnbereich Funk-Störungen verursachen; in diesem Fall kann vom Betreiberverlangt werden, angemessene Maßnahmen zu ergreifen und dafür aufzukommen.″

Deutschland: Einhaltung des Gesetzes über die elektromagnetischeVerträglichkeit von Geräten

Dieses Produkt entspricht dem “Gesetz über die elektromagnetische Verträglichkeitvon Geräten (EMVG)“. Dies ist die Umsetzung der EU-Richtlinie 89/336/EWG in derBundesrepublik Deutschland.

Zulassungsbescheinigung laut dem Deutschen Gesetz über dieelektromagnetische Verträglichkeit von Geräten (EMVG) vom 18. September1998 (bzw. der EMC EG Richtlinie 89/336) für Geräte der Klasse A.

Dieses Gerät ist berechtigt, in Übereinstimmung mit dem Deutschen EMVG dasEG-Konformitätszeichen - CE - zu führen.

Verantwortlich für die Konformitätserklärung nach Paragraf 5 des EMVG ist die IBMDeutschland GmbH, 70548 Stuttgart.

Informationen in Hinsicht EMVG Paragraf 4 Abs. (1) 4:

Das Gerät erfüllt die Schutzanforderungen nach EN 55024 und EN 55022Klasse A.

update: 2004/12/07



People’s Republic of China Class A Compliance Statement

This is a Class A product. In a domestic environment, this product may cause radiointerference in which case the user may need to perform practical actions.

Japan Class A Compliance Statement

This product is a Class A Information Technology Equipment and conforms to thestandards set by the Voluntary Control Council for Interference by InformationTechnology Equipment (VCCI). In a domestic environment, this product may causeradio interference in which case the user may be required to take adequatemeasures.

Korean Class A Compliance Statement

Taiwan Class A Compliance Statement

Warning: This is a Class A product. In a domestic environment, this product maycause radio interference in which case the user will be required to take adequatemeasures.


Appendix D. Notices D-5

Glossary

This glossary includes terms and definitions from:

v The Dictionary of Computing, SC20-1699.

v The American National Standard Dictionary forInformation Systems, ANSI X3.172-1990,copyright 1990 by the American NationalStandards Institute (ANSI). Copies can bepurchased from the American NationalStandards Institute, 1430 Broadway, New York,New York 10018. Definitions are identified bythe symbol (A) after the definition.

v The ANSI/EIA Standard - 440A: Fiber OpticTerminology, copyright 1989 by the ElectronicsIndustries Association (EIA). Copies can bepurchased from the Electronic IndustriesAssociation, 2001 Pennsylvania Avenue N.W.,Washington, D.C. 20006. Definitions areidentified by the symbol (E) after the definition.

v The Information Technology Vocabulary,developed by Subcommittee 1, Joint TechnicalCommittee 1, of the International Organizationfor Standardization and the InternationalElectrotechnical Commission (ISO/IECJTC1/SC1). Definitions of published parts of thisvocabulary are identified by the symbol (I) afterthe definition; definitions taken from draftinternational standards, committee drafts, andworking papers being developed by ISO/IECJTC1/SC1 are identified by the symbol (T) afterthe definition, indicating that final agreementhas not yet been reached among theparticipating National Bodies of SC1.

The following cross-references are used in thisglossary:

v Contrast with.This refers to a term that has anopposed or substantively different meaning.

v See. This refers the reader to multiple-wordterms in which this term appears.

v See also. This refers the reader to terms thathave a related, but not synonymous, meaning.

v Synonym for. This indicates that the term hasthe same meaning as a preferred term, which isdefined in the glossary.

Aabend. Abnormal end of task

abnormal end of task (abend). Ending a task beforeits completion because of an error condition that cannotbe resolved by recovery facilities while the task is beingexecuted.

action. One of the defined tasks that an applicationperforms. Actions modify the properties of an object ormanipulate the object in some way.

activate logical partition. An operator-initiatedprocedure that performs a system reset to a logicalpartition and assigns the previously defined hardware tothat partition. It causes an automatic IPL of the systemcontrol program to occur in the partition unless theoperator performs the IPL manually. Contrast withdeactivate logical partition.

active configuration. In an ESCON environment, theESCON Director configuration determined by the statusof the current set of connectivity attributes. Contrast withsaved configuration.

active subchannel. A subchannel that is locked andeither busy or has a pending interrupt, and is indicatedby subchannel status word (SCSW) bit 24 equals 1. Thecontrol information resides in the channel subsystembecause it is necessary for the current operation.Contrast with inactive subchannel. See also busysubchannel.

Note: An active subchannel can also reside in the localworking storage of an IOP or channel.

active window. The window that users are currentlyinteracting with. This is the window that receiveskeyboard input.

address. (1) A value that identifies a register, aparticular part of storage, data source, or a data sink.The value is represented by one or more characters. (T)(2) To refer to a device or an item of data by itsaddress. (I) (A) (3) The location in the storage of acomputer where data is stored. (4) In datacommunication, the unique code assigned to eachdevice or workstation connected to a network. (5) Theidentifier of a location, source, or destination.

address translation. (1) A value that identifies aregister, a particular part of storage, a data source, or adata sink. The value is represented by one or morecharacters. (T) (2) In virtual storage systems, theprocess of changing the address of an item of data oran instruction from its virtual storage address to its realstorage address. See also dynamic address translation.

ADMF. Asynchronous Data Mover Facility.

alert. (1) A unit of information, usually indicating theloss of a system resource, passed from one machine or


© Copyright IBM Corp. 2005, 2009 E-1

program to a host to signal an error. (2) An errormessage sent to the system services control point(SSCP) at the host system.

allocate. To assign a resource, such as a disk or adiskette file to perform a task. Contrast with deallocate.

American National Standard Code for InformationInterchange (ASCII). The standard code, using acoded character set consisting of 7-bit coded characters(8 bits including parity), used for information interchangeamong data processing systems, data communicationsystems, and associated equipment. The ASCII setconsists of control characters and graphics characters.(A)

Note: IBM has defined an extension to ASCII code(characters 128 - 255).

ANSI. American National Standards Institute.

AOC/MVS. automated operations control/multiplevirtual storage.

APAR. Authorized program analysis report.

API. application programming interface

application. (1) The use to which an informationprocessing system is put, for example, a payrollapplication, an airline reservation application, a networkapplication. (2) A collection of software componentsused to perform specific types of work on a computer.

Application Assist Processor (AAP). A specialprocessor configured for running Java applications onz9 EC, z990 and z890 class machines.

application program. (1) A program that is specific tothe solution of an application problem.(T) (2) A programwritten for or by a user that applies to the user's work,such as a program that does inventory control orpayroll. (3) A program used to connect andcommunicate with stations in a network, enabling usersto perform application-oriented activities.

architected . Defined in the z/Architecture Principlesof Operation or IBM Enterprise SystemsArchitecture/390 Principles of Operation.

ARP. Address Resolution Protocol

ASCII. American National Standard Code forInformation Interchange.

asynchronous. (1) Pertaining to two or moreprocesses that do not depend upon the occurrence ofspecific events such as common timing signals. (T) (2)Without regular time relationship; unexpected orunpredictable with respect to the execution of programinstructions. Contrast with synchronous.

ATM. Asynchronous Transfer Mode

ATM. Automatic Teller Machine.

authorized program analysis report (APAR). Arequest for correction of a problem caused by a defectin a current release of a program unaltered by the user.

auto-answer. In data communication, the ability of astation to respond automatically to a call that it receivesover a switched line.

auto-call. In data communication, the ability of astation to initiate a call automatically over a switchedline.

Bbasic mode. A central processor mode that does notuse logical partitioning. Contrast with logicallypartitioned (LPAR) mode.

batch. (1) An accumulation of data to be processed.(2) A group of records or data processing jobs broughttogether for processing or transmission. (3) Pertaining toactivity involving little or no user action. Contrast withinteractive.

BBU. Battery Back-up Unit

BL. Parallel block multiplexer channel.

block. A string of data elements recorded ortransmitted as a unit. The element may be characters,words, or physical records. (T)

block multiplexer channel. A multiplexer channel thatinterleaves blocks of data. Contrast with selectorchannel. See also byte multiplexer channel

BPA. Bulk Power Assembly

buffer. (1) A routine or storage used to compensate fora difference in rate of flow of data, or time of occurrenceof events, when transferring data from one device toanother. (A) (2) To allocate and schedule the use ofbuffers. (A) (3) A portion of storage used to hold input oroutput data temporarily. See fiber buffer.

burst. In data communication, a sequence of signalscounted as one unit in accordance with some specificcriterion or measure. (A)

bus. (1) A facility for transferring data between severaldevices located between two end points, only onedevice being able to transmit at a given moment. (T) (2)A network configuration in which nodes areinterconnected through a bidirectional transmissionmedium. (3) One or more conductors used fortransmitting signals or power. (A)

BY. Parallel byte multiplexer channel.

byte. (1) A string that consists of a number of bits,treated as a unit, and representing a character. (T) (2) A


E-2 z9 EC System Overview

binary character operated upon as a unit and usuallyshorter than a computer word. (A) (3) A string thatconsists of a particular number of bits, usually eight,that is treated as a unit, and that represents a character.(4) A group of eight adjacent binary digits that representone extended binary-coded decimal interchange code(EBCDIC) character.

byte multiplexer channel. A multiplexer channel thatinterleaves bytes of data.Contrast with selector channel.See also block multiplexer channel.

cCA. channel adapter (card).

cache. (1) A special purpose buffer storage, smallerand faster than main storage, used to hold a copy of theinstructions and data obtained from main storage andlikely to be needed next by the processor. (T) (2) Abuffer storage that contains frequently accessedinstructions and data; it is used to reduce access time.

cache structure. A coupling facility structure thatenables high-performance sharing of cached data bymultisystem applications in a Parallel Sysplex.Applications can use a cache structure to implementseveral different types of caching systems, including astore-through or a store-in cache.

CAW. channel address word.

CBU. Capacity Back Up

CBY. ESCON byte multiplexer channel.

CCC. Channel control check.

CCW. Channel command word.

CDC. Channel data check.

central processor (CP). The part of the computer thatcontains the sequencing and processing facilities forinstruction execution, initial program load, and othermachine operations.

central processor complex (CPC). The boundaries ofa system, exclusive of I/O control units and devices,that can be controlled by a single operating system. ACPC consists of main storage, one or more centralprocessor units, time-of-day clocks, and channels, whichare or can be placed in a single configuration. A CPCalso includes channel subsystems, service processors,and expanded storage where installed.

central storage. Storage that is an integral part of theprocessor and includes both main storage and thehardware system area.

CF. coupling facility (CF).

CFS. coupling facility sender channel.:gd.couplingfacility sender channel.

CFR. coupling facility receiver channel.

CFRM. coupling facility resource management (policy).

change. An alteration (addition, deletion, ormodification) of one or more information systemcomponents, of one of the following types: hardware(may include internal code), or software (system orapplication). The term change also refers to an SNA/FileServices data object containing internal code, internalcode customizing data, software, software customizingdata, applications data, procedures, or documentation.

CH. Channel Card

change management. The management disciplinethat defines the process of planning (for example,scheduling) and controlling (for example, distributing,installing, and tracking) changes in an SNA network.

channel. (1) A path along which signals can be sent,for example, input/output channel. (2) The systemelement that controls one channel path, whose mode ofoperation depends on the type of hardware to which it isattached.

channel adapter. (1) A communication controllerhardware unit used to attach the controller to a datachannel. (2) Hardware that attaches a group ofchannels to the secondary data stager and prioritizesand stages data between the channels and the channelcontrol element.

channel address . In S/370 mode, the 8 leftmost bitsof an input/output address that identify the channel. Seealso device address and input/output address.

channel address word (CAW). An area in storagethat specifies the location in main storage at which achannel program begins.

channel-attached. (1) Pertaining to attachment ofdevices directly by data channels (I/O channels) to acomputer. (2) Pertaining to devices attached to acontrolling unit by cables rather than bytelecommunication lines. Contrast with link-attached.

channel command word (CCW). A doubleword at thelocation in main storage specified by the channeladdress word. One or more CCWs make up thechannel program that directs data channel operations.

channel control check. A category of I/O errorsaffecting channel controls and sensed by the channel towhich a device is attached. See also channel datacheck.

channel data check. A category of I/O errors,indicating a machine error in transferring data to or from


Glossary E-3

storage and sensed by the channel to which a device isattached. See also channel control check.

channel data rate. The rate at which a channel canmove data between a transmission link and processorstorage during the data transfer portion of an I/Ooperation.

channel Licensed Internal Code. That part of thechannel subsystem Licensed Internal Code used tostart, maintain, and end all operations on the I/Ointerface. See also IOP Licensed Internal Code.

channel path (CHP). A single interface between acentral processor and one or more control units alongwhich signals and data can be sent to perform I/Orequests.

channel path configuration. In an ESCONenvironment, the connection between a channel and acontrol unit or between a channel, an ESCON Director,and one or more control units. See also link,point-to-point channel path configuration, and switchedpoint-to-point channel path configuration.

channel path identifier (CHPID). In a channelsubsystem, a value assigned to each installed channelpath of the system that uniquely identifies that path tothe system.

channel status word (CSW). An area in storage thatprovides information about the termination ofinput/output operations.

channel subsystem (CSS). A collection ofsubchannels that directs the flow of information betweenI/O devices and main storage, relieves the processor ofcommunication tasks, and performs path managementfunctions.

channel subsystem (CSS) Licensed Internal Code.Code that consists of the IOP Licensed Internal Codeand the channel Licensed Internal Code.

channel-to-channel (CTC). Refers to thecommunication (transfer of data) between programs onopposite sides of a channel-to-channel adapter (CTCA).

channel-to-channel adapter (CTCA). An input/outputdevice that is used by a program in one system tocommunicate with a program in another system.

check stop. The state that occurs when an errormakes it impossible or undesirable to continue theoperation in progress.

CHD. channel driver (card).

CHN. Channel.

choice. An item that users may select. Choicesappear in the selection fields, action bars, and pull-downmenus.

CHP. Channel path.

CHPID. See channel path identifier.

CI. console integration.

CICS. Customer Information Control System.

CICS/ESA. Customer Information ControlSystem/Enterprise Systems Architecture.

CIU. Customer Initiated Upgrade

CKD. count key data.

click. To press and release a mouse button withoutmoving the mouse pointer off the choice.

CLIST (command list). A data set in which commandsand possibly subcommands and data are stored forsubsequent execution.

CMOS. Complementary metal-oxide semiconductor.

CNC. Mnemonic for an ESCON channel attached toan ESCON-capable device.

command. (1) A character string from a sourceexternal to a system that represents a request forsystem action. (2) A request from a terminal forperformance of an operation or execution of a pro gram.(3) A value sent on an I/O interface from a channel to acontrol unit that specifies the operation to be performed.

command chaining. The fetching of a new channelcommand word (CCW) immediately following thecompletion of the previous CCW.

command entry field. An entry field in which a usertypes commands.

command list. See CLIST.

command retry. A channel and control unit procedurethat causes a command to be retried without requiringan I/O interrupt.

communication control unit. A communication devicethat controls transmission of data over lines in anetwork.

communication controller. (1) A device that directsthe transmission of data over the data links of anetwork; its operation can be controlled by a programexecuted in a processor to which the controller isconnected or it may be controlled by a programexecuted within the device. (T) (2) A type ofcommunication control unit whose operations arecontrolled by one or more programs stored andexecuted in the unit. It manages the details of linecontrol and the routing of data through a network.



complementary metal-oxide semiconductor(CMOS). A technology that combines the electricalproperties of positive and negative voltage requirementsto use considerably less power than other types ofsemiconductors.

concurrent maintenance. Hardware maintenanceactions performed by a service representative whilenormal operations continue without interruption. Seealso nondisruptive installation and nondisruptiveremoval.

configuration. (1) The arrangement of a computersystem or network as defined by the nature, number,and the chief characteristics of its functional units. Morespecifically, the term configuration may refer to ahardware configuration or a software configuration. (I)(A) (2) In an ESCON Director, the physical connectioncapability determined by a set of attributes. The attributevalues specify the connectivity control status andidentifiers associated with the ESCD and its ports. Seealso active configuration, configuration matrix,connectivity attribute, and saved configuration.

configure. To describe to the system the devices andoptional features installed on the system.

connectivity. A term used to describe the physicalinterconnections of multiple devices/computers/networksemploying similar or different technology or architecturetogether to accomplish effective communicationbetween and among connected members involving dataexchange or resource sharing.

console. A logical device used for communicationbetween the user and the system. (A) See displaystation, monitor console, operator console, programmode console, programming support console, serviceconsole, and system console.

console integration (CI). The hardware and softwarefacilities used to bring operating systems managementand hardware systems management under a singlecontrol point.

control program. A computer program designed toschedule and to supervise the execution of programs ofa computer system. (I) (A)

control unit. A hardware unit that controls the reading,writing, or displaying of data at one or more input/outputunits.

control unit data rate. The rate at which a control unitcan move data between itself and a transmission linkduring the data transfer portion of an I/O operation.

controller. A unit that controls input/output operationsfor one or more devices.

conversational monitor system (CMS). A virtualmachine operating system that provides general

interactive time sharing, problem solving, and programdevelopment capabilities, and operates only under theVM control program.

Coordinated Server Time (CST). Represents the timein a CTN. Timekeeping messages carried over thecoupling links determine the CST at each server.

Coordinated Timing Network (CTN). A collection ofservers that are time synchronized to CoordinatedServer Time (CST). All STP-configured servers in aCTN must have the same CTN ID.

coupling facility. A special logical partition thatprovides high-speed caching, list processing, andlocking functions in a Parallel Sysplex.

coupling facility channel. A high bandwidth fiberoptic channel that provides the high-speed connectivityrequired for data sharing between a coupling facility andthe central processor complexes directly attached to it.

CP. (1) Control program. (2) Central processor.

CPC. Central processor complex.

CPCID. CPC identifier.

CPC Image. The set of CPC resources that support asingle control program.

CPU. central processor unit.

CPUID. CPU identifier.

critical resource. A resource required for systemoperation. See also system resource.

CRW. Channel report word.

CS. (1) Central storage. (2) Channel set. (3) Controlstorage.

CSS. Channel subsystem.

CST. Coordinated Server Time.

CSW. Channel status word.

CTC. (1) Channel-to-channel. (2) Mnemonic for anESCON or FICON channel attached to another ESCONor FICON channel respectively.

CTCA. channel-to-channel adapter.

CTN. Coordinated Timing Network

CU. Control unit.

CUA®. Control unit address.

CUADD. Control unit logical address.

CUoD. Capacity Upgrade on Demand


Glossary E-5

cursor. (1) A movable, visible mark used to indicatethe position at which the next operation will occur on adisplay screen. (A) (2) A visual cue that shows the userwhere keyboard input will appear on the screen.

Customer Information Control System (CICS). AnIBM licensed program that enables transactions enteredat remote terminals to be processed concurrently byuser-written application programs. It includes facilitiesfor building, using, and maintaining data bases.

customize. To change a data processing installation ornetwork to meet the needs of particular users. Definition

CVC. Mnemonic for an ESCON channel attached to a9034.

DDASD. See direct access storage device.

DASD subsystem. A storage control and its attacheddirect access storage devices.

DAT. Dynamic address translation.

data processing (DP). The systematic performance ofoperations upon data; for example, arithmetic or logicoperations upon data, merging or sorting of data,assembling or compiling of programs. (T)

data rate. See channel data rate, control unit datarate, device data rate, effective data rate, andinput/output subsystem data rate. See also link rate.

data sharing. The ability of concurrent subsystems(such as DB2 or IMS DB) or application programs todirectly access and change the same data whilemaintaining data integrity.

data streaming. In an I/O interface, a mode ofoperation that provides a method of data transfer at upto 4.5 MB per second. Data streaming is not interlockedbetween the sender and the receiver. Once datatransfer begins, the sender does not wait foracknowledgment from the receiver before sending thenext byte. The control unit determines the data transferrate.

data transfer. (1) The result of the transmission ofdata signals from any data source to a data receiver. (2)The movement, or copying, of data from one locationand the storage of the data at another location.

data transfer mode. The method of informationexchange used on an I/O interface. See data streaming.

DB2. DATABASE 2.

DCA. Distributed Converter Assembly

DCAF. distributed console access facility.

deactivate logical partition. An operator-initiatedprocedure that releases the hardware assigned to alogical partition, making it available to other partitions.Contrast with activate logical partition.

Note: The operator should first deactivate the systemcontrol program, if possible or necessary, andthen reactivate the partition, which could providea reset to that partition, if required.

deallocate. To release a resource assigned to a task.Contrast with allocate.

DES. Data Encryption Standard

device. A mechanical, electrical, or electroniccontrivance with a specific purpose.

device address. In S/370 mode, the 8 rightmost bitsof an I/O address that identify a particular I/O deviceand a control unit on the designated channel. Seechannel address, device-level addressing, andinput/output address.

DFSMS™. Data Facility Storage ManagementSubsytem.

DH. Diffie Hellman

direct access storage. A storage device that providesdirect access to data. (I) (A) See also random accessmemory.

direct access storage device (DASD). (1) A storagedevice in which the location of each data record can bedirectly addressed. (2) A device in which the accesstime is effectively independent of the location of thedata. (Restriction: Does not refer to diskette drive.)

DP. data processing.

DSS. Digital Signature Standard

dynamic address translation (DAT). In virtual storagesystems, the change of a virtual storage address to areal storage address during execution of an instruction.See also address translation.

dynamic reconfiguration management. In MVS, theability to modify the I/O configuration definition withoutneeding to perform a power-on reset (POR) of thehardware or an initial program load (IPL).

dynamic storage reconfiguration. A PR/SM LPARfunction that allows central or expanded storage to beadded or removed from a logical partition withoutdisrupting the system control program operating in thelogical partition.



EEC. engineering change.

ECC. error checking and correction.

ECKD. extended count key data.

EEPROM. electrically erasable programmable readonly memory.

EIA. Electronics Industries Association. One EIA unit is1.75 inches or 44.45mm.

Enterprise Systems Architecture/390 (ESA/390).IBM ESA/390 principles of operation.

Enterprise Systems Connection (ESCON). A set ofproducts and services that provides a dynamicallyconnected environment using optical cables as atransmission medium.

EPO. Emergency power off.

error checking and correction (ECC). In a processor,the detection and correction of all single-bit errors, plusthe detection of double-bit and some multiple-bit errors.

ESA. (1) Enterprise Systems Architecture. (2)Expanded storage array.

ESA/370. Enterprise Systems Architecture/370™.

ESA/390. Enterprise Systems Architecture/390.

ESCD. Enterprise Systems Connection (ESCON)Director.

ESCM. Enterprise Systems Connection Manager.

ESCON. Enterprise Systems Connection.

ESCON channel. A channel having an EnterpriseSystems Connection channel-to-control-unit I/O interfacethat uses optical cables as a transmission medium.Contrast with parallel channel.

ESCON Director (ESCD). A device that providesconnectivity capability and control for attaching any twolinks to each other.

ESCON environment. The data processingenvironment having an Enterprise Systems Connectionchannel-to-control-unit I/O interface that uses opticalcables as a transmission medium.

ESCON Manager (ESCM). A licensed program thatprovides host control and intersystem communicationcapability for ESCON Director connectivity operations.

Ethernet Definition. A communication network (USA,Xerox 1975).

ETR. External Time Reference

event. (1) An occurrence or happening. (2) Anoccurrence of significance to a task; for example, thecompletion of an asynchronous operation, such as aninput/output operation.

expanded storage. Optional high-speed storage thattransfers 4KB pages to and from central storage.

Ffacility. (1) An operational capability, or the means forproviding such a capability. (T) (2) A service provided byan operating system for a particular purpose; forexample, the checkpoint/restart facility.

FCP. Fibre Channel Protocol for SCSI

FDDI. Fiber Distributed Data Interface(100Mbits/second fiber optic LAN).

fiber. See optical fiber.

FICON. FIber CONnection channel card

frame. (1) For a zSystem or zSeries microprocessorcluster, a frame contains one or two central processorcomplexes (CPCs), Support Elements, and AC powerdistribution. (2) A housing for machine elements. (3) Thehardware support structure, covers, and all electricalparts mounted there in that are packaged as one entityfor shipping. (4) A formatted display. See display frameand transmission frame.

FRU. Field-replaceable unit.

ft. Foot.

GGb. Gigabit

GB. Gigabyte.

GbE. Gigabit Ethernet.

gigabit (Gb). A unit of measure for storage size. Onegigabit equals one billion bits.

Gigabit Ethernet. An OSA channel (type OSD)

gigabyte (GB). (1) A unit of measure for storage size.One gigabyte equals 1,073,741,824 bytes. (2) Loosely,one billion bytes.

GMT. Greenwich mean time.

HHardware Management Console. A console used tomonitor and control hardware such as the zSystem z9or zSeries z990 microprocessors.


Glossary E-7

hardware system area (HSA). A logical area ofcentral storage, not addressable by applicationprograms, used to store Licensed Internal Code andcontrol information.

HCA. Host Channel Adapter

HCA1-O fanout. The HCA1-O (optical) fanout card isused for coupling using an InfiniBand connection on az9 server. The HCA1-O fanout is designed to support atwo-port 12x IB-SDR optical link operating at a link rateof 3 GBps.

HCD. hardware configuration definition.

HDD. HDD

HSA. hardware system area.

HMCA. Hardware Management Console Application.

IIBB. Internal Bus Buffer

IBF. Internal Battery Feature

IC. Internal Coupling link

ICB. Integrated Cluster Bus link

ICF. Internal Coupling Facility

ICMF. Integrated Coupling Migration Facility.

ICRF. Integrated Cryptographic Facility

ICSF. Integrated Cryptographic Service Facility

IDAW. indirect data address word.

IFCC. interface control check.

IFL. Integrated Facility for Linux

IML. Initial machine load.

IMS. Information Management System.

initial machine load (IML). A procedure that preparesa device for use.

initial program load (IPL). (1) The initializationprocedure that causes an operating system tocommence operation. (2) The process by which aconfiguration image is loaded into storage at thebeginning of a work day or after a system malfunction.(3) The process of loading system programs andpreparing a system to run jobs.

initialization. (1) The operations required for setting adevice to a starting state, before the use of a datamedium, or before implementation of a process. (T) (2)Preparation of a system, device, or program for

operation. (3) To set counters, switches, addresses,latches, or storage contents to zero or to other startingvalues at the beginning of, or at the prescribed pointsin, a computer program or process.

Integrated Facility for Applications (IFA). A generalpurpose assist processor for running specific types ofapplications. See Application Assist Processor (AAP).

input/output (I/O). (1) Pertaining to a device whoseparts can perform an input process and an outputprocess at the same time. (I) (2) Pertaining to afunctional unit or channel involved in an input process,output process, or both, concurrently or not, and to thedata involved in such a process. (3) Pertaining to input,output, or both.

input/output configuration. The collection of channelpaths, control units, and I/O devices that attach to theprocessor complex.

input/output configuration data set (IOCDS). Thedata set that contains an I/O configuration definition builtby the I/O configuration program (IOCP).

input/output configuration program (IOCP). Aprogram that defines to a system all the available I/Odevices and the channel paths.

input/output configuration source. The file used asinput to the IOCP when an IOCDS is created. It is an80-column card-image, and is initially provided inEBCDIC form on a system tape, or in ASCII form on a3.5-inch diskette.

interrupt. (1) A suspension of a process, such asexecution of a computer program caused by an externalevent, and performed in such a way that the processcan be resumed. (A) (2) To stop a process in such away that it can be resumed. (3) In data communication,to take an action at a receiving station that causes thesending station to end a transmission. (4) To temporarilystop a process.

I/O. See input/output.

IOCDS. I/O configuration data set.

IOCP. I/O configuration program.

IODF. input/output definition file.

IPL. See initial program load.

IPv6. Internet Protocol Version 6

ISC. InterSystem Coupling link

ISDN. Integrated-Services Digital Network

KKB. kilobyte.



|

kilobyte. (1) A unit of measure for storage size. (2)Loosely, one thousand bytes.

km. kilometer

LLAN. See local area network.

laser. A device that produces optical radiation using apopulation inversion to provide light amplification bystimulated emission of radiation and (generally) anoptical resonant cavity to provide positive feedback.Laser radiation can be highly coherent temporally, orspatially, or both. (E)

LCSS. logical channel subsystem.

LED. Light-emitting diode.

LIC. Licensed Internal Code.

Licensed Internal Code (LIC). Software provided foruse on specific IBM machines and licensed tocustomers under the terms of IBM's CustomerAgreement. Microcode can be Licensed Internal Codeand licensed as such.

light-emitting diode (LED). A semiconductor chip thatgives off visible or infrared light when activated.

local area network (LAN). A computer networklocated on a user's premises within a limitedgeographical area. Communication within a local areanetwork is not subject to external regulations; however,communication across the LAN boundary can besubject to some form of regulation.

Note: A LAN does not use store and forwardtechniques.

logical address. The address found in the instructionaddress portion of the program status word (PSW). Iftranslation is off, the logical address is the real address.If translation is on, the logical address is the virtualaddress. See also absolute address, physical address,real address, and virtual address.

logical control unit. A group of contiguous words inthe hardware system area that provides all of theinformation necessary to control I/O operations througha group of paths that are defined in the IOCDS. Logicalcontrol units represent to the channel subsystem a setof control units that attach common I/O devices.

logical partition (LP). A subset of the processorhardware that is defined to support the operation of asystem control program (SCP). See also logicallypartitioned (LPAR) mode.

logical processor. In LPAR mode, central processorresources defined to operate in a logical partition like aphysical central processor.

logical unit (LU). In SNA, a port to the networkthrough which an end user accesses the SNA networkand the functions provided by system services controlpoints (SSCPs). An LU can support at least twosessions - one with an SSCP and one with another LU -and may be capable of supporting many sessions withother LUs.

logically partitioned (LPAR) mode. A centralprocessor complex (CPC) power-on reset mode thatenables use of the PR/SM feature and allows anoperator to allocate CPC hardware resources (includingcentral processors, central storage, expanded storage,and channel paths) among logical partitions. Contrastwith basic mode.

LP. Logical partition.

LU. logical unit.

LUPS. Local Uninterruptible Power Supply

MMAC. Message Authentication Code

main storage. A logical entity that represents theprogram addressable portion of central storage. Seecentral storage.

main storage. (1) Program-addressable storage fromwhich instructions and other data can be loaded directlyinto registers for subsequent processing. (I) (A) (2) Thatpart of internal storage into which instructions and otherdata must be loaded for subsequent execution orprocessing. (3) The part of a processor unit whereprograms are run. See central storage.

Notes:

1. Main storage refers to the wholeprogram-addressable execution space and caninclude one or more storage devices.

2. The term main storage is generally used in largeand intermediate computers. The term memory isprimarily used in microcomputers, calculators, andsome minicomputers.

maintenance change level (MCL). A change tocorrect a single licensed internal code design defect.Higher quality than a patch, and intended for broaddistribution. Considered functionally equivalent to asoftware PTF.

MAU. multistation access unit.

Mb. Megabit

MB. Megabyte.

MBA. memory bus adapter.

MBA fanout. On a z9 server, an MBA fanout card isused for coupling using ICB-4 and for connecting an I/O


Glossary E-9

cage to a CEC. On a z10 EC server, MBA fanout isused for coupling using ICB-4.

MCCU. multisystem channel communication unit.

MCL. See maintenance change level.

megabit (Mb). A unit of measure for storage size. Onemegabit equals 1,000,000 bits.

megabyte (MB). (1) A unit of measure for storagesize. One megabyte equals 1,048,576 bytes. (2)Loosely, one million bytes.

menu bar. The area at the top of the primary windowthat contains keywords that give users access to actionsavailable in that window. After users select a choice inthe action bar, a pull-down menu appears from theaction bar.

MIDAW. Modified Data Indirect Address Word

MIF. Multiple Image Facility.

modem (modulator/demodulator). A device thatconverts digital data from a computer to an analogsignal that can be transmitted on a telecommunicationline, and converts the analog signal received to data forthe computer.

mouse. In computer graphics, a pointing deviceoperated by moving it on a surface.

Multiple Image Facility (MIF). A facility that allowschannels to be shared among PR/SM logical partitionsin an ESCON or FICON environment.

multiplexer channel. A channel designed to operatewith a number of I/O devicessimultaneously.Several I/Odevices can transfer records at the same time byinterleaving items of data. See block multiplexer channeland the multiplexer channel.

MVS. Multiple Virtual Storage.

MVS image. A single occurrence of the MVS/ESA™

operating system that has the ability to process work.

MVS system. An MVS image together with itsassociated hardware, which collectively are oftenreferred to simply as a system, or MVS system.

NNetBIOS. local area network basic input/outputsystem.

network. (1) An arrangement of nodes and connectingbranches. (T) (2) A configuration of data processingdevices and software connected for informationexchange.

NIC. numerically intensive computing.

OOn/Off Capacity on Demand (On/Off CoD). Used totemporarily turn on CPs, IFLs, ICFs, zIIPs, and zAAPs.

operate. To do a defined action, such as adding orcomparing, performed on one or more data items.

operating system (OS). Software that controls theexecution of programs and that may provide servicessuch as resource allocation, scheduling, input/outputcontrol, and data management. Although operatingsystems are predominantly software, partial hardwareimplementations are possible. (T)

operator console. (1) A functional unit containingdevices that are used for communications between acomputer operator and a computer. (T) (2) A displayused for communication between the operator and thesystem, used primarily to specify information concerningapplication programs and I/O operations and to monitorsystem operation.

optical cable. A fiber, multiple fibers, or a fiber bundlein a structure built to meet optical, mechanical, andenvironmental specifications. (E) See also jumper cable,optical cable assembly, and trunk cable.

optical fiber. Any filament made of dielectric materialsthat guides light, regardless of its ability to send signals.(E) See also fiber optics and optical waveguide.

OS. Operating system.

OSA. Open Systems Adapter (OSA-Express,OSA-Express2, and OSA-2). The OSA is an integratedhardware feature that provides direct connection toclients on local area networks (LANs).

OSA/SF. Open Systems Adapter/Support Facility

OS/2. Operating System/2®.

Pparallel channel. (1) A channel having a S/360™ andS/370 channel-to-control-unit I/O interface that usesbus-and-tag cables as a transmission medium. Contrastwith ESCON channel. (2) A data path along which agroup of signals representing a character or any otherentity of data can be sent simultaneously.

Parallel Sysplex. A set of MVS systemscommunicating and cooperating with each other throughcertain multisystem hardware components and softwareservices to process customer workloads. See also MVSsystem.

PIN. Personal Identification Number

PKA. Public-Key-Algorithm



PKSC. Public-Key Secure Cable

point-to-point channel path configuration. In an I/Ointerface, a configuration that consists of a single linkbetween a channel and one control unit. Contrast withswitched point-to-point channel path configuration.

point-to-point connection. A connection establishedbetween two data stations for data transmission.Contrast with multipoint connection.

Note: The connection may include switching facilities.

POR. Power-on reset.

power-on reset. A function that re-initializes all thehardware in the system and loads the internal code thatenables the machine to load and run an operatingsystem. This function is intended as a recovery function.

power-on reset state. The condition after a machinepower-on sequence and before an IPL of the controlprogram.

problem analysis. The process of identifying thesource of a problem; for example, a programcomponent, a machine failure, telecommunicationfacilities, user or contractor-installed programs orequipment, an environment failure such as a powerloss, or a user error.

problem determination (PD). The process ofdetermining the source of a problem; for example, aprogram component, machine failure,telecommunication facilities, user or contractor-installedprograms or equipment, environmental failure such as apower loss, or user error.

problem management. The management disciplinethat handles a problem from its detection through itsfinal resolution. Problem management is composed ofthe following:

v Problem determination

v Problem diagnosis

v Problem bypass and recovery

v Problem resolution

v Problem tracking and control.

processor. (1) In a computer, a functional unit thatinterprets and executes instructions. A processorconsists of at least an instruction control unit and anarithmetic and logic unit. (T) (2) The functional unit thatinterprets and executes instructions. (3) The boundariesof a system, exclusive of I/O control units and devices,that can be controlled by a single operating system. Aprocessor consists of main storage, one or more centralprocessors, time-of-day clocks, and channels, whichare, or can be, placed in a single configuration. Aprocessor also includes channel subsystems, andexpanded storage where installed.

processor complex. A system configuration thatconsists of all the machines required for operation; forexample, a processor unit, a processor controller, asystem display, a service support display, and a powerand coolant distribution unit.

processor console. The workstation from which anoperator can control and observe system hardwareoperation. See also system console.

Processor Resource/Systems Manager (PR/SM)facility. The feature that allows the processor to useseveral system control programs (SCPs) simultaneously,provides logical partitioning capability for the realmachine, and provides support for multiple preferredguests.

profile. (1) A description of the characteristics of anentity to which access is controlled. (2) Data thatdescribes the significant characteristics of a user, agroup of users, or one or more computer resources.

program. Sequence of instructions for a computer. Aprogram interacts and relies on either the hardware orother programs.

program status word (PSW). An area in storage usedto indicate the sequence in which instructions areexecuted, and to hold and indicate the status of thecomputer system.

program temporary fix (PTF). A temporary solution orbypass of a problem diagnosed by IBM as resultingfrom a defect in a current, unaltered release of theprogram.

PR/SM. Processor Resource/Systems Manager.

PSC. Power Sequence Controller

PSP. preventive service planning.

PSW. Program status word.

PTF. Program temporary fix.

RRAS. reliability, availability, serviceability.

reconfiguration. (1) A change made to a givenconfiguration in a computer system; for example,isolating and bypassing a defective functional unit orconnecting two functional units by an alternative path.Reconfiguration is effected automatically or manuallyand can be used to maintain system integrity. (T) (2)The process of placing a processor unit, main storage,and channels offline for maintenance, and adding orremoving components. (3) Contrast with nondisruptiveinstallation and nondisruptive removal.

recovery. To maintain or regain system operation aftera failure occurs. Generally, to recover from a failure is to


Glossary E-11

identify the failed hardware, to deconfigure the failedhardware, and to continue or restart processing.

remote service facility (RSF). (1) A control programplus associated communication equipment that allowslocal personnel to connect to an IBM service center, andallows remote personnel to operate the remote systemor send new internal code fixes to it, if properlyauthorized. (2) A system facility invoked by LicensedInternal Code that provides procedures for problemdetermination and error detection.

Remote Technical Assistance and InformationNetwork (RETAIN). A database, accessible to servicerepresentatives, of information relating to IBM-installedproducts.

RETAIN. Remote Technical Assistance and InformationNetwork.

REXX. restructured extended executor language.

ring network. A network configuration in whichdevices are connected by unidirectional transmissionlinks to form a closed path.

Note: A ring of an IBM Token-Ring Network is referredto as a LAN segment or as a Token-RingNetwork segment.

RMF. Resource Measurement Facility.

RPQ. Request for Price Quotation.

RPS. rotational positional sensing/sensor.

RSA. Rivest-Shamir-Adelman

RSF. Remote support facility.

SSAD. system activity display.

SAP. system assist processor

SCP. system control program.

scroll. To move a display image vertically orhorizontally to display data that cannot be observedwithin the boundaries of the display frame.

scroll bar. A window component associated with ascrollable area that provides users a visual cue thatmore information is available and that the unseeninformation can be manipulated into view using themouse. Users scroll the information in the window byinteracting with the scroll bar.

SCSI. Small Computer System Interface

SDLC. See synchronous data link control.

SE. Support Element.

Server Time Protocol (STP). A message basedprotocol designed to enable multiple servers to maintaintime synchronization with each other. The timekeepinginformation is passed over data links (externally definedcoupling links) between servers. It provides timesynchronization for the z10 EC, z10 BC, z9 EC, z9 BC,z990, and z890 servers and CFs without requiring theSysplex Timer.

service representative. A person who performsmaintenance services for IBM hardware products orsystems. See also IBM program support representative.

SIE. Start Interpretive Execution

single point of control. The characteristic a ParallelSysplex displays when you can accomplish a given setof tasks from a single workstation, even if you needmultiple IBM and vendor products to accomplish thatparticular set of tasks.

single system image. The characteristic a productdisplays when multiple images of the product can beviewed and managed as one image.

SNA. See systems network architecture.

SNA/Management Services (SNA/MS). Functionsdistributed among network components to operate,manage, and control the network.

SNA/MS. SNA/Management Services.

SNA network. The part of a user-application networkthat conforms to the formats and protocols of SystemsNetwork Architecture. It enables reliable transfer of dataamong end-users and provides protocols for controllingthe resources of various network configurations. TheSNA network consists of network addressable units(NAUs), boundary function components, and the pathcontrol network.

SNMP. simple network management protocol.

STI. Self-Timed Interconnect

STI-MP (Self-Timed Interconnect Multiplexer). ForSystem z9, used for an I/O cage intraconnection.

STP. Server Time Protocol.

storage. (1) A functional unit into which data can beplaced, in which they can be retained, and from whichthey can be retrieved. (T) (2) The action of placing datainto a storage device. (I) (A)

structure. A construct used by MVS to map andmanage storage on a coupling facility. See cachestructure, list structure, and lock structure.

subchannel. In 370-XA, ESA/390 modes, andz/Architecture modes, the facility that provides all of theinformation necessary to start, control, and complete anI/O operation.



subchannel number. A system-unique 16-bit valueused to address a subchannel. See also channel pathidentifier, device identifier, and device number.

subsystem. A secondary or subordinate system, orprogramming support, usually capable of operatingindependently of or asynchronously with a controllingsystem. (T) See DASD subsystem. and storagesubsystem.

subsystem storage. Synonym for cache.

Support Element (SE). (1) an internal control elementof a processor that assists in many of the processoroperational functions. (2) A hardware unit that providescommunications, monitoring, and diagnostic functions toa central processor complex (CPC).

synchronous data link control (SDLC). A form ofcommunication line control that uses commands tocontrol data transfer over a communication line.

Sysplex Timer. An IBM unit that synchronizes thetime-of-day (TOD) clocks in multiple processors orprocessor sides. External Time Reference (ETR) is theMVS generic name for the IBM Sysplex Timer (9037).

system. Comprises the processor complex and allattached and configured I/O and communicationdevices.

system area. A logical area of central storage used tostore Licensed Internal Code and control information(not addressable by application programs).

Systems Network Architecture (SNA). Thedescription of the logical structure, formats, protocols,and operational sequences for transmitting informationunits through, and controlling the configuration andoperation of, networks.

S/370. IBM System/370™

S/390. IBM System/390®.

Ttarget logical partition. In LPAR mode, the targetlogical partition is the logical partition that is selected onthe System Monitor panel. It is the current or immediatelogical partition; the logical partition that the operator isworking with or looking at.

target processor. The processor that controlsexecution during a program restart, instruction trace,stand-alone dump, or IPL, and whose ID is identified byhighlighting on the status line.

TCP/IP. Transmission Control Protocol/InternetProtocol.

TDES. Triple Data Encryption Standard

time-of-day (TOD) clock. A system hardware featurethat is incremented once every microsecond, andprovides a consistent measure of elapsed time suitablefor indicating date and time. The TOD clock runsregardless of whether the processor is in a running,wait, or stopped state.

TKE. Trusted Key Entry

TOD. Time of day.

token. A sequence of bits passed from one device toanother on the token-ring network that signifiespermission to transmit over the network. It consists of astarting delimiter, an access control field, and an enddelimiter. The access control field contains a bit thatindicates to a receiving device that the token is ready toaccept information. If a device has data to send alongthe network, it appends the data to the token. Whendata is appended, the token then becomes a frame.

token-ring network. (1) A ring network that allowsunidirectional data transmission between data stations,by a token passing procedure, such that the transmitteddata return to the transmitting station. (T) (2) A networkthat uses ring topology, in which tokens are passed in acircuit from node to node. A node that is ready to sendcan capture the token and insert data for transmission.

Note: The IBM Token-Ring Network is a baseband LANwith a star-wired ring topology that passestokens from network adapter to network adapter.

TPF. Transaction processing facility.

transaction processing. In batch or remote batchprocessing, the processing of a job or job step. Ininteractive processing, an exchange between a terminaland another device that does a particular action; forexample, the entry of a customer's deposit and theupdating of the customer's balance.

TSCF. target system control facility.

UUCW. unit control word.

UPC. universal power controller.

UPS. uninterruptible power supply.

user interface. Hardware, software, or both thatallows a user to interact with and perform operations ona system, program, or device.

VVLAN. Virtual Local Area Network

VSE. Virtual Storage Extended.


Glossary E-13

WWAC. Wide Area Connector (card)

window. (1) An area of the screen with visibleboundaries through which information is displayed. Awindow can be smaller than or equal in size to thescreen. Windows can overlap on the screen and givethe appearance of one window being on top of another.(2) A choice in the action bar of some applications.Users select it to arrange the display of severalwindows or to change the active window. (3) A choice inthe action bar of multiple-document interfaceapplications. (4) A choice in an action bar that allowsthe user to arrange the display of all open windows andto change the active window. (5) A choice in an actionbar of multiple-document interface applications thatallows a user to arrange the display of all open windowsand to change the active window.

workstation. (1) A functional unit at which a userworks. A workstation often has some processingcapability. (T) (2) A terminal or microcomputer, usuallyone that is connected to a mainframe or network, atwhich a user can perform applications. (3) See alsodisplay station and terminal.

ZzIIP. System z9 Integrated Information Processor.

zAAP. System z9 Application Assist Processor. SeeApplication Assist Processor (AAP).

z800. ERserver zSeries 800



z9 BC. IBM System z9 Business Class server.

z9 EC. IBM System z9 Enterprise Class server.

Numerics370-XA. IBM System/370 extended architecture



Index

Numerics2501 Card Reader 5-233380 Direct Access Storage 5-233390 5-1964-bit addressing 6-7

AAAP (System z9 Application Assist Processor) 1-4AAP (zSeries Application Assist Processor) 2-5access

data deferred 5-23activation 9-8addressing

FCP 5-10network concentrator 5-31

API (Application Programming Interfaces) 9-7Application Assist Processor (zAAP) 1-4Application Programming Interfaces (API) 9-7architecture

z/Architecture 1-7assignments

CHPID 4-7PCHID 4-7

availability guidelines 4-4

BBackup Time Server 10-6Base Power Interface (BPI) 2-12basic floating point extensions 2-22benefits

Linux 3-4Bimodal Accommodation offering 3-2binary floating-point extensions 2-23block multiplexer

operation 5-13book

PCHID 4-9books 2-2

positions 2-3BPA (Bulk Power Assembly) 2-12BPF (Bulk Power Fan) 2-12BPI (Base Power Interface) 2-12buffered device 5-23Bulk Power Assembly (BPA) 2-12Bulk Power Fan (BPF) 2-12burst mode 5-14byte multiplexer

mode 5-14operation 5-14

Ccable ordering 8-2cabling

ESCON 8-2

cabling (continued)fiber optic 1-7FICON LX 8-2report 8-3responsibilities 8-3

Cabling 8-1cache hit rates 5-22cages

I/O 2-10capacity backup C-4Capacity Backup (CBU) 10-9Capacity Upgrade on Demand (CUoD) C-7capacity upgrades C-1cards

DCA 2-2DCA-CX 2-12DCA-X 2-9ETR 2-9ISC-D 6-3ISC-M 6-3memory 2-7OSC 2-2, 2-9PSC24V 2-12

cascaded directors 10-5CBU (Capacity Backup) 10-9CCC (Channel Control Check) 5-17CCW (Channel Command Word) 5-16CDC (Channel Data Check) 5-17Central Processor (CP) 2-4, 2-13Central Processor Complex (CPC) 2-2central storage 2-6certification

FIPS 7-4LPAR security 7-4

CF (Coupling Facility) 6-6CF Duplexing 6-8, 6-11CFCC (Coupling Facility Control Code) 6-6

48 internal tasks 6-864-bit addressing 6-7CF Duplexing 6-8Level 11 6-8Level 12 6-7Level 13 6-7Level 14 6-7Level 15 6-7message time ordering 6-8patch apply 6-7

chainingcheck 5-24command 5-16data 5-16, 5-24definition of 5-16operation 5-16type of 5-16

channelcommand word 5-16commands 5-15path definition 4-14


© Copyright IBM Corp. 2005, 2009 X-1

channel (continued)time out

active 5-18inactive 5-18

Channel Command Word (CCW) 5-16Channel Control Check (CCC) 5-17Channel Data Check (CDC) 5-17channel hardware

FCP 5-9channel sharing

FCP 5-11channel subsystem

planning 4-7Channel Subsystem (CSS) 2-13, 4-1, 4-2

performance 5-19performance characteristics, summary 5-19

channel-to-channel connection 5-18channels 2-14

dedicated 2-14ESCON 5-12HiperSockets 4-13Integrated Cluster Bus-3 (ICB-3) 6-4Integrated Cluster Bus-4 (ICB-4) 6-4Internal Coupling (IC) 4-13, 6-5InterSystem Coupling-3 (ISC-3) 6-3IOCP definitions 4-2maximums 4-1Peer 4-3reconfigurable 2-14shared 2-14, 2-15spanned 2-14, 4-13

characteristicschannel program 5-22channel subsystem workload 5-21device I/O 5-21ESCON performance 5-22

CHPIDassignments 4-7types 4-2

CHPID Mapping Tool 4-11CICS

availability 2-24translation isolation 2-24

CIU C-1process C-2

classesdevice 5-15, 5-24

clustering 6-15command

chaining 5-16deferred access 5-24

commentssending xviii

compatibilityprogramming 3-1

concurrentchannel upgrade 10-5maintenance

channels 10-6connectivity links 10-6cryptographic 10-6

concurrent (continued)maintenance (continued)

Hardware Management Console/SE 10-6OSA 10-6power 10-6

undo CBU 10-9, C-5configurations

ICB 4-6InfiniBand 4-6system 2-1

connectionCTC 5-17

connectivitysubchannel 4-3

considerationsCFCC 6-6coupling connection 6-8FICON 5-5HCD 4-15I/O configuration 6-8IOCP 4-14

controlcheck 5-17priority 5-18

converter operationESCON 5-13

Coupling Facility (CF) 6-6continuous availability 9-12duplexing 6-11

Coupling Facility Control Code (CFCC) 6-6coupling link

peer channels 4-3CP (Central Processor) 2-4, 2-13CP Assist for Cryptographic Function (CPACF) 1-4,

7-1CPACF (CP Assist for Cryptographic Function) 1-4,

7-1CPC (Central Processor Complex) 2-2critical time 5-14, 5-23, 5-24Crypto Express2 7-2cryptography 7-1

CPACF 7-1CSS (Channel Subsystem) 2-13, 4-1, 4-2CTC connection 5-18CUoD (Capacity Upgrade on Demand) C-7

Reserved CP support C-6Customer Initiated Upgrade (CIU) C-1customization 9-3

Ddata

chaining 5-16check 5-17compression 2-23deferred access 5-23representation 2-23streaming 5-25streaming protocol 5-13transfer rate 5-21


X-2 z9 EC System Overview

data rateschannel 5-21control unit 5-21device 5-21I/O subsystem 5-21

DCA (Distributed Converter Assembly) 2-2DCA-CX (Distributed Converter Assembly-CX) 2-12DCA-N (Distributed Converter Assembly-N) 2-9dedicated channels 2-14Dedicated Move Page 2-25deferred access 5-23

command 5-24definition

channel path 4-14I/O device 4-15logical partition 4-14

degradation 5-24degrade indicator 9-5device

buffered 5-23class 5-15, 5-24I/O 5-10, 5-14performance 5-14sharing 2-15

device sharing 2-14FCP 5-11

Distributed Converter Assembly (DCA) 2-2Distributed Converter Assembly-CX (DCA-CX) 2-12Distributed Converter Assembly-X (DCA-X) 2-9Dyamic Oscillator Switchover 1-4Dynamic CF Dispatching (DYNDISP) 10-4Dynamic Channel Path Management 6-16Dynamic I/O Configuration 2-24, 10-4dynamic ICF expansion 2-14, 10-3dynamic link aggregation 5-29dynamic reconnection 5-19dynamic SAP sparing 10-2dynamic storage reconfiguration 2-14

EECC (Error Checking and Correction) 2-8, 10-4elapsed time 5-23ending status 5-16Error Checking and Correction (ECC) 2-8Error Correction Code (ECC) 10-4error handling 5-16ESCON

channel commands 5-15channel multiplexing modes 5-13channels 5-12converter operation 5-13I/O operations control 5-15port sharing 10-5

Ethernet switch 2-17ETR (External Time Reference) 2-9, 10-5expanded storage 2-6External Time Reference (ETR) 2-9, 10-5, 10-6

FFast Sync/DataMover 2-25FCP (Fibre Channel Protocol) 5-7

addressing 5-10channel hardware 5-9channel sharing 5-11device sharing 5-11positioning 5-11

featuresI/O 2-10

fiber optic cabling 1-7Fiber Quick Connect (FQC) 8-2Fibre Channel Protocol (FCP) 5-7Fibre Channel Protocol for SCSI (FCP) 5-7FICON

bridge channels 5-5cascaded directors 10-5feature codes 5-4link data rate 5-5

FICON Express2 5-3FICON Express4 5-2floating-point

basic floating-point extensions 2-22binary floating-point extensions 2-23extensions 2-22floating-point-support (FPS) extensions 2-23hexadecimal floating-point (HFP) extensions 2-23

floating-point-support (FPS) extensions 2-23FQC (Fiber Quick Connect) 8-2frames

A 2-1Z 2-1

frames, system 2-1functions

Parallel Sysplex 6-1

GGDPS (Geographically Dispersed Parallel

Sysplex) 6-12Geographically Dispersed Parallel Sysplex

(GDPS) 6-12Graphically Dispersed Parallel Sysplex (GDPS) 10-9

HHardware Configuration Definition (HCD) 4-15, 9-6,

9-11Hardware Management Console 2-17, 9-1

availability 10-2capabilities 9-1function 9-1

Hardware Management Console and Support Elementcapabilities 9-2wiring options 9-8

Hardware Management Console Application(HWMCA) 9-3

hardware messages 9-4Hardware System Area (HSA) 2-7

estimation tool 2-8


Index X-3

HCA1-0 (Host Channel Adapter) 2-8HCD (Hardware Configuration Definition) 4-15, 9-6,

9-11HCD panels 5-1hexadecimal floating-point (HFP) extensions 2-23highlights 1-2HiperSockets 1-4, 2-12, 4-13, 5-30

network concentrator 5-31Host Channel Adapter (HCA1-0) 2-8HSA (Hardware System Area) 2-7HSA Estimator B-2hub 2-17HWMCA (Hardware Management Console

Application) 9-3

II/O

cages 2-10layout 4-5PCHID 4-9

configuration 9-11device 5-10, 5-14device characteristics 5-21device definition 4-15features 2-10interface mode 5-13interface protocol 5-17interruptions 5-16

CCC 5-16CDC 5-16IFCC 5-16

operations control 5-15performance 5-14priority queuing 6-15Subsystem Priority Queuing 6-15system reset 5-17

I/O Priority Queuing (IOPQ) 6-15I/O Subsystem (IOSS) 4-2IBF (Internal Battery Feature) 2-18

hold up times 2-19IBM Resource Link B-1ICB-3 (Integrated Cluster Bus-3) 6-3ICB-4 (Integrated Cluster Bus-4) 6-3, 6-4ICF (Internal Coupling Facility) 2-4, 2-13IFA (Integrated Facility for Applications) 1-4, 2-5IFB (InfiniBand) links 6-3, 6-5IFCC (Interface Control Check) 5-17IFL (Integrated Facility for Linux) 2-13, 3-4Immediate and Relative Instruction facility 2-25InfiniBand

configurations 4-6instructions

Cancel I/O (XSCH) 5-16Clear Subchannel (CSCH) 5-16Halt Subchannel (HSCH) 5-16Modify Subchannel (MSCH) 5-16Reset Channel Path (RCHP) 5-16Resume Subchannel (RSCH) 5-16Set Channel Monitor (SCHM) 5-16SIE (start interpretive execution) 3-1

instructions (continued)Start Subchannel (SSCH) 5-16Store Report Word (STCRW) 5-16Store Subchannel (STSCH) 5-16Test Pending Interruption (TPI) 5-16Test Subchannel (TSCH) 5-16

Integrated Cluster Bus-3 (ICB-3)channels 6-4

Integrated Cluster Bus-4 (ICB-4) 6-4Integrated Facility for Applications (IFA) 1-4, 2-5Integrated Facility for Linux (IFL) 2-13, 3-4Intelligent Resource Director (IRD) 6-14interface

protocol, I/O 5-13Interface Control Check (IFCC) 5-17interface mode

data streaming 5-13interlocked 5-13

interlockedmode 5-25protocol 5-13

Internal Battery Feature (IBF) 2-18Internal Coupling (IC)

channels 4-13, 6-5Internal Coupling Facility (ICF) 2-4, 2-13interruptions

control check 5-16data check 5-16I/O 5-16interface control check 5-16machine check 5-16

InterSystem Coupling-3 (ISC-3) channels 6-3IOCP

channel definitions 4-2considerations 4-14

IOPQ (I/O Priority Queuing) 6-15IOSS (I/O Subsystem) 4-2iQDIO (internal Queued Direct Input/Output) 5-30IRD (Intelligent Resource Director) 6-14ISC-3 (InterSystem Coupling-3) 6-3ISC-D 6-3ISC-M 6-3

Kkey-controlled storage protection 2-6

LLarge Send 5-28Layer 2 (Link Layer) 5-28Layer 3 Virtual MAC 5-29LCSS (Logical Channel Subsystem) 4-1Licensed Internal Code (LIC) 9-6link data rate

FICON 5-5links

IC 6-5ICB-2 6-4ICB-3 6-4ICB-4 6-4



links (continued)IFB 6-5ISC-3 6-3

Linux 3-4benefits 3-4VLAN support 5-30

logicalpartition definition 4-14string assist 2-25

Logical Channel Subsystem (LCSS) 4-1logical partition (LPAR)

increased 1-3operating mode 2-13

logically partitioned operating mode 2-13LPAR (logical partition)

clustering 6-15mode 2-13

MMachine Information B-2machine-check interruptions 5-17MAU (Multistation Access Unit) 2-16maximums

channel 4-1MBA (Memory Bus Adapter) 2-8MCM (Multichip Module) 2-4MCU (Modular Cooling Unit) 2-12MDA (Motor Drive Assembly 2-12memory 2-5

central storage 2-6expanded storage 2-6rules 2-7

Memory Bus Adapter (MBA) 2-8failure containment 10-2

memory cardscharacteristics 2-7

memory scrubbing 10-4message time ordering 6-8, 6-11MIDAW facility 5-1MIF (Multiple Image Facility) 4-12model capacity indicator 1-6models

S08 1-6S18 1-6S28 1-6S38 1-6S54 1-6structure of 1-5z990 2-4

modesburst 5-14byte multiplexer 5-14channel multiplexing 5-13interlocked 5-25LPAR 2-13operation 5-15peer 6-3

Modified Indirect Data Address Word (MIDAW) 5-1Modular Cooling Unit (MCU) 2-12Modular Refrigeration Unit (MRU) 2-12

Motor Drive Assembly (MDA) 2-12Motor Scroll Assembly (MSA) 2-12MRU (Modular Refrigeration Unit) 2-12MSA (Motor Scroll Assembly) 2-12Multichip Module (MCM) 2-4multipath IPL 5-2, 5-13Multiple Image Facility (MIF) 4-12

Nnetwork concentrator 5-31nonsynchronous operation 5-21

OOn/Off CoD (On/Off Capacity on Demand) C-3online information xviiiOpen Systems Adapter (OSA) 5-26Open Systems Adapter-Express (OSA-Express)

features 5-26operating system messages 9-5operation

block multiplexer 5-13byte multiplexer 5-14chaining 5-16ESA/390 5-15modes 2-13nonsynchronous 5-21retry 5-16synchronous 5-20

OSA (Open Systems Adapter) 5-26features 5-26

OSA LAN idle timer 5-29OSA Support Facility (OSA/SF) 5-26OSA-Express (Open Systems Adapter-Express) 5-26OSA-Express Network Traffic Analyzer Trace

facility 5-29OSA-Express2 5-27OSA/SF (OSA Support Facility) 5-26OSC (oscillator) cards 2-2, 2-9oscillator (OSC) cards 2-2, 2-9

PParallel Sysplex 6-1

coupling link connectivity 6-2partial memory restart 10-4patch apply 6-7PCHID

assignments 4-7book 4-9I/O cage 4-9report, sample 4-10

peer channels 4-3peer mode 6-3Perform Locked Operation (PLO) 2-25performance 1-6

device 5-14ESCON channel 5-19ESCON characteristics 5-22I/O 5-14


Index X-5

permanent upgrades C-2planning

channel subsystem 4-7PLO (Perform Locked Operation) 2-25positioning

FCP 5-11positions

books 2-3Power Estimation tool B-2power sequence control (PSC) 2-19PR/SM (Processor Resource/Systems Manager) 3-4,

6-14PR/SM LPAR

CPU Management (Clustering) 6-15Timer Offset 2-15

Preferred Time Server 10-6priority

control 5-18Priority Queuing 6-15problem analysis and reporting 9-5Processor Resource/Systems Manager (PR/SM) 2-13,

3-4, 6-14processor unit (PU) 2-4, 2-13

sparing 10-2programming

compatibility 3-1support 3-1

protocolI/O

data streaming 5-13interlocked 5-13

interlocked 5-13PSC (power sequence control) 2-19PSC24V card 2-12PU (processor unit) 2-4, 2-13publications

Cryptographic Coprocessor Feature xviimiscellaneous xviiOSA-Express xviiparallel sysplex xviprerequisite xvrelated xv

purge path extended 5-6

QQueued Direct I/O Diagnostic Synchronization

(QDIOSYNC) 5-29

RRAS

availability 10-1application preservation 10-3Backup Time Server 10-6CBU 10-9CBU smart reactivation 10-9CF Duplexing 10-7CHPID Mapping Tool 10-10concurrent channel upgrade 10-5concurrent hardware maintenance 10-6

RAS (continued)availability (continued)

concurrent LIC patch enhancements 10-6concurrent model upgrade 10-1concurrent undo CBU 10-9CUoD 10-9dual power feeds 10-5dynamic CF dispatching 10-4dynamic I/O configuration 10-4dynamic ICF expansion 10-3ECC 10-4Electronic Service Agent 10-7ESCON port sparing 10-5fault tolerant interconnect design 10-1FCP full-fabric connectivity 10-5fiber optic cabling 10-9FICON cascaded directors 10-5GDPS 10-9Hardware Management Console (HMC) 10-2IBF 10-7LPAR dynamic storage reconfiguration 10-4maintenance/upgrade for coupling 10-5MBA failure containment 10-2memory scrubbing 10-4memory sparing 10-4OSC and ETR cards 10-5partial memory restart 10-4Preferred Time Server 10-6processor design 10-2PU sparing 10-2redundant coupling links 10-7redundant flexible service processors 10-5redundant power and thermal subsystems 10-5sparing for storage protect keys 10-1subsystem storage protect 10-4Support Element (SE) 10-2two SAP standard 10-3

reliability 10-1serviceability 10-10

RAS (Reliability, Availability, Serviceability) 10-1reconfigurable channels 2-14reconnection, dynamic 5-19related

publications xvweb sites xvii

remote automated operations 9-15Remote Keyload 1-4remote operations 9-13

manual 9-14using a Hardware Management Console 9-14using a web browser 9-15

overview 9-13Remote Support Facility (RSF) 9-7reports

cabling 8-3PCHID 4-10

reset, system 5-17Resource Link 1-7retry

operations 5-16RMF Monitoring 7-3



RSAavailability

On/Off CoD 10-8RSF (Remote Support Facility) 9-7rules

memory 2-7

SS08 1-6S18 1-6S28 1-6S38 1-6S54 1-6sample

PCHID report 4-10SAP (System Assist Processor) 2-5, 10-2scalability 1-3scheduled operations 9-6SCSI (Small Computer Systems Interface) 5-7SE (Support Element) 2-16, 9-1security 9-10

CPACF 7-1cryptographic coprocessor 7-1

Self-Timed Interconnect (STI)STI-3 6-4

sending comments xviiiServer Time Protocol (STP)

description 2-16, 6-9service required state 9-4shared channels 2-14SIE (start interpretive execution) instruction 3-1Small Computer Systems Interface (SCSI) 5-7software

supportLinux 3-4VSE/ESA 3-3z/OS 3-2z/OS.e 3-2

spanned channels 2-14, 4-13status reporting 9-3STI (Self-Timed Interconnect)

configurations 4-6STI-3 card 6-4storage 2-14

central 2-6expanded 2-6z/Architecture 2-6

subchannelconnectivity 4-3

Subspace Group Facility support 2-24support

broadcast 5-30operating systems 3-1software 3-2

Support Element (SE) 2-16, 9-1synchronous operation 5-20sysplex functions

parallel 6-1system

configurations 2-1

system (continued)I/O reset 5-17

System Assist Processor (SAP) 2-5, 10-2system frames

Internal Battery Feature (IBF) 2-1system power supply 2-12System Time Protocol (STP)

availability 10-6System-Managed Coupling Facility Structure Duplexing

(CF Duplexing) 6-11

Ttime-out functions

channelactive 5-18inactive 5-18

Timer Offset 2-15timing-only links 4-3TKE (Trusted Key Entry) 7-3tools

CHPID mapping 4-11translation isolation 2-24Trusted Key Entry (TKE) 7-3

Uunsupported features 1-8upgrade progression 1-8upgrades

non-disruptive C-6permanent C-2

Vvirtual RETAIN 9-5VSE/ESA 3-3

Wweb sites xviiWorkload Manager 6-16

Zz/Architecture 1-7, 2-20

enhancements 1-7z/OS 3-2z/OS.e 3-2z/VM

dynamic link aggregation 5-29support 3-3

z990books 2-2memory 2-5models 2-4

zAAP (System z9 Application Assist Processor) 1-4zAAP (zSeries Application Assist Processor) 2-5zSeries Application Assist Processor (zAAP) 2-5


Index X-7

��

Level 06a, April 2009Printed in USA

SA22-6833-06

z9 EC System Overview - IBM

Documents

Transcript of z9 EC System Overview - IBM