iCluster Manual End-User 5.1

IBM® DataMirror iCluster®

Version 5.1

End-User Documentation

Table of Contents

Table of ContentsiCluster Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17

iCluster Enterprise Edition and iCluster SMB Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17Installation and Upgrade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19

Before You Install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19Minimum Requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19

Hardware Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Software Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Communication Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Program Temporary Fix Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19Authorization Codes and Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20Required Information for the Installation Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20Creating a Journal Receiver and Journal QAUDJRN in QSYS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20Setting the QALWUSRDMN OS/400 System Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21Installing, Re-installing, and Upgrading iCluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21Re-installing iCluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21Applying iCluster Updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21

Installing, Re-Installing, or Upgrading iCluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23Overview of the Installation Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23iCluster Enterprise Edition and iCluster SMB Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23Installing the iBalance Feature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23Signing on to your System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24Preparing for an Upgrade or Reinstallation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24Restoring the Installation Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25

To Restore Objects from CD-ROM Device . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25To Restore Objects from Save File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Running the Installation Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25Upgrading iCluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26

After You Install . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29Starting the XDMCLUSTER Subsystem on IPL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29Performance Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29Adding an iCluster User (Optional) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29Securing the iCluster Product Library (Optional) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30Scheduling iCluster Startup and Shutdown (Optional) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30Adding Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31

Uninstalling iCluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33Uninstalling iCluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33

Migrating HA Suite to iCluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35Introduction to the Migration Procedures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35

© DataMirror, an IBM Company Printed in Canada 3

Table of Contents

General Constraints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35Pre-migration Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35Running the Migration Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35Post-migration Tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36

Running iCluster Under TCP/IP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37Adding a New Entry 'dmcluster' to the TCP/IP Service Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37

Method 1 - Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37Method 2 - Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Restricting the Port Number from Being Used by Other Applications . . . . . . . . . . . . . . . . . . . . . . . . . . .37Method 1 - Menu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37Method 2 - Command. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Verifying Domain Name Server (DNS) Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38Removing the iCluster Service Table Entries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39

Removing the dmcluster Table Entry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39Signing on to your System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Removing dmcluster from the TCP/IP Service Table. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Removing Port Number Restrictions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39Other Considerations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41

Additional Message Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41Batch Processes and Job Logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41

iCluster Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43Working in a Clustered Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43

Nodes in a Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43Replication Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44Failover Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Failovers and Switchovers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .47Scenario 1: SOS Cannot Communicate with the Primary Node . . . . . . . . . . . . . . . . . . . . . . . . . . 49Scenario 2: IBM CRS Detects a Distress Signal from the Primary Node. . . . . . . . . . . . . . . . . . . . 49Scenario 3: IBM CRS Loses Communication with the Primary Node . . . . . . . . . . . . . . . . . . . . . . 50

Object Specifiers and Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50Constructing Object Specifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51Path Object Specifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52Generic Object Specifiers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52Rules of Precedence for Object Specifiers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52Native Object Specifier Changes While Replication is Active . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Path Object Specifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53Journaled and Non-journaled Path Object Specifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

iCluster Quick Start . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55Auto-configuration of iCluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55

DMAUTOCFG—Auto-configure iCluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55

4 Printed in Canada iCluster—Version 5.1

Table of Contents

Quick Start for the iCluster Command Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .57Starting iCluster from the Command Line . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .58

To invoke iCluster from the command line: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59Quick Start for iCluster Administrator for Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .59Logging on to iCluster Administrator for Windows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60

To log on to iCluster Administrator for Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60iCluster Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63

Command Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .63Operating System Authority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63Commands Available to any OS/400 User. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63Commands Available to OS/400 Security Officers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64Commands Available to *ALLOBJ Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65iCluster Authority . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65iCluster User Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65iCluster Operator Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65iCluster Administrator Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

Adding and Configuring Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67Removing Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

Initial Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .68Application Resiliency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .68Staging Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .69Locked Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .70Avoiding Contention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .70Commitment Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .71Suspended Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .72Choosing a Failover Mechanism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .73

Key Differences in Failover Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74Configuring iCluster to use SwitchOver System (SOS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74Configuring iCluster to use IBM Cluster Resource Services (CRS) . . . . . . . . . . . . . . . . . . . . . . . . 75

Starting Replication and Journal Positions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .75To start replication after setting journal positions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Monitoring Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76Monitoring Out-of-Sync Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77

To start a Sync Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77To view Sync Check results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Monitoring Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .78Restarting iCluster after Restarting a Node. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .79Upgrading the Cluster Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .80Changing IP Addresses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .80Journaling in iCluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .81

Remote Journaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83


Table of Contents

Configuring Remote Journaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84Role Switching with Remote Journals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

Passing Arguments to Sync Point User Exit Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .86iCluster Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .87

Replicating Database *FILE Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .87Replicating BSF Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .89Replicating Configuration Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .90Replicating User Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91Replicating LOBs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .92Replicating Triggers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .92Replicating Save Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .92Replicating QDLS Objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .94Replicating WebSphere MQ Objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .95

To enable WebSphere MQ support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96iCluster Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .97

Node Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .99DMADDNODE—Add Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .99DMDSPNODE—Display Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .103DMCHGNODE—Change Node. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .104DMRMVNODE—Remove Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .109DMWRKNODE—Work with Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .110

Group Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113DMADDGRP—Add Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .113DMDSPGRP—Display Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .124DMCHGGRP—Change Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .124DMRMVGROUP—Remove Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .134DMADDBACK—Add Backup Node to Recovery Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .135DMRMVBACK—Remove Backup Node from Recovery Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . .135DMCHGROLE—Change Group Primary Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .136DMWRKGRP—Work with Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .137

Native AS/400 Object Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .141DMSELOBJ—Select Objects to Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .141DMCHGOBJSL—Change Object Selection to Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .147DMDSELOBJ—De-select Objects from Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .151DMWRKOBJ—Work with Object Specifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .152

Byte Stream File (BSF) Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .155DMADDBSF—Add Path Object Specifier to Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .155DMRMVBSF—Remove Path Object Specifier to Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .158DMCHGBSF—Change Path Object Specifier to Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .159DMWRKBSF—Work with Path Object Specifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .162

Switchable Device Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .163


Table of Contents

DMADDSWDEV—Add Switchable Device Entry to Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .163DMDSPASPGP—Display Switchable Device Entry to Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .164DMCHGSWDEV—Change Switchable Device Entry for Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .164DMRMVSWDEV—Remove Switchable Device Entry for Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .165

Resilient Application Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .167DMADDAPP—Add Resilient Application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .167DMDSPAPPGP—Display Resilient Application Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .170DMUPDAPP—Update Resilient Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .172DMCHGAPP—Change Resilient Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .172DMRMVAPP—Remove Resilient Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .175DMADDBACK—Add Backup Node to Recovery Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .176DMRMVBACK—Remove Backup Node from Recovery Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . .176DMWRKAPP—Work with Resilient Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .177

MQSeries Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .179RBDHAMQM—Rebuild iCluster MQSeries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .179

Administration Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .181DMSETSVAL—Set Cluster System Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .181DMCHGTIME—Change System Date and Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .193DMDSPLOG—Display Cluster Event Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .195DMCLRLOG—Clear Cluster Event Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .199HAPNGTCP—Ping Using TCP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .200JRNHADADQ—Journal Data Areas and Data Queues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .201STRHATCP—Start TCP/IP Listener . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .202WRKHAJOB—Work with Active Cluster Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .203DMSTRDM—Start Definition Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .203DMDLTCLSTR—Delete Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .203DMSYSINF—System Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .204DMWRKSSPLF—Work with Suspended Spooled Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .205DMACTSPLF—Activate Spooled File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .205

Cluster Operation Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .207DMSTRNODE—Start Cluster Operations at Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .207DMENDNODE—End Cluster Operations at Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .208DMSTRGRP—Start Cluster Operations at Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .208DMSTRREPL—Start Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .211DMENDGRP—End Cluster Operations for Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .213DMSTRWO—Switchover Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .215DMREJOIN—iCluster Rejoin Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .216DMSTRAPP—Start Cluster Operations for Resilient Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . .217DMENDAPP—End Cluster Operations for Resilient Application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .218DMSWOAPP—Switchover Resilient Application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .220


Table of Contents

DMSETPRIM—Prepare Primary Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .220DMCHGROLE—Change Group Primary Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .221DMGENEXC—Generate Command Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .222

Replication Operation Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .225DMSETPOS—Set Journal Start Position . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .225DMMRKPOS—Mark Current Journal Positions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .228CHGHAJRN—Change Journal Receiver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .229DSPHAPOS—Display Journal Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .230RTVHAPOS—Retrieve Journal Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .230VFYHAJRN—Verify Audit Journal. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .232STRHABRCD—Start BSF Recording . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .233DSPHABRCD—Display Recording for BSF Object. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .234ENDHABRCD—End Recording for BSF Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .234DMSTRAPY—Start Replication Apply Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .235DMENDAPY—End Replication Apply Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .236DMACTOBJ—Activate Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .238DMACTBSF—Activate BSF Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .240DMSUSOBJ—Suspend Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .241DMSUSBSF—Suspend BSF Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .242DMSNDOBJ—Send Object Immediately. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .243DMSETSYNC—Set Sync Point. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .245CRTCFGOBJ—Create Configuration Objects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .248INITHAOBJ—Initialize Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .249RTVRCVPT—Retrieve Recovery Checkpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .250RTVRCVPTR—Retrieve Recovery Checkpoint (CL Program) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .251SYNCHATRG—Synchronize Triggers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .252DMLOGENT—Log Journal Entry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .253

Status and History Monitor Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .255DSPHASMON—Display Source Monitor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .255WRKHASMON—Work with Status Monitor on Primary Node. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .255WRKHATMON—Work with Status Monitor on Backup Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .256CHGHASMON—Change History Monitor on Primary Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .257PRGHASMON—Purge History Monitory on Primary Node. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .259WRKHAOBJST—Work with Object Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .260WRKHABSFST—Work with BSF Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .260DMWRKOBJST—Work with Object Status by Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .261

Sync Check Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .263DSPHASC—Display Sync Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .263

*FILEATTR Sync Check Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263*LIST Sync Check Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264


Table of Contents

*DTAARA Sync Check Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264*BSF Sync Check Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265

SELSCATTR—Select Sync Check Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .265STRHASC—Start Sync Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .267STRHASCUSR—Start User Sync Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .270DMSCRPT—Sync Check Report for IFS Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .274DMSCRPTNTV—Sync Check Report for Native Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .275PRGHASC—Purge Sync Check Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .276

Registered iCluster User Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .277DMADDUSR—Add User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .277DMCHGUSR—Change User. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .278DMRMVUSR—Remove User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .280DMWRKUSR—Work with Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .281

Exit Program Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .283ADDPFEXPGM—Add Exit Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .283RMVPFEXPGM—Remove Exit Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .283

iCluster for Supported External Storage Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .285DMRBDNODE—Rebuild Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .285DMREGPOS—Register Positions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .286

Other Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .289SETHAREG—Restore Communications Registries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .289

Using the Status Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .291Working with the Status Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .291Simplified Status Monitor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .292Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .292Status of Apply Jobs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .292Real Time and Historical Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .292Interactive Inquiry Screens . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .292Real Time Statistics Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .293

Starting the Status Monitor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293Cycling Through the Views. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293Common Information on all Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294Common Options for all Views . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295Monitoring Real Time Overall Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297Monitoring Real Time Object Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299Monitoring Real Time Object Position and Totals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301Monitoring Real Time Object Throughput . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302Reading Status Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303

Working with Object Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .306Working with BSF Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310Working with Object Status for Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312


Table of Contents

Monitoring Historical Statistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .314Monitoring Historical Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314Monitoring Historical Object Position and Totals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316Monitoring Historical Object Throughput . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318

iBalance and Master-Master Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .321Configuration of Master-Master Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .322

To configure master-master replication:. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322Master-Master Replication—Operations and Monitoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .323Overview of Conflict Detection and Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .323Configuration of Conflict Detection and Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .325Parameter List Descriptions for a Conflict Resolution User Exit Program. . . . . . . . . . . . . . . . . . . . . . .326

Parameters Passed to the User Exit Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326Image Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328LOB Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329Control Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329Field Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335

Conflict Logging File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .337High Availability Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .340

Switchover for IBM Cluster Resource Services (CRS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .341iSeries Clustering Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .341

iSeries Clustering Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .341iCluster Concepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .342

iCluster and OS/400 Cluster Resource Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .342iCluster Groups and Role Switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .342

ROLESWITCH Parameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343What iCluster Does During a Switchover and Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343

Switchover Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .344Failover Overiew . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .344

Failovers and the ROLESWITCH parameter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .344ROLESWITCH *NO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .344ROLESWITCH *YES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .345

Preparing the Cluster to Handle Switchovers and Failovers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .345ROLESWITCH Parameter. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .345User Exits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .345Journal Entry Processing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .345Synchronous and Asynchronous Remote Journaling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .346Failover Message Queue . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .347Enabling Alerts in OS/400 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .347Line Controller Heartbeat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .347

Passing Arguments to Role Switch User Exit Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .347Failed State Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .348


Table of Contents

Node Failure Detection in iCluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .348FAILED State Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .348Detecting a FAILED State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .349Group Failover in iCluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .349

Failed State Recovery Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .350Recovery Paths for a FAILED State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .351

Partition State Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .355False and True PARTITIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .355

Partition State Recovery Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .356Recovery Paths for a PARTITION State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .357

Restarting Active Groups with Mirroring in the Original Direction with ROLESWITCH *NO . . . . . . . . . . .359Restarting Active Groups with Mirroring in the Original Direction with ROLESWITCH *YES . . . . . . . . . .360Restarting Active Groups with Mirroring in the Opposite Direction with ROLESWITCH *YES . . . . . . . . .361Restarting Active Groups with Mirroring in the Opposite Direction with ROLESWITCH *NO . . . . . . . . . .361Rejoining a Non-Functioning Node with a Role Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .362Rejoining a Non-Functioning Node without a Role Switch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .363Removing a Failed Node from the Cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .364iCluster and Resilient Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .364iCluster and Resilient Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .365Alertable Messages for Clustering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .366Switching Over to Another Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .367

Switchover for SwitchOver System (SOS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .369SwitchOver System Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .369

SwitchOver System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .369User Actions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .369Detecting Node Failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .370Node Failure Walkthroughs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .370

Manual Failover Walkthrough. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372Automatic Failover Walkthrough . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374

Configuring Operational Switching. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .376Operational Switching Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .376Defining User Exit Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .376Sample User Exits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .378Configuring Nodes for Operational Switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .379Configuring Groups for Operational Switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .380

Administering Operational Switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .381Performing a Manual Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .381

To perform a manual failover: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381Recovering from Failovers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .381

To restore a group from this state: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382Performing a Switchover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .382


Table of Contents

To perform a switchover: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382iCluster Administrator for Windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .383

Minimum Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .383Hardware Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .383Software Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .383

Installing iCluster Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .383Upgrading iCluster Administrator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .383Uninstalling iCluster Administrator. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .383

iCluster Administrator and Event Viewer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .384Monitoring Operations in the iCluster Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .385

Cluster Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .385Setting System Values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .386

To set system values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386Cluster Operation Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .395

Rejoin Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .395To restart node operations so that it rejoins the cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395

Delete Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .396To end cluster operations and delete cluster definitions on all active nodes . . . . . . . . . . . . . . . . 396

Node Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .396Working with Nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .397Add Node. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .397

To add a node to the cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397Remove Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .402

To remove a node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402Start Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .403

To start cluster operations at the specified node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403End Node. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .403

To end cluster operations at the specified node. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404Change Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .404

To change the specified node in the cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404Send Object. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .410

To replicate one or more objects to a target node:. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411Group Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .412

Working with Replication Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .413Add Full Replication Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .413

To add a full replication group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414Start Full Replication Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .421

To start cluster operations for a full replication group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421End Full Replication Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .423

To end cluster operations for a full replication group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423Remove Full Replication Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .424


Table of Contents

To remove a full replication group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424Change Full Replication Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .424

To change a full replication group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424Convert to Refresh Only Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .431

To convert a full replication group to a refresh-only group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431Switchover Full Replication Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .433

To initiate a switchover for a full replication group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434Add Refresh-Only Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .434

To add a refresh-only group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434Start Refresh-Only Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .437

To start a refresh-only group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437End Refresh-Only Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .437

To end a refresh-only group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438Remove Refresh-Only Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .438

To remove a refresh-only group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438Change Refresh-Only Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .439

To change a refresh-only group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439Convert to a Full Replication Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .441

To convert a refresh-only group to a full replication group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441Add Master-Master Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .448

To add a master-master group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448Start Master-Master Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .452

To start a master-master group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452End Master-Master Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .454

To end a master-master group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454Remove Master-Master Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .455

To remove a master-master group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455Change Master-Master Group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .455

To change a master-master group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 456Resilient Application Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .459

Working with Resilient Applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .459Add Resilient Application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .459

To add a resilient application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460Change Resilient Application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .462

To change a resilient application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 463Update Resilient Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .465

To update a resilient application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465Remove Resilient Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .465

To remove a resilient application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466Start Resilient Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .466


Table of Contents

To start a resilient application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 466End Resilient Application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .468

To end a resilient application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468Switchover Resilient Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .469

To switchover a resilient application. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469MQSeries Group Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .470

Working with MQSeries Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .470Add MQSeries Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .471

To add an MQSeries group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 471Start MQSeries Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .476

To start an MQSeries Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476End MQSeries Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .478

To end an MQSeries Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478Remove MQSeries Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .478

To remove an MQSeries Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479Change MQSeries Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .479

To change an MQSeries group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 479Switchover MQSeries Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .483

To switchover an MQSeries group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483Working with Recovery Domains . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .484

Add Backup Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .484To add a backup node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 484

Remove Backup Node. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .485To remove the backup node. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485

Working with Object Specifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .486Select/Add Object Specifier. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .486

To select/add an object specifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486Deselect Object Specifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .492

To deselect an object specifier. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492Change Object Specifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .492

To change an object specifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 493iCluster Operations Commands. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .496

Mark Journal Positions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .496To mark current journal positions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 496

Set Journal Positions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .497To set journal positions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497

Set Synchronization Point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .498To set a synchronization point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499

Set Primary Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .502To set a primary node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502


Table of Contents

Change Role . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .502To change the role of the primary node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502

Start Apply Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .503To start the apply process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503

End Apply Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .504To end the apply process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504

Suspended Object Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .505Activate Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .505

To activate an object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 506Suspend Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .507

To suspend an object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 508iBalance and Master-Master Replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .508

Configuring Master-Master Replication in iCluster Administrator . . . . . . . . . . . . . . . . . . . . . . . . . . . . .509To configure master-master replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509

Viewing Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .510Starting the iCluster Event Viewer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .510

To start the iCluster Event Viewer:. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 511Change Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .512

To change the event log filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 512Export Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .514

To export the event log. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514Clear Log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .515

To clear the event log. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 515Set Message Frame Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .516

To set the message Frame Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516Detailed Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .516

Monitoring iCluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .516Using the Backup Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .516

To use the Backup Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517Using the Object Status Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .518

To activate or suspend objects from the Object Status Monitor. . . . . . . . . . . . . . . . . . . . . . . . . . 519Using the iCluster Performance Monitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .520

To launch the Performance Monitor for a group. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 520Performance Monitor Display . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .520

Source Status and Latency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523Updating the Performance Monitor Display. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .526

To update the performance monitor display. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 526Object Types Replicated by iCluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .529iCluster Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .533iCluster Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .535

The Apply Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .535


Table of Contents

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .537Contacting DataMirror . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .547

Troubleshooting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .547Contacting Technical Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .547Training and Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .547Send us your Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .547

Copyright Notice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .549


iCluster Enterprise Edition and iCluster SMB Edition

iCluster OverviewiCluster is a replication solution that allows you to link computers together for a continuously available system. This creates a system that can continue to operate and provide services during anticipated or unexpected interruptions. iCluster can be installed on distributed systems to maintain operations after recovering from planned (switchover) or unplanned (failover) interruptions. It is designed for organizations that have the necessary hardware and communications to offload processing to other iSeries systems defined in the cluster. A cluster administrator will use the iCluster commands to define a configuration that supports high availability in a distributed iSeries environment. iCluster users that are defined by the administrator can perform certain iCluster operations based on the level of authority assigned to each user.iCluster provides the functionality that is required to support high availability in an iSeries clustered environment. A cluster consists of a network of iSeries computers (or nodes) that work together to provide seamless iSeries operations. If one node in the cluster that provides a service can no longer perform this role, operations can be automatically switched to a designated backup node. From outside of the cluster, there is no distinction as to which node in the cluster is actually performing these operations. After a switchover or failover, resilient applications behave as though operations are being provided by the same node in the cluster. To achieve this objective, you must replicate objects and data from the primary node to the backup node so that operations can be immediately moved to this node when a switchover or failover occurs. For more information on failovers and switchovers, see Failovers and Switchovers on page 47. See Choosing a Failover Mechanism on page 73 for more information on the available failover mechanisms in iCluster.DataMirror iCluster provides a set of commands and a Java-based workstation application that allows you to define cluster components and initiate replication activities within the cluster. In addition to defining nodes within a cluster, you are also able to define replication groups, initiate cluster operations, define resilient applications, and set cluster security levels.See Working in a Clustered Environment on page 43 for more information on iCluster.This section contains the following topics:• iCluster Enterprise Edition and iCluster SMB Edition on page 17

iCluster Enterprise Edition and iCluster SMB EditionDepending on your business requirements, you can choose between the following two versions of iCluster: • iCluster Enterprise Edition• iCluster SMB (Small and Medium Business) Edition.iCluster Enterprise Edition is appropriate for complex iSeries environments that require richer configuration options. More flexibility is provided for groups and the ability to deal with them independently.iCluster SMB Edition is appropriate for less complicated environments with fewer applications, data, groups, and journals. Configuration is simpler for ease of set up and administration. It is limited to two replication groups with two data journals per group.The iBalance feature for workload balancing is available for both versions of iCluster. For more information on iBalance, see iBalance and Master-Master Replication on page 321.For more information on installing these editions of iCluster, see Installation and Upgrade on page 19.


Before You Install

Installation and UpgradeThis section describes how to install and configure iCluster on an iSeries node.

Before You InstallThis section identifies the information and parameters that you need to know before installing, re-installing, or upgrading iCluster on an iSeries node.

Minimum RequirementsThe following sections list minimum hardware and software requirements for iCluster.

Hardware RequirementsDisk Space:• 200 megabytes (for each installation of iCluster)• 90 megabytes (recommended for running iCluster) • 512 megabytes (minimum for the staging store)Note: It is recommended that additional disk space be allocated to accommodate large save files.

Software RequirementsOperating System:• OS/400 Version 5 Release 1 and greaterNote: At the time of this release, iCluster supports V5R1 to V5R4 of the operating system. To install iCluster on a

different version of the operating system, contact DataMirror Technical Support. See Contacting Technical Support on page 547 for more information.

Note: You must make sure that TCP/IP servers are installed on the system before you install and configure iCluster.

Communication Protocol• TCP/IP

Program Temporary Fix RequirementsiCluster optionally uses IBM Cluster Resource Services. If you are using this failover mechanism, then any issues in Cluster Resource Services may affect iCluster. See Choosing a Failover Mechanism on page 73 for more information.Contact DataMirror Technical Support for the current list of PTFs that are necessary for iCluster. For more information, see Contacting Technical Support on page 547.Note: For information regarding the download and installation of any required PTFs, visit the support page at the

IBM web site.The following V5R1 PTFs are required for the QDBRPLAY API functionality that is required by the apply jobs on the backup node:• SI07279• SI06408


Note: The QDBRPLAY API functionality is built-in to OS/400 V5R2 and above, and therefore the PTFs mentioned above are not required.

Authorization Codes and LicensingiCluster requires a valid authorization code to ensure that it runs only on licensed iSeries systems. Serial numbers are recognized by the application. DataMirror trusts our clients to respect our proprietary rights.iCluster is provided by license only. You are required to purchase licenses for every node on which iCluster will be used. Licenses are issued on a machine class basis.Note: New authorization codes are required for new product releases only. For example, from iCluster V4R0 to

V5R0. Authorization codes are not required for pack ID updates such as iCluster V2R0 from pack ID 4 to pack ID 5. The existing authorization code can continue to be used for pack ID updates.

You will require a valid authorization code if you are installing a feature such as iBalance or one of the two editions of iCluster: iCluster Enterprise Edition or iCluster SMB Edition. For more information, see iCluster Enterprise Edition and iCluster SMB Edition on page 23 and Installing the iBalance Feature on page 23.You can obtain authorization codes either from DataMirror Technical support or from the DataMirror web site at http://www.datamirror.com. Note that the authorization codes obtained from the web site are temporary.

Required Information for the Installation ProcedureBefore starting the installation procedure, you need the following information:• Product library name: The name you want to assign to the iCluster product library. Note that DMCLUSTER is the default

library name for iCluster. • Device or save file name: The CD-ROM device name or save file name that will be used to restore iCluster. • Authorization code: The iCluster authorization code for each system where you will install iCluster.

For more information about authorization codes, see Authorization Codes and Licensing on page 20.

Creating a Journal Receiver and Journal QAUDJRN in QSYSYou need to make sure that the audit journal exists. An error will occur if the journal QAUDJRN cannot be found in the library QSYS. Follow the procedure below to create the QAUDJRN journal in library QSYS if it does not already exist.1 Create a journal receiver by issuing the following command:

CRTJRNRCV

The Create Journal Receiver (CRTJRNRCV) screen appears.2 In the Journal receiver field, enter the name of the journal receiver that you want to create.3 In the Library field, enter the name of the library where the new journal receiver will be located.4 Enter values in the other fields on this screen. If necessary, use F1 help to assist you in specifying these values.5 Press Enter.6 Create a journal for the receiver by issuing the following command:

CRTJRN

The Create Journal (CRTJRN) screen appears.7 In the Journal field, enter QAUDJRN.8 In the Library field for the journal, enter QSYS.


Before You Install

9 In the next two fields, enter the journal receiver name and library that you specified on the Create Journal Receiver (CRTJRNRCV) screen.

10 Enter values in the other fields on this screen. If necessary, use F1 help to assist you in specifying these values. 11 Press Enter.

Setting the QALWUSRDMN OS/400 System ValueThe allow user domain (QALWUSRDMN) system value determines which libraries can contain user-domain user objects. The default value for QALWUSRDMN is *ALL. Your system administrator can change this system value on individual machines to be one library or a list of libraries.Before proceeding with the installation, you must ensure that the QALWUSRDMN OS/400 system value is set to *ALL or specifies the DMCLUSTER or staging library. This setting is required to complete the installation successfully.

Installing, Re-installing, and Upgrading iClusterYou have the option of installing, re-installing, or upgrading iCluster. This section describes the differences between these three operations.• Installation: Installs iCluster into a new product library that previously did not exist. iCluster creates new metadata. You will

need to add nodes, groups, and other cluster objects before you can start replicating using iCluster.• Re-installation: Installs iCluster into an existing product library (any version) that is an iCluster production library. All existing

metadata is deleted from that library. You will need to add nodes, groups, and other cluster objects before you can replicate with iCluster.

• Upgrade: Upgrades an existing iCluster installation. A product library must exist, and the library specified during the upgrade must be an iCluster production library. An upgrade does not delete existing metadata. You do not need to re-add nodes, groups, and other cluster objects.

The subsystem must be ended prior to performing the upgrade.You can also migrate from HA Suite Version 3.6 or greater to iCluster. For more information, see Migrating HA Suite to iCluster on page 35.

Re-installing iClusterBefore re-installing iCluster, you need to end all replication activities and close all iCluster menus.Note: You also need to make sure that the iCluster product library is not in your library list.To re-install iCluster, you need to:1 Run the iCluster installation program.

The installation program automatically runs the VFYHAJRN—Verify Audit Journal command. For more information, see VFYHAJRN—Verify Audit Journal on page 232.

2 Re-add nodes and groups.For more information about adding and removing nodes and groups, see Node Commands and Group Commands.

Applying iCluster UpdatesYou can update your current version of iCluster with service packs. Service packs contain additional fixes or enhancements for your current version of iCluster. To obtain the latest service pack, contact DataMirror Technical Support. See Contacting Technical Support on page 547 for contact information.


Installing, Re-Installing, or Upgrading iCluster

Installing, Re-Installing, or Upgrading iClusterThis section identifies the steps required to install, re-install, or upgrade iCluster.

Overview of the Installation ProcedureThe installation/upgrade procedure should take between 20 and 30 minutes to complete. Before you get started, you need to sign on to the system.The installation procedure consists of the steps listed below: 1 Sign on to the system. For more information, see Signing on to your System on page 24.2 If you are upgrading or reinstalling iCluster, then cleanly stop the current installation. For more information, see Preparing for

an Upgrade or Reinstallation on page 24.3 Restore the installation program. For more information, see Restoring the Installation Program on page 25.4 Run the installation program. For more information, see Running the Installation Program on page 25.5 If you are upgrading an iCluster installation, then perform upgrade tasks. For more information, see Upgrading iCluster on

page 26.6 Configure TCP/IP for iCluster. For more information, see Running iCluster Under TCP/IP on page 37.

This step does not apply if you are upgrading or re-installing iCluster.7 Perform post-installation procedures. For more information, see After You Install on page 29.The material provided in After You Install on page 29 to Other Considerations on page 41 contains information about adding nodes to your clustered environment through iCluster and removing the software from your iSeries nodes. It also provides important information about messaging, batch processes, and job logs.Note: It is recommended that you read Before You Install on page 19 for important considerations about iCluster

before you start the installation program.

iCluster Enterprise Edition and iCluster SMB EditionDepending on your business requirements, you can choose between the following two versions of iCluster: • iCluster Enterprise Edition• iCluster SMB (Small and Medium Business) EditioniCluster Enterprise Edition is appropriate for complex iSeries environments that require richer configuration options. More flexibility is provided for groups and the ability to deal with them independently.iCluster SMB Edition is appropriate for less complicated environments with fewer applications, data, groups, and journals. Configuration is simpler for ease of set up and administration. It is limited to two replication groups with two data journals per group.The iBalance feature for workload balancing is available for both versions of iCluster.You will require a valid authorization code from your DataMirror representative if you are installing iCluster Enterprise Edition or iCluster SMB Edition. For more information, see Authorization Codes and Licensing on page 20.

Installing the iBalance FeatureiBalance is an additional feature for iCluster that allows you to perform master-master replication. With master-master replication, you can perform application updates to the same objects on both the source and target, and these updates can be replicated in both directions. Depending on the replication rules used, identical copies of your data can be maintained on both the source and


target systems, or conflict detection and resolution rules can be set up to control the updates to ensure that data is not overwritten on one of the servers. For more information on iBalance and master-master replication, see the iCluster iBalance and Master-Master Replication on page 321.You can install the iBalance feature as part of an installation/upgrade or on top of an existing iCluster installation. If you are installing or upgrading to iCluster with the iBalance feature, you can follow the steps outlined in Overview of the Installation Procedure on page 23. You can also follow these same steps if you are installing the iBalance feature into an existing iCluster library.Note: To install the iBalance feature, you must obtain an iBalance authorization code from your DataMirror

representative. For more information, see Authorization Codes and Licensing on page 20.

Signing on to your SystemSign on to the system with a user profile that has all of the following authorities:• *JOBCTL• *SECADM• *SERVICE• *ALLOBJ• *AUDIT• *SPLCTL• *SAVSYS• *IOSYSCFGNote: If you have already signed on to the system and run any programs, other than iCluster, we recommend that

you sign off and then sign on again.

Preparing for an Upgrade or ReinstallationIf you do not have an active iCluster installation on your system, then you can skip this section. Otherwise, perform the following tasks to cleanly exit your current installation.1 End all iCluster groups by running the following command for every group:

DMENDGRP GROUP(<group>)

2 Drain the staging store for every group in the cluster by running the following command:DMSTRAPY GROUP(<group>) FRCDRN(*YES)

3 Clear the staging store completely with the following command:CLRLIB <staging store library>

The default staging store library is DMSTORE. This value was set when the node was added to the cluster.4 End all iCluster nodes by running the following command for every node:

DMENDNODE NODE(<node>)

5 Exit the iCluster menus on all clients and terminals accessing the current installation.6 End the XDMCLUSTER subsystem on all nodes by running the following command:

ENDSBS SBS(XDMCLUSTER)

After performing these tasks you can continue with the installation process. See iCluster iCluster Commands on page 97 for more information on performing these tasks.



Restoring the Installation ProgramYou can restore the installation objects from a CD-ROM device or from a save file using the RSTOBJ command. There are two save files required for installation.Note: The RSTOBJ command may take a few moments to complete. It does not provide any feedback while the

installation objects are being restored.

To Restore Objects from CD-ROM Device1 Insert the iCluster CD-ROM into the CD-ROM drive.2 Run the following command:

RSTOBJ OBJ(*ALL) SAVLIB(CSSINS) DEV(<CD-ROM device>) MBROPT(*ALL) ALWOBJDIF(*ALL) RSTLIB(QTEMP) OPTFILE('/ICLUSTER/CSINITIAL/CSSINS')

To Restore Objects from Save File1 Transfer the installation files to the machine where you will install or upgrade iCluster.2 Run the following command:

RSTOBJ OBJ(*ALL) SAVLIB(CSSINS) MBROPT(*ALL) ALWOBJDIF(*ALL) DEV(*SAVF) SAVF(<save file library>/<save file name>) RSTLIB(QTEMP)

Note: You must specify the save file library and name in this command. DataMirror names the save file CSSINS, but your administrator may have renamed the file.

Running the Installation ProgramThe installation screens in the following procedure are for an iCluster Enterprise installation. The text on the installation screens may vary depending on the edition of iCluster that you install (iCluster Enterprise or iCluster SMB) and the features you select (iBalance). Where important, the differences are noted in this procedure.1 To run the installation program, enter the following command:

?QTEMP/CSINSTALL

The DM iCluster Installation (CSINSTALL) screen appears where you can specify the device name or save file name.2 Specify the CD-ROM device name (blank by default) or save file that will be used to restore the iCluster components, and

press Enter. DataMirror names the file CSSMAS, but your administrator may have renamed the file.Note: The Savefile name for iCluster and Library Name fields are displayed only if you enter *SAVF in the Device

name field.A screen appears that displays the start of the DataMirror software license agreement. Read this document by pressing Page Down a number of times to scroll down to the bottom of the agreement.

3 Press F2 to accept the terms expressed in the software license agreement.The screen displays the version of iCluster that you are about to install.

4 Press Enter to advance to the next screen in the installation program.The iCluster Authorization Code screen is displayed.

5 In the iCluster for the iSeries Authorization Code field, enter the iCluster authorization code that you have received from DataMirror. You cannot continue with the installation without a valid authorization code.For more information about obtaining authorization codes, see Authorization Codes and Licensing on page 20.

6 Press Enter to continue the installation/upgrade process.The Installation/Upgrade screen is displayed.


7 In the selection field, enter I if you want to install iCluster and proceed to Step 8 below.Note: To upgrade iCluster, enter U and then proceed to Upgrading iCluster on page 26.8 After you select I and press Enter on the Installation/Upgrade screen in the previous step, the Confirm Installation screen is

displayed.This screen confirms that you have decided to install iCluster.

9 Press Enter to continue.The iCluster Product Library screen is displayed.

10 In the iCluster for the iSeries Product Library field, specify the library name where you want to install iCluster.Before specifying the library, you should note the following:• The library that you specify will indicate whether you are installing or re-installing iCluster. See Installing, Re-installing,

and Upgrading iCluster on page 21 for information about the differences between the two procedures. The default iCluster product library is DMCLUSTER.

• If you are installing the iBalance feature on top of an existing iCluster installation, you must specify the existing iCluster product library and press Enter. A confirmation screen will inform you that you are about to install the iBalance feature into the iCluster product library. To complete the installation, you can proceed to Step 16 in this procedure.

11 Press Enter to continue.If you specified a new library name, the New Library Installation screen is displayed.

12 Press Enter.iCluster will be installed into the specified library.If you specified an existing iCluster installation library name, then the Existing Library Installation screen is displayed.

13 In the selection field, enter N if you do not want to over-write the contents of the specified library, or enter Y if you want to re-install iCluster into the specified library.

Note: If you decide to re-install (Y), all existing objects in the specified library will be deleted, and new programs configuration information will be installed. The selection N is the default.

14 Press Enter.If you selected N, you will return to the iCluster Product Library screen (Step 10) where you can enter the name of a new library.If you selected Y, the Confirm Existing Library screen is displayed.This screen confirms that you want to re-install iCluster into the existing product library. By default, iCluster will be installed into the existing library DMCLUSTER. Note that all objects that currently reside in the specified library will be deleted.

Note: When re-installing iCluster, you may receive a message indicating that the HADRCV and HABSFRCV journal receivers are never fully saved. This is a system message that is displayed when iCluster tries to delete a journal receiver. You simply need to acknowledge the message to continue with the installation.

15 Press Enter.A set of iCluster objects will be placed on your node after you press Enter. This process may take a few moments to complete. When this is done, the Confirmation screen is displayed.

16 Press Enter to complete the installation.

Upgrading iClusterIf you selected to upgrade iCluster (by selecting U from in Step 7 above), the following screen appears.

This screen confirms that you have decided to upgrade to iCluster.



Note: Before beginning the upgrade process, you must complete the tasks described in Preparing for an Upgrade or Reinstallation on page 24.

1 Press Enter to continue the upgrade process.The iCluster Product Library screen is displayed.

2 In the iCluster for the iSeries Product Library field, specify the library name where iCluster is currently installed. The default installation library is DMCLUSTER.

3 Press Enter to continue.The following Confirm Upgrade screen is displayed.This screen confirms that you are upgrading the existing version of iCluster in the specified installation library.

4 Press Enter to continue.A set of iCluster objects will be placed on your system after you press Enter. This process will take some amount of time to complete. As part of this process, the corresponding metadata will be upgraded in the DMCLUSTER library.

5 Once the upgrade is complete, the following iCluster Installation/Upgrade Complete screen is displayed.6 Press Enter to complete the upgrade.7 Start the subsystem and then the node, for every node in the cluster.

You can now proceed to Running iCluster Under TCP/IP on page 37.


After You Install

After You InstallThis section contains information about a number of post-installation activities that you should perform to ensure that iCluster will function correctly.

Starting the XDMCLUSTER Subsystem on IPLIn most environments, you will want to configure the XDMCLUSTER subsystem to start automatically when the iSeries computer it is installed on starts. To do this, you must run the STRSBS command as a part of the QSTRUPPGM start up value. This is an optional configuration.To configure your system to start the XDMCLUSTER subsystem automatically with the computer, add the following command to the source code for the program that is specified in the QSTRUPPGM system value to allow the program to run whenever the machine is re-booted:QSYS/STRSBS <iCluster installation library name>/XDMCLUSTER

If the machine does not have a program specified in the QSTRUPPGM system value, then perform the following tasks:1 Create a startup program by modifying the QSTRUP member in the QGPL/QCLSRC source file with the command specified

above, and compile the program with the CRTCLPGM command.2 Run the following command to display the Change System Values screen:

CHGSYSVAL QSTRUPPGM

3 Enter the name and library of the program created in Step 1 and press Enter. This instructs the machine to run the program every time the machine starts.

See your iSeries documentation for more information on this task.

Performance ConsiderationsiCluster allows you to optimize the flow of data into and out of a high-speed software cache. This enables you to move data from the primary system to the backup system's cache at a much higher rate than previously possible. This performance enhancement allows you to attain near zero latency for system backup and recovery.To obtain these benefits, you should configure iCluster to run in its own storage pool, or at least run in a storage pool other than *BASE. To perform this configuration, see your system administrator.

Adding an iCluster User (Optional)After installing iCluster on an iSeries node, it is recommended that you create one or more iCluster users on this node through the DMADDUSR—Add User command. This step will allow you to work with iCluster without having to sign on to the system as QSECOFR or a user with QSECOFR authority. If you do not know the password for QSECOFR, adding at least one iCluster user allows you to work with iCluster through the specified user name after you have logged off from QSECOFR. Each iCluster user should have security access that is appropriate for their use of iCluster. Perform the following steps to add an iCluster user on a node where iCluster has been installed:1 Sign on to the system as QSECOFR or a user with *SECADM authority.2 Change the current library to the iCluster product library.3 Issue the following iCluster command:

DMADDUSR USER(<OS/400 user name>) AUTH(<iCluster authority level>) PASSWORD(<iCluster Administrator log on password>)

where:


• <OS/400 user name> is an OS/400 user profile name that is defined on the node where the command is issued. The OS/400 user name must be defined on the node and must have *IOSYSCFG authority.

• <iCluster Administrator log on password> is the iCluster password that you want to associate with the user name specified through the USER parameter.

Note: The iCluster password does not refer to the password associated with the specified OS/400 user name. You need to specify a password only if you will be using the iCluster Administrator. It is suggested that the specified password be different from the password associated with the OS/400 user name.

• <iCluster authority level> is the authority level that you will grant to the user. For example, *ADMIN for administrative privileges.

You should grant administrative (*ADMIN) privileges to at least one iCluster user so that all product operations can be performed by this user.

4 Press Enter.

Securing the iCluster Product Library (Optional)The iCluster product library was created during the installation process. Its default name is DMCLUSTER and it is owned by DMCLUSTER. Use the following procedure to authorize the use of iCluster to specific users.1 Edit the authority of the iCluster library object using the following command:

EDTOBJAUT OBJ(QSYS/<iCluster product library>) OBJTYPE(*LIB)

where <iCluster product library> is the library where iCluster was installed.Note: If you revoke authority for the user *PUBLIC, you should make sure that you add the user DMCLUSTER and

any other authorized user(s) with object authority *CHANGE.2 Press Enter to complete the operation.

Scheduling iCluster Startup and Shutdown (Optional)Note: This section assumes that the subsystem XDMCLUSTER is active.To schedule iCluster to start up or shut down automatically at scheduled times using the IBM supplied command ADDJOBSCDE (Add Job Schedule Entry), follow these steps: 1 Create a Control Language (CL) program to set the current library to the iCluster product library name, and to execute the

desired iCluster command. The sample CL program below sets the current library to DMCLUSTER and starts cluster operations to the backup node in the cluster group GRP1 an initial refresh is performed before cluster operations are started. For a detailed description of all available commands, see iCluster iCluster Commands on page 97.

CL Program: OBJ Start***********************Beginning of data *****************0001.00 PGM0002.00 CHGCURLIB CURLIB (DMCLUSTER)0003.00 DMSTRGRP GROUP (GRP1) REFRESH(*YES) STRAPY(*YES)0004.00ENDPGM***********************End of data************************

2 Add an entry to the OS/400 job scheduler by issuing the ADDJOBSCDE command.An example is the following:ADDJOBSCDE JOB(OBJ_START) CMD(Call <library name 1>/OBJ_START) FRQ(*WEEKLY) SCDDATE(*NONE) SCDDAY(*ALL) SCDTIME(080000) JOBD(<iCluster product library>/CSJOBD)


After You Install

where:• <library name 1> is the library where you created the CL program OBJ_START.• <iCluster product library> is the library where you installed the iCluster product.

Cluster operations are scheduled to start daily at 8 a.m.

Adding NodesBefore you can start performing cluster operations with iCluster, you must add at least one node to your cluster. For more information on performing this task, see Node Commands.Note: If you are using OS/400 Cluster Resource Services to manage your cluster, you must install the QGY system

library on every node in the cluster.


Uninstalling iCluster

Uninstalling iClusterThis chapter identifies the procedure that you can use to uninstall iCluster from your nodes.

Uninstalling iClusterBefore you run the uninstall program, you must do the following:• End all replication activity and close all open iCluster menus.• End all queue managers for MQSeries groups that are replicating in the cluster. This will ensure that journaling ends for

objects that are selected to MQSeries groups.Note: iCluster will end journaling before the uninstall program begins.Perform the following steps to remove iCluster from your node.1 Sign on to the system as QSECOFR. 2 Enter the following command to change the current library, and then press the Enter key:

CHGCURLIB CURLIB(<iCluster product library>)

where <iCluster product library> is the library where iCluster was installed.3 Enter the following command to start the uninstall program, and then press Enter:

?CSUNINST

The iCluster Uninstall screen is displayed.After issuing the CSUNINST command, a screen appears that displays the product library name.

4 Verify that you have specified the iCluster product library name (this is the library where iCluster is installed).5 Press Enter to start the uninstall program.Note: You will have to acknowledge system messages during the uninstall process.

The uninstall program starts. The screen will be blank for several minutes. As iCluster is being removed from your node, messages will appear briefly on the screen. You do not need to perform any actions while this is taking place.You may receive a message indicating that the HADRCV and HABSFRCV journal receivers are never fully saved. This is a system message that is displayed when iCluster attempts to delete a journal receiver. Acknowledge the message to continue with the uninstall.When the uninstall is complete, you will return to the screen where you entered the uninstall command. Messages will be displayed to confirm whether or not iCluster was successfully removed.

Proceed to Removing the iCluster Service Table Entries on page 39 to complete the uninstall process.


Migrating HA Suite to iCluster

Migrating HA Suite to iClusterThis section identifies the steps required to migrate HA Suite to iCluster.

Introduction to the Migration ProceduresTo migrate HA Suite Version 3.7 or greater to iCluster, you must run a command line migration utility (DMCVTHATGT) that will define iCluster replication groups based on metadata in your version of the HA Suite installation library. System parameters and SwitchOver System information defined in HA Suite will be migrated where possible.Note: You must install iCluster on your system before you can migrate from HA Suite.

General ConstraintsCertain aspects of the HA Suite configuration cannot be ported to iCluster during the migration process. These include the following:• SNA communications: This type of communication protocol is not supported in iCluster.• SwitchOver System (SOS) user exits: The argument list passed to the switchover user exits (before/after) differs between

HA Suite and iCluster. You must change the user exit programs in order to conform to the argument list expected by iCluster.

Pre-migration TasksBefore running the migration utility, ensure the following tasks are completed:1 iCluster is installed on all machines involved in replication. For more information, see Installing, Re-Installing, or Upgrading

iCluster on page 23.2 The user running the migration utility has iCluster administrative rights for the current machine. Issue the DMADDUSR—Add

User command for the user who will run the utility.3 Cluster nodes are defined. For more information, see iCluster iCluster Commands.

Running the Migration UtilityTo begin the migration process from HA Suite to iCluster, complete the following tasks:1 To run the HA Suite to iCluster migration utility, enter the following command on the iCluster command line:Note: When executing this command, the iCluster product library should be the current library and the HA Suite

product library should not be in the library list.DMCLUSTER/DMCVTHATGT HALIB(<HA Suite library>) TARGET(<target name>) BACKUP(<backup node name>)

where:• <HA Suite library> is the name of the library containing the HA Suite installation.• <target name> is the name of a single HA Suite target.• <backup node name> is the name of the backup node with which the target is associated.You can issue this command only on an HA Suite source machine. It will perform metadata migration from HA Suite to iCluster. Upon successful completion of this command, iCluster replication groups, object specifiers, and system values (group/object) based on HA Suite metadata will be configured.Note: The DMCVTHATGT command will only operate on a single target and must be run separately for each target

on every source machine. The HA Suite target's role must be set to *PRIMARY if it is HA-enabled.


2 Check the job log for errors.If there are no errors in the job log, the migration is complete. You can now proceed to the next section to complete the migration process.If there are errors in the job log, you may have to run the migration utility again.

Post-migration TasksAfter completing the migration, perform the following tasks:1 The source system must have minimal or no activity. An idle state is recommended.2 Ensure there are no suspended objects. If the objects were purposefully suspended in HA Suite, these objects will have to be

manually suspended in iCluster. For more information, see the DMSUSOBJ—Suspend Object command. 3 If HA Suite is running, make sure the apply has caught up and end replication. The Transactions to process value in the HA

Suite Status Monitor should be zero. For more information, see the iCluster iCluster Commands on page 97.4 Clear the HA Suite staging store library if it is to be re-used in iCluster.5 Issue the DMREGPOS—Register Positions command. This command will set the positions of the journals scraped by

iCluster.6 Issue the DMSTRGRP—Start Cluster Operations at Group command to start replication in iCluster for each group converted

by the migration.Note: After migrating to iCluster, it is recommended that you do not remove the HA Suite default journal

(HASUITE/HADJRN) from your system. It is likely that you will still have a number of files journaled to this journal. When you begin using iCluster, these files will continue to be journaled to the HASUITE/HADJRN journal.


Running iCluster Under TCP/IP

Running iCluster Under TCP/IPThis section provides instructions to help you configure iCluster so that it can communicate with an iSeries node under TCP/IP.

Adding a New Entry 'dmcluster' to the TCP/IP Service TableYou need to add a new entry (dmcluster) to the TCP/IP service table, and specify an unused port number. You can perform this task using Method 1 - Menu or Method 2 - Command.

Method 1 - Menu1 On the command line of the Command Entry screen, enter CFGTCP.

The Configure TCP/IP screen is displayed.2 Select option 21 - Configure related tables.

The Configure Related Tables screen is displayed.3 Select option 1 - Work with service table entries.

The Work with Service Table Entries screen appears.4 Enter 1 in the Opt column and press Enter.

The Add Service Table Entry screen is displayed.5 In the Service field, you must enter 'dmcluster' (in single quotes).6 In the Port field, enter an unused port number in the range of 1 to 65535.7 In the Protocol field, enter 'tcp' (in single quotes).8 In the Text 'description' field, enter a description to indicate that this port will be used by an iCluster TCP/IP node.9 After entering the values, press Enter.

You will receive a confirmation message if you successfully added a service table entry.

Method 2 - Command1 On the command line, enter:

ADDSRVTBLE SERVICE('dmcluster') PORT(<port number>) PROTOCOL('tcp') TEXT('<user description>')

2 Press Enter.You will receive a confirmation message if you successfully added a service table entry.

Restricting the Port Number from Being Used by Other ApplicationsTo ensure that the port number that you specified in the installation program will not be used by other applications, you can add a restriction on this port using Method 1 - Menu or Method 2 - Command as follows:

Method 1 - Menu1 On the command line of the Command Entry screen, enter CFGTCP.

The Configure TCP/IP screen is displayed.2 Select Work with TCP/IP port restrictions (option 4).

The Work with TCP/IP Port Restrictions screen is displayed.


3 Enter 1 on the first row under Opt and press Enter.4 On the first line, enter the same port number you used when adding the service table entry.5 In the Protocol field, enter *TCP.6 In the User Profile field, enter DMCLUSTER.

This ensures that the iCluster subsystem (XDMCLUSTER) can also access this port number.7 Press Enter to issue the command.8 Repeat this method for all other user profiles that can issue the STRHATCP—Start TCP/IP Listener command.


ADDTCPPORT PORT(<port number>) PROTOCOL (*TCP) USRPRF(DMCLUSTER)

This ensures that the iCluster subsystem (XDMCLUSTER) can also access this port number.2 Press Enter to issue the command.3 Repeat this method, but replace "DMCLUSTER" with each user profile that can issue the STRHATCP command.

See the STRHATCP—Start TCP/IP Listener command for more information.There are also commands and command menus to remove the entry and restriction you have just added. For more information, see Removing the iCluster Service Table Entries on page 39 and Removing Port Number Restrictions on page 39.

Verifying Domain Name Server (DNS) ConfigurationIf you are using the local host table for host name resolution, then you must verify the domain name server (DNS) configuration to ensure that the local host table is initially searched for the host name.1 On the command line of the Command Entry screen, enter CFGTCP.

The Configure TCP/IP screen is displayed.2 Select option 12 - Change TCP/IP domain information.

The Change TCP/IP Domain (CHGTCPDMN) screen is displayed.3 Verify that the Host name search priority field is set to *LOCAL.

*LOCAL indicates that you want to use the local host table. If you want to search a domain name server before searching the local host table, set this field to *REMOTE (this is the default setting), and specify the IP address of the domain name server in the configuration.

4 Exit the screen.


Removing the iCluster Service Table Entries

Removing the iCluster Service Table EntriesThis section explains how to remove the iCluster service table entries that were created after the installation.

Removing the dmcluster Table EntryPerform the following steps to remove the dmcluster entry from the TCP/IP service table.You can use Method 1 - Menu or Method 2 - Command.

Signing on to your SystemSign on to the system as QSECOFR.

Removing dmcluster from the TCP/IP Service TableMethod 1 - Menu1 On the command line of the Command Entry screen, enter CFGTCP.

The Configure TCP/IP screen is displayed.2 Select option 21 - Configure related tables.

The Configure Related Tables screen is displayed.3 Select option 1 - Work with service table entries.

The Work with Service Table Entries screen is displayed.Note: You need to remember the port number associated with this entry because it is required when removing port

restrictions. For more information, see Removing Port Number Restrictions on page 39.4 Enter 4 in the Opt field beside the dmcluster service and press the Enter key to remove the entry from the service table.


RMVSRVTBLE SERVICE(dmcluster) PORT(<port number>) PROTOCOL (*TCP)

2 Press Enter.

Removing Port Number Restrictions1 On the command line of the Command Entry screen, enter CFGTCP.

The Configure TCP/IP screen is displayed.2 Select option 4 - Work with TCP/IP Port Restrictions.

The Work with TCP/IP Port Restrictions screen is displayed.3 Enter 4 beside all entries for the port number used by iCluster. 4 Press Enter on the Confirm Delete screen.


Other Considerations

Other ConsiderationsThis section contains important considerations for upgrading the operating system, and provides information about messaging and system logs.

Additional Message InformationExcept for cases of communication or machine failures, a message is displayed on the screen when a problem occurs.If a screen operator needs more information about a message in the message area, the operator can display the Additional Message Information by moving the cursor to the message area and pressing the F1 (HELP) key. The Additional Message Information display lists extended message text, the probable cause of the message, and a list of resolutions.If the problem cannot be solved, press F6 (PRINT) to log the error. Contact DataMirror Technical Support to obtain additional assistance. For more information, see Contacting Technical Support on page 547.

Batch Processes and Job LogsFor communications and other batch processes in iCluster, system job logs are produced for both primary and backup jobs that provide messages about the operation of each particular function.1 To access the job logs, use the following command on either the primary or backup node:

WRKSPLF USER(DMCLUSTER)

Note that DMCLUSTER should be specified if the default iCluster job description is used. Otherwise, specify the user profile associated with the job description that you are using.


iCluster BasicsThis section contains the following topics:• Working in a Clustered Environment on page 43• Failovers and Switchovers on page 47• Object Specifiers and Types on page 50• Path Object Specifiers on page 53

Working in a Clustered EnvironmentThis section provides a high-level view of iSeries clusters by examining the basic elements of a clustered environment. It describes the following components:• Nodes• Replication Objects • Groups • Failover Mechanisms

Nodes in a ClusterA node is a single iSeries computer. To use iCluster, you must add one or more nodes to the same cluster. This cluster becomes your replication environment.The replication environment identifies each node by a name entered by the iCluster Administrator. To successfully cluster a set of nodes, each node must have a unique name and be able to communicate with every other node in the cluster. You can add up to 128 nodes to a cluster.Figure 1shows a cluster with four nodes. The lines indicate network connections between nodes.

Figure 1 A Four Node Cluster


Master NodeThe first node you add to a cluster becomes the master node. In addition to the features available to all of the other nodes in the cluster, this node also maintains a repository of configuration information for the cluster. The following table illustrates how cluster membership affects the master node.

Replication ObjectsReplication objects are the data, applications, and other objects on a node that you want to replicate. You can replicate data stored in native iSeries objects or Integrated File System (IFS) files. See Object Specifiers and Types on page 50 for a complete list of the objects you can replicate.

GroupsA group is a replication relationship between nodes in a cluster. When you create a group, you typically define the primary node and backup node in the relationship. iCluster provides the following types of groups for your replication environment:• Replication groups, which support the replication of objects and data for high availability. If you are using iCluster

Administrator, see Working with Replication Groups on page 413.• MQSeries groups, which provide a high availability solution for WebSphere MQ. If you are using iCluster Administrator, see

MQSeries Group Commands on page 470.• Application groups, which support application resiliency on clusters using IBM Cluster Resources. See Application

Resiliency on page 68 for more information.• Switchable device groups, which use IBM Cluster Resources to support switchable disks.• Refresh-only groups, which perform a refresh of the objects selected to the group and then shut down. These groups are

not eligible for role switch. If you are using iCluster Administrator, see Working with Replication Groups on page 413.• Master-master groups, which support the mirroring of changes from the source to the target and vice-versa. For more

information, see iBalance and Master-Master Replication on page 321.

Table 1 Cluster Membership

Action Nodes in Cluster Master Node

1. TORONTO joins cluster TORONTO TORONTO

2. NEW YORK joins cluster TORONTO NEW YORK TORONTO

3. DALLAS joins cluster TORONTO NEW YORK DALLAS TORONTO

4. TORONTO leaves cluster NEW YORK DALLAS NEW YORK

5. TORONTO rejoins cluster NEW YORK DALLAS TORONTO NEW YORK

6. NEW YORK fails DALLAS TORONTO DALLAS


The replication process copies data and applications from the primary node to the backup node. For example, in the cluster in Figure 1 on page 43, we can create a group that replicates data between the TORONTO and NEWYORK nodes. In this group, TORONTO is the primary node and NEWYORK is the backup node.

The primary node keeps a record of the data it has sent to the backup node. If the backup node goes offline, then the primary node will queue the replication requests and send them to the backup node when it becomes available again.In a cluster using the SwitchOver System (SOS) (see Failover Mechanisms on page 47, below), the backup node in a group regularly polls its connection with the primary node. By configuring node and group properties, you can control what the backup node does when it cannot verify its connection with the primary node. You can also define how many attempts the backup node makes to establish communication before taking action.To use your SOS cluster as a high availability solution, you can configure the backup node to become the primary node when it cannot communicate with the previous primary node. For example, consider the group that replicates data from TORONTO to NEWYORK. NEWYORK is configured to test the link to TORONTO every minute and wait for thirty seconds for Toronto to respond. If NEWYORK does not get a response within thirty seconds for five consecutive attempts, then it considers TORONTO to be unavailable and runs a user exit application that notifies the system administrator. The system administrator performs a switchover, which causes all applications to use NEWYORK instead of TORONTO without requiring application changes. These properties were configured when NEWYORK was added to the cluster.Nodes in a cluster can belong to multiple groups and can replicate the same or different objects in these groups. Figure 2 shows the sample cluster with its groups. The arrows indicate the groups and the group names appear in italics.

Figure 2 All Groups in the Cluster

Figure 2 shows the following groups:• TNGRP: Replicates from the TORONTO (primary) node to the NEWYORK (backup) node.• DTGRP: Replicates from the DALLAS node to the TORONTO node.


• TSGRP: Replicates from the TORONTO node to the SANJOSE node.• NSGRP: Replicates from the NEWYORK node to the SANJOSE node.This example also illustrates that the same node can serve multiple roles in the same cluster. In the example:• DALLAS is a primary node (for the DTGRP group).• TORONTO is both a primary node (for the TNGRP and TSGRP groups) and a backup node (for the DTGRP).• NEWYORK is both a primary node (for the NSGRP group) and a backup node (for the TNGRP).• SANJOSE is exclusively a backup node (for the TSGRP and NSGRP groups).After creating a group, you can specify the objects that the group replicates from the primary to the backup node. You can specify multiple objects for replication in the same group. You can also specify the same object in multiple groups from the same primary node. For example, we can replicate INVENTORY/BOOKS from TORONTO to both NEWYORK and SANJOSE by specifying it in each of the TNGRP and TSGRP groups. We can also replicate an object to only one node, even though that node is the primary node to multiple nodes. For example, we can replicate HR/PAYROLL from TORONTO to only SANJOSE by only specifying it in TSGRP. Figure 3 shows these objects and the groups they are specified for.

Figure 3 Specifying Replication Objects

Chaining GroupsYou can create replication sequences that make your cluster more resilient by chaining a group across multiple nodes. In our basic group scenario, when the TORONTO node fails, the NEWYORK node replaces it. You can then replicate from NEWYORK to SANJOSE. This provides another layer of resiliency to your cluster.

Group IndependenceReplication groups operate independently of each other. You can start, stop, and configure one group without affecting the other groups in the cluster. For example, if the TORONTO node loses power, the NSGRP group is unaffected.

Local Loopback ReplicationiCluster can replicate objects across separate physical systems or between environments on the same physical system. The latter approach is known as local loopback. Since replication is within the same physical system, iCluster cannot replicate an object on top of itself and does not allow you to perform recursive updates. You must specify a different target library for local loopback replication.Local loopback replication is available for native library-based objects. You cannot use local loopback replication with BSF objects, resilient applications, MQ Series groups, and switchable device groups.


Failover MechanismsiCluster uses your failover mechanism to detect and respond to outages in a cluster. iCluster supports the following failover mechanisms:• SwitchOver System (SOS) has the backup node in each group test its connection with the primary node regularly. If the

primary node is unavailable, then the backup node declares the primary node as failed. When the backup node does this, you can also have it notify an administrator and run a user exit.The Switchover for SwitchOver System (SOS) on page 369 contains more information about this failover mechanism.

• IBM Cluster Resource Services (IBM CRS) uses the iSeries Cluster Resource Group to maintain the status of each node in the cluster. IBM CRS handles outages by either partitioning the cluster or failing the primary node, depending on how the outage occurred. The partition and failed states require administrators to develop multiple recovery strategies. The Switchover for IBM Cluster Resource Services (CRS) on page 341 contains more information about this failover mechanism.

See Failovers and Switchovers on page 47 for more information about how each failover mechanism handles planned and unplanned outages. See Choosing a Failover Mechanism on page 73 for more information on using these failover mechanisms in iCluster.

Failovers and SwitchoversA failover occurs when the primary (or production) node in a group unexpectedly becomes unavailable. When this happens, the system can either automatically move client applications and users to the backup node as a high availability solution or the system can wait for an administrator to take action. Power failures or network failures are examples of unexpected issues that can cause a node to become unavailable.A switchover occurs when an administrator decides to swap the roles of the nodes in a group. This causes the original backup node to replicate data to the original primary node. After a switchover, client applications and users use the new primary node (formerly the backup node).Failover and switchover handling is not supported for the following types of groups:• Groups that are refresh-only.• Master-master replication groups• Groups that are local loopback.• Groups that have a target library other than *PRIMARY.• Groups that have an object specifier selected whose target library is not *GRPSEL.


The following sections describe the different types of failover and switchover and how each failover mechanism handles them. They use the following figures to illustrate the concepts using the group from Working in a Clustered Environment that replicates data from TORONTO to NEWYORK.

Figure 4 Initial Group Replicates from TORONTO to NEW YORK

Figure 5 NEWYORK Becomes the Primary Node after TORONTO Fails

Figure 6 After Repairing TORONTO, it Becomes NEWYORK's Backup Node


Automatic FailoverWith both failover mechanisms, administrators can configure the backup node to automatically take over the role of the primary node during a failure. This is known as an automatic failover. In an automatic failover, the backup node assumes the role of the primary node to clients. If the primary node later becomes active, then objects are replicated from the former backup node to the former primary node. For example, consider the group that replicates from TORONTO to NEWYORK (Figure 4 on page 48). If TORONTO fails, then NEWYORK becomes the primary node in the group. Clients now connect to NEWYORK (Figure 5 on page 48). After correcting the problem, TORONTO is available again and, by default, will be the group's backup node for replication (Figure 6 on page 48).

Manual FailoverA manual failover situation occurs when the primary node of a group becomes unavailable and you, as an administrator, configures the group to wait for a response instead of performing an automatic failover. This allows administrators to try to fix the problem instead of changing the characteristics of their group and cluster. Each failover mechanism handles manual failover situations differently, depending on how the primary node became unavailable. The following scenarios illustrate the differences between how failover mechanisms handle manual failovers. Each scenario uses the group that replicates from TORONTO to NEWYORK (Figure 4 on page 48).

Scenario 1: SOS Cannot Communicate with the Primary NodeSwitchOver System (SOS) handles all primary node outages in the same manner. SOS does not distinguish between node or communication failures. Once the backup node cannot communicate with the primary node (determined by a set of configurable timeouts and thresholds), it changes the status of the primary node to *FAILED and suspends replication.To recover from this scenario, the administrator can either failover control to the backup node (the end result is the same as an automatic failover, see Figure 5 on page 48) or can repair the primary node and restart the group.

Scenario 2: IBM CRS Detects a Distress Signal from the Primary NodeIBM CRS has the ability to detect distress signals from nodes in a cluster. These signals warn the cluster of imminent outages. For example, if TORONTO loses power and switches to a backup power supply, it can notify the cluster that it is about to go offline as the power drains from the backup power supply.


After receiving the distress signal, IBM CRS sets the status of the primary node to *FAILED and suspends replication. The administrator can either failover control to the backup node (the end result is the same as an automatic failover, see Figure 5 on page 48) or can fix the problem on the primary node and restart the group.See Switchover for IBM Cluster Resource Services (CRS) on page 341 for more information recovery options for this scenario.

Scenario 3: IBM CRS Loses Communication with the Primary NodeIf IBM CRS loses communications with the primary node and did not receive a distress signal, then it partitions the cluster so that the nodes in each partition are aware of the communication problems to the other partition.If the administrator repairs the problem, then the cluster will automatically merge the partition and replication will continue as it did before the failure (Figure 4 on page 48). Otherwise, the administrator must either failover control to the backup node (the end result is the same as an automatic failover, see Figure 5 on page 48) or can fix the problem on the primary node and rejoin the group.See Switchover for IBM Cluster Resource Services (CRS) on page 341 for more information recovery options for this scenario.

SwitchoverA switchover allows administrators to reverse the nodes in a group intentionally and under controlled conditions. A common switchover scenario is upgrading a computer that is part of a replication group. The switchover allows administrators to move clients off the computer before removing it from the network.When you perform a switchover on a group, the backup node takes over for the primary node. In our example (Figure 7), NEWYORK would become the primary node and TORONTO would be the backup. NEWYORK would replicate data to TORONTO. At this point, the group appears as it did after an automatic failover (Figure 9).Depending on your failover mechanism, the failover and switchover options available to you will differ. See Choosing a Failover Mechanism on page 73 for more information.

Object Specifiers and TypesIn Working in a Clustered Environment on page 43, the concept of a group was introduced to illustrate how different replication paths can be defined in a cluster. A method of addressing objects that reside on a node is needed to identify the objects that are selected to groups for replication to backup nodes. iCluster uses object specifiers to achieve this objective. Object specifiers selected to groups are shown in Table 4.An object specifier consists of the following elements:• Object nameGeneric and special values for the object name can be specified to reference multiple objects.• Library nameThe name of the library where this object is stored.• Object typeA special value for the object type can be specified to reference all object types that can be replicated by iCluster.• Object attributeNote: This element of the object specifier only applies to *FILE objects. Object attributes allow you to specify the type of *FILE object. A special value for the object attribute can be specified to reference all types of *FILE objects that can be replicated by iCluster.


Constructing Object SpecifiersFor example, assume that the following set of objects exist on an iSeries node:

Table 6 contains a number of object specifiers and identifies the objects that are addressed by each specifier (the number assigned to each object in Table 1 is used in Table 3 to identify the addressed objects).

Table 2 Sample Objects on iSeries Node

# Object Library Object Type Physical File

1 EMG LIB1 *MSGF Not applicable

2 HLPATS HP1 *PNLGRP Not applicable

3 HLPAG HP1 *PNLGRP Not applicable

4 HLPRTS HP2 *PNLGRP Not applicable

5 M3024 TEST *FILE Source physical file

6 M3909 TEST *FILE Data physical file

7 HASETUP TEST *PGM Not applicable

8 HAFSYS SLIB *PGM Not applicable

9 S1 COM *ALRTBL Not applicable

10 S1 COM *STMF Not applicable

11 SRCJD LIB1 *JOBD Not applicable

12 SRCLG OLG *FILE Data physical file

13 UKK YLIB *USRIDX Not applicable

Table 3 List of Objects Addressed by Example Specifiers

Object Specifier Addressed Objects (See # in Table 2)

Object Library Object Type Object Attribute

SRCLG OLG *FILE PFDTA 12

M* TEST *FILE PF 5, 6

M3909 TEST *PNLGRP Not applicable No objects are addressed

HLPA* HP1 *PNLGRP Not applicable 2, 3

HLPAT* HP1 *PNLGRP Not applicable 2

HLPAT HP1 *PNLGRP Not applicable No objects are addressed

*ALL LIB1 *JOBD Not applicable 11


Note the use of generic (for instance, M* and HLPA*) and special values (for instance, *ALL) in some of the specifiers.iCluster can replicate different types of objects within a group. It is important for you to recognize the object types that are supported by this product and to be aware of certain issues that address specific types. For more information about object types, see Object Types Replicated by iCluster on page 529.

Path Object SpecifiersA path object specifier identifies a set of Byte Stream File (BSF) objects, stored in the Integrated File System (IFS), for replication. See Replicating BSF Objects on page 89 for more information.You identify a path object specifier in iCluster by supplying the full path where the BSF objects are located. It is important to identify the correct path so that you address the correct set of objects.Like object specifiers, you can select path object specifiers to groups for replication. You can select both object specifiers and path object specifiers to the same group, although DataMirror recommends that you set up two groups.For more information about path object specifiers, see Path Object Specifiers on page 53.

Generic Object SpecifiersA generic object specifier has one or more of the following properties:• A generic name component. For example, AB*.• Name component of *ALL.• Type component of *ALL or *FILE.• Type component of *FILE and attribute component of *ALL.Note: Note that the attribute component is ignored for object types other than *FILE.

Rules of Precedence for Object SpecifiersThe following numbered list outlines the rules of precedence that iCluster applies when an object matches several object specifiers (1 = highest precedence, 3 = lowest precedence):1 Exact specifiers (non-generic) have the highest precedence. INCLUDE and EXCLUDE specifiers have the same precedence

when they are exact specifiers.2 Generic EXCLUDE specifiers have the next highest precedence.3 Generic INCLUDE specifiers have the lowest precedence.

*ALL LIB1 *ALL Not applicable 1, 11

M3 *TEST *ALL Not applicable 5, 6

*ALL TEST *ALL Not applicable 5, 6, 7

S1 COM *ALL Not applicable 9, 10

H *SLIB *PGM Not applicable 8

Table 3 List of Objects Addressed by Example Specifiers

Object Specifier Addressed Objects (See # in Table 2)

Object Library Object Type Object Attribute


There is no precedence among generic specifiers based on the length of the non-generic part. For example, A* as an EXCLUDE specifier has higher precedence than AB* as an INCLUDE specifier. An object named ABBA will therefore be excluded from replication.Note that there are parameters that you can specify on the object specifier that will determine, depending on the object type, how the objects matching the object specifier are replicated. These include the following:• Object polling interval - POLLINT parameter• File update method - PFUPDMTD parameterSee DMDSELOBJ—De-select Objects from Group (Deselect Object Specifier for iCluster Administrator) for more information on these parameters and how to select an object specifier for a replication group.

Native Object Specifier Changes While Replication is ActiveiCluster supports the following with respect to native object specifier changes while replication is active:• Addition of new *INCLUDE object specifiers while replication is active for native object specifiers.• Activation (with or without refresh) of objects coming into replication scope due to the addition of new *INCLUDE object

specifiers.• Metadata maintenance for objects due to the addition of new *INCLUDE object specifiers. As a result, there is no need for a

user-specified starting journal position.

Path Object SpecifiersPath object specifiers are used to refer to objects within the Integrated File System (IFS) that you want to replicate.IFS is a OS/400 feature that supports a hierarchical directory structure similar to that found on personal computers and UNIX systems. The IFS integrates support for a number of file systems, such as QDLS, the "root" file system, and the QOpenSys file system. You can use IFS to port PC or UNIX-based applications to an iSeries system. IFS is the name used to refer to the file system, but references to the actual BSF objects that reside in this system will be through path object specifiers. See Replicating BSF Objects on page 89 for more information.

Naming ConventionsWhen creating a path object specifier, you indicate the full path name where the BSF objects that you want to replicate reside on the primary node. For example, '/Dir1/Dir2/Dir3/Dir4/file' and '/Dir1/Dir2/Dir5/Dir6/*. The path name must be contained in single quotes.Note the following regarding the definition of path object specifiers:• The path must start with a '/' (forward slash) character ('/Dir1').• The path cannot end with a '/' (forward slash) character ('/Dir3/Dir4/file').• The path can have a maximum of 5000 characters. You must enter at least two characters for the path.• Generic path names of the form '/mydir*' are supported, where the generic indicator '*' is the final character of the path name.Note: DataMirror strongly recommends that you do not specify '/*' because this specifier matches the entire IFS

system, including objects that should not be replicated from one system to another.• If you are creating a path object specifier that references multiple directories, you can create an exclude object specifier by

using the following methods:• Exclude a particular directory and its contents, including its sub-directories, from replication by adding an asterisk (*) to

the last character of the file that you want to replicate.• Create an exclude object specifier that matches the object that you do not want replicated.


• To exclude from replication a directory named '/Dir3/Dir4', then you should specify the following path object specifier EXC '/Dir3/Dir4*' (where EXC indicates an exclusion). In contrast, specifying EXC '/Dir3/Dir4/*' will exclude the contents of the '/Dir3/Dir4' directory, but not the directory itself.

• The longer the non-generic portion of the path name, the higher the precedence it has.

Journaled and Non-journaled Path Object SpecifiersNon-journaled BSF support is appropriate for applications that create large numbers of files that are not changed after they are created or, if the file is changed, it is actually replaced with a new version of the file. For example, an application that includes photographs of objects is unlikely to ever edit the photographs. The photograph is added and may be deleted later, but will not be changed. Journaling of the files that contain the photographs is not required for replication and the overhead of journaling can be avoided by mirroring the files using the non-journaled BSF support.Journaled BSF support is appropriate for applications that store modifiable data in BSF objects. These applications change the contents of the data contained in the BSF objects and, therefore, journaling is required to mirror those changes in real time.See DMADDBSF—Add Path Object Specifier to Group (Add Full Replication Group for iCluster Administrator) for more information on non-journaled BSF support or Replicating BSF Objects on page 89 for more information on BSF objects.


Auto-configuration of iCluster

iCluster Quick StartThis section contains the following topics:• Auto-configuration of iCluster on page 55• Quick Start for the iCluster Command Line on page 57• Starting iCluster from the Command Line on page 58• Quick Start for iCluster Administrator for Windows on page 59• Logging on to iCluster Administrator for Windows on page 60

Auto-configuration of iClusterThe DMAUTOCFG—Auto-configure iCluster command enables you to configure iCluster automatically. This command is not available from the iCluster menus but runs immediately and interactively from the command line. For an example of how to configure your cluster by manually running the commands, see Quick Start for the iCluster Command Line.

DMAUTOCFG—Auto-configure iClusterDMAUTOCFG PRMNODE( ) PRMIPADR( ) BCKNODE( ) BCKIPADR( ) PRMGRP( ) SECGRP( ) SELOBJO( )

This command automatically configures iCluster based on a typical installation. It also displays a list of all libraries on your system. By using this command, you do not have to issue seperate commands to create nodes, groups, object specifiers for each library, default journals, or set the iCluster system values. Once the DMAUTOCFG command is run, you can modify the configuration to fit the business requirements for High Availability.For this command to work, the iCluster product subsystem (XDMCLUSTER) and the DMCHATL job must be active on both nodes. The command will prompt for primary and secondary node names and IP addresses. Optionally, you can specify primary and secondary group names or use PRIMARY and SECONDARY as the default group names. To use this command, the TCP port for the dmcluster service must be 4545. This is the default port setting. For more information, see Running iCluster Under TCP/IP on page 37.This command does the following:• Creates the iCluster primary and secondary nodes using the names and IP addresses specified.• Creates the default journals HADJRN and HABSFJRN and journal receivers HAD0000001 and BSF0000001 in a library

called AAAJRNLIB.• Sets the iCluster product system values to use the journals in AAAJRNLIB as the default journals.• Creates two replication (type *REPL) groups with the names defined in the PRMGRP and SECGRP parameters. Both

groups use the default journals that are created in the AAAJRNLIB library. A third group called SYSTEM will be created if the current installation is an iCluster Enterprise installation.

• Creates object specifiers for all user profiles, authorization lists (excluding those whose name begins with the letter 'Q'), output queues (excluding those whose name begins with the letter 'Q'), and the QGPL library (excluding all objects whose name begins with the letter 'Q') and these specifiers will be added to either the SYSTEM group for iCluster Enterprise or into the Secondary group for iCluster SMB.

The command displays all user libraries (excluding objects whose name begins with the letter 'Q' and DataMirror libraries) and allow you to choose object specifiers for the Primary and Secondary groups by library name, or just use F13 to place everything into the Primary group.


Once iCluster has been configured, you can also run this command by specifying the additional parameter for select object specifiers only (SELOBJO). This option will bypass the steps mentioned above (adding nodes, configuring groups, etc.) and just display the list of libraries for selecting object specifiers.Adding object specifiers to the iCluster groups will turn on object auditing for the libraries, thus modifying the last updated date and time, so the next SAVCHGOBJ may save more objects then expected. It is recommended that a full save be done before the next SAVCHGOBJ.

Input Parameters• PRMNODE

Specify the name of the primary node for iCluster. If you have multiple OS levels within the cluster, this should be the node with the lowest (earliest) i5/OS release.This is a required parameter.

• PRMIPADRSpecify the IP address or the fully qualified host name of the primary node specified for the PRMNODE parameter specified above.The IP address of the node can be specified in dotted quad notation (142.4.15.133) or in domain name form (prod400.mycorp.com).This is a required parameter.

• BCKNODESpecify the name of the secondary node for iCluster.This is a required parameter.

• BCKIPADRSpecify the IP address or the fully qualified host name of the secondary node specified for the BCKNODE parameter above.The IP address of the node can be specified in dotted quad notation (142.4.15.133) or in domain name form (prod400.mycorp.com).This is a required parameter.

• PRMGRPSpecify the name of the primary iCluster group. The command will display a list of all user libraries (*ALLUSR) on the system. You can use F13 to select all of these libraries by default to this group, or you will have the option of choosing which libraries should be selected to this group by placing a P (for primary) next to the library name.Enter a group name or use the following default value:• PRIMARY—The default name of the primary group.

• SECGRPSpecify the name of the secondary iCluster group. If iCluster SMB is installed, all system objects will be added to this group. You will be presented with a display of all user libraries (*ALLUSR) on the system and you will have the option of using F13 to add all libraries to the previously specified Primary group or choose which libraries should be selected to this group by placing an S (for Secondary) next to the library name.Enter a group name or use the following default value:• SECONDARY—The default name of the secondary group.

• SELOBJOSpecify whether you want to select object specifiers when running this command.Specify one of the following values:


Quick Start for the iCluster Command Line

• *YES—Indicates that you want to run this command but only select object specifiers. The groups, nodes, and other objects must have already been created by a prior run of this command. The command assumes all nodes, groups and journals have been created and will go directly to the display of all user libraries and will allow the user to select which libraries should be selected to which group.

• *NO—Indicates that the command will run in its entirety and will create nodes, groups, journals, and set iCluster system values.

The default setting for this parameter is N.

ExamplesDMAUTOCFG PRMNODE(400P) PRMIPADR(400P) BCKNODE(400Q) BCKIPADR(400Q) PRMGRP(PRIMARY) SECGRP(SECONDARY) SELOBJO(N)

Creates a primary node named 400P.Creates a backup node named 400Q.Creates a primary group using the default value of PRIMARY.Creates a secondary group using the default value of SECONDARY.The command will run in it’s entirety and will create nodes, groups, journals, and set iCluster system values.

Minimum Security LevelAdministrator (*ADMIN)

Menu AccessThis is an interactive command and it is not available from the menus.

Quick Start for the iCluster Command LineThis topic describes some sample steps to help you create a cluster with a replication group. After completing the procedure, you will have a working cluster with data that has been replicated from one node to another. Many of the commands in this topic have parameters that are not included in this tutorial. See iCluster Commands on page 97 for a complete list of parameters. Before starting, you must be logged onto a node as a user with iCluster *ADMIN authority. This node will be the master node in the cluster and the primary node for the replication group. You will also need to have values for the following placeholders.Table 4 Placeholders for Quick Start

Placeholder Description

<lib> The iCluster product library. This value was defined during the iCluster installation process. The default library name is DMCLUSTER.

<port> The iCluster TCP/IP listener port. This value was defined during the iCluster installation process. The default port is 4545.

<primary> The name of the primary node. You will use this name in both the node name and IP address fields.

<backup> The name of the backup node. You will use this name in both the node name and IP address fields.


Perform the following tasks on the primary node to complete this tutorial:1 Change to the iCluster library by entering the following commands:> CHGCURLIB <lib>

> GO DMCLUSTER

2 Create a library and data area object to replicate by entering the following commands:> CRTLIB LIB(MYLIB) TYPE(*TEST) TEXT('TEST LIBRARY')

> CRTDTAARA DTAARA(MYLIB/MYDATA) TYPE(*CHAR)

> CHGDTAARA DTAARA(MYLIB/MYDATA) VALUE(HELLO)

3 Add a primary node to the cluster. This node must be the node you are currently logged into. Enter the following command to do this:

> DMADDNODE NODE(<primary>) IPADDR(<primary>) PORT(<port>) PRODLIB(<lib>)

4 Add another node to the cluster. This node will be the backup node for the replication group.> DMADDNODE NODE(<backup>) IPADDR(<backup>) PORT(<port>) PRODLIB(<lib>)

5 Add a replication group between the primary and backup nodes.> DMADDGRP GROUP(MYGROUP) GRPTYPE(*REPL) PRIMNODE(<primary>) BACKUPS(<backup>)

6 Select the object you want to replicate with the group. For this tutorial, the object is the data area you created in Step 2.> DMSELOBJ GROUP(MYGROUP) OBJ(MYLIB/MYDATA) OBJTYPE(*DTAARA)

7 Start the replication group. This mirrors the data area to the backup node.> DMSTRGRP GROUP(MYGROUP) REFRESH(*YES) USEMARKED(*NO)

8 Verify the replication by viewing the data area on the backup node. You can do this by running the following command on the backup node:

> DSPDTAARA DTAARA(MYLIB/MYDATA)

This displays the data area you replicated from the primary node.

Related TopicsFor more information on the steps in this topic, see the following topics:• Working in a Clustered Environment on page 43• Starting iCluster from the Command Line on page 58• DMADDNODE—Add Node on page 99• DMADDGRP—Add Group on page 113• DMSELOBJ—Select Objects to Group on page 141 • DMSTRGRP—Start Cluster Operations at Group on page 208Refer to your IBM documentation for more information on the other commands in this tutorial.

Starting iCluster from the Command LineTo invoke iCluster commands, you must change your current library to the iCluster product library that you specified during installation. For more information, see Step 10 in the installation procedure.


Quick Start for iCluster Administrator for Windows

To invoke iCluster from the command line:1 Issue the following command on an OS/400 command line:> CHGCURLIB <iCluster product libcrary>

where <iCluster product library> is the name of the library where iCluster was installed.iCluster also provides menus from where you can issue commands.2 To display the DataMirror iCluster Main Menu, issue the following command:> GO DMCLUSTER

In addition to the main menu, which lists the commands that you are most likely to use on a regular basis, another menu (DataMirror iCluster Commands) is provided that lists all supported iCluster commands. From most iCluster screens, you can access this menu by pressing F22 (SHIFT+F10).For each command described in this document, the menu option number(s) that you can use to invoke the command are identified.

Quick Start for iCluster Administrator for WindowsThe iCluster Administrator provides functions for refining a cluster configuration. Below is a set of basic steps that you should follow to start cluster operations quickly in your working environment. Perform the following steps after you have installed iCluster on each node, and installed the iCluster Administrator and iCluster Event Viewer on a Windows workstation:1 Register your existing OS/400 user name in iCluster.

The OS/400 user name must be defined on a node in the cluster, and registered in iCluster through the DMADDUSR—Add User command.You must have system administrator or *SECOFR authority to issue this command. The iCluster password that you specify will be used in Step 2.

Note: It is recommended that OS/400 user names be registered in iCluster immediately after installing the product on each node in the cluster. See the Installation and Upgrade for more information.

2 Start and log on to the iCluster Administrator.The iSeries system that you must specify during the log on process must be the one where the DMADDUSR—Add User command was issued (see Step 1). For more information, see Logging on to iCluster Administrator for Windows on page 60.

3 Add the master node to the cluster.The first node added to the cluster is considered as the master node. This node must be the iSeries system that you specified during the login process (see Step 2). See the Add Node command.See Working in a Clustered Environment on page 43 for more information about the master node.

4 Add other nodes to the cluster.When adding other nodes to the cluster, ensure the iCluster Administrator is connected to an active node in the cluster. To add other nodes to the cluster, use the Add Node command again.

5 Add replication groups, MQSeries groups, and resilient applications.When adding replication groups, MQSeries groups, and resilient applications, you will identify the nodes in the recovery domain for new replication groups, resilient applications, and MQSeries groups.


To add replication groups, use the Add Full Replication Group command. To add resilient applications, use the Add Resilient Application command. To add MQSeries groups, use the Add MQSeries Group command.

6 Select object specifiers to replication groups.This step is required only if you added replication groups in Step 5.To select object specifiers to replication groups, see Select/Add Object Specifier.

7 Synchronize objects on primary and backup nodes.Objects on the primary node must be synchronized with the same objects on the backup node before mirroring is started. You can perform this step in one of the following ways:On the primary node, save the objects to a save file or tape, and restore them on the backup node. Issue the Mark Journal Positions command for the replication groups and resilient applications defined earlier. When you start cluster operations for the groups and applications (see Step 8), set Use Marked Journal Positions to *YES.When you start cluster operations for the replication groups and resilient applications (see Step 8), set Refresh Selected Objects to *YES, and set Use Marked Journal Positions to *NO.

8 Start cluster operations.This step starts mirroring objects. The command you need to issue depends on the objects that you want to mirror.To start mirroring within replication groups, use the Start Full Replication Group command. To start mirroring for resilient applications, use the Start Resilient Application command. To start mirroring for MQSeries groups, see Start MQSeries Group.

Logging on to iCluster Administrator for WindowsThis section explains how to log on to the iCluster Administrator.

To log on to iCluster Administrator for Windows1 From the Windows Start button, select the program group you specified for iCluster during installation. 2 Select iCluster, then iCluster Administrator from the submenu.

The iCluster Administrator Login dialog box opens.


Logging on to iCluster Administrator for Windows

This dialog box allows you to log on to a specific iSeries node from the workstation where you have installed iCluster. A successful log in is required to work with the different elements in your cluster configuration.The version of iCluster Administrator is displayed under Version Information.

3 In the User Name box (under User Information), specify the OS/400 user name that you want to use to log in to the system.The specified user name must be defined as a user profile on the iSeries system that is identified in the Node List box (see below), and the same name must also have been defined in iCluster through the DMADDUSR—Add User command. You can issue this iCluster command only locally on the iSeries system, and not through the iCluster Administrator.

4 In the Password box (under User Information), specify the iCluster password that is associated with the user name specified in the User Name box (see above).

The password that you enter in this box is the one that was specified through the DMADDUSR—Add User (see above) or DMCHGUSR—Change User command when the user name was defined in iCluster on the iSeries system identified in the Node List box (see below). It is not the password that is used to log on to the iSeries system under that name. To change an iCluster password, the DMCHGUSR—Change User command has to be issued on the iSeries system with the PASSWORD parameter set to the new password.Note: The password is case-sensitive.5 In the Node List box (under Connection Information), specify the name of the iSeries system where the login procedure will

be performed.You can select system names specified in previous login attempts.

Note: If you are logging in for the first time, you will have to enter the system name in the IP Address box (see below).

6 In the Host Name box (under Connection Information), specify the Host Name or the IP address of the iSeries system where the login procedure will be performed.You can also enter an iSeries system name (for example, NODE1) in this box. The IP address of the node can be specified in dotted quad notation (for example, 125.4.3.55) or in domain name form (for example, as400sys1a.abccorp.com).


7 In the Port box (under Connection Information), specify the port number on the iSeries node that will be used for communications with the workstation where iCluster Administrator is running.This port number was specified during iCluster installation on the iSeries node. See Running iCluster Under TCP/IP in the iCluster Installation and Upgrade for information about the port number specification. The default port number is 4545, but consult your iSeries system administrator to obtain the port number you should specify.

8 Ensure that the iCluster Administrator box (under Windows) is checked.By default, this box is checked.

9 Check the iCluster Event Viewer box if you want to display the iCluster Event Viewer.See Starting the iCluster Event Viewer for more information.

10 Click Execute.If the attempt to log on is successful, the iCluster Administrator window opens.

This window allows you to configure your clustered environment by defining the nodes, replication groups, objects, resilient applications, and other elements that interact to support high availability.An explorer-style interface is provided so that you can view the relationship between different elements, and work with specific items that have been defined in iCluster Administrator. The iCluster Administrator consists of two main panes:• iCluster tree view: Displays the relationship of nodes, groups, and object specifiers. The tree view represents the

iCluster hierarchy and status in real time. Each entity may display different icons depending on status.• iCluster table view: Displays detailed information concerning the currently selected item on the left-hand side of the

window.


iCluster OperationsThis section discusses the following topics:• Command Security on page 63• Adding and Configuring Nodes on page 67• Initial Synchronization on page 68• Application Resiliency on page 68• Staging Objects on page 69• Locked Objects on page 70• Avoiding Contention on page 70• Commitment Control on page 71• Suspended Objects on page 72• Choosing a Failover Mechanism on page 73• Starting Replication and Journal Positions on page 75• Monitoring Out-of-Sync Objects on page 77• Monitoring Latency on page 78• Restarting iCluster after Restarting a Node on page 79• Upgrading the Cluster Version on page 80• Changing IP Addresses on page 80• Journaling in iCluster on page 81• Passing Arguments to Sync Point User Exit Programs on page 86

Command SecurityiCluster ensures operational security by restricting the ability to run commands to users based on user access levels. This topic lists the commands that are available to users with each access level.

Operating System AuthorityThis section lists commands that require OS/400 authority to run.

Commands Available to any OS/400 UserAny OS/400 user with access to the iCluster library can run the following commands:• ADDPFEXPGM• DMDSPAPPGP • DMSNDOBJ • DMSYSINF • DSPHASC • INITHAOBJ • RBDHAMQM• RTVRCVPTR


• STRHASC• VFYHAJRN• WRKHASMON • CHGHAJRN • DMDSPASPGP • DMWRKOBJST• DSPHASMON• JRNHADADQ• RMVPFEXPGM • SELSCATTR • STRHASCUSR • WRKHABSFST • WRKHATMON• CHGHASMON • DMSCRPT • DSPHABRCD• ENDHABRCD • PRGHASC • RTVHAPOS • SETHAREG • STRHATCP • WRKHAJOB• CRTCFGOBJ• DMSCRPTNTV • DSPHAPOS • HAPNGTCP • PRGHASMON • RTVRCVPT • STRHABRCD • SYNCHATRG • WRKHAOBJST

Commands Available to OS/400 Security OfficersThe following iCluster commands can only be run by users who have *SECADM authority are signed in as QSECOFR:• DMADDUSR • DMCHGUSR • DMRMVUSR • DMWRKUSR


Commands Available to *ALLOBJ UsersOnly users who have *ALLOBJ authority or are signed in as QPGMR, QSYSOPR, or QSRV can run the DMCHGTIME command.

iCluster AuthorityWhen adding iCluster users with the DMADDUSR command, the system administrator assigns an authority level to the user. The possible authority levels are *USER, *OPERATOR, and *ADMINISTRATOR.

iCluster User CommandsiCluster users with *USER authority can monitor and inspect objects in the cluster. They have access to the following commands:• DMCLRLOG • DMWRKAPP • DMWRKOBJ • DMDSPGRP • DMWRKBSF • DMDSPLOG • DMWRKGRP • DMDSPNODE • DMWRKNODE

iCluster Operator CommandsiCluster operators, users with *OPERATOR authority, can initiate cluster operations, view journal positions, and see cluster system values. Operators can run all *USER authority commands as well as the following commands:• DMACTBSF • DMENDGRP • DMLOGENT • DMSETPOS • DMSTRGRP • DMSUSOBJ • DMACTOBJ • DMENDNODE • DMSETSYNC • DMSTRNODE • DMGENEXC • DMENDAPP • DMMRKPOS • DMSTRAPP • DMSTRREPL • DMENDAPY • DMREGPOS • DMSTRAPY


• DMSUSBSF

iCluster Administrator CommandsiCluster administrators, users with *ADMINISTRATOR authority, can configure all cluster aspects except for user operations. Administrators can run all * OPERATOR and *USER authority commands as well as the following commands:• DMADDAPP • DMADDNODE• DMAUTOCFG• DMCHGGRP • DMCHGSWDEV • DMREJOIN • DMRMVBSF • DMSELOBJ • DMSTRSWO• DMADDBACK • DMADDSWDEV • DMCHGNODE • DMDLTCLSTR • DMRMSWDEV • DMRMVGRP • DMSETPRIM • DMSWOAPP• DMADDBSF • DMCHGAPP • DMCHGOBJSL • DMDSELOBJ • DMRMVAPP • DMRMVNODE • DMSETSVAL • DMUPDAPP• DMADDGRP • DMCHGBSF • DMCHGROLE • DMRBDNODE • DMRMVBACK • DMRMVSWDEV • DMSTRDM


Adding and Configuring NodesThe following information pertains to the configuration and addition of nodes to your clustered environment through iCluster.

PrerequisitesNodes must meet the following requirements before you add them to a cluster:• The first node you add to a cluster must be able to communicate with every other node in the cluster.• The node's Internet Daemon (INETD) must be running. You can start this by running the following command:STRTCPSVR SERVER(*INETD)

DataMirror also recommends adding this command to the autostart job that runs when then the machine loads.• The XDMCLUSTER subsystem must be running on all of the other nodes currently in the cluster. The DMCHATL listener job

must also be running in this subsystem. See STRHATCP—Start TCP/IP Listener on page 202 for more information on the listener job.

Note: Never stop the XDMCLUSTER subsystem on a node that is currently in a cluster.• DataMirror recommends setting the event log's message generation level to *ALL. This ensures that all cluster information is

recorded in the node operations fail. After configuring your cluster, you can lower this level to meet your business needs. See DMSETSVAL—Set Cluster System Values on page 181 for more information.

If you are using the IBM CRS failover mechanism, then nodes must also meet the following requirement:• Nodes must have the ALWADDCLU network attribute set to either *ANY or *RQSAUT. If *RQSAUT is the network attribute,

then you must also have the Digital Certificate Manager and a Cryptographic Access Provider installed on the node. See your IBM documentation for more information.

See Choosing a Failover Mechanism on page 73 for more information.

Adding NodesTo add a node to a cluster, call the DMADDNODE—Add Node command. After the command completes, the node will be automatically started in the cluster. See the DMADDNODE—Add Node command for more information.

Changing NodesTo change a node that is currently in a cluster, call the DMCHGNODE—Change Node command. You cannot change a node if it has any active replication groups. See the DMCHGNODE—Change Node command for more information.

Removing NodesNode information is shared between the nodes in a cluster when they are active. This requires that at least one node in the cluster must be active before you remove any nodes from the cluster. If all of the nodes in a cluster have stopped, then the nodes will be unaware of the change. This can corrupt your cluster configuration.The following procedure illustrates a possible way to safely remove nodes.1 Open the Work with Nodes screen by running DMWRKNODE—Work with Nodes from the command line. Alternatively, you

can select 1 on the commands menu.2 Verify that at least one node has a status of *ACTIVE. This does not include the status of the node you are removing. There

is no restriction on the status of the node you are removing.3 Enter option 6 next to the node you want to remove from the cluster. See DMRMVNODE—Remove Node for additional

considerations before you remove the node.4 Press the Enter key to confirm the operation.After performing these steps, the node will be removed from the cluster.


Initial SynchronizationThere are several issues that you should consider before starting replication with iCluster.

System ValuesiCluster supports system values that can be used to control different behaviors of your cluster. System values should be reviewed and initialized prior to starting replication for the first time. See DMSETSVAL—Set Cluster System Values (Setting System Values for iCluster Administrator) for more information on setting and viewing system values.

Refresh (Before Mirroring)Initiating a refresh operation replicates the selected objects in a group from the primary node to the backup node before mirroring begins. Since mirroring only replicates changes made to objects on the primary node, refreshing the data ensures the object on the backup node has changes made before mirroring began. Use the REFRESH parameter of DMSTRGRP—Start Cluster Operations at Group (Start Full Replication Group for iCluster Administrator) to initiate this operation.When refreshing physical files in iCluster, you can choose whether these files on the primary node are locked or can be modified during refresh operations when all data is sent across the network.

MirroringAfter synchronizing objects on primary and backup nodes through a refresh, mirroring ensures that updates applied to group objects on the primary node are replicated immediately to the backup node. Mirroring occurs automatically in any group that is started and has a status of *ACTIVE. You can start a group with DMSTRGRP—Start Cluster Operations at Group (Start Full Replication Group for iCluster Administrator).Mirroring maintains an up-to-date image of objects on the backup node. This ensures that a switchover or failover to the backup node can be performed without disrupting business operations. See Failovers and Switchovers on page 47 for more information. Mirroring is a conceptually continuous operation that is only stopped as a result of anticipated or unplanned interruptions.

Application ResiliencyIn addition to moving objects in your clustered environment, you will also want to have the same applications running on a new primary node after a switchover or a failover occurs. Applications that can be re-started automatically after a switchover or failover are called resilient.Resilient applications are designed by their software vendors to operate in a clustered environment. This means that if a primary node stops, the same resilient application installed on the backup node can be started with minimal or no disruptions to service. Application vendors provide the necessary data areas and files that allow their applications to operate in a clustered environment. You should check with the application vendor whether the application has been designed to operate in a clustered environment.In iCluster, you do not have to be aware of the configuration information of the resilient application. Instead, commands are provided in iCluster to add and change a resilient application. Other commands allow you to start and end cluster operations for resilient applications. You can also update a resilient application in iCluster when the data areas and files provided by the application vendor change on the primary node.Note: The resilient application and configuration information must reside on all nodes in the recovery domain in

order to support a resilient application switchover or failover.See the following topics for more information:• DMADDAPP—Add Resilient Application on page 167, Add Resilient Application for iCluster Administrator• DMCHGAPP—Change Resilient Application on page 172, Change Resilient Application for iCluster Administrator• DMSTRAPP—Start Cluster Operations for Resilient Application on page 217, Start Resilient Application for iCluster

Administrator• DMENDAPP—End Cluster Operations for Resilient Application on page 218, End Resilient Application for iCluster

Administrator• DMUPDAPP—Update Resilient Application on page 172, Update Resilient Application for iCluster Administrator


Staging ObjectsStaging is a storage mechanism of iCluster that holds journal entries on a backup node before they are applied to the target objects. This allows the apply jobs to be ended and backup and other maintenance operations to be performed that require exclusive access to files and objects. Cluster operations for the group are allowed to continue, but changes will not be applied on the backup node until the apply jobs are started again.Ending the apply process for a backup node does not affect processing on the node that is replicating objects until all available staging storage has been filled. The amount of space available for the staging store is set for each node with the staging store size parameter on the DMADDNODE—Add Node (Add Node in iCluster Administrator) and DMCHGNODE—Change Node (Change Node in iCluster Administrator) commands.If cluster operations for the group are active when the apply process is stopped, the backup node will continue to receive journal information which will be placed into the staging store. If the staging store becomes full, then journal scraping will stall.The apply process on a backup node can be stopped or started independently of any cluster operation. If the group is not currently active, the apply process will end once the staging store has been drained.

Staging Store and AllocationThe staging store is a non-volatile storage area that is managed on a node basis. The size of the staging store must be set for each physical node in the cluster.The staging store is a non-volatile storage mechanism for backup nodes that is managed on a node-by-node basis. The size of the staging store must be set for each backup node in the cluster.All information transferred to backup nodes will travel through the staging store and assuming the apply process is active, will be applied. If the apply process is not active, the staging store will store all journal entries until the apply process is started.You can increase the size of the staging store at any time, and you can change the maximum size of the staging store. iCluster will use extra space allocated if needed while active. A decrease in the size of the store will only be made if the total size of transactions in the store is less than the maximum size of the store and all iCluster replication is ended on the node.If necessary, you can also choose to force drain the staging store when the apply process is started. Normally, the apply process merges entries from the audit and database channels and applies them in sequence. When one channel becomes empty, the apply process will stop regardless of any entries remaining in the other channel. When the staging store is force drained, then the merge will stop once the last entry of the channel with fewer entries is reached, and the apply process will then drain the other channel until it is empty.

Clearing the Staging StoreiCluster will clear the portion of the staging store reserved for a group, regardless of whether those entries have been applied on the backup node, in the following instances:• When you start replication with the Use marked journal positions parameter set to *YES.• When you start replication after issuing the DMSETPOS—Set Journal Start Position (Set Journal Positions in iCluster

Administrator) command.• When you start replication with a refresh before mirroring (Refresh before mirror parameter is set to *YES).If you want the entries in the staging store to be applied on the backup node, then you need to apply them before starting replication in any of the three cases listed above. See the DMSTRAPY—Start Replication Apply Process (Start Apply Process in iCluster Administrator) command for more information.

LOB Staging StoreLOB data is stored in a directory in IFS which is separate from the regular staging store. See Replicating LOBs on page 92 for more information about the LOB staging store.


ConsiderationsStaging increases system resources because journal data will be placed into and extracted from the staging store. Also, the system storage requirements may need to be increased to hold the staged journal entries. There will be no effect on performance or resource requirements on primary nodes.The scrape latency will not be affected while the apply process is suspended as long as the staging store is large enough to hold all the journal information. Stalling may occur if the staging store is not large enough to hold the journal entries. For example, transferring large objects can cause current products to stall as communication buffers back up due to a slow apply process.

Locked ObjectsCluster operations will sometimes lock objects to ensure that only one process can affect the object at a time. This topic provides information about locked objects in iCluster. For more general information on locked objects, see the IBM documentation.

Setting the Retry LimitYou can specify the number of attempts the system makes to save a locked object by creating a data area on the primary node. The data area must have a decimal 10.0 format and its value will be the maximum number of times you want the system to try to save locked objects.For example, the following command sets the retry limit in the current product library to 30 attempts:CRTDTAARA DTAARA(HASAVRETRY) TYPE(*DEC) LEN(10 0) VALUE(30)

iCluster waits for about one to two seconds between each attempt to save the locked object. Setting the retry limit to 30 allows iCluster to retry the operation 30 times, which will last approximately 60 seconds.

Locked Object ContentionIf the apply processes on the backup node cannot apply a journal entry to a file, since it is exclusively locked either by another application or by the operating system, you can configure the apply processes on the backup node to wait a certain amount of time and retry the apply process a specified number of times. This is achieved by creating a data area object in the iCluster library on the backup node. The data area should be named HA_OBJLCK, be of type *CHAR, and it must be long enough to contain the wait/retry specification string. Specify the following parameters:WAIT <n1> RETRY <n2>

where:‘WAIT’ and ‘RETRY’ cannot be in mixed case (only all uppercase or all lowercase).<n1> can be any positive integer except zero (0). It represents the number of seconds to wait.<n2> can be any positive integer or *NOMAX. It represents the number of times to retry the apply process.The target apply job will retry the record update every <n1> seconds until the uniquely keyed access path has been rebuilt or <n2> attempts have been made. If the last attempt fails, the file will be suspended.If the HA_OBJLCK data area is not created, the default is to wait one second and retry five more times.

Avoiding ContentionIf an application upgrade procedure is running at the same time that replication is active, changes to files during the upgrade may interfere with replication or vice-versa. Sometimes it is necessary to delay the refresh of a file by iCluster to allow the upgrade process to complete before proceeding with the refresh.You can create the HA_DELAY data area in the product library to modify the default behaviour of iCluster so that if a refresh of a file is required, iCluster will delay processing of the refresh by the amount of time you specify.


The HA_DELAY data area affects any refresh of a file. iCluster will refresh a file when it processes a journal entry that indicates the file was created, restored, activated, or moved/renamed so that it now matches the object specifiers of the group. You can set up the HA_DELAY data area in the following way:1 Stop mirroring.2 Create a character data area called HA_DELAY in product library.3 Put delay info into this DTAARA in a following format:

<*ALL++++++><GROUP NAME1><DELAY1 IN SECONDS>*CHAR(10) *CHAR(10) *CHAR(5) <*ALL++++++><GROUP NAME2><DELAY2 IN SECONDS> <*ALL++++++><GROUP NAME3><DELAY3 IN SECONDS>

Group name may be specific or *ALL.

Example 1'MYTARGET1+MYGROUP7++100++MYTARGET2+*ALL++++++90+++'

Here + is a <Space> padding and for the group MYGROUP7 delay is set to 100 seconds, whereas for all groups selected to the backup node delay is 90 sec.

Example 2'*ALL++++++MYGROUP+++120++'

Commitment ControlCommitment control stages database transactions so that they are assembled before being applied, or additionally, opened in a commitment control environment to ensure that only complete transactions are applied. It also makes sure that the changes are applied in the correct sequence. Commitment control staging is performed on backup nodes. You can specify commitment control at the replication group and system levels.iCluster supports *LEVEL1, and *LEVEL2 commitment control, or no commitment control at all.

*NONEIf there is no commitment control, the update processes on the backup node will not perform commitment control staging, will not open the files under commitment control and will not apply updates under commitment control.

*LEVEL1 Commitment ControliCluster assembles all updates that comprise a transaction before applying them on the backup node. *LEVEL1 does not require that the backup files are journaled, it will not open the files under commitment control and will not apply the updates under commitment control. If the backup node ends abnormally in the middle of applying a transaction, the transaction will become partially applied and iCluster will complete the transaction at next startup. This option may be useful to those who do not want to journal the backup node files for performance reasons but do want transaction consistency.

*LEVEL2 Commitment ControliCluster makes sure that all updates in a transaction are opened in a commitment control environment to ensure that only complete transactions are applied. *LEVEL2 requires that backup files are journaled with both images. It will then open the files under commitment control and apply the updates under commitment control. If the backup or replicate node ends abnormally in the middle of applying a transaction, the system will automatically rollback the transaction, ensuring that the database is still consistent. This option provides true commitment control.

Commitment Control ConsiderationsA suspended file cannot be involved in transactions under commitment control. Updates to suspended files will not be applied as part of a committed transaction. For more information about suspended files, see Suspended Objects on page 72.


Suspended ObjectsDuring normal replication, all changes to replicated objects are applied on the backup node. However, there are circumstances where objects may become suspended, meaning that one or more changes could not be processed. Potentially, the primary and backup nodes could become unsynchronized.If a suspended object is a part of the transaction under Commitment Control, the transaction will not be applied on the backup system in its entirety.

Suspending ObjectsAn object can become suspended in several ways. The following list describes some possible causes:• If it cannot be refreshed or if an iCluster file or member level operation fails.• If you set a cluster system value to indicate the maximum size of a file that can be refreshed through the network. Any file that

is larger than this parameter will not be refreshed if required, and instead, will be marked as suspended. See DMSETSVAL—Set Cluster System Values on page 181 (Setting System Values for the iCluster Administrator) for more information.

• If a manual refresh is specified when selecting an object specifier to a replication group. You may want to perform a manual refresh to control the size of objects being transferred through the network or to have some control over the number of "stalled" journal entries (an entry that has been delayed in being sent to the backup node). Note that you are responsible for a manual refresh from the primary node to the backup node.

• If you select a certain object to be suspended. The reason why an object was suspended by the user will be displayed through the Status Monitor.

• If it cannot be restored to the backup node because it is locked by an application.In most cases, suspended objects can be automatically reactivated by iCluster. You can reactivate any object that is not automatically reactivated using the DMACTOBJ—Activate Object (Activate Object in the iCluster Administrator) command for native objects and DMACTBSF—Activate BSF Object (Activate Object in the iCluster Administrator)command for IFS objects. See Activating Objects and Automatic Reactivation for more information.

Activating ObjectsMirroring of suspended objects begins when you activate the objects. You can activate one or more suspended objects through the DMACTOBJ—Activate Object (Activate Object in the iCluster Administrator) command for native objects, DMACTBSF—Activate BSF Object (Activate Object in the iCluster Administrator) command for BSF objects, or from the Status Monitor. Objects can be refreshed as a part of the activation or mirror can begin on activation. You do not have the option to specify a particular time or journal entry at which to start mirroring for the file being activated.Note: If you do not refresh the objects when activating them, then you must ensure any logical files associated with

a suspended physical file are created on the backup node before activating the physical file, which starts mirroring. The system will only mirror dependent files that exist on the target.

Note: You must also ensure that no changes are made to an object between the time of the save and the time the file is activated. Changes made since the time of the save will not be applied to the object on the backup node. Also, you will be responsible for replicating any suspended objects.

Automatic ReactivationAutomatic reactivation allows iCluster to refresh automatically those objects that become unsynchronized on either the primary or backup system. If an error occurs while replicating an object, it will become unsynchronized and iCluster will suspend it. Once an object is suspended, subsequent updates to that object will not be applied until it is automatically reactivated or you manually activate it. See the DMACTOBJ—Activate Object (Activate Object in the iCluster Administrator) command for more information about activating objects manually.


Automatic reactivation attempts to resynchronize suspended objects without user intervention. This feature decreases the amount of user effort required to keep your primary and backup nodes synchronized, and decreases the time that your nodes could be unsynchronized if an object is suspended. It also reduces the time you need to monitor the replication status of their system and to activate suspended objects.iCluster automatically reactivates the following objects that are suspended on both the primary and backup nodes:• Non-journaled native objects• Physical files (data)• Logical filesIFS files are also automatically reactivated on the primary node.You have the option of indicating whether you want to enable automatic reactivation. Also, you can specify the number of reactivation attempts for an object and the size of an object that will be tried for reactivation. See the DMSETSVAL—Set Cluster System Values command for more information about the parameters that you can set for automatic reactivation.For the automatic reactivation of database files, refresh of objects, and the activation of objects (see the DMACTOBJ—Activate Object command), use the Maximum Refresh Size parameter in the DMSETSVAL—Set Cluster System Values (Setting System Values for the iCluster Administrator) command. For all other objects, use the Maximum Reactivation Size parameter in the DMSETSVAL—Set Cluster System Values command.Objects suspended with the following reason codes cannot be automatically reactivated. All of the other reason codes are eligible for automatic reactivation.• Native objects: EJF, JPF, MRR, NGP, RBC, SBU, SIZ, SPF• IFS objects: EJF, INE, INS, LNK, SBU, SIZSee Working with Object Status on page 306 for a complete list of reason codes and their meanings.

Choosing a Failover MechanismThis topic offers criteria for choosing a failover mechanism and provides instructions for changing to a different system. iCluster uses its failover mechanism to detect connectivity problems and manage node status. The following failover mechanisms are available in iCluster:

SwitchOver SystemSwitchOver System (SOS) integrates the existing failover mechanism from DataMirror HA Suite into iCluster. Its purpose is to simplify node and replication group management and support while removing some of the limitations of IBM Cluster Resource Services (IBM CRS). SOS provides a comparable set of functionality to IBM CRS, with the exception of support for resilient applications and switched disks.For new installations, SOS is the default failover mechanism. If you are upgrading from iCluster 2.0 or earlier, then your installation will continue to use IBM Cluster Resource Services. You can change your installation to use SOS after upgrading iCluster. See Configuring iCluster to use SwitchOver System (SOS) on page 74, below, for more information.

IBM Cluster Resource ServicesIBM Cluster Resource Services (IBM CRS) is the native iSeries cluster management framework. These services provide a standard platform for managing resilient resources, including resilient applications.See Switchover for IBM Cluster Resource Services (CRS) on page 341 and your IBM documentation for more information on Cluster Resource Services.


Key Differences in Failover MechanismsThe following table lists key differences between the failover mechanisms. You can use it to determine which system meets the needs of your environment.

DataMirror recommends that you use SOS unless your environment requires support for resilient applications or switchable disk storage devices. If it does, then you must use IBM CRS.

Configuring iCluster to use SwitchOver System (SOS)New iCluster installations use SOS by default. To change an existing cluster to use SOS, you must perform the following tasks:1 On the Work with Cluster Nodes (DMWRKNODE—Work with Nodes on page 110) screen, ensure all nodes have a status of

*ACTIVE.

Table 1 Key Differences in Failover Mechanisms

Switchover System (SOS) IBM Cluster Resource Services (IBM CRS

• Supports consecutive and non-adjacent OS/400 versions in the same cluster. For example, you can have nodes running on OS/400 V5R1 and V5R3 in the same cluster.

• Only supports consecutive OS/400 versions in the same cluster. For example, you can have nodes running on OS/400 V5R2 and V5R3 in the same cluster, but not V5R1 and V5R3.

• Allows control over automatic switching of nodes after detecting a failure (optional failover).

• Performs automatic switching of nodes on failover (mandatory failover) in the cluster resource group.

• Uses TCP/IP to test connectivity between nodes.

• Uses ICMP (ping) to test connectivity between nodes.

• Handles communication failures through a single *FAILED state. This simplifies recovery from failures.

• Uses *FAILED and *PARTITION states to handle communication failures.

• Does not support resilient applications nor takeover IP addresses.

• Supports resilient applications, including the ability to takeover IP addresses.

• Does not support switchable disk storage devices on the primary and backup nodes.

• Supports switchable disk storage devices.

• Both SOS and IBM CRS support calling user exit programs before and after switchover operations.


2 On the Work with iCluster Groups (DMWRKGRP—Work with Groups on page 137) screen, ensure all groups have a replication status of *INACTIVE.

3 On the iCluster System Values (DMSETSVAL—Set Cluster System Values on page 181) screen, change the Use OS/400 Cluster Services parameter to *NO and press Enter to save the change.

4 Change the default switchover settings for each node using the DMCHGNODE—Change Node command to ensure they meet your recovery needs.

5 On the Work with Cluster Nodes (DMWRKNODE—Work with Nodes on page 110) screen, restart all of the nodes in the cluster.

6 On the Work with iCluster Groups (DMWRKGRP—Work with Groups on page 137) screen, restart all of the groups.Your replication environment will now run with its previous configuration and SOS.Note: After performing these tasks, you will no longer be able to use DMSETPRIM—Prepare Primary Node to

perform a switchover. Instead, use the DMCHGROLE—Change Group Primary Node command.

Configuring iCluster to use IBM Cluster Resource Services (CRS)After installing iCluster for the first time on a computer, run DMSETSVAL—Set Cluster System Values and set the CLUSTER parameter to *YES. Alternatively, perform the following tasks to change an existing cluster to use IBM CRS. Note that you must recreate the cluster as a part of this process.1 Record the configuration of your current clustering environment including nodes, groups, and object specifiers. You will need

this information when you recreate the cluster.2 Delete the current cluster by running DMDLTCLSTR—Delete Cluster on every node in the cluster.3 On the iCluster System Values (DMSETSVAL—Set Cluster System Values on page 181) screen, change the Use OS/400

Cluster Services parameter to *YES and press Enter to save the change.4 Delete residual cluster information by running DMDLTCLSTR—Delete Cluster on every node in the cluster, again.5 Repeat Step 2 and Step 3 on every node in the cluster.6 Add each node back to the cluster using the DMADDNODE—Add Node command. The first node you add will become the

master node.Note: If you have nodes with different operating system versions, then you must add the node with the lowest

version first. For example, if your clusters will have nodes running on OS/400 V5R2 and V5R3, then the first node you add must be running V5R2.

7 Create the groups that will replicate objects using the DMADDGRP—Add Group command. Use the information you recorded in Step 1 to help with this step.

8 Specify the objects you want to be in each group using the DMSELOBJ—Select Objects to Group and DMADDBSF—Add Path Object Specifier to Group commands. Use the information you recorded in Step 1 to help with this step.

9 On the Work with iCluster Groups (DMWRKGRP—Work with Groups on page 137) screen, restart all of the groups.Your replication environment will now run using IBM CRS.

Starting Replication and Journal PositionsThis topic outlines some of the differences between DMSETPOS—Set Journal Start Position and DMMRKPOS—Mark Current Journal Positions, and how to start replication after setting or marking journal positions with these commands.

How are DMSETPOS and DMMRKPOS Different?It is important to understand the differences between the DMSETPOS—Set Journal Start Position and DMMRKPOS—Mark Current Journal Positions commands:


• The DMSETPOS—Set Journal Start Position command requires you to specify a journal position for all journals in the specified group. The specific journal entry can be entered directly or determined by the command if a date and time is entered. DMSETPOS also allows you to use the last applied journal position.DMSETPOS is typically used for recovery purposes when you would like to resume mirroring from a journal position back in time.

• In contrast, the DMMRKPOS—Mark Current Journal Positions command does not require you to specify a journal position for all journals in the specified group. DMMRKPOS—Mark Current Journal Positions automatically determines the journal position by using the current sequence number(s) in all relevant journals for the group.DMMRKPOS—Mark Current Journal Positions is typically used for initial synchronization. For example, after a full tape save/restore to the backup when mirroring is started from the current point in time.

Both of these commands help to prevent unsynchronized start-ups when starting replication for a group.

To start replication after setting journal positionsIf you want to start replication with DMSTRGRP or DMSTRREPL after having set or marked journal positions with DMSETPOS or DMMRKPOS, note the following:• If you issue the DMMRKPOS command for a group and then start replication, you must specify *YES for the USEMARKED

parameter in DMSTRGRP and DMSTRREPL. This starts replication at the journal positions you marked with the DMMRKPOS command.

• If you issue the DMSETPOS command for a group and then start replication, you must specify *NO for the USEMARKED parameter in DMSTRGRP and DMSTRREPL. This starts replication at the journal positions you set with the DMSETPOS command.

See DMSTRGRP—Start Cluster Operations at Group and DMSTRREPL—Start Replication for more detailed information on these commands.

Monitoring ReplicationThis topic describes the tools in iCluster you can use to monitor replication. Monitoring the replication status of your objects is very important and should be performed regularly.

Using the Status MonitorYou can use the iCluster Status Monitor to monitor the current status of replication. The Status Monitor allows you to determine if replication is active, if any latency exists for the journal scrape or backup apply processes, and the through put and runtime totals for replication processes.The Status Monitor also allows you to view and activate suspended objects.See Working with the Status Monitor on page 291 for more information.

Using Sync CheckA sync check will allow you to check whether the objects within a replication group, are equivalent on the primary and backup nodes. All objects for the specified backup node/replication group combination are checked.You can perform a sync check through the Status Monitor and the STRHASC—Start Sync Check and STRHASCUSR—Start User Sync Check commands.You should perform regular sync checks and review the sync check reports to ensure that the replicated objects on the primary and backup systems are equivalent.

Using Event LogsIn iCluster, an event log is used to maintain messages generated during replication, communication, and cluster activities. The event log is maintained on each node and contains messages about events involving that node.


To view all events in iCluster, display the event log on the backup node. All communication and replication events are sent to the backup node.iCluster provides the DMDSPLOG—Display Cluster Event Log command that displays the messages that have been placed in the event log. You can use the DMCLRLOG—Clear Cluster Event Log command to remove specific categories of messages from the event log.

Monitoring Out-of-Sync ObjectsDuring normal replication, all changes to replicated objects are applied on the backup node. Sync check lets you check whether the primary and backup nodes have become unsynchronized. You can perform a sync check through the Status Monitor and by issuing STRHASC—Start Sync Check on page 267 and STRHASCUSR—Start User Sync Check on page 270.STRHASC checks all objects for the specified replication group, while STRHASCUSR allows you to perform a sync check on specific objects that are replicated in the replication group. The following are just some of the sync check options provided by these commands:• Type of output for the sync check. You can output the sync check to a spool file or database file.• Type of sync check.• Object specifiers and path object specifiers that allow you to filter the objects you want to sync check.• Date and time when the sync check will start.See To view Sync Check results on page 78 for more information.The Out-of-Sync (OOS) Count Column in the Status Monitor also lets you determine if objects on the backup node are the same as the objects on the primary node within a replication group. See Out-of-Sync (OOS) Count Column for more information.

To start a Sync CheckTo use sync check, perform the following tasks:1 Use the SELSCATTR—Select Sync Check Attributes command to select the file attributes that you want to include in your

sync check.Note: You can also choose to use the default attributes supplied by this command.2 Start the sync check program with the STRHASC or STRHASCUSR command.

See the STRHASC—Start Sync Check and STRHASCUSR—Start User Sync Check commands for more detailed information on the parameters in these commands. You can also issue these commands from the Status Monitor. See Common Options for all Views on page 295 for more information.

3 Use one of the following commands on the backup node to view the sync check:• DSPHASC • DMSCRPT• DMSCRPTNTV

See To view Sync Check results on page 78 for more information on viewing sync checks.

Out-of-Sync (OOS) Count ColumnThe OOS Count Column (Figure 5) displays the sum of the OS/400 native objects and IFS objects that are out-of-sync for each group. This information is current as of the last sync check.Note: The OOS field is only available on backup monitor screens in a 132-column terminal session.See Reading Status Information on page 303 for more information on the OOS column in the Status Monitor.


To view Sync Check resultsThe command that you use to view the sync check is determined by the OUTPUT parameter in STRHASC—Start Sync Check on page 267 and STRHASCUSR—Start User Sync Check on page 270. This parameter determines whether output is sent to a spooled file or a database file. Use the following to determine which command you should use:• If you ran the STRHASC or STRHASCUSR commands with the OUTPUT (*MISMATCH) or OUTPUT (*ALL) options, iCluster

places the results of the sync check in a spool file on the primary node.Use the DSPHASC—Display Sync Check command to view the spool file.

• If you ran the STRHASC or STRHASCUSR commands with the OUTPUT (*NONE) option, the output of the sync check is sent to a database file.Use the DMSCRPT—Sync Check Report for IFS Objects command (*BSF sync check type) or DMSCRPTNTV—Sync Check Report for Native Objects command ( *FILEATTR, *DTAARA, and *LIST sync check types) to view the sync check report in the database file.

Note: If your sync check displays out-of-sync objects that are suspended, you can use DMACTOBJ—Activate Object and DMACTBSF—Activate BSF Object to activate and refresh the object(s) to the backup node. See Suspended Objects on page 72 for general information about suspended objects.

Note: Use the PRGHASC—Purge Sync Check Results on page 276 command to clear the sync check database files of obsolete records.

Monitoring LatencyThis topic describes the tools that are available in iCluster that let you monitor latency. Latency is the time interval between a journal entry being written on the primary node and it being applied on the backup node. iCluster provides statistics about real time latency so that you can examine the speed of replication from the primary node to the backup node and take any necessary action. You can adjust the LATENCY system values to the level of latency that you are willing to tolerate in your cluster.

LATENCY System ValuesThe LATENCY system values affect the latency threshold settings within iCluster. You can use these system values to determine the threshold or upper limit of latency that iCluster will tolerate before issuing a warning message. You can set the LATENCY system values with the DMSETSVAL—Set Cluster System Values command:• The Source Receive Threshold system value indicates the amount of source receive latency that can be tolerated before the

latency warning message, OMI0308, is issued. Source receive latency indicates the difference between the timestamp of the journal entry last processed by the backup receiver for the journal, and the timestamp of the last journal entry deposited in the journal on the primary (source) node.

• The Target Apply Threshold system value indicates the amount of backup apply latency that can be tolerated before the latency warning message, OMI0308, is issued. The backup apply latency is the difference in the timestamps of the last entry received by the journal's receive process and the last entry applied by the journal's apply process.

• The Latency Check Interval system value lets you specify how often iCluster checks for latencies (source receive and target apply) and compares them with their corresponding thresholds.

See the DMDSPLOG—Display Cluster Event Log command for more information on how to display the OMI0308 latency message in the event log.


Monitoring Latency with the Real Time Overall Latency ScreenAfter setting the latency threshold system values (above), you can monitor latency in your cluster with the Real Time Overall Latency view in the Status Monitor.

Figure 1 Real Time Overall Latency View in the Status Monitor

Source latency, apply latency, and total latency are displayed in real time in HH:MM:SS format for all primary/backup/group/journal combinations. Visual indicators let you know when you have exceeded latency thresholds or total latency for your cluster.Figure 1 displays three latency scenarios that you can monitor in iCluster:• The first group, denoted by [1], contains very little or no latency. None of the latency thresholds set in the DMSETSVAL—Set

Cluster System Values command have been exceeded since only one '.' character appears under Total Latency Status.Note: If latency thresholds have not been exceeded, the bar graph displays the '.' character. If either or both latency

thresholds have been exceeded, the bar graph displays asterisks '*'.• The second group, denoted by [2], contains more latency than [1] since the bar graph under Total Latency Status now has

three '.' characters. Latency thresholds have not been exceeded.• The third group, denoted by [3], has exceeded one of the latency threshold values set in the DMSETSVAL—Set Cluster

System Values command. Brackets '< >' are displayed around the Source Latency value that has exceeded the threshold. The bar graph under Total Latency Status now displays asterisks '*' since one of the latency thresholds has been exceeded.

See Monitoring Real Time Overall Latency on page 297 for more information about using the Real Time Overall Latency screen in the Status Monitor.

Restarting iCluster after Restarting a NodeNote: See your IBM documentation for more information on managing clusters, including the steps in this topic.After restarting a node in a cluster, you must perform the following tasks on the node you want to add:1 Initialize TCP/IP processing by running the STRTCP command.2 Start the TCP/IP server by running the STRTCPSVR command. You must minimally start the *INETD server. For example:


> STRTCPSVR SERVER(*INETD)

3 Start the XDMCLUSTER subsystem by running the STRSBS command. For example: > STRSBS SBSD(XDMCLUSTER)

4 Start the TCP/IP listener job on the local node by running the STRHATCP—Start TCP/IP Listener command.5 Rejoin the node by running the DMREJOIN—iCluster Rejoin Cluster command.If you want to automatically restart iCluster for a node, then you must include the commands in the autostart job files for each node in the cluster.

Upgrading the Cluster VersionThis topic describes the tasks you must perform when upgrading the operating system on one or more nodes in a cluster. This information only applies to iCluster installations that use IBM Cluster Resource Services as their failover mechanism. The concept of cluster versions does not apply to installations using SwitchOver System.Note: See your IBM documentation for more information on managing clusters, including the steps in this topic.Every node in a cluster has a potential node version, which is determined by the operating system version installed on the node. A cluster's version has a limit of the minimum potential node version of the nodes in the cluster. For example, consider the following cluster:

The highest possible version of this cluster is 2, because 2 is the lowest common potential node version that it can support. It is technically possible for this cluster to be at version 1.Note: DataMirror recommends upgrading your cluster to its highest possible version. This reduces the likelihood of

versioning problems after future node upgrades.To see the current version of a cluster and its nodes, run the DSPCLUINF command.

To upgrade the version of a cluster, perform the following tasks:1 Upgrade the operating system for nodes in the cluster. The potential node version for each node must be greater than or

equal to the version you want the cluster to have.2 Change the cluster version by running the CHGCLUVER command.

Changing IP AddressesThis topic describes the tasks you must perform to change the IP address of a node in a cluster. The steps you must follow depend on if your iCluster installation uses SwitchOver System or IBM Cluster Resource Services as its failover mechanism.You must perform the complete set of steps for every IP address that you are changing. Do not try to change more than one node's IP addresses at a time.

Table 2 iCluster Version

Node OS/400 Version Potential Node Version

ALPHA V5R2 2

BRAVO V5R2 2

CHARLIE V5R3 3


SwitchOver System ProcedureTo change the IP address of a node when the cluster is using the SwitchOver System failover mechanism, perform the following steps:1 Stop all groups that use the node by running the DMENDGRP—End Cluster Operations for Group command.2 Change the node's IP address using the DMCHGGRP—Change Group command. The node you are changing must be

active when you run this command.3 End the node by running the DMENDNODE—End Cluster Operations at Node command.4 Refresh the XDMCLUSTER subsystem by ending and then starting it using the following commands:> ENDSBS SBS(XDMCLUSTER) OPTION(*IMMED)

> STRSBS SBSD(XDMCLUSTER)You must end the subsystem with the *IMMED option.

5 Start the node using the DMSTRNODE—Start Cluster Operations at Node command.6 Start all groups that use the node using the DMSTRGRP—Start Cluster Operations at Group command.

The cluster will now use the new IP address to connect to the node.

IBM Cluster Resource Services ProcedureTo change the IP address of a node when the cluster is using IBM Cluster Resource Services, perform the following steps:1 Ensure at least one other node in the cluster is active. The active node cannot be the node whose IP address you are

changing.2 End all groups that use the node whose IP address you are changing by running the DMENDGRP—End Cluster Operations

for Group command.3 End the node by running the DMENDNODE—End Cluster Operations at Node command.4 Change the node's IP address by configuring your network. See your operating system documentation for more information

on this step.5 Change the node's IP address in iCluster using the DMCHGNODE—Change Node command on an active node.6 Start the node whose IP address you changed by running the DMSTRNODE—Start Cluster Operations at Node command.7 Start all groups that use the node using the DMSTRGRP—Start Cluster Operations at Group command.The cluster will now use the new IP address to connect to the node.

Journaling in iClusterThis topic explores some of the differences between traditional local journaling and remote journaling in iCluster.


Local JournalingWith local journaling, applications on the primary system generate database changes, and these changes generate journal entries that are written to a local journal receiver. iCluster then transmits these changes asynchronously to the backup system where the same changes are made. Local journaling is used by default. Figure 2 gives you a more detailed view of the local journaling process in iCluster.

Figure 2 Local Journaling in iCluster

Consider the following when using local journaling:• With local journaling, iCluster lets you be more selective in what you journal. For example, you can choose to avoid journaling

BSF's with non-journaled BSF support. See Path Object Specifiers on page 52 for more information.• Asynchronous data updates result in very little impact to applications on the primary system.• Local journaling is only available in asynchronous mode. This can result in a small amount of data latency since control

returns to the applications as the journal entries are prepared for transmission to the backup system.

Remote JournalingWith remote journaling, journal entries are transmitted directly from the primary system to journals and their associated receivers on the backup system. This can improve system performance on the primary system because the remote node is used to do more of the replication processing.


Note: The default data update method for remote journaling is asynchronous mode. See Configuring Remote Journaling on page 84 for more information on how to set the delivery modes for remote journaling.

Figure 3 Remote Journaling in iCluster

Figure 3 gives you a more detailed view of the remote journaling process in iCluster.Consider the following when using remote journaling:• Remote journaling does not let you be selective about what journal entries are transmitted to the backup system. All entries in

the primary system journal are sent to the remote journal.• Synchronous update of data can have an impact on applications and journaling throughput on your primary system.• In synchronous mode, there is no data latency with remote journaling since journal entries are applied to the backup system

before updating the primary system.• In asynchronous mode there is a small amount of data latency since control returns to the applications as the journal entries

are prepared for transmission to the backup system.• Remote journaling is configured at the journal level. See Configuring Remote Journaling on page 84 for more information.

Remote JournalingRemote journaling is an OS/400 feature that replicates journal entries for an object on the primary node (local journal) to be placed on a journal on the backup node (remote journal). In this case, the local and remote journals are identical. This occurs independently from any iCluster operations and provides an alternative method for transporting data from the primary node to a backup node.iCluster can apply transactions from a remote journal on the backup node to the replicated objects on the backup node. Using a remote journal in a replication group can improve system performance on the primary node because remote journaling uses the backup node to do more of the replication processing. In addition, you can configure your local journal to update the remote journal either synchronously or asynchronously. Traditional iCluster journaling using only a local journal only supports asynchronous updates.


See Journaling in iCluster on page 81 for a comparison of local journaling and remote journaling.See your IBM documentation for more information on remote journaling concepts, including synchronous and asynchronous journal updates

Configuring Remote JournalingThis topic describes the tasks that you must perform to configure a group or resilient application to apply journal entries from a remote journal. The examples show the required parameters you must set to perform each step. See the documentation for each command for a complete list of parameters.To configure remote journaling, perform the following tasks:1 On the backup node, add a relational database directory entry by running the ADDRDBDIRE command. The IBM

documentation describes this procedure.iCluster requires that you give this database entry the same name as the node. For example, the following command creates the relational database directory entry on a node named NEWYORK:> ADDRDBDIRE RDB(NEWYORK) RMTLOCNAME(*LOCAL)

2 On the primary node, add a relational database entry for the database on the backup node by running the ADDRDBDIRE command. For example,> ADDRDBDIRE RDB(NEWYORK) RMTLOCNAME(192.168.0.240)

3 On the primary node, create a journal receiver by running the CRTJRNRCV command. For example, the following command creates a receiver with a threshold of 5000 KB:> CRTJRNRCV JRNRCV(TORLIB/MYJRNRCV) THRESHOLD(5000)

4 On the primary node, create a journal that uses the journal receiver by running the CRTJRN command. You must configure the journal to have the system manage journal receiver changing. For example, the following command creates a journal:> CRTJRN JRN(TORLIB/MYJRN) JRNRCV(TORLIB/MYJRNRCV) MNGRCV(*SYSTEM)

5 On the primary node, add a remote journal by running the ADDRMTJRN command. iCluster requires that you do the following for this command:

• Set the relational database to the database directory entry you created in Step 1.• Set the names of the source journal and target journal to the journal you created in Step 4. The target journal name must be

the same value as the source journal.• Set the source journal library to the library of the journal you created in Step 4.• Set the target journal library to a library that is different from the source journal library. The source and target libraries must be

different.• Create both the source journal library and the target journal library on the backup node.• Set the remote journal type to *TYPE1. iCluster does not support any other remote journal types.

For example, the following command creates the remote journal NYLIB/MYJRN on the backup node:> ADDRMTJRN RDB(NEWYORK) SRCJRN(TORLIB/MYJRN) TGTJRN(NYLIB/MYJRN) RMTJRNTYPE(*TYPE1)

6 On the primary node, activate the remote journal by running the CHGRMTJRN command with the JRNSTATE(*ACTIVE) parameter. For example, the following command activates the remote journal:> CHGRMTJRN RDB(NEWYORK) SRCJRN(TORLIB/MYJRN) TGTJRN(NYLIB/MYJRN) JRNSTATE(*ACTIVE)

If you want to configure your journal to perform synchronous updates, then use the DELIVERY(*SYNC) parameter with this command.


Note: When a remote journal group starts, iCluster automatically activates the remote journal (if it is not activated) using the default value for the DELIVERY parameter in the CHGRMTJRN command.

See the IBM CHGRMTJRN documentation for more information about this command and setting.7 On any node, add your group or application by running the DMADDGRP—Add Group or DMADDAPP—Add Resilient

Application command, respectively. Alternatively, you can use the DMCHGGRP—Change Group or DMCHGAPP—Change Resilient Application commands to modify existing groups or applications. When running this command, you must do the following to use the remote journal:

• Set the default databse journal to use the journal you created in Step 4.• Set the journal location to *REMOTE.For example, the following command creates a group that replicates from TORONTO to NEWYORK using the MYJRN journal.

> DMADDGRP GROUP(MYGRP) GRPTYPE(*REPL) PRIMNODE(TORONTO) BACKUPS(NEWYORK) DFTDBJRN(TORLIB/MYJRN) JRNLOC(*REMOTE)

8 Perform the normal configuration and administration tasks for your replication group. This includes selecting objects to be replicated, setting or marking journal positions, and starting the applications and groups. These tasks are the same for both local and remote journaling.

Your group or application will now use the remote journal.

Role Switching with Remote JournalsIf you intend to perform failover or switchover operations on a group using remote journaling, then you must perform additional configuration tasks. For example, consider a group that replicates data from a node named TORONTO to a node named NEWYORK. In this group, TORONTO is the primary node and NEWYORK is the backup node. Since the group uses remote journaling, TORONTO has a remote journal on NEWYORK.Since a switchover or failover reverses the roles of the nodes in a group, NEWYORK must be able to become the primary node in the group. This requires that NEWYORK also has a remote journal on TORONTO.During the switchover or failover, the nodes must perform the following tasks:1 TORONTO must deactivate its remote journal on NEWYORK.2 NEWYORK must activate its remote journal on TORONTO.You can create a user exit that performs these tasks. See the RMTJRNSWO file member in <lib>/QACLSRC, where <lib> is the iCluster product library, for a sample user exit.

Switchovers and Failovers with Synchronous Remote JournalingRemote journals that use the synchronous delivery mode can have suspended journaled objects after a failover or switchover, if they are in a replication group that only includes after images. To prevent this suspension, you must configure your group to include both before and after images. To do this, perform one of the following tasks:• To change this property for one group, set the JRNBA property to *BOTH by running the DMCHGGRP—Change Group

command. All new objects that are added to this group will use this setting, but any existing objects in the group must be manually changed to include both before and after images.Perform the following tasks to change this setting for individual objects:• If you are using OS/400 V5R3, then run the CHGJRNOBJ command with the IMAGES(*BOTH) parameter.• If you are using OS/400 V5R2 or earlier release, then you must stop journaling on the object by running the ENDJRN

command and then restart journaling with before and after images by running the STRJRN command with the IMAGES(*BOTH) parameter. To ensure data integrity, we recommend that you do this when no users are accessing the node.


• To change this property globally for all groups, set the journal images attribute of the PF property to *BOTH by running the DMSETSVAL—Set Cluster System Values command. All new groups will use this setting by default, but any groups that were explicitly configured to include only after images will need to have their JRNBA property set to either *BOTH or *CLUSTER by running the DMCHGGRP—Change Group command.

Passing Arguments to Sync Point User Exit ProgramsIf you define a user exit program that will be called at a sync point set with the DMSETSYNC—Set Sync Point command, the following arguments will be passed to the program:

User Exit PointType: CharacterLength: 1The point where the user exit program was called. One of the following values will be passed through this argument to indicate the exit point:• S: At journal entry scrape on the primary node• R: At journal entry receive on the backup node• A: At journal entry apply on the backup nodeIf you want to stop mirroring after completion of the user exit program, return the value '9' through this argument.

Backup Node NameType: CharacterLength: 10The name of the backup node in the replication group.

Replication Group NameType: CharacterLength: 10The replication group that had its jobs synchronized at the checkpoint journal entry.

User DataType: CharacterLength: 400The user-defined data that is specified through the DMSETSYNC—Set Sync Point command.


iCluster ReplicationThis section contains the following topics:• Replicating Database *FILE Objects on page 87• Replicating BSF Objects on page 89• Replicating Configuration Objects on page 90• Replicating User Profiles on page 91• Replicating LOBs on page 92• Replicating Triggers on page 92• Replicating Save Files on page 92• Replicating QDLS Objects on page 94• Replicating WebSphere MQ Objects on page 95

Replicating Database *FILE ObjectsThis section describes important considerations that should be noted when replicating physical files.

Object Specifiers for Database FilesWhen defining object specifiers for *FILE objects, an attribute of *ALL matches all types of files, including database files and device files. To include only database files, you need to create two object specifiers, one with an attribute of PFDTA (to include physical files) and the other with an attribute of LF (to include logical files). Note that the attribute PF (physical files) includes both source physical files and database files. When creating an object specifier for a physical file, specifying an object attribute of PFDTA allows access to certain iCluster parameters, such as the update method (by relative record number or by unique key), that cannot be accessed if PF or PFSRC (source physical file) is specified.

Refreshing Physical FilesiCluster uses a refresh while active method of refresh of physical files to the backup node. In other words, *FILE objects can be refreshed while they are still being updated. This is the only refresh option available for physical files in iCluster.While iCluster is refreshing a database file, changes to the content of the file may continue. Changes at the member level and file level (a rename of move operations, for example) may be delayed or they may fail while iCluster is refreshing the file. The length of time required to refresh a file depends upon the number of members in the file, the amount of data in the file, and the communications bandwidth available to the iCluster job performing the refresh.

Refreshing Logical FilesThe refresh of logical files does not include saving and restoring the access paths associated with the logical files. The access paths are rebuilt on the secondary system when the logical file is restored.Dependent logical files are automatically refreshed when the based on physical file is refreshed. In this case, the logical files on the secondary system are deleted prior the restore of the based on physical file, and restored after the data is refreshed to the physical file. Note that there may be a relatively long period of time between deleting the logical file and restoring the new version of the logical file. During this period, the logical file will appear in the monitor as in a pending state.If a logical file has a unique key, updates to the based on physical file will be delayed until the access path is completely built after the logical file is restored. Updates to all files are applied in time order relative to the primary system, so the rebuild of a uniquely keyed access path may delay the updates to other files associated with the same journal.


Unique Key Update MethodThe use of a unique key update method allows you to reorganize physical files without affecting the backup files.If a physical file is a multi-member file, the members of the unique index used for its replication must have the same names as the members in the physical file.You can change the backup node update method for a physical file from *UKEY to *RRN only while the replication group it belongs to is inactive. The file also must be resynchronized on source and target (by refresh or save/restore operation) before mirroring is restarted to ensure the relative record numbers on the source correspond to those on the target. Note that you can change the backup node update method of an object specifier only when the object specifier has been selected to a replication group.

Member SupportIf a file is selected for replication, all members of the file will be replicated. You cannot exclude individual members of a file from an object specifier.

Journaled ObjectsIn this manual, the term "journaled objects" refers to database files, data areas, and data queues. BSF files can also be journaled, but they are handled differently than other journaled objects. See Replicating BSF Objects on page 89 for more information about journaled BSF objects.

Referential Integrity (RI) ConstraintsWhen replicating RI constraints in iCluster, you should note the following:• Create and delete operations are replicated.• RI constraints are disabled on the backup node.• When performing a switchover, RI constraints are enabled on the new primary node.• When starting replication after a switchover, RI constraints are disabled on the new backup node.

Check ConstraintsiCluster replicates all operations for check constraints. If you add, remove, enable, or disable constraints on the primary system, iCluster will replicate these changes to the backup system.

Other Considerations• iCluster creates a journal on the backup node for physical files if one has not been created already. The backup node journal

will have the same name and reside in the library with the same name as the corresponding primary node journal.• iCluster creates a work library on the backup node for each apply job. These libraries are used to temporarily hold some

replicated objects prior to restoring them. These libraries follow a naming convention of HAxxxxxxxx, where xxxxxxxx are digits. The text associated with these libraries includes the logical backup node, group name, journal name, journal library, and the journal entry. These libraries should not be deleted and are considered part of the staging store. Work libraries are also created on the primary node for each journal scrape job which follow the same naming convention, however, these libraries are temporary. iCluster will delete these libraries when they are no longer needed.

• iCluster only replicates authority holders for *FILE objects only.• The CHGPF command is not supported when a source file is specified. The following attributes are mirrored for the CHGPF

and CHGLF commands: MAXMBRS, MAINT, RECOVER, FRCRATIO, and TEXT.• When mirroring files that have user defined SQL data types (SQL DISTINCT TYPEs), you must ensure that the *SQLUDT

object is also in mirroring scope. The file will become suspended unless the corresponding *SQLUDT object exists on the backup system.


Replicating BSF ObjectsByte Stream File (BSF) objects reside in the Integrated File System (IFS) and can be replicated by iCluster through the use of path object specifiers. See Path Object Specifiers on page 53 for more information. Note the following important issues and considerations concerning BSF objects.

Mirroring BSF ObjectsiCluster supports two types of replication for BSF objects: object-level mirroring, and content-level mirroring. The type of mirroring used for BSF objects matching a particular object specifier is determined by the JOURNAL parameter of the object specifier. See DMADDBSF—Add Path Object Specifier to Group (Add Full Replication Group for iCluster Administrator) or DMCHGBSF—Change Path Object Specifier to Group (Change Full Replication Group for iCluster Administrator) for more information.The following points are worth considering when deciding on the type of replication that is most appropriate for your needs:• iCluster supports content-level and object-level mirroring in the "root" ("/") and QopenSys file systems. Content-level mirroring

and object-level mirroring is also supported in the QDLS (DLO objects) file system.• Content-level mirroring includes object-level mirroring support and also mirrors changes to the contents of BSF objects.

Content-level mirroring requires that the BSF objects are journaled and is, therefore, more resource intensive than object-level mirroring.

• BSF polling allows content-level mirroring of objects in the QDLS file system. No journaling is required. See DMADDBSF—Add Path Object Specifier to Group (Add Full Replication Group for iCluster Administrator) for more information.

• Object-level mirroring will replicate changes to the object at the object-level only including: creates, deletes, moves, renames, restores, authority changes, ownership changes, and primary group changes.

• In object-level mirroring, changes to the contents of the object are not mirrored. Object-level mirroring is suitable for applications that create BSF objects but do not change the contents of those objects, or change the contents by creating a completely new version of the object.

• The overlap of journaled (*GROUP) and non-journaled (*NONE) path object specifiers is restricted in iCluster. For example, using both '/home/user/*' (non-journaled) and '/home/user/employees/*' (journaled) is not permitted. See Path Object Specifiers on page 53 for more information.

• The number of objects (BSF and/or native) that may be journaled to a particular journal is limited. See the IBM documentation for more information. Consider using object-level mirroring if you have a large number (100,000 or more) of BSF objects to be replicated.

Initial Synchronization of Path ObjectsInitial synchronization of path objects between the primary and backup nodes involves the same considerations required for the initial synchronization of database files. Mirroring path objects involves sending to the backup node only the portion of the object that was changed, therefore the changed object must exist on the backup node prior to applying the change.If you are starting mirroring for existing path objects, you can use one of two methods to synchronize the primary and backup nodes:• Refresh the objects through iCluster. The easiest way to synchronize the primary and backup nodes is to initially refresh all

objects before mirroring is started. Thereafter, when replication is started, only the mirroring of changes is required. The disadvantage of this approach is that the amount of data to refresh across the communications line may be large.

• Manually synchronize all objects. The second method to synchronize the primary and backup nodes is to manually save the objects on the primary node and restore them on the backup node. The starting journal position is established using DMMRKPOS—Mark Current Journal Positions (Mark Journal Positions for iCluster Administrator) after saving the primary node objects and before changes are made to these objects.BSF objects that are created on the primary node after the initial synchronization will be automatically refreshed to the backup node by iCluster if they are referenced by path object specifiers selected to groups.


Journaling Path ObjectsiCluster will start journaling an object referenced by a path object specifier to the default BSF journal if the object is refreshed and it is not already journaled. iCluster will also start journaling an object referenced by a path object specifier to the default BSF journal when you invoke DMMRKPOS—Mark Current Journal Positions (Mark Journal Positions for iCluster Administrator). After journaling has been started, most changes to that object can be mirrored without accessing the object on the primary node.Note: Journaling changes to path objects requires resources on the primary node both to record (CPU) and store

(disk) the changes. Journaling changes to path object specifiers will have an impact on the performance of applications that use those path object specifiers just as journaling database files has a performance impact on applications that use those database files.

For more information about journaling BSF objects, see DSPHABRCD—Display Recording for BSF Object and ENDHABRCD—End Recording for BSF Object.

Refresh (Before Mirroring) of BSF ObjectsThe following points should be considered when doing a refresh (before mirror) of BSF objects:• Prior to the refresh, iCluster will attempt to delete all objects on the secondary system that match the object specifiers.• Refresh before mirroring will create base directories if they do not exist. For an object specifier of '/home/objects/journaled/*',

iCluster will create the following base directories: '/home', '/home/objects', and '/home/objects/journaled'.

Status Monitor Support for BSF ObjectsSuspended BSF objects will be shown in the Status Monitor and may be activated from the object status screen or activated automatically by iCluster. For more information, see Working with the Status Monitor (Using the Object Status Monitor for iCluster Administrator).

iCluster Limitations for BSF ObjectsiCluster currently does not provide support in the following areas:• Hard links are not supported.• BSF include/exclude specifiers cannot have symbolic links to directories as part of their paths. This is only permitted when the

symbolic link is at the end of a non-generic specifier that points directly to the symbolic link. iCluster does not support using symbolic links to directories to make BSF specifiers shorter.

• Replication must be to the same directory on the backup node. iCluster does not support directory redirection of BSF objects.• DataMirror recommends that you replicate all BSF objects within a group to the same journal if they are journaled. When

using separate journals there is no guarantee that replication of files will occur in the same order in which they were created/renamed on the primary node.

• The maximum length of path names is 5000 bytes long (OS/400 supports 16 megabyte path names). • Two cluster system values, Lock File on Backup Node and Maximum Refresh Size, are not supported for path object

specifiers. See DMSETSVAL—Set Cluster System Values (Setting System Values for iCluster Administrator) for more information about these system values.

• DLO extended object attributes are not replicated by iCluster.• iCluster does not currently support suspension of non-journaled library objects on the backup system. The event log must be

monitored to track replication errors with BSF objects on the backup system.• Journaled BSF objects are suspended on the primary and backup nodes, while non-journaled BSF objects are suspended on

the primary node only.

Replicating Configuration ObjectsConfiguration objects collectively refer to the following object types:• Connection lists (*CNNL)


• Class-of-service descriptions (*COSD)• Controller descriptions (*CTLD)• Device descriptions (*DEVD)• Internet package exchange descriptions (*IPXD)• Line descriptions (*LIND)• Mode descriptions (*MODD)• NetBIOS descriptions (*NTBD)• Network interface descriptions (*NWID)• Network server descriptions (*NWSD)

Creating Configuration ObjectsWhen configuration objects are replicated to a node, you can request iCluster to automatically create these objects immediately after they are received or create them at a later time. If a configuration object already exists on a node, the DMSETSVAL—Set Cluster System Values (Setting System Values for iCluster Administrator) and DMADDNODE—Add Node (Add Node for iCluster Administrator) have options that allow you to avoid creating a configuration object if it is currently in use. As a result, DataMirror provides a set of source physical files (named CNNL, COSD, CTLD, DEVD, IPXD, LIND, MODD, NTBD, NWID, and NWSD) that are used to hold the source code for generating configuration objects on each node in the cluster. These source physical files are all located in the product library. If, at a later time, you want to create these objects, issue the CRTCFGOBJ command. See CRTCFGOBJ—Create Configuration Objects on page 248 for more information.

Identifying Configuration ObjectsWhen defining object specifiers, no special treatment is required to identify configuration objects. The configuration object type (see list above) can be specified in exactly the same way as all other supported types. However, you cannot use the generic type *ALL to identify configuration objects.

Replicating Configuration ObjectsFor configuration objects to be successfully replicated by iCluster, any dependent lines, devices, modes, and so on, must already exist on the backup system.You can also refer to DMSETSVAL—Set Cluster System Values (Setting System Values for iCluster Administrator), DMADDNODE—Add Node (Add Node for iCluster Administrator), and DMCHGNODE—Change Node (Change Node for iCluster Administrator) for more information.

Replicating User ProfilesWhen replicating user profiles with iCluster, you should note the following:• When user profiles (*USRPRF) are replicated to backup nodes, iCluster does not replicate associated document passwords.

The user must explicitly arrange for document passwords to be defined and updated on the backup node.• DataMirror recommends that all user profiles (*USRPRF) in library QSYS are replicated to backup nodes. This will ensure

that the owners of replicated objects are the same on both primary and backup nodes.• When mirroring changes to user profiles (*USRPRF) that belong to a replication group, you need to set the parameter

JOBMSGQFL to *WRAP for job description CSJOBD on the backup node. Setting this option prevents the job message queue from filling up.

• Since one or more group profiles can be associated with a single user profile object (*USRPRF), it is important to note that all related group profiles are also sent to the backup node when a user profile is replicated by iCluster.


Replicating LOBsA LOB (Large Object) provides the ability to store a large amount of data in a database.Previously, you may have excluded the files that contained LOBs because they potentially could have been suspended. You can include these files with LOB fields and they will be replicated just as any other data file. To replicate LOBs, you perform the same steps as for replicating other data in iCluster.

ConsiderationsYou should be aware of the following considerations for LOBs in iCluster: • The staged LOB data is stored on the backup node in IFS in BSF objects. iCluster stores LOB data in the following location:/home/DataMirror/iCluster/LOB/<00000000 >/<node name>/<group name>/<journal library>/<journal name>

• Currently, the maximum store size on the target system is not respected when it is creating the BSF objects for LOB staging. LOB data can be replicated as long as there is sufficient disk space on the target system to contain the LOB data.

• iCluster does not impose limitations in addition to those imposed by OS/400 for LOB fields in a record format. There can be a total of 1023 LOB fields in a record format. The total size of the LOB fields cannot exceed two gigabytes.

• We recommend setting all journals to use the highest receiver size possible to ensure successful journaling of files containing LOB data. To do this, set the receiver size to *MAXOPT2.

Replicating TriggersWhen replicating triggers in iCluster, you should note the following:• iCluster supports the replication of system triggers and SQL triggers. System triggers are handled by the ADDPFTRG and

RMVPFTRG operating system commands. SQL triggers are created and dropped through SQL statements on tables.• All triggers in a file are disabled when the file is replicated to the backup node. When a switchover occurs, all triggers are

once again enabled on the new primary node regardless of their original status.• Up to 300 triggers is supported per file.• You can add trigger names of up to 128 characters.Note that for OS/400 V5R1, there are a number of PTFs that are required for QDBRPLAY API functionality and replicating triggers:• SI07279• SI06408Note: The QDBRPLAY API functionality is built-in to OS/400 V5R2 or later and is not required for replicating

triggers.

Replicating Save FilesiCluster supports the replication of save file (SAVF) objects and allows you to automatically perform actions on the backup node when the save file is replicated to the backup node. Other types of objects, such as document library objects (DLO), and folders, and Integrated File System (IFS) objects (that cannot be replicated through path object specifiers), can be saved in a save file and then replicated by transferring the save file to the backup node.When a save file is received on a backup node, iCluster automatically invokes a predefined program called SAVFEXIT. You can write this program to perform specific actions on the save file. The name and library of the replicated save file are passed as parameters to SAVFEXIT. The program can then use these parameters to determine what kind of actions should be performed on the save file. For example, if the replicated save file contains a document library object, SAVFEXIT can be modified to detect the save file and then restore the DLO by issuing the RSTDLO command.


Note that SAVFEXIT program has to be created after the product is installed. Issue the following command to create this program.CRTCLPGM PGM(<iCluster product library>/SAVFEXIT) SRCFILE<iCluster product library>/QACLSRC)

The <iCluster product library> is the library where iCluster was installed. If you want iCluster to invoke the SAVFEXIT program automatically, it must be placed in the product library on the backup node before replication is started.If you do not want to perform any actions on the save file, remove or delete the SAVFEXIT program after stopping all replication. To replace the current SAVFEXIT program with a new program, you must first stop replication to ensure that the new program is invoked when replication is re-started.DataMirror provides sample source code that can be used to replicate document library objects, folders, and IFS objects. These code segments are located in the file QACLSRC in your product library. Table 12 identifies and briefly describes each code segment.

The following describes the process of replicating these types of objects through save files:1 Determine which objects on the primary node will be refreshed or mirrored.

These objects can be of any type that can be saved in a save file.2 Create a save file that the objects will be saved to (use the OS/400iSeries command CRTSAVF).

Table 1 Sample CL Programs

Name Description

SAVE_DLO Saves document library objects (DLO) into a named save file. This program resides on the primary node. When SAVE_DLO is called, the specified DLOs are saved into the save file. These DLOs are then saved at specified intervals. If the save file is being mirrored to a backup node, all specified DLOs will be replicated after they are saved in the save file.

SAVE_IFS Saves Integrated File System objects (IFS) into a named save file. This program resides on the primary node. When SAVE_IFS is called, the specified directory is saved into the save file (this includes all directories and objects contained within the directory). Subsequent updates applied to these directories and objects are saved at specified intervals. If the save file is being mirrored to a backup node, any subsequent updates will be replicated through the save file.

SAVFEXIT01 Restores document library objects from save files received on backup nodes. To invoke this routine when a save file is received on a backup node, rename this program to SAVFEXIT and compile it in the product library on the backup node.

SAVFEXIT02 Restores document library objects and Integrated File System objects from save files received on backup nodes. To invoke this routine when a save file is received on a backup node, rename this program to SAVFEXIT and compile it in the product library on the backup node.

SAVFEXIT03 Restores generic Integrated File System objects from save files received on backup nodes. To invoke this routine when a save file is received in a backup node, rename this program to SAVFEXIT and compile it into the product library on the backup node.


3 Create a CL program that will save these objects into the created save file.Note that a CL program can be written to use the autostart job or delay job capabilities to schedule when objects should be saved. DataMirror provides two sample CL programs (SAVE_DLO and SAVE_IFS - see Table 1) that can be used to save document library objects and Integrated File System objects into the save file.

4 Write the SAVFEXIT program to apply user actions to replicated save files on backup nodes.This program must reside in the iCluster product library on the backup node. Use the CRTCLPGM command (see above) to create the SAVFEXIT program. Note that DataMirror provides three sample programs (SAVFEXIT01, SAVFEXIT02, and SAVFEXIT03) that can be renamed to SAVFEXIT. These sample programs restore document library objects and Integrated File System objects on backup nodes.

5 Define the backup node that will be destination of the save files.6 Create a replication group that includes the backup node defined in the previous step.7 Create an object specifier that references the save files and select the specifier to the replication group created in the previous

step.8 Write the SAVFEXIT program to apply user actions to replicated save files on backup nodes to make sure that the program is

created on both nodes of your replication group.9 Call the CL program (for example, SAVE_DLO or SAVE_IFS - see Table 1) that saves the objects into the save file you

previously created.10 Start node and replication group operations.

When a save file is received on a backup node, SAVFEXIT is called. Typically, this program will restore the objects in the save file.

Replicating QDLS ObjectsThis topic describes the commands you must run to add QDLS objects to an existing replication group. See the DMADDGRP—Add Group on page 113 for more information on creating replication groups.

Replicating the QDLS Folder and its SubfoldersTo replicate the QDLS folder or any combination of its subfolders, you must add each folder to the objects replicated by a group using the DMADDBSF—Add Path Object Specifier to Group command. For example, the following commands add the PUBLIC and GLOBAL subfolders to the MYGROUP replication group:> DMADDBSF GROUP(MYGROUP) PATH('/QDLS/PUBLIC/*')

> DMADDBSF GROUP(MYGROUP) PATH('/QDLS/GLOBAL/*')

Excluding QDLS FoldersiCluster allows you to exclude QDLS folders from replication. However, to exclude any QDLS folders, you must replicate the entire QDLS folder and then specify which folders you want to exclude. You can only exclude folders from replication and not the individual objects in the folders.To exclude QDLS folders from your replication group, perform the following tasks:1 Add the entire QDLS folder to the replication group using the DMADDBSF command. You must include the entire QDLS

folder and cannot specify a subfolder in this step. For example,> DMADDBSF GROUP(MYGROUP) PATH('/QDLS/*')

2 Specify which QDLS folders you do not want to replicate using the DMADDBSF command and setting the INCFLG parameter to *EXCLUDE. You must end each path entry with /*. For example,

> DMADDBSF GROUP(MYGROUP) PATH('/QDLS/PRIVATE/*') INCFLG(*EXCLUDE)

3 Repeat step 2 for any other folders you do not want to replicate from the QDLS file system.


When you start the replication group, it will mirror all of the QDLS file system folders, except the ones you specified in Step 2 and Step 3. The replication process will create the folders you excluded on the backup node, but these folders will be empty.

Replicating WebSphere MQ ObjectsWebSphere MQ is an application-to-application program that exchanges data contained in messages via queues. WebSphere MQ facilitates the exchange of information between applications that would not otherwise communicate with each other, such as to multiple and potentially remote systems in different geographical regions, and dissimilar systems. It provides assured, once-only delivery of messages.

iCluster and WebSphere MQiCluster has the ability to mirror WebSphere MQ objects, enabling WebSphere MQ to continue operating on a secondary system should a failover occur. This support allows interdependent applications to continuously communicate within enterprises and between enterprises. If you use WebSphere MQ to integrate your applications, you can use iCluster to protect your applications from primary system outages since WebSphere MQ integration jobs are preserved.Note: iCluster mirrors BSF objects that represent queues, processes, name lists, and so on. Remote journaling is

used to guarantee that all WebSphere MQ transactions are replicated to the backup system and applied to the backup system queues.

iCluster supports WebSphere MQ V5R2, V5R3, and V6R0. Earlier versions of WebSphere MQ are not supported.The following commands in iCluster are important for a switchover with MQSeries groups:• RBDHAMQM—Rebuild iCluster MQSeries rebuilds the WebSphere MQ messages for the queue manager. This command

brings the WebSphere MQ objects on the backup node up-to-date based on the information recorded in the primary node's WebSphere MQ journal.

• DMSTRGRP—Start Cluster Operations at Group starts the mirroring for the queue manager.Multiple queue managers are supported in iCluster. Use the DMADDGRP—Add Group command to add a group for each of the queue managers to be mirrored.

iCluster LimitationsAuthority changes made through the GRTMQMAUT and RVKMQMAUT commands are not journaled by WebSphere MQ. Therefore, iCluster does not mirror these changes. IBM recommends that the operator use a CL program of their own to apply the two commands to any WebSphere MQ objects on the original primary machine instead of issuing them on the command line. This means the operator may have to update and run the CL program periodically in order to handle newly created WebSphere MQ objects. iCluster can mirror this particular program to the target. The program can be run after a switchover. All authority changes on the primary system are properly applied on the backup.

WebSphere MQ Support: Pre-requisitesBefore you can mirror WebSphere MQ objects with iCluster, you must perform the following steps:• Verify that you have applied the pre-requisite PTFs for the supported versions of WebSphere MQ. See PTFs for WebSphere

MQ on page 96 for more information.• Create a local relational database directory entry for each node in the cluster. See Creating a Local Relational Database

Directory Entry on page 96 for more information.• Add the QMQMADM administrative group to the DMCLUSTER user profile as either the group profile or as a supplemental

group. This must be done on both the primary and backup nodes.• Make sure that you have completed the miscellaneous steps outlined in Additional Pre-requisites for WebSphere MQ Support

(V5R2M0 and V5R3M0) on page 96.• Complete the steps in To enable WebSphere MQ support on page 96 below.


PTFs for WebSphere MQYour system administrator should install the appropriate system PTFs before you can mirror WebSphere MQ objects with iCluster. See Replicating WebSphere MQ Objects on page 95 for a service summary for OS/400 in IBM's online documentation for an up-to-date listing of the necessary system PTFs.Before installing the necessary system PTFs, make sure of the following:• WebSphere MQ must be installed on both the primary and the backup machines.• WebSphere MQ queue managers on both machines can start and stop properly.

Creating a Local Relational Database Directory EntryEach node in the cluster must contain a valid local relational database directory entry. These entries must be unique across the cluster.To determine if a node has a local database directory entry, issue the following OS/400 command for each node in the cluster:

WRKRDBDIRE

The local relational database directory entries are displayed as "*LOCAL" under the "Remote Location" field. If a "*LOCAL" value is not present, use the following command to create a local relational database directory entry for each node in the cluster:

ADDRDBDIRE RDB(<chosen_name>) RMTLOCNAME(*LOCAL)

Where <chosen name> can be the system name usually used for identifying the machine.Note: If two machines have the same local relational database directory value, remove one of them and recreate it

with a value that closely reflects the system name.

Additional Pre-requisites for WebSphere MQ Support (V5R2M0 and V5R3M0)• The WebSphere MQ queue manager is active on the primary node, and the WebSphere MQ queue manager on the backup

node exists, but is inactive before starting mirroring.You can use the WRKMQM command to check whether a certain WebSphere MQ queue manager is active or not. An error message will tell you that the manager is not active.

• Check the journal AMQAJRN on both the primary and backup machines (with the command WRKJRNA). Make sure the currently attached journal receiver has the highest number on the journal receiver chain.

To enable WebSphere MQ support1 Use the DMADDGRP—Add Group command to set up a group for WebSphere MQ replication.

The appropriate object specifiers are added automatically and selected to this group by this command.See the DMADDGRP—Add Group command for more information.

2 Use the DMSTRGRP—Start Cluster Operations at Group command to start replicating WebSphere MQ objects. See the DMSTRGRP—Start Cluster Operations at Group command for more information.

3 After a switchover, you need to rebuild the MQSeries environment by issuing the RBDHAMQM command.You should only use RBDHAMQM on the new primary node after a switchover or failover, and the WebSphere MQ queue manager has not been activated. See RBDHAMQM—Rebuild iCluster MQSeries on page 179 for more information.

4 If you are mirroring the MQSeries group(s) back to the original primary system as part of a switchover, mirroring must be started with the RFSH parameter set to *YES. For more information, see DMSTRGRP—Start Cluster Operations at Group on page 208.After a switchover, you should also stop the MQSeries queue manager on the original primary system.

Note: MQSeries groups do not support checkpoint user exits.


iCluster CommandsThis section contains information on the commands available in iCluster and how to use them. This guide also introduces iCluster concepts and provides general configuration information.This section provides full descriptions of each command that is supported by iCluster.For each command that is described, the following items of information are provided:• Input Parameters: Describes each parameter in the command and identifies the values that can be specified.• Examples: Provides one or more examples of invoking the command.• Use: Identifies limitations that determine where and when you can invoke the command.• Minimum Security Level: Where applicable, identifies the minimum security level (administrator, operator, or user) that is

required to invoke the command.• Menu Access: Identifies the option number(s) that you can use to invoke the command through supported iCluster menus.Note: Several historical HA Suite commands still exist within iCluster. If a command does not appear in the

documentation or the menus (F22 - Shift+F10), then its use is not supported by DataMirror.This section contains the following topics:• Node Commands on page 99• Group Commands on page 113• Native AS/400 Object Commands on page 141• Byte Stream File (BSF) Commands on page 155• Switchable Device Commands on page 163• Resilient Application Commands on page 167• MQSeries Commands on page 179• Administration Commands on page 181• Cluster Operation Commands on page 207• Replication Operation Commands on page 225• Status and History Monitor Commands on page 255• Sync Check Commands on page 263• Registered iCluster User Commands on page 277• Exit Program Commands on page 283• iCluster for Supported External Storage Systems on page 285• Other Commands on page 289


Node Commands

Node CommandsThis section contains the following commands:• DMADDNODE—Add Node on page 99• DMDSPNODE—Display Node on page 103• DMCHGNODE—Change Node on page 104• DMRMVNODE—Remove Node on page 109• DMWRKNODE—Work with Nodes on page 110

DMADDNODE—Add NodeDMADDNODE NODE( ) IPADDR( ) IPADDR2( ) PORT( ) PRODLIB( ) DESC( ) REPLJOBD( ) USRPRFSTS( ) CFGSRCHLD( ) STGSTORSZ( ) STGSTORLIB( ) SWITCHRES( ) CHKPRIMLNK( ) CHKALTLNK( ) LNKCHKFRQ( ) LNKCHKRTO( ) LNKCHKTRY( ) MSGUSREXIT( ) MSGQUEUE( ) MSGACTWAIT( ) NUMMSGACT( )

Use this command to add a node to the cluster.When a node is added through this command, it is automatically activated in the cluster. You can de-activate cluster operations on the node by issuing the DMENDNODE—End Cluster Operations at Node command.If no node has been added to the cluster, then you must issue this command on the system that you want to define in the cluster (see Use on page 103 if the node you are adding is not the first node in the cluster). The first node added to the cluster is considered as the master node. For more information about the master node, see Working in a Clustered Environment on page 43.If you are using IBM Cluster Resources as your failover mechanism and have nodes with different operating system versions, then the first node you add to a cluster must use the lower operating system version.See Adding Nodes on page 31 for important pre-requisite information concerning nodes in your clustered environment.

Input Parameters• NODE

The name that identifies the node in the cluster.• IPADDR

The primary IP address of the node being added to the cluster.You can specify the IP address of the node in dotted quad notation (for example, 125.4.3.55) or in domain name form (for example, as400sys1a.abccorp.com).

• IPADDR2The secondary IP address of the node being added to the cluster.You can specify the IP address of the node in dotted quad notation (for example, 125.4.3.56) or in domain name form (for example, as400sys1b.abccorp.com).The secondary IP address is used by the failover mechanism to determine if a node in the cluster is not operational or the communication link connecting the node has failed. The secondary IP address is not used by iCluster for replication operations.

• PORTThe TCP/IP port number on the node that has been reserved for iCluster communications.This port number was specified when defining the dmcluster service to the TCP/IP service table. For more information about adding this service to the table, see Adding a New Entry 'dmcluster' to the TCP/IP Service Table on page 37.


The default port number is 4545.• PRODLIB

The library on the node where iCluster has been installed.The default product library is DMCLUSTER.

• DESCA short description that allows you to identify this node among all others that have been defined in the cluster.The description can be a maximum of 50 characters long.

• REPLJOBDThe name of the job description that you want to associate with jobs that handle replication activity on the specified node.Specify the name of the job description or the following value:• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL—Set

Cluster System Values command.The library where the job description resides must be identified if you do not specify *CLUSTER. Prefix the job description with the name of the library where the job description is located (for example, LIB1/RJD).The default setting for this parameter is *CLUSTER.

• USRPRFSTSThe status that you want to assign to user profiles that are replicated to the node you are adding to the cluster.Specify one of the following values:• *CLUSTER—Refers to the cluster system value for this parameter you specify through the DMSETSVAL—Set

Cluster System Values command.• *PRIMARY—Sets the user profile status to the same status that is currently assigned to the corresponding user

profile on the primary node.• *DISABLED—Sets all user profiles to a status of *DISABLED.• *NOCHG—Indicates that the current status of each user profile will not be changed by iCluster and the user profile

status on the backup node is set to *DISABLED by default.The default setting for this parameter is *CLUSTER.

• CFGSRCHLDIndicates whether you want iCluster to automatically create configuration objects immediately after they are received on the specified node or hold the commands for creating them in specific physical files so that they can be created at a later time.Specify one of the following values:• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL—Set

Cluster System Values command.• *YES—Holds commands to create configuration objects in specific physical files so that they can be created at a

later time with the CRTCFGOBJ command.• *NO—Automatically creates configuration objects as soon as they are received on the node you are adding to the

cluster.If a configuration object that is being replicated already exists on the node you are adding to the cluster, you should set this value to *YES in order to prevent iCluster from trying to create the object when it is in use.The default setting for this parameter is *CLUSTER.For more information about configuration objects, see Replicating Configuration Objects on page 90.


Node Commands

• STGSTORSZIndicates the size (in megabytes) that you want to allocate for the staging store. Note that the size of the store changes dynamically but it cannot exceed the size specified through this parameter. The size of the staging store can range from a minimum of 512 megabytes to a maximum of 1,048,576 megabytes.All objects and data replicated between nodes will travel through the staging stores on the backup nodes. Assuming the backup node update process is active, the journal entries will be ready to be applied. If the node update process is not active, the staging store will hold all journal entries until the node apply process is re-started.The default setting for this parameter is 512 megabytes.For more information about staging stores, see Staging Objects on page 69.

• STGSTORLIBThe library where the staging store is located.

• SWITCHRESIndicates whether you will be using switchable resources on the current node.Enter one of the following values:• *YES—Enables the node to use switchable resources. If you specify this value, then you must have installed

OS/400 option 41 HA Switchable Resources on the node and there must be a valid license key for the option.• *NO—Does not enable the node for switchable resources.

• CHKPRIMLNKThis parameter indicates if you want to test the primary IP address for communication failures between the primary and backup nodes. The possible values are:• *YES—Test the primary IP address for communication failures.• *NO—Do not test the primary IP address for communication failures.

The default value for this parameter is *YES.This parameter has no effect in clusters that use IBM Cluster Resource Services.

• CHKALTLNKThis parameter indicates if you want to test the alternate IP address for communication failures between the primary and backup nodes. The possible values are:• *YES—Test the alternate IP address for communication failures.• *NO—Do not test the alternate IP address for communication failures.

The default value for this parameter is *NO.This parameter has no effect in clusters that use IBM Cluster Resource Services.

• LNKCHKFRQSet this parameter to how often, in seconds, you want to poll the primary and alternate links for communication failures. The default value is 60 seconds.This parameter has no effect in clusters that use IBM Cluster Resource Services.

• LNKCHKRTOSet this parameter to the number of seconds you want to wait for a response when polling links for failures. The default value is 30 seconds.This parameter has no effect in clusters that use IBM Cluster Resource Services.

• LNKCHKTRY


Set this parameter to the number of consecutive failures you want to allow before iCluster considers the primary node to have failed. For example, if you set this parameter to 3, then after the fourth consecutive failure, iCluster will change the status of the primary node to *FAILED. The default value is 5.This parameter has no effect in clusters that use IBM Cluster Resource Services.

• MSGUSREXITSet this parameter to the name and library of the user exit program to run after the number of consecutive communication failures, specified with the LNKCHKTRY parameter, is exceeded. You can either specify a name and library or *NONE. The default value is *NONE.This user exit is run once per message sent to the message queue.This parameter has no effect in clusters that use IBM Cluster Resource Services.

• MSGQUEUESet this parameter to the name and library of the message queue that will receive messages after the number of consecutive communication failures, specified with the LNKCHKTRY parameter, is exceeded. You can either specify a name and library or *NONE. The default value is *NONE.This parameter has no effect in clusters that use IBM Cluster Resource Services.

• MSGACTWAITSet this parameter to the number of minutes to wait before sending messages to the message queue, specified with the MSGQUEUE parameter, after detecting a failure. The default value is 0 minutes.

Note: Both MSGACTWAIT and NUMMSGACT must be set to non-zero values for messages to be sent after a failure.

This parameter has no effect in clusters that use IBM Cluster Resource Services.• NUMMSGACT

Set this parameter to the maximum number of messages to send through the message queue, specified with the MSGQUEUE parameter, after detecting a failure. The system will wait for the time specified in the MSGACTWAIT parameter between sending each message. The default value is 0.


This parameter has no effect in clusters that use IBM Cluster Resource Services.

ExamplesDMADDNODE NODE(NODE1) IPADDR(155.4.5.20) IPADDR2(155.4.5.21) PORT(4141) PRODLIB(DMICLUS) DESC('New York') REPLJOBD(LIB1/RJD) USRPRFSTS(*DISABLED) CFGSRCHLD(*YES) STGSTORSZ(1024) SWITCHRES(*NO) CHKALTLNK(*YES) MSGQUEUE(QUEUES/NYQ) MSGACTWAIT(10) NUMMSACT(6)

Adds the node named NODE1 to a cluster.NODE1 has a primary IP address (expressed in dotted quad notation) of 155.4.5.20, a secondary IP address (also expressed in dotted quad notation) of 155.4.5.21, and will use port number 4141 for iCluster replication within the cluster.iCluster was installed in the library DMICLUS.Description indicates that the node is located in New York.


Node Commands

The job description associated with replication jobs on NODE1 will be RJD in library LIB1.User profiles replicated to NODE1 will all be set to a status of *DISABLED.Commands to create configuration objects will be held in specific source physical files so that they can be created at a later time.The maximum size of the staging store on NODE1 will be 1,024 megabytes.Switchable resources will not be enabled on the node.This node will test the alternate IP address to detect primary node failures.This node will send messages regarding primary node failures to the NYQ queue in the QUEUES library.This node will send messages to QUEUES/NYQ every ten minutes if it detects a failure.This node will send up to 6 messages to QUEUES/NYQ.

DMADDNODE NODE(NODE2) IPADDR(sys1a.abc123corp.com) IPADDR2(sys1b.abc123corp.com) PORT(3333) PRODLIB(DMICLUS) DESC('Los Angeles') REPLJOBD(LIB1/RJD) USRPRFSTS(*PRIMARY) CFGSRCHLD(*NO) STGSTORSZ(2048) SWITCHRES(*NO)

Adds the node named NODE2 to the cluster.NODE2 has a primary IP address (expressed in domain name form) of sys1a.abc123corp.com, a secondary IP address (also expressed in domain name form) of sys1b.abc123corp.com, and will use port number 3333 for iCluster replication within the cluster.iCluster was installed in the library DMICLUS.Description indicates that the node is located in Los Angeles.The job description associated with replication jobs on NODE2 will be RJD in library LIB1.User profiles replicated to NODE2 will be set to the same status that is currently assigned to the corresponding user profile on the primary node.Configuration objects replicated to NODE2 will be automatically created as soon as they are received.The maximum size of the staging store on NODE2 will be 2,048 megabytes.Switchable resources will not be enabled on the node.

UseIf at least one node has been defined in the cluster, this command must be invoked from an active node that can communicate with the master node. You can define a maximum of 128 nodes in the cluster.


Menu AccessF22 (Shift+F10) - Option 1Work With Nodes screen - Option F6

DMDSPNODE—Display NodeDMDSPNODE NODE( )

Use this command to display the current settings for the specified node.

Input Parameter• NODE


The name of the node that you want to examine.

ExampleDMDSPNODE NODE(NODE1)

Displays the current settings for the node NODE1.

UseYou can issue this command on an active node in the cluster.

Minimum Security LevelUser (*USER)

Menu AccessF22 (Shift+F10) - Option 4Work With Nodes screen - Option 5

DMCHGNODE—Change NodeDMCHGNODE NODE( ) IPADDR( ) IPADDR2( ) DESC( ) REPLJOBD( ) USRPRFSTS( ) CFGSRCHLD( ) STGSTORSZ( ) SWITCHRES( ) CHGSTATUS( ) CHKPRIMLNK( ) CHKALTLNK( ) LNKCHKFRQ( ) LNKCHKRTO( ) LNKCHKTRY( ) MSGUSREXIT( ) MSGQUEUE( ) MSGACTWAIT( ) NUMMSGACT( ) AUTHCODE( )

Use this command to change one or more attributes of the specified node in the cluster. The node that you change must be active in the cluster, with the following exception: If you are using OS/400 Cluster Resource Services, the node must be *INACTIVE when changing its IP address(es) for iCluster.If replication is active and you are using Cluster Resource Services, the node's IP addresses and job description will not be changed. You can change all other node parameters when replication is active on the node. However, some parameters used by replication jobs will only come into effect when new replication jobs are started. Existing replication jobs will continue to use the previous parameters, with the exception of the staging store size. The staging store size comes into use by all backup replication jobs as soon as it is changed.If you are not using OS/400 Cluster Resource Services, the node must be *ACTIVE when changing its IP address(es) or any other parameter of the node.


The node in the cluster that will have one or more attributes changed by this command.Note that the node you specify must be in the cluster. If you specify a node that does not exist in the cluster, this command will fail. Therefore, this parameter is simply used to reference an existing node in the cluster so that other attributes of the node can be changed.

• IPADDRThe primary IP address of the node that you want to change. See Changing IP Addresses on page 80 before changing a node's IP address.The IP address of the node can be specified in dotted quad notation (for example, 125.4.3.55) or in domain name form (for example, as400sys1a.abccorp.com).Specify the primary IP address of the node or the following value:• *SAME—Uses the current setting for this parameter.

The default setting for this parameter is *SAME.• IPADDR2


Node Commands

The secondary IP address of the node that you want to change.The IP address of the node can be specified in dotted quad notation (for example, 125.4.3.56) or in domain name form (for example, as400sys1b.abccorp.com).The secondary IP address is used by the failover mechanism to determine if a node in the cluster is not operational or the communication link connecting the node has failed. The secondary IP address is not used by iCluster for replication operations.Specify the secondary IP address of the node or the following value:• *SAME—Uses the current setting for this parameter.

The default setting for this parameter is *SAME.• DESC

A short description that allows you to identify this node amongst all others that have been defined in the cluster.The description can be a maximum of 50 characters long.Specify a short description or the following value:• *SAME—Uses the current setting for this parameter.

The default setting for this parameter is *SAME.• REPLJOBD

The name of the job description that you want to associate with jobs that handle replication activity on the specified node.Specify the name of the job description or one of the following values:• *SAME—Uses the current setting for this parameter.• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL

command.The default setting for this parameter is *SAME.The library where the job description resides must be identified if you do not specify *CLUSTER or *SAME. Prefix the job description with the name of the library where the job description is located (for example, LIB1/RJD).

• USRPRFSTSThe status that you want to assign to user profiles that are replicated to the node you are changing in the cluster.Specify one of the following values:• *SAME—Uses the current setting for this parameter.• *PRIMARY—Sets the user profile status to the same status that is currently assigned to the corresponding user

profile on the primary node.• *DISABLED—Sets all user profiles to a status of *DISABLED.• *NOCHG—Indicates that the current status of each user profile will not be changed by iCluster and the user profile

status on the backup node is set to *DISABLED by default.• *CLUSTER—Refers to the cluster system value for this parameter that you specify through the DMSETSVAL

command.The default setting for this parameter is *SAME.

• CFGSRCHLDIndicates whether you want iCluster to automatically create configuration objects immediately after they are received on the specified node or hold the commands for creating them in specific physical files so that they can be created at a later time.Specify one of the following values:


• *SAME—Uses the current setting for this parameter.• *YES—Holds commands to create configuration objects in specific physical files so that they can be created at a

later time with the CRTCFGOBJ command.• *NO—Automatically creates configuration objects as soon as they are received on the node you are changing in the

cluster.• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL

command.The default setting for this parameter is *SAME.If a configuration object that is being replicated already exists on the node you are changing in the cluster, you can set this value to *YES to prevent iCluster from trying to create the object when it is in use.For more information about configuration objects, see Replicating Configuration Objects on page 90.

• STGSTORSZIndicates the size (in megabytes) that you want to allocate for the staging store. Note that the size of the store changes dynamically but it cannot exceed the size specified through this parameter. The size of the staging store can range from a minimum of 512 megabytes to a maximum of 1,048,576 megabytes.The staging store is a non-volatile storage mechanism for backup nodes that is managed on a node-by-node basis. The size of the staging store must be set for each backup node in the cluster.All objects and data replicated between nodes will travel through the staging stores on the backup nodes. Assuming the node update process is active, the journal entries will be ready to be applied. If the node update process is not active, the staging store will hold all journal entries until the node apply process is re-started.Specify the size of the staging store or the following value:• *SAME—Uses the current setting for this parameter.

The default setting for this parameter is *SAME.For more information about staging stores, see Staging Objects on page 69.

Note: Unlike the DMADDNODE—Add Node command, you cannot modify the staging store library through this command. To change the existing staging store library for the specified node, the node must be removed and then re-added to the cluster. See DMRMVNODE—Remove Node on page 109 and DMADDNODE—Add Node on page 99 for more information.

• SWITCHRESIndicates whether you will be using switchable resources on the node.Enter one of the following values:• *SAME—Keeps the present setting for this parameter. This is the default value.• *YES—Enables the node to use switchable resources. If you specify this value, then the OS/400 option 41, HA

Switchable Resources, must be installed on the node and there must be a valid license key for the option.• *NO—Does not enable the use of switchable resources.

• CHGSTATUSIndicates whether the status of the specified node should be changed.Enter one of the following values:• *SAME—Keeps the present setting for this parameter. This is the default value.• *FAILED—Use this value when a node has failed but its status in the cluster is *PARTITION. After the status of the

node has been changed to *FAILED, it can either be removed from the cluster or re-started if it is capable of rejoining the cluster. For more information on failovers and switchovers, see Failovers and Switchovers on page 47. To identify your failover mechanism, see Failover Mechanisms on page 47.


Node Commands

• CHKPRIMLNKThis parameter indicates if you want to test the primary IP address for communication failures between the primary and backup nodes. The possible values are:• *SAME—Use the current setting for this parameter.• *YES—Test the primary IP address for communication failures.• *NO—Do not test the primary IP address for communication failures.

The default setting for this parameter is *SAME.This parameter has no effect in clusters that use IBM Cluster Resource Services.

• CHKALTLNKThis parameter indicates if you want to test the alternate IP address for communication failures between the primary and backup nodes. The possible values are:• *SAME—Use the current setting for this parameter.• *YES—Test the alternate IP address for communication failures.• *NO—Do not test the alternate IP address for communication failures.


• LNKCHKFRQSet this parameter to how often, in seconds, you want to poll the primary and alternate links for communication failures. The default setting for this parameter is *SAME.This parameter has no effect in clusters that use IBM Cluster Resource Services.

• LNKCHKRTOSet this parameter to the number of seconds you want to wait for a response when polling links for failures. The default setting for this parameter is *SAME.This parameter has no effect in clusters that use IBM Cluster Resource Services.

• LNKCHKTRYSet this parameter to the number of consecutive failures you want to allow before iCluster considers the primary node to have failed. For example, if you set this parameter to 3, then after the fourth consecutive failure, iCluster will change the status of the primary node to *FAILED. The default setting for this parameter is *SAME.This parameter has no effect in clusters that use IBM Cluster Resource Services.

• MSGUSREXITSet this parameter to the name and library of the user exit program to run after the IBM Cluster Resource Services number of consecutive communication failures, specified with the LNKCHKTRY parameter, is exceeded. This user exit is run once per message. The possible values are:• *SAME—Use the current settings for this parameter.• LIBRARY/NAME—The name and library of the message queue.• *NONE—Do not queue messages.


• MSGQUEUE


Set this parameter to the name and library of the message queue that will receive messages after the IBM Cluster Resource Services number of consecutive communication failures, specified with the LNKCHKTRY parameter, is exceeded. The possible values are:• *SAME—Use the current settings for this parameter.• LIBRARY/NAME—The name and library of the message queue.• *NONE—Do not queue messages.


• MSGACTWAITSet this parameter to the number of minutes to wait before sending messages to the message queue, specified with the MSGQUEUE parameter, after detecting a failure. The default setting for this parameter is *SAME.


This parameter has no effect in clusters that use IBM Cluster Resource Services.• NUMMSGACT

Set this parameter to the maximum number of messages to send through the message queue, specified with the MSGQUEUE parameter, after detecting a failure. The system will wait for the time specified in the MSGACTWAIT parameter between sending each message. The default setting for this parameter is *SAME.


This parameter has no effect in clusters that use IBM Cluster Resource Services.• AUTHCODE

The new authorization code for the node that was provided by DataMirror Technical Support. See Contacting Technical Support on page 547 for more information.The authorization code was specified during iCluster installation, and is required to run iCluster on the node. Therefore, use this parameter to change the existing authorization code for the node if it no longer allows you to run iCluster.Specify the authorization code or the following value:• *SAME—Uses the current setting for this parameter.

The default setting for this parameter is *SAME.

ExamplesDMCHGNODE NODE(NODE1) IPADDR(155.4.5.44) IPADDR2(155.4.5.45) DESC(Chicago) REPLJOBD(LIB1/RJD) USRPRFSTS(*DISABLED) CFGSRCHLD(*YES) STGSTORSZ(4096) SWITCHRES(*NO) CHGSTATUS(*SAME) AUTHCODE(XCEFAASQLPJUIHE)

Changes the attributes of NODE1 in the cluster.Changes the primary IP address of NODE1 to 155.4.5.44 (expressed in dotted quad notation), and the secondary IP address to 155.4.5.45 (also expressed in dotted quad notation).Changes the description associated with the node to indicate that the system is located in Chicago.Changes the job description associated with replication jobs on NODE1 to RJD in library LIB1.User profiles replicated to NODE1 will all be set to a status of *DISABLED.Commands to create configuration objects will be held in specific physical files so that they can be created at a later time.Changes the maximum size of the staging store on NODE1 to 4,096 megabytes.


Node Commands

Switchable resources will not be enabled on the node, and the status of the node will remain the same.Changes the iCluster authorization code for NODE1.

DMCHGNODE NODE(NODE2) IPADDR(sys2a.abc123corp.com) IPADDR2(sys2b.abc123corp.com) DESC(London) REPLJOBD(LIB1/RJD) USRPRFSTS(*PRIMARY) CFGSRCHLD(*NO) STGSTORSZ(8192) SWITCHRES(*NO) CHGSTATUS(*SAME) AUTHCODE(*SAME)

Changes the attributes of NODE2 in the cluster.Changes the primary IP address of NODE2 to sys2a.abc123corp.com (expressed in domain name form), and the secondary IP address to sys2b.abc123corp.com (also expressed in domain name form).Changes the description associated with the node to indicate that the system is located in London.Changes the job description associated with replication jobs on NODE2 to RJD in library LIB1.User profiles replicated to NODE2 will be set to the same status that is currently assigned to the corresponding user profile on the primary node.Configuration objects replicated to NODE2 will be automatically created as soon as they are received.Changes the maximum size of the staging store on NODE2 to 8,192 megabytes.Switchable resources will not be enabled on the node, and the status of the node will remain the same.The iCluster authorization code remains the same.DMCHGNODE NODE(NODE3) LNKCHKTRY(3)Changes the attributes of NODE3 in the cluster.Sets the SwitchOver System to allow three failures before changing the status of the primary node to *FAILED.

UseYou must issue this command from an active node that can communicate with the master node.



DMRMVNODE—Remove NodeDMRMVNODE NODE( )

Use this command to remove a node that was previously defined in the cluster. Cluster operations at the node are also stopped. A reference cannot be made to the node after it has been removed from the cluster.If you are removing the only node in the cluster, the entire cluster is deleted. In this case, the removal of the node is equivalent to the operations performed when the DMDLTCLSTR—Delete Cluster command is invoked.You can use this command to remove the backup node for replication groups and applications. Note that you cannot remove the primary node of a replication group or resilient application. If you wish to remove such a node from the cluster, you can take one of two actions:• Remove all replication groups and resilient applications having this node as their primary node. See DMRMVGROUP—

Remove Group and DMRMVAPP—Remove Resilient Application for more information.• End cluster operations at the node before removing it. See DMENDNODE—End Cluster Operations at Node for more

information. After ending cluster operations, ensure that you remove the cluster definitions from the inactive node by issuing the DMDLTCLSTR—Delete Cluster command at that node.



The name of the existing node that you want to remove from the cluster.

ExampleDMRMVNODE NODE(NODE1)

Removes the node NODE1 from the cluster.

UseYou must issue this command from an active node. If you issue this command on a node other than the specified node, the specified node must be active. If the specified node is inactive, the node will only be removed from the active part of the cluster. Therefore, an attempt to add the same node to the cluster at a later time will fail unless you issue the DMDLTCLSTR—Delete Cluster command on the specified node only before attempting to re-add it to the cluster.



DMWRKNODE—Work with NodesDMWRKNODE BY( ) GROUP( ) APPL( )Use this command to list the nodes that have been defined in the cluster. The BY, GROUP, and APPL parameters allow you to filter the list by displaying the nodes that are included in the recovery domain for a specific replication group or resilient application.The status of each node is indicated on the displayed screen, and can be one of the following:• *ACTIVE—Indicates that cluster operations are running on the node.• *ACT_PEND—Indicates that cluster operations are in the process of being started on the node.• *INACTIVE—Indicates that cluster operations are not running on the node.• *INACT_PND—Indicates that cluster operations are in the process of being stopped on the node.• *NEW—Indicates that the node has been defined in the cluster, but cluster operations have not been started on the node.• *RMV_PEND—Indicates that the node is in the process of being removed from the cluster.• *FAILED—Indicates that a system failure has occurred on the node.• *PARTITION—Indicates that a system failure has occurred on the node or communications with the node have been lost. If a

primary node in an active group is in this state, the group will assume an *INACTIVE status and no role change will occur. If a backup node in an active group is in this state, the group will remain in an *ACTIVE state and no role change will occur. See DMWRKGRP—Work with Groups on page 137 for more information about group states.For general information about system failures on nodes within your cluster configuration, see Failovers and Switchovers on page 47. See Failover Mechanisms on page 47 for information on identifying your failover mechanism.

• *IN_ERROR—Indicates that an internal iCluster error may have occurred. If this status persists, contact DataMirror Technical Support. See Contacting Technical Support on page 547 for more information.

• *UNKNOWN—Indicates that the node is not recognized in iCluster or the cluster is not recognized on the system or the current node is inactive in the cluster.


Node Commands

You should consult Adding Nodes on page 31 for important pre-requisite information concerning nodes in your clustered environment.

Input Parameters• BY

The set of nodes that are listed by this command.Specify one of the following values:• *NONE—Lists all nodes defined in iCluster.• *GROUP—Lists the nodes that are included in the recovery domain for the group specified through the GROUP

parameter (see below).• *APPL—Lists the nodes that are included in the recovery domain for the resilient application identified through the

APPL parameter (see below).The default setting for this parameter is *NONE.

• GROUPThe name of an existing group. Nodes included in the recovery domain for the named group are listed. This parameter is only applicable when the BY parameter (see above) is set to *GROUP.

• APPLThe name of an existing resilient application. Nodes included in the recovery domain for the named application are listed. This parameter is only applicable when the BY parameter (see above) is set to *APPL.

ExamplesDMWRKNODE BY(*NONE)

Lists all nodes defined in iCluster.DMWRKNODE BY(*GROUP) GROUP(GRP1)

Lists all nodes in the recovery domain for replication group GRP1.DMWRKNODE BY(*APPL) APPL(OEPACK)

Lists all nodes included in the recovery domain for the resilient application OEPACK.

UseYou must issue this command on an active node in the cluster.


Menu AccessMain Menu - Option 1F22 (Shift+F10) - Option 5


Group Commands

Group CommandsThis section contains the following commands:• DMADDGRP—Add Group on page 113• DMDSPGRP—Display Group on page 124• DMCHGGRP—Change Group on page 124• DMRMVGROUP—Remove Group on page 134• DMADDBACK—Add Backup Node to Recovery Domain on page 135• DMRMVBACK—Remove Backup Node from Recovery Domain on page 135• DMCHGROLE—Change Group Primary Node on page 136• DMWRKGRP—Work with Groups on page 137

DMADDGRP—Add GroupDMADDGRP GROUP( ) GRPTYPE( ) DMNSRC( ) PRIMNODE( ) PRIMIASP( ) BACKUPS( ) BACKIASP( ) TGTLIB( ) MQVERSION( ) QMNAME( ) MSGQUEUE( ) ROLESWITCH( ) CHKJRN( ) BSWUSREXIT( ) ASWUSREXIT( ) EXITDATA( ) POLLINT( ) SAVACT( ) MAXSPLWAIT( ) SPLACTWAIT( ) DFTDBJRN( ) JRNLOC( ) JRNBACKUP( ) CMTLVL( ) JRNBA( ) OPTIMZAPY( ) DFTBSFJRN( ) RCVRYEXP( ) JRNKEY( ) LCKBACKUP( ) CFGSRCHLD( ) SWDEV( ) ONLINE( ) CRWNR( ) CRUSREXIT( ) DESC( )

Use this command to add a group consisting of one primary node and possibly one backup node.After creating a group, use the DMSELOBJ—Select Objects to Group and/or DMADDBSF—Add Path Object Specifier to Group commands to specify the objects it will replicate.Using this command you can create object specifiers and indicate the target library where the objects that match the object specifier are replicated.

Input Parameters• GROUP

The name of the group that you want to define in iCluster.The specified group name must be unique in the cluster.

• GRPTYPESpecifies the type of group that you want to define.Specify one of the following values:• *REPL—Indicates that you want to create a group for continuous object replication between the primary and backup

nodes.• *RFSH—Indicates that you want to create a refresh-only group.• *SWDEV—Indicates that you want to create a group for the switchable disk storage devices on the primary and

backup nodes.• *MQSERIES—Indicates that you want to create a WebSphere MQ group for object replication between the primary

and backup nodes.• *MSTRMSTR—Indicates that you want to create a master-master replication group that replicates application

updates on the primary into active tables on the backup node.


For bi-directional replication, two master-master groups can be set up to mirror updates to the same objects in both directions. For more information on this type of replication, see iBalance and Master-Master Replication on page 321.

• DMNSRCThe name of an existing group or resilient application that you want to use to define the recovery domain of the group you are adding.This parameter allows you to add a group with the same set of primary and backup nodes as the recovery domain for an existing group or resilient application. Defining two or more replication groups with the same set of nodes is useful when you want to replicate objects between the same nodes, but you want to define different replication group behaviors for different sets of objects.Specify the name of an existing group, resilient application, or the following value:• *LIST—Indicates that you want to explicitly identify the primary and backup nodes in the replication group instead of

selecting an existing replication group or resilient application.The default setting for this parameter is *LIST.

• PRIMNODEThe name of a node that will be the primary node in the group you are adding.The node that you specify must have been added to the cluster using the DMADDNODE—Add Node command. This parameter applies only if the DMNSRC parameter (see above) is set to *LIST.For *SWDEV groups, the primary and backup nodes must have the switchable disk storage devices enabled (SWITCHRES parameter). For more information, see the DMADDNODE—Add Node command.

• PRIMIASPThe name of an independent auxiliary storage pool (IASP) on the primary node from which you want to replicate.Specify the name of an existing IASP or the following value:• *SYSBAS—Indicates that you are using a system ASP rather than an independent ASP for storing the objects that

you want to replicate.The default setting for this parameter is *SYSBAS.

• BACKUPSThe name of a node that will be the backup node in the group you are adding.At this time, only one backup node can be specified for a group.The node that you specify must have been added to the cluster using the DMADDNODE—Add Node command. This parameter applies only to the DMNSRC parameter (see above) is set to *LIST.

• BACKIASPThe name of an independent auxiliary storage pool (IASP) on the backup node to which you want to replicate.Specify the name of an existing IASP or the following value:• *SYSBAS—Indicates that you are using a system ASP rather than an independent ASP for storing the objects to be

replicated.The default setting for this parameter is *SYSBAS.

• TGTLIBThe name of the library on the backup system that receives the replicated objects.Specify the name of a target library or the following value:• *PRIMARY—Sets the target library so that it is the same as the primary node library where the object resides.


Group Commands

If the primary and backup environments reside on the same physical system (local loopback replication), the target library that you specify cannot be the same library where a selected object in the group currently resides. This restriction means that iCluster does not permit the special value *PRIMARY in this situation.*PRIMARY is the default setting for this parameter.

Note: Switchovers and role changes are not supported for groups that use a target library other than *PRIMARY. Consequently, because a non-primary target library is required for local loopback replication, switchovers and role changes are not supported for local loopback replication.

• MQVERSIONIndicates the product version of WebSphere MQ that you are using. WebSphere MQ Versions V5R2M0, V5R3M0, and V6R0M0 are supported.The default setting for this parameter is V5R3M0.

Note: This parameter only applies to groups of type *MQSERIES.• QMNAME

Indicates the name of the WebSphere MQ queue manager that must be mirrored. Multiple queue managers are not supported. If you want to mirror multiple queue managers, use the DMADDGRP—Add Group command to define a group for each of them.Enter specific names. Do not use *DFT or generic names.

Note: This parameter only applies to groups of type *MQSERIES.• MSGQUEUE

The name of the message queue on the new primary node that will receive messages generated when a failover occurs.Note that this parameter is only valid for replication groups.Specify the name of a message queue or the following value:• *NONE—Indicates that you do not want to specify a message queue.

The default setting for this parameter is *NONE.The library where the message queue resides must be identified if you do not specify *NONE. Prefix the message queue with the name of the library where the queue is located. For example, LIB1/MSGQ1.

• ROLESWITCHIndicates if you want the role of the backup node to change automatically so that it becomes the primary node in the replication group when a failover occurs.This parameter is valid only for replication groups which have *PRIMARY as the target library, and to MQSeries groups.The functions of the primary node will be moved to the backup node if this parameter is set to *YES when a failover occurs.Specify one of the following values:• *YES—Automatically changes the role of the backup node in the replication group to the primary node when a

failover occurs. In this case, user exit programs specified through the BSWUSREXIT and ASWUSREXIT parameters (see below) are called.

• *NO—Does not automatically change the role of the backup node in the replication group to the primary node when a failover occurs. Setting the ROLESWITCH parameter to *NO allows you to decide if you want to continue with a role switch, or continue with your current configuration.

With this setting, a failure of the group's primary node is declared by the failover mechanism, although iCluster does not prepare the backup node to take over as the primary node. Replication does not start automatically once the failed primary node is again active in the cluster.


When using IBM Cluster Resource Services, only groups which have a cluster status of *ACTIVE are switched over if the primary node fails.When IBM Cluster Resource Services are not being used for iCluster operations, all groups that have the ROLESWITCH parameter set to *YES are switched over when the primary node fails, regardless of whether they are active or not.The default setting for this parameter is *NO.For more information about the ROLESWITCH parameter, see Switchover for IBM Cluster Resource Services (CRS) on page 341.

• CHKJRNSpecifies whether or not when switching to the new primary node, iCluster checks that the objects for the specified group are being properly journaled on that node, and starts journaling them. Certain types of objects must be journaled before replication starts. However, because journaling objects can be time-consuming, this option allows you to specify whether or not journaling is started for objects during a switchover, depending on your business requirements.Depending on the value of the group's CHKJRN parameter, iCluster may perform processing that is equivalent to the DMMRKPOS—Mark Current Journal Positions command when a role switch occurs. If the value of the group's CHKJRN parameter is *NO, iCluster does not perform DMMRKPOS—Mark Current Journal Positions processing and the replication metadata for the new primary node is generated based on the existing replication metadata on the current backup node. In case of an error in processing the backup replication metadata, or non-existent backup replication metadata, iCluster performs DMMRKPOS—Mark Current Journal Positions processing regardless of the CHKJRN parameter's value.

Note: The DMMRKPOS—Mark Current Journal Positions command establishes the set of mirrored objects based on the actual objects that match specifiers on the backup system, regardless if they were previously mirrored or not, and starts journaling for the mirrored objects that should be journaled.

If the value of the group's CHKJRN parameter is *YES, iCluster will always perform DMMRKPOS—Mark Current Journal Positions processing during a role switch, and the backup replication metadata is not used for setting up the replication metadata for the new primary node.You can specify one of the following values:• *YES—Specifies that iCluster checks if all mirrored objects that should be journaled are being journaled on the new

primary node. If not, the objects will be journaled.• *NO—Specifies that iCluster does not check if all mirrored objects that should be journaled are journaled on the new

primary node. You must make sure that all objects that need to be journaled are being journaled to the correct journals, and that the objects have been journaled before replication starts.

The default setting for this parameter is *YES.• BSWUSREXIT

The name of the user exit program that you want to call immediately before the role change is performed.The user exit program is called on both nodes of the group for a switchover, but only on the new primary node for a failover.This parameter is valid only for replication groups that have their target library set to *PRIMARY.If you specify a user exit program, the ROLESWITCH parameter (see above) must be set to *YES if you want to call the program when a failover occurs. The specified user exit program will always be called (regardless of the ROLESWITCH parameter setting) when a switchover is initiated.Specify the name of a user exit program or the following value:• *NONE—Indicates that you do not want to specify a user exit program.

The default setting for this parameter is *NONE.The library where the user exit program resides must be identified if you do not specify *NONE. Prefix the name of the user exit program name with the name of the library where the program is located (for example, LIB1/USREXIT1).


Group Commands

For more information about the arguments that can be passed to the user exit program, see Passing Arguments to Role Switch User Exit Programs on page 347.

• ASWUSREXITThe name of the user exit program that you want to call immediately after the role change is performed. After a role change occurs, the user exit program will be called on both nodes of the group.This parameter applies only to replication groups that have their target library set to *PRIMARY.If you specify a user exit program, the ROLESWITCH parameter (see above) must be set to *YES if you want to call the program when a failover occurs. The specified user exit program will always be invoked (regardless of the ROLESWITCH parameter setting) when a switchover is initiated.Specify the name of a user exit program or the following value:• *NONE—Indicates that you do not want to specify a user exit program.

The default setting for this parameter is *NONE.The library where the user exit program resides must be identified if you do not specify *NONE. Prefix the name of the user exit program name with the name of the library where the program is located (for example, LIB1/USREXIT1).See Passing Arguments to Role Switch User Exit Programs on page 347 for more information about the arguments that can be passed to the user exit program.

• EXITDATAIdentifies the user-defined data that you want to pass to the user exit programs specified through the BSWUSREXIT and ASWUSREXIT parameters (see above).This parameter applies only to replication groups that have their target library set to *PRIMARY.If you specify two different user exit programs (one program before the role switch, and a different program after the role switch), the same user-defined data is passed to both programs. A maximum of 256 bytes of data can be passed to the user exit programs. If your data contains spaces or non-alphanumeric characters (commas, periods, and so on), you must enclose your data in single quotes.

• POLLINTThe time interval (expressed in HHMMSS format) that determines how often iCluster should check for content changes to user spaces.This parameter applies only to replication groups.You set the interval by specifying the number of hours (first two digits), minutes (middle two digits) and the seconds (last two digits) between consecutive polls. The polling interval applies only when user spaces (*USRSPC) are selected to the replication group through the DMSELOBJ—Select Objects to Group command.Specify a time interval or one of the following values:• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL—Set

Cluster System Values command.• *NONE—Specifies that user spaces should not be polled at the group level.

Not polling user spaces can ease the polling resources on a system that has many user spaces that have not been changed.

Note: If *NONE is selected as the polling interval for a *USRSPC object specifier but the group's polling interval is not *NONE, polling occurs for all user spaces selected to the group that do not match the *USRSPC object specifier that has a polling interval of *NONE.

The default setting for this parameter is *CLUSTER.• SAVACT


Indicates if an object can be updated and saved at the same time it is being transferred to the backup node. This parameter does not apply to physical files because they cannot be modified while they are being saved.This parameter applies only to replication groups.Specify one of the following values:• *YES—Allows objects that are in use by another job to be saved and updated. Objects may reach checkpoints at

different times and may not be in a consistent state in relationship to each other.• *NO—Does not allow objects that are in use to be saved and updated at the same time. Objects cannot be updated

while being saved.• *SYNC—Allows objects that are in use by another job to be saved and updated. All of the objects reach a

checkpoint together and are saved in a consistent state in relationship to each other.• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL—Set

Cluster System Values command.The default setting for this parameter is *CLUSTER.

• MAXSPLWAITThe time period (expressed in HHMMSS format) you want iCluster to wait before abandoning the replication of a spool file.This parameter applies only to replication groups.A spool file cannot be replicated unless it has a status of *READY. Therefore, if iCluster is up-to-date with the audit journal, it may start processing a spool file before it is ready. iCluster will only wait the amount of time set in this parameter for the spool file to have a status of *READY. If the spool file is not ready after waiting the amount of time specified in this parameter, iCluster abandons the replication of this spool file and issues a message in the event log.Specify a waiting period from 000001 to 235959, or the following value:• *CLUSTER—Refers to the cluster system value for this parameter that is specified using the DMSETSVAL—Set


• SPLACTWAITThe time period (expressed in HHMMSS format) you want iCluster to wait before abandoning the replication of a spool file if no data is being written to it.This parameter applies only to replication groups.In some cases, a job may be writing to a spool file. However, the data is first placed in a temporary buffer and only written to the spool file upon the completion of the job. In such a case, the spool file will be open and have zero pages until the job ends.This parameter allows you to specify the time to wait for changes to occur to an open spool file. If no additional data is written to an open spool file (that is, it is open and has zero pages) within the time specified by this parameter, iCluster abandons the replication of this spool file and issue a message. This parameter provides added flexibility as you can specify a shorter interval over which only spool file activity is monitored. As a result, an open spool file that has no data being written to it will be abandoned quickly, as opposed to waiting the amount of time specified in the MAXSPLWAIT parameter (see above).Specify a waiting period from 000001 to 235959, or the following value:• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL—Set


• DFTDBJRN


Group Commands

The name of the database journal that you want to use as your default. This parameter is valid only for replication groups.The journal that you specify is used for files that are to be mirrored but are not yet journaled. iCluster starts journaling automatically in this journal.Specify the name of the database journal or the following value:• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL—Set

Cluster System Values command.The default setting for this parameter is *CLUSTER.The library where the journal resides must be identified if you do not specify *CLUSTER. Prefix the journal with the name of the library where the journal is located (for example, LIB1/JRN1).

Note: If this group replicates data from an IASP device, then the journal must exist on the IASP device before the group can start replicating objects.

• JRNLOCThe location of the database journal where scraping occurs.Specify one of the following values:• *LOCAL—Indicates that the database journal(s) on the primary node are scraped during mirroring.• *REMOTE—Indicates that the remote database journal(s) are scraped on the backup node during mirroring.

Note: When a remote journal group starts, iCluster activates the remote journal automatically (if it is not already active) using the default value for the DELIVERY parameter in the CHGRMTJRN command.

The default setting for this parameter is *LOCAL.After changing the journal location with this parameter, you must set the position of the local journal to the position of the last applied entry by using the DMSETPOS command with the JRNPOS(*LASTAPY) parameter.

• JRNBACKUPIndicates whether replicated physical files, data areas, and data queues will be journaled on backup nodes.Specify one of the following values:• *CLUSTER—Indicates that this parameter uses the system value that is specified through the DMSETSVAL—Set

Cluster System Values command.• *YES—Indicates that the replicated physical files, data areas, and data queues will be journaled on backup nodes.• *NO—Indicates that the replicated physical files, data areas, and data queues will not be journaled on backup

nodes.The default setting for this parameter is *CLUSTER.

• CMTLVLThe level of commitment control you want to use when replicating *FILE objects.This parameter applies only to replication groups.Commitment control stages database transactions so that they are assembled before being applied, or additionally, opened in a commitment control environment to ensure that all updates are received. It also makes sure that the changes are applied in the correct sequence. For more information, see Commitment Control on page 71.Specify one of the following values:• *NONE—Indicates that the update process on the backup node will not perform commitment control staging.• *LEVEL1—Indicates that all updates that comprise a transaction are assembled before being applied on the backup

node.


• *LEVEL2—Indicates that all updates in a transaction are opened in a commitment control environment to ensure that all updates are received. This option provides true commitment control.

• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL—Set Cluster System Values command.

The default setting for this parameter is *CLUSTER.If you select *LEVEL2, but a file on the backup node cannot be opened under commitment control, *LEVEL1 commitment control is used for that file.

• JRNBAIndicates if default journaling includes both before and after images.This parameter applies only to replication groups.Before images are necessary to remove applied or keyed replication updates. Also, if you specified unique index refresh method for a file, then it must be journaled with both before and after images. If a file is replicated using relative record number, it may be journaled with after images only.For unique key updates, both before and after images must be journaled if the CMTLVL parameter (see above) is set to *LEVEL2.Specify one of the following values:• *BOTH—Indicates that you want to journal both the before and after images.• *AFTER—Indicates that you want to journal the after image only.• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL—Set


• OPTIMZAPYIndicates whether or not you want to optimize database apply entries. Optimization can increase the apply performance for large files that are significantly larger than the shared memory pool, and have a large number of random updates applied to them.Specify one of the following values:• *NO—Indicates that you do not want optimization for database apply updates.• *YES—Indicates that you want optimization for database apply updates.

The default setting for this parameter is *NO.• DFTBSFJRN

The name of the journal that you want to use as your default for journaling BSF files that will be replicated by this group. This parameter only applies to replication groups.


The journal that you specify will be used for BSF files that are to be mirrored and journaled. See the DFTBSFJRN parameter on the DMADDBSF—Add Path Object Specifier to Group command for more information.Specify the name of the BSF journal or one of the following values:• *NONE—Indicates that you do not require journaling for BSF replication.• *CLUSTER—BSF objects selected to this group use the journal specified at the product level. See the

DMSETSVAL—Set Cluster System Values command for more information.The default setting for this parameter is *CLUSTER.


Group Commands

The library where the journal resides must be identified if you do not specify *NONE or *CLUSTER. Prefix the journal with the name of the library where the journal is located (for example, LIB2/JRN2).

• RCVRYEXPSpecifies the number of journal entries applied between writing recovery checkpoints. Specifying a lower number decreases the performance of the apply, but provides a greater safeguard in recovery situations. Specifying a higher number lessens the impact of recovery checkpoints on the apply.This parameter applies only to replication groups. Enter a recovery exposure value (number of journal entries) or the following value:• *DISABLED—Indicates that recovery checkpoints will not be generated. This is the default setting for this

parameter.• JRNKEY

Specifies the key, or the identifier, that will be assigned to a recovery checkpoint for the group combination. You must also specify this value in the RTVRCVPT—Retrieve Recovery Checkpoint or RTVRCVPTR—Retrieve Recovery Checkpoint (CL Program) command.This parameter only applies to replication groups.

• LCKBACKUPIndicates whether journaled files should be locked on the backup node. When locked, no other users can modify these files on the backup node.Specify one of the following values:• *CLUSTER—Indicates that this parameter uses the system value that is specified through the DMSETSVAL—Set

Cluster System Values command.• *YES—Indicates that files cannot be modified on the backup node when the group is active.• *NO—Indicates that files will not be locked when the group is active, meaning that users can modify these files on

the backup node.The default setting for this parameter is *CLUSTER.

• CFGSRCHLDIndicates whether you want iCluster to automatically create configuration objects immediately after they are received on a backup node or hold the commands for creating them in specific physical files so that they can be created at a later time with the CRTCFGOBJ command.If configuration objects that are being replicated already exist on backup nodes, this parameter should be set to *YES in order to prevent iCluster from trying to create objects when they are in use.Specify one of the following values:• *BACKUP—Indicates that this parameter uses the Hold configuration object source setting on the backup node

of the group that is specified through the DMADDNODE—Add Node command.• *CLUSTER—Indicates that this parameter uses the system value that is specified through the DMSETSVAL—Set

Cluster System Values command.• *YES—Holds commands to create configuration objects in specific physical files so that they can be created at a

later time with the CRTCFGOBJ command.• *NO—Automatically creates configuration objects as soon as they are received on the node.

The default setting for this parameter is *BACKUP.• SWDEV

The name of a switchable disk storage device that is associated with this group.


Note: This parameter applies only when the GRPTYPE parameter is set to *SWDEV.• ONLINE

Indicates whether the device associated with the switchable resource group is varied on or varied off when the group is switched over from one node to another or when it is failed over to another node.This parameter applies only to groups of type *SWDEV.Specify one of the following values:• *YES—The device that is associated with the group is varied on when the group is switched over from one node to

another or when it is failed over to another node. This is the default value for this parameter.• *NO—The device that is associated with the group is not varied on when the group is switched over from one node

to another or when it is failed over to another node.• CRWNR

Indicates the group-level rules by which iCluster will resolve conflicts for the objects selected to a *MSTRMSTR replication group.Specify one of the following values:• *SRC—Indicates that the source wins if there is a conflict. Source data is applied to the target, which ensures that

target data matches source data and mirroring continues.• *TGT—Indicates that the target wins if there is a conflict. iCluster does not apply data to the target. This rule

preserves data on the target and mirroring continues.• *EXIT—Indicates that you would like to resolve data conflicts through a user exit program. With this option, you

must specify a user exit program and library in the CRUSREXIT parameter below.• *NONE—Indicates that you do not want to enable conflict detection and resolution rules. If there is a conflict,

iCluster suspends the file from replication and sends a message to the event log. This is the default behavior for iCluster.

The default setting for this parameter is *NONE.Note: This parameter only applies to iBalance and *MSTRMSTR replication groups. See iBalance and Master-

Master Replication on page 321 for more information on this feature.• CRUSREXIT

The name of the user exit program that you want to call to resolve data conflicts. A user exit program allows you to define a conflict resolution strategy that is not covered by one of the standard resolution methods in the CRWNR parameter.Specify the name of a user exit program or the following value:• *NONE - Indicates that you do not want to specify a user exit program.

The library where the user exit program resides must be identified if you do not specify *NONE. Prefix the name of the user exit program name with the name of the library where the program is located (for example, LIB1/USREXIT1).See Parameter List Descriptions for a Conflict Resolution User Exit Program on page 326 for more information about the values you can pass to the user exit program.The default setting for this parameter is *NONE. You cannot specify *NONE for this parameter if you specify *EXIT for the CRWNR parameter.

Note: This parameter only applies to iBalance and *MSTRMSTR replication groups. See iBalance and Master-Master Replication on page 321 for more information on this feature.

• DESCA short description that allows you to identify this group amongst all others that have been defined in iCluster.The description cannot exceed 50 characters in length.


Group Commands

ExampleDMADDGRP GROUP(GRP1) GRPTYPE(*REPL) DMNSRC(*LIST) PRIMNODE(NODE1) PRIMIASP(DMC_IASP1) BACKUPS(NODE2) BACKIASP(*SYSBAS) TGTLIB(*PRIMARY) MSGQUEUE(LIB1/MSGQ1) ROLESWITCH(*YES) CHKJRN(*YES) BSWUSREXIT(LIB1/BFEXIT) ASWUSREXIT(LIB1/AFEXIT) EXITDATA(ARJ123908KPJ230982) POLLINT(003000) SAVACT(*NO) MAXSPLWAIT(000200) SPLACTWAIT(000030) DFTDBJRN(LIB1/JRN1) JRNLOC(*LOCAL) CMTLVL(*LEVEL2) JRNBA(*BOTH) OPTIMZAPY(*YES) DFTBSFJRN(LIB2/JRN2) DESC('NY/LA')

Adds the replication group GRP1. The primary and backup nodes in the replication group are explicitly identified through the PRIMNODE and BACKUP parameters.The primary node in the replication group is NODE1, and the backup node is NODE2. The switchable disk storage device on NODE1 is DMC_IASP1. The target library (TGTLIB) is set so that it is the same as the primary node library where the object resides. Messages generated when a failover occurs will be added to the message queue MSGQ1 in library LIB1. The backup node in the replication group will become the primary node if a failover occurs. iCluster will check if all mirrored objects that should be journaled are being journaled on the new primary node in the event of a failover or switchover. User exit programs specified through the BSWUSREXIT and ASWUSREXIT parameters will be called when failover or switchover occurs.The user exit program BFEXIT in library LIB1 will be called immediately before a role change is performed.The user exit program AFEXIT in library LIB1 will be called immediately after a role change is performed.User-defined data consisting of a sequence of characters will be passed to the user exit programs.The polling interval for user spaces selected for replication within the group is 30 minutes.Objects that are in use cannot be saved and updated at the same time. Objects cannot be updated while being saved.iCluster waits a maximum of two minutes for a spool file to have a status of *READY before abandoning the replication of a spool file.iCluster waits 30 seconds for changes to occur to an open spool file before abandoning the replication of a spool file.The default database journal is JRN1 in library LIB1. Changes are applied with *LEVEL2 commitment control.The journal location is set to *LOCAL. Database journal scraping occurs on the group's primary node.Both before and after images are journaled for changes to database files.iCluster will optimize database apply entries.The default BSF journal is JRN2 in library LIB2.Description indicates that the nodes in the replication group are located in New York and Los Angeles.

DMADDGRP GROUP(GRP1) GRPTYPE(*MQSERIES) DMNSRC(*LIST) PRIMNODE(NODE1) BACKUPS(NODE2) MQVERSION (V5R2M0 ) QMNAME(QMANAGER) MSGQUEUE(LIB1/MSGQ1) BSWUSREXIT(LIB1/BFEXIT) ASWUSREXIT(LIB1/AFEXIT) EXITDATA(ARJ123908KPJ230982) DESC('NY/LA')

Adds the group GRP1 for WebSphere MQ objects. The primary and backup nodes in the group are explicitly identified through the PRIMNODE and BACKUP parameters.The primary node in the group is NODE1, and the backup node is NODE2. iCluster supports WebSphere MQ Versions V5R2M0 and V5R3M0. The queue manager name for WebSphere MQ is QMANAGER. Messages generated when a failover occurs will be added to the message queue MSGQ1 in library LIB1.The backup node in the group will become the primary node if a failover occurs. User exit programs specified through the BSWUSREXIT and ASWUSREXIT parameters will be called if a failover or switchover occurs.Description indicates that the nodes in the group are located in New York and Los Angeles.

UseYou must issue this command from an active node, and all specified nodes in the added group must be active.



Menu AccessF22 (Shift+F10) - Option 6

DMDSPGRP—Display GroupDMDSPGRP GROUP( )Use this command to display the current settings for the specified replication group.

Input Parameter• GROUP

The name of the replication group that you want to examine.

ExampleDMDSPGRP GROUP(GRP1)

Displays the current settings for the replication group GRP1.



Menu AccessF22 (Shift+F10) - Option 11Work With Groups screen - Option 5

DMCHGGRP—Change GroupDMCHGGRP GROUP( ) GRPTYPE( ) TGTLIB( ) MSGQUEUE( ) ROLESWITCH( ) CHKJRN( ) BSWUSREXIT( ) ASWUSREXIT( ) EXITDATA( ) POLLINT( ) SAVACT( ) MAXSPLWAIT( ) SPLACTWAIT( ) DFTDBJRN( ) JRNLOC( ) JRNBACKUP( ) CMTLVL( ) JRNBA( ) OPTIMZAPY( ) DFTBSFJRN( ) RCVRYEXP( ) JRNKEY( ) LCKBACKUP( ) CFGSRCHLD( ) CRWNR( ) CRUSREXIT( ) DESC( )Use this command to change one or more attributes of an existing replication group.It does not allow you to change the primary and backup nodes in the replication group. You can add and remove the backup nodes using the DMADDBACK—Add Backup Node to Recovery Domain and DMRMVBACK—Remove Backup Node from Recovery Domain commands.


The name of the replication group that you want to change in iCluster.• GRPTYPE

Specifies the type of group that you want to define.Note: This parameter does not apply to switchable device groups and MQSeries groups.

Specify one of the following values:


Group Commands

• *SAME—Uses the current setting for this parameter.• *REPL—Indicates that you want to change a refresh-only group to a replication group.• *RFSH—Indicates that you want to change a replication group to a refresh-only group.• *MSTRMSTR—Indicates that you want to define a master-master replication group that replicates application

updates on the primary into active tables on the backup node.For bi-directional replication, two master-master groups can be set up to mirror updates to the same objects in both directions. For more information on this type of replication, see iBalance and Master-Master Replication on page 321.

Note: You cannot change a *MSTRMSTR group to another group type or vice versa.This value is only available with iBalance. See iBalance and Master-Master Replication on page 321 for more information on this feature.The default setting for this parameter is *SAME.

• TGTLIBThe name of the library on the backup system that will receive the replicated objects.Specify the name of a target library or one of the following values:• *SAME—Uses the current setting for this parameter.• *PRIMARY—Sets the target library so that it is the same as the primary node library where the object resides.

The default setting for this parameter is *SAME.If the primary and backup environments reside on the same physical system (local loopback implementation), the target library that you specify cannot be the same library where a selected object in the group currently resides. This restriction means that iCluster does not permit the special value *PRIMARY in this situation.

Note: Switchovers and role changes are not supported for groups that use a target library other than *PRIMARY. Also, because a non-primary target library is required for local loopback replication, switchovers and role changes are not supported for local loopback replication either.

• MSGQUEUEThe name of the message queue that will receive messages generated when a failover occurs.Note that this parameter is valid only for replication groups.Specify the name of a message queue or one of the following values:• *SAME—Uses the current setting for this parameter.• *NONE—Indicates that you do not want to specify a message queue.

The default setting for this parameter is *SAME.The library where the message queue resides must be identified if you do not specify *SAME or *NONE. Prefix the message queue with the name of the library where the queue is located (for example, LIB1/MSGQ1).

• ROLESWITCHIndicates whether you want the role of the backup node to automatically change so that it becomes the primary node in the replication group when a failover occurs.This parameter is valid only for replication groups whose target library is *PRIMARY, and to MQSeries groups.The functions of the primary node will be moved to the backup node if this parameter is set to *YES when a failover occurs.Specify one of the following values:• *SAME—Uses the current setting for this parameter.


• *YES—Automatically changes the role of the backup node in the replication group to the primary node when a failover occurs. In this case, user exit programs specified through the BSWUSREXIT and ASWUSREXIT parameters (see below) will be called.

• *NO—Does not automatically change the role of the backup node in the replication group to the primary node when a failover occurs. Setting the ROLESWITCH parameter to *NO gives you the opportunity to decide if you would like to continue with a role switch, or continue with your current configuration.

With this setting, a failure of the group's primary node is declared by the failover mechanism, although iCluster will not prepare the backup node to take over as the primary node. Replication will not start up automatically once the failed primary node is again active in the cluster.When using IBM Cluster Resource Services, only groups whose cluster status is *ACTIVE are switched over if the primary node fails.When IBM Cluster Resource Services are not being used for iCluster operations, all groups that have the ROLESWITCH parameter set to *YES are switched over when the primary node fails, regardless of whether they are active or not.The default setting for this parameter is *SAME.See Switchover for IBM Cluster Resource Services (CRS) on page 341 for more information about the ROLESWITCH parameter.

• CHKJRNSpecifies whether (when switching to the new primary node) iCluster will check whether the objects for the specified group are being properly journaled on that node, and start journaling them. Certain types of objects must be journaled before replication starts. However, because journaling objects can be time-consuming, this option allows you to specify whether or not journaling should be started for objects during a switchover, depending on your business requirements.Depending on the value of the group's CHKJRN parameter, iCluster may perform processing that is equivalent to the DMMRKPOS—Mark Current Journal Positions command when a role switch occurs. If the value of the group's CHKJRN parameter is *NO, iCluster will not perform DMMRKPOS—Mark Current Journal Positions processing and the replication metadata for the new primary node is generated based upon the existing replication metadata on the current backup node. In the case of an error in processing the backup replication metadata, or non-existent backup replication metadata, iCluster performs DMMRKPOS—Mark Current Journal Positions processing regardless of the CHKJRN parameter's value.The DMMRKPOS—Mark Current Journal Positions command establishes the set of mirrored objects based on the actual objects that match specifiers on the backup system, regardless if they were previously mirrored or not, and starts journaling for the mirrored objects that should be journaled.If the value of the group's CHKJRN parameter is *YES, iCluster will always perform DMMRKPOS—Mark Current Journal Positions processing during a role switch, and the backup replication metadata is not used for setting up the replication metadata for the new primary node.You can specify one of the following values:• *SAME—Keeps the present setting for this parameter.• *YES—Specifies that iCluster will check if all mirrored objects that should be journaled are being journaled on the

new primary node. If not, the objects will be journaled.• *NO—Specifies that iCluster will not check if all mirrored objects that should be journaled are journaled on the new

primary node. You will need to make sure that all objects that need to be journaled are being journaled to the correct journals, and that the objects have been journaled before replication starts.

The default setting for this parameter is *SAME.• BSWUSREXIT

The name of the user exit program that you want to call immediately before the role change is performed.This parameter is valid only for replication groups whose target library is *PRIMARY.


Group Commands

If you specify a user exit program, the ROLESWITCH parameter (see above) must be set to *YES if you want to call the program when a failover occurs. The specified user exit program will always be invoked (regardless of the ROLESWITCH parameter setting) when a switchover is initiated.Specify the name of a user exit program or one of the following values:• *SAME—Uses the current setting for this parameter.• *NONE—Indicates that you do not want to specify a user exit program.

The default setting for this parameter is *SAME.The library where the user exit program resides must be identified if you do not specify *SAME or *NONE. Prefix the name of the user exit program name with the name of the library where the program is located (for example, LIB1/USREXIT1).See Passing Arguments to Role Switch User Exit Programs on page 347 for more information about the arguments that can be passed to the user exit program.

• ASWUSREXITThe name of the user exit program that you want to call immediately after the role change is performed.This parameter is valid only for replication groups whose target library is *PRIMARY.If you specify a user exit program, the ROLESWITCH parameter (see above) must be set to *YES if you want to call the program when a failover occurs. The specified user exit program will always be invoked (regardless of the ROLESWITCH parameter setting) when a switchover is initiated.Specify the name of a user exit program or one of the following values:• *SAME—Uses the current setting for this parameter.• *NONE—Indicates that you do not want to specify a user exit program.

The default setting for this parameter is *SAME.The library where the user exit program resides must be identified if you do not specify *SAME or *NONE. Prefix the name of the user exit program name with the name of the library where the program is located (for example, LIB1/USREXIT1).See Passing Arguments to Role Switch User Exit Programs on page 347 for more information about the arguments that can be passed to the user exit program.

• EXITDATAIdentifies the user-defined data that you want to pass to the user exit programs specified through the BSWUSREXIT and ASWUSREXIT parameters (see above).This parameter is valid only for replication groups whose target library is *PRIMARY.If you specify two different user exit programs (one program before the role switch, and another program after the role switch), the same user-defined data is passed to both programs. A maximum of 256 bytes of data can be passed to the user exit programs. If your data contains spaces or non-alphanumeric characters (commas, periods, and so on), you must enclose your data in single quotes.Specify user-defined data or the following value:• *SAME—Uses the current setting for this parameter.

The default setting for this parameter is *SAME.• POLLINT

The time interval (expressed in HHMMSS format) that determines how often iCluster should check for content changes to user spaces.Note that this parameter is valid only for replication groups.


You set the interval by specifying the number of hours (first two digits), minutes (middle two digits) and the seconds (last two digits) between consecutive polls. The polling interval is only applicable when user spaces (*USRSPC) are selected to the replication group through the DMSELOBJ—Select Objects to Group command.Specify a time interval or one of the following values:• *SAME—Uses the current time interval setting.• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL—Set

Cluster System Values command.• *NONE—Specifies that user spaces should not be polled at the group level.

Not polling user spaces can ease the polling resources on a system that has many user spaces that have not been changed.

Note: If *NONE is selected as the polling interval for a *USRSPC object specifier but the group's polling interval is not *NONE, polling will occur for all user spaces selected to the group that do not match the *USRSPC object specifier whose polling interval is *NONE.

The default setting for this parameter is *SAME.• SAVACT

Indicates whether an object can be updated and saved at the same time it is being transferred to the backup node. Note that this parameter does not apply to physical files because they cannot be modified while they are being saved.Note that this parameter is valid only for replication groups.Specify one of the following values:• *SAME—Uses the current setting for this parameter.• *YES—Allows objects that are in use by another job to be saved and updated. Objects may reach checkpoints at

different times and may not be in a consistent state in relationship to each other.• *NO—Does not allow objects that are in use to be saved and updated. Objects cannot be updated while being

saved.• *SYNC—Allows objects that are in use by another job to be saved and updated. All of the objects reach a

checkpoint together and are saved in a consistent state in relationship to each other.• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL—Set

Cluster System Values command.The default setting for this parameter is *SAME.

• MAXSPLWAITThe time period (expressed in HHMMSS format) you want iCluster to wait before abandoning the replication of a spool file.Note that this parameter is valid only for replication groups.A spool file cannot be replicated unless it has a status of *READY. Therefore, if iCluster is up-to-date with the audit journal, it may start processing a spool file before it is ready. iCluster will only wait the amount of time set in this parameter for the spool file to have a status of *READY. If the spool file is not ready after waiting the time specified in this parameter, iCluster will abandon the replication of this spool file and issue a message in the event log.Specify a waiting period from 000001 to 235959, or one of the following values:• *SAME—Uses the current setting for this parameter.• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL—Set


• SPLACTWAIT


Group Commands

The time period (expressed in HHMMSS format) you want iCluster to wait before abandoning the replication of a spool file if no data is being written to it.Note that this parameter is valid only for replication groups.In some cases, a job may be writing to a spool file. However, the data is first placed in a temporary buffer and only written to the spool file upon the completion of the job. In such a case, the spool file will be open and have zero pages until the job ends.This parameter allows you to specify the time to wait for changes to occur to an open spool file. If no additional data is written to an open spool file (that is, it is open and has zero pages) within the time specified by this parameter, iCluster will abandon the replication of this spool file and issue a message. This parameter provides added flexibility as you can specify a shorter interval over which only spool file activity is monitored. As a result, an open spool file that has no data being written to it will be abandoned quickly, as opposed to waiting the full time specified in the MAXSPLWAIT parameter (see above).Specify a waiting period from 000001 to 235959, or one of the following values:• *SAME—Uses the current setting for this parameter.• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL

command.The default setting for this parameter is *SAME.

• DFTDBJRNThe name of the database journal that you want to use as your default. This parameter is valid only for replication groups.The journal that you specify will be used for objects that are to be mirrored but are not yet journaled.Specify the name of the database journal or one of the following values:• *SAME—Uses the current setting for this parameter.• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL—Set

Cluster System Values command.The default setting for this parameter is *SAME.The library where the journal resides must be identified if you do not specify *CLUSTER or *SAME. Prefix the journal with the name of the library where the journal is located (for example, LIB1/JRN1).

Note: Note that if this group replicates data from an IASP device, then the journal must exist on the IASP device before the group can start replicating objects.

• JRNLOCThe location of the database journal where scraping will occur.You can specify the following value:• *LOCAL—Indicates that the database journal(s) on the primary node are scraped during mirroring.• *REMOTE—Indicates that the remote database journal(s) are scraped on the backup node during mirroring.• *SAME—Keeps the present setting for this parameter.

The default setting for this parameter is *SAME.After changing the journal location with this parameter, you must set the position of the local journal to the position of the last applied entry by running the DMSETPOS—Set Journal Start Position command with the JRNPOS(*LASTAPY) parameter.

• JRNBACKUPIndicates whether replicated physical files, data areas, and data queues will be journaled on backup nodes.


Specify one of the following values:• *SAME—Keeps the present setting for this parameter.• *CLUSTER—Indicates that this parameter uses the system value that is specified through the DMSETSVAL—Set

Cluster System Values command.• *YES—Indicates that the replicated physical files, data areas, and data queues will be journaled on backup nodes.• *NO—Indicates that the replicated physical files, data areas, and data queues will not be journaled on backup

nodes.The default setting for this parameter is *SAME.

• CMTLVLThe level of commitment control you want to use when replicating *FILE objects.This parameter is valid only for replication groups.Commitment control stages database transactions so that they are assembled before being applied, or additionally, opened in a commitment control environment to ensure that all updates are received. It also makes sure that the changes are applied in the correct sequence. For more information, see Commitment Control on page 71.Specify one of the following values:• *SAME—Uses the current setting for this parameter.• *NONE—Indicates that the update process on the backup node will not perform commitment control staging.• *LEVEL1—Indicates that all updates that comprise a transaction are assembled before being applied on the backup

node.• *LEVEL2—Indicates that all updates in a transaction are opened in a commitment control environment to ensure

that all updates are received. This option provides true commitment control.• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL—Set

Cluster System Values command.The default setting for this parameter is *SAME.Note that if you select *LEVEL2 but a file on the backup node cannot be opened under commitment control, *LEVEL1 commitment control will be used for that file.

• JRNBAIndicates whether default journaling should include both before and after images.Note that this parameter is valid only for replication groups.Before images are necessary to remove applied or keyed replication updates. Also, if you specified unique index refresh method for a file, then it must be journaled with both before and after images. If a file is replicated using relative record number, it may be journaled with after images only.For unique key updates, both before and after images must be journaled if the CMTLVL parameter (see above) is set to *LEVEL2.Specify one of the following values:• *SAME—Uses the current setting for this parameter.• *BOTH—Indicates that you want to journal both the before and after images. • *AFTER—Indicates that you want to journal the after image only.• *CLUSTER—Refers to the cluster system value for this parameter that is specified through the DMSETSVAL—Set


• OPTIMZAPY


Group Commands

Indicates whether you want to optimize database apply entries. Optimization can increase the apply performance for large files that are significantly larger than the shared memory pool, and have a large number of random updates applied to them.Specify one of the following values:• *SAME—Uses the current setting for this parameter.• *NO—Indicates that you do not want optimization for database apply updates.• *YES—Indicates that you want optimization for database apply updates.

The default setting for this parameter is *SAME.• DFTBSFJRN

The name of the journal that you want to use as your default for journaling BSF files that will be replicated by this group. This parameter only applies to replication groups.


The journal that you specify will be used for BSF files that are to be mirrored and journaled. See the DFTBSFJRN parameter on the DMADDBSF—Add Path Object Specifier to Group command for more information.Specify the name of the BSF journal or one of the following special values:• *SAME—Uses the current setting for this parameter.• *NONE—Indicates that you do not require journaling for BSF replication.• *CLUSTER—BSF objects selected to this group will use the journal specified at the product level. See the

DMSETSVAL—Set Cluster System Values command for more information.The default setting for this parameter is *CLUSTER.The library where the journal resides must be identified if you do not specify *NONE or *CLUSTER. Prefix the journal with the name of the library where the journal is located (for example, LIB2/JRN2).

Note: Note that If this group replicates data from an IASP device, then the journal must exist on the IASP device before the group can start replicating objects.

• RCVRYEXPSpecifies the number of journal entries applied between writing recovery checkpoints. Specifying a lower number decreases the performance of the apply but provides a greater safeguard in recovery situations. Specifying a higher number lessens the impact of recovery checkpoints on the apply.

Note: This parameter only applies to replication groups.Enter a recovery exposure value (number of journal entries) or one of the following values:• *SAME—Uses the current setting for this parameter.• *DISABLED—Indicates that recovery checkpoints will not be generated.

The default setting for this parameter is *SAME.• JRNKEY

Specifies the key, or the identifier, that will be assigned to a recovery checkpoint for the group combination. This value must be specified for the RTVRCVPT—Retrieve Recovery Checkpoint or RTVRCVPTR—Retrieve Recovery Checkpoint (CL Program) command.

Note: This parameter only applies to replication groups.Specify the key or the following value:• *SAME—Uses the current setting for this parameter.


The default setting for this parameter is *SAME.• LCKBACKUP

Indicates whether journaled files should be locked on the backup node. When locked, no other users can modify these files on the backup node.Specify one of the following values:• *SAME—Keeps the present setting for this parameter.• *CLUSTER—Indicates that this parameter uses the system value that is specified through the DMSETSVAL—Set

Cluster System Values command.• *YES—Indicates that files cannot be modified on the backup node when the group is active.• *NO—Indicates that files will not be locked when the group is active, meaning that users can modify these files on

the backup node.The default setting for this parameter is *SAME.

• CFGSRCHLDIndicates whether you want iCluster to automatically create configuration objects immediately after they are received on a backup node or hold the commands for creating them in specific physical files so that they can be created at a later time with the CRTCFGOBJ command.If configuration objects that are being replicated already exist on backup nodes, this parameter should be set to *YES in order to prevent iCluster from trying to create objects when they are in use.Specify one of the following values:• *SAME—Keeps the present setting for this parameter.• *BACKUP—Indicates that this parameter uses the Hold configuration object source setting on the backup node

of the group that is specified through the DMADDNODE—Add Node command.• *CLUSTER—Refers to the system value for this parameter that is specified through the DMSETSVAL—Set Cluster

System Values command.• *YES—Holds commands to create configuration objects in specific physical files so that they can be created at a

later time with the CRTCFGOBJ command.• *NO—Automatically creates configuration objects as soon as they are received on the node.

The default setting for this parameter is *SAME.• CRWNR

Indicates the group-level rules by which iCluster will resolve conflicts for the objects selected to a *MSTRMSTR replication group.Specify one of the following values:• *SAME—Uses the current setting for this parameter.• *SRC—Indicates that the source wins if there is a conflict. Source data is applied to the target, which ensures that






Group Commands

The default setting for this parameter is *SAME.Note: This parameter only applies to iBalance and *MSTRMSTR replication groups. See iBalance and Master-


The name of the user exit program that you want to call to resolve data conflicts. A user exit program allows you to define a conflict resolution strategy that is not covered by one of the standard resolution methods in the CRWNR parameter.Specify the name of a user exit program or one of the following values:• *SAME—Uses the current setting for this parameter.• *NONE—Indicates that you do not want to specify a user exit program.

The library where the user exit program resides must be identified if you do not specify *NONE. Prefix the name of the user exit program name with the name of the library where the program is located (for example, LIB1/USREXIT1).See Parameter List Descriptions for a Conflict Resolution User Exit Program on page 326 for more information about the values you can pass to the user exit program.The default setting for this parameter is *SAME. You cannot specify *NONE for this parameter if you specify *EXIT for the CRWNR parameter.


• DESCA short description that allows you to identify this group among all others that have been defined in iCluster.The description can be a maximum of 50 characters long.Specify a description or the following value:• *SAME—Uses the current setting for this parameter.


ExampleDMCHGGRP GROUP(GRP1) TGTLIB(TGT1) MSGQUEUE(LIB1/MSGQ1) ROLESWITCH(*YES) BSWUSREXIT(*NONE) ASWUSREXIT(LIB1/AFEXIT) EXITDATA('ARJ 123908 KPJ 230982') POLLINT(*SAME) SAVACT(*SYNC) MAXSPLWAIT(000500) SPLACTWAIT(*CLUSTER) DFTDBJRN(LIB1/JRN1) JRNLOC(*LOCAL) CMTLVL(*NONE) JRNBA(*AFTER) OPTIMZAPY(*YES) RCVRYEXP(5000) JRNKEY(MYGROUP) DESC('LON/PAR')

Changes will be made to one or more attributes of the replication group GRP1.TGT1 is the name of the library on the backup system that will receive the replicated objects.Messages generated when a failover occurs will be added to the message queue MSGQ1 in library LIB1.The backup node in the replication group will become the primary node if a failover occurs. The user exit program specified through the ASWUSREXIT parameter will be called.No user exit program will be called immediately before a role change is performed.The user exit program AFEXIT in library LIB1 will be called immediately after a role change is performed.User-defined data consisting of a sequence of characters will be passed to the user exit program. Note that single quotes are required to enclose data that includes spaces and other non-alphanumeric characters.The polling interval for user spaces selected for replication within the group is the current setting.Objects can be updated and saved when they are being replicated within the group. All of the objects will reach a checkpoint together and will be saved in a consistent state in relationship to each other.iCluster will wait a maximum of five minutes for a spool file to have a status of *READY before abandoning the replication of a spool file.


iCluster will reference the corresponding cluster system value to determine how long to wait for changes to occur to an open spool file before abandoning the replication of a spool file.The default database journal will be JRN1 in library LIB1.The journal location is set to *LOCAL. Journal scraping will occur on the group's primary node.Changes will be applied with no commitment control.Only after images will be journaled.iCluster will optimize database apply entries.There will be 5000 journal entries applied between recovery checkpoints.MYGROUP is assigned to a recovery checkpoint for the group combination.The description associated with the replication group indicates that the nodes are located in London and Paris.

DMCHGGRP GROUP(GRP2) GRPTYPE(*RFSH)Changes the replication group GRP2 to a refresh-only group.

UseYou must issue this command from an active node, and the specified replication group must be inactive.



DMRMVGROUP—Remove GroupDMRMVGRP GROUP( )Use this command to remove a replication group that was previously defined. A reference cannot be made to the group after it has been removed.If the group is active, group operations are automatically stopped before the group is removed.


The name of the existing group that you want to remove.

ExampleDMRMVGRP GROUP(GRP1)

Removes the replication group GRP1.





Group Commands

Work With Groups screen - Option 6

DMADDBACK—Add Backup Node to Recovery DomainDMADDBACK NAME( ) NODE( ) BACKIASP( )Use this command to add a backup node to the recovery domain for an existing group or resilient application.You can add only a single backup node through this command. The recovery domain for the group or resilient application must already have a defined primary node. You can only add a backup node to the recovery domain for inactive groups or resilient applications that are not currently running.You must add a backup node to the recovery domain before replication can be started for a group or resilient application.

Input Parameters• NAME

The name of the group or resilient application that will have a backup node added to its recovery domain.• NODE

The name of the node in the cluster that will be added to the recovery domain for the specified group or resilient application.You must specify a node that has been added to the cluster.For an *SWDEV group, the node must have switchable disk storage devices enabled (SWITCHRES parameter). See the DMADDNODE—Add Node command for more information.

• BACKIASPThe name of the independent auxiliary storage pool (IASP) on the backup node to which you want to replicate.Specify the name of an existing IASP or the following value:• *SYSBAS—Indicates that you are using a system ASP rather than an independent ASP for storing the objects that

will be replicated.The default setting for this parameter is *SYSBAS.

ExampleDMADDBACK NAME(GRP1) NODE(NODE1) BACKIASP(DMC_3)

Adds the backup node NODE1 to group GRP1. The name of the IASP on the backup node is DMC_3.

UseYou must issue this command on an active node in the cluster. The specified node must also be active.


Menu AccessF22 (Shift+F10) - Option 9 or Option 27Accessible from the Work With Groups screen (Option 22) and the Work With Resilient Applications screen (Option 22).

DMRMVBACK—Remove Backup Node from Recovery DomainDMRMVBACK NAME( ) NODE( )Use this command to remove the backup node from the recovery domain for an existing group or resilient application.


Note that you can remove a backup node only using this command. The recovery domain for the group or resilient application must already have defined primary and backup nodes. You can remove backup nodes only from recovery domains for inactive groups or resilient applications that are not currently running.


The name of the group or resilient application that will have a backup node removed from the recovery domain.• NODE

The name of the backup node in the cluster that will be removed from the recovery domain for the specified group or resilient application.

ExampleDMRMVBACK NAME(GRP1) NODE(NODE1)Removes the backup node NODE1 from group GRP1.




DMCHGROLE—Change Group Primary NodeDMCHGROLE GROUP( ) PRIMNODE( )

This command changes the specified groups' or resilient applications' primary node to the current first backup node. The groups or resilient applications will remain *INACTIVE until they are started with the DMSTRGRP or DMSTRAPP commands.The current backup is prepared to become the primary node in the same way as occurs if a failover happens when the group is active.

Note: A message (CSI1314) stating whether or not the role switch completed successfully is placed into the event log on all nodes of the cluster. For more information about event logs, see Monitoring Replication on page 76.

Note: This command supports spooled file replication on the primary and backup node(s) as of i5/OS V5R4.


The name of the group or resilient application. If you specify a group or resilient application, the group (or groups of the resilient application) has to be in *INACTIVE status.Specify the name of the group or resilent application or one of the following values:• *ALL—Performs a role switch for all groups and resilient applications in the cluster. Changes the primary node of

all groups and resilient applications in the cluster to the current first backup node.• *PRIMNODE—Performs a role switch for all groups and resilient applications whose primary node is specified on

the PRIMNODE parameter. If you specify this option, you must specify the name of the primary node in the PRIMNODE parameter (see below).

• PRIMNODE


Group Commands

Indicates the name of the primary node of the groups and resilient applications for which you are performing a role switch. This parameter is only applicable when the GROUP parameter (see above) is set to *PRIMNODE.

ExamplesDMCHGROLE GROUP(GROUP1)

Changes the primary node to the backup node and the backup node to the primary node. The group or resilient application must have a status of *INACTIVE.

DMCHGROLE GROUP(*ALL)Changes the primary node to the backup node and the backup node to the primary node for all groups and resilient applications in the cluster. The groups or resilient applications must have a status of *INACTIVE.

UseYou can invoke the DMCHGROLE command on an active node in the cluster for groups or applications in *INACTIVE status.

Minimum Security LeveAdmin (*ADMIN)

Menu AccessF22 (Shift+F10) - Option 51Work With Groups screen - Option 20Work With Resilient Applications screen - Option 20

DMWRKGRP—Work with GroupsDMWRKGRP BY( ) NODE( ) APPL( )

Use this command to list the groups that have been defined. The BY, NODE, and APPL parameters allow you to filter the list by displaying the groups that contain a specific node or are associated with a specific resilient application.The cluster status of each group is indicated on the displayed screen, and can be one of the following:• *ACTIVE—Indicates that low-level cluster support within the group is running.• *INACTIVE—Indicates that low-level cluster support within the group is not running.• *INIT_PEND—Indicates that the group is in the process of being defined in iCluster.• *ADDN_PEND—Indicates that a node is in the process of being added to the group.• *RMVN_PEND—Indicates that a node is in the process of being removed from the group.• *DLT_PEND—Indicates that the group is in the process of being removed from iCluster.• *STRT_PEND—Indicates that low-level cluster operations are in the process of being started.• *END_PEND—Indicates that low-level cluster operations are in the process of being stopped.• *SWO_PEND—Indicates that a role switch is being performed within the group.• *RESTORED—Indicates that the replication group object (*CRG) has been restored to a system (a *CRG object

identifies all of the nodes contained in the group).• *CHG_PEND—Indicates that the replication group object (*CRG) for the group is in the process of being changed.• *DLTCMD_PN—Indicates that the replication group object (*CRG) for the group is in the process of being deleted by the

OS/400 command DLTCRG.• *INDOUBT—Indicates that the status of the group cannot be determined.


• *IN_ERROR—Indicates that an internal iCluster error may have occurred. If this state persists, contact DataMirror Technical Support. See Contacting Technical Support on page 547 for more information.

• *UNKNOWN—Indicates that the group is not recognized in iCluster.In addition, the replication status that is currently assigned to the replication group is displayed, and can be one of the following:

• *ACTIVE—Indicates that iCluster replication is currently being performed within the replication group.• *INACTIVE—Indicates that iCluster replication is not currently being performed within the replication group.• *INDOUBT—Indicates that some, but not all of the replication groups are running.• *IN_ERROR—Indicates that an internal iCluster error may have occurred. If this state persists, contact DataMirror

Technical Support. See Contacting Technical Support on page 547 for more information.• *UNKNOWN—Indicates that the replication status of the replication group cannot be determined. This status may arise

if the primary node in the group is currently inactive or if there is no backup node in the group.• *SWOSTART—Indicates that iCluster is starting switchover processing.• *BEFUEXIT—Indicates that iCluster is executing the before roleswitch user exit.• *STSDRAIN—Indicates that iCluster is draining the staging store.• *STRJRN—Indicates that iCluster is starting journaling on the journaled objects in replication scope.• *TRIGGERS—Indicates that triggers are being enabled on the new primary machine.• *CONSTR—Indicates that iCluster is enabling the referential integrity constrains on the new primary machine.• *CHGROLES—Indicates that iCluster is changing the roles of the primary and backup nodes.• *MRKPOS—Indicates that iCluster is marking the starting position for mirroring on the new primary node.• *AFTUEXIT—Indicates that iCluster is executing the after roleswitch user exit.


The set of groups that are listed by this command.Specify one of the following values:• *NONE—Lists all groups defined in iCluster.• *NODE—Lists the groups that contain the node specified through the NODE parameter (see below).• *APPL—Lists the groups that are associated with the resilient application identified through the APPL parameter

(see below).The default setting for this parameter is *NONE.

• NODEThe name of an existing node in the cluster. Groups that contain the specified node are listed. This parameter is only applicable when the BY parameter (see above) is set to *NODE.

• APPLThe name of an existing resilient application. Groups associated with the identified application are listed. This parameter is only applicable when the BY parameter (see above) is set to *APPL.

ExamplesDMWRKGRP BY(*NONE)

Lists all groups.DMWRKGRP BY(*NODE) NODE(NODE1)


Group Commands

Lists all groups that contain the node NODE1.DMWRKGRP BY(*APPL) APPL(OEPACK)

Lists all groups that are associated with the resilient application OEPACK.





Native AS/400 Object Commands

Native AS/400 Object CommandsThis section contains the following commands:• DMSELOBJ—Select Objects to Group on page 141• DMCHGOBJSL—Change Object Selection to Group on page 147• DMDSELOBJ—De-select Objects from Group on page 151• DMWRKOBJ—Work with Object Specifiers on page 152

DMSELOBJ—Select Objects to GroupDMSELOBJ GROUP( ) OBJ( ) OBJTYPE( ) OBJATTR( ) TGTLIB( ) DESC( ) INCFLG( ) POLLINT( ) NEWOBJACT( ) TRUNCATE( ) REPLACELFS( ) MIRRCNTS( ) PFUPDMTD( ) PFKEY( ) CRWNR( ) CRUSREXIT( )

Use this command to select an object specifier to the replication group identified through the first parameter. The objects referenced by selected specifiers are replicated from the primary node to the backup node in the replication group.Through the use of generic names, you can select any number of object specifiers to a replication group.You must define the replication group in iCluster before selecting object specifiers to the replication group.

Note: This command can be issued when adding new object specifiers for active replication groups. See the NEWOBJACT parameter in this command for more information.

Note: If you decide to synchronize objects on the primary and backup nodes through a save file or tape transfer, it is important that you select the object specifiers referencing these objects before saving the objects. This ensures that changes to the objects will be audited between the time of the save and the time when replication is started.

Using this command you can create object specifiers and indicate the specific target library where the objects that match the object specifier should be replicated. Target library support at the group level is a requirement for local loopback replication.


The name of the defined replication group that will have objects selected to it.• OBJ

The object name component of the specifier that you want to select.Enter a specific or generic name (to identify multiple objects in a library), or the following value:• *ALL—Specifies all objects in a library.

The library where the objects reside must be identified. Prefix the object specification with the name of the library where the objects are located (for example, LIB1/OBJ1).

• OBJTYPEThe object type component of the specifier that you want to select.Specify an object type or the following value:• *ALL—Specifies all object types.

For a complete list of all object types that iCluster can replicate, see Object Types Replicated by iCluster on page 529.• OBJATTR

The attribute component of the specifier you want to select.This parameter is only applicable when the OBJTYPE parameter (see above) is set to *FILE.


This field is free-form. Consequently, you can enter any value you want to describe the object, as long as the value conforms to OS/400 naming conventions. However, there are values that have special meaning to iCluster. They are PFSRC (source physical file), PFDTA (data physical file), and PF (any physical file). Other values listed are standard OS/400 file sub-types. If PF is used, the object specifier will match either PFSRC or PFDTA files.Press F4 for a list of values or enter the following value:• *ALL—Specifies all object attributes. *ALL is not a valid OS/400 object attribute but allows iCluster to gather all

objects regardless of their attribute.The default setting for this parameter is *ALL.

• TGTLIBThe name of the library on the backup system that will receive the replicated objects.Specify the name of a target library or one of the following values:• *GROUP—Specifies the same target library as specified in the DMADDGRP—Add Group or DMCHGGRP—

Change Group commands.This is the default setting for this parameter.

• *PRIMARY—Sets the target library so that it is the same as the library where the object resides on the primary node.

If the primary and backup environments reside on the same physical system (local loopback implementation), the target library that you specify cannot be the same library where a selected object in the group currently resides. This restriction means that iCluster does not permit the special value *PRIMARY in this situation.

Note: Switchovers and role changes are not supported for groups that have objects selected to them that have a target library other than *GROUP or *PRIMARY. Also, because a non-primary target library is required for local loopback replication, switchovers and role changes are not supported for local loopback replication either.

• DESCA short description that allows you to identify this object specifier selection amongst all others that have been defined in iCluster.The description can be a maximum of 50 characters long.

• INCFLGIndicates whether the objects referenced by the specifier will be replicated within the replication group.Specifying *INCLUDE means that the referenced objects will be replicated to a backup environment when object replication is started. Specifying *EXCLUDE prevents the referenced objects from being replicated to a backup environment.

Note: If the specifier is being added to an active group, only *INCLUDE may be specified.For the rules of precedence for object specifiers, see Object Specifiers and Types on page 50 for more information.Specify one of the following values:• *INCLUDE—Specifies that the referenced objects will be replicated within the replication group when cluster

operations are started.• *EXCLUDE—Specifies that the referenced objects will not be replicated within the replication group when cluster

operations are started.The default setting for this parameter is *INCLUDE.

• POLLINTThe time interval (expressed in HHMMSS format) that determines how often iCluster should check for content changes to user spaces.



You set the interval by specifying the number of hours (first two digits), minutes (middle two digits) and the seconds (last two digits) between consecutive polls. The polling interval is only applicable when the object type is set to *USRSPC (user space).Specify a time interval or one of the following values:• *GROUP—Refers to the replication group value for this parameter that is specified when adding or changing a

replication group. See DMADDGRP—Add Group and DMCHGGRP—Change Group for more information about these two commands.


• *NONE—Specifies that user spaces should not be polled at the object specifier level.Not polling user spaces can ease the polling resources on a system that has many user spaces that have not been changed.


The default setting for this parameter is *GROUP.• NEWOBJACT

Indicates the method by which replication is to begin for the objects that come into replication scope when an object specifier is added.Specify one of the following values:• *NONE—This value does not change the replication status of new, in-scope objects, and is intended to support

initial group configuration. If this value is selected, mirroring must be started with a refresh of the entire group, or with the DMMRKPOS—Mark Current Journal Positions or DMREGPOS—Register Positions commands. The group's replication status cannot be *ACTIVE if you use this value.

• *CURRENT—Replication of journal entries for new, in-scope objects will begin at the time the object specifier is added. Journal entries related to newly in-scope objects created after that time will be replicated. Journal entries related to new, in-scope objects created before that time will not be replicated. The group's replication status must be *ACTIVE for this option to take effect.If this value is selected, no changes should occur on the new, in-scope objects until the OMI0320 event for the object specifier appears in the event log.

• *REFRESH—Newly in-scope objects will be refreshed one at a time. Replication of journal entries for new, in-scope objects begins for each object as it is refreshed. The group's replication status must be *ACTIVE for this option to take effect.

The default setting for this parameter is *NONE.• TRUNCATE

Indicates whether you want to remove existing data before performing a refresh.Specify one of the following values:• *YES—Indicates that you want to remove and recreate existing data files as part of a refresh. This is the default

behavior for iCluster and is appropriate for standard iCluster replication. You can use this option for a master-master group if the data on the source system is known to be complete in order to ensure a synchronized starting point.With this value, logical files are replaced automatically because the deletion of a physical file requires the deletion of all logical files upon which it is based.

• *NO—Indicates that you do not want to remove existing data before performing a refresh. This option is appropriate for master-master replication.


The default setting for this parameter is *NO.See Master-Master Replication—Operations and Monitoring on page 323 for more information on the refresh options that are specific to master-master replication groups.

Note: This parameter is only available with iBalance. See iBalance and Master-Master Replication on page 321 for more information on this feature.

• REPLACELFSIndicates whether you want to replace logical files during a refresh.Specify one of the following values:• *YES—Indicates that you want to replace logical files during a refresh. This is the default behavior for iCluster and

is appropriate for standard replication. You can use this option for a master-master group if the logical files on the source system are the complete set required for your applications in order to ensure a synchronized starting point.

• *NO—Indicates that you want to use existing logical files during a refresh. This option is appropriate for master-master replication.

Note: iCluster does not require logical files based on the replicated physical file to be the same on source and target systems if they are not in replication scope.


Note: This parameter is only available with iBalance. See iBalance and Master-Master Replication on page 321 for more information on this feature.

• MIRRCNTSIndicates whether or not the contents of the object are mirrored. This parameter allows you to mirror object-level operations such as CREATE, DELETE, RENAME, and MOVE, but not the contents of the object.You can only use this parameter with the following type and attribute combinations:• OBJTYPE parameter is *DTAQ• OBJTYPE parameter is *DTAARA• OBJTYPE parameter is *FILE and OBJATTR parameter is PFDTA

Specify one of the following values:• *YES—Indicates the contents of the object are refreshed and mirrored.• *NO—Indicates the contents of objects are not refreshed or mirrored. Refresh creates an empty object at target,

and only the object-level operations such as CREATE, DELETE, RENAME, and MOVE are mirrored.The default setting for this parameter is *YES.

Note: *FILEATTR and *DTAARA sync checks will not report object contents mismatches if you decide not to refresh or mirror the object contents with MIRRCNTS(*NO). See STRHASC—Start Sync Check on page 267 for more information on starting a sync check.

• PFUPDMTDThe method to update physical data files. This parameter is applicable when *FILE object types with an attribute of PFDTA are selected, and when the INCFLG parameter (see above) is set to *INCLUDE.You must specify the update method, either by relative record number or by unique key.Relative record number specifies the location of a record in relation to the beginning of a file.



Unique key specifies a unique index used to update a file. It allows you to perform reorganizations of physical files at different times on the primary node and the backup node. This option requires that both before and after images are journaled on the backup node because before images are required to remove applied or keyed replication updates. Also, if updating multi-member files, the members of the unique index must have the same name as the members of the physical file.Specify one of the following values:• *RRN—Indicates an update by relative record number.• *UKEY—Indicates an update by unique index. If you are updating by unique index, you then need to specify the

name of the unique index (logical file) in the PFKEY parameter (see below). If a unique index cannot be specified, then you must choose update by relative record number.

The default setting for this parameter is *RRN.Note: You cannot change the replication method in an object specifier for a *FILE object from *RRN to *UKEY or

vice-versa.• PFKEY

Indicates a physical file's unique key. An object name and library defines which file to use as the physical file's unique key.This parameter is available only if the PFUPDMTD parameter (see above) is set to *UKEY and the INCFLG parameter (see above) is set to *INCLUDE. It must be specified when the update method is by unique key. The object specifier must identify a unique index for the file.You can specify the unique key and its library or the following value:• *AUTO—Indicates that you want iCluster to automatically determine a unique key for every physical file being

replicated by the object specifier.Note: If the unique key cannot be found on the target, iCluster suspends the physical file on the target with the UKM

suspension code.You must identify the unique key and its library if you do not specify *AUTO. Prefix the physical file key with the name of the library where the key is located (for example, LIB1/OBJ1).

• CRWNRIndicates the object-level rules by which iCluster will resolve conflicts for the objects selected to a *MSTRMSTR replication group.Specify one of the following values:• *GROUP—Indicates that the object specifier inherits conflict resolution rules and the user exit program configured at

the group-level.• *SRC—Indicates that the source wins if there is a conflict. Source data is applied to the target, which ensures that





The default setting for this parameter is *NONE.Note: This parameter only applies to iBalance and *MSTRMSTR replication groups. See iBalance and Master-

Master Replication on page 321 for more information on this feature.


• CRUSREXITThe name of the user exit program that you want to call to resolve data conflicts. A user exit program allows you to define a conflict resolution strategy that is not covered by one of the standard resolution methods in the CRWNR parameter.Specify the name of a user exit program or the following value:• *NONE—Indicates that you do not want to specify a user exit program.



ExamplesDMSELOBJ GROUP(GRP1) OBJ(LIB1/OBJ1) OBJTYPE(*JOBD) DESC('Job Description OBJ1 in LIB1') INCFLG(*INCLUDE) NEWOBJACT (*CURRENT)

Selects the object OBJ1 in library LIB1 that has an object type of *JOBD (job description).Provides a description to be associated with the object selection.The object specifier will be replicated in the replication group GRP1 when cluster operations are started.Replication of journal entries for new, in-scope objects will begin at the time the object specifier is added, provided that the group is currently active.

DMSELOBJ GROUP(GRP2) OBJ(LIB2/OBJ*) OBJTYPE(*USRSPC) TGTLIB(*GROUP) INCFLG(*EXCLUDE)Selects the individual objects in library LIB2 that have names which start with OBJ (for example, OBJ2, OBJTEST, and so on) and an object type of *USRSPC (user spaces).The objects referenced by the specifier will be selected to the replication group GRP2.The target library is the same as specified in the DMADDGRP—Add Group or DMCHGGRP—Change Group commands.The referenced objects will not be replicated within the replication group when cluster operations are started.

DMSELOBJ GROUP(GRP3) OBJ(LIB3/OBJ3) OBJTYPE(*FILE) OBJATTR(PFDTA) TGTLIB(*PRIMARY) INCFLG(*INCLUDE) PFUPDMTD(*UKEY) PFKEY(LIB3/KEY3)

Selects the object OBJ3 in the library LIB3 that has an object type of *FILE and attribute of PFDTA.The object referenced by the specifier will be replicated in the replication group GRP3 when cluster operations are started.The target library is set to *PRIMARY so that it is the same as the primary node library where the object resides.The physical file will be updated by unique key using KEY3 in library LIB3.UseYou must issue this command on an active node in the cluster.


Menu AccessF22 (Shift+F10) - Option 13Work With Object Specifier screen - F6



DMCHGOBJSL—Change Object Selection to GroupDMCHGOBJSL GROUP( ) OBJ( ) OBJTYPE( ) OBJATTR( ) TGTLIB( ) DESC( ) POLLINT( ) MIRRCNTS( ) PFUPDMTD( ) PFKEY( ) CRWNR( ) CRUSREXIT( )

Use this command to change specific attributes of an object specifier selected to a replication group.Through this command, you can change the description currently associated with the object specifier (DESC) and modify the flag that determines whether the referenced objects are replicated within the group (INCFLG). In addition, the polling interval for user spaces can be changed (POLLINT), and the target library (TGTLIB) can be changed.


The name of the defined replication group that will be affected by the change to the object specifier.• OBJ

The object name component of the specifier that you want to change.Note that you cannot change the object name component through this command. It must be identified in order to change the other parameters that can be modified by this command.Enter a specific or generic name (to identify multiple objects in a library), or the following value:• *ALL—Specifies all objects in a library.

The library where the objects reside must be identified. Prefix the object specifier with the name of the library where the objects are located (for example, LIB1/OBJ1).

• OBJTYPEThe object type component of the object specifier that you want to change.Note that you cannot change the object type component through this command. It must be identified in order to change the other parameters that can be modified by this command.Specify an object type or the following value:• *ALL—Specifies all object types.

For a complete list of all object types that iCluster can replicate, see Object Types Replicated by iCluster on page 529.• OBJATTR

The attribute component of the specifier you want to select.This parameter is only applicable when the OBJTYPE parameter (see above) is set to *FILE.Note that you cannot change the object attribute component through this command. It must be identified in order to change the other parameters that can be modified by this command.Press F4 for a list of values or enter the following value:• *ALL—Specifies all object attributes. *ALL is not a valid system attribute but allows iCluster to gather all objects

regardless of their attribute.The default setting for this parameter is *ALL.

• TGTLIBThe name of the library on the backup system that will receive the replicated objects.Specify the name of a target library or one of the following values:• *SAME—Keeps the present setting for this parameter. This is the default value.• *PRIMARY—Sets the target library so that it is the same as the library where the object resides on the primary

node.


If the primary and backup environments reside on the same physical system (local loopback implementation), the target library that you specify cannot be the same library where a selected object in the group currently resides. This restriction means that iCluster does not permit the special value *PRIMARY in this situation.• *GROUP—Specifies the same target library as specified in the DMADDGRP—Add Group or DMCHGGRP—

Change Group commands.Note: Switchovers and role changes are not supported for groups that have objects selected to them that have a

target library other than *GROUP or *PRIMARY. Also, because a non-primary target library is required for local loopback replication, switchovers and role changes are not supported for local loopback replication either.

• DESCThe short description that you want to use to identify this object specifier selection amongst all others that have been defined in iCluster.The description can be a maximum of 50 characters long.Specify a short description or the following value:• *SAME—Uses the description that is currently associated with the object specifier selection to the replication group.

The default setting for this parameter is *SAME.• INCFLG

Indicates whether the objects referenced by the specifier will be replicated within the group.Specifying *INCLUDE means that the referenced objects will be replicated to a backup environment when object replication is started. Specifying *EXCLUDE prevents the referenced objects from being replicated to a backup environment.For the rules of precedence for object specifiers, see Object Specifiers and Types on page 50 for more information.Specify one of the following values:• *SAME—Uses the current setting for this parameter.• *INCLUDE—Indicates that the referenced objects will be replicated within the replication group when cluster

operations are started.• *EXCLUDE—Indicates that the referenced objects will not be replicated within the replication group when cluster

operations are started.The default setting for this parameter is *SAME.

Note: If you change this parameter from *EXCLUDE to *INCLUDE for an existing object specifier, you must issue the INITHAOBJ—Initialize Objects command.

• POLLINTThe time interval (expressed in HHMMSS format) that determines how often iCluster should check for content changes to user spaces.You set the interval by specifying the number of hours (first two digits), minutes (middle two digits), and the seconds (last two digits) between consecutive polls. The polling interval is only applicable when the object type is set to *USRSPC (user space).Specify a time interval or one of the following values:• *SAME—Uses the current polling interval setting.• *GROUP—Refers to the replication group value for this parameter that is specified when adding or changing a

replication group. See DMADDGRP—Add Group and DMCHGGRP—Change Group respectively for more information.




• *NONE—Specifies that user spaces should not be polled at the object specifier level.Not polling user spaces can ease the polling resources on a system that has many user spaces that have not been changed.


The default setting for this parameter is *SAME.• MIRRCNTS

Indicates whether or not the contents of the object are mirrored. This parameter allows you to mirror object-level operations such as CREATE, DELETE, RENAME, and MOVE, but not the contents of the object.You can only use this parameter with the following type and attribute combinations:• OBJTYPE parameter is *DTAQ• OBJTYPE parameter is *DTAARA• OBJTYPE parameter is *FILE and OBJATTR parameter is PFDTA

Specify one of the following values:• *SAME—Uses the current setting for this parameter.• *YES—Indicates the contents of the object are refreshed and mirrored.• *NO—Indicates the contents of objects are not refreshed or mirrored. Refresh creates an empty object at target,

and only the object-level operations such as CREATE, DELETE, RENAME, and MOVE are mirrored.The default setting for this parameter is *YES.*FILEATTR and *DTAARA sync checks will not report object contents mismatches if you decide not to refresh or mirror the object contents with MIRRCNTS(*NO). See STRHASC—Start Sync Check on page 267 for more information on starting a sync check.

• PFUPDMTDThe method to update physical data files. This parameter is applicable when *FILE object types with an attribute of PFDTA are selected, and when the INCFLG parameter (see above) is set to *INCLUDE.You must specify the update method, either by relative record number or by unique key.Relative record number specifies the location of a record in relation to the beginning of a file.Unique key specifies a unique index used to update a file. It allows you to perform reorganizations of physical files at different times on the primary node and the backup node. This option requires that both before and after images are journaled on the backup node because before images are required to remove applied or keyed replication updates. Also, if updating multi-member files, the members of the unique index must have the same name as the members of the physical file.Specify one of the following values:• *SAME—Uses the current setting for this parameter.• *RRN—Indicates an update by relative record number.• *UKEY—Indicates an update by unique index. If you are updating by unique index, you then need to specify the

name of the unique index (logical file) in the PFKEY parameter (see below). If a unique index cannot be specified, then you must choose update by relative record number.


The default setting for this parameter is *SAME.Note: You cannot change the replication method in an object specifier for a *FILE object from *RRN to *UKEY or

vice-versa.• PFKEY

Indicates a physical file's unique key. An object name and library defines which file to use as the physical file's unique key.This parameter is available only if the PFUPDMTD parameter (see above) is set to *UKEY and the INCFLG parameter (see above) is set to *INCLUDE. It must be specified when the update method is by unique key. The object specifier must identify a unique index for the file.You can specify the unique key and its library or the following value:• *SAME—Uses the current setting for this parameter.• *AUTO—Indicates that you want iCluster to automatically determine a unique key for every physical file being

replicated by the object specifier.Note: If the unique key cannot be found on the target, iCluster suspends the physical file on the target with the UKM

suspension code.You must identify the unique key and its library if you do not specify *AUTO. Prefix the physical file key with the name of the library where the key is located (for example, LIB1/OBJ1).

• CRWNRIndicates the object-level rules by which iCluster will resolve conflicts for the objects selected to a *MSTRMSTR replication group.Specify one of the following values:• *SAME—Uses the current setting for this parameter.• *GROUP—Indicates that the object specifier inherits conflict resolution rules and the user exit program configured

at the group-level.• *SRC—Indicates that the source wins if there is a conflict. Source data is applied to the target, which ensures that





The default setting for this parameter is *SAME.Note: This parameter only applies to iBalance and *MSTRMSTR replication groups. See iBalance and Master-


The name of the user exit program that you want to call to resolve data conflicts. A user exit program allows you to define a conflict resolution strategy that is not covered by one of the standard resolution methods in the CRWNR parameter.Specify the name of a user exit program or one of the following values:• *SAME—Uses the current setting for this parameter.• *NONE—Indicates that you do not want to specify a user exit program.




Note: This parameter only applies to iBalance and *MSTRMSTR replication groups. See iBalance and Master-Master Replication on page 321 for more information on installing this feature.

ExamplesDMCHGOBJSL GROUP(GRP1) OBJ(LIB1/OBJ1) OBJTYPE(*AUTL) DESC('Authorization list OBJ1 in LIB1')

Changes the description associated with selection of object OBJ1 in library LIB1 that has an object type of *AUTL (authorization list) to replication group GRP1.

DMCHGOBJSL GROUP(GRP2) OBJ(LIB2/OBJ*) OBJTYPE(*USRSPC) INCFLG(*EXCLUDE)Changes the selection of individual objects in library LIB2 that have names which start with OBJ (for example, OBJ2, OBJTEST, and so on) and an object type of *USRSPC (user spaces) to replication group GRP2.The referenced objects will not be replicated within the replication group when cluster operations are started.

DMCHGOBJSL GROUP(GRP3) OBJ(LIB3/OBJ3) OBJTYPE(*FILE) OBJATTR(PFDTA) INCFLG(*INCLUDE)Changes the selection of object OBJ3 in the library LIB3 that has an object type of *FILE to replication group GRP3.Object OBJ3 will be replicated within the replication group when cluster operations are started.

UseYou must issue this command on an active node in the cluster. The specified replication group must be inactive when you issue this command.


Menu AccessF22 (Shift+F10) - Option 15Work With Object Specifiers screen - Option 2

DMDSELOBJ—De-select Objects from GroupDMDSELOBJ GROUP( ) OBJ( ) OBJTYPE( ) OBJATTR( )Use this command to de-select an object specifier from the replication group identified through the first parameter. De-selecting an object specifier from a replication group prevents referenced objects from being replicated within the group.


The name of the defined replication group that will have objects de-selected from it.• OBJ

The object name component of the specifier that you want to de-select.Enter a specific or generic name (to identify multiple objects in a library), or the following value:• *ALL—Specifies all objects in a library.


You must identify the library where the objects reside. Prefix the object specification with the name of the library where the objects are located (for example, LIB1/OBJ1).

• OBJTYPEThe object type component of the specifier that you want to de-select.Specify an object type or the following value:• *ALL—Specifies all object types.

For a complete list of all object types that iCluster can replicate, see Object Specifiers and Types on page 50.• OBJATTR

The attribute component of the specifier you want to select.This parameter is only applicable when the OBJTYPE parameter (see above) is set to *FILE.Press F4 for a list of values or enter the following value:• *ALL—Specifies all object attributes. *ALL is not a valid system attribute but allows iCluster to gather all objects

regardless of their attribute.The default setting for this parameter is *ALL.

ExamplesDMDSELOBJ GROUP(GRP1) OBJ(LIB1/OBJ1) OBJTYPE(*MSGF)

De-selects the object OBJ1 in library LIB1 that has an object type of *MSGF (message file).The object specifier will be de-selected from the replication group GRP1.

DMDSELOBJ GROUP(GRP2) OBJ(LIB2/OBJ*) OBJTYPE(*SRVPGM)De-selects the objects in library LIB2 that have names which start with OBJ (for example, OBJ2, OBJTEST, and so on) and an object type of *SRVPGM (service program).The individual objects referenced by the specifier will be de-selected and will no longer be replicated.

DMDSELOBJ GROUP(GRP3) OBJ(LIB3/OBJ3) OBJTYPE(*FILE) OBJATTR(DSPF)De-selects the object OBJ3, which resides in library LIB3, from the replication group GRP3.The object specifier is a physical file object of the OS/400 standard type DSPF.



Menu AccessF22 (Shift+F10) - Option 14Work With Object Specifiers screen - Option 4

DMWRKOBJ—Work with Object SpecifiersDMWRKOBJ BY( ) GROUP( )

Use this command to list the object specifiers that have been defined in iCluster. The BY and GROUP parameters allow you to filter the list by displaying the object specifiers that have been selected to a specific group.For more information about object specifiers, see Object Specifiers and Types on page 50.




The set of object specifiers that are listed by this command.Specify one of the following values:• *NONE—Lists all object specifiers defined in iCluster.

*GROUP—Lists the object specifiers that have been selected to the group specified through the GROUP parameter (see below).The default setting for this parameter is *NONE.

• GROUPThe name of an existing group. Object specifiers selected to the named group are listed. This parameter is only applicable when the BY parameter (see above) is set to *GROUP.

ExamplesDMWRKOBJ BY(*NONE)

Lists all object specifiers defined in iCluster.DMWRKOBJ BY(*GROUP) GROUP(GRP1)

Lists all object specifiers selected to the group GRP1.





Byte Stream File (BSF) Commands

Byte Stream File (BSF) CommandsThis section contains the following commands:• DMADDBSF—Add Path Object Specifier to Group on page 155• DMRMVBSF—Remove Path Object Specifier to Group on page 158• DMCHGBSF—Change Path Object Specifier to Group on page 159• DMWRKBSF—Work with Path Object Specifiers on page 162

DMADDBSF—Add Path Object Specifier to GroupDMADDBSF GROUP( ) PATH( ) DESC( ) INCFLG( ) JOURNAL( ) POLLINT( ) NEWOBJACT( ) CRWNR( ) CRUSREXIT( )

Use this command to select one or more Byte Stream File (BSF) objects residing in the Integrated File System (IFS) to the specified replication group. The referenced BSF objects are replicated from the primary node to the backup node in the replication group.

Note: This command can be issued when adding new object specifiers for active replication groups. See the NEWOBJACT parameter in this command for more information.

For more information about BSF objects, see Replicating BSF Objects on page 89.


The name of the defined replication group that will have BSF objects selected to it. You must define the replication group in iCluster before selecting BSF objects to the replication group.

• PATHThe path that identifies the location of the BSF objects.The path must be enclosed in single quotes and start with a '/' (forward slash) character ('/Dir3/Dir4/file'). The path can be a maximum of 5000 characters, and you must enter at least two characters for the path.Generic path names of the form '/mydir*' are supported, where the generic indicator '*' is the final character of the path name. When using generic path names, all sub-directories are included recursively.For more information about path object specifiers, see Path Object Specifiers on page 53.

• DESCA short description that allows you to identify this path object specifier selection amongst all others that have been defined in iCluster.The description can be a maximum of 50 characters long.

• INCFLGIndicates whether the BSF objects referenced by the path object specifier will be replicated within the replication group.Specify one of the following values:• *INCLUDE—Indicates that the referenced BSF objects will be replicated within the replication group when cluster

operations are started.• *EXCLUDE—Indicates that the referenced BSF objects will not be replicated within the replication group when

cluster operations are started.The default setting for this parameter is *INCLUDE.


Note: BSF include/exclude specifiers cannot have symbolic links to directories as part of their paths. This is only permitted when the symbolic link is at the end of a non-generic specifier that points directly to the symbolic link. iCluster does not support using symbolic links to directories to make BSF specifiers shorter.

• JOURNALIndicates how the BSF objects referenced by the path object specifier will be replicated with the replication group.Specify one of the following values:• *NONE—BSF objects matching this object specifier will not be journaled by iCluster. Only object-level changes to

objects matching this specifier will be replicated. Changes to the contents of the objects matching this specifier will not be replicated unless the entire object is replaced. See Path Object Specifiers on page 53 for more information.

• *GROUP—BSF objects matching this object specifier will use the journal specified at the group level. See the DMADDGRP—Add Group command for more information.

The default setting for this parameter is *NONE.Note: The overlap of journaled and non-journaled path object specifiers is restricted in iCluster. For example, using

both '/home/user/*' (non-journaled) and '/home/user/employees/*' (journaled) is not permitted.Note: Objects within the QDLS directory cannot be journaled. iCluster will enforce the *NONE option for the

JOURNAL parameter of any path specifier matching QDLS.• POLLINT

Indicates the current polling interval for the BSF objects referenced by the path object specifier. Using this parameter, you can indicate whether or not BSF objects will be polled.Specify one of the following values:• *GROUP—Indicates that BSF objects will use the corresponding replication group value that is specified when

adding or changing a replication group. See DMADDGRP—Add Group on page 113 for more information.• *NONE—Indicates that BSF objects will not be polled.

The default setting for this parameter is *GROUP.Note: Note that it is only possible to poll BSF objects in the QDLS folder.

• NEWOBJACTIndicates the method by which replication is to begin for the objects that come into replication scope when an object specifier is added.Specify one of the following values:• *NONE—This value does not change the replication status of new, in-scope objects, and is intended to support

initial group configuration. If this value is selected, mirroring must be started with a refresh of the entire group, or at a user-specified position with the DMSETPOS—Set Journal Start Position, DMMRKPOS—Mark Current Journal Positions, or DMREGPOS—Register Positions commands.

Note: This value is not allowed if replication of the group is active.• *CURRENT—Replication of journal entries for new, in-scope objects will begin at the time the object specifier is

added. Journal entries related to new, in-scope objects created after that time will be replicated. Journal entries related to new, in-scope objects created before that time will not be replicated.

Note: This value is only permitted when the group is active.Note: If this value is selected, no changes should occur on the new, in-scope objects until the OMI0320 event for the

object specifier appears in the event log.• *REFRESH—New in-scope objects will be refreshed one at a time. Replication of journal entries for new, in-scope

objects begins for each object as it is refreshed.Note: This value is only permitted when the group is active.



Note: If this option is selected, all the objects that match the specifier and do not match an exclude specifier for the group will be refreshed. Objects that were already in replication scope, but also match the new object specifier, will be refreshed as part of the DMADDBSF—Add Path Object Specifier to Group command processing.

The default setting for this parameter is *NONE.• CRWNR

Indicates the object-level rules by which iCluster will resolve conflicts for the non-journaled BSF objects selected to a *MSTRMSTR replication group.

Note: See the JOURNAL parameter in this command for more information on how to specify a non-journaled BSF.Specify one of the following values:• *GROUP—Indicates that the object specifier inherits conflict resolution rules and the user exit program configured at

the group-level.Note: If you specify *GROUP for the CRWNR parameter and set the CRWNR parameter at the group level to *TGT,

iCluster will enforce the *NONE conflict resolution rule for all objects matching this specifier.• *SRC—Indicates that the source wins if there is a conflict. Source data is applied to the target, which ensures that

target data matches source data and mirroring continues.• *EXIT—Indicates that you would like to resolve data conflicts through a user exit program. With this option, you must

specify a user exit program and library in the CRUSREXIT parameter in this command.• *NONE—Indicates that you do not want to enable conflict detection and resolution rules. If there is a conflict,


The default setting for this parameter is *GROUP.If you want to resolve conflicts through a user exit program, you must change the logic in your user exit program to use the correct BSF path.


• CRUSREXITThe name of the user exit program that you want to call to resolve data conflicts. A user exit program allows you to define a conflict resolution strategy that is not covered by one of the standard resolution methods in the CRWNR parameter.Specify the name of a user exit program or one of the following values:• *NONE—Indicates that you do not want to specify a user exit program.

The library where the user exit program resides must be identified if you do not specify *NONE. Prefix the name of the user exit program name with the name of the library where the program is located (for example, LIB1/USREXIT1).See Parameter List Descriptions for a Conflict Resolution User Exit Program on page 326 for more information about the values you can pass to the user exit program.You cannot specify *NONE for this parameter if you specify *EXIT for the CRWNR parameter.


ExamplesDMADDBSF GROUP(GRP1) PATH('QDLS/DIR1/DIR2/DIR3/FILEA') DESC('BSF Object') INCFLG(*INCLUDE) JOURNAL(*NONE) POLLINT(*GROUP) NEWOBJACT (*CURRENT)

Selects the BSF object FILEA in /DIR1/DIR2/DIR3 to replication group GRP1.Provides a description to be associated with the object selection.


FILEA will be included for replication within group GRP1 when cluster operations are started.BSF objects matching this object specifier will not be journaled to the default BSF journal for the cluster.The polling interval that was set in the DMADDGRP—Add Group command will be used.Replication of journal entries for new, in-scope objects will begin at the time the object specifier is added.

DMADDBSF GROUP(GRP2) PATH('QDLS/DIR1/DIR2/*') INCFLG(*INCLUDE) JOURNAL(*GROUP) POLLINT(*NONE) NEWOBJACT (*REFRESH)

Selects the generic path name '/DIR1/DIR2/*' to replication group GRP2.This allows iCluster to include all path objects in the directory /DIR1/DIR2 for replication when cluster operations are started.BSF objects matching this object specifier will use the journal specified at the group level.BSF objects will not be polled and therefore do not have content-level mirroring.New in-scope objects will be refreshed one at a time. Replication of journal entries for new, in-scope objects begins for each object as it is refreshed.

UseYou need to issue this command on an active node in the cluster.


Menu AccessF22 (Shift+F10) - Option 18Work With Path Object Specifiers screen - F6

DMRMVBSF—Remove Path Object Specifier to GroupDMRMVBSF GROUP( ) PATH( )

Use this command to de-select BSF objects from the replication group identified through the first parameter. De-selecting BSF objects from a replication group prevents referenced objects from being replicated within the group.For more information about BSF objects, see Replicating BSF Objects on page 89.


The name of the defined replication group that will have BSF objects de-selected from it.• PATH

The path object specifier that identifies the location of the BSF objects. The path object specifier must be currently selected to the replication group specified through the GROUP parameter (see above).The path must be enclosed in single quotes and start with a '/' (forward slash) character ('/Dir3/Dir4/file'). The path can be a maximum of 5000 characters, and you must enter at least two characters for the path.Generic path names of the form '/mydir*' are supported, where the generic indicator '*' is the final character of the path name. When using generic path names, all sub-directories are included recursively.For more information about path object specifiers, see Path Object Specifiers on page 53.

ExamplesDMRMVBSF GROUP(GRP1) PATH('/DIR1/DIR2/DIR3/FILEA')

De-selects the BSF object FILEA in /DIR1/DIR2/DIR3 from replication group GRP1.



DMRMVBSF GROUP(GRP2) PATH('/DIR1/DIR2/*')De-selects the generic path name '/DIR1/DIR2/*' from replication group GRP2.



Menu AccessF22 (Shift+F10) - Option 19Work With Path Object Specifiers screen - Option 4

DMCHGBSF—Change Path Object Specifier to GroupDMCHGBSF GROUP( ) PATH( ) DESC( ) INCFLG( ) JOURNAL( ) POLLINT( ) CRWNR( ) CRUSREXIT( )

Use this command to change specific attributes of a BSF object selection to a replication group.You can change the description currently associated with the object selection (DESC), modify the flag that determines whether the referenced objects are replicated within the group (INCFLG), and change the polling interval (POLLINT).The journaling option for BSF objects matching this specifier can also be changed with this command.For more information about BSF objects, see Replicating BSF Objects on page 89.


The name of the defined replication group that will be affected by the change to the object selection. The BSF objects referenced by the PATH parameter (see below) must be currently selected to the replication group.

• PATHThe path object specifier that identifies the location of the BSF objects that are currently selected to the replication group identified through the GROUP parameter (see above).The path must be enclosed in single quotes and start with a '/' (forward slash) character ('/Dir3/Dir4/file'). The path can be a maximum of 5000 characters, and you must enter at least two characters for the path.Generic path names of the form '/mydir*' are supported, where the generic indicator '*' is the final character of the path name. When using generic path names, all sub-directories are included recursively.For more information about path object specifiers, see Path Object Specifiers on page 53.

• DESCA short description that allows you to identify this path object specifier selection amongst all others that have been defined in iCluster.The description can be a maximum of 50 characters long.Specify a short description or the following value:• *SAME—Uses the description that is currently associated with the object selection to the replication group.

The default setting for this parameter is *SAME.• INCFLG

Indicates whether the BSF objects referenced by the path object specifier will be replicated within the replication group.


Specify one of the following values:• *SAME—Uses the current setting for this parameter.• *INCLUDE—Indicates that the referenced BSF objects will be replicated within the replication group when cluster

operations are started.• *EXCLUDE—Indicates that the referenced BSF objects will not be replicated within the replication group when

cluster operations are started.The default setting for this parameter is *SAME.

• JOURNALIndicates how the BSF objects referenced by the path object specifier will be replicated with the replication group.Specify one of the following values:• *SAME—Uses the current setting for this parameter.• *GROUP—BSF objects matching this object specifier will use the journal specified at the group level. See the

DMADDGRP command for more information.• *NONE—BSF objects matching this object specifier will not be journaled by iCluster. Only object-level changes to

objects matching this specifier will be replicated. Changes to the contents of the objects matching this specifier will not be replicated unless the entire object is replaced. See Path Object Specifiers on page 53 for more information.

The default setting for this parameter is *SAME.Note: The overlap of journaled and non-journaled path object specifiers is restricted in iCluster. For example, using

both '/home/user/*' (non-journaled) and '/home/user/employees/*' (journaled) is not permitted.Note: DLO objects in the QDLS directory will be replicated in real-time using object-level only support (*NONE).

• POLLINTIndicates the current polling interval for the BSF objects referenced by the path object specifier. Using this parameter, you can indicate whether or not BSF objects will be polled.Specify one of the following values:• *SAME—Uses the current setting for this parameter.• *GROUP—Indicates that BSF objects will use the corresponding replication group value that is specified when

adding or changing a replication group. See DMADDGRP—Add Group on page 113 for more information.• *NONE—Indicates that BSF objects will not be polled.

The default setting for this parameter is *SAME.Note: Note that is only possible to poll BSF objects in the QDLS folder.

• CRWNRIndicates the object-level rules by which iCluster will resolve conflicts for the non-journaled BSF objects selected to a *MSTRMSTR replication group.

Note: See the JOURNAL parameter in this command for more information on how to specify a non-journaled BSF.Specify one of the following values:• *SAME—Uses the current setting for this parameter.• *GROUP—Indicates that the object specifier inherits conflict resolution rules and the user exit program configured

at the group-level.Note: If you specify *GROUP for the CRWNR parameter and set the CRWNR parameter at the group level to *TGT,

iCluster will enforce the *NONE conflict resolution rule for all objects matching this specifier.• *SRC—Indicates that the source wins if there is a conflict. Source data is applied to the target, which ensures that

target data matches source data and mirroring continues.



• *EXIT—Indicates that you would like to resolve data conflicts through a user exit program. With this option, you must specify a user exit program and library in the CRUSREXIT parameter in this command.

• *NONE—Indicates that you do not want to enable conflict detection and resolution rules. If there is a conflict, iCluster suspends the file from replication and sends a message to the event log. This is the default behavior for iCluster.

The default setting for this parameter is *SAME.If you want to resolve conflicts through a user exit program, you must change the logic in your user exit program to use the correct BSF path.


• CRUSREXITThe name of the user exit program that you want to call to resolve data conflicts. A user exit program allows you to define a conflict resolution strategy that is not covered by one of the standard resolution methods in the CRWNR parameter.Specify the name of a user exit program or one of the following values:• *SAME—Uses the current setting for this parameter.• *NONE—Indicates that you do not want to specify a user exit program.

The library where the user exit program resides must be identified if you do not specify *NONE. Prefix the name of the user exit program name with the name of the library where the program is located (for example, LIB1/USREXIT1).See Parameter List Descriptions for a Conflict Resolution User Exit Program on page 326 for more information about the values you can pass to the user exit program.The default setting for this parameter is *SAME. You cannot specify *NONE for this parameter if you specify *EXIT for the CRWNR parameter.


ExamplesDMCHGBSF GROUP(GRP1) PATH('/DIR1/DIR2/DIR3/FILEA') DESC('Text File') INCFLG(*SAME) JOURNAL(*SAME)

Changes the description associated with the selection of BSF object FILEA in /DIR1/DIR2/DIR3 to replication group GRP1.The parameter setting that determines whether the BSF object is replicated within the group is not changed from its current value.The JOURNAL parameter uses the settings that were set in the DMADDBSF—Add Path Object Specifier to Group command.

DMCHGBSF GROUP(GRP2) PATH('/DIR1/DIR2/*') DESC(*SAME) INCFLG(*INCLUDE) JOURNAL(*NONE) POLLINT(*SAME) CRWNR(*SRC)

Changes the INCFLG parameter setting associated with the selection of a generic path name '/DIR1/DIR2/*' to replication group GRP2. This allows iCluster to exclude all path objects in the directory /DIR1/DIR2 from replication when cluster operations are started.The description associated with the BSF object selection is not changed. BSF objects matching this object specifier will not be journaled by iCluster.The polling interval that was set in the DMADDGRP—Add Group command will be used.The source will win if there is a conflict.




Menu AccessF22 (Shift+F10) - Option 20Work With Path Object Specifiers screen - Option 2

DMWRKBSF—Work with Path Object SpecifiersDMWRKBSF BY( ) GROUP( )

Use this command to list the path object specifiers that have been defined in iCluster. The BY and GROUP parameters allow you to filter the list by displaying the path object specifiers that have been selected to a specific replication group.For more information about path object specifiers, see Path Object Specifiers on page 53.


The set of path object specifiers that are listed by this command.Specify one of the following values:• *NONE—Lists all path object specifiers defined in iCluster.• *GROUP—Lists the path object specifiers that have been selected to the replication group specified through the

GROUP parameter (see below).The default setting for this parameter is *NONE.

• GROUPThe name of an existing replication group. Path object specifiers selected to the named replication group are listed. This parameter is only applicable when the BY parameter (see above) is set to *GROUP.

ExamplesDMWRKBSF BY(*NONE)

Lists all path object specifiers defined in iCluster.DMWRKBSF BY(*GROUP) GROUP(GRP1)

Lists all path object specifiers selected to the replication group GRP1.





Switchable Device Commands

Switchable Device CommandsThis section contains the following topics:• DMADDSWDEV—Add Switchable Device Entry to Group on page 163• DMDSPASPGP—Display Switchable Device Entry to Group on page 164• DMCHGSWDEV—Change Switchable Device Entry for Group on page 164• DMRMVSWDEV—Remove Switchable Device Entry for Group on page 165

DMADDSWDEV—Add Switchable Device Entry to GroupDMADDSWDEV GROUP( ) SWDEV( ) ONLINE( )

Use this command to add a switchable disk storage device entry to a switchable device replication group.A replication group of type *SWDEV can have a switchable disk storage device associated with it. A switchable disk storage device (called an Independent Auxiliary Storage Pool (IASP) device) controls a disk unit that is assigned to the primary node of a group but can be re-assigned to the new primary node after a switchover or failover of the group.


Specifies the name of the switchable resource group (type *SWDEV) to which you want to add a switchable disk storage device entry.

• SWDEVSpecifies the name of the switchable disk storage device which you want to add to the group. You can associate a maximum of 256 switchable disk storage devices with a single group.

• ONLINEIndicates whether the switchable disk storage device being added to the switchable resource group is varied on or varied off when the group is switched over from one node to another or when it is failed over to another node.Enter one of the possible values:• *YES—Indicates that the switchable disk storage device will be varied on when the group is switched over from one

node to another, or when it is failed over to another node.• *NO—Indicates that the switchable disk storage device will not be varied on when the group is switched over from

one node to another or when it is failed over to another node.

ExampleDMADDSWDEV GROUP(ASP33) SWDEV(IASP1) ONLINE(*YES)

Indicates that the switchable disk storage device IASP1 will be added to group ASP33, and that the IASP1 will be varied on.


Menu AccessWork with Groups screen - Option 25

Minimum Security Level*ADMIN


DMDSPASPGP—Display Switchable Device Entry to GroupDMDSPASPGP GROUP( ) GRPTYPE( ) PRIMNODE( ) BACKUP( ) DESC( ) DEVICES( )

This command displays the parameters of a switchable resource group (type *SWDEV) in the cluster.


Specifies the name of the switchable resource group that you want to display.• GRPTYPE

Indicates the type of group you want to display. The only possible value is: • *SWDEV—Indicates that you want to display a group for switchable devices.

• PRIMNODEThe primary node in the recovery domain for the specified switchable resource group.

• BACKUPLists the backup node in the recovery domain of the specified switchable resource group.

• DESCSpecifies a short description that allows you to identify this switchable resource group amongst all others that have been defined in iCluster.

• DEVICESThe name of a switchable device that is associated with this group. Online at switchoverIndicates whether the switchable device associated with the switchable resource group is varied on or left off when the group is switched over from one node to another or when it is failed over to another node. The possible values are: • *YES—The switchable device that is associated with the group will be varied on when the group is switched over

from one node to another or when it is failed over to another node. • *NO—The switchable device that is associated with the group will be not varied on when the group is switched over

from one node to another or when it is failed over to another node.

ExampleDMDSPASPGP GROUP(ASP33)

Displays the ASP33 group.


DMCHGSWDEV—Change Switchable Device Entry for GroupDMCHGSWDEV GROUP( ) SWDEV( ) ONLINE( )

Use this command to change the parameters of a switchable disk storage device entry for a switchable device replication group.A replication group of type *SWDEV can have a switchable disk storage device associated with it. A switchable disk storage device (called an Independent Auxiliary Storage Pool (IASP) device) controls a disk unit that is assigned to the primary node of a group but can be re-assigned to the new primary node after a switchover or failover of the group.


Switchable Device Commands


Specifies the name of the switchable resource group (type *SWDEV) for which you want to change a switchable disk storage device entry.

• SWDEVSpecifies the name of a switchable disk storage device which you want to change for the group.

• ONLINEIndicates whether the switchable disk storage device being changed for the switchable resource group is varied on or varied off when the group is switched over from one node to another or when it is failed over to another node.Enter one of the following values:• *YES—Specifies that the switchable disk storage device will be varied on when the group is switched over from one

node to another or when it is failed over to another node.• *NO—Specifies that the switchable disk storage device will not be varied on when the group is switched over from

one node to another or when it is failed over to another node.

ExampleDMCHGSWDEV GROUP(ASP33) SWDEV(IASP1) ONLINE(*NO)

Indicates that the switchable disk storage device IASP1 for group ASP33 will be varied off at switchover.



UseYou must issue this command on an active node in the cluster. The nodes in the group must be enabled for switchable resources.

DMRMVSWDEV—Remove Switchable Device Entry for GroupDMRMSWDEV GROUP( ) SWDEV( )

Use this command to remove a switchable disk storage device entry from a switchable device replication group.


Specifies the name of the switchable resource group (type *SWDEV) from which you want to remove a device entry.• SWDEV

Specifies the name of a switchable disk storage device which you want to remove from the group.

ExampleDMRMVSWDEV GROUP(ASP33) SWDEV(IASP1)

Indicates that the switchable disk storage device IASP1 will be removed from group ASP33.



Resilient Application Commands

Resilient Application CommandsThis section contains the following topics:• DMADDAPP—Add Resilient Application on page 167• DMDSPAPPGP—Display Resilient Application Group on page 170• DMUPDAPP—Update Resilient Application on page 172• DMCHGAPP—Change Resilient Application on page 172• DMRMVAPP—Remove Resilient Application on page 175• DMADDBACK—Add Backup Node to Recovery Domain on page 176• DMRMVBACK—Remove Backup Node from Recovery Domain on page 176• DMWRKAPP—Work with Resilient Applications on page 177

DMADDAPP—Add Resilient ApplicationDMADDAPP APPNAME( ) PRODLIB( ) TKOVRIPADR( ) DMNSRC( ) PRIMNODE( ) BACKUPS( ) JRNLOC( ) BSWUSREXIT( ) ASWUSREXIT( ) EXITDATA( ) DESC( )

Use this command to add a new resilient application to iCluster.To identify a resilient application, you must specify the library where the QCSTHAAPPI data area for the application is located on the primary node. This data area is defined by the application vendor and contains settings that are referenced by iCluster to define the resilient application. Another data area (QCSTHAAPPO) within the same library will contain information that confirms the addition of a resilient application through this command.

Input Parameters• APPNAME

The name of the resilient application that you are adding.The name that you specify does not have to be the actual name of the application that is defined as being resilient in iCluster. However, DataMirror recommends that the specified name allows you to identify the application easily. If necessary, use the DESC parameter (see below) to identify the application.• PRODLIB

The name of the library on the primary node that contains the definition of the resilient application as specified by the vendor.This library contains the data area (QCSTHAAPPI) that is provided by the application vendor so that the application can operate in a clustered environment. Within the same library, the QCSTHAAPPO data area will contain information that confirms the addition of a resilient application through this command.The library specified through the PRODLIB parameter (see above) must exist on this node.

• TKOVRIPADRThe takeover IP address that will be associated with the resilient application on all nodes in the recovery domain. When the resilient application is started, the takeover IP address will be activated on the application's primary node.When a failover or switchover occurs, the takeover IP address will be moved to the resilient application's new primary node. Moving the takeover IP address avoids having to modify client configurations with the resilient application after operations have been activated on another node. Client interaction with the resilient application is essentially seamless even in the event of a failover or switchover.


In the absence of a takeover IP address, a different IP address would have to be used for the same application on each node in the recovery domain. After a failover or switchover in this case, a new IP address must be specified on the client to support interaction with the application on the new primary node. Therefore, a takeover IP address simplifies the process of working with the resilient application after a failover or switchover, since no IP address changes are required on the client.The takeover IP address must be specified in dotted quad notation (for example, 125.4.3.55).

Note: This IP address is not the same IP address of the primary or backup node that is specified when adding a node to the cluster. See DMADDNODE—Add Node for more information.

• DMNSRCThe name of an existing resilient application or replication group that you want to use to define the recovery domain for the new resilient application.This parameter allows you to conveniently define a recovery domain for the new resilient application that has the same primary and backup nodes as the recovery domain for an existing resilient application or replication group. Defining two or more recovery domains with the same nodes is useful when you want multiple applications to be resilient across the same nodes.Specify the name of a replication group, resilient application, or the following value:• *LIST—Indicates that you want to explicitly identify the primary and backup nodes in the recovery domain (see

PRIMNODE and BACKUPS below) as opposed to basing this selection on an existing resilient application or replication group.

The default setting for this field is *LIST.• PRIMNODE

The name of a node that will be the primary node in the recovery domain for the resilient application that is being defined.The node you specify must have been added to the cluster, and must be currently active. See DMADDNODE—Add Node on page 99 for more information. This parameter is only applicable if the DMNSRC parameter (see above) is set to *LIST.

Note: Both primary and backup (see BACKUPS below) nodes in a resilient application must be located in the same subnet.

• BACKUPSThe name of a node that will be the backup node in the recovery domain for the resilient application you are defining.At this time, only one backup node can be specified.The node you specify must have been added to the cluster, and must be currently active. See DMADDNODE—Add Node on page 99 for more information. This parameter is only applicable if the DMNSRC parameter (see above) is set to *LIST.

Note: Both primary (see PRIMNODE above) and backup nodes in a resilient application must be located in the same subnet.

• JRNLOCThe location of the database journal where scraping will occur.You can specify the following value:• *LOCAL—Indicates that the database journal(s) on the primary node are scraped during mirroring.• *REMOTE—Indicates that the remote database journal(s) are scraped on the backup node during mirroring.

The default setting for this parameter is *LOCAL.After changing the journal location with this parameter, you must set the position of the local journal to the position of the last applied entry by running the DMSETPOS—Set Journal Start Position command with the JRNPOS(*LASTAPY) parameter.



• BSWUSREXITThe name of the user exit program that you want to call immediately before the role change is performed.The user exit program will be called on both nodes of the resilient application for switchover, but only on the new primary node for a failover.Specify the name of a user exit program or the following value:• *NONE—Indicates that you do not want to specify a user exit program.


• ASWUSREXITThe name of the user exit program that you want to call immediately after the role change is performed. After a role change occurs, the user exit program will be called on both nodes of the resilient application.Specify the name of a user exit program or the following value:• *NONE—Indicates that you do not want to specify a user exit program.


• EXITDATAIdentifies the user-defined data that you want to pass to the user exit programs specified through the BSWUSREXIT and ASWUSREXIT parameters (see above).If you specify two different user exit programs (one program before the role switch, and a different program after the role switch), the same user-defined data is passed to both programs. A maximum of 256 bytes of data can be passed to the user exit programs. If your data contains spaces or non-alphanumeric characters (commas, periods, and so on), you must enclose your data in single quotes.

• DESCA short description that allows you to identify this resilient application among all others that have been defined in iCluster.Use this parameter to identify the application that will be resilient in the recovery domain. The description can be a maximum of 50 characters long.

ExamplesDMADDAPP APPNAME(OEPACK) PRODLIB(LIB1) TKOVRIPADR(125.32.2.4) DMNSRC(*LIST) PRIMNODE(NODE1) BACKUPS(NODE2) JRNLOC(*LOCAL) BSWUSREXIT(LIB1/USREXIT1) ASWUSREXIT(*NONE) EXITDATA(ARJ123908KPJ230982) DESC('Order Entry Application')

Adds the resilient application OEPACK to iCluster.Library LIB1 contains the data areas and files that are provided by the application vendor to define the resilient application.The floating IP address 125.32.2.4 can be used exclusively for communications with the application on the primary node in the recovery domain.The primary and backup nodes in the recovery domain are explicitly specified through the PRIMNODE and BACKUPS parameters.


The node NODE1 in the cluster is the primary node for the resilient application, while NODE2 in the cluster assumes the role of backup node.The database journal(s) on the primary node are scraped during mirroring.The user exit program USREXIT1 in library LIB1 will be called immediately before a role change is performed.No user exit programs will be called immediately after a role change is performed.User-defined data consisting of a sequence of characters will be passed to the user exit programs.The description associated with the resilient application indicates that the application provides support for order entry.

DMADDAPP APPNAME(INVPACK) PRODLIB(LIB1) TKOVRIPADR(125.44.7.1) DMNSRC(OEPACK) JRNLOC(*LOCAL) BSWUSREXIT(*NONE) ASWUSREXIT(LIB1/USREXIT1) EXITDATA(ARJ123908KPJ230982) DESC('Inventory Application')

Adds the resilient application INVPACK to iCluster.Library LIB1 contains the data areas and files that are provided by the application vendor to define the resilient application.The floating IP address 125.44.7.1 can be used exclusively for communications with the application on the primary node in the recovery domain.The primary and backup nodes in the recovery domain for the resilient application are the same nodes that are in the recovery domain for the resilient application OEPACK.The database journal(s) on the primary node are scraped during mirroring.No user exit programs will be called immediately before a role change is performed.The user exit program USREXIT1 in library LIB1 will be called immediately after a role change is performed.User-defined data consisting of a sequence of characters will be passed to the user exit programs.The description associated with the resilient application indicates that the application provides support for inventory.

UseYou must issue this command on an active node in the cluster, and all specified nodes in the recovery domain must be active.


Menu AccessF22 (Shift+F10) - Option 23Work With Resilient Applications screen - Option 6 or F6

DMDSPAPPGP—Display Resilient Application GroupDMDSPAPPGP GROUP( ) GROUPTYPE( ) DESC( ) PRIMNODE( ) BACKUP( ) TKOVRIPADR( ) APPNAME( ) JRNLOC( ) BSWUSREXIT( ) ASWUSREXIT( ) EXITDATA( )

This command displays the parameters of a group that is associated with a resilient application. This command can be used on any active node of the cluster.


The name of a group that is associated with a resilient application.• GROUPTYPE

The type of a group that is associated with a resilient application. The possible values are: • *APPL—Indicates an application resource group



• *REPL—Indicates a replication resource group• DESC

The description that is assigned to the resilient application with which this group is associated. The description can be a maximum of 50 characters long.

• PRIMNODEThe name of the group's primary node.

• BACKUPThe name of the group's first backup node.

• TKOVRIPADRThe takeover IP address for an application group. This field is left blank for replication groups.

• APPNAMEThe name of the resilient application defined in iCluster with which this group is associated.

• JRNLOCThe location of the database journal where scraping will occur. The possible values are: • *LOCAL—Indicates that the database journal(s) on the primary node are scraped during mirroring. *LOCAL is the

default setting for this parameter. • *REMOTE—Indicates that the remote database journal(s) are scraped on the backup node during mirroring.

• BSWUSREXITSpecifies the name of the user exit program that you want to call immediately before a role switch is performed. The possible values are:• *NONE—Indicates that you do not want to specify a user exit program. • user-exit-program-name—Refers to the name of the user exit program to be called.

You must specify the library where the user exit program resides if you do not specify *NONE. Prefix the name of the user exit program name with the name of the library where the program is located (for example, LIB1/USREXIT1).

• ASWUSREXITSpecifies the name of the user exit program that you want to call immediately after a role switch is performed. The possible values are: • *NONE—Indicates that you do not want to specify a user exit program. • user-exit-program-name—Refers to the name of the user exit program to be called.

Yo must specify the library where the user exit program resides if you do not specify *NONE. Prefix the name of the user exit program name with the name of the library where the program is located (for example, LIB1/USREXIT1).

• EXITDATAIdentifies the user-defined data that you want to pass to the user exit programs specified through the BSWUSREXIT and ASWUSREXIT parameters.If you specify two different user exit programs, the same user-defined data is passed to both programs. A maximum of 256 bytes of data can be passed to the user exit programs. If your data contains spaces or non-alphanumeric characters (commas, periods, and so on), you must enclose your data in single quotes.

ExamplesDMDSPAPPGP GROUP(APPGRP1)


Displays informtion for the APPGRP1 group.

DMUPDAPP—Update Resilient ApplicationDMUPDAPP APPNAME( ) PRODLIB( )

Use this command to update a resilient application in iCluster when the data areas and files provided by the application vendor change on the primary node.When the QCSTHAAPPI data area, along with related data areas and files, is updated on the application's primary node, this command must be used to update the corresponding resilient application in iCluster. You can use this command to remove and re-define the application groups currently associated with the resilient application. The recovery domain takeover IP address and the description of the application are not changed by this command.


The name of the resilient application that you are updating.The resilient application that you specify must have been added to iCluster through the DMADDAPP—Add Resilient Application command.

• PRODLIBThe name of the library on the primary node that contains the updated definition of the resilient application as specified by the vendor.This library contains the data areas and files that are provided by the application vendor so that the application can operate in a clustered environment. When these data areas and files are modified, this command must be used to update the corresponding resilient application in iCluster.

ExampleDMUPDAPP APPNAME(OEPACK) PRODLIB(LIB1)Updates the resilient application OEPACK in iCluster.Library LIB1 contains the data areas and files that are provided by the application vendor, but have been changed on the application's primary node.

UseYou must issue this command on an active node in the cluster. The resilient application must not be running when you issue this command.


Menu AccessF22 (Shift+F10) - Option 24Work With Resilient Applications screen - Option 3

DMCHGAPP—Change Resilient ApplicationDMCHGAPP APPNAME( ) TKOVRIPADR( ) BSWUSREXIT( ) ASWUSREXIT( ) EXITDATA( ) JRNLOC( ) DESC( )

Use this command to change the takeover IP address or description for the specified resilient application. You can also change the name of the user exit programs that are called before and after a role change, and the user-defined data that is passed to these programs.



Note that the recovery domain for an existing resilient application cannot be changed through this command. However, you can modify the recovery domain for a resilient application by using the commands for adding or removing backup nodes. See DMADDBACK—Add Backup Node to Recovery Domain on page 135 and DMRMVBACK—Remove Backup Node from Recovery Domain on page 135.


The name of the resilient application that will have its takeover IP address and/or description changed by this command.The resilient application that you specify must have been added to iCluster through the DMADDAPP—Add Resilient Application command.

• TKOVRIPADRThe takeover IP address that will be associated with the resilient application on all nodes in the recovery domain (see note below). When the resilient application is started, the take over IP address will be activated on the application's primary node.When a failover or switchover occurs, the takeover IP address will be moved to the resilient application's new primary node. Moving the takeover IP address avoids having to modify client configurations with the resilient application after operations have been activated on another node. Client interaction with the resilient application is essentially seamless even in the event of a failover or switchover.In the absence of a takeover IP address, a different IP address would have to be used for the same application on each node in the recovery domain. After a failover or switchover in this case, a new IP address must be specified on the client to support interaction with the application on the new primary node. Therefore, a takeover IP address simplifies the process of working with the resilient application after a failover or switchover, since no IP address changes are required on the client.It must be specified in dotted quad notation (for example, 125.4.3.55).Specify the IP address for the application or the following value:• *SAME—Uses the current setting for this parameter.

The default setting for this parameter is *SAME.Note: This IP address is not the same IP address of the primary or backup node that is specified when adding a

node to the cluster. See DMADDNODE—Add Node on page 99 for more information.• BSWUSREXIT

The name of the user exit program that you want to call immediately before the role change is performed.The user exit program will be called on both nodes of the resilient application for switchover, but only on the new primary node for a failover.Specify the name of a user exit program or the following value:• *SAME—Uses the current setting for this parameter.• *NONE—Indicates that you do not want to specify a user exit program.

The default setting for this parameter is *SAME.The library where the user exit program resides must be identified if you do not specify *NONE. Prefix the name of the user exit program name with the name of the library where the program is located (for example, LIB1/USREXIT1).See Passing Arguments to Role Switch User Exit Programs on page 347 for more information about the arguments that can be passed to the user exit program.

• ASWUSREXITThe name of the user exit program that you want to call immediately after the role change is performed. After a role change occurs, the user exit program will be called on both nodes of the resilient application.


Specify the name of a user exit program or the following value:• *SAME—Uses the current setting for this parameter.• *NONE—Indicates that you do not want to specify a user exit program.

The default setting for this parameter is *SAME.The library where the user exit program resides must be identified if you do not specify *NONE. Prefix the name of the user exit program name with the name of the library where the program is located (for example, LIB1/USREXIT1).See Passing Arguments to Role Switch User Exit Programs on page 347 for more information about the arguments that can be passed to the user exit program.

• EXITDATAIdentifies the user-defined data that you want to pass to the user exit programs specified through the BSWUSREXIT and ASWUSREXIT parameters (see above).If you specify two different user exit programs (one program before the role switch, and a different program after the role switch), the same user-defined data is passed to both programs. A maximum of 256 bytes of data can be passed to the user exit programs. If your data contains spaces or non-alphanumeric characters (commas, periods, and so on), you must enclose your data in single quotes.Specify the user-defined data that you want to pass to the user exit programs or the following value:• *SAME—Uses the current setting for this parameter.

The default setting for this parameter is *SAME.• JRNLOC

The location of the database journal where scraping will occur.You can specify the following value:• *SAME—Uses the current setting for this parameter.• *LOCAL—Indicates that the database journal(s) on the primary node are scraped during mirroring.• *REMOTE—Indicates that the remote database journal(s) are scraped on the backup node during mirroring.

The default setting for this parameter is *SAME.After changing the journal location with this parameter, you must set the position of the local journal to the position of the last applied entry by running the DMSETPOS—Set Journal Start Position command with the JRNPOS(*LASTAPY) parameter.

• DESCA short description that allows you to identify this resilient application amongst all others that have been defined in iCluster.Use this parameter to identify the application that will be resilient in the recovery domain. The description can be a maximum of 50 characters long.Specify a short description or the following value:• *SAME—Uses the current setting for this parameter.


ExampleDMCHGAPP APPNAME(OEPACK) TKOVRIPADR(125.32.2.5) JRNLOC(*LOCAL) BSWUSREXIT(LIB1/USREXIT1) ASWUSREXIT(*NONE) EXITDATA(ARJ123908KPJ230982) DESC('Order Entry Application')

Changes the resilient application OEPACK in iCluster.



The floating IP address 125.32.2.5 can be used exclusively for communications with the application on all nodes in the recovery domain.The database journal(s) on the primary node are scraped during mirroring.The user exit program USREXIT1 in library LIB1 will be called immediately before a role change is performed.No user exit programs will be called immediately after a role change is performed.User-defined data consisting of a sequence of characters will be passed to the user exit programs.The description associated with the resilient application indicates that the application provides support for order entry.




DMRMVAPP—Remove Resilient ApplicationDMRMVAPP APPNAME( )

Use this command to remove the specified resilient application in iCluster. This command does not delete the software application itself.

Input Parameter• APPNAME

The name of the resilient application that you are removing.The resilient application that you specify must have been added to iCluster through the DMADDAPP command.

ExampleDMRMVAPP APPNAME(OEPACK)

Removes the resilient application OEPACK in iCluster.





DMADDBACK—Add Backup Node to Recovery DomainDMADDBACK NAME( ) NODE( ) BACKIASP( )

Use this command to add a backup node to the recovery domain for an existing replication group.You can add only a single backup node through this command. The recovery domain for the replication group or resilient application must already have a defined primary node. You can only add a backup node to the recovery domain for inactive replication groups or resilient applications that are not currently running.You must add a backup node to the recovery domain before replication can be started for a resilient application or replication group.


The name of the group or resilient application that will have a backup node added to its recovery domain.• NODE

The name of the node in the cluster that will be added to the recovery domain for the specified group or resilient application.You must specify a node that has been added to the cluster.For an *SWDEV group, the node must have switchable disk storage devices enabled (SWITCHRES parameter). See the DMADDNODE—Add Node on page 99 for more information.

• BACKIASPThe name of the independent auxiliary storage pool (IASP) on the backup node to which you want to replicate.Specify the name of an existing IASP or the following value:• *SYSBAS—Indicates that you are using a system ASP rather than an independent ASP for storing the objects that

will be replicated.The default setting for this parameter is *SYSBAS.

ExampleDMADDBACK NAME(GRP1) NODE(NODE1) BACKIASP(DMC_3)

Adds the backup node NODE1 to replication group GRP1. The name of the IASP on the backup node is DMC_3.

UseYou must issue this command on an active node in the cluster. The specified node must also be active.



DMRMVBACK—Remove Backup Node from Recovery DomainDMRMVBACK NAME( ) NODE( )

Use this command to remove the backup node from the recovery domain for an existing group or resilient application.



Note that you can remove a backup node only using this command. The recovery domain for the group or resilient application must already have defined primary and backup nodes. You can remove backup nodes only from recovery domains for inactive groups or resilient applications that are not currently running.


The name of the group or resilient application that will have a backup node removed from the recovery domain.• NODE

The name of the backup node in the cluster that will be removed from the recovery domain for the specified group or resilient application.

ExampleDMRMVBACK NAME(GRP1) NODE(NODE1)

Removes the backup node NODE1 from group GRP1.




DMWRKAPP—Work with Resilient ApplicationsDMWRKAPP BY( ) NODE( ) GROUP( )

Use this command to display the resilient applications in iCluster. The BY, NODE, and GROUP parameters allow you to filter the list by displaying the resilient applications that contain a specific node in their recovery domains, or the application that is associated with a specific group.In addition, the application status that is currently assigned to the resilient application is displayed, and can be one of the following:• *ACTIVE—Specifies that the cluster statuses of all the groups under the resilient application are active. The groups'

recovery domains are the same.• *INACTIVE—Specifies that the cluster statuses of all the groups under the resilient application are inactive. The groups'

recovery domains are the same.• *INDOUBT—Indicates if the groups' recovery domains are different, or if one of the cluster or replication status is

different from other groups.• *UNKNOWN—Indicates that the status of the resilient application cannot be determined.• *IN_ERROR—Indicates that an internal iCluster error may have occurred. If this state persists, contact DataMirror

Technical Support. See Contacting Technical Support on page 547 for more information.• *NONE—Applications that have the necessary data to be defined as a resilient application in the cluster but are not yet

defined as a resilient application.



The set of resilient applications.Specify one of the following values:• *NONE—Lists all resilient applications in iCluster.• *NODE—Lists the resilient applications that contain the node specified through the NODE parameter (see below) in

their recovery domains.• *GROUP—Identifies the resilient application that is associated with the group specified through the GROUP

parameter (see below). The group can be a resilient application group or a data replication group associated with the resilient application.

The DMWRKAPP—Work with Resilient Applications panel will display the full list of resilient application definitions on the current system, whether or not they have been defined in iCluster as resilient applications.The default setting for this parameter is *NONE.

• NODEThe name of an existing node. Resilient applications that contain the named node in their recovery domains are listed. This parameter is only applicable when the BY parameter (see above) is set to *NODE.• GROUP

The name of an existing replication group. The resilient application associated with the named group is identified. This parameter is only applicable when the BY parameter (see above) is set to *GROUP.

ExamplesDMWRKAPP BY(*NONE)

Lists all resilient applications in iCluster.DMWRKAPP BY(*NODE) NODE(NODE1)

Lists all resilient applications that contain the node NODE1 in their recovery domains.DMWRKAPP BY(*GROUP) GROUP(GRP1)

Identifies the resilient application that is associated with the group GRP1.





MQSeries Commands

MQSeries CommandsThis section contains the following topics:• RBDHAMQM—Rebuild iCluster MQSeries on page 179

RBDHAMQM—Rebuild iCluster MQSeriesRBDHAMQM MQSRLS( ) QMNAME( )Use this command on the backup node that is being turned into a primary node to rebuild the WebSphere MQ environment after a switchover, and re-creates all WebSphere MQ objects from the information stored in the WebSphere MQ journal and mirrored WebSphere MQ BSF files.The operations performed by this command may take a long period of time (over 20 minutes). The length of time depends on the number of WebSphere MQ objects originally present on the primary node, and the number of messages stored on the WebSphere MQ queues if a refresh before mirror was done on the primary node. An increase in the number of operations (for example, creation, deletion, attribute change, message addition, and removal) performed on the WebSphere MQ objects while mirroring was active will increase the length of time required for both mirroring and refresh before mirroring.Note: If you issue RBDHAMQM for a particular queue manager on the backup node for backup purposes or other

reasons, the group that mirrors this queue manager must always be restarted with a refresh. This will ensure data integrity on the backup node the next time the WebSphere MQ queue manager starts.

Input Parameters• MQSRLS

The product version of WebSphere MQ that you are using. WebSphere MQ versions V5R2M0, V5R3M0, and V6R0M0 are supported.The default setting for this parameter is V5R3M0.

• QMNAMEThe name of the WebSphere MQ queue manager that must be rebuilt. Multiple queue managers are not supported. If you want to mirror multiple queue managers, use the DMADDGRP—Add Group on page 113 command to define a group for each of them.Enter specific names. Do not use *DFT or generic names.

OutputRelevant operating system messages to the job log.The output can be long and can contain warnings if WebSphere MQ attempts to rebuild many objects.

ExamplesRBDHAMQM MQSRLS(V5R3M0) QMNAME(QM1)

This command rebuilds the WebSphere MQ environment on a machine running WebSphere MQ version V5R3M0. QM1 is the name of the queue manager being rebuilt and activated.

UseYou can issue this command on the current backup node to rebuild and activate a WebSphere MQ queue manager that has been successfully replicated after ending your groups. You should only use this command on the new primary node after a switchover or failover, and the WebSphere MQ queue manager has not been activated.


Administration Commands

Administration CommandsThis section contains the following topics:• DMSETSVAL—Set Cluster System Values on page 181• DMCHGTIME—Change System Date and Time on page 193• DMDSPLOG—Display Cluster Event Log on page 195• DMCLRLOG—Clear Cluster Event Log on page 199• HAPNGTCP—Ping Using TCP on page 200• JRNHADADQ—Journal Data Areas and Data Queues on page 201• STRHATCP—Start TCP/IP Listener on page 202• WRKHAJOB—Work with Active Cluster Jobs on page 203• DMSTRDM—Start Definition Manager on page 203• DMDLTCLSTR—Delete Cluster on page 203• DMSYSINF—System Information on page 204• DMWRKSSPLF—Work with Suspended Spooled Files on page 205• DMACTSPLF—Activate Spooled File on page 205

DMSETSVAL—Set Cluster System ValuesDMSETSVAL OPER( ) ACT( ) OBJ( ) SPLF( ) PF( ) BSF( ) EVNTLOG( ) LATENCY( ) CLUSTER( ) PERFORM( )

Use this command to set certain cluster-wide values that affect different aspects of the clustered environment you have defined through iCluster.These cluster system values allow you to control different behaviors of your cluster. In addition, some of these values are used when corresponding parameters in other commands are set to *CLUSTER.Each cluster system value that can be set through this command belongs to one of the following categories:• Operational (OPER)—Cluster system values that affect operations within the cluster.• Automatic Reactivation (ACT)—Cluster system values that affect the automatic reactivation of suspended objects.• Object (OBJ)—Cluster system values that pertain to objects replicated within the cluster.• Spool File (SPLF)—Cluster system values that affect the replication of spool files within the cluster.• Physical File (PF)—Cluster system values that affect the mirroring of physical files within the cluster.• Byte Stream File (BSF)—Cluster system values that affect the replication of Byte Stream File (BSF) objects within the

cluster.• Event Log (EVNTLOG)—Cluster system values that affect the information displayed through the event log.• Latency (LATENCY)—Cluster system values that affect the latency threshold settings within the cluster.• Cluster Values (CLUSTER)—Controls whether or not iCluster uses IBM Cluster Resource Services as its failover

mechanism.• Performance (PERFORM)—Controls the performance of iCluster in certain scenarios.

Unless otherwise stated, you must restart your groups for the changes made with this command to take effect.

Input Parameters• OPER


Specifies settings for operational parameters.Default Polling IntervalThe time interval (expressed in HHMMSS format) that determines how often iCluster should check for content changes to user spaces.You set the interval by specifying the number of hours (first two digits), minutes (middle two digits) and the seconds (last two digits) between consecutive polls. The polling interval only applies to user spaces (*USRSPC).Specify a polling interval from 000010 to 235959, or one of the following values:• *SAME—Uses the current setting for this cluster system value.• *NONE—Specifies that user spaces should not be polled.

The default setting for this parameter is 5 minutes (000500).Communications TimeoutThe waiting period that has to expire before a communications failure can be declared by iCluster.During the waiting period specified through this parameter, iCluster will attempt to establish any communications between the nodes. If communications cannot be established during this time, a communications failure is reported.Specify a waiting period from 000015 to 235959, or the following value:• *SAME—Uses the current setting for this cluster system value.

The default setting for this parameter is 1 hour (010000).Object CompressionIndicates whether you want to compress objects in save files that are generated by iCluster and replicated to backup nodes.Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *NO—Does not compress all objects.• *YES—Compresses all objects in save files before replication.

The default setting for this parameter is *NO.Compressing objects consumes CPU cycles, but it may lead to faster transfer times of the save files. If objects are compressed on the primary node, iCluster automatically de-compresses the objects when they are received on the backup node.Save ActiveIndicates whether an object can be updated and saved at the same time it is being transferred to the backup node.Specify one of the following values:• *SAME—Uses the current setting in the DMADDGRP—Add Group command for this cluster system value.• *NO—Does not allow objects that are in use to be saved and updated. Objects cannot be updated while being

saved.• *YES—Allows objects that are in use by another job to be saved and updated. Objects may not be in a consistent

state in relationship to each other.• *SYNC—Allows objects that are in use by another job to be saved and updated. All of the objects are saved in a

consistent state in relationship to each other.The default setting for this parameter is *NO.Save/Restore Operation Output



Indicates whether restore operations will create a spool file that identifies which objects were restored. This parameter only applies to refresh, since mirroring does not generate spool files. You can use this spool file to indicate which objects need to be refreshed.Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *ERROR—Generates a spool file for restore operations that produce an error (all other restore and save operations

will not generate a spool file).• *FULL—Generates a spool file to identify restored and not restored objects.• *NONE—Does not generate a spool file for restore operations.

The default setting for this parameter is *ERROR.Default Job DescriptionThe name of the job description that you want to designate as the default iCluster job description for replication jobs.Specify the name of a job description or the following value:• *SAME—Uses the current setting for this cluster system value.

The library where the job description resides must be identified if you do not specify *SAME. Prefix the job description with the name of the library where the job description is located (for example, LIB2/SJD), or one of the following values:• *PRODLIB—Specifies your iCluster installation library.• *LIBL—Specifies the set of libraries in your library list (the libraries are searched in order for the first occurrence of

the specified default job description).The default setting for this parameter is *PRODLIB/CSJOBD.Group Job Start DelayThe number of seconds (from 1 to 60) of delay between the start of replication processes when cluster operations are initiated for groups.This delay is only used when multiple replication processes are started through a single command invocation. It is intended for working environments that consist of relatively small systems. This setting can be used to distribute the start of replication processes over a period of time so as to avoid high peaks in CPU usage that would typically occur if these processes are started at the same time.Specify the number of seconds (between 1 and 60, inclusively), or the following value:• *SAME—Uses the current setting for this cluster system value.

The default setting for this parameter is two seconds.• ACT

Specifies settings for the automatic reactivation of suspended objects. The following parameters combine to define the automatic reactivation settings.For more information about automatic reactivation, see Suspended Objects on page 72.Automatic reactivationSpecifies whether iCluster should try to reactivate suspended objects.Enter one of the following values:• *SAME—Keeps the present setting for this parameter.• *YES—Specifies that iCluster should try to reactivate suspended objects. This is the default value for this

parameter.• *NO—Specifies that iCluster should not reactivate suspended objects.


Reactivation intervalThe number of minutes between automatic retries.Valid entries range between 1 and 1440 (one day).Enter the number of minutes between intervals or the following value:• *SAME—Keeps the present setting for this parameter.

The product default is 10 minutes.Max. reactivation attemptsThe number of automatic retries before halting activation. Valid entries range between 1 and 32767.Enter the maximum number of retries or one of the following values:• *SAME—Keeps the present setting for this parameter.• *NOMAX—Never gives up on automatic reactivation. This is the default value for this parameter.

Max. reactivation sizeThe largest object size (in bytes) to be included in the reactivation. The size of an object can affect network performance and introduce mirroring latency.Enter the maximum size or one of the following values:• *SAME—Keeps the present setting for this parameter.• *NOMAX—Includes objects of all sizes in the reactivation. This value could have serious performance issues on

the primary node and network bandwidth issues if very large objects are suspended frequently. This is the default value for this parameter.

*NOMAX is the default value for this parameter.• OBJ

Specifies settings for object parameters.Lock Files on Backup NodesIndicates whether journaled files should be locked on the backup node. When locked, no other users can modify these files on the backup node.Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *NO—Indicates that files will not be locked when the group is active, meaning that users can modify these files on

the backup node.• *YES—Indicates that files cannot be modified on the backup node when the group is active.

The default setting for this parameter is *NO.Maximum Refresh SizeIndicates the size of the largest physical file (in kilobytes) that can be refreshed automatically over the network. The size of an object can affect network performance and introduce mirroring latency.If a physical file is too large to be refreshed, it will be marked as suspended and will not be refreshed by iCluster. In this case, you are responsible for ensuring the physical file is refreshed to the backup node by some other means (tape, for example) and activated for replication.

Note: This parameter applies to the automatic reactivation of database files, refresh of objects, and the activation of objects. See the DMACTOBJ—Activate Object command.



Journal entries added after activating the suspended physical file will be sent to the backup node. Journal entries added before the file is activated will not be sent to the backup node. As a result, you will need to make sure that the file was not changed between the time it was saved on the primary node and the time it was activated.Specify a maximum refresh size or one of the following values:• *SAME—Uses the current setting for this cluster system value.• *NOMAX—Indicates that there is no maximum refresh size.

The default setting for this parameter is *NOMAX.Changes made to the maximum refresh size take effect immediately.Hold Configuration Object Source on Backup NodeIndicates whether you want iCluster to create configuration objects automatically and immediately after they are received on a backup node or hold the commands for creating them in specific physical source files so that they can be created at a later time.Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *YES—Holds commands to create configuration objects in specific physical source files so that they can be created

at a later time.• *NO—Automatically creates configuration objects as soon as they are received on the backup node.

The default setting for this parameter is *YES.If configuration objects that are being replicated already exist on backup nodes, this value should be set to *YES to prevent iCluster from trying to create objects when they are in use.For more information about configuration objects, see Replicating Configuration Objects on page 90.Replicated User Profile StatusThe status that you want to assign to user profiles that have been replicated to a backup node.Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *DISABLED—Indicates that all user profiles will be set to a status of *DISABLED.• *PRIMARY—Indicates that the user profile status will be set to the same status that is currently assigned to the

corresponding user profile on the primary node.• *NOCHG—Indicates that the current status of each user profile will not be changed by iCluster and the user profile

status on the backup node is set to *DISABLED by default.The default setting for this parameter is *DISABLED.User Profile Replication LevelIndicates whether or not dependent user profiles should be replicated when a main user profile is replicated.Dependent user profiles include object owner profiles, group profiles, supplemental profiles, and profiles on an authority list associated with the replicated profile.If you replicate dependent user profiles, then the relationship between user profiles is preserved so that there is an exact mirroring of user profiles. However, replicating all dependent user profiles may affect performance if the number of dependent user profiles is considerable.Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *FULL—Indicates that all dependent profiles are replicated. This includes group and supplemental profiles, profile

owners as well as dependent profiles that are dependent on dependent profiles, and so on.


• *BASIC—Indicates that only profiles that match the selected object specifier will be replicated. During replication, iCluster assumes that all dependent profiles already exist on the backup node. Error messages will be generated if the dependent profiles are not on the backup node, though replication will continue.

The default setting for this parameter is *FULL.Replicate User Profiles for Authorization ListsIndicates whether you want to replicate authorization lists with or without the user profiles.An authorization list object consists of a list of user profiles. This parameter dictates whether authorization lists are replicated with or without these user profiles. If user profiles are replicated, it is important to recognize which user profiles will be sent to the backup node, as you may want to restrict access to this node. On the other hand, if a user profile in an authorization list does not exist on the backup node, the replicated authorization list will not be the same as the authorization list on the primary node if user profiles are not replicated.Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *NO—Does not replicate user profiles with the authorization lists.• *YES—Replicates user profiles with the authorization lists.

The default setting for this parameter is *NO.Replicate Q* ProfilesIndicates whether you want to replicate user profiles that start with the letter Q.Usually, all profiles that start with the letter Q are system-owned user profiles (for example, QSECOFR) and should not be replicated from one node to another. However, in case you have defined user profiles that start with the letter Q that are not system-owned profiles, setting this parameter to *YES will allow you to replicate those profiles. Profiles that are owned by QSYS will not be replicated even when this parameter is set to *YES.Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *NO—Does not replicate any user profiles that start with the letter Q.• *YES—Replicates non-system-owned profiles that start with the letter Q.

The default setting for this parameter is *NO.• SPLF

Specifies settings for spool files.Maximum Wait for Spool FilesThe time period (expressed in HHMMSS format) you want iCluster to wait before abandoning the replication of a spool file.A spool file cannot be replicated unless it has a status of *READY. Therefore, if iCluster is up-to-date with the audit journal, it may start processing a spool file before it is ready. iCluster will only wait the amount of time set in this parameter for the spool file to have a status of *READY. If the spool file is not ready after waiting the time specified in this parameter, iCluster will abandon the replication of this spool file and issue a message in the event log.Specify a waiting period from 000001 to 235959, or the following value:• *SAME—Uses the current setting for this cluster system value.

The default setting for this parameter is 15 seconds (000015).Maximum Wait for Spool File ActivityThe time period (expressed in HHMMSS format) you want iCluster to wait before abandoning the replication of a spool file if no data is being written to it.



In some cases, a job may be writing to a spool file. However, the data is first placed in a temporary buffer and only written to the spool file upon the completion of the job. In such a case, the spool file will be open and have zero pages until the job ends.This parameter allows you to specify the time to wait for changes to occur to an open spool file. If no additional data is written to an open spool file (that is, it is open and has zero pages) within the time specified by this parameter, iCluster will abandon the replication of this spool file and issue a message. This parameter provides added flexibility as the user can specify a shorter interval over which only spool file activity is monitored. As a result, an open spool file that has no data being written to it will be abandoned quickly, as opposed to waiting the full time specified in the MAXSPLWAIT parameter (see above).Specify a waiting period from 000001 to 235959, or the following value:• *SAME—Uses the current setting for this cluster system value.

The default setting for this parameter is 5 seconds (000005).Replicate *OUTQ ContentsIndicates whether you want to mirror the contents of spool files. If you specify *NO, then iCluster will only replicate *OUTQ objects but will not replicate the spool files in them.Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *YES—Indicates that the spool files will be created and the contents will be mirrored.• *NO—Indicates that the spool files will be created, but the contents will not be mirrored.

The default setting for this parameter is *YES.• PF

Specifies settings for replicating physical files.Commitment Control LevelThe level of commitment control you want to use when replicating *FILE objects.Commitment control stages complete database transactions so that they are assembled before being applied, or additionally, opened in a commitment control environment to ensure backup database consistency. For more information, see Commitment Control on page 71.Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *NONE—Indicates that the update process on the backup node will not perform commitment control staging.• *LEVEL1—Indicates that all updates that comprise a transaction are assembled before being applied on the backup

node.• *LEVEL2—Indicates that all updates in a transaction are applied in a commitment control environment to ensure

backup database consistency. This option provides true commitment control.The default setting for this parameter is *NONE.Note that if you select *LEVEL2 but a file on the backup node cannot be opened under commitment control, *LEVEL1 commitment control will be used for that file.Journal ImagesIndicates whether default journaling should include both before and after images.If a file is replicated using relative record number, it may be journaled with after images only.If you are using commitment control on the primary database or unique key update method, both before and after images must be journaled.


Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *AFTER—Indicates that you want to journal the after image only.• *BOTH—Indicates that you want to journal both the before and after images.

The default setting for this parameter is *AFTER.Journal Objects on BackupIndicates whether replicated physical files, data areas, and data queues will be journaled on backup nodes.Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *NO—Indicates that the replicated physical files, data areas, and data queues will not be journaled on backup

nodes.• *YES—Indicates that the replicated physical files, data areas, and data queues will be journaled on backup nodes.

The default setting for this parameter is *NO.Default Database JournalThe name of the database journal that you want to use as your default.The journal that you specify will be used for files that are to be mirrored but are not yet journaled.Specify the name of the database journal or the following value:• *SAME—Uses the current setting for this cluster system value.

The library where the journal resides must be identified if you do not specify *SAME. Prefix the journal with the name of the library where the journal is located (for example, LIB1/JRN1), or one of the following values:

• *PRODLIB—Specifies your iCluster installation library.• *LIBL—Specifies the set of libraries in your library list (the libraries are searched in order for the first occurrence of

the specified default job description).The default setting for this parameter is *PRODLIB/HADJRN.Delete Dependent Non-Group Logical FilesIndicates whether you want to delete logical files that are dependent on physical files on the backup node during a refresh of the associated physical file. Note that the logical files are not necessarily the ones you have selected to a group.Physical files that have dependent logical files can only be deleted if the dependent logical files are deleted first.

Note: If some of the logical files do not belong to the same group as the physical files, you can only delete the logical files by setting this system parameter to *YES.

Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *NO—Indicates that you do not want to delete backup logical files that are dependent on physical files. You cannot

delete the physical files until you have first deleted the dependent logical files.• *YES—Indicates that you want to delete backup logical files that are dependent on physical files in the event that

the physical files are deleted.The default setting for this parameter is *NO.Maximum Record-Level ErrorsThe maximum number of record-level errors that are allowed to occur for a particular file before replication of the physical file is automatically stopped by iCluster and the file becomes suspended.



Record-level errors occur when updating, inserting, or deleting records in a file during replication to a backup node.Specify the maximum number of errors or one of the following values:• *SAME—Uses the current setting for this cluster system value.• *NOMAX—Specifies that replication of physical files will continue regardless of the number of record-level errors

that are generated.The default setting for this parameter is one error.

• BSFSpecifies settings for replicating Byte Stream File (BSF) objects. See Replicating BSF Objects on page 89 for more information.Default BSF JournalThe name of the journal that you want to use as the default for your BSF objects.Note that all BSF objects replicated within a group must be journaled to the same journal.Specify the name of the journal or one of the following values:• *SAME—Uses the current setting for this cluster system value.• HABSFJRN—Uses the default BSF journal supplied with iCluster.

The library where the journal resides must be identified if you do not specify *SAME. Prefix the journal name with the name of the library where the journal is located (for example, LIB2/JRN), or one of the following values:• *PRODLIB—Specifies your iCluster installation library.• *LIBL—Specifies the set of libraries in your library list. The libraries are searched in order for the first occurrence of

the specified default BSF journal.The default setting for this parameter is *PRODLIB/HABSFJRN.

• EVNTLOGSpecifies settings for the cluster event log. For more information about the cluster event log, see Monitoring Replication on page 76.Default Event Message QueueThe name of the message queue that will hold event messages generated by iCluster.If a message queue is identified, iCluster event messages will be placed in the message queue and the event log.Specify the name of the event message queue or one of the following values:• *NONE—Indicates that no message queue is specified. Event messages will be placed in the event log only.• HAMSGQ—Refers to the name of the event message queue that is supplied with iCluster.

The library where the event message queue resides must be identified if you do not specify *NONE. Prefix the message queue name with the name of the library where the message queue is located (for example, LIB2/MSGQ1), or one of the following values:• *PRODLIB—Specifies your iCluster installation library.• *LIBL—Specifies the set of libraries in your library list. The libraries are searched in order for the first occurrence of

the specified default event message queue.The product default for this parameter is *NONE.Remote Date/Time Log DisplayIndicates which date and time settings should be used to generate event messages that originate from a remote node. This value is relevant if local and remote nodes are located in different time zones.Specify one of the following values:


• *SAME—Uses the current setting for this cluster system value.• *LOCAL—Displays dates and times in the time zone where the local node is located.• *REMOTE—Displays dates and times in the time zone where the remote node is located.

The default setting for this parameter is *LOCAL.Logged Message LifetimeThe maximum length of time (in days) that messages in the event log should be kept before being removed. If the age of a specific message exceeds the time limit, this message is automatically removed when the user issues the DMDSPLOG—Display Cluster Event Log command.Specify a time limit or one of the following values:• *SAME—Uses the current setting for this cluster system value.• *NOMAX—Disables automatic removal of event messages. This means that messages will stay in the log until you

remove them. The default setting for this parameter is 30 days.Message Generation LevelsIndicates the levels of messages that will be generated by iCluster. By default, iCluster generates only those messages that have a severity of 20, 30 or 40. Messages in all levels can be generated by specifying *ALL.Specify a message level or one of the following values:• *SAME—Uses the current setting for this cluster system value.• 00—Generates information messages (Level 00) (for example, Savefile created…).• 10—Generates non-critical status messages (Level 10).• 20—Generates stop/start messages and warning (Level 20) (for example, Mirroring started…).• 30—Generates messages that report recoverable errors (Level 30).• *ALL—Generates messages in all levels.

The default settings for this parameter are 20 and 30.When specific message levels are identified, messages in other levels are not generated by iCluster. However, all fatal error messages (Level 40) are always generated.Note that more than one message level can be specified by using *ALL or by separating each message level in the parameter with a space. For example, MSGLVLS (10 20) specifies that only status and warning messages should be generated.

• LATENCYLatency in iCluster is the time difference between what is on the primary node and what is on the backup node.There are two types of latency in iCluster:• Source receive latency: The time difference between the last journal entry in the primary (source) node and the

last journal entry received and put into the staging store on the backup node.• Target apply latency: The time difference between the last journal entry in the staging store and the last journal

entry applied on the backup node.Total latency is the sum of source receive latency and target apply latency.iCluster checks latency and issues a warning message if latency is exceeded. The LATENCY system values let you configure iCluster's latency threshold settings.

Note: iCluster will issue a warning message only after the latency threshold has been exceeded for one minute.



Latency check interval (min)Specifies how often iCluster checks for latencies (source receive and target apply) and compares them with their corresponding thresholds.Enter a value ranging between 1 - 1440 (one day), or the following value:• *SAME—Keeps the present setting for this parameter.

The default value is 5 (minutes).Source Receive threshold (min)This threshold value indicates the amount of source receive latency that is tolerated before a latency warning message is issued. Source receive latency indicates the difference between the timestamp of the journal entry last processed by the backup receiver for the journal, and the timestamp of the last journal entry deposited in the journal on the primary (source) node.For idle or lightly used journals, iCluster determines the latency by putting an entry into a journal if it has been idle for one minute since the last journal entry.Enter a value ranging between 1 and 10080 (seven days), or the following value:• *SAME—Keeps the present setting for this parameter.

The default value is 60 (minutes).Target Apply threshold (min)This threshold value indicates the amount of apply latency that can be tolerated before a latency warning message is issued. The backup apply latency is the difference in the timestamps of the last entry received by the journal's receive process and the last entry applied by the journal's apply process.Enter a value ranging between 1 and 10080 (seven days) or the following value:• *SAME—Keeps the present setting for this parameter.

The default value is 240 (minutes).• CLUSTER

This setting allows you to specify which failover mechanism you want the cluster to use to detect node failures. See Choosing a Failover Mechanism on page 73 for more information about this parameter before changing its value.Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *NO—Indicates that you want to use SwitchOver System.• *YES—Indicates that you want to use IBM Cluster Resource Services.

The default setting for this parameter is *NO.• PERFORM

Specifies the cluster system values that affect the performance of iCluster in certain scenarios.Comms channelThis setting allows you to specify whether a separate communications channel is used for each group or one channel for all groups.Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *SHARED—Uses one communications channel for all groups.


• *PERGROUP—Each group has its own communications channel. With separate communications channels for each group, better line utilization can be obtained and problems in one group does not affect other groups. However, this requires more communications resources.

The default setting for this parameter is *SHARED.Staging store block managementThis setting allows you to control how iCluster manages used staging store blocks. iCluster normally re-allocates blocks to the memory pool after they are used, but now you can choose to not re-allocate the staging store blocks for a performance improvement in situations where you are using multiple apply jobs with large data volumes.Specify one of the following values:• *SAME—Uses the current setting for this cluster system value.• *SHARED—Staging store blocks are re-allocated to the memory pool after they are used. This is the default

behavior of iCluster.• *PERGROUP—Staging store blocks are not re-allocated to the memory pool after they are used. This can result in

a performance improvement when you are using multiple apply jobs.The default setting for this parameter is *SHARED.

ExampleDMSETSVAL OPER(003000 000020 NO NO FULL LIB1/DEFJD 5) ACT(*NO) OBJ(*NO *NOMAX *NO *DISABLED *FULL *YES *YES) SPLF(000100 000015 *YES) PF(*LEVEL2 *AFTER *YES *PRODLIB/HADJRN *NO 5) BSF(LIB1/BSFJRN) EVNTLOG(LIB1/MSGQ1 *LOCAL 120 00 20) LATENCY(5 60 240) PERFORM(*SHARED *SHARED)

OPERThe default polling interval is 30 minutes.The waiting period before a communications failure is declared is 20 seconds.Object compression will turned off.Objects are not saved when they are in use.A spool file is generated during all save operations to record what was saved by iCluster.The default job description is DEFJD in library LIB1.A five second delay between the start of replication processes will occur when cluster operations are initiated for groups.ACTAutomatic reactivation will not be enabled for suspended objects.OBJObjects will not be locked on backup nodes.There is no maximum refresh size for physical files.Configuration objects replicated to backup nodes will not be held in files for creation at a later time. iCluster will automatically create these objects immediately after they are received from a primary node.All replicated user profiles will be set to a status of *DISABLED.All dependent user profiles will be replicated.User profiles will be replicated with authorization lists.User profiles (except those owned by QSYS) starting with the letter Q will be replicated.SPLFiCluster will wait a maximum of one minute for a spool file to have a status of *READY before abandoning the replication of a spool file.



iCluster will wait 15 seconds for changes to occur to an open spool file before abandoning the replication of a spool file.Spool file contents will be mirrored.PF*LEVEL2 commitment control will be applied to replicated files.Only after images will be journaled.Physical files replicated to backup nodes will be journaled.Physical files will be journaled to the journal supplied with iCluster, HADJRN, which is located in the library where iCluster was installed.Dependent logical files will not be deleted when refreshing a physical file.After five record-level errors are produced, iCluster will suspend the file.BSFBSF objects will be journaled to the journal BSFJRN, which is located in library LIB1.EVNTLOGIn addition to the event log, iCluster messages will also be placed in the message queue MSGQ1 in library LIB1.Dates and times will be displayed in the time zone where the local node is located.All messages that have been in the cluster event log for more than 120 days will be automatically removed when the DMDSPLOG—Display Cluster Event Log command is issued.iCluster will only generate information (Level 00), warning (Level 20 and higher severity), and fatal error (Level 40) messages.LATENCYiCluster checks the latencies (source receive and target apply) and compares them to their respective thresholds every 5 minutes. The primary receive threshold is 60 minutes, and the backup apply threshold is 240 minutes.

Note: It is important that at least one space separates each item contained in the OPER, ACT, OBJ, SPLF, PF, BSF, and EVNTLOG parameters. Follow the example that is provided to specify these parameters correctly.

PERFORMOne communications channel is used for all groups.Staging store blocks are re-allocated to the memory pool after they are used.

UseYou can issue this command on any active node in the cluster. You can also issue it on an inactive node, as long as the cluster does not exist and that node is the first node that will be defined in the cluster (master node).



DMCHGTIME—Change System Date and TimeDMCHGTIME CHGTYPE( ) SYSVAL( ) VALUE( ) SECONDS( ) MINUTES( ) HOURS( )

Use this command to change the system clock's date and time. Since journal entries are processed based on their timestamp, changing the system clock can affect the order in which journal entries are processed. The DMCHGTIME command helps to ensure that journal entries are applied in the correct sequence after the system time changes backwards.


You must use DMCHGTIME instead of your operating system's time adjustment commands if the following conditions are both true:

• The operating system is OS/400 V5R2 or earlier.• The time change is less than half an hour in the past. For example, you are changing the time from 2:00 AM to 1:45

AM.If these conditions do not apply to your computer, then DataMirror recommends using your operating system's commands to change the system time. iCluster automatically detects and handles time changes in these situations and running this command is optional.You can change only one value at a time. To change both the system date and time, you must issue this command twice.If iCluster is currently scraping a large number of journals, then this command can take a long time to run. To reduce the processing time for this command, run DMCHGTIME when system activity is low.To use this command, you must be signed on as QPGMR, QSYSOPR, QSRV, or have *ALLOBJ authority.

Input Parameters• CHGTYPE

Specifies the method of changing the date and/or time system values.Enter one of the following values:• *SYSVAL—Indicates that the system date or time will be set explicitly using SYSVAL and VALUE parameters.• *FORWARD—Indicates that the system time will be changed by moving the system clock forward by a specified

number of hours, minutes, and/or seconds entered as separate parameters.• *BACKWARD—Indicates that the system time will be changed by moving the system clock backward by a specified

number of hours, minutes, and/or seconds entered as separate parameters.• SYSVAL

Specifies the name of the system value that you want to change. You can change either the system date or the system time.Note that this parameter is valid only when you specify *SYSVAL for the CHGTYPE parameter.Note that the new value that you enter must meet the format for the system value you specify that you want to change.Enter one of the following values:• QDATE—The system date. Changes take effect immediately. • QTIME—The system time. Changes take effect immediately.

• VALUESpecifies the new value for the system parameter (either the system date or time) that you want to change. The value that you enter must meet the format for the system parameter that you are changing.Note that this parameter is valid only when you specify *SYSVAL for the CHGTYPE parameter.

• SECONDSThe number of seconds that the system time is incremented or decremented if either *FORWARD or *BACKWARD respectively is selected for CHGTYPE parameter.Valid entries range from 0 - 59 seconds.

• MINUTESThe number of minutes that the system time is incremented or decremented if either *FORWARD or *BACKWARD respectively is selected for CHGTYPE parameter.Valid entries range from 0 - 59 minutes.



• HOURSThe number of hours that the system time is incremented or decremented if either *FORWARD or *BACKWARD respectively is selected for CHGTYPE parameter.Valid entries range from 0 - 23 hours.

OutputDepending on which system value you modified, either the system date or time will be modified.

ExamplesDMCHGTIME CHGTYPE(*SYSVAL) SYSVAL(QDATE) VALUE('111905')

Assuming the system date is in the format "mmddyy", this command changes the job date format to November 19, 2005.DMCHGTIME CHGTYPE(*BACKWARD) HOURS(1)

Changes the system time backward by one hour on any node in iCluster.

Minimum Security LevelYou must be signed on as QPGMR, QSYSOPR, or QSRV, or else have *ALLOBJ authority.


DMDSPLOG—Display Cluster Event LogDMDSPLOG EVNTTYPE( ) REPLEVNT( ) COMMEVNT( ) CLUSEVNT( ) STRDATE( ) STRTIME( ) ENDDATE( ) ENDTIME( ) MSGID( ) OUTPUT( ) DETAIL( ) MSGLVLS( )

Use this command to display event messages generated by iCluster during cluster operations. You can filter messages by replication, communication, and cluster type events. You can also filter event messages by time and date, message ID, level of detail, and message level.For more information about event logs, see Monitoring Replication on page 76.

Input Parameters• EVNTTYPE

The type of event messages that you want to display.Specify one of the following values:• *REPL—Displays all event messages that provide information about replication.• *ALL—Displays all types of event messages.• *COMM—Displays all event messages that provide information about iCluster communications between nodes.• *CLUSTER—Displays all event messages that provide information concerning cluster nodes and groups, as well as

the outcome of some iCluster setup and configuration.The default setting for this parameter is *REPL.

• REPLEVNTA parameter that determines which replication event messages are displayed. REPLEVNT only has to be specified when EVNTTYPE (see above) is set to *ALL or *REPL. The group parameter filters replication event messages.

Note: If EVNTTYPE is not *ALL or *REPL, the command ignores this parameter.Group Name


The name of an existing group that will be used to determine which replication event messages are displayed. Only replication event messages that address the referenced group will be displayed.Specify a group name or the following value:• *ALL—Displays all replication event messages regardless of group.

The default setting for this parameter is *ALL.• COMMEVNT

A parameter that determines which communication event messages are displayed. COMMEVNT only has to be specified when EVNTTYPE (see above) is set to *ALL or *COMM. The node name parameter filters communication event messages. You cannot filter communication event messages by group name.

Note: If EVNTTYPE is not *ALL or *COMM, the command ignores this parameter.Node NameThe name of the node that generated the communication event messages.Specify a node name or one of the following values:*ALL - Displays communication event messages generated by all nodes.*LOCAL - Displays communication event messages generated by the node you are currently using.The default setting for this parameter is *ALL.

• CLUSEVNTA parameter that determines which cluster event messages are displayed. CLUSEVNT only has to be specified when EVNTTYPE (see above) is set to *ALL or *CLUSTER. The node name parameter filters cluster event messages generated on the node you are currently using. You cannot filter cluster event messages by group name.

Note: If EVNTTYPE is not *ALL or *CLUSTER, the command ignores this parameter.Node NameThe name of the node that generated the cluster event messages.

Note: At this time, only *LOCAL can be specified. This setting displays cluster event messages generated by the node you are currently using.

• STRDATEThe earliest date from which iCluster will display event messages from the event log. The date must be entered in your local system date format.If you do not specify STRDATE, this command will filter event log messages starting from the beginning of the event log.

• STRTIMEThe earliest time from which iCluster will display event messages from the event log. The time must be entered in your local system time format.If you specify STRTIME with no STRDATE, this command will ignore this parameter.

• ENDDATEThe date at which iCluster will stop displaying event messages from the event log. The date must be entered in your local system date format.If you do not specify ENDDATE, this command will filter event messages to the end of the event log.

• ENDTIMEThe time at which iCluster will stop displaying event messages from the event log. The time must be entered in your local system time format.If you specify ENDTIME with no ENDDATE, this command will not filter on an ending date and time.



• MSGIDSpecifies whether you want to display those messages that match a message ID.You can find message IDs in the event log.Enter a message ID or one of the following values:• *ALL—Specifies that messages for all message IDs will be displayed.• *LATNCY—Specifies that only messages whose message ID is OMI0308 will be displayed. These messages are

generated when either the primary receive latency or the backup apply latency threshold has been exceeded.The default setting for this parameter is *ALL.

• OUTPUTIndicates whether messages are displayed on the terminal or directed to a spool file for printing.Specify one of the following values:• *—Displays messages on the terminal.• *PRINT—Directs messages to a spool file.

The default setting for this parameter is *.• DETAIL

Indicates the level of detail for each message that is displayed or directed to a spool file.Specify one of the following values:• *BASIC—Displays first level message text.• *FULL—Displays more detailed messages (first and second levels of message text).

The default setting for this parameter is *BASIC.• MSGLVLS

Specifies the severity levels of messages that will be displayed.Note that messages in more than one level can be displayed by specifying *ALL or by identifying each level (see below).

Note: To display the messages of a particular severity level in iCluster, you have to ensure that they are generated by setting an appropriate MSGLVLS system value in the DMSETSVAL—Set Cluster System Values command.

Specify one of the following values:• *ALL—Displays messages in all levels.• 00—Displays information messages (Level 00) (for example, Savefile created).• 10—Displays non-critical status messages (Level 10).• 20—Displays stop/start messages (Level 20) (for example, Start Mirroring).• 30—Displays messages that report recoverable errors (Level 30).

The default setting for this parameter is *ALL.When specific message levels are identified, messages in other levels are not displayed by iCluster. However, all fatal error messages (Level 40) are displayed.Note that more than one message level can be specified by using *ALL or by separating each message level in the parameter with a space. For example, MSGLVLS (10 20) specifies that only status and warning messages should be displayed.


Note: DataMirror recommends that you initially specify *ALL for this parameter. This ensures that you will see all messages being generated. At a later time, after you have defined nodes and groups, you can identify the specific level of messages that are displayed.

ExamplesDMDSPLOG EVNTTYPE(*ALL) REPLEVNT(*ALL) COMMEVNT(NODE2) CLUSEVNT(*LOCAL) OUTPUT(*) DETAIL(*FULL) MSGLVLS(*ALL)

Displays all types of event messages based on the specified filters.The replication event messages that will be displayed refer to any group in the cluster.The communication event messages that will be displayed are those that were generated by node NODE2.The cluster event messages that will be displayed were generated by the local node.Messages contain first and second level text and will be displayed on the terminal.Messages in all levels will be displayed.

DMDSPLOG EVNTTYPE(*REPL) REPLEVNT (*ALL) OUTPUT(*PRINT) DETAIL(*BASIC) MSGLVLS(00)Displays event messages that provide information about replication (communications and cluster event messages will be hidden).These replication event messages refer to any group in the cluster.Messages contain first level text, and will be directed to a spool file.Only information (Level 00) and fatal error (Level 40) messages will be displayed.

DMDSPLOG EVNTTYPE(*COMM) COMMEVNT(NODE3) OUTPUT(*) DETAIL(*BASIC) MSGLVLS(10 20)Displays event messages that provide information about communications between primary and backup nodes (replication and cluster event messages will be hidden).The communication event messages that will be displayed are those that were generated by node NODE3.Messages contain first level text and will be displayed on the terminal.Only status (Level 10), warning (Level 20), and fatal error (Level 40) messages will be displayed.

DMDSPLOG EVNTTYPE(*ALL) STRDATE('08/23/2003') STRTIME('09:00') ENDDATE('08/30/2003') ENDTIME('17:00') MSGID (*ALL) OUTPUT(*PRINT) DETAIL(*BASIC) MSGLVLS(00)

Displays all event messages starting August 23, 2002 at 9 am to August 30, 2003 at 5 pm on a system where the date format is 'mmddyyyy'.Messages contain first level text, and will be directed to a spool file.Only information (Level 00) and fatal error (Level 40) messages will be displayed.DMDSPLOG EVNTTYPE(*REPL) ENDDATE('08/30/2002') ENDTIME('17:00')Displays all replication event messages until August 30, 2003 at 5pm.

UseYou can issue this command from any node that is either active or inactive in the cluster.





DMCLRLOG—Clear Cluster Event LogDMCLRLOG EVNTTYPE( ) REPLEVNT( ) COMMEVNT( ) CLUSEVNT( )

Use this command to remove event messages generated by iCluster during cluster operations. The messages can be filtered by replication, communication and cluster event parameters.For more information about event logs, see Monitoring Replication on page 76.

Input Parameters• EVNTTYPE

The type of event messages that you want to remove.Specify one of the following values:• *REPL—Removes all event messages that provide information about replication.• *ALL—Removes all types of event messages.• *COMM—Removes all event messages that provide information about iCluster communications between nodes.• *CLUSTER—Displays all event messages that provide information concerning cluster nodes and groups, as well as

the outcome of some iCluster setup and configuration.The default setting for this parameter is *ALL.

• REPLEVNTA parameter that determines which replication event messages are removed. REPLEVNT only has to be specified when EVNTTYPE (see above) is set to *ALL or *REPL.

Note: If EVNTTYPE is not *ALL or *REPL, the command ignores this parameter.The parameter that filters replication event messages is as follows:Group NameThe name of a group that will be used to determine which messages are removed from the event log. Only messages that address the named group will be removed.Specify a group name or the following value:• *ALL—Removes all replication event messages regardless of group.

The default setting for this parameter is *ALL.• COMMEVNT

A parameter that determines the communication event messages that are removed from the log. COMMEVNT only has to be specified when EVNTTYPE (see above) is set to *ALL or *COMM. You cannot filter communication event messages by group name.

Note: If EVNTTYPE is not *ALL or *COMM, the command ignores this parameter.The node name parameter filters communication event messages.Node NameThe name of the node that generated the communication event messages.Specify a node name or one of the following values:• *ALL—Removes messages generated by all nodes.• *LOCAL—Removes messages generated by the node you are currently using.

The default setting for this parameter is *ALL.• CLUSEVNT


A parameter that determines which cluster event messages are removed. CLUSEVNT only has to be specified when EVNTTYPE (see above) is set to *ALL or *CLUSTER. You cannot filter cluster event messages by group name.

Note: If EVNTTYPE is not *ALL or *CLUSTER, the command ignores this parameter.The parameter that filters cluster event messages is as follows:Node NameThe name of the node that generated the cluster event messages.

Note: At this time, only *LOCAL can be specified. This setting removes cluster event messages generated by the node you are currently using.

ExamplesDMCLRLOG EVNTTYPE(*ALL) REPLEVNT(*ALL) COMMEVNT(NODE2) CLUSEVNT(*LOCAL)

Removes all types of event messages based on the specified filters.The replication event messages that will be removed refer to any group in the cluster.The communication event messages that will be removed are those that were generated by node NODE2.The cluster event messages that will be removed were generated by the local node.

DMCLRLOG EVNTTYPE(*REPL) REPLEVNT(*ALL)Removes event messages that provide information about replication (communications and cluster event messages will be kept).The replication event messages that will be removed refer to any group in the cluster.

UseYou can issue this command on a node that is active or inactive in the cluster.



HAPNGTCP—Ping Using TCPHAPNGTCP RMTNME( ) RMTPRT( ) PKTLEN( ) NBRPKT( ) RMTLIB( ) JOBD( )

Use this command to attempt to contact a specified remote node in the cluster to determine whether the node is available and operational. After entering this command, you may have to wait a few seconds before receiving a response. You can use this command as a diagnostic aid if you are having trouble replicating objects and data to the remote node. This command allows you to verify iCluster communications.

Input Parameters• RMTNME

The host name of a remote node in the cluster.• RMTPRT

The port number on the remote node that has been reserved for iCluster communications.You should specify the port number that was defined during installation. For more information about specifying the port number during installation, see Adding a New Entry 'dmcluster' to the TCP/IP Service Table on page 37.



The default remote port number is 4545.• PKTLEN

The length, in bytes, of the packets you are sending to the remote node in the cluster.The length can range from 32 bytes to 32,760 bytes.The default packet length is 256 bytes.

• NBRPKTThe number of packets you are sending to the remote node in the cluster.This number can range from 1 to 999 packets.The default number of packets is 5.

• RMTLIBThe name of the library on the remote node where iCluster was installed.The default library is DMCLUSTER.

• JOBDThe job description associated with iCluster jobs running on the remote node.The default job description is CSJOBD.

ExampleHAPNGTCP RMTNME(NYSYS1) RMTPRT(4545) PKTLEN(256) NBRPKT(5) RMTLIB(ICLUS) JOBD(ICJOBD)

Attempts to contact the remote node that is identified by the host name NYSYS1, and whose remote port number is 4545.Five packets will be sent, with each packet being 256 bytes in length.On the remote node, iCluster is installed in the library ICLUS and the job description ICJOBD will be associated with iCluster jobs running on the remote node.

UseTo invoke this command, iCluster must be installed on both the local and remote nodes, and the TCP listener job must be running on the remote node.


JRNHADADQ—Journal Data Areas and Data QueuesJRNHADADQ

Use this command to start data area and data queue journaling by iCluster on the objects on the system.Note: This command should be run for every machine in the cluster.

This command will start journaling for any unjournaled data areas and data queues that match an object specifier defined in your iCluster installation after the operating system is upgraded from any release prior to OS/400 V5R1.Journaling will be started to the default journal of the group to where the object specifier is selected. If you would prefer to use a different journal other than the default for the group, you must either change the group default before running this command, or manually journal the objects to another journal.After starting journaling on the objects manually, or by running this command, re-starting replication will automatically refresh the objects to achieve their initial synchronization. Through journaling, only changes to these objects are mirrored, which will increase throughput and reduce network bandwidth requirements.


Input ParametersNone.

OutputRelevant messages to the job log.

ExamplesJRNHADADQ

Starts journaling any unjournaled data areas and data queues that match a specified object specifier defined in your iCluster installation.

UseYou need to run this command only after the upgrade of the operating system is complete. If you do not run this command after the upgrade, data areas and data queues will no longer be mirrored.

STRHATCP—Start TCP/IP ListenerSTRHATCP JOBD( ) LIB( ) HOST( ) SERVICE( )

Use this command to start the iCluster TCP/IP listener job (DMCHATL) on the local node. This job must be running on all nodes that are defined in the cluster.

Input Parameters• JOBD

The job description that you want to associate with the TCP/IP listener job (DMCHATL) that is started by this command.The default setting for this parameter is CSJOBD.

• LIBThe library that contains the job description you want to associate with the TCP/IP listener job (DMCHATL) that is started by this command.The default setting for this parameter is DMCLUSTER.

• HOSTThe name of the local host to be used.

Note: This parameter is usually omitted to allow the listener job to accept incoming connection requests directed to any local address. You should specify this parameter only for a multi-homed site to restrict the listener to accept incoming connection requests to a single local address.

• SERVICEThe name of the service that identifies the local port on which the listener job will listen.This parameter is usually omitted to allow the default service dmcluster to be selected. You should specify this parameter only when it is necessary to start multiple listeners on different ports.

ExampleSTRHATCP JOBD(TCPJOBD) LIB(LIB1)

Starts the TCP/IP listener job with the job description TCPJOBD in library LIB1.The host name and TCP service name are retrieved by the job from a pre-defined data area.

UseYou must issue this command on the node where the operation will be performed.




WRKHAJOB—Work with Active Cluster JobsWRKHAJOB

Use this command to display the active cluster jobs for the subsystem that iCluster is running under (XDMCLUSTER), plus the subsystems QCMN and QSYSWRK.To view all active jobs regardless of the subsystem, execute the OS/400 command WRKACTJOB.


ExampleWRKHAJOB

Displays the Work with Active Jobs screen so that you can view active cluster jobs for the subsystem that iCluster is running under, plus the subsystems QCMN and QSYSWRK.



DMSTRDM—Start Definition ManagerDMSTRDM

Starts the definition manager on the current node.

Input PrametersNone.

ExampleDMSTRDM

Starts the definition manager on the node where the command was entered.

UseYou must issue this command on the node where the operation will be performed. The node must be active before you start the definition manager.

DMDLTCLSTR—Delete ClusterDMDLTCLSTR CLSTR( )

If invoked on an active node, this command ends cluster operations and deletes cluster definitions maintained by iCluster on all active nodes in the cluster.If this command is invoked on an inactive node, cluster operations are stopped and cluster definitions are deleted only on the node where the command is invoked. Therefore, to ensure that all cluster operations are stopped and all cluster definitions are deleted, issue this command on each node in the cluster.


If a communication failure occurs between two active nodes, issue this command on both partitioned nodes to ensure cluster operations are stopped and all cluster definitions are deleted.

Input Parameter• CLSTR

The name of the cluster that you want to delete. This command can be used to delete any iSeries cluster.In iCluster, the name of the cluster is set internally to DM_CLUSTER, and cannot be changed.

ExampleDMDLTCLSTR CLSTR(DM_CLUSTER)

Stops cluster operations and deletes cluster definitions in the cluster DM_CLUSTER. If invoked from an active node, cluster operations will be stopped and cluster definitions will be deleted on all active nodes in the cluster.

UseYou can issue this command on any node in the cluster.



DMSYSINF—System InformationDMSYSINF OUTPUT( )

Use this command to display the following iCluster system information:• Node names• System names• Node IP addresses• iCluster version• Additional features• Operating system version, model number, and serial number• System timesThis command tells you if you are using Cluster Resource Services.Note: This screen displays the nodes in the order that they were added to the cluster. The first node is the master

node.Note: If a node is down, the only information displayed for the node is the node name and IP address.

Input Parameter• OUTPUT

Indicates whether system information is displayed on the console or directed to a spool file for printing.Specify one of the following values:• *—Displays system information on the console.• *PRINT—Directs system information to a spool file for printing.

The default for this parameter is '*'.



OutputiCluster system information is sent to the console or a spool file.

ExampleDMSYSINF OUTPUT(*)

Displays system information on the console.



DMWRKSSPLF—Work with Suspended Spooled FilesDMWRKSSPLF GROUP( ) OUTQ( )The Work with Suspended Spooled Files (DMWRKSSPLF) command displays the suspended spooled files for a group or a group and a specific output queue. After the suspended spooled files are displayed, you can then activate selected spooled files.


Specifies the name of an existing replication group.• OUTQ

Specifies the name of an existing output queue that is in the replication scope of the specified group.Specify one of the following values:• *ALL—Specifies that suspended spooled files to be displayed can be in any output queue that is replicated by the

group.• output-queue-name—Specifies the name of the output queue whose suspended spooled files are to be displayed.

The library where the output queue resides must be identified, unless the special value *ALL is specified for the output queue. Prefix the output queue with the name of the library where the output queue is located (for example, LIB2/OUTQ1).

ExampleDMWRKSSPLF GROUP(GRP1) OUTQ(LIB1/OUTQ1)

Displays the suspended spooled files for group GRP1 and output queue OUTQ1.

Minimum Security LevelOperator (*Operator)

DMACTSPLF—Activate Spooled FileDMACTSPLF GROUP( ) OUTQ( ) SPLF( ) SPLFNBR( ) JOBNAME( ) JOBUSER( ) JOBNBR( ) SRCSYSTEM( ) TGTLIB( )The iCluster Activate Spooled File (DMACTSPLF) command activates a spooled file that is currently suspended from replication. After activating a spooled file, mirroring of the specified spooled file will start if the replication group is active.If the replication group is inactive, mirroring will start when the group is activated through the DMSTRGRP or DMSTRAPP command.


Note: The spooled file will be refreshed when it is activated.This command must be invoked on an active node in the cluster.


Specifies the name of the replication group to which the output queue of the spooled file is selected.• OUTQ

The name of the output queue that contains the suspended spooled file that you want to activate.The library where the output queue resides must be identified, unless the special value *ALL is specified for the output queue. Prefix the output queue with the name of the library where the output queue is located (for example, LIB2/OUTQ1).

• SPLFSpecifies the name of the suspended spooled file that you want to activate.

• SPLFNBRSpecifies the number of the suspended spooled file that you want to activate.

• JOBNAMESpecifies the name of the job that created the suspended spooled file that you want to activate.

• JOBUSERSpecifies the user of the job that created the suspended spooled file that you want to activate.

• JOBNBRSpecifies the number of the job that created the suspended spooled file that you want to activate.

• SRCSYSTEMSpecifies the machine name on which the suspended spooled file that you want to activate was created.

• TGTLIBSpecifies the library of the output queue on the backup node to which the suspended spooled file will be refreshed when activated.

ExampleDMACTSPLF GROUP(GRP2) OUTQ(LIB2/OUTQ2) SPLF(QPJOBLOG) SPLFNBR(000001) JOBNAME(MYJOB) JOBUSER(USER1) JOBNBR(654321) SRCSYSTEM(MACHINEA) TGTLIB(LIB3)

Activates the specified spooled file.

Minimum Security LevelOperator (*Operator)



Cluster Operation Commands

Cluster Operation CommandsThis section contains the following topics:• DMSTRNODE—Start Cluster Operations at Node on page 207• DMENDNODE—End Cluster Operations at Node on page 208• DMSTRGRP—Start Cluster Operations at Group on page 208• DMSTRREPL—Start Replication on page 211• DMENDGRP—End Cluster Operations for Group on page 213• DMSTRWO—Switchover Group on page 215• DMREJOIN—iCluster Rejoin Cluster on page 216• DMSTRAPP—Start Cluster Operations for Resilient Application on page 217• DMENDAPP—End Cluster Operations for Resilient Application on page 218• DMSWOAPP—Switchover Resilient Application on page 220• DMSETPRIM—Prepare Primary Node on page 220• DMCHGROLE—Change Group Primary Node on page 221• DMGENEXC—Generate Command Exceptions on page 222

DMSTRNODE—Start Cluster Operations at NodeDMSTRNODE NODE( )

Use this command to start cluster operations at the specified node. High availability support for active groups that have this node in their recovery domain is also started.This command sets the status of the specified node to *ACTIVE.If the node is the backup node of a group that has an active status, cluster operations for the group will also be started at the specified node.


The name of the node where cluster operations will be started.

ExampleDMSTRNODE NODE(NODE1)

Starts cluster operations at node NODE1.Sets the status of the node to active.If NODE1 is the backup node of an active group, cluster operations for the group will also be started at this node.

UseYou must issue this command on an active node in the cluster unless there are no active nodes in the cluster.

Note: If this command is invoked on different nodes, separate clusters are effectively activated. To ensure that only a single cluster is maintained, issue this command for each node in the cluster from a single node. The first time you invoke the command, reference the name of the single node. Then, from the single node, issue the command for each other node in the cluster.


Minimum Security LevelOperator (*OPERATOR)

Menu AccessMain Menu - Option 11F22 (Shift+F10) - Option 40Work With Nodes screen - Option 1

DMENDNODE—End Cluster Operations at NodeDMENDNODE NODE( )

Use this command to end cluster operations at the specified node. Use this command if you intend to re-start cluster operations at the specified node. Otherwise, use the DMRMVNODE—Remove Node command to remove the specified node from the cluster when it is active.This command sets the status of the specified node to *INACTIVE.If the node is the backup node of a replication group that has an active status, cluster operations for the group will also be stopped at the specified node before cluster operations at the node are ended. The replication group will still have a status of *ACTIVE even though replication operations have stopped.

Note: Cluster operations at the node cannot be stopped if the specified node is a primary node in an active replication group.


The name of the node where cluster operations will be stopped.

ExampleDMENDNODE NODE(NODE1)

Ends cluster operations at node NODE1.Sets the status of the node to inactive.If NODE1 is the backup node of an active replication group, cluster operations for the group will be ended at this node before cluster operations for the node are stopped.



Menu AccessMain Menu - Option 12F22 (Shift+F10) - Option 41Work With Nodes screen - Option 4

DMSTRGRP—Start Cluster Operations at GroupDMSTRGRP GROUP( ) PRIMNODE( ) STRAPY( ) REFRESH( ) TRUNCATE( ) REPLACELFS( ) USEMARKED( )



Use this command to start cluster operations for the specified cluster resource groups. The high availability support provided by iCluster is started for replication groups and WebSphere MQ groups.This command also sets the cluster status of the specified groups to *ACTIVE. If the groups are replication groups, the replication status of the groups will become *ACTIVE when replication processes are started.

Note: The STRAPY, REFRESH, and USEMARKED parameters are ignored if the group is a *SWDEV group. See DMADDGRP—Add Group on page 113 for more information.

Note: The REFRESH and USEMARKED parameters are ignored if the group is a *RFSH group. See DMADDGRP—Add Group on page 113 for more information.


The name of the group that has cluster operations started by this command.Specify the name of the group or one of the following values:• *ALL—Starts all groups on all nodes, regardless of their current recovery domain.• *PRIMNODE—Starts all groups with the primary node specified. If you specify this option, you must specify the

name of the primary node in the PRIMNODE parameter (see below).• PRIMNODE

Indicates the name of the primary node for the groups to be started. This parameter is only applicable when the GROUP parameter (see above) is set to *PRIMNODE.

• STRAPYIndicates whether the apply process for the replication group will be started on the backup node when mirroring begins.Specify one of the following values:• *NOCHG—Indicates that the last operational status of the apply jobs on the backup node remains in affect.• *YES—Indicates that the apply jobs on the backup node for the replication group will be started. This does not

apply to suspended files.• *NO—Indicates that the apply jobs on the backup node for the replication group will not be started.

The default for this parameter is *NOCHG.• REFRESH

Indicates whether an initial refresh of selected objects in the replication group is performed before mirroring is started.If the USEMARKED parameter is set to *YES, this parameter is ignored.Specify one of the following values:• *YES—Specifies that an initial refresh of selected objects in the replication group is performed.• *NO—Specifies that an initial refresh of selected objects in the replication group is not performed.• *MQRUNSTR—Refreshes objects on the backup node for Websphere MQ groups and is recommended for

environments where WMQ transactions may be started while the group is inactive. This option is similar to *YES, except that the apply jobs will process MQ journal entries from the time the group was last ended instead of from the time you run the command. You can only use this option with MQ Series groups that were previously active.

The default setting for this parameter is *NO.Warning: If you unexpectedly began a refresh (by selecting the *YES option for the REFRESH parameter) and then

ended the refresh before completion (due to system constraints), you must contact DataMirror technical support for assistance. See Contacting Technical Support on page 547 for more information. You must perform a number of steps following a discontinued refresh to allow mirroring to re-start properly. If these steps are not performed following a discontinued refresh, mirroring may not work as you intended.


If you intentionally began a refresh that ended unexpectedly (for example, power failure), you must re-start the refresh.• TRUNCATE


behavior for iCluster and is appropriate for standard iCluster replication. You can use this option for a master-master group if the data on the source system is known to be complete in order to ensure a synchronized starting point.

• With this value, logical files are replaced automatically because the deletion of a physical file requires the deletion of all logical files upon which it is based.










• USEMARKEDIndicates whether mirroring is started at the marked journal positions for the specified replication group.Note that these journal positions are marked through the DMMRKPOS—Mark Current Journal Positions command, and are maintained in the iCluster metadata.Specify one of the following values:• *NO—Indicates that iCluster replication will not start at positions marked using the DMMRKPOS—Mark Current

Journal Positions command. iCluster replication will start at either: • Journal positions set using the DMSETPOS—Set Journal Start Position command. • Journal positions registered using the DMREGPOS—Register Positions command.



• The journal entry following the last journal position received on the backup system when replication was previously stopped.

• *YES—Indicates that mirroring will start at the journal positions that have been marked for the specified replication group through the DMMRKPOS—Mark Current Journal Positions command. Note that the primary and backup nodes must have been synchronized at the time the DMMRKPOS—Mark Current Journal Positions command was issued. The REFRESH parameter is ignored when this parameter is set to *YES.

Note that specifying *YES on this parameter will overwrite any starting journal positions previously journaled by the DMSETPOS—Set Journal Start Position or DMREGPOS—Register Positions commands.The default setting for this parameter is *NO.

Note: If the DMMRKPOS—Mark Current Journal Positions command has not been invoked for the named replication group, and the value specified for this parameter is *YES, mirroring is started at the last replication position.

ExamplesDMSTRGRP GROUP(GRP1) STRAPY(*YES) USEMARKED(*YES)

Mirroring will start at the journal positions that have been marked for GRP1 through the DMMRKPOS command.Update/apply jobs on the backup node will be started.

DMSTRGRP GROUP(GRP2) STRAPY(*NO) REFRESH(*NO) TRUNCATE(*NO) REPLACELFS(*NO) USEMARKED(*NO)An initial refresh of selected objects in replication group GRP2 is not performed before mirroring is started.Existing logical files are not used during the refresh, and existing data is not removed before performing the refresh.Mirroring is started at the last replication positions for the replication group GRP2.The apply job on the backup node is not started when mirroring begins. Any updates that are in progress is stopped in a controlled manner.

DMSTRGRP GROUP(*PRIMNODE) PRIMNODE(AS400HQ) STRAPY(*NO) USEMARKED(*NO)All groups that have node AS400HQ as their primary node are started.An initial refresh is not performed before mirroring is started.Replication starts at the last replication positions for all groups of type *REPL and *MQSERIES that have the specified primary node. A full refresh is performed for groups of type *RFSH that have the specified primary node.The apply job on the backup node is not started when mirroring begins. Any updates that are in progress are stopped in a controlled manner.



Menu AccessMain Menu - Option 13F22 (Shift+F10) - Option 43Work With Groups screen - Option 1

DMSTRREPL—Start ReplicationDMSTRREPL GROUP( ) STRAPY( ) REFRESH( ) USEMARKED( )


Use this command to start replication for the specified replication group or resilient application.Note: This command should only be invoked for groups or resilient applications in *ACTIVE status. To start

replication for an inactive group, use the DMSTRGRP command.Before using this command, you must identify why replication is not active. If the reason for replication failing or not starting is unclear, contact DataMirror Technical Support for assistance. See Contacting Technical Support on page 547 for more information.


The name of the replication group or resilient application for which replication will be started by this command.• STRAPY

Indicates whether the apply process on the replication group's or resilient application's backup node will be started when mirroring begins.Specify one of the following values:• *NOCHG—Specifies that the current operational status of update/apply job on the backup node is not changed.• *YES—Indicates that the backup node update/apply job for the replication group or resilient application will be

started. This does not apply to suspended files.• *NO—Indicates that the backup node update/apply job for the replication group or resilient application will not be

started.The default for this parameter is *NOCHG.

• REFRESHIndicates whether an initial refresh of objects selected to the replication group or resilient application is performed before mirroring is started.If the USEMARKED parameter (see below) is set to *YES, this parameter is ignored.Specify one of the following values:• *YES—Specifies that an initial refresh of selected objects is performed.• *NO—Specifies that an initial refresh of selected objects is not performed.

The default setting for this parameter is *NO.Warning: If you unexpectedly began a refresh (by selecting the *YES option for the REFRESH parameter) and then

ended the refresh before completion (due to system constraints), you should contact DataMirror technical support for assistance. See Contacting Technical Support on page 547 for more information. You must perform a number of steps following a discontinued refresh to allow mirroring to re-start properly. If these steps are not performed following a discontinued refresh, mirroring may not work as you intended.

If you intentionally began a refresh that ended unexpectedly (for example, power failure), you should re-start the refresh.• USEMARKED

Indicates whether mirroring is started at the marked journal positions for the specified replication group or resilient application.Note that these journal positions are marked through the DMMRKPOS—Mark Current Journal Positions command, and are maintained in the iCluster metadata.Specify one of the following values:• *NO—Indicates that iCluster will start mirroring from either the saved journal positions when replication was

previously stopped, the journal positions marked through the DMSETPOS—Set Journal Start Position command prior to issuing this command, or if there are no saved or marked journal positions, mirroring will start at the last replication positions for objects selected to the replication group or resilient application.



• *YES—Starts mirroring at the journal positions that have been marked for the specified replication group or resilient application through the DMMRKPOS—Mark Current Journal Positions command. Note that the primary and backup nodes must have been synchronized at the time the DMMRKPOS—Mark Current Journal Positions command was issued. The REFRESH parameter (see above) will be ignored when this parameter is set to *YES.

The default setting for this parameter is *NO.Note: If the DMMRKPOS—Mark Current Journal Positions command has not been invoked for the named

replication group or resilient application, and the value specified for this parameter is *YES, mirroring is started at the last replication positions.

ExamplesDMSTRREPL GROUP(GRP1) STRAPY(*YES) USEMARKED(*YES)

Mirroring will be started at the journal positions that have been marked for GRP1 through the DMMRKPOS command.Update/apply jobs on the backup node will be started.

DMSTRREPL GROUP(APP2) STRAPY(*NO) REFRESH(*NO) USEMARKED(*NO)An initial refresh of objects selected to resilient application APP2 will not be performed before mirroring is started.Mirroring will be started at the last replication positions for the resilient application GRP2.The apply job on the backup node will not be started when mirroring begins. Any updates that are in progress will be stopped in a controlled manner.

UseYou can issue this command on an active node in the cluster when the replication group or resilient application has a status of *ACTIVE but is not replicating.



DMENDGRP—End Cluster Operations for GroupDMENDGRP GROUP( ) PRIMNODE( ) OPTION( ) DELAY( )

Use this command to end cluster operations for the specified groups.This command sets the cluster status of the specified groups to *INACTIVE.If replication for a group fails, the group should be ended using this command.


The name of the active group that will have cluster operations stopped by this command.Specify the name of the group or one of the following values:• *ALL—Ends all groups on all nodes, regardless of their current recovery domain.• *PRIMNODE—Ends all groups with the specified primary node. If you specify this option, you must specify the

name of the primary node in the PRIMNODE parameter (see below).• PRIMNODE

Indicates the name of the primary node of the groups that will end cluster operations. This parameter is only applicable when the GROUP parameter (see above) is set to *PRIMNODE.


• OPTIONIndicates how replication for the group is stopped. Replication within the group can be stopped immediately or in a controlled manner.An immediate stop concludes replication at the point when the command is invoked. As a result, iCluster may not be able to guarantee that replication is stopped in the desired manner. On the other hand, a controlled stop allows iCluster to perform the necessary tasks to ensure that replication is gracefully ended. However, the completion of these tasks may take some time. DataMirror recommends performing a controlled stop when possible.Specify one of the following values:• *CNTRLD—Ends replication for the group in a controlled manner.• *IMMED—Ends replication for the group immediately.

The default setting for this parameter is *CNTRLD.• DELAY

The maximum amount of time (expressed in seconds) for replication to stop in a controlled manner without intervention.This parameter allows you to specify a timeout period for stopping replication. If, for some reason, replication cannot be completed within the timeout period, it will be immediately stopped after the timeout period expires. This parameter is only applicable when the OPTION parameter (see above) is set to *CNTRLD.The default setting for this parameter is 3600 seconds.

Note: Long running refreshes or commitment control may require a longer timeout to ensure consistency of the data on the backup system.

Note that there are certain conditions where the apply job(s) for a group may not end quickly. For example, if commitment control is used, the apply cannot end until it reaches a point where there are no open commit cycles. For a busy system, this may take a very long time (if ever) to happen. Another example would be when a large refresh is taking place. It is therefore important that situations such as those mentioned be considered and a timeout is set appropriately to ensure the backup database is in a consistent state while the apply is inactive. In general, the default setting for this parameter should be long enough, but individual circumstances may require a longer timeout.

ExamplesDMENDGRP GROUP(GRP1) OPTION(*IMMED)

Cluster operations for group GRP1 will be immediately stopped.DMENDGRP GROUP(GRP2) OPTION(*CNTRLD) DELAY(120)

Cluster operations for group GRP2 will be stopped in a controlled manner.If replication cannot be stopped in a controlled manner within 120 seconds (two minutes) after the command was invoked, replication within GRP2 will be immediately stopped.

DMENDGRP GROUP(*ALL) OPTION(*IMMED)Cluster operations are ended immediately for all groups on all nodes, regardless of your current recovery domain.UseYou must issue this command on an active node in the cluster.





DMSTRWO—Switchover GroupDMSTRSWO GROUP( ) STRREPL( )

Use this command to initiate a role switch within the specified cluster resource group(s). Therefore, this command causes a switchover involving the primary and backup nodes in the group(s).When you issue this command, the current primary node becomes the backup node, and the current backup node assumes the role of the primary node. In addition, the user exit program(s) that may have been specified for the group will be invoked on the new primary node. See DMADDGRP—Add Group and DMCHGGRP—Change Group for more information.This command must be invoked on an active node in the cluster. However, it can only be applied to active groups.

Note: This command supports spooled file replication on the primary and backup node(s) as of i5/OS V5R4.Note: To initiate a group switchover, the status of low-level cluster support provided by IBM's OS/400 Cluster

Resource Services for the group must be *ACTIVE. See DMWRKGRP—Work with Groups for more information.

Note: A message stating whether or not the role switch completed successfully is placed into the event log on all nodes of the cluster. For more information about event logs, see Monitoring Replication on page 76.


The name of the cluster resource group where the roles of the primary and backup nodes in the group will be reversed.Specify the name of a cluster resource group or the following value:• *ALL—Switches the roles of primary and backup nodes in all active groups defined in iCluster that are not part of a

resilient application.• STRREPL

Indicates whether or not to re-start replication after a switchover has completed.Specify one of the following values:• *YES—Indicates that replication processes for the group will be re-started from the new primary node to the new

backup node when switchover processing is complete. This is the default setting.• *NO—Indicates that replication processes for the group will not be re-started from the new primary node to the new

backup node when switchover processing is complete. If this value is specified, the DMSTRREPL—Start Replication command can be used to re-start replication at a later time. The Use marked journal positions (USEMARKED) parameter must be set to *YES when invoking the DMSTRREPL—Start Replication command for the first time after a switchover.

ExamplesDMSTRSWO GROUP(GRP1) STRREPL(*YES)

Switches the roles of primary and backup nodes in the active replication group GRP1.With the STRREPL parameter set to *YES, replication processing will be re-started from the new primary node to the new backup node when switchover processing is complete. This is the default setting. If one of the user exit program(s) was specified for group GRP1 through the DMADDGRP—Add Group or DMCHGGRP—Change Group commands, they will be invoked on the new primary node.

DMSTRSWO GROUP(*ALL) STRREPL(*NO)Switches the roles of primary and backup nodes in all active groups defined in iCluster that are not part of a resilient application.With the STRREPL parameter set to *NO, replication processing will not be re-started from the new primary node to the new backup node when switchover processing is complete.


User exit programs that may have been specified for each group through the DMADDGRP—Add Group or DMCHGGRP—Change Group commands will be invoked on the new primary nodes.

UseYou must issue this command on an active node in the cluster. However, it can only be applied to active groups.


Menu AccessMain Menu - Option 15F22 (Shift+F10) - Option 46Work With Groups screen - Option 20

DMREJOIN—iCluster Rejoin ClusterDMREJOIN NUMTRIES( ) DELAY( ) STARTNEW( )

Use this command to re-start cluster node operations on the current machine in such a way that it rejoins the cluster of which it was a previously an active member. It also sets the status of the node on the current machine to active if the node successfully rejoins the cluster.If the current machine's node is the backup node of a group that has active status, cluster group operations will also be started at the current machine's node through this command.This command requires that the current machine is inactive in the cluster and is recognized as a node by at least one active node in the cluster.Note that this command may take a few minutes to complete.

Input Parameters• NUMTRIES

Specifies the number of attempts that will be made to rejoin an existing cluster. The default value is one attempt.• DELAY

Specifies the time interval in seconds between attempts to re-start cluster operations at the current node. The default delay time is 10 seconds.

• STARTNEWSpecifies whether or not a new cluster will be started on the current machine if attempts to rejoin an existing cluster fail.Enter one of the following values:• *YES—Indicates that you want to start a new cluster on the current machine if attempts to rejoin an existing cluster

fail. The new cluster will not be part of an existing cluster.• *NO—Indicates that you do not want to start a new cluster on the current machine if attempts to rejoin an existing

cluster fail. The current node will remain inactive if attempts to rejoin an existing cluster fail. This is the default value for this parameter.

OutputThe status of the node should change to *ACTIVE.

ExampleDMREJOIN NUMTRIES(5) DELAY(10) STARTNEW(*NO)

iCluster will attempt to re-start operations, to a maximum of five times, on the current machine.



UseYou can issue this command only on an inactive node in the cluster.



DMSTRAPP—Start Cluster Operations for Resilient ApplicationDMSTRAPP APPNAME( ) STRAPY( ) REFRESH( ) USEMARKED( )

Use this command to start cluster operations for the specified resilient application.Objects associated with the application are identified through groups that are defined in the data areas and files provided by the application vendor. Therefore, this command ensures that application objects are mirrored to nodes in the application group to support a resilient application switchover or failover. See DMSWOAPP—Switchover Resilient Application on page 220 for more information.You can perform an initial refresh of the application objects before starting cluster operations.Cluster operations for groups that identify application objects are started in an indeterminate order. If the application requires operations to be started in a specific order, the order must be defined through a user exit provided by the application vendor.For groups that have no object specifiers, the replication process will not be started, and the group, and resilient application, if it is part of a resilient application, will stay in *ACTIVE status (even though its replication status will be *INACTIVE). This allows a switchover to be performed on the group and its resilient application, if applicable.


The name of a resilient application that you have defined in iCluster.Cluster operations in application groups will be started.

• STRAPYIndicates whether the apply process for the replication groups associated with the resilient application will be started on the backup node when mirroring begins.Specify one of the following values:• *NOCHG—Indicates that the current operational status of the apply jobs on the backup node is not changed.• *YES—Indicates that the apply jobs on the backup node for the replication groups associated with the resilient

application will be started. This does not apply to suspended files.• *NO—Indicates that the apply jobs on the backup node for the replication groups associated with the resilient

application will not be started.The default setting for this parameter is *NOCHG.

• REFRESHIndicates whether an initial refresh of objects for replication groups is performed before mirroring is started.If the USEMARKED parameter (see below) is set to *YES, this parameter is ignored.Specify one of the following values:• *YES—Indicates that an initial refresh of objects for replication groups will be performed.• *NO—Indicates that an initial refresh of objects for replication groups will not be performed.


The default setting for this parameter is *NO.Warning: If you unexpectedly began a refresh (by selecting the *YES option for the REFRESH) and then ended the

refresh before completion (due to system constraints), you should contact DataMirror technical support for assistance. See Contacting Technical Support on page 547 for more information. You must perform a number of steps following a discontinued refresh to allow mirroring to re-start properly. If these steps are not performed following a discontinued refresh, mirroring may not work as you intended.

If you intentionally began a refresh that ended unexpectedly (for example, power failure), you should re-start the refresh.

• USEMARKEDIndicates whether mirroring is started at the marked journal positions for the specified resilient application.Note that these journal positions are marked through the DMMRKPOS—Mark Current Journal Positions command, and are maintained in the iCluster metadata.Specify one of the following values:• *NO—Indicates that iCluster will start mirroring from either the saved journal positions when replication was

previously stopped, the journal positions marked through the DMSETPOS—Set Journal Start Position command prior to issuing this command, or if there are no saved or marked journal positions, mirroring will start at the last replication positions for objects selected to the application groups.

• *YES—Indicates that mirroring is started at the journal positions that have been marked for the specified resilient application through the DMMRKPOS—Mark Current Journal Positions command. Note that the primary and backup nodes must have been synchronized at the time the DMMRKPOS—Mark Current Journal Positions command was issued. The REFRESH parameter (see above) will be ignored when this parameter is set to *YES.

The default setting for this parameter is *NO.Note: If the DMMRKPOS—Mark Current Journal Positions command has not been invoked for the named resilient

application, and the value specified for this parameter is *YES, mirroring is started at the last replication positions.

ExampleDMSTRAPP APPNAME(OEPACK) USEMARKED(*YES)

Cluster operations are started for the resilient application OEPACK.Mirroring will be started at the journal positions that have been marked for OEPACK through the DMMRKPOS—Mark Current Journal Positions command.



Menu AccessMain Menu - Option 22F22 (Shift+F10) - Option 47Work With Resilient Applications screen - Option

DMENDAPP—End Cluster Operations for Resilient ApplicationDMENDAPP APPNAME( ) OPTION( ) DELAY( )

Use this command to end cluster operations for the specified resilient application.



Objects associated with the application are identified through groups that are defined in the data areas and files provided by the application vendor. Therefore, this command stops mirroring to the resilient applications backup node. You can specify the manner in which replication is stopped (either immediately or in a controlled manner).Replication within groups that identify application objects is stopped in an indeterminate order. If the application requires replication to be stopped in a specific order, the order must be defined through a user exit provided by the application vendor.If replication for a resilient application fails, you should end the resilient application using this command.


The name of a resilient application that you have defined in iCluster.Cluster operations in application groups will be stopped.

• OPTIONIndicates how replication for the replication groups is stopped. Replication for the replication groups can be stopped immediately or ended in a controlled manner.An immediate stop concludes replication at the point when the command is invoked. As a result, iCluster may not be able to guarantee that replication is stopped in the desired manner. On the other hand, a controlled stop allows iCluster to perform the necessary tasks to ensure that replication is gracefully ended. However, the completion of these tasks may take some time. DataMirror recommends performing a controlled stop when possible.Specify one of the following values:• *CNTRLD—Ends replication for the replication groups in a controlled manner.• *IMMED—Ends replication for the replication groups immediately.

The default setting for this parameter is *CNTRLD.• DELAY

The maximum amount of time (expressed in seconds) for replication to stop in a controlled manner without intervention.This parameter allows you to specify a timeout period for stopping replication. If, for some reason, replication cannot be completed within the timeout period, it will be immediately stopped after the timeout period expires. This parameter is only applicable when the OPTION parameter (see above) is set to *CNTRLD.The default setting for this parameter is 3600 seconds.

ExampleDMENDAPP APPNAME(OEPACK) OPTION(*CNTRLD) DELAY(60)

Cluster operations are stopped for the resilient application OEPACK.Replication associated with OEPACK is stopped in a controlled manner.If replication cannot be stopped in a controlled manner within 60 seconds (one minute) after the command was invoked, replication will be immediately stopped.





Work With Resilient Applications screen - Option 4

DMSWOAPP—Switchover Resilient ApplicationDMSWOAPP APPNAME( ) DELAY( ) STRREPL( )

Use this command to initiate a role switch within the groups associated with the specified resilient application. Therefore, this command causes a switchover involving the primary and backup nodes in the resilient application.After using this command, the current primary node in each group becomes the backup node, and the current backup node in each group assumes the role of the primary node.



The name of a resilient application that you have defined in iCluster.A role switch will be performed in the groups associated with the specified resilient application.

• DELAYSpecifies the time, in seconds, that is allowed between a switchover of the application groups associated with the specified resilient application, and switchover of the replication groups associated with the specified resilient application.

• STRREPLIndicates whether or not to re-start replication after a switchover has completed.Specify one of the following values:• *YES—Indicates that replication will re-start after a switchover.• *NO—Indicates that replication will not re-start after a switchover.

The default setting for this parameter is *YES.

ExampleDMSWOAPP APPNAME(OEPACK) DELAY (30) STRREPL(*YES)

Switches the roles of primary and backup nodes in the groups associated with the resilient application OEPACK. There will be a delay of 30 seconds between the switchover of the associated application group and the replication group. Replication will re-start after the switchover.

UseYou must issue this command on an active node in the cluster.Minimum Security LevelAdministrator (*ADMIN)

Menu AccessMain Menu - Option 24F22 (Shift+F10) - Option 49Work With Resilient Applications screen - Option 20

DMSETPRIM—Prepare Primary NodeDMSETPRIM GROUP( )



Use this command to perform the necessary tasks to prepare a group or resilient application's primary node to be the source node for replication after a failover of the group's primary node has occurred. After preparation is complete, the node can be the source node for replication for the group or resilient application.This command is intended for groups that do not have the automatic role switch mechanism enabled (set to *NO). An automatic role switch is enabled or disabled by setting the ROLESWITCH parameter in the DMADDGRP—Add Group or DMCHGGRP—Change Group commands. In this situation, this command will prepare the primary node to become the source node for replication as part of its failover processing. In addition, the user exit program(s) that may have been specified for the group will be invoked on the new primary node. See DMADDGRP—Add Group or DMCHGGRP—Change Group for more information.


This command is not available when the Use OS/400 Cluster Services system value is *NO. See DMSETSVAL—Set Cluster System Values on page 181 for more information on system values.


The name of the replication group or application that will have its primary node prepared by this command.

ExamplesDMSETPRIM GROUP(GRP1)

After a failover (with *ROLESWITCH set to *NO), the GRP1 replication group will have its primary node prepared to be the active node for replication.If a user exit program was specified for group GRP1 through the DMADDGRP—Add Group or DMCHGGRP—Change Group commands, it will be invoked on the new primary node.


Minimum Security LevelOperator (*OPER)


DMCHGROLE—Change Group Primary NodeDMCHGROLE GROUP( ) PRIMNODE( )

This command changes the specified groups' or resilient applications' primary node to the current first backup node. The groups or resilient applications will remain *INACTIVE until they are started with the DMSTRGRP—Start Cluster Operations at Group or DMSTRAPP—Start Cluster Operations for Resilient Application commands.The current backup is prepared to become the primary node in the same way as occurs if a failover happens when the group is active.

Note: A message (CSI1314) stating whether or not the role switch completed successfully is placed into the event log on all nodes of the cluster. For more information about event logs, see Monitoring Replication on page 76.


The name of the group or resilient application. If you specify a group or resilient application, the group (or groups of the resilient application) has to be in *INACTIVE status.


Specify the name of the group or resilent application or one of the following values:• *ALL - Performs a role switch for all groups and resilient applications in the cluster. Changes the primary node of all

groups and resilient applications in the cluster to the current first backup node.• *PRIMNODE - Performs a role switch for all groups and resilient applications whose primary node is specified on

the PRIMNODE parameter. If you specify this option, you must specify the name of the primary node in the PRIMNODE parameter (see below).

• PRIMNODEIndicates the name of the primary node of the groups and resilient applications for which you are performing a role switch. This parameter is only applicable when the GROUP parameter (see above) is set to *PRIMNODE.

ExamplesDMCHGROLE GROUP(GROUP1)

Changes the primary node to the backup node and the backup node to the primary node. The group or resilient application must have a status of *INACTIVE.

DMCHGROLE GROUP(*ALL)Changes the primary node to the backup node and the backup node to the primary node for all groups and resilient applications in the cluster. The groups or resilient applications must have a status of *INACTIVE.

UseYou can invoke the DMCHGROLE—Change Group Primary Node command on an active node in the cluster for groups or applications in *INACTIVE status.

Minimum Security LevelAdmin (*ADMIN)

Menu AccessF22 (Shift+F10) - Option 51Work With Groups screen - Option 20Work With Resilient Applications screen - Option 20

DMGENEXC—Generate Command ExceptionsDMGENEXC GENEXC( )

Use this command to generate an exception whenever an iCluster command fails or results in no action. For example, if you are running iCluster commands in a user exit program, this command allows you to detect command failures by generating an exception.

Note: This command generates an exception with an ID of CSI9954. For more information on what command caused this exception, examine the previous messages in the job log or in the event log for each node in the cluster.

If you want to enable this feature in your role switch user exit program, you must place the call to DMGENEXC GENEXC *YES before any calls to commands in your user exit program. When iCluster encounters the CSI9954 exception, role switch processing is stopped and declared incomplete.

Input Parameter• GENEXC

Indicates whether exception generation is enabled or disabled for commands that fail or result in no action.Specify one of the following values:



• *NO—Does not generate exceptions for commands that fail.• *YES—Generates exceptions for commands that fail.

The default setting for this parameter is *NO.

ExampleDMGENEXC GENEXC(*YES)

iCluster generates an exception for commands that fail or result in no action.

UseThe scope of this command is limited to the job in which it is run. iCluster disables exception generation when the job ends.




Replication Operation Commands

Replication Operation CommandsThis section contains the following topics:• DMSETPOS—Set Journal Start Position on page 225• DMMRKPOS—Mark Current Journal Positions on page 228• CHGHAJRN—Change Journal Receiver on page 229• DSPHAPOS—Display Journal Information on page 230• RTVHAPOS—Retrieve Journal Information on page 230• VFYHAJRN—Verify Audit Journal on page 232• STRHABRCD—Start BSF Recording on page 233• DSPHABRCD—Display Recording for BSF Object on page 234• ENDHABRCD—End Recording for BSF Object on page 234• DMSTRAPY—Start Replication Apply Process on page 235• DMENDAPY—End Replication Apply Process on page 236• DMACTOBJ—Activate Object on page 238• DMACTBSF—Activate BSF Object on page 240• DMSUSOBJ—Suspend Object on page 241• DMSUSBSF—Suspend BSF Object on page 242• DMSNDOBJ—Send Object Immediately on page 243• DMSETSYNC—Set Sync Point on page 245• CRTCFGOBJ—Create Configuration Objects on page 248• INITHAOBJ—Initialize Objects on page 249• RTVRCVPT—Retrieve Recovery Checkpoint on page 250• RTVRCVPTR—Retrieve Recovery Checkpoint (CL Program) on page 251• SYNCHATRG—Synchronize Triggers on page 252• DMLOGENT—Log Journal Entry on page 253

DMSETPOS—Set Journal Start PositionDMSETPOS GROUP( ) JRN( ) JRNRCV( ) JRNPOSLRG( ) STRDATE( ) STRTIME( ) JRNPOS( )

Use this command to position iCluster to start replication at a specific entry in a database journal, system audit journal, or all journals scraped by a group or resilient application.The specific journal entry can be entered directly or determined by the command if a date and time is entered.

Note: If you add an object specifier that maps to non-journaled objects, and then want to start mirroring after a save and restore of those objects, you should set the starting position for QAUDJRN. For non-journaled BSF objects, set the JOURNAL parameter to *NONE in the DMADDGRP—Add Group command.

If you add an object specifier that maps to journaled objects and want to start mirroring after performing a save and restore of those objects, then set the starting position for the default journal. For journaled BSF objects, set the JOURNAL parameter to *CLUSTER in the DMADDGRP—Add Group command.




The name of a replication group or resilient application. The starting entry in the journal will be set on the primary node in the recovery domain for the specified group or resilient application.The replication group or resilient application must be inactive when this command is invoked.

• JRNThe name of a database or system audit journal that has its starting position set through this command.Enter the name of a database journal, system audit journal or the following value:• *ALL—Sets starting positions for all journals that are being scraped by the replication group or resilient application

identified in the GROUP parameter of this command. If you specify the *ALL option, the following conditions must be true:• The JRNRCV parameter is set to *CURRENT or *CURCHAIN.• A specific date and time is specified in the STRDATE/STRTIME parameters OR the *LASTAPY or *CURSRCPOS

options are specified in the JRNPOS parameter.If you specify a database journal, then the database and audit journal starting positions are set for journaled objects selected for replication.If you specify the system audit journal, then the audit journal starting position is set for all non-journaled objects selected for replication.If you do not specify *ALL, the library where the journal resides must be identified. Prefix the journal with the name of the library where the journal is located (for example, LIB1/JRN1).

• JRNRCVThe name of a database or system audit journal receiver.Specify the name of a journal receiver or one of the following values:• *CURRENT—Specifies the current journal receiver for the specified journal.• *CURCHAIN—Specifies the journal receiver that is determined from the timestamp information of the starting entry

that is provided through the STRDATE and STRTIME parameters (see below).If you specify *CURCHAIN, you cannot specify a starting sequence number with the JRNPOS parameter. In this case, only the STRDATE and STRTIME parameters can be set.If you do not use the *CURCHAIN value, then you must specify the library where the journal receiver resides. Prefix the journal receiver with the name of the library where the journal receiver is located (for example, LIB1/JRNRCV1), or with the following value:• *LIBL—Specifies the set of libraries in your library list (the libraries are searched in order for the first occurrence of

the specified journal receiver).The default setting for this parameter is *LIBL.

• JRNPOSLRGThe sequence number (up to 20 digits) of the entry in the journal that iCluster will start processing from when replication resumes. Any non-blank value in this parameter takes precedence over the JRNPOS parameter.You cannot use this parameter with STRDATE or STRTIME or if JRNRCV is set to *CURCHAIN.Specify the starting sequence number or one of the following values:• *LASTAPY—Start journal processing at the last journal entry in the source journal that was applied to the target.

You cannot use this value for a group that has never been replicated. Instead, refresh the group when you start it for the first time.



• *CURSRCPOS—Start journal processing at the last journal entry in the source journal.• STRDATE

The date of the entry that iCluster will start processing from when mirroring is re-started. The date format must adhere to the journal's date format, which can be determined by displaying attributes for the attached journal receiver on the node.If you specify a starting date, the JRNRCV parameter (see above) can be set to special values.

• STRTIMEThe time of the entry that iCluster will start processing from when mirroring is re-started.This parameter is only applicable when a value is specified for the STRDATE parameter (see above).If you specify a starting time, the JRNRCV parameter (see above) can be set to special values.

• JRNPOSThe sequence number of the entry in the journal that iCluster will start processing from when replication resumes.You cannot use this parameter with STRDATE or STRTIME or if JRNRCV is set to *CURCHAIN.Specify the starting sequence number or one of the following values:• *LASTAPY—Start journal processing at the last journal entry in the source journal that was applied to the target.

You cannot use this value for a group that has never been replicated. Instead, refresh the group when you start it for the first time.

• *CURSRCPOS—Start journal processing at the last journal entry in the source journal.

ExamplesDMSETPOS GROUP(GRP1) JRN(QSYS/QAUDJRN) JRNRCV(LIB1/JRNRCV1) JRNPOSLRG(25689653214569)

The starting journal position is set for replication group GRP1.The starting position is set for journal QAUDJRN in library QSYS, and journal receiver JRNRCV1 in the same library.Journal processing starts at sequence number 25689653214569 in the specified journal when mirroring is re-started.

DMSETPOS GROUP(GRP1) JRN(QSYS/QAUDJRN) JRNRCV(LIB1/JRNRCV1) JRNPOS(142345)The starting journal position is set for replication group GRP1.The starting position is set for journal QAUDJRN in library QSYS, and journal receiver JRNRCV1 in the same library.Journal processing starts at sequence number 142345 in the specified journal when mirroring is re-started.

DMSETPOS GROUP(GRP2) JRN(QSYS/QAUDJRN) JRNRCV(*CURRENT) JRNPOS(*LASTAPY)The starting journal position is set for replication group GRP2.The starting position is set for journal QAUDJRN in library QSYS and the current journal receiver.Journal processing starts at the last journal entry in the source journal that was applied to the target.

DMSETPOS GROUP(GRP3) JRN(QSYS/QAUDJRN) JRNRCV(*CURCHAIN) STRDATE('10/01/03') STRTIME('12:45:00')The starting journal position is set for replication group GRP3.The starting position is set for journal QAUDJRN in library QSYS.The journal receiver is determined from the starting time and date information specified through the STRDATE and STRTIME parameters.Journal processing starts at the entry timestamped October 1, 2003 at 12:45 p.m. in all journals when mirroring is re-started.

DMSETPOS GROUP(GRP4) JRN(*ALL) JRNRCV(*CURCHAIN) STRDATE('10/01/03') STRTIME('12:45:00')The starting journal position is set for replication group GRP4.The starting position is set for all journals scraped by the group GRP4.


The journal receiver is determined from the starting time and date information specified through the STRDATE and STRTIME parameters.When mirroring is re-started, journal processing starts at the entry timestamped October 1, 2003 at 12:45 p.m. in all journals.

UseYou must issue this command on an active node in the cluster when the specified replication group or resilient application is inactive.



DMMRKPOS—Mark Current Journal PositionsDMMRKPOS GROUP( )

Use this command to establish the starting position for mirroring for the specified replication group or resilient application. The journal positions are journaled in the iCluster metadata on the primary node associated with the specified group or resilient application. Marking a journal position takes a snapshot of the state of the source system and the objects that match object specifiers at the time you issue this command.Marking journal positions can help prevent unsynchronized start-ups when starting mirroring. See the DMSTRREPL—Start Replication command for more information.The marked positions are used in the DMSTRGRP—Start Cluster Operations at Group and DMSTRAPP—Start Cluster Operations for Resilient Application commands. A parameter provided in these commands allows you to start mirroring at the journal positions marked by this command. Therefore, this command is typically used to record starting points before starting replication.This command will journal any objects (that have not already been journaled) that match object specifiers for the group to the default journal for the group. To journal objects to a journal other than the default journal, it is your responsibility to start journaling for the objects prior to running this command.

Note: Each time you issue this command, the previous results of this command are overwritten. Only those replication groups or resilient applications that are specified in the current invocation will have their journal positions overwritten in the iCluster metadata.



The name of an existing replication group or resilient application.

ExampleDMMRKPOS GROUP(GRP1)

Marks the current positions for journals that are used by replication group GRP1.

UseYou must issue this command on an active node in the cluster when the replication group is inactive or the resilient application is not running.





CHGHAJRN—Change Journal ReceiverCHGHAJRN JOURNAL( ) DLTRCV( ) DAYS( )

Use this command to change journal receivers for the system audit journal or a database journal on the primary node.This command can also delete fully processed receivers. A processed receiver is one where all of the journal entries in the receiver are completely applied to the backup node. In addition, you can specify the number of days that fully processed receivers must be older than before they are deleted.

Note: If the specified journal is used as a remote journal, this command deletes fully processed journal receivers on remote systems.

Note: If you are using this command for journals with remote journals attached, you should issue this command on the source system of the journal. In this situation, the journal must be *ACTIVE when you issue this command.

This journal cleanup procedure can be invoked while mirroring is active.

Input Parameters• JOURNAL

The name of the journal on the primary node for which you will be generating journal receivers.Note: If the journal is used for remote journaling, you must specify the name of the source journal.

You also need to identify the library where the journal resides. Prefix the journal with the name of the library where the journal receiver is located (for example, LIB1/JRN1).

• DLTRCVIndicates whether you want to delete fully processed receivers that are associated with the named journal.Specify one of the following values:• *NO—Specifies that you do not want to delete fully processed journal receivers associated with the named journal.

These will remain until you delete or archive them.• *YES—Specifies that you want to delete fully processed receivers and remote receivers associated with the named

journal.The default setting for this parameter is *NO.

• DAYSIndicates the number of days that fully processed journal receivers must be older than before they are deleted. You must specify a positive number.

Note: The age of the fully processed journal receivers is based on the detach date and time attributes of the journal receiver.

Specify the number of days or the following value:• *NONE—Specifies that the age of the receivers is not considered when deleting fully processed journal receivers.

The default setting for this parameter is *NONE.

ExamplesCHGHAJRN JOURNAL (LIB1/HADJRN) DLTRCV(*YES) DAYS(4)

Generates new receivers for the default journal HADJRN located in the library LIB1 and deletes fully processed receivers older than 4 days.


CHGHAJRN JOURNAL (LIB1/JRN1) DLTRCV(*NO) DAYS(*NONE)Generates new receivers for the journal JRN1 located in the library LIB1. Fully processed receivers will not be deleted.

UseDataMirror recommends that you schedule journal management procedures to run at least once a day and if you are dealing with high volumes, you should schedule it to run several times daily. This assures that journal receivers can be purged off the system if they become too large or grow too quickly.You must issue this command on the node that the operation will be performed on. The node does not have to be active in the cluster.


DSPHAPOS—Display Journal InformationDSPHAPOS JOURNAL( )

Use this command to display the journal sequence number, journal receiver and library, and replication group of the oldest position in the journal that iCluster is currently processing.You can also use this command to determine whether a journal receiver associated with the specified journal is being used by iCluster or to let the user know which receivers may be deleted. Any receiver that is older than the reported position is no longer required by iCluster and can be deleted.

Input Parameter• JOURNAL

The name of a journal being used by iCluster.The library where the journal resides must be identified. Prefix the journal with the name of the library where the journal receiver is located (for example, LIB1/JRN1).

ExampleDSPHAPOS JOURNAL(LIB1/JRN1)

This command displays the journal sequence number, the journal receiver name and the journal receiver library for JRN1 (if it exists) in library LIB1.

UseYou must issue this command on the node where the journal resides. The node does not have to be active in the cluster.


RTVHAPOS—Retrieve Journal InformationRTVHAPOS JOURNAL( ) TARGET( ) GROUP( ) JRNENTLRG( ) JRNENTRY( ) JRNRCVNME( ) JRNRCVLIB( ) RESULT( )

Use this command to return the journal receiver, the library of the journal receiver, and the sequence number of the last fully processed entry for the specified journal. The node and replication group associated with the journal entry are also returned.Use this command to determine whether a journal receiver associated with the specified journal is being used by iCluster. Any receivers in the chain prior to the returned journal receiver that may have been fully processed and may be deleted.

Note: Since this command uses return variables, it can only be executed from a CL program. Use the DSPHAPOS—Display Journal Information command to obtain the same information from the command line.



Input Parameters• JOURNAL

The name of a journal.The library where the journal resides must be identified. Prefix the journal with the name of the library where the journal is located (for example, LIB1/JRN1).

• TARGETThe name of a 10 character CL variable into which the name of the backup node for the group associated with the last fully processed entry for the specified journal will be copied.

• GROUPThe name of a 10-character CL variable into which the name of the replication group associated with the last fully processed journal entry for the specified journal will be copied.

• JRNENTLRGThe name of the CL program variable into which the sequence number of the last fully processed journal entry is copied. The variable must be a character variable with a length of 20.Both the JRNENTLRG and JRNENTRY parameters will return the last applied journal position if the journal position is less than 10000000000. However, if the journal position is 10000000000 or greater, only the JRNENTLRG parameter will be returned, and the JRNENTRY parameter will return the decimal value -1.

• JRNENTRYThe name of the CL program variable into which the sequence number of the last fully processed journal entry is copied. The variable must be a decimal variable that has a length of 10 positions with no decimal positions.JRNENTRY returns the last applied journal position if the journal position is less than 10000000000, and returns -1 if the journal position is 10000000000 or greater.

• JRNRCVNMEThe name of a 10-character CL variable into which the name of the journal receiver that contains the last fully processed journal entry for the specified journal will be copied.

• JRNRCVLIBThe name of a 10-character CL variable into which the name of the library that contains the journal receiver identified through the JRNRCVNME parameter (see above) will be copied.

• RESULTThe name of a 10-character CL variable where the result of the command will be copied.If the command runs successfully, the result variable will contain the string '*SUCCESS'. Otherwise, the result variable will contain the string '*ERROR'.

ExampleRTVHAPOS JOURNAL(LIB1/JRN1) TARGET(&NODE) GROUP(&GRPVAR) JRNENTRY(&JRNEVAR) JRNRCVNME(&JRNRVAR) JRNRCVLIB(&JRNLVAR) RESULT(&RESVAR)

This command retrieves the journal receiver, the library of the journal receiver, and the last fully processed journal entry for journal JRN1 in library LIB1. This information is received through the variables &JRNRVAR, &JRNLVAR, and &JRNEVAR, respectively.The node and replication group associated with the sequence number are returned through the variable &GRPVAR.The backup node associated with the sequence number is returned through the variable &NODE.

Note: DataMirror recommends that you use the CL convention of '&' in front of the variable names.


UseYou must issue this command on the node where the journal resides. The node does not have to be active in the cluster.


VFYHAJRN—Verify Audit JournalVFYHAJRN AUDQTEMP( ) AUDSPLF( )

Use this command to verify that the audit journal exists in the library QSYS and the audit related OS/400 system values are set appropriately for iCluster. The VFYHAJRN command can only be used if the profile that is running the command has *ALLOBJ or *SECOFR authority.After installing iCluster on a primary node, it is required that the level of security auditing on the system is appropriately set. It is important that you issue this command before starting iCluster.

Note: This command sets two OS/400 system values (QAUDLVL, QAUDCTL) that are required for replication. Other system values that have been set for QAUDLVL and QAUDCTL are maintained and not affected by this command.

The VFYHAJRN command verifies that the audit journal exists in the library QSYS. The required system values for parameter QAUDLVL (security auditing level) are as follows:• *CREATE• *DELETE• *OBJMGT• *SAVRST• *SECURITY• *SPLFDTA (if the AUDSPLF parameter is set to *YES).

This command also activates object-level auditing before using iCluster. The required system values for parameter QAUDCTL (auditing control) are as follows:• *OBJAUD• *AUDLVL• *NOQTEMP (if the AUDQTEMP parameter is set to *YES).

Specifying *OBJAUD means that objects selected for audit via the CHGOBJAUD command will be audited. *AUDLVL ensures that object auditing changes controlled by the QAUDLVL system value will be performed. See the AUDQTEMP parameter for this command to obtain more information about *NOQTEMP.

This command will also ensure that the new object auditing level is set correctly for iCluster.

Input Parameters• AUDQTEMP

Specifies whether you want to audit objects in the library QTEMP. Warning: DataMirror recommends that you avoid the auditing of objects in this library by specifying *NO for this

parameter.Specify one of the following values:• *NO—Does not audit objects in the library QTEMP by specifying *NOQTEMP for QAUDCTL.• *YES—Audits objects in the library QTEMP by not specifying *NOQTEMP for QAUDCTL.




• AUDSPLFSpecifies whether you want to audit spool file functions (create spool file, delete spool file, display spool file, and so on).Specify one of the following values:• *NO—Does not audit spool file functions.• *YES—Audits spool file functions.


ExampleVFYHAJRN AUDQTEMP(*YES) AUDSPLF(*NO)

Verifies that the audit journal exists.Objects in the library QTEMP will not audited (the system value *NOQTEMP is set for QAUDCTL).Spool file functions will not be audited.

UseYou must issue this command on the node where the operation will be performed. The node does not have to be active in the cluster.


STRHABRCD—Start BSF RecordingSTRHABRCD PATH( )

Use this command to start journaling a BSF object referenced by a path object specifier to the default BSF journal for the cluster specified in the DMSETSVAL—Set Cluster System Values command. Note that you cannot specify generics for this parameter.Generally, iCluster starts journaling a BSF object automatically so that you do not need to do so yourself. However, in uncommon circumstances where you want to start journaling yourself (such as starting journaling in advance of starting iCluster, or assuring that files are not changed before starting iCluster), then you can issue this command to start journaling.You can also use the IBM STRJRN command to start BSF journaling. See your iSeries documentation for more information.

Note: This command starts journaling for a single file only.

Input Parameters• PATH

The full object path in the format of /Dir1/Dir2/file.

OutputIf journaling starts correctly, no messages are displayed.Error messages will be displayed if journaling cannot start.

ExamplesSTRHABRCD PATH(/Dir1/Dir2/file1)

This starts journaling the BSF object named file1 that resides in the base path of /Dir1/Dir2/.




DSPHABRCD—Display Recording for BSF ObjectDSPHABRCD PATH( )

Use this command to display information about the journaling of BSF objects. If journaling is active, then this command displays the journal where the specified BSF objects are being journaled. If journaling is not active, then this command will indicate its inactive state.For more information about BSF objects, see Replicating BSF Objects on page 89.

Input Parameter• PATH

The path that identifies the location of the BSF object for which journaled information will be displayed.

ExampleDSPHABRCD PATH(/DIR1/DIR2/DIR3/FILEA)

Displays information about the journaling of BSF object FILEA that resides in /DIR1/DIR2/DIR3/.



ENDHABRCD—End Recording for BSF ObjectENDHABRCD PATH( )

Use this command to end journaling of a Byte Stream File (BSF) object to its journal.You can also use the IBM ENDJRN command to end BSF journaling. See your iSeries documentation for more information.

Input Parameter• PATH

The path that identifies the location of the BSF object that will no longer be journaled to the default BSF journal.

ExampleENDHABRCD PATH(/DIR1/DIR2/DIR3/FILEA)

Ends journaling for the BSF object FILEA that resides in /DIR1/DIR2/DIR3/.

UseYou must issue this command on the node when the operation will be performed. The node does not have to be active in the cluster.




DMSTRAPY—Start Replication Apply ProcessDMSTRAPY GROUP( ) BACKUP( ) OVRSTOP( ) FRCDRN( )

This command starts the apply processes on a backup node. Starting apply processes allow journal entries placed in the staging store by receive processes to be applied. For more information about staging stores, see Staging Objects for more information.The apply processes handle each journal entry in the staging store in chronological order. They remain active until the stop date/time is specified through the previous or subsequent DMENDAPY—End Replication Apply Process command. The apply processes also end when the staging store has been drained and the send/receive processes are not running.


The name of the replication group or resilient application containing the backup node that will have its apply processes started.Specify the name of the group or resilient application or one of the following values:• *ALL—Starts the apply processes on the backup node for all groups and resilient applications in the cluster.• *BACKUP—Starts the apply processes for all groups and resilient applications on the backup node. If you specify

this option, you must specify the name of the backup node in the BACKUP parameter (see below).• BACKUP

Indicates the name of the backup node that starts the apply processes for all groups and resilient applications that have the specified node as their backup node. This parameter is only applicable when the GROUP parameter (see above) is set to *BACKUP.Specify the name of the backup node or the following value:• *ALL—Starts the apply processes for all groups and resilient applications on all backup nodes in the cluster.

The default setting for this parameter is *ALL.• OVRSTOP

Overrides a previously set stop date/time for the apply processes that is specified through the DMENDAPY—End Replication Apply Process command.Specify one of the following values:• *YES—The previous stop date/time is disregarded. The apply processes will continue until a new DMENDAPY—

End Replication Apply Process command is issued.• *NO—The apply processes will start, but the previous stop date/time specified through the DMENDAPY—End

Replication Apply Process command is still valid. Consequently, the processes will stop at that date/time unless it is reset by a subsequent invocation of the DMENDAPY—End Replication Apply Process command.

The default setting for this parameter is *YES.• FRCDRN

Specifies whether the staging store will be force drained.The apply processes merge entries from the audit and database journals. Depending on your selection, the apply processes will either continue draining the staging store even if one of the journals is empty (known as force draining), or it will stop draining when one of the journals is empty.Force draining stops once it reaches the stop date/time specified through the DMENDAPY—End Replication Apply Process command (if one is specified). Otherwise, it will stop once the staging store is empty.Specify one of the following values:


• *NO—Indicates that the staging store will not be force drained. The apply processes will merge entries from the audit and database journals and apply them in sequence. When one journal becomes empty, the apply processes will stop regardless of any entries remaining in the other journal.

• *YES—Indicates that the staging store will be force drained. The apply processes will merge entries from the audit and database journals. Once it reaches the last entry of the journal with fewer entries, the merge will stop, and the apply processes will drain the other journal until it is empty.


ExampleDMSTRAPY GROUP(GRP1) OVRSTOP(*YES) FRCDRN(*YES)

Starts the apply processes on the backup node for replication group GRP1.The staging store will be force drained.

DMSTRAPY GROUP(*BACKUP) BACKUP(AS400BU) OVRSTOP(*YES) FRCDRN(*YES)Starts the apply processes for all groups and resilient applications that have the AS400BU backup node.The staging store is force drained.

UseYou can issue this command on any node in the cluster.



DMENDAPY—End Replication Apply ProcessDMENDAPY GROUP( ) BACKUP( ) OPTION( ) ENDDATE( ) ENDTIME( )

Use this command to end applying journal entries from the staging store on a backup node. For more information about staging stores, see Staging Objects on page 69.Even though this command stops the apply processes, iCluster continues to scrape and send journal entries for the replication group if it is active. The journal entries remain in the staging store until the apply processes are started through the DMSTRAPY—Start Replication Apply Process command.


The name of the replication group or resilient application containing the backup node that will have its apply processes stopped by this command.Specify the name of the group or resilient application or one of the following values:• *ALL—Ends the apply processes on the backup node for all groups and resilient applications in the cluster.• *BACKUP—Ends the apply processes for all groups and resilient applications that have the specified node as their

backup node. If you specify this option, you must specify the name of the backup node in the BACKUP parameter (see below).

• BACKUP



Indicates the name of the backup node that will end the apply processes for all groups and resilient applications. This parameter is only applicable when the GROUP parameter (see above) is set to *BACKUP.Specify the name of the backup node or one of the following values:• *ALL—Ends the apply processes for all groups and resilient applications on all backup nodes in the cluster.

The default setting for this parameter is *ALL.• OPTION

Indicates how you want to stop the apply processes.Specify one of the following values:• *CTRLD—Indicates a controlled end of the apply processes, allowing them to complete their current processing.

DataMirror recommends performing a controlled end when possible.• *INVLD—Invalidates the pending end date/time for the apply processes. Applies will continue.• *DATETIME—Indicates the date and time when the apply processes will end. If you select this setting, you must

specify values for the ENDDATE and ENDTIME parameters (see below).• *IMMED—Indicates that the apply processes will stop immediately.

The default setting for this parameter is *CTRLD.Note: If either *CTRLD or *DATETIME is specified, the apply processes will stop once the current operation has

been completed. Depending on the operation (for example, restoring a save file), it may take some time for the apply processes to stop.

• ENDDATEThe date when the apply processes will be stopped. The date format must adhere to the system's date format.This parameter is only applicable if the OPTION parameter (see above) is set to *DATETIME.

• ENDTIMEThe time when the apply processes will be stopped. The time format must adhere to the system's time format.This parameter is only applicable if the OPTION parameter (see above) is set to *DATETIME.

ExamplesDMENDAPY GROUP(GRP1) OPTION(*CTRLD)

Ends the apply processes on the backup node in replication group GRP1.The apply processes are stopped in a controlled manner after the current operation has been completed.

DMENDAPY GROUP(*ALL) OPTION(*DATETIME) ENDDATE(12/31/05) ENDTIME(235959)Ends the apply processes for all groups and resilients applications on all backup nodes in the cluster.Stops the apply processes at the specified date and time.

UseYou can issue this command on any active node in the cluster.


Menu Access F22 (Shift+F10) - Option 70


DMACTOBJ—Activate ObjectDMACTOBJ GROUP( ) OBJ( ) OBJTYPE( ) MEMBER( ) RFSH( ) TRUNCATE( ) REPLACELFS( )

Use this command to activate an object that is currently suspended from replication. This command will also enable replication of objects that have been added into replication scope.After activating an object, mirroring of the specified object will start if the replication group is active. If the replication group is inactive, mirroring will start when the group is activated through the DMSTRGRP—Start Cluster Operations at Group command.

Note: Any changes made to an object while it is suspended will not be mirrored when it is activated through this command. Specify *YES for the RFSH parameter to verify that the object on the backup node is synchronized with the object on the primary node before activating the object for mirroring.

For more information about suspended objects, see Suspended Objects on page 72.


The name of the replication group to which the object is selected.• OBJ

The object name component of the suspended object that you want to activate.Enter a specific or generic name (to identify multiple objects in a library), or the following value:• *ALL—Specifies all objects in a library.

The library where the object resides must be identified. Prefix the object with the name of the library where the object is located (for example, LIB1/OBJ1).

Note: If you specify a generic name for this parameter, only *ALL is allowed for the MEMBER parameter. If you specify a specific name for this parameter, both *ALL and a specific name are allowed for the MEMBER parameter.

• OBJTYPEThe type of object being activated.Enter an object type supported by iCluster or the following value:• *ALL—Specifies all objects in the library which has the same object name component (OBJ parameter) but different

types.See Object Types Replicated by iCluster on page 529 for more information on object types supported by iCluster.

• MEMBERThis parameter is available only if you entered *FILE in the OBJTYPE parameter and the file is a source physical file.Specifies whether you want to activate a specific member of a source physical file. Enter a member name or the following value:• *ALL—Specifies all members for the specified source physical file.

• RFSHIndicates whether you want to refresh the suspended object to the backup node in the replication group when the object is activated. The refresh method used is the one that was specified when the object was selected for replication within the group. See DMSELOBJ—Select Objects to Group on page 141 for more information.This option allows you to activate an object without having to save and restore the object.Specify one of the following values:



• *NO—Indicates that you do not want the object to be refreshed on the backup node when the object is activated. In this case, you are responsible for refreshing the object and ensure that it is synchronized at the time the activate is performed.

• *YES—Indicates that you want the suspended object to be refreshed on the backup node when the object is activated.

The default setting for this parameter is *YES.• TRUNCATE


behavior for iCluster and is appropriate for standard iCluster replication. You can use this option for a master-master group if the data on the source system is known to be complete in order to ensure a synchronized starting point.With this value, logical files are replaced automatically because the deletion of a physical file requires the deletion of all logical files upon which it is based.










ExamplesDMACTOBJ GROUP(GRP1) OBJ(LIB1/OBJ1) OBJTYPE(*FILE) RFSH(*YES) TRUNCATE(*NO) REPLACELFS(*NO)

Activates the *FILE object named OBJ1 located in library LIB1 that is selected for replication within replication group GRP1.OBJ1 will be refreshed on the backup node before mirroring is started. Existing logical files are not used during the refresh, and existing data is not removed before performing the refresh.

DMACTOBJ GROUP(GRP1) OBJ(LIB1/OBJ1) OBJTYPE(*FILE) MEMBER(MEM1)


Activates the member MEM1 of object named OBJ1 that is located in the library LIB1 and is part of the group GRP1.DMACTOBJ GROUP(GRP1) OBJ(LIB1/A*) OBJTYPE(*ALL) MEMBER(*ALL)

Activates all objects located in library LIB1 and have 'A' as their first letter. These objects are also in the replication scope of GRP1.

DMACTOBJ GROUP(GRP1) OBJ(LIB1/*ALL) OBJTYPE(*ALL) MEMBER(*ALL) RFSH(*YES)Activates all objects located in library LIB1. These objects are also in the replication scope of GRP1.




DMACTBSF—Activate BSF ObjectDMACTBSF GROUP( ) PATH( ) TRUNCATE( )

Use this command to activate a suspended Byte Stream File (BSF) object that is currently suspended. It applies only to BSF objects residing in the Integrated File System (IFS). For more information about BSF objects, see Replicating BSF Objects on page 89.After activating a BSF object, it is refreshed to the backup node in the recovery domain if mirroring is active. The amount of latency in the system will also affect the refresh of a BSF object. Mirroring of the specified object will start if the replication group is active. If the replication group is inactive, mirroring will start when the group is activated through the DMSTRGRP—Start Cluster Operations at Group command.For more information about suspended objects, see Suspended Objects on page 72.


The name of the replication group to which the BSF object is selected.• PATH

The path object specifier that identifies the location of the BSF object that will be activated through this command.The path must be enclosed in single quotes and start with the '/' (forward slash) character ('/Dir3/Dir4/file'). The path can be a maximum of 5000 characters, and you must enter at least two characters for the path.Generic path names of the form '/mydir*' are supported, where the generic indicator '*' is the final character of the path name. When using generic path names, all sub-directories are included recursively.

Note: Only those objects that match the generic path specifier and are replicated by the group will be activated. For example, matching objects that are excluded from replication by the group will not be activated.

For more information about path object specifiers, see Path Object Specifiers on page 53.• TRUNCATE

Indicates whether you want to remove existing data before performing a refresh.Specify one of the following values:



• *YES—Indicates that you want to remove and recreate existing data files as part of a refresh. This is the default behavior for iCluster and is appropriate for standard iCluster replication. You can use this option for a master-master group if the data on the source system is known to be complete in order to ensure a synchronized starting point.With this value, logical files are replaced automatically because the deletion of a physical file requires the deletion of all logical files upon which it is based.




ExamplesDMACTBSF GROUP(GRP1) PATH('/DIR1/DIR2/DIR3/FILEA')

Activates the BSF object named FILEA in /DIR1/DIR2/DIR3 that is selected for replication within replication group GRP1.FILEA will be refreshed to the backup node in the replication group.

DMACTBSF GROUP(GRP2) PATH('QDLS/DIR1/DIR2/*')Activates all BSF objects in /DIR1/DIR2 that are selected for replication within replication group GRP2.The objects are refreshed to the backup node in the recovery domain if mirroring is active.




DMSUSOBJ—Suspend ObjectDMSUSOBJ GROUP( ) OBJ( ) OBJTYPE( ) MEMBER( )

Use this command to prevent an object from being replicated to a backup node in the specified replication group by suspending it.Journal entries created before an object was suspended will still be sent to the backup node.To activate an object that is suspended through this command, use the DMACTOBJ—Activate Object command.

Note: Any changes made to an object while it is suspended will not be mirrored when it is activated.For more information about suspended objects, see Suspended Objects on page 72.


The name of the replication group to which the object is selected.• OBJ


The name of a valid object.The library where the object resides must be identified. Prefix the object with the name of the library where the object is located (for example, LIB1/OBJ1).

• OBJTYPEThe type of object being suspended.

• MEMBER This parameter is available only if you entered *FILE in the OBJTYPE parameter.Specifies whether you want to suspend a specific member of a source physical file.Enter a member name or the following value:• *ALL—Specifies all members for the specified source physical file.

ExampleDMSUSOBJ GROUP(GRP1) OBJ(LIB1/OBJ1) OBJTYPE(*FILE) MEMBER(MEM1)

Suspends the member MEM1 of the *FILE object named OBJ1 located in library LIB1 that is selected for replication within replication group GRP1.




DMSUSBSF—Suspend BSF ObjectDMSUSBSF GROUP( ) PATH( )

Use this command to prevent a BSF object from being replicated to a backup node. The object will appear suspended with the SBU reason code. It applies only to BSF objects residing in the Integrated File System (IFS). For more information about BSF objects, see Replicating BSF Objects on page 89.Journal entries created before the object was suspended will still be sent to the backup node if the BSF object is journaled.To activate a BSF object that is suspended through this command, use the DMACTBSF—Activate BSF Object command.

Note: This command supports the suspension of directories and folders with the exception of those directories that match a BSF specifier with matching files that are journaled. See the DMADDBSF—Add Path Object Specifier to Group command for more information on journaled and non-journaled BSF files.

For more information about suspended objects, see Suspended Objects on page 72.


The name of the replication group to which the BSF objects are selected.• PATH

The path object specifier that identifies the location of the BSF objects that will be suspended through this command.The path must be enclosed in single quotes and start with a '/' (forward slash) character ('/Dir3/Dir4/file'). The path can be a maximum of 5000 characters, and you must enter at least two characters for the path.



For more information about path object specifiers, see Path Object Specifiers on page 53.

ExamplesDMSUSBSF GROUP(GRP1) PATH('/DIR1/DIR2/DIR3/FILEA')

Suspends the BSF object named FILEA in /DIR1/DIR2/DIR3 that is selected for replication within replication group GRP1.DMSUSBSF GROUP(GRP2) PATH('/DIR1/DIR2/FILEB')

Suspends the BSF object named FILEB in /DIR1/DIR2 that is selected for replication within replication group GRP2.




DMSNDOBJ—Send Object ImmediatelyDMSNDOBJ SRCNODE( ) TGTNODE( ) SNDTYPE( ) OBJ( ) OBJTYPE( ) OBJATTR( ) TGTLIB( ) PATH( )

Use this command to replicate one or more objects that are not necessarily part of a group to a target system immediately. This command allows you to send objects immediately without having to select them in a group. The referenced objects are essentially refreshed to the target node.The DMSNDOBJ command can be used to synchronize the backup node with the primary node before replication is started. Once replication is started, iCluster should detect the creation of new objects in replication scope and replicate them automatically. However, if replication is running and you have determined that some objects in replication scope are not found on the backup node, the DMSNDOBJ command should NOT be used to send these objects to the backup node if they are journaled objects. By doing this you will prevent these objects from being properly handled by the replication process, that is, setting up the metadata that iCluster requires for changes to these objects to be replicated and starting journaling of the objects if they are not journaled. The correct way to deal with this issue is to use the Activate Object command to activate and refresh the objects from the group's primary node to the backup node.

Note: Journaled objects are physical database files, logical files, data areas and data queues.Note: This command cannot be used to refresh data areas and data queues when the source object is at OS/400

Version 5.1 or later. You can use DMACTOBJ—Activate Object on page 238 to refresh data areas and data queues that are replicated by an iCluster group.

Input Parameters• SRCNODE

The name of the node where the source object is found.Enter the name of a node or the following value:• *CURRENT—The current node.

This is the default value for this parameter.• TGTNODE

The name of the node where the object will be sent.• SNDTYPE

The object type being sent.


Specify one of the following values:• *LIBRARY—A native OS/400 object that resides in an OS/400 library.• *PATH—A path object.

• OBJThe object name and library component of the object that you want to send.Enter a specific or generic name (to identify multiple objects in a library), or the following value:• *ALL—Specifies all objects in a library.

The library where the objects reside must be identified. Prefix the object specification with the name of the library where the objects are located (for example, LIB1/OBJ1).This parameter is used only when the SNDTYPE parameter is *LIBRARY.

• OBJTYPEThe object type component of the object that you want to send.Specify an object type or the following value:• *ALL—Specifies all object types.

This is the default setting for this parameter.For a complete list of all object types that iCluster can replicate, see Object Types Replicated by iCluster on page 529.This parameter is used only when the SNDTYPE parameter is *LIBRARY.

• OBJATTRThe attribute component of the object you want to send.This parameter is only applicable when the OBJTYPE parameter (see above) is set to *FILE.This field is free-form. Consequently, you can enter any value you want to describe the object, as long as the value conforms to OS/400 naming conventions. However, there are values that have special meaning to iCluster. They are PFSRC (source physical file), PFDTA (data physical file), and PF (any physical file). Other values listed are standard OS/400 file sub-types. If PF is used, the object specifier will match either PFSRC or PFDTA files.Press F4 for a list of values or enter the following value:• *ALL—Specifies all object attributes. *ALL is not a valid OS/400 object attribute but allows iCluster to gather all

objects regardless of their attribute.The default setting for this parameter is *ALL.This parameter is used only when the SNDTYPE parameter is *LIBRARY.

• TGTLIBThe name of the library on the backup system that will receive the replicated objects.Enter the name of the library or the following value:• *SRC—Sets the target library so that it is the same as the primary node library where the object resides.

If the primary and backup environments reside on the same physical system (local loopback implementation), the target library that you specify cannot be the same library where a selected object in the group currently resides. This restriction means that iCluster does not permit the special value *SRC in this situation.This parameter is used only when the SNDTYPE parameter is *LIBRARY.

• PATHThe path object specifier that identifies the location of the BSF objects.



Path object specifiers consist of a sequence of directories that identify the location of the BSF objects in the IFS. The path that is defined is similar to the path specification under Windows and UNIX operating systems. The defined path must be in single quotes. Generic path names of the form '/mydir*' are supported, where the generic indicator '*' is the final character of the path name. When using generic path names, all sub-directories are included recursively.For more information about path object specifiers, see Path Object Specifiers on page 53.This parameter is used only when the SNDTYPE parameter is *PATH.

OutputNone.

ExamplesDMSNDOBJ SRCNODE(*CURRENT) TGTNODE(TGT1) SNDTYPE(*LIBRARY) OBJ(LIB1/OBJ*) OBJTYPE(*FILE) OBJATTR(*ALL) TGTLIB(*SRC)

This command will cause all FILE objects in library LIB1 that match the generic specifier OBJ* to be refreshed from the current node to the node named TGT1, into the LIB1 library.

DMSNDOBJ SRCNODE(SRC1) TGTNODE(TGT1) SNDTYPE(*PATH) PATH('/home/mydir/ex*')This command will cause all path objects that in the /home/mydir directory that match the generic specifier 'ex*' to be refreshed from the node named SRC1 to the node named TGT1.

UseThe nodes named SRCNODE and TGTNODE must be active in the cluster.

DMSETSYNC—Set Sync PointDMSETSYNC GROUP( ) USREXIT( ) SCRAPE( ) RECEIVE( ) APPLY( ) USRDATA( )

Use this command to place checkpoint entries in journals that define sync points when journal entries are scraped and applied on the primary and backup nodes.Once the checkpoint condition is met, a user exit program can be called by one of the replication jobs on the primary or backup node. For scrape and apply jobs, the checkpoint condition implies that all jobs have reached the checkpoint and are waiting for the user exit program to be completed. Depending on the return code of the user exit program, these jobs will continue replication beyond the checkpoint or complete. For the receive jobs, the checkpoint condition is met when all the jobs except the last one have passed the checkpoint. In this case, it is the last job that fires the user exit program.

Note: A replication job is any iCluster replication job associated with replication group or resilient application activities.

You can use checkpoint journal entries when it is necessary to synchronize operations on primary and backup nodes. After defining a checkpoint journal entry on the primary node, synchronization is achieved when the checkpoint is reached on the backup node. A user exit program can then be called to perform some operation with the user-defined data that is passed to the program. For example, you may want to backup a consistent database or perform summary calculations after all entries have been replicated from the primary node and applied on the backup node.

Note: Note that a replication apply job on a backup node processes journal entries until it reaches the checkpoint journal entry that is generated by this command. At this point, the job waits for other specified replication jobs to reach this checkpoint journal entry. Synchronization is achieved when all active replication jobs reach the checkpoint journal entry. This means that one or more jobs will wait at the checkpoint journal entry until all replication jobs reach the sync point.

Note the following important considerations concerning the DMSETSYNC command:• After issuing the DMSETSYNC command, you cannot delete or change the checkpoint journal entry you defined.

Therefore, it is important that you are aware of this consideration and exercise caution before issuing this command.


• If the apply job has a checkpoint user exit, the update continues up to the point where you set the checkpoint. The update is resumed after the user exit program runs to completion, unless a special value is returned through one of the user exit program arguments that causes the apply job's completion. See Passing Arguments to Sync Point User Exit Programs on page 86.

Therefore, it may be necessary to minimize the execution time of your user exit program so that mirroring can resume as soon as possible.


The name of the replication group or resilient application that will synchronize at a checkpoint journal entry. Jobs associated with the specified replication group or resilient application will synchronize at the checkpoint journal entry. If you specify a group, it must be active. If you specify a resilient application, it must be running.

• USREXITThe name of the user exit program that will be invoked when all active replication jobs reach the checkpoint journal entry.Note that the same user exit program can be invoked at different synchronization points (journal entry scrape, receive, and apply). If it is invoked when journal entries are being scraped, the program must reside on the primary node. If it is also invoked at journal receive or apply times, it must also reside in the same location on the backup node. If you want to specify a different user exit program at multiple sync points, issue the DMSETSYNC command (once for each sync point) so that a different user exit program can be specified for each invocation of the command.

Note: User exit programs must be compiled with the Use adopted authority option set to *NO. Otherwise, the user exit programs cannot be executed and mirroring will continue.

Specify the name of a user exit program or the following value:• *NONE—Indicates that you do not want to specify a user exit program.

The library where the user exit program resides must be identified if you do not specify *NONE. Prefix the user exit program with the name of the library where the program is located (for example, LIB1/CHKPGM) or with the following value:• *PRODLIB—Specifies your iCluster installation library.

If iCluster cannot find the specified user exit program, an appropriate error message will be generated and mirroring will continue.See Passing Arguments to Sync Point User Exit Programs on page 86 for more information about the arguments that can be passed to the user exit program.

• SCRAPEIndicates whether you want to synchronize group replication jobs when journal entries are scraped on the primary node.At this sync point, the user exit program specified through the USREXIT parameter (see above) will be invoked when all active scrape jobs on the primary node scrape process synchronize at the checkpoint journal entry.You can use a user exit to end mirroring by setting a return value. Setting the value for the scrape affects both the scrape and receive processes. The apply process is not affected.Specify one of the following values:• *NO—Does not synchronize group replication jobs at checkpoint journal entry. The user exit program is not

invoked.• *YES—Synchronizes group replication jobs at checkpoint journal entry and then invokes user exit program.

The default setting for this parameter is *NO.Note: If the group uses a remote journal, then you must set this value to *NO.

• RECEIVE



Indicates whether you want the user exit program to be called when all the receive jobs have passed through the sync point.

Note: Unlike the scrape and apply jobs, the receive jobs are not synchronized at sync point journal entries. The receive jobs pass through the sync point at their own pace. The user exit program specified through the USEREXIT parameter will be called asynchronously by the receive job that is the last one to receive a sync point journal entry.

Unlike the scrape and apply jobs, you cannot end the receive jobs using a specific return code from the user exit program. However, the receive jobs will stop automatically if the scrape jobs are stopped.The possible values are:• *NO—The user exit program will not be called when all the receive jobs have passed the sync point (received sync

point journal entry).• *YES—Invokes the sync point user exit program.

• APPLYIndicates whether you want to synchronize replication jobs when journal entries are applied on the backup node.At this sync point, the user exit program specified through the USREXIT parameter (see above) will be invoked when all active replication jobs for the backup node apply processes synchronize at the checkpoint journal entry.You can use a user exit to end the apply jobs by setting a return value. Setting this value for the apply process affects only the apply process.Specify one of the following values:• *YES—Synchronizes group replication jobs at the checkpoint journal entry and then invokes user exit program.• *NO—Does not synchronize group replication jobs at the checkpoint journal entry and then the user exit program is

not invoked.The default setting for this parameter is *YES.

• USRDATAIdentifies the user-defined data that you want to pass to the user exit program specified through the USREXIT parameter (see above).You can pass a maximum of 400 bytes of data to the user exit program. If your data contains spaces or non-alphanumeric characters (commas, periods, and so on), you must enclose your data in single quotes.

Note: This parameter is ignored if no user exit program is specified.See Passing Arguments to Sync Point User Exit Programs on page 86 for more information.

ExamplesDMSETSYNC GROUP(GRP1) USREXIT(LIB1/CPUPGM) SCRAPE(*NO) APPLY(*YES) USRDATA(QTY400STS1PRC800)

Jobs associated with replication group GRP1 will be synchronized at the checkpoint journal entries.The checkpoint user exit program CPUPGM is located in library LIB1. This program must reside on the backup node.Replication jobs will only be synchronized when journal entries are applied on the backup node. No sync points are established for journal scraping.User-defined data consisting of a sequence of characters will be passed to the user exit program CPUPGM.

DMSETSYNC GROUP(GRP2) USREXIT(*NONE) SCRAPE(*YES) APPLY(*YES)Jobs associated with replication group GRP2 will be synchronized at the checkpoint journal entries.A user exit program has not been specified in this command.Replication jobs will be synchronized when journal entries are scraped on the primary node as well as applied on the backup node.


DMSETSYNC GROUP(GRP3) USREXIT(*PRODLIB/CPUPGM) SCRAPE(*YES) APPLY(*YES) USRDATA('DJONES 750098 ASMITH 912457')

Jobs associated with replication group GRP3 will be synchronized at the checkpoint journal entries.The checkpoint user exit program CPUPGM is located in the iCluster installation library. This program must reside in the same location on both the primary and backup nodes.Replication jobs will be synchronized when journal entries are scraped on the primary node and applied on the backup node. User-defined data consisting of a sequence of characters will be passed to the user exit program CPUPGM. Note that single quotes are required to enclose data that includes spaces and other non-alphanumeric characters.

UseYou must issue this command on an active node in the cluster when the replication group is active or the resilient application is running.



CRTCFGOBJ—Create Configuration ObjectsCRTCFGOBJ OBJ( ) OBJTYPE( ) OWNER( )

Use this command to create configuration objects that are replicated to a backup node.iCluster can automatically create configuration objects immediately after the data required for creating them is received on the backup node. Alternatively, the configuration objects can be created at a later time. If the latter approach is adopted, this command can be issued to create the configuration objects.For more information about configuration objects, see Replicating Configuration Objects on page 90.

Input Parameters• OBJ

The name of the configuration object that you want to create.Enter a specific or generic object name (to identify multiple objects), or the following value:• *ALL—Specifies all configuration objects whose data is being stored on the backup node. Once the objects have

been successfully created, the data is removed from the backup node.• OBJTYPE

The type of configuration object that you want to create.Specify *CNNL, *COSD, *CTLD, *DEVD, *IPXD, *LIND, *MODD, *NTBD, *NWID, *NWSD (to identify a specific type of configuration object) or the following value:• *ALL - Specifies all configuration object types.

See Object Types Replicated by iCluster on page 529 to identify the configuration object types that are listed above.• OWNER

The owner of the configuration object that is created.Specify the user name of the owner or the following value:• *CURRENT—Specifies the user name that is running this command is assigned ownership of the created object.



The default setting for this parameter is *CURRENT.

ExamplesCRTCFGOBJ OBJ(OBJ1) OBJTYPE(*MODD) OWNER(SMITHB)

Creates the mode description OBJ1.SMITHB will be assigned ownership of this mode description.

CRTCFGOBJ OBJ(*ALL) OBJTYPE(*DEVD) OWNER(BLOGGSJ)Creates all device descriptions whose data is being stored on the backup node.BLOGGSJ will be assigned ownership of these device descriptions.

CRTCFGOBJ OBJ(*ALL) OBJTYPE(*ALL) OWNER(*CURRENT)Creates all configuration objects whose data is being stored on the backup node.The user name that is running this command will be assigned ownership of all objects that are created.

UseIf the configuration object being created already exists on the backup node, this command can only be issued when the object is not in use.You must issue this command on the backup node where the configuration objects are created. However, the backup node does not have to be active.


INITHAOBJ—Initialize ObjectsINITHAOBJ TARGET( ) GROUP( )

Use this command to prepare objects for mirroring so that any changes made to the objects in questions are journaled. It changes the OS/400 parameter Object Auditing Value from *NONE to *CHANGE.Initializing objects should occur automatically, though you may need to issue this command in the following situations:• If the system auditing levels (QAUDLVL, and QAUDCTL) have been changed so that they no longer meet the required

values for iCluster. See the VFYHAJRN—Verify Audit Journal command for more information. • If iCluster could not start auditing on one or more objects at the time the object specifier was added or modified. In this

case a warning message will be logged in the job log to remind you to issue the INITHAOBJ command.Any objects that are replicated by a group will have object auditing set to *CHANGE. Auditing should begin whenever an included specifier (BSF or native) is added to the group or whenever a specifier is changed to include an object or BSF.The following restrictions apply:• Excluded BSF objects are not respected when there is a more generic include.

For example, /home/* *INC supersedes /home/dir1 *EXC or /home/dir1/* *EXC.• The OS/400 parameter Object Auditing Value for BSF objects will always be set to *CHANGE.

Input Parameters• TARGET

The name of the backup node in the recovery domain for the replication group identified through the GROUP parameter (see below).


• GROUPThe name of the replication group that will have its selected objects initialized through this command.

ExampleINITHAOBJ TARGET(NODE1) GROUP(GRP1)

Initializes all objects selected to the replication group GRP1 that has NODE1 as a backup node.

UseYou must issue this command only on a primary node. You should issue it before you start mirroring in iCluster for the very first time and after you have selected an object specifier to a replication group.This command ensures the system auditing values and the system audit level are set to the values that iCluster needs for mirroring to work correctly.


RTVRCVPT—Retrieve Recovery CheckpointRTVRCVPT JRNKEY( ) JRN( ) OLDRCV( ) OLDPOS( ) OLDPOSLRG( )

Use this command to retrieve the recovery checkpoint position from the journal on the backup node given a journal key, journal, receiver, and position from the primary node. The journal key (JRNKEY) is specified in the DMADDGRP—Add Group command.You can use the recovery checkpoint to find the approximate backup node journal position corresponding to a given primary node journal position. This journal position can be used as the starting journal position in a recovery situation.

Note: This command is intended to be issued from the command line only. To issue this command from a CL program, see the RTVRCVPTR—Retrieve Recovery Checkpoint (CL Program) command.

Parameters• JRNKEY

Specifies the key of a recovery checkpoint that was specified for the JRNKEY parameter of either the DMADDGRP—Add Group or DMCHGGRP—Change Group command.

• JRNSpecifies the name of the journal on the primary node. You need to identify the library where the journal resides. Prefix the journal with the name of the library where the journal is located (for example, LIB1/JRN1).

• OLDRCVIndicates the name of the journal receiver on the primary node.You need to identify the library where the journal receiver resides. Prefix the journal receiver with the name of the library where the journal receiver is located (for example, LIB1/JRNRCV1).

• OLDPOSIndicates the journal position on the primary node.

• OLDPOSLRGIndicates the journal position of up to 20 digits on the primary node.

OutputThe journal receiver and journal position of the recovery checkpoint on the backup system is displayed in the job log.



ExamplesRTVRCVPT JRNKEY(RCVRYID1) JRN(LIB1/JRN1) OLDRCV(LIB2/JRNRCV2) OLDPOSLRG(242)

Retrieves the recovery checkpoint based on the key RCVRYID1, journal JRN1, journal receiver JRNRCV2, and journal position 242.

RTVRCVPT JRNKEY(RCVRYID1) JRN(LIB1/JRN1) OLDRCV(LIB2/JRNRCV2) OLDPOSLRG(2424546454533345) Retrieves the recovery checkpoint based on the key RCVRYID1, journal JRN1, journal receiver JRNRCV2, and journal position 2424546454533345.

UseYou may issue this command on the backup system as part of a recovery situation, such as after performing a switchover.

RTVRCVPTR—Retrieve Recovery Checkpoint (CL Program)RTVRCVPTR JRNKEY( ) JRN( ) OLDRCV( ) OLDPOS( ) OLDPOSLRG( ) RTNCDE( ) RTNRCVNME( ) RTVRCVLIB( ) RTNPOS( ) RTNPOSLRG( )

Use this command to retrieve the recovery checkpoint position from the journal on the backup node given a journal key, journal, receiver, and position from the primary node. The journal key (JRNKEY) is specified in the DMADDGRP—Add Group or DMCHGGRP—Change Group command.You can use the recovery checkpoint to find the approximate backup node journal position corresponding to a given primary node journal position. This journal position can be used as the starting journal position in a recovery situation, such as after a switchover.

Note: You must issue this command as part of a CL program. To issue this command from the command line, see the RTVRCVPT—Retrieve Recovery Checkpoint command.

Input Parameters• JRNKEY

Specifies the key of a recovery checkpoint that was specified for the JRNKEY parameter of either a DMADDGRP or DMCHGGRP command.

• JRNSpecifies the name of the journal on the primary node.You need to identify the library where the journal resides. Prefix the journal with the name of the library where the journal is located (for example, LIB1/JRN1).

• OLDRCVIndicates the name of the journal receiver on the primary node.You need to identify the library where the journal receiver resides. Prefix the journal receiver with the name of the library where the journal receiver is located (for example, LIB1/JRN1).

• OLDPOSIndicates the journal position on the primary node.

• OLDPOSLRGIndicates the journal position of up to 20 digits on the primary node.

Output Parameters• RTNCDE

Indicates if the command succeeded or failed in retrieving a recovery checkpoint.One of the following values will be returned:


• *OK—Specifies that the command completed successfully.• *ERROR—Specifies that the command did not complete successfully.

• RTNRCVNMEIndicates the name of the journal receiver on recovery machine.

• RTVRCVLIBIndicates the library where the journal receiver is located.

• RTNPOSIndicates the recovery checkpoint journal position on the backup machine. This is the journal position that is used in the DMSETPOS—Set Journal Start Position command.

• RTNPOSLRGIndicates the recovery checkpoint journal position of up to 20 digits on the backup machine. This is the journal position that is used in the DMSETPOS—Set Journal Start Position command.

ExamplesRTVRCVPTR JRNKEY(RCVRYID1) JRN(LIB1/JRN1) OLDRCV(LIB2/JRNRCV2) OLDPOS(262) RTNCDE (&RTNVAR) RTNRCVNME (&RCVNMEVAR) RTNRCVLIB (&RCVLIBVAR) RTNPOS (&POSVAR)

Retrieves the recovery checkpoint position based on the key RCVRYID1, journal JRN1, journal receiver JRNRCV2, and journal position 262.

Note: &RTNVAR, &RCVNMEVAR, &RCVLIBVAR and &POSVAR are CL variables.RTVRCVPTR JRNKEY(RCVRYID2) JRN(LIB2/JRN2) OLDRCV(LIB2/JRNRCV2) OLDPOSLRG(5612364789525458) RTNCDE (&RTNVAR) RTNRCVNME (&RCVNMEVAR) RTNRCVLIB (&RCVLIBVAR) RTNPOSLRG (&POSVARLRG)

Retrieves the recovery checkpoint position based on the key RCVRYID2, journal JRN2, journal receiver JRNRCV2, and journal position 5612364789525458.

Note: &RTNVAR, &RCVNMEVAR, &RCVLIBVAR and &POSVARLRG are CL variables.

UseYou need to issue this command from a CL program on the backup system as part of a recovery situation, such as after performing a switchover.

SYNCHATRG—Synchronize TriggersSYNCHATRG TARGET( ) GROUP( ) FILE( ) FILELIB( )

Use this command to refresh the triggers on the database files for a valid backup node or group.This command disables all synchronized triggers on the backup node.


The name of the backup node for the files whose triggers are going to be synchronized.Enter the name of a valid backup node or the following value:• *ALL—All targets will be used. If you select this value, then the command ignores the GROUP parameter.

• GROUPThe name of the group whose files are going to be synchronized.Enter the name of a valid group belonging to the specified backup node or the following value:• *ALL—All groups attached to the specified backup node.



• FILEThe database files whose triggers will be synchronized. These files must be within the replication scope of the backup node and group specified above.Enter a specific or a generic name or the following value:*ALL - All of the database files of the specified backup node or group.

• FILELIBThe library for the files whose triggers will be synchronized.Enter a specific name or the following value:• *ALL—All libraries within replication scope of the backup node or group specified. The FILE parameter must be

*ALL if you select this value.

OutputRelevant messages are sent to the event and job logs.

ExamplesSYNCHATRG TARGET(*ALL) GROUP(*ALL) FILE(*ALL) FILELIB(*ALL)

Synchronizes triggers for all files within the replication scope for all libraries for all backup node and group combinations.SYNCHATRG TARGET(SITGT) GROUP(*ALL) FILE(*ALL) FILELIB(MYLIB1)

Synchronizes triggers for all files within the replication scope of library MYLIB1 for all groups attached to target SITGT.'

UseThis command must be used on the source machine of the specified groups. The groups can be active or inactive when issuing the command, but actual synchronization will only occur when the group is active.

DMLOGENT—Log Journal EntryDMLOGENT GROUP( ) ENTTYP( ) USRDTA( ) DTAARA( )

Use this command to record the times that data is scraped and applied, which allows you to gauge the performance of specific groups in your cluster. You can also use this command to gain insight into the progress of your mirroring activities during batch runs or troubleshooting.This command creates the DMLOG database file on the backup node if it does not already exist. The log file has the following format:• Group (char 10): The name of the group or resilient application.• ENTTYPE (4 bytes): Entry type• Journal (char 20): Journal with library• Receiver (char 20): Receiver with library• Journal position sequence number (numeric 20)• Scrape date/time: Filled in at run-time by a scraper job.• Apply date/time: Filled in at run-time by an apply job.• USRDTA (char 400): A user defined field.• Data area contents (char 2000): Filled in at run-time by a scraper job.



The name of the group or resilient application for which logging information is deposited and recorded.Note: *REPL and *REFRESH groups are supported, while *MQSERIES and *SWDEV are not supported.

• ENTTYPIndicates the type of logging information. You can choose from a predefined type or create a user defined type.Specify an integer value between 1 and 2000 or one of the following values:• *INFO—Indicates an informational entry.• *STRRUN—Indicates the beginning of mirroring.• *ENDRUN—Indicates the end of mirroring.

The default setting for this parameter is *INFO.• USRDTA

A user defined field of up to 400 characters that is optional.• DTAARA

The name of a data area that will be recorded in the log file. The data area cannot exceed 2000 bytes of character data.Specify the name of the data area or the following value:• *NONE—Indicates that no data area is specified.

The library where the data area resides must be identified if you do not specify *NONE. Prefix the data area name with the name of the library where the data area is located (for example, LIB2/DTAARA1), or the following value:• *PRODLIB—Specifies your iCluster installation library.

The default setting for this parameter is *NONE.

ExampleDMLOGENT GROUP(GRP1) ENTTYP(20) DTAARA(LIB3/DTAARA2)

Logging information is tracked for GRP1.A user defined entry type of 20.The contents of DTAARA2 in library LIB3 will be recorded in the log file.




Status and History Monitor Commands

Status and History Monitor CommandsThis section contains the following topics:• DSPHASMON—Display Source Monitor on page 255• WRKHASMON—Work with Status Monitor on Primary Node on page 255• WRKHATMON—Work with Status Monitor on Backup Node on page 256• CHGHASMON—Change History Monitor on Primary Node on page 257• PRGHASMON—Purge History Monitory on Primary Node on page 259• WRKHAOBJST—Work with Object Status on page 260• WRKHABSFST—Work with BSF Status on page 260• DMWRKOBJST—Work with Object Status by Group on page 261

DSPHASMON—Display Source MonitorDSPHASMON

Use this command to display a simplified version of the Status Monitor on the primary system. This simplified Status Monitor allows you to view only the details and the status of objects for latency, throughput, object position, and journal information. No operational control of iCluster is provided from the resulting screen.

Note: For additional functionality, you will need to issue the WRKHASMON—Work with Status Monitor on Primary Node command. See the WRKHASMON—Work with Status Monitor on Primary Node command for more information.



ExamplesDSPHASMON

Displays the simplified Status Monitor.

UseYou can issue this command at any time from the primary system.


WRKHASMON—Work with Status Monitor on Primary NodeWRKHASMON

Use this command to display the Status Monitor on a primary node, which allows the replication status of both nodes to be viewed. The monitor reporting is based on group and journal combinations, which provide for both real time inquiry of active replication processes and historical inquiry of past activity. You can check object status and open the communication links from the Status Monitor.The following statuses and statistics may be viewed using the Status Monitor:


• Replication status for both active and inactive groups.• Replication latency on the backup node.• Primary and backup node journal positions.• Run time totals for both primary and backup node replication processes.• Throughput in transactions per hour for the backup node.

For more information about the features, functions, and displays provided by the monitor facility, see Working with the Status Monitor on page 291.


ExampleWRKHASMON

Displays the Status Monitor on the primary node of the group(s)

UseYou must issue this command on the primary node of the group(s) you want to monitor.


WRKHATMON—Work with Status Monitor on Backup NodeWRKHATMON

Use this command to display the Status Monitor on a backup node, which allows the replication status of both nodes to be viewed. The monitor reporting is based on group and journal combinations, which provides for both real time inquiry of active replication processes and historical inquiry of past activity.Using this command you can check object status. No communication with the primary node is required.The following statuses and statistics may be viewed using the Status Monitor:• Replication status for both active and inactive groups.• Replication latency on the backup node.• Primary and backup node journal positions.• Run time totals for both primary and backup node replication processes.• Throughput in transactions per hour for the backup node.



ExampleWRKHATMON

Displays the Status Monitor allowing the replication status of groups as well as transaction information to be viewed.



UseYou must issue this command on the backup node of the groups you want to monitor.


CHGHASMON—Change History Monitor on Primary NodeCHGHASMON TARGET( ) GROUP( ) STRDTE( ) STRTIM( ) POLINT( ) ENDDTE( ) ENDTIM( )

Use this command to change the history monitor collection process on a primary node. The collection process can be modified so that replication information is captured for different groups within different collection intervals.Monitor reporting is based on groups that provide for both real time inquiry and historical inquiry of replication processes.The following status and statistics can be viewed using the Status Monitor:• Replication status for both active and inactive groups.• Replication latency on the backup node.• Primary and backup node journal positions.• Runtime totals for both primary and backup node replication processes.• Throughput in transactions per hour for the backup node.



The name of the backup node for which you will be starting the history monitor collection.• GROUP

The name of the group for which data will be collected.The primary node in the group must be the system where this command is invoked.

• STRDTEThe starting date for replication monitoring. The date must be supplied in a format specified in the user's job attributes. If the date specified precedes the current date, monitoring will start at the current date. Enter a specific date or one of the following values:• *TODAY—Starts monitoring on the current date.• *SAME—Uses the current setting for this parameter.

The default setting for this parameter is *TODAY.• STRTIM

The starting time for monitoring replication. The time must be entered as a six-digit integer, between 000000 (midnight) and 235959 (11:59:59 p.m.), that represents a valid time.Enter a specific time or one of the following values:• *IMMED—If the starting date (see STRDTE above) is set to *TODAY or an earlier date, monitoring starts

immediately. If the starting date is set to a later date, monitoring starts at 000000 on the specified date.• *SAME—Uses the current setting for this parameter.


The default setting for this parameter is *IMMED.Note that if the date specified in the parameter STRDTE (see above) precedes the current date, monitoring will start immediately.

• POLINTThe polling interval for monitoring replication. This is the time interval that the monitoring system will use to take a snapshot of replication activity. This parameter must be entered as a six-digit integer, between 000010 and 235959.Specify a polling interval or the following value:• *SAME—Uses the current setting for this parameter.

The default setting for this parameter is 001500 (15 minutes).• ENDDTE

The ending date for monitoring replication. The date must be supplied in a format specified in the user's job attributes.Enter a specific date or one of the following values:• *TODAY—Ends monitoring on the current date.• *NONE—Ends monitoring when all replication processes end.• *SAME—Uses the current setting for this parameter.

This default setting for this parameter is *NONE.Note: The history monitor collection process is always active when replication is active.

Note that if the date specified precedes the current date, an error message will be issued asking you to correct their input.

• ENDTIMThe ending time for replication monitoring. The time must be entered as a six-digit integer, between 000000 (midnight) and 235959 (11:59:59 p.m.), that represents a valid time.Enter a specific time or one of the following values:• *NONE—Ends monitoring when all replication processes end or when the CHGHASMON command is invoked. If

the end date (see ENDDTE above) is set to *TODAY or a date in the future, monitoring will end at 23:59:59 (11:59:59 p.m.) on that date.

• *SAME—Uses the current setting for this parameter.The default setting for this parameter is *NONE.

Note: The history monitor collection process is always active when replication is active.Note that if the time specified precedes the current time, an error message will be issued asking the user to correct their input. However, the time specified may precede the current time if the end date (see ENDDTE above) has been set to a date in the future.

ExampleCHGHASMON TARGET(NODE2) GROUP(GRP1) STRDTE(*TODAY) STRTIM(163000) POLINT(013000) ENDDTE('07/23/03') ENDTIM(175959)

The monitor facility will collect information pertaining to the replication status of group GRP1 whose backup node is NODE2.Replicated objects will be monitored between 4:30 p.m. on the day the command is issued, and 6:00 p.m. on July 23, 2003.Replication activity will be captured every 90 minutes within this time interval.

UseYou must issue this command on the primary node of the groups you want to monitor.




PRGHASMON—Purge History Monitory on Primary NodePRGHASMON TARGET( ) ENDDTE( ) ENDTIM( )Use this command to delete the historical information from the Status Monitor history database on the primary node. For more information about the features, functions, and displays provided by the monitor facility, see Working with the Status Monitor on page 291.


The name of the backup node for which records will be removed.Enter a specific name of a backup node or the following value:• *ALL—Records related to all backup nodes will be deleted.

This field requires an entry.• ENDDTE

The latest date for records to be deleted. All records timestamped before the specified date and time (see ENDTIM below) will be purged.The date must be supplied in a format specified in the user's job attributes.If the date specified follows the current date, all the historical information will be deleted.Enter a specific date or the following value:• *TODAY—Indicates that history log records for all dates will be deleted.

The default setting for this parameter is *TODAY.• ENDTIM

The latest time for records to be deleted. All records timestamped before the specified date (see ENDDTE above) and time will be purged.The time must be entered as a six-digit integer, between 000000 and 235959, that represents a valid time.Enter a specific time or the following value:• *NOW—The ending time for records to be deleted is the current time.

The default setting for this parameter is *NOW.

ExamplePRGHASMON TARGET(NODE2) ENDDTE('12/31/03') ENDTIM(235959)

Purges the monitor of iCluster historical records that were written before the end of 2003 and whose backup node is NODE2.

UseYou must issue this command on the primary node of the groups that you want to monitor.



WRKHAOBJST—Work with Object StatusWRKHAOBJST TARGET( ) GROUP( ) JOURNAL( ) LOCATION( ) SOURCE( )

This command displays the Work with Object Status screen. This screen shows the status of a journal. See Working with Object Status on page 306 for more information.


Specifies the backup node of the specified group.• GROUP

Specifies the name of an existing replication group.• JOURNAL

Specifies the name of an existing journal.The library where the journal resides must be identified. Prefix the journal with the name of the library where the journal is located (for example, LIB2/JRN1).The possible values are:• *PRODLIB—Specifies your iCluster installation library.• *UNKNOWN—Specifies a library-journal combination with the values *UNKNOWN/*UNKNOWN.

You would specify *UNKNOWN when you want to check whether there are suspended objects associated with unknown journals and libraries.

• library-name—Specifies the name of the library where the specified journal is located.• LOCATION

Specifies whether you want to work with object status on the primary or backup node.The possible values are:• *SRC—Indicates that you want to work with object status on the primary node.• *TGT—Indicates that you want to work with object status on the backup node.

• SOURCESpecifies the name of the primary node where the objects you want to view the status for are located.This command only uses the value for this parameter when LOCATION is *SRC.

ExampleWRKHAOBJST TARGET(TGT1) GROUP(GRP1) JOURNAL(LIB1/JRN1)

Displays the journal JRN1, located in the library LIB1, belonging to the group GRP1 and backup node TGT1.

WRKHABSFST—Work with BSF StatusWRKHABSFST TARGET( ) GROUP( ) JOURNAL( ) LOCATION( ) SOURCE( )

This command displays the Work with Object Status screen. This screen shows the status of a journal. See Working with BSF Status on page 310 for more information.


Specifies the backup node of the specified group.



• GROUPSpecifies the name of an existing replication group.

• JOURNALSpecifies the name of an existing journal.The library where the journal resides must be identified. Prefix the journal with the name of the library where the journal is located (for example, LIB2/JRN1).The possible values are:• *PRODLIB—Specifies your iCluster installation library.• *UNKNOWN—Specifies a library-journal combination with the values *UNKNOWN/*UNKNOWN.

You would specify *UNKNOWN when you want to check whether there are suspended objects associated with unknown journals and libraries.• library-name—Specifies the name of the library where the specified journal is located.

• LOCATIONSpecifies whether you want to work with object status on the primary or backup node.The possible values are:• *SRC—Indicates that you want to work with object status on the primary node.• *TGT—Indicates that you want to work with object status on the backup node.

The default value is *SRC.• SOURCE

Specifies the name of the primary node where the objects you want to view the status for are located.This command only uses the value for this parameter when LOCATION is *SRC.

ExampleWRKHABSFST TARGET(TGT1) GROUP(GRP1) JOURNAL(LIB1/JRN1)

Displays the journal JRN1, located in the library LIB1, belonging to the group GRP1 and backup node TGT1.

DMWRKOBJST—Work with Object Status by GroupDMWRKOBJST GROUP( ) STRWITH( )

This command displays the Work with Object Status by Group screen. This screen displays the status of objects for a specified group. You can also use this screen to activate suspended native objects.You must run this command from the backup node of the specified group.


The name of the group you want to monitor.• STRWITH

Specifies which view you want to open this screen with. After running this command, you can change views by pressing F16.The possible values are:• *NTV—Start with the native object view.• *BSF—Start with the BSF object view.


The default value is *NTV.

ExampleDMWRKOBJST GROUP(GRP1) STRWITH(*NTV)

Displays the status of objects in the GRP1 group. When the screen first appears, it shows native objects.


Sync Check Commands

Sync Check CommandsThis section contains the following topics:• DSPHASC—Display Sync Check on page 263• SELSCATTR—Select Sync Check Attributes on page 265• STRHASC—Start Sync Check on page 267• STRHASCUSR—Start User Sync Check on page 270• DMSCRPT—Sync Check Report for IFS Objects on page 274• DMSCRPTNTV—Sync Check Report for Native Objects on page 275• PRGHASC—Purge Sync Check Results on page 276

DSPHASC—Display Sync CheckDSPHASC

Use this command to access the results of the sync check. The results are contained in a spool file named <Group Name> Sync Check Report.You can display the results of the sync check by entering option 5 in the field beside the name of a selected spool file. The report contains the following sections that are organized by sync check type.

*FILEATTR Sync Check TypeThis type of sync check contains the following sections:

HA Sync Check SummaryThis section contains the following statistics about the sync check:• Total number of objects checked• Total number of objects not matching• Total number of members not matching• Total number of extra members on backup• Total number of objects not on backup• Total number of members not on backup• Total number of objects that cannot be allocatedNote that this section is not displayed if all objects exist on the backup system.

Object List Report This section displays the following columns:• Object/Member: Displays the name of the object and member. An asterisk to the left of this column indicates a mismatch

between the object on the primary and backup nodes. The type of mismatch is indicated in the last four columns.• Src Lib: Displays the name of the library where the source object is located on the primary node.• Tgt Lib: Displays the name of the library where the target object is located on the backup node.• Type: Indicates the type of the object.• Size: Indicates the size of the object on the primary node in bytes. Note that only mismatched objects are displayed (not in

sync).


• #Attr checked: Displays the number of attributes that were checked in the sync check. You can modify the number of attributes that are checked. See the SELSCATTR—Select Sync Check Attributes command for more information.

• #Attr match: Indicates the number of attributes that matched on the primary and backup nodes for the object.• #Attr not match: Indicates the number of attributes for the object that did not match on the primary and backup nodes. If you

have non-matching attributes, then this object is not in sync. You should check the Object Differences Detail section to determine which attribute(s) did not match.

• Not found: A 'B' in this column indicates that the object was not found on the backup node.

Object Differences Detail This section provides information about any objects that did not match. It displays the name of the object that had unmatching attributes, the library where the object resides on both the primary and backup nodes, the object size, and the type of file. It also indicates the attribute(s) that did not match, and the attribute settings on the primary and backup nodes.If all objects matched, then this section will not be displayed in the sync check results.

Unallocated Object ListThis section lists the objects that could not be sync checked because they were locked by another process and their attributes could not be retrieved. This section is displayed if one or more objects were locked during the sync check.

*LIST Sync Check TypeThis type of sync check contains the following sections:• Object List Comparison Statistics• Object List Comparison Details. This section displays information by object type and has an *OUTQ mismatch column that

indicates how many output queue (*OUTQ) objects do not match between the primary node and the backup node.• Objects Not Found or Mismatched on Backup. This section lists the objects and has a SPLF count column that indicates

whether or not the number of spooled files in an *OUTQ object is the same on the primary node and the backup node.These sections display the following information:• Total objects checked on primary• Total objects found on backup• Total objects not found or mismatched on backup.• Number of objects that could not be accessed for checking

*DTAARA Sync Check TypeThis type of sync check contains the following sections:

Data Area Sync Check SummaryThis section contains the following columns: • Data Area: The name of the data area.• Src lib: Displays the name of the library where the source data area is located on the primary node.• Tgt lib: Displays the name of the library where the target data area is located on the backup node.• Identical: Specifies whether the two objects match.• Not found: Specifies whether the data area exists on the backup node.• Attr differ: Indicates the object attributes that differ on the backup node from the primary node.• Contents differ: Specifies those data areas on the backup node whose contents are different from the primary node.


Sync Check Commands

• Comparison failed: Identifies the objects that could not be compared, for example, because one of the objects was locked.This section also displays the following items:• Number of data areas checked• Number of mismatched data areas

*BSF Sync Check TypeThis type of sync check contains the following section:

Object List Comparison StatisticsThis section contains the following statistics about the sync check:• Number of objects checked on primary• Total number of objects found on backup• Total number of objects not matching (checks for object existence, object authority, user/group/owner permission, and primary

group)• Number of objects not found on backup• Number of objects that could not be accessed for checkingNote: To view the output for this type of sync check, you can run the DMSCRPT—Sync Check Report for IFS

Objects command on the backup node.


OutputThis command displays the list of spool files generated by the sync check. A completion message for sync check will be put into the event log on the group's primary node. For more information about event logs, see Monitoring Replication on page 76.

ExampleDSPHASC

Displays all sync check spool files on the Work with All Spooled Files screen.

UseYou should issue this command from the group's primary node after performing a sync check.


SELSCATTR—Select Sync Check AttributesSELSCATTR USEDFT( )

Use this command to select file attributes that will be involved in a sync check operation.The file attributes that you can include are divided into four groups:• File attributes: These attributes relate to each file.• Member attributes: These attributes relate to each member within each file.• Record attributes: These attributes relate to the record formats contained within each file.


• Field attributes: These attributes relate to the fields of records contained within each file.When selecting attributes, you should note the following items:• You should select at least one file attribute at the file level.• If you select attributes at the member level, then the File Member Name attribute will automatically be selected for the

sync check.• The number of attributes that you select can affect the performance of the sync check. The more attributes you select,

then the longer the sync check will take to complete.• If you are unsure which attributes to select, DataMirror recommends using the default values. When you first issue this

command, all the defaults are selected.Some attributes are included, by default, in a check. While you do not need to use the default attributes, DataMirror recommends that you do because they are important in determining primary and backup node synchronization. The default sync check attributes are as follows:

File Level Attributes• Access path maintenance• File generic key field count• File generic key length• Increment number of records• Initial number of records• Maximum key length• Maximum members• Maximum number of fields• Maximum record length• Null value duplicate indicator• Number of constraints• Number of key fields• Primary key indicator• Public authority at creation• Record format level check• Reuse deleted record indicator• Total number of members• Total number of record formats• Unique constraint indicator

Member Level Attributes• Current number of records• File attribute• File member name• Number of deleted records


Sync Check Commands

You must specify the attributes you want to include in a sync check prior to launching the sync check program. Note that this attribute list is global, meaning that the attributes you include affect all sync checks. The list of attributes used by the sync check is stored in the database file scattrfile. The online help for the SELSCATTR—Select Sync Check Attributes command also contains a list of all the attributes that can be included during a sync check.Note that you can use this command for both the STRHASC—Start Sync Check and STRHASCUSR—Start User Sync Check commands.

Input Parameter• USEDFT

Indicates whether the default settings will be used.Enter one of the following values:• *YES—Uses the default value for each attribute. Any changes that you have selected for the attributes are ignored.• *NO—Does not use the default attribute selections, and instead use the values that you have selected.

Note: Use the Page Down key to view all of the attributes that you can select for a sync check.

ExamplesSELSCATTR USEDFT(*YES)

The default values will be used in the sync check.SELSCATTR USEDFT(*NO)

The selections that you made will be applied in the sync check.

UseUse this command to indicate the attributes that you want to use during a sync check.You should issue this command from a primary node.


STRHASC—Start Sync CheckSTRHASC TARGET( ) GROUP( ) LOCK( ) SCTYPE( ) DATE( ) TIME( ) OUTPUT( )

Use this command to start the sync check program.A sync check allows you to check whether the objects on the backup node are the same as the objects on the primary node within a replication group. All objects for the specified replication group are checked. By comparison, the STRHASCUSR—Start User Sync Check command allows you to perform a sync check on specific objects that are replicated in the replication group.You can perform the following types of sync checks:• The object list sync check compares the number of objects on the primary node with the number of objects on the backup

node. It does not compare the attributes of the objects.• The file attribute sync check checks for the existence of replicated *FILE objects on the backup node. It also compares

the attributes of the primary and backup node *FILE objects. Note that the object list sync check compares all object types while the file attribute check examines only *FILE objects.

• You can also use this command to perform a data area sync check and a BSF sync check for BSF objects. See DSPHASC—Display Sync Check for more information on the type of information that is displayed with a data area or BSF sync check.

• A check journaling sync check verifies that the native and BSF objects for a group are journaled on the backup node.


How often you perform a sync check depends on how often you want to check that the primary and backup node objects are the same. Through the use of the SELSCATTR—Select Sync Check Attributes command, you can indicate the attributes that you want to include in a file attribute check. Note that the more attributes you include in the sync check, the longer the check will take. The results of the sync check are displayed in a spool file on the primary node. See DSPHASC—Display Sync Check for more information.


The name of the backup node for the specified group on the GROUP parameter.• GROUP

The name of a replication group defined in iCluster.• LOCK

Indicates whether the files involved in the sync check will be locked during a file attribute sync check. Locking ensures that an accurate synchronization takes place between the primary and backup nodes, but it increases the amount of time required to complete the operation.

Note: If you decide to lock objects during a sync check, DataMirror recommends that you do not initiate a sync check when the objects in question are being heavily used.

DataMirror recommends that the DMSETSVAL—Set Cluster System Values parameter LOCKTGT is set to *YES to ensure that the backup node files are also not changed by users.Enter one of the following values:• *YES—Indicates that files will be locked during the sync check. If a file cannot be locked, then it will not be involved

in the sync check, and a message will be logged in a spool file.Note that locking files increases the time it takes to complete the sync check.

• *NO—Indicates that files will not be locked during the sync check. You will have to make sure that the files are not modified during the course of the sync check. This is the default value for this parameter.

Not locking files decreases the amount of time required to complete the sync check.• SCTYPE

The type of sync check that you want to perform.Specify one of the following values:• *FILEATTR—Checks whether the *FILE objects on the primary node exist on the backup node. It also checks that

the attributes of the primary node *FILE objects match the attributes of the backup node *FILE objects.This type of check locks each object until the check for the individual object has been completed. Therefore, this option produces a slower sync check as each object has to be individually locked and then unlocked. However, each object is locked for a shorter period of time, which means that access to the object is possible while other objects are being checked.*FILEATTR sync check requires that the files being checked are journaled, and that all of the group's required database journal scrape and apply processes are active. Source physical files can be sync checked with *FILEATTR sync check if either of the following two requirements are met:

• If the file is journaled and there is a scrape and apply process for the file's journal active for the group that is being sync checked.

• If the file is not journaled but there is a scrape and apply process active for the default database journal for the group that is being sync checked.

Note: *FILEATTR sync check will check objects of attribute type PF-SRC only if the group the object belongs to replicates at least one object that is being journaled. For example, having at least one PF-DTA object in the group will ensure *FILEATTR sync checks will include the PF-SRC objects.


Sync Check Commands

Note: *FILEATTR sync checks will not report object contents mismatches if you decide not to refresh or mirror the object contents with MIRRCNTS(*NO) in the DMSELOBJ—Select Objects to Group command.• *LIST—Checks whether the objects on the primary node exist on the backup node.

Note: The objects for a *LIST sync check are restricted to native OS/400 objects (library objects). To perform a sync check on BSF (path) objects, the *BSF sync check type should be selected.• *BSF—Checks all the BSF objects on the primary node that match the specified path object specifiers with the

matching objects on the backup node.Note: DataMirror recommends that you run sync check for BSF objects only when latency for the group is negligible,

or when little or no change is occurring for the objects being checked.• *DTAARA—Checks whether data area objects on the primary node are replicated to the backup node with the

correct attributes and contents.If a certain data area does not exist on the primary node when you issue this command, it will not be included in the sync check.

Note: *DTAARA sync checks will not report object contents mismatches if you decide not to refresh or mirror the object contents with MIRRCNTS(*NO) in the DMSELOBJ—Select Objects to Group command.• *CHKJRN—Verifies that the objects selected to this group are journaled on the backup node. This report considers

objects to be out-of-sync if any of the following conditions are met:• It is a *FILE, *DTAARA, or *DTAQ object that is not journaled on the backup node.• It is a non-directory *STMF object that is journaled on the primary node and not journaled on the backup node.• A journaled *STMF file or directory object exists on the primary node but not on the backup node.

You can only generate this report if the Journal Objects on Backup system value is set to *YES. See the Physical File values of the DMSETSVAL—Set Cluster System Values command for more information.The default setting for this parameter is *FILEATTR.

• DATEThe date when the sync check will be started. The date that you enter must be entered in MMDDYY format, where the first two digits represent the month, the second two digits represent the day, and the last two digits represent the year (for the year 2000, set YY to 00).Enter a specific date or the following value:• *CURRENT—Indicates that the sync check will be performed when you issue this command.

The default setting for this parameter is *CURRENT.• TIME

The time when the sync check will be started. The time that you enter must be in the six-digit format HHMMSS, where the first two digits represent hours, the second two represent minutes, and the last two represent seconds.Enter a specific time or the following value:• *CURRENT—Indicates that the sync check will be performed when you issue this command.

The default setting for this parameter is *CURRENT.• OUTPUT

The type of sync check report that should be generated.You can generate a complete sync check report that displays information about all objects included in the sync check, regardless of whether they are synchronized on the primary and backup nodes. Depending on the number of objects included in the sync check, this report can be long and difficult for navigation purposes.


Note: This parameter only applies to *FILEATTR, *LIST, and *DTAARA sync checks. *BSF sync check does not produce a listing of objects. Use the DMSCRPT—Sync Check Report for IFS Objects command to obtain a *BSF sync check report.

Alternately, you can generate a report that displays only those objects that are not synchronized on the primary and backup nodes.Enter one of the following values:• *MISMATCH—Displays only those objects that are not synchronized when writing the sync check report.• *ALL—Displays all objects, regardless of whether they match or not.• *NONE—Indicates that no spool file report is to be generated for the sync check. The output is still sent to a

database file. Use the DMSCRPT—Sync Check Report for IFS Objects or DMSCRPTNTV—Sync Check Report for Native Objects command to view the sync check report in the database file.

The default setting for this parameter is *MISMATCH.

ExamplesSTRHASC TARGET(TGT1) GROUP(GRP1) SCTYPE (*LIST) DATE(*CURRENT) TIME(*CURRENT) OUTPUT(*MISMATCH)

Indicates that an Object List sync check command will begin at the current date and time for the backup node TGT1 and group GRP1 for selected objects. The report displays only those objects that are not synchronized

UseYou must issue this command on the primary node of the specified replication group.


STRHASCUSR—Start User Sync CheckSTRHASCUSR TARGET( ) GROUP( ) LOCK( ) SCTYPE( ) OBJSPEC( ) PATH( ) DATE( ) TIME( ) OUTPUT( )

Use this command to test if a set of user-specified objects are synchronized between th primary and backup nodes in a group. This command allows you to perform a synchronization check on a single object or a subset of objects replicated within the specified replication group. This can be useful when a replication group is replicating an object specifier with the *ALL setting, but you want to ensure synchronization for a single object that is referenced by that object specifier. By comparison, the STRHASC—Start Sync Check command performs a sync check on all objects selected to the specified group.The following types of synchronization checks are available:• The object list check compares the number of objects on the primary node with the number of objects on the backup

node. It does not compare the attributes of the objects. • The file attribute check checks for the existence of replicated *FILE objects on the backup node. It also compares the

attributes of the primary and backup node *FILE objects. Note that the object list sync check compares all object types while the file attribute check examines only *FILE objects.

• You can also use this command to perform a data area check and a BSF check for native and BSF objects, respectively. See DSPHASC—Display Sync Check for more information on the type of information that is displayed with a data area or BSF synchronization check.

How often you test for synchronization depends on how often you want to check that the primary and backup node objects are the same. Through the use of the SELSCATTR—Select Sync Check Attributes command, you can indicate the attributes that you want to include in a file attribute sync check. Note that the more attributes you include in the sync check, the longer the check will take. The results of the sync check are displayed in a spool file on the primary node. See DSPHASC—Display Sync Check on page 263 for more information.


Sync Check Commands


The name of the backup node for the specified group on the GROUP parameter.• GROUP

The name of a replication group defined in iCluster.• LOCK

Indicates whether the files involved in the sync check will be locked during a file attribute sync check. Locking ensures that an accurate sync check takes place between the primary and backup nodes, but it increases the amount of time required to complete the operation.

Note: If you decide to lock objects during a sync check, DataMirror recommends that you do not initiate a sync check when the objects in question are being heavily used.

DataMirror recommends that the DMSETSVAL—Set Cluster System Values parameter LOCKTGT is set to *YES to ensure that the backup node files are also not changed by users.Enter one of the following values:• *YES—Indicates that files will be locked during the sync check. If a file cannot be locked, then it will not be involved

in the sync check, and a message will be logged in a spool file.Note that locking files increases the time it takes to complete the sync check.• *NO—Indicates that files will not be locked during the sync check. You will have to make sure that the files are not

modified during the course of the sync check. This is the default value for this parameter.Not locking files decreases the amount of time required to complete the sync check.

• SCTYPEThe type of sync check that you want to perform.Specify one of the following values:• *FILEATTR—Checks whether the *FILE objects on the primary node exist on the backup node. It also checks that

the attributes of the primary node *FILE objects match the attributes of the backup node *FILE objects.This type of check locks each object until the check for the individual object has been completed. Therefore, this option produces a slower sync check as each object has to be individually locked and then unlocked. However, each object is locked for a shorter period of time, which means that access to the object is possible while other objects are being checked.*FILEATTR sync check requires that the files being checked are journaled, and that all of the group's required database journal scrape and apply processes are active. Source physical files can be sync checked with *FILEATTR sync check if either of the following two requirements are met:

• If the file is journaled and there is a scrape and apply process for the file's journal active for the group that is being sync checked.

• If the file is not journaled but there is a scrape and apply process active for the default database journal for the group that is being sync checked.

Note: *FILEATTR sync check will check objects of attribute type PF-SRC only if the group the object belongs to replicates at least one object that is being journaled. For example, having at least one PF-DTA object in the group will ensure *FILEATTR sync checks will include the PF-SRC objects.• *BSF—Checks all the BSF objects on the primary node that match the specified path object specifier with the

matching objects on the backup node.You can enter a path name of up to 5000 characters in length. The path name can be generic, for example "/home/adamsmith/*". A sync check will be performed on all objects in the "/home/adamsmith" directory.


• *LIST—Checks whether the objects on the primary node exist on the backup node.Note: The objects for a *LIST sync check are restricted to native OS/400 objects (library objects). To perform a sync

check on BSF (path) objects, the *BSF sync check type should be selected.• *DTAARA—Checks whether data area objects on the primary node are replicated to the backup node with the

correct attributes and contents.If a certain data area does not exist on the primary node when you issue this command, it will not be included in the sync check.

The default setting for this parameter is *FILEATTR.• OBJSPEC

Note: The default setting for all object specifier parameters is *ALL.The set of parameters that will filter the objects you want to check.

Note: This parameter is ignored if the SCTYPE parameter is set to *BSF.ObjectThe name of the objects that you want to include in the sync check. Note that you can enter a generic name to identify multiple objects in a library.Specify the name of the object or the following value:• *ALL—Indicates that you want to include all object names in the sync check.

The examples illustrate how the library and object parameters are specified in OBJSPEC.LibraryThe name of the library where the objects are located. The examples illustrate how the library and object parameters are specified in OBJSPEC.Object typeThe type of objects that you want to include in the sync check.This parameter is only used if the SCTYPE parameter is set to *LIST.Specify the object type or the following value:• *ALL—Indicates that you want to include all object types in the sync check.

For a complete list of all object types that iCluster can replicate, see Object Types Replicated by iCluster on page 529.Object attributeThe attribute of the objects that you want to include in the sync check.Note that the object attribute is applicable only if you specified a *FILE object in the Object type parameter.This field is free-form. Consequently, you can enter any value you want to represent the object, as long as the value conforms to OS/400 naming conventions. However, there are values that have special meaning to iCluster. They are PFSRC (source physical file), PFDTA (data physical file), and PF (any physical file). Other values listed are standard OS/400 file sub-types. If PF is used, the object specifier will match either PFSRC or PFDTA files.Specify an object attribute or the following value:• *ALL—Indicates that you want to include all object attributes in the sync check.

• PATHThe path specifier that identifies the location of the BSF objects that you want to include in the sync check.The path must be enclosed in single quotes and start with a '/' (forward slash) character ('/Dir3/Dir4/file'). The path can be a maximum of 5000 characters, and you must enter at least two characters for the path.


Sync Check Commands

Generic path names of the form '/mydir*' are supported, where the generic indicator '*' is the final character of the path name. When using generic path names, all sub-directories are included recursively.This parameter is only used if the SCTYPE parameter is set to *BSF.For more information about path object specifiers, see Path Object Specifiers on page 53.

• DATEThe date when the sync check will be started. The date that you enter must be entered in MMDDYY format, where the first two digits represent the month, the second two digits represent the day, and the last two digits represent the year (for the year 2000, set YY to 00).Enter a specific date or the following value:• *CURRENT—Specifies that the sync check will be performed when you issue this command.

The default setting for this parameter is *CURRENT.• TIME

The time when the sync check will be started. The time that you enter must be in the six-digit format HHMMSS, where the first two digits represent hours, the second two represent minutes, and the last two represent seconds.Enter a specific time or the following value:• *CURRENT—Specifies that the sync check will be performed when you issue this command.

The default setting for this parameter is *CURRENT.• OUTPUT

The type of sync check report that should be generated.You can generate a complete sync check report that displays information about all objects included in the sync check, regardless of whether they are synchronized on the primary and backup nodes. Depending on the number of objects included in the sync check, this report can be long and difficult for navigation purposes.Alternately, you can generate a report that displays only those objects that are not synchronized on the primary and backup nodes.

Note: This parameter only applies to *FILEATTR, *LIST, and *DTAARA sync checks. *BSF sync check does not produce a listing of objects. Use the DMSCRPT—Sync Check Report for IFS Objects command to obtain a *BSF sync check report.

Enter one of the following values:• *MISMATCH—Displays only those objects that are not synchronized when writing sync check report.• *ALL—Displays all objects, regardless of whether they match or not.• *NONE—Indicates that no spool file report is to be generated for the sync check. The output is still sent to a

database file. Use the DMSCRPT—Sync Check Report for IFS Objects or DMSCRPTNTV—Sync Check Report for Native Objects command to view the sync check report in the database file.

The default setting for this parameter is *MISMATCH.

ExamplesSTRHASCUSR TARGET(TGT1) GROUP(GRP1) SCTYPE(*FILEATTR) OBJSPEC(*ALL LIB1 *FILE PF) DATE(*CURRENT) TIME(*CURRENT) OUTPUT(*MISMATCH)

Indicates that the sync check command will be begin at the current date and time for the backup node TGT1 and the group GRP1. It will check the selected attributes for all *FILE objects in the library LIB1. The sync check report will display only those objects that are not synchronized.

Note: It is important that at least one space separates each item contained in the OBJSPEC parameter. Follow the examples that are provided to specify this parameter correctly.


UseYou must issue this command on the primary node of the specified replication group.


DMSCRPT—Sync Check Report for IFS ObjectsDMSCRPT GROUP( ) SORTBY( ) OUTPUT( )

You can use this command to generate a report based on current information in the sync check database file. Using this command, you can:• Display the report on your screen or send the results to a spool file.• Sort output by group name or object name.• List groups and objects that are out-of-sync as of the last sync check that was run for the group(s).

This command will only report results for the backup node where it is issued and for the group(s) that you specify. The node can be active or inactive.

Note: This command can be issued anywhere, but only reports sync check results for the backup node where it is issued.

Note: You can only run this command for the *BSF sync check type.


The name of a group that has been sync checked and is used to generate a sync check report or the special value *LOCAL.Enter the name of the group or the following value:• *LOCAL—All the groups that have the current node as their backup node are used in the sync check report.

*LOCAL is the default setting for this parameter.• SORTBY

This parameter allows you to sort the output (alphabetically) of the sync check report by group name or object name.Enter one of the following values:• *GROUP—Sort the sync check results by group name.• *OBJECT—Sort the sync check results by object name.

*GROUP is the default setting for this parameter.• OUTPUT

This parameter allows you to specify whether you would like to view the sync check report on your screen or send the results to a spool file.Enter one of the following values:• *—The sync check report is displayed on your screen.• *PRINT—The sync check report is placed in a spool file.

'*' is the default setting for this parameter.

ExamplesDMSCRPT GROUP(*LOCAL) SORTBY(*GROUP) OUTPUT(*)


Sync Check Commands

All the groups that have the current node as the backup node are used in the sync check report.The output of the report is sorted (alphabetically) by group name.The output of the report is displayed on your screen.

UseThe command must be used on the backup node for the group(s) that you specify in this command. The node can be active or inactive.


DMSCRPTNTV—Sync Check Report for Native ObjectsDMSCRPTNTV GROUP( ) DETAIL( )

You can use this command to generate a report based on current information in the sync check database file for native, library-based objects on the backup node. Using this command, you can:• Send a report containing sync check results to a spool file.• List groups and objects that are out-of-sync as of the last sync check that was run for the group(s).

This command will only report results for the backup node where it is issued and for the group(s) that you specify. The node can be active or inactive.

Note: This command can be issued anywhere, but only reports sync check results for the backup node where it is issued.

Note: You can only run this command for the *FILEATTR, *DTAARA, *LIST, and *CHKJRN sync check types.


The name of a group that has been sync checked and is used to generate a sync check report or the special value *LOCAL.Enter the name of the group or the following value:• *LOCAL—All the groups that have the current node as their backup node are used in the sync check report.

*LOCAL is the default setting for this parameter.• DETAIL

Controls whether detailed information about attribute mismatches is displayed.Enter one of the following values:• *NO—Mismatching objects are listed without attribute details.• *YES—Mismatching objects are listed along with each of their attributes that differs.

Note: *YES is valid for *FILEATTR and *DTAARA sync checks only.*NO is the default setting for this parameter.

ExamplesDMSCRPTNTV GROUP(*LOCAL) DETAIL(*NO)

All the groups that have the current node as the backup node and for which a sync check has been performed are used in the sync check report.The output of the report is sent to a spool file named DMSCRPTNTV.


UseThe command must be used on the backup node for the group(s) that you specify in this command. The node can be active or inactive.

Menu AccessF22 (Shift + F10) - Option 95

PRGHASC—Purge Sync Check ResultsPRGHASC PURGE( )

Use this command to clear the sync check database files of obsolete records. Obsolete records are records for groups that no longer exist and objects that are no longer replicated. These records can accumulate when a group is removed or an object specifier is deselected from a group when a group's backup node is not available. This command also provides an option to clear all records from the sync check database files.

Note: This command must be issued on the node where the sync check database files are located. This is always the backup node of a group.

Input Parameters• PURGE

This parameter allows you to choose between removing obsolete records or all records from the sync check database files.Enter one of the following values:• *ALL—Purge all records from the sync check database files.• *OBSOLETE—Purge only obsolete records from the sync check database files.

*OBSOLETE is the default setting for this parameter.

ExamplesPRGHASC PURGE(*ALL)

Removes all records from the sync check database files.

UseThis command must be issued on the node where the sync check database files are located. This is always the backup node of a group.


Registered iCluster User Commands

Registered iCluster User CommandsThis section contains the following topics:• DMADDUSR—Add User on page 277• DMCHGUSR—Change User on page 278• DMRMVUSR—Remove User on page 280• DMWRKUSR—Work with Users on page 281

DMADDUSR—Add UserDMADDUSR USER( ) AUTH( ) DESC( ) PASSWORD( )

Use this command to register an OS/400 user name in iCluster so that an administrator, operator, or user can perform certain cluster operations. For each user name that is added through this command, a security level (Administrator, Operator, or User) must be identified in order to define the cluster operations that can be performed by the user.The specified user is only registered on the node where this command is issued.If you are using the iCluster Administrator to manage remotely your clustered environment, the user name (USER) and password (PASSWORD) specified through this command must also be entered on the iCluster Administrator Login dialog box in order to work with the application on a Windows workstation. See iCluster Administrator for Windows on page 383 for more information about the iCluster Administrator.For more information about security in iCluster, see Command Security on page 63.

Input Parameters• USER

An OS/400 user profile name that is defined on the node where this command is issued.• AUTH

The security level that you want to assign to the user name specified through the USER parameter (see above).Specify one of the following values:• *USER—Grants user privileges to the specified user.• *ADMIN—Grants administrative privileges to the specified user.• *OPERATOR—Grants operator privileges to the specified user.

Note: iCluster administrators and operators must have *IOSYSCFG (input/output system configuration) authority to invoke commands at these levels.

The default setting for this parameter is *USER.See Command Security on page 63 to identify the cluster operations that can be performed at each level.

• DESCA short description that allows you to identify this user among all others that have been defined in iCluster.The description can be a maximum of 50 characters long. Typically, this description will be used to identify the full name of the user.

• PASSWORDThe iCluster password that you want to associate with the user name specified through the USER parameter (see above).


This password is not the same one that is used to log on to the OS/400 under the same name. It is an iCluster password, and so it can be different from the current password used to log on to the OS/400 system. For security reasons, DataMirror recommends that a different password is used for iCluster. This password must be specified during the login process for iCluster Administrator.The password cannot exceed 10 characters in length, and must only contain alphanumeric characters. The specified password is also case-sensitive.Specify a password or the following value:• *—Indicates that no password has to be specified with the user name identified through the USER parameter (see

above). If you specify this value, you cannot log on to the iCluster Administrator, but you can use the iCluster commands identified in this guide.

The default setting for this parameter is *.

ExamplesDMADDUSR USER(MJONES) AUTH(*OPERATOR) DESC('Mary Jones') PASSWORD(LEMONADE)

Registers the OS/400 user name MJONES (Mary Jones) in iCluster with operator privileges.DMADDUSR USER(BSMITH) AUTH(*ADMIN) DESC('Bruce Smith') PASSWORD(APPLEJUICE)

Registers the OS/400 user name BSMITH (Bruce Smith) in iCluster with administrative privileges.

UseYou cannot issue this command by the DMCLUSTER, OMIRROR, or HASUITE user names. These names are created for the installation of iCluster and other DataMirror products.

Minimum Security LevelQSECOFR or a user name having *SECADM authority in the following situations:• As part of their basic user profile.• As part of their primary group profile.• As part of any of their supplemental profiles.


DMCHGUSR—Change UserDMCHGUSR USER( ) AUTH( ) DESC( ) PASSWORD( )

Use this command to change the security level of a user name that is currently defined in iCluster. The user name that is specified must have already been added through the DMADDUSR—Add User command.The specified user must be registered on the node where this command is issued.If you are using the iCluster Administrator to manage remotely your clustered environment, the user name (USER) and password (PASSWORD) specified through this command must also be entered on the iCluster Administrator Login dialog box in order to work with the application on a Windows workstation. See iCluster Administrator for Windows on page 383 for more information about iCluster Administrator.For more information about security in iCluster, see Command Security on page 63.

Input Parameters• USER

The iCluster user name that will have its security level changed by this command. This user name must already be registered on the node where this command is issued.



• AUTHThe security level that you want to change to for the user name specified through the USER parameter (see above).Specify one of the following values:• *USER—Changes the security level to user for the specified user.• *ADMIN—Changes the security level to administrative for the specified user.• *OPERATOR—Changes the security level to operator for the specified user.• *SAME—Uses the current setting for this parameter.

Note: iCluster administrators and operators must have *IOSYSCFG (input/output system configuration) authority to invoke commands at these levels.

The default setting for this parameter is *SAME.See Command Security to identify the cluster operations that can be performed at each level.

• DESCA short description that allows you to identify this user among all others that have been defined in iCluster.The description can be a maximum of 50 characters long. Typically, this description will be used to identify the full name of the user.Specify a short description or the following value:• *SAME—Uses the current setting for this parameter.

The default setting for this parameter is *SAME.• PASSWORD

The iCluster password that you want to associate with the user name specified through the USER parameter (see above). You can change the password associated with the user name or keep it the same (see * below).This password is not the same one that is used to log on to the OS/400 under the same name. It is an iCluster password, and so it can be different from the current password used to log on to the OS/400 system. For security reasons, DataMirror recommends that a different password is used for iCluster. This password must be specified during the log in process for iCluster Administrator.The password cannot exceed 10 characters in length, and must only contain alphanumeric characters. The specified password is also case-sensitive.Specify a password or one of the following values:• *NONE—Indicates that no password has to be specified with the user name identified through the USER parameter

(see above). If you specify this value, you cannot log on to the iCluster Administrator, but you can use the iCluster commands identified in this guide.

• *—Uses the current password for the user.The default setting for this parameter is '*'.

ExamplesDMCHGUSR USER(MJONES) AUTH(*OPERATOR) DESC('Mary Jones') PASSWORD(ROOTBEER)

Changes the security level of user name MJONES (Mary Jones) to operator, and her iCluster password to ROOTBEER.DMCHGUSR USER(BSMITH) AUTH(*ADMIN) DESC('Bruce Smith') PASSWORD(*)

Changes the security level of user name BSMITH (Bruce Smith) to administrative. The iCluster password associated with BSMITH is not changed.



Note: You must restart the iCluster Administrator before password changes will take effect.



DMRMVUSR—Remove UserDMRMVUSR USER( )

Use this command to remove the OS/400 user name registered in iCluster. When a user is removed by this command, that user can no longer able to work with the cluster environment through iCluster.Note that issuing this command does not delete the OS/400 user name from the node on which it is invoked. Only user registration on the node where you issue this command is removed.For more information about security in iCluster, see Command Security on page 63.

Input Parameter• USER

The OS/400 user name that will be removed from iCluster. This user name must already be registered in iCluster on the node where this command is issued.

ExampleDMRMVUSR USER(MJONES)

Removes the user name MJONES from iCluster.


Note: You must restart the iCluster Administrator before password changes will take effect.





DMWRKUSR—Work with UsersDMWRKUSR

Use this command to display a screen that allows you to work with the user names that have been granted certain privileges to work with nodes, groups, object specifiers, and other cluster components in iCluster.From the screen that is displayed, you can add or remove user names as well as change security attributes for a user. For more information security in iCluster, see Command Security on page 63.


ExampleDMWRKUSR

Displays a screen that allows you to work with user IDs that have been given certain privileges to view or modify iCluster components.





Exit Program Commands

Exit Program CommandsThis section contains the following topics:• ADDPFEXPGM—Add Exit Program on page 283• RMVPFEXPGM—Remove Exit Program on page 283

ADDPFEXPGM—Add Exit ProgramADDPFEXPGM

Use this command to register a DataMirror exit program that provides the ability for iCluster to replicate changes to the following attributes for both physical and logical files: Text description, Maximum number of members, Access path maintenance, Access path recovery, and Records to force a write. This command also enables replication of changes to the Size attribute for physical files.

Note: This command registers two exit programs (one for physical files and one for logical files) for the QIBM_QCA_RTV_COMMAND exit point. If there are more than 20 exit programs already defined for this exit point, then this command will not be able to add the exit programs.



ExamplesADDPFEXPGM

This command registers the DataMirror exit program.

UseYou can issue this command on the primary node and backup node.This command is issued by the iCluster installation process on Version 4 Release 5 or later of the operating system.

RMVPFEXPGM—Remove Exit ProgramRMVPFEXPGM

Use this command to remove a DataMirror exit program that provides the ability for iCluster to replicate changes to the following attributes for both physical and logical files: Text description, Maximum number of members, Access path maintenance, Access path recovery, and Records to force a write. Replication of changes to the Size attributes for physical files is also enabled by the ADDPFEXPGM—Add Exit Program command.Removing the exit program means that only the contents of physical and logical files will be replicated. Changes to the attributes listed above will not be replicated. You can re-add the exit program by issuing the ADDPFEXPGM—Add Exit Program command. See ADDPFEXPGM—Add Exit Program on page 283 for more information.




ExamplesRMVPFEXPGM

This command removes the DataMirror exit program.

UseYou can issue this command from the primary node and the backup node.This command is available only for Version 4, Release 5 or later of the operating system.


iCluster for Supported External Storage Systems

iCluster for Supported External Storage SystemsThis section contains the following topics:• DMRBDNODE—Rebuild Node on page 285• DMREGPOS—Register Positions on page 286

DMRBDNODE—Rebuild NodeDMRBDNODE NODE( )

Use this command to rebuild the specified node as a node in the cluster, based on recovery domain information for all groups that have the specified node as their backup node. Typically, this command should be issued after a refresh for a supported external storage system.

Note: This command is only available if the iCluster authorization code is installed. See Authorization Codes and Licensing on page 20 for more information.

Specifically, this command performs the following operations:• Saves the recovery domain for all groups with the backup node in the recovery domain.• Removes the backup node from the cluster.• Re-adds the backup node as a node in the cluster.• Establishes the proper iCluster authorization code for the backup node. • Restores the recovery domains for all groups with the backup node in the recovery domain prior to the refresh for

supported external storage systems.Warning: This command should be used only if the specified node is no longer part of the cluster after performing a

refresh for a supported external storage system.


The system name of the backup node as set in the IPL exit program.

OutputNone.

ExamplesDMRBDNODE NODE(NODE_B)

The backup node, NODE_B, will be restored as a node in the cluster, based on recovery domain information for groups relating to NODE_B.

UseYou should note the following considerations about issuing this command:• The node to be rebuilt must be active or the cluster on that node must be deleted prior to invoking this command.• The DMCLUSTER product library must be the current library when invoking this command.• Replication must be inactive for all groups associated with the specified node at the time this command is invoked.• This command assumes that no groups have the selected node as their primary node.

Warning: For any groups for which the specified node is the primary node in the group, a failover will occur. See Failovers and Switchovers on page 47 for general information about failovers. See Failover Mechanisms on page 47 for more information about the different failover mechanisms available to you.


DMREGPOS—Register PositionsDMREGPOS GROUP( ) NODE( )

This command registers starting journal positions for a specific group or for all groups that have the specified node as the primary node of the group. The registered starting journal positions are used as starting points before starting replication for a supported external storage system.

Note: Typically, you need to invoke this command manually during a refresh for a supported external storage system when the primary node is set to a restricted state.

Starting journal positions for a group registered with this command will be used automatically when replication is next started for the group. Once replication has been started successfully, the registered positions are not re-used. AIf you have used the DMREGPOS command to record registered starting journal positions, do not issue the CHGHAJRN command for any journals on the primary node prior to re-establishing replication. Once replication has started, you can then issue the CHGHAJRN command to remove fully processed journal receivers.The starting journal positions registered by this command will be overridden in the following situations:• Starting positions specified by a subsequent call to the DMSETPOS—Set Journal Start Position command.• Specifying *YES on the USEMARKED (Start with Marked Positions) parameter of the DMSTRGRP—Start Cluster

Operations at Group command.Note: This command supports spooled file replication on the primary and backup node(s) as of i5/OS V5R4.


The name of the group for which starting positions will be registered. Enter the name of a group or the following value:• *ALL—Specifies all groups.

• NODEThe name of a cluster node for which all groups having that node as the primary node of the group will have starting positions registered. This parameter may be specified only if the special value *ALL is specified on the GROUP parameter.Enter the name of the node or the following value:• *CURNODE—Specifies the current node. This is the default setting for this parameter.

OutputJob log messages indicating successful completion or problems are interactively displayed.

ExampleDMREGPOS GROUP(*ALL) NODE(*CURNODE)

Starting journal positions are registered for all groups on the current node.

UseYou should note the following considerations about issuing this command:• The primary database must be inactive when this command is issued. If the primary database is active at the time this

command is called, all transactions may not be replicated.• Before mirroring is started, you need to make sure that the image of the backup database is consistent with the image of

the primary database.• Replication must be inactive for all groups specified at the time this command is invoked.


iCluster for Supported External Storage Systems

• This command should not be invoked during the "duplicate time" if the system time is changed backwards. Unpredictable results may occur in these situations.


Other Commands

Other CommandsThis section contains the following topics:• SETHAREG—Restore Communications Registries on page 289

SETHAREG—Restore Communications RegistriesSETHAREG

Use this command to restore communications registries that have been corrupted through node failure, interruption, or other circumstances.


OutputNone.

ExamplesSETHAREG

Restores communications registries on the local active node.

UseYou must issue this command on the node where the operation will be performed.The iCluster product library must be the library of the object that runs the command.


Using the Status MonitorThis section contains the following topics:• Working with the Status Monitor on page 291• Simplified Status Monitor on page 292• Latency on page 292• Status of Apply Jobs on page 292• Real Time and Historical Statistics on page 292• Interactive Inquiry Screens on page 292• Real Time Statistics Views on page 293• Working with Object Status on page 306• Monitoring Historical Statistics on page 314

Working with the Status MonitoriCluster allows you to monitor replication on both the primary and backup nodes. This is especially useful when the primary and backup nodes are located in different geographical regions. Regardless of where you happen to be situated (in either the primary or backup node environment), replication can be monitored from either site.Monitoring is an important component of a high availability solution because it allows you to identify conditions that may require your attention. Monitor reporting is based on group/journal combinations. The following statuses and statistics are provided for both real time inquiry of replication processes and historical inquiry of past activity:• Replication status (both active and inactive)• Real time object latency• Real time overall latency• Backup node latency• Runtime totals for both primary and backup replication processes• Overall and current transactions per hourNote: The throughput statistics are presented in units called transactions. This number is greater than or equal to

the number of journal entries processed, due to the processing requirements of certain journal entries.You can also perform the following operations for specified group/journal combinations:• Start and end replication• Perform sync checks on objects (on the primary node only)• View event logs• View sync check results• View suspended objectsAll information for active or inactive groups is shown on the same screen. Except for the Status column, information about an active item will be highlighted. You can view expanded statistics to have complete information about replication performance and status.Note: The primary monitor must communicate with the backup node to retrieve its data, therefore there may be a

delay in displaying the primary monitor screen.


Simplified Status MonitorIn iCluster, you have the option of accessing the full Status Monitor, which contains sync check, activate, suspend operations, among others, or you can access the simplified Status Monitor, which displays only the details and the status of objects for latency, throughput, object position, and journal information. For more information about the simplified Status Monitor, see DSPHASMON—Display Source Monitor on page 255.

LatencyLatency is the time interval between a journal entry being written on the primary node and it being applied on the backup node. It is sometimes expressed as the number of entries between the last journal entry written and the last journal entry applied. High latency can occur when the volume of transactions to be sent exceeds the available bandwidth.Latency may become very large (several hours) if the apply process is suspended. The scrape latency will not be affected during apply suspension as long as the staging store is large enough to hold all the journal information. See Staging Objects on page 69 for more information.iCluster provides statistics about real time and historical latency so that you can examine the speed of replication from the primary node to the backup node and take any necessary action. See Monitoring Latency on page 78 for more information about the latency monitoring tools that iCluster provides.

Status of Apply JobsYou can use the Status Monitor to view the status of the apply jobs. These states are explained on each of the real time Status Monitor screens.

Real Time and Historical StatisticsiCluster provides interactive screens that allow you to view the runtime statistics of active and inactive replication processes. The interactive screens access replication processes to provide real time statistics. These statistics are recalculated upon screen refresh.iCluster also provides historical replication statistical screens. These screens allow you to view previous replication activity. The historical statistics consist of activity snapshots and job start and end statistics. With these historical statistics, you can determine replication trends (for example, periods of high activity) and reconcile replication processes.iCluster acquires historical statistics on the primary node by taking snapshots of the replication activity of group/journal combinations at a user-defined frequency. The replication process will read the replication statistics and write timestamped records to the Status Monitor history database.iCluster provides job start and end statistical records that are stored in the monitor database. These records are created at the start and end of processing, allowing the user to easily reconcile the replication activity.Normally, records will be written containing data for both the primary and backup nodes. In the event that data is unavailable, a partial data record (such as type SS, if only primary node start data is available) is written to the monitor database.You can control the frequency of data collection through the CHGHASMON—Change History Monitor on Primary Node command on the primary node.Note: If the process reporting frequency is set to a high rate (providing almost real time statistics) the performance

of the mirroring processes may be adversely affected.

Interactive Inquiry ScreensiCluster provides you with interactive screens that allow for the easy display of activity.From these screens, you can view the following replication information:


• Monitoring Real Time Object Latency• Monitoring Real Time Overall Latency• Monitoring Real Time Object Position and Totals• Monitoring Real Time Object Throughput• Monitoring Historical Object Position and Totals• Monitoring Historical Object Throughput

Real Time Statistics ViewsThe iCluster real time status monitor splits its information across multiple views. This section introduces the status monitor and provides information that is common across all of the status monitor views.

Starting the Status MonitorTo display statistics for the current node as a primary node, start the status monitor by either:• Selecting option 80 (Display the iCluster Primary Monitor) from the iCluster main menu.• Running the WRKHASMON—Work with Status Monitor on Primary Node command from the command line.To display statistics for the current node as a backup node, start the status monitor by either:• Selecting option 81 (Display the iCluster Backup Monitor) from the iCluster main menu.• Running the WRKHATMON—Work with Status Monitor on Backup Node command from the command line.

Cycling Through the ViewsWhile the status monitor is open, you can cycle through its views by pressing F11. This will display the next view.The views that are available depend on if your terminal is set for 132 column display or 80 column display. If you are viewing the primary or backup monitor on a 132 column display, then you will see the views shown in Figure 1.

Figure 1 View Sequence for 132 Column Terminals


If your terminal is set for 80 column display, then you will not see the Real Time Overall Latency view. Figure 2 shows the sequence of views you will see.

Figure 2 View Sequence for 80-Column Terminal

For more information on each of these views, see the following topics:• Monitoring Real Time Overall Latency on page 297• Monitoring Real Time Object Latency on page 299• Monitoring Real Time Object Position and Totals on page 301 • Monitoring Real Time Object Throughput on page 302

Common Information on all ViewsEach view shares columns and other informational fields. They are as follows:Table 1 Common Information on All Views

Name Description

Position to field When you type a name in the Position to field, starting from the top of the list, the cursor moves to either the first backup node that matches your entry, or if there is no match in the list, then to the first backup node that appears prior to the item in the Position to field. You can also move to the top or the bottom of the list by entering *TOP or *BOT respectively.You can use the Position to field to search for primary nodes, backup nodes, groups and journals. You need to prefix your entry in the Position to field with an S for primary nodes, a T for backup nodes, G for groups or J for journal. Using these prefixes, you can perform a sequential search through the list from the current position for a primary node "S:", a backup node "T:", a group "G:" or a journal "J:". Again, if a match is not found, then the first item prior to what would match the text in the Position to field is selected.To search for a particular backup node, type the name of the backup node in the Position to field. Status Monitor will scroll to the specified backup node.


Common Options for all ViewsThis topic describes the options and tasks that you can perform from each status monitor view.

Performing Tasks with Function KeysYou can perform the following tasks at any time by pressing its associated function key.

Elapsed time field Displays the amount of time that has passed since the status monitor was last restarted. See Common Options for all Views on page 295 for more information on the restart option.

Prm/Bkp/Grp/Jrn column When viewing the primary monitor, there will only be one entry for the primary node as you can monitor only one primary node at a time. When displaying the backup monitor, there can be more than one primary node entry, as there is conceptually no restriction on the number of primary nodes that can replicate to a particular backup node.This view lists all active and inactive primary/backup/group/journal combinations in the column. A primary system is indicated when the name is at the left of the column, a backup system is indicated when the name is indented by two spaces, a group is indicated when the name is indented by four spaces, and a journal is indicated when indented six spaces from the left side of the margin. Active backup nodes are highlighted.

Table 2 Function Keys

Key Label Description

F5 Refresh Refreshes the Status Monitor screen. Entries will stay in the same position in the list, and the cursor will remain in its current position.

F9 Sync Check Results Runs the DSPHASC—Display Sync Check command to display the Work with All Spooled Files screen. This shows the available sync check result files.

F10 Re-Start Resets the timer that populates the Elapsed Time field in the top-right corner of the status monitor. This also refreshes the Status Monitor.

F11 Next View Displays the next view for the status monitor. See Real Time Statistics Views on page 293 for more information on the sequence of views.

F21 (SHIFT+F9) Command Displays an OS/400 command prompt.

F22 (SHIFT+F10) DM Cmds Exits the status monitor and displays the iCluster Main Menu.

Table 1 Common Information on All Views

Name Description


Selecting OptionsYou can enter the following options in the Opt column to the left of a node, group, or journal to perform an action on a specific object.Table 3 Options in the Status Monitor

Opt Label Description

1 Start Starts replication for the selected group on the primary node. On the backup node, the apply process will start.This option runs DMSTRGRP—Start Cluster Operations at Group on page 208.

2 Wrk Obj Specs Show the object specifiers for the selected group.This option runs the DMWRKOBJ—Work with Object Specifiers command.

4 End Ends replication on the primary node, or ends the apply processes on the backup node, for the selected group. The Status column will display "I", indicating that it is now inactive.This option runs the DMENDGRP—End Cluster Operations for Group command.

5 Details Display the name of the journal receivers on the primary and backup nodes, the libraries where the receivers reside and their last replication positions for the selected group/journal combination.

6 Msgs Displays the cluster event log for the selected group.

7 BSF Sts Displays the WRKHABSFST—Work with BSF Status screen for the selected group or journal.

8 Obj Sts Displays the WRKHAOBJST—Work with Object Status screen for the selected group or journal.

9 Chk backup Opens a communications link to the backup node so that you can check the status of an apply job from the source monitor. Any group that is not active will display the unknown status in the Status column for the backup node. By default, the Status Monitor does not open the communications link so that it does not become delayed during initialization or refresh. However, once selected, the communication link stays in effect for as long as the screen is displayed or communication to that backup node is available.If communication fails for a backup node, the option is cancelled. This prevents a failed retry each time the screen is refreshed. You will need to select this option again to re-start communications.This option is only available from the primary monitor.

12 History Displays the Monitoring Historical Latency screen the selected journal.


Monitoring Real Time Overall LatencyThe Real Time Overall Latency view (Figure 3) displays the real time overall latency for all primary/backup/group/journal combinations. Since this is a real time display, the information is recalculated every time the screen is refreshed. This provides a useful summary of the latency information for all of the backup nodes of the current node.

Figure 3 Real Time Overall Latency View - Primary Monitor

Latency can only be calculated when the current node has an active connection to each backup node. This calculation will not occur for inactive backup nodes unless they are explicitly contacted using option 9. See Common Options for all Views on page 295 for more information on this option.

Displaying this ViewTo display this view, start either the primary or backup monitor with a 132 column terminal. From a running status monitor, press F11 to cycle through the views until you see this one.See Real Time Statistics Views on page 293 for more information.

54 Start Synchck Initiates a sync check for the selected group by running the STRHASC—Start Sync Check command. This option is only available from the primary monitor.

56 Start User Synchck Initiates a user sync check for the selected group by running the STRHASCUSR—Start User Sync Check command. This option is only available from the primary monitor.

Table 3 Options in the Status Monitor

Opt Label Description


Reading this ViewThe Real Time Overall Latency view displays the following information:

See Real Time Statistics Views on page 293 for descriptions of information available to all status monitor views.

Performing TasksSee Common Options for all Views on page 295 for a complete list of options and tasks you can perform from this view.See Monitoring Latency on page 78 for more information on how to monitor latency in iCluster.

Table 4 Information on the Real Time Overall Latency Screen

Source Latency Source latency indicates the difference between the timestamp of the journal entry last processed by the backup receiver for the journal, and the timestamp of the last journal entry deposited in the primary journal. It is displayed in HH:MM:SS format.For idle or lightly-used groups, the receive timestamp is resynchronized every minute to ensure that reported latencies remain below specified thresholds.If the primary latency threshold has been exceeded, then brackets, "<>", are displayed around the primary latency time. See the DMSETSVAL—Set Cluster System Values command for more information on setting latency thresholds.

Apply Latency Apply latency is the difference between the timestamp of the journal entry last processed by the backup apply process for the journal, and the timestamp of the last journal entry processed by the backup receive process. It is displayed in HH:MM:SS format.If the apply latency threshold has been exceeded, then brackets, "<>", are displayed around the apply latency time. See the DMSETSVAL—Set Cluster System Values command for more information on setting latency thresholds.

Total Latency Total Latency is the sum of the source and target apply latency times. It is displayed in HH:MM:SS format.

Total Latency Status The Total Latency Status graphically represents the total latency. The bar gives an approximate indication of the Total Latency. Each character represents one unit of time depending on the sum of the Primary and Apply threshold values. The sum of the two thresholds appears 2/3 of the way across the bar graph.If no latency thresholds have been exceeded, the bar graph displays '.'. If either or both latency thresholds have been exceeded, the bar displays asterisks '*'.

Status See Reading Status Information on page 303 for details on the status fields.


Monitoring Real Time Object LatencyThe Real Time Object Latency view (Figure 4) displays the real time latency for all group and journal combinations. Since this is a real time display, the information is recalculated every time the screen is refreshed. This provides a useful summary of the latency information for all of the nodes connected to the current node.

Figure 4 Real Time Object Latency View - Primary Monitor

Latency can only be calculated when the current node has an active connection to each backup node. This calculation will not occur for inactive backup nodes unless they are explicitly contacted using option 9. See Common Options for all Views on page 295 for more information on this option.

Displaying this ViewTo display this view, start either the primary or backup monitor and press F11 to cycle through the views until you see this one. See Real Time Statistics Views on page 293 for more information.

Reading this ViewThe Real Time Object Latency view displays the following information:Table 5 Real Time Object Latency

Backup J/E The last journal entry applied to the backup node by the apply process.

Curr Trans Sent The current number of transactions processed on the primary node and received into the staging store since the start of the current replication process. This entry does not include omitted entries.



Performing TasksSee Common Options for all Views on page 295 for a complete list of options and tasks you can perform from this view.

Curr Trans Applied The current number of transactions processed by the primary node since the start of the last reset of the transaction counters that have been sent to the backup node and applied by the apply processes on the backup node.Note that inactive apply processes are not counted in the backup node and group summaries.Transaction counts are reset under the following conditions:• After issuing the DMSETPOS—Set Journal Start Position,

DMMRKPOS—Mark Current Journal Positions, or DMREGPOS—Register Positions commands

• Refresh before mirroring• Counts exceed approximately 10 billion

Trans to Process The number of transactions which have been sent by the primary node, but have not yet been applied on the backup node.


Table 5 Real Time Object Latency


Monitoring Real Time Object Position and TotalsThe Real Time Object Position and Totals view (Figure 5) displays the real time position and totals for all active and inactive group/journal combinations. Active combinations will be highlighted. The positions and totals are retrieved prior to initial display as well as upon every refresh of the screen. This provides both primary and backup node processing positions and the total number of entries processed. The Elapsed time field displays the elapsed time since the last re-start (F10) or initial display of the screen after a refresh (F5).

Figure 5 Real Time Object Position and Totals View - Primary Monitor


Reading this ViewThe Real Time Object Position and Totals view displays the following information:Table 6 Real Time Object Position and Totals View

Last J/E Last J/EThe last journal entry written to the journal.

Primary J/E The journal entry where the primary node journal scraper process is currently scraping the journal and has been received into the staging store.


Trans to Process The number of transactions which have been sent by the primary node, but have not yet been applied on the backup node.





Monitoring Real Time Object ThroughputThe Real Time Object Throughput view (Figure 6) displays the real time throughput in transactions per hour for the backup node apply for all active group/journal combinations. Active combinations will be highlighted. It displays both overall and current throughputs, providing 'at a glance' throughput values for all active backup nodes. These figures provide throughput in transactions per hour. Inactive group/journal combinations will appear on the screen but will not have throughput values.

Figure 6 Real Time Object Throughput View - Primary Monitor

Overall throughput is calculated based on the backup node apply process using figures (time period and transactions processed) based on the elapsed run time of the backup node apply job. Therefore the time period used is the start of the job to the last time the information was updated. This throughput rate is expressed in transactions per hour.The time sample is calculated as follows:(Backup Node Update Timestamp) less (Backup Node Start Processing Timestamp).The number of transactions is the total of all transactions processed on the backup node.Current throughput will also be based on the throughput of the backup node apply process using figures based on when the screen was originally displayed or since the previous re-start, to when a refresh of the screen is requested (similar to WRKHAJOB).



Reading this ViewThe Real Time Object Throughput view displays the following information:



Reading Status InformationEach view screen has a set of columns that display status information for the nodes, groups, and journals being monitored. This topic describes each of the following status columns:• The RPB Status column on page 304• Suspended Objects (S/O) Status column on page 305• Out-of-Sync (OOS) Status Column on page 305• Operational (OP) Status Column on page 305• Current Object (Curr Object) Column on page 306

Table 7 Real time Object Throughput View

Apply Str/End The start/end date of the latest replication process for the backup node/ journal combination.If the Status column displays a status of I (inactive), then this field is displaying the end time. Otherwise, it indicates the start date.

Elapsed Days Hr Mn The elapsed time since the latest apply process for the journal of the backup node/group combination began.

Overall Trans/Hr The number of transactions per hour based on the elapsed time of the backup node apply job.

Current Trans/Hr The number of transactions per hour based on the transactions performed since the last time the status monitor timer was restarted by pressing F10.When you change to this view or restart the status monitor timer, the Current Trans/Hr column will contain a '-' because the elapsed time is zero (0), and therefore no value can be calculated. Press F5 to refresh the screen. This will recalculate the value for this column based on the time that has passed since you restarted the status monitor timer.See Common Options for all Views on page 295 for more information on restarting the status monitor timer.



The RPB Status columnThe RPB status column represents the status of replication processes for remote journals, primary and backup nodes. The ‘R’ represents the replication status of remote journals, the ‘P’ represents the status of the scrape process on the primary node, and the ‘B’ represents the status of the receive and apply process on the backup node. Table 8 lists the status codes and their meanings.Table 8 Remote Journal, Primary and Backup Status Codes

Status Meaning

A Indicates that the replication process is active on the primary node, backup node, or remote journal.

I Indicates that the replication process is inactive on the primary node, backup node, or remote journal. This state occurs whenever replication is inactive.

S Indicates that a group is starting but not yet fully active, or is in the process of shutting down.

R Indicates that a refresh is active for a non-mirroring group. This status is not available for refresh before mirror. During a refresh before mirror, the status is S. This field also indicates those journals that are refreshing a particular entry.

F Indicates that the staging store is full, and is shown only on the primary node.

U Indicates that the journal is not being used by this process.If a journal is no longer required by a group on both the primary and backup nodes, it will no longer be displayed in the Status Monitor. In other words, a status combination of 'U U' is never displayed.

- Indicates that the status is unknown for the primary node, backup node, or remote journal. The status can be unknown for the backup node when the primary node is not actively communicating with the backup node. It can also be unknown for a primary node that has not been involved in replication.

* Indicates that the journal may be unused. This is set at startup and can change to a status of 'A' or 'I' if the process uses the journal.

blank A blank value only applies to remote journals and indicates that it is not a remote journal.


Table 9 describes some sample RPB codes and then explains what they mean.

Suspended Objects (S/O) Status columnThis column shows the number of suspended objects in this group. These objects match the object specifier but cannot be replicated. If there are no suspended objects, then this field will appear blank.

Out-of-Sync (OOS) Status ColumnThis column displays the sum of the OS/400 native objects and IFS objects that are out-of-sync for each group. This information is current as of the last sync check.Note: The OOS column is only available on backup monitor screens in a 132-column terminal session.

Operational (OP) Status ColumnThis column indicates the current operational status for each journal being used. If an operation is being performed, then it will display one of the following codes:• RBD: The group is waiting for the system to finish rebuilding a logical file. • RFS: The operation is refreshing a file or IFS object. This can be the result of a group refresh or of an individual file or IFS

object being activated by a refresh. For database file members, the progress of the refresh is indicated as a percentage following this code.

• RGZ: The operation is reorganizing a file. • SWO: The group is performing a switchover.The OP column also indicates the switchover status for each group. This column displays a blank if a group is only being used for mirroring, but it will indicate if a switchover is in progress. The current step during the switchover is indicated in the adjacent Current Object column that is available on the backup monitor.

Table 9 Sample RPB Status Codes

RPB Meaning

-*- The status of the remote journal is unknown. The journal may be unused on the primary node, and the status is unknown on the backup node.

I*I The remote journal is inactive. The journal may not be used on the primary node, and the apply process is inactive on the backup node.

A*A The remote journal is active. The journal may not be used on the primary node, and the apply process is active on the backup node.

-** The status of the remote journal is unknown. The journal may not be used on both the primary and backup nodes.

AU- The remote journal is active. The journal is not being used on the primary node, and the status is unknown for the backup node.

IUI The remote journal is inactive. The journal is not being used on the primary node, and the latent apply on the now unused journal is inactive on the backup node.

UA The journal is not a remote journal. The journal is not being used on the primary node, and the latent apply on the now unused journal is active on the backup node.


Note: The OP field is only available on primary and backup monitor screens in a 132-column terminal session.

Current Object (Curr Object) ColumnAt the journal level, this column displays the name or path of the target object that the status in the OP column applies to. If the object is a native object, then this column will show the object as LIBRARY/FILENAME(FILEMEMBER). If the object is an IFS object, then the column will show the last thirty characters of the path and filename.Note: The Curr Object column is only available on backup monitor screens in a 132-column terminal session.

Working with Object StatusThe Work with Object Status screen (Figure 7) displays the status of objects within the Status Monitor. For suspended objects, this screen also displays the reason for the suspension. For more information about suspended objects, see Suspended Objects on page 72.

Figure 7 Work with Object Status Screen

Displaying the Object Status ScreenTo display the Object Status screen from the Status Monitor, enter option 8 (Obj Sts) in the Opt column next to the journal you want to display status information for. You can display this screen from either the primary or backup monitor.To display this screen from the command line, run the WRKHAOBJST—Work with Object Status command.

The Object Status ScreenThis screen lists all objects for the specified group/journal combination. It contains three views. The first view shows only suspended objects. By pressing F16 (SHIFT+F4), you can cycle through the views to show only suspended objects, then only active objects, and then all objects.The backup node, name, group, and library of the journal is displayed at the top of the screen. The Object Status screen also displays the following information:Status—Indicates the status of each object. The possible statuses are as follows:

• ACTIVE—The object is available for replication.• PNDACT—Active pending. The object is currently suspended and waiting to become active through a journal entry

that must be scraped.


• PNDSUS—The object is active and waiting to be suspended through a journal entry that must be scraped.• SUSPND—The object is currently suspended.

Objects—The replication object this status describes.Type—The object type of the object.Library—The library the object belongs to.Attribute—The object extended attribute for the object.Jrn Pos—The object's current journal position.Reason—Indicates the reason why the object was suspended. The three-character reason codes are as follows:

• ABU—The object was activated by the user.• AIS—The object will be activated as soon as this entry is scraped from the journal by iCluster.• APD—The object's activation has started but has not finished.• AUD—Object auditing cannot be started for a BSF object.• AUS—Private authorities associated with the BSF object could not be replicated or retrieved.• AUT—Private authorities for an object could not be retrieved.• BTN—Metadata for a BSF object cannot be found on the backup node.• CDA—The change content operation to a data area failed.• CHK—Temporary state showing that iCluster tried to refresh file during rename/move.• CIL—Unable to determine if a BSF object is a hard link.• CJN—A required journal entry associated with the object could not be found in the audit journal.• COM—An object could not be refreshed.• CPT—A journal entry required to refresh the object could not be added to the database journal.• CRT—The BSF object cannot be created on the backup node.• DLT—A BSF object could not be removed from the backup node.• DTA—The BSF object could not be opened on the primary node for refreshing.• EJF—An object was suspended on the primary node because journaling for the object ended on this node.• EXS—iCluster attempted to create an IFS folder that already existed.• INT—An object was suspended as a result of an internal failure. Synchronization between objects on the primary

and backup nodes cannot be assumed.• IOF—A read or write operation failed.• JPF—A logical file was suspended on the primary node because the associated physical file could not be journaled.• JPO—A required journal entry related to the object could not be found in the database journal.• JRN—Journal information for an object could not be retrieved, or the object could not be journaled.• LCK—A lock on an object could not be obtained.• LNK—The BSF object is a hard link (replication of hard links is not supported by iCluster).• MCN—The object is temporarily suspended during the renaming/moving of a database table, where the original

object is selected with MIRRCNTS(*NO) and the new object with MIRRCNTS(*YES). The object will be activated by iCluster when the renaming/moving of the database table is complete on the target. For more information on the MIRRCNTS parameter, see the DMSELOBJ—Select Objects to Group command.

• MDF—A problem occurred when creating or updating metadata related to the object.


• MLF—An object was suspended on the backup node because a member-level failure (rename, delete, reorganize, and so on) occurred during replication.

• MMC—A master-master data conflict occurred.• MML—The file is suspended on the primary node. LOB column replication is not supported in a master-master

group.• MMO—A master-master object-level error occurred.• MMR—A master-master refresh failed.• MRR—An object was suspended on the primary node. The object should have been refreshed manually, but it has

yet to be activated.• NFD—Object not found on the system.• NGP—A logical file was suspended on the primary node because the associated physical file was not replicated in

the same group as the logical file.• NRE—File has no record in the metadata.• NSI—A BSF object exists on the backup node, but not on the primary node.• NSO—Object replication is not supported in the current mode.• NTI—The state associated with a BSF object could not be found on the backup node.• OLF—An object level failure occurred while trying to rename or move a non-journaled object.• OWN—The owner of BSF object could not be changed on the backup node.• PCK—Compression of the BSF object path was unsuccessful.• PDA—Uncommited DO "delete object" entry is received for an active object under commitment control.• PDS—Uncommited DO "delete object" entry is received for a suspended object under commitment control.• PDU—Uncommited DO "delete object" entry is received for an object suspended by the user under commitment

control.• PGP—The primary group of a BSF object could not be changed on the backup node.• PND—Activate pending for the object engine.• RBC—The file was part of a cancelled rollback operation on the primary node.• RDQ—The receive operation to a data queue failed.• RFF—A refresh of a BSF object was unsuccessful.• RFS—A refresh of a BSF object could not be started from the primary node.• RLE—An object was suspended on the backup node because the number of I/O errors generated when replicating

to the object exceeded the value specified in the Max. record level errors cluster system value. See DMSETSVAL—Set Cluster System Values on page 181 for more information.

• RMV—A RMVJRNCHG journal entry was processed for the object. The object is suspended and will be refreshed later by the automatic re-activation function.

• RNM—Rename or move operation failed.• RSF—An object was suspended on the backup node. It should have been refreshed, but the restore operation

failed on the backup node.• RST—A BSF object was restored. The object needs to be refreshed.• RTN—An unexpected return code while dealing with an object.• RTV—The file's description could not be retrieved.• RWA—An object was suspended on the primary node because a record-by-record refresh failed.


• SBU—An object was explicitly suspended on the primary node as a result of issuing the DMSUSOBJ—Suspend Object or DMSUSBSF—Suspend BSF Object command.

• SCT—The contents of an object could not be replicated by iCluster.• SDQ—The send operation to a data queue failed.• SFD—Unable to retrieve the file identifier of a BSF object.• SIS—A suspend entry has been issued for an object.• SIZ—An object was suspended on the primary node. A refresh of the object was required, but the size of the object

was greater than the value specified in the Maximum refresh size cluster system value. See DMSETSVAL—Set Cluster System Values on page 181 for more information.

• SND—An object or its authorities could not be replicated.• SPF—A logical file was suspended on the primary node because the associated physical file was suspended.• SPL—An *OUTQ object has suspended spooled files. The *OUTQ object itself is not suspended and will not be

included in auto-reactivation processing (if enabled in iCluster).• SRF—A refresh of a BSF object has been started.• SSC—A BSF object was suspended on the primary node.• STR—Journaling for a BSF object has been started. The object must be refreshed.• SWA—Activate pending for files being refreshed manually. The file is waiting for the activate command.• SWO—The object was suspended on the backup prior to a switchover.• SVF—An object was suspended on the primary node. It should have been refreshed, but the save operation failed.

This reason code appears only when displaying primary node statistics.• TNE—The BSF object does not exist on the backup node.• TNR—The path of a BSF object could not be resolved on the backup node.• TNS—The BSF object is a type of object that cannot be replicated.• TRG—The trigger information for the object could not be retrieved.• UBI—An operation was found that could not be undone because the object in question does not have journaled

before-images.• UCC—An error occurred undoing a content change to a journaled object.• UKM—The file is suspended on the backup node. The file's object specifier has been added with PFKEY (*AUTO),

however a unique key could not be found for the file on the backup node.• UOC—An object level change was encountered that cannot be undone.• UPD—A member of PF-SRC is open for update. The file will no longer be suspended once the application closes

the file.• UUT—The object type in question is not eligible for undoing.

If a reason code is not displayed for the object, this indicates that the object is not suspended.

Performing TasksFrom this screen, you have the following options:1: Activate

Activates a suspended object. Replication of this object to the backup node for the specified group will begin.4: Suspend

Suspends an active object on the primary node and backup node. It will not be replicated when you start refresh or mirroring.


9: Event LogDisplays the suspension event message in the event log.

F5: RefreshRefreshes the Status Monitor screen. Entries will stay in the same position in the list, and the cursor will remain in its current position.

F11: Sort ObjectsBy default, file objects are sorted by library (Sort by Lib). To sort by file name, press F11 again (Sort by Object). Press the same key again to sort by file type (Sort by Type).

F16 (or Shift+F4): Next ViewCycles through the views for this screen. The views show only suspended objects, only active objects, and then all objects.

F22 (or Shift+F10): DM CmdsLists the iCluster commands. Select a command from the list to display a screen that prompts you for the command parameters.

Working with BSF StatusThe Work with BSF Status screen (Figure 8) displays the status of BSF objects within the Status Monitor. The reason why an object was suspended is also displayed. Suspended BSF objects may be activated from the Object Status screen or activated automatically by iCluster.

Figure 8 Work with BSF Status Screen

For more information about suspended objects, see Suspended Objects on page 72. For more information about working with the Object Status screen, see Working with Object Status on page 306.

Displaying the BSF Status ScreenTo display the Object Status screen from the Status Monitor, enter option 7 (BSF Sts) in the Opt column next to the journal for which you want to display status information. You can display this screen from either the primary or backup monitor.To display this screen from the command line, run the WRKHABSFST—Work with BSF Status command.


Reading the BSF Status ScreenThis screen lists all objects for the specified group/journal combination. It contains three views. The first view shows only suspended objects. By pressing F16 (SHIFT+F4), you can cycle through the views to show only suspended objects, then only active objects, and then all objects.The backup node, name, group, and library of the journal is displayed at the top of the screen. This screen also displays the following information:


Activates a suspended object, and replication to the backup node for the specified group begins.5: Display

Displays the group and the full path object specifier of the byte stream file (BSF) object.F5: Refresh

Refreshes the Status Monitor screen. Entries remain in the same position in the list, and the cursor remains in its current position.

F9: RetrieveRetrieves the command previously invoked from the command line.

F16 (or Shift+F4): Next ViewCycles through the views for this screen. The views show only suspended objects, only active objects, and then all objects.

F22 (or Shift+F10): DM Cmds

Table 10 The BSF Status Screen

Status Indicates the status of each object. The possible statuses are as follows:• ACTIVE—The object is available for replication.• PNDACT—Active pending. The object is currently suspended

and waiting to become active though a journal entry that must be scraped.

• PNDSUS—The object is active and waiting to be suspended though a journal entry that must be scraped.

• SUSPND—The object is currently suspended.

Type Indicates the object type component that belongs to the specified group/journal combination.

Objects Indicates the location of the Byte Stream File (BSF) object.

Jrn Pos Indicates the journal position where the object was suspended.

Reason Indicates the reason why the object was suspended. For a complete listing of all of the three-character reason codes, see Working with Object Status on page 306 for more information.If a reason code is not displayed for the object, this indicates that the object is not suspended.


Lists the iCluster commands. Select a command from the list to display a screen that prompts you for the command parameters.

Working with Object Status for GroupsThe Working with Object Status for Groups screen (Figure 9) displays the status of objects within a group on the backup node. You can use it to determine why objects were suspended or are unsynchronized. This screen displays information for both native and BSF objects.

Figure 9 Work with Object Status for Groups Screen

This screen only displays native objects that are either out-of-sync or suspended. Similarly, it only shows BSF objects that are out-of-sync. For more information about suspended objects, see Suspended Objects on page 72.

Displaying the Object Status for Groups ScreenTo display the Object Status screen from the backup monitor, enter option 7 (BSF Sts) or 8 (Obj Sts) in the Opt column next to the journal you want to display status information for. Your choice selects the default view for the Work with Objects by Group screen. You can only display this screen from the backup monitor.To display this screen from the command line, run the DMWRKOBJST—Work with Object Status by Group command on the backup node of the group you want to view.

Reading the Object Status for Groups ScreenThis screen lists all objects for the specified group/journal combination. It contains three views. The first view shows only suspended objects. By pressing F16 (SHIFT+F4), you can cycle through the views to show only suspended objects, then only active objects, and then all objects.The group's name and its primary and backup nodes are displayed at the top of the screen. This screen also displays the following information:Table 11 Object Status for Groups Screen

Objects (Native View only) The replication object this status describes.

Type (Native View only) The object type of the object.



Activates a suspended native object. Replication of this object to the backup node for the specified group will begin. You cannot use this option on a BSF object.

F5: RefreshRefreshes the Status Monitor screen. Entries will stay in the same position in the list, and the cursor will remain in its current position.

F9: RetrieveRetrieves the command previously invoked from the command line.

F16 (or Shift+F4): Next Object ViewAlternates between the native object view and the BSF object view.


Library (Native View only) The library the object belongs to.

Object Path (BSF View only) Indicates the location of the Byte Stream File (BSF) object.

OOS Reason The three character reason code for out-of-sync status. The possible reason codes are as follows:• ATR—The object's attributes are out-of-sync.• AUT—The object's authorities, owners, or permissions are out-of-sync.• CNT—The object's contents are out-of-sync.• LCK—The object was locked on the primary node during a sync check.• NFD—The object does not exist on either the backup or the primary

node.• NJT—The object is not journaled on the backup node.• SZE—The object's size attribute is out-of-sync.

OOS Date-time Specifies the date and time when the object went out-of-sync.

Susp Reason Indicates the reason why the object was suspended. For a complete listing of all of the three-character reason codes, see The Object Status Screen on page 306 for more information.If a reason code is not displayed for the object, this indicates that the object is not suspended.

Auto Recvy Specifies whether or not the object is eligible for auto-recovery. '*YES' indicates that the object is eligible for auto-recovery. '*NO' indicates the object is not eligible for auto-recovery and manual steps will need to be taken to re-synchronize the object.

Table 11 Object Status for Groups Screen


Monitoring Historical StatisticsThe iCluster history log displays historical information for a journal. It splits its information across multiple views.To view historical statistics from the status monitor, enter option 12 (History) next to the journal you want to see the statistics for. You can do this from either the primary or backup monitor.While the historical status monitor is open, you can cycle through its views by pressing F11. This will display the next view. Figure 10 shows the views for the history log.

Figure 10 History Log View Sequence

For more information on each of these views, see the following topics:• Monitoring Historical Latency on page 314 • Monitoring Historical Object Position and Totals on page 316• Monitoring Historical Object Throughput on page 318

Monitoring Historical LatencyThe Historical Latency view (Figure 11) displays the historical latency for a selected group/journal combination. Historical latency is obtained by reading the records created by the batch collection process.

Figure 11 Historical Latency View


Any existing activity snapshots and ending records for the selection will be displayed in ascending order according to the creation timestamp in the Type column. An asterisk (*) in this column denotes that a portion of the data required to display the entry is unavailable.When STR (starting record) appears in the Type column, the value displayed in the Last J/E column is the first entry being processed. If MON (monitor) appears in the Type column, the value displayed in the Last J/E column is the current entry being processed. If END (ending record) appears in the Type column, the value displayed in the Last J/E column is the last entry processed.

Displaying the Historical Latency ViewTo view historical statistics from the status monitor, enter option 12 (History) next to the journal you want to see the statistics for. You can do this from either the primary or backup monitor. You may need to cycle to this view by pressing F11. See Monitoring Historical Statistics on page 314 for more information on the history log views.

Reading the Historical Latency ViewThis screen lists all the historical activity for the selected group/journal combination, with the backup node and group name displayed at the top of the screen. This screen displays the following items of information that are briefly described along with the Position to date and Position to time fields:

Performing TasksFrom this screen, you have the following options:F5: Refresh

Refreshes the Status Monitor screen. Entries will stay in the same position in the list, and the cursor will remain in its current position.

F11: Totals (View 2)

Table 12 Historical Latency View

Type The type of entry that is displayed.

Date The date when the activity occurred or end record was defined.

Time The time when the activity occurred or end record was defined.

Last J/E The last journal entry written to the journal.

Primary J/E The journal entry where the primary node journal scrape process is currently scraping the journal and has been received into the staging store.


Trans to Process The number of transactions which have been sent by the primary node, but have not yet been applied by the apply process on the backup node.

Position to date Allows you to search for activity occurring on a particular date. The Status Monitor system will search for activities matching the entered date in the Date column.

Position to time Allows you to search for activity occurring at a particular time. The Status Monitor system will search for activities matching the entered time in the Time column.


Displays historical position and totals view.F22 (or Shift+F10): DM Cmds

Lists the iCluster commands. Select a command from the list to display a screen that prompts you for the command parameters.

Monitoring Historical Object Position and TotalsThe Historical Object Position and Totals view (Figure 12) displays the historical journal position and totals for a selected group/journal combination. Historical information is obtained by reading the records created by the batch collection process.

Figure 12 Historical Object Position and Totals View

Any existing activity snapshots and ending records for the selection will be displayed in ascending order according to the creation timestamp in the Type column. An asterisk (*) in this column denotes that a portion of the data required to display the entry is unavailable.

Displaying the Historical Object Position and Totals ViewTo view historical statistics from the status monitor, enter option 12 (History) next to the journal you want to see the statistics for. You can do this from either the primary or backup monitor. You may need to cycle to this view by pressing F11. See Monitoring Historical Statistics on page 314 for more information on the history log views.

Reading the Historical Object Position and Totals ViewThis screen lists all the historical activity for the selected group/journal combination. It displays the name of the backup node and group.This screen displays the following items of information that are briefly described along with the Position to date and Position to time fields:Table 13 Historical Object Position and Totals View






F11: Throughput (View 3)Displays historical throughput view.




Total Trans Sent The total number of transactions processed by the primary node since the last reset of the transaction counters, that will be sent to the backup node (entries that are associated with files to replicate). This number does not include omitted entries.Transaction counts are reset under the following conditions:• After issuing the DMSETPOS—Set Journal Start Position,

DMMRKPOS—Mark Current Journal Positions, or DMREGPOS—Register Positions commands.

• Refresh before mirroring.• Counts exceed approximately 10 billion.

Total Trans Applied The total number of transactions processed by the backup node since the last reset of the transaction counters that have been sent to the backup node and applied by the apply process.

Trans to Process The number of transactions which have been sent by the primary node, but have not yet been applied by the apply process on the backup node.

Position to date Allows you to search for activity occurring on a particular date. The monitor system will search for activities matching the entered date in the Date column.

Position to time Allows you to search for activity occurring at a particular time. The monitor system will search for activities matching the entered time in the Time column.

Table 13 Historical Object Position and Totals View


Monitoring Historical Object ThroughputThe Historical Object Throughput view (Figure 13) displays the historical throughput for a selected group/journal combination. Historical information is obtained by reading the records created by the batch collection process.

Figure 13 Historical Object Throughput View

Any existing activity snapshots and ending records for the selection will be displayed in ascending order according to the creation timestamp in the Type column. An asterisk (*) in this column denotes that a portion of the data required to display the entry is unavailable.

Displaying the Historical Object Throughput ViewTo view historical statistics from the status monitor, enter option 12 (History) next to the journal you want to see the statistics for. You can do this from either the primary or backup monitor. You may need to cycle to this view by pressing F11. See Monitoring Historical Statistics on page 314 for more information on the history log views.

Reading the Historical Object Throughput ViewThis screen lists all the historical activity for the selected group/journal combination. It displays the name of the backup node, as well as the name of the journal previously in use.This screen displays the following items of information that are briefly described along with the Position to date and Position to time fields:Table 14 Historical Object Throughput View




Elapsed - Days Hr Mn The elapsed time since the latest replication (mirroring) process for the backup node - journal combination began. This timestamp is used to calculate throughput.

Overall Trans/Hr The number of transactions performed in an hour based on the life of the backup job.




F11: Latency (View 1)Displays historical latency view.


Current Trans/Hr The number of transactions performed in an hour based on the transactions performed since the last monitor record.

Position to date Allows you to search for activity occurring on a particular date. The monitor system will search for activities matching the entered date in the Date column.

Position to time Allows you to search for activity occurring at a particular time. The monitor system will search for activities matching the entered time in the Time column.

Table 14 Historical Object Throughput View


iBalance and Master-Master ReplicationWith standard replication groups, iCluster replicates exact, synchronized copies of data from a source system to a target system. As your application performs updates on the source system, iCluster replicates identical updates to your target system (Figure 1):

Figure 1 Replicating from Source to Target

After replicating your data, iCluster allows you to perform synchronization checks to ensure that replicated objects on the source and target systems are equivalent.After installing the iBalance feature, you can perform application updates to the same objects on both the source and the target systems. iCluster replicates these updates in both directions (Figure 2). This is referred to as master-master replication. Depending on the replication rules used, you can maintain identical copies of your data on both the source and the target systems. In addition, you can set up rules to control the updates to ensure that data is not overwritten on one of the servers:

Figure 2 Master-Master Replication

For example, a company configures two servers, New York and Los Angeles, for master-master replication (Figure 2). If the same file is simultaneously updated on the New York server and on the Los Angeles server, master-master replication ensures that you can view the updates that originated on the other server. Conflict detection and resolution rules ensure data integrity on both servers and recursion prevention prevents updates from replicating back to the original system. For updates to data that is not synchronized on the two servers, a conflict log file located on your target machine allows you to monitor replication and make adjustments to your configuration when necessary.The iBalance feature and master-master replication is applicable to many different scenarios including load balancing and data distribution. It allows for active use of the second system for a variety of business needs, instead of reserving it for read-only operations like queries and backups, or for disaster recovery.Note: iBalance is a feature that requires different authorization codes. For more information on installing this

feature, see Installing the iBalance Feature on page 23.The following topics provide more information on how to configure, operate, and monitor master-master replication:• Configuration of Master-Master Replication on page 322• Configuring Master-Master Replication in iCluster Administrator on page 509• Master-Master Replication—Operations and Monitoring on page 323


• Overview of Conflict Detection and Resolution on page 323• Configuration of Conflict Detection and Resolution on page 325• Parameter List Descriptions for a Conflict Resolution User Exit Program on page 326• Conflict Logging File on page 337 • High Availability Considerations on page 340

Configuration of Master-Master ReplicationThis topic describes the tasks you must perform to configure master-master replication for two iSeries environments configured on system A and B. This example assumes that you are populating new data between these two systems while preserving the original data on these systems. The examples show the required parameters you must set to perform each step. For a complete list of parameters, see the documentation for each command.Note: Bold indicates values that are specific to master-master replication.

To configure master-master replication:1 Create two master-master replication groups that mirror in opposite directions (A->B and B->A):

DMADDGRP GROUP('MyGroupAB') GRPTYPE(*MSTRMSTR) PRIMNODE('NodeA') BACKUPS('NodeB')

DMADDGRP GROUP('MyGroupBA') GRPTYPE(*MSTRMSTR) PRIMNODE('NodeB') BACKUPS('NodeA')

You can also configure data conflict options and a logging file (on the backup node only) for each master-master replication group. For more information on these options, see Overview of Conflict Detection and Resolution on page 323.

For more information on the DMADDGRP command and a complete list of parameters, see DMADDGRP—Add Group on page 113.2 Select the object specifiers for each group that you created in the previous step. The object specifiers for each of the groups

must be the same for true master-master replication. The example below mirrors two physical files (LIB1/FILE1, LIB2/FILE2).DMSELOBJ GROUP('MyGroupAB') OBJ('LIB1/FILE1') OBJTYPE(*FILE) OBJATTR(PFDTA) PFUPMTD(*UKEY) PFKEY(*AUTO)

DMSELOBJ GROUP('MyGroupAB') OBJ('LIB2/FILE2') OBJTYPE(*FILE) OBJATTR(PFDTA) PFUPMTD(*UKEY) PFKEY(*AUTO)

DMSELOBJ GROUP('MyGroupBA') OBJ('LIB1/FILE1') OBJTYPE(*FILE) OBJATTR(PFDTA) PFUPMTD(*UKEY) PFKEY(*AUTO)

DMSELOBJ GROUP('MyGroupBA') OBJ('LIB2/FILE2') OBJTYPE(*FILE) OBJATTR(PFDTA) PFUPMTD(*UKEY) PFKEY(*AUTO)

Keyed replication (*UKEY) is required for physical data files (PF). You must also specify a value for the PFKEY parameter. The PFKEY (*AUTO) option for master-master replication scans the logical files (LF) associated with physical files (PF) and selects a uniquely keyed LF that can be used for mirroring. The logical files must already exist on the target when they are needed for replication (run-time resolution).

Note: You can use generics when selecting object specifiers for your groups.For more information on this command and a complete list of parameters, see DMSELOBJ—Select Objects to Group on page 141.Note: In addition to the database files configured in this example, you can also configure master-master replication

for non-journaled IFS files and data areas.3 Start both groups with a refresh. The options for master-master replication prevent the refresh from truncating existing data

and replacing or removing existing logical files.DMSTRGRP GROUP('MyGroupAB') STRAPY(*YES) REFRESH(*YES) TRUNCATE(*NO) REPLACELFS(*NO)


DMSTRGRP GROUP('MyGroupBA') STRAPY(*YES) REFRESH(*YES) TRUNCATE(*NO) REPLACELFS(*NO)

For more information on the refresh options for master-master replication groups, see Master-Master Replication—Operations and Monitoring on page 323.

As with the steps above, any subsequent operations with master-master replication groups such as ending and re-starting groups are done in pairs.

Master-Master Replication—Operations and MonitoringiCluster has specific refresh options for master-master replication groups. There are also specific considerations for sync check when using master-master replication groups. The following sections go through the options that are available to you when using master-master replication groups. As with all operations involving master-master replication groups, you typically execute commands in pairs.

Refresh Options for Master-Master Replication GroupsWhen starting master-master replication groups with a refresh, you have the following options:• The *NO option for the TRUNCATE parameter in the DMSTRGRP—Start Cluster Operations at Group and DMACTOBJ—

Activate Object commands prevents the truncation of existing data on the target system during a refresh. This option is appropriate for master-master replication and is used in the example configuration in Configuring Master-Master Replication.The *YES option for the TRUNCATE parameter in the DMSTRGRP—Start Cluster Operations at Group and DMACTOBJ—Activate Object commands removes and recreates existing data files as part of a refresh. This is the default option for iCluster and is appropriate for standard iCluster replication. You can use this option for a master-master group if the data on the source system is known to be complete in order to ensure a synchronized starting point.

• The *NO option for the REPLACELFS parameter in the DMSTRGRP—Start Cluster Operations at Group and DMACTOBJ—Activate Object commands uses existing logical files during a refresh. This option is appropriate for master-master replication and is used in the example configuration in Configuring Master-Master Replication.

Note: iCluster does not require logical files to be the same on source and target systems except for the uniquely keyed logical files used for mirroring.

The *YES option for the REPLACELFS parameter in the DMSTRGRP—Start Cluster Operations at Group and DMACTOBJ—Activate Object commands replaces logical files during a refresh. This is the default option for iCluster and is appropriate for standard iCluster replication. You can use this option for a master-master group if the logical files on the source system are the complete set required for your applications in order to ensure a synchronized starting point.

Refresh options for master-master replication groups follow conflict detection and resolution rules. See Overview of Conflict Detection and Resolution on page 323 for more information on these rules and configuring conflict detection and resolution.

Sync Check for Master-Master Replication GroupsDue to the nature of master-master replication, it is recommended that you initiate sync checks during maintenance windows or down time to get a more accurate picture of your state of synchronization.For more information about using sync check, see Monitoring Replication on page 76.

Overview of Conflict Detection and ResolutionWith master-master replication of groups in iCluster, a conflict is defined as any operation attempted on the target system where the target system is not in the same state as the source system. Conflict detection and resolution in iCluster allows you to detect, log, and act on inconsistent data. This ensures your iSeries environment handles data conflicts automatically and in accordance with your business rules.Conflict detection and resolution allows you to detect conflicts in the apply process and to automatically resolve the conflicts when they are detected. As conflicts are detected and resolved, iCluster logs them in a file which allows you to analyze your conflicts and make adjustments to your mirroring or applications.


Conflict Detection RulesTable 1 outlines the conflict detection rules for database files.Table 1 Conflict Detection Rules for Database Files

Source Operation

Target Row

Conflict No Conflict

Insert Row exists No row with that key

Update No target row OR Before image differs from target row

Before image matches

Delete Before image differs from target row

No target row OR Before image matches


Table 2 Conflict Detection Rules for Data Areas

Table 3 Conflict Detection Rules for IFS Files

Configuration of Conflict Detection and ResolutioniCluster resolves conflicts with various rules that you can configure at the group level in the DMADDGRP—Add Group command, or the object specifier level in the DMSELOBJ—Select Objects to Group command. The CRWNR parameter in both of these commands gives you the following options:• *SRC: The source wins. If there is a conflict, source data is applied to the target which ensures that target data matches

source data and mirroring continues.• *TGT: The target wins. If there is a conflict, iCluster does not apply any data to the target. This rule preserves data on the

target and mirroring continues.• *EXIT: Indicates that you would like to resolve data conflicts through a user exit program. With this option you will also have

to specify the user exit program and library in the CRUSREXIT parameter. The user exit program allows for more flexibility since you can choose the resolution method for data conflicts. See the DMADDGRP—Add Group and DMSELOBJ—Select Objects to Group commands for more information.

• *NONE: Indicates that conflict resolution rules are not enabled and iCluster suspends the file from replication if there is a conflict. In addition, a message is sent to the event log. This is the default behavior of iCluster.

Resolved conflicts are silent (source wins, target wins, user exit) and no messages are generated in the event log. However, this information is logged in a special database file. See Conflict Logging File on page 337 for more information.

Source Operation

Target Data Area


Update Before image of the changed part of the data area differs from the target data area.

Before image matches

Source Operation Target Row


Create Object Exists Object Does Not Exist

Move/Rename Into Scope

Object Exists Object Does Not Exist

Move/Rename Out of Scope

Object Does Not Exist Object Exists

Object Changed Object Does Not Exist Object Exists


Note: Conflict resolution configuration with the DMSELOBJ—Select Objects to Group command will override any conflict resolution configuration in the DMADDGRP—Add Group command. By using the *GROUP option in the DMSELOBJ—Select Objects to Group command, iCluster uses the user exits configured at the group-level.

See the DMADDGRP—Add Group and DMSELOBJ—Select Objects to Group commands for more information on the parameters available for conflict detection and resolution.

Parameter List Descriptions for a Conflict Resolution User Exit ProgramWhen a conflict is detected, you can resolve it by invoking a customized user exit program. This program allows you to define a conflict resolution strategy that is not covered by one of the standard resolution methods (source wins, target wins, or take no action). The user exit program can resolve the conflict by directing the apply job on what to do with the source image. The user exit program returns a conflict resolution code of source wins, target wins, apply the desired image, or take no action. If you are applying the desired image, the user exit program also returns a desired image that is applied to the target object.You can define the user exit program at the group-level in the DMADDGRP—Add Group command or the object specifier level in the DMSELOBJ—Select Objects to Group level. The object specifier level takes precedence when defining your user exit programs.Note: The iBalance feature includes a sample user exit program, cruesample.c, which sets up a source wins

resolution code for a database record and a target wins resolution code for a data area.The following sections outline the values you can pass to the user exit program. If the group has a database file, the user exit program may deal with the columns of a database row or the changed part of a data area since it is only provided with a raw image of the record.

Parameters Passed to the User Exit ProgramTable 4 describes each of the parameters that are passed to the conflict resolution user exit program for database files, data areas, and Byte Stream Files (BSF).Table 4 Parameters passed to the conflict resolution user exit program for Database files,

Data Areas, and Byte Stream Files (BSF)

Parameter Description

Control Structure (*pControl) A pointer to the control structure containing various items of information about the source and target, as well as various indicators.Note: This parameter applies to database files, data areas, and

BSFs.For more information about the structure, see Control Structure on page 329.

BSF Path Name (*pBsfPathName) A pointer to the string that specifies the BSF path terminated with a null character.Note: This parameter only applies to BSFs.


Source Before Image (*pBeforeImage) Database record:A pointer to the structure containing the image of the row in the source table before changes were applied to the row.

Data area:A pointer to the structure containing the source before image of the changed part of the data area.

Note: This parameter does not apply to BSF.For more information about the structure, see Image Structure on page 328.The control structure contains an indicator that you can use to determine whether or not this image was passed to the conflict resolution user exit program. For more information about this structure, see Control Structure.

Source After Image (*pAfterImage) Database record:A pointer to the structure containing the image of the row in the source table after changes were applied to the row.

Data area:A pointer to the structure containing the source after image of the changed part of the data area.


Table 4 Parameters passed to the conflict resolution user exit program for Database files, Data Areas, and Byte Stream Files (BSF)



For important considerations regarding large object (LOB) data in the images, see LOB Considerations on page 329.

Image StructureThe structure that is used to represent an image is as follows:

typedef_Packed struct CrUeImage_T{ int allocatedSize; int actualSize; unsigned int offsetToData; char data[1];} CrUeImage_t;

Target Image (*pTargetImage) Database record:A pointer to the structure containing the image of the row in the target table when the user exit program was called.

Data area:A pointer to the structure containing the target image of the changed part of the data area.


Result Image (*pDesiredImage) Database record:A pointer to the structure containing the image of the row that will be applied to the target table if the user exit applies the desired image.If the conflict is resolved by the user exit program, this is the image defined in the user exit program, returned to the calling environment, and subsequently applied to the target table.This parameter applies only to conflicts caused by rows being inserted or updated in the source table. For row deletes, do not assign an image to this parameter.

Data area:A pointer to the structure of the changed part of the data area that will be applied to the target if the user exit applies the desired image.

Note: This parameter does not apply to BSF.For more information about the structure, see Image Structure on page 328.The control structure contains an indicator that you can use to determine whether or not this image can be returned by the user exit program. For more information about this structure, see Control Structure.

Table 4 Parameters passed to the conflict resolution user exit program for Database files, Data Areas, and Byte Stream Files (BSF)



Parameters in the Image StructureTable 5 describes the parameters in the image structure for database records and data areas. This structure is not applicable to BSF.

LOB ConsiderationsColumns containing LOB data are not passed to conflict resolution user exit programs. In the row images passed to a user exit program, each LOB column is represented by a placeholder. The result image returned by the user exit program cannot return LOB data, and so the placeholder(s) should also be used in the result image. iCluster ensures that source LOB data is applied to the target table.

Control StructureThe structure that is used to pass and return various items of information to and from a user exit program is as follows:

typedef_Packed struct CrUeControl_T{

Table 5 Parameters in the Image Structure for Database Files and Data Areas


Allocated Size (allocatedSize) Database record: The allocated size for the array of null indicators and record data for database files.

Data area:The allocated size for the changed part of the data area.

Actual Size (actualSize) Database record:The number of bytes used for the null map and record data.

Data area:The number of bytes of the changed part of the data area. For *DEC data areas, this value equals the number of decimals divided by 2 plus 1.

(offsetToData) Offset in bytes from the starting address of the CrUeImage_t data structure to the buffer containing the image data.

Data (data [1]) The image data is represented by an array of characters whose starting address is pointed to by the offsetToData field value. The array has variable length.Database record:

It is the null map followed by the record buffer.Note: The first numberOfFields bytes are allocated for the null

map only in null-capable files, otherwise they hold meaningless values.

Data area:The changed part of the data area. The starting position and length in bytes of the data change is indicated in the fieldOffset and fieldLength fields of CrUeField_T structure. For more information about the structure, see Field Structure on page 335.


int revision; char groupName [10]; int allocatedSize char objectName[10]; char member[10]; char objectType[10]; char objectAttr [10]; char sourceLibrary[10]; char sourceIASP[10]; char targetLibrary[10]; char targetIASP[10]; char operationType; char replicationMode; char conflictType; char hasBeforeImage; char hasAfterImage; char hasTargetImage; char returnCode; char resolutionCode; int numberOfFields; char isNullCapable; unsigned int offsetToFields; CrUeField fields[1];} CrUeControl_t;

Parameters in the Control StructureTable 6 describes each of the parameters in the control structure.Table 6 Parameters in the Control Structure


Revision (revision) The internal revision number that is used to determine whether or not the user exit program is using the current structures.If the structures change in a future release, this parameter can be referenced in a user exit program to determine compatibility with those structures. In a user exit program, logic can be defined to take appropriate actions depending on the revision number.For iCluster Version 5.1, the revision number is set to 1 (DM_CR_UE_REVISION).

Group Name (groupName) The master-master (*MSTRMSTR) replication group name.

Object Name (objectName) The 10 character object name.Note: This field only applies to database records and

data areas.

Member (member) The member name.Note: This field only applies to database records.


Object Type (objectType) The 10 character object type.Database record:

This will be *FILE.Data area:

This will be *DTAARA.BSF:

This will be *STMF for a stream file or *DIR for a directory.

Object Attribute (objectAttr) Extended object attribute.Database record:

This will be *PFDTA for *FILE object type.Data area:

One of *CHAR, *DEC, or *LGL for *DTAARA object type.Note: This field is not applicable to BSFs.

Source Library (sourceLibrary) The source library name.Note: This field only applies to database records and

data areas.

Source IASP (sourceIASP) The source IASP name.

Target Library (targetLibrary) The target library name.Note: This field only applies to database records and

data areas.

Target IASP (targetIASP) The target IASP name.

Operation Type (operationType) The source object-level operation that caused the conflict resolution user exit program to be called.Possible values for database records, BSFs, and data areas:

• 'I' (DM_CR_INSERT): Inserting a database record, or creating a BSF.

• 'U' (DM_CR_UPDATE): Updating a database record, or renaming a BSF.

• 'D' (DM_CR_DELETE): Deleting a database record. Not applicable to resolution code 'D'.

• 'R' (DM_CR_REFRESH_INSERT): Insert a database record during refresh.

• 'C' (DM_CR_CHG_DTAARA): Changing a data area.

Table 6 Parameters in the Control Structure



Replication Mode The modes of replication.• 'R' (DM_CR_MODE_REFRESH): Refresh.• 'W' (DM_CR_MODE_RWA): Refresh while active.• 'M' (DM_CR_MODE_MIRRORING): Mirroring.

Note: This field only applies to database records.

Conflict Type (conflictType) The type of conflict that caused the conflict resolution user exit program to be called.Possible values for database records:

• 'E' (DM_CR_UE_EXIST): An attempt was made to insert a row into the source table, but the key value already exists in the target table.

• 'M' (DM_CR_UE_MISSING): An attempt was made to update or delete a row in the source table, but the key value does not exist in the target table.

• 'D' (DM_CR_UE_DIFF): An attempt was made to update or delete a row in the source table, but the images of the corresponding rows in the source and target tables do not match.

Possible values for data areas:• 'D' (DM_CR_UE_DIFF): An attempt was made to update or

delete an image in the source, but the images of the corresponding images in the source and target do not match.

For BSFs, this field is always:• ‘E’ (DM_CR_UE_EXISTING): BSF object of the same path

name already exists on the target.




Source Before Image Indicator (hasBeforeImage) An indicator of whether or not the before image from the source has been passed to the conflict resolution user exit program.Possible values for database records:

• 0 (DM_CR_UE_FALSE): The before image has not been passed to the user exit program.

• 1 (DM_CR_UE_TRUE): The before image has been passed to the user exit program. You can reference this image in the user exit program.

Possible values for data areas:• 1 (DM_CR_UE_TRUE): The after image has been passed

to the user exit program. This image can be referenced in the user exit program.

Note: This field only applies to database records and data areas.

For information about the image, see Image Structure on page 328.

Source After Image Indicator (hasAfterImage) An indicator of whether or not the after image from the source has been passed to the conflict resolution user exit program.Possible values for database records:

• 0 (DM_CR_UE_FALSE): Indicates that the row after image has not been passed to the user exit program.

• 1 (DM_CR_UE_TRUE): Indicates that the row after image has been passed to the user exit program. This image can be referenced in the user exit program.

Possible values for data areas:• 1 (DM_CR_UE_TRUE): Indicates that the after image has

been passed to the user exit program. This image can be referenced in the user exit program.






Target Image Indicator (hasTargetImage) An indicator of whether or not the image from the target has been passed to the conflict resolution user exit program.Returns either for database records:

• 0 (DM_CR_UE_FALSE): The image has not been passed to the user exit program.

• 1 (DM_CR_UE_TRUE): The image has been passed to the user exit program. This image can be referenced in the user exit program.

Returns the following for data areas:• 1 (DM_CR_UE_TRUE): The image has been passed to the

user exit program. This image can be referenced in the user exit program.



Return Code (returnCode) The code returned by the conflict resolution user exit program that indicates whether or not the program call was successful.In the user exit program, set this parameter to one of the following values:

• 0 (DM_CR_UE_FALSE): Indicates that the user exit program call was not successful. Returning this value ends all subsequent iCluster replication operations for the group.

• 1 (DM_CR_UE_TRUE): Indicates that the user exit program call was successful.

Note: This parameter applies to database files, data areas, and BSFs.

Resolution Code (resolutionCode) The action that iCluster must perform to resolve the conflict.Returns either:

• 'N' (DM_CR_UE_NONE): No action. Object is suspended with "CNR" reason code.

• 'D' (DM_CR_UE_DESIRED): iCluster performs operation with desired image. Not applicable to a row deletion operation.

Note: This value only applies to database records and data areas and does not apply to BSFs.

• 'T' (DM_CR_UE_TARGET): Target wins.• 'S' (DM_CR_UE_SOURCE): Source wins.

Note: This parameter applies to database files, data areas, and BSFs.




Field StructureThe structure that is used to pass information about the fields comprising the file record to the user exit program is as follows:

typedef_Packed struct CrUeField_T{ int fieldOffset; int fieldLength; char isNullCapable; char fieldType;} CrUeField_t;

Number of Fields (numberOfFields) Database record:The number of fields in a database record.

Data area:The number of fields is always 1 for a data area.

Note: This field does not apply to BSFs.

Null Capable (isNullCapable) Indicates whether or not the file is null capable.Returns either:

• 0 (DM_CR_UE_FALSE): Indicates that the field is not null capable.

• 1 (DM_CR_UE_TRUE): Indicates that the field is null capable.

Note: This field is only applies to database records.

(offsetToFields) Offset in bytes from the starting address of the CrUeControl_t data structure to the array of field data.Note: This field only applies to database records and

data areas.

Fields (fields[1]) This is an array of CrUeField_t elements whose starting address is pointed to by the offsetToFields field value. It is only meaningful for database records and data areas.Database record:

The array has variable number of CrUeField_t elements. Each element describes a field in the database record. Array of structures containing the file fields attributes. For more information, see Field Structure on page 335.

Data area:The array has only one CrUeField_t element, which describes the data area.




Parameters in the Field StructureTable 7 describes each of the parameters in the control structure for database records and data areas. This structure is not applicable to BSFs.Table 7 Parameters in Field Structure for Database Records and Data Areas


Field Offset (fieldOffset) Database record:The offset to the field in the record buffer.Data area:The starting position in the data area that is changed. This is the same as indicated by the journal entry.

Field Length (fieldLength) Database record:Length of the field.Data area:Length in bytes of the changed part of the data area. For *CHAR and *LGL data areas, this is the same as indicated by journal entry. For *DEC data areas, this value equals the length of the change in the journal entry divided by 2 plus 1, since the length of the change in the journal entry is not the number of bytes but the number of decimals.

Null Capable (isNullCapable) An indicator of whether or not the field is null capable.In the user exit program, set this parameter to one of the following values:

• 0 (DM_CR_UE_FALSE): Indicates that the field is not null capable.

• 1 (DM_CR_UE_TRUE): Indicates that the field is null capable.Note: This field is only used for database records.

Field Type (fieldType) Data type of the field.Possible values are:

• '1': BLOB data type.• '2': CLOB data type.• '3': DBCLOB data type.• '4': DATALINK data type.• 'A': ALPHANUMERIC data type.• 'B': BINARY data type.• 'G': GRAPHIC data type.• 'P': PACKED_DECIMAL data type.• 'S': ZONED_DECIMAL data type.• 'Z': TIMESTAMP data type.

Note: This field is only used for database records.


Conflict Logging FileWhen iCluster resolves a conflict between the source and target tables, it automatically records information about the nature of the resolution for database files and data areas in a conflict logging file. The log file is generated at run time in iCluster's product library when the first conflict is encountered. If there are no conflicts, iCluster does not generate this table.Note: For data areas, the images in the logging file are of the same format as the change data area journal entry.The DMCONFAUD table in the logging file allows you to track how conflict resolution affects your target table. For example, you can query the CNFTIME column to see when a change was made to the target table. Then you can review the contents of the BEFOREIMG and AFTERIMG columns to see the change on the source table that resulted in the data on the target table. This can help in identifying problems in your conflict resolution strategy.Table 8 describes the structure of the DMCONFAUD conflict resolution table for database files and data areas.Table 8 DMCONFAUD Conflict Resolution Table

Column Description

CNFTIME - timestamp The date and time on the source system when the conflict was detected.

SRCTIME - timestamp The time the conflicting data was applied to the source object.

GROUP - char(10) The name of the master-master group name.

SRCSCHEMA - char(30) The schema or library name for the source object.

SRCNAME - char(10) The name of the source table or data area.

SRCMEMBER - char(10) The source table member name. NULL for data areas.

TGTSCHEMA - char(30) The schema or library for the target object.

TGTNAME - char(10) The name of the target table or data area. The target name is the same.

TGTMEMBER - char(10) The target table member name. NULL for data areas. The target member is the same.

OBJTYPE - char(10) The object type.Database record:

This will be *FILE.Data area:

This will be *DTAARA.BSF:

This will be *STMF for a stream file or *DIR for a directory.

OBJATTR - char(10) The object attribute.


OPTYPE - numeric(2) The operation on the source server that caused the conflict.For database records, the value is one of the following:

• 1 - A row was inserted into the source table.• 2 - A row was updated on the source table.• 3 - A row was deleted from the source table.• 4 - A row was inserted during a refresh.

For data areas, the value can be the following:• 5 - A data area has changed.

For BSF objects, the value can be one of the following:• 1—Create a BSF object on the source.• 2—Rename a BSF object on the source.

CNFTYPE - numeric(2) The type of conflict that was detected. The value is one of the following for database records:

• 1 - A row was inserted into the source table. The key for that row already exists in the target table.

• 2 - A row was updated or deleted on the source table. The key for that row does not exist in the target table.

• 3 - A row was updated or deleted on the source table. The images of the source and target tables do not match.

• 4 - An unexpected conflict was detected.For data areas, the value can be the following:

• 3 - A data area was changed, the images of the source and target data areas do not match.

For BSF objects, the value is always the following:• 1—The BSF object already exists on the target.

RESMTD - numeric(2) The conflict resolution method that was used. The value is one of:• 1 - Source wins.• 2 - Target wins.• 3 - User exit with no decision. This is for a delete with no row on the

target.• 4 - User exit with a source wins decision.• 5 - User exit with a target wins decision.• 6 - User exit generates a desired image to apply on the target.

If the resolution method was None, then a row will not be entered into this table.See Configuration of Conflict Detection and Resolution on page 325 for more information on these methods.

Table 8 DMCONFAUD Conflict Resolution Table

Column Description


CNFRES - char(1) Indicates if the conflict was resolved. The value is one of:• Y - The conflict was resolved.• N - The conflict was not resolved.

BEFORETRNC - char(1) Indicates if the before image stored in BEFOREIMG was truncated. The value is one of:

• Y - The value was truncated.• N - The value was not truncated.

BEFOREIMG - varchar(8000) Database record:A representation of the row in the source table before it was changed. The representation is truncated to the first 8000 bytes if truncation occurs.

Data area:A representation of the before image of the changed part of the data area.

BSF objects:Used to store the BSF object path for BSF objects.

AFTERTRNC - char(1) Indicates if the after image stored in AFTERIMG was truncated. The value is one of:

• Y - The value was truncated.• N - The value was not truncated

AFTERIMG - varchar(8000) Database record:A representation of the row in the source table after it was changed. The representation is truncated to the first 8000 bytes if truncation occurs.

Data area:A representation of the after image of the changed part of the data area.

TGTTRNC - char(1) Indicates if the after image stored in TGTIMG was truncated. The value is one of:

• Y - The value was truncated.• N - The value was not truncated.

TGTIMG - varchar(8000) Database record:A representation of the row in the target table before replication occurred. The representation is truncated to the first 8000 bytes if truncation occurs.

Data area:A representation of the target data area before replication occurred.


Column Description


High Availability ConsiderationsThere are special considerations when using master-master replication groups in disaster recovery (DR) scenarios. Since they mirror changes in both directions, master-master replication groups do not permit fail over like standard iCluster replication groups where the backup system is reserved to take over from the primary system during a failover. With master-master replication groups, you will have to migrate users to the surviving system which you can do manually or with a stand-alone CL program. This will place a heavier load on your surviving system, but it will give you time to fix your failed system so that you can resume master-master replication.

WINTRNC - char(1) Indicates if the image stored in WINIMG was truncated. The value is one of:• Y - The value was truncated.• N - The value was not truncated.

WINIMG - varchar(8000) Database record:A representation of the final row in the target table after conflict resolution has occurred. The representation is truncated to the first 8000 bytes if truncation occurs.

Data area:The image of the data area on the target after conflict resolution has occurred.


Column Description


iSeries Clustering Overview

Switchover for IBM Cluster Resource Services (CRS)This section contains information about the switchover and failover mechanisms for clusters that use IBM Cluster Resource Services.This section is for iCluster installations using IBM Cluster Resource Services as the failover mechanism. See Choosing a Failover Mechanism on page 73 for more information.DataMirror's iCluster works closely with IBM's low-level cluster support (OS/400 Cluster Resource Services). For more information about OS/400 Cluster Resource Services, see iSeries Clustering Overview on page 341.Warning: If your cluster is NOT configured to use IBM Cluster Resource Services for cluster operations, then DO

NOT refer to this section. Instead, refer to Switchover for SwitchOver System (SOS) on page 369.

iSeries Clustering OverviewA cluster consists of a networked collection of iSeries systems (or nodes) that work together to provide seamless iSeries operations. iCluster is built on the underlying framework provided in the OS/400 operating system. This section introduces iSeries clustering concepts that are relevant to switchovers and failovers. Note: For more information about iSeries clustering concepts, see your iSeries documentation.

iSeries Clustering ConceptsThis section provides information about the concepts related to iSeries clustering:• OS/400 Cluster Resource Services: A set of OS/400 system functions that allow for iSeries cluster operations and

implementations.• Node: A node is any iSeries server that is a member of a cluster. In iCluster, each node is identified by its iSeries system

name. The iCluster node name is associated with one or more Internet Protocol addresses that represent an iSeries server. iCluster communications makes use of the TCP/IP protocol suite to provide the communications path between each node in the cluster.

• Cluster: A cluster consists of a set of iSeries systems that have been configured to work together in order to provide continuous, uninterrupted operations. A cluster of iSeries nodes is capable of recovering from anticipated or unexpected disruptions in the normal flow of operations by moving processing activities from one node in the cluster to another. Therefore, communications must exist between all nodes in the cluster.

• Cluster Resource Group (CRG): A CRG specifies a recovery domain that defines a group of nodes and the role of each node in the group.There are three kinds of CRGs: data, applications, and devices.

• Recovery domain: The ordered list of nodes in a Cluster Resource Group from the primary node to the first backup node to the second backup node, and so on.

• Switchover: When a user manually switches the role of a primary node over to the role of a backup node. A switchover is a planned procedure.

• Heartbeat: The mechanism within the cluster that verifies that every active node in the cluster is responding to signals across one or two networks.

• Distress message: A message, broadcast to all known nodes in the cluster subnet that indicates that a failure situation has been detected by a cluster node and the node will end clustering locally.


• Node failure: The detection of a distress message received from a failing cluster node before a heartbeat or message time-out is detected.

• Failover: When a group's primary node automatically switches over to a backup node because of a system failure. A failover is generally unplanned.Regardless of whether a failover or switchover occurs, OS/400 Cluster Resource Services provide the underlying mechanisms to switch operations to a backup node when the primary node is unable to provide business services.

• Cluster partition: The detection of a heartbeat time-out with no detection of a distress message.• Resilient applications: An application that is restarted on the new primary node after switchover or failover by OS/400

Cluster Resource Services. Application groups within a resilient application have a floating IP address associated with the group to allow users to access the application using the same IP address before and after the switchover or failover.For information about how iCluster handles switchovers of resilient applications, see iCluster and Resilient Applications on page 364.

• Takeover IP addresses: A takeover IP address allows multiple nodes in the recovery domain to have the same IP address at different times. It facilitates a transparent switchover in a TCP/IP environment.

• Resilient devices: A resilient device is a physical resource represented by a configuration object, such as a device description, that is accessible from more than one node in a cluster.For information about how iCluster handles switchovers of resilient devices, see iCluster and Resilient Devices on page 365.

iCluster builds upon the OS/400 Cluster Resource Services for high availability and switchover capabilities. See iCluster Concepts on page 342 for more information.

iCluster ConceptsThis section provides information about the clustering concepts related specifically to iCluster. For more information on the basic concepts in iCluster such as nodes in a cluster, groups, group types, failovers, and switchovers, see iCluster Basics on page 43.

iCluster and OS/400 Cluster Resource ServicesThere are two parts to the cluster definition: • Low-level definition maintained by OS/400 Cluster Resource Services. The OS/400 Cluster Resource Service definition

includes the status of each node of the cluster and status of each cluster group.• High-level definition maintained by iCluster. The iCluster portion of the definition includes the object specifiers, resilient

applications, and the replication status of replication groups.

iCluster Groups and Role SwitchingWhen OS/400 Cluster Resource Services determines that a node in the cluster has failed, it puts the node into *FAILED status on the other nodes of the cluster. If a group in *ACTIVE status has the failed node as its primary node, the group will be switched over at the operating system level, that is, the previous backup node becomes the primary node and the previous primary node becomes the backup node in the view of OS/400 Cluster Resource Services.Note: Inactive groups are not switched over, even if their primary node is the one that failed.


iCluster Concepts

In the following discussion, the terms "server A" and "server B" refer to physical machines as seen in the following figure.

Figure 1 Typical Node Configuration

The terms "primary node" and "backup node" refer to roles that a physical machine can adopt in a replication group (Figure 1). Hence, server A may be either the primary node or the backup node of a group. In this discussion, server A is the primary node and server B is the backup node before the failover happens. After the failover happens, server B is the primary node and server A is the backup node in the view of OS/400 Cluster Resource Services.

ROLESWITCH ParameterThe ROLESWITCH parameter allows you to make the decision about whether to switch roles for a node. Each replication group defined in iCluster with the DMADDGRP—Add Group command has a ROLESWITCH parameter which can be set to *YES or *NO. It gives the administrator control over a group beyond the behavior of OS/400 Cluster Resource Services.• ROLESWITCH *NO means that when a failure of the group's primary node (for example, server A) is declared by OS/400

Cluster Resource Services, iCluster will not prepare the backup node (server B) to take over as the primary node. Replication will not start up automatically once server A is again active in the cluster. Note that even though server B has not been prepared to take over as the primary node, it will still appear as the primary node for the groups of which it was the backup node prior to the failure.*NO is the default setting for the ROLESWITCH parameter. Setting the ROLESWITCH parameter to *NO gives you the opportunity to decide if you would like to continue with a role switch, or continue with your current configuration. With this setting, the failed primary node will retain its role and resume its functions as the primary node once the appropriate commands have been issued. For minor system failures with a minimum amount of down time, this is probably the optimal situation. See the DMSETPRIM—Prepare Primary Node command for more information.

• ROLESWITCH *YES means that when a failure of the group's primary node (for example, server A) is declared by OS/400 Cluster Resource Services, iCluster will prepare the backup node (server B) to take over as the primary node. Once server A is again active in the cluster, replication will take place from server B, which is now the primary node, to server A, which is now the backup node.

Note: Note that the ROLESWITCH parameter is defined at the group level. It is not necessary or advisable to have user exits called for each group. Should you choose to set the ROLESWITCH parameter to *YES, you must have a thoroughly defined and tested switch plan. *YES means iCluster will prepare the backup node to take over as the primary node. This often involves, but is not limited to, enabling user profiles, varying on devices, and changing TCP/IP configurations. These steps would have to be reversed and could be time-consuming should a true failover not have occurred.

What iCluster Does During a Switchover and FailoverFor both switchover and failover, the following steps prepare the backup node to become the primary node. The commands for switchover in iCluster are DMSTRWO—Switchover Group (for active independent groups, that is, groups that are not part of a resilient application), DMSWOAPP—Switchover Resilient Application (for active resilient applications), and DMCHGROLE—Change Group Primary Node (for inactive groups or resilient applications).The same steps are automatically performed when a failover occurs, and the group's ROLESWITCH parameter is set to *YES.


These commands automatically perform the following operations:1 End replication if the group or resilient application is active.2 Call the user-specified before role switch user exit for the group or resilient application.3 Drain the staging store on the backup node by applying all staged entries. 4 Start journaling on backup files and objects.5 Restore database triggers on backup files.6 Perform the role switch at the operating system level.7 Establish starting journal positions on the new primary node for the group or resilient application.8 Call the user-specified after role switch user exit for the group or resilient application.9 Start replication from the new primary to the new backup if the group or resilient application is active, and a backup node is

available.Note: Local loopback groups (groups where the primary and backup nodes are the same) and groups that have a

target library other than *PRIMARY, cannot undergo a role switch at the iCluster level.

Switchover OverviewA switchover occurs when a user manually switches a primary node over to a backup node. You would perform a switchover, for example, for maintenance or upgrade reasons. During a switchover, the primary node becomes the backup node and the backup node becomes the primary node. Note that a switchover procedure is planned by the user, while a failover is an unplanned procedure due to a system failure.You can use the commands DMSTRWO—Switchover Group (for replication groups) and DMSWOAPP—Switchover Resilient Application (for resilient applications) to perform manual switchovers.

Failover OveriewA failover is a switch in roles of the backup node to the primary node as a result of the failure of the original primary system. This may occur automatically if the ROLESWITCH group parameter is set to *YES, or it may be done at the user's discretion by using the DMSETPRIM—Prepare Primary Node command. The role switch in this case is unplanned.

Failovers and the ROLESWITCH parameterThe ROLESWITCH parameter in the DMADDGRP—Add Group command will control the behavior of the cluster in the case of a primary node failure.

ROLESWITCH *NOIf you would like the opportunity to decide if a role switch is necessary after a failover, set the ROLESWITCH parameter in the DMADDGRP—Add Group command to *NO. This setting gives you the opportunity to decide if you would like to proceed with a role switch, or continue with your current configuration. With this parameter set to *NO, the failed primary node will retain its role and resume its functions as the primary node once it recovers and rejoins the cluster. If you would like to complete a role switch after a failover has occurred, use the DMSETPRIM—Prepare Primary Node and DMCHGROLE commands to prepare the new primary node (previous backup node) for replication.Note: The before and after user exit programs are called as part of issuing the DMSETPRIM—Prepare Primary

Node command.


Preparing the Cluster to Handle Switchovers and Failovers

For more information about the ROLESWITCH parameter, see DMSETPRIM—Prepare Primary Node and DMCHGROLE—Change Group Primary Node.

ROLESWITCH *YESIf you would like to automatically change the role of the backup node in the replication group to the primary node when a failover occurs, set the ROLESWITCH parameter in the DMADDGRP—Add Group command to *YES. The user exit programs that are specified through the BSWUSREXIT and ASWUSREXIT parameters in the DMADDGRP—Add Group command will be called automatically.See the DMADDGRP—Add Group command for more information about the ROLESWITCH parameter.

Preparing the Cluster to Handle Switchovers and FailoversThis section presents the considerations that you should note when preparing nodes and groups for handling switchovers or failovers.

ROLESWITCH ParameterWhen creating or modifying groups using the DMADDGRP—Add Group or DMCHGGRP—Change Group commands, you can enter a value for the ROLESWITCH parameter to control the behavior of the system on a failover. See iSeries Clustering Concepts on page 341 for more information about this parameter and the values that you can specify.Note: The ROLESWITCH parameter is not available for resilient applications. The behavior for resilient applications

is always that of ROLESWITCH *YES.

User ExitsWhen adding a group or resilient application in iCluster, you can specify before and/or after user exits that you want to call immediately before the role change, or after the role change is performed.If you specify a user exit program for a group using the DMADDGRP—Add Group or DMCHGGRP—Change Group commands, it will be called when the role of the node is changed within the group. If a failover occurs and the ROLESWITCH parameter for the group is *YES, the user exits will be called automatically when the failure is detected. If the ROLESWITCH parameter for the group is *NO, the user exit will not be called unless you decide to complete the failover of the node by calling the DMSETPRIM—Prepare Primary Node command.Note: The specified user exit program is invoked for a switchover if you decide to prepare the backup node to be the

primary node by issuing the DMSETPRIM—Prepare Primary Node command. If you decide not to prepare the backup node to be the primary node, and just wait for the primary node to become active, the user exits are not called. See Passing Arguments to Role Switch User Exit Programs on page 347 for more information.

Journal Entry ProcessingYou may want to change your journal cleanup processes/schedule in your before role switch user exit program. You may have configured your target receivers to be cleaned up quickly on the backup node. If your group is configured for remote journaling, you will need the receivers created as a result of the failover-switch to properly re-synchronize your original primary node. You can change the schedule for journal receiver cleanup in the before role switch user exit program.


Synchronous and Asynchronous Remote JournalingFigure 2 shows the recommended configuration for remote journaling between two systems: System A (Primary Node) and System B (Backup Node).

Figure 2 Setup for Synchronous and Asynchronous Remote Journaling

The example in Figure 2 shows remote journals configured for synchronous remote journaling. Note that the example also applies to configurations that involve asynchronous remote journaling.As shown in Figure 2, a remote journal should be configured on both System A and System B. The purpose of these remote journals is to receive journal entries from the other system when it acts as a primary node. Whenever a system acts as a primary node (System A in Figure 2), the remote journal that resides on that system (Remote Jrn A) should be inactive. This will prevent the unnecessary transmission of journal entries from the backup system (System B) to the remote journal on the primary node (System A). Note: Note that these transactions are unnecessary since they originate on System A and will already reside in

Local Jrn A.The remote journal should be inactivated from the backup node (System B in this case) with the CHGRMTJRN command. See Configuring Remote Journaling on page 84 for more information on the CHGRMTJRN command. Whenever a switchover is performed, the remote journal residing on the new backup node should be activated from the new primary node. In addition, the remote journal residing on the new primary node should be de-activated from the new backup node. This can be done with the CHGRMTJRN command on each machine. Note: You can use the after role switch user exit program to activate (de-activate) the remote journals on the

appropriate nodes.Note: Note that a sample program, RMTJRNSWO, has been included with iCluster. This program is located in the

QACLSRC file in the product library. The name of the member is RMTJRNSWO.


Passing Arguments to Role Switch User Exit Programs

Failover Message QueueIn iCluster, you can specify a failover message queue that will receive messages generated when a failover occurs. Note that you can specify a failover message queue only for replication groups.

Enabling Alerts in OS/400To detect partition states more easily, the operator should set alert status to "on" using the appropriate OS/400 command. See your iSeries documentation for more information on enabling alerts.See Alertable Messages for Clustering on page 366 for a list of alertable messages that are generated by OS/400 Cluster Resource Services.

Line Controller HeartbeatThe communication line controller detects and records line errors. Within the line description are two parameters that determine at what point in time the system operator is notified of line errors. The default settings specify that if an error count limit of two is reached within the last five minute time interval, then the system operator is notified with a message that requires a response as to whether to continue retrying the line (G or R) or to no longer attempt retries (C).WRKLIND <Name of line description for TCP/IP line>

Recovery LimitsCount Limit: 2 (default)Time interval: 5 (default)For example, a LAN cable pull will result in one line error. Every 10 to 15 seconds a retry is attempted, resulting in additional errors. Thus with the default settings an operator message will be received within 20 to 30 seconds. This manual intervention should be eliminated so that the cluster heartbeat time-outs can take control.You may want to adjust these parameters to allow for a longer interval before the system message is generated. This would create a larger window in which OS/400 Cluster Resource Services can recover from line outages. Since increasing these parameters would mean that the system overall would detect line errors more slowly, the values that you specify depend on what is acceptable for your system.

Passing Arguments to Role Switch User Exit ProgramsIf you define a user exit program that will be called when a failover or switchover occurs, the following arguments will be passed to the program:Note: The order of the arguments passed to the user exit program is very important, and they will be passed in the

same order as they appear below.Note: User exit programs must be compiled with the Use adopted authority option set to *NO. Otherwise, the user

exit programs cannot be executed and mirroring will continue.

Group NameType: CharacterLength: 10 bytesThe name of the group in which the failover or switchover occurred.


ReasonType: CharacterLength: 10 bytesThe point where the user exit program was called. One of the following values will be passed through this argument:• *BEFORE: The user exit program is called immediately before a group is switched over at the operating system level on both

nodes of the group.• *AFTER: The user exit program is called immediately after a failover occurs on the new primary node of the group, or

immediately after a switchover occurs on both nodes of the group.

RoleType: CharacterLength: 10 bytesThe new role of the node that the user exit program is running on in the specified group. The value passed to the user exit program is one of the following:• *PRIMARY: The user exit is running on the primary node for the specified group.• *BACKUP: The user exit is running on the backup node for the specified group.

User DataType: CharacterLength: 256 bytesThe user-defined data can be specified through the DMADDGRP—Add Group or DMCHGGRP—Change Group commands.

Failed State OverviewThis section provides an overview of the FAILED state, indicates how to detect it, and specifies the method of recovery depending on certain conditions.

Node Failure Detection in iClusterIf a node fails and sends a distress signal, then OS/400 Cluster Resource Services reports the node failure to the other nodes in the cluster and initiates a failover.

FAILED State ScenariosThe following scenarios will result in a detected node failure if OS/400 Cluster Resource Services detects a distress message from the failing cluster node. All of the active resources that are managed by OS/400 Cluster Resource Services on that node will go through the failover process. This includes all application Cluster Resource Groups (CRG)s, data CRGs, and device CRGs.

Loss of utility power scenarios:• Internal Battery Backup Unit (BBU) installed and the internal battery timer times out. • Internal BBU installed and internal battery low signal detected.• UPS installed, UPS Utility Failure signal received by SPCN, and a timer set by system value QUPSDLYTIM system value

times out.• UPS installed and both UPS Utility Failure and UPS Battery Low signals received by SPCN.


Failed State Overview

Controlled shutdown scenarios:• Initial Program Load (white) button pushed to power down the system.• PWRDWNSYS *IMMED command issued.• PWRDWNSYS *CNTRLD command issued and cntrld function expires.

ENDSBS *ALL:• *IMMED or *CNTRLD where a time limit on the controlled function expires.

ENDSYS on controlling subsystem:• *IMMED or *CNTRLD where a time limit on the controlled function expires.

ENDSBS QSYSWRK: • *IMMED or *CNTRLD where a time limit on the controlled function expires.• Where OS/400 Cluster Resource Services and TCP/IP jobs reside.

ENDTCP:• *IMMED or *CNTRLD where a time limit on the controlled function expires.

ENDJOB QCSTCTL is issued:• *IMMED or *CNTRLD where a time limit on the controlled function expires.

ENDJOB QCSTCRGM is issued:• *IMMED or *CNTRLD where a time limit on the controlled function expires.

Detecting a FAILED StateYou can detect whether a node is in a FAILED state from either the iCluster Administrator interface or from the command line:• From the iCluster Administrator that is connected to a node in *ACTIVE status, select a node and examine the Node Status

value in the table view. If the value displays *FAILED, then the node is in a FAILED state. • From the iCluster command line, issue the DMWRKNODE—Work with Nodes command. The status of the node will be

displayed on the screen. If it displays *FAILED, then the node is in a FAILED state.Once a node is in a FAILED state, you will need to recover the node. See FAILED State Recovery Paths for information about how to recover from a FAILED state.

Group Failover in iClusterAn active replication group whose primary node goes into *FAILED state undergoes failover in one of two ways, depending on the group's ROLESWITCH parameter:Note: Inactive groups do not undergo failover.• If the group's ROLESWITCH parameter is set to *YES, the group's backup node will become the group's primary node, and

the group's new primary node will be prepared so that the group is ready to begin replication from the new primary node (the previous backup node) when a backup node for the group becomes available.See Preparing the Cluster to Handle Switchovers and Failovers on page 345 for more information on what is involved in the preparation.

• If the group's ROLESWITCH parameter is set to *NO (the default), the group's backup node will become the group's primary node at the cluster level. However, the new primary node will not be prepared to be the group's primary node for replication.


Setting the ROLESWITCH parameter to *NO gives you the opportunity to decide if you would like to continue with a role switch, or continue with your current configuration. With this setting, the failed primary node will retain its role and resume its functions as the primary node once it recovers and rejoins the cluster.It is the responsibility of the cluster administrator to decide whether to continue with the role switch processing and switch the group back to its original configuration, or to complete the failover processing and prepare the new primary node (the previous backup node) for replication as the primary node. For minor system failures with a minimum amount of down time, it is probably best to retain the original configuration. For more detailed information about these steps, see Failed State Recovery Paths.

Failed State Recovery PathsThis section indicates the recovery path that you should follow depending on certain conditions.


Failed State Recovery Paths

Recovery Paths for a FAILED StateDifferent recovery paths depend on certain conditions. Figure 3 displays the different options:

Figure 3 Recovery Paths for Failed State (Part 1)


Failed State Recovery Paths

The OS/400 Cluster Resource Services jobs that run in the QSYSWRK subsystem on an active cluster node are listed in the following table:Table 1 OS/400 Cluster Resource Services Jobs in the QSYSWRK Subsystem

Job Name Function

QCSTCRGM PGM-QCSTCRGMA

QCSTCTL PGM-QCSTCCJOB

QCSTINETD Not available.

DM_INTGRP PGM-QCSTCRGJOB

All other jobs whose function name is PGM-QCSTCRGJOB. The job names are the names of *CRG objects (cluster resource groups) in the cluster.


To view the job logs of these jobs if they end (Table 1), you can issue the command WRKSPLF QSYS and search for either the job name or the job function name.

Figure 4 Recovery Paths for FAILED State (Part 2)

Any underlined text in Figure 3 and Figure 4 indicates a topic in this document that you should read for information about how to proceed. The topics are as follows:Note: "Server A" and "Server B" in the following links are in reference to Figure 1 on page 343. The > indicates the

direction in which objects are replicated within the group.• (Server A > Server B): Restarting Active Groups with Mirroring in the Original Direction with ROLESWITCH *NO on page 359.• (Server A > Server B): Restarting Active Groups with Mirroring in the Original Direction with ROLESWITCH *YES on

page 360.• (Server B -->Server A): Restarting Active Groups with Mirroring in the Opposite Direction with ROLESWITCH *YES on

page 361.• Restarting Active Groups with Mirroring in the Opposite Direction with ROLESWITCH *NO on page 361.• Rejoining a Non-Functioning Node with a Role Switch on page 362• Rejoining a Non-Functioning Node without a Role Switch on page 363• Removing a Failed Node from the Cluster on page 364


Partition State Overview

Partition State OverviewThis section provides an overview of the PARTITION state, how to detect it, and specifies the method of recovery depending on certain conditions.OS/400 Cluster Resource Services checks every three seconds for communication among the nodes in a cluster. If communication between two nodes is lost, then OS/400 Cluster Resource Services attempts to re-establish communications. At this point, the node is in a PARTITION state.In contrast, a FAILED state occurs when the primary node will soon no longer be operational, but is able to inform other nodes of this situation by issuing a distress signal. If the primary node of an active group enters a FAILED state, then the OS/400 Cluster Resource Services initiates a failover for the group.

False and True PARTITIONSIf a node is in a PARTITION state, it can either be true or false.

True PARTITIONA true PARTITION (simply referred to as PARTITION) occurs when one node (node A) cannot communicate with another node (node B). However, all nodes are functioning; only the communication among nodes is lost. In a PARTITION state, it is assumed that communication will return soon, and that the node will become active again.Essentially, a PARTITION state is a waiting state, under the assumption that communication between nodes is down only temporarily. While the node is in a PARTITION state, replication does not take place. When communication between the partitioned nodes is restored, replication automatically resumes.

False PARTITIONA false PARTITION state occurs when communication among nodes is lost, but one of the nodes is no longer functioning. It is considered a false PARTITION because the node should really be in a FAILED state, but due to the lack of communication, node A does not know that node B is no longer functioning; node A is aware only that it cannot communicate with node B.

PARTITION State ScenariosThe OS/400 Cluster Resource Services detects a PARTITION state when a node repeatedly fails to respond to the heartbeat. By default, the PARTITION state is detected within 42 seconds on the OS/400.The following scenarios will result in a detected cluster partition if OS/400 Cluster Resource Services is active on the node detecting the cluster partition. The node(s) may or may not be operational but the cluster will stop all further operations and inform the cluster operator that a partition has been detected by an alertable message in the QSYSOPR message queue. See Alertable Messages for Clustering in Appendix B for the list of alertable messages issued by OS/400 Cluster Services.

False PARTITION Scenarios:CEC hardware failures:• SRC B6xx Terminate Immediate.• SRC B9xx XPF originated Terminate Immediate.Loss of Utility power scenarios: • Neither Battery Backup Unit nor UPS installed.• Remote EPO signal active.Power down from control panel (blue button):• Option 3 restart.• Option 8.Operating System software hard machine check:


• Jobs are not ended and a mainstore dump is initiated.

True PARTITION Scenarios:Communications hardware failure:• On all IP interface addresses defined for a node.• Comm adapter, line, hub or router failures.ENDTCPIFC command issued:• On all IP interfaces addresses defined for cluster heartbeat.

Detecting a PARTITION StateYou can detect whether a cluster is in a PARTITION state from either the iCluster Administrator or from the command line:• From the iCluster Administrator that is connected to a node in *ACTIVE status, select a node and examine the Node Status

value in the table view. If the value displays *PARTITION, then the cluster is in a PARTITION state. • From the iCluster command line, issue the DMWRKNODE—Work with Nodes command. The status of the node will be

displayed on the screen. If the status displays *PARTITION, then the cluster is in a PARTITION state.Verifying a PARTITION State in iClusterTo verify that your cluster is in a true *PARTITION state, issue the HAPNGTCP—Ping Using TCP command from all active nodes in the cluster to the nodes that are listed as being in a *PARTITION state. The following four parameters must be entered when issuing the HAPNGTCP—Ping Using TCP command:• IP address of the node• Port number of the node• iCluster product library on the node• Default replication job description of the nodeThe correct values for these four parameters are the ones listed for the node by the Display node option (option 5 on the Work with Nodes screen).If the HAPNGTCP—Ping Using TCP command succeeds, your cluster is not in a *PARTITION state and the nodes should return to a *ACTIVE state within 15 minutes.If the HAPNGTCP—Ping Using TCP command fails, there are two possible explanations:• If all the nodes in the cluster list their own status as being *ACTIVE with one or more of the other nodes in the cluster having

*PARTITION state, you have a true PARTITION state in the cluster.• If the HAPNGTCP—Ping Using TCP command fails and the node with *PARTITION status has actually failed, you have a

false PARTITION state in the cluster.See Partition State Recovery Paths on page 356 for information about how to proceed once you have detected a PARTITION State.

Partition State Recovery PathsThis section indicates the recovery path that you should follow depending on certain variables.


Partition State Recovery Paths

Recovery Paths for a PARTITION StateFigure 5 below indicates the recovery path for nodes in a PARTITION state, depending on variables, such as false or true PARTITION, whether the machine will be repaired quickly, and whether the failed node will become the backup node.

Figure 5 Recovery Paths for PARTITION State


Restarting Active Groups with Mirroring in the Original Direction with ROLESWITCH *NO

The OS/400 Cluster Resource Services jobs that run in the QSYSWRK subsystem on an active cluster node are listed in Table 2:

To view the job logs of these jobs if they end (Table 2), you can issue the command WRKSPLF QSYS and search for either the job name or the job function name.Any underlined text in Figure 5 references a topic in this document that you should read for information about how to proceed. These links are:• Rejoining a Non-Functioning Node with a Role Switch on page 362• Rejoining a Non-Functioning Node without a Role Switch on page 363• Removing a Failed Node from the Cluster on page 364

Restarting Active Groups with Mirroring in the Original Direction with ROLESWITCH *NOUse this procedure when the primary node has failed but you have determined that you do not want to make the backup node your production server. You would perform the procedure below for a group whose cluster status is *ACTIVE and whose primary node is in the FAILED state when all of the following points are true:• The status of the non-active node is *FAILED.• The failed node will return to the cluster in the near future.• The group's ROLESWITCH parameter is set to *NO.• The failed node will be the group's primary node when it rejoins the cluster.See Failed State Recovery Paths on page 350 for a graphical representation of this information.

Procedure1 Repair the failed node (machine) and re-start it.

You then need to return the node to the cluster by performing Step 2 - Step 5 (below).2 Make sure that TCP/IP and the *INETD TCP/IP server are active on the machine.3 Make sure that the XDMCLUSTER subsystem and the DMCHATL job are running.

Table 2 OS/400 Cluster Resource Services Jobs in the QSYSWRK Subsystem

Job Name Function

QCSTCRGM PGM-QCSTCRGMA

QCSTCTL PGM-QCSTCCJOB

QCSTINETD Not available.

DM_INTGRP PGM-QCSTCRGJOB

All other jobs whose function name is PGM-QCSTCRGJOB. The job names are the names of *CRG objects (cluster resource groups) in the cluster.


If not, you can start the subsystem using the STRSBS command. The DMCHATL job should start automatically. If it does not, then you can start it by issuing the STRHATCP—Start TCP/IP Listener command. Make sure that you specify "dmcluster" as the TCP/IP service name.

4 Re-start the node by running the DMREJOIN—iCluster Rejoin Cluster command on the failed node or by issuing the DMSTRNODE—Start Cluster Operations at Node command on another active node in the cluster.

Warning: Do not issue the DMSTRNODE command on the previous failed node.Replication for active groups with ROLESWITCH *NO and whose primary node failed will not re-start because iCluster did not complete the failover processing for these groups.

5 Run the DMSTRWO—Switchover Group or the DMSWOAPP—Switchover Resilient Application command for active groups whose primary node failed and rejoined the cluster.

This will cause the OS/400 Cluster Resource Services to perform a role switch at the operating system level and the node that failed will again be the primary node for these groups. This will also cause replication to restart automatically for these groups. Replication will be restarted at the journal positions last received in the staging store before the primary node failed.

Restarting Active Groups with Mirroring in the Original Direction with ROLESWITCH *YESYou would perform the procedure below for a group whose cluster status is *ACTIVE and whose primary node is in the FAILED state when all of the following points are true:• When the status of the non-active node is *FAILED.• The failed node will return to the cluster in the near future.• The group's ROLESWITCH parameter is set to *YES, or the group is part of a resilient application.• The previous backup node will not become the new primary node after the failed node rejoins the cluster. For example, the

failed node will be the group's primary node when it rejoins the cluster.See Failed State Recovery Paths on page 350 for a graphical representation of this information.


You then need to make sure that the node rejoins the cluster and that replication is re-started by performing Step 2 - Step 4.2 Make sure that TCP/IP and the *INETD TCP/IP server are active on the machine.3 Make sure that the XDMCLUSTER subsystem and the DMCHATL job are running.


4 Restart the node by running the DMREJOIN—iCluster Rejoin Cluster command on the failed node or the DMSTRNODE—Start Cluster Operations at Node on another active node. This should cause replication to restart automatically for the groups that were active at the time that the node failed.

After the node has rejoined the cluster and the replication has re-started, then you should perform Step 5. Warning: Users should not resume working on either node until Step 5 has been completed. 5 Run the DMSTRWO—Switchover Group command or the DMSWOAPP—Switchover Resilient Application command for the

active groups whose primary node failed and rejoined the cluster.


Restarting Active Groups with Mirroring in the Opposite Direction with ROLESWITCH *YES

Restarting Active Groups with Mirroring in the Opposite Direction with ROLESWITCH *YESYou would perform the procedure below for a group whose cluster status is *ACTIVE and whose primary node is in the FAILED state when all of the following points are true:• When the status of the non-active node is *FAILED.• The node will be returned to the cluster in the near future.• The group's ROLESWITCH parameter is set to *YES, or the group is part of a resilient application.• The previous backup will become the new primary node after the failed node rejoins the cluster.See Failed State Recovery Paths on page 350 for a graphical representation of this information.


You then need to make sure that the node rejoins the cluster and that replication is re-started by performing Step 2 - Step 4.2 Make sure that TCP/IP and the *INETD TCP/IP server are active on the machine.3 Make sure that the XDMCLUSTER subsystem and the DMCHATL job in the XDMCLUSTER subsystem are running.

If it is not started, you can start the subsystem using the STRSBS command. The DMCHATL job should start automatically. If it does not, then you can start it by issuing the STRHATCP—Start TCP/IP Listener command. Make sure that you specify "dmcluster" as the TCP/IP service name.

4 Restart the node by running the DMREJOIN—iCluster Rejoin Cluster command on the node that failed or the DMSTRNODE—Start Cluster Operations at Node on another active node.This should cause replication to restart automatically for the groups that were active at the time that the node failed.

Restarting Active Groups with Mirroring in the Opposite Direction with ROLESWITCH *NOYou would perform the steps on a group whose cluster status is *ACTIVE and whose primary node has failed when all of the following points are true:• When the status of the non-active node is *FAILED.• The failed node will return to the cluster in the near future.• The group's ROLESWITCH parameter is set to *NO.• The original backup node will become the new primary node after the failed node rejoins the cluster.See Failed State Recovery Paths on page 350 for a graphical representation of this information.

Procedure1 Run the DMSETPRIM—Prepare Primary Node command for the groups with ROLESWITCH *NO whose primary node has

failed. This will complete the failover processing by iCluster so that the previous backup node is now ready to act as the primary node for replication.

Note: You should not start making changes on the new primary system until this step is completed.2 Repair the machine (node) and re-start it.


You then need to make sure that the node rejoins the cluster and that replication is re-started by performing Step 3 - Step 5.3 Make sure that TCP/IP and the *INETD TCP/IP server are active on the machine.4 Make sure that the XDMCLUSTER subsystem and the DMCHATL job are running.


5 Re-start the node by running the DMREJOIN—iCluster Rejoin Cluster command on the failed node or the DMSTRNODE—Start Cluster Operations at Node on another active node.

Replication should restart automatically for the groups that were active at the time that the node failed.

Rejoining a Non-Functioning Node with a Role SwitchYou follow this procedure to recover a group whose primary node is functioning in either of the following scenarios:

Scenario 1 - False PARTITION• The machine can be repaired and returned to the cluster with minimal downtime.• The failed node will take over the backup role after it is repaired.

Scenario 2 - FAILED Node• The cluster status of the group is *INACTIVE. • The machine in question can be repaired and returned to the cluster with minimal downtime. • The failed node will take over the backup role after the repair.See Failed State Recovery Paths on page 350 and Partition State Recovery Paths on page 356 for a graphical representation of this information.


You then need to return the node to the cluster by performing Step 2 - Step 4.2 Make sure that TCP/IP and the *INETD TCP/IP server are active on the machine.3 Make sure that the XDMCLUSTER subsystem and the DMCHATL job are running.


4 Return the failed node to the cluster by running the DMSTRNODE—Start Cluster Operations at Node command for the failed node from another active node, or else by running the DMREJOIN—iCluster Rejoin Cluster command on the failed node.Note that it can take up to 15 minutes for the status of the failed node to become *ACTIVE when viewed from another node if it was previously in a *PARTITION state.The node should now be returned node to the cluster. Perform Step 5 - Step 6 (below).

5 Run the DMCHGROLE—Change Group Primary Node command for the groups whose primary node is the failed node. This action will cause the backup node for the groups to become the primary node, and likewise the primary node to become the backup node. Once this process has finished, users can resume their work on the new primary node(s).


Rejoining a Non-Functioning Node without a Role Switch

6 If you want, you can re-start the inactive groups for which you performed a role switch (see Step 5) using either the DMSTRGRP—Start Cluster Operations at Group or the DMSTRAPP—Start Cluster Operations for Resilient Application command. You should verify that mirroring re-starts properly.

Warning: You do not perform Step 6 for Scenario 2 (see above).

Rejoining a Non-Functioning Node without a Role SwitchYou follow this procedure to recover a group whose primary node is in a false PARTITION state without performing a role switch in either of the following scenarios:

Scenario 1 - False PARTITION• The status of the non-active node is *PARTITION.• The machine in question is not running.• The communication failure cannot be fixed quickly and without ending TCP/IP.• The machine in question can be repaired and returned to the cluster with minimal downtime.• The failed node will retain the primary node after it rejoins the cluster.

Scenario 2 - FAILED Node• The status of the non-active node is *FAILED.• The cluster status of the group is *INACTIVE. • The failed node will remain as the group's primary node after it rejoins the cluster.See Failed State Recovery Paths on page 350 and Partition State Recovery Paths on page 356 for a graphical representation of this information.


You then need to return the failed node to the cluster by performing Step 2 - Step 6.2 Make sure that TCP/IP and the *INETD TCP/IP server are active on the node.3 Make sure that the XDMCLUSTER subsystem and the DMCHATL job are running.


4 Return the failed node to the cluster by running the DMSTRNODE—Start Cluster Operations at Node command for the failed node from another active node, or else by running the DMREJOIN—iCluster Rejoin Cluster command on the failed node.Note that it can take up to 15 minutes for the status of the failed node to become *ACTIVE when viewed from another node.

5 If you want, you can re-start the groups that were active before the failure and whose primary node is the failed node. You should also verify that mirroring re-starts properly.

6 Verify that mirroring re-starts properly for active groups where the failed node is the backup node. For these groups, mirroring should re-start automatically when the failed node is re-started.

Note: For Scenario 2, mirroring will not re-start because the group is inactive.


Removing a Failed Node from the ClusterYou follow this procedure when you encounter one of scenarios listed below:

Scenario 1• The status of the non-active node is *FAILED. • The cluster status of each group that has the failed node as its primary node is *ACTIVE. • The node will not be returned to the cluster in the near future.

Scenario 2• The status of the non-active node is *FAILED. • The cluster status of each group that has the failed node as its primary node is *INACTIVE. • The node will not be returned to the cluster in the near future.See Failed State Recovery Paths on page 350 for a graphical representation of these scenarios.

Procedure1 On an active node, issue the DMRMVNODE—Remove Node command for the failed node on an active node.

This will remove the failed node from the cluster, and will cause an automatic role switch for any groups whose primary node was the failed node.

Note: If you want to re-start replication for the groups whose primary or backup node was the failed node, you may need to add a new backup node to the cluster and define that node as the backup node for the groups that do not have a backup node.

Once the groups have a backup node, you can re-start replication by performing Step 2 - Step 4. 2 Issue either the DMENDGRP—End Cluster Operations for Group or DMENDAPP—End Cluster Operations for Resilient

Application command to end all active groups.3 Perform a save/restore or refresh of replicated objects to the new backup node.Note: If a save/restore of replicated objects has been performed, issue the DMMRKPOS—Mark Current Journal

Positions command for the groups prior to starting the group. The group must be started using the marked positions.

4 Re-start replication by issuing either the DMSTRGRP—Start Cluster Operations at Group or DMSTRAPP—Start Cluster Operations for Resilient Application command with the USEMARKED parameter set to *YES.

Warning: If the failed node, which is now removed from the cluster, is to return to the cluster at a later time, then you must delete the cluster from the node before performing cluster operations on it. You can do this by running the DMDLTCLSTR—Delete Cluster command on the failed node only. You must not run this command on any other node.

iCluster and Resilient ApplicationsIn addition to replicating objects in your clustered environment, you may also want to have the same applications running on a new primary node after a switchover or a failover occurs. Applications that can be restarted on another cluster node are called resilient.


iCluster and Resilient Devices

Resilient applications are designed by their software vendors to operate in a clustered environment. This means that if a primary node is no longer operational, the same resilient application installed on the backup node can be started with minimal or no disruption to service. Application vendors provide the necessary data areas and files that allow their application to operate in a clustered environment. You should check with the application vendor whether the application has been designed to operate in a clustered environment.In iCluster, you do not have to be aware of the contents of these data areas and files. Instead, iCluster provides commands to add (DMADDAPP—Add Resilient Application), change (DMCHGAPP—Change Resilient Application), remove (DMRMVAPP—Remove Resilient Application), and update (DMUPDAPP—Update Resilient Application) a resilient application. The application vendor provides the data areas and object specifier files during installation for replication of its required data by iCluster. Other commands allow you to start (DMSTRAPP—Start Cluster Operations for Resilient Application) and end (DMENDAPP—End Cluster Operations for Resilient Application) cluster operations for resilient applications.Note: The resilient application, along with the necessary data areas and files, must reside on all nodes in the

recovery domain in order to support a resilient application switchover or failover.When you define a resilient application in your iCluster environment, iCluster will create several replication groups and an application group as part of the resilient application.If the resilient application exists on both nodes, then server B should take over processes of server A when server A is no longer operational.If you request to switch over a resilient application, then:• The resilient application's exit program that was running on the primary node is ended and is called again by OS/400 Cluster

Resource Services with the new recovery domain.• Replication is stopped and switched over using the process for data group switchover. See iCluster Concepts on page 342 for

more information for more information. During a failover or switchover, the takeover IP address is removed from the previous primary node and associated with the new primary node and activated on the new primary node. This allows users to log onto the new primary node using the same IP address.In iCluster, you can use the DMSWOAPP—Switchover Resilient Application command for manual switchovers to initiate a role switch within the groups associated with a resilient application. After issuing this command, the current primary node in each group becomes the backup node, and the current backup node in each group assumes the role of the primary node. It is important to note that groups associated with resilient applications are not switched over when using the DMSTRWO—Switchover Group command.If you want to specify a role switch user exit program for a resilient application, the resilient application must have one and only one replication group associated with it. Otherwise, the roleswitch user exit program(s) you specify will not be used for the resilient application. The list of replication groups associated with a resilient application can be seen by using the DMWRKGRP—Work with Groups command. The DMWRKGRP—Work with Groups command identifies the groups that are associated with a resilient application by highlighting them in a list. Make sure that only one of the highlighted groups is of type *REPL if you wish to specify a role switch user exit program for the resilient application. For resilient applications, the process for switching over is similar to that of data groups, except the GRP commands are APP commands. Also note that resilient applications do not have a role switch parameter, and therefore they behave as if the ROLESWITCH parameter is set to *YES.

iCluster and Resilient DevicesA resilient device is a physical resource represented by a configuration object, such as a device description, that is accessible from more than one node in a cluster. You need to define objects for inclusion in the resilient device, and you must include in the cluster any nodes that will use the resilient device. In the event of an outage, the point of access for the resource is switched to


the first backup node in the cluster resource group recovery domain. Currently, Independent Auxiliary Storage Pools (IASP) are the only type of device that can be defined as resilient. An IASP can go offline or online independently of the rest of the system storage.A resilient device cluster resource group can contain a list of switchable devices. Each device in the list identifies a switchable IASP. The entire collection of devices (up to 256) are switched to the backup node when an outage occurs.Note that resilient device groups can only be added through the iCluster iSeries DMADDGRP—Add Group command, and not through iCluster Administrator.iCluster supports commands for adding (DMADDSWDEV—Add Switchable Device Entry to Group), changing (DMCHGSWDEV—Change Switchable Device Entry for Group), and removing (DMRMVSWDEV—Remove Switchable Device Entry for Group) IASP devices from your cluster group.Once you have associated an IASP device with a switchable resource group by using the DMADDSWDEV—Add Switchable Device Entry to Group command, the IASP device is assigned to the primary node of a group, but is re-assigned to the new primary node after a switchover or failover of the group.

Alertable Messages for ClusteringOS/400 Clustering Resource Services generates several types of messages - alertable and informational - which are listed in Table 3 and Table 5 below.

Alertable MessagesThe following alertable messages are generated by OS/400 Cluster Resource Services:Table 3 Alertable Messages

Message ID Severity Alert Option Message Title

CPFBB20 40 *IMMED Cluster partition detected for cluster [name] by cluster node [node name].

CPFBB06 40 *IMMED Incoming request from cluster node &3 in cluster &2 rejected.

CPIBB10 50 *IMMED Cluster resource group exit program &1 in library &2 on node &3 failed.

CPIBB11 50 *IMMED Automatic recovery of cluster object &1 attempted.

CPFBB22 40 *IMMED OS/400 Cluster Resource Services communications failure on cluster node &2.

CPFBB47 50 *IMMED OS/400 Cluster Resource Services detected an error and may have ended abnormally.

CPFBB48 40 *IMMED OS/400 Cluster Resource Services error detected.


Switching Over to Another Node

Informational MessagesThe following informational messages are generated by OS/400 Cluster Resource Services:

Switching Over to Another NodeYou can use the following procedure to switchover a master node to another node.1 Make sure that all nodes of the cluster are in *ACTIVE status.

You can use the DMWRKNODE—Work with Nodes command to list the nodes in the cluster and their status.2 End the node that is the current master node. Another node in the cluster will assume the role of the master node.

See the DMENDNODE—End Cluster Operations at Node command for more information on ending a node.3 Restart the node that was ended. It will no longer be the master node.See the DMSTRNODE—Start Cluster Operations at Node command for more information on restarting a node.

Table 4 Informational Messages

Message ID Severity Alert Option Message Title

CPFBB4F 40 *NO Automatic fail over not started for cluster resource group [CRG name] in cluster [name].

CPIBB09 00 *NO Cluster resource group exit program [name] in library [name] on cluster node [system name] called.

CPFBB21 00 *NO Cluster partition condition no longer exists for cluster [cluster name].


SwitchOver System Concepts

Switchover for SwitchOver System (SOS)This section contains information about the switchover and failover mechanisms for clusters that use SwitchOver System (SOS).Warning: If your cluster is NOT configured to use SwitchOver System (SOS), then DO NOT refer to this section.

Instead, refer to Switchover for IBM Cluster Resource Services (CRS) on page 341.

SwitchOver System ConceptsThis section contains the following topics:• SwitchOver System Overview on page 369• User Actions on page 369• Detecting Node Failures on page 370• Node Failure Walkthroughs on page 370

SwitchOver System OverviewSwitchOver System (SOS) integrates the proven failover mechanism from DataMirror High Availability Suite into iCluster. Its purpose is to simplify node and replication group management and support while serving as an alternative failover mechanism to IBM Cluster Resource Services (IBM CRS). SOS provides a comparable set of functionality to IBM CRS, with the exception of support for resilient applications and switched disks.Organizations can use SwitchOver System (SOS) to support operational switching between two iSeries systems. It provides high availability when a production system is considered to be no longer operational. When this occurs, you can use SOS to switch operations to a backup system so that normal activities can resume as soon as possible. SOS is tightly integrated with iCluster for high availability when switching operations to a backup machine. It automatically detects when the communication link has failed and invokes the corrective measures that you specify. You can specify how frequently the communication links should be polled to verify that they are working properly. You can also define a switch condition by specifying the maximum waiting time to receive a response from a polled communication link and how many consecutive communication failures are tolerated. It also provides the necessary support for user exits and messaging.For more information about Failover Mechanisms in iCluster, see Failovers and Switchovers on page 47.

User ActionsUser actions consist of generating messages into one or more message queues and optionally invoking a user exit program each time SwitchOver System produces a message. This provides a way to configure your cluster to instantly notify administrators after detecting node failures. You can also use user actions to perform cleanup and other maintenance tasks both before and after a role switch occurs.

Message QueuesAfter declaring a switch condition, SwitchOver System can periodically generate messages and place them in a message queues. Each backup node can have its own message queue. iCluster generates the message HAS0220, which contains a single, ten character parameter that identifies the backup node that detected the node failure. Each time a message is produced, it is placed the queue.When you configure the backup node, you can specify how many messages you want to generate, how long to wait between messages, and in which queue to place the messages.


User ExitsUser exits offer the flexibility of invoking user-written programs to perform a specific task. Therefore, each time a message is generated, you can invoke a user exit program to accommodate requirements that are different for each working environment. You can configure nodes and groups to identify the name of the user exit program that you want to invoke. The delay period between consecutive messages also indicates the period of time between consecutive user exit calls.

Detecting Node FailuresClusters using SwitchOver System (SOS) detect node failures by having the backup node in each group poll the primary node at a configurable interval. After the backup node determines that it cannot communicate with the primary node, it changes the status of the primary node to *FAILED. Note: Since the backup node cannot communicate with the primary node after a node failure, it is unable to notify

the primary node of its status change. The primary node remains *ACTIVE.Each backup node has its own set of timeout and threshold values for declaring a failover. The basic properties that you can configure to detect a failure are:• Which IP addresses the backup node polls the primary node on. If you configure the primary node to poll both the primary

and alternate IP addresses and it does not receive a response on either poll, then it considers the poll a failure. For example, if the backup node receives a response from the primary IP address poll but not the alternate IP address poll, then the poll failed.

• How frequently the backup node polls the primary node.• How long the backup node waits for a response from the primary node.• How many consecutive polls must fail for the backup node to fail the primary node.After determining that the primary node is unavailable using these criteria, the backup node can automatically switch the group so the backup node becomes the primary node. This property is configured for each group and two groups with the same backup node can have different values. You can configure the cluster to do the following when detecting node failures or performing switchovers:• Place messages generated by the failover into a message queue.• Run user exit programs before and after switching the group. These user exits are configured at the group level and apply to

failovers and switchovers.See Node Failure Walkthroughs on page 370 for an example of the effect of these properties.See Configuring Nodes for Operational Switching on page 379 and Configuring Groups for Operational Switching on page 380 for more information.

Node Failure WalkthroughsThis topic illustrates the effects of the various configuration values on a failover. For a description of the parameters, see Detecting Node Failures on page 370. For more information on configuring these values, see Configuring Nodes for Operational Switching on page 379 and Configuring Groups for Operational Switching on page 380.This examples use the following configuration values:



Group Level Settings

Node Level Settings

Table 1 Group Level Settings

Setting Keyword and Value Meaning

Do roleswitch at failover Manual Failover Example:ROLESWITCH( *NO )

After the backup node changes the status of the primary node to *FAILED, replication is suspended and the group waits for an administrator to either repair the problem or initiate a manual failover.

Automatic Failover Example:ROLESWITCH( *YES )

After the backup node changes the status of the primary node to *FAILED it automatically takes over as the group's primary node.

Check journaling at roleswitch

CHKJRN( *NO ) Do not ensure each journaled object is journaled on the original backup node when switching roles.

Failover message queue name

MSGQUEUE( QUEUES/ROLECHG ) Messages generated by the failover are placed in the ROLECHG message queue.

User exit before roleswitch BSWUSREXIT( APPS/PREEXIT ) Active nodes call PREEXIT before switching roles.

User exit after roleswitch ASWUSREXIT( APPS/POSTEXIT ) Active nodes call POSTEXIT after switching roles.

Table 2 Node Level Settings


Check primary link CHKPRIMLNK( *YES ) The backup node polls the primary IP address of the primary node.

Check alternate link CHKALTLNK( *YES ) The backup node also polls the alternate IP address of the primary node.

Link check frequency (seconds) LNKCHKFRQ( 120 ) The backup node polls the primary node every 2 minutes (120 seconds).

Link check reply timeout (seconds)

LNKCHKRTO( 60 ) The backup node waits for one minute (60 seconds) before considering a primary node poll to be a failure.

Maximum link check retries LNKCHKTRY( 3 ) The backup node allows three consecutive polling failures before changing the status of the primary node to *FAILED.

Message action user exit MSGUSREXIT( APPS/MSGEXIT ) After failing the primary node, the backup node calls the MSGEXIT user exit.


Manual Failover WalkthroughThe figure below shows a timeline, for a group that replicates data from a node named TORONTO (primary) to a node named NEWYORK (backup), where the primary node becomes unavailable in a group configured for manual failover. The timeline shows the sequence of events from the backup node's perspective. The backup node performs all operations in the event of a failover.

Message queue name MSGQUEUE( QUEUES/NODEFAIL ) After failing the primary node, the backup node places a message in the NODEFAIL message queue.

Wait time after action (minutes) MSGACTWAIT( 1 ) The backup node places subsequent messages in the NODEFAIL message queue every minute. The backup node also calls MSGEXIT once for every queued message.

Number of message actions NUMMSGACT( 5 ) Five messages are placed in the NODEFAIL message queue, if the primary node fails. The backup node calls MSGEXIT once for every queued message.

Table 2 Node Level Settings



In this example, the administrator has chosen to perform a manual failover at 8:20. Alternatively, the administrator could have tried to repair TORONTO and restart the group. See Performing a Manual Failover on page 381 and Recovering from Failovers on page 381 for more information.

Automatic Failover WalkthroughThe figure below shows a timeline for an automatic failover. The configuration in this example is the same as the one described for a manual failover, except that the ROLESWITCH parameter is set to *YES. This indicates that the backup node must automatically failover to the primary node after changing its status to *FAILED.


This example shows how the type of failover affects the message actions configured for the backup node. Since this group automatically fails over, only one message is placed in the NODEFAIL message queue and the MSGEXIT user exit is only called once.This differs from the manual failover example, where five messages were placed in the NODEFAIL message queue and the MSGEXIT user exit was called five times.

Configuring Operational SwitchingThis section contains the following topics:• Operational Switching Overview on page 376• Defining User Exit Programs on page 376• Sample User Exits on page 378• Configuring Nodes for Operational Switching on page 379• Configuring Groups for Operational Switching on page 380

Operational Switching OverviewThe following sections describe the tasks that you must perform to configure your network to support using SwitchOver System for operational switching. You should perform these tasks in the following order:Step 1: Defining User Exit Programs on page 376 Step 2: Configuring Nodes for Operational Switching on page 379 Step 3: Configuring Groups for Operational Switching on page 380

Defining User Exit ProgramsUser exit programs allow you to execute a set of tasks from the backup node at various stages of node failure detection and role switching.Note: User exit programs must be compiled with the Use adopted authority option set to *NO. Otherwise, the user

exit programs cannot be executed and mirroring will continue.

User Exit EventsYou can configure your cluster to run user exits in conjunction with the following switchover events:• Before the backup node performs its failover processing. This user exit is run immediately before the role switch occurs.

Each group can call a different user exit when this happens. The user exit you configure with this parameter will run on all of the group's active nodes. For example, if you are performing a switchover, then this user exit will run on both the primary and backup nodes. For an automatic failover, this user exit will only run on the backup node because the primary node will have a status of *FAILED.

• After the backup node performs its failover processing. This user exit is run immediately after the role switch. Each group can call a different user exit when this happens. The user exit you configure with this parameter will run on all of the group's active nodes.

• When the backup node places a message in its queue to track repeated communication failures. Each node in the cluster can call a different user exit when this happens and you cannot have two different user exits for the same backup node.


Configuring Operational Switching

Role Switch User Exit ParametersWhen the backup node calls a user exit before or after performing a role switch, it passes the following parameters. You can use these values in the user exits you write.

Message Action User Exit ParametersWhen the backup node calls a user exit while queuing node failure messages, it passes the following parameters to the user exit. You can use these values in the user exits you write.

See Sample User Exits on page 378 for examples of user exits that use these parameters.

Detecting Problems in Role Switch ProcessingTo detect problems during role switch processing, you can use the following command in your role switch user exit program:SNDPGMMSG MSGID(CSI9953) MSGF(*LIBL/HAMSGF) TOPGMQ(*SAME)

After detecting problems with role switch processing, iCluster will stop role switch processing and declare that the role switch did not complete successfully by putting message CSI1357 (role switch for group &1 could not be completed) into the *CLUSTER portion of the event log.

Table 3 Role Switch User Exit Parameters

Position Description Length and Type

1 The name of the group the switch applies to. 10 characters

2 The state of the role switch. The possible values are *BEFORE and *AFTER depending on if the role switch has already occurred or not.

10 characters

3 The node that the exit is running on. The possible values are *PRIMARY and *BACKUP for the primary and backup nodes, respectively.

10 characters

4 User-specified data from the group's EXITDATA property.

256 characters

Table 4 Message Action User Exit Parameters

Position Description Length and Type

1 The name of the backup node. 10 characters

2 The role of the node initiating the switch. The value of this parameter is always *BACKUP for SOS.

10 characters

3 An event code indicating why the user exit was called. The value of this parameter is always *MSG.

10 characters

4 The number of messages that have been placed in the queue for this node failure.

2 characters


Sample User ExitsThe following sample user exits demonstrate how you can access the parameters that are passed to user exits. Additional examples are available in <iCluster Library>/QACLSRC. See Defining User Exit Programs on page 376 for more information on user exits and their parameters.

Sample Role Switch User ExitThis user exit runs before and after a group switches its roles and sends the parameter values to the parameters.PGM PARM(&GROUP &RSSTATE &ROLE &USRDATA)/* declare the parameters and other variables */DCL VAR(&GROUP) TYPE(*CHAR) LEN(10)DCL VAR(&RSSTATE) TYPE(*CHAR) LEN(10)DCL VAR(&ROLE) TYPE(*CHAR) LEN(10)DCL VAR(&USRDATA) TYPE(*CHAR) LEN(256)DCL VAR(&MSG) TYPE(*CHAR) LEN(50)MONMSG MSGID(CPF0000)/* display the group's name */CHGVAR VAR(&MSG) VALUE('Group name is ' *BCAT &GROUP *BCAT '.' )SNDPGMMSG MSG(&MSG)/* display the state of the role switch */IF COND(&RSSTATE *EQ '*BEFORE ') THEN(DO) CHGVAR VAR(&MSG) VALUE('Role switch pending.') SNDPGMMSG MSG(&MSG)ENDDOIF COND(&RSSTATE *EQ '*AFTER ') THEN(DO) CHGVAR VAR(&MSG) VALUE('Role switch complete.') SNDPGMMSG MSG(&MSG)ENDDO/* display the current node's role */CHGVAR VAR(&MSG) VALUE('Role is ' *BCAT &ROLE *BCAT '.')SNDPGMMSG MSG(&MSG)/* display the user data */SNDPGMMSG MSG(&USRDATA)ENDPGM

Sample Message Action User ExitThis user exit runs each time the backup node places a message in the node failure queue. It displays the name of the backup node and the number of queued messages.PGM PARM(&BUNODE &ROLE &REASON &COUNT)/* declare the parameters and other variables */DCL VAR(&BUNODE) TYPE(*CHAR) LEN(10)DCL VAR(&ROLE) TYPE(*CHAR) LEN(10) /* always *BACKUP */DCL VAR(&REASON) TYPE(*CHAR) LEN(10) /* always *MSG */DCL VAR(&COUNT) TYPE(*CHAR) LEN(2)DCL VAR(&MSG) TYPE(*CHAR) LEN(50)MONMSG MSGID(CPF0000)/* display the name of the backup node */CHGVAR VAR(&MSG) VALUE('Backup node ' *BCAT &BUNODE *BCAT '...')SNDPGMMSG MSG(&MSG)/* display the number of messages that have been placed in the queue */CHGVAR VAR(&MSG) VALUE('...has queued ' *BCAT &COUNT *BCAT ' messages.')SNDPGMMSG MSG(&MSG)ENDPGM


Configuring Operational Switching

Configuring Nodes for Operational SwitchingTo enable nodes for operational switching, you must configure a set of SOS properties. You can do this when you add the node to the cluster or when reconfiguring nodes with the DMADDNODE—Add Node or DMCHGNODE—Change Node commands, respectively. This topic only describes the specific properties you must set to use SOS.Table 5

Property Keyword Description

Check primary link CHKPRIMLNK Sets the backup node to poll the primary node on its primary IP address. The valid values for this property are *YES and *NO.If you set this property to *NO, then the backup node does not detect node failures on the primary node.

Check alternate link CHKALTLNK Sets the backup node to poll the primary node on its alternate IP address in addition to its primary address. The valid values are *YES and *NO.If you set this property to *YES, then the backup node polls the primary node on both its primary and secondary IP addresses, regardless of the value of CHKPRIMLNK.

Link check reply timeout LNKCHKRTO Sets how long the backup node should wait for a response from the primary node before considering the polling attempt to be a failure. Specify this value in seconds.

Link check frequency LNKCHKFRQ Sets how often the backup node should poll the primary node. Specify this value in seconds.

Maximum link check retries

LNKCHKTRY Sets how many consecutive polling failures are allowed before the backup node changes the status of the primary node to *FAILED.

Message queue name MSGQUEUE Sets the message queue to place messages regarding the status of the primary node after its status is changed to *FAILED.

Wait time after action MSGACTWAIT Sets how long the backup node should wait between placing messages in the message queue. Specify this value in minutes.


See Node Failure Walkthroughs on page 370 for an example of how these values affect a failover.

Configuring Groups for Operational SwitchingTo enable groups for operational switching, you must configure a set of properties. You can do this when you add the group or when reconfiguring groups with the DMADDGRP—Add Group or DMCHGGRP—Change Group commands, respectively. This topic only describes the specific properties you must set to configure operational switching.

The values of the CHKJRN, MSGQUEUE, BSWUSREXIT, and ASWUSREXIT properties are used for automatic failovers, manual failovers, and switchovers.

Message action user exit MSGUSREXIT Specifies a user exit to run when each message is placed in the queue. See Defining User Exit Programs on page 376 for more information.

Number of message actions

NUMMSGACT Sets the maximum number of message actions you want the backup node to place in the message queue. A message action consists of placing a message in the queue defined with MSGQUEUE and calling the user exit set with MSGUSREXIT.For automatic failovers, only one message action occurs and that is at the time of the failover.

Table 6


Do roleswitch at failover ROLESWITCH Configures the group to perform an automatic role switch after the backup node changes the status of the primary node to *FAILED. The possible values are *YES and *NO.

Check journaling at roleswitch

CHKJRN Sets the backup node to ensure its objects are journaled before switching roles.

Failover message queue name

MSGQUEUE Sets the message queue to place messages generated by the role switching process.

User exit before roleswitch

BSWUSREXIT Specifies the user exit to call before switching roles. This user exit runs on all active nodes in the group. See Defining User Exit Programs on page 376 for more information.

User exit after roleswitch ASWUSREXIT Specifies the user exit to call after switching roles. This user exit runs on all active nodes in the group. See Defining User Exit Programs on page 376 for more information.

Table 5



Administering Operational Switching

See Node Failure Walkthroughs on page 370 for an example of how these values affect a failover.

Administering Operational SwitchingThis section contains the following topics:• Performing a Manual Failover on page 381• Recovering from Failovers on page 381• Performing a Switchover on page 382

Performing a Manual FailoverIf you have configured a group to not perform an automatic failover, then you have the option of performing a manual failover after the backup node changes the state of the primary node to *FAILED and suspends replication. At this point, you have the option of either repairing the problem on the primary node and then resuming the group or performing a manual switchover and switching the roles of the primary and backup nodes for the group. This topic describes the steps for performing a manual failover. See Recovering from Failovers on page 381 for more information on the option.Note: The examples in this topic refer to a group that replicates from TORONTO (primary node) to NEWYORK

(backup node).

To perform a manual failover:1 Change the roles of the primary and backup nodes by running the following command on the backup node:

DMCHGROLE GROUP(<group name>)For example, this makes NEWYORK the primary node and TORONTO the backup node.

2 Repair the failed primary node. For example, plug TORONTO into an alternate power source.3 Start the failed primary node by running the following command:

DMSTRNODE NODE(<node name>)For example, enter DMSTRNODE NODE(TORONTO).

4 Restart replication by running the following command:DMSTRGRP GROUP(<group name>) USEMARKED(*YES)This starts replication from the backup node to the primary node because you changed the role of the group's nodes in step 1. You must specify the USEMARKED(*YES) property to ensure replication includes all changes since the failover. In our example, this command starts replication from NEWYORK to TORONTO.

The manual failover is complete. To revert the group back to its original roles (replicating TORONTO to NEWYORK) you must perform a switchover. See Performing a Switchover on page 382 for more information.

Recovering from FailoversIf you have configured a group to perform an automatic failover, then you must perform a switchover to restore the group to resume replication in the original direction. Before performing a switchover, both nodes must have a state of *ACTIVE. See Performing a Switchover on page 382 for more information.


If you have configured a group to not perform an automatic failover, then you have the option of performing a manual failover after the backup node changes the state of the primary node to *FAILED and suspends replication. At this point, you have the option of either repairing the problem on the primary node and then resuming the group or performing a manual switchover and switching the roles of the primary and backup nodes for the group. This topic describes the steps for repairing the primary node and resuming operations. See Performing a Manual Failover on page 381 for more information on the option.

To restore a group from this state:1 Repair the primary node. For example, if it was disconnected from the network, then restore the connection.2 Start the failed primary node by running the following command on the current, and active, primary node:

DMSTRNODE NODE(<node name>)3 Restart replication by running the following command:

DMSTRGRP GROUP(<group name>) USEMARKED(*NO)You must specify the USEMARKED(*NO) property to ensure replication restarts from the last position replicated.

The group is now restored and replication continues normally.

Performing a SwitchoverA switchover allows you to reverse the roles of the nodes in a group. This topic describes the tasks you must perform to complete a successful switchover.Note: The examples in this topic refer to a group that replicates from TORONTO (primary node) to NEWYORK

(backup node).

To perform a switchover:1 End the group by entering the following command on the backup node:

DMENDGRP GROUP(<group name>)2 Initiate the switchover by entering the following command:

DMCHGROLE GROUP(<group name>)3 Start the group by entering the following command:

DMSTRGRP GROUP(<group name>) USEMARKED(*YES)This starts replication from the backup node to the primary node because you changed the role of the group's nodes in step 2. You must specify the USEMARKED(*YES) property to ensure replication starts at the right positions.

The group now replicates from the original backup node to the original primary node. For example, NEWYORK now replicates data to TORONTO.


Minimum Requirements

iCluster Administrator for WindowsThis section describes how to install and use the iCluster Administrator and Event Viewer on the Windows platform.

Minimum RequirementsTo run the iCluster Administrator and Event Viewer on a Windows workstation, you need the hardware and software listed below.

Hardware Requirements• RAM: 2 megabytes in addition to the current disk space requirements for the Java 2 Runtime Environment (see below).

Software Requirements• Operating System: Microsoft Windows 95 and greater, Microsoft Windows NT 4.0 and greater.• Java 2 Runtime Environment: Version 1.2.0 and greater.Note: Since the minimum requirements for iCluster Administrator and Event Viewer are tightly coupled with the

requirements for running Java 2, consult http://www.java.com for the current requirements.

Installing iCluster AdministratorTo install iCluster software on your Windows workstation, perform the following steps:1 Locate the iCluster/Gui folder on the iCluster CD-ROM.2 To launch the iCluster installation program, double-click on the setup.exe file in this folder.

The Welcome dialog box appears.3 Proceed through the sequence of dialog boxes to complete the installation.After completing the installation, you are now ready to start the iCluster Administrator. See Logging on to iCluster Administrator for Windows on page 60 for more information.

Upgrading iCluster AdministratorIf you want to upgrade to a newer version of the iCluster Administrator, perform the following steps:1 Uninstall the previous iCluster Administrator. See Uninstalling iCluster Administrator on page 383 for more information.2 Perform the steps to install the iCluster Administrator (see above).It is recommended that you install the iCluster Administrator into the directory that was used for the previous installation.

Uninstalling iCluster AdministratorFollow the procedure below to uninstall the iCluster Administrator from your system.Note: The following steps are based on an iCluster uninstall procedure for the Windows XP operating system. The

uninstall procedure may be different depending on the supported operating system where you have installed iCluster Administrator.


To uninstall iCluster Administrator:1 From the Windows Start menu, select Programs > DataMirror > iCluster > uninstall iCluster Administrator.

The Uninstall iCluster Administrator dialog box opens.2 Perform the steps to uninstall iCluster Administrator.You may be prompted to re-start your system so that all changes can take effect.

iCluster Administrator and Event ViewerThe iCluster Administrator provides an interface through which you can easily manage and monitor activities in your clustered environment. From a Windows workstation, the iCluster Administrator offers a complete view of the nodes in your cluster, the groups that you have defined, the objects you are replicating, and the applications that are resilient in your environment.iCluster consists of two main components: the iCluster Administrator and the iCluster replication engine. The iCluster Administrator is the graphical user interface that you use to configure iCluster for replication and switchovers. The iCluster replication engine handles requests from the iCluster Administrator to configure the cluster and to replicate data. Together, these two components constitute a high availability solution for the iSeries machines.You can examine messages generated from any part of the cluster using the iCluster Event Viewer on the same Windows workstation. The iCluster Event Viewer, is used to maintain messages generated during replication, communication, and cluster activities. The event log is maintained on each node and contains messages about events involving that node. This makes it easier to identify and address issues and concerns that may arise during normal operations. Therefore, the iCluster Administrator and the iCluster Event Viewer provide you with the ability to manage your clustered environment from a single, central location through interfaces that conveniently organize nodes, groups, objects, resilient applications, and event messages.


Monitoring Operations in the iCluster Administrator

You must install the iCluster replication engine on each iSeries node in the cluster. For installation instructions, see Installation and Upgrade.See the following topics for more information about logging on to iCluster Administrator and starting the Event Viewer:• Logging on to iCluster Administrator for Windows on page 60• Starting the iCluster Event Viewer on page 510

Monitoring Operations in the iCluster AdministratorIn the iCluster Administrator, node icons change color to indicate different statuses. To confirm that cluster, node, resilient application, and group operations have started, you can inspect the status information shown on the table view. In the tree view, node, resilient application, and group icons change color to indicate its current status: • Blue node icons indicate a cluster status of *ACTIVE. • Red node icons indicate a cluster status of *INACTIVE or *UNKNOWN. • Yellow node icons indicate the other statuses.To confirm that a command has completed successfully, you can display event messages produced by iCluster. See Starting the iCluster Event Viewer on page 510 for more information. You can also examine the status of the entity in the table view. For example, to confirm that cluster operations at the specified node have stopped, inspect the node status shown on the table view.

Cluster SecuritySince a cluster consists of multiple iSeries nodes that have been configured to provide continuous operations, it is critical that the level of access to the cluster configuration is assigned to each user in an appropriate manner. iCluster contains security mechanisms and recognizes levels of security that can be associated with user IDs.

Security LevelsThe following security levels are defined:• Administrator: Administrators can modify any aspect of the cluster. This means that administrators are able to invoke any

supported command except cluster security commands. Cluster security commands, such as DMADDUSR—Add User and DMRMVUSR—Remove User, can only be invoked by QSECOFR or a user name having *SECADM authority from the command line. For more information, see Command Security on page 63.

• Operator: Operators can initiate cluster operations as well as inspect the nodes, groups and object specifiers that have been defined in the clustered environment. Operators can also view cluster system values and journal positions. Unlike administrators, operators are not able to define entities in the clustered environment or set security levels.

• User: Users are limited to inspecting the nodes, groups, object specifiers, and resilient applications in the clustered environment and the Event Viewer and Status Monitor.

• Other: To use these commands you do not have to be a registered iCluster user. Note: iCluster administrators and operators must have *IOSYSCFG (input/output system configuration) authority to

invoke commands at these levels.These security levels effectively enable or disable certain cluster commands for each individual user (see below). The security level(s) that permit a user to invoke a specific command are identified in this manual.


Privileges granted to each level decrease from administrator to operator to user. This implies that an administrator can issue commands at the administrator, operator, and user levels. An operator can issue commands at the operator and user levels, while a user can only issue commands at the user level.

Setting System ValuesUse this command to set certain cluster-wide values that affect different aspects of the clustered environment you have defined using iCluster Administrator.The minimum security level for this command is Administrator (*ADMIN).

To set system values1 From the iCluster Administrator window, select Set System Values from either the File or Command menu.

The Set System Values dialog box opens.


Setting System Values

You can set the following iCluster system values:• Default polling interval box—Specify the time interval that determines how often iCluster should check for content changes

to user spaces.You set the interval by specifying the number of hours (first two digits), minutes (middle two digits), and the seconds (last two digits) between consecutive polls. The polling interval only applies to user spaces. The specified interval must be between 10 seconds (000010) and 23 hours, 59 minutes, and 59 seconds (235959), inclusively.Special value:• *NONE—Specifies that user spaces should not be polled.

The default value for this parameter is five minutes (000500).• Communications timeout—Specify the waiting period that has to expire before iCluster can declare a communications

failure.During the waiting period specified in this box, iCluster will attempt to establish communications. If communications cannot be established during this time, a communications failure is reported.The default value for this parameter is one hour (010000).

• Object compression—Select whether you want to compress objects in save files that are generated by iCluster and replicated to backup nodes.Compressing objects consumes CPU cycles, but it may lead to faster transfer times. If objects are compressed on the primary node, iCluster automatically de-compresses the objects when they are received on the backup node.Possible values:• *NO—Does not compress all objects.• *YES—Compresses all objects in save files before replication.

The default value for this parameter is *NO.• Save active—Indicates whether an object can be updated and saved at the same time it is being transferred to the backup

node.Possible values:• *NO—Does not allow objects that are in use to be saved and updated. Objects cannot be updated while being saved.• *YES—Allows objects that are in use by another job to be saved and updated. Objects may reach checkpoints at

different times and may not be in a consistent state in relation to each other.• *SYNC—Allows objects that are in use by another job to be saved and updated. All of the objects reach a checkpoint

together and are saved in a consistent state in relationship to each other.The default value for this parameter is *NO.

• Save operation output—Indicates whether restore operations will create a spool file that identifies which objects were restored.This setting only applies to refresh operations, since mirroring does not generate spool files. You can use this spool file to refresh specific objects at a later time.Possible values:• *ERROR—Generates a spool file for save operations that produced an error. All other save operations will not generate

a spool file.• *FULL—Generates a spool file to identify save operations.• *NONE—Does not generate a spool file to identify save operations.

The default value for this parameter is *ERROR.


• Default job description—Specify the name of the job description that you want to designate as the default iCluster job description for replication jobs.The default value for this parameter is CSJOBD.

• Default job description library—Specify the name of the library where the default job description currently resides.Special values:• *PRODLIB—Specifies your iCluster installation library.• *LIBL—Specifies the set of libraries in your library list. The libraries are searched in order for the first occurrence of the

specified default job description.The default value for this parameter is DMCLUSTER.

• Group job start delay (secs)—Specify the number of seconds, between 1 and 60, of delay between the start of replication processes when cluster operations are initiated for groups.This delay is only used when multiple replication processes are started through a single command invocation. It is intended for working environments that consist of relatively small systems. You can use this setting to distribute the start of replication processes over a period of time so as to avoid high peaks in CPU usage that would typically occur if these processes were started without delay.The default value for this parameter is two seconds.

• Automatic reactivation—Indicate whether iCluster should try to reactivate suspended objects.Possible values:• *YES—Specifies that iCluster should try to reactivate suspended objects. • *NO—Specifies that iCluster should not reactivate suspended objects.

The default value for this parameter is *YES.• Reactivation interval (min)—Specify, in minutes, the number of minutes between automatic retries.

You can specify a value between 1 and 1440.The default value for this parameter is 10 minutes.

• Max. reactivation attempts—Specify the number of times that iCluster will try to perform an automatic reactivation of a suspended object before halting activation.Valid entries range between 1 and 32767.Special value:• *NOMAX—Indicates that there is no maximum value. iCluster will continue to perform an automatic reactivation until it

succeeds.The default value for this parameter is *NOMAX.

• Max. reactivation size (bytes)—Specify the maximum size, in bytes, of an object to be included in automatic reactivation.Special value:• *NOMAX—Indicates that there is no maximum value. This value could have serious performance issues on the primary

node if very large objects are locked frequently by other jobs.The default value for this parameter is *NOMAX.

• Lock files on backup nodes—Indicates whether physical files should be locked on the backup node when they are being updated.When locked, no other users can modify these files.Possible values:



• *NO—Specifies that physical files will not be locked when they are being updated, meaning that users can modify these files on the backup node.

• *YES—Specifies that physical files cannot be modified on the backup node when they are being updated.The default value for this parameter is *NO.

• Maximum refresh size—Specify the size of the largest physical file, in kilobytes, that can be refreshed automatically over the network.If a physical file is too large to be refreshed, it will be marked as suspended and will not be refreshed. In this case, you are responsible for ensuring the physical file is refreshed.Journal entries added after activating the suspended physical file will be sent to the backup node. Journal entries added before the file is activated will not be sent to the backup node. As a result, you will need to make sure that the file was not changed when it was suspended.Special value:• *NOMAX—Indicates that there is no maximum refresh size.

The default value for this parameter is *NOMAX.• Hold config obj src on backup—Indicates whether you want iCluster to create configuration objects automatically once they

are received on a backup node or hold the commands for creating them in specific physical files so that they can be created at a later time.If configuration objects that are being replicated already exist on backup nodes, you should set this value to *YES to prevent iCluster from trying to create objects when they are in use.Possible values:• *YES—Holds commands to create configuration objects in specific physical files so that they can be created at a later

time.• *NO—Automatically creates configuration objects as soon as they are received on the backup node.

The default value for this parameter is *YES.For more information about configuration objects, see Replicating Configuration Objects on page 90.

• Replicated user profile status—Specify the status that you want to assign to user profiles that have been replicated to a backup node.Possible values:• *DISABLED—Specifies that all replicated user profiles will be set to a status of *DISABLED.• *PRIMARY—Specifies that replicated user profile status will be set to the same status that is currently assigned to the

corresponding user profile on the primary node.• *NOCHG—Specifies that the current status of each user profile will not be changed by iCluster.

The default value for this parameter is *DISABLED.• User profile replication level—Indicate whether or not dependent user profiles should be replicated when a main user profile

is replicated.Dependent user profiles include object owner profiles, group profiles, supplemental profiles, and profiles on an authorization list associated with the replicated profile.If you replicate dependent user profiles, then the relationship between user profiles is preserved so that there is an exact mirroring of user profiles. However, replicating all dependent user profiles may affect performance if the number of dependent user profiles is considerable.Possible values:• *FULL—Indicates that all dependent profiles are replicated. This includes group and supplemental profiles, profile

owners, dependent profiles, and so on.


• *BASIC—Indicates that only profiles that match the selected object specifier will be replicated. During replication, iCluster assumes that all dependent profiles already exist on the backup node. Error messages will be generated if the dependent profiles are not on the backup node, though replication will continue.

The default value for this parameter is *FULL.• Replicate usrprf for *AUTL—Select whether you want to replicate authorization lists with or without the user profiles.

An authorization list object consists of a list of user profiles. This setting dictates whether authorization lists are replicated with or without these user profiles. If user profiles are replicated, it is important to recognize which user profiles will be sent to the backup node, as you may want to restrict access to this node. On the other hand, if a user profile in an authorization list does not exist on the backup node, the replicated authorization list will not be the same as the authorization list on the primary node if user profiles are not replicated.Possible values:• *NO—Does not replicate user profiles with the authorization lists.• *YES—Replicates user profiles with the authorization lists.

The default value for this parameter is *NO.• Replicate Q* profiles—Indicate whether you want to replicate user profiles that start with the letter Q.

Usually, all profiles that start with the letter Q are system-owned user profiles (for example, QSECOFR) and should not be replicated from one node to another. However, if you have defined user profiles that start with the letter Q that are not system-owned profiles, setting this box to *YES will allow you to replicate those profiles. Profiles that are owned by QSYS will not be replicated even when this box is set to *YES.Possible values:• *NO—Does not replicate any user profiles that start with the letter Q.• *YES—Replicates non-system-owned profiles that start with the letter Q.

The default value for this parameter is *NO.• Max. wait for spool files—Specify the time period, in HHMMSS format, you want iCluster to wait before abandoning the

replication of a spool file.You can replicate a spool file only if it has a status of *READY. If iCluster is up-to-date with the audit journal, it may start processing a spool file before it is ready. iCluster will only wait the amount of time specified in this box for the spool file to have a status of *READY. If the spool file is not ready after waiting the time specified in this box, iCluster will abandon the replication of this spool file and issue a message in the event log.The default value for this parameter is 15 seconds (000015).

• Max. wait for spool activity—Specify the time period, in HHMMSS format, you want iCluster to wait before abandoning the replication of a spool file if no data is being written to it.In some cases, a job may be writing to a spool file. However, the data is first placed in a temporary buffer and only written to the spool file upon the completion of the job. In such a case, the spool file will be open and have zero pages until the job ends.This box allows you to specify the time to wait for changes to occur to an open spool file. If no additional data is written to an open spool file (that is, it is open and has zero pages) within the time specified in this box, iCluster will abandon the replication of this spool file and issue a message. This setting provides added flexibility as you can specify a shorter interval over which only spool file activity is monitored. As a result, an open spool file that has no data being written to it will be abandoned quickly, as opposed to waiting the full time specified in the Max. wait for spool files box (see above).The default value for this parameter is 15 seconds (000015).

• Replicate *OUTQ contents—Select whether you want to mirror the contents of spool files. If you specify *NO, then iCluster will only replicate *OUTQ objects but will not replicate the spool files in them.Possible values:



• *YES—Indicates that the spool files will be created and the contents will be mirrored.• *NO—Indicates that the spool files will be created, but the contents will not be mirrored.

The default value for this parameter is *YES.• Commitment control level—Select the level of commitment control you want to use when replicating *FILE objects.

Commitment control stages database transactions so that they are assembled before being applied, or additionally, opened in a commitment control environment to ensure that all updates are received. It also makes sure that the changes are applied in the correct sequence. For more information, see Commitment Control on page 71.If you select *LEVEL2 but a file on the backup node cannot be opened under commitment control, *LEVEL1 commitment control will be used for that file.Possible values:• *NONE—Indicates that the update process on the backup node will not perform commitment control staging.• *LEVEL1—Indicates that all updates that comprise a transaction are assembled before being applied on the backup

node.• *LEVEL2—Indicates that all updates in a transaction are opened in a commitment control environment to ensure that all

updates are received. This option provides true commitment control.The default value for this parameter is *NONE.

• Journal images—Select whether default journaling should include both before and after images.Before images are necessary to remove applied or keyed replication updates. Also, if you specified the unique index refresh method for a file, then it must be journaled with both before and after images. If a file is replicated using relative record number, it may be journaled with after images only.

Note: For unique key updates, both before and after images must be journaled if commitment control is set to *LEVEL2.

Possible values:• *AFTER—Indicates that you want to journal the after image only.• *BOTH—Indicates that you want to journal both the before and after images.

The default value for this parameter is *AFTER.• Journal objects on backup—Indicates whether replicated physical files, data areas, and data queues will be journaled on

backup nodes.Before images are necessary to remove applied or keyed replication updates. Also, if you specified unique index refresh method for a physical file, then it must be journaled with both before and after images. If a physical file is replicated using relative record number, it may be journaled with after images only.For unique key updates, both before and after images must be journaled if Commitment Control Level (see above) is set to *LEVEL2.Possible values:• *NO—Specifies that replicated physical files, data areas, and data queues will not be journaled on backup nodes.• *YES—Specifies that replicated physical files, data areas, and data queues will be journaled on backup nodes.

The default value for this parameter is *NO.• Default database journal—Specify the name of the database journal that you want to use as your default.

The journal that you specify will be used for files that are to be mirrored but are not yet journaled.The default value for this parameter is HADJRN.

• Default database journal library—Specify the name of the library where the default database journal currently resides.Special values:


• *LIBL—Specifies the set of libraries in your library list. The libraries are searched in order for the first occurrence of the specified default database journal.

• *PRODLIB—Specifies your iCluster installation library.The default value for this parameter is *PRODLIB.

• Delete dependent non-group LF—Select whether you want to delete logical files that are dependent on physical files being replicated on the backup node.The logical files are not necessarily the ones you have selected to a group. Physical files that have dependent logical files can only be deleted if the dependent logical files are deleted first.

Note: If some of the logical files do not belong to the same group as the physical files, you can only delete the logical files by setting this system parameter to *YES.

Possible values:• *NO—Indicates that you do not want to delete backup logical files that are dependent on physical files. • *YES—Indicates that you want to delete backup logical files that are dependent on physical files in the event that the

physical files are deleted.The default value for this parameter is *NO.

• Max. record level errors—Specify the maximum number of record-level errors that must occur before iCluster stops replicating physical files automatically.Record-level errors are ones that occur when updating, inserting, or deleting records in a file during replication to a backup node.Special value:• *NOMAX—Specifies that replication of physical files will continue regardless of the number of record-level errors that are

generated.The default value for this parameter is one error.

• Default BSF journal—Specify the name of the journal that you want to use as the default journal for your Byte Stream File (BSF) objects.For more information, see Replicating BSF Objects on page 89.Special value:• HABSFJRN—Uses the default BSF journal supplied with iCluster.

Note: If you specify this value, then you must set the Default BSF journal library box (see below) to *PRODLIB.The default value for this parameter is HABSFJRN.

• Default BSF journal library—Specify the name of the library where the default BSF journal (see the Default BSF journal box above) resides.Special values:• *PRODLIB—Specifies your iCluster installation library.• *LIBL—Specifies the set of libraries in your library list. The libraries are searched in order for the first occurrence of the

specified default BSF journal.The default value for this parameter is *PRODLIB.

• Default event message queue—Specify the name of the message queue that will hold event messages generated by iCluster.If a message queue is identified, iCluster event messages will be placed in both the message queue and the event log. See Starting the iCluster Event Viewer on page 510 for more information.Special values:



• *NONE—Indicates that no message queue is specified. Event messages will be placed in the event log only.• HAMSGQ—Refers to the name of the event message queue that is supplied with iCluster. If you specify this value, then

you must set the Default message queue library box (see below) to *PRODLIB.The default value for this parameter is *NONE.

• Default message queue library—Specify the name of the library where the default event message queue (see the Default event message queue box above) resides.Special values:• *PRODLIB—Specifies your iCluster installation library.• *LIBL—Specifies the set of libraries in your library list. The libraries are searched in order for the first occurrence of the

specified default event message queue.• Remote date, time display—Specify the date and time settings that should be used to display event messages that originate

from a remote node.This value is relevant if local and remote nodes are located in different time zones.Possible values:• *LOCAL—Displays dates and times in the time zone where the local node is located.• *REMOTE—Displays dates and times in the time zone where the remote node is located.

The default value for this parameter is *LOCAL.• Logged msg lifetime (days)—Specify the maximum length of time (in days) that messages in the event log should be kept

before being removed.If the age of a message exceeds the time limit, this message is automatically removed when you display the event log using the DMDSPLOG—Display Cluster Event Log command.Special value:• *NOMAX—Disables automatic removal of event messages. Messages will remain in the log until they are explicitly

removed.The default value for this parameter is 30 days.

• Message generation levels—Specify the levels of messages that iCluster will generate. See Starting the iCluster Event Viewer on page 510 for more information.You can specify multiple levels in this box by separating the levels with commas.Possible values:• 00—Generates information messages (Level 00) (for example, Save File restored).• 10—Generates non-critical status messages (Level 10) (for example, Library created on target system).• 20—Generates stop/start messages (Level 20) (for example, Mirroring starting).• 30—Generates messages that report recoverable errors (Level 30) (for example, Could not rename object).• *ALL—Generates messages in all levels.

The default values for this parameter are 20 and 30.All fatal error messages (Level 40) are generated.

• Latency check interval (min)—Specify how often iCluster will check latencies (source receive and target apply) and compare them with their corresponding thresholds.Latency in iCluster is the time difference between what is on the primary node and what is on the backup node.Enter a value ranging between 1 -1440 (one day).


The default value is 5 (minutes).• Source receive threshold (min)—Specify the amount of latency that can be tolerated before a latency warning message is

issued. Source receive latency indicates the difference between the timestamp of the journal entry last processed by the backup receiver for the journal, and the timestamp of the last journal entry deposited in the journal on the primary (source) node.For idle or lightly used journals, iCluster determines the latency by putting an entry into a journal if it has been idle for one minute since the last journal entry.Enter a value ranging between 1 and 10080 (seven days).The default value is 60 (minutes).

• Target apply threshold (min)—Specify the amount of latency that can be tolerated before a latency warning message is issued. The target apply latency is the difference in the timestamps of the last entry received by the journal's receive process and the last entry applied by the journal's apply process.Enter a value ranging between 1 and 10080 (seven days).The default value is 240 (minutes).

• Use OS/400 Cluster Services—Select the failover mechanism you want you cluster to use to detect node failures. See Choosing a Failover Mechanism on page 73 for more information about this parameter before changing its value.Specify one of the following values:• *NO—Indicates that you want to use SwitchOver System.• *YES—Indicates that you want to use IBM Cluster Resource Services.

The default setting for this parameter is *NO.• Comms Channel—Specify whether a separate communications channel is used for each group or one channel for all groups.

Possible values:• *SHARED—Uses one communications channel for all groups.• *PERGROUP—Each group has its own communications channel. With separate communications channels for each

group, better line utilization can be obtained and problems in one group will not affect other groups. However, this requires more communications resources.

The default value for this parameter is *SHARED.• Staging Store Block Management—Specify how iCluster manages used staging store blocks. Normally, iCluster re-

allocates blocks to the memory pool after they are used, but now you can choose to not re-allocate the staging store blocks for a performance improvement in situations where you are using multiple apply jobs with large data volumes.Possible values:• *SHARED—Staging store blocks are re-allocated to the memory pool after they are used.• *PERGROUP—Staging store blocks are not re-allocated to the memory pool after they are used. This can result in a

performance improvement when you are using multiple apply jobs.The default value for this parameter is *SHARED.

2 Click Execute.Changes that you have made to cluster system values are registered. Depending on the cluster system values you have changed, the impact is immediate or takes effect when a command is subsequently issued.



Cluster Operation CommandsThis section contains the following topics:• Rejoin Cluster on page 395• Delete Cluster on page 396

Rejoin ClusterUse this command to re-start cluster node operations on the current machine so that it rejoins the cluster of which it was previously an active member. It also sets the status of the node on the current machine to active if the node successfully rejoins the cluster.If the current machine's node is the backup node of a group that has active status, cluster group operations will also be started at the current machine's node through this command.This command requires that the current machine is inactive in the cluster and is recognized as a node by at least one active node in the cluster.Note: You can issue this command only on an inactive node in the cluster.This command may take a few minutes to complete.The minimum security level for this command is Administrator (*ADMIN).

To restart node operations so that it rejoins the cluster1 From the iCluster Administrator window, select the Cluster heading, display its shortcut menu, and select Rejoin Cluster.

The Rejoin Cluster dialog box opens.

2 You can specify values in the following fields:• Number of Rejoin Attempts—Specify the number of attempts that will be made to rejoin an existing cluster.

The default value for this parameter is one attempt.• Delay Time Between Attempts—Specify the time interval, in seconds, between attempts to re-start cluster operations at the

current node.The default value for this parameter is 10 seconds.

• Start New Cluster box—Specify whether or not a new cluster will be started on the current machine if attempts to rejoin an existing cluster fail.Possible values:• *NO—Indicates that you do not want to start a new cluster on the current machine if attempts to rejoin an existing cluster

fail. The current node will remain inactive if attempts to rejoin an existing cluster fail.


• *YES—Indicates that you want to start a new cluster on the current machine if attempts to rejoin an existing cluster fail. The new cluster will not be part of an existing cluster.

The default value for this parameter is *NO.3 Click Execute.The status of the cluster should change to *ACTIVE.

Delete ClusterIf invoked on an active node, this command ends cluster operations and deletes cluster definitions maintained by iCluster on all active nodes in the cluster.If invoked on an inactive node, cluster operations are stopped and cluster definitions are deleted only on the node where the command is invoked. Therefore, to ensure that all cluster operations are stopped and all cluster definitions are deleted, you should issue this command on each node in the cluster.If a communication failure occurs between primary and backup nodes, issue this command on both partitioned nodes to ensure cluster operations are stopped and all cluster definitions are deleted.The minimum security level for this command is Administrator (*ADMIN).

To end cluster operations and delete cluster definitions on all active nodes1 On the iCluster Administrator window, select Delete Cluster from the Command menu.

The Delete Cluster dialog box opens.

The dialog box warns you that all operations on the selected node will be stopped, and the cluster definition will be deleted.2 Click Execute.All operations are stopped and the cluster definition is deleted.

Node CommandsThis section contains the following topics:• Working with Nodes on page 397• Add Node on page 397• Remove Node on page 402• Start Node on page 403• End Node on page 403• Change Node on page 404• Send Object on page 410


Node Commands

Working with NodesWhen you select a node in the tree view, the set of associated attributes is shown in the table view. You can define or change these attributes through the Add Node and Change Node commands respectively.For more information about nodes, see Working in a Clustered Environment on page 43.

Node StatusA status is always assigned to a node. It indicates the state of low-level cluster support provided by IBM Cluster Resource Services on the node. For more information on the different statuses that can be assigned to a node, see DMWRKNODE—Work with Nodes on page 110.

Monitoring OperationsIn the iCluster Administrator, node icons change color to indicate different statuses. See Monitoring Operations in the iCluster Administrator on page 385 for more information.

Add NodeUse this command to add a node to the cluster. Adding a node using this command automatically activates it in the cluster.The first node added to the cluster is considered as the master node. See Working in a Clustered Environment on page 43 for more information about the master node. You can define a maximum of 128 nodes in the cluster.The minimum security level for this command is Administrator (*ADMIN).

To add a node to the cluster1 From the iCluster Administrator window, select the Nodes heading in the tree view.2 Display the shortcut menu, and select Add Node.

The Add Node dialog box opens.


This dialog has the following tabs:• General, see Step 3.• SOS DCM Options, see Step 4.

3 On the General tab, you can specify values in the following fields:• Node Name—Specify the name that identifies the node in the cluster.• IP Address—Specify the primary IP address of the node being added to the cluster.

You can specify the IP address of the node in dotted quad notation (for example, 155.6.4.2) or in domain name form (for example, as400sys1a.abccorp.com).

• Alternate IP Address—Specify the secondary IP address of the node being added to the cluster.You can specify the IP address of the node in dotted quad notation (for example, 155.6.4.3) or in domain name form (for example, as400sys1b.abccorp.com).The secondary IP address is used by the failover mechanism to determine if a node in the cluster is not operational or the communication link connecting the node has failed. The secondary IP address is not used by iCluster for replication operations, and is an optional setting.

• Port—Specify the port number on the node that has been reserved for iCluster communications.


Node Commands

This port number was specified when defining the dmcluster service to the TCP/IP service table. For more information about adding this service to the table, see Adding a New Entry 'dmcluster' to the TCP/IP Service Table on page 37.The default value for this parameter is 4545.

• Product Library—Specify the library on the node where iCluster has been installed.The default value for this parameter is DMCLUSTER.

• Description—Specify a short description that allows you to identify this node among all others that have been defined in the cluster.The description can be a maximum of 50 characters long, and is an optional setting.

• Enable Switchable Resources—Specify whether you will be using switchable resources on the node. This parameter is used only on OS/400 V5R1 and greater. All the nodes in the cluster must also be at OS/400 V5R1 and greater.Possible values:• *YES—Enables the node to use switchable resources. If you specify this value, then the OS/400 option 41, HA

Switchable Resources, must be installed on the node and there must be a valid license key for the option.• *NO—Does not enable the node to use switchable resources.

The default value for this parameter is *NO.• Name (under Replication Job Description)—Specify the name of the job description that you want to associate with jobs

that handle replication activity on the specified node.Special value:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values on

page 386 for more information.The default value for this parameter is *CLUSTER.

• Library (under Replication Job Description)—Specify the name of the library where the specified job description currently resides.If you specified *CLUSTER in the Name box, a library name is not required or is ignored if it is specified. In this case, the corresponding cluster system value (Default job description library) will be used.

• User Profile Status—Select the status that you want to assign to user profiles that are replicated to the node you are adding to the cluster.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values on

page 386 for more information.• *PRIMARY—Sets the user profile status to the same status that is currently assigned to the corresponding user profile on

the source node.• *DISABLED—Sets all user profiles to a status of *DISABLED.• *NOCHG—Indicates that the current status of each user profile will not be changed by iCluster and the user profile status

on the backup node is set to *DISABLED by default.The default value for this parameter is *CLUSTER.

• Hold Config Object Source—Select whether you want iCluster to create configuration objects automatically once they are received on the specified node or hold the commands for creating them in specific physical files so that they can be created at a later time.Possible values:


• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values on page 386 for more information.

• *YES—Holds commands to create configuration objects in specific physical files so that they can be created at a later time. If a configuration object that is being replicated already exists on the node you are adding to the cluster, select *YES to prevent iCluster from trying to create the object when it is in use.

• *NO—Automatically creates configuration objects as soon as they are received on the node you are adding to the cluster.

The default value for this parameter is *CLUSTER.For more information about configuration objects, see Replicating Configuration Objects on page 90.

• Size (under Staging Store)—Specify the size, in megabytes, that you want to allocate for the staging store.The size of the store changes dynamically but it cannot exceed the size specified in this box. The size of the staging store can range from a minimum of 512 megabytes to a maximum of 1,048,576 megabytes.The default value for this parameter is 512 megabytes.For more information about staging stores, see Staging Objects.

• Library box (under Staging Store)—Specify the name of the library where the staging store will be located.If you specify a library that does not exist, iCluster will create it for you.The default value for this parameter is DMSTORE.

4 If your cluster uses IBM Cluster Resource Services, then proceed to Step 5. If your cluster uses SwitchOver System, then you can specify values in the following fields under the SOS DCM Options tab. For more information on SwitchOver System, see Switchover for SwitchOver System (SOS) on page 369.


Node Commands

• Check Primary Link—Select whether you want to test the primary IP address for communication failures between the primary and backup nodes. The possible values are: • *YES—Test the primary IP address for communication failures. • *NO—Do not test the primary IP address for communication failures.

The default value for this parameter is *YES.• Check Alternate Link—Select whether you want to test the alternate IP address for communication failures between the

primary and backup nodes. The possible values are:• *YES—Test the alternate IP address for communication failures.• *NO—Do not test the alternate IP address for communication failures.

The default value for this parameter is *NO.• Link Check Frequency—Enter how often, in seconds, you want to poll the primary and alternate links for communication

failures. The default value is 60 seconds.

• Link Check Reply Timeout—Enter the maximum number of seconds you want to wait for a response when polling links for failures. The default value is 30 seconds.


• Maximum Link Check Retries—Enter the number of consecutive failures you want to allow before iCluster considers the primary node to have failed. For example, if you set this parameter to 3, then after the fourth consecutive failure, iCluster will set the primary node to the *FAILED state. The default value is 5.

• Message Queue—Enter the name and library of the message queue that will receive messages after the number of consecutive communication failures is exceeded. You can either specify a name and library or select *NONE. The default value is *NONE.

Note: Both Number of Message Actions and Wait Time After Action must be set to non-zero values for messages to be sent to this queue after a failure.

• Message Action User Exit—Enter the name and library of the user exit program to run after the number of consecutive communication failures is exceeded. You can either specify a name and library or select *NONE. The user exit is run once for each message sent to the message queue. The default value is *NONE.

• Number of Message Actions—Enter the maximum number of messages to send through the message queue after detecting a failure. The system will wait for the time specified in the Wait Time After Action box between sending each message. The default value is 0.

• Wait Time After Action—Enter the number of minutes to wait before sending messages to the message queue after detecting a failure. The default value is 0 minutes.

5 Click Execute.On the iCluster Administrator window, the new node is added under the Nodes heading in the tree view.

Remove NodeUse this command to remove a node that was previously defined in the cluster. Cluster operations at the node are also stopped. You cannot reference a node after it has been removed from the cluster.If you are removing the only node in the cluster, the entire cluster is removed. In this case, the removal of the node is equivalent to the operations performed when you issue the Delete Cluster command.If the selected node is only a backup node in replication groups and applications, the node can be removed by this command. However, you cannot apply this command to the primary node of a replication group or resilient application. If you want to remove such a node from the cluster:• Remove all replication groups and resilient applications having this node as their primary node. See the Remove Full

Replication Group and Remove Resilient Application commands for more information.• End cluster operations at the node before removing it. See the End Node command for more information. After ending

cluster operations, ensure that you remove the cluster definitions from the inactive node by issuing the Delete Cluster command from that node.

The specified node must be active. If the node is inactive, the node will only be removed from the active part of the cluster. Therefore, an attempt to add the same node to the cluster at a later time will fail.The minimum security level for this command is Administrator (*ADMIN).

To remove a node1 From the iCluster Administrator window, expand the Nodes heading.2 Select the node in the tree view that you want to remove, display its shortcut menu, and select Remove Node.

The Remove Node dialog box opens.


Node Commands

This dialog box identifies the node that you selected on the iCluster Administrator window. 3 Click Execute.On the iCluster Administrator window, the node that you selected is removed from the tree view.

Start NodeUse this command to start cluster operations at the specified node. It also sets the status of the specified node to *ACTIVE. If the selected node is the backup node of a group that has an active status, cluster operations for the group will also be started.This command does not start high availability support provided by iCluster.You must issue this command on an active node in the cluster unless there are no active nodes in the cluster.Note: If this command is invoked on different nodes, separate clusters are effectively activated. To ensure that only

a single cluster is maintained, issue this command for each node in the cluster from a single node. The first time you invoke the command, reference the name of the single node. Then, from the single node, issue the command for all the remaining nodes in the cluster.

The minimum security level for this command is Administrator (*ADMIN).

To start cluster operations at the specified node1 From the iCluster Administrator window, expand the Nodes select the node in the tree view where cluster node operations will

be started.2 Display the shortcut menu, and select Start Node.

The Start Node dialog box opens.

This dialog box identifies the node that you selected on the iCluster Administrator window. 3 Click Execute.

End NodeUse this command to end cluster operations at the specified node. This command also sets the status of the specified node to *INACTIVE. Use this command if you intend to re-start cluster operations at the specified node. Otherwise, remove the specified node from the cluster when it is active. See Remove Node for more information.Warning: You must not end a node before removing it.


If the node is the backup node of a replication group that has an active status, cluster operations for the group will also be ended at the specified node before cluster operations at the node are ended. The replication group will still have a status of *ACTIVE even though replication operations have ended.Note: You cannot stop cluster operations if the specified node is a primary node in an active replication group.The minimum security level for this command is Administrator (*ADMIN).

To end cluster operations at the specified node1 From the iCluster Administrator window, select the node in the tree view where cluster node operations will be ended.2 Display the shortcut menu, and select End Node.

The End Node dialog box opens.

This dialog box identifies the node that you selected on the iCluster Administrator window. 3 Click Execute.

Change NodeUse this command to change one or more attributes of the specified node in the cluster. The node that you change must be active in the cluster, with the following exception: If you are using OS/400 Cluster Resource Services, the node must be inactive when changing its IP address(es) for iCluster.If replication is active on the node being changed and you are using Cluster Resource Services, the node's IP addresses and job description will not be changed. All other node parameters can be changed when replication is active on the node. However, some parameters used by replication jobs will only come into effect when new replication jobs are started. Existing replication jobs will continue to use the previous parameters, with the exception of the staging store size. The staging store size comes into use by all backup replication jobs as soon as it is changed.If you are not using OS/400 Cluster Resource Services, the node must be active when changing its IP address(es) or any other parameter of the node.Note: You cannot change the staging store library using this command. To change the existing staging store library

for the specified node, you must remove the node, and then re-add it to the cluster. The minimum security level for this command is Administrator (*ADMIN).

To change the specified node in the cluster1 From the iCluster Administrator window, select the Nodes heading in the tree view.2 Select the node that you want to change in the tree view, display the shortcut menu, and select Change Node.

The Change Node dialog box opens.


Node Commands

This dialog has the following tabs:• General, see Step 3.• SOS DCM Options, see Step 4.• Advanced, see Step 5.

3 On the General tab, you can specify values in the following fields:• IP Address—Specify the primary IP address of the node being added to the cluster.

You can specify the IP address of the node in dotted quad notation (for example, 155.6.4.2) or in domain name form (for example, as400sys1a.abccorp.com).

• Alternate IP Address—Specify the secondary IP address of the node being added to the cluster.You can specify the IP address of the node in dotted quad notation (for example, 155.6.4.3) or in domain name form (for example, as400sys1b.abccorp.com).


The secondary IP address is used by the failover mechanism to determine if a node in the cluster is not operational or the communication link connecting the node has failed. The secondary IP address is not used by iCluster for replication operations, and is an optional setting.

• Port—Specify the port number on the node that has been reserved for iCluster communications.This port number was specified when defining the dmcluster service to the TCP/IP service table. For more information about adding this service to the table, see Adding a New Entry 'dmcluster' to the TCP/IP Service Table on page 37.The default value for this parameter is 4545.

• Description—Specify a short description that allows you to identify this node among all others that have been defined in the cluster.The description can be a maximum of 50 characters long, and is an optional setting.

• Enable Switchable Resources—Specify whether you will be using switchable resources on the node. This parameter is used only on OS/400 V5R1 and greater. All the nodes in the cluster must also be at OS/400 V5R1 and greater.Possible values:• *SAME—Keeps the present setting for this parameter.• *YES—Enables the node to use switchable resources. If you specify this value, then the OS/400 option 41, HA

Switchable Resources, must be installed on the node and there must be a valid license key for the option.• *NO—Does not enable the node to use switchable resources.

The default value for this parameter is *SAME.• Change Node Status—Specify whether the status of the specified node should be changed.

This parameter is used only on OS/400 V5R1 and greater.Possible values:• *SAME—Keeps the present setting for this parameter. • *FAILED—Specifies that a node has failed but its status in the cluster is *PARTITION. After the status of the node has

been changed to *FAILED, it can either be removed from the cluster or re-started if it is capable of rejoining the cluster. See the Switchover for IBM Cluster Resource Services (CRS) for more information.

The default value for this parameter is *SAME.• Name (under Replication Job Description)—Specify the name of the job description that you want to associate with jobs

that handle replication activity on the specified node.Special value:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values on

page 386 for more information.The default value for this parameter is *CLUSTER.

• Library (under Replication Job Description)—Specify the name of the library where the specified job description currently resides.If you specified *CLUSTER in the Name box, a library name is not required or is ignored if it is specified. In this case, the corresponding cluster system value (Default job description library) will be used.

• User Profile Status—Select the status that you want to assign to user profiles that are replicated to the node you are adding to the cluster.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values on

page 386 for more information.


Node Commands

• *PRIMARY—Sets the user profile status to the same status that is currently assigned to the corresponding user profile on the source node.

• *DISABLED—Sets all user profiles to a status of *DISABLED.• *NOCHG—Indicates that the current status of each user profile will not be changed by iCluster and the user profile status

on the backup node is set to *DISABLED by default.The default value for this parameter is *CLUSTER.

• Hold Config Object Source—Select whether you want iCluster to create configuration objects automatically once they are received on the specified node or hold the commands for creating them in specific physical files so that they can be created at a later time.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values on

page 386 for more information.• *YES—Holds commands to create configuration objects in specific physical files so that they can be created at a later

time. If a configuration object that is being replicated already exists on the node you are adding to the cluster, select *YES to prevent iCluster from trying to create the object when it is in use.

• *NO—Automatically creates configuration objects as soon as they are received on the node you are adding to the cluster.The default value for this parameter is *CLUSTER.For more information about configuration objects, see Replicating Configuration Objects on page 90.

• Size (under Staging Store)—Specify the size, in megabytes, that you want to allocate for the staging store.The size of the store changes dynamically but it cannot exceed the size specified in this box. The size of the staging store can range from a minimum of 512 megabytes to a maximum of 1,048,576 megabytes.The default value for this parameter is 512 megabytes.For more information about staging stores, see Staging Objects.

4 If your cluster uses IBM Cluster Resource Services, then proceed to Step 5. If your cluster uses SwitchOver System, then you can specify values in the following fields under the SOS DCM Options tab. For more information on SwitchOver System, see Switchover for SwitchOver System (SOS) on page 369.


• Check Primary Link—Select whether you want to test the primary IP address for communication failures between the primary and backup nodes. The possible values are: • *YES—Test the primary IP address for communication failures. • *NO—Do not test the primary IP address for communication failures.

The default value for this parameter is *YES.• Check Alternate Link—Select whether you want to test the alternate IP address for communication failures between the

primary and backup nodes. The possible values are:• *YES—Test the alternate IP address for communication failures.• *NO—Do not test the alternate IP address for communication failures.

The default value for this parameter is *NO.• Link Check Frequency—Enter how often, in seconds, you want to poll the primary and alternate links for communication

failures. The default value is 60 seconds.

• Link Check Reply Timeout—Enter the maximum number of seconds you want to wait for a response when polling links for failures. The default value is 30 seconds.


Node Commands

• Maximum Link Check Retries—Enter the number of consecutive failures you want to allow before iCluster considers the primary node to have failed. For example, if you set this parameter to 3, then after the fourth consecutive failure, iCluster will set the primary node to the *FAILED state. The default value is 5.

• Message Queue—Enter the name and library of the message queue that will receive messages after the number of consecutive communication failures is exceeded. You can either specify a name and library or select *NONE. The default value is *NONE.

Note: Both Number of Message Actions and Wait Time After Action must be set to non-zero values for messages to be sent to this queue after a failure.

• Message Action User Exit—Enter the name and library of the user exit program to run after the number of consecutive communication failures is exceeded. You can either specify a name and library or select *NONE. The user exit is run once for each message sent to the message queue. The default value is *NONE.

• Number of Message Actions—Enter the maximum number of messages to send through the message queue after detecting a failure. The system will wait for the time specified in the Wait Time After Action box between sending each message. The default value is 0.

• Wait Time After Action—Enter the number of minutes to wait before sending messages to the message queue after detecting a failure. The default value is 0 minutes.

5 For more advanced options, select the Advanced tab.The Advanced tab of the Change Node dialog box opens.


• Authorization Code—Specify the new authorization code for the node that was provided by DataMirror Technical Support.The authorization code was specified during iCluster installation, and is required to run iCluster on the node. Therefore, use this parameter to change the existing authorization code for the node if it no longer allows you to run iCluster.Special value:• *SAME—Uses the current setting for this parameter.

The default setting for this parameter is *SAME.6 Click Execute.On the iCluster Administrator window, the new node is added under the Nodes heading in the tree view.

Send ObjectUse this command to replicate one or more objects that are not necessarily part of a group, to a target node immediately. The referenced objects are essentially refreshed to the target.


Node Commands

This command can be used to synchronize the backup node with the primary node before replication is started. Once replication is started, iCluster should detect the creation of new objects in replication scope and replicate them automatically. However, if replication is running and you have determined that some objects in replication scope are not found on the backup node, this command should NOT be used to send these objects to the backup node if they are journaled objects. By doing this you will prevent these objects from being properly handled by the replication process, that is, setting up the metadata that iCluster requires for changes to these objects to be replicated and starting journaling of the objects if they are not journaled. The correct way to deal with this issue is to use Activate Object on page 505 to activate and refresh the objects from the group's primary node to the backup node.Note: Journaled objects are physical database files, logical files, data areas and data queues.

To replicate one or more objects to a target node:1 From the iCluster Administrator window, expand the Nodes heading.2 Select the node in the tree view that contains the objects that you would like to send to the backup node, display its shortcut

menu, and select Send Object.The Send Object dialog box opens.

The Source Node Name value identifies the name of the node selected on the iCluster Administrator window. You cannot modify this value from this dialog box.You can specify values in the following fields:

• Target Node Name—Select the name of the node where the object will be sent.3 Select the Native Object Specifier or the Path Object Specifier option (Step 4).

If the objects you want to send to the target node are stored in the native OS/400 file system, select Native Object Specifier. If the objects you want to send to the target node are stored in the Integrated File System (IFS) or QDLS file system, select Path Object Specifier. See Path Object Specifiers on page 53 for more information.


Under Native Object Specifier, you can specify values in the following fields:• Object Specifier Name—Enter a specific or generic name (to identify multiple objects in a library).

Special value:• *ALL—Specifies all objects in a library.

The default value for this parameter is *ALL.• Object Specifier Library—Specify the library where the object resides.• Object Type—Specify an object type.

Special value:• *ALL—Specifies all object types.

The default value for this parameter is *ALL.For a complete list of all object types that iCluster can replicate, see Object Types Replicated by iCluster on page 529.

• Object Extended Attribute—Enter the attribute component of the object specifier you want to select.This parameter is only applicable when the OBJTYPE parameter (see above) is set to *FILE. Special value:• *ALL—Specifies all object attributes.

The default value for this parameter is *ALL.• Target Library list—Specify the name of the library on the backup system that will receive the replicated objects.

Special value:• *SRC—Sets the target library so that it is the same as the primary node library where the object resides.

The default value for this parameter is *SRC.If the primary and backup environments reside on the same physical system (local loopback implementation), the target library that you specify cannot be the same library where a selected object in the group currently resides. This restriction means that iCluster does not permit the special value *SRC in this situation.

4 In the Path box, specify the location of the path object specifier that you are defining.Path object specifiers identify the location of Byte Stream File (BSF) objects in the Integrated File System (IFS) and QDLS file system. See Path Object Specifiers on page 53 for more information about identifying the correct path.

5 Click Execute.

Group CommandsThis section contains the following topics:• Working with Replication Groups on page 413• Add Full Replication Group on page 413• Start Full Replication Group on page 421• End Full Replication Group on page 423• Remove Full Replication Group on page 424• Change Full Replication Group on page 424• Convert to Refresh Only Group on page 431• Switchover Full Replication Group on page 433


Group Commands

• Add Refresh-Only Group on page 434• Start Refresh-Only Group on page 437• End Refresh-Only Group on page 437• Remove Refresh-Only Group on page 438• Change Refresh-Only Group on page 439• Convert to a Full Replication Group on page 441• Add Master-Master Group on page 448• Start Master-Master Group on page 452• End Master-Master Group on page 454• Remove Master-Master Group on page 455• Change Master-Master Group on page 455

Working with Replication GroupsWhen a replication group is selected in tree view, the set of associated attributes is shown on the table view. In iCluster there are three types of replication groups:• *REPL—Identifies a full replication group. This type of group supports the continuous replication of objects and data between

nodes in the group.• *RFSH—Identifies a refresh-only group. This type of group performs a refresh of the objects selected to the group and then

shuts down. These groups are not eligible for role switch.• *MSTRMSTR—Identifies a master-master group. With these types of groups you can perform application updates to the same

objects on both the source and the target systems. iCluster replicates these updates in both directions. This is referred to as master-master replication.. These groups are not eligible for role switch. For more information, see iBalance and Master-Master Replication on page 321.

You can define or change the attributes of a full replication group the Add Full Replication Group and Change Full Replication Group commands.Likewise, you can define or change the attributes of a refresh-only group through the Add Refresh-Only Group and Change Refresh-Only Group commands.For more information about groups, see Working in a Clustered Environment.

Replication Group StatusesThere are two different statuses for replication groups in iCluster. The cluster status of a group indicates the state of its primary and backup nodes as reported by the failover mechanism. The replication status of a group indicates the state of object and data replication being performed by iCluster. See DMWRKGRP—Work with Groups on page 137 for more information on replication status.

Monitoring OperationsIn the iCluster Administrator, group icons change color to indicate different statuses. See Monitoring Operations in the iCluster Administrator on page 385 for more information.

Add Full Replication GroupUse this command to add a full replication group consisting of one primary node and possibly one backup node. All nodes in the replication group's recovery domain must be active.The minimum security level for this command is Administrator (*ADMIN).


To add a full replication group1 From the tree view in the iCluster Administrator window, select the Replication Groups heading.2 Right-click and select Add Replication Group.

The Add Replication Group dialog box opens. This dialog has the following tabs:• Properties, see Step 3.• Recovery Domain, see Step 4.• User Exit, see Step 5.• Objects, see Step 6• Spool File, see Step 7.• Physical File, see Step 8.• BSF, see Step 9.


Group Commands

3 On the Properties tab, you can specify the following values:• Replication Group Name—specify the name of the replication group that you want to define in iCluster.

The specified group name must be unique in the cluster.• Failover Message Queue Name—Specify the name of the message queue that will receive messages generated when a

failover occurs.Special value:• *NONE—Indicates that you do not want to specify a message queue.

The default value for this parameter is *NONE. • Failover Message Queue Library—Specify the name of the library where the specified message queue currently resides.

If you selected *NONE in the Failover Message Queue Name box, a library name is not required or is ignored if it is specified.• Do Role Switch at Failover—Select whether you want the role of the backup node to change automatically so that it

becomes the primary node in the replication group when a failover occurs.This parameter only applies to replication groups whose target library is *PRIMARY, and to MQSeries groups.If set to *YES, the functions of the primary node will be moved to the backup node when a failover occurs.Possible values:• *NO—Does not automatically change the role of the backup node in the replication group to the primary node when a

failover occurs. *NO gives you the opportunity to decide if you would like to continue with a role switch, or continue with your current configuration.

With this setting, a failure of the group's primary node is declared by the failover mechanism, although iCluster will not prepare the backup node to take over as the primary node. Replication will not start up automatically once the failed primary node is again active in the cluster.When using IBM Cluster Resource Services, only groups whose cluster status is *ACTIVE are switched over if the primary node fails. For more information on using IBM Cluster Services, see Switchover for IBM Cluster Resource Services (CRS) on page 341.When IBM Cluster Resource Services is not being used for iCluster operations, all groups that have the ROLESWITCH parameter set to *YES are switched over when the primary node fails, regardless of whether they are active or not.• *YES—Automatically changes the role of the backup node in the replication group to the primary node when a failover

occurs. In this case, user exit programs specified through the User Exit Before Role Switch and User Exit After Role Switch boxes (see below) will be called.

The default value for this parameter is *NO.Note: If you do not want an automatic role switch to be performed in the specified replication group (this parameter

set to *NO), use the Set Primary Node or the Change Role command to prepare the new primary node, that is, the previous backup node, for replication after a failover occurs. If you set this parameter to *YES, this step is not necessary.

• Target Library—Specify the name of the library on the backup system that will receive the replicated objects.Special value:• *PRIMARY—Sets the target library so that it is the same as the primary node library where the object resides.

The default value for this parameter is *PRIMARY.If the primary and backup environments reside on the same physical system (local loopback implementation), the target library that you specify cannot be the same library where a selected object in the group currently resides. This restriction means that iCluster does not permit the special value *PRIMARY in this situation.



• Journal Location—Specify the location of the database journal where scraping will occur.Special values:• *LOCAL—Indicates that the database journal(s) on the primary node are scraped during mirroring.• *REMOTE—Indicates that the remote database journal(s) are scraped on the backup node during mirroring.

The default value for this parameter is *LOCAL.Note: For more information on the configuration of remote journaling, see Configuring Remote Journaling on

page 84.• Check Journaling At Role Switch—Specify whether iCluster will check if all mirrored objects that should be journaled are

being journaled on the new primary node.Specifies whether (when switching to the new primary node) iCluster will check whether the objects for the specified group are being properly journaled on that node, and start journaling them. Certain types of objects must be journaled before replication starts. However, because journaling objects can be time-consuming, this option allows you to specify whether or not journaling should be started for objects during a switchover, depending on your business requirements.Depending on the value of the group's Check Journaling At Role Switch parameter, iCluster may perform processing that is equivalent to the Mark Journal Positions command when a roleswitch occurs.If the value of the group's Check Journaling At Role Switch parameter is *NO, iCluster will not perform Mark Journal Positions processing and the replication metadata for the new primary node is generated based upon the existing replication metadata on the current backup node. In the case of an error in processing the backup replication metadata, or non-existent backup replication metadata, iCluster performs Mark Journal Positions processing regardless of the Check Journaling At Role Switch parameter's value.

Note: The Mark Journal Positions command establishes the set of mirrored objects based on the actual objects that match specifiers on the backup system, regardless if they were previously mirrored or not, and starts journaling for the mirrored objects that should be journaled.

If the value of the group's Check Journaling At Role Switch parameter is *YES, iCluster will always perform Mark Journal Positions processing during a roleswitch, and the backup replication metadata is not used for setting up the replication metadata for the new primary node.Special values:• *YES—Specifies that iCluster will check if all mirrored objects that should be journaled are being journaled on the new

primary node. If they are not journalled, they will be.• *NO—Specifies that iCluster will not check if all mirrored objects that should be journaled are journaled on the new


The default value for this parameter is *YES.• Recovery Exposure—Specify the number of journal entries applied between writing recovery checkpoints. Specifying a

lower number decreases the performance of the apply, but provides a greater safeguard in recovery situations. Specifying a higher number lessens the impact of recovery checkpoints on the apply.Enter a recovery exposure value (number of journal entries) or the following value:• *DISABLED—Indicates that recovery checkpoints will not be generated. This is the default setting for this parameter.

The default value for this parameter is *DISABLED.• Recovery Journal Key—Specify the key, or the identifier, that will be assigned to a recovery checkpoint for the group

combination.


Group Commands

• Description—Enter a short description that allows you to identify this replication group among all others that have been defined in iCluster.The description can be a maximum of 50 characters long, and is an optional setting.

4 On the Recovery Domain tab, you can specify the following values:• Select recovery domain from existing group—Allows you to add a replication group with the same set of primary and

backup nodes as the recovery domain for an existing group. Defining two or more replication groups with the same set of nodes is useful when you want to replicate objects between the same nodes, but you would like to define different replication group behaviors for different sets of objects. In this case, select the name of an existing group that you want to use to define the recovery domain of the replication group you are adding.

• Specify the recovery domain—Indicates that you want to identify the nodes in the recovery domain. In this case, complete the following four fields.

The default option is Specify the recovery domain.• Primary Node—Select the name of a node that will be the primary node in the replication group you are adding.

The node that you specify must have been added to the cluster. See Add Node for more information. This box is only applicable if the Specify the recovery domain option is selected.

• Primary IASP Device Name—Specify the name of an existing independent auxiliary storage pool (IASP) on the primary node from which you want to replicate.Special value:• *SYSBAS—Indicates that you are using a system ASP rather than an independent ASP for storing the objects that you

want to replicate.The default value for this parameter is *SYSBAS.

• Local Loopback—Select if you would like the primary and backup environments to reside on the same physical system (local loopback implementation).This box is unchecked by default.

• Backup Node—Select the node that will be the backup node in the replication group you are adding.At this time, you can define only one backup node in a replication group.The node that you select must have been added to the cluster. See Add Node for more information. You can select a backup node only when the Specify the recovery domain option is selected.

Note: Both primary and backup nodes in a replication group must reside in the same subnet.• Backup IASP Device Name—Specify the name of an existing independent auxiliary storage pool (IASP) on the backup node

to which you want to replicate.Special value:• *SYSBAS—Indicates that you are using a system ASP rather than an independent ASP for storing the objects that will

be replicated.The default value for this parameter is *SYSBAS.

5 On the User Exit tab, you can specify the following values:Note: The parameters on this tab are not available for local loopback.• Name (under User Exit Before Role Switch)—Specify the name of the user exit program that you want to call immediately

before the role change is performed.The user exit program will be called on both nodes of the group for switchover, but only on the new primary node for a failover.This parameter only applies to replication groups whose target library is *PRIMARY, and to MQSeries groups.


If you specify a user exit program, the Do Role Switch at Failover box (see above) must be set to *YES if you want to call the program when a failover occurs.

Note: The specified user exit program is invoked for a switchover if you decide to prepare the backup node to be the primary node by issuing the DMSETPRIM—Prepare Primary Node command. If you decide not to prepare the backup node to be the primary node, and just wait for the primary node to become active, the user exits are not called.

Special value:• *NONE—Indicates that you do not want to specify a user exit program.

The default value for this parameter is *NONE.See the DMSETPRIM—Prepare Primary Node command and the arguments that can be passed to the user exit program.

• Library (under User Exit Before Role Switch)—Specify the name of the library where the before role switch user exit program currently resides.If you selected *NONE in the Name box, a library name is not required or ignored if it is specified.

• Name (under User Exit After Role Switch)—Specify the name of the user exit program that you want to call immediately after the role change is performed.After a role change occurs, the user exit program will be called on both nodes of the group.If you specify a user exit program, the Do Role Switch at Failover box (see above) must be set to *YES if you want to call the program when a failover occurs.




• Library (under User Exit After Role Switch)—Specify the name of the library where the after role switch user exit program currently resides.If you selected *NONE in the Name box, a library name is not required or is ignored if it is specified.

• Exit Data—Specify the user-defined data that you want to pass to the user exit programs specified on this tab (see above).This parameter only applies to replication groups whose target library is *PRIMARY, and to MQSeries groups.If you specify two different user exit programs (one program before the role switch, and a different program after the role switch), the same user-defined data is passed to both programs. A maximum of 256 bytes of data can be passed to the user exit programs. If your data contains spaces or non alpha-numeric characters (commas, periods, and so on), you must enclose your data in single quotes.

6 On the Objects tab, you can specify the following values:Under Group Polling Interval, specify the time interval that determines how often iCluster should check for content changes to user spaces.The polling interval is only applicable when you select user spaces to the replication group through the Select/Add Object Specifier command.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more

information.


Group Commands

• *NONE—Specifies that user spaces should not be polled.• Specify Values—This option allows you to set the time interval that determines how often iCluster should check for

content changes to user spaces. You set the interval by specifying the number of hours (Hours box), minutes (Minutes box), and seconds (Seconds box) between consecutive polls.

The default value for this parameter is *CLUSTER.• Save Active Objects—Select whether an object can be updated and saved at the same time it is being transferred to the

backup node.This setting does not apply to physical files because they cannot be modified while they are being saved.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more

information.• *YES—Allows objects that are in use by another job to be saved and updated. Objects may reach synchronization points

at different times and may not be in a consistent state in relation to each other.• *NO—Does not allow objects that are in use to be saved and updated. Objects cannot be updated while being saved.• *SYNC—Allows objects that are in use by another job to be saved and updated. All of the objects reach a

synchronization point together and are saved in a consistent state in relation to each other.The default value for this parameter is *CLUSTER.

7 On the Spool File tab, you can specify the following values:• Maximum Wait for Spool Files—Specify the time period (in hours, minutes, and seconds) you want iCluster to wait before

abandoning the replication of a spool file.You can specify a waiting period from 000001 to 235959.You can replicate a spool file only when it has a status of *READY. Therefore, if iCluster is up-to-date with the audit journal, it may start processing a spool file before it is ready. iCluster will only wait the specified amount of time for the spool file to have a status of *READY. If the spool file is not ready after waiting the specified time, iCluster will abandon the replication of this spool file and issue a message in the event log.Special value:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more

information. The default value for this parameter is *CLUSTER.

• Maximum Wait for Spool Activities—Specify the time period (in hours, minutes, and seconds) you want iCluster to wait before abandoning the replication of a spool file if no data is being written to it.In some cases, a job may be writing to a spool file. However, the data is first placed in a temporary buffer and only written to the spool file upon the completion of the job. In such a case, the spool file will be open and have zero pages until the job ends.This box allows you to specify the time to wait for changes to occur to an open spool file. If no additional data is written to an open spool file (that is, it is open and has zero pages) within the specified time, iCluster will abandon the replication of this spool file and issue a message. This option provides added flexibility as you can specify a shorter interval over which only spool file activity is monitored. As a result, an open spool file that has no data being written to it will be abandoned quickly, as opposed to waiting the full time specified under Maximum Wait for Spool Files (see above).Special value:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more


8 On the Physical File tab, you can specify the following values:


• Default Database Journal—Specify the name of the database journal that you want to use as your default.The journal that you specify will be used for files that are to be mirrored but are not yet journaled. iCluster will start journaling automatically to this journal.Special value:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more

information.The default value for this parameter is *CLUSTER.


• Default Database Journal Library—Specify the name of the library where the database journal currently resides.If you specified *CLUSTER in the Default Database Journal box, a library name is not required, as the corresponding cluster system value (Default database journal library) will be used.

• Commitment Control Level—Select the level of commitment control you want to use when replicating *FILE objects.Commitment control stages database transactions so that they are assembled before being applied, or additionally, opened in a commitment control environment to ensure that all updates are received. It also makes sure that the changes are applied in the correct sequence. For more information, see Commitment Control.If you select *LEVEL2 but a file on the backup node cannot be opened under commitment control, *LEVEL1 commitment control will be used for that file.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values on

page 386 for more information.• *NONE—Indicates that the update process on the backup node will not perform commitment control staging.• *LEVEL1—Indicates that all updates that comprise a transaction are assembled before being applied on the backup


updates are received. This option provides true commitment control.The default value for this parameter is *CLUSTER.

• Physical File Journal Images—Select whether default journaling should include both before and after images.Before images are necessary to remove applied or keyed replication updates. Also, if you specified unique index refresh method for a file, then it must be journaled with both before and after images. If a file is replicated using relative record number, it may be journaled with after images only.For unique key updates, both before and after images must be journaled if Commitment Control Level is set to *LEVEL2.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values on

page 386 for more information.• *BOTH—Indicates that you want to journal both the before and after images.• *AFTER—Indicates that you want to journal the after image only.

The default value for this parameter is *CLUSTER.9 On the BSF tab, you can specify values in the following fields:• Default BSF Journal—Specify the name of the journal that you want to use as your default for journaling BSF files that will be

replicated by this group.Note: This parameter only applies to replication group.


Group Commands

Note: This parameter is valid for BSF replication in system auxiliary storage pools or independent auxiliary storage pools.

The journal that you specify will be used for BSF files that are to be mirrored and journaled. See Select/Add Object Specifier for more information.Specify the name of the BSF journal or one of the following values:• *NONE—Indicates that you do not require journaling for BSF replication.• *CLUSTER—BSF objects selected to this group will use the journal specified at the product level. See Setting System

Values on page 386 for more information.The default value for this parameter is *CLUSTER.


• Default BSF Journal Library—Specify the library where the journal resides if you do not specify *NONE or *CLUSTER in the Default BSF Journal box above.

10 Click Execute.On the iCluster Administrator window, the new replication group is added under the Full Replication heading in the tree view.

Start Full Replication GroupUse this command to start cluster operations for the specified full replication group. This command also sets the cluster and replication statuses of the specified replication group to *ACTIVE.If the group is a replication group, the replication status of the group will become *ACTIVE when replication processes are started.The minimum security level for this command is Operator (*OPERATOR).

To start cluster operations for a full replication group1 From the iCluster Administrator window, expand the Replication Groups heading and the Full Replication heading in the

tree view.2 Right-click a Full Replication group and select Start Replication Group.

The Start Replication Group dialog box opens.

This dialog box identifies the replication group name that you selected on the iCluster Administrator window.


This dialog has the following tabs:• General, see Step 3• Advanced, see Step 4

3 On the General tab, you can specify the following values:• Start Replication Apply Jobs—Select whether the apply processes on the backup node in the replication group will be

started when mirroring begins.Possible values:• *NOCHG - Specifies that the last operational status of the apply jobs on the backup node is not changed.• *YES - Indicates that the backup node apply jobs for the replication group will be started. This does not apply to

suspended files.• *NO - Specifies that the backup node apply jobs for the replication group will not be started.

The default value for this parameter is *NOCHG.4 Select the Advanced tab for more advanced operations.

• Refresh—Select if you want an initial refresh of selected objects in the replication group to be performed before you start mirroring.

Note: If you check the Use Marked Position box (see below), this setting is ignored.By default, this box is not checked.

Warning: If you unexpectedly began a refresh (by selecting the *YES option in the Refresh Selected Objects box) and then ended the refresh before completion (due to system constraints), you should contact DataMirror Technical Support for assistance. See Contacting Technical Support for more information. You must perform a number of steps following a discontinued refresh to allow mirroring to re-start properly. If these steps are not performed following a discontinued refresh, mirroring may not work as you intended.

If you intentionally began a refresh that ended unexpectedly (for example, power failure), you should re-start the refresh.• Use Marked Position—Check if you want mirroring to start at the marked journal position for the specified replication group.

You can mark these journal positions using the Mark Journal Positions command. They are maintained in the iCluster metadata.If you do not check this box, then iCluster replication will not start at positions marked using the Mark Journal Positions command. iCluster replication will start at either: • Journal positions set using the Set Journal Positions command.


Group Commands

• Journal positions registered using the DMREGPOS—Register Positions command.• The journal entry following the last journal position received on the backup system when replication was previously

stopped.If you check this box, then mirroring will start at the journal positions that have been marked for the specified replication group through the Mark Journal Positions command. The primary and backup nodes must have been synchronized at the time the Mark Journal Positions command was issued. Checking this box on this parameter will overwrite any starting journal positions previously recorded by the Set Journal Positions or Mark Journal Positions commands.By default, this box is not checked.

5 Click Execute.

End Full Replication GroupUse this command to end cluster operations for the specified full replication group. This command also sets the cluster and replication statuses of the specified replication group to *INACTIVE.The minimum security level for this command is Operator (*OPERATOR).

To end cluster operations for a full replication group1 From the iCluster Administrator window, expand the Replication Groups heading and the Full Replication heading in the

tree view.2 Right-click a Full Replication group and select End Replication Group.

The End Replication Group dialog box opens.

This dialog box identifies the replication group name that you selected on the iCluster Administrator window.You can specify the following values:

• Option—Select how replication within the replication group is ended.You can end replication immediately or in a controlled manner. An immediate end concludes replication within the group at the point when you issue the command. As a result, iCluster may not be able to guarantee that replication is ended in the desired manner. On the other hand, a controlled end allows iCluster to perform the necessary tasks to ensure that replication is gracefully ended. However, the completion of these tasks may take some time. A controlled end is the recommended selection.Possible values:• *CNTRLD—Ends replication within the replication group in a controlled manner.• *IMMED—Ends replication within the replication group immediately.

The default value for this parameter is *CNTRLD.


• Delay Time—Specify the maximum amount of time, in seconds, for replication within the group to end in a controlled manner without intervention.You can specify a timeout period for ending replication. If, for some reason, it cannot be completed within the timeout period, replication will be immediately ended after the timeout period expires. This box is only applicable (enabled) when the Option box (see above) is set to *CNTRLD.The default value for this parameter is 3600 seconds.

3 Click Execute to end the replication group.

Remove Full Replication GroupUse this command to remove an existing full replication group.If the replication group is active, group operations are automatically stopped before the replication group is removed.The minimum security level for this command is Administrator (*ADMIN).

To remove a full replication group1 From the iCluster Administrator window, expand the Replication Groups heading and the Full Replication heading in the

tree view.2 Right-click a Full Replication group and select Remove Replication Group.

The Remove Replication Group dialog box opens.

This dialog box identifies the replication group name that you selected on the iCluster Administrator window.3 Click Execute.On the iCluster Administrator window, the replication group that you selected is removed from the tree view.

Change Full Replication GroupUse this command to change one or more attributes of an existing full replication group.The selected replication group must be inactive when you issue this command.The minimum security level for this command is Administrator (*ADMIN).

To change a full replication group1 From the iCluster Administrator window, expand the Replication Groups heading and the Full Replication heading in the

tree view.2 Right-click a Full Replication group and select Change Replication Group.

The Change Replication Group dialog box opens. This dialog has the following tabs:• Properties, see Step 3.


Group Commands

• User Exit, see Step 4.• Objects, see Step 5• Spool File, see Step 6.• Physical File, see Step 7.• BSF, see Step 8.

3 On the Properties tab, you can specify the following values:• Failover Message Queue Name—Specify the name of the message queue that will receive messages generated when a


The default value for this parameter is *NONE. • Failover Message Queue Library—Specify the name of the library where the specified message queue currently resides.


becomes the primary node in the replication group when a failover occurs.


This parameter only applies to replication groups whose target library is *PRIMARY, and to MQSeries groups.If set to *YES, the functions of the primary node will be moved to the backup node when a failover occurs.Possible values:• *NO—Does not automatically change the role of the backup node in the replication group to the primary node when a

failover occurs. *NO gives you the opportunity to decide if you would like to continue with a role switch, or continue with your current configuration.

With this setting, a failure of the group's primary node is declared by the failover mechanism, although iCluster will not prepare the backup node to take over as the primary node. Replication will not start up automatically once the failed primary node is again active in the cluster.When using IBM Cluster Resource Services, only groups whose cluster status is *ACTIVE are switched over if the primary node fails. For more information on using IBM Cluster Services, see Switchover for IBM Cluster Resource Services (CRS) on page 341.When IBM Cluster Resource Services is not being used for iCluster operations, all groups that have the ROLESWITCH parameter set to *YES are switched over when the primary node fails, regardless of whether they are active or not.• *YES—Automatically changes the role of the backup node in the replication group to the primary node when a failover

occurs. In this case, user exit programs specified through the User Exit Before Role Switch and User Exit After Role Switch boxes (see below) will be called.

The default value for this parameter is *NO.Note: If you do not want an automatic role switch to be performed in the specified replication group (this parameter

set to *NO), use the Set Primary Node or the Change Role command to prepare the new primary node, that is, the previous backup node, for replication after a failover occurs. If you set this parameter to *YES, this step is not necessary.






page 84.• Check Journaling At Role Switch—Specify whether iCluster will check if all mirrored objects that should be journaled are

being journaled on the new primary node.


Group Commands

Specifies whether (when switching to the new primary node) iCluster will check whether the objects for the specified group are being properly journaled on that node, and start journaling them. Certain types of objects must be journaled before replication starts. However, because journaling objects can be time-consuming, this option allows you to specify whether or not journaling should be started for objects during a switchover, depending on your business requirements.Depending on the value of the group's Check Journaling At Role Switch parameter, iCluster may perform processing that is equivalent to the Mark Journal Positions command when a roleswitch occurs.If the value of the group's Check Journaling At Role Switch parameter is *NO, iCluster will not perform Mark Journal Positions processing and the replication metadata for the new primary node is generated based upon the existing replication metadata on the current backup node. In the case of an error in processing the backup replication metadata, or non-existent backup replication metadata, iCluster performs Mark Journal Positions processing regardless of the Check Journaling At Role Switch parameter's value.



primary node. If they are not journalled, they will be.• *NO—Specifies that iCluster will not check if all mirrored objects that should be journaled are journaled on the new





combination.• Description—Enter a short description that allows you to identify this replication group among all others that have been

defined in iCluster.The description can be a maximum of 50 characters long, and is an optional setting.


before the role change is performed.The user exit program will be called on both nodes of the group for switchover, but only on the new primary node for a failover.This parameter only applies to replication groups whose target library is *PRIMARY, and to MQSeries groups.If you specify a user exit program, the Do Role Switch at Failover box (see above) must be set to *YES if you want to call the program when a failover occurs.











• Exit Data—Specify the user-defined data that you want to pass to the user exit programs specified on this tab (see above).This parameter only applies to replication groups whose target library is *PRIMARY, and to MQSeries groups.If you specify two different user exit programs (one program before the role switch, and a different program after the role switch), the same user-defined data is passed to both programs. A maximum of 256 bytes of data can be passed to the user exit programs. If your data contains spaces or non alpha-numeric characters (commas, periods, and so on), you must enclose your data in single quotes.

5 On the Objects tab, you can specify the following values:• Group Polling Interval—Specify the time interval that determines how often iCluster should check for content changes to

user spaces.The polling interval is only applicable when you select user spaces to the replication group through the Select/Add Object Specifier command.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more

information.• *NONE—Specifies that user spaces should not be polled.


Group Commands

• Specify Values—This option allows you to set the time interval that determines how often iCluster should check for content changes to user spaces. You set the interval by specifying the number of hours (Hours box), minutes (Minutes box), and seconds (Seconds box) between consecutive polls.

The default value for this parameter is *CLUSTER.• Save Active Objects—Select whether an object can be updated and saved at the same time it is being transferred to the





6 On the Spool File tab, you can specify the following values:• Maximum Wait for Spool Files—Specify the time period (in hours, minutes, and seconds) you want iCluster to wait before

abandoning the replication of a spool file.You can specify a waiting period from 000001 to 235959.You can replicate a spool file only when it has a status of *READY. Therefore, if iCluster is up-to-date with the audit journal, it may start processing a spool file before it is ready. iCluster will only wait the specified amount of time for the spool file to have a status of *READY. If the spool file is not ready after waiting the specified time, iCluster will abandon the replication of this spool file and issue a message in the event log.Special value:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more


• Maximum Wait for Spool Activities—Specify the time period (in hours, minutes, and seconds) you want iCluster to wait before abandoning the replication of a spool file if no data is being written to it.In some cases, a job may be writing to a spool file. However, the data is first placed in a temporary buffer and only written to the spool file upon the completion of the job. In such a case, the spool file will be open and have zero pages until the job ends.This box allows you to specify the time to wait for changes to occur to an open spool file. If no additional data is written to an open spool file (that is, it is open and has zero pages) within the specified time, iCluster will abandon the replication of this spool file and issue a message. This option provides added flexibility as you can specify a shorter interval over which only spool file activity is monitored. As a result, an open spool file that has no data being written to it will be abandoned quickly, as opposed to waiting the full time specified under Maximum Wait for Spool Files (see above).Special value:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more


7 On the Physical File tab, you can specify the following values:• Default Database Journal—Specify the name of the database journal that you want to use as your default.


The journal that you specify will be used for files that are to be mirrored but are not yet journaled. iCluster will start journaling automatically to this journal.Special value:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more




• Commitment Control Level—Select the level of commitment control you want to use when replicating *FILE objects.Commitment control stages database transactions so that they are assembled before being applied, or additionally, opened in a commitment control environment to ensure that all updates are received. It also makes sure that the changes are applied in the correct sequence. For more information, see Commitment Control.If you select *LEVEL2 but a file on the backup node cannot be opened under commitment control, *LEVEL1 commitment control will be used for that file.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more

information.• *NONE—Indicates that the update process on the backup node will not perform commitment control staging.• *LEVEL1—Indicates that all updates that comprise a transaction are assembled before being applied on the backup



• Physical File Journal Images—Select whether default journaling should include both before and after images.Before images are necessary to remove applied or keyed replication updates. Also, if you specified unique index refresh method for a file, then it must be journaled with both before and after images. If a file is replicated using relative record number, it may be journaled with after images only.For unique key updates, both before and after images must be journaled if Commitment Control Level is set to *LEVEL2.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more

information.• *BOTH—Indicates that you want to journal both the before and after images.• *AFTER—Indicates that you want to journal the after image only.

The default value for this parameter is *CLUSTER.8 On the BSF tab, you can specify values in the following fields:• Default BSF Journal—Specify the name of the journal that you want to use as your default for journaling BSF files that will be

replicated by this group.Note: This parameter only applies to replication group.


Group Commands

Note: This parameter is valid for BSF replication in system auxiliary storage pools or independent auxiliary storage pools.

The journal that you specify will be used for BSF files that are to be mirrored and journaled. See Select/Add Object Specifier for more information.Specify the name of the BSF journal or one of the following values:• *NONE—Indicates that you do not require journaling for BSF replication.• *CLUSTER—BSF objects selected to this group will use the journal specified at the product level. See Setting System

Values for more information.The default value for this parameter is *CLUSTER.


• Default BSF Journal Library—Specify the library where the journal resides if you do not specify *NONE or *CLUSTER in the Default BSF Journal box above.

9 Click Execute.On the iCluster Administrator window, the new replication group is added under the Full Replication heading in the tree view.

Convert to Refresh Only GroupUse this command to convert a full replication group to a refresh-only group. Refresh-only groups perform a refresh of the objects selected to the group and then shut down. These groups are not eligible for role switch.When converting full replication groups to refresh-only groups, iCluster fills the Convert to Refresh-Only Group dialog box with full replication group values that also apply to refresh-only groups. Values that do not apply to refresh-only groups are discarded after the conversion.The minimum security level for this command is Administrator (*ADMIN).

To convert a full replication group to a refresh-only group1 From the iCluster Administrator window, expand the Replication Groups heading and the Full Replication heading in the

tree view.2 Right-click a Full Replication group and select Convert to Refresh-Only Group.

The Convert to Refresh-Only Group dialog box opens with values from the full replication group that you want to convert.


The Replication Group Name value identifies the name of the full replication group that you selected in the iCluster Administrator window.This dialog has the following tabs:• Properties, see Step 3.• Objects, see Step 4

3 On the Properties tab, you can specify the following values:• Target Library—Specify the name of the library on the backup system that will receive the replicated objects.

Special value:• *PRIMARY—Sets the target library so that it is the same as the primary node library where the object resides.


• Description—Enter a short description that allows you to identify this refresh-only group among all others that have been defined in iCluster.The description can be a maximum of 50 characters long, and is an optional setting.

4 Select the Objects tab.


Group Commands

You can specify the following values on this tab:• Save Active Objects—Select whether an object can be updated and saved at the same time it is being transferred to the



at different times and may not be in a consistent state in relation to each other.• *NO - Does not allow objects that are in use to be saved and updated. Objects cannot be updated while being saved.• *SYNC - Allows objects that are in use by another job to be saved and updated. All of the objects reach a


5 Click Execute.On the iCluster Administrator window, the new settings for the refresh-only group are shown on the table view.

Switchover Full Replication GroupUse this command to initiate a switchover, involving the primary and backup nodes, within the specified full replication group.When this command is used, the current primary node becomes the backup node, and the current backup node assumes the role of the primary node. In addition, the role switch user exit programs that may have been specified for the replication group will be invoked on both nodes. See Add Full Replication Group and Change Full Replication Group for more information.Note: To initiate a replication group switchover, the group's status must be *ACTIVE. See Working with Replication

Groups for more information.The minimum security level for this command is Administrator (*ADMIN).


To initiate a switchover for a full replication group1 From the iCluster Administrator window, expand the Replication Groups heading and the Full Replication heading in the

tree view.2 Right-click a Full Replication group and select Switchover Replication Group.

The Switchover Replication Group dialog box opens.

The Replication Group Name value identifies the replication group selected on the iCluster Administrator window.You can specify the following values on this dialog:

• Re-start Replication—Indicate whether or not to re-start replication after a switchover has completed.Possible values:• *YES—Indicates that replication processes for the group will be re-started from the new primary node to the new backup

node when switchover processing is complete.• *NO—Indicates that replication processes for the group will not be re-started from the new primary node to the new

backup node when switchover processing is complete. If this value is specified, you can use the Start Replication Group command to re-start replication at a later time.

The default value for this parameter is *YES.3 Click Execute.

Add Refresh-Only GroupRefresh-only groups perform a refresh of the objects selected to the group and then shut down. These groups are not eligible for role switch.Use this command to add a refresh-only group consisting of one primary node and possibly one backup node. All nodes in the refresh only group's recovery domain must be active.The minimum security level for this command is Administrator (*ADMIN).

To add a refresh-only group1 From the tree view in the iCluster Administrator window, expand the Replication Groups heading.2 Right-click the Refresh-Only heading and select Add Refresh-Only Group.

The Add Refresh-Only Group dialog box opens.


Group Commands

This dialog has the following tabs:• Properties, see Step 3• Recovery Domain, see Step 4.• Objects, see Step 5.

3 On the Properties tab, you can specify the following values:• Replication Group Name—Specify the name of the refresh-only group that you want to define in iCluster.

The specified group name must be unique in the cluster.• Target Library—Specify the name of the library on the backup system that will receive the replicated objects.




4 On the Recovery Domain tab, you can specify the following values:


• Select recovery domain from existing group—Allows you to add a refresh-only group with the same set of primary and backup nodes as the recovery domain for an existing group. Defining two or more groups with the same set of nodes is useful when you want to replicate objects between the same nodes, but you would like to define different group behaviors for different sets of objects. In this case, select the name of an existing group that you want to use to define the recovery domain of the refresh-only group you are adding.

• Specify the recovery domain—Indicates that you want to identify the nodes in the recovery domain. In this case, perform the following four steps.The default option is Specify the recovery domain.

• Primary Node—Select the primary node for the refresh-only group you are adding.The node that you specify must have been added to the cluster. See Add Node for more information. This box is only applicable if the Specify the recovery domain option is selected.



• Local loopback—Select this option if you want the primary and backup environments to reside on the same physical system (local loopback implementation).This option is not selected by default.

• Remote Backup—Select this option if you want the primary and backup environments to reside on different physical systems.This option is selected by default.

• Backup Node—Select the backup node for the refresh-only group you are adding.At this time, you can define only one backup node in a refresh-only group.The node that you select must have been added to the cluster. See Add Node for more information. You can select a backup node only when the Specify the recovery domain option is selected.

Note: Both primary and backup nodes in a refresh-only group must reside in the same subnet.• Backup IASP Device Name—Specify the name of an existing independent auxiliary storage pool (IASP) on the backup node

to which you want to replicate.Special value:• *SYSBAS—Indicates that you are using a system ASP rather than an independent ASP for storing replicated objects.

The default value for this parameter is *SYSBAS.5 On the Objects tab, you can specify the following values:• Save Active Objects—Select whether an object can be updated and saved at the same time it is being transferred to the


information.• *YES—Indicates that objects in use by another job can be saved and updated. Objects may reach synchronization

points at different times and may not be in a consistent state in relation to each other.


Group Commands

• *NO—Indicates that objects that are in use cannot be saved and updated. Objects cannot be updated while being saved.• *SYNC—Indicates that objects that are in use by another job to be saved and updated. All of the objects reach a


6 Click Execute.On the iCluster Administrator window, the new refresh-only group is added under the Refresh-Only heading in the tree view.

Start Refresh-Only GroupUse this command to start cluster operations for the specified refresh-only group. This command also sets the cluster and replication statuses of the specified refresh-only group to *ACTIVE.If the group is a refresh-only group, the replication status of the group will become *ACTIVE when replication processes are started.The minimum security level for this command is Operator (*OPERATOR).

To start a refresh-only group1 From the iCluster Administrator window, expand the Replication Groups heading and the Refresh-Only heading in the tree

view.2 Right-click a Refresh-Only group and select Start Refresh-Only Group.

The Start Refresh-Only Group dialog box opens.

You can specify the following values:• Start Replication Apply Jobs—Select whether the apply processes on the backup node in the refresh-only group will be

started when mirroring begins.Possible values:• *NOCHG—Indicates that the last operational status of the apply jobs on the backup node is not changed.• *YES—Indicates that the apply jobs are started on the backup node for the refresh-only group. This does not apply to

suspended files.• *NO—Indicates that the apply jobs have not started on the backup node for the refresh-only group.

The default value for this parameter is *NOCHG.3 Click Execute to start the refresh-only group.

End Refresh-Only GroupUse this command to end cluster operations for the specified refresh-only group. This command also sets the cluster and replication statuses of the specified refresh-only group to *INACTIVE.


The minimum security level for this command is Operator (*OPERATOR).

To end a refresh-only group1 From the iCluster Administrator window, expand the Replication Groups heading and the Refresh-Only heading in the tree

view.2 Right-click a Refresh-Only group and select End Refresh-Only Group.

The End Refresh-Only Group dialog box opens.

You can specify the following values:• Option—Select how you want replication to end within the refresh-only group.

You can end replication immediately or in a controlled manner. An immediate end concludes replication within the group at the point when you issue the command. As a result, iCluster may not be able to guarantee that replication is ended in the desired manner. On the other hand, a controlled end allows iCluster to perform the necessary tasks to ensure that replication ends gracefully. However, the completion of these tasks may take some time. A controlled end is the recommended selection.Possible values:• *CNTRLD—Ends replication in a controlled manner within the refresh-only group.• *IMMED—Ends replication immediately within the refresh-only group.

The default value for this parameter is *CNTRLD.• Delay Time—Specify the maximum amount of time, in seconds, for replication within the group to end in a controlled manner

without intervention.You can specify a timeout period for ending replication. If replication does not end within the timeout period, replication will end immediately after the timeout period expires. This box is only applicable (enabled) when the Option box (see above) is set to *CNTRLD.The default value for this parameter is 3600 seconds.

3 Click Execute to end the refresh-only group.

Remove Refresh-Only GroupUse this command to remove an existing refresh-only group.If the refresh-only group is active, group operations are automatically stopped before the refresh-only group is removed.The minimum security level for this command is Administrator (*ADMIN).

To remove a refresh-only group1 From the iCluster Administrator window, expand the Replication Groups heading and the Refresh-Only heading in the tree

view.


Group Commands

2 Right-click a Refresh-Only group and select Remove Refresh-Only Group.The Remove Refresh-Only Group dialog box opens.

This dialog box identifies the refresh-only group name that you selected on the iCluster Administrator window.3 Click Execute.On the iCluster Administrator window, the refresh-only group that you selected is removed from the tree view.

Change Refresh-Only GroupUse this command to change one or more attributes of an existing refresh-only group.The selected refresh-only group must be inactive when you issue this command.The minimum security level for this command is Administrator (*ADMIN).

To change a refresh-only group1 From the iCluster Administrator window, expand the Replication Groups heading and the Refresh-Only heading in the tree

view.2 Right-click a Refresh-Only group and select Change Refresh-Only Group.

The Change Refresh-Only Group dialog box opens.


This dialog has the following tabs:• Properties, see Step 3• Objects, see Step 4.

3 On the Properties tab, you can specify the following values:• Replication Group Name—Specifies the name of the refresh-only group defined in iCluster.• Target Library—Specify the name of the library on the backup system that will receive the replicated objects.




4 On the Objects tab, you can specify the following values:• Save Active Objects—Select whether an object can be updated and saved at the same time it is being transferred to the

backup node.This setting does not apply to physical files because they cannot be modified while they are being saved.Possible values:


Group Commands

• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more information.

• *YES—Indicates that objects in use by another job can be saved and updated. Objects may reach synchronization points at different times and may not be in a consistent state in relation to each other.

• *NO—Indicates that objects that are in use cannot be saved and updated. Objects cannot be updated while being saved.• *SYNC—Indicates that objects that are in use by another job to be saved and updated. All of the objects reach a


5 Click Execute.

Convert to a Full Replication GroupUse this command to convert a refresh-only group to a full replication group.A full replication group consists of one primary node and possibly one backup node. All nodes in the replication group's recovery domain must be active.When converting refresh-only groups to full replication groups, iCluster fills the Convert to Replication Group dialog box will refresh-only group values that also apply to full replication groups. iCluster uses default values for those values in the Convert to Replication Group dialog box that only apply to full replication groups.The minimum security level for this command is Administrator (*ADMIN).

To convert a refresh-only group to a full replication group1 From the iCluster Administrator window, expand the Replication Groups heading and the Refresh-Only heading in the tree

view.2 Right-click a Refresh-Only group and select Convert to Full Replication Group.

The Convert to Replication Group dialog box opens with values from the refresh-only group that you want to convert.


This dialog box has the following tabs:• Properties, see Step 3.• User Exit, see Step 4• Objects, see Step 5.• Spool File, see Step 6.• Physical File, see Step 7.• BSF, see Step 8.


failover occurs.


Group Commands

Special value:• *NONE—Indicates that you do not want to specify a message queue.

The default value for this parameter is *NONE.• Failover Message Queue Library—Specify the name of the library where the specified message queue currently resides.

If you selected *NONE in the Failover Message Queue Name box, a library name is not required or is ignored if it is specified.• Do Role Switch at Failover—Indicate whether you want the role of the backup node to automatically change so that it

becomes the primary node in the replication group when a failover occurs.This parameter is valid only for replication groups whose target library is *PRIMARY.If set to *YES, the functions of the primary node will be moved to the backup node when a failover occurs.Possible values:• *NO—Does not automatically change the role of the backup node in the replication group to the primary node when a

failover occurs. This lets you decide if you want to continue with a role switch, or continue with your current configuration.By selecting *NO, a failure of the group's primary node is declared by the failover mechanism, although iCluster will not prepare the backup node to take over as the primary node. Replication is not started automatically once the failed primary node is again active in the cluster.

Note: Even though the backup node has not been prepared to take over as the primary node, it will still appear as the primary node for the groups of which it was the backup node prior to the failure. No user exit programs are called with this setting.

• *YES—Automatically changes the role of the backup node in the replication group to the primary node when a failover occurs. In this case, user exit programs specified through the User Exit Before Role Switch and User Exit After Role Switch boxes (see below) are called.

Note: If you do not want an automatic role switch to be performed in the specified replication group (this parameter set to *NO), use the Set Primary Node or the Change Role command to prepare the new primary node, that is, the previous backup node, for replication after a failover occurs. If you set this parameter to *YES, this step is not necessary.

The default value for this parameter is *NO.See the Switchover for IBM Cluster Resource Services (CRS) on page 341.


If the primary and backup environments reside on the same physical system (local loopback implementation), the target library that you specify cannot be the same library where a selected object in the group currently resides. This restriction means that iCluster does not allow the special value *PRIMARY in this situation.

Note: Switchovers and role changes are not supported for groups that use a target library other than *PRIMARY. Also, because a non-primary target library is required for local loopback replication, switchovers and role changes are not supported for local loopback replication.

• Journal Location—Specify the location of the database journal where you want scraping to occur.Possible values:• *LOCAL—Indicates that the database journal(s) on the primary node are scraped during mirroring.• *REMOTE—Indicates that the remote database journal(s) are scraped on the backup node during mirroring.

The default value for this parameter is *LOCAL.For more information on the configuration of remote journaling, see Configuring Remote Journaling on page 84.


• Check Journaling At Role Switch—Specify if you want iCluster to check if all mirrored objects are being journaled on the new primary node.Specifies whether (when switching to the new primary node) iCluster will check whether the objects for the specified group are being properly journaled on that node, and start journaling them. Certain types of objects must be journaled before replication starts. However, because journaling objects can be time-consuming, this option allows you to specify whether or not journaling should be started for objects during a switchover, depending on your business requirements.Depending on the value of the group's Check Journaling At Role Switch parameter, iCluster may perform processing that is equivalent to the Mark Journal Positions command when a roleswitch occurs. If the value of the group's Check Journaling At Role Switch parameter is *NO, iCluster will not perform Mark Journal Positions processing and the replication metadata for the new primary node is generated based upon the existing replication metadata on the current backup node. In the case of an error in processing the backup replication metadata, or non-existent backup replication metadata, iCluster performs Mark Journal Positions processing regardless of the Check Journaling At Role Switch parameter's value.



primary node. If they are not, the objects will be journaled.• *NO—Specifies that iCluster will not check if all mirrored objects that should be journaled are journaled on the new



lower number decreases the performance of the apply, but provides a greater safeguard in recovery situations. Specifying a higher number lessens the impact of recovery checkpoints on the apply.Enter a recovery exposure value (number of journal entries) or the following value:• *DISABLED—Indicates that you do not want to generate recovery checkpoints.


combination.• Description box—Specify a short description that allows you to identify this replication group among all others defined in

iCluster.The description can be a maximum of 50 characters long, and is an optional setting.


before the role change is performed.The user exit program is called on both nodes of the group for switchover, but only on the new primary node in the event of a failover.This parameter is valid only for replication groups whose target library is *PRIMARY.


Group Commands

If you specify a user exit program, the Do Role Switch at Failover box (see above) must be set to *YES if you want to call a user exit in the event of a failover.



See the DMSETPRIM—Prepare Primary Node command for more information about the arguments that can be passed to the user exit program.

• Library (under User Exit Before Role Switch), specify the name of the library where the before role switch user exit program currently resides.If you selected *NONE in the Name box, a library name is not required or is ignored if it is specified.




See the DMSETPRIM—Prepare Primary Node command for more information about the arguments that can be passed to the user exit program.


• Exit Data—Specify the user-defined data that you want to pass to the user exit programs specified on this tab (see above).This parameter is valid only for replication groups whose target library is *PRIMARY.If you specify two different user exit programs (one program before the role switch, and a different program after the role switch), the same user-defined data is passed to both programs.

Note: A maximum of 256 bytes of data can be passed to the user exit programs. If your data contains spaces or non-alphanumeric characters (commas, periods, and so on), you must enclose your data in single quotes.

5 On the Objects tab, you can specify the following values:• Group Polling Interval—Specify the time interval during which iCluster will check for content changes to user spaces.

The polling interval is only applicable when you select user spaces to the replication group through the Select/Add Object Specifier command.Special value:• *CLUSTER—Uses the value specified for the corresponding system parameter (Default polling interval). See Setting

System Values for more information.• *NONE—Specifies that user spaces should not be polled.


• Specify Values—Allows you to specify the time interval that determines how often iCluster should check for content changes to user spaces. You set the interval by specifying the number of hours (Hours box), minutes (Minutes box), and the seconds (Seconds box) between consecutive polls.

• Save Active Objects—Select if you want an object to be updated and saved at the same time it is being transferred to the backup node.This setting does not apply to physical files because they cannot be modified while they are being saved.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more




6 On the Spool File tab, you can specify the following values:• Maximum Wait for Spool Files—Specify the amount of time (in hours, minutes, and seconds) you want iCluster to wait

before abandoning the replication of a spool file and issuing a warning message in the event log.You can specify a waiting period from 000001 to 235959.You can replicate a spool file only if it has a status of *READY. If iCluster is up-to-date with the audit journal, it can start processing a spool file before it is ready.Special value:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more

information.• Maximum Wait for Spool Activities—Specify the amount of time (in hours, minutes, and seconds) you want iCluster to wait

before abandoning the replication of a spool file if no data is being written to it.In some cases, a job may be writing to a spool file. However, the data is first placed in a temporary buffer and only written to the spool file upon the completion of the job. In such a case, the spool file is open and has zero pages until the job ends.You can specify the time to wait for changes to occur to an open spool file. If no additional data is written to an open spool file (for instance, it is open and has zero pages) within the specified time, iCluster will abandon the replication of this spool file and issue a message. This option provides added flexibility so you can specify a shorter interval in which only spool file activity is monitored. As a result, an open spool file that has no data being written to it is abandoned quickly, as opposed to waiting the full time specified under Maximum Wait for Spool Files (see above).Special value:• *CLUSTER - Uses the value specified for the corresponding system parameter (Max. wait for spool activity). See Setting

System Values for more information.7 On the Physical File tab, you can specify the following values:• Default Database Journal—Specify the name of the database journal that you want to use as your default journal.

The journal you specify is used for files that are to be mirrored but are not yet journaled. iCluster automatically journals to this journal.Special value:• *CLUSTER—Uses the value specified for the corresponding system parameter (Default database journal). See Setting

System Values for more information.


Group Commands


• Default Database Journal Library—Specify the name of the library where the database journal currently resides.If you specified *CLUSTER in the Default Database Journal box, a library name is not required, as iCluster uses the corresponding cluster system value (Default database journal library).

• Commitment Control Level—Select the level of commitment control you want to use when replicating *FILE objects.Commitment control stages database transactions so that they are assembled before being applied, or additionally, opened in a commitment control environment to ensure that all updates are received. It also makes sure that changes are applied in the correct sequence. For more information, see Commitment Control on page 71.If you select *LEVEL2 but a file on the backup node cannot be opened under commitment control, *LEVEL1 commitment control is used for that file.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter (Commitment control level). See Setting

System Values for more information.• *NONE—Indicates that the update process on the backup node is not performing commitment control staging.• *LEVEL1—Indicates that all updates that comprise a transaction are assembled before being applied on the backup


updates are received. This option provides true commitment control.• Physical File Journal Images—Select whether default journaling should include both before and after images.

Before images are necessary to remove applied or keyed replication updates. Also, if you specified unique index refresh method for a file, then it must be journaled with both before and after images. If a file is replicated using relative record number, it may be journaled with after images only.For unique key updates, both before and after images must be journaled if Commitment Control Level is set to *LEVEL2.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter (Journal images). See Setting System

Values for more information.• *BOTH—Indicates that you want to journal both the before and after images. • *AFTER—Indicates that you want to journal the after image only.

8 On the BSF tab, you can specify the following values:• Default BSF Journal—Specify the name of the journal that you want to use as your default for journaling BSF files that are

replicated by this group.Note: This parameter only applies to replication groups.Note: This parameter is valid for BSF replication in system auxiliary storage pools or independent auxiliary storage

pools.The journal that you specify is used for BSF files that are to be mirrored and journaled. See Select/Add Object Specifier on page 486 for more information.Specify the name of the BSF journal or one of the following values:• *NONE—Indicates that you do not require journaling for BSF replication.• *CLUSTER—BSF objects selected to this group will use the journal specified at the product level. See Setting System

Values for more information.The default value for this parameter is *CLUSTER.



• Default BSF Journal Library—If you do not specify *NONE or *CLUSTER in the Default BSF Journal box above, specify the library where the journal resides.

9 Click Execute.On the iCluster Administrator window, the new settings for the replication group you have modified are shown on the table view.

Add Master-Master GroupUse this command to add a master-master replication group. The minimum security level for this command is Administrator (*ADMIN).Note: This command only applies to iBalance and Master-Master replication groups. See iBalance and Master-

Master Replication on page 321 for more information on installing this feature.

To add a master-master group1 From the tree view in the iCluster Administrator window, expand the Replication Groups heading and the Master-Master

heading.2 Right-click the Master-Master heading and select Add Master-Master Group.

The Add Master-Master Group dialog box opens.

This dialog has the following tabs:


Group Commands

• Properties, see Step 3.• Recovery Domain, see Step 4.• Physical File, see Step 5.• Conflict Resolution, see Step 6.

3 On the Properties tab, you can specify the following values:• Group Name—Specify the name of the group that you want to define in iCluster.

The specified group name must be unique in the cluster.• Target Library—Specify the name of the library on the backup system that will receive the replicated objects.






page 84.• Recovery Exposure—Specify the number of journal entries applied between writing recovery checkpoints. Specifying a





4 On the Recovery Domain tab, you can specify the following values:• Select recovery domain from existing group—Allows you to add a replication group with the same set of primary and

backup nodes as the recovery domain for an existing group. Defining two or more replication groups with the same set of nodes is useful when you want to replicate objects between the same nodes, but you would like to define different replication group behaviors for different sets of objects. In this case, select the name of an existing group that you want to use to define the recovery domain of the replication group you are adding.


• Specify the recovery domain—Indicates that you want to identify the nodes in the recovery domain. In this case, complete the following four fields.This is the default option.

• Primary Node—Select the name of a node that will be the primary node in the replication group you are adding.The node that you specify must have been added to the cluster. See Add Node for more information. This box is only applicable if the Specify the recovery domain option is selected.



• Local Loopback—Select if you would like the primary and backup environments to reside on the same physical system (local loopback implementation).This box is unchecked by default.

• Backup Node—Select the node that will be the backup node in the replication group you are adding.At this time, you can define only one backup node in a replication group.The node that you select must have been added to the cluster. See Add Node for more information. You can select a backup node only when the Specify the recovery domain option is selected.

Note: Both primary and backup nodes in a replication group must reside in the same subnet.• Backup IASP Device Name—Specify the name of an existing independent auxiliary storage pool (IASP) on the backup node

to which you want to replicate.Special value:• *SYSBAS—Indicates that you are using a system ASP rather than an independent ASP for storing the objects that will







• Commitment Control Level—Select the level of commitment control you want to use when replicating *FILE objects.


Group Commands

Commitment control stages database transactions so that they are assembled before being applied, or additionally, opened in a commitment control environment to ensure that all updates are received. It also makes sure that the changes are applied in the correct sequence. For more information, see Commitment Control on page 71.If you select *LEVEL2 but a file on the backup node cannot be opened under commitment control, *LEVEL1 commitment control will be used for that file.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more




• Physical File Journal Images—Select whether default journaling should include both before and after images.Before images are necessary to remove applied or keyed replication updates. Also, if you specified unique index refresh method for a file, then it must be journaled with both before and after images. If a file is replicated using relative record number, it may be journaled with after images only.For unique key updates, both before and after images must be journaled if Commitment Control Level is set to *LEVEL2. For more information, see Commitment Control on page 71.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more


The default value for this parameter is *CLUSTER.6 On the Conflict Resolution tab, you can specify the following values:• Data Conflict Winner—Select the group-level rules by which iCluster will resolve conflicts for the objects selected to a

master-master replication group.Possible values:• *SRC—Indicates that the source wins if there is a conflict. Source data is applied to the target, which ensures that target

data matches source data and mirroring continues.• *TGT—Indicates that the target wins if there is a conflict. iCluster does not apply data to the target. This rule preserves

data on the target and mirroring continues.• *EXIT—Indicates that you would like to resolve data conflicts through a user exit program. With this option, you must

specify a user exit program and library in the Conflict User Exit parameter below.• *NONE—Indicates that you do not want to enable conflict detection and resolution rules. If there is a conflict, iCluster

suspends the file from replication and sends a message to the event log. This is the default behavior for iCluster.The default value for this parameter is *NONE.

• Conflict User Exit—Select or type the name of the user exit program that you want to call to resolve data conflicts. A user exit program allows you to define a conflict resolution strategy that is not covered by one of the standard resolution methods in the Data Conflict Winner box.Specify the name of a user exit program or the following value:


• *NONE—Indicates that you do not want to specify a user exit program.See Parameter List Descriptions for a Conflict Resolution User Exit Program on page 326 for more information about the values you can pass to the user exit program.The default setting for this parameter is *NONE. You cannot specify *NONE for this parameter if you specify *EXIT for the Data Conflict Winner parameter.

• Conflict User Exit Library—Specify the library where the user exit program resides must be identified if you do not specify *NONE in the Conflict User Exit box above.

7 Click Execute.

Start Master-Master GroupUse this command to start cluster operations for the specified master-master group. This command also sets the cluster and replication statuses of the specified master-master group to *ACTIVE.Note: This command only applies to iBalance and Master-Master replication groups. See iBalance and Master-

Master Replication on page 321 for more information on installing this feature.The minimum security level for this command is Operator (*OPERATOR).

To start a master-master group1 From the iCluster Administrator window, expand the Replication Groups heading and the Master-Master heading in the tree

view.2 Right-click a Master-Master group and select Start Master-Master Group.

The Start Replication Group dialog box opens.

This dialog has the following tabs:• General, see Step 3.• Advanced, see Step 4.

3 On the General tab, you can specify the following values:• Start Replication Apply Jobs—Select whether the apply processes on the backup node in the replication group will be

started when mirroring begins.


Group Commands

Possible values:• *NOCHG—Specifies that the last operational status of the apply jobs on the backup node is not changed.• *YES—Indicates that the backup node apply jobs for the replication group will be started. This does not apply to

suspended files.• *NO—Specifies that the backup node apply jobs for the replication group will not be started.

The default value for this parameter is *NOCHG.4 On the Advanced tab, you can specify the following values:• Refresh—Select if you want an initial refresh of selected objects in the replication group to be performed before you start

mirroring.Note: If you check the Use Marked Position box (see below), this setting is ignored.

By default, this box is not checked.Warning: If you unexpectedly began a refresh (by selecting the *YES option in the Refresh Selected Objects box) and

then ended the refresh before completion (due to system constraints), you should contact DataMirror Technical Support for assistance. See Contacting Technical Support on page 547 for more information. You must perform a number of steps following a discontinued refresh to allow mirroring to re-start properly. If these steps are not performed following a discontinued refresh, mirroring may not work as you intended.

If you intentionally began a refresh that ended unexpectedly (for example, power failure), you should re-start the refresh.• Truncate Data—Specify whether you want to remove existing data before performing a refresh.

Specify one of the following values:• *YES—Indicates that you want to remove and recreate existing data files as part of a refresh. This is the default behavior

for iCluster and is appropriate for standard iCluster replication. You can use this option for a master-master group if the data on the source system is known to be complete in order to ensure a synchronized starting point.With this value, logical files are replaced automatically because the deletion of a physical file requires the deletion of all logical files upon which it is based.



• Replace Logical Files—Specify whether you want to replace logical files during a refresh.Specify one of the following values:• *YES—Indicates that you want to replace logical files during a refresh. This is the default behavior for iCluster and is

appropriate for standard replication. You can use this option for a master-master group if the logical files on the source system are the complete set required for your applications in order to ensure a synchronized starting point.




• Use Marked Position—Select if you want mirroring to start at the marked journal position for the specified replication group.


You can mark these journal positions using the Mark Journal Positions command. They are maintained in the iCluster metadata.If you do not check this box, then iCluster replication will not start at positions marked using the Mark Journal Positions command. iCluster replication will start at either: • Journal positions set using the Set Journal Positions command. • Journal positions registered using the DMREGPOS—Register Positions command.• The journal entry following the last journal position received on the backup system when replication was previously

stopped.If you check this box, then mirroring will start at the journal positions that have been marked for the specified replication group through the Mark Journal Positions command. The primary and backup nodes must have been synchronized at the time the Mark Journal Positions command was issued. Checking this box on this parameter will overwrite any starting journal positions previously recorded by the Set Journal Positions or Mark Journal Positions commands.By default, this box is not checked.

5 Click Execute.

End Master-Master GroupUse this command to end cluster operations for the specified master-master group. This command also sets the cluster and replication statuses of the specified master-master group to *INACTIVE.Note: This command only applies to iBalance and Master-Master replication groups. For more information on this

feature, see iBalance and Master-Master Replication on page 321.The minimum security level for this command is Operator (*OPERATOR).

To end a master-master group1 From the iCluster Administrator window, expand the Replication Groups heading and the Master-Master heading in the tree

view.2 Right-click a Master-Master group and select End Master-Master Group.

The End Master-Master Group dialog box opens.

You can specify the following values on this dialog:• Option—Select how replication within the replication group is ended.

You can end replication immediately or in a controlled manner. An immediate end concludes replication within the group at the point when you issue the command. As a result, iCluster may not be able to guarantee that replication is ended in the desired manner. On the other hand, a controlled end allows iCluster to perform the necessary tasks to ensure that replication is gracefully ended. However, the completion of these tasks may take some time. A controlled end is the recommended selection.


Group Commands

Possible values:• *CNTRLD—Ends replication within the replication group in a controlled manner.• *IMMED—Ends replication within the replication group immediately.


without intervention.You can specify a timeout period for ending replication. If, for some reason, it cannot be completed within the timeout period, replication will be immediately ended after the timeout period expires. This box is only applicable (enabled) when the Option box (see above) is set to *CNTRLD.The default value for this parameter is 3600 seconds.

3 Click Execute to end the master-master group.

Remove Master-Master GroupUse this command to remove an existing master-master group.Note: This command only applies to iBalance and Master-Master replication groups. For more information on this

feature, see iBalance and Master-Master Replication on page 321.If the master-master group is active, group operations are automatically stopped before the master-master group is removed.The minimum security level for this command is Administrator (*ADMIN).

To remove a master-master group1 From the iCluster Administrator window, expand the Replication Groups heading and the Master-Master heading in the tree

view.2 Right-click a Master-Master group and select Remove Master-Master Group.

The Remove Master-Master Group dialog box opens.

This dialog box identifies the replication group name that you selected on the iCluster Administrator window.3 Click Execute.On the iCluster Administrator window, the replication group that you selected is removed from the tree view.

Change Master-Master GroupUse this command to change one or more attributes of an existing master-master group. The selected master-master group must be inactive when you issue this command.The minimum security level for this command is Administrator (*ADMIN).Note: This command only applies to iBalance and Master-Master replication groups. For more information on this

feature, see iBalance and Master-Master Replication on page 321.


To change a master-master group1 From the iCluster Administrator window, expand the Replication Groups heading and the Master-Master heading in the tree

view.2 Right-click a Master-Master group and select Change Master-Master Group.

The Change Master-Master Group dialog box opens.

This dialog has the following tabs:• Properties, see Step 3.• Physical File, see Step 4.• Conflict Resolution, see Step 5.

3 On the Properties tab, you can specify the following values:• Target Library—Specify the name of the library on the backup system that will receive the replicated objects.




• Journal Location—Specify the location of the database journal where scraping will occur.


Group Commands

Special values:• *LOCAL—Indicates that the database journal(s) on the primary node are scraped during mirroring.• *REMOTE—Indicates that the remote database journal(s) are scraped on the backup node during mirroring.


page 84.• Recovery Exposure—Specify the number of journal entries applied between writing recovery checkpoints. Specifying a










• Commitment Control Level—Select the level of commitment control you want to use when replicating *FILE objects.Commitment control stages database transactions so that they are assembled before being applied, or additionally, opened in a commitment control environment to ensure that all updates are received. It also makes sure that the changes are applied in the correct sequence. For more information, see Commitment Control on page 71.If you select *LEVEL2 but a file on the backup node cannot be opened under commitment control, *LEVEL1 commitment control will be used for that file.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more


node.


• *LEVEL2—Indicates that all updates in a transaction are opened in a commitment control environment to ensure that all updates are received. This option provides true commitment control.

The default value for this parameter is *CLUSTER.• Physical File Journal Images—Select whether default journaling should include both before and after images.

Before images are necessary to remove applied or keyed replication updates. Also, if you specified unique index refresh method for a file, then it must be journaled with both before and after images. If a file is replicated using relative record number, it may be journaled with after images only.For unique key updates, both before and after images must be journaled if Commitment Control Level is set to *LEVEL2. For more information, see Commitment Control on page 71.Possible values:• *CLUSTER—Uses the value specified for the corresponding system parameter. See Setting System Values for more


The default value for this parameter is *CLUSTER.5 On the Conflict Resolution tab, you can specify the following values:• Data Conflict Winner—Select the group-level rules by which iCluster will resolve conflicts for the objects selected to a

master-master replication group.Possible values:• *SRC—Indicates that the source wins if there is a conflict. Source data is applied to the target, which ensures that target

data matches source data and mirroring continues.• *TGT—Indicates that the target wins if there is a conflict. iCluster does not apply data to the target. This rule preserves

data on the target and mirroring continues.• *EXIT—Indicates that you would like to resolve data conflicts through a user exit program. With this option, you must

specify a user exit program and library in the Conflict User Exit parameter below.• *NONE—Indicates that you do not want to enable conflict detection and resolution rules. If there is a conflict, iCluster


• Conflict User Exit—Select or type the name of the user exit program that you want to call to resolve data conflicts. A user exit program allows you to define a conflict resolution strategy that is not covered by one of the standard resolution methods in the Data Conflict Winner box.Specify the name of a user exit program or the following value:• *NONE—Indicates that you do not want to specify a user exit program.

See Parameter List Descriptions for a Conflict Resolution User Exit Program on page 326 for more information about the values you can pass to the user exit program.The default setting for this parameter is *NONE. You cannot specify *NONE for this parameter if you specify *EXIT for the Data Conflict Winner parameter.

• Conflict User Exit Library—Specify the library where the user exit program resides must be identified if you do not specify *NONE in the Conflict User Exit box above.

6 Click Execute.



Resilient Application CommandsThis section contains the following topics:• Working with Resilient Applications on page 459• Add Resilient Application on page 459• Change Resilient Application on page 462• Update Resilient Application on page 465• Remove Resilient Application on page 465• Start Resilient Application on page 466• End Resilient Application on page 468• Switchover Resilient Application on page 469

Working with Resilient ApplicationsWhen a resilient application is selected in the tree view, the set of associated attributes is shown on the table view. You can define or change these attributes through the Add Resilient Application and Change Resilient Application commands respectively.It is possible to have one or more of the following types of groups in a resilient application: • *REPL—Identifies a replication group. This group type handles the data replicated between the primary and backup node.

Object specifiers and suspended objects are displayed for this type of group.• *APPL—Identifies an application group. This group type handles application-specific actions defined by the application

vendor. Because *APPL groups deal with the application and not data, object specifiers and suspended objects do not apply for *APPL groups.

A typical configuration would be one *APPL group and one or more *REPL groups inside a resilient application. In the tree view, groups for resilient applications are listed under Group List.The application groups and object specifiers are pre-defined by the application vendor through the data areas and files that allow the application to operate in a clustered environment. However, you can define the nodes in the recovery domain for the application through iCluster Administrator, and you can activate or suspend specific application objects. See Activate Object and Suspend Object for more information.For more information about resilient applications, see Application Resiliency on page 68.

Resilient Application StatusesThere are two different statuses for resilient applications in iCluster. The cluster status of a *APPL or *REPL group indicates the state of its primary and backup nodes as reported by the failover mechanism. The replication status of a *REPL group indicates the state of object and data replication being performed by iCluster. Replication status only applies to *REPL groups. For more information on the statuses for groups in a resilient application, see DMWRKAPP—Work with Resilient Applications on page 177.

Monitoring OperationsIn the iCluster Administrator, group icons change color to indicate different statuses. See Monitoring Operations in the iCluster Administrator on page 385 for more information.

Add Resilient ApplicationUse this command to add a new resilient application to iCluster.You must issue this command on an active node in the cluster, and all specified nodes in the recovery domain must be active.For more information about application resiliency, see Application Resiliency on page 68.



Data AreasTo identify a resilient application, you must specify the library where the QCSTHAAPPI data area for the application is located on the primary node. This data area is defined by the application vendor and contains settings that are referenced by iCluster to define the resilient application. Another data area (QCSTHAAPPO) will contain information that confirms the addition of a resilient application through this command.

To add a resilient application1 From the iCluster Administrator window, select the Resilient Applications heading in the tree view.2 Right-click and select Add Resilient Application.

The Add Resilient Application dialog box opens.

This dialog box has the following tabs:• Properties, see Step 3.• Recovery Domain, see Step 4.• User Exit, see Step 5.

3 On the Properties tab, you can specify the following values:• Application Name—Specify the name of the resilient application that you are adding.



The name that you specify does not have to be the actual name of the application that is defined as being resilient in iCluster. However, it is recommended that the specified name allow you to identify the application easily. If necessary, use the Description box (see below) to identify the application.

• Application Data Library—Secify the name of the library on the primary node that contains the definition of the resilient application as specified by the vendor.This library contains the data area (QCSTHAAPPI) that is provided by the application vendor so that the application can operate in a clustered environment. Within the same library, the QCSTHAAPPO data area will contain information that confirms the addition of a resilient application through this command.

• Takeover IP Address—Specify the takeover IP address that will be associated with the resilient application on all nodes in the recovery domain (see note below).When a failover or switchover occurs, the takeover IP address will be moved to the resilient application's new primary node. Moving the takeover IP address avoids having to modify client configurations with the resilient application after operations have been moved to another node. Client interaction with the resilient application is essentially seamless even in the event of a failover or switchover.In the absence of a takeover IP address, a different IP address would have to be used for the same application on each node in the recovery domain. After a failover or switchover in this case, you must specify a new IP address on the client to support interaction with the application on the new primary node. Therefore, a takeover IP address simplifies the process of working with the resilient application after a failover or switchover, since no IP address changes are required on the application.You must specify the takeover IP address in dotted quad notation (for example, 125.4.3.55).

Note: This IP address is not the IP address of the primary or backup node that is specified when adding a node to the cluster. See Add Node for more information. In addition, this address must be reserved for the application to ensure it is resilient in the recovery domain. It cannot be used for any other purpose.



page 84.• Description—Specify a short description that allows you to identify this resilient application among all others that have been

defined in iCluster.Use this box to identify the application that will be resilient in the recovery domain. The description can be a maximum of 50 characters long, and is an optional setting.

4 On the Recovery Domain tab, you can specify the following values:• Select recovery domain from existing group—Select this option to define a recovery domain conveniently for the new

resilient application that has the same primary and backup nodes as the recovery domain for an existing node. Defining two or more recovery domains with the same nodes is useful when you want multiple applications to be resilient across the same nodes. If you select this option, you only have to specify the name of an existing replication group or resilient application that you want to use to define the recovery domain of the resilient application you are adding

• Specify the recovery domain—If you select this option, then you must specify the next two values.• Primary Node—Select the name of a node that will be the primary node in the recovery domain for the resilient application

you are defining.The node you want to select must have been added to the cluster, and is currently active. The library specified in the Application Data Library box (see above) must exist on this node.


The value specified in this box is only applicable if you selected the Specify the recovery domain option.• Nodes in Cluster—Select the node that will be the backup in the resilient application you are adding, and click the left arrow

(<) button to the left of this box.This action selects a backup node in the resilient application. At this time, you can define only one backup node in a resilient application.

Note: Both primary and backup nodes in a resilient application must reside in the same subnet.The node you want to select must have been added to the cluster. See Add Node for more information. You can select a backup node only when the Specify the recovery domain option is selected.

5 On the User Exit tab, you can specify the following values:Note: The parameters on this tab are not available for local loopback.

The User Exit Before Role Switch group box contains the Name drop-down list box and the Library box.• Name—Enter the name of the user exit program that you want to call immediately before a role change is performed.

The user exit program will be called on both nodes of the resilient application for a switchover, but only on the new primary node for a failover. See Switchover for IBM Cluster Resource Services (CRS) on page 341 for more information.The user exit program will be called on both nodes of the resilient application for switchover, but only on the new primary node for a failover.Special value:• *NONE—Indicates that you do not want to specify a user exit program.

The default value for this parameter is *NONE.• Library—Enter the library where the user exit program resides.

You must identify the library if you specify a user exit program.The User Exit After Role Switch group box contains the Name drop-down list box and the Library box.

• Name—Enter the name of the user exit program that you want to call immediately after a role change is performed.Special value:• *NONE—Indicates that you do not want to specify a user exit program.


You must identify the library if you specify a user exit program.• Exit Data—Enter the user-defined data that you want to pass to the user exit programs.

If you specify two different user exit programs (one program before the role switch, and a different program after the role switch), the same user-defined data is passed to both programs. A maximum of 256 bytes of data can be passed to the user exit programs. If your data contains spaces or non-alphanumeric characters (commas, periods, and so on), you must enclose your data in single quotes.

6 Click Execute.On the iCluster Administrator window, the new resilient application is added under Resilient Applications in the tree view.

Change Resilient ApplicationUse this command to change one or more attributes of an existing resilient application. The resilient application must not be running when you issue this command. The minimum security level for this command is Administrator (*ADMIN).For more information about application resiliency, see Application Resiliency on page 68.



For more information on data areas and resilient applications, see Data Areas on page 460.

To change a resilient application1 From the iCluster Administrator window, expand the Resilient Applications heading in the tree view.2 Right-click a Resilient Application and select Change Resilient Application.

The Change Resilient Application dialog box opens.

This dialog box has the following tabs:• Properties, see Step 3.• User Exit, see Step 4.

3 On the Properties tab, you can specify the following values:• Takeover IP Address—Specify the takeover IP address that will be associated with the resilient application on all nodes in

the recovery domain (see note below).When a failover or switchover occurs, the takeover IP address will be moved to the resilient application's new primary node. Moving the takeover IP address avoids having to modify client configurations with the resilient application after operations have been moved to another node. Client interaction with the resilient application is essentially seamless even in the event of a failover or switchover.


In the absence of a takeover IP address, a different IP address would have to be used for the same application on each node in the recovery domain. After a failover or switchover in this case, you must specify a new IP address on the client to support interaction with the application on the new primary node. Therefore, a takeover IP address simplifies the process of working with the resilient application after a failover or switchover, since no IP address changes are required on the application.You must specify the takeover IP address in dotted quad notation (for example, 125.4.3.55).

Note: This IP address is not the IP address of the primary or backup node that is specified when adding a node to the cluster. See Add Node for more information. In addition, this address must be reserved for the application to ensure it is resilient in the recovery domain. It cannot be used for any other purpose.



page 84.• Description—Specify a short description that allows you to identify this resilient application among all others that have been

defined in iCluster.Use this box to identify the application that will be resilient in the recovery domain. The description can be a maximum of 50 characters long, and is an optional setting.

4 On the User Exit tab, you can specify the following values:Note: The parameters on this tab are not available for local loopback.

The User Exit Before Role Switch group box contains the Name drop-down list box and the Library box.• Name—Enter the name of the user exit program that you want to call immediately before a role change is performed.

The user exit program will be called on both nodes of the resilient application for a switchover, but only on the new primary node for a failover. See Switchover for IBM Cluster Resource Services (CRS) on page 341 for more information.The user exit program will be called on both nodes of the resilient application for switchover, but only on the new primary node for a failover.Special value:• *NONE—Indicates that you do not want to specify a user exit program.


You must identify the library if you specify a user exit program.The User Exit After Role Switch group box contains the Name drop-down list box and the Library box.

• Name—Enter the name of the user exit program that you want to call immediately after a role change is performed.Special value:• *NONE—Indicates that you do not want to specify a user exit program.


You must identify the library if you specify a user exit program.• Exit Data—Enter the user-defined data that you want to pass to the user exit programs.




5 Click Execute.

Update Resilient ApplicationUse this command to update a resilient application in iCluster when the data areas and files provided by the application vendor are changed on the primary node.You must issue this command on an active node in the cluster. The resilient application must not be running when you issue this command.For more information about application resiliency, see Application Resiliency.The minimum security level for this command is Administrator (*ADMIN).For more information on data areas and resilient applications, see Data Areas on page 460.

To update a resilient application1 From the iCluster Administrator window, expand the Resilient Applications heading in the tree view.2 Right-click a Resilient Application and select Update Resilient Application.

The Update Resilient Application dialog box opens.

You can specify the following values on this dialog box:• Application Data Library—Specify the name of the library on the primary node that contains the updated definition of the

resilient application as specified by the vendor.This library contains the data areas and files that are provided by the application vendor so that the application can operate in a clustered environment. When these data areas and files are modified, you must use this command to update the corresponding resilient application in iCluster.

3 Click Execute.

Remove Resilient ApplicationUse this command to remove the specified resilient application in iCluster. Note: This command does not remove the software application associated with the resilient application.You must issue this command on an active node in the cluster. The resilient application must not be running when you issue this command.For more information about application resiliency, see Application Resiliency on page 68The minimum security level for this command is Administrator (*ADMIN).For more information on data areas and resilient applications, see Data Areas on page 460.


To remove a resilient application1 From the iCluster Administrator window, expand the Resilient Applications heading in the tree view.2 Right-click a Resilient Application and select Remove Resilient Application.

The Remove Resilient Application dialog box opens.

3 Click Execute.On the iCluster Administrator window, the resilient application that you selected is removed from the tree view.

Start Resilient ApplicationUse this command to start cluster operations for the selected resilient application.Objects associated with the application are identified through groups that are defined in the data areas and files provided by the application vendor. Therefore, this command ensures that application objects are mirrored to nodes in application groups to support a resilient application switchover or failover. See Switchover Resilient Application for more information. You can perform an initial refresh of the application objects before starting cluster operations.Cluster operations for groups that identify application objects are started in an indeterminate order. If the application requires operations to be started in a specific order, the order must be defined through a user exit provided by the application vendor.For more information about application resiliency, see Application Resiliency on page 68.The minimum security level for this command is Operator (*OPERATOR).For more information on data areas and resilient applications, see Data Areas on page 460.

To start a resilient application1 From the iCluster Administrator window, expand the Resilient Applications heading in the tree view.2 Right-click a Resilient Application and select Start Resilient Application.

The Start Resilient Application dialog box opens.



This dialog has the following tabs:• General, see Step 3.• Advanced, see Step 4.

3 On the General tab, you can specify the following values:• Start Replication Apply Jobs—Select whether the apply processes on the backup node in the resilient application will be

started when mirroring begins.Possible values:• *NOCHG—Specifies that the current operational status of the apply jobs on the backup node is not changed.• *YES—Indicates that the backup node apply jobs for the resilient application will be started. This does not apply to

suspended files.• *NO—Specifies that the backup node apply jobs for the resilient application will not be started.

The default value for this parameter is *NOCHG.4 Select the Advanced tab.


You can specify the following values on this tab:• Refresh Selected Objects—Select if you want an initial refresh of objects in application groups to be performed before

mirroring is started. If the Use Marked Journal Positions box (see below) is checked, this setting is ignored.By default, this box is not checked.

Warning: If you unexpectedly began a refresh (by selecting the *YES option in the Refresh Selected Objects box) and then ended the refresh before completion (due to system constraints), you should contact DataMirror Technical Support for assistance. See Contacting Technical Support on page 547 for more information. You must perform a number of steps following a discontinued refresh to allow mirroring to re-start properly. If these steps are not performed following a discontinued refresh, mirroring may not work as you intended.

If you intentionally began a refresh that ended unexpectedly (for example, power failure), you should re-start the refresh.• Use Marked Journal Positions—Select if you want mirroring to start at the marked journal positions for the specified resilient

application. These journal positions are marked through the Mark Journal Positions command, and are maintained in the iCluster metadata.If you do not check this box, then iCluster will start mirroring from either the saved journal positions when replication was previously stopped, the journal positions set through the Set Journal Position command prior to issuing this command, or if there are no saved journal positions, mirroring will start at the current journal positions for objects selected to the application groups.If you check this box, then mirroring will start at the journal positions that have been marked for the specified resilient application through the Mark Journal Positions command. The primary and backup nodes must have been synchronized at the time the Mark Journal Positions command was issued. By default, this box is not checked.

Note: If you do not invoke the Mark Journal Positions command for the named resilient application, and this box is checked, mirroring is started at the current journal positions.

5 Click Execute.

End Resilient ApplicationUse this command to end cluster operations for the selected resilient application.Objects associated with the application are identified through groups that are defined in the data areas and files provided by the application vendor. Therefore, this command ends mirroring application objects to nodes in application groups. You can specify the manner in which replication is ended either immediately or in a controlled manner.Replication within groups that identify application objects is ended in an indeterminate order. If the application requires operations to be ended in a specific order, you must define the order through a user exit provided by the application vendor.For more information about application resiliency, see Application Resiliency.The minimum security level for this command is Operator (*OPERATOR).For more information on data areas and resilient applications, see Data Areas on page 460.

To end a resilient application1 From the iCluster Administrator window, expand the Resilient Applications heading in the tree view.2 Right-click a Resilient Application and select End Resilient Application.

The End Resilient Application dialog box opens.



You can specify the following values in this dialog box:• Option—Select how replication within the application groups is ended.

You can end replication immediately or in a controlled manner. An immediate end concludes replication within the application groups at the point when you issue this command. As a result, iCluster may not be able to guarantee that replication is ended in the desired manner. On the other hand, a controlled end allows iCluster to perform the necessary tasks to ensure that replication is gracefully ended. However, the completion of these tasks may take some time. A controlled end is the recommended selection.Possible values:• *CNTRLD—Ends replication within the application groups in a controlled manner.• *IMMED—Ends replication within the application groups immediately.

The default value for this parameter is *CNTRLD.• Delay Time—Specify the maximum amount of time, in seconds, for replication within the application groups to end in a

controlled manner without intervention.You can specify a timeout period for ending replication. If, for some reason, replication cannot be completed within the timeout period, it will be immediately ended after the timeout period expires. This box is only applicable when the Option box (see above) is set to *CNTRLD.The default value for this parameter is 3600 seconds.

3 Click Execute.

Switchover Resilient ApplicationUse this command to initiate a switchover within the groups associated with the specified resilient application. This command causes a switchover involving the primary and backup nodes in the groups.When this command is used, the current primary node in each group becomes the backup node, and the current backup node in each group assumes the role of the primary node.For more information about application resiliency, see Application Resiliency.The minimum security level for this command is Administrator (*ADMIN).For more information on data areas and resilient applications, see Data Areas on page 460.

To switchover a resilient application1 From the iCluster Administrator window, expand the Resilient Applications heading in the tree view.2 Right-click a Resilient Application and select Switchover Resilient Application.

The Switchover Resilient Application dialog box opens.


You can specify the following values on this dialog box:• Repl. Group Switchover Delay—Select the amount of time in seconds that is allowed between switchover of the application

groups associated with the specified resilient application and switchover of the replication groups associated with the specified resilient application.

• Re-start Replication—Specify whether or not replication processes for the resilient application will be re-started after switchover processing is completed.Possible values:• *YES—Indicates that replication processes for the resilient application will be re-started from the new primary node to the

new backup node when switchover processing is complete. This is the default setting.• *NO—Indicates that replication processes for the resilient application will not be re-started from the new primary node to

the new backup node when switchover processing is complete.The default value for this parameter is *YES.

3 Click Execute.

MQSeries Group CommandsThis section contains the following topics:• Working with MQSeries Groups on page 470• Add MQSeries Group on page 471• Start MQSeries Group on page 476• End MQSeries Group on page 478• Remove MQSeries Group on page 478• Change MQSeries Group on page 479• Switchover MQSeries Group on page 483

Working with MQSeries GroupsMQSeries groups support the continuous replication of MQSeries objects from the primary node to the backup node. When an MQSeries group is selected in the tree view, the set of associated attributes is shown on the table view. You can define or change the attributes of an MQSeries group through the Add MQSeries Group and Change MQSeries Group commands, respectively.For more information about groups, see Working in a Clustered Environment on page 43.


MQSeries Group Commands

MQSeries Group StatusesThere are two different statuses for MQSeries groups in iCluster. The cluster status of a group indicates the state of low-level cluster support provided by the failover mechanism within the group. The replication status of a group indicates the state of object and data replication being performed by iCluster. For more information on the different statuses for MQSeries groups, see DMWRKGRP—Work with Groups on page 137.

Monitoring OperationsIn the iCluster Administrator, node icons change color to indicate different statuses. See Monitoring Operations in the iCluster Administrator on page 385 for more information.

Add MQSeries GroupUse this command to add an MQSeries group consisting of one primary node and one backup node.All nodes added to the MQSeries group must be active.The minimum security level for this command is Administrator (*ADMIN).

To add an MQSeries group1 From the iCluster Administrator window, select the MQSeries Groups heading in the tree view.2 Right-click and select Add MQSeries Group.

The Add MQSeries Group dialog box opens.


This dialog box has the following tabs:• Properties, see Step 3.• Recovery Domain, see Step 4.• User Exit, see Step 5.• Physical File, see Step 6.

3 On the Properties tab, you can specify the following values:• MQSeries Group Name—Specify the name of the MQSeries group that you want to define in iCluster.

The specified group name must be unique.• MQSeries Product Version—Specify the product version of MQSeries that you are currently using.

Possible values:• V5R2M0—Specifies V5R2M0 for the MQSeries group that you are creating.• V5R3M0—Specifies V5R3M0 for the MQSeries group that you are creating.• V6R0M0—Specifies V6R0M0 for the MQSeries group that you are creating.



The default setting for this parameter is V5R3M0.Note: This parameter only applies to groups of type *MQSeries.• Queue Manager Name—Specify the name of the MQSeries queue manager that needs to be mirrored.

Multiple queue managers are not supported. If you want to mirror multiple queue managers, you will need to create one group for each queue manager.You cannot use generic names in this box.

• Failover Message Queue Name—Specify the name of the message queue that will receive messages generated when a failover occurs.Special value:• *NONE—Indicates that you do not want to specify a message queue.



becomes the primary node in the MQSeries group when a failover occurs.If set to *YES, the functions of the primary node will be moved to the backup node when a failover occurs.Possible values:• *NO—Does not automatically change the role of the backup node in the MQSeries group to the primary node when a

failover occurs. The *NO setting gives you the opportunity to decide if you would like to continue with a role switch, or continue with your current configuration. No user exit programs are called with this setting.

With this setting, a failure of the MQSeries group's primary node is declared by the failover mechanism, although iCluster will not prepare the backup node to take over as the primary node. Replication will not start up automatically once the failed primary node is again active in the cluster.If you are using OS/400 Cluster Resource Services, a setting of *NO for this parameter will cause a failover of the MQSeries group's primary node to be declared by OS/400 Cluster Resource Services, although iCluster will not prepare the backup node to take over as the primary node. Replication will not start up automatically once the failed primary node is again active in the cluster.

Note: Even though the backup node has not been prepared to take over as the primary node, it will still appear as the primary node for the MQSeries groups of which it was the backup node prior to the failure.

• *YES—Automatically changes the role of the backup node in the MQSeries group to the primary node when a failover occurs. In this case, user exit programs specified through the User Exit Before Role Switch and User Exit After Role Switch boxes (see below) will be called.

Note: If you do not want an automatic role switch to be performed in the specified MQSeries group (this parameter set to *NO), use the Set Primary Node or the Change Role command to prepare the new primary node, that is, the previous backup node, for replication after a failover occurs. If you set this parameter to *YES, this step is not necessary.

The default value for this parameter is *NO.See the Switchover for IBM Cluster Resource Services (CRS) on page 341 for more information.

• Check Journaling At Role Switch—Specify whether iCluster will check if all mirrored objects that should be journaled are being journaled on the new primary node.Specifies whether (when switching to the new primary node) iCluster will check whether the objects for the specified group are being properly journaled on that node, and start journaling them. Certain types of objects must be journaled before replication starts. However, because journaling objects can be time-consuming, this option allows you to specify whether or not journaling should be started for objects during a switchover, depending on your business requirements.


Depending on the value of the group's Check Journaling At Role Switch parameter, iCluster may perform processing that is equivalent to the Mark Journal Positions command when a roleswitch occurs. If the value of the group's Check Journaling At Role Switch parameter is *NO, iCluster will not perform Mark Journal Positions processing and the replication metadata for the new primary node is generated based upon the existing replication metadata on the current backup node. In the case of an error in processing the backup replication metadata, or non-existent backup replication metadata, iCluster performs Mark Journal Positions processing regardless of the Check Journaling At Role Switch parameter's value.



primary node. If they are not, the objects will be journaled.• *NO—Specifies that iCluster will not check if all mirrored objects that should be journaled are journaled on the new


The default value for this parameter is *YES.• Description—Specify a short description that allows you to identify this MQSeries group among all others that have been


4 On the Recovery Domain tab, you can specify the following values:• Select recovery domain from existing group—Define a recovery domain for the new resilient application that has the same

primary and backup nodes as the recovery domain for an existing node. Defining two or more recovery domains with the same nodes is useful when you want multiple applications to be resilient across the same nodes. Select the name of an existing replication group or resilient application that you want to use to define the recovery domain of the resilient application you are adding

• Specify the recovery domain—If you select this option, then you must specify the next two values.• Primary Node—Select the name of a node that will be the primary node in the replication group you are adding.

The node you specify must have been added to the cluster. See Add Node for more information. This box is only applicable if the Specify the recovery domain option is selected.

• Nodes in Cluster—Select the node that will be the backup in the replication group you are adding, and click the left arrow (<) button to the left of this box.This action selects a backup node in the replication group. At this time, you can define only one backup node in an MQSeries group.

Note: Both primary and backup nodes in a replication group must reside in the same subnet.The node you select must have been added to the cluster. See Add Node for more information. You can select a backup node only when the Specify the recovery domain option is selected.

5 On the User Exit tab, you can specify the following values:• Name (under User Exit Before Role Switch)—Specify the name of the user exit program that you want to call immediately

before the role change is performed.The user exit program will be called on both nodes of the group for switchover, but only on the new primary node for a failover.



If you specify a user exit program, you must set the Do Role Switch at Failover box (see above) to *YES if you want to call the program when a failover occurs.



The default setting is *NONE.See the DMSETPRIM—Prepare Primary Node command for more information about the arguments that can be passed to the user exit program.


• Name (under User Exit After Role Switch)—Specify the name of the user exit program that you want to call immediately after the role change is performed.After a role change occurs, the user exit program will be called on both nodes of the group.If you specify a user exit program, you must set the Do Role Switch at Failover box (see above) to *YES if you want to call the program when a failover occurs.



The default value for this parameter is *NONE.See the DMSETPRIM—Prepare Primary Node command for more information about the arguments that can be passed to the user exit program.


• Exit Data—Specify the user-defined data that you want to pass to the user exit programs specified on this tab (see above).If you specify two different user exit programs (one program before the role switch, and a different program after the role switch), the same user-defined data is passed to both programs. A maximum of 256 bytes of data can be passed to the user exit programs. If your data contains spaces or non-alphanumeric characters (commas, periods, and so on), you must enclose your data in single quotes.

6 On the Physical File tab, you can specify the following values• Default Database Journal—Specify the name of the database journal that you want to use as your default.


information.


The default value for this parameter is *CLUSTER.• Default Database Journal Library—Specify the name of the library where the database journal currently resides.

If you specified *CLUSTER in the Default Database Journal box, a library name is not required, as the corresponding cluster system value (Default database journal library) will be used.

7 Click Execute.On the iCluster Administrator window, the new MQSeries group is added under the MQSeries Groups heading in the tree view.

Start MQSeries GroupUse this command to start cluster operations for the specified MQSeries group.This command also sets the cluster status of the specified group to *ACTIVE. If the group is a replication group, the replication status of the group will become *ACTIVE when replication processes are started.

To start an MQSeries Group1 From the iCluster Administrator window, expand the MQSeries Groups heading in the tree view.2 Right-click an MQSeries group and select Start MQSeries Group.

The Start MQSeries Group dialog box opens.

This dialog box has the following tabs:• General, see Step 3.• Advanced, see Step 4.

3 On the General tab, you can specify the following values:• Start Replication Apply Jobs—Select whether the apply processes on the backup node in the MQSeries group will be

started when mirroring begins.Possible values:• *NOCHG—Specifies that the current operational status of the apply jobs on the backup node is not changed.• *YES—Indicates that the backup node apply jobs for the MQSeries group will be started. This does not apply to

suspended files.• *NO—Specifies that the backup node apply jobs for the MQSeries group will not be started.



The default value for this parameter is *NOCHG.4 Select the Advanced tab.

On the Advanced tab, you can specify the following values:• Refresh—Select if you want an initial refresh of selected objects in the replication group to be performed before mirroring is

started.Note: If you check the Use Marked Journal Positions box (see below), this setting is ignored.

By default, this box is not checked.Warning: If you unexpectedly began a refresh (by selecting the *YES option in the Refresh Selected Objects box) and

then ended the refresh before completion (due to system constraints), you should contact DataMirror Technical Support for assistance. See Contacting Technical Support on page 547 for more information. You must perform a number of steps following a discontinued refresh to allow mirroring to re-start properly. If these steps are not performed following a discontinued refresh, mirroring may not work as you intended.

If you intentionally began a refresh that ended unexpectedly (for example, power failure), you should re-start the refresh.• Use Marked Journal Positions—Select if you want mirroring to start at the marked journal position for the specified

replication group.These journal positions are marked through the DMMRKPOS—Mark Current Journal Positions command, and are maintained in the iCluster metadata.If you do not check this box, then iCluster replication will not start at positions marked using the Mark Journal Positions command. iCluster replication will start at either: • Journal positions set using the Set Journal Positions command. • Journal positions registered using the DMREGPOS—Register Positions command.• The journal entry following the last journal position received on the backup system when replication was previously

stopped.If you check this box, then mirroring will start at the journal positions that have been marked for the specified replication group through the Mark Journal Positions command. The primary and backup nodes must have been synchronized at the time the Mark Journal Positions command was issued. Checking this box will overwrite any starting journal positions previously recorded by the Set Journal Positions or Mark Journal Positions commands.By default, this box is not checked.


5 Click Execute.

End MQSeries GroupUse this command to end cluster operations for the specified MQSeries group and set the cluster and replication statuses of the specified MQSeries group to *INACTIVE.

To end an MQSeries Group1 From the iCluster Administrator window, expand the MQSeries Groups heading in the tree view.2 Right-click an MQSeries group and select End MQSeries Group.

The End MQSeries Group dialog box opens.

You can specify the following values on this dialog box:• Option—Select how replication within the replication group is ended.

You can end replication immediately or in a controlled manner. An immediate end concludes replication within the group at the point when you issue this command. As a result, iCluster may not be able to guarantee that replication is ended in the desired manner. On the other hand, a controlled end allows iCluster to perform the necessary tasks to ensure that replication is gracefully ended. However, the completion of these tasks may take some time. A controlled end is the recommended selection.Possible values:• *CNTRLD—Ends replication within the replication group in a controlled manner.• *IMMED—Ends replication within the replication group immediately.


without intervention.This box allows you to specify a timeout period for ending replication. If, for some reason, replication cannot be completed within the timeout period, it will be immediately ended after the timeout period expires. This box is only applicable when the Option box (see above) is set to *CNTRLD.The default value for this parameter is 3600 seconds.

3 Click Execute.

Remove MQSeries GroupUse this command to remove an existing MQSeries group.If the MQSeries group is active, group operations are automatically stopped before the MQSeries group is removed.




To remove an MQSeries Group1 From the iCluster Administrator window, expand the MQSeries Groups heading in the tree view.2 Right-click an MQSeries group and select Remove MQSeries Group.

The Remove MQSeries Group dialog box opens.

3 Click Execute.On the iCluster Administrator window, the MQSeries group that you selected is removed from the tree view.

Change MQSeries GroupUse this command to change one or more attributes of an MQSeries group. It does not allow you to change the primary and backup nodes in the MQSeries group. You can achieve this through Add Backup Node and Remove Backup Node.To change attributes, the selected MQSeries group must be inactive.The minimum security level for this command is Administrator (*ADMIN).

To change an MQSeries group1 From the iCluster Administrator window, expand the MQSeries Groups heading in the tree view.2 Select an MQSeries group and select Change MQSeries Group.

The Change MQSeries Group dialog box opens.


This dialog box has the following tabs:• Properties, see Step 3.• User Exit, see Step 4.• Physical File, see Step 5.




If you selected *NONE in the Failover Message Queue Name box, a library name is not required or is ignored if it is specified.



• Do Role Switch at Failover—Select whether you want the role of the backup node to change automatically so that it becomes the primary node in the MQSeries group when a failover occurs.If set to *YES, the functions of the primary node will be moved to the backup node when a failover occurs.Possible values:• *NO—Does not automatically change the role of the backup node in the MQSeries group to the primary node when a

failover occurs. The *NO setting gives you the opportunity to decide if you would like to continue with a role switch, or continue with your current configuration.

With this setting, a failure of the MQSeries group's primary node is declared by the failover mechanism, although iCluster will not prepare the backup node to take over as the primary node. Replication will not start up automatically once the failed primary node is again active in the cluster.If you are using OS/400 Cluster Resource Services, a setting of *NO for this parameter will cause a failover of the MQSeries group's primary node to be declared by OS/400 Cluster Resource Services, although iCluster will not prepare the backup node to take over as the primary node. Replication will not start up automatically once the failed primary node is again active in the cluster.

Note: Even though the backup node has not been prepared to take over as the primary node, it will still appear as the primary node for the MQSeries groups of which it was the backup node prior to the failure. No user exit programs are called with this setting.

• *YES—Automatically changes the role of the backup node in the MQSeries group to the primary node when a failover occurs. In this case, user exit programs specified through the User Exit Before Role Switch and User Exit After Role Switch boxes (see below) will be called.

Note: If you do not want an automatic role switch to be performed in the specified MQSeries group (this parameter set to *NO), use the Set Primary Node or the Change Role command to prepare the new primary node, that is, the previous backup node, for replication after a failover occurs. If you set this parameter to *YES, this step is not necessary.

The default value for this parameter is *NO.See the Switchover for IBM Cluster Resource Services (CRS) on page 341 for more information.

• Check Journaling At Role Switch—Specify whether iCluster will check if all mirrored objects that should be journaled are being journaled on the new primary node.Specifies whether (when switching to the new primary node) iCluster will check whether the objects for the specified group are being properly journaled on that node, and start journaling them. Certain types of objects must be journaled before replication starts. However, because journaling objects can be time-consuming, this option allows you to specify whether or not journaling should be started for objects during a switchover, depending on your business requirements.Depending on the value of the group's Check Journaling At Role Switch parameter, iCluster may perform processing that is equivalent to the Mark Journal Positions command when a roleswitch occurs. If the value of the group's Check Journaling At Role Switch parameter is *NO, iCluster will not perform Mark Journal Positions processing and the replication metadata for the new primary node is generated based upon the existing replication metadata on the current backup node. In the case of an error in processing the backup replication metadata, or non-existent backup replication metadata, iCluster performs Mark Journal Positions processing regardless of the Check Journaling At Role Switch parameter's value.


If the value of the group's Check Journaling At Role Switch parameter is *YES, iCluster will always perform Mark Journal Positions processing during a roleswitch, and the backup replication metadata is not used for setting up the replication metadata for the new primary node.Special values:


• *YES—Specifies that iCluster will check if all mirrored objects that should be journaled are being journaled on the new primary node. If they are not, the objects will be journaled.

• *NO—Specifies that iCluster will not check if all mirrored objects that should be journaled are journaled on the new primary node. You will need to make sure that all objects that need to be journaled are being journaled to the correct journals, and that the objects have been journaled before replication starts.

The default value for this parameter is *YES.• Description—Specify a short description that allows you to identify this MQSeries group among all others that have been


4 On the User Exit tab, you can specify the following values:• Name (under User Exit Before Role Switch)—Specify the name of the user exit program that you want to call immediately

before the role change is performed.The user exit program will be called on both nodes of the group for switchover, but only on the new primary node for a failover.If you specify a user exit program, you must set the Do Role Switch at Failover box (see above) to *YES if you want to call the program when a failover occurs.



The default setting is *NONE.See the DMSETPRIM—Prepare Primary Node command for more information about the arguments that can be passed to the user exit program.


• Name (under User Exit After Role Switch)—Specify the name of the user exit program that you want to call immediately after the role change is performed.After a role change occurs, the user exit program will be called on both nodes of the group.If you specify a user exit program, you must set the Do Role Switch at Failover box (see above) to *YES if you want to call the program when a failover occurs.



The default value for this parameter is *NONE.See the DMSETPRIM—Prepare Primary Node command for more information about the arguments that can be passed to the user exit program.

• Library (under User Exit After Role Switch)—Specify the name of the library where the after role switch user exit program currently resides.



If you selected *NONE in the Name box, a library name is not required or is ignored if it is specified.• Exit Data—Specify the user-defined data that you want to pass to the user exit programs specified on this tab (see above).


5 On the Physical File tab, you can specify the following values• Default Database Journal—Specify the name of the database journal that you want to use as your default.




6 Click Execute to change the MQSeries group.

Switchover MQSeries GroupUse this command to initiate a switchover involving the primary and backup nodes for a specified MQSeries group. When this command is used, the current primary node becomes the backup node, and the current backup node assumes the role of the primary node. In addition, the user exit program that may have been specified for the MQSeries group will be invoked on the new primary node. See Add MQSeries Group and Change MQSeries Group for more information.The minimum security level for this command is Administrator (*ADMIN).

To switchover an MQSeries group1 From the iCluster Administrator window, expand the MQSeries Groups heading in the tree view.2 Right-click an MQSeries group and select Switchover MQSeries Group.

The Switchover MQSeries Group dialog box opens.

In this dialog box, you can specify the following values:• Re-start Replication—Indicate whether you want to re-start replication after a switchover has completed.

Specify one of the following values:• *YES—Indicates that replication processes for the group will be re-started from the new primary node to the new backup

node when switchover processing is complete.


• *NO—Indicates that replication processes for the group will not be re-started from the new primary node to the new backup node when switchover processing is complete. If you specify this value, you can use the Start MQSeries Group command to re-start replication at a later time. You must set the Use marked journal positions parameter to *YES when invoking the command for the first time after a switchover.

The default value for this parameter is *YES.3 Click Execute.

Working with Recovery DomainsA recovery domain consists of the nodes in a replication group (full replication and refresh-only), resilient application, or MQSeries group that maintain the same set of replicated objects for high availability. Currently, you can include a maximum of two nodes (one primary node and one backup node) in a recovery domain.For more information about recovery domains, see Working in a Clustered Environment on page 43.This section contains the following topics:• Add Backup Node on page 484• Remove Backup Node on page 485

Add Backup NodeUse this command to add a backup node to the recovery domain for an existing replication group (full replication or refresh-only) or resilient application.You can add only a single backup node through this command. The recovery domain for the replication group or resilient application must already have a defined primary node. You can add a backup node only to recovery domains for inactive replication groups or resilient applications that are not currently running.You must add a backup node to the recovery domain prior to replication being started within the domain.The specified node must be active when you issue this command.The minimum security level for this command is Administrator (*ADMIN).

To add a backup node1 From the iCluster Administrator window, select the replication group (full replication or refresh-only) or resilient application

where you want to add the recovery domain.2 Right-click the Recovery Domain heading and select Add Backup Node.

The Add Backup Node dialog box opens.


Working with Recovery Domains

You can specify the following values on this dialog box:• Node Name—Specify the name of the node in the cluster that will be added to the recovery domain for the specified

replication group or resilient application. You must specify a node that has been added to the cluster.

• Backup IASP Device Name—Specify the name of an existing independent auxiliary storage pool (IASP) on the backup node to which you want to replicate.Special value:• *SYSBAS—Indicates that you are using a system ASP rather than an independent ASP for storing the objects that will


3 Click Execute.The backup node is added to the recovery domain.

Remove Backup NodeUse this command to remove the backup node from the recovery domain for inactive replication groups (full replication or refresh-only) and resilient applications.You can remove a backup node only using this command. The recovery domain for the group or resilient application must already have defined primary and backup nodes. You can remove backup nodes only from recovery domains for inactive groups or resilient applications that are not currently running.The recovery domain for the replication group or resilient application must already have defined primary and backup nodes.The minimum security level for this command is Administrator (*ADMIN).

To remove the backup node1 From the iCluster Administrator window, select the group (full replication or refresh-only) or resilient application from where

you want to remove the recovery domain.2 Expand the Recovery Domain heading, and select a recovery domain.3 Right-click the Recovery Domain heading and select Remove Backup Node.

The Remove Backup Node dialog box opens.

4 Click Execute.The selected backup node is removed from the recovery domain.


Working with Object SpecifiersObject specifiers are definitions in iCluster that reference objects that you want to replicate within a recovery domain. You can define any number of object specifiers, and through generic names and special values, each object specifier can reference more than one object on a node. Object specifiers are listed on the table view. For more information about object specifiers, see Object Specifiers and Types on page 50 and Path Object Specifiers on page 53.This section contains the following topics:• Select/Add Object Specifier on page 486• Deselect Object Specifier on page 492• Change Object Specifier on page 492

Select/Add Object SpecifierUse this command to define a new object specifier and select it to a replication group (full replication, refresh-only, or master-master). The objects referenced by selected object specifiers are replicated from the primary node to the backup node in the replication group.Through the use of generic object specifiers, you can reference any number of objects through a single object specifier.You must issue this command on an active node in the cluster.Note: If you decide to synchronize objects on the primary and backup nodes through a save file or tape transfer, it is

important that you select the object specifiers referencing these objects before saving the objects. This ensures that changes to the objects will be audited between the time of the save and the time when replication is started.


To select/add an object specifier1 From the iCluster Administrator window, expand the Replication Groups heading.2 Under the Full Replication, Refresh-Only, or Master-Master heading, expand a group and select the Object Specifiers

heading.3 Right-click and select Select/Add Object Specifier.

The Select/Add Object Specifier dialog box opens.


Working with Object Specifiers

This dialog box has three tabs:• Select Native Object Specifier if the objects you want to replicate are stored in the native OS/400 file system. See

Step 4. See Object Specifiers and Types on page 50 for more information.• Select Path Object Specifier if the objects you want to replicate are stored in the Integrated File System (IFS), select

Path Object Specifier. See Step 5. For more information, see Path Object Specifiers on page 53.• Conflict Resolution, see Step 6.• General properties, see Step 7.

4 Select the Native Object Specifier to specify the following values:


• Object Specifier Name—Specify the object name component of the object specifier that you want to select.You can also specify a generic name to identify multiple objects in a library. See Object Specifiers and Types on page 50 for more information about generic names.Special value:• *ALL—Specifies all objects in the specified library.

The default value for this parameter is *ALL.• Object Specifier Library—Specify the name of the library where the objects are located.• Object Type—Select the object type component of the object specifier that you want to select.

Special value:• *ALL—Specifies all object types.

The default value for this parameter is *ALL.For a complete list of all object types that iCluster can replicate, see Object Types Replicated by iCluster on page 529.

• Object Extended Attribute—Specify the attribute component of the object specifier you want to select.This parameter is only applicable when the OBJTYPE parameter (see above) is set to *FILE.Special value:• *ALL—Specifies all object attributes. *ALL is not a valid system attribute, but allows iCluster to gather all objects

regardless of their attribute.The default value for this parameter is *ALL.

• Target Library—Specify the name of the library on the backup system that will receive the replicated objects.Special values:• *GROUP—Specifies the same target library as specified in the Add Full Replication Group or Change Full Replication

Group dialog boxes.• *PRIMARY—Sets the target library so that it is the same as the primary node library where the object resides.

The default value for this parameter is *GROUP.If the primary and backup environments reside on the same physical system (local loopback implementation), the target library that you specify cannot be the same library where a selected object in the group currently resides. This restriction means that iCluster does not permit the special value *PRIMARY in this situation.


• Polling Interval—Specify the time interval that determines how often iCluster should check for content changes to user spaces.You set the interval by specifying the number of hours (first two digits), minutes (middle two digits), and the seconds (last two digits) between consecutive polls.Possible values:• *GROUP—Refers to the corresponding replication group value that is specified when adding or changing a replication

group.• *CLUSTER—Uses the value specified for the corresponding system parameter (Default polling interval). See Setting


The default value for this parameter is *GROUP.



• Physical File Update Method—Select the method that will be used to update physical data files.You must specify the update method either by relative record number or by unique key. Relative record number specifies the location of a record in relation to the beginning of a file.Unique key specifies a unique index used to update a file. It allows you to perform reorganizations of physical files at different times on the primary node and the backup node. This option requires that both before and after images are journaled on the backup node because before images are required to remove applied or keyed replication updates. Also, if updating multi-member files, the members of the unique index must have the same name as the members of the physical file.If you change the replication method in an object specifier for a *FILE object from *UKEY to *RRN while the replication group is inactive, you must refresh the file before replication can continue. This box is applicable when *FILE object types with an attribute of PFDTA are selected, and when the Include or Exclude box is set to *INCLUDEPossible values:• *RRN—Indicates an update by relative record number.• *UKEY—Indicates an update by unique index. If you are updating by unique index, you then need to specify the name of

the unique index (logical file) in the Unique Key Name box (see below). If you cannot specify a unique index, then you must choose to update by relative record number.

The default value for this parameter is *RRN.• Unique Key Name—Specify the name of the physical file's unique key.

An object name and library defines which file to use as the physical file's unique key. You can specify the unique key or the following value:• *AUTO—Indicates that you want iCluster to automatically determine a unique key for every physical file being replicated

by the object specifier.Note: If the unique key cannot be found on the target, iCluster suspends the physical file on the target with the UKM

suspension code.This box is applicable if the Physical File Update Method box is set to *UKEY and the Include or Exclude box is set to *INCLUDE. You must specify this value when the update method is by unique key. The object specifier must identify a unique index for the file.

• Unique Key Library—Specify the name of the library where the physical file's unique key file is located.• Mirror Contents—Specify whether or not the contents of the object are mirrored. This parameter allows you to mirror object-

level operations such as CREATE, DELETE, RENAME, and MOVE, but not the contents of the object.Note: This parameter only applies to iBalance and Master-Master replication groups. For more information on this

feature, see iBalance and Master-Master Replication on page 321.You can only use this parameter with the following type and attribute combinations:• Object Type parameter is *DTAQ• Object Type parameter is *DTAARA• Object Type parameter is *FILE and Object Attribute parameter is PFDTA

Specify one of the following values:• *YES—Indicates the contents of the object are refreshed and mirrored.• *NO—Indicates the contents of objects are not refreshed or mirrored. Refresh creates an empty object at target, and

only the object-level operations such as CREATE, DELETE, RENAME, and MOVE are mirrored.The default setting for this parameter is *YES.

You can now proceed to Step 7 to set the general properties on this dialog box.


5 Select Path Object Specifier to specify the following values:• Path—Specify the location of the path object specifier that you are defining.

Path object specifiers identify the location of Byte Stream File (BSF) objects in the Integrated File System (IFS) or QDLS file system. See Path Object Specifiers on page 53 for more information about correctly identifying the path.

• Journal Name—Specify how BSF objects referenced by the path object specifier will be replicated with the replication group.Possible values:• *GROUP—BSF objects matching this object specifier will use the journal specified at the group level. See Add Full

Replication Group for more information.• *NONE—Specifies that BSF objects matching this object specifier will not be journaled by iCluster. Only object-level

changes to objects matching this specifier will be replicated. Changes to the contents of the objects matching this specifier will not be replicated unless the entire object is replaced.

The default value for this parameter is *NONE.Note: The overlap of journaled and non-journaled path object specifiers is restricted in iCluster. For example, using


JOURNAL parameter of any path specifier matching QDLS.• Polling Interval—Indicate whether or not BSF objects will be polled.

Possible values:• *GROUP—Indicates that BSF objects will use the corresponding replication group value that is specified when adding or

changing a replication group. See Add Full Replication Group or Change Full Replication Group for more information.• *NONE—Indicates that the BSF objects will not be polled.

The default value for this parameter is *GROUP.Note: Note that it is only possible to poll BSF objects in the QDLS folder.6 Select Conflict Resolution to specify the following values:Note: The parameters on this tab only apply to iBalance and Master-Master replication groups. For more

information on this feature, see iBalance and Master-Master Replication on page 321.• Data Conflict Winner—Select the group-level rules by which iCluster will resolve conflicts for the objects selected to a

master-master replication group.Possible values:• *GROUP—Indicates that the object specifier inherits conflict resolution rules and the user exit program configured at the

group-level. For more information, see Add Master-Master Group.Note: For path object specifiers, if you specify *GROUP for this value and then set the Data Conflict Winner value at

the group level to *TGT, iCluster will enforce the *NONE conflict resolution rule for all objects matching this specifier.

• *SRC—Indicates that the source wins if there is a conflict. Source data is applied to the target, which ensures that target data matches source data and mirroring continues.

• *TGT—Indicates that the target wins if there is a conflict. iCluster does not apply data to the target. This rule preserves data on the target and mirroring continues.

Note: This value is not valid for path object specifiers.• *EXIT—Indicates that you would like to resolve data conflicts through a user exit program. With this option, you must

specify a user exit program and library in the parameters below.• *NONE—Indicates that you do not want to enable conflict detection and resolution rules. If there is a conflict, iCluster

suspends the file from replication and sends a message to the event log. This is the default behavior for iCluster.



The default value for this parameter is *NONE.• Conflict User Exit—Select or type the name of the user exit program that you want to call to resolve data conflicts. A user

exit program allows you to define a conflict resolution strategy that is not covered by one of the standard resolution methods in the Data Conflict Winner box.Specify the name of a user exit program or the following value:• *NONE—Indicates that you do not want to specify a user exit program.

See Parameter List Descriptions for a Conflict Resolution User Exit Program for more information about the values you can pass to the user exit program.The default setting for this parameter is *NONE. You cannot specify *NONE for this parameter if you specify *EXIT for the Data Conflict Winner parameter.

• Conflict User Exit Library—Type the library where the user exit program resides must be identified if you do not specify *NONE in the Conflict User Exit box above.

7 You can specify the following general properties on this dialog box:• Include or Exclude—Indicate whether the objects referenced by the object specifier will be replicated within the group.

Possible values:• *INCLUDE—Specifies that the referenced objects will be replicated within the replication group when cluster operations

are started.• *EXCLUDE—Specifies that the referenced objects will not be replicated within the replication group when cluster

operations are started.The default value for this parameter is *INCLUDE.

• Activation of New Objects—Select the starting replication method for objects that come into replication scope when an object specifier is added. The possible values are: • *NONE—This value does not change the replication status of new, in-scope objects, and is intended to support initial

group configuration. If this value is selected, mirroring must be started with a refresh of the entire group, or with the Mark Journal Positions or Set Journal Positions commands. The group's replication status cannot be *ACTIVE if you use this value.

• *CURRENT—Replication of journal entries for new, in-scope objects will begin at the time the object specifier is added. Journal entries related to newly in-scope objects created after that time will be replicated. Journal entries related to new, in-scope objects created before that time will not be replicated. The group's replication status must be *ACTIVE for this option to take effect.If this value is selected, no changes should occur on the new, in-scope objects until the OMI0320 event for the object specifier appears in the event log.

• *REFRESH—Newly in-scope objects will be refreshed one at a time. Replication of journal entries for new, in-scope objects begins for each object as it is refreshed. The group's replication status must be *ACTIVE for this option to take effect.

The default setting for this parameter is *NONE.• Description—Specify a short description that allows you to identify this object specifier selection among all others that have

been defined in iCluster.The description can be a maximum of 50 characters long, and is an optional setting.

8 Click Execute.On the iCluster Administrator window, the attributes of the object specifier are shown on the table view.


Deselect Object SpecifierUse this command to de-select an object specifier from the selected replication group (full replication, refresh-only, and master-master). De-selecting an object specifier from a replication group prevents referenced objects from being replicated within the group.You must issue this command on an active node in the cluster. The specified replication group must be inactive when you issue this command.Note: This command does not apply to resilient applications or MQSeries groups.

To deselect an object specifier1 From the iCluster Administrator window, expand the Replication Groups heading.2 Under the Full Replication, Refresh-Only, or Master-Master heading, expand a group and select the Object Specifiers

heading.3 In the iCluster table view, select the object specifier that you want to deselect, right-click and select Deselect Object

Specifier.The Deselect Object Specifier dialog box opens.

The values in this dialog box identify the object specifier that you selected on the iCluster Administrator window. You cannot modify any of the boxes.

4 Click Execute.On the iCluster Administrator window, the object specifier is removed from the table view. This indicates that the object specifier has been deselected from the replication group.

Change Object SpecifierUse this command to change specific attributes of an existing object specifier for the selected replication group (full replication, refresh-only, and master-master



Through this command, you can change the description currently associated with the object selection, modify the flag that determines whether or not the referenced objects are replicated within the group, and change the polling interval for user spaces. The journaling option for BSF objects matching this specifier can be changed with this command, as can the target library for native objects.Note: This command does not apply to resilient applications or MQSeries groups.

To change an object specifier1 From the iCluster Administrator window, expand the Replication Groups heading.2 Under the Full Replication, Refresh-Only, or Master-Master heading, expand a group and select the Object Specifiers

heading.3 In the iCluster table view, right-click an object specifier and select Change Object Specifier.

The Change Object Specifier dialog box opens.

This dialog box has three tabs:• Select Native Object Specifier if the objects you want to replicate are stored in the native OS/400 file system. See

Step 4. See Object Specifiers and Types on page 50 for more information.• Select Path Object Specifier if the objects you want to replicate are stored in the Integrated File System (IFS), select

Path Object Specifier. See Step 5. For more information, see Path Object Specifiers on page 53.• Conflict Resolution, see Step 6.• General properties, see Step 7.

4 Select the Native Object Specifier to specify the following values:• Target Library—Specify the name of the library on the backup system that will receive the replicated objects.

Special values:• *GROUP—Specifies the same target library as specified in the Add Full Replication Group or Change Full Replication

Group dialog boxes.


• *PRIMARY—Sets the target library so that it is the same as the primary node library where the object resides.The default value for this parameter is *GROUP.If the primary and backup environments reside on the same physical system (local loopback implementation), the target library that you specify cannot be the same library where a selected object in the group currently resides. This restriction means that iCluster does not permit the special value *PRIMARY in this situation.


• Polling Interval—Specify the time interval that determines how often iCluster should check for content changes to user spaces.You set the interval by specifying the number of hours (first two digits), minutes (middle two digits), and the seconds (last two digits) between consecutive polls.Possible values:• *GROUP—Refers to the corresponding replication group value that is specified when adding or changing a replication

group.• *CLUSTER—Uses the value specified for the corresponding system parameter (Default polling interval). See Setting


The default value for this parameter is *GROUP.You can now proceed to Step 7 to set the general properties on this dialog box.5 Select Path Object Specifier to specify the following values:• Path—Specify the location of the path object specifier that you are defining.

Path object specifiers identify the location of Byte Stream File (BSF) objects in the Integrated File System (IFS) or QDLS file system. See Path Object Specifiers on page 53 for more information about correctly identifying the path.

• Journal Name—Specify how BSF objects referenced by the path object specifier will be replicated with the replication group.Possible values:• *GROUP—BSF objects matching this object specifier will use the journal specified at the group level. See Add Full

Replication Group for more information.• *NONE—Specifies that BSF objects matching this object specifier will not be journaled by iCluster. Only object-level

changes to objects matching this specifier will be replicated. Changes to the contents of the objects matching this specifier will not be replicated unless the entire object is replaced.

The default value for this parameter is *NONE.Note: The overlap of journaled and non-journaled path object specifiers is restricted in iCluster. For example, using


JOURNAL parameter of any path specifier matching QDLS.• Polling Interval—Indicate whether or not BSF objects will be polled.

Possible values:• *GROUP—Indicates that BSF objects will use the corresponding replication group value that is specified when adding or

changing a replication group. See Add Full Replication Group or Change Full Replication Group for more information.• *NONE—Indicates that the BSF objects will not be polled.

The default value for this parameter is *GROUP.



Note: Note that it is only possible to poll BSF objects in the QDLS folder.6 Select Conflict Resolution to specify the following values:Note: The parameters on this tab only apply to iBalance and Master-Master replication groups. For more

information on this feature, see iBalance and Master-Master Replication on page 321.• Data Conflict Winner—Select the group-level rules by which iCluster will resolve conflicts for the objects selected to a

master-master replication group.Possible values:• *GROUP—Indicates that the object specifier inherits conflict resolution rules and the user exit program configured at the

group-level. For more information, see Add Master-Master Group.Note: For path object specifiers, if you specify *GROUP for this value and then set the Data Conflict Winner value at

the group level to *TGT, iCluster will enforce the *NONE conflict resolution rule for all objects matching this specifier.

• *SRC—Indicates that the source wins if there is a conflict. Source data is applied to the target, which ensures that target data matches source data and mirroring continues.

• *TGT—Indicates that the target wins if there is a conflict. iCluster does not apply data to the target. This rule preserves data on the target and mirroring continues.

Note: This value is not valid for path object specifiers.• *EXIT—Indicates that you would like to resolve data conflicts through a user exit program. With this option, you must

specify a user exit program and library in the parameters below.• *NONE—Indicates that you do not want to enable conflict detection and resolution rules. If there is a conflict, iCluster


• Conflict User Exit—Select or type the name of the user exit program that you want to call to resolve data conflicts. A user exit program allows you to define a conflict resolution strategy that is not covered by one of the standard resolution methods in the Data Conflict Winner box.Specify the name of a user exit program or the following value:• *NONE—Indicates that you do not want to specify a user exit program.

See Parameter List Descriptions for a Conflict Resolution User Exit Program for more information about the values you can pass to the user exit program.The default setting for this parameter is *NONE. You cannot specify *NONE for this parameter if you specify *EXIT for the Data Conflict Winner parameter.

• Conflict User Exit Library—Type the library where the user exit program resides must be identified if you do not specify *NONE in the Conflict User Exit box above.

7 You can specify the following general properties on this dialog box:• Include or Exclude—Indicate whether the objects referenced by the object specifier will be replicated within the group.

Possible values:• *INCLUDE—Specifies that the referenced objects will be replicated within the replication group when cluster operations

are started.• *EXCLUDE—Specifies that the referenced objects will not be replicated within the replication group when cluster

operations are started.The default value for this parameter is *INCLUDE.

• Description—Specify a short description that allows you to identify this object specifier selection among all others that have been defined in iCluster.


The description can be a maximum of 50 characters long, and is an optional setting.8 Click Execute.On the iCluster Administrator window, the attributes of the object specifier are shown on the table view.

iCluster Operations CommandsThis section contains the following topics:• Mark Journal Positions on page 496• Set Journal Positions on page 497• Set Synchronization Point on page 498• Set Primary Node on page 502• Change Role on page 502• Start Apply Process on page 503• End Apply Process on page 504

Mark Journal PositionsUse this command to mark the current journal positions for the specified replication group or resilient application. The journal positions are journaled in the iCluster metadata on the primary node associated with the specified group or resilient application. Marking a journal position takes a snapshot of the state of the primary system and the objects that match object specifiers at the time you issue this command.Marking journal positions can help prevent unsynchronized startups when starting mirroring. A setting in the Start Full Replication Group, Start Master-Master Group, and Start Resilient Application allows you to start object and data mirroring at the journal positions marked by this command. Therefore, this command is typically used to record starting points before starting replication.This command will journal any objects (that have not already been journaled) that match object specifiers for the group to the default journal for the group. To journal objects to a journal other than the default journal, it is your responsibility to start journaling for the objects prior to running this command.Note: Each time you issue this command, the results from a previous invocation are overwritten. Only those

replication groups or resilient applications that are specified in the current invocation will have their journal positions overwritten in the iCluster metadata.

The minimum security level for this command is Operator (*OPERATOR).

To mark current journal positions1 From the tree view in the iCluster Administrator window, select the replication group or resilient application that will have its

current journal positions marked by this command.2 Display its shortcut menu, and select Mark Journal Positions.

The Mark Journal Positions dialog box opens.


iCluster Operations Commands

3 Click Execute.

Set Journal PositionsUse this command to position iCluster to start replication at a specific entry in a database or system audit journal on the full replication group, master-master group, or resilient application's primary node.You can enter a specific entry or it can be determined by the command according to the specified options.You must issue this command when the full replication group is inactive or the resilient application is not running.This command does not apply to MQSeries groups.The minimum security level for this command is Operator (*OPERATOR).

To set journal positions1 From the iCluster Administrator window, select the full replication group or resilient application that will have its journal position

set by this command.2 Display its shortcut menu, and select Set Journal Position.

The Set Journal Position dialog box opens.

You can specify the following values on this dialog box.• Journal Name—Specify the name of a database or system audit journal that will have its starting position set through this

command.


• If you specify a database journal, then the database and audit journal starting positions are set for both physical and logical files selected for replication.

• If you specify the system audit journal, then the audit journal starting position is set for all objects other than physical or logical files selected for replication.

• Journal Library—Specify the name of the library where the database or system audit journal currently resides.• Start Journal Receiver—Specify the name of a database or system audit journal receiver.

Special values:• *CURRENT—Specifies the current journal receiver for the specified journal.• *CURCHAIN—Specifies the journal receiver that is determined from the timestamp information of the starting entry that

is provided through the Start Date and Start Time settings (see below).If you specify *CURCHAIN, you cannot specify a starting sequence number in the Journal Start Position box (see below). In this case, you can set only the Start Date and Start Time values.The default value for this parameter is *CURRENT.

• Journal Receiver Library—Specify the name of the library where the database or system audit journal receiver currently resides.If you selected *CURCHAIN in the Start Journal Receiver box, a library name is not required.Special value:• *LIBL—Specifies the set of libraries in your library list. The libraries are searched in order for the first occurrence of the

specified journal receiver.The default value for this parameter is *LIBL.

• Journal Start Position—Specify the sequence number of the entry in the journal that iCluster will start processing from when replication resumes.You cannot specify this setting when at least one of the following is true:• When a value is specified for the Timestamp parameter (see below).• When Start Journal Receiver (see above) is set to *CURCHAIN.

Special values:• *LASTAPY—Start journal processing at the last journal entry in the source journal that was applied to the target.• *CURSRCPOS—Start journal processing at the last journal entry in the source journal.

The default value for this parameter is *LASTAPY.• Timestamp—Specify the date and time when the apply processes will be stopped.

If you specify a timestamp, you can set the Start Journal Receiver box (see above) to special values.3 Click Execute.

Set Synchronization PointUse this command to place entries in journals that will define synchronization points for the journal scrape processes on the primary node and the journal entry apply process on the backup node. You can invoke a user exit program when replication jobs arrive at the journal entries.A replication job is any iCluster replication job associated with group or resilient application activities. This includes (but is not limited to) the jobs OMGROUPJOB, HADDJS (database journal scrape), HADSFP (save file process), and HADTUP (target update process).This command does not apply to MQSeries groups.



You can use the synchronization point journal entries when it is necessary to synchronize operations on primary and backup nodes. After defining a synchronization journal entry on the primary node, synchronization is achieved when the synchronization point journal entry is reached on the backup node. You can then call a user exit program to perform some operation with the user-defined data that is passed to the program. For example, you may want to backup a consistent database or perform summary calculations after all entries have been replicated from the primary node and applied on the backup node.A replication job on a backup node processes journal entries until it reaches the synchronization journal entry that is generated by this command. At this point, the job waits for the group's other replication jobs to reach this synchronization point journal entry. Synchronization is achieved when all active replication jobs reach the synchronization point journal entry. This means that one or more jobs will wait at the synchronization point journal entry until all replication jobs for the group reach the synchronization point. You should note the following important considerations concerning this command:• After setting a synchronization point, you cannot delete or change the synchronization point journal entry you defined.

Therefore, it is important that you are aware of this consideration and exercise caution before using this command.• Normal mirroring operations will not be active during the time that the user exit program is running. Mirroring resumes after

the user exit program runs to completion, unless a special value is returned through one of the user exit program arguments. For more information, see Passing Arguments to Sync Point User Exit Programs on page 86.

You may need to minimize the execution time of your user exit program so that mirroring can resume as soon as possible.You can issue this command only when the group is active or the resilient application is running.The minimum security level for this command is Operator (*OPERATOR).

To set a synchronization point1 From the iCluster Administrator window, select the full replication group, master-master group, or resilient application that will

have synchronization points defined by this command.2 Display its shortcut menu, and select Set Synchronization Point.

The Set Synchronization Point dialog box opens.


You can specify the following values on this dialog box:• Synchronization Point Exit Program—Specify the name of the user exit program that will be invoked when all active

replication jobs reach the synchronization point journal entry.You can invoke the same user exit program at different synchronization points (journal entry scrape, receive, and apply). If it is invoked when journal entries are being scraped, the program must reside on the primary node. If it is invoked at journal receive or apply times, it must the user exit program reside in the same location on the backup node. If you want to specify different user exit programs at each synchronization point, use this command multiple times (one time for each synchronization point) so that you can specify a different user exit program.If iCluster cannot find the specified user exit program, an appropriate error message will be generated and mirroring will continue.

Note: You must compile user exit programs with the Use adopted authority option set to *NO.Special value:• *NONE—Indicates that you do not want to specify a user exit program.

There is no default value for this parameter.For more information, see Passing Arguments to Sync Point User Exit Programs on page 86.

• Synchronization Point Exit Library—Specify the name of the library where the specified user exit program currently resides.If you specified *NONE in the Synchronization Point Exit program box, a library name is not required or is ignored if it is specified.Special value:• *PRODLIB—Specifies your iCluster installation library.

There is no default value for this parameter.



• Synchronize at Scrape—Select whether you want to synchronize replication jobs when journal entries are scraped on the primary node.At this synchronization point, the user exit program specified in the Synchronization Point Exit Program box (see above) will be invoked when all active replication jobs for the primary node scrape process synchronize at the synchronization point journal entry.You can use a user exit to end mirroring by setting a return value. Setting this value for either the scrape or receive affects both the scrape and receive processes. The apply processes are not affected.Possible values:• *NO—Does not synchronize group replication jobs at synchronization point journal entry, meaning that the user exit

program is not invoked.• *YES—Synchronizes group replication jobs at synchronization point journal entry and then invokes user exit program.

The default value for this parameter is *NO.• Synchronize at Receive—Select whether you want to synchronize replication jobs when journal entries are received on the

backup node.At this synchronization point, the user exit program specified through the Synchronization Point Exit Program box (see above) will be invoked when all active replication jobs for the backup node receive process synchronize at the synchronization point journal entry.You can use a user exit to end mirroring by setting a return value. Setting this value for either the scrape or receive affects both the scrape and receive processes. The apply processes are not affected.Possible values:• *NO—Does not synchronize group replication jobs at synchronization point journal entry, meaning that the user exit

program is not invoked.• *YES—Synchronizes group replication jobs at synchronization point journal entry and then invokes user exit program.

The default value for this parameter is *NO.• Synchronize at Apply—Indicate whether you want to synchronize replication jobs when journal entries are applied on the

backup node.At this synchronization point, the user exit program specified through the Synchronization Point Exit Program box (see above) will be invoked when all active replication jobs for the backup node apply processes synchronize at the checkpoint journal entry.You can use a user exit to end mirroring by setting a return value. Setting this value for the apply processes affects only the apply processes.Possible values:• *YES—Synchronizes group replication jobs at synchronization point journal entry and then invokes user exit program.• *NO—Does not synchronize group replication jobs at synchronization point journal entry, meaning that the user exit

program is not invoked.The default value for this parameter is *YES.

• Exit Program Data—Specify the user-defined data that you want to pass to the user exit program specified through the Synchronization Point Exit Program box (see above).You can pass a maximum of 400 bytes of data to the user exit program. If your data contains spaces or non-alphanumeric characters (commas, periods, and so on), you must enclose your data in single quotes.

3 Click Execute.


Set Primary NodeUse this command to prepare a primary node in a full replication group, resilient application, or MQSeries group to be the source node for replication. The minimum security level for this command is Operator (*OPERATOR).

To set a primary node1 From the tree view in the iCluster Administrator window, select the full replication group, master-master group, resilient

application, or MQSeries group that will have its primary node prepared for replication by this command.2 Display its shortcut menu, and select Set Primary Node.

The Set Primary Node dialog box opens.

3 Click Execute.

Change RoleUse this command to change the primary node to the current first backup node for the specified full replication group, resilient application, or MQSeries group. The group or resilient application will remain *INACTIVE until you start it with the Start Replication Group or Start Apply Process command.You must issue this command when the selected group or resilient application is *INACTIVE.The current backup will be prepared to become the primary node in the same way as would occur if a failover had happened when the group was active.Note: A message (CSI1314) stating whether or not the role switch completed successfully is placed into the event

log on all nodes of the cluster. For more information about event logs, see Starting the iCluster Event Viewer on page 510.


To change the role of the primary node1 On the iCluster Administrator window, select the full replication group, MQSeries group, or resilient application in the tree view

that will have the primary node changed to the current first backup node.2 Display its shortcut menu, and select Change Role.

The Change Role dialog box opens.



3 Click Execute.The specified group's or resilient application's primary node is now changed to the current first backup node.

Start Apply ProcessUse this command to start the apply processes for a replication group (full replication and refresh-only), resilient application, or MQSeries group on a backup node. Starting apply processes allows journal entries placed in the staging store by the receive processes to be applied. For more information about staging stores, see Staging Objects on page 69.The apply processes handle each journal entry in the staging store in chronological order. They remain active until the stop date/time specified through the previous or subsequent request to end the processes. See End Apply Process for more information. The apply processes also end when the staging store has been drained and the send/receive processes are not running.The minimum security level for this command is Operator (*OPERATOR).The replication group must be active or the resilient application must be running when you issue this command.

To start the apply process1 From the iCluster Administrator window, select the full replication group, refresh-only group, master-master group, resilient

application, or MQSeries group that will have apply processes on the backup node started by this command.2 Display its shortcut menu, and select Start Apply Process.

The Start Apply Process dialog box opens.

You can specify the following values on this dialog box:• Override Current Stop Time—Select whether you want to override a previously set stop date/time for the apply processes

that was specified through the command that ends the apply processes. See End Apply Process for more information.Possible values:


• *YES—Specifies that the previous stop date/time is disregarded. The apply processes will continue until the End Apply Process command is invoked again.

• *NO—Specifies that the apply processes will start, but the previous stop date/time specified through this command is still valid. Consequently, the processes will stop at that date/time unless it is reset by a subsequent invocation of the End Apply Process command.

The default value for this parameter is *YES.• Drain Staging Store—Indicate whether the staging store will be force drained.

The apply processes merge entries from the audit journal and database journal. Depending on your selection, the apply processes will either continue draining the staging store even if one of the journal channels is empty (known as force draining), or it will stop draining when one of the journal channels is empty.Force draining stops once it reaches the stop date/time specified through the End Apply Process command (if this command has been invoked). Otherwise, it will stop once the staging store is empty.Possible values:• *NO—Indicates that the staging store will not be force drained. The apply processes will merge entries from the audit

and database journals and apply them in sequence. When one journal becomes empty, the apply processes will stop regardless of any entries remaining in the other journal.

• *YES—Indicates that the staging store will be force drained. The apply processes will merge entries from the audit and database journals. Once it reaches the last entry of the journal with fewer entries, the merge will stop, and the apply processes will drain the other journal until it is empty.

The default value for this parameter is *NO.3 Click Execute.

End Apply ProcessUse this command to end the journal entries applied from the staging store on a backup node at the specified date and time. For more information about staging stores, see Staging Objects on page 69.Even though this command ends the apply processes, iCluster continues to scrape and send journal entries for the replication groups (full replication and refresh-only), resilient applications, or MQSeries groups. The journal entries remain in the staging store until the apply processes are started through the Start Apply Process command.The group must be active or the resilient application must be running when you issue this command.The minimum security level for this command is Operator (*OPERATOR).

To end the apply process1 From the iCluster Administrator window, select the full replication group, refresh-only group, master-master group, resilient

application, or MQSeries group that will have apply processes on the backup node ended by this command.2 Display its shortcut menu, and select End Apply Process.

The End Apply Process dialog box opens.


Suspended Object Commands

You can specify the following values on this dialog box:• End Apply at Node—Specify the name of the backup node in the recovery domain for the specified group or resilient

application.You can set this box only to *ALL.

• Option—Select how you want to end the apply processes.Possible values:• *CTRLD—Indicates a controlled end of the apply processes, allowing them to complete their current processing. This is

the recommended setting for this parameter.• *INVLD—Invalidates the pending end date/time for the apply processes. Applies will continue.• *DATETIME—Indicates the date and time when the apply processes will end. If you select this setting, you must specify

values in the Timestamp box (see below).• *IMMED—Indicates that the apply processes will end immediately.

The default value for this parameter is *CTRLD.Note: If you select either *CTRLD or *DATETIME, the apply processes will end once the current operation has been

completed. Depending on the operation (for example, restoring a save file), it may take some time for the apply processes to end.

• Timestamp—Specify the date and time when the apply processes will be ended.This box is only applicable when the Option box (see above) is set to *DATETIME.

3 Click Execute.

Suspended Object CommandsThis section contains the following topics:• Activate Object on page 505• Suspend Object on page 507

Activate ObjectUse this command to activate native and BSF objects that are currently suspended. For more information about BSF objects, see Replicating BSF Objects on page 89.


After activating a native or BSF object, mirroring of the specified object will start if the replication group is active. If the replication group is inactive, mirroring will start when the group is activated through the Start Full Replication Group or Start Master-Master Group command.Note: Any changes made to an object while it is suspended will not be mirrored when it is activated through this

command. You should verify that the object on the backup node is synchronized with the object on the primary node before activating the object for mirroring, or set the Refresh Object to Recovery Domain box in this command to *YES.

For more information about suspended objects, see Suspended Objects on page 72.The minimum security level for this command is Operator (*OPERATOR).

To activate an object1 From the iCluster Administrator window, expand the Replication Groups heading.2 Under the Full Replication or Master-Master heading, expand a group and select the Suspended Objects heading.3 In the iCluster table view, select the object that you want to activate, right-click and select Activate Object.

The Activate Object dialog box opens.

See Suspend Object for an explanation of the boxes in the Native Object or Path Object options.You can specify the following values on this dialog box:

• Refresh Object to Recovery Domain—Select whether you want to refresh the suspended object on the backup node in the replication group when the file is activated. The refresh method used is the one that was specified when the object was selected for replication within the group. See Select/Add Object Specifier for more information.This box allows you to activate an object without having to save and restore the object.Possible values:• *NO—Indicates that you do not want the file to be refreshed on the backup node when the object is activated. In this

case, you are responsible for refreshing the object.


Suspended Object Commands

• *YES—Indicates that you want the suspended object to be refreshed on the backup node when the object is activated.The default value for this parameter is *YES.

Note: BSF objects are always refreshed regardless of whether this box is set to *YES or *NO.• Truncate Data—Specify whether you want to remove existing data before performing a refresh.Note: This parameter only applies to iBalance and Master-Master replication groups. For more information, see

iBalance and Master-Master Replication on page 321.Specify one of the following values:• *YES—Indicates that you want to remove and recreate existing data files as part of a refresh. This is the default behavior

for iCluster and is appropriate for standard iCluster replication. You can use this option for a master-master group if the data on the source system is known to be complete in order to ensure a synchronized starting point.With this value, logical files are replaced automatically because the deletion of a physical file requires the deletion of all logical files upon which it is based.



• Replace Logical Files—Specify whether you want to replace logical files during a refresh.Note: This parameter only applies to iBalance and Master-Master replication groups. For more information, see

iBalance and Master-Master Replication on page 321.Specify one of the following values:• *YES—Indicates that you want to replace logical files during a refresh. This is the default behavior for iCluster and is

appropriate for standard replication. You can use this option for a master-master group if the logical files on the source system are the complete set required for your applications in order to ensure a synchronized starting point.




4 Click Execute.

Suspend ObjectUse this command to prevent an object from being replicated to a backup node in a full replication group by suspending it. For more information about BSF objects, see Replicating BSF Objects on page 89.Journal entries created before an object was suspended will still be sent to the backup node.Note: Any changes made to an object while it is suspended will not be mirrored when it is activated.For more information about suspended objects, see Suspended Objects on page 72.The minimum security level for this command is Operator (*OPERATOR).


To suspend an object1 From the iCluster Administrator window, expand the Replication Groups heading.2 Under the Full Replication or Master-Master heading, expand a group and select the Suspended Objects heading.3 Right-click and select Suspend Object.

The Suspend Object dialog box opens.

You can specify the following values on this dialog box:• Select Native Object if the object you want to suspend is stored in the native OS/400 file system. See Step 4.• Select Path Object if the object you want to suspend is stored in the Integrated File System (IFS). See Step 5.

4 In the Native Object area, you can specify the following values:• Object Name—Specify the name of a valid native object that is selected to the replication group.• Object Library—Specify the name of the library where the specified native object resides.• Object Type—Select the object type that you want to suspend. 5 In the Path Object area, you can specify the following values:• Path—Specify the location of the path object specifier identifying the BSF object you want to suspend.

See Path Object Specifiers on page 53 for more information about correctly identifying the path.6 Click Execute.

iBalance and Master-Master ReplicationThis section contains the following topic:• Configuring Master-Master Replication in iCluster Administrator on page 509


iBalance and Master-Master Replication

Configuring Master-Master Replication in iCluster AdministratorThis topic describes the tasks you must perform to configure master-master replication for two iSeries environments configured on system A and B. This example assumes that you are populating new data between these two systems while preserving the original data on these systems. The examples show the required parameters you must set to perform each step. For a complete list of parameters, see the documentation for each command.Note: Bold indicates values that are specific to master-master replication.

To configure master-master replication1 Create two master-master groups that mirror in opposite directions (A to B and B to A). In the Add Master-Master Group

dialog box, use the following required parameters:

Create the first master-master group:• Group Name: MyGroupAB• Primary Node: NodeA• Backup Node: NodeB

Create the second master-master group:• Group Name: MyGroupBA• Primary Node: Node B• Backup Node: Node AYou can also configure data conflict options and a logging file (on the backup node only) for each master-master replication group. For more information on these options, see Overview of Conflict Detection and Resolution on page 323.2 Next, you must select the object specifiers for each group that you created in the previous step. The object specifiers for each

of the groups must be the same for true master-master replication. The example below mirrors one physical file (LIB1/FILE1). In the Select/Add Object Specifier dialog box, use the following required parameters:

Select the object specifiers for the first group:• Group Name: MyGroupAB• Object Specifier Name: FILE1• Object Specifier Library: LIB1• Object Type: *FILE• Object Extended Attribute: PFDTA• Physical File Update Method: *UKEY• Unique Key Name: *AUTO

Select the object specifiers for the second group:• Group Name: MyGroupBA• Object Specifier Name: FILE1• Object Specifier Library: LIB1• Object Type: *FILE• Object Extended Attribute: PFDTA• Physical File Update Method: *UKEY• Unique Key Name: *AUTO


Keyed replication, Physical File Update Method (*UKEY), is required for physical data files. You must also specify a value for the Unique Key Name parameter. The Unique Key Name (*AUTO) option for master-master replication scans the logical files associated with physical files and selects a uniquely keyed logical files that can be used for mirroring. The logical files must already exist on the target when they are needed for replication (run-time resolution).Note: You can use generics when selecting object specifiers for your groups.For more information on this command and a complete list of parameters, see Select/Add Object Specifier.Note: In addition to the database files configured in this example, you can also configure master-master replication

for non-journaled IFS files and data areas.3 Start both groups with a refresh. The options for master-master replication prevent the refresh from truncating existing data

and replacing or removing existing logical files. In the Start Master-Master Group dialog box, use the following required parameters:

Start the first group with a refresh:• Group Name: MyGroupAB• Start Replication Apply Jobs: *YES• Refresh: Yes• Truncate: *NO• Replace Logical Files: *NO

Start the second group with a refresh:• Group Name: MyGroupBA• Start Replication Apply Jobs: *YES• Refresh: Yes• Truncate: *NO• Replace Logical Files: *NOFor more information on the refresh options for master-master replication groups, see Master-Master Replication—Operations and Monitoring on page 323.As with the steps above, any subsequent operations with master-master replication groups such as ending and re-starting groups are done in pairs.

Viewing EventsThis section contains the following topics:• Starting the iCluster Event Viewer on page 510• Change Filter on page 512• Export Log on page 514• Clear Log on page 515• Set Message Frame Size on page 516

Starting the iCluster Event ViewerThis section explains how to launch the iCluster Event Viewer and describes the functions that are available from the menu and toolbars.


Viewing Events

To start the iCluster Event Viewer:1 Select New Event Viewer from the Window menu2 To examine messages on the Event Viewer window, click Execute on the Change Filter dialog box.

To examine the Event Viewer window without displaying messages, click Cancel on the Change Filter dialog box.For more information about the Change Filter dialog box, see Change Filter.The iCluster Event Viewer window opens.

This window allows you to view event messages that have been generated by iCluster during replication, communication, and cluster activities. The event messages are retrieved from the selected node. If you have not selected a node, then the Event Viewer will display messages for the node that you specified on the iCluster Administrator Login dialog box. If the Event Viewer is already open, then opening a new Event Viewer will switch to the one currently open.

Note: If you want to display messages from a different node in the cluster, you can select the node in the tree view and then select New Event Viewer from the Window menu.

A list of event messages is displayed. You can set the maximum number of event messages that can be displayed in the Event Viewer at one time (the size of the message frame). Through menu bar and toolbar commands (see below), this message frame can be moved up or down to view a different part of the event message list. To control which event messages are displayed in the viewer, you can filter event messages based on type (replication, communication, and cluster), severity, and time period settings. See Change Filter for more information. The movement of the message frame through the list of event messages is always constrained to the time period specified through the Change Filter command. See Set Message Frame Size for information on how to set the size of the message frame.You can display detailed information about a specific message by double-clicking on the message in the list. This action displays second-level text that augments the first-level text that is displayed for each message in the list.


Messages are categorized based on the severity of the condition that caused them to be generated. For those messages that are displayed in the viewer, the total number of messages in each category is provided. See the Message Display Level setting for the Change Filter command for more information about the different severity levels that are presented on the iCluster Event Viewer window.You can remove messages in the log, and switch date/time information between the time zones of the local iSeries node or remote workstation where the iCluster Event Viewer is being displayed.

Change FilterUse this command to filter the event log messages that are displayed or cleared. See Clear Log for more information. Filtering is based on event type, severity, and timestamp attributes.This command is especially useful when you want to limit the number of messages that are displayed or cleared.

To change the event log filter1 From the iCluster Event Viewer window, click Change.

The Change Filter dialog box opens.

Note: This dialog box is automatically displayed when the iCluster Event Viewer is started.You can specify the following values on this dialog box:

• Event Type—Select the type of message that you want to display or clear.


Viewing Events

Possible values:• Replication—Displays or clears all messages that provide information about object replication.• Communication—Displays or clears all messages that provide information about communications between nodes.

You cannot filter communication events by group name.• Cluster—Displays or clears all messages that provide information concerning cluster setup, configuration, and

operations.You cannot filter cluster event messages by group name.• All—Displays or clears all types of messages.

The default value for this parameter is All.• Replication Group Name—Specify the name of a group that will be used to determine which replication event messages are

displayed or cleared.Only replication event messages that address the referenced groups will be displayed or cleared. This box is only applicable when Event Type (see above) is set to Replication or All.Special value:• *ALL—Displays or clears all replication event messages regardless of group.

The default value for this parameter is *ALL.• Communication Event Node Name—Specify the name of the node that will be used to determine which communication

event messages are displayed or cleared.This box is only applicable when Event Type (see above) is set to Communication or All.Special values:• *ALL—Displays or clears communication event messages generated by all nodes.• *LOCAL—Displays or clears communication event messages generated by the system you specified during the log in

process. See Starting the iCluster Event Viewer for more information.The default value for this parameter is *ALL.

• Cluster Event Node Name—Specify the name of the node that will be used to determine which cluster event message are displayed or cleared.This box is only applicable when Event Type (see above) is set to Cluster or All.At this time, only *LOCAL can be specified. This setting displays or clears cluster event messages generated by the system you specified during the login process. See Starting the iCluster Event Viewer for more information.

• Message Display Level—Select the boxes of the message severity levels that you want to be displayed or cleared.When some (but not all) message severity levels are selected, messages in other levels are not displayed or cleared through iCluster Event Viewer. However, all fatal error messages (Level 40) are displayed or cleared.Possible values:• 00—Displays or clears information messages (Level 00).• 10—Displays or clears non-critical status messages (Level 10).• 20—Displays or clears stop/start messages (Level 20). For example, Start Mirroring.• 30—Displays or clears messages that report recoverable errors (Level 30).

Note: It is strongly recommended you initially specify all severity levels. This ensures that you will see all messages being generated. At a later time, after you have defined your clustered environment and have been actively replicating between nodes, you can identify the specific level of messages that are displayed or cleared.

All boxes are checked by default.


• Use Time Filter—Select if you want to filter messages that are displayed or cleared by the timestamp applied to the message.The time interval is defined by the From Time and To Time settings (see below). Messages with timestamps within the specified range will be displayed or cleared in the Event Viewer, while those messages generated outside the time interval are not displayed or cleared.

• Today—Select if you want to automatically set the time in the From Time box (see below) to the current date at 12:00 AM.This selection only affects the current setting in the From Time box, and not the current setting in the To Time box (see below).

• From Time—Specify the earliest timestamp for a message that will be displayed or cleared in the Event Viewer.To change the current date or time, insert the cursor bar in the desired field, and click the arrows to move the setting up or down. Alternatively, select the field you want to change, and type the new date or time component. Any changes that you make in this box will replace the previous automatic setting caused by selecting the Today check box (see above).The timestamp that you specify must be chronologically before the timestamp in the To Time box (see below).The default value for this parameter is the date and time when you issued this command.

• To Time—Specify the latest timestamp for a message that will be displayed or cleared in the Event Viewer.To change the current day, month, year, hours, minutes, seconds, or AM/PM specification, insert the cursor bar in the desired field, and click the arrows to move the setting up or down. Alternatively, select the value that you want to change, and enter the new date or time.The timestamp that you specify must be chronologically after the timestamp in the From Time box (see above).The default value for this parameter is the date and time when you issued this command.

2 Click Execute.

Export LogUse this command to capture event messages selected on the Event Viewer window or the entire log of the node to which you are connected.You can select the format in which the messages are exported, and you can direct output to a file, the clipboard, or the default printer.

To export the event log1 From the iCluster Event Viewer window, select Export from the File menu.

The Export Log dialog box opens.


Viewing Events

You can specify the following values on this dialog box:• Export—Select an option button that indicates you want to capture event messages that are currently selected in the viewer.

Possible values:• Selected Entries• Complete Event Log—All messages in the event log.

If no messages were selected on the iCluster Event Viewer window when you issued this command, you can only select Complete Event Log.

• Export Format—Select an option button to indicate the format that you want to use to capture the event messages.You can capture messages in plain text, HTML, or CSV (comma-separated values) formats.

• Export To—Select an option button to indicate that captured event messages are to be directed to a file, the clipboard, or the printer.If no messages were selected on the iCluster Event Viewer window when you issued this command, the Clipboard option button will be disabled.

2 Click Execute.If you selected File in the Export To group box, the standard Save As dialog box will appear to allow you to save the messages in a file. The file type is dictated by your selection in the Export Format group box.If you selected Printer in the Export To group box, the standard Print dialog box will appear to allow you to select a printer in your local network.

Clear LogUse this command to delete all event messages based on the current filter settings. See Change Filter for more information about changing filter settings.

To clear the event log1 From the iCluster Event Viewer window, select Clear Log from the Command menu.

The Clear Log message box is displayed to confirm your decision to delete all event messages based on current filter settings.2 Click Execute to remove certain messages or Cancel to maintain all existing messages.


Set Message Frame SizeBy default, the maximum number of messages that can be listed on an iCluster Event Viewer window is 200. This number represents the message frame size, but you can change it by altering a setting contained in an iCluster file (icProfile.properties). This file is created after closing the iCluster Administrator for the first time, and it is placed in your iCluster installation folder. See Installing and Upgrading iCluster Administrator for more information. Therefore, you cannot change the message frame size setting prior to using iCluster Administrator for the first time.

To set the message Frame Size1 Using Notepad or a different text editor, open the icProfile.properties file in your iCluster installation folder.2 Locate the line in the file containing the following setting:

EvDspBufSize=<number>

If you have not modified this file, <number> will be 200.3 Change <number> to the maximum number of messages that you want to list on an iCluster Event Viewer window.4 Save and close the file to establish the change to the setting.This change takes effect when you open new Event Viewer windows.

Detailed MessageThis dialog displays first and second level information that describes and explains the message that was selected on the iCluster Event Viewer window.When contacting DataMirror Technical Support, reference the message identifier and provide the information that is presented on the Message Text tab.Two buttons (Prev and Next) allow you to display details of the previous or next message in the log without having to return to the iCluster Event Viewer window. This allows you to scan the messages quickly in sequence.The Prev button is disabled when the first message in the log is open, and the Next button is disabled when the last message in the log is open.

Monitoring iClusterThis section contains the following topics:• Using the Backup Monitor on page 516• Using the Object Status Monitor on page 518• Using the iCluster Performance Monitor on page 520

Using the Backup MonitorThe Backup Monitor provides the following information for each group:• Latency and status information• Suspended objects• Out-of-sync objects• Overall transactions per hour• Time of the last status change


Monitoring iCluster

You can use this information to activate or suspend objects, and ensure synchronization between the primary and backup systems.

To use the Backup Monitor1 Select the backup node, display the pop-up menu, and select Backup Monitor.

The Backup Monitor opens.

The Backup Monitor displays the following information:• Backup node—The name of the backup node that is being monitored.• Group—The name of each group on the backup node that is being monitored.• Suspended Objects—The number of suspended for each group. These numbers are displayed in red if there are

suspended objects.


• Out-of-sync Objects—The number of out-of-sync objects for each group. These numbers are displayed in red if there are out-of-sync objects.

• Overall Transactions Per Hour—The overall transactions per hour. This is the number of transactions per hour based on the elapsed time of the backup node apply job.

• Send/Receive Latency—The send/receive latency for each group in HHH:MM:SS format. This is the time difference between the last journal entry in the primary (source) node and the last journal entry that has been received and put into the staging store on the backup node.

• Apply Latency—The apply latency for each group in HHH:MM:SS format. This is the time difference between the last journal entry put into the staging store and the last journal entry applied on the backup node.

• Status—The status of each group including roleswitch states.• Time of Last Status Change—The time of the last status change.

2 Place the cursor over any group name and display its pop up menu to launch the Performance Monitor for that group.See Performance Monitor Display for more information.The Status column displays the group status in green for *ACTIVE groups, red for *INACTIVE groups, and yellow for all other statuses. Roleswitch states are also displayed in this column and are as follows:• *SWOSTART—Indicates that iCluster is starting switchover processing.• *BEFUEXIT—Indicates that iCluster is executing the before roleswitch user exit.• *STSDRAIN—Indicates that iCluster is draining the staging store.• *STRJRN—Indicates that iCluster is starting journaling on the journaled objects in replication scope.• *TRIGGERS—Indicates that triggers are being enabled on the new primary machine.• *CONSTR—Indicates that iCluster is enabling the referential integrity constrains on the new primary machine.• *CHGROLES—Indicates that iCluster is changing the roles of the primary and backup nodes.• *MRKPOS—Indicates that iCluster is marking the starting position for mirroring on the new primary node (MRKPOS

command).• *AFTUEXIT—Indicates that iCluster is executing the after roleswitch user exit.

Note: When a roleswitch is in progress, you will have to display the monitor for the original backup node (now the primary node) for accurate status information.

3 Place the cursor over any of the suspended object or out-of-sync object counts and display its pop up menu to launch the Object Status Monitor.See Using the Object Status Monitor for more information.

4 To manually update the information that is displayed in the Backup Monitor, press F5, or select Manual Refresh from the View menu.

Note: You can change the refresh rate for the Backup Monitor by selecting Options from the View menu.5 To reset the overall transactions per hour, press F10 or select Reset Transaction Throughput in the View menu. This value

is calculated automatically when the Backup Monitor is opened.6 To exit the Backup Monitor, select Close from the File menu.

Using the Object Status MonitorThe Object Status Monitor displays status information for out-of-sync and suspended objects for each group on the primary or backup node.From the Object Status Monitor you can activate or suspend objects.


Monitoring iCluster

To activate or suspend objects from the Object Status Monitor1 From the Backup Monitor window, select an out-of-sync or suspended object (in red), display its pop up menu, and select

Object Status Monitor.The Object Status Monitor window opens.

The Object Status Monitor displays status information about suspended and out-of-sync objects for a selected group:• Summary Information—Number of suspended objects on the primary node, backup node, and the number of out-of-

sync objects.• Object Name—Name of the object• Object Library (Primary)—Object library on the primary node• Object Library (Backup)—Object library on the backup node• Object Type—The type of object• Suspended At—The location where the object was suspended• Reason for Suspension—The reason for the suspension of the object.

See Working with Object Status on page 306 for more information on the suspension reason codes that are displayed here.


• OOS Reason—The reason why the object was declared out-of-sync (OOS). See Working with Object Status for Groups on page 312 for more information on the OOS reason codes that are displayed here.

• OOS Date-Time—The date and time when the object was declared out-of-sync• Refresh Size (kilobytes)—The size of the suspended object in kilobytes

2 Select an object, display its pop up menu, and you can choose to activate or suspend the object. See Activate Object or Suspend Object for more information.

3 To exit the Object Status Monitor, select Close from the File menu.

Using the iCluster Performance MonitorThe Performance Monitor displays status, latency, and other information for all journals used by a group on the primary and backup nodes.

To launch the Performance Monitor for a group1 Right-click a replication group (full replication, refresh-only, master-master), MQSeries group, or resilient application (*REPL

group type) and select Performance Monitor.The Performance Monitor opens.

Note: You can repeat this process to open multiple instances of the Performance Monitor.For an overview of the information displayed in the Performance Monitor, see Performance Monitor Display.

Performance Monitor DisplayThe following figure identifies the key elements of the Performance Monitor.


Monitoring iCluster

This section describes the key elements of the Performance Monitor display:

Group [1]Name of the group for which latency information is displayed in the Performance Monitor.

Primary Node [2]Name of the primary node for the group.

Backup Node [3]Name of the backup node for the group.

Out-of-sync Objects [4]The sum of OS/400 native objects and IFS objects that are out-of-sync for each group. The Performance Monitor displays a green number if the number of out-of-sync objects is 0, otherwise the number is red.Note: This information is current as of the last sync-check.

Suspended Objects [5]The number of suspended objects in the group. The Performance Monitor displays a green number if the number of suspended objects is 0, otherwise the number is red.

Receive Process [6]The arrow indicates the direction of data flow as well as the current status of the receive process for the group in a tooltip.


Note: Place your cursor on the receive process arrow and display the pop up menu to start a replication group or end a replication group. The available commands will vary depending on the type and state of the group that is displayed in the Performance Monitor.

Table 1 identifies and describes the different replication status symbols for the receive and apply processes (see above) that are displayed in the Performance Monitor.

Apply Process [7]The arrow indicates the direction of data flow as well as the current status of the apply process for the group in a tooltip.Note: Place your cursor on the apply process arrow and display the pop up menu to start a replication group, end a

replication group, start the apply process, or end the apply process. The available commands will vary depending on the type and state of the group that is displayed in the Performance Monitor.

Table 1

Symbol Color Status Description

Green *ACTIVE Receive Process: Indicates the journal scrape and receive processes for the group are active between the primary and backup node.Apply Process: The replication status of the apply process for the group is active on the backup node.

Red *INACTIVE Receive Process: Indicates the journal scrape and receive processes for the group are inactive on the primary and backup node.Apply Process: The replication status of the apply process for the group is inactive on the backup node.

Yellow and Red Gradient *INDOUBT Receive Process: Indicates that some, but not all, of the journal scrape and receive processes for the group are running.Apply Process: Indicates that some, but not all, of the apply processes are running.

Red *UNKNOWN Receive Process: Indicates the replication status of the journal scrape and receive process for the group cannot be determined.This status may arise if the primary node in the group is currently inactive or if there is no backup node in the group.Apply Process: The replication status of the apply process for the group cannot be determined on the backup node.This status may arise if the primary node in the group is currently inactive or if there is no backup node in the group.


Monitoring iCluster

Source Status and LatencyFor the specified group, this area of the Performance Monitor displays the names of the journals, journal statuses, and latency information for all journals on the primary node.

Journal Name [8]The names of the journals on the primary node associated with the group.

Journal Status [9]Table 2 identifies and provides a brief description for the different journal status symbols that appear in the Performance Monitor.Table 2

Source Receive Latency [10]The source receive latency of the group's journal scraper and receive process.Source receive latency is the difference between the timestamp of the journal entry last processed by the backup receiver for the journal, and the timestamp of the last journal entry deposited in the journal on the primary node. It is displayed in HHH:MM:SS format.You can display the timestamp of the last journal entry deposited in the journal on the primary node. From the View menu in the Performance Monitor, select Time Display and then the following option:


Green *ACTIVE The journal scraper is active on the primary node.

Red *INACTIVE The journal scraper is inactive on the primary node.

Yellow and Red Gradient *INDOUBT The journal may no longer be used by the group.

Red *UNKNOWN The status of the journal scraper is unknown.


• Time Stamp—The time stamp of the last entry deposited in the journal on the primary node. The time stamp is displayed in HH:MM:SS format.

Note: The Total Latency and Apply Latency options do not apply to the primary node.

Staging Store [11]The percentage of the maximum staging store size that has been filled.

Target Status and LatencyFor the specified group, this area of the Performance Monitor displays the names of the journals, journal statuses, current operations and objects, and latency information for all journals on the backup node.

Journal Name [12]The names of the journals on the backup node associated with the group.

Journal Status [13]Table 3 identifies and provides a brief description of the different journal status symbols that appear in the Performance Monitor.


Monitoring iCluster

Table 3

Current Operation [14]The current operation for each journal is displayed next to the journal status. For each of the current operations in Table 4, you can display the current object being processed in a tooltip by placing your cursor over the journal status or current operation. If you are refreshing the object, the progress of the refresh is displayed as a percentage in the tooltip. Table 4 identifies and provides a brief description of the different operations that appear in the Performance Monitor.


Green *ACTIVE The apply process for the journal is active on the backup node.

Red *INACTIVE The apply process is inactive on the backup node.

Yellow and Red Gradient *INDOUBT The journal may be unused on the backup node.

Red *UNKNOWN The status of the journal is unknown on the backup node.

Table 4

Current Operation Description

*MIRROR Mirroring the current object.

*REFRESH Refreshing the current object.Note: If you want to determine the progress of the object

refresh, right-click *REFRESH to open a pop up menu and select Details. The Refresh Details dialog box displays the progress of the refresh as a percentage.


Target Latency [15]Indicates the latency of the journal processes on the backup node.Apply latency is the difference between the timestamp of the journal entry last processed by the backup apply process for the journal, and the timestamp of the last journal entry processed by the backup receive process. It is displayed in HHH:MM:SS format.On the backup node, you can display latency in several different formats. From the View menu in the Performance Monitor, select Time Display and then one of the following three formats for displaying latency information:• Total Latency—The difference between the timestamps of the last journal entry put into the primary node and the last journal

entry applied on the backup node. This is the default setting.• Apply Latency—The difference between the timestamps of the last journal entry put into the staging store and the last journal

entry applied to the backup node.• Time Stamp—The time stamp of the last journal entry applied on the backup node. The time stamp is displayed in

HH:MM:SS format.

Note: If latency is above the target apply threshold, a warning icon is displayed. See LATENCY System Values on page 78 or Setting System Values on page 386 for more information on the target apply threshold.

Updating the Performance Monitor DisplayBy default, replication latency information is updated in the Performance Monitor every 30 seconds. You can change the amount of time between updates by increasing or decreasing the refresh rate.

To update the performance monitor display1 From the View menu, select Options.

The Refresh Rate dialog box opens.

2 In the Refresh Rate (seconds) box, enter the number of seconds between updates of latency information in the Performance Monitor.

Note: The minimum value for the refresh rate is one (1) second.3 Click OK.The new refresh rate is applied to all open and future instances of the Performance Monitor.

*RGZPFM Re-organizing the physical file member in the current object.

*ACCPATHRBD Access path rebuild in the current object.

Table 4

Current Operation Description


Monitoring iCluster

Note: You can also perform a manual refresh from the View menu bar option or by pressing F5.


Object Types Replicated by iClusterThe following table lists the object types that iCluster can replicate.Table 1 Object Types Replicated by iCluster

Object Type System Object Type

Alert table *ALRTBL

Authorization list *AUTL

Authority holder (see Step 8) *AUTHLR

Block special file *BLKSF

Bind directory *BNDDIR

Chart format *CHTFMT

C locale description *CLD

Class *CLS

Command definition *CMD

Connection list (see Step 5) *CNNL

Class-of-service description (see Step 5) *COSD

Change request descriptor *CRQD

Communications side information *CSI

Cross-system product map *CSPMAP

Cross-system product table *CSPTBL

Controller description (see Step 5) *CTLD

Device description (see Step 5) *DEVD

Directory *DIR

Document Library Object *DOC

Data area *DTAARA

Data queue *DTAQ

Edit description *EDTD

Exit registration *EXITRG

Forms control table *FCT

Folder *FLR


File (see Step 12) *FILE

Font resource *FNTRSC

Forms definition *FORMDF

Font table resource *FTR

Graphics symbol set *GSS

Double-byte character set dictionary *IGCDCT

Double-byte character set font table *IGCSRT

Double-byte character set sort table *IGCTBL

Internet packet exchange description (see Step 5) *IPXD

Job description *JOBD

Job queue (see Step 9) *JOBQ

Job scheduler *JOBSCD

Journal (see Step 10) *JRN

Library (see Step 1and Step 2) *LIB

Line description (see Step 5) *LIND

Machine *M36

Advanced 36 machine configuration *M36CFG

Menu *MENU

Mode description (see Step 5) *MODD

Module *MODULE

Message file *MSGF

Message queue (see Step 9) *MSGQ

Node group *NODGRP

Node list *NODL

NetBIOS description (see Step 5) *NTBD

Network interface description (see Step 5) *NWID

Network server description (see Step 5) *NWSD

Table 1 Object Types Replicated by iCluster



Output queue *OUTQ

Overlay *OVL

Page definition *PAGDFN

Page segment *PAGSEG

Printer description group *PDG

Program *PGM

Panel group *PNLGRP

Product availability *PRDAVL

Print Services Facility Configuration *PSFCFG

Query form *QMFORM

Query manager query *QMQRY

Query definition *QRYDFN

Reference code translation table *RCT

System/36 machine description *S36

Subsystem description *SBSD

Search index *SCHIDX

Spelling help dictionary *SPADCT

SQL package *SQLPKG

User defined SQL data type *SQLUDT

Service program *SRVPGM

Session description *SSND

Byte Stream File *STMF

Symbolic links *SYMLNK

Table *TBL

User index *USRIDX

User profile (see Replicating User Profiles on page 91 for more information)

*USRPRF

User queue *USRQ




Note: Since changes to user spaces (*USRSPC) are not usually journaled, iCluster uses polling to keep track of content changes for this type of object.

Notes:1 iCluster does not replicate system-supplied user profiles.2 It is important to recognize that replicating library objects (*LIB) does not automatically replicate objects contained in the

library. Only the library object is sent to the backup node. To replicate all objects in a library, use the pre-defined value *ALL to define an object specifier that addresses all objects and all object types in a specific library.

3 iCluster does not replicate OS/400 system values.4 iCluster supports user actions that can be applied to save file objects that are replicated to a backup node. For more

information about user actions and how other types of objects can be replicated via save files, see Replicating Save Files on page 92.

5 For configuration objects to be successfully replicated by iCluster, any dependent lines, devices, modes, and so on must already exist on the backup node.

6 Spool files in a replicated output queue (*OUTQ) will only be mirrored when they are created or deleted. iCluster does not mirror spool file updates to backup nodes.

7 Due to OS/400 limitations, spool files containing Intelligent Printer Data Stream (IPDS) data will not be properly replicated.8 To replicate authority holders, you must specify *AUTHLR as the object type when defining an object specifier (authority

holders are not referenced in an object specifier that uses the generic value of *ALL for the object type). iCluster only replicates authority holders for objects of type *FILE.

9 iCluster replicates job queues (*JOBQ) and message queues (*MSGQ), but does not replicate changes to their contents.10 When replicating journals, each journal can only be mirrored to a library that has the name of the originating library.11 iCluster does not currently support the mirroring of system-provided authorization lists (*AUTL). iCluster ignores change (ZC)

journal entries for authorization list objects.12 When replicating *FILE objects, there are certain considerations. These considerations are discussed in Replicating

Database *FILE Objects on page 87.13 The following object types cannot be replicated in a local loopback configuration: *JRN, *LIB, *USRPRF, and configuration

objects.14 For user spaces iCluster supports object-level only change replication and object and content-level change replication.

Object-level only support will replicate creation, deletion, restore, and move/rename operations on user spaces. This is suitable for user spaces that are created and filled with data, but the data never changes.Content-level change support for user spaces is based on polling the objects at regular intervals to determine if they have changed. Note that since replication is based on polling, the Status Monitor cannot be relied upon to determine if the user spaces on the backup node are up-to-date.

User space (see Step 14) *USRSPC

Workstation customization *WSCST




iCluster LimitsThis topic describes various limits, maximum capacities, and design recommendations that are specific to iCluster. Exceeding these limits can produce unexpected results.Note: See your IBM documentation for iSeries limits and capacities. All iSeries limits apply to iCluster unless stated

in this topic.

iCluster Maximum CapacitiesThis section describes the maximum capacities for iCluster.

Cluster LimitsIf your cluster uses the SwitchOver System failover mechanism, then each node cannot have more than 200 links. A link is created by having a node be either the primary or backup node in a group. Local loopback groups count as two links for a node.For example, Figure 1 shows a cluster with four nodes. The TORONTO node has three links, DALLAS has one link, and the NEWYORK and SANJOSE nodes each have two links.

Figure 1 Cluster Links

If your cluster uses IBM Cluster Resource Services as its failover mechanism, then you cannot have more than 128 nodes in a cluster.

Minimum Staging Store SizeThe minimum size of the staging store is directly related to the number of database journals on a node. Use the following formula to calculate the minimum size you need for a staging store:Minimum Size = ( 2 x <database journals> + 1 ) * 16 MB

In this calculation, <database journals> is the number of database journals on the node. For example, if you had six database journals on a node, then the node would require a minimum of 208 MB of staging store space.

Uncommitted TransactionsEach journal in a group can have up to one terabyte of uncommitted transactions.

IFS LimitationsiCluster does not support the following for IFS objects:• Replicating hard links.


• Replicating to a different directory on the backup node.• Replicating DLO extended object attributes.• Suspending non-journaled IFS objects on the backup node. You must monitor the event log to track replication errors with

IFS objects on the backup node. You can still suspend non-journaled library objects on the primary node.• The maximum length of path names is 5000 bytes long.• The Lock Files on Backup Node and Maximum Refresh Size system values are not supported for path object specifiers. See

on page 181 for more information about these system values.For more information about path object specifiers, see Path Object Specifiers on page 53.

Large Object (LOB) ConstraintsThe following constraints apply to LOBs:• iCluster stores staged LOB data on the backup node in IFS objects in the following directory:

/home/DataMirror/HASUITE/LOB/<primarySN>/<backupNode>/<group>/<journalLib>/<journalName>

where sourceSN is the serial number of the primary node, backupNode is the name of the backup node, group is the name of the group, journalLib is the journal library, and journalName is the journal name. This does not affect non-LOB data being replicated from the same database table.

• Currently, the maximum store size on the target system does not include IFS space used when creating the BSF objects for LOB staging. LOB data can be replicated as long as there is sufficient disk space on the target system to contain the LOB data.

• iCluster does not impose limitations in addition to those imposed by OS/400 for LOB fields in a record format. There can be a total of 1023 LOB fields in a record format. The total size of the LOB fields cannot exceed two gigabytes. The default journal iCluster uses for these files must have the RCVSIZOPT parameter of the CHGJRN command set to *MAXOPT2, otherwise iCluster will suspend the file.Consequently, DataMirror recommends that all journals use the highest receiver size option possible.

MQSeries RecommendationsAuthority changes made through the GRTMQMAUT and RVKMQMAUT commands are not recorded by MQSeries and are, therefore, not mirrored by iCluster.IBM recommends that the operator use a CL program to apply the two commands to any MQSeries objects on the primary node instead of issuing them on the command line. This means the operator may have to update and run the CL program periodically in order to handle newly created MQSeries objects. iCluster can replicate this program to the backup node, where it can be run after a switchover. This ensures all authority changes on the primary node are properly applied to the backup node.


The Apply Process

iCluster TroubleshootingThis section contains troubleshooting and problem-solving information.

The Apply ProcessIf the apply processes on the backup node cannot apply a journal entry to a file because it is exclusively locked either by another application or by the operating system, then you can configure the apply processes on the backup node to wait a certain amount of time and retry the apply a specified number of times. This is achieved by creating a data area object in the iCluster library on the backup node. The data area should be named HA_OBJLCK, be of type *CHAR, and it must be long enough to contain the wait/retry specification string. Specify the following parameters:

WAIT <n1> RETRY <n2>

where:• 'WAIT' and 'RETRY' cannot be in mixed case (only all uppercase or all lowercase).• <n1> can be any positive integer except zero (0). It represents the number of seconds to wait.• <n2> can be any positive integer or *NOMAX. It represents the number of times to retry the apply process.The target apply job will retry the record update every <n1> seconds until the uniquely keyed access path has been rebuilt or <n2> attempts have been made. If the last attempt fails, the file will be suspended.If the HA_OBJLCK data area is not created, the default is to wait one second and retry five more times.


Index

IndexAactivating an object 505adding a backup node 484adding a node 397adding an iCluster user 29adding nodes 31additional message information 41administering iCluster

commands for 181Administrator and Event Viewer

overview 384application resiliency 68apply jobs

status of 292apply process

ending 504starting 503

AS/400 objectscommands for 141

authorization codes 20automatic failover 49avoiding contention 70

Bbackup monitor

using 516backup node

adding 484removing 485

batch process 41BSF objects

replicating 89BSF status

working with 310Byte Stream File (BSF)

commands for 155

Cchaining groups 46changing IP addresses 80changing the event log filter 512choosing a failover mechanism 73commands

format of 97commands for AS/400 objects 141commitment control 71common information on views in the Status Monitor 294common options on views in the Status Monitor 295conflict resolution user exit program

parameter list descriptions for 326contacting

DataMirror 547technical support 547


Index

courses at DataMirror 547current journal position

mark 496

DDataMirror

contacting 547training and education 547

dmcluster table entryremoving 39

DNS configurationverifying 38

Eeducation at DataMirror 547ending the apply process 504event log

changing the filter 512clearing 515exporting 514maximum number of messages 516

Event Viewerstarting the 510

exit programscommands for 283

external storage systemscommands for 285

Ffailover

automatic 49manual 49

failover mechanismchoosing 74

failover mechanismschoosing 73

full replication groupchanging 424converting to 441ending 423removing 424starting 421switchover 433

full replication groupsadding 413

Ggeneric object specifiers 52groups 44

chaining 46commands for 113independence of 46object status for 312


Index

Hhardware requirements 19historical latency

monitoring 314historical object position and totals

monitoring 316historical object throughput

monitoring 318historical statistics

monitoring 314

IiBalance 321

installing 23iBalance feature 23IBM Cluster Resource Services (IBM CRS) 47IBM CRS

configuring iCluster to use 75iCluster

adding an iCluster user 29and master-master replication 321authorization codes 20commands for groups 113commitment control 71failover mechanisms 47failovers and switchovers 47groups in 44IBM Cluster Resource Services (IBM CRS) 47initial synchronization 68journaling in 81latency in 292limits of 533master node in 44monitoring replication 76nodes in 43object specifiers and types 50object types replicated by 529overview 17path object specifiers 52performance considerations 29quick start 55reinstalling 21remote journaling 83replicating configuration objects 90replicating database objects 87replicating LOBs 92replicating QDLS objects 94replicating save files 92replicating triggers 92replicating user profiles 91restarting after restarting a node 79scheduling startup and shutdown 30staging objects in 69starting from the command line 58SwitchOver System (SOS) 47uninstalling 33upgrading 26using the Status Monitor 291


Index

iCluster Administratordelete cluster 396installing 383minimum requirements 383monitoring 516monitoring operations 385rejoin cluster 395unnstalling 383upgrading 383

iCluster Administrator for Windowslogging on 60quick start 59

iCluster Enterprise Edition 17, 23iCluster product library

securing 30iCluster SMB Edition 17, 23independence of groups 46installation

minimum requirements 19restrict installation port from being used by other applications 37signing on to your system 24

installation portrestrict from being used by other applications 37

installation procedureoverview 23

installation programrestoring 25running the 25

Jjob logs 41journal position

set 497journal receiver

creating 20journal scrape processes

define synchronization point 498journaling

local 82remote 82, 83

journaling in iCluster 81JRE 383

Kknowledge base 547

Llatency 292

monitoring 78local loopback replication 46locked objects 70

Mmanual failover 49mark journal positions 496master node 44

adding to the cluster 59


Index

master-master groupadding 448changing 455, 456ending 454removing 455starting 452

master-master replication 321configuration of 322configuration of conflict detection and resolution 325configuring in iCluster Administrator 509conflict logging file 337high availability considerations 340operations and monitoring 323overview of conflict detection and resolution 323

migrating HA Suite to iCluster 35migration

HA Suite to iCluster 35introduction to 35

migration proceduregeneral constraints 35

migration utilityrunning 35

minimum RAM 383minimum requirements 19

hardware 383software 383

monitoring replication 76MQSeries

commands for 179MQSeries group

adding 471changing 479ending 478removing 478starting 476switchover 483

Nnodes

adding 397adding and configuring 67changing 404commands for 99ending 403removing 67, 402starting 403

nodes in iCluster 43

Oobject

activating 505object specifiers

changing 492deselecting 492rules of precedence 52selecting/adding 486

object specifiers and types 50constructing 51


Index

object statusworking with 306

object status monitorusing 518

object types replicated by iCluster 529operating system 383operations in iCluster

commands for 207out-of-sync objects

monitoring 77overview

iCluster SMB and iCluster Enterprise 23installation procedure 23installing, re-installing, and upgrading iCluster 21

Ppassing arguments

to role switch user exit programs 347path object specifers 53path object specifiers 52

journaled and non-journaled 54performance monitor

key elements 520updating the display 526using 520

port number restrictionsremoving 39

post-migration tasks 36pre-migration tasks 35primary node

change role 502prepare 502

PTF 19

QQALWUSRDMN

setting system value 21QAUDJRN

journal in QSYS 20quick start 55

Rreading status information 303real time and historical statistics 292real time object latency 299real time object position and totals 301real time object throughput 302real time overall latency 297real time statistics views 293refresh only group

changing 439ending 437removing 438starting 437

refresh-only groupadding 434converting 431


Index

register iCluster userscommands for 277

registering your existing OS/400 user name 59remote journaling 83

configuring 84remote journals

role switching with 85removing a backup node 485removing the dmcluster table entry 39remving a node 402replicating

BSF objects 89configuration objects 90LOBs 92QDLS objects 94save files 92triggers 92user profiles 91

replicating database objects 87replicating objects

sending one or more objects to a target node 410replication

monitoring 76replication operations

commands for 225requirements

communication protocol 19hardware and software 19information for the installation procedure 20PTF 19

resilient applicationadding 459changing 462ending 468removing 465starting 466switchover 469updating 465

resilient applicationscommands for 167

restoringfrom a save file 25from CD-ROM 25the installation program 25

restoring communications registries 289

Ssecuring the iCluster product library 30security

administrator commands 66iCluster Administrator 385levels 385of commands 63operator commands 65security officer commands 64user commands 65


Index

sending objects 410set journal position 497set primary node 502set synchronization point 498setting the QALWUSRDMN OS/400 system value 21staging objects 69starting replication and journal positions 75starting the apply process 503status and history monitor

commands for 255Status Monitor

simplified 292using 291

suspended objects 72suspending

objects 507suspending an object 507switchable devices

commands for 163switchover

choosing a system 74SwitchOver System

configuring iCluster to use 74SwitchOver System (SOS) 47sync check

commands for 263viewing 78

sync point user exit programspassing arguments to 86

system valuessetting in iCluster Administrator 386

Ttable view 62TCP/IP service table

adding a new entry to 37technical support, procedures 547temporary fix requirements 19training at DataMirror 547tree view 62troubleshooting information 547

Uupdates

applying iCluster updates 21upgrade or reinstallation

preparing for 24upgrading iCluster 26upgrading the iCluster version 80user exits

for role switch 347users

adding 29

WWebSphere MQ support

enabling 96


Index

workload balancing 321

XXDMCLUSTER subsystem

starting on IPL 29


Index


Contacting DataMirrorTroubleshootingTroubleshooting and diagnostic information is provided in the DataMirror Knowledge Base that you can access from the DataMirror web site at www.datamirror.com. Articles in the knowledge base are categorized by product, but you can also search for information using keywords and other attributes. The knowledge base also contains articles translated into different languages.You must complete a registration process before accessing the knowledge base.

Contacting Technical SupportIf you encounter problems using iCluster, follow the procedure outlined below:1 Consult the relevant sections of this guide, or consult the DataMirror Knowledge Base at www.datamirror.com.2 Consult your system and product event logs for messages that may contain relevant information for your problem.3 Attempt to re-create the problem. Document the steps used to do so.4 If the problem persists, record the full details of the clustered environment by issuing the DMSYSINF—System Information

command from the command line or select option 7 in the iCluster Main Menu. This command allows you to view or print system information.

5 Contact DataMirror Technical Support by visiting the DataMirror web site at www.datamirror.com for technical support email addresses and telephone numbers.

You can also access updates and platform compatibility information from the DataMirror web site.

Training and EducationFor hands-on training, DataMirror offers public education courses regularly at education centers in different parts of the world. During the training, participants will learn from experienced trainers the basic building blocks in implementing DataMirror technology and will be given the opportunity to use DataMirror products in guided lab exercises.You can find course outlines and schedules on the DataMirror web site at www.datamirror.com/education. For more information, send email to [email protected].

Send us your CommentsDataMirror welcomes your suggestions on how to enhance iCluster documentation. Send your suggestions or comments by contacting us at:Customer CommentsDataMirror Corporation3100 Steeles Avenue East, Suite 1100Markham, Ontario, CanadaL3R 8T3Telephone: 1-905-415-0310Facsimile: 1-905-415-0340Email: [email protected]


http://www.datamirror.com



http://www.datamirror.com/education

mailto:[email protected]

mailto:[email protected]

Copyright NoticeThis manual is © DataMirror Corporation, an International Business Machines Corporation Company ("DataMirror"). All rights reserved. No part of this manual may be reproduced, distributed or transmitted, in whole or in part, in paper, electronic or any other form or by any means other than as expressly permitted in the applicable DataMirror Corporation Software License Agreement or Software License and Maintenance Agreement, or as otherwise expressly permitted by DataMirror Corporation, an International Business Machines Corporation Company.

Trademark NoticeCONSTELLAR, DATA FROM WHERE IT IS TO WHERE IT NEEDS TO BE, DATAMIRROR, DATAMIRROR DB/XML TRANSFORM, DATAMIRROR DB/XML VISION, DATAMIRROR SYNAPSE MOBILITY, DATAMIRROR TRANSFORMATION SERVER, DBMIRROR, ENTERPRISE ADMINISTRATOR, ERP GATEWAY, HA SUITE, HIGH AVAILABILITY SUITE, ICLUSTER, ICLUSTER FOR EMC SYMMETRIX, IDELIVER, IREFLECT, ITRANSMIT, JOBSCHEDULER, MANAGEMENT CONSOLE, OBJECTMIRROR, QUICKMARTS, SWITCHOVER SYSTEM, THE EXPERIENCE OF NOW, TRANSFORMATION SERVER, TRANSFORMATION SERVER/ES, TRANSFORMATION SERVER/EVENT SERVER, TRANSFORMATION SERVER MANAGEMENT CONSOLE, and XTREMECACHE are registered, unregistered or pending trademarks of DataMirror Corporation and may not be used without the express written permission of DataMirror Corporation. POINTBASE, POINTBASE EMBEDDED, POINTBASE MICRO, POINTBASE UNISYNC, and TRANSFORMATION SERVER FOR MOBILE are trademarks of DataMirror Mobile Solutions Inc. ("PointBase") and DataMirror Corporation's use of these trademarks is by way of license with PointBase. This list of trademarks may not be complete; other trademarks or registered trademarks may be owned by DataMirror Corporation from time to time and may be used in this manual.All other trademarks or service marks are the properties of their respective owners.

Proprietary and Confidential Information NoticeDataMirror Corporation software products contain valuable trade secrets and proprietary information and are protected internationally, including without limitation, by Canadian, United States and international copyright, trademark, and other intellectual property laws and treaties.DataMirror Corporation’s use of certain copyrights and trademarks in this manual is authorized by license from their respective owners and licensors. Unauthorized use of this manual or DataMirror Corporation software products is strictly prohibited and may result in civil damages and criminal prosecution. See the applicable DataMirror Corporation Software License Agreement or Software License and Maintenance Agreement for additional information.

DisclaimersDataMirror reserves the right to revise this manual and make periodic changes to its content without obligation on DataMirror Corporation’s part to notify any person of such revisions or changes. DataMirror does not assume responsibility for the use of this manual. DataMirror makes no representation or warranty as to the accuracy of the contents of this manual. All statements made and information provided in this manual is provided on an errors and omission excepted (E. & O.E.) basis only.

IBM DATAMIRROR iCluster® Version 5.1DataMirror Corporation, an International Business Machines Corporation CompanyJanuary 3, 2008


iCluster Manual End-User 5.1

Documents

Transcript of iCluster Manual End-User 5.1