Replicating Tenants and Name Spaces

Product Version

Document Organization

Getting Help

Table of Contents

FASTFIND LINKS

MK-98ARC015-05

Hitachi Content PlatformReplicating Tenants and Namespaces

Copyright © 2008–2011 Hitachi Data Systems Corporation, ALL RIGHTS RESERVED

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording, or stored in a database or retrieval system for any purpose without the express written permission of Hitachi Data Systems Corporation (hereinafter referred to as “Hitachi Data Systems”).

Hitachi Data Systems reserves the right to make changes to this document at any time without notice and assumes no responsibility for its use. This document contains the most current information available at the time of publication. When new and/or revised information becomes available, this entire document will be updated and distributed to all registered users.

Some of the features described in this document may not be currently available. Refer to the most recent product announcement or contact your local Hitachi Data Systems sales office for information about feature and product availability.

Notice: Hitachi Data Systems products and services can be ordered only under the terms and conditions of the applicable Hitachi Data Systems agreement(s). The use of Hitachi Data Systems products is governed by the terms of your agreement(s) with Hitachi Data Systems.

By using this software, you agree that you are responsible for:

a) Acquiring the relevant consents as may be required under local privacy laws or otherwise from employees and other individuals to access relevant data; and

b) Ensuring that data continues to be held, retrieved, deleted, or otherwise processed in accordance with relevant laws.

Hitachi is a registered trademark of Hitachi, Ltd. in the United States and other countries. Hitachi Data Systems is a registered trademark and service mark of Hitachi, Ltd. in the United States and other countries.

All other trademarks, service marks, and company names are properties of their respective owners.

Contents

Preface........................................................................................................viiIntended audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiProduct version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiDocument organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiSyntax notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ixRelated documents. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ixGetting help. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xiComments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xii

1 Replication overview...........................................................................1-1About replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2

How replication works. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3What is replicated. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3

Replication of HCP tenants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-4Replication of the default tenant . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5

Replicated systems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-7Replication topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-8

Bidirectional replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-9Many-to-one replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-9Chained replication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-12Many-to-one replication with disaster recovery support . . . . . . . . . . . . . .1-14

Replication service processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-15Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-17Recovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-18Resolving recovery conflicts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1-19

2 Configuring SSL for replication...........................................................2-1Downloading an SSL server certificate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2Uploading a trusted replication server certificate . . . . . . . . . . . . . . . . . . . . . . 2-2Deleting a trusted replication server certificate . . . . . . . . . . . . . . . . . . . . . . . 2-3

Contents iii

Replicating Tenants and Namespaces

3 Working with replication links .............................................................3-1Creating a replication link . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2

Considerations for creating a replication link . . . . . . . . . . . . . . . . . . . . . . 3-3What you do . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4

Item selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-7Advanced configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-9

Canceling a link request . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-10Accepting or rejecting a link request. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-11Modifying a replication link . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-12

Considerations for modifying a replication link . . . . . . . . . . . . . . . . . . . . .3-12What you do . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-14

Deleting a replication link . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3-15

4 Monitoring replication .........................................................................4-1Link information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2Link overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-4

Data transmission rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-5Operation rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-5Space used on the replica . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-6

Tenants view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-6Tenant replication information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-7Tenant replication details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-8

Controlling the Tenant Management Console replication display . . . . . . . . . . . 4-9System Management Console alerts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4-10

5 Managing replication ..........................................................................5-1Scheduling activity on a replication link. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2

About scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2What you do . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-3

Suspending and resuming activity on a replication link . . . . . . . . . . . . . . . . . . 5-4Pausing and resuming tenant replication or recovery . . . . . . . . . . . . . . . . . . . 5-5

6 Managing failover and recovery .........................................................6-1Failover and recovery workflows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-2

Primary system failure workflow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-2Replica failure workflow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-3

Failing over to a replica. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-4Recovering from a failure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-5

Restoring a link . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-5Recovering the data after a primary system failure . . . . . . . . . . . . . . . . . 6-6

Failover and recovery considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-8Failover and recovery in a many-to-one topology. . . . . . . . . . . . . . . . . . . 6-8

iv Contents


Failover and recovery in a chained topology . . . . . . . . . . . . . . . . . . . . . . 6-9Scenario: System A fails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-10Scenario: System B fails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-11Scenario: System C fails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-12Scenario: Systems A and B fail . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-12Scenario: Systems A and C fail . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-14Scenario: Systems B and C fail . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-14

Failover and recovery in a many-to-one topology with disaster recovery support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6-15

A Reenabling user accounts disabled on the replica ............................ A-1

B Setting advanced replication options................................................. B-1Changing the custom performance level . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-2Changing the broken-link notification interval . . . . . . . . . . . . . . . . . . . . . . . . B-3

Glossary

Index

Contents v


vi Contents


Preface

This book is your guide to data replication with Hitachi Content Platform (HCP). Replication is an HCP service that copies selected tenants and namespaces from one HCP system to another.

This book describes how replication works and contains complete instructions for setting up replication between two HCP systems. It also provides information on monitoring and managing replication activity, including failover and recovery.

Intended audience

This book is intended for HCP system administrators who are responsible for configuring replication links, monitoring replication, and managing failover and recovery with a replica. It assumes you are familiar with HCP concepts and functionality and have some experience with the HCP System Management Console.

Product version

This book applies to release 4.1 of HCP.

Preface vii


Document organization

Document organization

This book contains six chapters and two appendixes.

Chapter/Appendix Description

Chapter 1, “Replication overview” Introduces replication concepts, describes the network connection between replicated systems, explains the supported replication topologies, and provides an overview of replication processing

Chapter 2, “Configuring SSL for replication”

Explains how to install the SSL server certificates required to secure replication activity

Chapter 3, “Working with replication links”

Contains instructions for creating, modifying, and deleting replication links

Chapter 4, “Monitoring replication” Discusses how to monitor replication activity in the System Management Console; also explains how to control whether the Tenant Management Console displays information about replication of the tenant

Chapter 5, “Managing replication” Explains how to schedule, suspend, and resume all activity on a replication link and how to pause and resume replication activity of an individual tenant

Chapter 6, “Managing failover and recovery”

Explains how to perform failover and recovery tasks and contains considerations for failover and recovery in many-to-one and chained replication topologies

Appendix A, “Reenabling user accounts disabled on the replica”

Explains how to reenable tenant-level user accounts that have been disabled on the replica

Appendix B, “Setting advanced replication options”

Contains information about setting a custom performance level for replication and changing the SNMP reporting interval for broken links

viii Preface


Syntax notation

Syntax notation

The table below describes the conventions used for the syntax of commands, expressions, URLs, and object names in this book.

Related documents

The following documents contain additional information about Hitachi Content Platform:

• Administering HCP — This book explains how to use an HCP system to monitor and manage a digital object repository. It discusses the capabilities of the system, as well as its hardware and software components. The book presents both the concepts and instructions you need to configure the system, including creating the tenants that administer access to the repository. It also covers the processes that maintain the integrity and security of the repository contents.

• Managing a Tenant and Its Namespaces — This book contains complete information for managing the HCP tenants and namespaces created in an HCP system. It provides instructions for setting up both administrative user accounts and data access accounts, configuring the HTTP protocol, which allows access to namespaces, managing search, and downloading installation files for HCP Data Migrator and the HCP client tools. It also explains how to work with retention classes and the privileged delete functionality.

• Managing the Default Tenant and Namespace — This book contains complete information for managing the default tenant and namespace in an HCP system. It provides instructions for changing tenant and namespace settings, configuring the protocols that allow access to the namespace, managing search, and downloading installation files for HCP Data Migrator and the HCP client tools. It also explains how to work with retention classes and the privileged delete functionality.

Notation Meaning Example

boldface Type exactly as it appears in the syntax (if the context is case insensitive, you can vary the case of the letters you type)

This book shows:replication.hcp-name.domain-name

You enter: replication.hcp-ca.example.com

italics Replace with a value of the indicated type

Preface ix


Related documents

• HCP Management API Reference — This book contains the information you need to use the HCP management API. This REST API enables you to create and manage tenants and namespaces programmatically. The book explains how to use the API to access an HCP system, specify resources, and update and retrieve resource properties.

• Using a Namespace — This book describes the properties of objects in HCP namespaces. It provides instructions for accessing namespaces by using the HTTP protocol for the purpose of storing, retrieving, and deleting objects, as well as changing object metadata such as retention and shred settings. It also explains how to manage namespace content and view namespace information in a web browser.

• Using the Default Namespace — This book describes the file system HCP uses to present the contents of the default namespace. It provides instructions for accessing the namespace by using the HCP-supported protocols for the purpose of storing, retrieving, and deleting objects, as well as changing object metadata such as retention and permissions.

• Searching Namespaces — This book describes the HCP Search Console. It explains how to search namespaces for objects that satisfy criteria you specify. It also explains how to manage and manipulate queries and search results. The book contains many examples, which you can use as models for your own searches.

• Using HCP Data Migrator — This book contains the information you need to install and use the HCP Data Migrator (HCP-DM) utility distributed with HCP. This utility enables you to copy data between local file systems, HCP namespaces, and earlier HCAP archives. It also supports bulk delete operations. The book describes both the interactive window-based interface and the set of command-line tools included in HCP-DM.

• Using the HCP Client Tools — This book contains the information you need to install and use the legacy set of client command-line tools distributed with HCP. These tools enable you to find files and to copy and move files to and from namespaces. The book contains many examples that show command-line details and the overall workflow.

Note: For most purposes, the HCP client tools have been superseded by HCP Data Migrator. However, they have some features, such as finding files, that are not available in HCP-DM.

x Preface


Getting help

• Installing an HCP System — This book provides the information you need to install the software for a new HCP system. It explains what you need to know to successfully configure the system and contains step-by-step instructions for the installation procedure.

• Third-Party Licenses and Copyrights — This book contains copyright and license information for third-party software distributed with or embedded in HCP.

• HCP-DM Third-Party Licenses and Copyrights — This book contains copyright and license information for third-party software distributed with or embedded in HCP Data Migrator.

• Installing an HCP 500 System — Final On-site Setup — This book contains instructions for deploying an assembled and configured HCP 500 system at a customer site. It explains how to make the necessary physical connections and reconfigure the system for the customer computing environment.

• Installing an HCP 300 System — Final On-site Setup — This book contains instructions for deploying an assembled and configured HCP 300 system at a customer site. It explains how to make the necessary physical connections and reconfigure the system for the customer computing environment.

Getting help

The Hitachi Data Systems® customer support staff is available 24 hours a day, seven days a week. If you need technical support, please call:

• United States: (800) 446-0744

• Outside the United States: (858) 547-4526

Note: If you purchased HCP from a third party, please contact your authorized service provider.

Preface xi


Comments

Comments

Please send us your comments on this document:

[email protected]

Include the document title, number, and revision, and refer to specific sections and paragraphs whenever possible.

Thank you! (All comments become the property of Hitachi Data Systems.)

xii Preface


mailto:[email protected]

1
Replication overview
Hitachi Content Platform (HCP) is the distributed, fixed-content, data storage system from Hitachi Data Systems. An HCP system maintains a repository of objects that encapsulate fixed-content data and metadata that describes the data. You can configure an HCP system to copy selected tenants and namespaces to another HCP system. This copying process is carried out by the replication service through a replication link.

This chapter provides an overview of how replication works and how it supports business continuity and disaster recovery. For an introduction to HCP systems and how they work, see Administering HCP.

Notes:

• Replication is an add-on feature to HCP. If your HCP system doesn’t include this feature and you would like to add it, please contact your HCP sales representative.

• HCP supports replication between two 4.x systems and between 4.x and 3.x but not earlier systems.

Replication overview 1–1


About replication

About replication

Replication, an HCP service, is the process of keeping selected tenants and namespaces in two HCP systems in sync with each other. This entails copying object creations, object deletions, metadata changes, and other information between the two systems. The HCP system in which the objects are initially created is called the primary system. The second system is called the replica. Typically, the two systems involved are in separate geographic locations and connected by a high-speed wide area network.

Replication has several purposes:

• If the primary system becomes unavailable (for example, due to network issues), the replica can provide continued data availability.

• If the primary system suffers irreparable damage, the replica can serve as a source for disaster recovery.

• If the two systems are widely separated, the replica may be able to provide faster data access for some locations than the primary system can.

• If an enterprise has several satellite offices, a replica at a central facility can consolidate data from the HCP systems at those outlying locations.

• If the content verification or protection service discovers objects it cannot repair on a primary system or replica, it can use object data and metadata from the other system to make the needed repairs. For more information on these services, see Administering HCP.

• If an object cannot be read from the primary system (for example, because a node is unavailable), HCP can try to read it from the replica. If the namespace on the replica is being replicated to a third HCP system in a chain configuration and HCP cannot read the object from the replica, HCP will try to read the object from that third system.

For more information on the read-from-replica feature, see Managing a Tenant and Its Namespaces or Managing the Default Tenant and Namespace.

1–2 Replication overview


About replication

How replication works

To start the replication service, you create a replication link between the primary system and the replica. A replication link is a secure trust relationship between the two systems. It determines what is replicated and how data is transmitted between the primary system and the replica.

A link created on a primary system must be accepted by the replica before replication can begin. To the primary system, this link is an outbound link. To the replica, it’s an inbound link.

You can also create a link remotely on the replica. When you do this, replication from the primary system to the replica begins right away, without the primary system having to accept the link. To the primary system, the link is still an outbound link. To the replica, it’s still an inbound link.

Replication relies on SSL to ensure the integrity and security of transmitted data. Before you can create a replication link, the primary system and the replica must each have a valid SSL server certificate. Each system must also have the server certificate from the other system installed as a trusted replication server certificate.

Replication is asynchronous with other tenant and namespace activity on both the primary system and the replica. An HCP system can, therefore, develop a backlog of objects to be replicated.

You can choose to have the replication service work on objects with the oldest changes first, regardless of which namespaces they’re in. Alternatively, you can have the service balance its processing time evenly across the namespaces being replicated. In this case, the service may replicate recent changes in some namespaces before older changes in others.

What is replicated

Replication works by tenant. When you create a replication link, you select the HCP tenants to replicate and, if you have administrative access to the tenant, which namespaces to include in replication of that tenant. For the default tenant, you select the directories to replicate. What is replicated for each tenant differs between HCP tenants and the default tenant.

Note: This book refers to nondefault tenants as HCP tenants when necessary to distinguish them from the default tenant.



About replication

Replication of HCP tenants

Part of the configuration of an HCP tenant is whether it’s eligible to be replicated. A replication link can include only HCP tenants that are eligible to be replicated.

Replicated namespacesWhen an HCP tenant is replicated, the replication service works only on selected namespaces owned by that tenant. The tenant administrator can select and deselect namespaces for replication at any time. If the tenant is being replicated, its selected namespaces are replicated along with it. If the tenant is not being replicated, the selected namespaces are not replicated either.

If an HCP tenant has granted system-level users administrative access to itself, you can select and deselect its namespaces for replication when you create the replication link or by modifying the link at a later time. You should coordinate namespace selection with the tenant administrator to ensure that you both have the same understanding of what should be replicated.

Other replicated itemsIn addition to object creations, object deletions, and metadata changes, the replication service replicates these items for an HCP tenant:

• Old versions of objects in the namespaces being replicated

• Retention class creations, modifications, and deletions for the namespaces being replicated

• All tenant-level log messages

• All namespace-level log messages for the namespaces that are being replicated

Note: Replicated namespaces do not count toward the total number of namespaces allowed on a replica as long as their owning tenant is included in a replication link. If you remove the tenant from the link, the namespaces are then included in the total on the replica.

Note: The amount of time before old versions of objects are pruned can differ between the primary system and the replica. HCP always ensures that old versions needed for the replica are replicated before pruning them on the primary system. For more information on version pruning, see Managing a Tenant and Its Namespaces.



About replication

• The tenant configuration

• The configuration of each namespace being replicated

• Administrative user accounts defined for the tenant

• Data access accounts defined for the tenant

Tenants on the replicaOn the replica, replicated HCP tenants are read-only. While a tenant is read-only, administrators cannot make any configuration changes to it or to any of the namespaces, nor can users and applications make any changes to namespace content.

HCP tenants can be defined on both the primary system and the replica, so the two systems can have tenants with the same name. You cannot, however, replicate an HCP tenant defined on the primary system if a tenant with the same name already exists on the replica. To replicate the tenant in this case, you first need to change the name of one of the tenants involved.

Replication of the default tenant

For the default tenant, the replication service works on one or more selected directories directly under fcfs_data in the default namespace. The service replicates the entire directory hierarchy under each selected directory.

Default namespace propertiesDirectories in the default namespace can be replicated only to the default namespace on the replica. They cannot be replicated to HCP namespaces. Additionally:

• Before you can include any default-namespace directories in a replication link, the default namespace must exist on the replica.

• The default namespaces on the primary system and replica must have the same cryptographic hash algorithm and retention mode.



About replication

• If the default namespace on the primary system supports appendable objects, the default namespace on the replica should, too. This ensures that, in case of failover, users and applications can continue to add data to the replicated appendable objects on the replica. For information on failover, see “Failover” on page 1-17.

The default namespace can be configured to prevent writes and/or deletes. This configuration can differ between the default namespace on the primary system and the default namespace on the replica. However, even if the replica is configured to prevent these actions, changes to namespace content on the primary system are replicated.

Replicated directoriesYou select directories for replication when you create the replication link. You can select additional directories or deselect directories by modifying the link at a later time.

Other replicated itemsIn addition to object creations, object deletions, and metadata changes, the replication service replicates:

• Retention class creations, modifications, and deletions for the default namespace

• All log messages relating to retention classes and privileged delete operations

On the replica, replicated directories are read-only. While a directory is read-only, users and applications cannot make any changes to its content.

Retention class conflict resolutionRetention classes can be defined on both the primary system and the replica, so the default namespace in the two systems can have classes with the same name and different values. If such a class is replicated:

• If the class on the primary system was created or modified more recently than the class on the replica and:

– The value of the class on the primary system is greater than the value of the class on the replica, the value of the class on the replica changes to the value of the class on the primary system

Note: Regardless of whether the default namespace on the replica supports appendable objects, when these objects are recovered to the primary system, they are still appendable.



Replicated systems

– The value of the class on the primary system is less than the value of the class on the replica and:

• The replica is in enterprise mode, the value of the class on the replica changes to the value of the class on the primary system

• The replica is in compliance mode, the value of the class on the replica does not change

• If the class on the replica was created or modified more recently than the class on the primary system, the value of the class on the replica does not change.

Replicated systems

Replication happens between two separate HCP systems, each of which is complete in its own right. Because replication is a software function, the two systems can have entirely different hardware configurations, including differing amounts of storage.

Note: An exception to this rule is when the value of the class on the primary system is -2 (Initial Unspecified) and the value of the class on the replica is not 0 (Deletion Allowed). In this case, the value of the class on the replica does not change.

Note: A retention class value of -1 (Deletion Prohibited) is greater than a value that’s a specific duration. A retention class value of 0 (Deletion Allowed) or -2 (Initial Unspecified) is less than a value that’s a specific duration.



Replication topologies

The two systems in a replicated pair are connected through the front-end network infrastructure, as shown in the figure below.


An HCP system can have one outbound link and up to five inbound links. This enables HCP to support various replication topologies. Replication can occur:

• In one direction between two HCP systems

• In both directions between two HCP systems (bidirectional replication)

• From multiple HCP systems to a single HCP system (many-to-one replication)

• From one HCP system to a second HCP system and from that second system to a third HCP system, such that the same HCP tenants and namespaces and default-namespace directories that are replicated to the second system are then replicated to the third system (chained replication)

Additionally, these configurations can be combined to form more complex replication topologies.

High-bandwidthnetwork

Note: Replication is a network bandwidth-intensive process. To ensure sufficient front-end network capacity when replicating, you need to give extra consideration to planning the front-end infrastructure.




Bidirectional replication

For any given pair of HCP systems, each system can serve as both a primary system and a replica for the other system, as long as different HCP tenants and different default-namespace directories are being replicated in each direction. A system that’s serving as both a primary system and a replica at the same time has two links — one outbound and one inbound. This topology is called bidirectional replication.

For each link, the selected HCP tenants and namespaces and default-namespace directories are read-write on the primary system and read-only on the replica. For an illustration of bidirectional replication, see “Replication service processing” on page 1-15.

Many-to-one replication

In a many-to-one replication topology, multiple HCP systems replicate to a single other HCP system. Each of the replicating systems is the primary system for a replication link. The other system is the replica for each of those links.

In this topology, you can think of the replica as being a hub, with the primary systems being at the ends of spokes connected to that hub. The spokes themselves are the replication links. The hub can have at most five spokes.

Each HCP tenant and default-namespace directory selected for replication on a primary system must be unique on the replica. For example, if two of the primary systems each have a tenant named Finance, the Finance tenant can be replicated from only one of the systems. For both tenants to be replicated, they must have different names.

To replicate the email directory in the default namespace on multiple primary systems, the SMTP protocol on each system must specify a different email directory name.

Note: For full functionality, all HCP systems in a many-to-one replication topology should be at release 4.0 or higher.




What this looks likeThe figure below shows a many-to-one replication topology in which three primary systems (A, B, and C) are replicating to a single replica (D).

In this figure:

• From system A, two HCP tenants are being replicated to system D. In the first tenant, two of three namespaces were selected for replication. In the second tenant, one of two namespaces was selected for replication.

Tenant-1

Tenant-2

Tenant-3

Tenant-4

Tenant-5

Tenant-6

Tenant-7

Tenant-1

Tenant-2

Tenant-3

Tenant-4

Tenant-6

Tenant-8

Replicating tenant Non-replicating tenant Replicating namespace Non-replicating namespace

Replicated tenantsLocally created tenants Replication links




• From system B, two of three HCP tenants are being replicated to system D. In the first tenant being replicated, two of three namespaces were selected for replication. In the second tenant being replicated, one of two namespaces was selected for replication.

• From system C, one of two HCP tenants is being replicated to system D. In that tenant, both namespaces were selected for replication.

• System D has one HCP tenant of its own that is not the result of replication. This tenant has two namespaces.

UsesIn a many-to-one replication topology, the replica is typically the largest HCP system. The data center in which it resides is usually fully staffed with IT personnel. The primary systems are typically smaller. Because replication links can be created and managed remotely on the replica, the data centers where these systems reside can be minimally staffed with less experienced people.

Many-to-one replication supports a scenario in which the outlying sites are branch offices of an enterprise that has an HCP system at a primary data center. Applications running at each of those offices connect over a local area network (LAN) to an HCP system at the local data center. Because they are close to the storage they use, the applications perform better than they would if they were using a wide area network (WAN) to connect to the HCP system at the hub. Also, the distribution of the network load among the outlying sites further enhances application performance.

With many-to-one replication, you can consolidate similar data from multiple sources in a single HCP system. Using a search application, you can then perform federated queries across the replicated namespaces in that system. For example, suppose each of the branch offices of an enterprise stores completed orders in a namespace in its own HCP system. If each of those namespaces is replicated to a single HCP system, you could query that system to find all orders placed for a specific product at any of the branch offices.

Many-to-one replication enables a single HCP system to be used for backup and recovery of multiple other HCP systems, so the outlying sites don’t need backup systems of their own. For additional data protection, the hub system can be backed up, for example, to tape or to another HCP system (see “Many-to-one replication with disaster recovery support” on page 1-14).

For information about backup and recovery in a many-to-one replication topology, see “Failover and recovery considerations” on page 6-8.




Chained replication

In a chained replication topology, HCP tenants and namespaces and default-namespace directories in one HCP system (A) are replicated to a second HCP system (B) and then from the second system to a third HCP system (C). To make this happen:

1. When you create the replication link from A to B, select the tenants, namespaces, and directories you want to replicate. In this link, A is the primary system and B is the replica.

2. When you create the replication link from B to C, select the link from A to B to be included in that link. This causes the HCP tenants, namespaces, and directories replicated on the link from A to B to be replicated again from B to C. In this link, B is the primary system and C is the replica.

The link from B to C can also include tenants, namespaces, and directories that were originally created on B.

You cannot individually select the replicated tenants, namespaces, and directories on B for inclusion in the link from B to C.

In a chained replication topology with the configuration A B C, B is the replica for the link from A to B. Therefore, on B, the tenants and directories that were replicated from A are read-only, even though B is also the primary system for the link from B to C.

HCP supports replication chains that consist of three HCP systems. Longer chains are not supported.

Note: For full functionality, all HCP systems in a chained replication topology must be at release 4.0 or higher.




What this looks likeThe figure below shows a chained replication topology in which system A is replicating to system B and system B is replicating to system C.

In this figure:

• From system A, two HCP tenants are being replicated to system B. In the first tenant, two of three namespaces are selected for replication. In the second tenant, one of two namespaces is selected for replication.

• From system B, the tenants and namespaces created by replication from system A are being replicated to system C because the link from A to B is included in the link from B to C.

• One tenant that was originally created on system B is also being replicated to system C. Both the namespaces in that tenant are selected for replication.

UsesIn a chained replication topology, all three HCP systems are typically located at full-scale data centers. For example, suppose a corporation has three major locations — New York, Los Angeles, and Tokyo. Each of these locations runs an application that is vital to the corporation, but only New York generates the data for these applications. Replicating the data from New York to Los Angeles and from Los Angeles to Tokyo enables applications at each location to access the data through a local area network. Because those applications are not accessing the data on the HCP system in New York, this topology also reduces the load on that system.

Chained replication helps ensure continuous data availability. If any one or even two of the HCP systems in the chain become unavailable, applications still have access to the stored data.

Tenant-1

Tenant-2

Tenant-1

Tenant-2

Tenant-3

Tenant-1

Tenant-2

Tenant-3


Locally created tenants Replicated tenants Replication links




In a replication chain, the third HCP system provides disaster recovery functionality for the second system, and the second system provides disaster recovery functionality for the first system. For information on data recovery in a chained replication topology, see “Failover and recovery considerations” on page 6-8.

In a chained replication topology, you can create and manage both replication links from the HCP system in the middle of the chain. To this system, the link from the first system is an inbound link, and the link to the third system is an outbound link.

Many-to-one replication with disaster recovery support

You can combine the many-to-one and chained replication topologies to create a configuration in which multiple HCP systems replicate to a single HCP system (many-to-one), which, in turn, replicates to another HCP system (chained). The last system provides additional assurance of continuous data availability for all the HCP tenants and namespaces and default-namespace directories originally selected for replication. It also provides disaster recovery functionality for the HCP system to which those tenants, namespaces, and directories are initially replicated.

For example, suppose HCP systems A, B, and C replicate to system D, which replicates to system E. To create this combined topology:

1. When you create the replication links from A to D, from B to D, and from C to D, select the tenants, namespaces, and directories you want to replicate.

2. When you create the replication link from D to E, select the links from A to D, from B to D, and from C to D to be included in that link. This causes the tenants, namespaces, and directories replicated on all three of those links to be replicated again from D to E.

In effect, you’re creating three replication chains: A D E, B D E, and C D E.



Replication service processing

The figure below shows a many-to-one topology with disaster recovery support.


When the replication service starts on a primary system, it immediately copies the applicable HCP tenant and namespace configurations to the replica. It then begins copying the objects from those namespaces and from any selected directories in the default namespace. The service starts with the objects with the oldest metadata changes either across all namespaces or within each namespace, depending on the link configuration.

Tenant-1

Tenant-2

Tenant-3

Tenant-4

Tenant-5

Tenant-6

Tenant-7

Tenant-1

Tenant-2

Tenant-3

Tenant-4

Tenant-6

Tenant-8

Tenant-1

Tenant-2

Tenant-3

Tenant-4

Tenant-6

Tenant-8


Locally created tenants Replicated tenants Replication links




The replication service enforces the read-write or read-only status of the tenants and directories being replicated, as applicable, on the primary system and the replica. The figure below shows this for replication that’s occurring in one direction only.

If replication is bidirectional, the tenants and directories being replicated on the primary system for each link are read-write, while the corresponding tenants and directories on each replica are read-only.

Tenantsand/or

directories

Tenantsand/or

directories

Primary system Replica

Link

Read-write

Read-only

Client

Tenantsand/or

directories

Tenantsand/or

directories

Primary system/replica

Replica/primary system

Link

Read-only

Read-only

Client

Link

Read-write

Read-write




Failover

If the primary system in a replicated pair fails, replication stops. The system administrator for the replica then needs to fail the link over to the replica. This causes the replicated tenants and directories to become read-write on the replica. The system administrator for the primary system needs to tell the administrators of the replicated tenants and directories to direct all client access requests to the replica.

Tenantsand/or

directories

Tenantsand/or

directories


LinkRead-write

Client

Important: Only one email server at a time can write to an HCP system. If the replication link includes the email directory on the primary system, in case of failover:

• If the replica has an active email directory, do not redirect the email server from the primary system to the replica.

• If the replica does not have an active email directory, be sure to redirect email to the same directory on the replica as on the primary system. For information on setting the email directory, see Managing the Default Tenant and Namespace.




If replication is bidirectional and one system fails, replication in both directions stops. When the system administrator for the surviving system fails over the link for which that system is the replica, the tenants and or directories involved become read-write on that system. The system administrator for the failed system needs to tell the applicable tenant administrators to direct all client access requests to the surviving system.

For instructions on failing over a link, see “Failing over to a replica” on page 6-4. For information about failover in a many-to-one or chained replication topology, see “Failover and recovery considerations” on page 6-8.

Recovery

After the catastrophic loss of a replicated HCP system, you need to reinstall and reconfigure the system. Once this is done, you need to restore the link configuration to that system. Then you can start the process of recovering objects, HCP tenant and namespace configurations, and other data from the replica.

During the first part of recovery processing, the applicable tenants and directories on the primary system are read-only, and the corresponding tenants and directories on the replica remain read-write. Clients must continue to direct write requests to the replica during this period.

Tenantsand/or

directories

Tenantsand/or

directories

Primary system/replica Replica/primary system

Link

Read-write

Client

Link

Read-write

Tenantsand/or

directories

Tenantsand/or

directories


Link

Read-only

Read-write

Client




While the recovery process is finishing, the applicable tenants and directories on both the primary system and the replica are read-only for a short period of time. The system administrator for the replica controls when this happens.

When recovery is complete, the applicable tenants and directories on the primary system become read-write while those on the replica become read-only again, and replication resumes. At this point, the system administrator needs to tell the tenant administrators to redirect client write requests to the primary system.

For instructions on managing the recovery process, see “Recovering from a failure” on page 6-5. For information about recovery in a many-to-one or chained replication topology, see “Failover and recovery considerations” on page 6-8.

Resolving recovery conflicts

Before a failed-over replication link is restored, clients should not be allowed to write to the replicated tenants and directories on the primary system. If they are allowed to do so, object conflicts can occur during recovery.

Tenantsand/or

directories

Tenantsand/or

directories


Link

Read-only

Read-only

Client

Tip: Because the tenants and directories on both the primary system and the replica are read-only during final recovery, this period should be scheduled for a time that will be least disruptive to clients.

Note: The replication service also handles situations in which the replica fails. For more information on this, see Chapter 6, “Managing failover and recovery.”




Similarly, before a failed-over replication link is restored, tenant administrators should not be allowed to make configuration changes to tenants, namespaces, retention classes, data access accounts, and user accounts on the primary system. If they are allowed to do so, configuration conflicts can occur during recovery.

An object conflict occurs when an object in a namespace on the replica and an object in the same namespace on the primary system have the same directory path, name, and version ID (HCP tenants only) and either of these applies:

• The objects have different creation times.

• The objects have different cryptographic hash values, which indicates that the object data doesn’t match.

If the existing and incoming objects have the same directory path, name, hash value, and creation time, they are assumed to be the same object. In this case, the replication service updates the metadata on the existing object to match that of the incoming object.

When an object copied back from the replica conflicts with an object already in the primary system, the replication service puts the incoming object in the rest/.lost+found/replication/link-id/object-path directory for an HCP namespace or the fcfs_data/.lost+found/replication/link-id/object-path directory for the default namespace. link-id is the internal identifier of the replication link and object-path is the directory path to the object below fcfs_data or rest.

You can delete a conflicting incoming object only when it’s not under retention. For more information on the .lost+found directory, see Using a Namespace or Using the Default Namespace.

A configuration conflict occurs when the configuration of a tenant, namespace, retention class, data access account, or user account differs between the replica and primary system. In this case, if the incoming item has a more recent configuration change than the item on the primary system, the incoming item replaces that item. Otherwise, the incoming item is ignored.

Note: Although the replication service can handle recovery conflicts, the need for it to do so should not arise. If you see conflicts occurring, please contact your authorized HCP service provider.



2
Configuring SSL for replication
Before replication can occur, the primary system and the replica each need to have installed a valid SSL server certificate. Each system also needs to have installed the server certificate from the other system as a trusted replication server certificate.

To install the trusted replication server certificates, the HCP administrator for each system needs to download the installed server certificate and give it to the administrator for the other system, who then uploads it. Because what’s downloaded is only the public portion of each server certificate, you can transfer these certificates unsecured.

For any given replication link, the two systems directly involved must share SSL server certificates. In a replication chain, for example, from A to B to C, A and B must share certificates, and B and C must share certificates, but A and C don’t need to do this.

This chapter provides instructions for downloading, uploading, and deleting SSL server certificates for replication. For general information on SSL server certificates for HCP, see Administering HCP.

Roles: For you to view the SSL configuration for replication, your user account must include the monitor or administrator role. For you to configure SSL for replication, your user account must include the administrator role.

Note: If you change the SSL server certificate for an HCP system, you need to download the new server certificate and have the system administrator for the other system upload it as a new trusted replication server certificate.

Configuring SSL for replication 2–1


Downloading an SSL server certificate

Downloading an SSL server certificate

To download the currently installed SSL server certificate for use with replication:

1. In the top-level menu in the HCP System Management Console, mouse over Services to display a secondary menu.

2. In the secondary menu, click on Replication.

3. On the Replication page, click on Certificates.

The Replication Server section opens. The section shows this information about the currently installed SSL server certificate:

– Server Certificate Domain — The distinguished name for the certificate

– Valid On — The date and time at which the certificate goes (or went) into effect

– Expires On — The date and time at which the certificate expires (or expired)

4. Click on the download control ( ) for the certificate. Then save the downloaded certificate in the location of your choice.

Uploading a trusted replication server certificate

To upload a downloaded SSL server certificate as a trusted replication server certificate:




4. In the Certificates panel, click on Trusted Replication.

5. In the Trusted Replication section, click on the Browse button. Then select the file containing the downloaded SSL server certificate.

6. Click on Upload Certificate button.

2–2 Configuring SSL for replication


Deleting a trusted replication server certificate

The Trusted Replication panel displays the uploaded certificate.


Occasionally, you may need to delete a replication certificate. For example, if the primary system for a replication link failed and was rebuilt, you would delete the old primary-system certificate on the replica and upload the new one.

You can delete a trusted replication server certificate only while all links in which the system participates are suspended. For example, in a chained topology A B C, if A fails and is then rebuilt, before you can delete the trusted replication server certificate on B, you need to suspend both the failed-over link to A and the healthy link to C. For instructions on suspending a link, see “Suspending and resuming activity on a replication link” on page 5-4.

To delete a trusted replication server certificate:




4. In the Certificates panel, click on Trusted Replication.

5. In the Trusted Replication section, click on the delete control ( ) for the certificate you want to delete.

6. In response to the confirming message, click on the Remove Certificate button.

Tip: You can also download a trusted replication server certificate. To do this, click on the download control ( ) for the certificate in the Trusted Replication panel.

Configuring SSL for replication 2–3



2–4 Configuring SSL for replication


3
Working with replication links
A replication link provides all the information the replication service needs to perform its functions. You can create a link in the HCP System Management Console for either the primary system or the replica.

When you initially configure a link on the primary system, you specify the replica system. When you submit the link configuration, HCP sends a link request to the replica. The replica administrator needs to accept the link before replication can start.

When you initially configure a link on the replica, replication starts when you submit the link. In this case, HCP does not send a link request to the other system.

Once a link exists, you can modify its configuration at any time. You can also delete a replication link. However, this action should be taken only with a full understanding of the consequences.

This chapter provides instructions for performing the activities mentioned above.

Roles: For you to view a replication link, your user account must include the monitor or administrator role. For you to create, accept, reject, modify, or delete a replication link, your user account must include the administrator role.

Working with replication links 3–1


Creating a replication link


You can create a replication link on either the primary system or the replica. In either case, you need to:

• Name the link.

• Indicate whether you’re creating the link on the primary system, in which case it’s an outbound link, or on the replica, in which case it’s an inbound link.

• Identify the other system.

• Specify whether you want to compress and/or encrypt data before transmitting it to the other system.

• Set the rate of activity on the link.

• Specify whether priority should be given to the objects with the oldest changes, regardless of namespace, or processing should be balanced across the namespaces.

• Identify what to replicate. This can include any number of:

– HCP tenants and, if you have administrative access to the tenant, any number of the namespaces it owns, including none

– Directories defined in the default namespace

– Replication links that are inbound to the primary system for the link you’re creating

Additionally, you can change the port on which the other system listens for transmissions from the system on which you’re creating the link. Also, if the other system uses network address translation (NAT), you need to specify how that system should communicate with the system on which you’re creating the link.

You can create a replication link only after you have installed the required SSL certificates, as described in Chapter 2, “Configuring SSL for replication.”

3–2 Working with replication links



Considerations for creating a replication link

These considerations apply to creating replication links:

• Before you can create a replication link, at least one of the following must exist on the primary system:

– An HCP tenant that you want to replicate

– The default namespace with at least one directory that you want to replicate

– An inbound replication link that you want to include in a replication chain

• You cannot include an HCP tenant in a replication link if a tenant with the same name already exists on the replica. To work around this, change the name of the tenant on either the primary system or the replica.

• You cannot include a default-namespace directory in a replication link if a directory with the same name already exists on the replica.

• You cannot include an inbound link in a replication link if the inbound link includes any HCP tenants or default-namespace directories with the same name as an existing tenant or directory on the replica.

• You can choose which namespaces to replicate for any given HCP tenant only if that tenant has granted system-level users administrative access to itself.

• If any of the namespaces owned by an HCP tenant have an explicit DPL setting that’s higher than what the replica supports, you cannot include those namespaces in the replication link. If you cannot exclude those namespaces because you don’t have administrative access to the tenant, you cannot include the tenant in the replication link.

• You can replicate a namespace with an explicit DPL setting of one to an HCP RAIN system that doesn’t support DPL one. In this case, the namespace on the replica retains its DPL setting of one despite the restriction.




• If the primary system for a replication link is release 4.0 or later and the replica is an earlier release:

– All namespaces owned by each HCP tenant included in the link are automatically selected for replication.

– The link cannot include HCP tenants that own any namespaces with a DPL setting of Dynamic.

– The DPL of an HCP namespace that’s included in the link cannot be changed after replication starts.

• If you don’t want HCP tenant-level users to see replication details, ensure that the option that allows this is deselected before you create the replication link. For more information on this, see “Controlling the Tenant Management Console replication display” on page 4-9.

What you do

To create a replication link on either the primary system or the replica:



3. On the Replication page, click on Create Link (or Create Inbound Link if the HCP system already has an outbound link).

4. In the Create Link or Create Inbound Link panel:

– In the Link Name field, type a name for the link. This name must be from one through 64 characters long and can contain any valid UTF-8 characters, including white space. Link names are not case sensitive.

For any given HCP system, each replication link in which the system participates as either the primary system or the replica must have a unique name.

– In the Create Link panel only, under Link Type, select Outbound if you’re creating the link on the primary system or Inbound if you’re creating the link on the replica.




– In the Replica Addresses field for an outbound link or the Source Addresses field for an inbound link, type either the DNS name of the other system or one or more comma-separated IP addresses of storage nodes in that system. For the DNS name, use this format:

replication.hcp-name.domain-name

For example:

replication.hcp-ca.example.com

– In the Performance Level field, select Low, Medium, High, or Custom to set the rate of replication or Use Schedule to have the replication rate vary according to a schedule. The performance level determines how many concurrent replication operations can occur on each node.

For more detailed information about performance levels, see Appendix B, “Changing the custom performance level.” For information on scheduling performance levels, see “Scheduling activity on a replication link” on page 5-2.

Note: Regardless of whether you specify a DNS name or IP addresses, the primary system transmits data to all storage nodes in the replica. Likewise, during data recovery, the replica transmits data to all storage nodes in the primary system.

Notes:

• Performance level is a per-link setting. In a replication chain, the two links can have different performance levels.

• When data recovery begins on a replication link, the link performance level automatically changes to High. When data recovery is complete, the level automatically returns to its previous setting, even if you’ve changed it manually during data recovery.

Tip: If the system load from other activities is light, consider increasing the rate of replication to reduce any backlog. If the load is heavy, consider lowering the rate to free up resources for other system activity.




– Optionally, select the Compress data option to compress data before it’s transmitted from one system to the other.

Compressing data increases network throughput but also increases processing time. Therefore, because compression applies to all transmitted data, you should enable it only if most of the data being replicated is compressible.

– Optionally, select the Encrypt data option to encrypt data before it’s transmitted from one system to the other. Encrypting data keeps it confidential during transmission. The transmitted data is automatically decrypted on the target system.

If you don’t select this option, data is transmitted unencrypted, even if the data is encrypted on disk in the system sending it. However, regardless of whether the data is encrypted before transmission, it is still secured by SSL during transmission.

– In the Replication Priority field:

• To replicate objects with the oldest changes first, regardless of which namespaces they’re in, select Oldest Object First.

• To have the replication service balance its processing time evenly across the namespaces being replicated, select Balanced Across Tenants.

– Optionally, specify a different port for the other system to listen on and/or network address translation (NAT) information for the system on which you’re creating the link. For instructions on this, see “Advanced configuration” on page 3-9.

5. If you’re creating the link on the replica, click on the Discover button. When you do this, the primary system sends information about its HCP tenants, default-namespace directories, and inbound links to the replica.

6. Select the items you want to replicate. For instructions on this, see “Item selection” below.

Note: For network configurations that use NAT, multiple IP addresses must be configured to expose the HCP nodes. You cannot use multiple ports on the same IP address.




7. Click on the Create Link button:

– If you’re creating the link on the primary system, HCP sends a link request to the replica. In the System Management Console for both the primary system and the replica, the Replication page lists the link with a mode of Pending.

While a link request is pending, you can change the link configuration in the Configuration panel on the primary system. When you click on the Update Pending Link button, HCP sends the changes to the replica.

For information on how to handle a pending link request, see “Accepting or rejecting a link request” on page 3-11.

– If you’re creating the link on the replica, the replication service on the primary system immediately begins transmitting data to the replica.

Item selection

For both outbound and inbound links, the bottom section of the Create Link or Create Inbound Link panel lists this information about the primary system for the link you’re creating:

• All HCP tenants in the primary system that are eligible for replication

• All top-level directories in the default namespace in the primary system

• All replication links that are inbound to the primary system

Selecting HCP tenants and namespacesTo select the HCP tenants and namespaces you want to include in the replication link:

1. Click on the Tenants tab.

Tip: If HCP cannot create the requested link, check that the other system is healthy and that you’ve correctly shared SSL server certificates between the two systems. If the default tenant is included in the link, check that the default namespaces on the two systems use the same cryptographic hash algorithm and have the same retention mode.




The Console displays a list of the HCP tenants that are eligible for replication. This list does not include tenants that are currently included in an inbound link.

If a tenant has granted system-level users administrative access to itself, the tenant row shows the number of namespaces selected for replication out of the total number of namespaces the tenant owns. To see a list of all the namespaces, click in the tenant row. The namespaces that are currently selected for replication are checked.

To see the namespaces owned by all the tenants to which you have administrative access, click on the expand all link.

2. For each tenant you want to include in the replication link:

a. Select the tenant. To select all the listed tenants, select Tenants at the top of the list.

b. Optionally, if the tenant has granted system-level users administrative access to itself, display the list of namespaces owned by the tenant. Then select the namespaces you want to replicate and deselect the ones you don’t want to replicate.

To undo the changes you’ve made to the tenant and namespace selections, click on the reset link.

Selecting default-namespace directoriesTo select the default-namespace directories you want to replicate:

1. Click on the Default Tenant tab.

The Console displays a list of all the top-level directories in the default namespace, except for those that are currently included in an inbound link.

2. Select each directory you want to include in the replication link. To select all the listed directories, select Directories at the top of the list.

To undo the changes you’ve made to the directory selections, click on the reset link.

Note: If the default namespace contains a large number of top-level directories, the Console may take a long time to display them.




Selecting inbound linksTo select the inbound links you want to include in the replication link:

1. Click on the Inbound Links tab if you’re creating an outbound link or the Cascading Inbound Links tab if you’re creating an inbound link.

The Console displays a list of the replication links that are inbound to the primary system. The row for each link shows the number of HCP tenants and default directories included in the link. To see a list of these items, click in the link row. To see the lists of items for all the inbound links, click on the expand all link.

If a listed tenant has granted system-level users administrative access to itself, the tenant row shows the number of namespaces selected for replication. To see a list of these namespaces, click in the tenant row.

2. Select each inbound link you want to include in the replication link. To select all the listed links, select Inbound Links at the top of the list.

To undo the changes you’ve made to the link selections, click on the reset link.

Advanced configuration

To specify a different port for the other system to listen on and/or NAT information for the system on which you’re creating the replication link:

1. In the Create Link or Create Inbound Link panel, click on the Advanced Configuration link.

2. Do either or both of these:

– In the Replica Port field for an outbound link or the Source Port field for an inbound link, type the number of the port on which the other system will listen for data from the system on which you’re creating the link. The default port is 5748. Typically, you specify a different port only if other port usage makes it necessary.

Note: If an inbound link contains a large number of top-level directories from the default namespace, the Console may take a long time to display them.



Canceling a link request

– If the system on which you’re creating the link uses NAT to communicate with the other system, specify the target for data transmissions from the other system:

• In the Local Addresses field, type either the DNS name of the system on which you’re creating the link or one or more comma-separated IP addresses of storage nodes in that system (as those addresses are known to the other system). Make sure the other system can resolve the DNS name or IP addresses you specify.

• In the Local Port field, type the number of the port on which the system on which you’re creating the link will listen for data from the other system. This is the port that’s exposed to the other system.

Canceling a link request

To cancel the request for an outbound replication link before it’s accepted:



3. In the list of replication links, click on the link for which you want to cancel the request.

The link Configuration panel opens. You can review the link configuration in this panel before deciding whether to delete the request.

4. In the Configuration panel, click on the Delete Link Request button.

Note: The other system will transmit data only to the nodes identified by the DNS name or IP addresses you specify. Therefore, if you’re using IP addresses, you should specify the addresses of all the storage nodes in the system.



Accepting or rejecting a link request

Accepting or rejecting a link request

When you create an outbound replication link, HCP sends a link request to the system identified as the replica in the link configuration. In the HCP System Management Console for that system:

• The Overview page displays an alert indicating that a link request is pending.

• The Replication page lists the link with a mode of Pending.

To review and accept or reject an replication link request:

1. Display the Replication page in the System Management Console for the target system. You can do this in either of these ways:

– In the alerts section on the Overview page, click on the link name below the alert about the pending request.

– In the top-level menu, mouse over Services. Then click on Replication in the secondary menu.

2. In the list of replication links, click on the pending link request.

The link Configuration panel opens.

3. Review the link configuration. For descriptions of the configuration fields, see “Creating a replication link” above.

Note: In the Configuration panel on the system receiving the link request:

• The Replica Addresses field is replaced by a Primary Addresses field.

• The Replica Port field is replaced by a Primary Port field.

• The Local Addresses field is replaced by a Replica Addresses field.

• The Local Port field is replaced by a Replica Port field.



Modifying a replication link

4. Do one of these:

– To accept the replication link, click on the Accept Link button.

HCP sends an acceptance message to the primary system, which immediately starts replicating.

– To reject the replication link, click on the Reject Link button.

HCP sends a rejection message to the primary system, and both systems delete the pending link.


You can change the configuration of a replication link at any time in the System Management Console for either the primary system or the replica except while either system is being upgraded. In this case, do not modify the link until the upgrade has been finalized.

Considerations for modifying a replication link

Most of the considerations for creating a replication link (described in “Considerations for creating a replication link” on page 3-3) apply as well to modifying a link. Additionally, when you modify a link:

• You can change any property of a link except the link direction (outbound or inbound).

• If you change the DNS name or IP addresses for either system, the new DNS name or IP addresses must identify the same system. The exception to this is when you’re reconfiguring the link before restoring it to a new or rebuilt system.

Note: If both the primary system and the replica are already running HCP release 4.0 or higher, the System Management Console on both systems does not allow you to modify the link during an upgrade. If one system is being upgraded to release 4.0 or higher and the other system is running a release earlier than 4.0, the System Management Console on the system being upgraded does not allow you to modify the link, but the Console on the other system does.




• You can add and remove HCP tenants and default-namespace directories from a link at any time, except while the link is failed over to the replica or during an upgrade of either the primary system or the replica.

• You can add HCP namespaces to a link at any time except during an upgrade of either the primary system or the replica. You can remove HCP namespaces from a link at any time, except while the link is failed over to the replica or during an upgrade of either system.

• In a replication chain with the configuration A B C, if you add an HCP tenant to the link from A to B and a tenant with the same name already exists on C, replication of that tenant from B to C is automatically paused. To recover from this situation, you can either rename or delete the tenant on C and then resume replication of the tenant on the link from B to C, or you can remove the tenant from the link from A to B.

Renaming the tenant on C is a solution only if the tenant was originally created on C rather than being the result of a previous replication from A to B to C.

For information on resuming replication of an automatically paused tenant, see “Pausing and resuming tenant replication or recovery” on page 5-5.

• In a replication chain with the configuration A B C, if the default namespace doesn’t exist on C and you add a default-namespace directory to the link from A to B, replication of the default tenant from B to C is automatically paused. To recover from this situation, you need to create the default tenant and namespace on C and then resume replication of the default tenant on the link from B to C.

Replication of the default tenant from B to C is also automatically paused if the default namespace exists on C but has a different retention mode or cryptographic hash algorithm from the default namespaces on A and B.

Important: Once you remove a tenant or directory from a link, you cannot add it back unless you first delete it from the replica. This is because the tenant or directory now exists on both the primary system and the replica.

In this case, unlike when you’re creating a link, renaming the tenant on either the primary system or the replica doesn’t help because the two tenants have the same internal ID.




• If the DPL of an HCP namespace that’s included in a link is changed to an explicit DPL that’s not supported on the replica, replication of the tenant that owns the namespace is automatically paused. To recover from this situation, have the tenant administrator change the DPL back to a value that the replica supports (or to Dynamic). Then resume replication of the tenant.

• When you add one or more default-namespace directories to a link, some objects in directories that were already included in the link may be reprocessed. This can increase the replication backlog for the default namespace.

• While updating the link configuration, HCP temporarily suspends activity on the link.

What you do

To change the configuration of a replication link:



3. On the Replication page, click on the name of the link you want to modify.

4. In the left side of the panel that opens, click on Configuration.

5. In the Configuration panel, make the changes you want:

– To change the link name, identification of the other system, compression and encryption options, performance level, or replication priority, click on Link Settings.

Note: When modifying a link on the primary system, the field labels in the Advanced Configuration section are, in order, Replica Port, Primary Addresses, and Primary Port.

When modifying a link on the replica, the Source Addresses field label is replaced by Primary Addresses, and the field labels in the Advanced Configuration section are, in order, Primary Port, Replica Addresses, and Replica Port.



Deleting a replication link

– To change the HCP tenants or namespaces, default-tenant directories, or inbound links included in the link, select or deselect the applicable items.

6. Click on the Update Link button.

If you deselected an HCP tenant, default-namespace directory, or inbound link, the Console displays a confirming message.

In the window with the confirming message, click in the checkbox to confirm that you understand the consequences of your action. Then click on the Update Link button.

HCP suspends replication or data recovery, whichever is currently happening, and makes the change to the link configuration. Once the change is complete, activity on the link returns to its previous state.


When you delete a replication link, replication on that link immediately stops. The HCP tenants and default-namespace directories included in the link remain or become read-write on both the primary system and the replica.

Note: When you click on the Default Namespace tab, if the default namespace contains a large number of top-level directories, the Console may take a long time to display them. Similarly, if you expand an inbound link that contains a large number of top-level directories, the Console may take a long time to display them.

Important: Once you delete a replication link, you can no longer replicate the tenants and directories included in it to the same replica unless you first delete them on the replica. When you create a new link with the same replica, HCP doesn’t let you include those tenants and directories because they now exist on both systems. If the possibility exists that you will want to replicate those tenants and directories again to the same replica, you should either pause replication for the individual tenants or suspend activity on the link rather than delete the link.

HCP uses internal IDs to identify tenants. Therefore, even if you change the name of a previously replicated tenant on either the primary system or the replica, you cannot include it in the new link.




You can delete a replication link only while activity on it is suspended. For information on suspending activity on a link, see “Suspending and resuming activity on a replication link” on page 5-4.

To delete a replication link, you can use the HCP System Management Console for either the primary system or the replica:



3. On the Replication page, click on the name of the link you want to delete.

4. In the left side of the panel that opens, click on Management.

5. In the Management panel, click on the Delete button.

6. In response to the confirming message, click on the Delete Link button in the message window.



4
Monitoring replication
You can monitor the status of each currently configured replication link, as well as replication and recovery activity on the link, in the HCP System Management Console for either the primary system or the replica. By periodically reviewing link activity, you can decide whether to change the replication rate in the link configuration to accommodate other loads on system performance. You can also determine whether you need to add more storage capacity to the replica.

Under normal circumstances, the replication service works without any intervention required. System Management Console alerts inform you of conditions, such as network connection problems, that may require action on your part to allow the service to continue processing.

For HCP tenants, you can control whether information about replication activity is displayed in the Tenant Management Console. For the default tenant, this information is always displayed.

This chapter describes the information available to you for the links you’ve created.

Roles: For you to monitor replication, your user account must include the monitor or administrator role.

Monitoring replication 4–1


Link information

Link information

The Replication page in the HCP System Management Console lists the currently defined replication links for which the system is either the primary system or the replica. To display this page:



For each listed link, the Replication page shows:

• The link name.

• The type of link — outbound or inbound.

• The current mode for the link. Possible values are:

– Pending — The HCP system administrator for the replica has not yet responded to the request to create the link.

– Replicating — For outbound links during replication, the primary system is copying data to the replica.

– Receiving — For inbound links during replication, the replica is receiving data from the primary system. For outbound links during data recovery, the primary system is receiving data from the replica.

– Failed Over — For outbound and inbound links during failover, the link is failed over to the replica.

– Recovering — For inbound links during data recovery, the replica is copying data to the primary system.

– Final Recovery — For inbound links during data recovery, the data recovery process is being completed. Both the primary system and the replica are read-only.

Note: Link mode is separate from link status, which is described below. For example, if the mode of a link is Replicating and you suspend the link, its status is Suspended, but its mode is still Replicating.

4–2 Monitoring replication


Link information

• The number of nodes on the other system that can receive replication transmissions from the current system.

• The current status of the link, represented by one or more icons. Possible icons are:

– (Link is suspended) — Activity on the link has been manually suspended. No replication or recovery processing is occurring.

– (Scheduled suspend) — Activity on the link has been suspended because the scheduled performance level for the current time period is Off. No replication or recovery processing is occurring.

– (Tenant(s) paused from replication) — Replication or recovery processing has been paused for one or more tenants included in the link.

– (System automatically paused tenant(s) from replication) — HCP has automatically paused replication or recovery processing for one or more tenants included in the link. This is typically due to a tenant name collision. For example, consider this scenario:

1. System A replicates to system B on link AB, which replicates to system C on link BC, where BC includes the inbound link AB.

2. A and C each have a tenant named T2, where T2 was created independently on each system.

3. You add T2 to link AB. T2 is replicated to B.

4. Because link BC includes link AB, the replication service tries to replicate T2 to C. The replication is unsuccessful because T2 already exists on C. As a result, the service automatically pauses replication of T2 on BC.

– (Replica storage full (link suspended)) — The replica does not have enough free storage to accept any more data transmissions. HCP has automatically suspended activity on the link.

– (High error rate) — Errors are occurring at a high rate. If you see this status during replication, check the health of the replica. If you see it during data recovery, check the health of the primary system. In either case, also check the health of the network. If you cannot find the problem, please contact your authorized HCP service provider.



Link overview

– (Link is unresponsive) — The other system doesn’t recognize the link. Restore the link. Then, if applicable, start data recovery. For these procedures, see “Recovering from a failure” on page 6-5.

– (Link is broken) — The replication service cannot contact any nodes on the other system. If you see this status during replication, check the health of the replica. If you see it during data recovery, check the health of the primary system. In either case, also check the health of the network. If you cannot find the problem, please contact your authorized HCP service provider.

If no status icons are present, the link is functioning normally.

Link overview

The list of links on the Replication page shows the current status of each replication link. To further monitor the status of and activity on a link, click on the name of the link in the list. The Console displays the Overview panel for that link.

The Overview panel shows:

• The link mode.

• For outbound links only, the approximate number of objects currently waiting to be replicated (Objects pending). This is the sum of the numbers of objects waiting to be replicated in each HCP namespace included in the link. This number does not include objects in the default namespace.

• For outbound links only, the approximate amount of data currently waiting to be replicated (Data pending). This is the sum of the amounts of data waiting to be replicated in each HCP namespace included in the link. This value does not include data in the default namespace.

• For outbound links only, the backlog time (Backlog time). Backlog time is the difference between these:

– The oldest change time for objects and configuration changes not yet replicated or recovered, as applicable

– The current time

Note: The System Management Console may not immediately reflect certain changes in link status.



Link overview

• A graph showing the rate of data transmission on the link.

• A graph showing the rate of replication operations on the link.

• For outbound links only, a pie chart showing the amount of used and free storage space on the replica.

While a link is failed over to the replica, the Overview panel for the link on the primary system is blank.

Data transmission rate

The Transfer Rate graph in the Overview panel for an outbound or inbound link shows the history of the rate of replication or recovery data transmissions per second, as applicable. This rate is cumulative for all the HCP tenants and default-tenant directories being replicated or recovered on the link. If the graph is not currently visible, click on Transfer Rate to display it

The x-axis in the Transfer Rate graph marks the passage of time. It shows 30 days (or less if the replication link was created less than 30 days ago). The y-axis measures the data transmission rate. As the transmission rate varies, the measurement units on the y-axis grow or shrink as needed (for example, from KB to MB to GB).

The graph heading displays the current data transmission rate. Factors that affect this rate include the amount of other traffic on the network, the current load on both the primary system and the replica, and whether data is being compressed or encrypted. Also, larger objects have higher rates of throughput.

Operation rate

The Operations per Second graph in the Overview panel for an outbound or inbound link shows the history of the rate of replication or recovery operations per second. This rate is cumulative for all the HCP tenants and default tenant directories being replicated or recovered on the link. If the graph is not currently visible, click on Operations per Second to display it.

An operation is the replication of any of these:

• A data object, data directory, symbolic link, metadata change, or object deletion

• A tenant, namespace, data access account, or configuration change to or deletion of a tenant, namespace, or data access account



Tenants view

• The creation, modification, or deletion of a retention class

• For HCP tenants only, the creation, modification, or deletion of an administrative user account

• A tenant log message

The x-axis in the Operations per Second graph marks the passage of time. It shows 30 days (or less if the replication link was created less than 30 days ago). The y-axis measures the operation rate. As the transmission rate varies, the intervals on the y-axis grow or shrink as needed (for example, from tens to hundreds).

The graph heading displays the current rate of replication operations. Factors that affect the operation rate are the same as those that affect the data transmission rate.

Space used on the replica

The Space Available on Replica chart in the Overview panel for an outbound link during replication shows the amounts of free and used storage space in the replica. Each amount is also shown as a number of bytes.

A primary system and its replica do not necessarily have the same storage capacity. As the replica storage fills up, you need to evaluate whether you need to increase its capacity.

When the replica storage is almost full, the Overview page in the HCP System Management Console for the primary system displays an alert. When the storage is full, the Console displays a different alert, and HCP automatically suspends replication on the link. For more information on these alerts, see “System Management Console alerts” below.

Tenants view

For any given outbound or inbound replication link, you can monitor the replication or recovery status and backlog time for each tenant included in the link. To see this information for a tenant:



3. On the Replication page, click on the link name in the list of links.



Tenants view

4. In the left side of the panel that opens, click on Tenants.

The top of the Tenants panel shows the same information as the top of the Overview panel shows: the link mode and for outbound links only, the approximate number of objects pending in HCP namespaces, the approximate amount of data pending in HCP namespaces, and the backlog time.

While a link is failed over to the replica, the Tenants panel for the link on the primary system is blank.

Tenant replication information

The Tenants panel lists the tenants included in the replication link, including the default tenant if any directories in it are included in the link. For each tenant, the list shows:

• The tenant name.

• A progress bar that measures the average backlog time for the replicating namespaces owned by the tenant, along with this time as a numeric value in days or in hours if it’s less than a day. The length of the progress bar represents 30 days, with the right end representing the current time.

For the default tenant, this is the average backlog time among the directories included in the link.

Tips:

• If some tenants have a significantly greater backlog than others, set the replication priority for the link to Oldest Object First to reduce those backlogs.

• If the backlog time is consistently increasing for one or more tenants, even with the replication performance level set to High, you may need to add more processing power (for example, additional nodes) to your system or increase the bandwidth between the primary system and the replica.



Tenants view

• If the tenant is included in a replication chain, this icon:

• The current replication status of the tenant, represented by one of these icons:

– — If the icon is animated, replication or recovery of the tenant is proceeding normally. If the icon is static, all activity on the replication link has been suspended.

– — Replication of the tenant was paused.

– — Replication of the tenant was paused automatically. When you mouse over this icon, the Console displays the reason why the replication of the tenant was paused.

Tenant replication details

To see more details about the replication of a tenant, click on the tenant name in the list in the Tenants panel. The information shown is:

• The shortest backlog time among the namespaces owned by the tenant (Minimum backlog). This time is shown in days or in hours if it’s less than a day.

For the default tenant, this is the shortest backlog time among the directories included in the replication link.

• The longest backlog time among the namespaces owned by the tenant (Maximum backlog). This time is shown in days or in hours if it’s less than a day.

For the default tenant, this is the longest backlog time among the directories included in the replication link.

• The approximate number of objects currently waiting to be replicated in the namespaces owned by the tenant (Objects pending). This value is shown only for HCP tenants in an outbound link.

• The approximate amount of data currently waiting to be replicated in the namespaces owned by the tenant (Data pending). This value is shown only for HCP tenants in an outbound link.



Controlling the Tenant Management Console replication display

Controlling the Tenant Management Console replication display

If a tenant is eligible for replication, the Tenant Management Console on both the primary system and the replica always shows the mode of the tenant replication. This mode can be Replicating, Receiving, or Not Replicating, which means the tenant is eligible for replication but is not currently included in a replication link.

For HCP tenants, you can control whether the Tenant Management Console also displays information about the tenant replication rate and backlog. The Tenant Management Console for the default tenant always displays this information.

To do this:



3. On the Replication page, click on Settings.

4. Do either of these:

– To have the Tenant Management Console display replication and status and backlog information for all HCP tenants, select the Allow tenants to monitor replication of their namespaces option.

– To have the Tenant Management Console hide replication and status and backlog information for all HCP tenants, deselect the Allow tenants to monitor replication of their namespaces option.



System Management Console alerts


The Overview page in the HCP System Management Console has a section in which it displays alerts about abnormal conditions. Each alert consists of an icon and accompanying text that identifies the problem. The table below describes the alerts that relate to replication.

Icon Text Description

Replication link pending Either the system has sent a request for a replication link to another system and is waiting for a response, or the system has received a request for a replication link and has not yet responded.

Replication link failure An outbound replication link is not working as expected. Check the network connection between the primary system and the replica. If the connection appears to be working properly, please contact your authorized HCP service provider.

Replication link stalled Replication has stopped on a replication link. Check the network connection between the primary system and the replica. If the connection appears to be working properly, please contact your authorized HCP service provider.

Replica storage almost full The replica storage space is at least 90% used. Consider adding more storage capacity to the replica.

This alert appears only in the System Management Console for the primary system.

Replica storage full (link suspended) The replica storage space is 94% used. The replica cannot act on any more replication data transmissions from the primary system. HCP has automatically suspended activity on the replication link.

Add more storage capacity to the replica. Then resume activity on the replication link.

This alert appears only in the System Management Console for the primary system.

System auto-paused tenant(s) from replication

HCP has automatically paused replication of one or more HCP tenants. For each tenant, correct the situation that caused replication to be paused. Then resume replication of the tenant.




For more information on alerts, see Administering HCP.

Front-end connection error All front-end connections to one or more nodes are unavailable. If this system is currently the source for replication, some objects will not be replicated. If the system is the replication target, replication performance will be degraded.

(Continued)

Icon Text Description



5
Managing replication
Part of the configuration of a replication link is the performance level for replication and recovery activity on that link. You can set this level explicitly, or you can use a schedule to change the performance level automatically at specific times.

Occasionally, you may need to temporarily stop all replication activity on a link. Or, you may want to temporarily stop replication activity only for particular tenants.

This chapter explains how to perform these tasks.

Managing replication 5–1


Scheduling activity on a replication link


The performance level for a replication link determines how much load replication or recovery processing puts on the HCP systems involved, as well as on the network connecting them. You can change the performance level yourself at any time by modifying the link configuration. Alternatively, you can schedule performance-level changes to happen automatically at specified times.

Replication schedules are link specific. Multiple links can have the same schedule, but you need to set the schedule individually for each link.

To have the replication service use the schedule for a link, set the performance level for the link to Use Schedule. For information on changing link properties, see “Modifying a replication link” on page 3-12.

For more detailed information about performance levels, see Appendix B, “Changing the custom performance level.”

About scheduling

You use the Schedule panel on the Replication page to set the schedule for activity on a replication link. This panel contains a grid in which the weekdays from Sunday through Saturday are each broken out into 24 hours.

To set a schedule, you assign performance levels to time periods. The Schedule panel makes it easy for you to do this for the whole week, individual days, individual hours, or ranges of hours within a day.

You can set the performance level for any given time period to Low, Medium, High, Custom, or Off. While the performance level is Off, all activity on the link is suspended. Activity is resumed automatically when the next scheduled performance-level change occurs.

You cannot explicitly resume activity on a link while this activity is suspended due to the schedule. However, you can change the performance level in the link configuration to something other than Use Schedule. Link activity then resumes automatically at the new level.

5–2 Managing replication



Performance levels are color coded on the schedule grid so you can easily see which levels are assigned to which time periods:

• Low: (orange)

• Medium: (white)

• High: (green)

• Custom: (blue)

• Off: (pink)

The first hour of each time period with a given performance level displays the start and end times for that period (for example, 8 am to 6 pm). These times are in the time zone of the primary system.

What you do

By default, in the schedule for a replication link, the performance level for the whole week is Medium. To change the schedule for a link:



3. On the Replication page, click on the link name in the list of links.

4. In the left side of the panel that opens, click on Schedule.

5. Do any of these as many times as needed to set the schedule you want:

– To set the performance level for the entire week:

a. Mouse over All to display the list of performance levels.

b. Click on the performance level you want.

– To set the performance level for an individual day:

a. Mouse over the name of the day to display the list of performance levels.

b. Click on the performance level you want.



Suspending and resuming activity on a replication link

– To set the performance level for a single hour or a range of hours:

a. Either click on an hour, or click and drag from one hour to another in the same day.

The Set Performance Level window opens.

b. Optionally, select different start and end times in the Start Time and End Time fields.

c. In the Level field, select the performance level you want.

d. Click on the Submit button.

6. Click on the Update Schedule button.

Suspending and resuming activity on a replication link

You can suspend and resume all activity on a replication link. You need to suspend link activity, for example, before upgrading HCP to a new release or making changes to system hardware or to the network through which the primary system and replica communicate with each other. While replication or recovery activity is suspended, each system remains read-write or read-only, as applicable.

The replication service periodically checkpoints its progress. When you suspend activity on a link, no special checkpoint occurs. When you resume link activity, therefore, processing starts from the last checkpoint before the suspension.

To suspend or resume link activity, you can use the HCP System Management Console for either the primary system or the replica:

1. In the top-level menu in the System Management Console, mouse over Services to display a secondary menu.


3. On the Replication page, click on the name of the link on which you want to suspend or resume activity.

Tip: To include the last hour of a day in a range of hours, you need to drag down to the exact bottom of that hour. An easier way to include that hour is to drag down to the previous hour and reset the end time in the Set Performance Level window.



Pausing and resuming tenant replication or recovery

4. In the left side of the panel that opens, click on Management.

5. In the Management panel, click on the Suspend or Resume button, as applicable.


You can pause and resume replication or recovery for an individual tenant that’s included in a replication link. You might pause replication of some tenants, for example, to give more processing time to other tenants with greater backlogs.

Replication of a tenant can also be paused automatically due to certain events, such as the DPL of one of its replicating namespaces being changed to a value that’s not supported on the replica. In this case, you cannot resume replication of the tenant until the issue that caused replication to pause is resolved.

The replication service periodically checkpoints its progress. When you pause replication or recovery processing for a tenant, no special checkpoint occurs. When you resume processing, therefore, processing starts from the last checkpoint before the pause.

To pause or resume replication or recovery for an individual tenant:

1. In the top-level menu in the System Management Console, mouse over Services to display a secondary menu.


3. On the Replication page, click on the name of the link that includes the tenant for which you want to pause or resume replication or recovery.

4. In the left side of the panel that opens, click on Tenants.

5. In the list of tenants, click on the pause control ( ) or resume control ( ), as applicable, for the tenant for which want to pause or resume activity.

Tip: The HCP MIB contains a field named replicationLinkSuspended that you can use to suspend and resume replication. You can schedule activity on a replication link by using SNMP to change the value in this field. For more information on the HCP MIB, see Administering HCP.



6
Managing failover and recovery
In the case of a catastrophic failure of the primary system in a replicated pair, clients of the primary system can be directed to read replicated content from the replica. However, the replicated HCP tenants and namespaces and default-namespace directories on the replica are still read-only, so clients cannot make any changes to the replicated content.

To enable clients to write to the replicated entities on the replica, you need to manually fail the replication link over to the replica. Failing over a link makes the replicated entities read-write on the replica.

After the primary system is repaired or rebuilt, you need to restore the link and start data recovery. When data recovery is almost complete, you need to explicitly start final recovery processing.

If the system that failed was the replica, the applicable tenants and directories remain read-write on the primary system, so no failover needs to occur. After the replica is repaired or rebuilt, you need to restore the link. When you do that, replication restarts automatically.

This chapter provides instructions for managing both failover and recovery. It also describes considerations for failover and recovery with many-to-one and chained replication topologies.

For more information on failover and recovery, see “Replication service processing” on page 1-15.

Roles: For you to fail over a replication link, restore a link, or recover replicated data, your user account must include the administrator role.

Managing failover and recovery 6–1


Failover and recovery workflows


Two failover and recovery scenarios are possible — one in which the primary system fails, the other in which the replica fails. The following sections describe the basic workflows for these scenarios. For information on failover and recovery workflows in many-to-one and chained replication topologies, see “Failover and recovery considerations” on page 6-8.

Primary system failure workflow

The table below outlines what happens when the primary system in a replicated pair fails.

Step What you do What happens

Primary system fails

1 On the replica, fail over the link Applicable tenants and directories on the replica become read-write

2 Direct clients to write only to the replica

Primary system comes back online

3 If the primary system has been rebuilt:

• On the primary system, upload the replication SSL server certificate from the replica

• On the replica, upload the replication SSL server certificate from the primary system

4 On the replica, update the link configuration as needed

5 On the replica, send a request to restore the link

Pending link request appears on the primary system

6 On the primary system, accept the restored link

Replication link is recreated in the suspended state

7 On the replica, begin data recovery Applicable tenants and directories on the replica remain read-write; applicable tenants and directories on the primary system are read-only; the link is resumed; data recovery from the replica to the primary system begins

6–2 Managing failover and recovery



Replica failure workflow

The table below outlines what happens when the replica system in a replicated pair fails.

8 Wait for the data recovery backlog time to approach zero

9 On the replica, complete data recovery

Applicable tenants and directories on the replica become read-only; applicable tenants and directories on the primary system remain read-only; data recovery from the replica to the primary system continues to completion

Data recovery finishes

10 Nothing Applicable tenants and directories on the replica remain read-only; applicable tenants and directories on the primary system become read-write; replication from the primary system to the replica resumes

11 After you see this message in the system log, direct clients to write only to the primary system: Replication data recovery completed

(Continued)



Replica fails

1 Suspend replication on the link

Replica comes back online

2 If the replica has been rebuilt:

• On the replica, upload the replication SSL server certificate from the primary system

• On the primary system, upload the replication SSL server certificate from the replica

3 On the primary system, update the link configuration as needed



Failing over to a replica

Failing over to a replica

If a primary system fails, you need to manually fail over the replication link in the HCP System Management Console for the replica. Failing over the link causes the applicable HCP tenants and namespaces and default-namespace directories on the replica to become read-write.

To fail over a replication link:

1. In the top-level menu in the System Management Console for the replica, mouse over Services to display a secondary menu.


3. On the Replication page, click on the name of the link you want to fail over.

4. In the left side of the panel that opens, click on Advanced.

4 On the primary system, send a request to restore the link

Pending link request appears on the replica

5 On the replica, accept the restored link

Replication link is recreated; applicable tenants and directories on the primary system remain read-write; applicable tenants and directories on the replica are read-only; the link is resumed; replication from the primary system to the replica restarts from the beginning

(Continued)


Important: Once failover has occurred:

• You need to direct all clients to write to the replicated namespaces and directories on the replica only. Do not allow clients to write to the replicated namespaces and directories on the primary system until the data recovery process is complete.

• The only way to return to normal replication is by going through the data recovery procedure. You need to perform this procedure even if you don’t need to restore the link and even if no changes have been made to the configuration or content of the replicated entities. Even though nothing has changed, the data recovery process in this case can take upwards of five minutes.



Recovering from a failure

5. In the Advanced panel, click on the Fail Over button.

6. In response to the confirming message, click on the Fail Over Link button.


During a catastrophic failure, an HCP system can lose all configuration information. If the system is involved in replication, you need to restore the link configuration after the system is rebuilt. If the primary system failed, you then need to recover namespace content and other applicable information from the replica. If the replica failed, replication automatically restarts, beginning again with the objects with the oldest metadata changes either across all namespaces or within each namespace, depending on the link configuration.

Restoring a link

Before you can restore a replication link, you need to ensure that both the primary system and the replica have the required SSL certificates installed, as described in Chapter 2, “Configuring SSL for replication.”

To restore a replication link after a system failure:

1. In the top-level menu in the HCP System Management Console for the primary system or the replica, as applicable, mouse over Services to display a secondary menu.


3. On the Replication page, click on the name of the link you want to restore.

4. If necessary, update the link configuration. You need to do this only if the DNS name or IP addresses of the other system have changed.

Tip: To prevent the content verification and protection services from trying to repair from the primary system after you’ve failed over to the replica, suspend the replication link.

Note: If the HCP system that failed still has the link configuration when it becomes available again, you don’t need to restore the link.




For instructions on updating the link configuration, see “Modifying a replication link” on page 3-12.

5. If activity on the link is currently suspended, resume it, as described in “Suspending and resuming activity on a replication link” on page 5-4.

6. In the left side of the panel, click on Advanced.

7. In the Advanced panel, click on the Restore Link button.

A pending link request appears on the other system. You respond to this the same way you do to an initial request to create a link. For information on responding to a link request, see “Accepting or rejecting a link request” on page 3-11.

When the other system accepts the link:

• If that system is the primary system, the link is recreated in a suspended state, and you need to begin data recovery.

• If that system is the replica, replication restarts automatically when the link is accepted.

Recovering the data after a primary system failure

To restore data and other applicable information to a primary system from a replica:

1. In the top-level menu in the HCP System Management Console for the replica, mouse over Services to display a secondary menu.


3. On the Replication page, click on the name of the link on which you want to recover data.

4. In the left side of the panel that opens, click on Advanced.

Note: If you don’t see the Restore Link button, wait a few minutes and then redisplay the Replication page. Before the System Management Console can display this button, HCP needs to recognize that the other system is available, has the necessary SSL certificates installed, and isn’t aware of the existing link.




5. In the Advanced panel, click on the Begin Recovery button.

The replication service starts copying the applicable configuration and objects from the replica to the primary system. As with replication from the primary system to the replica, the service starts with the objects with the oldest metadata changes either across all namespaces or within each namespace, depending on the link configuration.

6. Monitor the recovery process by periodically reviewing the backlog time on the Replication page in the System Management Console for either the primary system or the replica.

7. When data recovery is almost synchronized with current tenant and namespace activities on the replica, return to the Advanced panel in the System Management Console for the replica. Synchronization is nearing completion when the maximum backlog time for each tenant is close to zero hours.

8. In the Advanced panel, click on the Complete Recovery button.

The applicable tenants and directories on the replica immediately become read-only. The tenants and directories on both systems then remain read-only until the replication service finishes the data recovery. The amount of time this takes depends on how much data is left to recover.

When recovery is complete, the tenants and directories on the primary system become read-write, those on the replica remain read-only, and the replication service on the primary system starts copying objects to the replica again.

9. Wait for this message to appear in the system log:

Replication data recovery completed

Note: After uploading new trusted replication server certificates, you may need to wait more than ten minutes for the Begin Recovery button to become active.

Note: As long as clients continue writing to the replica, synchronization won’t reach 100 percent. Synchronization doesn’t need to be complete for you to finish the recovery process.

Tip: You can schedule completion of the data recovery process for a time when client usage of the repository is low.



Failover and recovery considerations

10.Redirect clients to write to the replicated namespaces and directories on the primary system.


These basic rules apply to replication, failover, and recovery on replication links, regardless of the replication topology:

• Failover occurs from the primary system to the replica for the same link. It cannot occur from the primary system for one link to the replica for a different link.

• Recovery occurs from the replica to the primary system for the same link. It cannot occur from the replica for one link to the primary system for another link.

• During replication or recovery or while a link is failed over, the HCP tenants and namespaces and default-namespace directories included in the link are read-write on at most one HCP system at a time.

• When a link fails over to the replica, only the HCP tenants and namespaces and default-namespace directories that were read-write on the primary system become read-write on the replica.

Additional considerations apply to failover and recovery in many-to-one and chained replication topologies.

Failover and recovery in a many-to-one topology

In a many-to-one replication topology, multiple HCP systems replicate to a single other HCP system. For an explanation of this topology, see “Many-to-one replication” on page 1-9.

To recover from a primary system failure in a many-to-one replication topology, you follow the normal pattern:

1. Fail over the link between the failed primary system and the replica.

2. When the primary system becomes available again, restore the link from the replica.

3. Begin and complete data recovery from the replica to the primary system.




If more than one primary system fails in a many-to-one topology, you need to fail the link from each failed system over to the replica. Multiple inbound links on the replica can be in the failed-over state at the same time.

When the failed systems become available again, you can restore the links at any time. However, you can perform the data recovery step on only one link at a time. (You don’t need to wait for all the failed systems to become available before recovering data to the first one.)

For example, suppose systems A, B, and C all replicate to system D. If both A and B fail, you can return to normal replication with these steps after the failed systems become available again:

1. On system D, restore the link from A to D.

2. On system D, begin and complete data recovery from D to A.

3. On system D, restore the link from B to D.

4. After the recovery of data from D to A is complete and replication from A to D has restarted, on system D, begin and complete data recovery from D to B.

Failover and recovery in a chained topology

In a chained replication topology, one HCP system replicates to a second HCP system, which replicates to a third HCP system. For an explanation of this topology, see “Chained replication” on page 1-12.

The way you manage failover and recovery in a chained replication topology depends on which system or systems have failed. The links from the first system in the chain to the second system and from the second system to the third system function independently of each other, but order matters when you fail them over or restore and perform data recovery on them.

The following sections outline the steps you need to take to return a replication chain to normal operation after the failure of any one or two of the systems in the chain. These sections assume a replication topology in which, when all three systems are healthy:

• System A replicates to system B on link AB.

• Link AB includes HCP tenant T1 and default-namespace directory D1, both of which were originally created on system A.




• System B replicates to system C on link BC.

• Link BC includes link AB and HCP tenant T2, which was originally created on system B.

• T1 and D1 are read-write on system A and read-only on systems B and C.

• T2 is read-write on system B and read-only on system C.

The figure below shows this topology.

Scenario: System A fails

To return to normal operation after system A fails:

1. On system B, fail link AB over to B.

T1 and D1 become read-write on B.

2. When system A becomes available again, on system B, restore link AB.

3. On system A, accept the restored link.

4. On system B, begin and complete data recovery on link AB.

When data recovery is complete, T1 and D1 become read-write on A and read-only on B, and replication resumes on link AB.

A B C

AB

BCAB

Read-write Read-only

T1

D1

T2

T1

D1

T1

D1

T2




Scenario: System B fails

To return to normal operation after system B fails:

1. On system C, fail link BC over to C.

T1 and D1 remain read-write on A and read-only on C. T2 becomes read-write on C.

2. When system B becomes available again, do either of these:

– Restore link AB first:

1. On system A, restore link AB.

2. On system B, accept the restored link.

T1 and D1 are read-write on A and read-only on B and C, and replication restarts on link AB.

3. On system C, restore link BC.


5. On system C, begin and complete data recovery on link BC.

When data recovery is complete, T1 and D1 are read-only on B and C. T2 becomes read-write on B and read-only on C, and replication resumes on link BC.

– Restore link BC first:

1. On system C, restore link BC.




System B automatically recreates link AB without the primary system IP addresses or hostname from the inbound link AB in link BC.




4. Follow the steps for option one or option two in the table below.

T1 and D1 are read-write on A and read-only on B and C, and replication restarts on link AB.

Scenario: System C fails

To return to normal operation after system C fails:

1. When system C becomes available again, on system B, restore link BC.

2. On system C, accept the restored link.

T1 and D1 remain read-only B and C. T2 remains read-write on B and read-only on C, and replication restarts on link BC.

Scenario: Systems A and B fail

To return to normal operation after systems A and B fail:

1. On system C, fail link BC over to C.

T1 and D1 remain read-only on C. T2 becomes read-write on C.

2. When system B becomes available again, on system C, restore link BC.




Option one Option two

1. On system B, update the configuration of link AB to include the IP addresses or hostname for system A.


2. On system B, restore link AB. 2. On system B, accept the restored link.


N/A




System B automatically recreates link AB without the primary system IP addresses or hostname from the inbound link AB in link BC.

5. When system A becomes available again, do either of these:

– If link AB still exists on A:




When data recovery is complete, T1 and D1 become read-write on A and remain read-only on B, and replication resumes on link AB.

– If link AB no longer exists on A, follow the steps for option one or option two in the table below.


1. On system B, update the configuration of the automatically recreated link AB to include the IP addresses or hostname for system A.

1. On system B, suspend and then delete link AB.

T1 and D1 become directly included in link BC, which still includes T2, and become read-write on B.

2. On system B, restore link AB. 2. Optionally, create a new replication chain B C A:

a. Reinstall HCP on A.

b. On system C, create outbound link CA, including link BC as an inbound link.

c. On system A, accept the new link.

T1, D1, and T2 are read-write on B and read-only on C and A, and replication starts on link CA.




Scenario: Systems A and C fail

To return to normal operation after systems A and C fail:

1. On system B, fail link AB over to B.

T1 and D1 become read-write on B. T2 remains read-write on B.

2. When system C becomes available again, on system B, restore link BC.

3. On system C, accept the restored link.

T1, D1, and T2 are read-write on B and read-only on C, and replication restarts on link BC.

4. When system A becomes available again, on system B, restore link AB.



When data recovery is complete, T1 and D1 become read-write on A and read-only on B, and replication resumes on link AB.

Scenario: Systems B and C fail

To return to normal operation after systems B and C fail:

1. When system B becomes available again, on system A, restore link AB.


T1 and D1 are read-write on A and read-only on B, and replication restarts.


N/A


When data recovery is complete, T1 and D1 are read-write on A and read-only on B and C, and replication restarts on link AB.

(Continued)





3. When system C becomes available again, if T1 and D1 still exist on system C, delete them. (This requires that all objects in the namespaces owned by T1 and in D1 be deleted first.)

4. On system B, create outbound link BC, including link AB as an inbound link. If T2 still exists on B, also include T2 in the link.

5. On system C, accept the new link.

T1 and D1 are read-only on B and C. If T2 is included in link BC, T2 is read-write on B and read-only on C. Replication restarts on link BC.

Failover and recovery in a many-to-one topology with disaster recovery support

When HCP systems fail in a many-to-one topology with disaster recovery support, you need to combine the recovery patterns for the many-to-one and chained topologies. For an explanation of this topology, see “Many-to-one replication with disaster recovery support” on page 1-14.

For example, assume a replication topology in which, when all five systems are healthy:

• System A replicates to system D on link AD. Link AD includes HCP tenant T1, which was originally created on system A.

• System B replicates to system D on link BD. Link BD includes HCP tenant T2, which was originally created on system B.

• System C replicates to system D on link CD. Link CD includes HCP tenant T3, which was originally created on system C.

• System D replicates to system E on link DE. Link DE includes links AD, BD, and CD and tenant T4, which was originally created on system D.

• T1, T2, and T3 are read-write on systems A, B, and C, respectively, and read-only on systems D and E.

• T4 is read-write on system D and read-only on system E.

Note: If T2 still exists on systems B and C, delete it from C. If T2 still exists on system C and not on system B, you can recover it to B by creating a link from C to B. However, you cannot then replicate T2 from B to C unless you first delete it from C.




The figure below shows this topology.

To return to normal operation after systems A, B, and D fail:

1. On system E, fail link DE over to E.

T1, T2, and T3 are read-only on E. T4 becomes read-write on E. T3 remains read-write on C.

2. When system D becomes available again, on system E, restore link DE.

3. On system D, accept the restored link.

4. On system E, begin and complete data recovery on link DE.

When data recovery is complete, T1, T2, and T3 are read-only on D and E. T4 becomes read-write on D and read-only on E, and replication resumes on link DE.

5. On system C, restore link CD.

6. On system D, accept the restored link.

T3 is read-write on C and read-only on D, and replication restarts on link CD.

7. When system A becomes available again, on system D, update the configuration of the automatically recreated link AD.

8. On system D, restore link AD.

A

D E

DEBD

Read-write Read-only

T1

T1

T2

T3

T4

T1

T2

T3

T4

B

T2

C

T3

AD

BD

CD

AD

CD





10.On system D, begin and complete data recovery on link AD.

When data recovery is complete, T1 becomes read-only on A and remains read-write on D. Replication resumes on link AD.

11.When system B becomes available again and data recovery on link AD is complete, on system D, update the configuration of the automatically recreated link BD.

12.On system D, restore link BD.

13.On system B, accept the restored link.

14.On system D, begin and complete data recovery on link BD.

When data recovery is complete, T2 becomes read-only on A and remains read-write on D. Replication resumes on link BD.



A
Reenabling user accounts
disabled on the replica

For HCP tenants, tenant-level user accounts are replicated and are available for use on the replica. If an account becomes disabled on the replica due to consecutive failed login attempts, it cannot be reenabled on the replica because the tenant is read-only.

This appendix explains how to reenable a tenant-level user account that has been disabled on a replica.

Reenabling user accounts disabled on the replica A–1


To reenable a tenant-level user account that’s disabled on the replica, in the Tenant Management Console for the applicable tenant on the primary system:

1. In the top-level menu, mouse over Security to display a secondary menu.

2. In the secondary menu, click on Users.

3. In the list of user accounts on the Users page, click on the edit control ( ) for the account you want to modify.

4. Click on the Update User Account button.

5. Wait for the updated account to be replicated.

When the account is replicated, it becomes enabled on the replica.

A–2 Reenabling user accounts disabled on the replica


B
Setting advanced replication
options

Two advanced options are available for replication:

• Setting the number of threads for the custom performance level

• Setting the interval at which HCP sends SNMP notifications about broken links

This appendix describes these options.

Roles: For you to perform the activities listed above, your user account must include the service role.

Setting advanced replication options B–1


Changing the custom performance level

Changing the custom performance level

When you create a replication link, you set a performance level for it. The performance level controls the number of replication threads that can run on each node. For the low, medium, and high performance levels, the numbers of threads are one, five, and ten, respectively. For the custom performance level, the default number of threads is 25, but you can change this number.

The performance level is a maximum. On any given node, fewer threads can be running than the number indicated by the performance level.

The performance level applies to the system that’s currently transmitting data. That is, during replication, it applies to the primary system. During data recovery, it applies to the replica.

If a system is transmitting data on more than one link (that is, if it’s replicating on one link and recovering on the other), the maximum number of threads per node on that system is the total allowed by both links.

To change the number of threads for the custom performance level:

1. In the top-level menu in the HCP System Management Console, mouse over Configuration to display a secondary menu.

2. In the secondary menu, click on Miscellaneous Settings.

3. In the Replication Threads for Custom Performance Level field, type the new number of threads for the custom performance level. Valid values are integers in the range one through 25. The default is 25.

4. Click on the Update Settings button.

Note: If a replication link is already set to the custom performance level when you change the number of threads, you need to suspend and resume the link for the change to take effect. For information on suspending and resuming a replication link, see “Suspending and resuming activity on a replication link” on page 5-4.

B–2 Setting advanced replication options


Changing the broken-link notification interval


HCP supports the use of SNMP for reporting certain events and conditions. When this capability is enabled, HCP reports broken links on a periodic basis. That is, while a link is broken, HCP sends an SNMP notification about it every n minutes.

You can change the SNMP reporting interval for broken links. To do this:

1. In the top-level menu in the HCP System Management Console, mouse over Configuration to display a secondary menu.

2. In the secondary menu, click on Miscellaneous Settings.

3. In the SNMP Broken-link Reporting Interval field, type the number of minutes you want HCP to wait between SNMP notifications about broken links. Valid values are integers greater than or equal to one. The default is 60.

4. Click on the Update Settings button.

Setting advanced replication options B–3



B–4 Setting advanced replication options


Glossary

A

alert

A graphic that indicates the status of some particular element of an HCP system in the System Management Console.

appendable object

An object to which data can be added after it has been successfully stored. Appending data to an object does not modify the original fixed-content data, nor does it create a new version of the object. Once the new data is added to the object, that data also cannot be modified.

Appendable objects are supported only with the CIFS and NFS protocols.

C

capacity

The total amount of storage space in HCP, excluding the space required for system overhead and the operating system. This is the amount of space available for all data in the repository, including the fixed-content data, metadata, and any redundant data required to satisfy namespace DPLs.

CIFS

Common Internet File System. One of the protocols HCP uses to provide access to the contents of the default namespace. CIFS lets Windows clients access files on a remote computer as if they were part of the local file system.

Glossary–1


compliance mode

compliance mode

The retention mode in which objects under retention cannot be deleted through any mechanism. This is the more restrictive retention mode.

content verification service

The HCP service that ensures the integrity of each object by checking that the object data still matches its cryptographic hash value.

cryptographic hash value

A system-generated metadata value calculated by a cryptographic hash algorithm from object data. This value is used to verify that the content of an object has not changed.

D

data access account

A set of credentials that give a user or application access to one or more HCP namespaces. For each namespace, the account specifies which operations the user or application can perform.

data protection level (DPL)

The number of copies of an object HCP must maintain in the repository. Each namespace has its own DPL setting that applies to all objects in that namespace.

default namespace

A namespace that supports the HTTP, WebDAV, CIFS, NFS, SMTP, and NDMP protocols and does not require user authentication for data access. An HCP system can have at most one default namespace.

default tenant

The tenant that manages the default namespace.

DNS

See domain name system (DNS).

domain name system (DNS)

A network service that resolves domain names into IP addresses for client access.

DPL

See data protection level (DPL).

Glossary–2


HCP node

dynamic DPL

A namespace data protection level that, at any given time, matches the system-level DPL setting.

enterprise mode

The retention mode in which these operations are allowed:

• Privileged delete

• Changing the retention class of an object to one with a shorter duration

• Reducing retention class duration

• Deleting retention classes

F

fixed-content data

A digital asset ingested into HCP and preserved in its original form as the core part of an object. Once stored, fixed-content data cannot be modified.

H

hash value

See cryptographic hash value.

HCAP

See Hitachi Content Archive Platform (HCAP).

HCP

See Hitachi Content Platform (HCP).

HCP namespace

A namespace that requires user authentication for data access. An HCP system can have multiple HCP namespaces.

HCP node

See node.

Glossary–3


HCP service

HCP service

See service.

HCP tenant

A tenant created to manage HCP namespaces.

Hitachi Content Archive Platform (HCAP)

The predecessor to Hitachi Content Platform.

Hitachi Content Platform (HCP)

A distributed object-based storage system designed to support large, growing repositories of fixed-content data. HCP provides a single scalable environment that can be used for archiving, business continuity, content depots, disaster recovery, e-discovery, and other services. With its support for multitenancy, HCP securely segregates data among various constituents in a shared infrastructure. Clients can use a variety of industry-standard protocols and various HCP-specific interfaces to access and manipulate objects in an HCP repository.

I

inbound link

A replication link from the perspective of the HCP system that’s specified as the target in the link configuration.

L

link

See replication link.

M

metadata

System-generated and user-supplied information about an object. Metadata is stored as an integral part of the object it describes, thereby making the object self-describing.

Glossary–4


outbound link

N

namespace

A logical partition of the objects stored in an HCP system. A namespace consists of a grouping of objects such that the objects in one namespace are not visible in any other namespace. Namespaces are configured independently of each other and, therefore, can have different properties.

NAT

See network address translation (NAT).

network address translation (NAT)

The translation of a set of IP addresses used within a local area network to a different set of IP addresses used within another network.

NFS

Network File System. One of the protocols HCP uses to provide access to the contents of the default namespace. NFS lets clients access files on a remote computer as if they were part of the local file system.

node

A server running HCP software and networked with other such servers to form an HCP system.

O

object

An exact digital representation of data as it existed before it was ingested into HCP, together with the system and custom metadata that describes that data. An object is handled as a single unit by all transactions and services, including shredding, indexing, versioning, and replication.

outbound link

A replication link from the perspective of the HCP system on which the link is created.

Glossary–5


privileged delete

P

privileged delete

A delete operation that works on objects regardless of their retention settings, except for objects on hold. This operation is available only to users and applications with explicit permission to perform it.

protection service

The HCP service that ensures the stability of the repository by maintaining a set level of data redundancy, as specified by the data protection level (DPL) for each namespace.

R

replica

The target system to which the replication service copies objects and other information from the primary system during replication.

replication

The process of keeping selected tenants and namespaces in two HCP systems in sync with each other. Basically, this entails copying object creations, deletions, and metadata changes from one system to the other. HCP also replicates tenant and namespace configuration, data access accounts, retention classes, all compliance log messages, and all HCP tenant log messages.

replication link

A configurable association that specifies the HCP systems from and to which the replication service should copy objects and other data.

replication service

The HCP service that performs replication.

repository

The aggregate of the namespaces defined for an HCP system.

retention class

A named retention setting. The value of a retention class can be a duration, Deletion Allowed, Deletion Prohibited, or Initial Unspecified.

Glossary–6


SSL server certificate

retention mode

A namespace property that affects which operations are allowed on objects under retention. A namespace can be in either of two retention modes: compliance or enterprise.

retention period

The period of time during which an object cannot be deleted (except by means of a privileged delete).

retention setting

The property that determines the retention period for an object.

role

A named collection of permissions that can be associated with an HCP user account, where each permission allows the user to perform some specific interaction or set of interactions with the HCP System Management Console. Roles generally correspond to job functions.

S

service

A background process that performs a specific function that contributes to the continuous tuning of the HCP system. In particular, services are responsible for optimizing the use of system resources and maintaining the integrity and availability of the data stored in the HCP repository.

SMTP

Simple Mail Transfer Protocol. The protocol HCP uses to receive and store email data directly from email servers.

SSL

Secure Sockets Layer. A key-based Internet protocol for transmitting documents through an encrypted link.

SSL server certificate

A file containing cryptographic keys and signatures. When used with the SSL protocol, an SSL server certificate helps verify that the web site holding the certificate is authentic. An SSL server certificate also helps protect data sent to or from that site.

Glossary–7


storage node

storage node

An HCP node that runs the complete HCP software (except the HCP search facility software) and stores the objects added to HCP.

System Management Console

The system-specific web application that lets you monitor and manage HCP.

T

tenant

An administrative entity created for the purpose of owning and managing namespaces and data access accounts. Tenants typically correspond to customers, business units, or individuals.

Tenant Management Console

The tenant-specific web application that lets you monitor and manage tenants and namespaces.

U

user account

A set of credentials that gives a user access to one or more of the System Management Console, Tenant Management Console, and HCP management API.

user role

See role.

Glossary–8


Index

Symbols.lost+found directory 1-20

Aaccepting link requests 3-11–3-12adding

directories to replication links 3-13–3-14HCP namespaces to replication links 3-13HCP tenants to replication links 3-13

advanced configuration, replication links 3-9–3-10

Advanced panel (Replication page)beginning data recovery 6-7completing data recovery 6-7failing over 6-5restoring replication links 6-6

alertsdescriptions 4-10–4-11pending replication links 3-11

appendable objects 1-6automatically paused replication of tenants 4-3,

5-5

Bbacklog time

about 4-4maximum for tenants 4-8minimum for tenants 4-8tenants 4-6–4-7

beginning data recovery 6-7bidirectional replication

about 1-9failover 1-18replicating 1-16

broken replication link 4-4broken-link notification interval B-3

Ccanceling replication link requests 3-10certificates

See SSL server certificatesCertificates panel (Replication page)

deleting trusted replication server certificates 2-3

downloading SSL server certificates 2-2downloading trusted replication server

certificates 2-3uploading trusted replication server

certificates 2-2chained replication

about 1-12–1-14adding directories 3-13adding HCP tenants 3-13considerations for failover and recovery 6-9–

6-15changing

custom performance level B-2SSL server certificates 2-1

checkpoints, link activity 5-4completing data recovery 6-7–6-8compression 3-6configuration conflicts during recovery 1-20Configuration panel (Replication page)

accepting link requests 3-11–3-12canceling link requests 3-10modifying replication links 3-14–3-15rejecting link requests 3-11–3-12

conflictsHCP tenant names 1-5resolving for configuration differences during

recovery 1-20resolving for objects during recovery 1-19–

1-20considerations

creating replication links 3-3–3-4failover and recovery rules 6-8

Index–1


considerations, failover and recovery with chained topology

failover and recovery with chained topology 6-9–6-15

failover and recovery with many-to-one topology 6-8–6-9

failover and recovery with many-to-one topology with disaster recovery support 6-15–6-17

modifying replication links 3-12–3-14content verification service 1-2, 6-5controlling Tenant Management Console

replication display 4-9Create Inbound Link panel (Replication

page) 3-4–3-7Create Link panel (Replication page) 3-4–3-7creating replication links

considerations 3-3–3-4procedure 3-4–3-7

cryptographic hash algorithm, default namespace 1-5

custom performance level, changing B-2

Ddata access accounts

replicating 1-5resolving recovery conflicts 1-20

data compression 3-6data encryption 3-6data recovery

See also replicationabout 1-18–1-19beginning 6-7chained topology 6-9–6-15completing 6-7–6-8many-to-one topology 6-8–6-9many-to-one topology with disaster recovery

support 6-15–6-17performance level 3-5primary system failure 6-6–6-8resolving conflicts 1-19–1-20rules 6-8

data transmission rate 4-5default namespace

See also default tenantappendable objects 1-6cryptographic hash algorithm 1-5retention mode 1-5writes/deletes disabled 1-6

default tenantSee also default namespace; tenantsselecting directories for replication 3-8what is replicated 1-5–1-6

deletingpending links 3-10replication client certificates 2-3replication links 3-15–3-16

directories.lost+found 1-20adding to replication links 3-13–3-14email 1-17removing from replication links 3-13selecting for replication 3-8status during failover 1-17–1-18status during recovery 1-18–1-19status during replication 1-16

disaster recovery with many-to-one replication 1-14–1-15

DNS names in replication links 3-5downloading

replication client certificates 2-3SSL server certificates 2-2

DPL, unsupported on replica 3-3, 3-14

Eemail directories 1-17encryption 3-6

FFailed Over (link mode) 4-2failover

bidirectional replication 1-18chained topology 6-9–6-15failing over to replica 6-4–6-5many-to-one topology 6-8–6-9many-to-one topology with disaster recovery

support 6-15–6-17one-way replication 1-17rules 6-8

failureprimary system, workflow 6-2–6-3replica, workflow 6-3–6-4

Final Recovery (link mode) 4-2finishing data recovery 6-7–6-8

Ggraphs

Operations per Second 4-5–4-6Transfer Rate 4-5

Hhardware, replicated systems 1-7HCP namespaces

adding to replication links 3-13

Index–2


performance level

configuration, replicating 1-5DPL unsupported on replica 3-3DPL, changing 3-14removing from replication links 3-13resolving recovery conflicts 1-20selecting for replication 1-4, 3-7–3-8total on replica 1-4

HCP systemsrecovering from failure 6-5replication between different releases 3-4

HCP tenantsSee also tenantsadding to replication links 3-13configuration, replicating 1-5name conflicts on primary system and

replica 1-5removing from replication links 3-13resolving recovery conflicts 1-20selecting for replication 3-7–3-8status during failover 1-17–1-18status during recovery 1-18–1-19status during replication 1-16Tenant Management Console replication

display 4-9what is replicated 1-4–1-5

high error rate 4-3

Iinbound links

See also replication linksabout 1-3creating 3-2–3-7including in replication links 3-3pending 3-11selecting for replication 3-9

Llog messages, replicating 1-4, 1-6

MManagement panel (Replication page)

deleting replication links 3-16resuming link activity 5-5suspending link activity 5-5

many-to-one replicationabout 1-9–1-11considerations for failover and recovery 6-8–

6-9with disaster recovery 1-14–1-15

maximum backlog time 4-8minimum backlog time 4-8

Miscellaneous Settings pagechanging the broken-link notification

reporting interval B-3changing the custom performance level B-2

modes, replication links 4-2modifying replication links

considerations 3-12–3-14procedure 3-14–3-15

monitoring link activityalerts 4-10–4-11link level 4-2–4-6tenant level 4-6–4-8in Tenant Management Console 4-9

Nname conflicts

HCP tenants 1-5retention classes 1-6–1-7

names, replication link 3-4namespaces, read from replica 1-2

See also default namespace; HCP namespaces

NAT with replication 3-10network address translation with replication 3-10networking for replication 1-8

Oobjects

conflicts during recovery 1-19–1-20repairing 1-2replicating versions 1-4

one-way replicationabout 1-16failover 1-17

operation rate 4-5–4-6outbound links

See also replication linksabout 1-3creating 3-2–3-7

Overview page, replication alerts 4-10–4-11Overview panel (Replication page) 4-4–4-6

Ppausing tenant replication or recovery 5-5Pending (link mode) 4-2pending links

about 3-7, 3-11deleting 3-10

performance levelabout 3-5custom, changing B-2

Index–3


performance level, Off

Off 5-2scheduling 5-2–5-4

ports, replication 3-9–3-10primary system

See also replication; replication linksabout 1-2creating outbound links 3-2–3-7data recovery after failure 6-6–6-8failure workflow 6-2–6-3modifying outbound links 3-12–3-15

priority, replicationabout 1-3setting 3-6

protection service 1-2, 6-5

Rread from replica 1-2Receiving (link mode) 4-2Recovering (link mode) 4-2recovering from system failure 6-5

See also replicationrecovery

See data recoveryreenabling user accounts disabled on

replica A-1–A-2rejecting link requests 3-11–3-12remote link configuration 1-3, 1-11

See also inbound linksremoving

directories from replication links 3-13HCP namespaces from replication links 3-13HCP tenants from replication links 3-13

repairing objects 1-2replica

See also replication; replication linksabout 1-2creating inbound links 3-2–3-7failing over to 6-4–6-5failure workflow 6-3–6-4modifying inbound links 3-12–3-15reenabling user accounts for A-1–A-2space available 4-6

Replicating (link mode) 4-2replication

See also bidirectional replication; chained replication; data recovery; many-to-one replication; replication links

about 1-2–1-3automatically paused for tenants 4-3backlog time for link 4-4backlog time for tenants 4-6–4-7bidirectional 1-9chained 1-12–1-14

checkpoints 5-4data transmission rate 4-5failing over to replica 6-4–6-5failover 1-17–1-18failover considerations 6-8–6-17hardware 1-7many-to-one 1-9–1-11many-to-one with disaster recovery 1-14–

1-15modifying link during online upgrade 3-12monitoring 4-2–4-11network address translation 3-10network connections 1-8operation rate 4-5–4-6performance level 3-5priority 1-3, 3-6recovering data after system failure 6-6–6-8recovery 1-18–1-20recovery considerations 6-8–6-17replicating 1-15–1-16resuming 5-4–5-5scheduling 5-2–5-4security 1-3space available on replica 4-6suspending 5-4–5-5between systems with different HCP

releases 3-4what is replicated 1-3–1-7

replication client certificatesSee also SSL server certificatesabout 1-3, 2-1deleting 2-3downloading 2-3uploading 2-2–2-3

replication linksSee also replicationabout 1-3, 3-1accepting 3-11–3-12adding directories 3-13–3-14adding HCP namespaces 3-13adding HCP tenants 3-13advanced configuration 3-9–3-10broken 4-4canceling requests 3-10considerations for creating 3-3–3-4considerations for modifying 3-12–3-14creating 3-4–3-7deleting 3-15–3-16high error rate 4-3modes 4-2modifying 3-14–3-15names 3-4pending 3-7, 3-11

Index–4


workflows

properties 3-2rejecting 3-11–3-12removing directories 3-13removing HCP namespaces 3-13removing HCP tenants 3-13restoring 6-5–6-6selecting items for replication 3-7–3-9status 4-3–4-4

Replication pageAdvanced panel 6-5, 6-6, 6-7Configuration panel 3-10, 3-11–3-12, 3-14–

3-15Create Inbound Link panel 3-4–3-7Create Link panel 3-4–3-7link information 4-2–4-4Management panel 5-5Overview panel 4-4–4-6Schedule panel 5-2–5-4Settings panel 4-9Tenants panel 4-7–4-8, 5-5

replication serviceSee also replicationabout 1-2starting 1-3

reporting interval for broken links B-3resolving recovery conflicts 1-19–1-20restoring replication links 6-5–6-6resuming

replication link activity 5-4–5-5tenant replication or recovery 5-5

retention classesreplicating in default namespace 1-6–1-7replicating in HCP namespaces 1-4resolving recovery conflicts 1-20

retention mode, default namespace 1-5

SSchedule panel (Replication page) 5-2–5-4scheduling

about 5-2–5-3procedure 5-3–5-4

selectingdirectories for replication 3-8HCP namespaces for replication 1-4, 3-7–3-8HCP tenants for replication 3-7–3-8inbound links for replication 3-9

server certificatesSee SSL server certificates

servicescontent verification 1-2, 6-5protection 1-2, 6-5replication 1-2, 1-3

Settings panel (Replication page) 4-9

SMTP protocol with replication 1-17SNMP broken-link notification interval B-3space available on replica chart 4-6SSL server certificates

See also replication client certificatesabout 2-1changing 2-1downloading 2-2required 1-3uploading 2-2–2-3

statusreplication links 4-3–4-4tenant replication 4-8

suspending replication link activity 5-4–5-5

TTenant Management Console, replication

display 4-9tenants

See also default tenant; HCP tenantsautomatically paused replication 4-3maximum backlog time 4-8minimum backlog time 4-8pausing replication or recovery 5-5replication backlog time 4-6–4-7replication status 4-8resuming replication or recovery 5-5

Tenants panel (Replication page)monitoring tenant replication or

recovery 4-7–4-8pausing tenant replication or recovery 5-5resuming tenant replication or recovery 5-5

threads for replication B-2topologies, replication 1-8–1-15transfer rate 4-5

Uuploading SSL server certificates 2-2–2-3user accounts

reenabling for replica A-1–A-2replicating 1-5resolving recovery conflicts 1-20

Vversions of objects, replicating 1-4

Wworkflows

primary system failure 6-2–6-3replica failure 6-3–6-4

Index–5


Index–6


MK-98ARC015-05

Hitachi Data Systems

Corporate Headquarters750 Central ExpresswaySanta Clara, California 95050-2627U.S.A.Phone: 1 408 970 [email protected]

Asia Pacific and Americas750 Central ExpresswaySanta Clara, California 95050-2627U.S.A.Phone: 1 408 970 [email protected]

Europe HeadquartersSefton ParkStoke PogesBuckinghamshire SL2 4HDUnited KingdomPhone: + 44 (0)1753 [email protected]

Replicating Tenants and Name Spaces

Documents

Transcript of Replicating Tenants and Name Spaces