Huawei HyperImage Technical White Paper.pdf

Doc. code

HyperImage

Technical White Paper

Issue 01

Date 2012-04-20

HUAWEI TECHNOLOGIES CO., LTD.

HyperImage Technical White Paper CONFIDENTIAL

2013-07-29 Copyright Huawei Technologies Co., Ltd. 2012. All rights reserved Page2, Total15

Copyright Huawei Technologies Co., Ltd. 2012. All rights reserved.

No part of this document may be reproduced or transmitted in any form or by any means

without prior written consent of Huawei Technologies Co., Ltd.

Trademarks and Permissions

and other Huawei trademarks are trademarks of Huawei Technologies Co., Ltd.

All other trademarks and trade names mentioned in this document are the property of their

respective holders.

Notice

The purchased products, services and features are stipulated by the contract made between

Huawei and the customer. All or part of the products, services and features described in this

document may not be within the purchase scope or the usage scope. Unless otherwise

specified in the contract, all statements, information, and recommendations in this document

are provided "AS IS" without warranties, guarantees or representations of any kind, either

express or implied.

The information in this document is subject to change without notice. Every effort has been

made in the preparation of this document to ensure accuracy of the contents, but all

statements, information, and recommendations in this document do not constitute the

warranty of any kind, express or implied.

Huawei Technologies Co., Ltd.

Address: Huawei Industrial Base

Bantian, Longgang

Shenzhen 518129

People's Republic of China

Website: http://www.huawei.com

Email: [email protected]



Chapter 1 Overview

As the requirements of centralized storage applications become stricter, users

need online protection for data to make the backup window shorter. The snapshot

technology is effective in preventing data loss on online storage devices. More

and more storage devices use the snapshot technology. What is a snapshot?

How does a snapshot work? What is the function of a snapshot in a storage array?

Defined by the Storage Networking Industry Association (SNIA), a snapshot is a

fully usable copy of a defined collection of data that contains an image of the data

as it appeared at the point in time at which the copy was initiated. A snapshot may

be either a duplicate or a replicate of the data it represents.

Many technologies can be used to implement the snapshot function. The most

widely used two are the virtual snapshot technology and the split mirror

technology. Huawei OceanStor storage arrays support both the virtual snapshot

technology and the split mirror technology. So, users have multiple ways to

protect data online. This document only describes the virtual snapshot technology.

For information on the split mirror technology, see the OceanStor Storage Array

HyperClone Technical White Paper.

Chapter 2 Virtual Snapshot of the OceanStor

Storage Array: HyperImage

2.1 Basic Work Principle of the HyperImage

The virtual snapshot of the OceanStor storage array is called HyperImage. The

HyperImage can generate a virtual consistent image of source LUN at a certain

point in time, thus quickly obtaining a consistent duplicate of data on the source

LUN without interruption to services. The duplicate is available immediately after

being generated. Writing data to or reading data from the duplicate have no

impact on the source data. Therefore, the snapshot technology can implement

online backup, data analysis, application tests, and so on. The HyperImage is

implemented through the combination of the mapping table and the copy-on-write

technology. The work principle is described as follows:

1) Writing data to a storage array on which no snapshot is created is the same

as writing data to a storage array without the snapshot function, that is,



modifications to data are directly written to the disk block where the original

data is. The original data is overwritten and not saved.

2) Before being activated in a storage array, the HyperImage must be

configured properly. First, a snapshot resource pool must be created, and

then a source LUN must be selected. A snapshot resource pool consists of

several resource LUNs. When the capacity of the resource pool is

inadequate, new resource LUNs can be added to the resource pool

dynamically.

3) When the HyperImage is activated (this time is called the snapshot point in

time), the system creates a mapping table. The mapping table records the

mappings between pieces of original data on the source LUN and their

physical addresses, and resource LUNs store the original data in various

disk blocks of the first update after the snapshot point in time. Original data

of any later update is overwritten rather than saved in the resource LUNs.

Before new data is written to the source LUN, all address pointers in the

mapping table point to the source LUN, and resource LUNs are empty, as

shown in Figure 2-1.

Figure 2-1 Activating the HyperImage

4) When new data is to be written to the source LUN, the original data in the

place where the new data is to be written onto is moved to a resource LUN,

and the mapping table changes the corresponding mapping at the same time

and records the new location of the original data. Then, the new data is

written to the source LUN. The copy-on-write technology is applied in this

process. As shown in Figure 2-2, when data s is to be written to block 3 of

the source LUN, data d on the source LUN is moved to a resource LUN first.

At the same time, the mapping on the snapshot LUN is changed. The

address pointer points to the new position of data d on the resource LUN.



Then data s is written onto the source LUN. Data on the source LUN is

changed. Data d that is previously stored on the source LUN at the snapshot

point in time is now stored on the resource LUN.

Figure 2-2 Writing data onto the source LUN

5) If new data is written to the same block of the source LUN later, the system

checks the mapping table to find that the data in the block at the snapshot

point in time has been moved to a resource LUN, and then, the new data is

directly written to the same disk block. The original data is overwritten and

not moved onto a resource LUN, that is, only one copy-on-write is

implemented in the same position. Therefore, the data consistency at the

snapshot point in time is effectively protected. See Figure 2-3.

Figure 2-3 Overwriting data in the same position since the second update



6) If the data at the snapshot point in time is required to be recovered, the

recovery can be immediately implemented by the rollback function of the

snapshot data. Through the rollback, the data in the storage array can be

recovered to that at the snapshot point in time. Therefore, data is protected

from being lost even if the source LUN is damaged by misoperations or

viruses after the snapshot point in time. As shown in Figure 2-3, if data is

required to be rolled back to that at the snapshot point in time, data d

replaces data m in the block 3 of the source LUN. It must be noted that the

snapshot rollback is irreversible. Through the rollback, data can be

recovered to a specified time, but data processed between the specified time

and the error time is lost.

7) When the HyperImage stops, data in the mapping table and resource LUNs

is emptied, and data at the snapshot point in time becomes unusable.

Judged from the previous process, after the HyperImage is activated, no matter

how data is read, written, or changed, data at the snapshot point in time can be

obtained timely provided that the HyperImage does not stop.

For some hosts using the cache management mechanism and the OceanStor

storage arrays, the HyperImage is inadequate to keep data consistency. The

reason is that some data in the cache of a host using the cache management

mechanism is possibly not delivered to disks. In this type of applications, the

HyperImage has to work with the snapshot agent on the host to keep data

consistency. For the work principle of the HyperImage working with the snapshot

agent, see the description of application scenarios.

2.2 Features of the HyperImage

2.2.1 Zero Backup Window

The traditional backup reduces the performance of an application server (AS),

even to an unacceptable level. So traditional backup tasks have to be processed

when the AS is shut down or the service amount is comparatively small. The

backup window refers to an interval of time during which a set of data can be

backed up and that can be accepted by the applications. That is to say, the

backup window is the maximum downtime accepted by the applications. A

backup task through the HyperImage can be processed online, and the backup

window is near to zero. The AS is not required to be down.

2.2.2 Saving Disk Capacity

When obtaining a consistent duplicate of the source LUN at the snapshot point in

time, the user only needs to save the original data of the first update after the

snapshot point in time on a resource LUN. The capacity of the resource LUN has



nothing to do with that of the source LUN, but is determined by the variation of

data on the source LUN after the snapshot point in time. The capacity of the

resource LUN never exceeds that of the source LUN. When the variation of data

on the source LUN is small, the HyperImage obtains a consistent duplicate of the

source LUN by using a small disk space. The consistent duplicate can be used by

other test services.

Supporting sharing of resource LUNs (originated with Huawei), the OceanStor

storage array enables the snapshot services of the entire system to occupy less

disk capacity. Sharing of resource LUNs means that multiple source LUNs can

share one resource LUN or use different resource LUNs, but all resource LUNs

are in the resource pool, and data on each resource LUN is shared by the others.

2.2.3 Quick Data Recovery

Backup data generated by traditional offline backup tasks cannot be read online.

Obtaining a usable duplicate of the source data at the backup point in time

requires a long-time data recovery process. The HyperImage can obtain the data

on the source LUN at the snapshot point in time by reading the snapshot LUN

directly. When data on source LUN is damaged, data on the source LUN at the

snapshot point in time can be directly recovered from the snapshot LUN through

the convenient data rollback.

2.2.4 Continuous Data Protection Through Cyclical Timing Snapshots

The OceanStor storage array supports creating virtual snapshots of a source LUN

at multiple points in time. The user can make a timing policy to activate and stop

the HyperImage. When automatic operations upon snapshots at multiple points in

time move forward along the time axis, continuous data protection at a low cost is

achieved.

2.2.5 Snapshot Consistent Group

In the application of the online transaction processing (OLTP), snapshots of

multiple source LUNs are required to be created to keep related data of the

application on different LUNs at the same time. For example, in the Oracle

database, management data, service data, and log information are scattered on

different source LUNs. The snapshots of the source LUNs of the three parts of

data should be created at the same point in time, so that the management data,

service data, and logs can be recovered to the same point in time during data

recovery. Otherwise, data relevance is lost because the data of three parts cannot

be recovered to the same time, and the data recovery becomes meaningless.

The snapshot consistent group of the OceanStor storage array solves this

problem. It freezes data on multiple source LUNs at the same snapshot point in



time and then obtains consistent snapshots of these source LUNs at the same

point in time.

2.2.6 Influence of Snapshots on Performance

Using the HyperImage in a disk array has an influence on the system

performance. When the HyperImage is activated, the write performance is not

influenced, but the write operations become more complex. When the variation of

data on the source LUN is great, write operations increase, thus reducing the

system performance to some degree.

2.3 Applications of the HyperImage

2.3.1 Rapid Data Backup and Data Recovery

The OceanStor HyperImage is differentiated from other backup methods by its

speed. Using the HyperImage does not involve data replication. Therefore, no

matter how large the amount of data is, it is very quick to create a snapshot.

Similar to a photo, a snapshot through the HyperImage is created instantly, but it

records the data at the moment when it is created. The HyperImage minimizes

the backup window (nearly eliminated) when backing up data.

When data on the source LUN is damaged due to a virus invasion or human

factors, the HyperImage can quickly roll back to the data at the specified snapshot

point of time to implement rapid data recovery.

It must be noted that the data backed up by the HyperImage disappears when the

HyperImage stops. Therefore, in many backup scenarios, the snapshot

technology is used together with backup software to back up data. The backup

software reads data from the snapshot and backs data up. In this way, the backup

window of traditional backup is eliminated.

2.3.2 Continuous Data Protection

Another significant feature that differentiates the HyperImage from other backup

methods is that the backup data takes up a very small storage space. This feature

enables users to create multiple snapshots of the same piece of source data.

Users can recover the source data at different points in time by using different

snapshots. The HyperImage does not only inherit the merits of the traditional

virtual snapshot technology, but also uses multiple new technologies to flexibly

support the service continuity solution. Continuous data protection can be

implemented through multiple snapshots automatically created by the

HyperImage at different points in time in a specified period. Figure 2-4 shows the

work principle:

Figure 2-4 Using the HyperImage to implement continuous data protection



The following description assumes that the system is set to create a snapshot

hourly from 1:00 pm. If from 1:00 pm to 2:00 pm, a and d (names of two data

blocks) change into a' and d'. Before a' and d' are written onto the source LUN, a

and d are saved on the snapshot resource LUN. From 2:00 pm to 3:00 pm, g

(name of a data block) changes to g'. Before g' is written onto the source LUN, g

is saved on the resource LUN. From 3:00 pm to 4:00 pm, c and i (names of two

data blocks) change into c' and i'. Before c' and i' are written onto the source LUN,

c and i are saved on the resource LUN.

To recover data at different points in time through the snapshot rollback, do as

follows:

1) To recover the data to that at 3:00 pm, replace c and i in the current source

volume.

2) To recover the data to that at 2:00 pm, replace g in the source volume of

3:00 pm.

3) To recover the data to that at 1:00 pm, replace a and d in the source volume

of 2:00 pm.

Continuous data protection through the HyperImage is not real, for real-time

snapshots of the source LUN cannot be created through the HyperImage. The

minimum interval between two snapshots determines the granularity of the

continuous data protection. The OceanStor HyperImage uses an advanced



algorithm to make full use of the resources when providing continuous data

protection. Multiple snapshots of the same source LUN share a resource pool, so

that storage capacity is saved and unnecessary copy-on-write is prevented. In

addition, the overall performance of the storage array is improved.

2.3.3 Redefining the Usage of the Data

The consistent images created by the OceanStor HyperImage can be read

directly. They are available for test, archiving, and query. They protect the

production system and increase the usage of the backup data, as shown in Figure

2-5.

Figure 2-5 Snapshot copy

2.3.4 Snapshot Agent of the OceanStor Series Storage Product:

Snapshot Agent

As known to all, databases or certain applications use the cache management

mechanism to make a large amount of data stay in the memory or storage media

in which data is transferred quicker, so that the response and access

performance of the system are enhanced. However, the HyperImage is dedicated

to storage arrays. Data in the host memory is not written onto hard disks in time

when the HyperImage starts. As a result, data inconsistency is inevitable when

the HyperImage is applied to these applications.

The Snapshot Agent is a tailored application for databases and these applications.

Its core function is to ensure the integrity and consistency of data on the hard

disks when data protective measures, such as the HyperImage, are working. In

this way, the Snapshot Agent ensures that the data is available immediately after

the recovery. The Snapshot Agent is installed on the database or hosts that use



the cache management mechanism. Together with the HyperImage, it ensures

data consistency at the snapshot point in time. Figure 2-6 shows the work

principle.

Figure 2-6 HyperImage based on the Snapshot Agent

1) Before starting the HyperImage, the system notifies the Snapshot Agent

installed on the host first.

2) The Snapshot Agent notifies the application of the database that the

HyperImage is going to start. The database system goes to the backup

mode, and then performs the "commit" operation, that is, the database

system delivers the data stored by the application in the host memory to the

production volumes. This process is called "disk flushing." During the disk

brushing, data cannot be written to the database.

3) When the disk brushing succeeds, the Snapshot Agent notifies the system

that the HyperImage can start.

4) As soon as the HyperImage starts successfully, the Snapshot Agent is

notified of recovering the application of the database to the normal mode.



Caution:

The Snapshot Agent developed by Huawei is applicable to only the Suse 9

platform and the Oracle 10i database. Now it is going through the certification on

Veritas-related backup software. It is estimated that the Snapshot Agent can

obtain the certificate in the second quarter of 2009.

2.3.5 VSS and HyperImage

Microsoft has introduced the Visual Source Safe (VSS) since Windows 2003 SP1.

The VSS defines the backup application framework on the Windows platform.

With the VVS, the parties involved in the backup can collaborate efficiently with

each other. The VVS provides the application programming interface (API) for the

administrator to automatically create snapshots and perform the backup. With the

collaboration of the VSS, the storage, backup software, and applications from a

third party can be smoothly used in the storage and backup process. Thus the

time and complexity of backup are reduced greatly.

The VSS defines three different roles to describe the different functions of the

software entities in the backup process. The three roles are the writer, requester,

and provider. A writer is an entity that uses data objects, such as a database and

a mail server. A requester mainly refers to the backup software, such as Veritas

NetBackup (NBU) and Windows NTBackup. A provider is a snapshot method

provider, which can be an operating system (OS), volume management software,

or hardware (array). Taking the backup using the NBU as an example, Figure 2-7

shows the realization process.

Figure 2-7 VSS invoking the HyperImage in the backup

1) The backup software NBU (requester) sends a backup request to the VSS

and specifies the writer that needs to be backed up.



2) After receiving the backup request forwarded by the VVS, the database

server (writer) writes the data in the buffer area onto the hard disks and

notifies the VSS.

3) The VSS queries whether the array (provider) has the snapshot ability (the

default priority that the VSS invokes the snapshots: the hardware snapshot,

the software snapshot, and then the snapshot of the VSS itself). At the same

time, the VSS notifies the NBU of issuing the snapshot activation command.

4) The NBU sends the snapshot activation command to the VSS.

5) The VSS locks the applications on the host temporarily to prevent

inconsistency of data at the snapshot point in time resulting from the read

and write operations performed by the host.

6) The VSS forwards the snapshot activation command to the OceanStor

storage array for executing.

7) After the snapshot is activated, the VSS notifies the host of starting the

applications.

8) The NBU accesses and backs up data saved on the snapshot LUN of the

OceanStor storage array, and then writes the data onto the tapes.

2.3.6 Backup Software and HyperImage

The backup software NBU/BE implements the backup service in the mode of

invoking the snapshot volume. This mode can be used to effectively avoid the

backup window problem and can be used to back up the data online. Different

backup modes such as LAN-Base, LAN-Free, and Server-Free can be

implemented through different optional components of the backup software. As

for the array, the problem is how to map the snapshot volume to the backup

system so that the backup system can access the data. This problem can be

solved in the following three ways: The first method is that the backup software

directly invokes the snapshots of the array. This is the simplest way, but the

prerequisite is that the backup software and the array are tightly coupled and they

can provide interfaces. The second method is to use an intermediate to act as the

coordinator. The typical solution is the VSS. The last method is to use a

loose-coupling solution agreed by both parties. For example, both parties agree

to perform the snapshot of the array at 9:50 a.m. and map the snapshot volume to

the backup system. The backup software then performs the backup at 10:00 a.m.

Taking the LAN-Base backup based on the NBU as an example, Figure 2-8

shows the backup process using the agreement mode.

Figure 2-8 LAN-base backup based on the NBU and the snapshot



1) Create snapshots of the production volume in the mode of setting timing

policy-creation by using the Snapshot Agent or in manual mode. Then map

the snapshot to the client. As shown in Figure 2-8, the snapshot is created at

9:50 a.m. and the backup operation is performed at 10:00 a.m.

2) Set the backup policy on the master server. (The timing backup performed

by the NBU is later than the timing backup set by the agent to ensure that the

agent already creates the snapshots of the production volume when the

NBU starts to back up.)

3) The master server sends the backup notification.

4) The media server accesses the data saved on the client.

5) The media sever writes the data onto the tape devices.

Taking the Server-Free backup based on the NBU as an example, Figure 2-9

shows the backup process based on the intermediate VSS.

Figure 2-9 Server-free backup based on the NBU, the VSS and the snapshot



1) Before backing data up, create the configuration file to record the source

LUN ID, snapshot LUN ID, snapshot type, and array serial number; and

configure the path of the in-band command (IC) and the host ID that is

mapped to the array.

2) The backup application NBU issues the backup command to the source

LUN.

3) The VSS framework (the VSS system service provided by the OS) queries

the configuration information about the current snapshot.

4) The master server notifies the client of creating the snapshots on the disk

array.

5) The VSS framework queries the information about the snapshot LUN.

6) The VSS framework mounts the snapshot LUN on the backup system, that is,

to the client and the media server, and notifies the NBU of the information on

the snapshot LUN.

7) The media server writes the data onto the tape devices by accessing the

snapshot volume.

Huawei HyperImage Technical White Paper.pdf

Documents

Transcript of Huawei HyperImage Technical White Paper.pdf