Gerd W. Becker, EMPALIS GmbH TSM Symposium … W. Becker, EMPALIS GmbH ... NAS device Tivoli Storage...

Autor 1

How to backup NAS-Filers with (and without) NDMPHow to backup NAS-Filers with (and without) NDMPHow to backup NAS-Filers with (and without) NDMPHow to backup NAS-Filers with (and without) NDMP

Gerd W. Becker, EMPALIS GmbHTSM Symposium 2009 Petersberg

TSM Symposium 2009 Empalis GmbH 2

Gerd Becker is a Project Manager for EMPALIS GmbH, an IBM Business Partner in Germany. He has more than 30 years of IT experience, including over 15 years experience with storage management products such as DFSMS and Tivoli Storage Manager. His areas of expertise include IBM Tivoli Storage Manager implementation projects and education at customer sites, including mainframe environments (OS/390®, VSE, VM, and Linux® for zSeries®). He holds several certifications, including technical and sales, and is an IBM Tivoli Certified Instructor. He has developed and taught several storage classes for IBM Education Services in Germany, Switzerland and Austria and for qSkills in Nuernberg. He has been Chairman of the Guide Share Europe (GSE) Stoage-Usergroup for more than 6 years. He is author of the Redbooks „IBM Tivoli Storage Manager Technical Guide 5.3, IBM Tivoli Storage Manager Technical Guide 6.1, IBM Tivoli Storage Manager Certification Guide 6.1“, did the betatest for TSM Version 5.5 and 6.1 and is member of the TSM Advisory Council.

Personal Data

� Gerd BeckerGerd BeckerGerd BeckerGerd BeckerGerd BeckerGerd BeckerGerd BeckerGerd Becker� Empalis GmbHEmpalis GmbHEmpalis GmbHEmpalis GmbHEmpalis GmbHEmpalis GmbHEmpalis GmbHEmpalis GmbH� [email protected]@[email protected]@[email protected]@[email protected]@empalis.com


What methods can we use, to backup NAS Filers like IBM N series / NetAppwith IBM Tivoli Storage Manager.

Details about● Snapshots● NDMP● Filer-to-Server● Snap Mirror to Tape● Incremental with Snapdiff.

This presentation will show, how it fits all together to design a powerful backup and disaster recovery solution

Abstract

The Team who wrote the RedbookIBM Tivoli Storage Manager Technical Guide Version 6.1IBM Tivoli Storage Manager Technical Guide Version 6.1IBM Tivoli Storage Manager Technical Guide Version 6.1IBM Tivoli Storage Manager Technical Guide Version 6.1


Agenda – NDMP Enhancements Tivoli Storage Manager and NDMP Filer-to-attached-library Filer-to-server Administration, Backup and Restore New Functions


Tivoli Storage Manager and NDMP

The NDMP Client-Server architecture separates the control and data flow: Data Management Application Data Management Application Data Management Application Data Management Application (DMA) also known as the NDMP client which is typically a

backup application. Since NDMP Version 4 support, NDMP clients are called DMAs.– The DMA initiates, controls, and monitors NDMP sessions for backup and recovery of

data and tracks and manages backups and media. NDMP server NDMP server NDMP server NDMP server performs one or more services– The Data Service transfers data to and from primary storage (typically a NAS file system)– The Tape Service transfers data to and from secondary storage (typically a tape drive)

and allows the DMA to manipulate and access secondary storage (tape positioning and I/O)

– The SCSI Service passes low-level SCSI commands from the DMA to a SCSI device (typically a media changer)




Filer-to-attached-library


Overview of NDMP Offsite Vaulting

– Tivoli Storage Manager server backend-data movements are enhanced to use NDMP tape-to-tape copy functions for tape volumes containing NDMP-generated backup images.

– Allows for the backup and restore storage pools and restoration of individual volumes in an NDMP storage pool.

– Allows for Move Data within an NDMP storage pool.

– Allows use of Copy Storage Pool Volumes when needed for a Restore Node process (no simultaneous-write support)

– Allows for integrating Disaster Recovery Manager (DRM) functions with NDMP storage pools.


Requirements

� Hardware and software requirements::::– Tape-to-tape Copy – Data Mover must be able to

read and write to both source and target drives. For example, they must have correct paths, mount points, and attributes in pools.

– Data Movers must support NDMP version 4 for all new function.

� Tivoli Storage Manager requirements – Tivoli Storage Manager 5.4 server on platforms other

than z/OS (no storage agent support).


Miscellaneous Features

� DRM– SET DRMPRIMSTGPOOL and SET DRMCOPYSTGPOOL may include storage pools

containing NAS data and TOC files.� BACKUP STGPOOL

– Incremental is based on backing up NAS images – There is no sub-image backup or continuation.

– Ensure that the TOC storage pool also has a copy storage pool. Since there was no copy storage pool for NAS pools in prior releases, it could be missed in implementing the new functionality. Best practice would be a separate TOC-only pool with a separate copy storage pool.

� NAS Workload Influence– The server keeps track of server NAS operations. When a request is made which can

be satisfied by multiple data movers (having paths to all needed devices), the server takes into account the workload in choosing a data mover.

– 3-Way Function is not used in this new function. Paths to all needed devices must be available to perform a backend operation.




NDMP: Filer-to-Server Configurations

Data

NDMP control (TCP/IP)

Roboticscontrol

Tape libraryData(TCP/IP)

Data

NAS file system

TSM Server

NAS Device

� Allows centralization of tape resource� Full capability of TSM storage hierarchy (migration, reclamation, copypool, etc.)


Overview

� NDMP Filer-to-server backup allows you to backup NAS devices to the native Tivoli Storage Manager storage hierarchy (storage pools with data formats of NATIVE or NONBLOCK).

� No need for a tape library connected to NAS device. Backup is through the LAN.

� All Tivoli Storage Manager backend data management functions can be performed on NAS backup data in native storage pools.

� All existing NAS backup functions are supported for Filer-to-server backups:– TOC creation– File level restore (FLR)– Direct Access Recovery (DAR)


Requirements

Hardware and Software Requirements:– NAS file server which supports NDMP version 4.0 and NDMP three-

way backup– Tivoli Storage Manager 5.4 server (Unix or Windows) with enough

free storage capacity for NAS backup images (typically very large)– Tivoli Storage Manager 5.4 Administration Center is needed to

configure filer-to-server operations via GUI.

Note: Once Tivoli Storage Manager native storage pools have Note: Once Tivoli Storage Manager native storage pools have Note: Once Tivoli Storage Manager native storage pools have Note: Once Tivoli Storage Manager native storage pools have been used for NAS backups, previous versions of the been used for NAS backups, previous versions of the been used for NAS backups, previous versions of the been used for NAS backups, previous versions of the Tivoli Storage Manager server cannot be used to restore Tivoli Storage Manager server cannot be used to restore Tivoli Storage Manager server cannot be used to restore Tivoli Storage Manager server cannot be used to restore the resulting backups.the resulting backups.the resulting backups.the resulting backups.


Classic NDMP Implementation

NDMP Local Backup ConfigurationNDMP Local Backup ConfigurationNDMP Local Backup ConfigurationNDMP Local Backup Configuration

NDMP DMA

NDMP tape server

control

data

Datacontrol

NAS device Tivoli Storage Manager server

NAS file system

NDMP data server

tape subsystem

data


Background: NDMP Components

� Tape Server:Tape Server:Tape Server:Tape Server: Accepts backup data from an NDMP data stream and writes to tape, or Accepts backup data from an NDMP data stream and writes to tape, or Accepts backup data from an NDMP data stream and writes to tape, or Accepts backup data from an NDMP data stream and writes to tape, or reads restore data from a tape and writes it to an NDMP data stream. Two main sub-reads restore data from a tape and writes it to an NDMP data stream. Two main sub-reads restore data from a tape and writes it to an NDMP data stream. Two main sub-reads restore data from a tape and writes it to an NDMP data stream. Two main sub-components:components:components:components:– Mover Interface: Controls reading and writing of data from the NDMP

data server and a tape device.– Tape Interface: supports tape read/write and tape positioning

operations� Data Management Application (DMA):Data Management Application (DMA):Data Management Application (DMA):Data Management Application (DMA): Backup software application which initiates Backup software application which initiates Backup software application which initiates Backup software application which initiates

NDMP backup and restore operations on NDMP tape and data serversNDMP backup and restore operations on NDMP tape and data serversNDMP backup and restore operations on NDMP tape and data serversNDMP backup and restore operations on NDMP tape and data servers� Data Server: Data Server: Data Server: Data Server:

1. Reads data from a file system and generates an NDMP data stream in a vendor specific backup format.

2. Sends file metadata to DMA during backup. 3. Reads an NDMP data stream and restores it back to a file system.


Tivoli Storage Manager Filer-to-server Backup Implementation

NDMP DMA

NDMP tape server

control

data

server bit file servicesdata

Tivoli Storage Managerstorage

data

control

NDMP Filer-to-Server ConfigurationNDMP Filer-to-Server ConfigurationNDMP Filer-to-Server ConfigurationNDMP Filer-to-Server ConfigurationNAS device Tivoli Storage Manager 5.4 server

NAS file system

NDMP data server

data


Tivoli Storage Manager NDMP Tape Server

� A modified NDMP tape server is implemented in the Tivoli Storage Manager server.

� Only the Mover Interface is implemented and has no direct access to tape devices.

� It stores NAS backup images in Tivoli Storage Manager Storage hierarchy.

� Tivoli Storage Manager tape server can only be accessed from within Tivoli Storage Manager server. No other DMA may communicate with it – including DMAs of other Tivoli Storage Manager servers.

� No user configuration is needed unless the default NDMP tape server port is unavailable.

� It supports multiple concurrent NDMP operations.


New Functions

� No new server commandsNo new server commandsNo new server commandsNo new server commands� QUERY NASBACKUP now displays the storage pool in which each image is stored now displays the storage pool in which each image is stored now displays the storage pool in which each image is stored now displays the storage pool in which each image is stored� New Server Options:New Server Options:New Server Options:New Server Options:

– NDMPControlport port-number• Port used by Tivoli Storage Manager tape server and Tivoli Storage Manager

DMA to communicate internally. Default is 10000.– NDMPPortrange port-number-low[ ,port-number-high ]

• The range of port numbers from which Tivoli Storage Manager tape server will select a port for accepting connections from a NAS device’s data server. Default is to let the OS provide a port (ephemeral port).

– NDMPPREFDATAinterface ip-address• With the NDMPPREFDATAINTERFACE option enabled, you can specify which

network interface theTivoli Storage Manager server uses to receive NDMP backup data during NDMP filer-to-server backup operations.

Tivoli Storage Manager Administration Center Tivoli Storage Manager Administration Center Tivoli Storage Manager Administration Center Tivoli Storage Manager Administration Center – NAS Configuration for Storage

• Wizard is updated to allow for pointing NAS backups to a native storage pool.


Setup for NDMP Filer-to-server Backup

1. Register a NAS node.2. Define a data mover for NAS node (no change from today).3. Determine target native storage pool (for example:

BACKUPPOOL or tape pool).4. Determine location for TOCs. 5. Set or update policy appropriately. Can use same policy as

traditional Tivoli Storage Manager clients.


Special considerations in a filer-to-server configurationSpecial considerations in a filer-to-server configurationSpecial considerations in a filer-to-server configurationSpecial considerations in a filer-to-server configuration

� Simultaneous write is not supported for NAS backup images.� The backup mode for NDMP-generated data in the filer-to-

server configuration remains image-based. This means that NDMP-generated data will not be created in progressive incremental mode.

� NAS backup data cannot be moved between NAS type storage pools (e.g. NETAPPDUMP) and native storage pools.

� Export and import of NAS images and NAS nodes is not supported

� The communication session from the filer to the server will be established on the network that was identified through the gethostbyname gethostbyname gethostbyname gethostbyname function of the Tivoli Storage Manager server. Make sure you have appropriate name resolution in place.


Filer-to-server Backup: Things to Remember

� The estimated size of a NAS backup image is always the current occupancy of the entire file system – even for differential backups.

� Connectivity can be affected by firewalls. NAS device now opens outbound connection to Tivoli Storage Manager tape server during backup.

� All backend data movement is supported for NAS Images stored in native storage pools.

� Full plus Differential restores can use a mix of storage pool types. Example: Full in NAS pool, differential in native pool.


NDMP Wizards Overview

� Create the backup storage pool as either NAS (in a tape library attached to the NAS File Server) or as Native (using a device class).

� Create backup storage pools as well as primary storage pools.

� Add additional tape libraries to existing NAS File Server.

Main Enhancements allow user to configure NDMP to:


Requirements to Use NDMP Wizards

� Hardware: special file names (device names) of the library and drives being attached, and element numbers

� Element numbers of drives (for SCSI tape)� ACS id's (for ACSLS)� NAS file server's TCP name or IP address and port,

and the username and password� Device class name (for native storage pools)


Wizard Changes

� The existing NDMP wizard is called Add a NAS Node and is a table action in the Nodes Table.

� This wizard is renamed to Protect a NAS File Server and is an action in the Servers, Libraries, Nodes, Storage Pools, Datamovers, and Device classes tables (all the tables on the Storage Devices portlet).

� A new, similar wizard is called Add Storage to NDMP operations and is located in all the tables on the Storage Devices portlet.

� This wizard assumes that the NAS file server is already fully configured. Its purpose is strictly to: – Add a new library to it.– Connect it to another existing library with new drive paths. – Add a new primary and copy storage pool.


NDMP Configuration Wizard


Protect a NAS File Server: Define NAS Node


Protect a NAS File Server: Select Destination


Protect a NAS File Server: Select Policy Domain


Protect a NAS File Server: Select an Existing Domain

• Only domains that already have NAS file server's nodedefined to them are shown in the select action box. This is a

best practice, to avoid mixing NAS and non-NAS data.• If an existing domain is selected, that domain's policies including storage pools, are used. New storage pools are not defined.

• If no such domains exist, the ability to choose a domainis not shown, and the user must define a new domain for theNAS data.


Protect a NAS File Server: Define a Native Storage Pool


Protect a NAS File Server: Native Copy Storage Pool


Protect a NAS File Server: Wizard Summary


Add Storage to NDMP Operations: Select Data Mover


Add Storage to NDMP Operations: Select a Library


Add Storage to NDMP Operations: Add a Drive Path


Add Storage to NDMP Operations: Create Storage Pools


Protect a NAS File Server: Wizard Summary


… here again with commands

� REGISTER NODE na3025 ?***? PASSEXP=0 CONTACT= TYPE=NAS� DEFINE DATAMOVER na3025 TYPE=NAS HLADDRESS=na3025

LLADDRESS=10000 USERID=ndmpuser PASSWORD=?***? DATAFORMAT=NETAPPDUMP

� DEFINE STGPOOL TOCPOOL DISK pooltype=PRIMARY description="Stg-Pool fuer NDMP-TOC„

� DEFINE VOLUME TOCPOOL e:\tsm\data\tocvol01.dsm formatsize=100� DEFINE DOMAIN PD_NAS DESCRIPTION="Policy_Domaene fuer NDMP-

Backups" BACKRETENTION=30 ARCHRETENTION=365� DEFINE POLICYSET PD_NAS standard desc="Standard policy set„� DEFINE MGMTCLASS PD_NAS standard standard desc="Standard

management class fuer Filer-to-Server"� DEFINE COPYGROUP PD_NAS standard standard destination=NDMPPOOL� ASSIGN DEFMGMTCLASS PD_NAS standard standard� ACTIVATE POLICYSET PD_NAS standard� UPDATE NODE NA3025 domain=PD_NAS


First Test

• Administative TSM-Command:

• A process will be started


Backup with Web-Client




Two-Tier Concept -Data recovery from Snapshot


Two-Tier Concept - Data recovery from NDMP-Dump

� Image RestoreImage RestoreImage RestoreImage Restore– Restore complete Volume– Directorey-based restore

� Single File or Folder RestoreSingle File or Folder RestoreSingle File or Folder RestoreSingle File or Folder Restore– Web-Client– Command-Line Interface– Point-in-Time


Two-Tier Concept - Recovery form NDMP-Dump with Web-Klient


Setup Administrator for Web-Client

We need an Administrator with following privileges:– Node Owner– Policy resticted / unrestricted– System

Define Node Owner Administrator At registration of the node, with parameter userid=<admin-name>

– Manuell, by register adminand grant authority node owner for one or more nodes

Administrative command:– Register node ... Userid=node name– Register admin– Grant auth


Privileges for Web-Client


Backup and Restore of NAS-Data


Restore on File-Level

Restore singel Files or Direcotries with Web-Client or Server-Command

– Restore with Web-Client for Backup / Archive• Table of Content available• Web-Client on Windows2000 oder Windows2003• Temporary load TOC in Server Database• examine directories and files contained in one or more file system images, and select individual

files or directories to restore directly from the backup images generated

– .

– Restore using Server Interface• QUERY NASBACKUP• RESTORE NODE with parameter FILELIST• Without TOC the contents of the backup image cannot be displayed. You can restore individual

files, directories, or both, if you know the name of the file or directory, and in which image the backup is located.


Restore on File-Level with Web client

�Select FILELEVEL �Load TOC


Restore on File-Level with Web client

�Select files for restore�Select destination for restored objects


Troubleshooting

� Are the options correctly set on the filer?ndmpd status

� Have a look into the files on the filer?/etc/log/backup

� Set up NDMP Trace priv set advanced (take care, super-user permissions)ndmpd debug 70 (reset after test to 0)

� Result in /etc/log/ndmpdlog� Set up Trace in TSM:� trace disable * � trace enable spi spid sessremote addmsg � trace begin pathandfilenamehere � ...run the test � trace flush � trace end � trace disable *




General VSS Architecture

� Volume Shadow Copy Service - A service that coordinates various components to create consistent A service that coordinates various components to create consistent A service that coordinates various components to create consistent A service that coordinates various components to create consistent shadow copies of one or more volumes.shadow copies of one or more volumes.shadow copies of one or more volumes.shadow copies of one or more volumes.

� Requestor - An application that requests that volume shadow copy be taken. A backup application An application that requests that volume shadow copy be taken. A backup application An application that requests that volume shadow copy be taken. A backup application An application that requests that volume shadow copy be taken. A backup application is an example (This is DP for SQL or DP for Exchange)is an example (This is DP for SQL or DP for Exchange)is an example (This is DP for SQL or DP for Exchange)is an example (This is DP for SQL or DP for Exchange)

� Writer – A component of an application that stores persistent information on one or more volumes A component of an application that stores persistent information on one or more volumes A component of an application that stores persistent information on one or more volumes A component of an application that stores persistent information on one or more volumes that participate in shadow copy synchronization. Typically this is a database application like SQL that participate in shadow copy synchronization. Typically this is a database application like SQL that participate in shadow copy synchronization. Typically this is a database application like SQL that participate in shadow copy synchronization. Typically this is a database application like SQL Server, Exchange Server, or a system service like Active DirectoryServer, Exchange Server, or a system service like Active DirectoryServer, Exchange Server, or a system service like Active DirectoryServer, Exchange Server, or a system service like Active Directory

� Provider – A component that creates and maintains the shadow copies. Examples are the system A component that creates and maintains the shadow copies. Examples are the system A component that creates and maintains the shadow copies. Examples are the system A component that creates and maintains the shadow copies. Examples are the system provider included with the operating system and the hardware providers included with storage provider included with the operating system and the hardware providers included with storage provider included with the operating system and the hardware providers included with storage provider included with the operating system and the hardware providers included with storage Arrays.Arrays.Arrays.Arrays.

� Source Volume – The volume that contains the data to be shadow copied.The volume that contains the data to be shadow copied.The volume that contains the data to be shadow copied.The volume that contains the data to be shadow copied.� Storage/Target Volume – The volume that holds the shadow copy storage files for the system The volume that holds the shadow copy storage files for the system The volume that holds the shadow copy storage files for the system The volume that holds the shadow copy storage files for the system

copy-on-write software provider.copy-on-write software provider.copy-on-write software provider.copy-on-write software provider.



Snapshot Difference API Snap-Mirror-to-Tape


Issues for Implementations up to TSM 5.5.x


TSM 6.1 N series Snapshot Exploitation

� Progressive Incremental Backup with Snapshot Differencing

� NDMP Snapmirror-to-Tape (SM2T) Backup and Restore


TSM Backup Client with Snapshot Differencing


Incremental/Snapdiff Backup - NetApp Snapshot API

1. NetApp has extended the Snapshot API to include the Snapshot Difference API (SnapDiff)

2. When called, Snapdiff returns a list of “changed” files, based on successive snapshots (“base” and “diff”)

3. Files are deemed “changed” if they have been created, modified or deleted (as determined by Snapdiff)


Incremental/Snapdiff Backup - NetApp Snapshot API

1. NetApp has extended the Snapshot API to include the Snapshot Difference API (SnapDiff)

2. When called, Snapdiff returns a list of “changed” files, based on successive snapshots (“base” and “diff”)

3. Files are deemed “changed” if they have been created, modified or deleted (as determined by Snapdiff)


Incremental/Snapdiff Backup - Processing (1)

Initial Incremental (Backup)1. Incremental (using Snapdiff) initiated by TSM2. nSeries/NetApp machine generates a file list to use

for the TSM incremental3. Incremental backup is completed and sent to TSM

server4. Snapshot name is also sent to TSM, (stored in

client’s filespace)





Subsequent Incrementals

1. Name of the last snapshot retrieved from TSM server

2. New snapshot taken by nSeries/NetApp machine

3. Snapdiff compares the new and last snapshots and reports changes to TSM

4. Incremental of new snapshot is performed (based on #3 changes) and sent to TSM server

5. New snapshot name is sent to TSM server for use in next incremental


Incremental Snapdiff Backups – New Options (1)

– Snapdiff – no parameters

Examples:• incremental -snapdiff /net/home1 • incremental -snapdiff H:

– Diffsnapshot• Create• Latest

Example:• incremental -snapdiff H: -diffsnapshot=CREATE


Incremental Snapdiff Backups– New GUI Elements (1)

– SnapDiff is triggered by a new backup type selection, in the backup window

– This selection will only be allowed with mounted and eligible network volumes


Snapdiff Incremental – New PasswordPasswordPasswordPassword Parameters

The Set Password Set Password Set Password Set Password command has been extended to allow setting a userid and password to be used for SnapDiff operations on a NAS volume. A new parameter TYPE has been added to the set password command.

� Type Type Type Type – Specifies if this password is for the workstation or filer. Use TYPE=TSM to specify the Tivoli

Storage Manager password for your workstation. Use TYPE=FILER to specify this password is for snapdiff operations on a filer. The default is TSM.

� OtherserverOtherserverOtherserverOtherserver– Specifies the filer host name if TYPE=FILER. The filer host name must be the fully qualified DNS

name and can not be the IP address in the dotted decimal format or the short local hostname. � OtheruseridOtheruseridOtheruseridOtheruserid

– Specifies the userid to be used for the entity named by the otherserver parameter.� Otherpassword Otherpassword Otherpassword Otherpassword

• Specifies the password to be used for entity named by the otherserver parameter.

Example: dsmc set password –type=filer filer101 fileuser filer101pwExample: dsmc set password –type=filer filer101 fileuser filer101pwExample: dsmc set password –type=filer filer101 fileuser filer101pwExample: dsmc set password –type=filer filer101 fileuser filer101pwtsm> set password -type=filer sim1 administratorPlease enter password for user id "administrator@sim1": *****Re-enter the password for verification:*****ANS0302I Successfully done.

tsm> set password -type=filer sim1 administratorPlease enter password for user id "administrator@sim1": *****Re-enter the password for verification:*****ANS0302I Successfully done.


Incremental/Snapdiff Backups - Additional Information (1)

� No external or internal dependencies

� Storage Agent – LANfree data movement is supported

� No migration of data is involved– For existing TSM protected filers, the same rules apply. The first

time SNAPDIFF backup runs, it will create the base snapshot, and perform a full scan incremental backup of that snapshot. Subsequent backups will then use the snapdiff method for backup.



� Any and all changes to any and all filesystem objects are tracked and acted upon by SnapDiff. In this regard, TSM will rely on Snapdiff to provide a list of files eligible for backup

� This work item is applicable to NetApp/N-Series volumes that are NAS attached to the host on which TSM client is running. At this time, the host must reside on either AIX (NFS) or Windows (CIFS)

� SAN attached NetApp/N-Series volumes are not supported at this time



� Currently, entire NAS volumes are supported. Qtrees or subdirectories of a NAS volume are not supported. SnapDiff won't be allowed for any filespace not mounted or mapped to the root of the volume

� NetApp/N-Series filers running Data ONTAP 7.3 or later is supported

� The Java and web client GUIs are supported� Native windows GUI (MFC GUI) is being phased out

and is not supported� Both traditional and FlexVol filer volumes are

supported


Considerations - Incremental/Snapdiff Scenarios (1)

� Given: A file is excluded by incl/excl rules

– TSM performs an Incr/Snapdiff backup

– The user removes the exclude rule for that file

� Result: SnapDiff will not detect this incl/excl change because it only detects file changes between 2 snapshots


Considerations - Incremental/Snapdiff Scenarios (2)

� Given: A user adds an include statement to the option file� Result: Only if Snapdiff detects a change will that include rule

take affect

� Given: A Policy changes from Mode=modified to mode=absolute

� Result: This change will not be detected by Snapdiff

� Given: A copy group policy (mode) changes from “modified” to “absolute.”

� Result: SnapDiff incremental backups do not reference this policy attribute. After the initial backup, unless files change, they will not be backup up


Considerations - Snapdiff internationalization (3)

� Snapdiff does not support Unicode characters outside of 7-bit ASCII – Snapdiff with ONTAP UTF8 enabled volumes will not

return the filename in UTF8– Will be addressed by NetAPP in a future revision of

ONTAP– TSM dsmerror.log will contain "File Not Found" errors

for these files ANS1626E An unexpected error was encountered when processing a TSM operation using a hardware or snapshot function.

TSM function name : NAHWInterface::naPerformSnapDiffTSM function : Error invoking ONTAP API snapdiff-iter-next: Parsing error in

results: Input is not proper UTF-8, indicate encoding !Bytes: 0xF6 0xE4 0xFC 0xD6

TSM return code : 13001TSM file : NAHWInterface.cpp (6776)

07/22/2009 15:57:28 ANS5250E An unexpected error was encountered.TSM function name : baPerformSnapDiffTSM function : failed to perform snapdiffTSM return code : 653TSM file : backsnap.cpp (4678)

07/22/2009 15:57:28 ANS2832E Incremental by snapshot difference failed for \\filer1\winvol01. Please see error log for details.07/22/2009 15:57:30 ANS5283E The operation was unsuccessful.


Test Environment

The performance data contained in this document was measured in a controlled environment. Results obtained in other operating environments may vary significantly depending on factors such as system workload and configuration. Accordingly, this data does not constitute a performance guarantee or warranty

Filer: N5300 with Data ONTAP Release 7.3 TSM Server: 5.5.1.0 on a 2-way 3.4 GHz, 4GB Windows Server 2003 TSM Client: 6.1.0.0 connected to local TSM Server Storage Pool: File pool on fibre-attached DS8K Small Workload: 8 thousand 100KB files Large Workload: 1.2 million 10KB files Huge Workload: 12 million 10KB files

Performance issues and Test Results


Test results with small workloadThe performance data contained in this document was measured in a controlled environment. Results obtained in other operating environments may vary significantly depending on factors such as system workload and configuration. Accordingly, this data does not constitute a performance guarantee or warranty


In the Small Workload test, when 10% data changed, using NetApp snapshot difference improved throughput from 1538 KB/sec to 2758 KB/sec (80% improvement).

In the Small Workload test, when 20% data changed, using NetApp snapshot differenceimproved throughput from 2501 KB/sec to 3139 KB/sec (26% improvement).

In the Small Workload test, when 80% data changed, using NetApp snapshot differenceimproved throughput from 3501 KB/sec to 3530 KB/sec (1% improvement).

Test results with small workload




In the Large Workload test, when 10% data changed, using NetApp snapshot differenceimproved throughput from 228 KB/sec to 676 KB/sec (196% improvement) (about 3x as fast).

In the Large Workload test, when 20% data changed, using NetApp snapshot differenceimproved throughput from 433 KB/sec to 756 KB/sec (75% improvement).

In the Large Workload test, when 80% data changed, using NetApp snapshot differenceimproved throughput from 472 KB/sec to 913 KB/sec (93% improvement).

Test results with large workload




In the Huge Workload test, when 10% data changed, using NetApp snapshot difference improved throughput from 199 KB/sec to 537 KB/sec (170% improvement).

As an example of reduced backup window, NetApp reduced the backup window of this test from almost 17 hours to about 6.25 hours (37% of the time).

The improvement is quite significant when only a small percentage of the data changes. Specific results will vary depending on the hardware configuration. If the percentage of changed data is lower than 10%, an even greater improvement will be expected.

Test conclusion



Snapshot Difference API Snap-Mirror-to-Tape


NDMP backups of very large NAS devices can be slow SM2T is a block level movement of a filesystem Similar to BA Client BACKUP IMAGE command

Data written to tape is a block by block copy of the file system Disk geometry taken into account Restoration of traditional volume to unlike geometry can be very slow

SM2T operations are only supported for Netapp and IBM nSeriesfile servers Implementation provided by extending NDMP functions

WHY ?

NDMP Snapmirror-to-Tape (SM2T) Backup and Restore


Block level movement of a filesystem Similar to BA Client BACKUP IMAGE command

Data written to tape is a block by block copy of the file system Disk geometry taken into account Restoration of traditional volume to unlike geometry can be very

slow The destination of the retrieval must use the same or later

version of Data ONTAP The traditional volume file system format is different from

the FlexVol volume file system format FlexVol volume cannot be restored to a traditional volume Traditional volume cannot be restored to a FlexVol volume.

What is SnapMirror to Tape (SM2T)


SnapMirror to tape support for Nseries/NetApp filers

SM2T operations are only supported for Netapp and IBM nSeries file serversImplementation provided by extending NDMP functionsThe following specific functions will be provided:

1.Backup a Netapp file system to media using the Netapp NDMP SnapMirror to Tape function. � TSM admin command: BACKUP NODE� Both Direct to tape and TSM Native stgpool using filer-to-server

2.Retrieve a SnapMirror copy of a Netapp file system� TSM admin command: RESTORE NODE

3.Display information about SnapMirror copies.� TSM admin command: QUERY NASBACKUP type=snapm

4.TSM back-end data movement for SnapMirror copies � Same support as currently provided for NDMP backup images.� BACKUP STGPOOL, MOVE DATA


With “block” based data transfer the from/to disk geometry matters Restore to the same disk geometry as backup for best performance

Cannot restore Flexvol volume to traditional volume and vice versa Recommended for disaster recovery or offsite vaulting use cases

SM2T is not a replacement for traditional full/differential TSM NDMP backup See “ONTAP Data Protection Online Backup and Recovery Guide” for

specific ONTAP SM2T considerations For example – the ONTAP release level on the restore to system must be the same or higher

ONTAP release level of the system when it was backed up

TSM NDMP SnapMirror-to-Tape Considerations


What is not Available

Admin Command Line only No BA Client interface, neither Web GUI nor

command line No support through Admin Center

No NDMP differential backup of SnapMirror images No NDMP Directory level backup using SnapMirror to

tape TSM will not permit an SM2T backup operation on a

server virtual filespace mapping No Generation of a Table of Contents for SM2T images No NDMP file level restore from SM2T images Cannot be used as baseline to prime SnapMirror

relationship


SM2T Usage

Command BACKUP NODE nodename fsname type=snapmirror

Must be MODE=FULL Must be TOC=NO

RESTORE NODE nodename fsname [destination_fsname] type=snapmirror

QUERY NASBACKUP nodename fsname Environment variables passed to NetApp / nSeries for:

Implied Snapshots are deleted when the operation completes The SnapMirror relationship is broken when the operation

completes On restore, the destination must be available and prepared

NetApp command: “vol create ... ” NetApp command: “vol restrict ... ”


Tapebackup Technique Comparism (what is available today under ONTAP)

no (no SnapMirror License), noLicense costs NetApp?

Disaster protection focus

Netbackup, Commvault, Bakbone, Syncsort, Atempo,

TSM

yesallno

Full Volume only2 Ontap-Versions onlySM2T

yes

highhigh

Block level

SM2T Ontap 7.x

nearly all (10)supported by backup software

Full / Incr / Diffbackup / restores modes

nonoPVR for DP usage at NetApp?

File levelFile- or block level backup ?

1# included Snapshots (backup & restore)

SMTapeDumpTape format

noKeeping dedupe benefit kept (transfer & media)?

Longtime (10 Years)Tape format backward support

yessingle file / directory restore?

Archival focusrecommended backup use case

limited (file-level)Backupspeed Volumes with millions of small filesgoodBackupspeed Volumes with medium/large files

SMTape Ontap 8.0DUMPNDMP Engines


TSM 6.1 N series integration enhancements aredesigned to address the backup challenges associated with large filers

TSM NDMP SnapMirror-to-Tape provides an efficient disaster recovery or vaulting solution

TSM progressive incremental with Snapshot differencing provides an efficient file level backup/recovery solution with disaster recovery and vaulting options


Documents published in 2008 and 2009

Redbooks 2008 and 2009IBM Tivoli Storage Manager Version 5.4 & 5.5 Technical Guide, SG24-7447-0IBM Tivoli Storage Manager Version 6 Technical Guide (coming soon)

Using the IBM System Storage N Series with IBM Tivoli Storage Manager, SG24-7243-00Redbooks, published 10 July 2006, last updated 25 June 2007Tivoli Field Guides 2007http://www-306.ibm.com/software/sysmgmt/products/support/doc.html


qSkills-Class BR205: IBM Tivoli Storage Manager Backup & Recovery for NAS-FilersIBM Tivoli Storage Manager Backup & Recovery for NAS-FilersIBM Tivoli Storage Manager Backup & Recovery for NAS-FilersIBM Tivoli Storage Manager Backup & Recovery for NAS-Filers

NAS Konfiguration am Beispiel von Network Appliance

NDMP Begriffe NDMP Protokololl

Grundfunktionalität der Filer von Network Appliance Volumes Snapshots Snap restore Auswirkungen auf das Backup und Restore-Konzept Übungen

IBM Tivoli Storage Manager Konfiguration für NAS-Komponenten Device und Library Konfiguration Client Konfiguration Backup und Recovery TSM als NDMP Tape-Server Filer-to-Server Konfiguration Disaster Recovery Manager für NDMP-Backups Übungen am Beispiel von NetApp Filern

http://www.qskills.de/qs/workshops/backuprecovery/http://www.qskills.de/qs/workshops/backuprecovery/http://www.qskills.de/qs/workshops/backuprecovery/http://www.qskills.de/qs/workshops/backuprecovery/


Overview of the Workshops "Backup & Recovery"Overview of the Workshops "Backup & Recovery"Overview of the Workshops "Backup & Recovery"Overview of the Workshops "Backup & Recovery"

http://www.qskills.de/qs/workshops/backuprecovery/http://www.qskills.de/qs/workshops/backuprecovery/http://www.qskills.de/qs/workshops/backuprecovery/http://www.qskills.de/qs/workshops/backuprecovery/

Gerd W. Becker, EMPALIS GmbH TSM Symposium … W. Becker, EMPALIS GmbH ... NAS device Tivoli Storage...

Documents

Transcript of Gerd W. Becker, EMPALIS GmbH TSM Symposium … W. Becker, EMPALIS GmbH ... NAS device Tivoli Storage...