Table of - Education Services Home, · 2014 EMC Proven Professional Knowledge Sharing 6 In this...
Transcript of Table of - Education Services Home, · 2014 EMC Proven Professional Knowledge Sharing 6 In this...
2014 EMC Proven Professional Knowledge Sharing 2
Table of Contents
Introduction ........................................................................................................................................................ 3
First phase: Analysis ......................................................................................................................................... 4
Second phase: Planning & Design .................................................................................................................... 5
Greenplum backup on Data Domain using NFS or DD*boost ......................................................................... 5
Standard database backup using proxy host with Oracle RMAN & Data Domain in VTL ................................ 7
NDMP VNX backup using snapshot and Data Domain NDMP server with NetWorker ................................... 9
Third phase: Implementation and Test ............................................................................................................. 11
Fourth phase: Monitoring and Change Management ....................................................................................... 12
Conclusion....................................................................................................................................................... 12
Glossary .......................................................................................................................................................... 13
Disclaimer: The views, processes, or methodologies published in this article are those of the author. They do
not necessarily reflect EMC Corporation’s views, processes, or methodologies.
2014 EMC Proven Professional Knowledge Sharing 3
Introduction
Backing up big data environments is a huge challenge for storage administrators faced with backup window,
SLA, and RTO/RPO requirements. Considerations are complex and change depending on the type of
environment, backup flow, and so on.
In this article, I will explain some general considerations along with a case study developed during a recent
customer site engagement. While not presuming to cover all possible cases, I can provide my direct
experience in the field.
This article will explain different process phases needed to define backup strategy and implement it on a
customer site, including;
analysis
planning and design
test and implementation
monitoring
2014 EMC Proven Professional Knowledge Sharing 4
First phase: Analysis
During this phase it is very important to collect all needed information about the environment to back up,
keeping in mind that backup performance is impacted by all elements involved during operation (storage, SAN,
LAN, Host, etc.). To do that, customers need to have clear answers to the following questions:
What type of data are we managing?
o Big Data appliance (Greenplum®, Teradata, Netezza, etc.)
o Traditional Database (Oracle, DB2, SQL, etc.)
o NAS (VNX, NetApp, etc.); Is NDMP applicable?
o Unstructured data on server
Which compliance is needed?
o Backup windows
o Success rate required
o RPO
o RTO
Size and data characteristics?
o Data size
o Regarding appliances: how many appliances, how many instances, full/incremental backup
management of appliance, etc.
o Regarding standard databases: type of databases (Oracle Exadata, RAC, etc.), type of
database; transactional? not transactional?, archive log management, backup host
characteristics (CPU, RAM, SAN, connection, etc.), storage and SAN characteristics, backup
destination, etc
o Regarding NDMP: NAS model, disk structure, snapshot are allowed? If yes, number of backup
snapshots in parallel, number of file systems, number of files inside, LAN/SAN connections,
change rate
o Regarding unstructured data: number of file systems, number of files, backup host
characteristics (CPU, RAM, SAN, connection, etc.), storage and SAN characteristics, backup
destination, change rate
2014 EMC Proven Professional Knowledge Sharing 5
Second phase: Planning & Design
With all data collected during the analysis phase, it is necessary to design the solution in terms of:
Backup flow: backup direct from production, from snapshot, from proxy host
Requirements: LAN/SAN connectivity, storage, host, backup destination, backup server
Policy to implement: number of sessions, scripts needed, backup server and storage node
configurations, etc.
While this is the most critical phase, it is not possible to give a complete answer to all scenarios in one article
because all change in base of environment characteristics. For this reason, examples of one backup type
(Appliance, Standard database, NDMP) are provided below.
Greenplum backup on Data Domain using NFS or DD*boost
Parameters:
Dimension: 50TB for database (2 databases, 100 TB total), 10GB connection, 1 DD990 used in DD*Boost, 1
full backup every Friday for DB1 and every Sunday for DB2. RPO: 24 Hours, RTO: 2 days.
To manage this backup, consider size, backup windows, parallel segments, and boost reduction.
Parameters Value
Requirements Value
Size 50
TB/h 4.17
Backup windows 12
MB/s Read 1213.63
Parallel segments 60
MB/s LAN/Write NFS 1213.63
Boost reduction 3
MB/s LAN/Write Boost 404.54
MB/s Single channel NFS 20.23
MB/s Single channel Boost 6.74
2014 EMC Proven Professional Knowledge Sharing 6
In this case, you need to manage 1214MB/s in read from the Greenplum appliance. As well, you need
1214MB/s write to DDwith NFS protocol, or 406MB/s in write in the case of DD*Boost.
This means that you need one 10GB connection for DD*Boost and two 10GB connection for NFS
Considering a DD990, you need at least 5-6 shelves in order to attain the performance needed. Fifty percent of
DD990 sustained throughput will be used in NFS and 25% using DD*Boost.
In this case, restore considerations are more restrictive because RTO is 2 days.
Parameters Value
Requirements Value
Size 50
TB/h 1.04
Backup windows 48
MB/s Read 303.41
Parallel segments 60
MB/s LAN/Write NFS 303.41
MB/s Single channel NFS 5.06
In this scenario, no backup software is used but all is managed by Greenplum and Data Domain®.
2014 EMC Proven Professional Knowledge Sharing 7
Standard database backup using proxy host with Oracle RMAN & Data Domain in VTL
Parameters:
Dimension: 80TB in one database, backup window is 24 hours, 1 full backup every day, backup managed by
proxy host, RTO: 2 days, RPO: 2 hours (managed by archive logs direct from production) This real case is
based on a transactional database.
In this case, it is necessary to consider that backup windows are used for different operations; for writing data
to the Data Domain system, the window is 12 hours.
In this scenario, we need to consider storage performance, channel between storage and proxy host, proxy
host, and channel between proxy host and Data Domain system as show below.
2014 EMC Proven Professional Knowledge Sharing 8
Requirements are an initial indication of prerequisites based on NetWorker® Performance and Tuning Guide
and from Oracle SGA requirements. Based on my experience, the values provided are good for Linux on Intel
platform. In any case, it gives the customer an indication of proxy host requirements.
CPU Requirements: (MB/s/100)*2+% DB overhead
SGA requirements: fixed size+ (n°Channel*(16+(4*device block size))
Parameters Value
Requirements Value
Size (TB) 80
Read from storage TB/h 6.666666667
Backup windows hours 12
Read From Storage MB/s 1942
%CPU DB overhead 15%
Required CPU 45
N° channel 64
Standard Target Write MB/s 1942
Device Block size GB 0.5
MB/s Single Channel 30
Retention (day) 30
SGA PARAMETERS:
Number of datafile 10000
Fixed size 4096
Channel/block size calculation 1152
MIN TOT. SGA REQUIRED 5248
Networker Index Size (GB) 0.07
Min IOPS ( large server ) 1900
Disk Latency max 25 ms
In this scenario, NetWorker (a virtual backup server in this example), RMAN, and script for disk operations are
used to support backup operations.
Storage needs to sustain 1650 MB/s for read, proxy host that can manage 3300MB/s transfer rate, and Data
Domain DD990 used in VTL with a minimum of 8 shelves needed.
On the proxy host, four 8GB Fiber Channel ports are needed for storage connection and four 8GB Fiber
Channel ports are needed for write to Data Domain. Each part of the architecture is critical in this scenario and
needs to be tuned and set up with very high level of attention to obtain required performance.
2014 EMC Proven Professional Knowledge Sharing 9
Restore considerations are more restrictive in this case because RTO is 2 days:
Parameters Value
Requirements Value
Size (TB) 80
Read from storage TB/h 1.666666667
Backup windows hours 48
Read From Storage MB/s 485
%CPU DB overhead 15%
Required CPU 11
N° channel 64
Standard Target Write MB/s 485
Device Block size GB 0.5
MB/s Single Channel 8 .
NDMP VNX backup using snapshot and Data Domain NDMP server with NetWorker
Considerations for NAS backup are a bit more complex and require a good NAS configuration in terms of
volumes and file system structure.
Parameters: 10 million files (1M for volume), 30 TB of data (10 Volumes of 3TB), NAS with a maximum of eight
backup/restore operations in parallel, mix of standard file systems and deduplicated, 10 GB connection
available for backup, backup windows; 24 hours on weekend, one full backup every month and incremental
backup on other days.
2014 EMC Proven Professional Knowledge Sharing 10
Parameters Value
Requirements Value
Size (TB) 30
MAX SIZE per Day 5.7
Backup windows hours 24
TB/h 0.24
Daily change rate 10%
Read/write MB/s 69
Number of stream 8
Full single stream MB/S 36
Device Block size GB 0.5
BACKUP SERVER
Retention (day) 30
Index size (GB) 17
Number of datafile 10000000
CPU 8GHz
Number of Volume 10
RAM (GB) 32
max latency max 25ms
IOPS 2800.00
In this scenario Data Domain is used as a NDMP server and is configured on NetWorker server as storage
node. Production NAS will direct data to Data Domain system via 10GB LAN and Index to Backup server.
You need to create a VTL on Data Domain with a minimum of 8 drives and set up 8 as max parallelism on NAS
client defined on NetWorker.
It is important to consider NAS configuration and free space:
It is strongly recommended to back up data from snapshot in this scenario. This requires 10-15% of
free space in NAS volume in order to manage snapshot
If you have deduplicated volume and you write to a Data Domain system, it is recommended to send
data not deduplicated to backup
If you have a deduplicated file system, VBB is not supported and you must use DUMP to manage
backup operations
2014 EMC Proven Professional Knowledge Sharing 11
In this case we will manage this schedule:
In order to have better performance, you need to consider that if you have large volume with millions of files it
is better to divide the structure into different folders in order to manage full backups within the backup windows.
In this case, IOPS / CPU / RAM on backup server are very important because NDMP backup uses more
resources than a standard backup (120 IOPS for each NDMP session).
Third phase: Implementation and Test
Inside this phase, it is very important to plan some tests in order to verify infrastructure performance and tune
parameters such as:
Number of sessions for appliance and standard database
DAY FS1 FS2 FS3 FS4 FS5 FS6 FS7 FS8 FS9 FS10
1° Mon Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
1° Tue Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
1° Wed Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
1° Tue Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
1° Fri FULL Incr Incr Incr Incr Incr Incr Incr Incr Incr
1° Sat Incr FULL Incr Incr Incr Incr Incr Incr Incr Incr
1° Sun Incr Incr FULL Incr Incr Incr Incr Incr Incr Incr
2° Mon Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
2° Tue Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
2° Wed Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
2° Tue Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
2° Fri Incr Incr Incr FULL Incr Incr Incr Incr Incr Incr
2° Sat Incr Incr Incr Incr FULL Incr Incr Incr Incr Incr
2° Sun Incr Incr Incr Incr Incr FULL Incr Incr Incr Incr
3° Mon Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
3° Tue Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
3° Wed Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
3° Tue Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
3° Fri Incr Incr Incr Incr Incr Incr FULL Incr Incr Incr
3° Sat Incr Incr Incr Incr Incr Incr Incr FULL Incr Incr
3° Sun Incr Incr Incr Incr Incr Incr Incr Incr FULL Incr
4° Mon Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
4° Tue Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
4° Wed Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
4° Tue Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
4° Fri Incr Incr Incr Incr Incr Incr Incr Incr Incr FULL
4° Sat Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
4° Sun Incr Incr Incr Incr Incr Incr Incr Incr Incr Incr
2014 EMC Proven Professional Knowledge Sharing 12
Type of backup for DUMP or VBB for NDMP backup
Device block size
Tune host kernel parameters
Oracle and DB parameters, etc.
Tune storage configuration, FC/LAN connections, etc.
It is important to plan enough tests in order to have enough time to manage tuning phases and have complete
architecture tuning.
This is the most difficult phase, requiring a very high level of attention and time in order to avoid customer
dissatisfaction with the backup configuration.
Fourth phase: Monitoring and Change Management
Monitoring the daily situation and changes inside the environment is crucial.
A basic trace can be managed by monitoring Data Domain with EMC Data Protection Advisor in order to track
daily backup ingestion, deduplication ratio, and LAN/SAN performance. Additionally, it is possible to add FC
and LAN switches with detail on port usage and monitoring. Finally, adding a backup application and operating
system monitoring provides a complete scenario of all infrastructure.
It is strongly suggested to add notifications such as backup failure, backup windows not respected, growth
size, etc.
Conclusion
Though this is all very hard work, it is what is required to achieve cost-effective and efficient backup!
2014 EMC Proven Professional Knowledge Sharing 13
Glossary
RTO = recovery time objective
RPO = recovery point objective
SLA = service level agreement
DPA = data protection advisor
DD = Data Domain
NDMP = Network Data Management Protocol
NAS = Network Attached Storage
EMC believes the information in this publication is accurate as of its publication date. The information is
subject to change without notice.
THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION MAKES NO
RESPRESENTATIONS OR WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS
PUBLICATION, AND SPECIFICALLY DISCLAIMS IMPLIED WARRANTIES OF MERCHANTABILITY OR
FITNESS FOR A PARTICULAR PURPOSE.
Use, copying, and distribution of any EMC software described in this publication requires an applicable
software license.