USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT...

20
WHITE PAPER USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD STORAGE ABSTRACT This white paper explains how to use Dell EMC ECS with Veritas Enterprise Vault December, 2017

Transcript of USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT...

Page 1: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

WHITE PAPER

USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD STORAGE

ABSTRACT

This white paper explains how to use Dell EMC ECS with Veritas Enterprise Vault

December, 2017

Page 2: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

2

The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with

respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a

particular purpose.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

EMC2, EMC, the EMC logo, are registered trademarks or trademarks of EMC Corporation in the United States and other

countries. All other trademarks used herein are the property of their respective owners. © Copyright 2016 EMC Corporation. All

rights reserved. Published in the USA. October 2016 white paper part no H15309

EMC believes the information in this document is accurate as of its publication date. The information is subject to change without

notice.

EMC is now part of the Dell group of companies.

Part Number H15309

Page 3: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

3

TABLE OF CONTENTS

INTRODUCTION ................................................................................................................... 5

Dell EMC ECS Streamer Overview .............................................................................................. 5

Dell EMC ECS - Multi-protocol, API-accessible Storage .............................................................. 6

Simple Multi-Tenancy ................................................................................................................... 6

Multi-site, Active-Active Architecture and Access ......................................................................... 6

Audience ...................................................................................................................................... 6

Terminology.................................................................................................................................. 6

ECS CONFIGURATION DETAILS ....................................................................................... 7

Namespace – compliance ............................................................................................................ 7

Bucket Default Retention Setting .................................................................................................. 7

ECS STREAMER INSTALLATION ...................................................................................... 8

Download the Dell EMC ECS Streamer Driver ............................................................................. 8

ECS STREAMER CONFIGURATION .................................................................................. 9

Add Dell EMC ECS as a Storage Option in Enterprise Vault ....................................................... 9

Configuring a Dell EMC ECS Streamer based Enterprise Vault Store Partition ........................... 9

DETAILS OF THE ECS STREAMER DRIVER .................................................................. 13

Network Ports ............................................................................................................................. 13

Retention .................................................................................................................................... 13

Retention Policies ....................................................................................................................... 13

Capacity Information .................................................................................................................. 13

WORM and NON-WORM Support ............................................................................................. 14

Safe Copy Check ....................................................................................................................... 15

SUPPORTED ENVIRONMENTS ........................................................................................ 16

Veritas Enterprise Vault .............................................................................................................. 16

Dell EMC Elastic Cloud Storage ................................................................................................. 16

IP LOAD BALANCER REQUIREMENT ............................................................................. 17

Overview .................................................................................................................................... 17

HAProxy Setup ........................................................................................................................... 17

PERFORMANCE INFORMATION ...................................................................................... 18

Testing Configuration ................................................................................................................. 18

Results ....................................................................................................................................... 18

APPENDIX A – EXAMPLE HAPROXY CONFIG FILE ...................................................... 19

Page 4: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

4

Page 5: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

5

INTRODUCTION

Dell EMC ECS Streamer Overview

Veritas has developed a Storage Streamer API for Enterprise Vault (EV) which archive storage vendors must integrate with to allow

Enterprise Vault to archive files/emails/etc. to their storage systems. Dell EMC has developed the ECS Storage Streamer driver to

allow Veritas Enterprise Vault to archive to Dell EMC Elastic Cloud Storage (ECS). The Dell EMC ECS Streamer driver “translates”

the Streamer API calls to the ECS S3 API.

Customers can now create Vault Store partitions of type Dell EMC ECS Streamer within the Veritas Enterprise Vault VAC. These

Vault Store partitions are associated with S3 buckets on the ECS cluster where files archived to the Vault Store will reside.

DELL EMC ECS buckets reside within ECS namespaces that may be compliant or non-compliant namespaces. Both compliant

and non-compliant namespaces allow retention to be propagated from Enterprise Vault to the ECS cluster, however compliant

namespaces store archive data in tamper-resistant storage which meets strict SEC 17a-4(f) rules for electronic record-keeping.

Figure 1 shows an example of a Dell EMC ECS based storage solutions for Veritas Enterprise Vault.

Figure 1. Veritas Enterprise Vault support of Dell EMC Elastic Cloud Storage

The Veritas Enterprise Vault system has archive policies that archive files to a Vault Store in which an EMC ECS based partition

has been defined. The ECS Streamer driver uses the Dell EMC S3 API to store/access objects in the S3 bucket on the Dell EMC

ECS cluster.

Dell EMC ECS systems are clusters of 4 or more individual ECS nodes and currently in order to allow all nodes to be accessed via

the Dell EMC Streamer driver a Network IP Load Balancer should be deployed “between” the Veritas Enterprise Vault server and

the Dell EMC ECS cluster.

Page 6: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

6

Dell EMC ECS - Multi-protocol, API-accessible Storage

ECS is a massively scalable, software-defined object storage platform that enables any organisation to store, access and

manipulate unstructured data as objects. ECS provides support for industry standard APIs such as Amazon S3. In addition ECS

extends the Amazon S3 API with support for retention, byte range updates and atomic appends.

Simple Multi-Tenancy

ECS delivers a flat software architecture ideal for multi-tenant environments. Configuring, provisioning, creating a namespace and

managing a multitenant platform has never been easier. Key metrics and reporting are provided for capacity, object count, objects

created, objects deleted and inbound/outbound bandwidth. These activities are all visible via the ECS GUI and through the REST

API.

Multi-site, Active-Active Architecture and Access

ECS features a truly geo-efficient architecture that stores, distributes and protects data both locally and geographically. This

eliminates any single point of failure and provides seamless failover from site to site with no impact to the business. ECS

automatically maximises throughout, maintains high availability and data durability, and increases capacity and the reliability of

applications. In terms of geographic limitations – there are none – providing users with a single global namespace with anywhere

access to content

Today’s modern applications demand a different architecture. Managing both cost and accessibility as storage environments grow

and become more complex is one of the biggest challenges organisations face today. Developers are finding it easier to go to

public cloud alternatives putting data at risk and increasing management costs. ECS provides a cloud-scale storage architecture

that is specifically designed to support modern applications with unparalleled availability, protection, simplicity and scale.

Audience

This document is intended for Storage Administrators and/or Dell EMC Professional Services personnel who wish to learn how to

deploy and configure Dell EMC’s ECS object storage with Veritas Enterprise Vault.

Terminology

Abbreviation Description

ECS Elastic Cloud Storage

EV Veritas Enterprise Vault

S3 Simple Storage Service (API)

LB IP Network Load Balancer

VDC ECS Virtual Data Centre

Bucket A bucket is a logical unit of storage on an Dell EMC ECS system in which objects (including their

metadata) are stored

Table 1. Terminology

Page 7: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

7

ECS CONFIGURATION DETAILS

How the ECS is configured (e.g. number of ECS nodes, number of VDC’s, GEO protection and application location access) is

decided by the Solution Architect designing the ECS configuration and for the most part Veritas Enterprise Vault is unaware of the

design. However, the Solution Architect should take into account the compliance requirements of the Customers Enterprise Vault

solution.

Namespace – compliance

If the customer wishes to have an SEC compliant solution then the Namespace that the buckets that all Enterprise Vault partitions

will use should be configured as compliant

When creating the Namespace in the ECS Management GUI ensure the Compliance option is selected

Bucket Default Retention Setting

Create the bucket you are creating for the EV Vault Store partition in the compliant Namespace.

Ensure that Bucket Retention is set to a minimum of 1 second. This must be set or an error will occur when you attempt to create

the bucket.

Page 8: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

8

Administrators should take care when setting ECS bucket level retention. Enterprise Vault will expect to be able to delete expired

content and although EV does cleanly handle errors on deleting content that is still under ECS retention, it will post an error in the

Windows Event Log on each failure. Always set the bucket level retention to 1 second for Enterprise Vault Partitions.

ECS STREAMER INSTALLATION

Download the Dell EMC ECS Streamer Driver

Download the ECS Streamer driver ahead of the installation. The ECS Streamer installer can be found on http://support.emc.com

Once you have downloaded the Dell EMC ECS Streamer driver onto the Veritas Enterprise Vault server, double-click the

installation file and it will install itself.

Click Next to installer the ECS Streamer driver.

If the ECS Streamer driver is already installed on the server it will be re-installed by this process.

The ECS Streamer driver should be installed on all Enterprise Vault Servers.

If you are re-installing the ECS Streamer driver you may be required to reboot your Enterprise Vault server.

Page 9: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

9

ECS STREAMER CONFIGURATION

Add Dell EMC ECS as a Storage Option in Enterprise Vault

Refer to the procedure for this documented on Veritas’ support web site at

https://www.veritas.com/support/en_US/article.000114337

this is only required to be performed on one Enterprise Vault server instance.

Configuring a Dell EMC ECS Streamer based Enterprise Vault Store Partition

In the VAC, expand the Enterprise Vault Site and Vault Store Groups and Right-Click on the Vault Store you wish to create a new

Vault Store Partition for and click New->Partition then Next.

Choose an appropriate Name and Description and click Next>

Choose Dell EMC Elastic Cloud Storage from the Storage Type: drop down

Page 10: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

10

Then Next >

Enter the connectivity details for the ECS cluster to be connected to and select Test to check the configuration details.

Property Description

Host Enter the IP address or FDQN of the LB service. See HOST

PROPERTY DISCUSSION BELOW

Port Default 9020. Enter here the port number being used for data

connection to the ECS.

Bucket Name Enter the name of the ECS bucket to be used

Access key Enter the ECS access key (object user) to be used

Secret key Enter the access key’s secret key

Safe copy check 0 or 2 See the Safe Copy Check discussion below. Do not use

option 1 at present

Use proxy Set to 0 if no proxy is being used or port 8888 for use with a

proxy

Namespace Enter the name of the namespace in which the bucket named

above resides

Click the Test button to perform a check of the configuration settings. If you get an error recheck the settings as entered.

The namespace property is only used in the dtquery request (used when safe copy check is 2, see the section below). The

namespace property is irrelevant to the actual archiving process and it is not used in S3 object requests. The object user is globally

unique and belongs to a namespace, therefore the namespace is implicitly identified.

If the Test is successful then select NEXT >

Page 11: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

11

In the next GUI form, select the appropriate WORM setting for the partition. The ECS Streamer driver has been validated for NON-

WORM and well as WORM Enterprise Vault partitions, check or un-check this radio button appropriately.

Complete the following Forms as required.

HOST PROPERTIES

With version 1.0.10 of the Dell ECS Streamer driver some additions were made to the syntax of the Host property to allow the

administrator to set some internal settings of the Dell ECS Streamer driver. These new options are

- MAX_TRIES,

- DISABLE_WRITE_TEST,

- MAX_CONNS,

- DTHOST

- RDT

- SHOWCHUNKS

In order to use the options, the Host IP (or name) must come first and the other options can be specified using a semicolon

separator in <name>=<value> format e.g.

10.1.83.51;MAX_TRIES=2;MAX_CONNS=25;DISABLE_WRITE_TEST=1

MAX_TRIES

When any server failure response is received, the streamer will attempt a total of 5 tries by default before returned bad status to

Evault. There is a standard exponential back-off algorithm between retries. This value can be overridden by the MAX_TRIES option

which must be greater than 0 and less than 5.

MAX_CONNS

This controls the maximum number of simultaneous safe copy checks that can occur at one time. The default is 65 as of rev 1.0.14

of the Streamer Driver, customers should not need to change this number however is may be set between 2 and 100

DISABLE_WRITE_TEST

Page 12: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

12

When creating a Vault Store Partition for the first time when there is a Temporary Site Outage (TSO), the tests performed when the

user clicks on the Test Button will fail because of the TSO. We recommend that you do not create Vault Store Partitions whilst a

TSO is in effect but if you must then setting DISABLE_WRITE_TEST=1 will disable the test.

DTHOST=<hostname> or <IP>

This allows the user to specify the endpoint for the DTQUERY geo replication checking to a different host than the one specified for

the Vault Store partition. This should only be changed under the instruction of Dell EMC support personnel.

RDT=1

When set this will set "&random=<someUniqueGuid>" to the end of the DTQUERY URL to make sure the URL response is never

cached. This <someUniqueGuid> is never reused to ensure the request URL is unique. Although additional flags are now used in

the request constructor to prevent caching, but this variation is still available for use.

SHOWCHUNKS=1

This will include the chunkIDs and chunk ranges for files written in the DTRACE log, only if the files was archived to the bucket

owning site. Do not enable this unless told to by Dell EMC support personnel. If enabled there will be an effect on performance.

Page 13: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

13

DETAILS OF THE ECS STREAMER DRIVER

Network Ports

The following network ports are used to communicate between the Enterprise Vault Server and ECS.

Port No Port

9020 or 9021 Default http connection or default https port. TCP to the ECS.

9101 Required to perform the Enterprise Vault “safe copy” check.

TCP to the ECS

If the customer uses an IP Load Balancer they may choose to use different HTTP or HTTPS ports from the Enterprise Vault server

to the IP Load Balancer. Ports used should be configured in any IP Load Balancer for TCP to the ECS nodes.

At present the port # 9101 cannot be mapped to another port by any Load Balancer being used.

Retention

Enterprise Vault allows users to associate retention policies to archived objects/ files/emails etc. The ECS Streamer driver provides

Enterprise Vault with the capability of associating these retention policies with the objects stored on the ECS. The ECS object store

will enforce the objects retention and will not allow objects to be deleted until its retention has expired.

Objects archived to ECS cannot have their retention shortened or extended by Enterprise Vault.

Enterprise Vault will not any apply retention periods to objects stored in a Dell ECS Streamer Partition that is configured as NON-

WORM.

Retention Policies

The Dell EMC ECS Streamer driver does not support Dell EMC ECS Retention Policies.

Capacity Information

The ECS Streamer driver does not at this stage implemented the Enterprise Vault Storage Streamer API Capacity method.

Dell EMC ECS is a scale-out architecture and each EV Vault Store Partition has available to it all the capacity of the entire ECS

cluster. ECS also provides the end-user capacity consumption metrics which will display current stats e.g.

Page 14: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

14

WORM and NON-WORM Support

Veritas defines WORM and NON-WORM as follows

WORM – Enterprise Vault will set the retention period defined in an Enterprise Vault Retention Category to the objects written to an

ECS Streamer Partition.

NON-WORM – Enterprise Vault will not set the retention period defined in an Enterprise Vault Retention Category to the objects

written to an ECS Streamer Partition.

When a NON-WORM partition is created this allows the following

- Even if a Vault Store is using a Retention Category that has a retention period, users using the Enterprise Vault Search

Browser can select a document(s) for deletion and as there is no-retention on the object(s) on the ECS, the ECS will allow

the deletion to succeed. Administrators for example use this to set documents to be Retained Forever in Enterprise Vault

so that the Enterprise Vault Expiration Task will not be run on the Partition but still allow users to delete individual

documents.

- Enterprise Vault Administrators can change the retention period of a Retention Category and can shorten it. Subsequent

runs of the Enterprise Vault Expiration Task will act on the new shortened retention period and will attempt to delete the

content which will succeed as there is no retention on the object on the ECS.

Dell EMC ECS 3.x is supported as a WORM or NON-WORM enabled Enterprise Vault archive device for EV11 and EV12. When

any Veritas Vault Store Partition is created and uses the ECS Streamer driver, check the radio box below to enable WORM

storage, un-check to use NON-WORM.

This settings cannot be changed once the Partition has been created.

Page 15: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

15

Safe Copy Check

Enterprise Vault has a “Safe Copy Check” feature which can be enabled by customers at the Vault Store level. This check is to

ensure that content archived by Enterprise Vault is not replaced in its original location by an HTM link until it has been backed up to

a second archive device via the devices replication functionality.

This approach is commonly used by customers who use archive devices with replication functionality that have an “eventual”

consistency model. With Centera, for example, it can be several minutes before replication has created the second copy of the

archive content and the “Safe Copy Check” is commonly used.

ECS however implements a strong consistency model and the latencies that exist in the Centera replication queues do not exist in

ECS. Customer can be assured that an object that is written to one Virtual Data Centre in a Replication Group will also be

available to all other Virtual Data Centres in that Replication Group. The more advanced multi-site architecture implemented by

ECS therefore eliminates the need for the Enterprise Vault “Safe Copy” verification.

The concept of a “safe” copy means that an archived object is geo replicated on a secondary VDC. However, within a VDC,

archived files are always stored in a resilient manner (i.e. Erasure coding and a copy). Please read the ECS storage documentation

for more information.

Note at present the ECS S3 API doesn’t support safe copy check however the ECS Streamer Driver has been written to

accommodate this in future revisions of the ECS S3 API. Customers who wish to perform this check will need to specify option 2 in

the table following. This will cause the ECS Streamer Driver to use dtQuery to perform the check. It is anticipated that in the future

when the ECS S3 API supports the safe copy check it will be more efficient and will use the same port as the EMC S3 API.

If safe copy check is set to 2 the dtQuery request to validate a safe archive copy (i.e. Geo replication) will be sent. The test will only

check that a valid response is received.

The ECS streamer does not control the message in the failure alert window. ECS streamer will attempt to log a specific reason.

Should a customer wish to use the Enterprise Vault safe copy check they can enable this as before on the Vault Store and then

choose option 2. This is set on the Partition setup Management GUI

The following table shows the 3 possible values for the “Safe Copy Check” parameter.

Safe copy check value Has the file been “backed” up

0 Always return “true”. The archived object is always considered

safe.

1 Reserved for future use. Do not use at present

2 Uses the ECS dtQuery API check to determine status.

This will require that port 9101 is open in any firewall between

the Enterprise Vault server and the ECS.

Notes

1. When using the dtQuery option, port 9101 must be open in any firewall between the Enterprise Vault Server and ECS

cluster.

2. There is a restriction that port 9101 is fixed and so any LB must be set up with this taken into consideration.

3. Ensure that ECS Streamer driver 1.0.10 or above is installed.

4. ECS buckets used by Enterprise Vault should not be file system enabled.

Page 16: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

16

SUPPORTED ENVIRONMENTS

Veritas Enterprise Vault

Veritas as a detailed support matrix for all Enterprise Vault including 3rd party Software and Hardware components at

https://www.veritas.com/support/en_US/article.TECH38537

Dell EMC Elastic Cloud Storage

Veritas EV10, EV11 and EV12 support Dell EMC Elastic Cloud Storage from revision 2.2 upwards.

STREAMER RELEASE INFORMATION

Streamer Release # Description

1.0.14 Default MAX_CONNS changed to 65

Added optional SHOWCHUNKS confuration parameter to assist debugging

Added optional RDT parameter to ensure requests are not cached

Added option DTHOSTS parameter to assist with debugging

Issue caused by caching in the Windows HTTP library resolved.

1.0.12 Modified the Test() function. Since Evault itself also calls the Test functionality, it is

not reasonable to fail the test if the dtquery call fails. This information is logged but

does not fail the connectivity test. This portion of the test is only done when

safecopy==2. Unfortunately dtquery has problems during TSO. So any call to the Test

functionality by Evault when safecopy==2 during TSO, would cause archiving to stop.

During archiving any failure in dtquery simply sets the safe copy boolean to false and

archiving continues.

1.0.11 Added a fix for a potential Read() problem where Windows may requires a larger

buffer

Page 17: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

17

IP LOAD BALANCER REQUIREMENT

Overview

The Dell EMC ECS Streamer driver uses the Dell EMC S3 API to communicate with the backend ECS storage cluster. At present

the configuration of the Dell EMC ECS Streamer driver takes only one IP address so in order for the driver to utilise all the nodes in

the ECS cluster an IP load balancer is required.

Dell EMC has no preference in the IP load balancer used, it may be hardware based or software based.

The Dell EMC ETD-SA group used the open source software load balancer HAProxy in their labs to perform the Veritas Enterprise

Vault self-certification testing.

Future revisions of the ECS Streamer driver may address the requirement to have an external IP load balancer by implementing

load balancing in the driver itself.

HAProxy Setup

HAProxy is an open source IP Load Balancer which is available free. It can be installed on a minimal Linux host (Virtual or

Hardware based) with as little of 4GB of RAM, single CPU, 50GB Disk and a single NIC.

For example with CentOS, once you have installed the base CentOS distribution (available at https://www.centos.org/download/)

you can install HAProxy with the following commands

# yum –y install haproxy

Next edit the file /etc/haproxy/haproxy.cfg, Appendix A contains an example of the HAPRoxy used in the ETD SA lab for testing.

Once you have configured the haproxy.cfg file you can start haproxy with the command

# service haproxy start

# chkconfig haproxy on

This service start is nor persistent over a reboot, so you should include these commands in the systems init file.

You can obtain service details by running the command

# service haproxy status

If you see errors indicating that haproxy could not bind to its ports use this command and restart haproxy

# setenforce 0

# service haproxy restart

# service haproxy status

Note; This is included purely as an illustration of how HAProxy was used in the running of the Veritas Enterprise Vault Self

Certification tests in our lab. Customer implementations should consider issues like single point of failure etc in designing and

implementing LB configuration for Production use.

Page 18: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

18

PERFORMANCE INFORMATION

At present no performance testing has been performed in order to provide details that can be used to assist architects sizing and

designing Enterprise Vault solutions. However, as part of the Veritas Enterprise Vault Self Certification testing performed by Dell

EMC, some basic throughput measurements were taken.

Testing Configuration

The following system configurations were used for testing with Enterprise Vault revisions tested.

EV10 EV11 EV12

SQL Server DB 8 x vCPU, 32Gb RAM

3 x 100GB SAN Disk

10GbE Nic

Windows 2008R2 STD

SQL Server 2008R2

8 x vCPU, 32Gb RAM

3 x 100GB SAN Disk

10GbE Nic

Windows 2008R2 STD

SQL Server 2008R2

8 x vCPU, 32Gb RAM

3 x 100GB SAN Disk

10GbE Nic

Windows 2012R2 STD

SQL Server 2014

EVServer 1 8 x vCPU, 32Gb RAM

3 x 100GB SAN Disk

10GbE Nic

Windows 2008R2 STD

8 x vCPU, 32Gb RAM

3 x 100GB SAN Disk

10GbE Nic

Windows 2008R2 STD

8 x vCPU, 32Gb RAM

3 x 100GB SAN Disk

10GbE Nic

Windows 2012R2 STD

EVServer 2 8 x vCPU, 32Gb RAM

3 x 100GB SAN Disk

10GbE Nic

Windows 2008R2 STD

8 x vCPU, 32Gb RAM

3 x 100GB SAN Disk

10GbE Nic

Windows 2008R2 STD

8 x vCPU, 32Gb RAM

3 x 100GB SAN Disk

10GbE Nic

Windows 2012R2 STD

Results

EV10 EV11 EV12

Ingest – Files/Hour 117630 175730 139134

Ingest – Mbytes/hour 6420 6760 6820

Read Test – Files/Hour 545270 642551 639047

Expiration Test – Files/hr 16630 21660 14389

File sizes used in these tests was on average 64Kbytes.

Page 19: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

19

APPENDIX A – EXAMPLE HAPROXY CONFIG FILE

The settings specific to the ECS in our test lab are highlighted

#---------------------------------------------------------------------

# Global settings

#---------------------------------------------------------------------

global

log 127.0.0.1 local2

chroot /var/lib/haproxy

pidfile /var/run/haproxy.pid

maxconn 4000

user haproxy

group haproxy

daemon

# turn on stats unix socket

stats socket /var/lib/haproxy/stats

# Default SSL material locations

ca-base /etc/ssl/certs

crt-base /etc/ssl/private

# Default ciphers to use on SSL-enabled listening sockets.

# For more information, see ciphers(1SSL).

ssl-default-bind-ciphers kEECDH+aRSA+AES:kRSA+AES:+AES256:!kEDH:!LOW:!EXP:!MD5:!aNULL:!eNULL

ssl-default-bind-options no-sslv3

#---------------------------------------------------------------------

# common defaults that all the 'listen' and 'backend' sections will

# use if not designated in their block

#---------------------------------------------------------------------

defaults

mode http

log global

option httplog

option dontlognull

option http-server-close

option forwardfor except 127.0.0.0/8

option redispatch

retries 3

timeout http-request 10s

timeout queue 1m

timeout connect 5000

timeout client 50000

timeout server 50000

timeout http-keep-alive 10s

timeout check 10s

maxconn 3000

#---------------------------------------------------------------------

# main frontend which proxys to the backends

#---------------------------------------------------------------------

frontend ecs05_s3_http_in 10.246.27.200:9020

default_backend ecs05_s3_http

#---------------------------------------------------------------------

# round robin balancing between the various backends

#---------------------------------------------------------------------

backend ecs05_s3_http

balance leastconn

# Set the X-Forwarded-For header with source IP

option forwardfor

# Use the s3 Ping ViPR extension to make sure node is alive

option httpchk GET /?ping HTTP/1.1\r\nHost:\ haproxy\r\nX-Emc-Namespace:\ foo

# Workaround for AWS JS SDK and Storage-2177

reqrep ^(X-Amz-Date:)(.*)$ x-amz-date:\2

# List of data nodes. S3 runs on port 9020.

server ecs05-01-s3-http 10.246.22.167:9020 check

server ecs05-02-s3-http 10.246.22.168:9020 check

server ecs05-03-s3-http 10.246.22.169:9020 check

server ecs05-04-s3-http 10.246.22.170:9020 check

Page 20: USING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC · PDF fileUSING VERITAS ENTERPRISE VAULT WITH DELL EMC ELASTIC CLOUD ... the Dell EMC Streamer driver a Network IP Load Balancer

20

listen stats :1936

mode http

log global

maxconn 10

clitimeout 100s

srvtimeout 100s

contimeout 100s

timeout queue 100s

stats enable

stats hide-version

stats refresh 30s

stats show-node

stats auth admin:password

stats uri /haproxy?stats