DELL EMC TECHNICAL VALIDATION OF FILECLOUD WITH ECS 3 · Dell EMC Confidential: Internal and...

TECHNICAL GUIDE – ISV PARTNER VALIDATION

DELL EMC TECHNICAL VALIDATION OF FILECLOUD WITH ECS 3.1

ASD Solution Architecture Compatibility and Performance

Testing Validation Brief

ABSTRACT

This document captures details on the technologies, results, and environment used to

perform various functionality tests to demonstrate compatibility between Dell EMC and

the FileCloud technologies described herein.

November 2017

Revision 1.0

Dell EMC Confidential: Internal and Authorized Partner Use Only 2

The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with respect

to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a particular purpose.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

EMC2, EMC, the EMC logo are registered trademarks or trademarks of EMC Corporation in the United States and other countries. All

other trademarks used herein are the property of their respective owners. © Copyright 2017 EMC Corporation. All rights reserved.

Published in the USA part number; i.e., HXXXXX>

EMC believes the information in this document is accurate as of its publication date. The information is subject to change without

notice.

EMC is now part of the Dell group of companies.


TABLE OF CONTENTS

INTRODUCTION ........................................................................................................................5

Technology Overview ........................................................................................................................ 5

Purpose ............................................................................................................................................. 5

Relevant Partner Features ................................................................................................................ 5

Relevant DELL EMC Features .......................................................................................................... 5

VALIDATION SCOPE ................................................................................................................5

Partner Technologies ........................................................................................................................ 5

DELL EMC Technologies .................................................................................................................. 6

Data Access Methods ....................................................................................................................... 6

TESTING ENVIRONMENT ........................................................................................................6

Architecture Illustration ...................................................................................................................... 6

Resources ......................................................................................................................................... 6

Supporting Information and Considerations ...................................................................................... 7

STEP-BY-STEP CONFIGURATION STEPS .............................................................................7

Optional Load Balancer ..................................................................................................................... 7

Data Access Ports ............................................................................................................................. 7

ECS Configuration ............................................................................................................................. 7

FileCloud Managed Storage .............................................................................................................. 7

Amazon S3 ................................................................................................................................................ 7

FILECLOUD FUNCTIONALITY .................................................................................................9

Web Interface .................................................................................................................................... 9

Automatically Syncing Files using the Sync Client .......................................................................... 11

Cloud Drive...................................................................................................................................... 12

VALIDATION TESTS .............................................................................................................. 13

Compatibility Testing ....................................................................................................................... 13

S3 Compatibility Testing .................................................................................................................. 13

PERFORMANCE TESTING .................................................................................................... 15

Object Upload Throughput ....................................................................................................................... 15

RECOMMENDATIONS ........................................................................................................... 16

General Notes ................................................................................................................................. 16

Configuration and Tuning ................................................................................................................ 16


CONCLUSION ........................................................................................................................ 16

Synopsis .......................................................................................................................................... 16

Testing Engineers ........................................................................................................................... 17

Testing Acceptance ......................................................................................................................... 17

REFERENCES ........................................................................................................................ 17

DOCUMENT VERSION HISTORY ......................................................................................... 17

APPENDIX .............................................................................................................................. 17

5

INTRODUCTION

TECHNOLOGY OVERVIEW

FileCloud is a self-hosted Enterprise File Access, Sharing and Sync solution that can be deployed on-premise (or) on cloud, using

storage system of customer's choice. FileCloud allows complete control and unmatched security and access to your data using a

variety of clients regardless of your location. FileCloud also provides mobile access clients for iOS, Android and Windows Phone and

Sync clients for Windows, Mac and Linux OS. You also have the option of mounting your FileCloud data as a Windows drive using our

custom client or as Mac/Linux Drive using WebDAV clients.

Dell EMC Elastic Cloud Storage (ECS) is a massively scalable enterprise-grade private cloud solution that provides extremely high

levels of storage efficiency, resiliency and simplicity. All data is automatically balanced which optimizes capacity and performance and

allows for performance to scale with capacity growth. ECS enables combinations of data type protocols, and provides access to object

services via several APIs. Of these, the S3 and OpenStack Swift REST APIs are supported by FileCloud.

PURPOSE

The intent of this document is to capture details pertaining to the technologies and environment used to confirm compatibility between

the Partner technologies and Dell EMC technologies described below.

RELEVANT PARTNER FEATURES

Table 1. List of Partner technology features relevant to the validation consideration

DESCRIPTION DETAIL

S3 API Integration FileCloud lists Amazon S3 as an available Cloud Storage integration type.

RELEVANT DELL EMC FEATURES

Table 2. List of Dell EMC technology features relevant to the validation consideration

DESCRIPTION DETAIL

S3 Protocol Support Arguably one of the more popular cloud storage protocols, the ECS implementation supports

additional capabilities such as Byte Range Updates and Rich ACLS

VALIDATION SCOPE

PARTNER TECHNOLOGIES

Table 3. List of Partner technologies used in the validation process

TECHNOLOGY - MODEL TECHNOLOGY VERSION TECHNOLOGY INSTANCE (PHYSICAL/VIRTUAL)

FileCloud Enterprise File Sharing and Sync Solution

15.0.0.35818 Virtual

FileCloud Sync Client 15.90.0.35620 Virtual

FileCloud Drive for Windows 15.90.0.35620 Virtual


DELL EMC TECHNOLOGIES

Table 4. List of Dell EMC technologies used in the validation process

TECHNOLOGY - MODEL TECHNOLOGY VERSION TECHNOLOGY INSTANCE (PHYSICAL/VIRTUAL)

ECS Multiple Protocol Support ECS 3.1 Physical

DATA ACCESS METHODS

Table 5. List of data access methods and technologies used in the validation process

DATA ACCESS METHOD DATA ACCESS STRUCTURE DETAILS

S3 API Used the native S3 API present in both FileCloud and Dell EMC ECS technologies

TESTING ENVIRONMENT

ARCHITECTURE ILLUSTRATION

All testing appliances were contained within a single lab environment, with the exception of the corporate DNS.. The FileCloud Sync

and Drive agents were installed on Virtual Windows machines. The ECS appliance was a physical system (8 Node U2000). (See

Figure 1 below)

Figure 1. Illustration of logical layout of Partner and Dell EMC technologies used for validation

FileCloud Server

ECS U2000FileCloud Sync and Drive

Client

Load Balancer

DNS Services & Active Directory

RESOURCES

Table 6. List of resources and technologies used to support the validation process

DESCRIPTION VERSION DETAILS & SPECIFICATIONS

FileCloud Server 15.0.0.35818 Windows 2012 R2 (64 Bit) – Virtual Machine 8 Cores, 16GB Memory, 128GB HDD, VMXNET3 Network Adapter

FileCloud Sync and Drive Client 15.90.0.35620 Windows 10 (64 Bit) – Virtual Machine 4 Cores, 8GB Memory, 500GB HDD, VMXNET3 Network Adapter

HAProxy Load Balancer V1.6.3 CentOS7 (64 Bit) – Virtual Machine 16 Cores, 32GB Memory, 126GB HDD, VMXNET3 Network Adapter

ECS 3.1 8 Node U2000


SUPPORTING INFORMATION AND CONSIDERATIONS

The Table below (Table 7) contains pertinent information relating to supporting the testing environment.

Table 7. List of supporting information and considerations relevant to the validation process

DESCRIPTION DETAIL

Cyberduck Amazon S3 and OpenStack Swift browser. Used to verify that data was written and deleted

from Cloud Storage.

STEP-BY-STEP CONFIGURATION STEPS

The sections that follow provide the necessary steps when connecting ECS with FileCloud as the Cloud Storage target. Refer to the

FileCloud documentation in the reference section for up-to-date information on Setting up FileCloud Managed Storage.

OPTIONAL LOAD BALANCER

The Amazon S3 and OpenStack Swift APIs expect a single ECS Hostname or IP Address. They do not support DNS round robin or

multiple addresses. Best practice for distribution of REST traffic among the ECS nodes requires a 3rd party load balancer. The following

resource describes one approach to configuring a load balancer with ECS: https://community.emc.com/docs/DOC-46375.

The ECS Object Services communicate on different ports depending on the protocol and Data Access API.

DATA ACCESS PORTS

Table 8. List of ECS S3 Data Access Ports

DATA ACCESS METHOD HTTP HTTPS

S3 9020 9021

ECS CONFIGURATION

ECS requires a storage pool, virtual data center, replication group, namespace and an object user with an S3 and Swift password.

Refer to the ECS Administrators guide in the reference section for up-to-date information on creating object users.

FILECLOUD MANAGED STORAGE

FileCloud supports storing user data into Cloud Storage using both the Amazon S3 and OpenStack Swift APIs. This validation focused

on the Amazon s3 implementation.

Amazon S3 Amazon S3 must be enabled in FileCloud as the backend server In order to use ECS Amazon S3 for Cloud Storage.

Note: In Windows WWWROOT is typically c:\xampp\htdocs and in Linux is its /var/www/

Step 1:

Edit the file "WWWROOT/config/cloudconfig.php" and change the line

define("TONIDOCLOUD_STORAGE_IMPLEMENTATION", "local");

to read as

define("TONIDOCLOUD_STORAGE_IMPLEMENTATION", "amazons3");

Step 2:

Rename file "WWWROOT/config/amazons3storageconfig-sample.php" to "WWWROOT/config/amazons3storageconfig.php"

Nothing needs to be added or edited in amazons3storageconfig.php

https://www.getfilecloud.com/supportdocs/display/cloud/Setting+up+FileCloud+Managed+S3+Storage

https://community.emc.com/docs/DOC-46375


1. Login to the FileCloud web-based User Interface and navigate to Settings > Storage

Figure 2. ECS S3 Storage Configuration 2. Click ‘Save S3 Settings’

S3 Key: The ECS Object User Name S3 Secret: The ECS Object user S3 Password S3 Bucket Name: Specify the bucket name to store data in. (The bucket will be created if it does not exist)

This field is optional, FileCloud will auto-generate a bucket if left blank. S3 Endpoint URL: The ECS endpoint URL. (The above example is using the S3 alias of the Load Balancer).

Connectivity was also testing using the direct URL to an ECS node to verify that FileCloud supports non-standard ports.


FILECLOUD FUNCTIONALITY

Web Interface

A user can access their files from any browser using the Web Interface. This allows users to upload, download, delete and share files.

Figure 3. FileCloud Browser Login Screen

Upload files

Files can be uploaded from the Web Interface by double clicking the ‘My Files’ folder

As you can see by looking at the bucket contents from Cyberduck that it is currently empty.

Click the ‘Upload’ icon and select the file(s) you want to upload

The file is uploaded successfully and Cyberduck reflects the data has been uploaded to ECS


Delete Files

Select the file, click the ‘More’ dropdown and select ‘Delete’ to delete the file

The Recycle Bin option is disabled so the file is immediately removed from ECS

The upload and delete operation is also reflected in the activity log on the User Dashboard


Automatically Syncing Files using the Sync Client

The FileCloud Sync Client will automatically sync selective files/folders, network folders or backup folders to Cloud Storage. The

following example configures a folder to be backed up to ECS. Note that the Endpoint Backup Settings ‘Allow users to Backup’ must be

enabled in order to configure backups.

Open the Sync client and select the ‘Settings’ tab. Click the ‘Edit’ button next to the Backup option

Then click ‘Add Backup Folder’ and select the folder to backup then click ‘Add Folder’.

There’s also the option to schedule the folder for backup

Click ‘Run Now’ to start the backup. The Sync client will update the backup progress and pop-up a system message to inform the user

that the backup is complete.


The Sync client shows the size and number of files that were backed up which matches the size and files we backed up.

The data in the ECS bucket matches the size and number of files as well.

Cloud Drive

FileCloud Drive is a client application available for Windows and Mac users. It allows you to access your FileCloud server files like you normally access any other file on a Widows or Mac.


VALIDATION TESTS

COMPATIBILITY TESTING

S3 COMPATIBILITY TESTING

The following tests attempt to validate the FileCloud Platform’s S3 compatibility when integrated with ECS as the Cloud Storage target.

Table 9. S3 Storage Functionality

TEST#

1.

NAME/GOAL/PASS CRITERIA/PLAN/ RESULTS OBSERVATIONS PASS

FAIL

DATE

COMPLETE

1.1 Test Name

S3 Storage Node Setup Test Goal

Ensure that S3 Storage can be setup with ECS Pass Criteria

Verify that S3 Storage can be setup and configured Test Plan

Enable S3 storage in the FileCloud backend

Configure the S3 Storage settings and save the

configuration.

Ensure the settings are verified successfully, a

confirmation message will be displayed

Reference the configuration steps in the Step-by-Step

Configuration section.

Test Results

ECS S3 Storage was successfully configured

Passed 10/09/2017

1.2 Test Name

Upload files from the Web Interface Test Goal

Verify that files can be uploaded to ECS successfully. Pass Criteria

Files are uploaded to ECS successfully Test Plan

Login to the FileCloud Web Interface as a user Upload files to the My Files folder Test Results

Files were uploaded successfully

Passed 10/09/2017


1.3 Test Name

Upload files using the Cloud drive Test Goal

Verify that objects can be uploaded to ECS successfully. Pass Criteria

Files are uploaded to ECS successfully Test Plan

Copy files to the Cloud drive Test Results


Passed 10/09/2017

1.4 Test Name

Download files from the Web Interface Test Goal

Verify that objects can be downloaded from ECS successfully. Pass Criteria

Files are downloaded from ECS successfully Test Plan

Login to the FileCloud Web Interface as a user Upload files to the My Files folder Test Results


Passed 10/09/2017

1.5 Test Name

Backup files using the Sync Client Test Goal

Verify that objects can be backed up to ECS successfully. Pass Criteria

Files are backed up to ECS successfully Test Plan

Install the FileCloud Sync Application Test Results

Passed 11/06/2017

1.6 Test Name

Delete files from the Cloud drive Test Goal

Verify that objects can be deleted from ECS successfully. Pass Criteria

Files are successfully deleted from ECS Test Plan

Delete one or more files from the mapped cloud drive Test Results

Files are deleted from ECS successfully. Note that FileCloud has the notion of a recycle bin so files will not be immediately purged when this option is enabled.

Passed 10/09/2017


PERFORMANCE TESTING

Object Upload Throughput

Backup throughput tests were conducted using two data sets with different characteristics.

Table 10. Data Service Scaling Validation

TEST#

2.

NAME/GOAL/PASS CRITERIA/PLAN/ RESULTS OBSERVATIONS PASS

FAIL

DATE

COMPLETE

2.1 Test Name

Backup a folder from the FileCloud Sync client to ECS. Test Goal

Measure the throughput while backing up data sets to ECS using the Sync Client Pass Criteria

The backup is successful Test Plan

Add a backup folder to the Sync Client and perform an immediate backup

Test Results

The below results show the throughput for a single client. A

larger payload looks to exhibit better throughput than one with

smaller files. Note that FileCloud utilizes MPU and will start

uploading parts once ~5MB of data is received.

Set Size # Files File Size Range File Type(s)

1 48.8 GB 28 100KB - 8GB Mixed

2 1.56 GB 130 1KB – 80KB Plain Text

Set Size # Files Throughput File Type(s)

1 48.8 GB 28 43.19 MB/s Mixed

2 1.56 GB 130 38.05 MB/s Plain Text

Passwd 11/7/2017


RECOMMENDATIONS

GENERAL NOTES

CONFIGURATION AND TUNING

Table 12. Configuration and tuning recommendations relevant to the validation process

DESCRIPTION DETAIL

S3 API Configuration Details The Amazon S3 API was straightforward to set up. The configuration settings accepted either the IP address or hostname of the load balancer and also accepted a non-standard port as part of the Endpoint address .

Optimize Mongo Database

Performance

For large implementations, FileCloud suggests using SSD storage to host the Mongo Database and increasing the number of CPUs on the host system.

S3 API Pre-signed URLs Pre-signed URLs are used so clients can download files from the Object Store directly rather than through the server eliminating potential performance bottlenecks, etc. This functionality can be disabled.

CONCLUSION

SYNOPSIS

As seen by the integration tests, FileCloud Amazon S3 integration performs well with the Dell EMC ECS S3 interface. Performance

testing showed that uploading larger files was consistently faster than smaller files. FileCloud does not provide a way to set the chunk

size so various chunk size tests were not included in the testing, it does however use Multipart upload with a part size of about 5MB.

FileCloud provides numerous ways for users to interact with their files some of which were not testing during the validation (mobile

apps, browser plugins) however the ones that were used worked flawlessly.

Table 11. General notes and advisements relevant to the validation process

DESCRIPTION DETAIL

Dell EMC Technology S3 Integration Analysis

Although FileCloud configuration lists Amazon S3 and OpenStack Swift, Dell EMC ECS Product Management has indicated that S3 is the preferred interface for ECS Nodes, therefore the Amazon S3 interface is the only interface that was tested. The S3 integration testing shows that the FileCloud implementation is compatible with ECS however the redirect option had to be disabled in order for clients to download objects. ECS supports Pre-signed URLs however a Signature Mismatch error is returned when enabled. This is being investigated further by the ECS Team.

Amazon S3 Pre-signed URLs

A signature mismatch error occurs when a user downloads a file or when the sync client is used and the redirect option is enabled. ECS returns a signature mismatch error. This issue is currently being investigated further. The validation tests were performed with the redirect flag disabled: define("TONIDOCLOUD_DISABLE_S3_REDIRECT","1");


TESTING ENGINEERS

Table 13. List of technical participants who participated in the validation process

NAME TITLE COMPANY PHONE EMAIL

Rich Paulson Advisory Solutions Architect

Dell EMC +1 925-457-8614 [email protected]

TESTING ACCEPTANCE

Table 14. List of technical participants who accepted the validation process results and methodology

NAME TITLE COMPANY PHONE EMAIL

Anis Abdul Codelathe [email protected]

Venkataragavan Ramasamy

COO Codelathe 832-512-8605 [email protected]

REFERENCES

Table 15. References relevant to the validation process

DESCRIPTION DETAIL / LINKS

FileCloud Documentation FileCloud Support Documentation

https://www.getfilecloud.com/supportdocs/display/cloud/Home

Performance Optimization FileCloud Performance considerations

https://www.getfilecloud.com/supportdocs/display/cloud/Performance

Elastic Cloud Storage (ECS) 3.1 Administration Guide

ECS 3.1 Administration Guide https://www.emc.com/collateral/TechnicalDocument/docu86293.pdf

DOCUMENT VERSION HISTORY

Table 16. Document revision history

VERSION # REVISION DATE REVISION AUTHOR REVISION DETAIL

Version 1.0 October 6, 2017 Rich Paulson Initial Document

APPENDIX

Table 17. Appendix lists

ID DESCRIPTION DETAIL

https://www.getfilecloud.com/supportdocs/display/cloud/Home

https://www.getfilecloud.com/supportdocs/display/cloud/Performance

https://www.emc.com/collateral/TechnicalDocument/docu86293.pdf

DELL EMC TECHNICAL VALIDATION OF FILECLOUD WITH ECS 3 · Dell EMC Confidential: Internal and...

Documents

Transcript of DELL EMC TECHNICAL VALIDATION OF FILECLOUD WITH ECS 3 · Dell EMC Confidential: Internal and...