WHITE PAPER

GuardianOS Data MigrationTool Enables Ease of Snap Server Deployment and Data Consolidation

1

Introduction

Powered by the GuardianOS™ operating system, the Overland Storage® Snap Server®family of NAS storage servers support numerous file access protocols that provide clientaccess to files in Windows, UNIX, and AFP networks, as well as remote access via FTP andHTTP. Since Snap Servers are a cost-effective file server replacement for so manyplatforms, a utility that simplifies data migration to Snap Servers was designed as part ofGuardianOS.

This white paper will describe how to utilize and best leverage the data migrationutility built into GuardianOS.

Overview of the Data Migration Utility

The data migration utility allows administrators to easily copy or move data from any fileserver that has an accessible Windows (CIFS/SMB) share or UNIX (NFS) mount. It can alsopreserve permissions in order to make the transition to a new Snap Server seamless toend users and data owners. Previously, only native tools were readily available to movedata from older file servers. Now, for consolidating data from multiple sources orperforming a technology refresh, the data migration utility simplifies the migrationprocess and reduces the implementation time for the new Snap Server.

The Old WayIn the past, if an administrator was moving data from a Windows system he could simplydrag-and-drop the data if he was unconcerned with preserving permissions. However, ifhe wanted to maintain any user and group permissions while migrating, he was forcedto use other utilities, such as XCOPY, Robocopy, SCOPY, XXCOPY, etc. Each of thesetools has its own set of subtle nuances that can make it difficult to work with and canproduce inconsistent and unpredictable results. Not to mention that these tools lack theability to verify the data transfers or restart without overwriting all data that waspreviously copied after a failed data movement.

If the administrator was moving data from a UNIX-based operating system, the task wasnot as unpredictable, but consolidating multiple systems could, nevertheless, becomeincredibly cumbersome.

In many cases, administrators are replacing or offloading data from existing networkattached storage (NAS) systems that do not have local utilities or commands formigrating data from them, requiring the use of a third-party Windows or UNIX system tomove that data. Permissions on both the source and target must be painstakingly set toallow the operation. The third-party system will become bogged down moving datainstead of doing other productive work. This can especially be a problem if theadministrator’s own workstation is used.

A Better WayNow, with an integrated data migration utility on GuardianOS-powered Snap Servers,administrators can easily move data, streamlining the integration of a new Snap Server.This makes it less daunting to offload shared data from servers that were not originallyintended for file serving, perform a technology refresh on older file servers that cannotkeep up with the current performance and capacity requirements, or even to centralizethe storage of multiple file servers to one high-capacity and high-performance solution.

2

The benefits of using the embedded data migration utility over other copy or move toolsare simplicity and centralization. The simplicity comes from having a centralized singleinterface for copying or moving data from any server or storage platform that can bemounted using either CIFS/SMB (Windows) or NFS (UNIX/Linux/Solaris/Mac OS X).Administrators will no longer have to learn any command-line switches to make sure thatthe permissions are included. No manual verification that the data was successfullymigrated is required. Everything an administrator needs to successfully perform datamigration from other servers is on one easy-to-use screen within the web browseradministration tool of Snap Server systems. Since the data migration utility runs nativelyon the Snap Server, all migration processing is done locally on the Snap Server and doesnot require any other system to facilitate the migration of data from the source.

Just getting the data where it needs to be on a new file server can be the most difficultpart of integrating a new data storage device. With the data migration utility,administrators can now easily accomplish this task.

Use Case Scenarios for the Data Migration Utility

There are several scenarios where the data migration utility can simplify the task ofgetting data moved over to the new Snap Server. The following use case scenariosoutline some of the ways that in which the data migration utility can be used. Figure 3.1shows just two examples of these use cases.

Figure 3.1

The data migration utility is designed to be a single-direction, single-instance tool formoving data from a source system to a target system. In the first usage case above(Performing a Technology Refresh with the Data Migration Utility) the usage of the datamigration utility is illustrated as a single operation for each server that is being replacedwith a more current Snap Server system. In the second usage case above(Consolidating Multiple File Servers with the Data Migration Utility) the data migrationutility is run repeatedly for each source system that is being consolidated and/orreplaced by the more current Snap Server system.

Snap Server 520, 620 or

Snap Server with Expansion

3

Use Case #1 – Migrating Data from a Server Running Windows (CIFS/SMB)Migrating data from a Windows source server requires the existence of a Windows shareto provide access to the data by the Snap Server. The source server can be in aWindows Workgroup, an NT Domain, or an Active Directory Domain. The data migrationutility can retain permissions for servers in a Windows NT or Active Directory Domain aslong as the Snap Server is joined to the domain prior to executing the migration. If theWindows source is part of a Windows Workgroup, permissions can not be retained as alllocal user and group accounts are only relevant on the source server. Local Windowsuser and group security identifiers (SIDs) are not transferable, and therefore, thepermissions cannot be recreated. In all cases, a proper administrative user account willbe required to access the data on the source machine in order to properly transfer thedata and permissions.

If the source data contains filenames with extended characters other than commonWestern European characters (CP 1252), it may be advisable to enable Unicode on theSnap Server. Then, convert the file system to the proper mode before migration in orderto properly retain the extended characters.

Note: SMB must be enabled on the Snap Server to perform migration using the Windowsnetwork protocol option. This is enabled by default in the Network – Windows screen ofthe web browser administration tool. A valid Windows share must also exist that allowswrite access to the target path in order to properly preserve permissions and DOSattributes.

Use Case #2 – Migrating Data from a Server Running UNIX or Linux (NFS)When moving data from a UNIX-based file server via NFS, the source server must have avalid NFS mount point for access to the data by the Snap Server. The file server can bepart of a NIS Domain, but does not have to be. If permission preservation is enabled, thedata migration utility preserves permissions and ownership for all user identifiers (UIDs)and group identifiers (GIDs) verbatim, regardless of whether an equivalent UID/GID isknown to the Snap Server. This means that if all UIDs/GIDs are created on the target andmatch the source, permissions will be properly retained. A proper administrative useraccount will be required to access the data on the source machine in order to properlytransfer the data and permissions. The migration will take place with the UID of the localSnap Server user logged into the UNIX machine as the migration user (note that thename may not have an equivalent on the target machine – only the UID is relevant).

Filenames are transferred over NFS verbatim as a “bag of bytes”. Therefore, extendedcharacters in filenames may not be properly retained if encoded on the source with acharacter set that does not match the character set used on the Snap Server's filesystem (ISO 8859-1/CP 1252 if Unicode is disabled, or UTF-8 if Unicode is enabled).

Note: When connecting to the NFS source, the Snap Server tries to connect using NFSv3by default, but will fall back to NFSv2 if necessary. It will, likewise, try to use TCP as thedefault transport protocol, but fall back to UDP.

Use Case #3 – Migrating Data from a GuardianOS-Powered Snap ServerMigrating data from another GuardianOS-powered Snap Server to a new Snap Serverwith the data migration utility works just as if you were moving data from any other fileserver source. Be sure to choose the appropriate protocol for the migration job withrespect to the security model of the data being migrated. For example, if the data is in

a Windows SnapTree, use Windows, and if the data is in a UNIX SnapTree, use NFS tomigrate the correct permissions. The rest works the same as any other Windows or UNIXdata migration as described in Use Cases #1 and #2.

To properly retain any extended characters in filenames, make sure the target SnapServer is configured for the same Unicode state (enabled/disabled) as the source SnapServer.

Note: If the source location contains both Windows and UNIX SnapTrees and you wantto preserve permissions, split the migration operation into multiple jobs, one for eachSnapTree for the given security model, using the appropriate protocol per job.

Use Case #4 – Migrating Data from a SnapOS-Powered Snap ServerLike GuardianOS-to-GuardianOS migrations, moving data off of a SnapOS-poweredSnap Server is as simple as defining the data type (Windows or UNIX) that you will bemigrating. Permission preservation, however, is generally not recommended whenmigrating from a SnapOS-powered Snap Server. SnapOS native file system permissionsare only properly visible and configurable in the SnapOS web UI. Though modeled afterWindows-style permissions, SnapOS permissions cannot be communicated over SMB orany of the file access network protocols. In addition, UNIX permissions reported over NFSby SnapOS servers are loosely interpreted from the SnapOS file system permissions, anddo not necessarily properly represent them. Therefore, preserving them when migratingover NFS may not result in equivalent effective permissions on the target, even ifpreserved verbatim with the same UIDs/GIDs.

SnapOS does not enforce a standardized character set for filenames and relies on DOScode pages for communicating filenames over SMB. Therefore, extended charactersmay not be properly retained using either SMB or NFS migration, depending on themethod and protocol by which the filenames were originally written at the source.Because NFS migration transfers filenames verbatim as a “bag of bytes”, extendedcharacters are likely to be copied improperly, especially if the target GuardianOS SnapServer is configured for UTF-8.

Use Case #5 - Migrating Data from Mac OS XMac OS X migrations can be done using either the Windows or NFS network proto-col, but they cannot retain the extended ACLs or resource forks of OS X 10.4’sextended ACLs or resource forks. If neither of these is a concern, then the appropri-ate migration protocol should be selected based on the need to retain UNIX-stylepermissions, whether there are extended characters in filenames on the source, andwhether the GuardianOS-powered Snap Server is in Unicode mode.• A NFS migration can retain UNIX-style permissions from OS X, but characters in

filenames are migrated verbatim as a “bag of bytes”. Extended characters onsource filenames will, therefore, only be retained properly if Unicode is enabled on theSnap Server.

• A Windows migration cannot retain permissions from OS X, but it can retain commonWestern-European extended characters in filenames if Unicode is not enabled on theSnap Server target. It can retain all extended characters if Unicode is enabled on theSnap Server.

Use Case #6 – Migrating Data from Other SourcesMigrating data from any other source is possible, as long as the source file server or NASsystem has an NFS mount or CIFS/SMB share available. Moving data off of other sourcesonly requires that you have an administrative login with appropriate access privileges.For customers whose NAS offerings are not performing adequately or meeting all of their

4

5

needs, the data can be easily migrated from these systems using the data migrationutility.

If NFS migration is chosen, note that filenames are transferred over NFS verbatim as a“bag of bytes”. Therefore, extended characters in filenames may not be properlyretained if encoded on the source with a character set that does not match thecharacter set used on the Snap Server’s file system (ISO 8859-1/CP 1252 if Unicode isdisabled, or UTF-8 if Unicode is enabled).

Note: Similar to a migration from GuardianOS to GuardianOS, migrating data from alocation that has mixed Windows-style and UNIX-style file permissions – e.g. a mixed-mode qtree on a NetApp filer – can only properly preserve permissions that correspondto the protocol used for the migration job. For example, if Windows is used, onlyWindows permissions will properly be preserved; and if NFS is used, only UNIX permissionswill be properly preserved.

Procedures for Using the Data Migration Utility

The data migration utility can be found on the bottom of the main Administration webpage of the web browser administration tool (see Screenshot 4-1) for any GuardianOS-powered Snap Server. The latest version of GuardianOS can be found at the supportportal at www.overlandstorage.com

Screenshot 4-1

It is important that, before any data migration, the administrator carefully plan out theprocess and allow enough time to move the data and test access prior to turning thenew Snap Server live to the end users. Roughly estimating a data migration can bedone by creating a “dry run” to copy (not move) the data; let the migration job run forsome time; and then check the estimated time remaining on the Data Migration StatusPage. The job can then be stopped and restarted when there is enough time tocomplete an uninterrupted migration. This will give a very rough estimate of themigration time and can in no way be used as an accurate gauge for the actualmigration. See Screenshot 4-2 for a look at the Data Migration page.

6

Screenshot 4-2

Requirements:- GuardianOS 4.4.045 or later- Valid Windows/CIFS share or NFS mount point with administrative access- Dedicated access to the source data (no clients accessing the source during

migration)- Minimum 100Base-T network for both source and target on the same local LAN (Does

not support WAN-based migrations)- Sufficient available capacity on the target volume on the Snap Server

The following is an overview of the steps that should be followed to set up and executea successful data migration:

Step 1: Synchronize User AccountsIf it is critical to preserve permissions, join the appropriate Windows Domain (with trusteddomain support enabled, if applicable) or NIS Domain, and add any local users andgroups that are necessary. Also make sure if there are any ID mappings that need to becreated between Local or NIS users/groups and Windows users/groups, that this is doneprior to beginning any migration jobs. ID Mapping instructions can be found in theGuardianOS Administrators Guide or in the online help.

Note: All of the users and groups that own or have permissions assigned at the sourceneed to be properly added and configured on the Snap Server.

Step 2: Choose Network ProtocolDecide which network protocol will be used for the data transfer. If you are transferringfrom a Windows file share, then you would choose the “Windows” option that uses theCIFS/SMB network protocol. If you are transferring data from a UNIX mount point,choose the “NFS” option. If either protocol is an option, choose whichever is moreconvenient, or, if preserving permissions, choose the protocol that matches the filepermission type of the source data.

Step 3: Identify Source User AccountIdentify an administrative user account that has the appropriate access on the sourcesystem to move the data. If performing a move operation, note that this user needs tobe able to delete source data, and if preserving permissions, needs to be able to readpermissions on source files. For Windows migration jobs, the user “Administrator” or an

7

equivalent account – e.g. a domain administrator such as “mydomain\administrator” –is recommended, as is the “root” account for NFS migration. If using “root” for NFS,make sure to use the no_root_squash option on the source export. In general, it is agood idea to test access to the source’s share or mount point with the identified useraccount before proceeding.

Step 4: Create the Security Model on the Snap ServerUsing the SnapTree feature, ensure that the appropriate security model for themigration type (Windows or NFS) is configured on the target path for the SnapServer. The following are some examples to help illustrate the importance of settingthe proper security model:• If moving Windows data, set up a Windows SnapTree at the target, and if moving

UNIX data, set up a UNIX SnapTree. If the SnapTree type does not match the migrationjob’s Network Protocol (e.g. a Windows job configured to migrate to a Unix SnapTree),a warning will be returned by the data migration utility.

• If preserving permissions when migrating Windows data, each file or folder written to afolder with the UNIX security model will result in a “permission denied” error output tothe migration log.

• If the target location is a volume root directory that contains various SnapTrees ofdifferent security models, permissions will be properly maintained for files and foldersplaced in directories with the security model that matches the protocol of themigration job, whereas files and folders placed in directories with a mismatchedsecurity model will not be properly retained (and may return an error per the above).

Step 5: Create a Share on the Snap ServerCreate the target share(s) or mount point(s) on the Snap Server exactly as you wantthem to be. Make sure that at least one share is available that can access the targetpath that gives write access to the Snap Server’s local root user.

Step 6: Move or Copy?Decide whether you want to copy the data (leave the data on the source) or move thedata (delete the data from the source). This is a very important decision. For example:In your environment it may make the most sense to copy all of the data first and then todisable access to the old file server and activate the new Snap Server share. This wouldallow you to test and make sure everything is working as desired before removing thedata from the old server. If you decide to “move” the data instead, it is highlyrecommended that you ensure there is no other user access to the source or targetserver during the migration. It is also highly recommended that you use the Verify optionto ensure the data has been properly migrated. The Verify option will take twice as long,but will ensure that the data was moved or copied correctly.

Note: If verification is enabled, files will only be removed from the source after successfulverification; otherwise they’ll be removed from the source immediately after copying.

Step 7: Restrict Access to the Source DataMake sure that the source location is quiescent. In other words, at the very least no usersor services can change or add data at the source. It is always best to restrict all otheraccess except for the Snap Server system doing the migration.

Step 8: Run the Migration JobOnce all of the necessary information is gathered, configure and run the migration job.At a minimum, always restrict access to the source from which the data is being moved.It is always best to restrict all other access except for the Snap Server doing themigration.

8

Step 9: Check the LogsAfter the job runs, always check the log for errors by clicking on the View Log button atthe bottom of the data migration screen.

Step 10: Verify the Data MigrationIf you used the Verify option, the Data Migration Log will help you determine success forthe migration. If you chose not to use the Verify option, make sure to use some othermethod to validate the success of the data migration.If you are executing another migration after a previous failed migration, it is crucial thatverification is enabled, to ensure that any files that may be corrupt due to a migrationfailure are identified. Corrupted data can only be found by utilizing the Verify option. If afile comparison fails in the verification phase, the target file will be moved to a“quarantine” directory on the same volume. The log message for the comparison failurewill indicate the path.

Step 11: Take the Old Server OfflineOnce migration of the data has been successfully verified, bring the old file server sharesor mount points offline. This will prevent new data from inadvertently being added to theold server.

What Permissions are Preserved?

The types of permissions retained will differ, depending on which of the followingmigration scenarios is applied:

Migrating from a System Using a Windows Security Model:When migrating from a Windows server or any other server using a Windows SecurityModel to a Snap Server, permissions will be retained as follows:• Known domain user owners are retained as owners if the Snap Server belongs to the

same or trusting domain of users assigned permissions on the source data. To retainpermissions for users of a trusted domain, Trusted Domain support must be enabled onthe Snap Server. To enable, navigate to Network > Windows.

• Permissions for unknown users and groups and for local users and groups on theWindows server are NOT retained.

• Access Control Entries (ACEs) for users and groups will be applied on the target filesapproximately as they were on the source files. Due to differences between WindowsAccess Control Lists (ACLs) and POSIX ACLs, the effective permissions may not bethose you expect. For more information, see “How the GuardianOS and WindowsDiffer in Processing File-Level Access Permissions” in the Snap Server AdministratorGuide. For example:– If there is no ACE for everyone on the source, it will be assigned No Access for all

users who are not covered by higher POSIX ACL checks.– If there is a Deny All ACE for everyone on the source, it will be applied to all ACEs on

the ACL (that is, everyone is denied access, although the owner always has implicitchange permissions).

– If there is a Deny All ACE for non-owning users or groups, it will be removed (i.e.,enforced as a lack of permission rather than as an explicit Deny).

– If there are Deny ACEs on users, the denied bits are set as No Access rather thanenforced as explicit Denies.

– If there are Deny ACEs on groups, the denied bits are set as No Access on thegroup ACE, and are also propagated as No Access bits to ACEs for individualgroup members (if any exist).

9

Migrating from a System Using a UNIX Security ModelWhen migrating from a UNIX server using a UNIX Security Model to a UNIX SnapTree, UNIXpermissions for UIDs/GIDs are copied exactly from the source to the target; thus,identities of the users and groups will be best retained if the Snap Server belongs to thesame NIS domain as the UNIX server. If an NIS Domain does not exist all of the sameUIDs/GIDs must exist and be configured properly as local users on the Snap Server toretain the permissions properly.

Migrating between conflicting Security ModelsIt is not recommended that permissions be preserved when migrating from a WindowsSecurity Model to a UNIX SnapTree, or vice versa. Permissions will not be applied orenforced as expected.

Migrating from a GuardianOS-Powered Snap ServerWhen migrating from one GuardianOS-powered Snap Server to another, it isrecommended that the same Security Model is used on the target server that was usedon the source.• If the source server uses a Windows SnapTree and has permissions assigned to

Windows domain users, use a Windows connection for migration.

Note: Permissions for local users on the source Snap Server will not be retained. Ifpermissions or ownership is largely assigned to local users, it is not recommended touse the Preserve Permissions option because the permissions will all be dropped.

• If the source server uses a UNIX SnapTree and has permissions assigned to local or NISusers, use an NFS connection for migration.

• Generic POSIX ACLs or Solaris POSIX ACLs will not be retained. Only standard UNIXpermissions (User/Group/Other) are maintained as part of data migration process.

Note: Local users that have UNIX permissions on the source will not be created on thetarget with the same UIDs. These local user accounts and UIDs must be createdmanually. During a NFS migration, permissions, and ownership are preserved verbatim– if local user name and/or group name equivalents for these UIDs/GIDs are needed,those users and groups with corresponding UIDs/GIDs need to be created manually.

When Not to use the Data Migration Utility

In some cases there are better options for moving or copying data to a Snap Server. Thefollowing describes some common scenarios that have better alternative methods formoving data:• Data migration is not a replication replacement – The data migration utility is not

schedulable and therefore is not a good tool for repetitive data movement. The datamigration utility also is not optimized by way of bandwidth throttling or copying byte-level changes only, which are highly beneficial for WAN-based data movement.Purpose-built tools, such as Snap Enterprise Data Replicator™ (Snap EDR), are bettersuited for this type of task.

• Data migration is not a backup replacement – For the same reasons that the datamigration utility is not suited as an ongoing replication tool, it is not designed with databackup in mind. Features such as scheduling, incremental capabilities, versioning,and backup cataloging among many other things make the data migration tool apoor choice as a backup tool.

10

• Mixed Windows and UNIX security models – For any data that is accessed by endusers from both Windows and UNIX machines, the data migration tool can onlyeffectively retain the permissions of one or the other security model, based on theprotocol used for the migration job. Therefore, it is recommended to choose thesecurity model that is predominantly used by the end users. Once the data migrationhas completed, the other security model permissions will have to be manuallyadjusted.

Note: When migrating from an older GuardianOS-powered Snap Server, Snap EDRcan be used to make sure all Windows and UNIX permissions are properly retainedduring the data movement.

• Migration of AFP files from SnapOS-powered Snap Servers – Macintosh files stored on aSnapOS-powered Snap Server may have resource forks associated with them thatmay not be properly accessible on the GuardianOS target after migration. Therefore,a manual copy or move of the data from a native Macintosh is the best option formigrating Macintosh files.

Notable Caveats

• NFS data migration is very memory-intensive, scaling with the number of files andfolders processed. Jobs should therefore be configured to divide up the source datainto data sets not exceeding one million total files and folders per job. There are nolimits for CIFS/SMB migration jobs.

• It is important to note the following behavior for migrating symbolic and hard linksfrom UNIX-based source servers using NFS:– Symbolic or soft links (also known as symlinks) are preserved verbatim, symbolically –

They are not followed (to prevent getting stuck in a recursive symlink loop). As aconsequence, symlinks at the source migrated to locations outside of the data setmay be broken at the target.

– Hard links are retained as hard links – Multiple files pointing to the same inode at thesource will result in those same files pointing to the same inode at the target.

• NetApp systems using a version of Data ONTAP® older then version 6.5.7 may notallow the Windows (CIFS/SMB) data migration engine to authenticate to the NetAppbox via a Windows Domain. This was verified as fixed in version 6.5.7.

• A NetApp bug has been noted during testing that can result in failure to migrate ifNFSv3 is enabled and TCP is disabled on the NetApp server. If necessary, configurethe NetApp server to either allow NFS over TCP or force NFSv2.

© 2008 Overland Storage, Inc. All trademarks and registered trademarks are the property of their respective owners. EDRDM-WP0908-04

Overland Storage provides affordable end-to-enddata protection solutions that are engineered tostore smarter, protect faster and extend anywhere— across networked storage, media types, andmulti-site environments.

Overland Storage products include award- win-ning NEO SERIES® and ARCvault™ tape libraries, REOSERIES® disk-based backup and recovery applianceswith VTL capabilities, Snap Server® NAS appliances,and ULTAMUS™ RAID high-performance, high-densitystorage.

For more information, visit www.overlandstorage.com

About Overland Storage

WORLDWIDE HEADQUARTERS4820 Overland AvenueSan Diego, CA 92123 USATEL 1-800-729-8725

1-858-571-5555FAX 1-858-571-3664

UNITED KINGDOM (EMEA OFFICE)Overland House, Ashville Way Wokingham, BerkshireRG41 2PL EnglandTEL +44 (0) 118-9898000FAX +44 (0) 118-9891897

www.overlandstorage.com