Quantum Stornext 4

8/10/2019 Quantum Stornext 4

1/12

P R O D U C T B R I E F

NOTICEThis product brief may contain proprietary information protected by copyright. Information in thisproduct brief is subject to change without notice and does not represent a commitment on the partof Quantum. Although using sources deemed to be reliable, Quantum assumes no liability for anyinaccuracies that may be contained in this product brief. Quantum makes no commitment to updateor keep current the information in this product brief, and reserves the right to make changes to ordiscontinue this product brief and/or products without notice. No part of this document may bereproduced or transmitted in any form or by any means, electronic or mechanical, includingphotocopying, recording, or information storage and retrieval systems, for any person other thanthe purchasers personal use, without the express written permission of Quantum.

StorNext 4.0 Technical Product Brief


2/12


3/12

StorNext 4.0 Product Brief 3

StorNext 4.0 Product BriefP R O D U C T B R I E F

Methodology

StorNext software delivers flexible enterprise data management solutions, such as high speed data accessand cost-effective storage tiering and archiving with two core components, the StorNext shared SAN filesystem (SN SAN Clients) and StorNext Storage Manager.

StorNext File System is enabled by SAN clients which reside on a Windows, Linux, UNIX, or MAC OS serverconnected to a SAN. A StorNext Meta Data Controller (MDC) sits on a Windows or Linux server and acts asthe traffic cop for shared heterogeneous server access to the data pool on the SAN storage. Where the SANshares out the storage to a server community, the StorNext file system, enabled by the MDC, shares a commonname space, and data within, out to the applications simultaneously across different server platforms.

StorNext Storage Manager, residing on the MDC, is a policy-based data mover that automatically andtransparently copies or moves data to another tier of storage. Files can be truncated in the primary filesystem, leaving behind a pointer, or can remain whole with a copy in a lower storage tier. Since thismovement is all done within the StorNext instance, the data remains local to the application accessing thefile system. If an archived file is requested, the Storage Manager fulfils the request and presents the fileback to the application as if it was resident in the primary file system.

There are many other features, including replication and deduplication, which further enhance these twocore components ability to manage enterprise data.

File System Background

To understand StorNext, it is helpful to understand some file system basics. For purposes of this product briefwe will discuss file systems used for normal business operations (such as the file system on your laptop). A filesystem is a structure for storing and organizing data. NTFS, EXT3, FAT are all examples of file systems in whicha user creates directory structures, assigns access permissions, and stores files. File systems are created on topof one to many disk systems (sometimes aggregated into LUNs) that may be internal to a server or a part of alarger disk array accessed via a LAN or SAN. In general, file systems are mounted and owned by a single server(for example, Windows server ABC owns D:\ which is a RAID 5 volume on a disk array).

While this configuration allows a server to read and/or write data to/from the file system very quickly,the disk capacity is available only to that server. In environments with many servers, this results in largeamounts of under-utilized capacity and over spending on storage budgets. It also implies that only usersand applications on a given server can access data in that file system. In many situations this is acceptable(e.g., user PCs, specialized systems that require dedicated resources, etc.). However, there are manyinstances where it is useful to have a file system, and its stored content, accessible to many individuals andapplications. In this case, network file system protocols like NFS (Linux/UNIX) and CIFS (Windows) areutilized to make a servers file system available to a larger group of servers, users and applications.This allows storage consolidation and the ability to easily share content between multiple applications. NFSand CIFS are well established protocols that have been utilized on servers and specialized NAS appliancesto make file systems available over an IP (Internet Protocol) network. While the performance of theseprotocols is not very high (50 70 MB/s generally for NFS, 35 MB/s generally for CIFS over Gigabit Ethernetconnections), they are most commonly used for general purpose file serving of office files and documents.

For larger files or content that requires low latency, a different solution is required. To meet these needs,shared file systems and centralized storage are utilized. Client-based software allows multiple servers toshare a common storage pool with a single namespace. Applications send data to the shared file systemdirectory (or folder) just like any other file system (e.g. a Windows NTFS). The difference is that the client filesystem software, not a file server, processes read and write operations locally and coordinates access toall other servers and clients. Most file systems lack volume management features and must utilize other 3rdparty volume management tools. Simple shared file systems are homogeneous, allowing many servers of asingle OS type (usually Linux) to share the file system. While this can be sufficient in small environments, itlacks scalability within organizations that have servers running different operating systems like Windows,Linux, UNIX and even Mac.


4/12



StorNext File System High-Performance Shared Data Access

StorNext is unique from other file systems in that it is a heterogeneous shared file system that allowsmultiple servers access to a common disk repository regardless of OS type. Nearly all serverswhether theyare Windows, Linux, UNIX, and/or Mac OS (via Apples Xsan)can run a StorNext client and obtain accessto the shared file system. StorNext software is developed for each OS type to ensure the file system ispresented coherently and data requests are properly processed (e.g. a Windows kernel behaves differentlyfrom a Solaris kernel).

Because multiple servers are accessing a single file system, some form of traffic cop is required to preventtwo applications or users from writing to the same disk location and guarantee that a server reading afile is not accessing stale content because another server is updating the file. In StorNext, this function isperformed using a MetaData Controller (MDC).

The MDC is a light weight server that sits outside the file system data path and is responsible fornegotiating access while maintaining coherency between clients (critical for allowing real-time parallel R/Woperations to a single file). Clients communicate with the MDC via an IP connection to obtain informationabout file location and block allocation, providing direct, block level access to the disk (see Figure 2).

Data access breaks down as follows:

1. Application issues a write command and StorNext client sends an op request to the MDC via the LAN2. MDC processes and responds via LAN with the disk blocks where the client can write data3. StorNext client receives the data block locations and StorNext client writes directly to the file system

at wire speeds

StorNext is designed as a heterogeneous SAN file system, allowing servers and clients attached to a

Fibre Channel (FC) or iSCSI SAN to directly access the same file system. This provides the highest level ofthroughput and the lowest possible latency in a mixed OS environment. StorNext utilizes one or more FCconnections on a server to access disk resources. This produces single server performance scalability from100s of MB/s to multiple GB/s simply by adding FC Host Bus Adapters (HBAs). Of course, performance isnot only based on host and network capabilities, but also the performance of disk that makes up the filesystem. For this reason a StorNext file system provides volume management of multiple disk LUNs andmultiple arrays. This is similar to the functionality available on most OS types that allow administrators tocreate a single volume from multiple RAID disk devices.

PRIMARY TIER

13

2

StorNext

SAN ClientsMetadataController

GigE TCP/IP

Infiniband IP

FC/iSCSI

Figure 2


5/12



An additional performance factor taken into consideration by StorNext, but not by most traditionalfile systems, is metadata location. A file is composed of content (the actual data) and metadata (theinformation about the file which includes attributes, permissions, etc.). Whenever a file is written, updated,or deleted the metadata is changed. This means that whenever a file is written, not only must the data bewritten but metadata must be written as well. With larger files, write performance can be reduced becausea file may be written sequentially for high performance, but the metadata write operations requires diskhead movement in bursts to another location on disk. For this reason, StorNext allows metadata to bewritten to a different disk volume so that the sequential writes of data are isolated from more randommetadata writes, providing the best possible throughput.

In addition, StorNext includes cache coherency mechanisms to ensure that data is coherent and up-to-dateeven when data is changed by multiple clients. These mechanisms also limit the required metadataoperations by having the clients communicate with only the MDC for certain requests, such as file creation,deletes or allocations.

Distributed LAN Clients (DLC)

While SAN file systems are excellent for applications requiring extremely high-speed and low latency access,not all applications fit this profile. In fact, an increasing number of environments are composed of a mixof applications requiring different access methods. For example, while online editing applications requirehigh-speed and low latency access, compute nodes and rendering applications require CPU-intensivecomputational operations with less dependency on data access performance. StorNext Distributed LANClient provides a NAS interface to the shared storage for these latter types of applications.

DLC allows servers on an IP network to connect to StorNext volumes through SAN clients designated asclustered gateway servers. These gateways are SAN clients that connect directly to the shared storage poolover FC and/or iSCSI, but primarily service DLC I/O instead of running customer applications. DLC is ideal forcustomers who have farms of servers (rendering, compute nodes, web servers, etc.) all needing to accessa shared set of files, but not at FC speeds. DLC is also used when customers have a variety of performanceneeds and want to mix SAN and NAS characteristics (see Figure 3).

While this might sound similar to traditional CIFS/NFS data sharing, Distributed LAN Client is unique inthat it utilizes a specialized block-based IP protocol designed for higher per-stream performance andresilient communication. This specialized protocol is optimized for StorNext and can achieve near line-ratethroughputs over standard IP network connections. The protocol also provides resiliency by having each DLCautomatically attach to multiple clustered gateways.

SECONDARY TIER

Vendor Agnostic

PRIMARY TIER

TAPE LIBRARY ARCHIVE

OptionalStorNext

PolicyEngine

SCALESTOT

HOUSANDSOFSTORN

EXTDISTRIBUTEDL

ANC

LIENTS

StorNext DLC

Gateway Servers

Vendor Agnostic

GigE TCP/IP

Infiniband IP

StorNext DLC

ateway ServersGigE TCP/IP

Infiniband IP

MetadataController

FC/iSCSI

Figure 3


6/12



Load balancing and clustering is provided when the DLC requests I/O across the attached gateways. If agateway system is unavailable or the network connection is lost, DLC transparently fails over I/O to anotherattached gateway. Additionally, a DLC load balances data requests across all available network interfaceson the client. In the chart below you can see a comparison between NFS, DLC and SAN Client performanceusing the exact same system (see Figure 4). Those numbers are not maximum throughput numbers;however, they show how throughput increases dramatically from NFS to DLC (nearly 3x the throughput onthe same network).

Distributed LAN Client is an important part of the StorNext data sharing offering, allowing lower cost highspeed LAN connectivity for applications connected via the network. This is especially attractive to customerswith High Performance Computing (HPC) and rendering projects where a large dataset is broken intosegments and processed by multiple servers.

For more information on Distributed LAN Client please see the Quantum StorNext Distributed LAN ClientProduct Brief (PB00021).

Tiered Storage and Data Preservation

SAN and NAS file systems are typically used for ingest and as a working area for most file types. However,there are many files that are candidates for movement beyond primary storage. These files are described asreference or static data types; data that is rarely, if ever, accessed or changed. Similarly, there are somedata that only needs to be preserved, whether its for protection or long-term archiving.

Most people are familiar with the protection component of a data management strategy. Almost everycompany employs some product to safeguard file system content. This might include applications likebackup software or integrated data protection features like snapshots. To protect content, these productsmake duplicate copies of files (or data blocks) enabling point-in-time recovery in the event of data loss.While these products are critical to provide continuance of business operations, it is important to notethat they do not reduce the amount of primary storage. This is especially important to consider today asindividual file sizes and number of files maintained by an organization are increasing exponentially. As moredata is stored for longer durations, it becomes increasingly cost prohibitive to keep every file on primarydiskboth in terms of capacity costs, power and cooling costs and data protection costs. For instance, asmore data is maintained, backup windows get longer and more tape/disk is required to backup files.

Storing all data on primary disk might be acceptable if all data in the environment is actively used, but inmost organizations only a fraction of data is actively in use or changes frequently. In fact, the majority ofdata in most organizations is used for only a short time after creation.

0

50

100

150

200

250

300

350

NFS (2x 1Gb) LAN Client (1x 1Gb) LAN Client (2x 1Gb) SAN Client (2x 4Gb)

Throughput (MB/s)

Figure 4


7/12



This is where a tiered data approach becomes important. Tiered strategies are aimed at reducing the costof storing content while still keeping it accessible to users for re-purposing. Archiving is a specific exampleof how data retention can be implemented to prevent expensive primary storage from being overwhelmedby inactive data. Archiving in this regard is not necessarily for compliance adherence but instead for cost-effective data storage. Data management software helps control storage costs by moving data off ofprimary storage and on to lower-cost secondary storage tiers. This allows users to reduce the amount ofexpensive, primary disk in play and instead use more cost effective mass storage devices for infrequentlyused content (i.e. tape technologies, VTLs or MAID). Tiered and archived data is kept in a near-line stateand accessible to users so it can be re-purposed for future revenue-generating projects.

StorNext Replication

General file-based replication has many use cases including data consolidation, data distribution,repurposing, and data protection for disaster recovery, just to name a few. StorNext Replication is anasynchronous, host based replication feature that is highly flexible and configurable. Replication isimplemented at the directory level and is policy driven to initiate timed or user initiated copies of data.The policy engine within StorNext also includes the capability to specify how copies are maintained andmanaged on both the source and target locations. For example, replicated files that have been deleted onthe source can be kept at the target location. With StorNext Replication, environments can be designed toreplicate up to five source StorNext file systems into one centralized StorNext target location (see Figure 5).

Similarly, StorNext replication can also be configured to replicate from one StorNext file system to as manyas three target StorNext file systems (see Figure 6). The target file system can be the same machine as thesource, different machines on the SAN, or in a remote StorNext environment over a LAN or WAN.

For customers who already have a StorNext Storage Manager environment and want to ingest those filesinto a replicated environment, files will need to be retrieved from the disk or tape device and passedthrough the Replication policy engine to be replicated to a new target location. Environments can bereplicated in many different combinations, including:

StorNext File System to StorNext File System StorNext File System to StorNext Storage Manager StorNext Storage Manager to StorNext Storage Manager

Remote Office

SOURCE TARGET SOURCE TARGET

Central Datacenter

WAN

Remote Office

Remote OfficeCentral Datacenter

Remote Office

WANRemote Office

Remote Office

Remote Office

Remote OfficeRemote Off

Remote Off

Remote Off

Remote Off

Remote Off

acenter Central D

ote Office

ote Office

ote Offce

Figure 5 Figure 6

DATA CONSOLIDATION AND PROTECTION DATA DISTRIBUTION AND PROTECTION


8/12



Replication policies can be assigned to more than one directory at a time for easy management. For addedflexibility, StorNext Replication supports the concept of cascading (or multi-hop) replication, meaningany directory can be designated as a source and/or target for replication. Files are replicated only whenthe primary copy changes, and has the ability to truncate (remove) the primary copy of the file once thereplicated copy is stored, thus freeing up valuable capacity on primary disk systems.

Benefits of StorNext Replication over array based or 3rd party replication products include:

Flexibility StorNext can replicate multiple OS types, multiple vendor SANs and can be combined withtiered data environments

Cost StorNext replication is license-based on per server basis, versus a capacity based model

Web services in the StorNext Management Console can be utilized to integrate Replication more closelywith 3rd party and custom applications

Deduplication used with Replication provides enhanced replication performance in low bandwidthenvironments making recovery time much faster because you dont have to pull as much data back fromthe remote site to rebuild the files

StorNext Deduplication

Quantum has long been an innovator in data management software, including data deduplicationtechnology. We developed the pioneering technology known as variable-length hash-based datadeduplication that has become an industry leader in the deduplication market. The introduction ofDeduplication in the StorNext File System now embraces this technology in the primary storage tier and ischanging the way tiered storage is used in file systems.

StorNext Deduplication reduces the amount of capacity consumed by data on disk. Unlike compressiontechnologies that use static pattern table methodologies based on an individual file being compressed,StorNext Deduplication builds its redundancy index based on all the data being deduplicated, not just thefile being processed at that moment. StorNext Deduplication will find redundancies within a file making itextremely efficient at saving storage space (up to 93% in some documented use cases).

With StorNext Deduplication (see Figure 7), any area of primary storage can become the repository forthe deduplicated dataset. StorNext Deduplication utilizes the same file system policy engine running on aLinux MDC that is put in place for StorNext Replication. The system allows the administrator to set policiesthrough the Management Console or command line based upon:

Directories to deduplicate How quickly deduplication will occur after files are created or changed When, and if removal of primary file occurs Meeting file size requirements in order for a file to be deduplicated

A typical use case for StorNext Deduplication is for reducing the capacity utilization due to having multiplecopies of the same, or even similar, file. Imagine how much disk space can be saved by deduplicating largestatic files, images, and video that have been intentionally or unknowingly stored in multiple locations.Capacity reductions also occur for customers who push pre-production copies of deduplicated data to targetlocations (using StorNext Replication), then push production copies of the same data to the target locationlater. With StorNext Deduplication only small amounts of production data are transferred at this time.

StorNext Deduplication can also take advantage of WAN acceleration techniques during replication. Beforenewly created data is replicated from the source to the target location, the replication engine will check tosee what deduplicated data already exists at the target site and only send the unique bits of the file acrossthe WAN, greatly reducing bandwidth and transfer time.


9/12



For those who need the maximum file retrieval performance available, StorNext Deduplication allowsapplication and user access to the file before the entire file is completely retrieved. Similarly, utilizing SolidState Disk (SSD) storage devices to house the block pool index can optimize performance.

For customers who utilize StorNext Deduplication as part of a tiered storage methodology with StorageManager, file retrieval will occur from the Deduplication block pool before retrieving from StorageManager. The only reason files would be retrieved from Storage Manager is if the block pool is unavailable.This increases the performance of file retrievals while maintaining the highest possible file preservationtechniques. Protection of the blockpool is achieved through the use of StorNext Replication.

Affinities

The simplest example of a tiered storage practice using the StorNext File System is the use of Affinities.As mentioned before, StorNext can simultaneously manage multiple types of storage (FC, iSCSI, SAS, and

SATA) from different vendors under one namespace. Affinities are used to steer data within the file systemto a specific stripe group which is made of one or multiple LUNs. Assigned at a directory level, Affinitiescan be used to steer data to the most appropriate device. An example is steering very important, highthroughput files to a FC disk volume while temporary files are stored in a directory that is tied to SATA disk.Users and applications do not see this operation: they simply see the directory structure of the file system,thus reaping the benefit of a single namespace combined with cost-optimal data location.

The example below shows various data-flows in a StorNext environment (See Figure 8). In this exampledata is first stored in directories 1 and 2. Affinities are steering data stored in Directory 1 to StorageGroup A, which is a high-performance, mirrored FC Storage System, while data in Directory 2 is stored onStorage Group B, which is lower-cost SATA-based storage system.

PRIMARY TIER

Vendor Agnostic

SECONDARY TIER

DEDUPLICATED

REPOSITORY

StorNext

PolicyEngine

Vendor Agnostic

Vendor Agnostic

StorNext

SAN Clients

Figure 7

SINGLE NAMESPACE

STORAGE GROUP A

STORAGE GROUP B

FILE SYSTEM A

Directory 1

Directory 2

StorNextClientStorNextCl ent

Figure 8


10/12



StorNext Storage Manager Intelligent Archiving and Data Preservation

StorNext Storage Manager is an optional software component of StorNext that provides advanced datapreservation services. Storage Manager is responsible for automated policy-based transparent datamovement as well as capacity management of storage devices. Storage Manager accomplishes this throughan integrated policy engine which administrators can easily customize based on specific needs.

Policies are assigned at a directory level allowing users to create different tiered storage strategies based onunique file and project requirements within a single namespace. For instance, certain files could be copiedto tape for data protection but never removed from primary disk because of their need for continuous use.Other files might be copied to SATA or NAS disk and then truncated (removed from primary disk so only thecopy remains) after 30 days of inactivity to create a tiered storage practice. Other files may never be copied(e.g. temp files) and would be cleared out by users as needed. Storage Managers flexible features are usedfor moving files to tiered storage for cost reduction, data protection and long-term archiving.

Additionally, when a file is retrieved from the archive, it can be placed on a primary disk tier that isdifferent than where it was originally located. Optionally, StorNext has the ability to leave up to 2TBof stubs files (headers) on the primary storage device for files that have been truncated. This providesimmediate access from the beginning of a file for applications expecting header information to beretrieved at primary disk speeds.

Data movement policies occur in a the following process:

Create target destination copy(ies) of file at a designated time Mark copy of file on primary disk as candidate for truncation (removal) Remove file from primary storage so only archive copy(ies) remain Future access of the file will trigger a transparent retrieval of the file from tiered storage or the archive

StorNext Distributed Data Movers

In most software archive architectures there often exists some sort of bottleneck when moving data (I/O)to and from the archive. These architectures are problematic causing unknown server sizing as wellas limitations to load level as performance needs grow. In Storage Manager, the actual data movement

is handled by a Distributed Data Mover (DDM) which acts as the conduit for passing data to and fromprimary storage to the storage tiers (see Figure 9). As the archive becomes more active, you can simply addmore DDMs to scale the system performance. These movers enable reads and writes of file copies for datapreservation and retrieval while still providing fast access to data via the copy on primary disk.

TAPE LIBRARY ARCHIVE

PRIMARY TIER

SECONDARY TIER

DDM

Server

SAN

Client

StorNext

SAN Clients

with DDMs

DDM

erver

MDC &

DDM

SAN

Cl ent

DDM

Server

Sto

SAN

witMDC &

DDM

s

Next

Client

DDMs

DD

er

DDM

ServerFC

FC

FC

FC/iSCSI

FC/iSCSI

Figure 9


11/12


12/12


For contact and product information,visit quantum.com, call800-677-6268 or sendan email [email protected]

Preserving the Worlds Most Important Data.Yours.

2010 Quantum Corporation. All rights reserved. Quantum, the Quantum logo, and all otherlogos are registered trademarks of Quantum Corporation or of their respective owners.

About Quantum

Quantum Corp. (NYSE:QTM) is the leading global storage company

specializing in backup, recovery and archive. Combining focused expertise,

customer-driven innovation, and platform independence, Quantum

provides a comprehensive range of disk, tape, media and software

solutions supported by a world-class sales and service organization.

This includes the DXi-Series, the first disk backup solutions to extend

the power of data deduplication and replication across the distributed

enterprise. As a long-standing and trusted partner, the company works

closely with a broad network of resellers, OEMs and other suppliers to

meet customers evolving data protection needs.

Frequently asked questions

Where can I find more information on StorNext?Check out Quantums StorNext website at http://www.quantum.com/StorNext/

Where can I download a StorNext Evaluation?Check out Quantums StorNext website at http://www.quantum.com/StorNext/

Where can I download product documentation and data sheets?Check out Quantums Product Information website athttp://www.quantum.com/ServiceandSupport/SoftwareandDocumentationDownloads/Index.aspx

Where can I find operating system and tape library/drive compatibility information?Check out Quantums Compatibility Guides athttp://www.quantum.com/ServiceandSupport/CompatibilityGuides/Index.aspx

What primary disk storage systems does StorNext support?StorNext is disk agnostic supporting many different vendors FC, iSCSI and NAS attached diskstorage systems.

What storage virtualization features does StorNext offer?StorNext offers a variety of storage virtualization features, including:

Spanning file systems across multiple, heterogeneous storage systems Dynamically move data from one to another storage system while data is being accessed Dynamically replace out-of-service storage systems while data is being accessed Dynamically enlarge file systems to incorporate increased capacity or additional storage systems Automatically and transparently move data between on-line and near-line storage tiers, including

disk and tape All clients have access to the same data at the same time, regardless of Operating System and

connectivity type

How does StorNext work with NAS devices?While some data might be suitable to reside on NAS for primary data access, StorNext as a highperformance file system leverages NAS devices as tiered storage through policy-based movement.

Additional Information

For additional details on StorNext File System and StorNext Storage Manager, including integration optionsfor your environment, consult Quantums Software Architecture Group or contact the Software team [email protected].

Quantum Stornext 4

Documents

Transcript of Quantum Stornext 4