DEP393Enabling File Server Consolidation with DFS and Microsoft® Windows Server™ 2003

Intelligent File Services

Brian DeweyLead Program Manager, Core File ServicesMicrosoft Corporation

File Server ConsolidationDo More With Less

What do you have today?Probably a lot of NT4 file servers

Where do you want to go?More availability, performance

Fewer servers, less TCO

High-level consolidation process:Size your new consolidated file servers

Move the files

Let people find the files

Plan for availability & recoverability

1. File Server SizingDisk and Memory

Maximum volume sizeDepends upon size of recovery window & backup device speed

Maximum number of files per volumeLargest factor affecting chkdsk time

See Veritest report for data points

Common max volume size: 200-500 GBVolume mount points mean terabytes per server

Memory: More is betterServer connection preallocation: See Q317249

Server memory usage: See Q312362

CHKDSK

Reminder – NTFS is a mature journaling file systemNTFS uses database-like logging techniques in order to remain consistent, even in case of a power fail or crashCHKDSK required only if

hardware does not honor ‘write-through’ of the journal fileHardware corrupts data in some wayThere’s an NTFS bug – hence the emphasis on ‘mature’

CHKDSK in Windows Server 2003 CHKDSK performance gains

See http://www.veritest.com/clients/reports/microsoft/mschkdsk.pdf...showing > 1271% performance gain in their largest test configurations.

Online CHKDSK filesystem verificationIn Windows Server 2003 RESKIT: ‘verfydsk.exe’Uses VSS to create a shadow copy of volume

Consider 64-Bit Servers

32 bits → 2 GB kernel address spaceLimits number of open files

Limits number of network connections

Limits cached file views

64 bits removes all of these limitsKernel “paged pool” goes from 540 MB to 241 GB

No need to wait for 64-bit applications64-bit antivirus products available

Slower clock speeds not an issue

Customer Examples

Customer example (clustered file server):Each node has 4 x 700Mhz CPUs, 4GB RAMEach node has 11 x 170GB volumesAntiVirus software~3000 active users per node, ‘no perf issues’

Tuned per Q312362 to avoid ‘2020’ event messages

Customer example (non-clustered):“great reliability, so no need to cluster”0.5 GB RAM*

Note – 4GB would be better: (a) file cache, (b) CHKDSK

1TB of disk3rd-party Quota product~1500 active users

2. Move the Data“A Miracle Occurs”

Several tools to choose fromXcopy, robocopy, richcopy

Backup / restore

3rd party replication

Be sure to verify permissions before / after

Shares now read-only by default

“Effective Permissions” makes this easier

3. Locate the Files

Congratulations! Your files are consolidated on new servers.Umm… how are people supposed to find them?

Send them emailChange drive letter mappingsDFS

What? You don’t use DFS?A consolidation effort is a great time to plan and deploy DFS

Benefits of DFS Virtual Paths

DFS provides virtual pathsThese paths are independent of the physical resources backing these paths

Administrator-defined

Example:\\corp\docs\2002

\\corp\docs\2003

\\corp\tools\sales

\\corp\tools\engineering

\\corp\projects\foobar

DFS

demodemo

DFS For ConsolidationNew for Service Pack 1

Consolidated Storage

Legacy Name Server using DFS

Client

Existing S

torage

\\oldserver1\public

\\oldmarketing\public

\\newdfs\public

\\newdfs\marketing

\\legacy\oldserver1\public -> \\newdfs\public

\\legacy\oldmarketing\public -> \\newdfs\marketing

After decommissioning existing storage, a single DFS server will be able redirect all of the old names to the new location!

Multi-

host

Microsoft DFS - Architecture

DFS ServerDFS Server

DFS

DFS ClientDFS Client

NetwareNetware

WindowsWindows

LAN or

WAN

UnixUnix

Multi-protocol– Uses SMB/LM protocol to

communicate between DFS client and DFS server

– Then, uses any underlying remote filesystem protocol to connect to remote share

SMB/LM for Windows Anything client has a RDR

installed for (e.g. NCP for Netware, NFS for Unix Servers)

Secure– DFS only involved in name

redirection

– Existing security of underlying OS and file system is used to access shares

Benefits of DFSMultiple Link Targets

DFS Links can have Multiple targetsA ‘link’ is a mapping from the ‘virtual’ DFS path to one or more shares (known as ‘targets’)

Example:\\corp\docs\2002 -> \\docs1\2002, and \\docs2\2002

A DFS path can point to another DFS pathExample:

\\corp\sales\demos -> \\demosrus\public

Benefits of DFSTarget selection – load balancing, site selection

In case of multiple targets, the DFS server sorts targets in 2 phases:

Phase 1: Sort into groups by ‘site’Windows® 2000 Site selection

Group #1 is same site,

Group #2 is rest-of-the-world

Windows® Server 2003 ‘closest-site-selection’Groups #1 … #N sorted by AD-defined site costing

Phase 2: within each ‘group’, randomize orderProvides load balancing

DFS client behavior:Use the first available target in the referral list

Failover to next in list when necessary

Deploying DFSChoosing a DFS type: Domain or Stand-alone

Use stand-alone DFS namespaces whenNo Active Directory available:

Your organization does not use Active Directory, or A domain admin will not create a DFS in their AD

You need to create a single namespace with more than 5,000 links.

If you can divide your links among two or more namespaces, then domain-based DFS is an option

Use clustering to Ensure the availability of the stand-alone DFS namespace

Use domain-based DFS namespaces when You plan to use FRS to replicate data You want to ensure the namespace is available across multiple sites

Deploying DFSScaling Hints & Tips

Keep comment fields to a minimumThese take space in the DFS metadata & limit size

Cascade DFSesBuild layers of DFS for large namespaces

A top-level DFS, pointing to subordinate DFSes3-tiers is common for Large deployments

8 tiers is maximum supported

Use ‘root scalability mode’This is available for Windows® Server 2003

Enable using DFSUTIL /RootScalabilityDFS root servers will get updates from a local domain controller instead of the server acting as the PDC reduces network traffic to the PDC at the expense of tight consistency

Deploying DFSSite selection Hints & Tips

Site is only determined by IP address rangeWindows® 2000 Site selection hints:

Note that a target’s site is defined when it is added to the DFSTo refresh

Manually remove/add the targetOr, use DFSUTIL /refresh from Windows Server 2003

Windows® Server 2003 Site selectionNote that a target’s site is dynamically definedSite information in the DFS metadata is ignored

The Windows .NET Server DFS UI does not store it thereUse DFSUTIL /refresh if using Windows® 2000 DFS servers

“Closest site selection” mode requires Windows® Server 2003 on all ISTGs

4a. Availability

Different techniques for a highly available file service

1. Clustering2. DFS with FRS file replication,3. DFS + Storage on a SAN – reallocate to another

server in case of failureNeed to evaluate ways to make data highly available

Multiple NICsFor SAN, Multiple paths from file server to disks

(MPIO: multiple independent fiber channel paths)

Different RAID levelsOnline volume growth, backup

DFS FailoverDFS Failover

demodemo

Availability: ISV Driver Quality

You run anti-virus, quota, replication productsThese use ‘file system filter drivers’

These are complex drivers - how can you ensure quality?

Answer: MS Plugfests, better dev kits, logo testsPlugfests: Quarterly week-long ISV workshops in Redmond

8 weeks of events held since mid-1999

Focus on robustness, performance, transparency

WHQL Certification for Antivirus products:For Windows XP and Windows .NET Server 2003.

We’ve measured significant progressNo FS filters in Windows ‘OCA’ top 100 ISV bluescreens

Huge change from Windows NT 4.0

4b. Data Recovery

Windows Server 2003 goes far beyond backup/restore

Shadow Copies of Shared FoldersEmpower users to undo their own mistakes

Open File Backup Backup open files without extra device drivers

Available to backup products which use ‘VSS’

Automated System Recovery (ASR)Bare-metal restore of systems

Supported by NTBackup

Supported by 3rd-party backup products

demodemo

Shadow Copies of Shadow Copies of Shared FoldersShared Folders

Appendix…Appendix…

Deploying DFSChoosing a DFS type: Domain or Stand-aloneCharacteristic Domain-based Stand-Alone

Administrator access

Only Domain Admins can create new domain-based DFS roots. Local Administrators group members on each of the root targets can add/delete links or root targets.

Local Administrators group members on the local server can create new stand-alone DFS roots and add links to the roots.

Where DFS root information is stored

In Active Directory. In the registry of the root server.

DFS namespace size restrictions

5,000 links. 50,000 links.

Methods to ensure DFS root availability

Create multiple DFS root targets in the same domain.

Create stand-alone DFS root on a clustered file server.

Methods to ensure link target availability

Create multiple link targets and replicate files by using one of the following methods: Enabling FRS Copying files manually or by using scripts Using a third-party replication tool

Create multiple link targets and replicate files by using one of the following methods: Copying files manually or by using scripts Using a third-party replication tool

Deploying DFSSize limits

If you do not enable “root scalability mode”, Microsoft recommends using 16 or fewer root targets to limit traffic to the server acting as the primary domain controller (PDC).

No fixed limitNumber of root targets per domain-based DFS root

The size of the Active Directory object is determined by the number and path length of roots, links, comments, and targets in the namespace. Microsoft recommends using no more than 5,000 links in a domain-based namespace to prevent the Active Directory object from exceeding 5 MB.

5 megabytes (MB) Size of each DFS Active Directory object (applies to domain-based DFS namespaces only).

These two stores (Active Directory, Registry) have different restrictions on the size of their objects

5,000 for domain-based DFS50,000 links for stand-alone DFS

Number of links per DFS namespace

Windows® 2000 and Windows® Server 2003, Standard Edition:

•One namespace root per server.Windows Server 2003, Enterprise Edition or Windows Server 2003, Datacenter Editions

•No Limit.

VariesNumber of DFS roots per server

ExplanationRecommendation*Description

Shadow Copy Restore Client Side Server Side

Note: Flash demo posted on www.microsoft.com:http://www.microsoft.com/windows.netserver/techinfo/overview/overview6.asp

..and click on the ‘flash demo’ link for ‘Shadow Copy Restore’

Tools - VSSAdmin

Ships with Windows Server 2003Command line scriptable utility for setting up or configuring shadow Copies

C:\>vssadmin /?vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool(C) Copyright 2001 Microsoft Corp.

---- Commands Supported ----

Add ShadowStorage - Add a new volume shadow copy storage associationCreate Shadow - Create a new volume shadow copyDelete Shadows - Delete volume shadow copiesDelete ShadowStorage - Delete volume shadow copy storage associationsList Providers - List registered volume shadow copy providersList Shadows - List existing volume shadow copiesList ShadowStorage - List volume shadow copy storage associationsList Volumes - List volumes eligible for shadow copiesList Writers - List subscribed volume shadow copy writersResize ShadowStorage - Resize a volume shadow copy storage association

Tools – VolRestAllows restore of files without knowing path of deleted fileShips in the Resource Kit

C:\Program Files\Windows Server 2003 Resource Kit>volrestVOLREST 1.1 - Timewarp Previous Version command-line tool(C) Copyright 2003 Microsoft Corp.

Usage: VOLREST [options] FileName

Options are: /? - Displays this help. /A - Includes files with specified attributes. /AD Directories (only). /AS System files. /AH Hidden files. /B - Uses bare format (no heading information or summary). /S - Includes files in specified directory and all subdirectories. /R:<DirectoryName> - Restore all previous versions in target directory. /E - Restores empty directories (use with /R). /SCT - Decorates restored file names with the shadow copy timestamp. Use with /R. For example: "foo (Wednesday, January 01, 2003, 14.00.00).doc"

Examples: VOLREST Z:\MYDIRECTORY\MYFILE.DOC VOLREST \\server\share\MYDIRECTORY\*.DOC VOLREST Z:\*.* /s /r:C:\OLDFILES VOLREST Z:\*.DOC /s /r:C:\OLDFILES /SCT

Tools – Shadow Copy Performance Counters

Provides detailed information on Shadow copy metrics and measures

Nb of Shadow Copies

% of Diff Area Used

Used Space (MB)

Ships with Windows Server 2003 ReskitVolperf.exe /install

Ask The ExpertsGet Your Questions Answered

Find me at the Storage booth, July 3 from 1 to 4 PM

Community Resources

Community Resourceshttp://www.microsoft.com/communities/default.mspxhttp://www.microsoft.com/windows2000/community/centers/fileservices

Most Valuable Professional (MVP)http://www.mvp.support.microsoft.com/

NewsgroupsConverse online with Microsoft Newsgroups, including Worldwidehttp://www.microsoft.com/communities/newsgroups/default.mspx

microsoft.public.win2000.file_system microsoft.public.windowsnt.dfs

User GroupsMeet and learn with your peershttp://www.microsoft.com/communities/usergroups/default.mspx

evaluationsevaluations

© 2003 Microsoft Corporation. All rights reserved.© 2003 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.