SYSVOL Replication: FRS or DFS-R??? Rhonda Layfield [email protected] Contents Copyright Rhonda J....
-
Upload
eugene-lester -
Category
Documents
-
view
219 -
download
1
Transcript of SYSVOL Replication: FRS or DFS-R??? Rhonda Layfield [email protected] Contents Copyright Rhonda J....
SYSVOL Replication: FRS or DFS-R???
Rhonda [email protected]
Contents Copyright Rhonda J. Layfield 2009
Rhonda Layfield
• IT industry 25+ years• NT/2000/2003 MCSE, MCT, MCSE+Security• Contribute articles to Redmond and
Windows IT Pro magazines• Setup and Deployment MVP• Desktop Deployment Product
Specialist (DDPS)• Co-Author of 2 Mastering Windows Server
books (2003 & 2008)
SYSVOL
Uptown DC Downtown DC
LScript.vbsLScript.vbs
Prehistory to Today• Always been a need for a tool to keep two or
more server’s folders in sync• NT -> LMRepl (Directory Replication)• 2000 introduced FRS (2003 used this also)
• Sysvol• DFS
• 2003 R2: a new Replication engine• Sysvol uses “old” FRS• R2’s DFS – which is called “DFS Namespace” – uses the
“new” Replication engine called “DFS Replication” or “DFS-R”
• Server 2008 can be configured to use DFS-R for SYSVOL replication
Sysvol• Created during dcpromo on domain
controllers• Automatically shared• C:\Windows\SYSVOL\sysvol\
Bigfirm.com (DNSDomainName)• Group Policy Template (Settings)• System Policies• Scripts (NT & AD)
• Must reside on an NTFS volume
FRS Sysvol Terminology
DC 1
DC 2
Upstream Partner
DownstreamPartner
Direct Replication
DC 3
Transitive Replication
Upstream Partner
DownstreamPartner
FRS Process (10,000 foot view)• NTFS logs changes to the “NTFS Change
Journal” • FRS monitors NTFS Change Journal• FRS places changed files in staging area• Replication partners notified
“something’s changed”• Partners request replication• Files are transferred
NTFS Change Journal (aka USN Journal)
• What does it do?Logs all changes to an NTFS volumeSeparate log on each NTFS volume
• Doesn’t it take a lot of space?• Sure; in fact it would eventually fill up a
drive• So Microsoft limits its size
So how large is it, and can I change it?• W2K SP2 32 MB• W2K SP3 512 MB• 2003 pre-SP1 128 MB• 2003 Hotfix 823230 or SP1/R2 512
MB• Server 2008 512 MB• Maximum size is 2 TB• MS Recommends increasing by 128
MB for every 100,000 files/folders
What happens when the change journal fills up?• Then NTFS just goes back up to the
top and starts overwriting the oldest entries
• So… if FRS hasn’t checked in with the change journal in a while, then FRS may get lost
• This is called a “journal_wrap” Ooh No!
FRS monitors NTFS Change Journal
NTFS Change Journal
1 GUID1
2 GUID2
3 GUID3
4 GUID4
Received 1-4
NTFRS FileIDTable
1 GUID1
2 GUID2
3 GUID3
4 GUID4
FRS monitors NTFS Change Journal
NTFS Change Journal
1 GUID1
FileRef#: 0x000f000000003a6f
USN: 0x000000000034cf40
NTFRS FileIDTable
GUID1
FID4222124650674799
USN
3460928
Tying the Change journal and FRS database together• FileRef# & USN in the NTFS Change Journal – NTFS Utility:
FSUtil• FSUtil USN ReadData C:\WINDOWS\SYSVOL\sysvol\
Domain\Policies\GUID1Major Version: 0x2Minor Version: 0x0FileRef#: 0x000f000000003a6fParent FileRef#: 0x0002000000002f45Usn: 0x000000000034cf40Time Stamp: 0x0000000000000000 12:00:00 AM
1/1/1601Reason: 0x0Source Info: 0x0Security Id: 0x308File Attributes: 0x20File Name Length: 0xeFile Name Offset: 0x3cFileName: GUID1
Dump GUID1 from FRS FileIDTable
wmic /namespace:\\root\microsoftdfs path dfsridrecordinfo where filename=“GUID1"
Attributes: 32Clock: 20060906155126.906250-000CreateTime: 20060906155119.203125-000Fence: 3Fid: 4222124650674799FileHash: 0000000000000000 0000000000000000FileName: GUID1Flags: 1GVsn: {55FDBBB9-0E2C-495C-8416-
7CE2706D62C7}-v1468Index: 729ParentUid: {3AB160AD-E505-492F-9C3B-09382DDB0CCC}-v1ReplicatedFolderGuid: 3AB160AD-E505-492F-9C3B-09382DDB0CCCUid: {55FDBBB9-0E2C-495C-8416-
7CE2706D62C7}-v1467UpdateTime: 20060906155126.937500-000Usn: 3460928Volume: \\.\C:
And So The Journal Begins…
NTFS Change Journal
1 GUID1
2 GUID2
3 GUID3
4 GUID4
NTFRS FileIDTable
1 GUID1
2 GUID2
3 GUID3
4 GUID4
Journal Wraps - Good
NTFS Change Journal
5 GUID5
6 GUID6
7 GUID7
4 GUID4
NTFRS FileIDTable
1 GUID1
2 GUID2
3 GUID3
4 GUID4
Journal Wraps - Good
NTFS Change Journal
5 GUID5
6 GUID6
7 GUID7
4 GUID4
5 GUID5
6 GUID6
7 GUID7
NTFRS FileIDTable
1 GUID1 2 GUID2 3 GUID3 4 GUID4
5 GUID5 6 GUID6 7 GUID7
Journal Wraps gone Bad
NTFRS FileIDTable
1 GUID1
2 GUID2
3 GUID3
4 GUID4
NTFS Change Journal
5 GUID5
6 GUID6
7 GUID7
8 GUID8
FRS Database is lost!
This is a Journal Wrap
Error
Journal Wrap Error
• Likely Causes• Turning off FRS for an extended period
of time• Disk intensive DCs
• Fixes• Increase NTFS change journal size• Self-Healing • Non-Authoritative Restore
Non-Authoritative Restore• “Flush everything in Sysvol and ask
my upstream partner for its entire Sysvol”
• Stop ntfrs.exe• HKLM\System\CCS\Services\ntfrs\
Parameters\Backup/Restore\Process at Startup
• Modify existing REG_DWORD entry Burflags, set to hex D2
• Start ntfrs.exe
Non-Authoritative Restores Automagically • W2k or W2k SP1– only happens
“manually” – when you set BurFlags=D2• W2k SP2 – occurs automatically whenever
a journal wrap is detected, no Reg hacking required
• W2k SP3 – Back to manual-only but a Reg entry will make it happen automatically again• HKLM\System\CCS\Services\ntfrs\Parameters
• “Enable journal wrap automatic restore” key to 1
• 2003/2008: MS says not to make automatic (KB 292438)
FRS keeps track of things via a database…• Database lives in C:\Windows\ntfrs\
jet\ntfrs.jdb• Consists of 5 tables• Connection Record table• Version vector table• File ID table (“IDtable”)• Inbound log (“inlog”)• Outbound log (“Ntfrs Outlog”)
Viewed using
Ntfrsutl
How FRS Handles New or Modified Files• FRS classifies files as new, modified or
deleted• Handles new and modified files similarly• FRS creates a Change Order (CO) in the
“Inbound log” table• FRS uses the Backup API to create a
compressed copy of the file in a “staging area” folder
• FRS creates a Change Order (CO) in the “Outbound log” table
• FRS creates a new entry in the IDTable table… but only for new files
FRS Database ntfrs.exe
GPO created
In Log
CO (C)GPO
Out LogCO (C)GPO
NTFS CJ
GPO – (C)
IDTable
GPO 123456748USN
DC
GPO
Partner Notification• Come and get it!!!!• KCC AD Connection Objects & Site
Links• FRS polls AD at startup (and every 5
minutes after) to check for list of replication partners
• Ntfrsutl poll /now (forces polling)• Ntfrsutl poll /now Computer (forces
polling remotely)• Intra-Site (immediate replication)• Inter-Site (replication schedule)
FRS Database
GPO created
In Log
CO New GPO Out Log
CO New GPO
NTFS CJ
GPO - Create
Change Notification
Psstt…I have changes
GPO
FRS Database
In Log
CO New GPO Out Log
CO New GPO
GPO
DC 1 DC 2
Entire File is Copied!!
FRS Issues
• Relocating the FRS database & logs• Relocating SYSVOL• Authoritative Restore• Morphed files/folders• FRSDiag• Ultrasound• Sonar
Relocate FRS database & log files• Stop FRS (net stop ntfrs)• Copy ntfrs folder to new destination• Modify Working Directory to reflect new path
• HKLM\SYSTEM\CCS\Services\ntfrs\Parameters• Confirm Administrator/System accounts have full
control to:• ntfrs• ntfrs\Jet• ntfrs\Jet\Log• ntfrs\Jet\Sys• ntfrs\Jet\Temp
• Start FRS (net start ntfrs)• Verify Inbound and Outbound replication with
partners
Relocate Sysvol
• Confirm replication is working properly
• Dcpromo down (demoting)• Wait for the removal of the DC
from all DC’s (NTDS file system settings object deletion propagates, ADSS)
• Dcpromo back up (re-promoting)• KB 842162 (manual – may the force
be with you!)
Sysvol Authoritative Restore• When would I do this?
• When many DC’s SYSVOL are inconsistent• Multiple DCs have journal_wrap errors
• How do I do it?• Stop FRS on ALL DCs• Select one DC to be the reference machine (this copy of
SYSVOL will be copied to all other DCs)• On the reference machine copy all folders/files that should
reside in SYSVOL to a temporary folder• Restart FRS on the reference machine with the Burflag set to
D4 HKLM\SYSTEM\CCS\Services\ntfrs\Parameters\Cumulative Replica Sets\GUID
• Restart all other DCs with the Burflag set to D2• On the reference machine copy files/folders from the
temporary location into the root of SYSVOL• Monitor that replication is consistent• Detailed steps KB 315457
FRS debug logs
• C:\Windows\Debug• ntfrs_0001.log - ntfrs_0005.log
• FRS Transactions & event details• Settings range from 0-5 (5 highest – most information
logged)• Default setting is 2• Log detail controlled by HKLM\System\CCS\Services\
ntfrs\Parameters• Debug Log Severity• Debug Log Files
• Can also be set using FRSDiag• Must stop and start ntfrs.exe
FRS Conflicts!Morphed Files/Folders• File/folder created on 2 different DC’s in the
exact same folder with the same name prior to a replication cycle
• When replication occurs – the inbound file will be renamed• Logon.vbs becomes
Logon.vbs_ntfrs_0001ab39• How does this happen?
• Administrators are attempting to make data consistent with manual copies
• A server’s FRS was not stopped prior to an authoritative restore on another server
• During an authoritative restore ONE server did not have it’s BurFlag set to D2
Resolve Morphed files/folders• Rename the original file/folder and the
morphed file/folder to different names• Wait for this to propagate to all servers• After propagation – choose the
file/folder you would like to keep and rename it back to the original name
• Delete all unwanted copies of the file/folder
• KB 328492
Tools• FRSDiag• Dump event logs• Find members GUID numbers
• Ultrasound• Requires a SQL database• Installs WMI providers on each DC• Polls DCs at defined intervals on FRS status
and places information in a SQL database• Configure Ultrasound to email or log an
event in the application log whenever an error condition is detected
Sonar
• Sonar-d.htm • installs with Sonar & is a great document on
troubleshooting FRS• Specific event ids which will help in resolving issues
• Find out which servers FRS service have been disabled or are not running
• SYSVOL not shared• Staging area full• Staging files being regenerated• Burflags set• NTFS change journal size• Morphed folders/files
Replicating Sysvol via DFS-R• All DC’s must be running Server
2008 • DFS-Replication can be managed
from:• XP-Pro SP2 workstation• Server 2008• Vista
DFS-Replication Terminology• Replication group - A set of servers, called
members, that participate in replicating one or more folders.
• Replicated folder - sysvol• Connection topology - Which members replicate
with other members.• Schedule - When replication is available.• Upstream partner - The partner who sends the
notification that it has changes for a replicating partner.
• Downstream partner - The partner who received the notification from an upstream partner and initiates replication.
DFSR and Journal Wraps• DFSR also monitors the NTFS change
journal• What’s different? DFSR always heals
itself
• And here is how…
Self-Healing Journal Wraps• DFS-R stops processing changes
from the NTFS change journal• Replication Stops• All entries in the DFS-R FileIDTable
receive a JWED (Journal Wrapped) flag
• The Directory Walker thread (DIRW) compares all Update Sequence Numbers (USNs) between the NTFS change journal and the DFS-R database
Upon Comparison - 3 Possible Conditions1. If the USNs in the NTFS change journal and the
DFS-R Database are the same:• DFS-R has the latest changes and removes the JWED
flag2. If the USNs are NOT the same:
• The USN in the NTFS change journal is incremented• The JWED flag is cleared from the DFS-R Database• The file will be replicated normally
3. Once all files in the DFS-R Database have been compared to the NTFS change journal and their JWED flags are cleared• One last scan is performed to find any files still flagged
JWED• If a file was deleted from the NTFS change journal, but
still exists in the DFS-R database. The files are tombstoned and will eventually be deleted from the replication process.
Do I have all the changes?• Version Vectors (VVs) are compared
Houston
Dallas
Server VVHouston 20
Dallas 30
Server VVDallas 31
Houston 20
RDC in Action!
Data (16 Bytes)
MD4
Row Row Row 27your house 42gently dow 17
Houston DC
Dallas DC
Data (16 Bytes)
MD4
Row Row Row 27
your house 42
gently dow 17
RDC in Action!
Data (16 Bytes)
MD4
Row Row Row 27your house 42gently dow 17
Dallas DC Houston DC
Data (16 Bytes)
MD4
Row Row Row 27
your boat g 35
ently down 22
What if there is a Conflict?• What causes a conflict?• The same file/folder (same UID) being
modified on two different servers, within one replication cycle
• A file/folder is created on two different servers in the same folder with the same name (different UID), within one replication cycle
File and Folder Conflicts Resolved• File with the Same Name & UID• Last writer wins (based on time stamps)
• File with the Same Name but different UID• First created wins (based on time stamps)
• Folder with the Same Name and UID• First created wins (based on time stamps)
• Folder with the Same Name but different UID• The contents are consolidated
Performance Monitor• DFS Replicated Folders• Bandwidth Savings• Number of conflicts that have occurred
• In bytes, or number of files/folders• RDC information• Staging Files
• DFS Replication Connections• Number and size of files replicated
• DFS Replication Service Volume• Number of entries read and accepted from the
NTFS change journal (USN Journal)
Why Use DFSR vs FRS?• FRS silently fails if the volume
SYSVOL resides on < 1GB of free space
• Copies changes (RDC) not entire files
• Version Vector tables• Journal Wraps are self-healing• Morphed files and folders
automagically taken care of
Migrating Sysvol Replication to DFS-R• Pre-Migration• Migration• Demo• PDC EM - Server 2003 that’s been
upgraded to 2008• Or a 2008 / 2008-R2 Server• Domain mode = Server 2008
Pre-Migration• On 2008 Server (PDC)• Pop in the Server 2008 DVD
• Adprep /forestprep from Sources/Adprep folder
• Raise domain functional level to Server 2003• Active Directory Domains and Trusts
• From Server 2008 DVD• Adprep /domainprep
• Upgrade PDC & all DCs to Server 2008• Raise domain Functional level to Server 2008• Backup SYSVOL
Migration Process has 2 Types of States• Stable states• Processes are complete• Can Rollback to a point
• Transitional states• Processes are in a working state
Stable State
T
Migration Process
START(State 0)
PREPARED(State 1)
RE-DIRECTED(State 2)
ELIMINATED(State 3)
4 5
6
7
Meet DfsrMig.exe• DfsrMig.exe lives in Windows\
System32 on Server 2008• Run dfsrmig from the PDC Emulator• State 0 - All DCs begin here• Replicating SYSVOL via NTFRS
• State 1 “Prepared”• Dfsrmig /SetGlobalState 1
• To confirm a state has been set…• Dfsrmig /GetGlobalState
What Happens…
• New Windows\SYSVOL_DFSR on all DC’s• Contents of Windows\SYSVOL copied to new
SYSVOL_DFSR folder• Windows\SYSVOL_DFSR
• domain & sysvol folders
• Netlogon share still points to Windows\SYSVOL\sysvol\Bigfirm.Com\Scripts
• SYSVOL share still points to Windows\SYSVOL\sysvol
• Both SYSVOL and SYSVOL_DFSR are being replicated
ADUC
• Advanced View• System• DFSR-GlobalSettings is created
• Domain System Volume (Replication Group)• DFSR-Replication Group
• DFSR-Content – SYSVOL Share• DFSR-Topology – List DCs
Re-Directed State…
• Dfsrmig /SetGlobalState 2
SYSVOL_DFSR
UptownDC
DownTownDC
DFS-RSYSVOL_DFSR
• Dfsrmig /GetGlobalState• Current DFSR global state:
Redirected
SYSVOL
SYSVOLFRS
SYSVOL_DFSR & SYSVOL out of sync…• The original copy of SYSVOL to
SYSVOL_DFSR was performed by Robocopy
• This copy is only done once• If you need to run it again you’ll
have to do it manually
Copying SYSVOL
• ROBOCOPY c:\Windows\Sysvol\Domain c:\Windows\Sysvol_DFSR\Domain /Copyall /MIR /B /R:0 /XD “Do_Not_Remove_NtFrs_PreInstall_Directory” “DfsrPrivate” “NtFrs_Prexisting__See_Eventlog” “ NTFRS_CMD_FILE_MOVE_ROOT” /XF “DO_NOT_REMOVE_NtFrs_PreInstall_Directory” “DfsrPrivate” “NtFrs_PreExisting__See_Eventlog” “NTFRS_CMD_FILE_MOVE_ROOT”
• Windows\Debug\SYSVOL_DFSR-Robocopy.txt
Re-Directed State
• Redirects the SYSVOL share to the new SYSVOL_DFSR folder
• HKLM\Sys\CCS\Services\Netlogon\Params
• Sets SysvolReady to False• Sets Sysvol = C:\WINDOWS\
SYSVOL_DFSR\sysvol• Sets SysvolReady to True
Migration Process
START(State 0)
PREPARED(State 1)
RE-DIRECTED(State 2)
ELIMINATED(State 3)
4 5
6
7
X
The Eliminated State• Dfsrmig /SetGlobalState 3• Deletes the NTFRS replica set from
AD• Deletes the old SYSVOL folder• Leaves NTFRS on• There is no going back!
Health Report
Create a Diagnostic Report• Health Report• Propagation Test• __DFSR_DIAGNOSTICS_TEST_FOLDER__• FRSRIP@A13948E4-5E44-483A-B56B-
65A075C446C0@Domain System Volume-SYSVOL Share.xml
• Propagation Report
Create a Diagnostic Report using
• The following example will generate a health report that:• Gathers information ALL servers in the sysvol
replication group.• Houston is the Reference Member. • The report will be named HealthReport.html stored in
the C:\Reports folder of the local machine.
• DfsrAdmin Health New /RgName: ”Domain System Volume” /RefMemName:Bigfirm\Houston /RepName:C:\Reports\HealthReport.html /FsCount:true
More Diagnostic Reports• The following example will generate a
health report that:• Gathers information only from Dallas• Houston is the Reference Member for the
sysvol replication group. • The report will be named DallasHealth.html
stored in the C:\Reports folder of the local machine.
• DfsrAdmin Health New /RgName:”Domain System Volume” /MemName:Bigfirm\Dallas /RefMemName:Houston /RepName:C:\Reports\DallasHealth.html /FsCount:true
The last Diagnostic Report… I promise• The following example will generate a
health report that:• Gathers information from all Servers listed in
the C:\Servers.txt file for the sysvol replication group.
• Houston is the Reference Member. • The report will be named ServersHealth.html
stored in the C:\Reports folder of the local machine.
• DfsrAdmin Health New /RgName:”Domain System Volume” /MemberListFile:C:\Servers.txt /RefMemName:Bigfirm\Houston /RepName:C:\Reports\ServersHealth.html /FsCount:true
In Summary
• FRS – RIP• DFS-R Rocks!• DFSRMig Works!
Hope you Enjoyed This Session
Please Don’t forget to fill out your evaluations
• Thank You!