Post on 02-Apr-2015
Impact of Cloning and Virtualization on Active Directory Domain Services
Dean WellsActive Directory Product GroupMicrosoft
SIM406
Session Objectives and Takeaways
Session Objective(s): Convey the technical challenges surrounding Windows & Active Directory in a virtual world
logistical and other valid concerns beyond scope for now
Highlight fundamental Windows & Active Directory concepts & assumptionsidentity, replication, time, etc.
Provide an understanding of the risks stemming from virtualization
Key Takeaways:Improved comprehension of…
core Active Directory and Windows components impacted by cloning & virtualizationbest practices when virtualizing DCs and domain memberswhat qualifies as “successfully cloning a Windows machine” and what doesn’t
Windows Concepts
Machine Identities
Computer-identity comprises…
NameStored locally, suffixed with a $
IP addressNetwork identifierName/IP information stored in DNS
SIDsWhat are these?What is that for?
What is a SID?
Protocol Documentation – Glossary:An identifier for security principals in Windows that is used to identify an account or a groupConceptually, a SID is composed of three parts:1. a SID prefix
revision (1 = revision 1) + an identifier authority (5 = NT Authority)
2. an account-authority portion (typically the domain’s SID) principals created in the same domain share the same prefix and authority-portion
3. an integer uniquely representing an identity relative to the account-authority commonly known as the relative identifier (RID)a 30-bit address-space (~1 billion principals per domain-lifetime)
S-1-5-21-2000478354-492864223-854245397-19221
SID assignment
Machine SIDsHow is it assigned? See [MS-SAMR], section 3.1.1.9.2How many individual SIDs does a computer serving as a domain-member have?
Domain SIDWhere does that come from?
SID UsageAuthorization
Deployment ScenariosLets walk through a few potential usage scenarios that, at first glance, may appear perfectly acceptable…
Deployment Scenarios
Pay close attention; this gets tricky…
Scenario 1
1. Start with a domain joined machine named M12. Clone it and boot-up the clone (e.g. copy its VHD)
• Can the clones co-exist?
3. What about if we “offline” unjoin the clone, rename it to M2 and join it (M2) back to the domain• And now?
Scenario 2
1) A template VHD file that is used to deploy new Windows servers is copied
4) Another copy is made
of the template VHD. It is renamed to LUIGI & joined to the PRINCESS domain
6) The PEACH\Administrator is added as a member of CHILD\SuperMarioBros
2) The cloned VM is renamed & promoted to a DC creating the PEACH domain
3) A child domain (PRINCESS) is promoted from a clean OS-install in a branch office
7) PEACH\Administrator logs on to a PEACH domain-member and tries to map a drive to:
\\luigi.princess.peach.com\Gameboy
What happens?
2k8r2.VHD
WindowsServer
5) CHILD\SuperMarioBros is granted READ/WRITE access to the Gameboy share on LUIGI
Scenario 3
1. Setup a machine M12. Clone M1 to get M23. Promote both in different
domains in different forests• Result: 2 domains share the
same SID space
4. Establish trust between the 2 domains/forests• What happens?
Computer: M1SID: S-10
Computer: M2SID: S-10
Forest1.comSID: S-10
Forest2.comSID: S-10
Trust?M1 is cloned M2
M1 & M2 promoted as first DCs in two forests
Scenario 4
1. Create domain from machine M1 (dom1.lab)2. Install a new machine M23. Clone M2 to get new machine: M34. Promote M2 as a replica in dom1.lab5. Join M3 to dom1.lab domain hosted by M1 and M2
• Anything wrong here?
Windows Concepts
Active Directory Replication
Update Sequence Numbers (USN)
What’s a USN?64 Bit QWORD Logical clock, per DC (USNs are local to a DC)Never re-used and SHOULD NEVER rollback
When are USNs assigned? (i.e. when does the clock tick?)Assigned to new objects / update transaction
if transaction is aborted USN skipped, remains unused
Independent from system time
Update Sequence Numbers (USN)
DS1
P1: 4711
Version#
<time>Value 1
Originating GUID
4711DS1
Property Value USN Timestamp Orig. USN
P2: 4711 <time>Value 1 4711DS1
P3: 4711 <time>Value 1 4711DS1
P4: 4711 <time>Value 1 4711DS1
Object usnCreated = 4711 Object usnChanged = 4711
Object creation & metadata
USN: 4710USN: 4711
• Add new user on DS1• DS1 USN increases to 4711• DS1 object metadata below
DS1
Object replication & metadata
USN: 4711
DS2
USN: 2051
P1: 2052
Version#
<time>Value 1
Originating GUID
4711DS1
Property Value USN Timestamp Orig. USN
P2: 2052 <time>Value 1 4711DS1
P3: 2052 <time>Value 1 4711DS1
P4: 2052 <time>Value 1 4711DS1
Object usnCreated = 2052 Object usnChanged = 2052
USN: 2052
• User replicated to DS2• DS2 USN increases to 2052• DS2 object metadata below
High Watermark vector table
Table per NC per DCMaintains
replication partners using DC’s DC-GUIDhighest known USN from last replication
Used to detect recent changes on replication partnersso that DCs only replicate that which changed since the last replication cycle
USN: 3388
DS4
USN: 1217
DS3
USN: 2052
DS2
USN: 4711
DS1
DC GUID Highest known USN
DS1 GUID 4711
DS3 GUID 1217
High Watermark vector table
• DS4’s high-watermark vector• assumes that DS1 and DS3 are its
replication partners
Database identity
Domain Controllers are machines with machine identitiesName, SID
Domain Controllers host a database with an identityInvocation ID, stored on NTDS Settings ObjectWhen is it assigned/updated?
Usage of the invocation IDReplication metadata (UTD Vector)
Up-To-Dateness (UTD) vector table
Table per NC per DCUsed to detect updates already received via another replication routeMaintains
originating DC’s invocation IDhighest originating USNtimestamp of last successful replication cycle
Which DCs have an entry in UTD vectors?
USN: 3388
DS4
USN: 1217
DS3
USN: 2052
DS2
USN: 4711
DS1
Invocation ID
Highest originating USN
DS1 GUID 4691
DS2 GUID 2052
Replication timestamp
12:02.31
12:02.29
DS3 GUID 1216 12:02.36
Up-To-Dateness (UTD) vector table
• DS4’s up-to-dateness vector• assumes that DS1, DS2 and DS3 have
all originated writes against the partition
Making the UTD vector “up-to-date”
DC2 initiates replication from DC1DC1 determines what changes to send:
Local USN higher than the one stored by DC2 in its high watermark tableOriginating USN higher than values in the UTD vector stored by DC2
At the end of replication:Increase DC2’s high watermark for DC1 to new DC1’s highest local USNDC2’s UTD vector becomes the max-merge of DC1 and DC2’s UTD vectors
Lingering Objects
An object on DC1 is lingering if:It is not present on DC2 that fully hosts the same NCIt is not “about to” be garbage collectedThe creation of that object is not part of any upcoming replication cycle
in other words, USNcreated on DC1 is lower than highest exchanged USN - as stored in High Watermark Vector for DC1 on DC2
Detection happens when DC2 receives from DC1 an update or deletion event for the object.
Events 1388, 1988
The fact that an object is lingering doesn’t necessarily make it “wrong”
USN rollback
What is a USN rollback?corresponds to the situation where a USN which had previously been allocated to an update gets re-used
Such a phenomenon breaks the strongest assumption made in our replication algorithmDetection:
DC2’s UTD vector indicates that it has replicated all originating updates from DC1 up to USN X1Next time DC2 pulls updates from DC1, DC1 “thinks” that its highest originating USN is X2<X1. Since DC1 realizes that it has previously sent out udpates with higher USN than what it’s currently using, it quarantines itselfEvent 2095
USN rollback detected
USN rollback
USN bubbles… how a USN rollback can turn really bad
USN rollback detected USN rollback NOT
detected!
Improper Backup/Restore
What can go wrong with an improper backup/restore?Summary of a real-world case:
2500 users not able to log onusers having access to resources they should not have access to anymoreschema mismatches after Schema Master rolled backExchange server failingRID pool allocated twice after RID master rolled back
Application – Backup/Restore
Resetting the invocation ID Use supported backup/restore solutions
VSS writers, whether in Windows backup or 3rd party solutions
Last resort option… (and not formally tested)before you apply the snapshot, disable the network adapters on the VMapply the snapshotset registry value Database Restored from Backup = 1rebootverify that the DC has a new invocation IDre-enable network adapters
Application – P2V migration
Is it enough to reset the invocation ID on the newly created Virtual DC?Online or offline P2V? Lab creation via P2V
What happens if various DCs are P2V’d at different times and placed in test network?
Recommendations:Use P2V in SCVMM, it has a few checks in placeReset the invocation IDDo not place physical and P2V’d VM on same network… ever!
Application – RODCs
Virtualization of RODCsCan I take snapshots of RODCs and use them?
Mostly but with various ramifications, e.g.lastLogon and other logon-statistics-attributes written only locally on RODC
Can I clone RODCs in a branch site?No
Miscellaneous Considerations
TimeSync, Security, Performance, Going all virtual, etc.
Time SynchronizationIf you have followed our existing guidance…
we’ve changed our minds documentation changes are on the way (or already published)
Windows Time Service has a well-defined algorithm for time synchronization within a domain (Domain Hierarchy)
let it do its thingand ensure the HyperVisor participates in the same timesync hierarchy
minimizes/eliminates large deltas in time
Are we suggesting you disable Virtual Machine Integration Services completely?
no… absolutely NOT! Virtual Machine Integration Services are still needed, e.g.
while the VM is booting or in the midst of other VM-specific operations such as Resume
Instead, disable the VMIC timesync provider in the guestKEY: HKLM\SYSTEM\CurrentControlSet\Services\W32Time\TimeProviders\VALUE: [REG_DWORD] VMICTimeProvider: 0 (NOTE: that’s a zero)
Security considerations
Hosts of domain controllers should be handled with same care as the DCs they host
Possible EoP from host administrator to Domain/Enterprise Admin
As possible, reduce attack surface on hostServer Core
A guest DC has admin privileges over domain members, including Hyper hosts, if joined to the domain
Possibility: make the host a DC
Performance considerations
In testing conducted in a W2K8 Hyper-V environmentVirtual DCs perform at about 90% compared to physical DCs
Is that still true?No, virtualization technologies improve. We’re now almost at par
assuming, of course, that the host isn’t running too many VMs
Going all virtual – a good idea?
Key: Avoid single points of failuresSame messaging for the past 10 years
Do not place all your DCs on the same hostwe have seen this
Diversify host’s hardware if possibleoftentimes, this is simply not realistic, but it remains optimal nonetheless
Maintain 1-2 physical DCs per domain?as above
Others
Disk Write Caching (FUA)Disk write caching setting on guest is honored by the host
Machines running hotHost running 5 VMs gets (too) hot and shuts down VMs
Antivirus Runs on the host, “locks” VM files (cannot boot)KB 961804
Snapshots and host’s disk space What if a snapshot takes up the whole disk?What if snapshot files improperly deleted?
Recap
Complete an evaluation on CommNet and enter to win!
Recap
Cloning non Domain Controllers?Perhaps, risks for 3rd-party software remain an unknown quantityBest Practice: SYSPREP instead
Cloning Domain Controllers?ABSOLUTELY NOT!What if it’s the only DC in the entire forest? Still a concern:
it won’t naturally replicateWhat happens to apps that understand the replication fabric, etc.
HyperV host snapshotting on Domain Controllers guestsWriteable: practically guarantees a USN rollback situationRODCs: perhaps… but untested the risks are undetermined
TimeSync in virtualized environmentsDisable the VMIC timesync provider within the guest
Questions?
Track Resources
Don’t forget to visit the Cloud Power area within the TLC (Blue Section) to see product demos and speak with experts about the Server & Cloud Platform solutions that help drive your business forward.
You can also find the latest information about our products at the following links:
Windows Azure - http://www.microsoft.com/windowsazure/
Microsoft System Center - http://www.microsoft.com/systemcenter/
Microsoft Forefront - http://www.microsoft.com/forefront/
Windows Server - http://www.microsoft.com/windowsserver/
Cloud Power - http://www.microsoft.com/cloud/
Private Cloud - http://www.microsoft.com/privatecloud/
Resources
www.microsoft.com/teched
Sessions On-Demand & Community Microsoft Certification & Training Resources
Resources for IT Professionals Resources for Developers
www.microsoft.com/learning
http://microsoft.com/technet http://microsoft.com/msdn
Learning
http://northamerica.msteched.com
Connect. Share. Discuss.
Scan the Tag to evaluate this session now on myTech•Ed Mobile
© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to
be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS
PRESENTATION.