www.novell.com

eDirectory™ In DeptheDirectory™ In DepthDuane BussSenior Software EngineerNovell, Inc.dbuss@novell.com

Tom DomanSenior Software EngineerNovell, Inc.tdoman@novell.com

Steve McLainSenior Software EngineerNovell, Inc.smclain@novell.com

Vision…one NetA world where networks of all types—corporate and public, intranets, extranets, and the Internet—work together as one Net and securely connect employees, customers, suppliers, and partners across organizational boundaries

MissionTo solve complex business and technical challenges with Net business solutions that enable people, processes, and systems to work together and our customers to profit from the opportunities of a networked world

Introduction

• Directory concepts and definitions• Object synchronization• External reference synchronization• Obituary synchronization• Schema synchronization• Agent configuration synchronization• Additional background processes• Database concepts and details• Security concepts and details

Concepts and Definitions

• General directory concepts Partition—A distinct portion of the directory tree that

stores and replicates directory information

Replica—A single instance of a partition Synchronization—The propagation of directory

information from one replica to another so the information in each partition is consistent with the other

Schema—Defines the types of objects that can be created in your tree (such as users, printers, and groups) and what information is required or optional at the time the object is created—every object has a defined schema class for that type of object

Background Process—A task or set of tasks that happens without user intervention to maintain directory information (such as synchronization)

Object Synchronization Concepts

• General directory terms and methods Convergence—The act of making an object

consistent among all instances of that object Hierarchy—A graded series of objects in which

each element may contain other objects Replication Styles—Single Instance, Master-

Slave, Multi-Master, Hybrids Replication Methods—None, Copy and Replace,

Change Log, State Based, Hybrids

• Novell eDirectory™ Multi-Master-Slave Hybrid, State-Based

Object Synchronization Concepts

• Novell eDirectory terms and methods Replica Types—Master, Read\Write, Read Only, Filtered Replica Ring—The set of eDirectory agents that hold a replica

of a given partition and, therefore, participate in the synchronization of objects contained with that partition

Deltas—Time based differences between copies of a given object

Change Cache—The collection of objects with deltas for a given replica

Transitive Synchronization—A method for providing convergence of data without requiring an eDirectory agent with changes (i.e., state based deltas) to directly contact and synchronize those changes with every other agent in the replica ring

Object Synchronization: Transitivity

eDirectory Agent

Server 1

eDirectory Agent

Server 2 eDirectory Agent

Server 3

Transitive Synchronization

ΔUp-To Ack

Up-To Ack

Communication

Object Synchronization:eDirectory Representation

• Partition Root Object Attributes Replica—A synchronizing multi-valued attribute where

each attribute value represents an eDirectory agent that is participating in replication of this partition and how it can be contacted

Transitive Vector—A synchronizing multi-valued attribute where each attribute value represents the state in time that the specified eDirectory agent has received changes up to

Local Received Up To—A non-synchronizing single valued attribute whose value represents the current state in time that the local eDirectory agent has received changes up to

Purge Vector—A non-synchronizing single valued attribute whose value represents the oldest state in time that has been seen by each eDirectory agent in the replica ring—uses the Transitive Vector to find the oldest state seen by all the agents in the ring for each replica in the ring

Local Received Up To (LRUT)

Preparing to Outbound

• Read the last TimeStamp issued on the local replica from the partition record

• Update the row in the Local Received Up To corresponding to the local replica number

• Update the Transitive Vector value corresponding to the local agent (“server”)

What is the Local Received Up To attribute used for?What is the Local Received Up To attribute used for?• To keep track of what changes have been received from other replicas

• To inform inbounding replica’s the current synchronization state

• To advertise the local state when the agent is ready to do so (outbound sync)

Partition Records

Transitive VectorTimeStamp = Date\Time, Replica Number, Event

- Changes to Send- No Changes to Send

When a synchronization cycle is started, the destination replica exchanges its LRUT with the source to determine the exact deltas that need to be sent in case some changes have already been received transitively

Change Cache

Values Needing Synchronization

eDirectory 8.6 Synchronization Improvements (Patents Pending)

• Incremental Replication of ChangesAllAll changes for the entireentire state difference between replicas of a given partition is still required, but a progress marker (“synchronization point”) is kept so that work is not lost and redone in the event of an error (usually communication) during a synchronization cycle

• Multi-Threaded OutboundThe outbounding eDirectory agent can update more than one agent for more than one partition at a time

• Reduced ChattinessCommunication of the Transitive Vector between replicas in no longer delayed until each replica’s outbound synchronization cycle—the destination replica’s Transitive Vectors are exchanged with the source replica at the end of a replication cycle

Per Replica Attribute Time Stamps no longer cause extra needless synchronization attempts

• More Efficient Object AnalysisImproved handling of large multi-valued attributes inbound and outbound

Incremental Replication

Window State (or Vector)—The state (in time) where the source replica is trying to move the destination replica up to—it is based on a configurable window size plus the destination replica’s current state (namely, the destination replica’s Local Received Up To)

Distributed Consistent Ordering of Objects—An ordering (such as a database index) that produces objects in the same order on every replica

Synchronization Pane Point—A representation of the following The set of orderings to be used to produce objects to be considered for synchronization Which ordering is currently being used to produce objects An element or “key” from the current distributed consistent ordering of objects that can be used to reposition to the location of that element in the distributed consistent ordering

Window Pane—A discrete non-state based unit of work—different units of measurement can be used to specify the size of a Window Pane. Examples include number of objects analyzed, number of objects sent, time spent analyzing and sending changes, current time within a specified time range (like WANMAN), number of bytes sent, number of attributes sent, all pertinent changes have been sent, etc.

Synchronization Point• Generated by the Source Replica

• Stored on the Partition Root Object of the Destination Replica

• Once established, able to be picked up and continued by any other Source Replica

Synchronization Configuration

• Pre 8.6 behavior—one thread in by partition mode

• Max threads and method are automatically chosen but can be overridden here

• Default max threads: 8

Multi-Threaded Outbound Synchronization

Deployed Versions Novell eDirectory and Novell Directory Services® (NDS®)

Product Version Build Version

Platforms

NetWare 5.1 SP4 (NDS 7) DS.nlm v7.57 NetWare 5.1

NetWare 5.1 SP 4 (NDS 8) DS.nlm v8.79 NetWare 5.1

eDirectory 8 DS.nlm & DS.dlm v8.79

NetWare 5.0,Win NT/2K

eDirectory 8.5.x DS v85.23 NetWare 5.x,Win,Solaris

NetWare 6 (eDirectory 8.6) DS.nlm v10110.20 NetWare 6

eDirectory 8.6.1 DS v10210.43 NW 5.1,NW 6,Win,Solaris,Linux

NetWare 6 SP1 (eDirectory 8.6.2)

DS.nlm v10310.17 NetWare 6

eDirectory 8.6.2 DS v103xx.xx NW 5.1,NW 6,Win,Solaris,Linux

eDirectory 8.7 DS v10410.xx NW 5.1,NW 6,Win,Solaris,Linux,AIX

Differences between eDirectory and Novell Directory Services (NDS)

NetWare 6

NetWare

NDS eDirectory

NOS directory focused on managing NetWare® servers

A cross-platform, scalable, standards-based directory

used for managing identities that span all aspects of the network—eDirectory

is the foundation for eBusiness

NetWare 5

External Reference Synchronization

What is an External Reference?What is an External Reference?A name needing to be tracked by the local DIB—it may contain a partial cache of attributes from the real object and/or results of local operations

What causes an External Reference to be created?What causes an External Reference to be created?1. Authentication, 2. It is referenced by another eDirectory object, 3. File rights or other OS dependency, 4. eDirectory itself has a dependency

How are they maintained?How are they maintained?Reference Check Process (AKA Backlinker and DRL processor)—On real replicas, this process maintains Uses, Used By, and Back Link attributes

What is maintained?What is maintained?Depends on the object and the version of eDirectory—the base class, name, and certain attributes are maintained. Some examples of attributes include Public Key and GUID for User objects, Replica for Partition Root objects, and Status and NDS Version for NCP Server objects

External Reference

Back Link

…

External References Synchronization

Why do I care about external references?Why do I care about external references?

1. If you have a lot of external references from one partition, you may want to consider placing a replica of that partition

2. They must be maintained properly for those subsystems that are dependant on them

3. They effect the amount and types of communication required between eDirectory agents

4. Referential Integrity

How can I tell if there are problems?How can I tell if there are problems?

NDS iMonitor—Agent Process Status

External Reference Status

Obituary Synchronization

What is an Obituary’s purpose?What is an Obituary’s purpose?To ensure referential integrity during delete, move, rename, etc. operations

What are the most common types?What are the most common types?

Primary—Dead, Restored, Moved, Inhibit Move, New RDN

Secondary—Back Link, Used By

How are obituaries processed?How are obituaries processed?

The Obituary Process—This is one of the processes that uses Purge Vector. It is not manually schedulable; it is automatically scheduled at the end of an inbound synchronization cycle for the just synchronized partition. Obituaries are moved through a series of states (i.e., Notified, OK To Purge, Purgeable, etc.) before they are purged from the system to ensure all interested parties are notified and can make the proper adjustments and modifications.

Obituary Synchronization

Who is responsible for moving obituaries through Who is responsible for moving obituaries through these state transitions?these state transitions?

For back link obituaries, the Master replica—for used by obituaries, the replica which created it, if that replica no longer exists, then the Master will take ownership

How are obituaries synchronized?How are obituaries synchronized?

As they are moved through their various states, those changes are synchronized (using the Obituary Index) to agents holding a replica of the effected objects

How can I tell if there are problems?How can I tell if there are problems?

NDS iMonitor—Agent Process Status, NDS iMonitor—Obituary Report

How do I resolve problems?How do I resolve problems?

Attend TUT229 and TUT223

Obituary Status

Schema Synchronization

Where does schema reside?Where does schema reside?All agents have a copy of the schema—This copy will be a cached copy (not modifiable) unless the agent also holds a writeable copy of the tree root partition

How does schema propagate?How does schema propagate?Through the schema synchronization process, sideways and down, never upwards, based on replica depth. Replica depth is the replica with the fewest number of containers held by a given agent. NDS iMonitor—Server Information Report, shows replica depth for all agents.

What happens to schema when the first replica is What happens to schema when the first replica is added to added to an agent?an agent?The agent resets the schema and adds itself to a poll list to receive a new copy of the schema—This operation must complete before the replica add can proceed

How can I tell if there are problems?How can I tell if there are problems?NDS iMonitor—Agent Process Status, NDS iMonitor—Schema Browse, Schema Root Browse

Schema Root

Schema Synchronization List

Service List Types

Replica—Shares a replica in common with this agent—Schema changes will be synchronized between the local agent and the specified agent

Service—The specified agent has requested that the local agent synchronize schema to it until the expiration time—normally, this is done by agents that do not hold a replica

Poll—The specified agent is being installed and has requested immediate schema synchronization

Agent Configuration Synchronization

What is Agent Configuration Synchronization and What is Agent Configuration Synchronization and why why is it needed?is it needed?

Not every agent in an eDirectory environment will hold a copy of it’s own NCP Server object. This object contains information needed to control the agent’s behavior—In order to have this information available and reduce network bandwidth, a local partial cache of the NCP Server object is maintained. Also, Agent Configuration Synchronization updates the NCP Server object with changes that occur on the local agent (ie. Network Address, NCP Server Name)

Where is the Agent Configuration stored?Where is the Agent Configuration stored?

The Pseudo Server—every agent has it’s own copy of this object in which to store it’s own agent configuration, regardless of whether it hold any replicas or not

Pseudo Server

Agent Configuration Synchronization

What is the process that performs Agent What is the process that performs Agent Configuration Synchronization?Configuration Synchronization?

Limber (also referred to as Connectivity Check)—Limber may trigger other processes to refresh their configuration (i.e., LDAP, etc.) Limber is scheduled to run every 4 hours by default (configurable in iMonitor) It will also run when requested or when a change is noticed that needs to be synchronized out

What kind of data is maintained by this process?What kind of data is maintained by this process?

Network Address, Index Definitions, NCP Server’s RDN and location in the tree, Tree Name, Permanent Configuration Parameters

How can I tell if there are problems?How can I tell if there are problems?

NDS iMonitor—Agent Process Status, NDS iMonitor—Pseudo Server Browse (eDirectory 8.6 or greater)

Additional Background Processes

JanitorJanitor—The Janitor process checks for Synthetic Time, Updates Inherited ACL’s, Updates Known NCP Server Status and Version, Purges Unneeded Bindery Entries, Purges Expired Temporary Agent Configuration Parameters

PurgerPurger—Using the Change Cache together with the Purge Vector for each replica held, the Purger process removes values and entries that are no longer needed in the system because their removal has been seen by all replicas

How can I see what Background Processes are How can I see what Background Processes are scheduled and/or running in a given eDirectory scheduled and/or running in a given eDirectory agent?agent?

NDS iMonitor—Background Process Schedule

How can I see what activity is occurring on a given How can I see what activity is occurring on a given eDirectory agent?eDirectory agent?

NDS iMonitor—Agent Activity, NDS iMonitor—Verb Statistics

Background Process Schedule

Agent Activity

Database Concepts and Details

How is the data stored in eDirectory?How is the data stored in eDirectory?In a true database, not a flat file, using a patented Novell proprietary technology similar to that used in GroupWise

What should I know if I’m an Administrator?What should I know if I’m an Administrator?How to tell if you have an appropriate amount of Database Cache

What should I know if I’m a Developer or an What should I know if I’m a Developer or an Administrator?Administrator?When, how, and where to use eDirectory Indexes—indexes have a cost, misuse can adversely effect performance, judicious use will improve performance

What should I know if I’m a Developer?What should I know if I’m a Developer?Plan your usage as you would a distributed database: carefully design your data model (i.e., when to use objects, attributes, and containers), carefully design your meta-data (schema)—in general, understand how eDirectory works and how your application’s interaction will effect it

Database Cache

Database Cache Configuration

Index Definitions

Database Concepts and Details

What is the difference between a standard database What is the difference between a standard database and eDirectory?and eDirectory?

Standard databases are hierarchical, relational, or object oriented; eDirectory is kind of a hybrid of all three and, at the same time, it’s none…it’s an X.500-like directory

• Object-Oriented—an eDirectory Object is the transactional unit

• Relational—based on the schema and referential integrity, the attribute values of an object are like relational tables in an SQL database only more flexible

• Hierarchical—each entry has a location in a hierarchy and can be referenced in that hierarchy

•And with eDirectory it’s all distributed: Attending “IO115—Directory or Database, Choosing the Right Tool for the Job” may help in further understanding the differences, advantages, and drawbacks of each

Security Concepts and Details

What are Access Control Lists (ACLs) and what are What are Access Control Lists (ACLs) and what are they they used for?used for?

ACLs are used to grant or block access to a particular subtree, object, or attribute

• ACLs must be unique—uniqueness is defined by the combination of the trustee and the attribute

• ACLs are inherited at the partition boundary

• They are not required to propagate to every container down the tree— they are calculated from the partition boundary down when needed

• The Janitor process calculates inherited ACLs using the “modifiedACLEntry” attribute on the Psuedo Server

• Inherited Rights Mask

Security Details and Concepts

How do ACLs affect system performance and how can I How do ACLs affect system performance and how can I optimize?optimize?

The number of ACLs that must be considered will increase the access time to each

object below those ACLs in the tree—Try to reduce the number of ACLs used using

the following priority1. .[This].2. Containers3. Groups or Organizational Roles4. Dynamic Groups5. Grant explicit rights

How are rights computed for an authenticated object?How are rights computed for an authenticated object?Using the Security Equivalence Vector (SEV)—When evaluating ACLs, each

trustee iscompared with the identity’s SEV to determine if the ACL is applicable. The

SEVincludes .[Public]., .[Root]., .[This]., and every container between the object

andthe root plus anything listed in the “Security Equals” attribute of the

authenticatedobject plus any cached dynamic group equivalents.

Access Control List

Inherited ACL

Security Equivalence Vector

Conclusion

• Directory concepts and definitions• Object synchronization• External reference synchronization• Obituary synchronization• Schema synchronization• Agent configuration synchronization• Additional background processes• Database concepts and details• Security concepts and details