AdminP Advanced Topics - AdminCamp Administration server must be set for the Domino Directory as...

1

®

IBM Software Group

© 2003 IBM Corporation

AdminP Advanced Topics

Susan Bulloch - IBM ISV Technical Enablement Engineer

IBM Software Group | Lotus software

2

Agenda …

AdminP history

AdminP processing and operations

Tuning AdminP

Monitoring AdminP

Defining best practices

Implementing tips and tricks

What’s coming in Domino 8

Wrap-up

2


3

What Was AdminP Designed to Do?

AdminP is a server task (adminp) that automates many administrative tasks

You initiate the tasks, and the Administration Process completes them for you

It was introduced in R4 to manage name changesThere were 19 requests when introduced in R4

And now?AdminP is a required server task and an integral part of the Domino systemIt’s taking on more work with each release

180+ requests in Domino 8With each release, it’s becoming more efficient, too!


4

What Does AdminP Do?

AdminP automates things that can be done over timeMoving filesDeleting filesChanging namesCreating replicas on remote servers

It automates things that need to be exactServer build numbersServer port namesClient buildsIf the data needs to be exact, AdminP can often do it

3


5

What Else Can AdminP Do?

Help manage user mail accessAllows the user to be set down to EditorPerforms various functions that formerly required manager access

Sets Out-of-Office status, mail & calendar delegation

Manage registration and recertification using the CA processAllows Web-based user registrationAllows ID management with no user actionsAllows more secure administration Integrates with many 3rd party tools


6

AdminP is Self-Configuring!

If you leave it alone, AdminP configures itself!A database replica stub is created on each new server

The ADMIN4.NSF database is created upon startup of each new serverThe replica ID of Admin4.nsf is based on the Directory Replica ID– So is unique to each environmentReplication must be allowed from admin hub to spokes– Either directly or through other hubs

If you interfere, it can cause problemsAttempts to change Replica ID will usually fail!

The replica ID needs to be set as designed

4


7

AdminP Requirements

The AdminP server task must be running on the serverLoad adminp at startup using servertasks= in the ini file

It’s there by defaultBest practice is to leave it this way

There must be an Administration Requests database (ADMIN4.NSF) on each server

Users and Admins need appropriate access to this databaseThe databases must be well maintained and replicating properly

More details later …


8

AdminP Requirements (cont.)

An Administration server must be set for the Domino Directory as well as ADMIN4.NSF

The setting “Do not modify Names fields” for Domino Directory and ADMIN4.NSF is required

A copy of CERTLOG.NSF must exist on your administration serverYou can have copies elsewhere too if you wish

An administration server set in the ACL of databasesAny database that you want AdminP to maintain

You probably want them all maintainedThere’s a command to know if all databases are set properly– Details in a few minutes

5


9

Where AdminP Works

On the administration server of a databaseChanges are made on this copy of a databaseThis minimizes chance of replication conflicts

On the administration server of the DirectoryOften the “main” server of a system

“All Servers” or *Every server in the domainFor example, name changes are processed by all servers

“Named” serverA specific server to perform a requestFor example, the move replica request works on the “target” server for the move, but no others


10

Processing Requests

Most processes are timedSequential actions trigger the next action

Process continues until all steps are completeThe shortest interval is one minute (immediate requests)

Something starts the processUsually an Administrator

Examples of admin-initiated processes are user renames, deletions, replica creation

A response is expected by DominoExample: User authenticates with home server, replica stub created on target server

The next step is startedExample: Unread marks change, group entries change

6


11

Processing Requests (cont.)

The processes continueSome can continue for a week

But you can speed this up– There’s even more control in Domino 8

Some processes can stay active for more than a weekMail movesName change requests– These are monitored and controlled in Person Documents– You do NOT need to keep documents 21 days in ADMIN4.NSF

– Any processes that need to continue will re-start based on the person documents


12

Automated Processing — Almost

Interim steps sometimes require human touchAnything affecting Directory documents or database files

Also name change reversions!Anything that must be approved along the way by someone with rights to the document or file

In other words, an Administrator:– This allows delegation to less experienced employees– Protects Directory data from employees in groups who are not Notes

Admins– Security teams often perform renames– They often have limited Domino training– This tiered approval process protects your system

7


13

Examples of AdminP Tasks

Delegate mail files

Set end-user agents to run

Manage CA administration

Manage roaming users

Create and rename rooms and resources

Find users

Manage policies

Change HTTP password

Create new mail files in the background


14

Examples of AdminP Tasks (cont.)

Add/remove servers in cluster

Change user password in Domino Directory

Add Internet Certificate to Person Record

Configure Domain Catalog

Enable server’s SSL ports in Domino Directory

Move mail files

Rename groups

But this isn’t all …

8


15

AdminP Operations

Every hour, by default, AdminP checks for workThis is a tuneable parameter

Only requests that are “new” are processed on a serverOn AdminP start-up, task requests with no response document (log) or entry in a hidden ID table are flagged to be processedWhen AdminP is already running, new entries (based on time/date stamp) to the ADMIN4 database are flagged

This can cause problems if “old” data is replicated back into newer databases– This must be prevented– We’ll tell you how


16

AdminP Operations (cont.)

Immediate requests are performed within a minute of posting to the ADMIN4 database

No option exists to change the immediate request interval (1 minute)

Typically these requests should be processed quickly:Create replicaChange user password in Domino DirectoryUpdate client information in Person RecordChange HTTP password in Domino Directory

Immediate requests are denoted in ADMIN4 with a “bolt” icon

9


17

AdminP Batched Requests

These were introduced in 6.0 to increase efficiency

They perform certain modifications for many users at once

The database is accessed onceSeveral user changes can be made Example: Four user names can be changed in the Access Control List (ACL)

Pre-batch methods caused 4 accesses

Currently 18 requests are batched

Interval times should be long enough to accumulate multiple batch types

Interval should be short on the admin hub, longer on spokesIf the interval is too long, the server won’t have time to accumulate similar requests.


18

List of Batched Requests

Rename in ACLDelete in Person DocumentsDelete in ACLDelete in Reader/Author fieldsRename in Person DocumentsRename in Reader/Author fieldsRename Group in ACLRename Group in Reader/Author fieldsRename Person in Unread List

10


19

List of Batched Requests (cont.)

Rename Web User in ACLRename Web User in Person DocumentsRename Web User in Reader/Author fieldsRename Web User in Unread ListDelete Person in Unread List Rename in Design ElementsDelete in Design ElementsRename Web User in Design ElementsRename Group in Design Elements


20

AdminP — The Database (ADMIN4.NSF)

Contains processing action requestsAdminP requests

Contains processing action resultsKnown as AdminP response (log)

Administration approval requests are there alsoExamples:

Confirm database deletionCertification requests for change hierarchy

Provides views to help with troubleshooting Use Domino Domain Monitoring (DDM) to monitor the database in Domino 7!

Finds stalled requests

11


21

AdminP – The Database (ADMIN4.NSF) (cont.)

AdminP is designed to be managedWorkflow requires attention/approvalThe database will grow in size if ignored

Sufficient access is neededDefault is Author with Create for users

Can be No Access in ND6 and later– Requests from users are mailed to the database– Default mail-in database is called Administration RequestsCan use wildcard if Default needs to be No Access

Administrators need Author, minimumEditor access to approve requests


22

AdminP – The Database (Admin4.nsf) (cont.)

Proper replication is requiredAdmin4.nsf should replicate as often as Directory

The size can grow unacceptably if it doesn’t

Replication retention should be standardizedThe default is 7 days

10 is acceptable, as is 14 or 21Anything longer is unnecessary and dangerous!

Improper replication causes old requests to “come back”Causes server slowdowns– Replication “storms” can occurThis is the number one cause of AdminP meltdowns!Easily controllable, preventable

12


23

Tuning the AdminP SystemDefault settings will work in small companies

AdminP default interval is 60 minutesEvery hour, AdminP checks for work to be doneDaily processes run at midnightDelayed processes run on Sunday at midnight

Because they are processor intensive

Large organizations need to tune the AdminP systemVirtually everything is configurableStart in the Server Document


24

Deep Dive into Tuning: Server Document Settings

13


25

Deep Dive into Tuning: Server Document Settings (cont.)

IntervalDefault is 60 minutes (blank in Server doc)You can reduce this as needed15 minutes on administrative server is acceptable

Be sure to increase replication interval also

Store Admin Process log entries when status of no change is recordedChange from “Yes” to “No”This will reduce the admin4.nsf database size

By as much as 20%!“No” is the default beginning in Domino 6.5.5, 7.0


26


Delayed Request SettingsThe default is SundayConsider running these requests more oftenThis is the Reader/Author name change

You can run this every night

Delayed requests generate messages in the server log18-10-2002 19:57:04 Begin MIME to CD Conversion (Process: ? (000004C4:00000002), Database: D:\data\mail\xxx.nsf, Note: 0000766E)Set converter_log_level=10 in server ini file to shut off these messages

It’s AdminP preparing data to work onIt was always there but not always logged

14


27


Maximum number of threads Multiple threads are supportedDefault is 3, maximum is 10

One thread is used to dispatch requestsThree threads to process the requestsThreads are only activated when required to process requestTest incrementally if you increase

Notes 8 offers more thread options


28

Tips for Tuning

Speed up replicationEspecially if you reduce interval timingRequests will replicate out faster, be processed quicker

Skipping databasesReader/Author name renames take a long time — they’re resource intensiveSkip databases using $Adminp hidden view

Use a selection formula to show only documents with Reader/Author fields– All others are skippedIf view is blank, the entire database is skippedYou can see a sample in PERNAMES.NTF– Modify to suit your needs

15


29

Tuning Tools: Server Console Commands

You may need to use Server Console command when troubleshootingUse with caution unless you’re sure of the impact

Tell AdminP Process NewProcesses all new requestsUse to jump-start a process

Use this one instead of almost any other you want to use

Tell AdminP Process PeopleProcesses Person Document changes

Tell AdminP Process TimeUsed for shared mail systems Used for load balancing mail moves


30

Tuning Tools: Server Console Commands (cont.)

Tell AdminP Process AllProcesses all new and modified requests

Includes immediate, interval, delayed, and daily requestsThis is probably not what you want to do when using this command

Causes requests to back up until “ALL” are finishedUse with extreme cautionNever use during production hours

Tell AdminP Process DailyProcesses all new and modified daily requests to Person Documents

16


31


Tell AdminP Process DelayedProcesses all new and modified delayed requestsBased on start executing on/at settingThis is a “Sunday morning process” because it is processor intensiveBut it doesn’t delay new requests

Like Tell AdminP Process All does

Tell AdminP Process IntervalProcesses all immediate and interval requests


32


Tell AdminP Show Databases Lists databases with and without a designated admin server See your server log for the list

You can ensure all databases are protected this way

Tell AdminP Process MAValidates whether mail policies were updated Not a new request type, but a new AdminP thread (Domino 7 only)

Tell AdminP QuitStops AdminP task

Load AdminPStarts AdminP task

17


33

Bonus Trick: How You Can Use AdminP

Tell AdminP Process Daily exampleYou change a user’s name using AdminPThe process rolls alongThe user calls you saying, “My unread marks are all messed up! You broke my Lotus Notes!”You tell the user “I can fix this. I need you to log out of Notes for 10 minutes”

I’m thinking we should tell them to turn off the PC just to be sureYou type “tell adminp process daily” at the Server Console

When the user logs back in, the unread marks are fixed


34

Monitoring AdminP

AdminP is designed to be managedSome database views offer you information

Administrative attention requiredThese are informational, there’s a button to remove them from viewSome end-user notifications can be automated– Select Action – Enable/Disable end-user notification

Other views require an actionIndividual approval required

File deletions require approvalName change reversions– No more “21-day” issue

Pending by age/server will show older requests that may need attention today

18


35

Monitoring AdminP (cont.)

Documents that need attention or action will stay in the database until:

You look at them orYou process them or You delete them

They are protected by a $NoPurge FieldYour database will grow and grow

Assign rotating responsibility for ADMIN4 monitoringOr let the new admins do it all!


36

New Feature for AdminP: DDM

DDM (Domino Domain Monitoring) can monitor the progress of requests

Monitors 11 different types of AdminP requestsSee me later for how to add more

New in Domino 7

The default server probe is the “Administration” type

Any error in AdminP processing will create a notification in DDMStalled rename requests will notify DDM

You don’t have to monitor the database as closelyBut you have to start using DDM

19


37

New Feature in AdminP: DDM (cont.)

AdminP requests monitored by default in DDM


38

Best Practices

Learn from the mistakes of others

The ADMIN4.NSF database must replicate throughout your system

It must have the Replica ID assigned by Domino

Old or test servers should not exist in production domainsADMIN4.NSF exists on all serversWhen old servers are turned back on, databases replicate

In addition to ruining NAMES.NSF, you ruin ADMIN4

20


39

Best Practices (cont.)

Never restart a server that has been out of service for more than the purge interval of ADMIN4

Old documents replicate back inOld requests are read by AdminPServers send error messages stating that the requests are too oldCustomers have clogged their systems this way

Never run test servers in your production domainThey, too, have a copy of ADMIN4.NSF


40

Best Practices (cont.)

Keep the database size downDo it for your serverProcess the requests that require your touch regularlyMonitor replication

Rules of thumbAll copies should have the same Replica ID and ACLAll copies should be nearly the same sizeNumber of documents should be nearly the same

Exceptions:– Admin server can store more information– If you use a selective replication formula, sizes will differ

21


41

Selective Replication

Selective replication formulas can help in large systemsThey work best when created and maintained on the spoke servers

You’ll need a process to add these when the database is replaced– Customers who use them, love them

This limits the size of the spoke databasesAlso limits the amount of data replicated

Especially useful over slow linksAdmin hub receives all requests, so can do the processing needed

Designed to allow the spoke server to receive only what it needsAnything it or cluster mate needs to processSpoke will send anything it originates to the admin hub


42

Selective Replication (cont.)

Sample codeAll disclaimers apply with this codeTEST, TEST, TEST

SELECT @Contains(@UpperCase(ProxyServer) ; ″server":″clustermate" ) |

@Contains(@UpperCase(ProxyServerName) ; ″server":″clustermate" ) |

@Contains(@UpperCase(ProxyActionRequestor) ; ″server":″clustermate" ) |

@Contains(@UpperCase(InboundReplicaServers) ; ″server":″clustermate" ) |

@Contains(@UpperCase(ProxyServer) ; "*" ) |

@Contains(@UpperCase(ProxyServerName) ; "*" )

22


43

Tweaking Name Changes

Increasing the time a user can accept name changesNecessary in EuropeChange the default

Allowable values are 14 to 60Allows the user to go on holiday


44

Names Fields

Use caution when implementing feature: All Names fieldsUsing the “Modify All Names Fields” in ACLs may have unexpected effectsIf used in mail files, AdminP will remove users from “Sent” fields when you delete users

Do NOT change the default AdminP settings in mail database (or in the Domino Directory)– Everything is coded to work as set by Domino/Notes

If used in other databases, the Creator name is removedThis could be a compliance issue

One more thingIf the last person in any Reader/Author field is removed, the document becomes publicUse this feature with care!

23


45

Programmability

Custom AdminP code can be written in LotusScript

Notes Administration Process ClassIntroduced in Domino 6.0

There are 6 properties and 39 methodsUseful you want to automate certain things

Like user-generated rename processes

Use with caution and test your codeProblems have occurred with third-party tools that weren’t thoroughly tested


46

Things to Watch Out For

Renames can take a long timeSemaphore gets locked doing ACL changes

Other changes cannot be processedFixed in 6.5.4 with code and ini setting

TN 1174405ADMINP_ENABLE_CASCADE_DESIGN_ELEMENTS=1

Mail file moves to a large, empty SAN using AIX can failAdminP reports insufficient disk spaceFixed in 7.0

Had problems with scientific notation

24


47

Things to Watch Out For (cont.)

Notes has problems with short names in Location documentsBoth AdminP and Dynamic Client Configuration have failed if the server name is short

Example: Notes1 instead of Notes1/Acme

CA-Process registered users have certificates in ADMIN4.NSFNot in certlogThis can create a lot of documentsIBM/Lotus is researching this

®

IBM Software Group

© 2003 IBM Corporation

AdminP Improvements in Domino 8

25


49

Direct Deposit of AdminP Requests

Works for the “Named Server” requestsMail file moves, etc.

Replication of ADMIN4 is skippedIf a connection is available

Reduces replication and time lagSpeedy

If a direct connection is not availableRegular process occurs

You can disable itADMINP_DONT_ATTEMPT_DIRECT_DEPOSIT=1


50

Special Purpose Threads

Remember the maximum number of threads for AdminP?It’s 10, with a default of 3In Domino 8, you can specify some of those 10 threads to certain process types

ADMINP_IMMEDIATE_THREAD=XADMINP_INTERVAL_THREAD=X

Works like an overflow valveOnly used when neededOnly used for those 2 types of requests

Other types are processed normally

26


51

Override Default Run Intervals

Use this with careCan cause problems if done wrong

If you want to change how certain items run, you can: ADMINP_IMMEDIATE_OVERRIDE = x, x, xADMINP_INTERVAL_OVERRIDE = x, x, xADMINP_DAILY_OVERRIDE = x, x, xADMINP_DELAYED_OVERRIDE = x, x, x

Domino 8 Admin Help has the list of numbers


52

Override Default Run Intervals (cont.)

Why would you do this?Use to change actions like “Rename in Unread List” to Interval instead of Daily

ADMINP_INTERVAL_OVERRIDE = 68.00If you’re doing a lot of name changes

Change Rename in Person Documents to Immediate instead of Interval– ADMINP_IMMEDIATE_OVERRIDE=16.00

You’ll fly through the changes!

27


53

Improved Rename Processing

A new, per database names listIf a name being processed is not in this list, the database is skippedLimited to 4K per databaseNo support for “Modify All Names Fields” choice in ACL

Requires optional new ODSODS change is not automatic or requiredYou have to enable it with an ini setting

Create_R8_Databases=1Then run copy-style compact


54

Synchronize Unread Marks

Inconsistencies are caused by AdminP replica creation methodsManual per-user synchronization via Notes Client is not practicalCreate and move replica

In 7.02, ADMINP_EXCHANGE_ALL_UNREAD_MARKS=1In Domino Admin 8.0, “Exchange Unread Marks” is a UI option

Move mail fileIn 7.02, ADMINP_EXCHANGE_ALL_UNREAD_MARKS=1In 8.0, automatic synchronization

Synchronization may impact overheadMail files

With limited users, synchronization should have limited impactApplications

With numerous users, this may significantly change creation time

28


55

Database RedirectDomino 8.0 introduced “Database Redirect File” (.NRF)

Placeholder file directs client to the new databaseClean up stale bookmarks and open alternate replica

Found in the Admin Client “Move Database” ToolOptionally create the redirect to new replica

Admin Client “Delete Database” Tool


56

Database Redirect (cont.)New Admin Client Processes also

Create “Database Redirect File”Update “Database Redirect File”

29


57

Automatic Inbox Maintenance

There is a significant decrease in server I/O with small inboxesFor information about the impact of large inboxes:

http://www.ibm.com/developerworks/lotus/library/notes-mail-files/

You beg and plead for users to file mail in foldersThey never doWe give you a new tool

AdminP will move the mail for themAge-based document trimming via mail policies or Server document

WARNING: Get management permission first!


58

Automatic Inbox Maintenance (cont.)

AdminP poll thread executes LotusInboxCleanup mail file agentTell adminp process mb

This task does not remove documents from the mail fileThey will still be available in All Documents view

Your users will still call you

It may take a while to get permissionBut you now have a tool to use

30


59

Improved Server Commands

tell adminp process allChanged in 8.0

Requeue all new and modified requests– No waiting for requests to finish

tell adminp process restartWaits for all requests to finish, rebuilds all queues

Formerly, tell adminp process all did thisUse with care, not during prime hours


60

For More Information about AdminP

TechnotesKnowledge Collection — the Administration Process in Domino 6.0x and 6.5x

http://www.ibm.com/support/docview.wss?uid=swg21213224

Frequently Asked Questions — the AdminP Process

http://www.ibm.com/support/docview.wss?rs=899&uid=swg21212760

developerWorks articles“All About AdminP,” Parts 1 and 2

http://www.ibm.com/developerworks/lotus/library/ls-AllAboutAdminP_1/http://www.ibm.com/developerworks/lotus/library/ls-AllAboutAdminP_2/index.html

LotusScript: The NotesAdministrationProcess Class in Notes/Domino 6http://www.ibm.com/developerworks/lotus/library/ls-LS_AdminProcess/

Creating a Custom Administration Process Request Handlerhttp://www.ibm.com/developerworks/lotus/library/ls-Custom_AdminP_Handler/

31


61

How to contact me:Susan Bulloch

[email protected]://notesgoddess.net

Questions?

AdminP Advanced Topics - AdminCamp Administration server must be set for the Domino Directory as...

Documents

Transcript of AdminP Advanced Topics - AdminCamp Administration server must be set for the Domino Directory as...