NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

58
Novell Storage Services File System Tuning, Troubleshooting and Auditing in Novell® Open Enterprise Server on Linux Adam Jerome Senior Software Engineer Novell, Inc. [email protected] Vijai Babu Madhavan Filesystem Engineer Novell, Inc. [email protected]

description

In this advanced technical session, we'll discuss techniques for optimizing and troubleshooting the Novell Storage Services (NSS) file system in Novell Open Enterprise Server 2 on Linux. You'll also learn how to design, configure and troubleshoot a clustered NSS environment. Finally, find out how integration with Novell Sentinel in Novell Open Enterprise Server 2 SP2 opens the door to NSS file auditing.

Transcript of NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

Page 1: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

Novell Storage Services™ File SystemTuning, Troubleshooting and Auditing in Novell® Open Enterprise Server on Linux

Adam JeromeSenior Software EngineerNovell, [email protected]

Vijai Babu MadhavanFilesystem EngineerNovell, [email protected]

Page 2: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.2

Agenda

• Novell Storage Services™ - Feature Overview

• Novell® Storage Services/NCP™ - Recent Changes, Tuning and Troubleshooting

• Novell Storage Services - Auditing (Vigil)

Page 3: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

Novell® Storage SystemsFeature Overview

Page 4: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.4

Novell® Storage Systems Features

• Trustee Model– Inherited Rights

– Visibility

• Salvage• Directory Quotas• User Quotas• Compression• Data Shredding• Immediate Flush

Page 5: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.5

Novell® Storage Systems Features

• Multiple Name Spaces

• Distributed File Services (DFS)

• Multiple Server Activation Prevention

• Archive / Versioning enabled

• Logical Volumes and Pools

• Encryption

• Pool Snapshot

• File Snapshot (COW)

Page 6: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.6

Novell Storage Services™ on Linux

Virtual File Services (VFS)

Device Drivers

EXT3 Reiser XFS

Novell Storage Services

User

Kernel

Device Mapper - EVMS

Media Manager Logic

Linux SemanticAgent

EdirHandler

_AdminManagement

Upcall

NCP™Linux Applications(including protocols like SAMBA)

Page 7: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.7

Novell Storage Services™ and LUM

• Novell® Storage Services uses eDirectory™ GUIDs internally

• Linux passes UIDs to Novell Storage Services

• LUM enabled users link the eDirectory user to a Linux UID

• Novell Storage Services maintains multiple ID caches

– G2I, I2G and SEV Caches

Page 8: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.8

NCP™ Server – Novell Storage Services™

NCP Client eDirectory™

NCP Server

Novell Storage Services

1) User and Groupinformation forthe connectionJoe is using

2) Create file foo.txtlogged in as Joe

3) Apply trusteerights-basedon path andconnection

4) Create foo.txt as root

5) Change ownerto Joe (by GUID)

Page 9: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.9

Novell Storage Services™ and NCP™

• NCP has its own trustee store– Synced with Novell® Storage Services trustees at run time.

– Ability to re-sync on demand

• NCP does not require LUM enablement• NCP runs in user space (ndsd addr space) as root and

uses various methods to communicate with nss– Normal VFS layer calls (i.e. open, close, read, write)– Extended attributes and zAPIs for special operations– _admin commands for special operations– NEB events generated by Novell Storage Services– Ncp2nss daemon facilitates this communication

Page 10: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.10

Novell Storage Services™ and NCP™

• OES1 considerations– Owner, modifier, archiver and deleter are all root– User quotas will not work

• OES2 considerations– Owner will be correct, and user quotas will work– Modifier, archiver, and deleter will still be root

• NCP implements LONG name space• NCP maintains the UIDMapping cache to speed up G2I

lookups

Page 11: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.11

Novell Storage Services

eDirectory™Client LUM

VFS – Novell Storage Services™

1) Create file foo.txt logged in as Joe— request comes to Novell Sorage Services as UID 705

2) Novell Storage Services requests the eDirectory name for UID 705—LUM returns joe.acme

3) Novell Storage Services requests security equivalence information for UID for joe.acme and eDirectory return a list of equivalent users and group memberships

4) Novell Storage Services does normal trustee checking based on the users and groups returned by eDirectory

Page 12: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.12

VFS Applications

• Novell Storage Services™ is registered as a Linux file system

• Protocols– Access protocols, NCP™,and Novell® CIFS use both the VFS layer and

special interfaces

– AFP uses zAPI interface

– Other protocols use the VFS layer

• Other Applications– Must have LUM enabled users

• Trustee rights are enforced– Does not use the group ID

• All Novell Storage Services capabilities are available

Page 13: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.13

• The POSIX read/write/execute attributes represent NetWare® attributes:

– Read: NetWare hidden attribute

– Write: Set unless it is a read-only file

– Execute: NetWare execute attribute (cannot copy) / or subdirectory

VFS Applications

Page 14: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

Novell Storage Services™

Recent Changes, Tuning and Troubleshooting

Page 15: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.15

Improved NCP™/NSS Integration to Support Salvage for Non-LUM users• NSS and NCP enhanced to retain deleter information

for files deleted by non-LUM users

• With latest OES2SP1 patches (or) OES2 SP2

– Owner, modifier and deleter will be correct

– User quotas and salvage will work

• Both LUM and non-LUM scenarios are nearly the same

– Users NO longer need to be LUM enabled

Page 16: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.16

Improved NCP/NSS Integration for LONG name space• NCP™ implements LONG name space• NCP access optimized if NSS volumes are mounted

with LONG name space – with latest patches for OES2SP2/SP1

• Significant improvement in performance/scalability

• Reduction of load/utilization on NCP/ndsd/ndp• LONG as default NSS mount name space since SP1

– Novell® GroupWise® had issues with LONG name space, but those issues are fixed with recent hot fixes from GroupWise

Page 17: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.17

Novell Storage Services™ ID Cacheon Novell® Open Enterprise Server 2 Linux

• NCP™ supports non-LUM map entries with its cache– With latest patches for SP1/SP2

• NSS ID Cache Tuning Paremeters– Reset various ID caches

> nss /ResetIDCache

– Modify the reset intervals of ID caches> nss /IDCacheResetInterval=value> Default – 25 hours

– Modify the size of ID caches (Latest patches for SP1/SP2)> nss /IDCacheSize=value> Default - 16K

– Monitor ID cache statistics (OES2SP3)

Page 18: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.18

NCP™ Dir Cache Tuningon Novell® Open Enterprise Server 2 Linux

• Monitor usage and evictions – LOG_CACHE_STATISTICS

• Configure based on working set and available memory– MAXIMUM_CACHED_FILES_PER_VOLUME

> Default - 20000– MAXIMUM_CACHED_SUBDIRECTORIES_PER_VOLUME

> Default - 50000– MAXIMUM_CACHED_FILES_PER_SUBDIRECTORY

> Default - 2048– Cache Entry memory usage - ~216 bytes + Full path name

• Additional Information> http://www.novell.com/documentation/oes2/file_ncp_lx/data/bc06ts8.html> TID 7004888 – NCP Performance Tuning on OES2 Linux

Page 19: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.19

Novell Storage Services™ IO SystemImprovements on Novell® Open Enterprise Server 2 Linux

• OES2 SP2– Improved Read-Ahead Algorithm (Stepping Window)

• OES2 SP3– Improved IO Scheduler Interactions – Multi-Page BIO Support

• Recommended Configuration– With latest patches for OES2 SP2/SP1

> nss /noUnplugAlways

– XEN Guest> Change the default noop IO scheduler to cfq

– Modify Read Ahead Blocks based on access pattern> Default readAheadBlks count for NetWare®/OES1 was 2

Page 20: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.20

Novell Storage Services™ Caching

• Files– Objects in memory (Beasts / Inodes)

– Packed on disk in 4K Blocks (Metadata Blocks)

• Metadata– 4K Pages in Memory

– Unpacked into Objects

• User Data– 4K Pages in Memory

– Saved on disk as 4K Blocks (User data Blocks)

Page 21: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.21

File Caching

• Novell Storage Services™ on NetWare®

– nss /ClosedFileCacheSize> Maximum Number of “notInUse” Files in memory> Based on the available memory in the system> NSS does its own balancing

• Novell Storage Services on Linux– nss /ClosedFileCacheSize– Linux Memory Pressure

> Inodes in memory> /proc/slabinfo

» lsa_inode_cache

> Linux will influence the total number of Files in memory

Page 22: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.22

Linux Page Cache

• Linux memory (Novell Storage Services™ is akernel module)

– Kernel Memory > Low Memory – directly addressable by the kernel

– User Memory > High Memory – Kernel has to map it before using

• 32-bit v/s 64-bit– 32-bit OES Linux Servers

> 1 Gig of Low Memory (by default)> Custom Linux Kernels can increase the default

– 64-bit OES Linux Servers> All memory is Low Memory

Page 23: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.23

Novell Storage Services™ Page Cacheon Novell® Open Enterprise Server 2 Linux

• OES2 Linux– 64-bit Servers

> Not Low Memory Constrained, so HiMem Cache not needed

> nss /MinBufferCacheSize to configure Metadata Cache

> User Data integrated into Linux Cache

– 32-bit Servers> Use HiMem to cache Novell Storage Services metadata

> nss /HighMemoryCacheType=cachetype (private, linux, OR none)

> Default is private

> Similar to OES1 Linux SP2 with the latest patch

> User Data integrated into Linux Cache

Page 24: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.24

Novell Storage Services™ Memory on Novell® Open Enterprise Server 2 Linux

• OES2SP3

– Ability to look at comprehensive memory usage

– Ability to look at memory consumed for different data structures

– Ability to tune the configurations

– Should help troubleshoot memory build-up issues

Page 25: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.25

Novell Storage Services™/NCP™ Issueson Novell® Open Enterprise Server 2 Linux

• Resource Migration Hangs

– Check Unload.out file

– Stack dump of user space processes (gstack)

> adminusd, ndsd, ncp2nss

– Stack dump of kernel threads

> echo 1 > /proc/sys/kernel/sysrq

> echo t > /proc/sysrq-trigger

– Core (user-space processes and kernel) if necessary

Page 26: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.26

Novell Storage Services™/NCP™ Issueson Novell® Open Enterprise Server 2 Linux

• Slow File Access

– Apply ALL the SP1/SP2 Patches

> Optimized for NSS volumes mounted in LONG name space

> Optimized for both LUM and non-LUM users access

> Reduced memory usage

– Tune NCP Dir Cache

– Tune NSS ID Cache

– Process of elimination (NCP, NSS, ...)

Page 27: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.27

Novell Storage Services™/NCP™ Issueson Novell® Open Enterprise Server 2 Linux

• Random Reboots/Multiple Poison Pills– Apply ALL latest patches

– Ensure there are NO corruptions or double activations> Verify

> MSAP Steal

– Determine poison pill is an after effect of the reboot or it caused the reboot

> Setup kdump to capture core

> Setup sysrq to capture core if hung

Page 28: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.28

nsscon

• NetWare® like console

– nss /help

– nss /find

– nss options for tuning, configuration

– Can be set persistently in /opt/novell/nss/conf/nssstart.cfg

Page 29: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.29

Ravsui

• Pool in maintenance state– nsscon

> nss /poolMaintenance=<POOLNAME>

• ravsui verify <POOLNAME>– Verify a pool– ravview vbfn <POOLNAME>

> Will show the results from verify

• ravsui rebuild <POOLNAME>– Rebuild a pool– ravview rtfn <POOLNAME>

> Will show the results from rebuild

Page 30: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.30

Novell Storage Services™ Future Work

• Support for >8TB Pools– Requires media change, NOT backwards compatible– Challenges

> Co-existence of old and new media formats> Migration of data from old to new media formats

• Support for >2TB Partitions• SLES11 support• NO release vehicle determined yet

Page 31: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

NSS Auditing (Vigil)

Page 32: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.32

NCP Handling ...(w/o auditing)

User-address space

Kernel-address space

eDirectoryEngine

POSIX

SystemCall

Table

VFSNSS

NCPEngine

NCP_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

NCP_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

Page 33: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.33

NCP Handling ...(vigil_nss.ko)

User-address space

Kernel-address space

eDirectoryEngine

POSIX

SystemCall

Table

VFSNSS

NCPEngine

vigil_nss.ko

NCP_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

NCP_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

Page 34: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.34

NCP Handling ...(vigil.ko)

User-address space

Kernel-address space

eDirectoryEngine

POSIX

SystemCall

Table

VFSNSS

NCPEngine

vigil_nss.ko vigil.ko

NCP_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

NCP_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

Page 35: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.35

NCP Handling ...(vigil.ko API)

User-address space

Kernel-address space

eDirectoryEngine

POSIX

SystemCall

Table

VFSNSS

NCPEngine

vigil_nss.ko

sysfsAPI

vigil.ko

NCP_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

NCP_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

Page 36: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.36

NCP Handling ...(libvigil)

User-address space

Kernel-address space

eDirectoryEngine

POSIX

SystemCall

Table

VFSNSS

NCPEngine

vigil_nss.ko

sysfsAPI

vigil.ko

libvigil

NCP_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

NCP_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

Page 37: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.37

NCP Handling ...(Auditing Client)

User-address space

Kernel-address space

eDirectoryEngine

POSIX

SystemCall

Table

VFSNSS

NCPEngine

vigil_nss.ko

sysfsAPI

vigil.koAudit Clients

Filter(s)

AuditStream

AuditApp.libvigil

NCP_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

NCP_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

SIG

IO

Page 38: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.38

NCP Handling ...(Multiple Auditing Clients)

User-address space

Kernel-address space

eDirectoryEngine

POSIX

SystemCall

Table

VFSNSS

NCPEngine

vigil_nss.ko

sysfsAPI

vigil.koAudit Clients

Filter(s)

AuditStream

AuditApp.libvigil

NCP_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

NCP_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

Filter(s)

AuditStream

Filter(s)

AuditStream

Filter(s)

AuditStream

SIG

IO

Page 39: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.39

NCP Handling ...(Identity Issue)

User-address space

Kernel-address space

eDirectoryEngine

POSIX

SystemCall

Table

VFSNSS

NCPEngine

vigil_nss.ko

sysfsAPI

vigil.koAudit Clients

Filter(s)

AuditStream

AuditApp.libvigil

NCP_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

NCP_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

Filter(s)

AuditStream

Filter(s)

AuditStream

Filter(s)

AuditStream

SIG

IO

User=root

Page 40: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.40

NCP Handling ...(Process Metadata)

User-address space

Kernel-address space

eDirectoryEngine

POSIX

SystemCall

Table

VFSNSS

NCPEngine

pmd.ko

ProcessMeta-data

Table

vigil_nss.ko

sysfsAPI

vigil.koAudit Clients

Filter(s)

AuditStream

AuditApp.libvigil

NCP_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

NCP_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

Filter(s)

AuditStream

Filter(s)

AuditStream

Filter(s)

AuditStream

SIG

IO

Page 41: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.41

NCP Handling ...(Process Metadata Table)

User-address space

Kernel-address space

eDirectoryEngine

POSIX

SystemCall

Table

VFSNSS

NCPEngine

pmd.ko

ProcessMeta-data

Table

vigil_nss.ko

sysfsAPI

vigil.koAudit Clients

Filter(s)

AuditStream

AuditApp.libvigil

NCP_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

NCP_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

Filter(s)

AuditStream

Filter(s)

AuditStream

Filter(s)

AuditStream

SIG

IO

Process Metadata Table

PID NDS GUID10532 12 3FE46A399BD8C12099 252 24 9CAD53B412EE2533 85 1 502DFE7A3C1011300 4 3DEADBEEF111

NCP Conn#

NCP Task#

Page 42: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.42

NCP Handling ...(Process Metadata Logging)

User-address space

Kernel-address space

eDirectoryEngine

POSIX

SystemCall

Table

VFSNSS

NCPEngine

pmd.ko

ProcessMeta-data

Table

vigil_nss.ko

sysfsAPI

vigil.koAudit Clients

Filter(s)

AuditStream

AuditApp.libvigilLinux Process

NCP/NDSClient

Metadata

NCP_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

NCP_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

Filter(s)

AuditStream

Filter(s)

AuditStream

Filter(s)

AuditStream

SIG

IO

Page 43: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.43

NCP Handling ...(Process Metadata Retrieval)

User-address space

Kernel-address space

eDirectoryEngine

POSIX

SystemCall

Table

VFSNSS

NCPEngine

pmd.ko

ProcessMeta-data

Table

vigil_nss.ko

sysfsAPI

vigil.koAudit Clients

Filter(s)

AuditStream

AuditApp.libvigilLinux Process

NCP/NDSClient

Metadata

NCP_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

NCP_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

Filter(s)

AuditStream

Filter(s)

AuditStream

Filter(s)

AuditStream

SIG

IO

Page 44: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.44

NCP Handling ...(Internal NCP Engine Event handling)

User-address space

Kernel-address space

eDirectoryEngine

POSIX

SystemCall

Table

VFSNSS

NCPEngine

pmd.ko

ProcessMeta-data

Table

vigil_nss.ko

sysfsAPI

vigil.koAudit Clients

Filter(s)

AuditStream

AuditApp.libvigil

Lazy-CloseDup-Open

Events

Linux ProcessNCP/NDS

ClientMetadata

NCP_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

NCP_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

Filter(s)

AuditStream

Filter(s)

AuditStream

Filter(s)

AuditStream

SIG

IO

Page 45: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.45

NCP Handling ...(vigil_ncp.ko)

User-address space

Kernel-address space

eDirectoryEngine

POSIX

SystemCall

Table

VFSNSS

NCPEngine

pmd.ko

ProcessMeta-data

Table

vigil_nss.kovigil_ncp.ko

sysfs sysfsAPI

vigil.koAudit Clients

Filter(s)

AuditStream

AuditApp.libvigil

Lazy-CloseDup-Open

Events

Linux ProcessNCP/NDS

ClientMetadata

NCP_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

NCP_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

Filter(s)

AuditStream

Filter(s)

AuditStream

Filter(s)

AuditStream

SIG

IO

Page 46: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.46

CIFS HandlingUser-address space

Kernel-address space

POSIX

SystemCall

Table

VFSNSS

CIFSEngine

pmd.ko

ProcessMeta-data

Table

vigil_nss.kovigil_ncp.ko

sysfs sysfsAPI

vigil.koAudit Clients

Filter(s)

AuditStream

AuditApp.libvigil

Lazy-CloseDup-Open

Events

Linux ProcessNCP/NDS

ClientMetadata

CIFS_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

CIFS_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

Filter(s)

AuditStream

Filter(s)

AuditStream

Filter(s)

AuditStream

SIG

IO

Page 47: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.47

AFP HandlingUser-address space

Kernel-address space

POSIX

SystemCall

Table

VFSNSS

vigil_nss.ko

sysfsAPI

vigil.koAudit Clients

Filter(s)

AuditStream

AuditApp.libvigil

NSSzAPIs

AFP_REQEST: OpenFile User:Joe File:fs1/sys:usr/Joe/readme.txt Mode: read_only

AFP_REPLY: OpenFile Status: 0 (OK) Handle: 21 (NCP Generated)

Filter(s)

AuditStream

Filter(s)

AuditStream

Filter(s)

AuditStream

SIG

IO

AFPEngine

Page 48: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.48

Auditing Client Applications

Blue Lance – LT Auditor+ 9 for SUSE® Linux

NetVision – NVMonitor Version 7.0

Novell® Sentinel™ – Log Manager

Novell Open Enterprise Server – vlog

Page 49: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.49

<AUDIT vlogRecNo="2" vigilRecNo="2" pid="11442" TimeStamp="2010-03-04 09:46:25.416900" Type="3 NSS"> <NSS Event="64 MODIFY" TaskID="0" Zid="1C2E" ParentZid="9A" OpRetCode="0" FileType="3 NAMED_DATA_STREAM" FileAttributes="0x40000021 0-READ_ONLY 5-ARCHIVE 30-ATTR_ARCHIVE" VolID="B87E55ADF559DE018000D7BDDBFE9C09" VolDn="VOL1" UserID="03000000000000000000000000000000" UserDn="Supervisor" Uid="0" Uid_name="root" Euid="0" Euid_name="root" Suid="0" Suid_name="root" Fsuid="0" Fsuid_name="root" Gid="0" Gid_name="root" Egid="0" Egid_name="root" Sgid="0" Sgid_name="root" Fsgid="0" Fsgid_name="root" Comm="ndsd"> <PATH Type="1 target" NameSpace="2 unicode">VOL1:/ajerome/myFile</PATH> <MODIFY_INFO ModifyInfoMask="0x00000021 0-FILE_ATTRIBUTES 5-METADATA_MODIFIED_TIME" modifyTypeInfoMask="0x21" /> <FILE_ATTRIBUTES FileAttributes="0x00000000" FileAttributesModMask="0x4000000B" /> <METADATA_MODIFIED_TIME Time="03/04/2010 09:46:25" /> </NSS></AUDIT>

<AUDIT vlogRecNo="3" vigilRecNo="3" pid="11442" TimeStamp="2010-03-04 09:46:25.417052" Type="3 NSS"> <NSS Event="4 OPEN" TaskID="0" Zid="1C2E" ParentZid="9A" OpRetCode="0" FileType="3 NAMED_DATA_STREAM" FileAttributes="0x40000020 5-ARCHIVE 30-ATTR_ARCHIVE" VolID="B87E55ADF559DE018000D7BDDBFE9C09" VolDn="VOL1" UserID="03000000000000000000000000000000" UserDn="Supervisor" Uid="0" Uid_name="root" Euid="0" Euid_name="root" Suid="0" Suid_name="root" Fsuid="0" Fsuid_name="root" Gid="0" Gid_name="root" Egid="0" Egid_name="root" Sgid="0" Sgid_name="root" Fsgid="0" Fsgid_name="root" Comm="ndsd"> <PATH Type="1 target" NameSpace="2 unicode">VOL1:/ajerome/myFile</PATH> <OPEN key="0x9552AE6B440F959" RequestedRights="0x00000013 0-READ_ACCESS 1-WRITE_ACCESS 4-SCAN_ACCESS" Accessed="03/04/2010 09:46:25" Created="02/05/2010 14:35:01" Modified="02/05/2010 14:35:01" MetaDataModified="03/04/2010 09:46:25" /> </NSS></AUDIT>

<AUDIT vlogRecNo="4" vigilRecNo="4" pid="11442" TimeStamp="2010-03-04 09:46:25.417154" Type="3 NSS"> <NSS Event="64 MODIFY" TaskID="0" Zid="1C2E" ParentZid="9A" OpRetCode="0" FileType="3 NAMED_DATA_STREAM" FileAttributes="0x40000020 5-ARCHIVE 30-ATTR_ARCHIVE" VolID="B87E55ADF559DE018000D7BDDBFE9C09" VolDn="VOL1" UserID="03000000000000000000000000000000" UserDn="Supervisor" Uid="0" Uid_name="root" Euid="0" Euid_name="root" Suid="0" Suid_name="root" Fsuid="0" Fsuid_name="root" Gid="0" Gid_name="root" Egid="0" Egid_name="root" Sgid="0" Sgid_name="root" Fsgid="0" Fsgid_name="root" Comm="ndsd"> <PATH Type="1 target" NameSpace="2 unicode">VOL1:/ajerome/myFile</PATH> <MODIFY_INFO ModifyInfoMask="0x00000021 0-FILE_ATTRIBUTES 5-METADATA_MODIFIED_TIME" modifyTypeInfoMask="0x21" /> <FILE_ATTRIBUTES FileAttributes="0x00000001 0-READ_ONLY" FileAttributesModMask="0x4000000B" /> <METADATA_MODIFIED_TIME Time="03/04/2010 09:46:25" /> </NSS></AUDIT>

<AUDIT vlogRecNo="5" vigilRecNo="5" pid="11442" TimeStamp="2010-03-04 09:46:25.417323" Type="2 NCP"> <NCP Event="16 OPEN"> <WHO_LINUX Uid=”0” UidName=”root” Euid=”0” EuidName=”root” Suid=”0” SuidName=”root” Fsuid=”0” FsuidName=”root” Gid=”0” GidName=”root” Egid=”0” EgidName=”root” Sgid=”0” SgidName=”root” Fsgid=”0” FsguidName=”root” Comm=”ndsd” /> <PMD_NCP ConnID="12" TaskID="4" Guid="A053F649CB3CD94AE5A3A053F649CB3C" Dn=".CN=ajerome.O=novell.T=AJEROME-OES2-64-TREE." /> <NCP_LOCAL__OPENFILE searchAttributes="0x26" DesiredAccessRights="0x01" LinuxPath="/media/nss/VOL1/ajerome/myFile" Status="0" FileHandle="000037623030" FileAttributes="0x00000021 0-READ_ONLY 5-ARCHIVE" FileExecuteType="00" FileLen="14" CreationDate=" 5/02/2010" LastAccessDate=" 4/03/2010" LastUpdateDate=" 5/02/2010" LastUpdateTime="14:35:00" /> </NCP></AUDIT>

<AUDIT vlogRecNo="6" vigilRecNo="6" pid="11442" TimeStamp="2010-03-04 09:46:25.423618" Type="2 NCP"> <NCP Event="32 CLOSE"> <WHO_LINUX Uid=”0” UidName=”root” Euid=”0” EuidName=”root” Suid=”0” SuidName=”root” Fsuid=”0” FsuidName=”root” Gid=”0” GidName=”root” Egid=”0” EgidName=”root” Sgid=”0” SgidName=”root” Fsgid=”0” FsguidName=”root” Comm=”ndsd” /> <PMD_NCP ConnID="12" TaskID="4" Guid="A053F649CB3CD94AE5A3A053F649CB3C" Dn=".CN=ajerome.O=novell.T=AJEROME-OES2-64-TREE." /> <NCP_LOCAL__CLOSEFILE FileHandle="0x000089007B00" Status="0" /> </NCP></AUDIT>

Page 50: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.50

<AUDIT VlogRecNo="3" VigilRecNo="3" Pid="11442" TimeStamp="2010-03-04 09:46:25.417052" Type="3 NSS" > <NSS Event="4 OPEN" TaskID="0" Zid="1C2E" ParentZid="9A" OpRetCode="0" FileType="3 NAMED_DATA_STREAM" FileAttributes="0x40000020 5-ARCHIVE 30-ATTR_ARCHIVE" VolID="B87E55ADF559DE018000D7BDDBFE9C09" VolDn="VOL1" UserID="03000000000000000000000000000000" UserDn="Supervisor" Uid="0" Uid_name="root" Euid="0" Euid_name="root" Suid="0" Suid_name="root" Fsuid="0" Fsuid_name="root" Gid="0" Gid_name="root" Egid="0" Egid_name="root" Sgid="0" Sgid_name="root" Fsgid="0" Fsgid_name="root" Comm="ndsd" > <PATH Type="1 target" NameSpace="2 unicode">VOL1:/ajerome/myFile</PATH> <OPEN Key="0x9552AE6B440F959" RequestedRights="0x00000013 0-READ_ACCESS 1-WRITE_ACCESS 4-SCAN_ACCESS" Accessed="03/04/2010 09:46:25" Created="02/05/2010 14:35:01" Modified="02/05/2010 14:35:01" MetaDataModified="03/04/2010 09:46:25" /> </NSS></AUDIT>

Page 51: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.51

<AUDIT VlogRecNo="4" VigilRecNo="4" Pid="11442" TimeStamp="2010-03-04 09:46:25.417154" Type="3 NSS" > <NSS Event="64 MODIFY" TaskID="0" Zid="1C2E" ParentZid="9A" OpRetCode="0" FileType="3 NAMED_DATA_STREAM" FileAttributes="0x40000020 5-ARCHIVE 30-ATTR_ARCHIVE" VolID="B87E55ADF559DE018000D7BDDBFE9C09" VolDn="VOL1" UserID="03000000000000000000000000000000" UserDn="Supervisor" Uid="0" Uid_name="root" Euid="0" Euid_name="root" Suid="0" Suid_name="root" Fsuid="0" Fsuid_name="root" Gid="0" Gid_name="root" Egid="0" Egid_name="root" Sgid="0" Sgid_name="root" Fsgid="0" Fsgid_name="root" Comm="ndsd" > <PATH Type="1 target" NameSpace="2 unicode">VOL1:/ajerome/myFile</PATH> <MODIFY_INFO ModifyInfoMask="0x00000021 0-FILE_ATTRIBUTES 5-METADATA_MODIFIED_TIME" ModifyTypeInfoMask="0x21" /> <FILE_ATTRIBUTES FileAttributes="0x00000001 0-READ_ONLY" FileAttributesModMask="0x4000000B" /> <METADATA_MODIFIED_TIME Time="03/04/2010 09:46:25" /> </NSS></AUDIT>

Page 52: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.52

<AUDIT VlogRecNo="5" VigilRecNo="5" Pid="11442" TimeStamp="2010-03-04 09:46:25.417323" Type="2 NCP" > <NCP Event="16 OPEN" > <WHO_LINUX Uid=”0” UidName=”root” Euid=”0” EuidName=”root” Suid=”0” SuidName=”root” Fsuid=”0” FsuidName=”root” Gid=”0” GidName=”root” Egid=”0” EgidName=”root” Sgid=”0” SgidName=”root” Fsgid=”0” FsguidName=”root” Comm=”ndsd” /> <PMD_NCP ConnID="12" TaskID="4" Guid="A053F649CB3CD94AE5A3A053F649CB3C" Dn=".CN=ajerome.O=novell.T=AJEROME-OES2-64-TREE." /> <NCP_LOCAL__OPENFILE SearchAttributes="0x26" DesiredAccessRights="0x01" LinuxPath="/media/nss/VOL1/ajerome/myFile" Status="0" FileHandle="000037623030" FileAttributes="0x00000021 0-READ_ONLY 5-ARCHIVE" FileExecuteType="00" FileLen="14" CreationDate=" 5/02/2010" LastAccessDate=" 4/03/2010" LastUpdateDate=" 5/02/2010" LastUpdateTime="14:35:00" /> </NCP></AUDIT>

Page 53: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.53

<AUDIT VlogRecNo="6" VigilRecNo="6" Pid="11442" TimeStamp="2010-03-04 09:46:25.423618" Type="2 NCP" > <NCP Event="32 CLOSE"> <WHO_LINUX Uid=”0” UidName=”root” Euid=”0” EuidName=”root” Suid=”0” SuidName=”root” Fsuid=”0” FsuidName=”root” Gid=”0” GidName=”root” Egid=”0” EgidName=”root” Sgid=”0” SgidName=”root” Fsgid=”0” FsguidName=”root” Comm=”ndsd” /> <PMD_NCP ConnID="12" TaskID="4" Guid="A053F649CB3CD94AE5A3A053F649CB3C" Dn=".CN=ajerome.O=novell.T=AJEROME-OES2-64-TREE." /> <NCP_LOCAL__CLOSEFILE FileHandle="0x000089007B00" Status="0" /> </NCP></AUDIT>

Page 54: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.54

Related Sessions

• CL116

– File Access in Novell® Open Enterprise Server 2 SP2

• CL115

– Novell Open Enterprise Server: Roadmap and Futures

• CL306

– High Availability with Novell Cluster Services for Novell Open Enterprise Server on Linux

• CL114

– Novell Open Enterprise Server Architecture

Page 55: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

© Novell, Inc. All rights reserved.55

Reference

• NCP™ Tuning> http://www.novell.com/documentation/oes2/file_ncp_lx/data/bc06ts8.html

• Novell Storage Services™ Tuning> http://www.novell.com/documentation/oes2/stor_nss_lx_nw/data/btbkjyi.html

• Auditing> http://developer.novell.com/wiki/index.php/NSS_Auditing_SDK

• TIDs> TID#7004888 - NCP Performance Tuning

> TID#7004877 – NSS takes time to load and slow access

> http://support.novell.com

Page 56: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

Question and Answer

Page 57: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux
Page 58: NSS File System Performance, Clustering and Auditing in Novell Open Enterprise Server on Linux

Unpublished Work of Novell, Inc. All Rights Reserved.This work is an unpublished work and contains confidential, proprietary, and trade secret information of Novell, Inc. Access to this work is restricted to Novell employees who have a need to know to perform tasks within the scope of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of Novell, Inc. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability.

General DisclaimerThis document is not to be construed as a promise by any participating company to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. Novell, Inc. makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for Novell products remains at the sole discretion of Novell. Further, Novell, Inc. reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All Novell marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.